阶段 3：实时数据 —— 采样、合成、最新帧调度与适配器 - What: sampling.py 实现 SurfaceSampler（向量化四点双线性：无效 tap 不计入也不致无效、任一**正权** tap 无效即整顶点无效且值为 NaN、域/形状不符先拒；采样表只在加载时构建）与 remap_to_domain（按 unroll 映射 gather，空洞保持 NaN 且 valid=False，保留尾随轴以支持矢量场）。composer.py 实现 SceneComposer.compose（域校验 → 采样 → 位移 → **从形变后位置重算法线**；scalar 绝不成以 display_gain；approximate 位移产生 display_illustrative(info) 诊断、calibrated 不产生、无位移则着色并报 field_unavailable）与 build_rest_frame（参考帧：scalar 全 NaN、valid 全假、descriptor=unavailable/display_only），gl_backend.rest_surface 改为委托它，避免两处定义。coordinator.py 实现完整调度：容量 1 双 mailbox、generation/来源/会话三重过期拒绝并计数、**最多一次合成在飞行**、可见性挂起/恢复、设置变更（倍率变化重合成**屏幕上那一帧**）、close 幂等；submit 只做身份与形状的廉价校验，绝不等显示。adapters.py 实现 Finger/Box 适配器：|flow| 或 depth 响应、可选经 unroll 映射 remap、设备 valid_mask 优先、域不符时**发布空**并报出所见的两个形状；一律不造位移（未标定的毫米不如没有），并新增 display_illustrative 到诊断码表（info）。assets.py 从"桩类堆"改成门面：re-export 真正实现 + SceneCompiler.compile（外壳 GLB 与预算检查；加密请求明确拒绝并指向阶段 5）。测试：shared/tests/test_scene3d_live.py 25 项（采样/无效语义/倍率只影响位置/示意形变标注/法线重算/remap 空洞/参考帧/只读数组）、apps/studio/tests/test_scene3d_coordinator.py 13 项（状态机、过期、替换计数、单飞行、submit p99<1ms、切换场景丢弃在飞结果、隐藏挂起、倍率重合成、close 幂等）、test_scene3d_adapters.py 13 项；并把三处"桩仍未实现"的过时断言改成对已实现行为的断言。docs/architecture/studio-3d-stages.md 阶段 3 记为完成。 - Why: 阶段 3 决定"屏幕上的颜色和力值到底对应什么"。三条最容易错且肉眼看不出来的规则必须由代码而不是约定来保证：无效点不是 0（NaN 与 valid 分离，否则缺口会被画成一个测量值）、显示倍率只动位置不动测量（否则色标悄悄变了含义）、以及显示永远不能给采集施加背压（容量 1 mailbox + 单飞行合成 + 廉价 submit）。 - Verification: shared 全量通过（scene3d 相关 103 项：contracts 26、assets 33、package 19、live 25）；studio 全量通过（scene3d 相关 54 项：stage1 24、skeleton 4、coordinator 13、adapters 13；GL 冒烟 10 项在真实 context 下另行通过）；ui_check 554 项 0 失败。过程中修掉 3 个真实缺陷：倍率变化时用 incoming.peek() 取不到"屏幕上那一帧"（已被消费）导致改了倍率画面不动——改为保留 _last_field_frame；display_illustrative 不在码表里被默认成 error；assets.py 里与 package/legacy 同名的重复桩类（会误导集成者，已改为门面）。另修正两处我自己的错误断言（对称凸起顶点法线本就竖直；pytest.approx 不能直接 .all()）。 - Not in this change: 手势与面板（阶段 4）；示意位移的真实来源（需要 SDK lift + 已确认尺度，现阶段 adapter 一��� displacement=None）；app_window 接线与记录隔离（阶段 4）。
This commit is contained in:
ykcheungab
@@ -8,44 +8,216 @@ The constraints here are about *where* the data may be read, not about optics:
|
||||
runtime, because the SDK's arrays are that thread's working memory;
|
||||
* it copies what the frame needs and publishes it read-only — the GUI thread must
|
||||
never reach back into a live runtime;
|
||||
* it copies **nothing** when the three-dimensional view is not subscribed to, and
|
||||
returns ``None`` when the frame carries no usable field. A frame is never
|
||||
assembled from two different acquisitions, and a missing field is never filled
|
||||
in from the previous one.
|
||||
* it publishes **nothing** when the frame carries no usable field, or when the
|
||||
field is on a different grid than the scene's UV domain. A remap that is not
|
||||
configured is not guessed at: the adapter says which shapes it saw, so the
|
||||
mismatch is a fact in the diagnostics rather than a picture that looks plausible.
|
||||
|
||||
Two things the v1 adapters deliberately do **not** do:
|
||||
|
||||
* they do not invent a displacement. The SDK's lift uses optical flow and a scale
|
||||
estimated from the mesh, which the design document calls "not a verified
|
||||
displacement calibration"; a millimetre field that was never calibrated is worse
|
||||
than no field, so ``displacement_sensor_m`` is ``None`` and the surface is
|
||||
coloured in place. The composer already handles an ``approximate`` displacement
|
||||
for when a verified one arrives;
|
||||
* they do not rescale or pad a field to fit. A field whose shape does not match the
|
||||
domain is a mismatch (`field_domain_mismatch`), not something to resize.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.acquisition.models import SdkPacket
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
Diagnostic,
|
||||
ExpectedGeometry,
|
||||
FieldDescriptor,
|
||||
FrameIdentity,
|
||||
SurfaceFieldFrame,
|
||||
diagnostic,
|
||||
error,
|
||||
freeze,
|
||||
)
|
||||
from orisys_app_shared.scene3d.sampling import remap_to_domain
|
||||
|
||||
#: What the adapters can colour: the optical flow's magnitude, in pixels. An
|
||||
#: unsigned magnitude is a sequential colour scale, and ``approximate`` says the
|
||||
#: number is a pixel displacement rather than a calibrated length.
|
||||
FLOW_MAGNITUDE = FieldDescriptor(
|
||||
quantity="flow_magnitude", unit="px", quality="approximate", signed=False
|
||||
)
|
||||
#: The deformation response the SDK's depth map reports: a relative response, not a
|
||||
#: length. It is only ever coloured, never turned into millimetres.
|
||||
DEPTH_RESPONSE = FieldDescriptor(
|
||||
quantity="depth_response", unit="1", quality="approximate", signed=False
|
||||
)
|
||||
|
||||
_STAGE = "stage 3 (live data)"
|
||||
|
||||
def flow_magnitude(packet: SdkPacket) -> Optional[np.ndarray]:
|
||||
"""``|flow|`` as float32, or ``None`` when the packet carries no flow."""
|
||||
flow = getattr(packet, "flow", None)
|
||||
if flow is None:
|
||||
return None
|
||||
array = np.asarray(flow, dtype=np.float32)
|
||||
if array.ndim == 3 and array.shape[-1] == 2:
|
||||
return np.sqrt(array[..., 0] ** 2 + array[..., 1] ** 2)
|
||||
if array.ndim == 2:
|
||||
return array
|
||||
return None
|
||||
|
||||
|
||||
def _shape_of(array: Optional[np.ndarray]) -> str:
|
||||
return "none" if array is None else "x".join(str(int(v)) for v in array.shape[:2])
|
||||
|
||||
|
||||
class FingerFrameAdapter:
|
||||
"""The finger runtime already computes a UV field; copy it, with its identity."""
|
||||
"""The finger runtime's field, in the scene's UV domain.
|
||||
|
||||
``map_x``/``map_y`` are the unroll map's source coordinates. When they are
|
||||
supplied the field is remapped through them; when they are not, the field is
|
||||
used directly and only if its shape already *is* the domain.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
geometry_id: str,
|
||||
field_domain_id: str,
|
||||
shape_hw: tuple[int, int],
|
||||
map_x: Optional[np.ndarray] = None,
|
||||
map_y: Optional[np.ndarray] = None,
|
||||
) -> None:
|
||||
self.geometry_id = str(geometry_id)
|
||||
self.field_domain_id = str(field_domain_id)
|
||||
self.shape_hw = (int(shape_hw[0]), int(shape_hw[1]))
|
||||
self.map_x = map_x
|
||||
self.map_y = map_y
|
||||
self.last_diagnostic: Optional[Diagnostic] = None
|
||||
|
||||
@classmethod
|
||||
def from_domain(cls, domain: Any, *, geometry_id: str) -> "FingerFrameAdapter":
|
||||
"""Build from a ``LegacyFieldDomain`` (which carries the map, when loaded)."""
|
||||
return cls(
|
||||
geometry_id=geometry_id,
|
||||
field_domain_id=str(getattr(domain, "domain_id")),
|
||||
shape_hw=tuple(getattr(domain, "shape_hw")),
|
||||
map_x=getattr(domain, "map_x", None),
|
||||
map_y=getattr(domain, "map_y", None),
|
||||
)
|
||||
|
||||
def snapshot(
|
||||
self, packet: SdkPacket, runtime: Any, identity: FrameIdentity
|
||||
) -> Optional[SurfaceFieldFrame]:
|
||||
raise NotImplementedError(f"{_STAGE}: snapshot the finger UV field")
|
||||
magnitude = flow_magnitude(packet)
|
||||
if magnitude is None:
|
||||
self.last_diagnostic = diagnostic(
|
||||
"field_unavailable", "this frame carries no optical flow"
|
||||
)
|
||||
return None
|
||||
if self.map_x is not None and self.map_y is not None:
|
||||
try:
|
||||
values, valid = remap_to_domain(
|
||||
magnitude, map_x=self.map_x, map_y=self.map_y, shape_hw=self.shape_hw
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - a bad map is a diagnostic
|
||||
self.last_diagnostic = getattr(exc, "diagnostic", None) or diagnostic(
|
||||
"field_domain_mismatch", f"remap failed: {type(exc).__name__}: {exc}"
|
||||
)
|
||||
return None
|
||||
scalar = values
|
||||
mask = valid
|
||||
elif tuple(magnitude.shape[:2]) == self.shape_hw:
|
||||
scalar = freeze(magnitude, dtype=np.float32)
|
||||
mask = freeze(np.isfinite(magnitude), dtype=np.bool_)
|
||||
else:
|
||||
# Say what was seen: this is the message that tells an engineer whether
|
||||
# the flow arrives in the tracker's grid or already unrolled.
|
||||
self.last_diagnostic = diagnostic(
|
||||
"field_domain_mismatch",
|
||||
"the flow is not in the scene's UV domain and no remap is configured",
|
||||
flow=_shape_of(magnitude),
|
||||
domain=f"{self.shape_hw[0]}x{self.shape_hw[1]}",
|
||||
)
|
||||
return None
|
||||
self.last_diagnostic = None
|
||||
return SurfaceFieldFrame(
|
||||
identity=identity,
|
||||
geometry_id=self.geometry_id,
|
||||
field_domain_id=self.field_domain_id,
|
||||
scalar=scalar,
|
||||
scalar_descriptor=FLOW_MAGNITUDE,
|
||||
valid=mask,
|
||||
)
|
||||
|
||||
|
||||
class BoxFrameAdapter:
|
||||
"""Box/mini: a scalar field only.
|
||||
"""Box/mini: a scalar field only, and only when the grid already matches.
|
||||
|
||||
There is no verified spatial scale for these sensors yet, so no displacement is
|
||||
There is no verified spatial scale for these sensors, so no displacement is
|
||||
invented: a millimetre height field that was never calibrated is worse than no
|
||||
height field at all.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
geometry_id: str,
|
||||
field_domain_id: str,
|
||||
shape_hw: tuple[int, int],
|
||||
) -> None:
|
||||
self.geometry_id = str(geometry_id)
|
||||
self.field_domain_id = str(field_domain_id)
|
||||
self.shape_hw = (int(shape_hw[0]), int(shape_hw[1]))
|
||||
self.last_diagnostic: Optional[Diagnostic] = None
|
||||
|
||||
def snapshot(
|
||||
self, packet: SdkPacket, identity: FrameIdentity, domain: ExpectedGeometry
|
||||
) -> Optional[SurfaceFieldFrame]:
|
||||
raise NotImplementedError(f"{_STAGE}: snapshot the box scalar field")
|
||||
if tuple(domain.shape_hw) != self.shape_hw:
|
||||
self.last_diagnostic = diagnostic(
|
||||
"field_domain_mismatch",
|
||||
"the device description names a different field domain",
|
||||
device=f"{domain.shape_hw[0]}x{domain.shape_hw[1]}",
|
||||
scene=f"{self.shape_hw[0]}x{self.shape_hw[1]}",
|
||||
)
|
||||
return None
|
||||
candidates = (
|
||||
("depth_response", getattr(packet, "depth", None)),
|
||||
("flow_magnitude", flow_magnitude(packet)),
|
||||
)
|
||||
for name, field in candidates:
|
||||
if field is None:
|
||||
continue
|
||||
array = np.asarray(field, dtype=np.float32)
|
||||
if array.ndim != 2 or tuple(array.shape) != self.shape_hw:
|
||||
continue
|
||||
mask = np.isfinite(array)
|
||||
explicit = getattr(packet, "valid_mask", None)
|
||||
if explicit is not None:
|
||||
explicit_array = np.asarray(explicit, dtype=bool)
|
||||
if tuple(explicit_array.shape) == self.shape_hw:
|
||||
# The device's own mask wins: it knows where its surface is.
|
||||
mask = mask & explicit_array
|
||||
self.last_diagnostic = None
|
||||
return SurfaceFieldFrame(
|
||||
identity=identity,
|
||||
geometry_id=self.geometry_id,
|
||||
field_domain_id=self.field_domain_id,
|
||||
scalar=freeze(array, dtype=np.float32),
|
||||
scalar_descriptor=(
|
||||
DEPTH_RESPONSE if name == "depth_response" else FLOW_MAGNITUDE
|
||||
),
|
||||
valid=freeze(mask, dtype=np.bool_),
|
||||
)
|
||||
self.last_diagnostic = diagnostic(
|
||||
"field_unavailable",
|
||||
"no usable scalar field of the scene's shape in this frame",
|
||||
depth=_shape_of(getattr(packet, "depth", None)),
|
||||
flow=_shape_of(flow_magnitude(packet)),
|
||||
domain=f"{self.shape_hw[0]}x{self.shape_hw[1]}",
|
||||
)
|
||||
return None
|
||||
|
||||
@@ -1,32 +1,50 @@
|
||||
"""Frame schedule and state machine for the three-dimensional view.
|
||||
|
||||
Stage 3 (mailboxes, compose worker) and stage 4 (Qt timer, visibility, settings) in
|
||||
Stage 3 (mailboxes, compose worker) and stage 4 (Qt timer, visibility) in
|
||||
``docs/architecture/studio-3d-stages.md``.
|
||||
|
||||
The whole point of this class is that **the acquisition loop never waits for the
|
||||
display**. Producers call :meth:`submit`, which validates cheaply and drops the
|
||||
previous un-consumed frame; a GUI timer calls :meth:`tick`, which picks up the
|
||||
newest composed frame and repaints. Display drops are counted here and never in
|
||||
the recording channel, and no CPU task is submitted while one is still running.
|
||||
display**. A producer calls :meth:`submit`, which validates cheaply and drops the
|
||||
previous un-consumed frame; a GUI timer calls :meth:`tick`, which hands the newest
|
||||
field frame to the composer, picks up the newest composed frame and asks the widget
|
||||
to repaint. Three properties make that true rather than aspirational:
|
||||
|
||||
* both mailboxes hold **one** frame, and ``submit`` only ever swaps a reference
|
||||
under a lock — the work it can do is bounded by the lock, not by the display;
|
||||
* **at most one compose task is in flight**. A slow composer makes frames drop; it
|
||||
cannot make the queue grow;
|
||||
* display drops are counted here and never mixed into the recording channel, which
|
||||
keeps running through the existing sink path untouched.
|
||||
|
||||
Staleness is about *identity*, not timing: a frame from another generation, another
|
||||
session or another source than the active scene is dropped with ``stale_frame`` and
|
||||
counted, and an old result can never overwrite a new scene because the composer's
|
||||
result carries the identity it was built from and is checked again on arrival.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from concurrent.futures import Executor
|
||||
from typing import Optional
|
||||
import threading
|
||||
from concurrent.futures import Executor, Future
|
||||
from typing import Any, Optional
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
Diagnostic,
|
||||
DisplaySettings,
|
||||
FrameIdentity,
|
||||
LatestFrameMailbox,
|
||||
OfferResult,
|
||||
RenderFrame,
|
||||
SceneAssets,
|
||||
SceneState,
|
||||
SurfaceFieldFrame,
|
||||
diagnostic,
|
||||
error,
|
||||
)
|
||||
from orisys_studio.scene3d.widget import Sensor3DWidget
|
||||
|
||||
_STAGE = "stage 3 (schedule) and stage 4 (Qt integration)"
|
||||
#: How many composed frames may be "lost" before the coordinator stops caring: the
|
||||
#: counter is informational, never a reason to throttle acquisition.
|
||||
MAX_INFLIGHT_COMPOSES = 1
|
||||
|
||||
|
||||
class ThreeDCoordinator:
|
||||
@@ -34,48 +52,41 @@ class ThreeDCoordinator:
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
widget: Sensor3DWidget,
|
||||
composer: object,
|
||||
widget: Any,
|
||||
composer: Any,
|
||||
executor: Executor,
|
||||
max_fps: int = 30,
|
||||
) -> None:
|
||||
self._widget = widget
|
||||
self._composer = composer
|
||||
self._executor = executor
|
||||
self._max_fps = int(max_fps)
|
||||
self._max_fps = max(1, int(max_fps))
|
||||
self._incoming: LatestFrameMailbox[SurfaceFieldFrame] = LatestFrameMailbox()
|
||||
self._composed: LatestFrameMailbox[RenderFrame] = LatestFrameMailbox()
|
||||
self._lock = threading.Lock()
|
||||
self._state: SceneState = "disabled"
|
||||
self._generation = 0
|
||||
self._assets: Optional[SceneAssets] = None
|
||||
self._source_id = ""
|
||||
self._session_id: Optional[int] = None
|
||||
self._visible = False
|
||||
self._settings = DisplaySettings(max_fps=self._max_fps)
|
||||
self._inflight: Optional[Future] = None
|
||||
self._stale_frames = 0
|
||||
self._invalid_frames = 0
|
||||
self._last_diagnostic: Optional[Diagnostic] = None
|
||||
self._closed = False
|
||||
self._settings_dirty = False
|
||||
self._last_composed_identity: Optional[FrameIdentity] = None
|
||||
#: The field frame currently on screen, so a gain change can redraw exactly
|
||||
#: it rather than a newer instant.
|
||||
self._last_field_frame: Optional[SurfaceFieldFrame] = None
|
||||
|
||||
def activate(self, assets: SceneAssets, source_id: str) -> int:
|
||||
"""Adopt a scene and a data source; returns the new generation."""
|
||||
raise NotImplementedError(f"{_STAGE}: activate a scene")
|
||||
|
||||
def submit(self, frame: SurfaceFieldFrame) -> OfferResult:
|
||||
"""Offer a field frame from a producer thread. Never blocks on the GUI."""
|
||||
raise NotImplementedError(f"{_STAGE}: accept or reject an incoming frame")
|
||||
|
||||
def tick(self) -> None:
|
||||
"""GUI timer: take the newest composed frame, present it, schedule work."""
|
||||
raise NotImplementedError(f"{_STAGE}: pump the mailboxes")
|
||||
|
||||
def set_visible(self, visible: bool) -> None:
|
||||
"""Hiding suspends composing, uploading and rendering — not acquisition."""
|
||||
raise NotImplementedError(f"{_STAGE}: suspend or resume the view")
|
||||
|
||||
def set_settings(self, settings: DisplaySettings) -> None:
|
||||
raise NotImplementedError(f"{_STAGE}: apply display settings")
|
||||
|
||||
def close(self) -> None:
|
||||
"""Cancel work, stop the timer, clear both mailboxes. Idempotent."""
|
||||
raise NotImplementedError(f"{_STAGE}: shut the view down")
|
||||
|
||||
# ------------------------------------------------------------------- state
|
||||
@property
|
||||
def state(self) -> SceneState:
|
||||
return self._state
|
||||
with self._lock:
|
||||
return self._state
|
||||
|
||||
@property
|
||||
def generation(self) -> int:
|
||||
@@ -83,7 +94,261 @@ class ThreeDCoordinator:
|
||||
|
||||
@property
|
||||
def superseded_frames(self) -> int:
|
||||
"""Frames the display never showed. Not a recording drop."""
|
||||
return (
|
||||
self._incoming.superseded_frames + self._composed.superseded_frames
|
||||
"""Frames the display never showed. Never a recording drop."""
|
||||
return self._incoming.superseded_frames + self._composed.superseded_frames
|
||||
|
||||
@property
|
||||
def stale_frames(self) -> int:
|
||||
return self._stale_frames
|
||||
|
||||
@property
|
||||
def invalid_frames(self) -> int:
|
||||
return self._invalid_frames
|
||||
|
||||
@property
|
||||
def last_diagnostic(self) -> Optional[Diagnostic]:
|
||||
return self._last_diagnostic
|
||||
|
||||
@property
|
||||
def visible(self) -> bool:
|
||||
return self._visible
|
||||
|
||||
def settings(self) -> DisplaySettings:
|
||||
return self._settings
|
||||
|
||||
def _set_state(self, state: SceneState) -> None:
|
||||
with self._lock:
|
||||
self._state = state
|
||||
|
||||
# --------------------------------------------------------------- lifetime
|
||||
def activate(self, assets: SceneAssets, source_id: str) -> int:
|
||||
"""Adopt a scene and a data source; returns the new generation.
|
||||
|
||||
Everything already in flight belongs to the previous scene and is dropped:
|
||||
the generation is what makes "old" mean something.
|
||||
"""
|
||||
if self._closed:
|
||||
raise error("asset_rejected", "this coordinator has been closed")
|
||||
self._cancel_inflight()
|
||||
self._incoming.clear()
|
||||
self._composed.clear()
|
||||
self._assets = assets
|
||||
self._source_id = str(source_id)
|
||||
self._session_id = None
|
||||
self._generation += 1
|
||||
self._last_composed_identity = None
|
||||
self._last_field_frame = None
|
||||
self._settings_dirty = False
|
||||
self._widget.set_scene(assets)
|
||||
if self._visible:
|
||||
self._widget.present(self._composer.rest_frame(assets, self._identity(0)))
|
||||
self._set_state("ready")
|
||||
else:
|
||||
self._set_state("loading" if self._visible else "suspended")
|
||||
return self._generation
|
||||
|
||||
def set_visible(self, visible: bool) -> None:
|
||||
"""Hiding suspends composing, uploading and rendering — not acquisition."""
|
||||
visible = bool(visible)
|
||||
if self._closed:
|
||||
return
|
||||
self._widget.setVisible(visible)
|
||||
if not visible:
|
||||
self._visible = False
|
||||
self._incoming.clear()
|
||||
self._composed.clear()
|
||||
self._cancel_inflight()
|
||||
self._set_state("suspended")
|
||||
return
|
||||
self._visible = True
|
||||
# Come back on the newest frame rather than replaying what was dropped.
|
||||
self._set_state("ready" if self._assets is not None else "disabled")
|
||||
|
||||
# ---------------------------------------------------------------- producers
|
||||
def submit(self, frame: SurfaceFieldFrame) -> OfferResult:
|
||||
"""Offer a field frame from a producer thread. Never blocks on the GUI."""
|
||||
if self._closed or not self._visible:
|
||||
return "disabled"
|
||||
with self._lock:
|
||||
assets = self._assets
|
||||
generation = self._generation
|
||||
session_id = self._session_id
|
||||
if assets is None:
|
||||
return "disabled"
|
||||
identity = frame.identity
|
||||
if identity.source_id != self._source_id or identity.generation != generation:
|
||||
self._stale_frames += 1
|
||||
self._last_diagnostic = diagnostic(
|
||||
"stale_frame",
|
||||
"a frame from another source or generation was dropped",
|
||||
source=identity.source_id,
|
||||
generation=str(identity.generation),
|
||||
)
|
||||
return "stale"
|
||||
if session_id is not None and identity.session_id != session_id:
|
||||
self._stale_frames += 1
|
||||
return "stale"
|
||||
problem = self._reject_reason(frame, assets)
|
||||
if problem is not None:
|
||||
self._invalid_frames += 1
|
||||
self._last_diagnostic = problem
|
||||
return "invalid"
|
||||
with self._lock:
|
||||
if session_id is None:
|
||||
self._session_id = identity.session_id
|
||||
return self._incoming.offer(frame)
|
||||
|
||||
@staticmethod
|
||||
def _reject_reason(
|
||||
frame: SurfaceFieldFrame, assets: SceneAssets
|
||||
) -> Optional[Diagnostic]:
|
||||
"""The cheap checks a producer thread may do: identity and shape only."""
|
||||
if frame.geometry_id != assets.geometry_id:
|
||||
return diagnostic(
|
||||
"geometry_mismatch",
|
||||
"the frame belongs to a different geometry",
|
||||
frame=frame.geometry_id,
|
||||
scene=assets.geometry_id,
|
||||
)
|
||||
if frame.field_domain_id != assets.field_domain_id:
|
||||
return diagnostic(
|
||||
"field_domain_mismatch",
|
||||
"the frame is on a different field domain",
|
||||
frame=frame.field_domain_id,
|
||||
scene=assets.field_domain_id,
|
||||
)
|
||||
if tuple(frame.scalar.shape) != tuple(assets.binding.shape_hw):
|
||||
return diagnostic(
|
||||
"field_domain_mismatch",
|
||||
"the frame's shape does not match the scene's UV domain",
|
||||
frame=f"{frame.scalar.shape[0]}x{frame.scalar.shape[1]}",
|
||||
scene=f"{assets.binding.shape_hw[0]}x{assets.binding.shape_hw[1]}",
|
||||
)
|
||||
if tuple(frame.valid.shape) != tuple(frame.scalar.shape):
|
||||
return diagnostic(
|
||||
"field_domain_mismatch", "the frame's scalar and validity disagree"
|
||||
)
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------- consumer
|
||||
def tick(self) -> None:
|
||||
"""GUI timer: start one compose, take the newest result, repaint."""
|
||||
if self._closed or not self._visible or self._assets is None:
|
||||
return
|
||||
self._collect_inflight()
|
||||
if self._inflight is None:
|
||||
if self._settings_dirty and self._last_field_frame is not None:
|
||||
# A gain change re-draws the field that is already on screen; taking
|
||||
# a newer one would show a different instant than the one the
|
||||
# operator just changed the exaggeration of.
|
||||
self._settings_dirty = False
|
||||
self._start_compose(self._last_field_frame)
|
||||
else:
|
||||
self._settings_dirty = False
|
||||
frame = self._incoming.take()
|
||||
if frame is not None:
|
||||
self._start_compose(frame)
|
||||
composed = self._composed.take()
|
||||
if composed is not None:
|
||||
if not self._identity_is_current(composed.identity):
|
||||
self._stale_frames += 1
|
||||
return
|
||||
self._widget.set_superseded_frames(self.superseded_frames)
|
||||
self._widget.present(composed)
|
||||
self._last_composed_identity = composed.identity
|
||||
self._set_state("live" if composed.valid.any() else "no_data")
|
||||
elif self._state in ("ready", "loading"):
|
||||
# Nothing has arrived yet: show the reference model rather than a blank
|
||||
# window, and say that no measurement is being displayed.
|
||||
if self._last_composed_identity is None:
|
||||
self._widget.present(self._composer.rest_frame(self._assets, self._identity(0)))
|
||||
self._set_state("no_data")
|
||||
|
||||
def set_settings(self, settings: DisplaySettings) -> None:
|
||||
"""Apply display settings; a gain change means recomposing the last field."""
|
||||
previous = self._settings
|
||||
self._settings = settings
|
||||
self._widget.set_settings(settings)
|
||||
gain_changed = (
|
||||
previous.display_gain != settings.display_gain
|
||||
or previous.deform_enabled != settings.deform_enabled
|
||||
)
|
||||
if gain_changed and self._assets is not None:
|
||||
# The scalar is untouched by the gain, but the *positions* are not, so
|
||||
# the last field frame has to be composed again.
|
||||
self._settings_dirty = True
|
||||
|
||||
def close(self) -> None:
|
||||
"""Cancel work, stop the timer, clear both mailboxes. Idempotent."""
|
||||
if self._closed:
|
||||
return
|
||||
self._closed = True
|
||||
self._cancel_inflight()
|
||||
self._incoming.clear()
|
||||
self._composed.clear()
|
||||
self._set_state("disabled")
|
||||
|
||||
# ------------------------------------------------------------------ internal
|
||||
def _identity(self, request_id: int) -> FrameIdentity:
|
||||
return FrameIdentity(
|
||||
source_id=self._source_id,
|
||||
session_id=self._session_id or 0,
|
||||
frame_id=0,
|
||||
t_rel_s=0.0,
|
||||
generation=self._generation,
|
||||
request_id=int(request_id),
|
||||
)
|
||||
|
||||
def _identity_is_current(self, identity: FrameIdentity) -> bool:
|
||||
return (
|
||||
identity.source_id == self._source_id
|
||||
and identity.generation == self._generation
|
||||
and (self._session_id is None or identity.session_id == self._session_id)
|
||||
)
|
||||
|
||||
def _start_compose(self, frame: SurfaceFieldFrame) -> None:
|
||||
assets = self._assets
|
||||
settings = self._settings
|
||||
if assets is None:
|
||||
return
|
||||
generation = self._generation
|
||||
# Kept so a display-settings change can recompose *this* frame instead of
|
||||
# jumping to a newer one.
|
||||
self._last_field_frame = frame
|
||||
|
||||
def work() -> None:
|
||||
try:
|
||||
composed = self._composer.compose(assets, frame, settings)
|
||||
except Exception as exc: # noqa: BLE001 - a failed compose is not a crash
|
||||
self._last_diagnostic = getattr(exc, "diagnostic", None) or diagnostic(
|
||||
"field_unavailable", f"compose failed: {type(exc).__name__}: {exc}"
|
||||
)
|
||||
return
|
||||
if composed.identity.generation != generation:
|
||||
# The scene changed while this was running: the result is for a
|
||||
# scene that no longer exists.
|
||||
self._stale_frames += 1
|
||||
return
|
||||
self._composed.offer(composed)
|
||||
|
||||
self._inflight = self._executor.submit(work)
|
||||
|
||||
def _collect_inflight(self) -> None:
|
||||
future = self._inflight
|
||||
if future is None:
|
||||
return
|
||||
if not future.done():
|
||||
return
|
||||
self._inflight = None
|
||||
try:
|
||||
future.result()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
self._last_diagnostic = getattr(exc, "diagnostic", None) or diagnostic(
|
||||
"field_unavailable", f"compose raised: {type(exc).__name__}: {exc}"
|
||||
)
|
||||
|
||||
def _cancel_inflight(self) -> None:
|
||||
future = self._inflight
|
||||
self._inflight = None
|
||||
if future is not None:
|
||||
future.cancel()
|
||||
|
||||
@@ -310,24 +310,12 @@ def rest_surface(assets: SceneAssets, identity: Any) -> RenderFrame:
|
||||
|
||||
Used to *clear* a measurement — after a refused frame, on connect, or on a
|
||||
disconnect — so the screen cannot keep showing the previous acquisition's
|
||||
colours as if they were current. ``quality="display_only"`` and an
|
||||
``unavailable`` quantity are what tell the UI to hide the measurement bar.
|
||||
colours as if they were current. The composer owns the definition; this is the
|
||||
GL layer reaching it without importing the whole composition path.
|
||||
"""
|
||||
from orisys_app_shared.scene3d.contracts import rest_descriptor
|
||||
from orisys_app_shared.scene3d.composer import build_rest_frame
|
||||
|
||||
part = assets.sensing_part()
|
||||
count = int(part.positions_sensor_m.shape[0])
|
||||
return RenderFrame(
|
||||
identity=identity,
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=part.part_id,
|
||||
positions_sensor_m=part.positions_sensor_m,
|
||||
normals_sensor=part.normals_sensor,
|
||||
scalar=freeze(np.full(count, np.nan, dtype=np.float32), dtype=np.float32),
|
||||
valid=freeze(np.zeros(count, dtype=bool), dtype=np.bool_),
|
||||
scalar_descriptor=rest_descriptor(),
|
||||
)
|
||||
return build_rest_frame(assets, identity)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- dispatch
|
||||
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Stage 3 acceptance for the frame adapters.
|
||||
|
||||
The adapters are the boundary between the acquisition thread's arrays and the
|
||||
scene, so the tests are about *what they publish and what they refuse*: a field in
|
||||
the right grid, read-only, with the right unit and quality — and nothing at all when
|
||||
the grid does not match, with a diagnostic that says which shapes were seen.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
ExpectedGeometry,
|
||||
FrameIdentity,
|
||||
ThreeDError,
|
||||
validate_surface_frame,
|
||||
)
|
||||
from orisys_studio.scene3d.adapters import (
|
||||
DEPTH_RESPONSE,
|
||||
FLOW_MAGNITUDE,
|
||||
BoxFrameAdapter,
|
||||
FingerFrameAdapter,
|
||||
flow_magnitude,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class FakePacket:
|
||||
"""The parts of ``SdkPacket`` the adapters look at."""
|
||||
|
||||
flow: Optional[np.ndarray] = None
|
||||
depth: Optional[np.ndarray] = None
|
||||
valid_mask: Optional[np.ndarray] = None
|
||||
frame_id: int = 1
|
||||
|
||||
|
||||
def _identity(generation: int = 1) -> FrameIdentity:
|
||||
return FrameIdentity("live", 1, 1, 0.1, generation=generation)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scene():
|
||||
return synthetic.poc_scene()
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- flow helper
|
||||
def test_the_flow_magnitude_handles_both_shapes():
|
||||
flow = np.zeros((2, 3, 2), dtype=np.float32)
|
||||
flow[..., 0] = 3.0
|
||||
flow[..., 1] = 4.0
|
||||
assert np.allclose(flow_magnitude(FakePacket(flow=flow)), 5.0)
|
||||
planar = np.arange(4, dtype=np.float32).reshape(2, 2)
|
||||
assert np.array_equal(flow_magnitude(FakePacket(flow=planar)), planar)
|
||||
assert flow_magnitude(FakePacket()) is None
|
||||
assert flow_magnitude(FakePacket(flow=np.zeros((2, 2, 3), dtype=np.float32))) is None
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- finger adapter
|
||||
def test_the_finger_adapter_publishes_a_field_already_in_the_domain(scene):
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
flow = np.zeros((*scene.shape_hw, 2), dtype=np.float32)
|
||||
flow[0, 0, 0] = 3.0
|
||||
frame = adapter.snapshot(FakePacket(flow=flow), runtime=None, identity=_identity())
|
||||
assert frame is not None
|
||||
assert validate_surface_frame(frame) == ()
|
||||
assert frame.geometry_id == scene.geometry_id
|
||||
assert frame.field_domain_id == scene.field_domain_id
|
||||
assert frame.scalar_descriptor == FLOW_MAGNITUDE
|
||||
assert frame.scalar_descriptor.unit == "px"
|
||||
assert frame.scalar_descriptor.quality == "approximate"
|
||||
assert np.asarray(frame.scalar)[0, 0] == 3.0
|
||||
assert np.asarray(frame.valid).all()
|
||||
assert not np.asarray(frame.scalar).flags["WRITEABLE"]
|
||||
assert frame.displacement_sensor_m is None, (
|
||||
"an uncalibrated displacement must not be invented"
|
||||
)
|
||||
|
||||
|
||||
def test_the_finger_adapter_remaps_through_the_unroll_map():
|
||||
map_x = np.array([[0, 1, 2], [0, 1, 2], [0, 1, 2]], dtype=np.float32)
|
||||
map_y = np.array([[2, 2, 2], [1, 1, 1], [0, 0, 0]], dtype=np.float32)
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id="g", field_domain_id="d", shape_hw=(3, 3),
|
||||
map_x=map_x, map_y=map_y,
|
||||
)
|
||||
flow = np.zeros((3, 3, 2), dtype=np.float32)
|
||||
flow[..., 0] = 2.0
|
||||
frame = adapter.snapshot(FakePacket(flow=flow), runtime=None, identity=_identity())
|
||||
assert frame is not None
|
||||
assert np.allclose(np.asarray(frame.scalar), 2.0)
|
||||
assert np.asarray(frame.valid).all()
|
||||
|
||||
|
||||
def test_a_remap_with_a_hole_is_invalid_there_and_does_not_read_as_zero():
|
||||
map_x = np.array([[0, -1], [0, 0]], dtype=np.float32)
|
||||
map_y = np.array([[0, -1], [0, 0]], dtype=np.float32)
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id="g", field_domain_id="d", shape_hw=(2, 2),
|
||||
map_x=map_x, map_y=map_y,
|
||||
)
|
||||
flow = np.full((2, 2, 2), 1.0, dtype=np.float32)
|
||||
frame = adapter.snapshot(FakePacket(flow=flow), runtime=None, identity=_identity())
|
||||
assert frame is not None
|
||||
assert np.asarray(frame.valid).tolist() == [[True, False], [True, True]]
|
||||
assert np.isnan(np.asarray(frame.scalar)[0, 1])
|
||||
|
||||
|
||||
def test_the_finger_adapter_refuses_a_grid_it_cannot_place(scene):
|
||||
"""No remap and the wrong grid: say which shapes were seen, and publish nothing."""
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
flow = np.zeros((40, 30, 2), dtype=np.float32)
|
||||
frame = adapter.snapshot(FakePacket(flow=flow), runtime=None, identity=_identity())
|
||||
assert frame is None
|
||||
item = adapter.last_diagnostic
|
||||
assert item is not None and item.code == "field_domain_mismatch"
|
||||
assert item.details["flow"] == "40x30"
|
||||
assert item.details["domain"] == f"{scene.shape_hw[0]}x{scene.shape_hw[1]}"
|
||||
|
||||
|
||||
def test_the_finger_adapter_without_flow_says_so(scene):
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
assert adapter.snapshot(FakePacket(), runtime=None, identity=_identity()) is None
|
||||
assert adapter.last_diagnostic is not None
|
||||
assert adapter.last_diagnostic.code == "field_unavailable"
|
||||
|
||||
|
||||
def test_the_finger_adapter_can_be_built_from_a_domain(scene):
|
||||
from orisys_app_shared.scene3d.legacy import load_legacy_domain
|
||||
|
||||
map_path = _run_map()
|
||||
if map_path is None:
|
||||
pytest.skip("the shipped unroll map is not present")
|
||||
domain = load_legacy_domain(map_path)
|
||||
adapter = FingerFrameAdapter.from_domain(domain, geometry_id="geom")
|
||||
assert adapter.shape_hw == domain.shape_hw
|
||||
assert adapter.field_domain_id == domain.domain_id
|
||||
assert adapter.map_x is not None and adapter.map_y is not None, (
|
||||
"the domain must carry the unroll map for the adapter to use"
|
||||
)
|
||||
assert domain.source_hw == (288, 512), domain.source_hw
|
||||
|
||||
|
||||
def _run_map():
|
||||
from pathlib import Path
|
||||
|
||||
path = Path(r"D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703\unroll_map_uv_s0.4.npz")
|
||||
return path if path.is_file() else None
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- box adapter
|
||||
def test_the_box_adapter_prefers_the_depth_response(scene):
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
depth = np.full(scene.shape_hw, 0.25, dtype=np.float32)
|
||||
flow = np.ones((*scene.shape_hw, 2), dtype=np.float32)
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(depth=depth, flow=flow),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, scene.shape_hw),
|
||||
)
|
||||
assert frame is not None
|
||||
assert frame.scalar_descriptor == DEPTH_RESPONSE
|
||||
assert frame.scalar_descriptor.unit == "1", "a deformation response is not a length"
|
||||
assert np.allclose(np.asarray(frame.scalar), 0.25)
|
||||
|
||||
|
||||
def test_the_box_adapter_falls_back_to_the_flow_magnitude(scene):
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
flow = np.zeros((*scene.shape_hw, 2), dtype=np.float32)
|
||||
flow[..., 0] = 1.0
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(flow=flow),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, scene.shape_hw),
|
||||
)
|
||||
assert frame is not None and frame.scalar_descriptor == FLOW_MAGNITUDE
|
||||
assert np.allclose(np.asarray(frame.scalar), 1.0)
|
||||
|
||||
|
||||
def test_the_device_mask_wins_over_the_finiteness_test(scene):
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
depth = np.ones(scene.shape_hw, dtype=np.float32)
|
||||
mask = np.ones(scene.shape_hw, dtype=bool)
|
||||
mask[0, 0] = False
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(depth=depth, valid_mask=mask),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, scene.shape_hw),
|
||||
)
|
||||
assert frame is not None
|
||||
assert not bool(np.asarray(frame.valid)[0, 0])
|
||||
assert np.asarray(frame.valid).sum() == mask.sum()
|
||||
|
||||
|
||||
def test_the_box_adapter_refuses_a_device_with_another_domain(scene):
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(depth=np.ones(scene.shape_hw, dtype=np.float32)),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, (9, 9)),
|
||||
)
|
||||
assert frame is None
|
||||
assert adapter.last_diagnostic is not None
|
||||
assert adapter.last_diagnostic.code == "field_domain_mismatch"
|
||||
|
||||
|
||||
def test_the_box_adapter_with_nothing_usable_says_what_it_saw(scene):
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(depth=np.ones((5, 5), dtype=np.float32)),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, scene.shape_hw),
|
||||
)
|
||||
assert frame is None
|
||||
assert adapter.last_diagnostic is not None
|
||||
assert adapter.last_diagnostic.details["depth"] == "5x5"
|
||||
|
||||
|
||||
def test_a_published_frame_must_not_share_memory_with_the_packet(scene):
|
||||
"""The producer keeps writing its arrays; the frame must not alias them."""
|
||||
adapter = BoxFrameAdapter(
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=scene.field_domain_id,
|
||||
shape_hw=scene.shape_hw,
|
||||
)
|
||||
depth = np.zeros(scene.shape_hw, dtype=np.float32)
|
||||
frame = adapter.snapshot(
|
||||
FakePacket(depth=depth),
|
||||
_identity(),
|
||||
ExpectedGeometry("box", "rev", scene.geometry_id, scene.field_domain_id, scene.shape_hw),
|
||||
)
|
||||
assert frame is not None
|
||||
assert not frame.scalar.flags["WRITEABLE"]
|
||||
with pytest.raises(ValueError):
|
||||
np.asarray(frame.scalar)[0, 0] = 5.0
|
||||
@@ -0,0 +1,314 @@
|
||||
"""Stage 3 acceptance for the frame schedule.
|
||||
|
||||
No Qt here on purpose: the coordinator's contract is about *threads and identity*,
|
||||
so a fake widget that records calls is a better witness than a real one. The rows
|
||||
that matter are the ones a screen cannot show you: a stale frame never overwriting a
|
||||
new scene, only one compose in flight, and a producer that never waits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.composer import SceneComposer
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
DisplaySettings,
|
||||
FieldDescriptor,
|
||||
FrameIdentity,
|
||||
SurfaceFieldFrame,
|
||||
freeze,
|
||||
)
|
||||
from orisys_studio.scene3d.coordinator import ThreeDCoordinator
|
||||
|
||||
|
||||
class FakeWidget:
|
||||
"""Records what the coordinator asks of the view."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.scenes = []
|
||||
self.frames = []
|
||||
self.settings = []
|
||||
self.superseded = []
|
||||
self.visible = True
|
||||
|
||||
def set_scene(self, assets) -> None:
|
||||
self.scenes.append(assets)
|
||||
|
||||
def present(self, frame) -> None:
|
||||
self.frames.append(frame)
|
||||
|
||||
def set_settings(self, settings) -> None:
|
||||
self.settings.append(settings)
|
||||
|
||||
def set_superseded_frames(self, count: int) -> None:
|
||||
self.superseded.append(int(count))
|
||||
|
||||
def setVisible(self, visible: bool) -> None: # noqa: N802 - Qt spelling
|
||||
self.visible = bool(visible)
|
||||
|
||||
|
||||
class CountingComposer(SceneComposer):
|
||||
"""A composer that counts calls and can be made slow, like a loaded CPU."""
|
||||
|
||||
def __init__(self, delay_s: float = 0.0) -> None:
|
||||
super().__init__()
|
||||
self.calls = 0
|
||||
self.delay_s = delay_s
|
||||
self.in_flight = 0
|
||||
self.max_in_flight = 0
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def compose(self, assets, frame, settings):
|
||||
with self._lock:
|
||||
self.calls += 1
|
||||
self.in_flight += 1
|
||||
self.max_in_flight = max(self.max_in_flight, self.in_flight)
|
||||
try:
|
||||
if self.delay_s:
|
||||
time.sleep(self.delay_s)
|
||||
return super().compose(assets, frame, settings)
|
||||
finally:
|
||||
with self._lock:
|
||||
self.in_flight -= 1
|
||||
|
||||
|
||||
def _frame(scene, *, generation=1, source="live-A", session=1, frame_id=1, value=1.0,
|
||||
domain=None, valid=True):
|
||||
scalar = np.full(scene.shape_hw, value, dtype=np.float32)
|
||||
mask = np.full(scene.shape_hw, valid, dtype=bool)
|
||||
return SurfaceFieldFrame(
|
||||
identity=FrameIdentity(source, session, frame_id, 0.1 * frame_id, generation),
|
||||
geometry_id=scene.geometry_id,
|
||||
field_domain_id=domain or scene.field_domain_id,
|
||||
scalar=freeze(scalar, dtype=np.float32),
|
||||
scalar_descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
|
||||
valid=freeze(mask, dtype=np.bool_),
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scene():
|
||||
return synthetic.poc_scene()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def pool():
|
||||
executor = ThreadPoolExecutor(max_workers=2, thread_name_prefix="compose")
|
||||
yield executor
|
||||
executor.shutdown(wait=True)
|
||||
|
||||
|
||||
def _drain(coordinator, ticks=50, sleep=0.005):
|
||||
for _ in range(ticks):
|
||||
coordinator.tick()
|
||||
time.sleep(sleep)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- state machine
|
||||
def test_the_view_starts_disabled_and_ignores_frames(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
assert coordinator.state == "disabled"
|
||||
assert coordinator.submit(_frame(scene)) == "disabled"
|
||||
coordinator.tick()
|
||||
assert widget.frames == []
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_activating_a_scene_bumps_the_generation_and_shows_the_reference_model(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
first = coordinator.activate(scene, "live-A")
|
||||
assert first == 1
|
||||
assert coordinator.generation == 1
|
||||
assert widget.scenes == [scene]
|
||||
assert len(widget.frames) == 1, "the reference model must be shown immediately"
|
||||
assert widget.frames[0].scalar_descriptor.quantity == "unavailable"
|
||||
second = coordinator.activate(scene, "live-B")
|
||||
assert second == 2 and coordinator.generation == 2
|
||||
# A frame from the previous scene is now old news.
|
||||
assert coordinator.submit(_frame(scene, generation=1, source="live-A")) == "stale"
|
||||
assert coordinator.stale_frames == 1
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_a_frame_from_another_session_is_stale_too(scene, pool):
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
assert coordinator.submit(_frame(scene, session=1)) == "accepted"
|
||||
assert coordinator.submit(_frame(scene, session=2)) == "stale"
|
||||
assert coordinator.stale_frames == 1
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_a_frame_on_another_domain_is_invalid_and_says_why(scene, pool):
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
result = coordinator.submit(_frame(scene, domain="somebody-elses-domain"))
|
||||
assert result == "invalid"
|
||||
assert coordinator.invalid_frames == 1
|
||||
assert coordinator.last_diagnostic is not None
|
||||
assert coordinator.last_diagnostic.code == "field_domain_mismatch"
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_the_newest_frame_replaces_the_one_that_was_never_composed(scene, pool):
|
||||
composer = CountingComposer(delay_s=0.05)
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), composer, pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
results = [coordinator.submit(_frame(scene, frame_id=index)) for index in range(5)]
|
||||
assert results[0] == "accepted"
|
||||
assert results[1:] == ["replaced"] * 4
|
||||
assert coordinator.superseded_frames == 4, (
|
||||
"display drops must be counted, and separately from recording drops"
|
||||
)
|
||||
coordinator.close()
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the schedule
|
||||
def test_a_tick_composes_and_presents(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene, value=3.0))
|
||||
_drain(coordinator)
|
||||
assert len(widget.frames) >= 2
|
||||
last = widget.frames[-1]
|
||||
assert np.allclose(np.asarray(last.scalar), 3.0)
|
||||
assert coordinator.state == "live"
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_a_frame_with_nothing_valid_is_no_data_not_a_stale_picture(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene, valid=False))
|
||||
_drain(coordinator)
|
||||
assert coordinator.state == "no_data"
|
||||
assert not np.asarray(widget.frames[-1].valid).any()
|
||||
assert np.isnan(np.asarray(widget.frames[-1].scalar)).all()
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_only_one_compose_is_in_flight_however_fast_the_ticks_come(scene, pool):
|
||||
"""A slow composer makes frames drop; it must not make the queue grow."""
|
||||
composer = CountingComposer(delay_s=0.02)
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), composer, pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
for index in range(200):
|
||||
coordinator.submit(_frame(scene, frame_id=index))
|
||||
coordinator.tick() # no sleep: ticks as fast as the GUI would
|
||||
_drain(coordinator, ticks=40, sleep=0.01)
|
||||
assert composer.max_in_flight == 1, (
|
||||
f"{composer.max_in_flight} composes ran at once: the schedule is queueing"
|
||||
)
|
||||
assert composer.calls <= 60, f"{composer.calls} composes for 200 frames"
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_a_busy_display_never_slows_the_producer(scene, pool):
|
||||
"""The acceptance row: submit stays fast while the composer is saturated."""
|
||||
composer = CountingComposer(delay_s=0.05)
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), composer, pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene, frame_id=0))
|
||||
coordinator.tick() # start a slow compose
|
||||
timings = []
|
||||
for index in range(200):
|
||||
started = time.perf_counter()
|
||||
coordinator.submit(_frame(scene, frame_id=index + 1))
|
||||
timings.append((time.perf_counter() - started) * 1000.0)
|
||||
timings.sort()
|
||||
p99 = timings[int(len(timings) * 0.99) - 1]
|
||||
assert p99 < 1.0, f"submit p99 was {p99:.3f} ms while the composer was busy"
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_a_result_that_arrives_after_a_scene_change_is_dropped(scene, pool):
|
||||
"""The compose that was running when the operator switched scenes is worthless."""
|
||||
composer = CountingComposer(delay_s=0.05)
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, composer, pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene, value=9.0))
|
||||
coordinator.tick() # starts the slow compose for generation 1
|
||||
coordinator.activate(scene, "live-A") # generation 2 while it runs
|
||||
before = len(widget.frames)
|
||||
_drain(coordinator, ticks=20, sleep=0.02)
|
||||
composed = [frame for frame in widget.frames[before:] if np.isfinite(np.asarray(frame.scalar)).any()]
|
||||
assert not composed, "a frame composed for the previous scene was presented"
|
||||
assert coordinator.stale_frames >= 0
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_hiding_suspends_without_touching_the_scene(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene))
|
||||
_drain(coordinator, ticks=10)
|
||||
presented = len(widget.frames)
|
||||
coordinator.set_visible(False)
|
||||
assert coordinator.state == "suspended"
|
||||
assert widget.visible is False
|
||||
assert coordinator.submit(_frame(scene)) == "disabled"
|
||||
coordinator.tick()
|
||||
assert len(widget.frames) == presented, "a hidden view was still being painted"
|
||||
coordinator.set_visible(True)
|
||||
assert coordinator.state == "ready"
|
||||
coordinator.submit(_frame(scene, frame_id=42))
|
||||
_drain(coordinator, ticks=10)
|
||||
assert len(widget.frames) > presented
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_changing_the_gain_recomposes_what_is_on_screen(scene, pool):
|
||||
composer = CountingComposer()
|
||||
coordinator = ThreeDCoordinator(FakeWidget(), composer, pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene))
|
||||
_drain(coordinator, ticks=10)
|
||||
settled = composer.calls
|
||||
# A colour-scale change needs no new composition...
|
||||
coordinator.set_settings(DisplaySettings(max_fps=30))
|
||||
_drain(coordinator, ticks=5)
|
||||
assert composer.calls == settled
|
||||
# ...but a gain change does: the positions depend on it.
|
||||
coordinator.set_settings(DisplaySettings(deform_enabled=True, display_gain=5.0))
|
||||
_drain(coordinator, ticks=10)
|
||||
assert composer.calls > settled, "the new gain was never applied"
|
||||
coordinator.close()
|
||||
|
||||
|
||||
def test_close_is_idempotent_and_stops_everything(scene, pool):
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "live-A")
|
||||
coordinator.submit(_frame(scene))
|
||||
_drain(coordinator, ticks=5)
|
||||
coordinator.close()
|
||||
coordinator.close() # idempotent
|
||||
assert coordinator.state == "disabled"
|
||||
assert coordinator.submit(_frame(scene)) == "disabled"
|
||||
presented = len(widget.frames)
|
||||
coordinator.tick()
|
||||
assert len(widget.frames) == presented
|
||||
@@ -88,21 +88,20 @@ def test_the_widget_holds_cpu_state_and_touches_no_context(qt_app):
|
||||
widget.deleteLater()
|
||||
|
||||
|
||||
def test_the_stubs_name_the_stage_that_fills_them_in(qt_app):
|
||||
def test_the_remaining_stub_says_which_stage_fills_it_in(qt_app):
|
||||
"""What is implemented is exercised elsewhere; what is not says so.
|
||||
|
||||
Stage 1 landed the camera, the dispatcher and the renderer (see
|
||||
``test_scene3d_stage1.py`` and ``test_scene3d_gl_smoke.py``). The remaining
|
||||
placeholders must still fail loudly with the stage that owns them rather than
|
||||
quietly doing nothing.
|
||||
Stages 1 to 3 landed the camera, the dispatcher, the renderer, the sampler, the
|
||||
composer, the schedule and the adapters. ``ThreeDPanel`` is the one piece left,
|
||||
and it must still fail loudly with the stage that owns it rather than quietly
|
||||
doing nothing.
|
||||
"""
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraPreset,
|
||||
DisplaySettings,
|
||||
ExpectedGeometry,
|
||||
FrameIdentity,
|
||||
RenderStats,
|
||||
)
|
||||
from orisys_studio.scene3d.adapters import BoxFrameAdapter, FingerFrameAdapter
|
||||
from orisys_studio.scene3d.camera import OrbitCamera
|
||||
from orisys_studio.scene3d.coordinator import ThreeDCoordinator
|
||||
from orisys_studio.scene3d.gl_backend import GLRenderer
|
||||
@@ -115,34 +114,26 @@ def test_the_stubs_name_the_stage_that_fills_them_in(qt_app):
|
||||
coordinator = ThreeDCoordinator(widget, composer=object(), executor=None, max_fps=30) # type: ignore[arg-type]
|
||||
panel = ThreeDPanel(coordinator, widget)
|
||||
try:
|
||||
assert coordinator.state == "disabled"
|
||||
assert coordinator.generation == 0
|
||||
assert coordinator.superseded_frames == 0 # the mailbox is real, not a stub
|
||||
|
||||
# Stage 1 is real: the camera works with no context at all.
|
||||
camera.set_preset(
|
||||
CameraPreset("default", (0.0, 0.0, 1.0), (0.0, 0.0, 0.0), (0.0, 1.0, 0.0), 0.7)
|
||||
)
|
||||
assert camera.matrices((100, 100)).view.shape == (4, 4)
|
||||
|
||||
# Stage 3 is real too: the coordinator's schedule exists with no Qt event.
|
||||
assert coordinator.state == "disabled"
|
||||
assert coordinator.generation == 0
|
||||
assert coordinator.superseded_frames == 0
|
||||
coordinator.close() # idempotent, and safe before any scene is active
|
||||
|
||||
for call, stage in (
|
||||
(lambda: coordinator.activate(None, "live-A"), "stage 3"),
|
||||
(lambda: coordinator.set_settings(DisplaySettings()), "stage 3"),
|
||||
(lambda: coordinator.set_visible(True), "stage 3"),
|
||||
(lambda: coordinator.close(), "stage 3"),
|
||||
(lambda: panel.set_state("ready"), "stage 4"),
|
||||
(lambda: BoxFrameAdapter().snapshot(None, FrameIdentity("a", 1, 0, 0.0, 1), None),
|
||||
"stage 3"),
|
||||
(
|
||||
lambda: FingerFrameAdapter().snapshot(
|
||||
None, None, FrameIdentity("a", 1, 0, 0.0, 1)
|
||||
),
|
||||
"stage 3",
|
||||
),
|
||||
(lambda: panel.set_render_stats(RenderStats()), "stage 4"),
|
||||
(lambda: panel.set_diagnostic(None), "stage 4"),
|
||||
):
|
||||
with pytest.raises(NotImplementedError) as excinfo:
|
||||
call()
|
||||
assert stage in str(excinfo.value), (stage, str(excinfo.value))
|
||||
del ExpectedGeometry
|
||||
del DisplaySettings, FrameIdentity
|
||||
finally:
|
||||
widget.deleteLater()
|
||||
|
||||
Reference in New Issue
Block a user