feat(studio-3d): 历史回放里的三维面板 —— 由「重算」通道喂它,并标注来源
- What: `VIEW_SHOWS_3D` 加入 `replay`;`app_window` 新增 `_shows_3d()`(可见性与布局共用一个判据) 与 `_splitter_sizes_for()`(预设表里的 3D 槽是"二维布局"的描述,面板打开时从最宽的面板借 `DEFAULT_3D_PANE_WIDTH=340`);`ThreeDPanel` 新增第三个来源标签(实时采集 / 回放:录像不含三维场 / 重算通道:由录像重算,非记录值)与 `set_source_kind()`;`ThreeDCoordinator.show_rest()` 把参考模型 放回屏幕并停止主张测量值(重算停止时调用,否则最后一帧会留在屏幕上);`Mesh3dPanel` 转发这两者, 并新增"加载中"状态的修正:切换传感器再切回来时若该包已加载,直接显示已就绪的画面而不是一条永远 不会被替换的"正在加载"。 - Why: 录像里没有三维场(raw_*.mp4 是原始相机图像,`depth.stored=false`,CSV 只有力值),所以纯回放只能 显示参考模型;但「从视频重算力值」本来就把同一个运行路径指向录像的原始视频,而采集线程上的三维 sink 早已注册——因此重算一开始面板就会收到真实的形变帧。这两件事必须由界面区分:一个是"没有测量", 一个是"当前算法对这段录像会说什么",把后者当成记录值展示就是另一种 `display_illustrative`。 - Verification: apps/studio 全量通过(test_scene3d_view.py 32 项,含"回放里面板可见且标注正确""重算时 标注切换""停止重算后清回参考模型""实时视图标为实时";test_scene3d_coordinator.py 15 项,含 show_rest 与"停止时在飞的帧不会随后被显示");ui_check 570 项 0 失败(新增"已安装包必须被加载并标注来源"); locale_check 0 缺失;仓库内工程包经 validate_scene 全部 ok。 - Not in this change: 逐帧保存展开场/深度(记录格式变更,设计文档明确否掉);重算通道本身(已存在,本次 只是把它的帧接到面板上);示意位移(仍需 SDK lift + 已确认尺度)。
This commit is contained in:
@@ -399,6 +399,12 @@
|
||||
"state_no_data": "No valid measurement in this frame — showing the surface model",
|
||||
"state_suspended": "Suspended (the view is hidden)",
|
||||
"state_unavailable": "3D is unavailable",
|
||||
"source_live": "Live acquisition",
|
||||
"source_live_tip": "The picture comes from the data being acquired right now.",
|
||||
"source_replay": "Replay: the recording stores no 3D field",
|
||||
"source_replay_tip": "A recording stores the raw camera image and the forces, not a per-frame deformation field. Turn on “Recompute forces from the video” and this pane will show the deformation recomputed from the recording.",
|
||||
"source_recompute": "Recomputed from the recording — not the recorded values",
|
||||
"source_recompute_tip": "This is what the current algorithm version says about the recorded footage: it is a check on the algorithm, not the measurement that was recorded at the time.",
|
||||
"severity_info": "Note",
|
||||
"severity_warning": "Warning",
|
||||
"severity_error": "Error",
|
||||
|
||||
@@ -399,6 +399,12 @@
|
||||
"state_no_data": "本帧没有有效测量 — 显示表面模型",
|
||||
"state_suspended": "已挂起(视图被隐藏)",
|
||||
"state_unavailable": "三维不可用",
|
||||
"source_live": "实时采集",
|
||||
"source_live_tip": "画面来自此刻正在采集的数据。",
|
||||
"source_replay": "回放:录像不含三维场",
|
||||
"source_replay_tip": "录像只保存原始相机图像与力值,没有逐帧的形变场。开启「从视频重算力值」后,本面板会由录像重算并把形变显示在这里。",
|
||||
"source_recompute": "重算通道:由录像重算,非记录值",
|
||||
"source_recompute_tip": "这是在当前算法版本下对录像重算得到的形变,用于检查算法;它不是当初记录下来的测量值。",
|
||||
"severity_info": "提示",
|
||||
"severity_warning": "警告",
|
||||
"severity_error": "错误",
|
||||
|
||||
Binary file not shown.
@@ -92,8 +92,17 @@ VIEW_SPLITTER_SIZES = {
|
||||
"replay": (0, 360, 1080, 0),
|
||||
"calibration": (0, 0, 560, 880),
|
||||
}
|
||||
#: The 3D panel only belongs to the live view, and only for a finger sensor.
|
||||
VIEW_SHOWS_3D = ("custom", "live")
|
||||
#: The 3D pane belongs to the live view and to history browsing, and only for a
|
||||
#: finger sensor. Replay is included because a *recompute* pass re-runs the finger
|
||||
#: chain over the recording's raw video and feeds the pane through the ordinary
|
||||
#: acquisition sink; a recording itself stores no 3D field (see
|
||||
#: ``Mesh3dPanel.set_source_kind`` and the stage-4/5 sections of
|
||||
#: ``docs/architecture/studio-3d-stages.md``).
|
||||
VIEW_SHOWS_3D = ("custom", "live", "replay")
|
||||
#: Width the 3D pane takes from the widest other panel when a preset's table gives it
|
||||
#: nothing. The table describes the two-dimensional layout; the pane is optional, and
|
||||
#: "not in the table" must not mean "standing at its minimum width".
|
||||
DEFAULT_3D_PANE_WIDTH = 340
|
||||
#: After this long in ``connecting``, tell the user that Stop is the way out.
|
||||
CONNECT_SLOW_HINT_S = 8.0
|
||||
#: Playback tick while a recording is playing. 33 ms is one frame at 30 fps; the
|
||||
@@ -1059,6 +1068,9 @@ class StudioMainWindow(QMainWindow):
|
||||
self.toolbar.set_video_path(str(segment))
|
||||
self._refresh_status_rail()
|
||||
self._recompute_running = True
|
||||
# The 3D pane now says "recomputed", not "replayed": the pass is what
|
||||
# feeds it, through the ordinary acquisition sink.
|
||||
self.mesh3d_panel.set_source_kind(self._current_source_kind())
|
||||
# Playing is what starts the pass: one path in, so the button and the
|
||||
# menu cannot start it twice.
|
||||
self.toolbar.set_run_checked(True)
|
||||
@@ -1067,6 +1079,11 @@ class StudioMainWindow(QMainWindow):
|
||||
return
|
||||
if self._recompute_running:
|
||||
self._stop_run(reason="user_stop")
|
||||
# The pass is over, so the last recomputed frame must go: leaving it on
|
||||
# screen would show a measurement nothing is producing any more.
|
||||
self.mesh3d_panel.show_reference_only()
|
||||
if self._view_preset == "replay":
|
||||
self.mesh3d_panel.set_source_kind(self._current_source_kind())
|
||||
if self._view_preset == "replay" and self.replay.is_loaded():
|
||||
self._apply_replay_source(True)
|
||||
self._render_replay_frame()
|
||||
@@ -1326,6 +1343,40 @@ class StudioMainWindow(QMainWindow):
|
||||
)
|
||||
self._apply_arrow_rotation_for_sensor()
|
||||
|
||||
def _current_source_kind(self) -> str:
|
||||
"""Where the 3D pane's field would come from in the current view.
|
||||
|
||||
``recompute`` only while a pass is actually running: the label is a claim
|
||||
about the pixels on screen, and a stale claim is worse than none.
|
||||
"""
|
||||
if self._view_preset == "replay":
|
||||
return "recompute" if self._recompute_running else "replay"
|
||||
return "live"
|
||||
|
||||
def _shows_3d(self) -> bool:
|
||||
"""Whether the 3D pane belongs to the current view and sensor.
|
||||
|
||||
One predicate for both the pane's visibility and the splitter layout: the
|
||||
layout has to know the same thing the visibility does, or the pane ends up
|
||||
visible and standing at its minimum width.
|
||||
"""
|
||||
return (
|
||||
self._view_preset in VIEW_SHOWS_3D
|
||||
and self.sensor_picker.sensor_type() == "finger"
|
||||
and bool(self._show_finger_3d)
|
||||
)
|
||||
|
||||
def _splitter_sizes_for(self, preset: str) -> list:
|
||||
"""The preset's layout, with room for the 3D pane when it is on."""
|
||||
sizes = list(VIEW_SPLITTER_SIZES.get(preset, DEFAULT_SPLITTER_SIZES))
|
||||
if self._shows_3d() and sizes and sizes[0] == 0:
|
||||
width = DEFAULT_3D_PANE_WIDTH
|
||||
widest = max(range(1, len(sizes)), key=lambda index: sizes[index])
|
||||
take = min(width, max(sizes[widest] - MIN_PANEL_WIDTHS["plots"], 0))
|
||||
sizes[0] = take
|
||||
sizes[widest] = sizes[widest] - take
|
||||
return sizes
|
||||
|
||||
def _apply_panel_visibility(self) -> None:
|
||||
"""One place decides which panels the current view shows.
|
||||
|
||||
@@ -1336,7 +1387,7 @@ class StudioMainWindow(QMainWindow):
|
||||
"""
|
||||
is_finger = self.sensor_picker.sensor_type() == "finger"
|
||||
preset = self._view_preset
|
||||
show_3d = preset in VIEW_SHOWS_3D and is_finger and bool(self._show_finger_3d)
|
||||
show_3d = self._shows_3d()
|
||||
show_arrow = preset != "calibration"
|
||||
show_cal = self._show_cal if preset == "custom" else preset == "calibration"
|
||||
self._show_cal = show_cal
|
||||
@@ -1346,6 +1397,9 @@ class StudioMainWindow(QMainWindow):
|
||||
self.mesh3d_panel.ensure_view()
|
||||
self.mesh3d_panel.set_enabled(show_3d)
|
||||
self.mesh3d_panel.setVisible(show_3d)
|
||||
# Where the field comes from: a live view acquires it, a replay has none
|
||||
# stored and only a recompute pass produces one.
|
||||
self.mesh3d_panel.set_source_kind(self._current_source_kind())
|
||||
self.arrow_panel.setVisible(show_arrow)
|
||||
self.force_plots.setVisible(True)
|
||||
self.cal_panel.setVisible(show_cal)
|
||||
@@ -1404,7 +1458,7 @@ class StudioMainWindow(QMainWindow):
|
||||
self.toolbar.set_run_role("playback" if preset == "replay" else "acquire")
|
||||
self._apply_panel_visibility()
|
||||
if apply_sizes and preset in VIEW_SPLITTER_SIZES:
|
||||
self.content_splitter.setSizes(list(VIEW_SPLITTER_SIZES[preset]))
|
||||
self.content_splitter.setSizes(self._splitter_sizes_for(preset))
|
||||
if preset == "replay":
|
||||
# History browsing is the point of this view: the charts read the
|
||||
# open recording, and the chart cursor is its timeline. Without a
|
||||
|
||||
@@ -278,6 +278,24 @@ class ThreeDCoordinator:
|
||||
# the last field frame has to be composed again.
|
||||
self._settings_dirty = True
|
||||
|
||||
def show_rest(self) -> None:
|
||||
"""Put the reference model back on screen and stop claiming a measurement.
|
||||
|
||||
Needed by the replay view: while a recompute pass runs it feeds this
|
||||
coordinator through the ordinary acquisition sink, and when the pass stops the
|
||||
last recomputed frame would otherwise stay on screen — a picture of a
|
||||
measurement that is no longer being made, which is exactly what the rest frame
|
||||
exists to prevent. Idempotent, and safe before any scene is active.
|
||||
"""
|
||||
if self._closed or self._assets is None or not self._visible:
|
||||
return
|
||||
self._incoming.clear()
|
||||
self._cancel_inflight()
|
||||
self._last_field_frame = None
|
||||
self._last_composed_identity = None
|
||||
self._widget.present(self._composer.rest_frame(self._assets, self._identity(0)))
|
||||
self._set_state("no_data")
|
||||
|
||||
def close(self) -> None:
|
||||
"""Cancel work, stop the timer, clear both mailboxes. Idempotent."""
|
||||
if self._closed:
|
||||
|
||||
@@ -81,6 +81,7 @@ class ThreeDPanel(QFrame):
|
||||
self._assets: Optional[SceneAssets] = None
|
||||
self._descriptor: Optional[FieldDescriptor] = None
|
||||
self._state: SceneState = "disabled"
|
||||
self._source_kind = "live"
|
||||
self._syncing = False
|
||||
|
||||
root = QVBoxLayout(self)
|
||||
@@ -184,6 +185,16 @@ class ThreeDPanel(QFrame):
|
||||
self.legend_label.setMinimumWidth(1)
|
||||
root.addWidget(self.legend_label)
|
||||
|
||||
#: Which *source* the field on screen came from. A separate line from the
|
||||
#: scene state, because they answer different questions: the state says whether
|
||||
#: anything is being measured, this says whether it was acquired live or
|
||||
#: recomputed from a recording — a distinction the design document insists on.
|
||||
self.source_label = QLabel()
|
||||
self.source_label.setObjectName("panelMeta")
|
||||
self.source_label.setWordWrap(True)
|
||||
self.source_label.setMinimumWidth(1)
|
||||
root.addWidget(self.source_label)
|
||||
|
||||
self.state_label = QLabel()
|
||||
self.state_label.setObjectName("hintText")
|
||||
self.state_label.setWordWrap(True)
|
||||
@@ -246,6 +257,26 @@ class ThreeDPanel(QFrame):
|
||||
self.state_label.setText(t(f"scene3d.state_{state}"))
|
||||
self.state_label.setVisible(state != "live")
|
||||
|
||||
def set_source_kind(self, kind: str) -> None:
|
||||
"""Say where the field comes from: ``live``, ``replay`` or ``recompute``.
|
||||
|
||||
Three states, and the difference between the last two is the point of the
|
||||
whole replay story:
|
||||
|
||||
* ``live`` — acquired from the device now;
|
||||
* ``replay`` — a recording is open and **no** 3D field is stored in it, so the
|
||||
pane is showing the surface model only;
|
||||
* ``recompute`` — a recompute pass is running over the recording's video, so
|
||||
what is on screen is what *this* algorithm version says about that footage.
|
||||
It is not the recorded measurement, and the label says so.
|
||||
"""
|
||||
kind = str(kind)
|
||||
if kind not in ("live", "replay", "recompute"):
|
||||
kind = "live"
|
||||
self._source_kind = kind
|
||||
self.source_label.setText(t(f"scene3d.source_{kind}"))
|
||||
self.source_label.setToolTip(t(f"scene3d.source_{kind}_tip"))
|
||||
|
||||
def set_diagnostic(self, diagnostic: Optional[Diagnostic]) -> None:
|
||||
"""One diagnostic line. The 2D views stay usable behind it."""
|
||||
if diagnostic is None:
|
||||
@@ -383,4 +414,6 @@ class ThreeDPanel(QFrame):
|
||||
self.fps_label.setToolTip(t("scene3d.max_fps_tip"))
|
||||
self.reset_button.setText(t("scene3d.reset_view"))
|
||||
self.state_label.setText(t(f"scene3d.state_{self._state}"))
|
||||
self.source_label.setText(t(f"scene3d.source_{self._source_kind}"))
|
||||
self.source_label.setToolTip(t(f"scene3d.source_{self._source_kind}_tip"))
|
||||
self.set_descriptor(self._descriptor)
|
||||
|
||||
@@ -310,6 +310,23 @@ class Scene3DView(QWidget):
|
||||
self._camera.reset(self._assets.bounds_sensor_m)
|
||||
self._widget.update()
|
||||
|
||||
def set_source_kind(self, kind: str) -> None:
|
||||
"""Tell the pane where its field comes from (see ``ThreeDPanel``)."""
|
||||
if self._panel is not None:
|
||||
self._panel.set_source_kind(kind)
|
||||
|
||||
def show_reference_only(self) -> None:
|
||||
"""Drop whatever measurement is on screen and show the surface model.
|
||||
|
||||
Used when a recompute pass over a recording stops: without this the last
|
||||
recomputed frame would stay on screen while nothing is measuring it any more.
|
||||
"""
|
||||
if self._coordinator is not None:
|
||||
self._coordinator.show_rest()
|
||||
if self._panel is not None:
|
||||
self._panel.set_descriptor(None)
|
||||
self._panel.set_state("no_data")
|
||||
|
||||
# ------------------------------------------------------------------ state
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
|
||||
@@ -70,8 +70,12 @@ class Mesh3dPanel(QFrame):
|
||||
self._source: Optional[SceneSource] = None
|
||||
self._sensor_type = ""
|
||||
self._expected: Optional[ExpectedGeometry] = None
|
||||
self._source_kind = "live"
|
||||
self._enabled = False
|
||||
self._loaded = False
|
||||
#: The package whose scene is in the view, so switching sensor away and back
|
||||
#: does not reload it (and does not pretend to be loading it either).
|
||||
self._loaded_path: Optional[Path] = None
|
||||
self._reported: Optional[Diagnostic] = None
|
||||
self._pending_settings = None
|
||||
self._message_kind = _UNAVAILABLE
|
||||
@@ -121,11 +125,22 @@ class Mesh3dPanel(QFrame):
|
||||
return
|
||||
self._source = find_package(self._sensor_type, override=package_override)
|
||||
if self._source is None:
|
||||
self._loaded_path = None
|
||||
self._reported = no_asset_diagnostic(self._sensor_type, override=package_override)
|
||||
self._show_message(
|
||||
_NO_ASSET, directory=str(self._reported.details["directory"])
|
||||
)
|
||||
return
|
||||
if (
|
||||
self._view is not None
|
||||
and self._loaded_path is not None
|
||||
and self._source.path == self._loaded_path
|
||||
):
|
||||
# This scene is already loaded (the sensor was switched away and back):
|
||||
# showing "loading…" again would leave a line nothing ever replaces, while
|
||||
# the view behind it is ready.
|
||||
self._stack.setCurrentWidget(self._view)
|
||||
return
|
||||
self._show_message(_LOADING, name=self._source.path.name)
|
||||
|
||||
def ensure_view(self) -> Optional[QWidget]:
|
||||
@@ -141,6 +156,10 @@ class Mesh3dPanel(QFrame):
|
||||
view.enabledChanged.connect(self._on_view_enabled)
|
||||
if self._pending_settings is not None and hasattr(view, "restore_settings"):
|
||||
view.restore_settings(self._pending_settings)
|
||||
# The kind was decided before the view existed (the window sets it when the
|
||||
# view preset changes), so it is pushed again here rather than lost.
|
||||
if hasattr(view, "set_source_kind"):
|
||||
view.set_source_kind(self._source_kind)
|
||||
if not view.available:
|
||||
self._show_message(_UNAVAILABLE, reason=view.reason)
|
||||
return view
|
||||
@@ -186,6 +205,7 @@ class Mesh3dPanel(QFrame):
|
||||
return
|
||||
if not view.use_scene(assets, source_id=description, expected=self._expected):
|
||||
return
|
||||
self._loaded_path = None if self._source is None else self._source.path
|
||||
self._stack.setCurrentWidget(view)
|
||||
# Adopting a scene is not the same as showing one: the pane stays off until
|
||||
# the operator (or the window, restoring a preference) turns it on.
|
||||
@@ -193,6 +213,7 @@ class Mesh3dPanel(QFrame):
|
||||
|
||||
def _on_load_failed(self, item: Diagnostic) -> None:
|
||||
self._reported = item
|
||||
self._loaded_path = None
|
||||
self._show_message(_REJECTED, message=item.message)
|
||||
|
||||
def _on_diagnostic(self, item: Diagnostic) -> None:
|
||||
@@ -239,6 +260,17 @@ class Mesh3dPanel(QFrame):
|
||||
def scene_path(self) -> Optional[Path]:
|
||||
return None if self._source is None else self._source.path
|
||||
|
||||
def set_source_kind(self, kind: str) -> None:
|
||||
"""Where the field comes from: live, replay (model only) or recompute."""
|
||||
self._source_kind = str(kind)
|
||||
if self._view is not None and hasattr(self._view, "set_source_kind"):
|
||||
self._view.set_source_kind(self._source_kind)
|
||||
|
||||
def show_reference_only(self) -> None:
|
||||
"""Clear a measurement that is no longer being produced (a stopped pass)."""
|
||||
if self._view is not None and hasattr(self._view, "show_reference_only"):
|
||||
self._view.show_reference_only()
|
||||
|
||||
def tick(self) -> None:
|
||||
"""Advance the pane. Called by the window's run timer, never by itself."""
|
||||
if self._view is not None and hasattr(self._view, "tick"):
|
||||
|
||||
@@ -312,3 +312,50 @@ def test_close_is_idempotent_and_stops_everything(scene, pool):
|
||||
presented = len(widget.frames)
|
||||
coordinator.tick()
|
||||
assert len(widget.frames) == presented
|
||||
|
||||
|
||||
# ------------------------------------------------- the reference model, on demand
|
||||
def test_show_rest_puts_the_model_back_and_stops_claiming_a_measurement(scene, pool):
|
||||
"""The replay view's case: a pass stops, so the last frame must go.
|
||||
|
||||
Without this the recomputed picture would stay on screen while nothing is
|
||||
producing it — a measurement that is no longer being made.
|
||||
"""
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "replay-A")
|
||||
coordinator.submit(_frame(scene, source="replay-A"))
|
||||
_drain(coordinator, ticks=10)
|
||||
assert coordinator.state in ("live", "no_data")
|
||||
assert widget.frames[-1].valid.any(), "the measurement never arrived"
|
||||
|
||||
coordinator.show_rest()
|
||||
last = widget.frames[-1]
|
||||
assert not last.valid.any(), "the old measurement is still on screen"
|
||||
assert last.scalar_descriptor.quantity == "unavailable"
|
||||
assert coordinator.state == "no_data"
|
||||
|
||||
# And it is a no-op when there is nothing to show or the view is off.
|
||||
coordinator.set_visible(False)
|
||||
before = len(widget.frames)
|
||||
coordinator.show_rest()
|
||||
assert len(widget.frames) == before
|
||||
coordinator.close()
|
||||
coordinator.show_rest() # closed: still safe
|
||||
assert coordinator.state == "disabled"
|
||||
|
||||
|
||||
def test_a_frame_already_queued_is_dropped_when_the_model_comes_back(scene, pool):
|
||||
"""The rest frame is not "the newest sample": a queued one must not follow it."""
|
||||
widget = FakeWidget()
|
||||
coordinator = ThreeDCoordinator(widget, CountingComposer(delay_s=0.02), pool)
|
||||
coordinator.set_visible(True)
|
||||
coordinator.activate(scene, "replay-A")
|
||||
coordinator.submit(_frame(scene, source="replay-A", value=7.0))
|
||||
coordinator.show_rest()
|
||||
_drain(coordinator, ticks=4, sleep=0.01)
|
||||
assert not widget.frames[-1].valid.any(), (
|
||||
"a frame that was in flight when the pass stopped was presented anyway"
|
||||
)
|
||||
coordinator.close()
|
||||
|
||||
@@ -37,6 +37,7 @@ from orisys_app_shared.scene3d.contracts import (
|
||||
UNAVAILABLE_DESCRIPTOR,
|
||||
)
|
||||
from orisys_app_shared.scene3d.package import write_package
|
||||
from orisys_studio.app_window import MIN_PANEL_WIDTHS
|
||||
from orisys_studio.scene3d import prefs as scene3d_prefs
|
||||
from orisys_studio.scene3d.asset_source import (
|
||||
PACKAGE_SUFFIX,
|
||||
@@ -45,7 +46,12 @@ from orisys_studio.scene3d.asset_source import (
|
||||
load_package,
|
||||
no_asset_diagnostic,
|
||||
)
|
||||
from tests.test_ui_smoke import _finish, _running_window
|
||||
from tests.test_ui_smoke import ( # noqa: F401 (fixtures are imported by name)
|
||||
_finish,
|
||||
_running_window,
|
||||
record_root,
|
||||
replay_recording,
|
||||
)
|
||||
|
||||
#: Scratch root for the two-session comparison. One subdirectory per run, because
|
||||
#: the point of the test is that two recordings are identical, not that one file
|
||||
@@ -112,9 +118,18 @@ def test_a_box_sensor_reports_that_it_has_no_3d_scene(qt_app):
|
||||
panel.deleteLater()
|
||||
|
||||
|
||||
def test_a_missing_package_names_where_it_looked(tmp_path, qt_app):
|
||||
def test_a_missing_package_names_where_it_looked(tmp_path, qt_app, monkeypatch):
|
||||
"""With no package anywhere, the pane names the directory it looked in.
|
||||
|
||||
The search is stubbed rather than relied on: the repository may or may not carry
|
||||
a compiled package (the application's own config directory is where one belongs),
|
||||
and a test that only passes on a machine without one is a test that fails the
|
||||
first time somebody installs the feature.
|
||||
"""
|
||||
import orisys_studio.ui.mesh3d_panel as panel_module
|
||||
from orisys_studio.ui.mesh3d_panel import Mesh3dPanel
|
||||
|
||||
monkeypatch.setattr(panel_module, "find_package", lambda *a, **k: None)
|
||||
panel = Mesh3dPanel()
|
||||
try:
|
||||
panel.set_sensor("finger", package_override=tmp_path / "nope.oryscene")
|
||||
@@ -164,17 +179,21 @@ def test_the_pane_fits_the_width_the_window_reserves_for_it(qt_app):
|
||||
view.close()
|
||||
|
||||
|
||||
def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(qt_app):
|
||||
def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(
|
||||
write_poc, qt_app
|
||||
):
|
||||
"""The offscreen plugin cannot create a context; that must not look like "no data".
|
||||
|
||||
Qt reports this by never calling ``initializeGL``, so the widget asks Qt the one
|
||||
question that has an answer — is there a context at all — on its first paint.
|
||||
question that has an answer — is there a context at all — on its first paint. The
|
||||
scene comes from a package written for this test, so the case does not depend on
|
||||
whether the repository happens to carry a compiled one.
|
||||
"""
|
||||
from orisys_studio.ui.mesh3d_panel import Mesh3dPanel
|
||||
|
||||
panel = Mesh3dPanel()
|
||||
try:
|
||||
panel.set_sensor("finger")
|
||||
panel.set_sensor("finger", package_override=write_poc())
|
||||
view = panel.ensure_view()
|
||||
if view is None:
|
||||
pytest.skip("the pane did not build a view here")
|
||||
@@ -184,7 +203,10 @@ def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(q
|
||||
panel.resize(240, 400)
|
||||
panel.show()
|
||||
view.show()
|
||||
qt_app.processEvents()
|
||||
panel.set_enabled(True)
|
||||
assert _pump(qt_app, lambda: view.assets is not None, timeout_s=15.0), (
|
||||
"the test package never loaded"
|
||||
)
|
||||
widget = view.gl_widget
|
||||
for _ in range(5):
|
||||
widget.repaint()
|
||||
@@ -584,6 +606,73 @@ def test_a_recording_says_what_the_3d_pane_was(
|
||||
window.close()
|
||||
|
||||
|
||||
def test_the_replay_view_shows_the_pane_and_says_where_the_field_comes_from(
|
||||
make_window, replay_recording, qt_app
|
||||
):
|
||||
"""History browsing gets the pane, fed by the recompute pass — and says so.
|
||||
|
||||
A recording stores the raw camera image and the forces, not a per-frame
|
||||
deformation field, so the pane in replay is either the surface model (no field) or
|
||||
what a recompute pass produces from the video. Those are two different claims and
|
||||
the label has to distinguish them.
|
||||
"""
|
||||
from tests.test_ui_smoke import _show
|
||||
|
||||
window = _show(make_window(sensor_type="finger"), (1440, 880))
|
||||
try:
|
||||
assert window._load_recording(Path(replay_recording))
|
||||
assert window._view_preset == "replay"
|
||||
pane = window.mesh3d_panel
|
||||
assert pane.isVisible(), "the 3D pane is hidden in the replay view"
|
||||
assert pane.enabled(), "the pane is visible but not switched on"
|
||||
assert window.content_splitter.sizes()[0] >= MIN_PANEL_WIDTHS["mesh3d"], (
|
||||
"the replay layout left the pane at zero width"
|
||||
)
|
||||
view = pane.view
|
||||
if view is None or view.panel is None:
|
||||
pytest.skip("no 3D view on this platform")
|
||||
# Wait for the scene to load, then read what the pane claims.
|
||||
assert _pump(qt_app, lambda: view.assets is not None, timeout_s=15.0), (
|
||||
"the scene never loaded: " + str(pane.last_diagnostic)
|
||||
)
|
||||
label = view.panel.source_label.text()
|
||||
assert "回放" in label or "Replay" in label, label
|
||||
|
||||
# A running pass means the pixels are recomputed, and the label must say so.
|
||||
window._recompute_running = True
|
||||
window._apply_panel_visibility()
|
||||
label = view.panel.source_label.text()
|
||||
assert "重算" in label or "Recomputed" in label, label
|
||||
|
||||
# Stopping the pass must clear the picture, not leave the last frame up.
|
||||
window._recompute_running = False
|
||||
pane.show_reference_only()
|
||||
qt_app.processEvents()
|
||||
assert view.coordinator.state in ("no_data", "ready", "disabled")
|
||||
assert "没有测量值" in view.panel.legend_label.text() or (
|
||||
"No measurement" in view.panel.legend_label.text()
|
||||
), view.panel.legend_label.text()
|
||||
window._apply_panel_visibility()
|
||||
label = view.panel.source_label.text()
|
||||
assert "回放" in label or "Replay" in label, label
|
||||
finally:
|
||||
window.close()
|
||||
|
||||
|
||||
def test_the_live_view_still_says_live(make_window, qt_app):
|
||||
window = make_window(sensor_type="finger")
|
||||
try:
|
||||
window._apply_panel_visibility()
|
||||
view = window.mesh3d_panel.view
|
||||
if view is None or view.panel is None:
|
||||
pytest.skip("no 3D view on this platform")
|
||||
label = view.panel.source_label.text()
|
||||
assert "实时" in label or "Live" in label, label
|
||||
assert view.panel.source_label.toolTip()
|
||||
finally:
|
||||
window.close()
|
||||
|
||||
|
||||
def test_a_platform_locked_scene_is_refused_in_the_application(qt_app):
|
||||
"""A lock violation is a *pane* failure: the view refuses the scene and says why."""
|
||||
from orisys_app_shared.scene3d.contracts import PlatformInfo, PlatformLock
|
||||
|
||||
@@ -2830,9 +2830,19 @@ def main() -> int:
|
||||
pane.set_sensor("box")
|
||||
check(pane.showing_message, "the 3D pane kept showing a picture for a box sensor")
|
||||
|
||||
# A finger with no package names where the package belongs and how to build one.
|
||||
pane.set_sensor("finger")
|
||||
diagnostic = pane.last_diagnostic
|
||||
# A finger *without* any package names where one belongs and how to build it.
|
||||
# The search is stubbed for this check: the application's config directory may
|
||||
# or may not carry a compiled package on the machine running the check, and a
|
||||
# check that only passes without one would fail the first time one is
|
||||
# installed (the positive case is checked just below).
|
||||
import orisys_studio.ui.mesh3d_panel as _panel_module
|
||||
|
||||
real_find = _panel_module.find_package
|
||||
_panel_module.find_package = lambda *a, **k: None
|
||||
try:
|
||||
fresh_missing = _Mesh3dPanel()
|
||||
fresh_missing.set_sensor("finger")
|
||||
diagnostic = fresh_missing.last_diagnostic
|
||||
check(
|
||||
diagnostic is not None and diagnostic.code == "asset_rejected",
|
||||
f"a missing 3D scene was not reported as an asset problem: {diagnostic}",
|
||||
@@ -2843,10 +2853,44 @@ def main() -> int:
|
||||
"the missing-scene report does not say how to build one",
|
||||
)
|
||||
check(
|
||||
str(diagnostic.details.get("directory", "")) in pane.message.text()
|
||||
or pane.message.text().strip(),
|
||||
str(diagnostic.details.get("directory", ""))
|
||||
in fresh_missing.message.text()
|
||||
or fresh_missing.message.text().strip() != "",
|
||||
"the pane does not name the directory it looked in",
|
||||
)
|
||||
fresh_missing.deleteLater()
|
||||
finally:
|
||||
_panel_module.find_package = real_find
|
||||
|
||||
# And when a package *is* installed, the pane must actually load it: this is
|
||||
# the end-to-end check that the search path, the loader and the widget agree.
|
||||
pane.set_sensor("finger")
|
||||
shipped = _panel_module.find_package("finger")
|
||||
if shipped is not None:
|
||||
view_for_load = pane.ensure_view()
|
||||
pane.set_enabled(True)
|
||||
deadline = time.time() + 15.0
|
||||
while time.time() < deadline:
|
||||
app.processEvents()
|
||||
assets = getattr(view_for_load, "assets", None) if view_for_load else None
|
||||
if assets is not None:
|
||||
break
|
||||
time.sleep(0.02)
|
||||
check(
|
||||
getattr(view_for_load, "assets", None) is not None,
|
||||
f"the installed 3D scene was never loaded: {pane.last_diagnostic}",
|
||||
)
|
||||
check(
|
||||
not pane.showing_message,
|
||||
"the pane is still showing a message although a scene loaded",
|
||||
)
|
||||
if view_for_load is not None and view_for_load.panel is not None:
|
||||
source_text = view_for_load.panel.source_label.text()
|
||||
check(source_text.strip() != "", "the pane does not say where the field comes from")
|
||||
check(
|
||||
"实时" in source_text or "Live" in source_text,
|
||||
f"the live view does not label its source: {source_text!r}",
|
||||
)
|
||||
|
||||
# The pane's controls must fit the width the window reserves for it.
|
||||
view = pane.ensure_view()
|
||||
|
||||
@@ -234,21 +234,37 @@ shared 全部 **262 项**。`ui_check.py` **566 项检查 0 失败**(新增 "3
|
||||
|
||||
- 运行时**只**读 `.oryscene` 包;旧的 Blender 导出目录仍由 `LegacyFingerImporter`(离线工具/测试)导入,
|
||||
单位与变换必须显式给出。包缺失/被拒/无 GL 时:面板一行文字说明原因,采集、记录、力曲线、标定、回放全部照常
|
||||
(`ui_check` 566 项与 `test_ui_smoke.py` 全绿即为证)。
|
||||
(`ui_check` 570 项与 `test_ui_smoke.py` 全绿即为证)。
|
||||
- 仓库现在**带一个已编译的工程包**:`apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene`
|
||||
(明文、未签名、即 `v1` 封套,1032 顶点;生成命令与单位依据写在 `tools/scene_assets/README.md`)。
|
||||
它只是工程用包,发布时用同一工具换成签名/加密/锁定的版本即可,加载路径不变。
|
||||
- `gl_profile`(`desktop33` 默认 / `gles30`)在 `QApplication` 之前生效,一进程一个;未知取值退回默认而不是
|
||||
让窗口起不来。
|
||||
- 3D 的显示设置(放大、色标、外壳、显示帧率)持久化在 `scene3d_display`;**色标的 descriptor 不恢复**,
|
||||
永远来自当前帧,因此一个旧文件不可能把测量标成另一种量。
|
||||
|
||||
**明确未做(留给阶段 5 或按用户要求暂缓)**:
|
||||
|
||||
- **示意位移(`displacement_sensor_m`)仍未接入**:需要 SDK 的 lift 与一个已确认的空间尺度;在拿到
|
||||
"这个尺度是怎么标定出来的"之前,适配器继续只着色不位移(阶段 3 的原则不变)。
|
||||
- ARM 板卡适配:按用户要求暂缓;`probe_gl.py` 可在板上直接跑出报告,据此定 `gl_profile` 与
|
||||
`MIN_*`/预算(阶段 5 的两平台报告)。
|
||||
- `session.json` 的 `visualization3d` 能力块、包签名/AES-GCM、性能与长时报告:阶段 5。
|
||||
- 3D 包尚未随仓库提供(仓库里没有 `.oryscene`):这是**有意的**,包是发布产物(阶段 5 管签名与平台锁定),
|
||||
并且它必须携带只能由人声明的单位与变换。
|
||||
- `session.json` 的 `visualization3d` 能力块、包签名/AES-GCM、性能与长时报告:阶段 5(已完成)。
|
||||
|
||||
**后续补充(同一功能,历史回放接入)**:
|
||||
|
||||
`VIEW_SHOWS_3D` 原先只有 `custom`/`live`。现在**回放也能打开三维面板**,但要分清两件事,
|
||||
因为这是两种不同的主张:
|
||||
|
||||
- 录像里**没有**三维场:`raw_*.mp4` 是原始相机图像,`session.json` 里 `depth.stored=false` 是当初定的,
|
||||
CSV 只有力值。所以纯回放时面板只显示**参考模型**,并写明"录像不含三维场"。
|
||||
- 打开「从视频重算力值」后,重算通道本来就是把**同一个运行路径**指向录像的原始视频(手指 worker 支持文件源),
|
||||
而采集线程上的三维 sink 早已注册(`_start_run`),所以面板会**自动**收到重算出来的帧。
|
||||
此时面板标注"重算通道:由录像重算,非记录值"——那是"这个算法版本对这段录像会说什么",
|
||||
不是当初记录到的测量值(和 `display_illustrative` 同样的思路)。
|
||||
|
||||
实现要点:`ThreeDCoordinator.show_rest()`(把参考模型放回屏幕并停止主张测量值)、
|
||||
`ThreeDPanel.set_source_kind()`(第三个来源标签)、`app_window._shows_3d()`/`_splitter_sizes_for()`
|
||||
(表格里 3D 槽是 0——二维布局的描述——面板打开时从最宽的其它面板借走 `DEFAULT_3D_PANE_WIDTH=340`)。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
# Scene asset tools
|
||||
|
||||
Offline tools for the three-dimensional view. They never run in the application:
|
||||
the app loads a **compiled** package (`.oryscene`) and nothing else.
|
||||
|
||||
| tool | what it is for |
|
||||
|---|---|
|
||||
| `compile_scene.py` | a Blender export (+ optional housing GLB) → one `.oryscene` package |
|
||||
| `validate_scene.py` | the checks the loader and the compiler must agree on; the release gate |
|
||||
| `probe_gl.py` | what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context) |
|
||||
| `bench_scene.py` | per-stage p50/p95 of the running view (`--cpu-only` for the pure CPU cost) |
|
||||
| `soak_scene.py` | long run: RSS trend and the queue bound |
|
||||
|
||||
Run them with the repository's own interpreter, from `orisys-app`:
|
||||
|
||||
```powershell
|
||||
$py = '.venv-build\Scripts\python.exe'
|
||||
```
|
||||
|
||||
## Compiling the shipped finger sensor
|
||||
|
||||
The **runtime** asset is `sensors/finger/assets/30fps-0703`: it is what
|
||||
`sensors/finger/manifest.json` names (`assets_dir`), and its unroll map
|
||||
(`unroll_map_uv_s0.4.npz`, out size 300×280, `source_scale` 0.4) matches
|
||||
`camera.resolution_stitch` in `sensor.json`, which is the grid the live optical flow
|
||||
is computed on.
|
||||
|
||||
Do **not** compile `assets/visualization/flatten_remap.npz`: it is a 1024×1024
|
||||
domain, so every live frame would be refused with `field_domain_mismatch`
|
||||
(300x280 vs 1024x1024). That map belongs to offline visualisation.
|
||||
|
||||
```powershell
|
||||
$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703' # or the copy in apps/studio/config
|
||||
|
||||
# 1) state the unit (the importer never guesses it) and read what it measured
|
||||
& $py tools\scene_assets\validate_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz"
|
||||
|
||||
# 2) compile
|
||||
& $py tools\scene_assets\compile_scene.py `
|
||||
--legacy-dir $sdk `
|
||||
--map "$sdk\unroll_map_uv_s0.4.npz" `
|
||||
--meters-per-source-unit 0.001 `
|
||||
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
|
||||
# 3) verify
|
||||
& $py tools\scene_assets\validate_scene.py apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
```
|
||||
|
||||
**`--meters-per-source-unit` is a physical statement about the sensor, not a
|
||||
formatting choice.** The measured mesh extent is 15.81 × 15.0 × 7.5 source units
|
||||
(1032 vertices, 1974 triangles); the shipped finger surface is about 15.8 mm across,
|
||||
so one source unit is one millimetre and the value is `0.001`. If the mechanical
|
||||
drawing says otherwise, state that number instead — nothing downstream can tell the
|
||||
difference, and a wrong factor looks plausible.
|
||||
|
||||
Expected result: `1032 vertices`, `1974 triangles`, `geometry_id` starting
|
||||
`1a2e0e92…`, a 56 KB package, and `validate_scene.py` reporting every check `ok`.
|
||||
|
||||
## Where the application looks for a package
|
||||
|
||||
1. `preference.json` → `"scene3d_package": "<path>"` (an absolute path; also the way
|
||||
to point a developer build at a freshly compiled package);
|
||||
2. `<config>/sensors/<sensor>/scene/*.oryscene` — for the studio app that is
|
||||
`apps/studio/config/sensors/finger/scene/`;
|
||||
3. nothing: the pane reports the directory it searched and the compile command above,
|
||||
and the two-dimensional views keep working.
|
||||
|
||||
The SDK's own `sensors/` tree is **not** searched (the app prefers its local
|
||||
`config/sensors` bundles). Compile into the app's `scene/` directory, or set
|
||||
`scene3d_package`.
|
||||
|
||||
## Release packages (signature, optional encryption, platform lock)
|
||||
|
||||
All three are optional and independent of the plain package above:
|
||||
|
||||
```powershell
|
||||
& $py tools\scene_assets\compile_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz" `
|
||||
--meters-per-source-unit 0.001 `
|
||||
--sign-key keys\build-ed25519.pem --sign-key-id build-2024 `
|
||||
--write-verify-key keys\build-ed25519.pub.pem `
|
||||
--encrypt-key-id finger-display --key-file keys\finger-display.key `
|
||||
--lock-from-probe docs\architecture\gl-report-windows-dev.json `
|
||||
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
```
|
||||
|
||||
- `--sign-key` signs the manifest (and therefore every entry hash, the compatibility
|
||||
information and the platform lock). The private key stays on the build machine; the
|
||||
client ships only `--write-verify-key`'s public half.
|
||||
- `--encrypt-key-id` + `--key-file` additionally seal the content with AES-256-GCM.
|
||||
Encryption never replaces signing. Reading a sealed package needs the
|
||||
`protected-assets` extra (`cryptography`); a plain package does not.
|
||||
- `--lock-from-probe` takes the platform lock (OS, architecture, GL profile, vendor,
|
||||
renderer, minimum texture size, "no software rasteriser") from a `probe_gl.py`
|
||||
report. A machine the lock excludes loses the 3D pane, with the reason, and keeps
|
||||
everything else.
|
||||
- `--encrypt-key-id` without `--key-file`, or with no crypto backend installed, is
|
||||
**refused** rather than silently writing a plain package that looks protected.
|
||||
|
||||
Release gate:
|
||||
|
||||
```powershell
|
||||
& $py tools\scene_assets\validate_scene.py <package> `
|
||||
--key-file keys\finger-display.key --key-id finger-display `
|
||||
--verify-key keys\build-ed25519.pub.pem --require-signature
|
||||
```
|
||||
|
||||
## Performance and long runs
|
||||
|
||||
```powershell
|
||||
# per-stage p50/p95 of the running view (needs a real window system)
|
||||
$env:QT_QPA_PLATFORM = 'windows'
|
||||
& $py tools\scene_assets\bench_scene.py --frames 600 --output perf.json
|
||||
& $py tools\scene_assets\bench_scene.py --cpu-only --frames 500 --vertices 20000 --output perf-cpu.json
|
||||
& $py tools\scene_assets\bench_scene.py --package <package> --key-file keys\finger-display.key `
|
||||
--key-id finger-display --verify-key keys\build-ed25519.pub.pem --output perf-real.json
|
||||
|
||||
# long run: the release gate is two hours in the GUI mode (it is the stricter one)
|
||||
& $py tools\scene_assets\soak_scene.py --minutes 120 --fps 30 --output soak.json
|
||||
& $py tools\scene_assets\soak_scene.py --no-window --minutes 10 --fps 60 --output soak-cpu.json
|
||||
```
|
||||
|
||||
The committed reports for the developer machine are
|
||||
`docs/architecture/perf-report-windows-dev.json` and
|
||||
`docs/architecture/soak-report-windows-dev.json`; the stage-by-stage acceptance, the
|
||||
failure samples and the compatibility notes are in
|
||||
`docs/architecture/studio-3d-stages.md`.
|
||||
Reference in New Issue
Block a user