feat(studio-3d): 历史回放里的三维面板 —— 由「重算」通道喂它,并标注来源

- What: `VIEW_SHOWS_3D` 加入 `replay`;`app_window` 新增 `_shows_3d()`(可见性与布局共用一个判据)
  与 `_splitter_sizes_for()`(预设表里的 3D 槽是"二维布局"的描述,面板打开时从最宽的面板借
  `DEFAULT_3D_PANE_WIDTH=340`);`ThreeDPanel` 新增第三个来源标签(实时采集 / 回放:录像不含三维场 /
  重算通道:由录像重算,非记录值)与 `set_source_kind()`;`ThreeDCoordinator.show_rest()` 把参考模型
  放回屏幕并停止主张测量值(重算停止时调用,否则最后一帧会留在屏幕上);`Mesh3dPanel` 转发这两者,
  并新增"加载中"状态的修正:切换传感器再切回来时若该包已加载,直接显示已就绪的画面而不是一条永远
  不会被替换的"正在加载"。
- Why: 录像里没有三维场(raw_*.mp4 是原始相机图像,`depth.stored=false`,CSV 只有力值),所以纯回放只能
  显示参考模型;但「从视频重算力值」本来就把同一个运行路径指向录像的原始视频,而采集线程上的三维 sink
  早已注册——因此重算一开始面板就会收到真实的形变帧。这两件事必须由界面区分:一个是"没有测量",
  一个是"当前算法对这段录像会说什么",把后者当成记录值展示就是另一种 `display_illustrative`。
- Verification: apps/studio 全量通过(test_scene3d_view.py 32 项,含"回放里面板可见且标注正确""重算时
  标注切换""停止重算后清回参考模型""实时视图标为实时";test_scene3d_coordinator.py 15 项,含 show_rest
  与"停止时在飞的帧不会随后被显示");ui_check 570 项 0 失败(新增"已安装包必须被加载并标注来源");
  locale_check 0 缺失;仓库内工程包经 validate_scene 全部 ok。
- Not in this change: 逐帧保存展开场/深度(记录格式变更,设计文档明确否掉);重算通道本身(已存在,本次
  只是把它的帧接到面板上);示意位移(仍需 SDK lift + 已确认尺度)。
This commit is contained in:
ykcheungab
2026-09-14 17:13:37 +08:00
parent 2226bdde68
commit 981c250559
13 changed files with 515 additions and 27 deletions
+6
View File
@@ -399,6 +399,12 @@
"state_no_data": "No valid measurement in this frame — showing the surface model",
"state_suspended": "Suspended (the view is hidden)",
"state_unavailable": "3D is unavailable",
"source_live": "Live acquisition",
"source_live_tip": "The picture comes from the data being acquired right now.",
"source_replay": "Replay: the recording stores no 3D field",
"source_replay_tip": "A recording stores the raw camera image and the forces, not a per-frame deformation field. Turn on “Recompute forces from the video” and this pane will show the deformation recomputed from the recording.",
"source_recompute": "Recomputed from the recording — not the recorded values",
"source_recompute_tip": "This is what the current algorithm version says about the recorded footage: it is a check on the algorithm, not the measurement that was recorded at the time.",
"severity_info": "Note",
"severity_warning": "Warning",
"severity_error": "Error",
+6
View File
@@ -399,6 +399,12 @@
"state_no_data": "本帧没有有效测量 — 显示表面模型",
"state_suspended": "已挂起(视图被隐藏)",
"state_unavailable": "三维不可用",
"source_live": "实时采集",
"source_live_tip": "画面来自此刻正在采集的数据。",
"source_replay": "回放:录像不含三维场",
"source_replay_tip": "录像只保存原始相机图像与力值,没有逐帧的形变场。开启「从视频重算力值」后,本面板会由录像重算并把形变显示在这里。",
"source_recompute": "重算通道:由录像重算,非记录值",
"source_recompute_tip": "这是在当前算法版本下对录像重算得到的形变,用于检查算法;它不是当初记录下来的测量值。",
"severity_info": "提示",
"severity_warning": "警告",
"severity_error": "错误",
+58 -4
View File
@@ -92,8 +92,17 @@ VIEW_SPLITTER_SIZES = {
"replay": (0, 360, 1080, 0),
"calibration": (0, 0, 560, 880),
}
#: The 3D panel only belongs to the live view, and only for a finger sensor.
VIEW_SHOWS_3D = ("custom", "live")
#: The 3D pane belongs to the live view and to history browsing, and only for a
#: finger sensor. Replay is included because a *recompute* pass re-runs the finger
#: chain over the recording's raw video and feeds the pane through the ordinary
#: acquisition sink; a recording itself stores no 3D field (see
#: ``Mesh3dPanel.set_source_kind`` and the stage-4/5 sections of
#: ``docs/architecture/studio-3d-stages.md``).
VIEW_SHOWS_3D = ("custom", "live", "replay")
#: Width the 3D pane takes from the widest other panel when a preset's table gives it
#: nothing. The table describes the two-dimensional layout; the pane is optional, and
#: "not in the table" must not mean "standing at its minimum width".
DEFAULT_3D_PANE_WIDTH = 340
#: After this long in ``connecting``, tell the user that Stop is the way out.
CONNECT_SLOW_HINT_S = 8.0
#: Playback tick while a recording is playing. 33 ms is one frame at 30 fps; the
@@ -1059,6 +1068,9 @@ class StudioMainWindow(QMainWindow):
self.toolbar.set_video_path(str(segment))
self._refresh_status_rail()
self._recompute_running = True
# The 3D pane now says "recomputed", not "replayed": the pass is what
# feeds it, through the ordinary acquisition sink.
self.mesh3d_panel.set_source_kind(self._current_source_kind())
# Playing is what starts the pass: one path in, so the button and the
# menu cannot start it twice.
self.toolbar.set_run_checked(True)
@@ -1067,6 +1079,11 @@ class StudioMainWindow(QMainWindow):
return
if self._recompute_running:
self._stop_run(reason="user_stop")
# The pass is over, so the last recomputed frame must go: leaving it on
# screen would show a measurement nothing is producing any more.
self.mesh3d_panel.show_reference_only()
if self._view_preset == "replay":
self.mesh3d_panel.set_source_kind(self._current_source_kind())
if self._view_preset == "replay" and self.replay.is_loaded():
self._apply_replay_source(True)
self._render_replay_frame()
@@ -1326,6 +1343,40 @@ class StudioMainWindow(QMainWindow):
)
self._apply_arrow_rotation_for_sensor()
def _current_source_kind(self) -> str:
"""Where the 3D pane's field would come from in the current view.
``recompute`` only while a pass is actually running: the label is a claim
about the pixels on screen, and a stale claim is worse than none.
"""
if self._view_preset == "replay":
return "recompute" if self._recompute_running else "replay"
return "live"
def _shows_3d(self) -> bool:
"""Whether the 3D pane belongs to the current view and sensor.
One predicate for both the pane's visibility and the splitter layout: the
layout has to know the same thing the visibility does, or the pane ends up
visible and standing at its minimum width.
"""
return (
self._view_preset in VIEW_SHOWS_3D
and self.sensor_picker.sensor_type() == "finger"
and bool(self._show_finger_3d)
)
def _splitter_sizes_for(self, preset: str) -> list:
"""The preset's layout, with room for the 3D pane when it is on."""
sizes = list(VIEW_SPLITTER_SIZES.get(preset, DEFAULT_SPLITTER_SIZES))
if self._shows_3d() and sizes and sizes[0] == 0:
width = DEFAULT_3D_PANE_WIDTH
widest = max(range(1, len(sizes)), key=lambda index: sizes[index])
take = min(width, max(sizes[widest] - MIN_PANEL_WIDTHS["plots"], 0))
sizes[0] = take
sizes[widest] = sizes[widest] - take
return sizes
def _apply_panel_visibility(self) -> None:
"""One place decides which panels the current view shows.
@@ -1336,7 +1387,7 @@ class StudioMainWindow(QMainWindow):
"""
is_finger = self.sensor_picker.sensor_type() == "finger"
preset = self._view_preset
show_3d = preset in VIEW_SHOWS_3D and is_finger and bool(self._show_finger_3d)
show_3d = self._shows_3d()
show_arrow = preset != "calibration"
show_cal = self._show_cal if preset == "custom" else preset == "calibration"
self._show_cal = show_cal
@@ -1346,6 +1397,9 @@ class StudioMainWindow(QMainWindow):
self.mesh3d_panel.ensure_view()
self.mesh3d_panel.set_enabled(show_3d)
self.mesh3d_panel.setVisible(show_3d)
# Where the field comes from: a live view acquires it, a replay has none
# stored and only a recompute pass produces one.
self.mesh3d_panel.set_source_kind(self._current_source_kind())
self.arrow_panel.setVisible(show_arrow)
self.force_plots.setVisible(True)
self.cal_panel.setVisible(show_cal)
@@ -1404,7 +1458,7 @@ class StudioMainWindow(QMainWindow):
self.toolbar.set_run_role("playback" if preset == "replay" else "acquire")
self._apply_panel_visibility()
if apply_sizes and preset in VIEW_SPLITTER_SIZES:
self.content_splitter.setSizes(list(VIEW_SPLITTER_SIZES[preset]))
self.content_splitter.setSizes(self._splitter_sizes_for(preset))
if preset == "replay":
# History browsing is the point of this view: the charts read the
# open recording, and the chart cursor is its timeline. Without a
@@ -278,6 +278,24 @@ class ThreeDCoordinator:
# the last field frame has to be composed again.
self._settings_dirty = True
def show_rest(self) -> None:
"""Put the reference model back on screen and stop claiming a measurement.
Needed by the replay view: while a recompute pass runs it feeds this
coordinator through the ordinary acquisition sink, and when the pass stops the
last recomputed frame would otherwise stay on screen — a picture of a
measurement that is no longer being made, which is exactly what the rest frame
exists to prevent. Idempotent, and safe before any scene is active.
"""
if self._closed or self._assets is None or not self._visible:
return
self._incoming.clear()
self._cancel_inflight()
self._last_field_frame = None
self._last_composed_identity = None
self._widget.present(self._composer.rest_frame(self._assets, self._identity(0)))
self._set_state("no_data")
def close(self) -> None:
"""Cancel work, stop the timer, clear both mailboxes. Idempotent."""
if self._closed:
@@ -81,6 +81,7 @@ class ThreeDPanel(QFrame):
self._assets: Optional[SceneAssets] = None
self._descriptor: Optional[FieldDescriptor] = None
self._state: SceneState = "disabled"
self._source_kind = "live"
self._syncing = False
root = QVBoxLayout(self)
@@ -184,6 +185,16 @@ class ThreeDPanel(QFrame):
self.legend_label.setMinimumWidth(1)
root.addWidget(self.legend_label)
#: Which *source* the field on screen came from. A separate line from the
#: scene state, because they answer different questions: the state says whether
#: anything is being measured, this says whether it was acquired live or
#: recomputed from a recording — a distinction the design document insists on.
self.source_label = QLabel()
self.source_label.setObjectName("panelMeta")
self.source_label.setWordWrap(True)
self.source_label.setMinimumWidth(1)
root.addWidget(self.source_label)
self.state_label = QLabel()
self.state_label.setObjectName("hintText")
self.state_label.setWordWrap(True)
@@ -246,6 +257,26 @@ class ThreeDPanel(QFrame):
self.state_label.setText(t(f"scene3d.state_{state}"))
self.state_label.setVisible(state != "live")
def set_source_kind(self, kind: str) -> None:
"""Say where the field comes from: ``live``, ``replay`` or ``recompute``.
Three states, and the difference between the last two is the point of the
whole replay story:
* ``live`` — acquired from the device now;
* ``replay`` — a recording is open and **no** 3D field is stored in it, so the
pane is showing the surface model only;
* ``recompute`` — a recompute pass is running over the recording's video, so
what is on screen is what *this* algorithm version says about that footage.
It is not the recorded measurement, and the label says so.
"""
kind = str(kind)
if kind not in ("live", "replay", "recompute"):
kind = "live"
self._source_kind = kind
self.source_label.setText(t(f"scene3d.source_{kind}"))
self.source_label.setToolTip(t(f"scene3d.source_{kind}_tip"))
def set_diagnostic(self, diagnostic: Optional[Diagnostic]) -> None:
"""One diagnostic line. The 2D views stay usable behind it."""
if diagnostic is None:
@@ -383,4 +414,6 @@ class ThreeDPanel(QFrame):
self.fps_label.setToolTip(t("scene3d.max_fps_tip"))
self.reset_button.setText(t("scene3d.reset_view"))
self.state_label.setText(t(f"scene3d.state_{self._state}"))
self.source_label.setText(t(f"scene3d.source_{self._source_kind}"))
self.source_label.setToolTip(t(f"scene3d.source_{self._source_kind}_tip"))
self.set_descriptor(self._descriptor)
@@ -310,6 +310,23 @@ class Scene3DView(QWidget):
self._camera.reset(self._assets.bounds_sensor_m)
self._widget.update()
def set_source_kind(self, kind: str) -> None:
"""Tell the pane where its field comes from (see ``ThreeDPanel``)."""
if self._panel is not None:
self._panel.set_source_kind(kind)
def show_reference_only(self) -> None:
"""Drop whatever measurement is on screen and show the surface model.
Used when a recompute pass over a recording stops: without this the last
recomputed frame would stay on screen while nothing is measuring it any more.
"""
if self._coordinator is not None:
self._coordinator.show_rest()
if self._panel is not None:
self._panel.set_descriptor(None)
self._panel.set_state("no_data")
# ------------------------------------------------------------------ state
@property
def available(self) -> bool:
@@ -70,8 +70,12 @@ class Mesh3dPanel(QFrame):
self._source: Optional[SceneSource] = None
self._sensor_type = ""
self._expected: Optional[ExpectedGeometry] = None
self._source_kind = "live"
self._enabled = False
self._loaded = False
#: The package whose scene is in the view, so switching sensor away and back
#: does not reload it (and does not pretend to be loading it either).
self._loaded_path: Optional[Path] = None
self._reported: Optional[Diagnostic] = None
self._pending_settings = None
self._message_kind = _UNAVAILABLE
@@ -121,11 +125,22 @@ class Mesh3dPanel(QFrame):
return
self._source = find_package(self._sensor_type, override=package_override)
if self._source is None:
self._loaded_path = None
self._reported = no_asset_diagnostic(self._sensor_type, override=package_override)
self._show_message(
_NO_ASSET, directory=str(self._reported.details["directory"])
)
return
if (
self._view is not None
and self._loaded_path is not None
and self._source.path == self._loaded_path
):
# This scene is already loaded (the sensor was switched away and back):
# showing "loading…" again would leave a line nothing ever replaces, while
# the view behind it is ready.
self._stack.setCurrentWidget(self._view)
return
self._show_message(_LOADING, name=self._source.path.name)
def ensure_view(self) -> Optional[QWidget]:
@@ -141,6 +156,10 @@ class Mesh3dPanel(QFrame):
view.enabledChanged.connect(self._on_view_enabled)
if self._pending_settings is not None and hasattr(view, "restore_settings"):
view.restore_settings(self._pending_settings)
# The kind was decided before the view existed (the window sets it when the
# view preset changes), so it is pushed again here rather than lost.
if hasattr(view, "set_source_kind"):
view.set_source_kind(self._source_kind)
if not view.available:
self._show_message(_UNAVAILABLE, reason=view.reason)
return view
@@ -186,6 +205,7 @@ class Mesh3dPanel(QFrame):
return
if not view.use_scene(assets, source_id=description, expected=self._expected):
return
self._loaded_path = None if self._source is None else self._source.path
self._stack.setCurrentWidget(view)
# Adopting a scene is not the same as showing one: the pane stays off until
# the operator (or the window, restoring a preference) turns it on.
@@ -193,6 +213,7 @@ class Mesh3dPanel(QFrame):
def _on_load_failed(self, item: Diagnostic) -> None:
self._reported = item
self._loaded_path = None
self._show_message(_REJECTED, message=item.message)
def _on_diagnostic(self, item: Diagnostic) -> None:
@@ -239,6 +260,17 @@ class Mesh3dPanel(QFrame):
def scene_path(self) -> Optional[Path]:
return None if self._source is None else self._source.path
def set_source_kind(self, kind: str) -> None:
"""Where the field comes from: live, replay (model only) or recompute."""
self._source_kind = str(kind)
if self._view is not None and hasattr(self._view, "set_source_kind"):
self._view.set_source_kind(self._source_kind)
def show_reference_only(self) -> None:
"""Clear a measurement that is no longer being produced (a stopped pass)."""
if self._view is not None and hasattr(self._view, "show_reference_only"):
self._view.show_reference_only()
def tick(self) -> None:
"""Advance the pane. Called by the window's run timer, never by itself."""
if self._view is not None and hasattr(self._view, "tick"):
@@ -312,3 +312,50 @@ def test_close_is_idempotent_and_stops_everything(scene, pool):
presented = len(widget.frames)
coordinator.tick()
assert len(widget.frames) == presented
# ------------------------------------------------- the reference model, on demand
def test_show_rest_puts_the_model_back_and_stops_claiming_a_measurement(scene, pool):
"""The replay view's case: a pass stops, so the last frame must go.
Without this the recomputed picture would stay on screen while nothing is
producing it — a measurement that is no longer being made.
"""
widget = FakeWidget()
coordinator = ThreeDCoordinator(widget, SceneComposer(), pool)
coordinator.set_visible(True)
coordinator.activate(scene, "replay-A")
coordinator.submit(_frame(scene, source="replay-A"))
_drain(coordinator, ticks=10)
assert coordinator.state in ("live", "no_data")
assert widget.frames[-1].valid.any(), "the measurement never arrived"
coordinator.show_rest()
last = widget.frames[-1]
assert not last.valid.any(), "the old measurement is still on screen"
assert last.scalar_descriptor.quantity == "unavailable"
assert coordinator.state == "no_data"
# And it is a no-op when there is nothing to show or the view is off.
coordinator.set_visible(False)
before = len(widget.frames)
coordinator.show_rest()
assert len(widget.frames) == before
coordinator.close()
coordinator.show_rest() # closed: still safe
assert coordinator.state == "disabled"
def test_a_frame_already_queued_is_dropped_when_the_model_comes_back(scene, pool):
"""The rest frame is not "the newest sample": a queued one must not follow it."""
widget = FakeWidget()
coordinator = ThreeDCoordinator(widget, CountingComposer(delay_s=0.02), pool)
coordinator.set_visible(True)
coordinator.activate(scene, "replay-A")
coordinator.submit(_frame(scene, source="replay-A", value=7.0))
coordinator.show_rest()
_drain(coordinator, ticks=4, sleep=0.01)
assert not widget.frames[-1].valid.any(), (
"a frame that was in flight when the pass stopped was presented anyway"
)
coordinator.close()
+95 -6
View File
@@ -37,6 +37,7 @@ from orisys_app_shared.scene3d.contracts import (
UNAVAILABLE_DESCRIPTOR,
)
from orisys_app_shared.scene3d.package import write_package
from orisys_studio.app_window import MIN_PANEL_WIDTHS
from orisys_studio.scene3d import prefs as scene3d_prefs
from orisys_studio.scene3d.asset_source import (
PACKAGE_SUFFIX,
@@ -45,7 +46,12 @@ from orisys_studio.scene3d.asset_source import (
load_package,
no_asset_diagnostic,
)
from tests.test_ui_smoke import _finish, _running_window
from tests.test_ui_smoke import ( # noqa: F401 (fixtures are imported by name)
_finish,
_running_window,
record_root,
replay_recording,
)
#: Scratch root for the two-session comparison. One subdirectory per run, because
#: the point of the test is that two recordings are identical, not that one file
@@ -112,9 +118,18 @@ def test_a_box_sensor_reports_that_it_has_no_3d_scene(qt_app):
panel.deleteLater()
def test_a_missing_package_names_where_it_looked(tmp_path, qt_app):
def test_a_missing_package_names_where_it_looked(tmp_path, qt_app, monkeypatch):
"""With no package anywhere, the pane names the directory it looked in.
The search is stubbed rather than relied on: the repository may or may not carry
a compiled package (the application's own config directory is where one belongs),
and a test that only passes on a machine without one is a test that fails the
first time somebody installs the feature.
"""
import orisys_studio.ui.mesh3d_panel as panel_module
from orisys_studio.ui.mesh3d_panel import Mesh3dPanel
monkeypatch.setattr(panel_module, "find_package", lambda *a, **k: None)
panel = Mesh3dPanel()
try:
panel.set_sensor("finger", package_override=tmp_path / "nope.oryscene")
@@ -164,17 +179,21 @@ def test_the_pane_fits_the_width_the_window_reserves_for_it(qt_app):
view.close()
def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(qt_app):
def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(
write_poc, qt_app
):
"""The offscreen plugin cannot create a context; that must not look like "no data".
Qt reports this by never calling ``initializeGL``, so the widget asks Qt the one
question that has an answer — is there a context at all — on its first paint.
question that has an answer — is there a context at all — on its first paint. The
scene comes from a package written for this test, so the case does not depend on
whether the repository happens to carry a compiled one.
"""
from orisys_studio.ui.mesh3d_panel import Mesh3dPanel
panel = Mesh3dPanel()
try:
panel.set_sensor("finger")
panel.set_sensor("finger", package_override=write_poc())
view = panel.ensure_view()
if view is None:
pytest.skip("the pane did not build a view here")
@@ -184,7 +203,10 @@ def test_a_platform_without_a_context_says_so_instead_of_showing_an_empty_view(q
panel.resize(240, 400)
panel.show()
view.show()
qt_app.processEvents()
panel.set_enabled(True)
assert _pump(qt_app, lambda: view.assets is not None, timeout_s=15.0), (
"the test package never loaded"
)
widget = view.gl_widget
for _ in range(5):
widget.repaint()
@@ -584,6 +606,73 @@ def test_a_recording_says_what_the_3d_pane_was(
window.close()
def test_the_replay_view_shows_the_pane_and_says_where_the_field_comes_from(
make_window, replay_recording, qt_app
):
"""History browsing gets the pane, fed by the recompute pass — and says so.
A recording stores the raw camera image and the forces, not a per-frame
deformation field, so the pane in replay is either the surface model (no field) or
what a recompute pass produces from the video. Those are two different claims and
the label has to distinguish them.
"""
from tests.test_ui_smoke import _show
window = _show(make_window(sensor_type="finger"), (1440, 880))
try:
assert window._load_recording(Path(replay_recording))
assert window._view_preset == "replay"
pane = window.mesh3d_panel
assert pane.isVisible(), "the 3D pane is hidden in the replay view"
assert pane.enabled(), "the pane is visible but not switched on"
assert window.content_splitter.sizes()[0] >= MIN_PANEL_WIDTHS["mesh3d"], (
"the replay layout left the pane at zero width"
)
view = pane.view
if view is None or view.panel is None:
pytest.skip("no 3D view on this platform")
# Wait for the scene to load, then read what the pane claims.
assert _pump(qt_app, lambda: view.assets is not None, timeout_s=15.0), (
"the scene never loaded: " + str(pane.last_diagnostic)
)
label = view.panel.source_label.text()
assert "回放" in label or "Replay" in label, label
# A running pass means the pixels are recomputed, and the label must say so.
window._recompute_running = True
window._apply_panel_visibility()
label = view.panel.source_label.text()
assert "重算" in label or "Recomputed" in label, label
# Stopping the pass must clear the picture, not leave the last frame up.
window._recompute_running = False
pane.show_reference_only()
qt_app.processEvents()
assert view.coordinator.state in ("no_data", "ready", "disabled")
assert "没有测量值" in view.panel.legend_label.text() or (
"No measurement" in view.panel.legend_label.text()
), view.panel.legend_label.text()
window._apply_panel_visibility()
label = view.panel.source_label.text()
assert "回放" in label or "Replay" in label, label
finally:
window.close()
def test_the_live_view_still_says_live(make_window, qt_app):
window = make_window(sensor_type="finger")
try:
window._apply_panel_visibility()
view = window.mesh3d_panel.view
if view is None or view.panel is None:
pytest.skip("no 3D view on this platform")
label = view.panel.source_label.text()
assert "实时" in label or "Live" in label, label
assert view.panel.source_label.toolTip()
finally:
window.close()
def test_a_platform_locked_scene_is_refused_in_the_application(qt_app):
"""A lock violation is a *pane* failure: the view refuses the scene and says why."""
from orisys_app_shared.scene3d.contracts import PlatformInfo, PlatformLock
+49 -5
View File
@@ -2830,9 +2830,19 @@ def main() -> int:
pane.set_sensor("box")
check(pane.showing_message, "the 3D pane kept showing a picture for a box sensor")
# A finger with no package names where the package belongs and how to build one.
pane.set_sensor("finger")
diagnostic = pane.last_diagnostic
# A finger *without* any package names where one belongs and how to build it.
# The search is stubbed for this check: the application's config directory may
# or may not carry a compiled package on the machine running the check, and a
# check that only passes without one would fail the first time one is
# installed (the positive case is checked just below).
import orisys_studio.ui.mesh3d_panel as _panel_module
real_find = _panel_module.find_package
_panel_module.find_package = lambda *a, **k: None
try:
fresh_missing = _Mesh3dPanel()
fresh_missing.set_sensor("finger")
diagnostic = fresh_missing.last_diagnostic
check(
diagnostic is not None and diagnostic.code == "asset_rejected",
f"a missing 3D scene was not reported as an asset problem: {diagnostic}",
@@ -2843,10 +2853,44 @@ def main() -> int:
"the missing-scene report does not say how to build one",
)
check(
str(diagnostic.details.get("directory", "")) in pane.message.text()
or pane.message.text().strip(),
str(diagnostic.details.get("directory", ""))
in fresh_missing.message.text()
or fresh_missing.message.text().strip() != "",
"the pane does not name the directory it looked in",
)
fresh_missing.deleteLater()
finally:
_panel_module.find_package = real_find
# And when a package *is* installed, the pane must actually load it: this is
# the end-to-end check that the search path, the loader and the widget agree.
pane.set_sensor("finger")
shipped = _panel_module.find_package("finger")
if shipped is not None:
view_for_load = pane.ensure_view()
pane.set_enabled(True)
deadline = time.time() + 15.0
while time.time() < deadline:
app.processEvents()
assets = getattr(view_for_load, "assets", None) if view_for_load else None
if assets is not None:
break
time.sleep(0.02)
check(
getattr(view_for_load, "assets", None) is not None,
f"the installed 3D scene was never loaded: {pane.last_diagnostic}",
)
check(
not pane.showing_message,
"the pane is still showing a message although a scene loaded",
)
if view_for_load is not None and view_for_load.panel is not None:
source_text = view_for_load.panel.source_label.text()
check(source_text.strip() != "", "the pane does not say where the field comes from")
check(
"实时" in source_text or "Live" in source_text,
f"the live view does not label its source: {source_text!r}",
)
# The pane's controls must fit the width the window reserves for it.
view = pane.ensure_view()
+21 -5
View File
@@ -234,21 +234,37 @@ shared 全部 **262 项**。`ui_check.py` **566 项检查 0 失败**(新增 "3
- 运行时**只**读 `.oryscene` 包;旧的 Blender 导出目录仍由 `LegacyFingerImporter`(离线工具/测试)导入,
单位与变换必须显式给出。包缺失/被拒/无 GL 时:面板一行文字说明原因,采集、记录、力曲线、标定、回放全部照常
(`ui_check` 566 项与 `test_ui_smoke.py` 全绿即为证)。
(`ui_check` 570 项与 `test_ui_smoke.py` 全绿即为证)。
- 仓库现在**带一个已编译的工程包**:`apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene`
(明文、未签名、即 `v1` 封套,1032 顶点;生成命令与单位依据写在 `tools/scene_assets/README.md`)。
它只是工程用包,发布时用同一工具换成签名/加密/锁定的版本即可,加载路径不变。
- `gl_profile`(`desktop33` 默认 / `gles30`)在 `QApplication` 之前生效,一进程一个;未知取值退回默认而不是
让窗口起不来。
- 3D 的显示设置(放大、色标、外壳、显示帧率)持久化在 `scene3d_display`;**色标的 descriptor 不恢复**,
永远来自当前帧,因此一个旧文件不可能把测量标成另一种量。
**明确未做(留给阶段 5 或按用户要求暂缓)**:
- **示意位移(`displacement_sensor_m`)仍未接入**:需要 SDK 的 lift 与一个已确认的空间尺度;在拿到
"这个尺度是怎么标定出来的"之前,适配器继续只着色不位移(阶段 3 的原则不变)。
- ARM 板卡适配:按用户要求暂缓;`probe_gl.py` 可在板上直接跑出报告,据此定 `gl_profile` 与
`MIN_*`/预算(阶段 5 的两平台报告)。
- `session.json` 的 `visualization3d` 能力块、包签名/AES-GCM、性能与长时报告:阶段 5。
- 3D 包尚未随仓库提供(仓库里没有 `.oryscene`):这是**有意的**,包是发布产物(阶段 5 管签名与平台锁定),
并且它必须携带只能由人声明的单位与变换。
- `session.json` 的 `visualization3d` 能力块、包签名/AES-GCM、性能与长时报告:阶段 5(已完成)。
**后续补充(同一功能,历史回放接入)**:
`VIEW_SHOWS_3D` 原先只有 `custom`/`live`。现在**回放也能打开三维面板**,但要分清两件事,
因为这是两种不同的主张:
- 录像里**没有**三维场:`raw_*.mp4` 是原始相机图像,`session.json` 里 `depth.stored=false` 是当初定的,
CSV 只有力值。所以纯回放时面板只显示**参考模型**,并写明"录像不含三维场"。
- 打开「从视频重算力值」后,重算通道本来就是把**同一个运行路径**指向录像的原始视频(手指 worker 支持文件源),
而采集线程上的三维 sink 早已注册(`_start_run`),所以面板会**自动**收到重算出来的帧。
此时面板标注"重算通道:由录像重算,非记录值"——那是"这个算法版本对这段录像会说什么",
不是当初记录到的测量值(和 `display_illustrative` 同样的思路)。
实现要点:`ThreeDCoordinator.show_rest()`(把参考模型放回屏幕并停止主张测量值)、
`ThreeDPanel.set_source_kind()`(第三个来源标签)、`app_window._shows_3d()`/`_splitter_sizes_for()`
(表格里 3D 槽是 0——二维布局的描述——面板打开时从最宽的其它面板借走 `DEFAULT_3D_PANE_WIDTH=340`)。
---
+126
View File
@@ -0,0 +1,126 @@
# Scene asset tools
Offline tools for the three-dimensional view. They never run in the application:
the app loads a **compiled** package (`.oryscene`) and nothing else.
| tool | what it is for |
|---|---|
| `compile_scene.py` | a Blender export (+ optional housing GLB) → one `.oryscene` package |
| `validate_scene.py` | the checks the loader and the compiler must agree on; the release gate |
| `probe_gl.py` | what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context) |
| `bench_scene.py` | per-stage p50/p95 of the running view (`--cpu-only` for the pure CPU cost) |
| `soak_scene.py` | long run: RSS trend and the queue bound |
Run them with the repository's own interpreter, from `orisys-app`:
```powershell
$py = '.venv-build\Scripts\python.exe'
```
## Compiling the shipped finger sensor
The **runtime** asset is `sensors/finger/assets/30fps-0703`: it is what
`sensors/finger/manifest.json` names (`assets_dir`), and its unroll map
(`unroll_map_uv_s0.4.npz`, out size 300×280, `source_scale` 0.4) matches
`camera.resolution_stitch` in `sensor.json`, which is the grid the live optical flow
is computed on.
Do **not** compile `assets/visualization/flatten_remap.npz`: it is a 1024×1024
domain, so every live frame would be refused with `field_domain_mismatch`
(300x280 vs 1024x1024). That map belongs to offline visualisation.
```powershell
$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703' # or the copy in apps/studio/config
# 1) state the unit (the importer never guesses it) and read what it measured
& $py tools\scene_assets\validate_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz"
# 2) compile
& $py tools\scene_assets\compile_scene.py `
--legacy-dir $sdk `
--map "$sdk\unroll_map_uv_s0.4.npz" `
--meters-per-source-unit 0.001 `
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
# 3) verify
& $py tools\scene_assets\validate_scene.py apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```
**`--meters-per-source-unit` is a physical statement about the sensor, not a
formatting choice.** The measured mesh extent is 15.81 × 15.0 × 7.5 source units
(1032 vertices, 1974 triangles); the shipped finger surface is about 15.8 mm across,
so one source unit is one millimetre and the value is `0.001`. If the mechanical
drawing says otherwise, state that number instead — nothing downstream can tell the
difference, and a wrong factor looks plausible.
Expected result: `1032 vertices`, `1974 triangles`, `geometry_id` starting
`1a2e0e92…`, a 56 KB package, and `validate_scene.py` reporting every check `ok`.
## Where the application looks for a package
1. `preference.json` → `"scene3d_package": "<path>"` (an absolute path; also the way
to point a developer build at a freshly compiled package);
2. `<config>/sensors/<sensor>/scene/*.oryscene` — for the studio app that is
`apps/studio/config/sensors/finger/scene/`;
3. nothing: the pane reports the directory it searched and the compile command above,
and the two-dimensional views keep working.
The SDK's own `sensors/` tree is **not** searched (the app prefers its local
`config/sensors` bundles). Compile into the app's `scene/` directory, or set
`scene3d_package`.
## Release packages (signature, optional encryption, platform lock)
All three are optional and independent of the plain package above:
```powershell
& $py tools\scene_assets\compile_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz" `
--meters-per-source-unit 0.001 `
--sign-key keys\build-ed25519.pem --sign-key-id build-2024 `
--write-verify-key keys\build-ed25519.pub.pem `
--encrypt-key-id finger-display --key-file keys\finger-display.key `
--lock-from-probe docs\architecture\gl-report-windows-dev.json `
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```
- `--sign-key` signs the manifest (and therefore every entry hash, the compatibility
information and the platform lock). The private key stays on the build machine; the
client ships only `--write-verify-key`'s public half.
- `--encrypt-key-id` + `--key-file` additionally seal the content with AES-256-GCM.
Encryption never replaces signing. Reading a sealed package needs the
`protected-assets` extra (`cryptography`); a plain package does not.
- `--lock-from-probe` takes the platform lock (OS, architecture, GL profile, vendor,
renderer, minimum texture size, "no software rasteriser") from a `probe_gl.py`
report. A machine the lock excludes loses the 3D pane, with the reason, and keeps
everything else.
- `--encrypt-key-id` without `--key-file`, or with no crypto backend installed, is
**refused** rather than silently writing a plain package that looks protected.
Release gate:
```powershell
& $py tools\scene_assets\validate_scene.py <package> `
--key-file keys\finger-display.key --key-id finger-display `
--verify-key keys\build-ed25519.pub.pem --require-signature
```
## Performance and long runs
```powershell
# per-stage p50/p95 of the running view (needs a real window system)
$env:QT_QPA_PLATFORM = 'windows'
& $py tools\scene_assets\bench_scene.py --frames 600 --output perf.json
& $py tools\scene_assets\bench_scene.py --cpu-only --frames 500 --vertices 20000 --output perf-cpu.json
& $py tools\scene_assets\bench_scene.py --package <package> --key-file keys\finger-display.key `
--key-id finger-display --verify-key keys\build-ed25519.pub.pem --output perf-real.json
# long run: the release gate is two hours in the GUI mode (it is the stricter one)
& $py tools\scene_assets\soak_scene.py --minutes 120 --fps 30 --output soak.json
& $py tools\scene_assets\soak_scene.py --no-window --minutes 10 --fps 60 --output soak-cpu.json
```
The committed reports for the developer machine are
`docs/architecture/perf-report-windows-dev.json` and
`docs/architecture/soak-report-windows-dev.json`; the stage-by-stage acceptance, the
failure samples and the compatibility notes are in
`docs/architecture/studio-3d-stages.md`.