orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(studio-3d): 历史回放里的三维面板 —— 由「重算」通道喂它，并标注来源

Browse Source 
- What: `VIEW_SHOWS_3D` 加入 `replay`；`app_window` 新增 `_shows_3d()`（可见性与布局共用一个判据）
  与 `_splitter_sizes_for()`（预设表里的 3D 槽是"二维布局"的描述，面板打开时从最宽的面板借
  `DEFAULT_3D_PANE_WIDTH=340`）；`ThreeDPanel` 新增第三个来源标签（实时采集 / 回放：录像不含三维场 /
  重算通道：由录像重算，非记录值）与 `set_source_kind()`；`ThreeDCoordinator.show_rest()` 把参考模型
  放回屏幕并停止主张测量值（重算停止时调用，否则最后一帧会留在屏幕上）；`Mesh3dPanel` 转发这两者，
  并新增"加载中"状态的修正：切换传感器再切回来时若该包已加载，直接显示已就绪的画面而不是一条永远
  不会被替换的"正在加载"。
- Why: 录像里没有三维场（raw_*.mp4 是原始相机图像，`depth.stored=false`，CSV 只有力值），所以纯回放只能
  显示参考模型；但「从视频重算力值」本来就把同一个运行路径指向录像的原始视频，而采集线程上的三维 sink
  早已注册——因此重算一开始面板就会收到真实的形变帧。这两件事必须由界面区分：一个是"没有测量"，
  一个是"当前算法对这段录像会说什么"，把后者当成记录值展示就是另一种 `display_illustrative`。
- Verification: apps/studio 全量通过（test_scene3d_view.py 32 项，含"回放里面板可见且标注正确""重算时
  标注切换""停止重算后清回参考模型""实时视图标为实时"；test_scene3d_coordinator.py 15 项，含 show_rest
  与"停止时在飞的帧不会随后被显示"）；ui_check 570 项 0 失败（新增"已安装包必须被加载并标注来源"）；
  locale_check 0 缺失；仓库内工程包经 validate_scene 全部 ok。
- Not in this change: 逐帧保存展开场/深度（记录格式变更，设计文档明确否掉）；重算通道本身（已存在，本次
  只是把它的帧接到面板上）；示意位移（仍需 SDK lift + 已确认尺度）。
This commit is contained in:
ykcheungab
2026-09-14 17:13:37 +08:00
parent 2226bdde68
commit 981c250559
13 changed files with 515 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/studio/src/orisys_studio/app_window.py
+58 -4
View File
@@ -92,8 +92,17 @@ VIEW_SPLITTER_SIZES = {
"replay": (0, 360, 1080, 0),
"calibration": (0, 0, 560, 880),
}
#: The 3D panel only belongs to the live view, and only for a finger sensor.
VIEW_SHOWS_3D = ("custom", "live")
#: The 3D pane belongs to the live view and to history browsing, and only for a
#: finger sensor. Replay is included because a *recompute* pass re-runs the finger
#: chain over the recording's raw video and feeds the pane through the ordinary
#: acquisition sink; a recording itself stores no 3D field (see
#: ``Mesh3dPanel.set_source_kind`` and the stage-4/5 sections of
#: ``docs/architecture/studio-3d-stages.md``).
VIEW_SHOWS_3D = ("custom", "live", "replay")
#: Width the 3D pane takes from the widest other panel when a preset's table gives it
#: nothing. The table describes the two-dimensional layout; the pane is optional, and
#: "not in the table" must not mean "standing at its minimum width".
DEFAULT_3D_PANE_WIDTH = 340
#: After this long in ``connecting``, tell the user that Stop is the way out.
CONNECT_SLOW_HINT_S = 8.0
#: Playback tick while a recording is playing. 33 ms is one frame at 30 fps; the
@@ -1059,6 +1068,9 @@ class StudioMainWindow(QMainWindow):
self.toolbar.set_video_path(str(segment))
self._refresh_status_rail()
self._recompute_running = True
# The 3D pane now says "recomputed", not "replayed": the pass is what
# feeds it, through the ordinary acquisition sink.
self.mesh3d_panel.set_source_kind(self._current_source_kind())
# Playing is what starts the pass: one path in, so the button and the
# menu cannot start it twice.
self.toolbar.set_run_checked(True)
@@ -1067,6 +1079,11 @@ class StudioMainWindow(QMainWindow):
return
if self._recompute_running:
self._stop_run(reason="user_stop")
# The pass is over, so the last recomputed frame must go: leaving it on
# screen would show a measurement nothing is producing any more.
self.mesh3d_panel.show_reference_only()
if self._view_preset == "replay":
self.mesh3d_panel.set_source_kind(self._current_source_kind())
if self._view_preset == "replay" and self.replay.is_loaded():
self._apply_replay_source(True)
self._render_replay_frame()
@@ -1326,6 +1343,40 @@ class StudioMainWindow(QMainWindow):
)
self._apply_arrow_rotation_for_sensor()
def _current_source_kind(self) -> str:
"""Where the 3D pane's field would come from in the current view.
``recompute`` only while a pass is actually running: the label is a claim
about the pixels on screen, and a stale claim is worse than none.
"""
if self._view_preset == "replay":
return "recompute" if self._recompute_running else "replay"
return "live"
def _shows_3d(self) -> bool:
"""Whether the 3D pane belongs to the current view and sensor.
One predicate for both the pane's visibility and the splitter layout: the
layout has to know the same thing the visibility does, or the pane ends up
visible and standing at its minimum width.
"""
return (
self._view_preset in VIEW_SHOWS_3D
and self.sensor_picker.sensor_type() == "finger"
and bool(self._show_finger_3d)
)
def _splitter_sizes_for(self, preset: str) -> list:
"""The preset's layout, with room for the 3D pane when it is on."""
sizes = list(VIEW_SPLITTER_SIZES.get(preset, DEFAULT_SPLITTER_SIZES))
if self._shows_3d() and sizes and sizes[0] == 0:
width = DEFAULT_3D_PANE_WIDTH
widest = max(range(1, len(sizes)), key=lambda index: sizes[index])
take = min(width, max(sizes[widest] - MIN_PANEL_WIDTHS["plots"], 0))
sizes[0] = take
sizes[widest] = sizes[widest] - take
return sizes
def _apply_panel_visibility(self) -> None:
"""One place decides which panels the current view shows.
@@ -1336,7 +1387,7 @@ class StudioMainWindow(QMainWindow):
"""
is_finger = self.sensor_picker.sensor_type() == "finger"
preset = self._view_preset
show_3d = preset in VIEW_SHOWS_3D and is_finger and bool(self._show_finger_3d)
show_3d = self._shows_3d()
show_arrow = preset != "calibration"
show_cal = self._show_cal if preset == "custom" else preset == "calibration"
self._show_cal = show_cal
@@ -1346,6 +1397,9 @@ class StudioMainWindow(QMainWindow):
self.mesh3d_panel.ensure_view()
self.mesh3d_panel.set_enabled(show_3d)
self.mesh3d_panel.setVisible(show_3d)
# Where the field comes from: a live view acquires it, a replay has none
# stored and only a recompute pass produces one.
self.mesh3d_panel.set_source_kind(self._current_source_kind())
self.arrow_panel.setVisible(show_arrow)
self.force_plots.setVisible(True)
self.cal_panel.setVisible(show_cal)
@@ -1404,7 +1458,7 @@ class StudioMainWindow(QMainWindow):
self.toolbar.set_run_role("playback" if preset == "replay" else "acquire")
self._apply_panel_visibility()
if apply_sizes and preset in VIEW_SPLITTER_SIZES:
self.content_splitter.setSizes(list(VIEW_SPLITTER_SIZES[preset]))
self.content_splitter.setSizes(self._splitter_sizes_for(preset))
if preset == "replay":
# History browsing is the point of this view: the charts read the
# open recording, and the chart cursor is its timeline. Without a
apps/studio/src/orisys_studio/scene3d/coordinator.py
+18
View File
@@ -278,6 +278,24 @@ class ThreeDCoordinator:
# the last field frame has to be composed again.
self._settings_dirty = True
def show_rest(self) -> None:
"""Put the reference model back on screen and stop claiming a measurement.
Needed by the replay view: while a recompute pass runs it feeds this
coordinator through the ordinary acquisition sink, and when the pass stops the
last recomputed frame would otherwise stay on screen — a picture of a
measurement that is no longer being made, which is exactly what the rest frame
exists to prevent. Idempotent, and safe before any scene is active.
"""
if self._closed or self._assets is None or not self._visible:
return
self._incoming.clear()
self._cancel_inflight()
self._last_field_frame = None
self._last_composed_identity = None
self._widget.present(self._composer.rest_frame(self._assets, self._identity(0)))
self._set_state("no_data")
def close(self) -> None:
"""Cancel work, stop the timer, clear both mailboxes. Idempotent."""
if self._closed:
apps/studio/src/orisys_studio/scene3d/panel.py
+33
View File
@@ -81,6 +81,7 @@ class ThreeDPanel(QFrame):
self._assets: Optional[SceneAssets] = None
self._descriptor: Optional[FieldDescriptor] = None
self._state: SceneState = "disabled"
self._source_kind = "live"
self._syncing = False
root = QVBoxLayout(self)
@@ -184,6 +185,16 @@ class ThreeDPanel(QFrame):
self.legend_label.setMinimumWidth(1)
root.addWidget(self.legend_label)
#: Which *source* the field on screen came from. A separate line from the
#: scene state, because they answer different questions: the state says whether
#: anything is being measured, this says whether it was acquired live or
#: recomputed from a recording — a distinction the design document insists on.
self.source_label = QLabel()
self.source_label.setObjectName("panelMeta")
self.source_label.setWordWrap(True)
self.source_label.setMinimumWidth(1)
root.addWidget(self.source_label)
self.state_label = QLabel()
self.state_label.setObjectName("hintText")
self.state_label.setWordWrap(True)
@@ -246,6 +257,26 @@ class ThreeDPanel(QFrame):
self.state_label.setText(t(f"scene3d.state_{state}"))
self.state_label.setVisible(state != "live")
def set_source_kind(self, kind: str) -> None:
"""Say where the field comes from: ``live``, ``replay`` or ``recompute``.
Three states, and the difference between the last two is the point of the
whole replay story:
* ``live`` — acquired from the device now;
* ``replay`` — a recording is open and **no** 3D field is stored in it, so the
pane is showing the surface model only;
* ``recompute`` — a recompute pass is running over the recording's video, so
what is on screen is what *this* algorithm version says about that footage.
It is not the recorded measurement, and the label says so.
"""
kind = str(kind)
if kind not in ("live", "replay", "recompute"):
kind = "live"
self._source_kind = kind
self.source_label.setText(t(f"scene3d.source_{kind}"))
self.source_label.setToolTip(t(f"scene3d.source_{kind}_tip"))
def set_diagnostic(self, diagnostic: Optional[Diagnostic]) -> None:
"""One diagnostic line. The 2D views stay usable behind it."""
if diagnostic is None:
@@ -383,4 +414,6 @@ class ThreeDPanel(QFrame):
self.fps_label.setToolTip(t("scene3d.max_fps_tip"))
self.reset_button.setText(t("scene3d.reset_view"))
self.state_label.setText(t(f"scene3d.state_{self._state}"))
self.source_label.setText(t(f"scene3d.source_{self._source_kind}"))
self.source_label.setToolTip(t(f"scene3d.source_{self._source_kind}_tip"))
self.set_descriptor(self._descriptor)
apps/studio/src/orisys_studio/scene3d/view.py
+17
View File
@@ -310,6 +310,23 @@ class Scene3DView(QWidget):
self._camera.reset(self._assets.bounds_sensor_m)
self._widget.update()
def set_source_kind(self, kind: str) -> None:
"""Tell the pane where its field comes from (see ``ThreeDPanel``)."""
if self._panel is not None:
self._panel.set_source_kind(kind)
def show_reference_only(self) -> None:
"""Drop whatever measurement is on screen and show the surface model.
Used when a recompute pass over a recording stops: without this the last
recomputed frame would stay on screen while nothing is measuring it any more.
"""
if self._coordinator is not None:
self._coordinator.show_rest()
if self._panel is not None:
self._panel.set_descriptor(None)
self._panel.set_state("no_data")
# ------------------------------------------------------------------ state
@property
def available(self) -> bool:
apps/studio/src/orisys_studio/ui/mesh3d_panel.py
+32
View File
@@ -70,8 +70,12 @@ class Mesh3dPanel(QFrame):
self._source: Optional[SceneSource] = None
self._sensor_type = ""
self._expected: Optional[ExpectedGeometry] = None
self._source_kind = "live"
self._enabled = False
self._loaded = False
#: The package whose scene is in the view, so switching sensor away and back
#: does not reload it (and does not pretend to be loading it either).
self._loaded_path: Optional[Path] = None
self._reported: Optional[Diagnostic] = None
self._pending_settings = None
self._message_kind = _UNAVAILABLE
@@ -121,11 +125,22 @@ class Mesh3dPanel(QFrame):
return
self._source = find_package(self._sensor_type, override=package_override)
if self._source is None:
self._loaded_path = None
self._reported = no_asset_diagnostic(self._sensor_type, override=package_override)
self._show_message(
_NO_ASSET, directory=str(self._reported.details["directory"])
)
return
if (
self._view is not None
and self._loaded_path is not None
and self._source.path == self._loaded_path
):
# This scene is already loaded (the sensor was switched away and back):
# showing "loading…" again would leave a line nothing ever replaces, while
# the view behind it is ready.
self._stack.setCurrentWidget(self._view)
return
self._show_message(_LOADING, name=self._source.path.name)
def ensure_view(self) -> Optional[QWidget]:
@@ -141,6 +156,10 @@ class Mesh3dPanel(QFrame):
view.enabledChanged.connect(self._on_view_enabled)
if self._pending_settings is not None and hasattr(view, "restore_settings"):
view.restore_settings(self._pending_settings)
# The kind was decided before the view existed (the window sets it when the
# view preset changes), so it is pushed again here rather than lost.
if hasattr(view, "set_source_kind"):
view.set_source_kind(self._source_kind)
if not view.available:
self._show_message(_UNAVAILABLE, reason=view.reason)
return view
@@ -186,6 +205,7 @@ class Mesh3dPanel(QFrame):
return
if not view.use_scene(assets, source_id=description, expected=self._expected):
return
self._loaded_path = None if self._source is None else self._source.path
self._stack.setCurrentWidget(view)
# Adopting a scene is not the same as showing one: the pane stays off until
# the operator (or the window, restoring a preference) turns it on.
@@ -193,6 +213,7 @@ class Mesh3dPanel(QFrame):
def _on_load_failed(self, item: Diagnostic) -> None:
self._reported = item
self._loaded_path = None
self._show_message(_REJECTED, message=item.message)
def _on_diagnostic(self, item: Diagnostic) -> None:
@@ -239,6 +260,17 @@ class Mesh3dPanel(QFrame):
def scene_path(self) -> Optional[Path]:
return None if self._source is None else self._source.path
def set_source_kind(self, kind: str) -> None:
"""Where the field comes from: live, replay (model only) or recompute."""
self._source_kind = str(kind)
if self._view is not None and hasattr(self._view, "set_source_kind"):
self._view.set_source_kind(self._source_kind)
def show_reference_only(self) -> None:
"""Clear a measurement that is no longer being produced (a stopped pass)."""
if self._view is not None and hasattr(self._view, "show_reference_only"):
self._view.show_reference_only()
def tick(self) -> None:
"""Advance the pane. Called by the window's run timer, never by itself."""
if self._view is not None and hasattr(self._view, "tick"):