From 1aa454311fa44d3ab8c52846275f50231a1879ed Mon Sep 17 00:00:00 2001 From: ykcheungab Date: Sun, 4 Oct 2026 16:04:02 +0800 Subject: [PATCH] Make the cached analysis views reachable over a recording MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replay moved the cursor onto the result's timeline, so the recording's own picture was not offered at all and ``use_recording()`` had no caller. Adopting a result now keeps the cursor on the recording, opens the panel on a derived view, and serves scrubs, playback and display changes from the cache (Part B §9). Verified: apps/studio 455 items / 0 failed / 18 skipped, shared 427 / 0 / 1, autocal 358, ui_check 628 checks / 0 failures, lint gate clean. --- apps/studio/docs/replay-analysis-part-b.md | 96 ++-- apps/studio/docs/replay-panel.md | 15 +- .../src/orisys_studio/replay_controller.py | 90 +++- apps/studio/tests/test_replay_analysis.py | 13 +- apps/studio/tests/test_replay_wiring.py | 507 ++++++++++++++++++ 5 files changed, 649 insertions(+), 72 deletions(-) create mode 100644 apps/studio/tests/test_replay_wiring.py diff --git a/apps/studio/docs/replay-analysis-part-b.md b/apps/studio/docs/replay-analysis-part-b.md index c44ea10..c5be65e 100644 --- a/apps/studio/docs/replay-analysis-part-b.md +++ b/apps/studio/docs/replay-analysis-part-b.md @@ -15,14 +15,18 @@ | `recording` | 录像自己的 `raw_*.mp4`,按需解码 | 这个画面是原始相机图像:它自己没有光流也没有深度 | | `analysis` | 存下来的分析结果,按帧读 | 有 pass 记录下来的光流样本、候选网格与深度图 | -模式说的是**画面从哪里读**,而"有没有可画的东西"是另一件事:只要有结果,二维面板的模式 -选择器就提供 深度图/箭头(时间轴在录像上时连"原始图像"一起提供),选了派生模式的那些帧 -一律从结果读、按窗口唯一的 2D 出口画 —— 绝不会把原始图像当成箭头图交出去。 +模式说的是画面从哪里读,"有没有可画的东西"是另一件事:只要有结果,选择器就提供箭头与深度图 +(连"原始图像"一起),派生模式那些帧一律从结果读、按窗口唯一的 2D 出口画。 图表、二维画布、三维面板都读这一个游标,模式由 `ReplayEngine.mode()` 回答;「从视频重算」 因此不再是第三种来源,而是**显式重跑**这一遍的开关(第 10 节)。时间基准是**信号时间** (Part A 第 5 节),进度、位置、单步与跳转都说它。 +**应用里的时间轴永远是录像的。** 引擎留在 `recording`:游标、帧 id 与时长都是录像自己的, +结果挂在游标后面当缓存(`available_result()`),`_adopt_session()` 调用 `use_recording()` 保证 +这一点。缓存有三个触点,不必先打开「从视频重算」:复用已有结果、pass 完成、选择器选中派生模式 +(时间轴不动)。 + **帧身份只来自录像自己的计数器。** 证据录像 id 是 20…52 而非 0…32,所以 `AnalysisSession.frame(frame_id)` 对结果里没有的 id 报 `AnalysisNotFoundError`:把索引当 id 传进去不会被"顺手改成能读的那一帧",那会返回一张看起来完全正确的图。按时间取帧走 @@ -43,15 +47,14 @@ **并发与工作目录。** 开链路会往当前目录写 2 MB 预览图(Part A 第 6 节),所以 pass 跑在**本 进程**的一个线程里,启动前要求没有采集会话、没有预览、也没有第二遍分析,并且**本模块从不改 -工作目录**。锁只覆盖开链路那一小段(`_working_directory` 在 `_OpenChain.open()` 内进出), -逐帧处理在锁之外,GUI 线程这期间正常刷新。关窗时先取消并等待 pass。 +工作目录**。锁只覆盖开链路那一小段,逐帧处理在锁之外,GUI 线程这期间正常刷新。 ## 11. 三视图如何被同一个位置喂 | 视图 | 读什么 | 怎么读 | |---|---|---| | 图表 | `forces.csv`(记录值) | `charts.set_cursor(t)` 用同一游标;结果里不存力 | -| 二维画布 | 结果的 `flow`、`depth` 与 `valid_mask` | `ImageFrame(source="analysis")` 走窗口唯一的 2D 出口 `_show_image_frame`,箭头、暗点、配色、域边界与深度视图与实时/重算**同一套代码** | +| 二维画布 | 结果的 `flow`、`depth` 与 `valid_mask` | `ImageFrame` 走窗口唯一的 2D 出口 `_show_image_frame`(录像时间轴上的派生视图带 `source="replay"` 与 `result_available=True`,因此"原始图像"仍在选择器里),箭头、暗点、配色、域边界与深度视图与实时/重算**同一套代码** | | 三维面板 | 同一帧的 `SdkPacket` | `session.packet(frame)` → `Scene3DView.submit_packet`,走同一个 adapter、域掩码 latch 与有效性求交 | **深度视图也由结果喂。** `AnalysedFrame.depth` 就是第 1 节那幅量化平面重建出来的 @@ -68,20 +71,20 @@ `dx`/`dy` 是**画出来的**位移,已经带上长度余量那一步的缩放:只用前两个因子会让每根箭头短 20%(finger 余量 0.8),330 个样本里 319 个不同、活动集也不同 —— 实测的,不是推导的。余量补 -上后,重建流场重新过一遍共享选区器,样本集与存下来的**逐元素相等**(33 帧全覆盖)。三维面板 -直接拿存下来的 `FlowSamples`,所以"两张图是同一个测量"由构造决定。 +上后,重建流场重新过一遍共享选区器,样本集与存下来的**逐元素相等**(33 帧全覆盖)。 ## 12. 显示设置是读 阈值、配色范围、暗点开关都不在失效键里(Part A 第 3 节),所以改动它们必须**只重画**:阈值 经 `AnalysisReader.samples(policy=…)` 作用在存下来的 `magnitude` 上,配色范围与暗点开关只 存在于绘制调用里。走线是 `Scene3DPanel.settingsChanged` → `on_display_settings()` → -`session.serve_policy()` → `ReplayEngine.redraw()`,重画不移动游标、不写文件。 +`session.serve_policy()` → `ReplayEngine.redraw()`,重画不移动游标、不写文件。这条走线问的是 +`available_result()` 而不是 `analysis_active()`(结果属于录像、游标在录像的时间轴上);被拒的 +栅格要由 `policy_divergence()` 报出是哪几项(第 14 节)。 **栅格类设置被拒绝**:`flow_grid_spacing`、`flow_edge_margin`、`max_arrows`、 -`flow_length_margin`、`flow_arrow_scale` 决定候选位置**在哪里**,而记录下来的位置是 pass 当时 -测的那些。`serve_policy()` 返回 `False` 并提示哪一项动了,画面继续按结果自己的栅格绘制 —— 画 -一份"看起来一样、其实采在别处"的箭头才是真正错误的选项。 +`flow_length_margin`、`flow_arrow_scale` 决定候选位置在哪里,而记录下来的位置是 pass 当时测的 +那些;画面继续按结果自己的栅格绘制,而不是画一份"看起来一样、其实采在别处"的箭头。 `app_window.py` 的接线只有三处,供 Part C 使用:`_analysis_service(recording, cancelled, *, on_progress)`(`chain_spec_for` 从录像自己的 `session.json` 取链路,因此结果可复现)、 @@ -92,36 +95,25 @@ on_progress)`(`chain_spec_for` 从录像自己的 `session.json` 取链路, ## 13. 未完成与取消 `AnalysisSession.__init__` 在 `reader.complete` 为假时抛 `ResultPolicyError`; -`ResultLibrary.open(..., allow_incomplete=True)` 也拒绝 ——「可查看」与「可回放」是两个不同的 -断言。面板的 `cancelled` 与 `failed` 是两个独立状态,都不会显示成完成;这两种情况下时间轴留 -在 `recording` 模式。 +`ResultLibrary.open(..., allow_incomplete=True)` 同样拒绝:可查看与可回放是两个断言。面板的 +`cancelled` 与 `failed` 是两个独立状态,都不会显示成完成;这两种情况下时间轴留在 `recording`。 ## 14. 验收证据 `apps/studio/tests/test_replay_analysis.py`(15 项,真实素材): * **跳转等于顺序播放**:顺序走过全部 33 帧记下每帧,再按 `time_of(frame_id)` 直接跳到 5 个 - 采样位置。容差是零:帧 id、`rows/cols/dx/dy/active` 逐元素相等,流场按位相等(非候选格按 - 构造是 `NaN`,用 `equal_nan` 比较)。跳转精确的原因是给它的时间就是那一帧自己的 `t_rel_s`。 -* **显示设置不写结果**:`scan_results(cache)` 的目录集合在改阈值(0.2 与 1e9)、改显示增益、 - 重复同一栅格前后完全相同。 -* **取消不被当完整**:pass 在第一帧后取消 → 面板报 `cancelled`、`analysis_active()` 为假、 - 引擎停在 `recording` 模式、目录里 `complete=false` 且默认读端抛 - `AnalysisIncompleteError`。 -* **回归**:`apps/studio/tests` 440 项 / 0 失败 / 18 跳过(原 425 项:新增 15 项,既有断言一条 - 未改)。把分析终态写进状态栏时, - `test_the_replay_reset_is_the_display_reset_and_never_the_device` 失败过 —— 合成录像的配置 - 缺失让 pass 在断言之前结束并覆盖了 `复位显示` 的提示;修的是消息落在哪里。 + 采样位置,帧 id、`rows/cols/dx/dy/active` 与流场逐项相等(非候选格按构造是 `NaN`)。 +* **取消不被当完整**:pass 在第一帧后取消 → 面板报 `cancelled`、时间轴留在 `recording`、目录里 + `complete=false` 且默认读端抛 `AnalysisIncompleteError`。 -**深度那一片加的验收**(同一个文件再 +8 项,另加 `shared/tests/test_analysis_depth.py` 10 项): +**深度那一片加的验收**(同一文件再 +8 项,另加 `shared/tests/test_analysis_depth.py` 10 项; +往返误差与尺寸见 Part A 第 1 节): -* **深度往返**:另跑一遍链路,逐帧比 `depth_map`,误差上界是该帧自己的 `(max − min) / 510`, - 在 box 录像(33 帧,1 帧非空、76 px)和 264 帧的 `long_8x.mp4`(241 帧空、23 帧非空、 - 最坏 336 px)上都是 0 失败;非零计数、min/max 与链路的完全相等。 -* **跳转等于顺序播放(深度模式)**:把上面那条测试扩到深度 —— 顺序播放记下每帧渲染出的深度 - 图,再跳到 5 个位置,帧 id、`depth`、统计与**渲染出的字节**逐项相等。 -* **尺寸**:空帧 4,048 B(固定 4,042 + 6 B 载荷头),最坏非空帧 4,534 B(+486 B); - 按实测比例 4,090 B/帧,十分钟 70.2 MB(上限 81.6 MB)。断言写死了这三个数。 +* **深度往返**:另跑一遍链路逐帧比 `depth_map`,在 box 录像与 264 帧的 `long_8x.mp4` 上都是 + 0 失败;非零计数、min/max 与链路的完全相等。 +* **跳转等于顺序播放(深度模式)**:顺序播放记下每帧渲染出的深度图,再跳到 5 个位置,帧 id、 + `depth`、统计与渲染出的字节逐项相等。 * **模式切换不写结果**:四个模式各画一次,`scan_results(cache)` 的目录集合不变。 * **"没什么可画"只在该说的时候说**:无结果时 `plan_image` 只给 `recorded` 并带 `replay_raw_note`;有结果时该行消失、四个模式都在,默认仍是原始图像。两种语言各断言一次, @@ -142,27 +134,41 @@ scratch cache、游标停在存下来的帧 id 上): 二维画面是候选网格上的暗点阵(该帧活动样本很少),三维是手指岛面加同一套样本点;"同一个样本 集"由上面的逐元素断言保证,快照证明这条链路真的走到了 GL。 -**深度那一片的快照**(`.scratch/replay-analysis-a2/snapshots/`,同一个窗口、同一个游标、 -同一个模式选择器;时间轴停在录像上,所以"原始图像"是真的相机图,两个派生模式读结果): +**深度那一片的快照**(`.scratch/replay-analysis-a2/snapshots/`,同一个窗口、同一个游标、同一个 +选择器;"原始图像"是真的相机图,两个派生模式读结果): | 文件 | 内容 | |---|---| | `analysis-finger-frame-33-2d-raw.png` / `-arrows.png` / `-depth.png` | finger 链路帧 33,深度非零 76 px(峰值 0.0242) | -| `analysis-box-frame-27-2d-raw.png` / `-arrows.png` / `-depth.png` | box 链路帧 27,`depth_map` 365×365 **恰好全零**,面板显示"本帧深度全为 0(133225 px 中 0 个非零)" | +| `analysis-box-frame-27-2d-raw.png` / `-arrows.png` / `-depth.png` | box 链路帧 27,`depth_map` 365×365 全零,面板显示"本帧深度全为 0(133225 px 中 0 个非零)" | 同一帧三张图的大小不同(raw 96.6 kB 是相机图,arrows 4.9 kB,depth 4.2 kB),这本身就是 "三个模式画的是三样东西"的证据。 +**接线那一片的验收**(`apps/studio/tests/test_replay_wiring.py`,7 项,真实录像 + 真实 pass;录像 +先复制到临时目录、结果写进它自己的 cache,窗口因此复用而不开链路):操作员路径断言引擎仍在 +`recording`、`available_result()` 非空、选择器给出"原始图像"与三个派生模式、屏上是复用的结果而 +「从视频重算」是关的(改前报 `'analysis' == 'recording'`;选择器里当时没有"原始图像")。 + +箭头与深度的字节分别与存下来的候选/`active` 集合和 `depth_map_to_bgr(存的 depth, …)` 相等 +(这份 box 结果每帧全零,面板用 `panels.depth_empty` 说明);跳转、单步、真定时器播放(0.4 s) +与 pass 完成四条都在录像时间轴上读结果。另两项:改阈值只重画、改栅格被拒但画面照画(改前抛 +`TypeError`)。 + +**接线那一片的快照**(`.scratch/replay-reachable/snapshots/`,真实 GL、真窗口、没有手工调用 +`use_recording()`):`wiring-operator-frame-40-2d-{raw,arrows,depth}.png` 是同一帧的三个模式 +(86 个活动样本;深度全零),`-playback-2d-arrows.png` 是播放到该时刻抓的,`-window.png` 为整窗。 + ## 15. 已知限制 * 结果里没有力:力曲线始终读记录的 `forces.csv`。深度有了(第 1、11 节),但**三维面板仍只 - 拿光流**:`session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False`, - 因为把深度接进三维会改变三维图形的构成,而那不在这一片里。二维深度视图是这一片交付的东西。 -* 结果覆盖的帧是 pass 走过的那些:录像比结果长时游标可以离开结果范围(面板说"这个分析结果里 - 没有可显示的帧"),不会静默重复最后一帧。 + 拿光流**:`session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False` —— 接进 + 三维会改变三维图形的构成,不在这一片里。 +* 结果覆盖的帧是 pass 走过的那些:**结果的第一帧可能晚于录像的起点**(box 链路构造时吃掉前 + 7 个解码帧,这份结果从 t=0.4162 s 开始),最后一批帧也可能早于录像的终点。游标落在这两段里 + 时,派生视图按 `frame_id_at()` 的"该时刻或之前、越界夹到端点"给帧:起点之前画的是最早那一帧 + 测量,终点之后画的是最后一帧。两段没有各自的提示,"不会静默重复最后一帧"在本片没有兑现。 * 栅格类显示设置不生效(第 12 节);换栅格就是重跑一遍,那是**另一个键**。 * 一个进程同时只有一遍分析在跑;采集会话运行期间不启动分析。 -* 时间轴与模式选择是两件事,而"有没有结果"是录像的属性、不是游标的属性: - `ReplayController.available_result()` 查的是**本录像**在 `ResultLibrary` 里的那一份,所以 - 即使 `ReplayEngine.use_recording()` 让引擎不再借用 session(目前 UI 没有第二条路会走到 - 这个状态),二维面板照样能画箭头和深度,`replay_raw_note` 也不会重新出现。 +* "有没有结果"是录像的属性、不是游标的属性:`available_result()` 查的是本录像在 + `ResultLibrary` 里的那一份,第 9 节的接线之后这是常态。 diff --git a/apps/studio/docs/replay-panel.md b/apps/studio/docs/replay-panel.md index ff80496..3af2573 100644 --- a/apps/studio/docs/replay-panel.md +++ b/apps/studio/docs/replay-panel.md @@ -5,9 +5,9 @@ capability states its requirement, data source and a falsifying check. The resul playback semantics are owned by Part A ([replay-analysis.md](replay-analysis.md)) and Part B ([replay-analysis-part-b.md](replay-analysis-part-b.md)). -Status: Parts A and B shipped at `cf49b08`; the depth channel — result format and 2D view — is -uncommitted; the cache-root and lifetime policy and the display-control separation (Part C) -are decided below, not implemented. +Status: Parts A and B shipped at `cf49b08`, the depth channel at `9e20fd0`; the result-aware +mode plan and the wiring that reaches it are in the working tree (Part B §9, §14). The cache-root +and lifetime policy and the display-control separation (Part C) are decided below, not implemented. ## 1. Load a video @@ -110,9 +110,12 @@ frame's depth is all zeros, `panels.depth_unavailable` says the frame has no dep all — different facts, neither drawing the processed image. A mode needing derived data is served from the result, never faked: with none, that view is not -offered and `panels.replay_raw_note` names the action that produces a result. +offered and `panels.replay_raw_note` names the action that produces a result. The state above is +what an operator reaches by opening a recording that has a result: the cursor stays on the +recording's own timeline and the panel opens on a derived view, the recorded one still offered. Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`, -`::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_analysis_depth.py::test_an_empty_map_costs_a_header_and_states_its_shape`. +`::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_analysis_depth.py::test_an_empty_map_costs_a_header_and_states_its_shape`, +`test_replay_wiring.py::test_the_operator_sees_the_cached_views_over_the_recording`. ## Cache and export policies @@ -151,7 +154,7 @@ possibility, belonging to export rather than the cache. Not implemented. |---|---|---| | Format, cache, reader, pass | `orisys_app_shared.analysis` | A/B shipped; depth channel uncommitted | | Timeline, cursor, policy, pass thread, progress surface | `replay_analysis.py`, `replay.py`, `replay_controller.py`, `analysis_worker.py`, `ui/analysis_progress.py` | shipped; the depth view and the result-aware mode plan are uncommitted | -| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams); the result-aware plan is uncommitted | +| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams); the wiring from the operator's open action, and its snapshots, are in the working tree | | Cache root and lifetime | `analysis/cache.py`, app preferences | decided, not implemented | | Display-control separation: rail reset as the measurement reference, a separate display baseline, the F8 view reset | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started; Part B §12 records these as unchanged. The pass accepts a `reference_frame` but `_analysis_service` passes none | diff --git a/apps/studio/src/orisys_studio/replay_controller.py b/apps/studio/src/orisys_studio/replay_controller.py index 2379062..4a96734 100644 --- a/apps/studio/src/orisys_studio/replay_controller.py +++ b/apps/studio/src/orisys_studio/replay_controller.py @@ -29,6 +29,7 @@ from typing import Any, Protocol import numpy as np from orisys_app_shared.acquisition.image_backend import ( + ARROW_PURE, DERIVED_MODES, SOURCE_ANALYSIS, SOURCE_REPLAY, @@ -45,6 +46,7 @@ from orisys_studio.replay_analysis import ( AnalysisSession, ResultLibrary, ResultPolicyError, + policy_divergence, ) logger = logging.getLogger(__name__) @@ -620,7 +622,13 @@ class ReplayController(QObject): # ---------------------------------------------------------------- analysis def analysis_session(self) -> AnalysisSession | None: - """The stored result the timeline is reading, if any.""" + """Whatever result is open: the cursor's own, else the library's. + + Not the question a view asks — that is :meth:`available_result`, which answers "is + there a measurement for *this recording*" and is what the app's timeline (always + the recording's) is served from. This one is the raw pair, kept for a caller that + wants the session itself. + """ session = self._engine.analysis() if session is not None: return session @@ -658,19 +666,25 @@ class ReplayController(QObject): ``test_replay_analysis`` that the result directory set is unchanged is a test of this method, not of the panel. + It is asked of ``available_result``, not of ``analysis_active``: a result is a + property of the recording, and the cursor is normally on the recording's own + timeline while the arrows and the depth map are drawn from it. Gating on the + timeline instead would make a threshold change stop mattering in exactly the + state this view is served in. + A policy that moves the sample *grid* cannot be served by the stored candidate - locations; :meth:`AnalysisSession.serve_policy` says so, the message names the - key, and the frame is still redrawn from the result's own grid, because + locations; :meth:`AnalysisSession.serve_policy` refuses it, the message names the + keys, and the frame is still redrawn from the result's own grid, because re-gridding it would draw samples nothing measured. """ - session = self.analysis_session() - if session is None or not self.analysis_active(): + session = self.available_result() + if session is None: return from orisys_app_shared.scene3d.visual_policy import policy_for_settings policy = policy_for_settings(settings) - moved = not session.serve_policy(policy) - if moved: + if not session.serve_policy(policy): + moved = policy_divergence(session.stored_policy(), policy) self._host._notify(t("playback.analysis_grid_mismatch", keys="·".join(moved))) self._engine.redraw() @@ -765,15 +779,54 @@ class ReplayController(QObject): return True def _adopt_session(self, session: AnalysisSession) -> None: - """One timeline: the cursor, the charts and both figures read this result.""" + """One timeline, one cache behind it: the cursor, the charts and both figures. + + **The engine stays on the recording.** The stored result is adopted *behind* the + cursor — :meth:`available_result` answers "is there a measurement for this + recording?" — and ``ReplayEngine.use_recording()`` is what keeps the cursor's own + timeline the recording's: its signal time, its own frame ids, and its whole span, + so a position the pass did not cover is still a position the operator can be at. + Moving the cursor onto the result instead (``use_analysis``) is what made the + recording's own picture unreachable: with the cursor on a result, the recorded + view has nothing to decode. + """ self.results.open_session(session) - self._engine.use_analysis(session) + self._engine.use_recording() self._charts.set_recording_source(self._engine.reader()) - self._image_panel.set_source_kind("analysis") + # The timeline is the recording's, so the panel is told "replay" — the same word + # ``apply_source`` uses for an open recording. It used to be told "analysis", + # which is not a 2D source kind at all: ``ArrowPanel`` knows live/recompute/replay + # and coerced it, so a stored result was worded as the live pipeline. + self._image_panel.set_source_kind("replay") self._mesh_panel.set_source_kind("analysis") if self._actions is not None: self._actions.recompute.setEnabled(True) self.render_frame() + # A result just became readable, so it is what the operator is shown — unless a + # derived view was already on screen (a depth view chosen while the pass ran). + self.serve_stored_view() + + def serve_stored_view(self) -> bool: + """Put the cached view on screen when a result becomes readable. + + The trigger for "draw the cache without being asked": a stored result is adopted + — on opening a recording that has one, or when a pass commits — and the panel is + showing the recording's own picture, so the operator would otherwise be left + looking at the camera while the measurement sits on disk. The recording's own + picture stays in the same selector, one step away, and the recompute switch is + not involved. + + Answers whether the selection moved. A derived selection is left alone: the mode + the operator chose is not this method's to override. + """ + current = str(getattr(self._image_panel, "view_mode", lambda: "")() or "") + if current in DERIVED_MODES: + return False + # The plan for the frame just drawn offers the derived modes (the frame carries + # ``result_available``), so this selection is one the panel can honour. + self._image_panel.set_view_mode(ARROW_PURE) + self.render_frame() + return True def _display_policy(self) -> Any: from orisys_app_shared.scene3d.visual_policy import policy_for_settings @@ -907,6 +960,10 @@ class ReplayController(QObject): map can be drawn at all. The session the library holds is checked against the recording that is open, so a previous recording's result cannot answer for this one. + + This is the normal state of a replay, not a fallback: the cursor stays on the + recording's own timeline (``_adopt_session``) and the result is the cache behind + it, so the answers and the view modes come from here. """ session = self._engine.analysis() if session is not None: @@ -950,12 +1007,13 @@ class ReplayController(QObject): def render_frame(self) -> None: """Draw what the 2D panel should show for the current replay cursor. - One method for both timeline modes, because it is one cursor: in analysis mode - the picture is the stored frame's own flow field and depth map (the pass's - candidate locations, on the domain grid), and in recording mode it is the - decoded video frame. The window's single 2D sink draws both through the *same* - backend, so the arrows, the idle dots, the colour range and the domain boundary - behave identically in the two modes. + One method for both timeline modes, because it is one cursor: with the cursor on + a stored result (``analysis_active``) the picture is the stored frame's own flow + field and depth map, and with the cursor on the recording's own timeline it is the + decoded video frame — unless the *view* the operator asked for is a measurement, + in which case it is read from the result for the same cursor. The window's single + 2D sink draws all of them through the *same* backend, so the arrows, the idle + dots, the colour range and the domain boundary behave identically. **A mode that needs derived data is served from the result, never faked.** The arrows and the depth map are the pass's measurements; with a result in hand they diff --git a/apps/studio/tests/test_replay_analysis.py b/apps/studio/tests/test_replay_analysis.py index bdaa64b..320ddbd 100644 --- a/apps/studio/tests/test_replay_analysis.py +++ b/apps/studio/tests/test_replay_analysis.py @@ -757,9 +757,12 @@ def test_opening_a_recording_with_a_stored_result_runs_no_pass(pass_root): controller, host, engine, parts = _controller(host=host) assert controller.load_recording(RECORDING) is True assert calls == [str(RECORDING)], f"the service was built {len(calls)} times" - assert controller.analysis_active(), "the timeline did not adopt the stored result" - assert engine.mode() == MODE_ANALYSIS - assert parts[2].kinds[-1] == "analysis", "the 2D panel was not told it is a result" + # Adoption is "the recording has a readable result", not "the cursor moved onto it": + # the engine keeps the recording's own timeline (Part B §9), so the session is held + # by the library and answered through ``available_result``. + assert controller.available_result() is not None, "the stored result was not adopted" + assert engine.mode() == MODE_RECORDING, "adopting a result moved the cursor's timeline" + assert parts[2].kinds[-1] == "replay", "the 2D panel was not told the cursor is a replay" assert parts[3].kinds[-1] == "analysis", "the 3D pane was not told it is a result" assert parts[4].states[-1] == "reused", "the surface did not report the reuse" assert stored.is_dir() @@ -782,8 +785,8 @@ def test_opening_a_recording_without_a_result_starts_the_pass(tmp_path, qt_app): assert controller.wait_for_analysis(120) _flush() assert parts[4].states[-1] == "complete", parts[4].states - assert controller.analysis_active(), "a complete result was not adopted" - assert engine.mode() == MODE_ANALYSIS + assert controller.available_result() is not None, "a complete result was not adopted" + assert engine.mode() == MODE_RECORDING, "a finished pass moved the cursor's timeline" def test_a_cancelled_pass_is_reported_and_never_served(tmp_path, qt_app): diff --git a/apps/studio/tests/test_replay_wiring.py b/apps/studio/tests/test_replay_wiring.py new file mode 100644 index 0000000..acc0baf --- /dev/null +++ b/apps/studio/tests/test_replay_wiring.py @@ -0,0 +1,507 @@ +"""Part B's wiring: the cached analysis views over a recording, in the running app. + +The result contract ([replay-analysis.md](../docs/replay-analysis.md)), the playback +contract ([replay-analysis-part-b.md](../docs/replay-analysis-part-b.md)) and the panel +contract ([replay-panel.md](../docs/replay-panel.md)) all describe one state: a recording +is open, the cursor is on the recording's own signal time, and the arrows and the depth +map are served from the stored analysis result while the operator scrubs and plays — with +the recording's own picture one step away in the mode selector, and the recompute switch +left as the explicit "run the chain over this footage again" action. + +What was missing was the wiring that reaches it. ``ReplayEngine.use_recording()`` had no +caller, so analysis-on-open moved the cursor onto the *result's* timeline instead: the +derived views were drawn, but the recording's own picture was not offered at all, and the +cursor could not leave the frames the result covers. These tests drive the window's own +「打开录像」 action over a recording whose result is already stored beside it — the result +is generated here into the recording's own cache, so the app *reuses* it and opens no SDK +chain — and assert on the picture the panel actually shows, not on a state a harness +built. + +The recording is copied before use: a pass writes its result into the recording's own +directory, and nothing may write into ``dist/``. +""" + +from __future__ import annotations + +import shutil +import time +from pathlib import Path + +import numpy as np +import pytest +from orisys_app_shared.acquisition.arrows import depth_map_to_bgr, flow_colormap +from orisys_app_shared.acquisition.image_backend import ( + ARROW_PURE, + ARROW_STITCHED, + DEPTH, + DERIVED_MODES, + RECORDED, +) +from orisys_app_shared.i18n import t +from orisys_app_shared.scene3d.contracts import DisplaySettings +from orisys_app_shared.scene3d.visual_policy import ( + select_flow_samples, +) +from orisys_studio.replay import MODE_RECORDING +from PySide6.QtWidgets import QApplication + +REPO = Path(__file__).resolve().parents[3] +STUDIO = REPO / "dist" / "OrisysStudio" +SOURCE_RECORDING = ( + STUDIO + / "outputs" + / "recordings" + / "20260930T015028Z_session0002_40cb8e7441434feab48bb646afdba011" +) +#: What makes a directory a recording for the application, and all a replay reads. +RECORDING_FILES = ( + "frames.csv", + "forces.csv", + "events.csv", + "session.json", + "raw_0001.mp4", + "preroll_0001.mp4", +) + + +def _require_inputs() -> None: + missing = [ + str(SOURCE_RECORDING / name) + for name in ("frames.csv", "raw_0001.mp4", "session.json") + if not (SOURCE_RECORDING / name).exists() + ] + if missing: + pytest.skip( + "the replay wiring tests need the evidence recording: " + ", ".join(missing) + ) + pytest.importorskip("orisys", reason="the analysis pass drives the SDK chain") + + +def _copy_recording(destination: Path) -> Path: + """The evidence recording, copied: nothing may write into ``dist/``.""" + destination.mkdir(parents=True, exist_ok=True) + for name in RECORDING_FILES: + source = SOURCE_RECORDING / name + if source.exists(): + shutil.copy2(source, destination / name) + return destination + + +def _run_the_pass(recording: Path): + """One real pass, with the chain the *application* derives from the recording.""" + from orisys_app_shared.analysis.service import AnalysisService, chain_spec_for + from orisys_app_shared.record import RecordingReader + + reader = RecordingReader(recording) + try: + chain = chain_spec_for(reader.session()) + finally: + reader.close() + return AnalysisService(recording, chain=chain).run() + + +@pytest.fixture(scope="module") +def stored_recording(tmp_path_factory) -> Path: + """A copy of the evidence recording whose own cache already holds a result.""" + _require_inputs() + recording = _copy_recording(tmp_path_factory.mktemp("replay-wiring") / "recording") + outcome = _run_the_pass(recording) + assert outcome.complete, outcome.reason + assert outcome.frames_written > 0 + return recording + + +@pytest.fixture +def window(make_window, qt_app): + """The application, in the replay view, as the operator meets it.""" + window = make_window(sensor_type="box", source_mode="camera") + window.resize(1440, 880) + window.show() + QApplication.processEvents() + window.brand_bar.btn_playback.click() + QApplication.processEvents() + assert window._in_replay_view(), "the window did not switch to the replay view" + return window + + +def _open(window, recording: Path, monkeypatch) -> None: + """Press the window's own 「打开录像」 entry, with the file dialog answered for it.""" + import orisys_studio.replay_controller as replay_controller + + video = Path(recording) / "raw_0001.mp4" + monkeypatch.setattr( + replay_controller.QFileDialog, + "getOpenFileName", + lambda *args, **kwargs: (str(video), ""), + ) + action = window.replay_ctl._actions.playback + assert action.isEnabled(), "the replay entries are not offered in the replay view" + action.trigger() + QApplication.processEvents() + assert window.replay.has_video(), "the recording did not open" + + +def _sink_frames(window, monkeypatch) -> list: + """Every frame handed to the window's single 2D sink, in order.""" + captured: list = [] + original = window._show_image_frame + + def spy(frame): + captured.append(frame) + return original(frame) + + monkeypatch.setattr(window, "_show_image_frame", spy) + return captured + + +def _panel_bgr(window) -> np.ndarray: + """The 2D panel's picture, in BGR, from the end of the real display path.""" + image = window.arrow_panel._source_image + assert image is not None and not image.isNull(), "the 2D panel shows no picture" + width, height = image.width(), image.height() + rows = np.frombuffer(bytes(image.constBits()), dtype=np.uint8) + rows = rows.reshape(height, image.bytesPerLine()) + return rows[:, : width * 3].reshape(height, width, 3)[:, :, ::-1].copy() + + +def _expected_depth(window, frame) -> np.ndarray: + """A stored frame's depth view, drawn the way the window's own sink draws it.""" + return depth_map_to_bgr( + frame.depth, + flow_colormap(window._theme), + value_range=window._depth_range(), + invalid_mask=window._domain_mask.mask, + ) + + +def _set_mode(window, mode: str) -> None: + index = window.arrow_panel.view_combo.findData(mode) + assert index >= 0, f"the mode {mode} is not offered: {window.arrow_panel.view_modes()}" + window.arrow_panel.view_combo.setCurrentIndex(index) + QApplication.processEvents() + assert window.arrow_panel.view_mode() == mode + + +def _pump(seconds: float) -> None: + """Let real time pass so the playback timer fires, as it does in the app.""" + deadline = time.perf_counter() + float(seconds) + while time.perf_counter() < deadline: + QApplication.processEvents() + time.sleep(0.01) + + +def _cells(rows, cols) -> set: + return {(int(row), int(col)) for row, col in zip(np.asarray(rows), np.asarray(cols))} + + +def _active_cells(samples) -> set: + return { + (int(row), int(col)) + for row, col, active in zip( + np.asarray(samples.rows), np.asarray(samples.cols), np.asarray(samples.active) + ) + if bool(active) + } + + +def test_the_operator_sees_the_cached_views_over_the_recording( + stored_recording, window, monkeypatch +): + """The state the contracts describe, reached the way an operator reaches it. + + A recording with a result beside it opens on the **recording's** timeline (the result + is a cache behind the cursor, not a second timeline), the panel offers the recording's + own picture *and* the three derived views, and what is on screen is the stored frame — + not the camera, and not a placeholder. + """ + captured = _sink_frames(window, monkeypatch) + _open(window, stored_recording, monkeypatch) + + assert window.replay.mode() == MODE_RECORDING, ( + "the cursor left the recording's own timeline; the result is a cache behind it" + ) + session = window.replay_ctl.available_result() + assert session is not None, "the stored result beside the recording was not adopted" + assert not window.replay_ctl.recompute_active(), ( + "the cache was only reachable by turning the recompute switch on" + ) + assert window.analysis_progress.state() == "reused", ( + "a pass ran over a recording that already had a result" + ) + + assert window.arrow_panel.view_modes() == ( + RECORDED, + ARROW_PURE, + ARROW_STITCHED, + DEPTH, + ), "the recording's own picture must stay one step away from the derived views" + assert window.arrow_panel.view_mode() in DERIVED_MODES, ( + "the operator was left on the camera picture although a result is served" + ) + assert window.arrow_panel.note_label.text() == "", ( + "the panel says there is nothing to draw while it is drawing a result" + ) + + # The arrow view is the stored measurement, at the cursor's own frame. + frames = list(session.frames()) + active_frame = next(frame for frame in frames if frame.active_count > 0) + window.replay.set_position(session.time_of(active_frame.frame_id)) + QApplication.processEvents() + assert captured, "nothing reached the 2D sink" + drawn = captured[-1] + assert drawn.frame_id == active_frame.frame_id + assert drawn.result_available is True + stored_cells = _cells(active_frame.samples.rows, active_frame.samples.cols) + field = np.asarray(drawn.flow) + finite = np.argwhere(np.isfinite(field[..., 0])) + assert _cells(finite[:, 0], finite[:, 1]) == stored_cells, ( + "the recovered flow field carries values outside the stored candidate locations" + ) + # The height last: the shared selector re-reads the same cells, in its own order, and + # comes to the same activity verdict as the pass stored. + reselected = select_flow_samples(field, drawn.valid_mask, session.display_policy) + assert _cells(reselected.rows, reselected.cols) == stored_cells + assert _active_cells(reselected) == _active_cells(active_frame.samples) + arrows_bgr = _panel_bgr(window) + raw_here = window.replay.video_frame_bgr() + assert raw_here is not None + assert not np.array_equal(arrows_bgr, np.asarray(raw_here)), ( + "the arrow view is the recording's raw picture" + ) + + # The depth view is the stored map, drawn through the window's own backend. This + # result's map is exactly zero on every frame it holds — the box chain's depth is + # flat on this footage, which the panel has its own sentence for — and it is still a + # stored array of the result's own shape, so the pixels are checked against it rather + # than against a flat placeholder. + _set_mode(window, DEPTH) + target = active_frame + assert target.depth is not None and target.depth_nonzero == 0 + QApplication.processEvents() + drawn = captured[-1] + assert drawn.frame_id == target.frame_id + assert np.array_equal( + np.asarray(drawn.depth), np.asarray(target.depth) + ), "the depth view is not the stored map" + shown = _panel_bgr(window) + expected = _expected_depth(window, target) + assert expected is not None + assert shown.shape == expected.shape + assert np.array_equal(shown, expected), ( + "the depth view's pixels are not depth_map_to_bgr of the stored frame" + ) + assert not np.array_equal(shown, arrows_bgr), "the depth view drew the arrow view" + assert t("panels.depth_empty", pixels=int(target.depth.size)) in ( + window.arrow_panel.depth_scale_label.text() + ), "an exactly-zero depth map is not stated as such" + + # …and the recording's own picture is still reachable, at the same cursor. + _set_mode(window, RECORDED) + raw = window.replay.video_frame_bgr() + assert raw is not None, "the recording's own frame could not be decoded" + assert np.array_equal(_panel_bgr(window), np.asarray(raw)), ( + "the recorded view is not the recording's own picture" + ) + assert window.arrow_panel.note_label.text() == "" + assert window.replay.mode() == MODE_RECORDING + + +def test_playing_forward_and_seeking_draw_the_same_cached_frame( + stored_recording, window, monkeypatch +): + """A seek is the frame playing forward passes through — on the operator's path.""" + captured = _sink_frames(window, monkeypatch) + _open(window, stored_recording, monkeypatch) + session = window.replay_ctl.available_result() + assert session is not None + _set_mode(window, DEPTH) + + ids = [int(value) for value in session.frame_ids] + target_id = ids[len(ids) // 4] + moment = session.time_of(target_id) + assert moment is not None + + # Play forward from the top, a tick's worth of cursor at a time, ending on the + # target instant itself. + positions = [step / 240.0 for step in range(int(moment * 240.0) + 1)] + positions.append(moment) + for position in positions: + window.replay.set_position(position) + played = _panel_bgr(window) + played_id = captured[-1].frame_id + + # …then seek straight to the same instant. + window.replay.set_position(moment) + QApplication.processEvents() + assert captured[-1].frame_id == played_id == target_id + assert np.array_equal(played, _panel_bgr(window)), ( + "a seek drew a different frame from playing forward to the same position" + ) + + # Stepping one stored frame on moves to that frame's own picture. + later = ids[ids.index(target_id) + 1] + window.replay.set_position(session.time_of(later)) + QApplication.processEvents() + assert captured[-1].frame_id == later + assert np.array_equal( + np.asarray(captured[-1].depth), np.asarray(session.frame(later).depth) + ) + + +def test_playback_draws_the_cached_frame_not_the_video( + stored_recording, window, monkeypatch +): + """Advance the real playback timer and check every drawn frame against the cache.""" + captured = _sink_frames(window, monkeypatch) + _open(window, stored_recording, monkeypatch) + session = window.replay_ctl.available_result() + assert session is not None + _set_mode(window, DEPTH) + + window.replay.set_position(0.0) + QApplication.processEvents() + captured.clear() + window.replay_ctl.start_playback() + assert window.replay_ctl.is_playing(), "playback did not start" + _pump(0.4) + position = window.replay.position() + assert position > 0.0, "the playhead did not advance" + assert captured, "playback drew nothing" + assert position <= session.duration_s(), "the playhead left the recording's timeline" + + expected_id = session.frame_id_at(position) + assert expected_id is not None + for frame in captured: + assert frame.depth is not None, "a frame drawn during playback carries no depth" + assert frame.result_available is True + assert session.has(int(frame.frame_id)), ( + f"playback drew frame {frame.frame_id}, which the result does not hold" + ) + # Pausing leaves the frame the cursor is on, and it is the cached one. + window.replay_ctl.pause_playback() + assert not window.replay_ctl.is_playing() + assert captured[-1].frame_id == expected_id + expected = _expected_depth(window, session.frame(expected_id)) + assert expected is not None + assert np.array_equal(_panel_bgr(window), expected), ( + "the paused frame is not the stored frame for the cursor" + ) + + +def test_a_finished_pass_puts_the_cache_on_screen(tmp_path, window, monkeypatch): + """The other trigger: the pass completes and the panel serves what it stored. + + Nothing is pre-placed here, so the window runs the pass over the copy (a box chain: + a couple of seconds for this recording) and the *completion* is what has to leave the + derived view on screen — with the cursor still on the recording. + """ + recording = _copy_recording(tmp_path / "recording") + _sink_frames(window, monkeypatch) + _open(window, recording, monkeypatch) + + assert window.replay_ctl.wait_for_analysis(180.0), "the pass did not finish" + QApplication.processEvents() + assert window.replay_ctl.analysis_error() == "", ( + f"the pass left no usable result: {window.replay_ctl.analysis_error()}" + ) + assert window.analysis_progress.state() == "complete" + session = window.replay_ctl.available_result() + assert session is not None, "the completed result was not adopted" + assert window.replay.mode() == MODE_RECORDING, ( + "a finished pass moved the cursor off the recording's own timeline" + ) + assert window.arrow_panel.view_mode() in DERIVED_MODES, ( + "the operator was left on the camera picture after the pass finished" + ) + assert not window.replay_ctl.recompute_active() + + frame_id = session.frame_id_at(window.replay.position()) + assert frame_id is not None + _set_mode(window, DEPTH) + expected = _expected_depth(window, session.frame(frame_id)) + assert expected is not None + assert np.array_equal(_panel_bgr(window), expected), ( + "the finished result is not what the panel draws" + ) + + +def test_the_mode_selector_is_the_trigger_for_a_derived_view( + stored_recording, window, monkeypatch +): + """On the recording's timeline, choosing a derived view serves it from the cache.""" + captured = _sink_frames(window, monkeypatch) + _open(window, stored_recording, monkeypatch) + session = window.replay_ctl.available_result() + assert session is not None + _set_mode(window, RECORDED) + + position = session.time_of(int(session.frame_ids[len(session.frame_ids) // 2])) + assert position is not None + window.replay.set_position(position) + QApplication.processEvents() + assert window.replay.mode() == MODE_RECORDING + assert captured[-1].depth is None, "the recorded view drew a stored frame" + + captured.clear() + _set_mode(window, DEPTH) + assert captured, "choosing a derived view drew nothing" + assert captured[-1].depth is not None, ( + "a view that needs derived data drew the recording's raw picture" + ) + assert captured[-1].frame_id == session.frame_id_at(position) + assert window.replay.mode() == MODE_RECORDING, ( + "choosing a view moved the cursor's timeline" + ) + + +def test_the_display_settings_still_redraw_from_the_cache( + stored_recording, window, monkeypatch +): + """A threshold change is a read: it redraws the stored frame and writes nothing.""" + from orisys_app_shared.analysis.cache import scan_results + + _open(window, stored_recording, monkeypatch) + session = window.replay_ctl.available_result() + assert session is not None + _set_mode(window, ARROW_PURE) + + cache_root = Path(stored_recording) / "analysis" + before = {entry.directory for entry in scan_results(cache_root)} + captured = _sink_frames(window, monkeypatch) + window.replay_ctl.on_display_settings(DisplaySettings(flow_on_px=0.9)) + QApplication.processEvents() + + assert session.threshold() == pytest.approx(0.9), ( + "the display threshold never reached the session" + ) + assert captured, "a display change did not redraw the frame" + assert captured[-1].frame_id == session.frame_id_at(window.replay.position()) + assert {entry.directory for entry in scan_results(cache_root)} == before, ( + "a display change wrote an analysis result" + ) + + +def test_a_setting_that_moves_the_grid_is_still_refused( + stored_recording, window, monkeypatch +): + """The stored candidate locations cannot be re-gridded; the frame is drawn anyway.""" + from orisys_app_shared.analysis.cache import scan_results + + _open(window, stored_recording, monkeypatch) + session = window.replay_ctl.available_result() + assert session is not None + _set_mode(window, ARROW_PURE) + before_threshold = session.threshold() + cache_root = Path(stored_recording) / "analysis" + before = {entry.directory for entry in scan_results(cache_root)} + + captured = _sink_frames(window, monkeypatch) + window.replay_ctl.on_display_settings(DisplaySettings(max_arrows=50)) + QApplication.processEvents() + + assert session.threshold() == pytest.approx(before_threshold), ( + "a refused policy changed the session" + ) + assert captured, "the frame was not redrawn from the result's own grid" + assert captured[-1].frame_id == session.frame_id_at(window.replay.position()) + assert {entry.directory for entry in scan_results(cache_root)} == before