Make the cached analysis views reachable over a recording

Replay moved the cursor onto the result's timeline, so the recording's own
picture was not offered at all and ``use_recording()`` had no caller. Adopting a
result now keeps the cursor on the recording, opens the panel on a derived view,
and serves scrubs, playback and display changes from the cache (Part B §9).
Verified: apps/studio 455 items / 0 failed / 18 skipped, shared 427 / 0 / 1,
autocal 358, ui_check 628 checks / 0 failures, lint gate clean.
This commit is contained in:
ykcheungab
2026-10-04 16:04:02 +08:00
parent fbf32b5303
commit 1aa454311f
5 changed files with 649 additions and 72 deletions
+51 -45
View File
@@ -15,14 +15,18 @@
| `recording` | 录像自己的 `raw_*.mp4`,按需解码 | 这个画面是原始相机图像:它自己没有光流也没有深度 |
| `analysis` | 存下来的分析结果,按帧读 | 有 pass 记录下来的光流样本、候选网格与深度图 |
模式说的是**画面从哪里读**,而"有没有可画的东西"是另一件事:只要有结果,二维面板的模式
选择器就提供 深度图/箭头(时间轴在录像上时连"原始图像"一起提供),选了派生模式的那些帧
一律从结果读、按窗口唯一的 2D 出口画 —— 绝不会把原始图像当成箭头图交出去。
模式说的是画面从哪里读,"有没有可画的东西"是另一件事:只要有结果,选择器就提供箭头与深度图
(连"原始图像"一起),派生模式那些帧一律从结果读、按窗口唯一的 2D 出口画。
图表、二维画布、三维面板都读这一个游标,模式由 `ReplayEngine.mode()` 回答;「从视频重算」
因此不再是第三种来源,而是**显式重跑**这一遍的开关(第 10 节)。时间基准是**信号时间**
(Part A 第 5 节),进度、位置、单步与跳转都说它。
**应用里的时间轴永远是录像的。** 引擎留在 `recording`:游标、帧 id 与时长都是录像自己的,
结果挂在游标后面当缓存(`available_result()`),`_adopt_session()` 调用 `use_recording()` 保证
这一点。缓存有三个触点,不必先打开「从视频重算」:复用已有结果、pass 完成、选择器选中派生模式
(时间轴不动)。
**帧身份只来自录像自己的计数器。** 证据录像 id 是 20…52 而非 0…32,所以
`AnalysisSession.frame(frame_id)` 对结果里没有的 id 报 `AnalysisNotFoundError`:把索引当 id
传进去不会被"顺手改成能读的那一帧",那会返回一张看起来完全正确的图。按时间取帧走
@@ -43,15 +47,14 @@
**并发与工作目录。** 开链路会往当前目录写 2 MB 预览图(Part A 第 6 节),所以 pass 跑在**本
进程**的一个线程里,启动前要求没有采集会话、没有预览、也没有第二遍分析,并且**本模块从不改
工作目录**。锁只覆盖开链路那一小段(`_working_directory` 在 `_OpenChain.open()` 内进出),
逐帧处理在锁之外,GUI 线程这期间正常刷新。关窗时先取消并等待 pass。
工作目录**。锁只覆盖开链路那一小段,逐帧处理在锁之外,GUI 线程这期间正常刷新。
## 11. 三视图如何被同一个位置喂
| 视图 | 读什么 | 怎么读 |
|---|---|---|
| 图表 | `forces.csv`(记录值) | `charts.set_cursor(t)` 用同一游标;结果里不存力 |
| 二维画布 | 结果的 `flow`、`depth` 与 `valid_mask` | `ImageFrame(source="analysis")` 走窗口唯一的 2D 出口 `_show_image_frame`,箭头、暗点、配色、域边界与深度视图与实时/重算**同一套代码** |
| 二维画布 | 结果的 `flow`、`depth` 与 `valid_mask` | `ImageFrame` 走窗口唯一的 2D 出口 `_show_image_frame`(录像时间轴上的派生视图带 `source="replay"` 与 `result_available=True`,因此"原始图像"仍在选择器里),箭头、暗点、配色、域边界与深度视图与实时/重算**同一套代码** |
| 三维面板 | 同一帧的 `SdkPacket` | `session.packet(frame)` → `Scene3DView.submit_packet`,走同一个 adapter、域掩码 latch 与有效性求交 |
**深度视图也由结果喂。** `AnalysedFrame.depth` 就是第 1 节那幅量化平面重建出来的
@@ -68,20 +71,20 @@
`dx`/`dy` 是**画出来的**位移,已经带上长度余量那一步的缩放:只用前两个因子会让每根箭头短
20%(finger 余量 0.8),330 个样本里 319 个不同、活动集也不同 —— 实测的,不是推导的。余量补
上后,重建流场重新过一遍共享选区器,样本集与存下来的**逐元素相等**(33 帧全覆盖)。三维面板
直接拿存下来的 `FlowSamples`,所以"两张图是同一个测量"由构造决定。
上后,重建流场重新过一遍共享选区器,样本集与存下来的**逐元素相等**(33 帧全覆盖)。
## 12. 显示设置是读
阈值、配色范围、暗点开关都不在失效键里(Part A 第 3 节),所以改动它们必须**只重画**:阈值
经 `AnalysisReader.samples(policy=…)` 作用在存下来的 `magnitude` 上,配色范围与暗点开关只
存在于绘制调用里。走线是 `Scene3DPanel.settingsChanged` → `on_display_settings()` →
`session.serve_policy()` → `ReplayEngine.redraw()`,重画不移动游标、不写文件。
`session.serve_policy()` → `ReplayEngine.redraw()`,重画不移动游标、不写文件。这条走线问的是
`available_result()` 而不是 `analysis_active()`(结果属于录像、游标在录像的时间轴上);被拒的
栅格要由 `policy_divergence()` 报出是哪几项(第 14 节)。
**栅格类设置被拒绝**:`flow_grid_spacing`、`flow_edge_margin`、`max_arrows`、
`flow_length_margin`、`flow_arrow_scale` 决定候选位置**在哪里**,而记录下来的位置是 pass 当时
测的那些。`serve_policy()` 返回 `False` 并提示哪一项动了,画面继续按结果自己的栅格绘制 —— 画
一份"看起来一样、其实采在别处"的箭头才是真正错误的选项。
`flow_length_margin`、`flow_arrow_scale` 决定候选位置在哪里,而记录下来的位置是 pass 当时测的
那些;画面继续按结果自己的栅格绘制,而不是画一份"看起来一样、其实采在别处"的箭头。
`app_window.py` 的接线只有三处,供 Part C 使用:`_analysis_service(recording, cancelled, *,
on_progress)`(`chain_spec_for` 从录像自己的 `session.json` 取链路,因此结果可复现)、
@@ -92,36 +95,25 @@ on_progress)`(`chain_spec_for` 从录像自己的 `session.json` 取链路,
## 13. 未完成与取消
`AnalysisSession.__init__` 在 `reader.complete` 为假时抛 `ResultPolicyError`;
`ResultLibrary.open(..., allow_incomplete=True)` 也拒绝 ——「可查看」与「可回放」是两个不同的
断言。面板的 `cancelled` 与 `failed` 是两个独立状态,都不会显示成完成;这两种情况下时间轴留
在 `recording` 模式。
`ResultLibrary.open(..., allow_incomplete=True)` 同样拒绝:可查看与可回放是两个断言。面板的
`cancelled` 与 `failed` 是两个独立状态,都不会显示成完成;这两种情况下时间轴留在 `recording`。
## 14. 验收证据
`apps/studio/tests/test_replay_analysis.py`(15 项,真实素材):
* **跳转等于顺序播放**:顺序走过全部 33 帧记下每帧,再按 `time_of(frame_id)` 直接跳到 5 个
采样位置。容差是零:帧 id、`rows/cols/dx/dy/active` 逐元素相等,流场按位相等(非候选格按
构造是 `NaN`,用 `equal_nan` 比较)。跳转精确的原因是给它的时间就是那一帧自己的 `t_rel_s`。
* **显示设置不写结果**:`scan_results(cache)` 的目录集合在改阈值(0.2 与 1e9)、改显示增益、
重复同一栅格前后完全相同。
* **取消不被当完整**:pass 在第一帧后取消 → 面板报 `cancelled`、`analysis_active()` 为假、
引擎停在 `recording` 模式、目录里 `complete=false` 且默认读端抛
`AnalysisIncompleteError`。
* **回归**:`apps/studio/tests` 440 项 / 0 失败 / 18 跳过(原 425 项:新增 15 项,既有断言一条
未改)。把分析终态写进状态栏时,
`test_the_replay_reset_is_the_display_reset_and_never_the_device` 失败过 —— 合成录像的配置
缺失让 pass 在断言之前结束并覆盖了 `复位显示` 的提示;修的是消息落在哪里。
采样位置,帧 id、`rows/cols/dx/dy/active` 与流场逐项相等(非候选格按构造是 `NaN`)。
* **取消不被当完整**:pass 在第一帧后取消 → 面板报 `cancelled`、时间轴留在 `recording`、目录里
`complete=false` 且默认读端抛 `AnalysisIncompleteError`。
**深度那一片加的验收**(同一个文件再 +8 项,另加 `shared/tests/test_analysis_depth.py` 10 项):
**深度那一片加的验收**(同一文件再 +8 项,另加 `shared/tests/test_analysis_depth.py` 10 项;
往返误差与尺寸见 Part A 第 1 节):
* **深度往返**:另跑一遍链路,逐帧比 `depth_map`,误差上界是该帧自己的 `(max − min) / 510`,
在 box 录像(33 帧,1 帧非空、76 px)和 264 帧的 `long_8x.mp4`(241 帧空、23 帧非空、
最坏 336 px)上都是 0 失败;非零计数、min/max 与链路的完全相等。
* **跳转等于顺序播放(深度模式)**:把上面那条测试扩到深度 —— 顺序播放记下每帧渲染出的深度
图,再跳到 5 个位置,帧 id、`depth`、统计与**渲染出的字节**逐项相等。
* **尺寸**:空帧 4,048 B(固定 4,042 + 6 B 载荷头),最坏非空帧 4,534 B(+486 B);
按实测比例 4,090 B/帧,十分钟 70.2 MB(上限 81.6 MB)。断言写死了这三个数。
* **深度往返**:另跑一遍链路逐帧比 `depth_map`,在 box 录像与 264 帧的 `long_8x.mp4` 上都是
0 失败;非零计数、min/max 与链路的完全相等。
* **跳转等于顺序播放(深度模式)**:顺序播放记下每帧渲染出的深度图,再跳到 5 个位置,帧 id、
`depth`、统计与渲染出的字节逐项相等。
* **模式切换不写结果**:四个模式各画一次,`scan_results(cache)` 的目录集合不变。
* **"没什么可画"只在该说的时候说**:无结果时 `plan_image` 只给 `recorded` 并带
`replay_raw_note`;有结果时该行消失、四个模式都在,默认仍是原始图像。两种语言各断言一次,
@@ -142,27 +134,41 @@ scratch cache、游标停在存下来的帧 id 上):
二维画面是候选网格上的暗点阵(该帧活动样本很少),三维是手指岛面加同一套样本点;"同一个样本
集"由上面的逐元素断言保证,快照证明这条链路真的走到了 GL。
**深度那一片的快照**(`.scratch/replay-analysis-a2/snapshots/`,同一个窗口、同一个游标、
同一个模式选择器;时间轴停在录像上,所以"原始图像"是真的相机图,两个派生模式读结果):
**深度那一片的快照**(`.scratch/replay-analysis-a2/snapshots/`,同一个窗口、同一个游标、同一个
选择器;"原始图像"是真的相机图,两个派生模式读结果):
| 文件 | 内容 |
|---|---|
| `analysis-finger-frame-33-2d-raw.png` / `-arrows.png` / `-depth.png` | finger 链路帧 33,深度非零 76 px(峰值 0.0242) |
| `analysis-box-frame-27-2d-raw.png` / `-arrows.png` / `-depth.png` | box 链路帧 27,`depth_map` 365×365 **恰好全零**,面板显示"本帧深度全为 0(133225 px 中 0 个非零)" |
| `analysis-box-frame-27-2d-raw.png` / `-arrows.png` / `-depth.png` | box 链路帧 27,`depth_map` 365×365 全零,面板显示"本帧深度全为 0(133225 px 中 0 个非零)" |
同一帧三张图的大小不同(raw 96.6 kB 是相机图,arrows 4.9 kB,depth 4.2 kB),这本身就是
"三个模式画的是三样东西"的证据。
**接线那一片的验收**(`apps/studio/tests/test_replay_wiring.py`,7 项,真实录像 + 真实 pass;录像
先复制到临时目录、结果写进它自己的 cache,窗口因此复用而不开链路):操作员路径断言引擎仍在
`recording`、`available_result()` 非空、选择器给出"原始图像"与三个派生模式、屏上是复用的结果而
「从视频重算」是关的(改前报 `'analysis' == 'recording'`;选择器里当时没有"原始图像")。
箭头与深度的字节分别与存下来的候选/`active` 集合和 `depth_map_to_bgr(存的 depth, …)` 相等
(这份 box 结果每帧全零,面板用 `panels.depth_empty` 说明);跳转、单步、真定时器播放(0.4 s)
与 pass 完成四条都在录像时间轴上读结果。另两项:改阈值只重画、改栅格被拒但画面照画(改前抛
`TypeError`)。
**接线那一片的快照**(`.scratch/replay-reachable/snapshots/`,真实 GL、真窗口、没有手工调用
`use_recording()`):`wiring-operator-frame-40-2d-{raw,arrows,depth}.png` 是同一帧的三个模式
(86 个活动样本;深度全零),`-playback-2d-arrows.png` 是播放到该时刻抓的,`-window.png` 为整窗。
## 15. 已知限制
* 结果里没有力:力曲线始终读记录的 `forces.csv`。深度有了(第 1、11 节),但**三维面板仍只
拿光流**:`session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False`,
因为把深度接进三维会改变三维图形的构成,而那不在这一片里。二维深度视图是这一片交付的东西。
* 结果覆盖的帧是 pass 走过的那些:录像比结果长时游标可以离开结果范围(面板说"这个分析结果里
没有可显示的帧"),不会静默重复最后一帧。
拿光流**:`session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False` —— 接进
三维会改变三维图形的构成,不在这一片里。
* 结果覆盖的帧是 pass 走过的那些:**结果的第一帧可能晚于录像的起点**(box 链路构造时吃掉前
7 个解码帧,这份结果从 t=0.4162 s 开始),最后一批帧也可能早于录像的终点。游标落在这两段里
时,派生视图按 `frame_id_at()` 的"该时刻或之前、越界夹到端点"给帧:起点之前画的是最早那一帧
测量,终点之后画的是最后一帧。两段没有各自的提示,"不会静默重复最后一帧"在本片没有兑现。
* 栅格类显示设置不生效(第 12 节);换栅格就是重跑一遍,那是**另一个键**。
* 一个进程同时只有一遍分析在跑;采集会话运行期间不启动分析。
* 时间轴与模式选择是两件事,而"有没有结果"是录像的属性、不是游标的属性:
`ReplayController.available_result()` 查的是**本录像**在 `ResultLibrary` 里的那一份,所以
即使 `ReplayEngine.use_recording()` 让引擎不再借用 session(目前 UI 没有第二条路会走到
这个状态),二维面板照样能画箭头和深度,`replay_raw_note` 也不会重新出现。
* "有没有结果"是录像的属性、不是游标的属性:`available_result()` 查的是本录像在
`ResultLibrary` 里的那一份,第 9 节的接线之后这是常态。
+9 -6
View File
@@ -5,9 +5,9 @@ capability states its requirement, data source and a falsifying check. The resul
playback semantics are owned by Part A ([replay-analysis.md](replay-analysis.md)) and Part B
([replay-analysis-part-b.md](replay-analysis-part-b.md)).
Status: Parts A and B shipped at `cf49b08`; the depth channel — result format and 2D view — is
uncommitted; the cache-root and lifetime policy and the display-control separation (Part C)
are decided below, not implemented.
Status: Parts A and B shipped at `cf49b08`, the depth channel at `9e20fd0`; the result-aware
mode plan and the wiring that reaches it are in the working tree (Part B §9, §14). The cache-root
and lifetime policy and the display-control separation (Part C) are decided below, not implemented.
## 1. Load a video
@@ -110,9 +110,12 @@ frame's depth is all zeros, `panels.depth_unavailable` says the frame has no dep
all — different facts, neither drawing the processed image.
A mode needing derived data is served from the result, never faked: with none, that view is not
offered and `panels.replay_raw_note` names the action that produces a result.
offered and `panels.replay_raw_note` names the action that produces a result. The state above is
what an operator reaches by opening a recording that has a result: the cursor stays on the
recording's own timeline and the panel opens on a derived view, the recorded one still offered.
Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`,
`::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_analysis_depth.py::test_an_empty_map_costs_a_header_and_states_its_shape`.
`::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_analysis_depth.py::test_an_empty_map_costs_a_header_and_states_its_shape`,
`test_replay_wiring.py::test_the_operator_sees_the_cached_views_over_the_recording`.
## Cache and export policies
@@ -151,7 +154,7 @@ possibility, belonging to export rather than the cache. Not implemented.
|---|---|---|
| Format, cache, reader, pass | `orisys_app_shared.analysis` | A/B shipped; depth channel uncommitted |
| Timeline, cursor, policy, pass thread, progress surface | `replay_analysis.py`, `replay.py`, `replay_controller.py`, `analysis_worker.py`, `ui/analysis_progress.py` | shipped; the depth view and the result-aware mode plan are uncommitted |
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams); the result-aware plan is uncommitted |
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams); the wiring from the operator's open action, and its snapshots, are in the working tree |
| Cache root and lifetime | `analysis/cache.py`, app preferences | decided, not implemented |
| Display-control separation: rail reset as the measurement reference, a separate display baseline, the F8 view reset | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started; Part B §12 records these as unchanged. The pass accepts a `reference_frame` but `_analysis_service` passes none |
@@ -29,6 +29,7 @@ from typing import Any, Protocol
import numpy as np
from orisys_app_shared.acquisition.image_backend import (
ARROW_PURE,
DERIVED_MODES,
SOURCE_ANALYSIS,
SOURCE_REPLAY,
@@ -45,6 +46,7 @@ from orisys_studio.replay_analysis import (
AnalysisSession,
ResultLibrary,
ResultPolicyError,
policy_divergence,
)
logger = logging.getLogger(__name__)
@@ -620,7 +622,13 @@ class ReplayController(QObject):
# ---------------------------------------------------------------- analysis
def analysis_session(self) -> AnalysisSession | None:
"""The stored result the timeline is reading, if any."""
"""Whatever result is open: the cursor's own, else the library's.
Not the question a view asks — that is :meth:`available_result`, which answers "is
there a measurement for *this recording*" and is what the app's timeline (always
the recording's) is served from. This one is the raw pair, kept for a caller that
wants the session itself.
"""
session = self._engine.analysis()
if session is not None:
return session
@@ -658,19 +666,25 @@ class ReplayController(QObject):
``test_replay_analysis`` that the result directory set is unchanged is a test of
this method, not of the panel.
It is asked of ``available_result``, not of ``analysis_active``: a result is a
property of the recording, and the cursor is normally on the recording's own
timeline while the arrows and the depth map are drawn from it. Gating on the
timeline instead would make a threshold change stop mattering in exactly the
state this view is served in.
A policy that moves the sample *grid* cannot be served by the stored candidate
locations; :meth:`AnalysisSession.serve_policy` says so, the message names the
key, and the frame is still redrawn from the result's own grid, because
locations; :meth:`AnalysisSession.serve_policy` refuses it, the message names the
keys, and the frame is still redrawn from the result's own grid, because
re-gridding it would draw samples nothing measured.
"""
session = self.analysis_session()
if session is None or not self.analysis_active():
session = self.available_result()
if session is None:
return
from orisys_app_shared.scene3d.visual_policy import policy_for_settings
policy = policy_for_settings(settings)
moved = not session.serve_policy(policy)
if moved:
if not session.serve_policy(policy):
moved = policy_divergence(session.stored_policy(), policy)
self._host._notify(t("playback.analysis_grid_mismatch", keys="·".join(moved)))
self._engine.redraw()
@@ -765,15 +779,54 @@ class ReplayController(QObject):
return True
def _adopt_session(self, session: AnalysisSession) -> None:
"""One timeline: the cursor, the charts and both figures read this result."""
"""One timeline, one cache behind it: the cursor, the charts and both figures.
**The engine stays on the recording.** The stored result is adopted *behind* the
cursor — :meth:`available_result` answers "is there a measurement for this
recording?" — and ``ReplayEngine.use_recording()`` is what keeps the cursor's own
timeline the recording's: its signal time, its own frame ids, and its whole span,
so a position the pass did not cover is still a position the operator can be at.
Moving the cursor onto the result instead (``use_analysis``) is what made the
recording's own picture unreachable: with the cursor on a result, the recorded
view has nothing to decode.
"""
self.results.open_session(session)
self._engine.use_analysis(session)
self._engine.use_recording()
self._charts.set_recording_source(self._engine.reader())
self._image_panel.set_source_kind("analysis")
# The timeline is the recording's, so the panel is told "replay" — the same word
# ``apply_source`` uses for an open recording. It used to be told "analysis",
# which is not a 2D source kind at all: ``ArrowPanel`` knows live/recompute/replay
# and coerced it, so a stored result was worded as the live pipeline.
self._image_panel.set_source_kind("replay")
self._mesh_panel.set_source_kind("analysis")
if self._actions is not None:
self._actions.recompute.setEnabled(True)
self.render_frame()
# A result just became readable, so it is what the operator is shown — unless a
# derived view was already on screen (a depth view chosen while the pass ran).
self.serve_stored_view()
def serve_stored_view(self) -> bool:
"""Put the cached view on screen when a result becomes readable.
The trigger for "draw the cache without being asked": a stored result is adopted
— on opening a recording that has one, or when a pass commits — and the panel is
showing the recording's own picture, so the operator would otherwise be left
looking at the camera while the measurement sits on disk. The recording's own
picture stays in the same selector, one step away, and the recompute switch is
not involved.
Answers whether the selection moved. A derived selection is left alone: the mode
the operator chose is not this method's to override.
"""
current = str(getattr(self._image_panel, "view_mode", lambda: "")() or "")
if current in DERIVED_MODES:
return False
# The plan for the frame just drawn offers the derived modes (the frame carries
# ``result_available``), so this selection is one the panel can honour.
self._image_panel.set_view_mode(ARROW_PURE)
self.render_frame()
return True
def _display_policy(self) -> Any:
from orisys_app_shared.scene3d.visual_policy import policy_for_settings
@@ -907,6 +960,10 @@ class ReplayController(QObject):
map can be drawn at all. The session the library holds is checked against the
recording that is open, so a previous recording's result cannot answer for this
one.
This is the normal state of a replay, not a fallback: the cursor stays on the
recording's own timeline (``_adopt_session``) and the result is the cache behind
it, so the answers and the view modes come from here.
"""
session = self._engine.analysis()
if session is not None:
@@ -950,12 +1007,13 @@ class ReplayController(QObject):
def render_frame(self) -> None:
"""Draw what the 2D panel should show for the current replay cursor.
One method for both timeline modes, because it is one cursor: in analysis mode
the picture is the stored frame's own flow field and depth map (the pass's
candidate locations, on the domain grid), and in recording mode it is the
decoded video frame. The window's single 2D sink draws both through the *same*
backend, so the arrows, the idle dots, the colour range and the domain boundary
behave identically in the two modes.
One method for both timeline modes, because it is one cursor: with the cursor on
a stored result (``analysis_active``) the picture is the stored frame's own flow
field and depth map, and with the cursor on the recording's own timeline it is the
decoded video frame — unless the *view* the operator asked for is a measurement,
in which case it is read from the result for the same cursor. The window's single
2D sink draws all of them through the *same* backend, so the arrows, the idle
dots, the colour range and the domain boundary behave identically.
**A mode that needs derived data is served from the result, never faked.** The
arrows and the depth map are the pass's measurements; with a result in hand they
+8 -5
View File
@@ -757,9 +757,12 @@ def test_opening_a_recording_with_a_stored_result_runs_no_pass(pass_root):
controller, host, engine, parts = _controller(host=host)
assert controller.load_recording(RECORDING) is True
assert calls == [str(RECORDING)], f"the service was built {len(calls)} times"
assert controller.analysis_active(), "the timeline did not adopt the stored result"
assert engine.mode() == MODE_ANALYSIS
assert parts[2].kinds[-1] == "analysis", "the 2D panel was not told it is a result"
# Adoption is "the recording has a readable result", not "the cursor moved onto it":
# the engine keeps the recording's own timeline (Part B §9), so the session is held
# by the library and answered through ``available_result``.
assert controller.available_result() is not None, "the stored result was not adopted"
assert engine.mode() == MODE_RECORDING, "adopting a result moved the cursor's timeline"
assert parts[2].kinds[-1] == "replay", "the 2D panel was not told the cursor is a replay"
assert parts[3].kinds[-1] == "analysis", "the 3D pane was not told it is a result"
assert parts[4].states[-1] == "reused", "the surface did not report the reuse"
assert stored.is_dir()
@@ -782,8 +785,8 @@ def test_opening_a_recording_without_a_result_starts_the_pass(tmp_path, qt_app):
assert controller.wait_for_analysis(120)
_flush()
assert parts[4].states[-1] == "complete", parts[4].states
assert controller.analysis_active(), "a complete result was not adopted"
assert engine.mode() == MODE_ANALYSIS
assert controller.available_result() is not None, "a complete result was not adopted"
assert engine.mode() == MODE_RECORDING, "a finished pass moved the cursor's timeline"
def test_a_cancelled_pass_is_reported_and_never_served(tmp_path, qt_app):
+507
View File
@@ -0,0 +1,507 @@
"""Part B's wiring: the cached analysis views over a recording, in the running app.
The result contract ([replay-analysis.md](../docs/replay-analysis.md)), the playback
contract ([replay-analysis-part-b.md](../docs/replay-analysis-part-b.md)) and the panel
contract ([replay-panel.md](../docs/replay-panel.md)) all describe one state: a recording
is open, the cursor is on the recording's own signal time, and the arrows and the depth
map are served from the stored analysis result while the operator scrubs and plays — with
the recording's own picture one step away in the mode selector, and the recompute switch
left as the explicit "run the chain over this footage again" action.
What was missing was the wiring that reaches it. ``ReplayEngine.use_recording()`` had no
caller, so analysis-on-open moved the cursor onto the *result's* timeline instead: the
derived views were drawn, but the recording's own picture was not offered at all, and the
cursor could not leave the frames the result covers. These tests drive the window's own
「打开录像」 action over a recording whose result is already stored beside it — the result
is generated here into the recording's own cache, so the app *reuses* it and opens no SDK
chain — and assert on the picture the panel actually shows, not on a state a harness
built.
The recording is copied before use: a pass writes its result into the recording's own
directory, and nothing may write into ``dist/``.
"""
from __future__ import annotations
import shutil
import time
from pathlib import Path
import numpy as np
import pytest
from orisys_app_shared.acquisition.arrows import depth_map_to_bgr, flow_colormap
from orisys_app_shared.acquisition.image_backend import (
ARROW_PURE,
ARROW_STITCHED,
DEPTH,
DERIVED_MODES,
RECORDED,
)
from orisys_app_shared.i18n import t
from orisys_app_shared.scene3d.contracts import DisplaySettings
from orisys_app_shared.scene3d.visual_policy import (
select_flow_samples,
)
from orisys_studio.replay import MODE_RECORDING
from PySide6.QtWidgets import QApplication
REPO = Path(__file__).resolve().parents[3]
STUDIO = REPO / "dist" / "OrisysStudio"
SOURCE_RECORDING = (
STUDIO
/ "outputs"
/ "recordings"
/ "20260930T015028Z_session0002_40cb8e7441434feab48bb646afdba011"
)
#: What makes a directory a recording for the application, and all a replay reads.
RECORDING_FILES = (
"frames.csv",
"forces.csv",
"events.csv",
"session.json",
"raw_0001.mp4",
"preroll_0001.mp4",
)
def _require_inputs() -> None:
missing = [
str(SOURCE_RECORDING / name)
for name in ("frames.csv", "raw_0001.mp4", "session.json")
if not (SOURCE_RECORDING / name).exists()
]
if missing:
pytest.skip(
"the replay wiring tests need the evidence recording: " + ", ".join(missing)
)
pytest.importorskip("orisys", reason="the analysis pass drives the SDK chain")
def _copy_recording(destination: Path) -> Path:
"""The evidence recording, copied: nothing may write into ``dist/``."""
destination.mkdir(parents=True, exist_ok=True)
for name in RECORDING_FILES:
source = SOURCE_RECORDING / name
if source.exists():
shutil.copy2(source, destination / name)
return destination
def _run_the_pass(recording: Path):
"""One real pass, with the chain the *application* derives from the recording."""
from orisys_app_shared.analysis.service import AnalysisService, chain_spec_for
from orisys_app_shared.record import RecordingReader
reader = RecordingReader(recording)
try:
chain = chain_spec_for(reader.session())
finally:
reader.close()
return AnalysisService(recording, chain=chain).run()
@pytest.fixture(scope="module")
def stored_recording(tmp_path_factory) -> Path:
"""A copy of the evidence recording whose own cache already holds a result."""
_require_inputs()
recording = _copy_recording(tmp_path_factory.mktemp("replay-wiring") / "recording")
outcome = _run_the_pass(recording)
assert outcome.complete, outcome.reason
assert outcome.frames_written > 0
return recording
@pytest.fixture
def window(make_window, qt_app):
"""The application, in the replay view, as the operator meets it."""
window = make_window(sensor_type="box", source_mode="camera")
window.resize(1440, 880)
window.show()
QApplication.processEvents()
window.brand_bar.btn_playback.click()
QApplication.processEvents()
assert window._in_replay_view(), "the window did not switch to the replay view"
return window
def _open(window, recording: Path, monkeypatch) -> None:
"""Press the window's own 「打开录像」 entry, with the file dialog answered for it."""
import orisys_studio.replay_controller as replay_controller
video = Path(recording) / "raw_0001.mp4"
monkeypatch.setattr(
replay_controller.QFileDialog,
"getOpenFileName",
lambda *args, **kwargs: (str(video), ""),
)
action = window.replay_ctl._actions.playback
assert action.isEnabled(), "the replay entries are not offered in the replay view"
action.trigger()
QApplication.processEvents()
assert window.replay.has_video(), "the recording did not open"
def _sink_frames(window, monkeypatch) -> list:
"""Every frame handed to the window's single 2D sink, in order."""
captured: list = []
original = window._show_image_frame
def spy(frame):
captured.append(frame)
return original(frame)
monkeypatch.setattr(window, "_show_image_frame", spy)
return captured
def _panel_bgr(window) -> np.ndarray:
"""The 2D panel's picture, in BGR, from the end of the real display path."""
image = window.arrow_panel._source_image
assert image is not None and not image.isNull(), "the 2D panel shows no picture"
width, height = image.width(), image.height()
rows = np.frombuffer(bytes(image.constBits()), dtype=np.uint8)
rows = rows.reshape(height, image.bytesPerLine())
return rows[:, : width * 3].reshape(height, width, 3)[:, :, ::-1].copy()
def _expected_depth(window, frame) -> np.ndarray:
"""A stored frame's depth view, drawn the way the window's own sink draws it."""
return depth_map_to_bgr(
frame.depth,
flow_colormap(window._theme),
value_range=window._depth_range(),
invalid_mask=window._domain_mask.mask,
)
def _set_mode(window, mode: str) -> None:
index = window.arrow_panel.view_combo.findData(mode)
assert index >= 0, f"the mode {mode} is not offered: {window.arrow_panel.view_modes()}"
window.arrow_panel.view_combo.setCurrentIndex(index)
QApplication.processEvents()
assert window.arrow_panel.view_mode() == mode
def _pump(seconds: float) -> None:
"""Let real time pass so the playback timer fires, as it does in the app."""
deadline = time.perf_counter() + float(seconds)
while time.perf_counter() < deadline:
QApplication.processEvents()
time.sleep(0.01)
def _cells(rows, cols) -> set:
return {(int(row), int(col)) for row, col in zip(np.asarray(rows), np.asarray(cols))}
def _active_cells(samples) -> set:
return {
(int(row), int(col))
for row, col, active in zip(
np.asarray(samples.rows), np.asarray(samples.cols), np.asarray(samples.active)
)
if bool(active)
}
def test_the_operator_sees_the_cached_views_over_the_recording(
stored_recording, window, monkeypatch
):
"""The state the contracts describe, reached the way an operator reaches it.
A recording with a result beside it opens on the **recording's** timeline (the result
is a cache behind the cursor, not a second timeline), the panel offers the recording's
own picture *and* the three derived views, and what is on screen is the stored frame —
not the camera, and not a placeholder.
"""
captured = _sink_frames(window, monkeypatch)
_open(window, stored_recording, monkeypatch)
assert window.replay.mode() == MODE_RECORDING, (
"the cursor left the recording's own timeline; the result is a cache behind it"
)
session = window.replay_ctl.available_result()
assert session is not None, "the stored result beside the recording was not adopted"
assert not window.replay_ctl.recompute_active(), (
"the cache was only reachable by turning the recompute switch on"
)
assert window.analysis_progress.state() == "reused", (
"a pass ran over a recording that already had a result"
)
assert window.arrow_panel.view_modes() == (
RECORDED,
ARROW_PURE,
ARROW_STITCHED,
DEPTH,
), "the recording's own picture must stay one step away from the derived views"
assert window.arrow_panel.view_mode() in DERIVED_MODES, (
"the operator was left on the camera picture although a result is served"
)
assert window.arrow_panel.note_label.text() == "", (
"the panel says there is nothing to draw while it is drawing a result"
)
# The arrow view is the stored measurement, at the cursor's own frame.
frames = list(session.frames())
active_frame = next(frame for frame in frames if frame.active_count > 0)
window.replay.set_position(session.time_of(active_frame.frame_id))
QApplication.processEvents()
assert captured, "nothing reached the 2D sink"
drawn = captured[-1]
assert drawn.frame_id == active_frame.frame_id
assert drawn.result_available is True
stored_cells = _cells(active_frame.samples.rows, active_frame.samples.cols)
field = np.asarray(drawn.flow)
finite = np.argwhere(np.isfinite(field[..., 0]))
assert _cells(finite[:, 0], finite[:, 1]) == stored_cells, (
"the recovered flow field carries values outside the stored candidate locations"
)
# The height last: the shared selector re-reads the same cells, in its own order, and
# comes to the same activity verdict as the pass stored.
reselected = select_flow_samples(field, drawn.valid_mask, session.display_policy)
assert _cells(reselected.rows, reselected.cols) == stored_cells
assert _active_cells(reselected) == _active_cells(active_frame.samples)
arrows_bgr = _panel_bgr(window)
raw_here = window.replay.video_frame_bgr()
assert raw_here is not None
assert not np.array_equal(arrows_bgr, np.asarray(raw_here)), (
"the arrow view is the recording's raw picture"
)
# The depth view is the stored map, drawn through the window's own backend. This
# result's map is exactly zero on every frame it holds — the box chain's depth is
# flat on this footage, which the panel has its own sentence for — and it is still a
# stored array of the result's own shape, so the pixels are checked against it rather
# than against a flat placeholder.
_set_mode(window, DEPTH)
target = active_frame
assert target.depth is not None and target.depth_nonzero == 0
QApplication.processEvents()
drawn = captured[-1]
assert drawn.frame_id == target.frame_id
assert np.array_equal(
np.asarray(drawn.depth), np.asarray(target.depth)
), "the depth view is not the stored map"
shown = _panel_bgr(window)
expected = _expected_depth(window, target)
assert expected is not None
assert shown.shape == expected.shape
assert np.array_equal(shown, expected), (
"the depth view's pixels are not depth_map_to_bgr of the stored frame"
)
assert not np.array_equal(shown, arrows_bgr), "the depth view drew the arrow view"
assert t("panels.depth_empty", pixels=int(target.depth.size)) in (
window.arrow_panel.depth_scale_label.text()
), "an exactly-zero depth map is not stated as such"
# …and the recording's own picture is still reachable, at the same cursor.
_set_mode(window, RECORDED)
raw = window.replay.video_frame_bgr()
assert raw is not None, "the recording's own frame could not be decoded"
assert np.array_equal(_panel_bgr(window), np.asarray(raw)), (
"the recorded view is not the recording's own picture"
)
assert window.arrow_panel.note_label.text() == ""
assert window.replay.mode() == MODE_RECORDING
def test_playing_forward_and_seeking_draw_the_same_cached_frame(
stored_recording, window, monkeypatch
):
"""A seek is the frame playing forward passes through — on the operator's path."""
captured = _sink_frames(window, monkeypatch)
_open(window, stored_recording, monkeypatch)
session = window.replay_ctl.available_result()
assert session is not None
_set_mode(window, DEPTH)
ids = [int(value) for value in session.frame_ids]
target_id = ids[len(ids) // 4]
moment = session.time_of(target_id)
assert moment is not None
# Play forward from the top, a tick's worth of cursor at a time, ending on the
# target instant itself.
positions = [step / 240.0 for step in range(int(moment * 240.0) + 1)]
positions.append(moment)
for position in positions:
window.replay.set_position(position)
played = _panel_bgr(window)
played_id = captured[-1].frame_id
# …then seek straight to the same instant.
window.replay.set_position(moment)
QApplication.processEvents()
assert captured[-1].frame_id == played_id == target_id
assert np.array_equal(played, _panel_bgr(window)), (
"a seek drew a different frame from playing forward to the same position"
)
# Stepping one stored frame on moves to that frame's own picture.
later = ids[ids.index(target_id) + 1]
window.replay.set_position(session.time_of(later))
QApplication.processEvents()
assert captured[-1].frame_id == later
assert np.array_equal(
np.asarray(captured[-1].depth), np.asarray(session.frame(later).depth)
)
def test_playback_draws_the_cached_frame_not_the_video(
stored_recording, window, monkeypatch
):
"""Advance the real playback timer and check every drawn frame against the cache."""
captured = _sink_frames(window, monkeypatch)
_open(window, stored_recording, monkeypatch)
session = window.replay_ctl.available_result()
assert session is not None
_set_mode(window, DEPTH)
window.replay.set_position(0.0)
QApplication.processEvents()
captured.clear()
window.replay_ctl.start_playback()
assert window.replay_ctl.is_playing(), "playback did not start"
_pump(0.4)
position = window.replay.position()
assert position > 0.0, "the playhead did not advance"
assert captured, "playback drew nothing"
assert position <= session.duration_s(), "the playhead left the recording's timeline"
expected_id = session.frame_id_at(position)
assert expected_id is not None
for frame in captured:
assert frame.depth is not None, "a frame drawn during playback carries no depth"
assert frame.result_available is True
assert session.has(int(frame.frame_id)), (
f"playback drew frame {frame.frame_id}, which the result does not hold"
)
# Pausing leaves the frame the cursor is on, and it is the cached one.
window.replay_ctl.pause_playback()
assert not window.replay_ctl.is_playing()
assert captured[-1].frame_id == expected_id
expected = _expected_depth(window, session.frame(expected_id))
assert expected is not None
assert np.array_equal(_panel_bgr(window), expected), (
"the paused frame is not the stored frame for the cursor"
)
def test_a_finished_pass_puts_the_cache_on_screen(tmp_path, window, monkeypatch):
"""The other trigger: the pass completes and the panel serves what it stored.
Nothing is pre-placed here, so the window runs the pass over the copy (a box chain:
a couple of seconds for this recording) and the *completion* is what has to leave the
derived view on screen — with the cursor still on the recording.
"""
recording = _copy_recording(tmp_path / "recording")
_sink_frames(window, monkeypatch)
_open(window, recording, monkeypatch)
assert window.replay_ctl.wait_for_analysis(180.0), "the pass did not finish"
QApplication.processEvents()
assert window.replay_ctl.analysis_error() == "", (
f"the pass left no usable result: {window.replay_ctl.analysis_error()}"
)
assert window.analysis_progress.state() == "complete"
session = window.replay_ctl.available_result()
assert session is not None, "the completed result was not adopted"
assert window.replay.mode() == MODE_RECORDING, (
"a finished pass moved the cursor off the recording's own timeline"
)
assert window.arrow_panel.view_mode() in DERIVED_MODES, (
"the operator was left on the camera picture after the pass finished"
)
assert not window.replay_ctl.recompute_active()
frame_id = session.frame_id_at(window.replay.position())
assert frame_id is not None
_set_mode(window, DEPTH)
expected = _expected_depth(window, session.frame(frame_id))
assert expected is not None
assert np.array_equal(_panel_bgr(window), expected), (
"the finished result is not what the panel draws"
)
def test_the_mode_selector_is_the_trigger_for_a_derived_view(
stored_recording, window, monkeypatch
):
"""On the recording's timeline, choosing a derived view serves it from the cache."""
captured = _sink_frames(window, monkeypatch)
_open(window, stored_recording, monkeypatch)
session = window.replay_ctl.available_result()
assert session is not None
_set_mode(window, RECORDED)
position = session.time_of(int(session.frame_ids[len(session.frame_ids) // 2]))
assert position is not None
window.replay.set_position(position)
QApplication.processEvents()
assert window.replay.mode() == MODE_RECORDING
assert captured[-1].depth is None, "the recorded view drew a stored frame"
captured.clear()
_set_mode(window, DEPTH)
assert captured, "choosing a derived view drew nothing"
assert captured[-1].depth is not None, (
"a view that needs derived data drew the recording's raw picture"
)
assert captured[-1].frame_id == session.frame_id_at(position)
assert window.replay.mode() == MODE_RECORDING, (
"choosing a view moved the cursor's timeline"
)
def test_the_display_settings_still_redraw_from_the_cache(
stored_recording, window, monkeypatch
):
"""A threshold change is a read: it redraws the stored frame and writes nothing."""
from orisys_app_shared.analysis.cache import scan_results
_open(window, stored_recording, monkeypatch)
session = window.replay_ctl.available_result()
assert session is not None
_set_mode(window, ARROW_PURE)
cache_root = Path(stored_recording) / "analysis"
before = {entry.directory for entry in scan_results(cache_root)}
captured = _sink_frames(window, monkeypatch)
window.replay_ctl.on_display_settings(DisplaySettings(flow_on_px=0.9))
QApplication.processEvents()
assert session.threshold() == pytest.approx(0.9), (
"the display threshold never reached the session"
)
assert captured, "a display change did not redraw the frame"
assert captured[-1].frame_id == session.frame_id_at(window.replay.position())
assert {entry.directory for entry in scan_results(cache_root)} == before, (
"a display change wrote an analysis result"
)
def test_a_setting_that_moves_the_grid_is_still_refused(
stored_recording, window, monkeypatch
):
"""The stored candidate locations cannot be re-gridded; the frame is drawn anyway."""
from orisys_app_shared.analysis.cache import scan_results
_open(window, stored_recording, monkeypatch)
session = window.replay_ctl.available_result()
assert session is not None
_set_mode(window, ARROW_PURE)
before_threshold = session.threshold()
cache_root = Path(stored_recording) / "analysis"
before = {entry.directory for entry in scan_results(cache_root)}
captured = _sink_frames(window, monkeypatch)
window.replay_ctl.on_display_settings(DisplaySettings(max_arrows=50))
QApplication.processEvents()
assert session.threshold() == pytest.approx(before_threshold), (
"a refused policy changed the session"
)
assert captured, "the frame was not redrawn from the result's own grid"
assert captured[-1].frame_id == session.frame_id_at(window.replay.position())
assert {entry.directory for entry in scan_results(cache_root)} == before