删除深度重算引擎;修好回放视频每帧重开文件导致的无法播放,运输控制改到状态轨主按钮 - What: 1) 彻底删除深度重算引擎:shared/.../record/depth.py(DepthRecomputer/DepthResult)与 depth_backends.py(BoxReplay/FingerReplay)、其专属测试 test_depth_replay.py,record/__init__ 不再导出,playback.py::replay_warnings 不再报 depth_recomputed;apps/studio/replay.py 由 515 行瘦身为只做读盘+游标+视频帧+提示,回放视图的 2D 面板只提供「原始图像」;session.json 的 depth.replay 由 recompute 改为 none;删除 31 个只剩深度使用的文案键(depth_*/warn_state_*/warn_reset_*),warn_config_changed 与 warn_sdk_changed 改为描述"由视频重算";ui_check 深度段、studio 两个深度用例、test_playback 三例、drive_shared 的 test_depth_replay 与 reproduce.py 的深度段一并清理(shared 套件 150→134 项)。2) 无法播放的根因是 RecordingReader.video_frame_at 每帧都 cv2.VideoCapture 重开并 seek:小片段已 ~10 ms/帧,1080p H.264 每次 seek 需从关键帧解码(上百 ms),而播放每 33 ms 要一帧且全在 UI 线程上,视频因此看着不动;改为按分段缓存解码器、顺序帧用 read() 步进(实测 10 ms → 0.5 ms/帧)、跳转才 seek,新增 RecordingReader.close() 释放。运输控制按用户建议上移到状态轨主按钮:toolbar.set_run_role("playback") 使回放视图的主按钮变成「▶ 回放」/「⏸ 暂停」(实时视图仍是「▶ 运行」),点按以 1× 推进游标(墙钟计时,漏拍不变慢)、到末尾自动停,拖动游标/框选/滚轮/拖总览条经 ForcePlotsPanel.timeline_taken_over 自动暂停,空格键同效;曲线表头不再放第二个播放按钮;按钮样式在回放角色下不再套用"停止"的红色语义。 - Why: 用户要求先清掉深度重算引擎、并反馈"现在无法播放视频",建议把上方播放按钮在历史回放形式下改为回放。 - Verification: ui_check.py 554 项 0 失败(新增 seek<250ms/步进<60ms 的解码预算、回放按钮文字随播放/暂停切换、手动拖游标即暂停、切回实时视图按钮恢复运行、按深度视图已移除);test_ui_smoke.py 71 项 0 失败;drive_shared.py 134 项 0 失败(深度引擎用例随代码删除);locale_studio.py 262 键齐全、zh==en;reproduce.py 退出 0;.repro/probe_play.py 实测按「回放」后 0.75 s 内游标推进 0.689 s、面板 12/12 帧都在更新;两张复核图(回放视图的「▶ 回放」、实时视图的「▶ 运行」)。 - Not in this change: 录制文件格式与 CSV 列序不变(raw_*.mp4 与 forces.csv 继续一起写);「由视频重算」复核模式保留(它复用实时管线,是有状态、只能顺序前进的一遍)。

This commit is contained in:
ykcheungab
2026-09-14 09:31:21 +08:00
parent 9b3c5fe46c
commit da8165cf03
19 changed files with 354 additions and 2056 deletions
+8 -30
View File
@@ -77,6 +77,11 @@
"run_unavailable": "Not available yet", "run_unavailable": "Not available yet",
"run_disabled_video": "Select a video file first", "run_disabled_video": "Select a video file first",
"run_disabled_sensor": "Run is not available for this sensor type", "run_disabled_sensor": "Run is not available for this sensor type",
"playback": "▶ Play",
"playback_pause": "⏸ Pause",
"playback_tip": "Play this recording at 1× (space bar does the same); dragging the cursor pauses",
"playback_pause_tip": "Pause playback (dragging the cursor, box zoom or the wheel also pauses)",
"playback_disabled": "Open a recorded video first (“Choose a recorded video…” at the top left)",
"reset": "Reset", "reset": "Reset",
"units": "Units", "units": "Units",
"units_newton": "Newton", "units_newton": "Newton",
@@ -142,10 +147,6 @@
"fx_name": "Shear X", "fx_name": "Shear X",
"fy_name": "Shear Y", "fy_name": "Shear Y",
"window_readout": "{start} – {end} s / {total} s total", "window_readout": "{start} – {end} s / {total} s total",
"play": "Play",
"play_tip": "Play this recording from the cursor at 1× (space bar); dragging the cursor pauses",
"pause": "Pause",
"pause_tip": "Pause playback (dragging the cursor, box zoom or the wheel also pauses)",
"window_tip": "Drag the highlighted region to move the window: this strip is the timeline of the whole session", "window_tip": "Drag the highlighted region to move the window: this strip is the timeline of the whole session",
"no_data": "—", "no_data": "—",
"mode_follow": "Following live", "mode_follow": "Following live",
@@ -180,37 +181,14 @@
"none_found": "No recording found under outputs/recordings", "none_found": "No recording found under outputs/recordings",
"frame_dropped": "This frame never reached the video (the record queue overflowed)", "frame_dropped": "This frame never reached the video (the record queue overflowed)",
"frame_unreadable": "This frame could not be decoded from the video", "frame_unreadable": "This frame could not be decoded from the video",
"depth_running": "Recomputing depth… (on a worker thread)",
"depth_busy": "The previous recomputation is still running",
"depth_empty": "No depth data for this frame",
"depth_failed": "Depth recomputation failed: {detail}",
"depth_sdk_unavailable": "Cannot recompute: the SDK is not available",
"depth_config_missing": "Cannot recompute: the recording has no sensor config reference",
"depth_config_file_missing": "Cannot recompute: the config file no longer exists at that path",
"depth_no_video": "Cannot recompute: this recording has no video segment",
"depth_not_recomputed": "Depth has not been recomputed for this cursor position\n(the previous frame's map is not reused)",
"depth_recomputing": "Recomputing depth for the current cursor position…",
"depth_frame_not_recorded": "This frame was not recorded: its data row was lost to a full queue and only the index row remains",
"depth_frame_not_in_video": "This frame never reached the video, so its depth cannot be recomputed",
"depth_state_across_segment": "Replay cannot carry the algorithm state across segments: this one restarts at its first frame",
"depth_state_approximate": "No reset or pre-roll history for this segment: the algorithm state is an approximation",
"depth_warnings": "Recomputation notes: {detail}",
"warnings": "Replay notes: {detail}", "warnings": "Replay notes: {detail}",
"decimated": "This recording is long, so the curves are a decimated view ({shown} of {total} rows; intervals are read from disk on demand)", "decimated": "This recording is long, so the curves are a decimated view ({shown} of {total} rows; intervals are read from disk on demand)",
"warn_config_changed": "the config differs from recording time, so the recomputed depth may differ", "warn_config_changed": "the config differs from recording time, so recomputing from the video may differ",
"warn_sdk_changed": "the SDK version differs from recording time, so the recomputation may differ", "warn_sdk_changed": "the SDK version differs from recording time, so recomputing may differ",
"warn_config_missing": "no config hash was recorded, so recomputation cannot be verified", "warn_config_missing": "no config hash was recorded, so recomputation cannot be verified",
"warn_depth_recomputed": "depth comes from offline recomputation, not from stored data",
"warn_rows_skipped": "some unparsable CSV rows were skipped", "warn_rows_skipped": "some unparsable CSV rows were skipped",
"warn_frames_dropped": "frames were lost to a full queue; only their index rows remain (shown as \"not recorded\")", "warn_frames_dropped": "frames were lost to a full queue; only their index rows remain (shown as \"not recorded\")",
"warn_out_of_order": "this recording was written by an older version: its rows are not in time order, so the replay re-sorts them and window edges may differ", "warn_out_of_order": "this recording was written by an older version: its rows are not in time order, so the replay re-sorts them and window edges may differ"
"warn_state_across_segment": "the algorithm state cannot be carried across a segment",
"warn_state_approximate": "the algorithm state is an approximation",
"warn_state_approximate_after_rewind": "the video was reopened to reach an earlier frame, so the state may differ",
"warn_state_too_far_back": "the zeroing is too far back: reading there would exceed the replay budget, so the state is approximate",
"warn_reset_frame_not_indexed": "the zeroing frame is not in the frame index, so the state cannot be rebuilt from it",
"warn_reset_frame_not_in_video": "the zeroing frame never reached the video, so the state cannot be rebuilt from it",
"warn_reset_not_applied": "the replay could not re-apply the zeroing, so the state may differ"
}, },
"record": { "record": {
"label": "Record", "label": "Record",
+8 -30
View File
@@ -77,6 +77,11 @@
"run_unavailable": "当前不可运行", "run_unavailable": "当前不可运行",
"run_disabled_video": "请先选择一个视频文件", "run_disabled_video": "请先选择一个视频文件",
"run_disabled_sensor": "此传感器类型暂不支持运行", "run_disabled_sensor": "此传感器类型暂不支持运行",
"playback": "▶ 回放",
"playback_pause": "⏸ 暂停",
"playback_tip": "按 1× 播放这段录制(空格键同效);拖动游标即暂停",
"playback_pause_tip": "暂停播放(拖动游标、框选或滚轮也会暂停)",
"playback_disabled": "先打开一段录制视频(左上「选择录制视频…」)",
"reset": "重置", "reset": "重置",
"units": "单位", "units": "单位",
"units_newton": "牛顿", "units_newton": "牛顿",
@@ -142,10 +147,6 @@
"fx_name": "切向力 X", "fx_name": "切向力 X",
"fy_name": "切向力 Y", "fy_name": "切向力 Y",
"window_readout": "{start} – {end} s / 全程 {total} s", "window_readout": "{start} – {end} s / 全程 {total} s",
"play": "播放",
"play_tip": "从游标处按 1× 播放这段录制(空格键);拖动游标即暂停",
"pause": "暂停",
"pause_tip": "暂停播放(拖动游标、框选或滚轮也会暂停)",
"window_tip": "拖动高亮区域即可移动视窗;这是整段会话的时间轴", "window_tip": "拖动高亮区域即可移动视窗;这是整段会话的时间轴",
"no_data": "—", "no_data": "—",
"mode_follow": "跟随实时", "mode_follow": "跟随实时",
@@ -180,37 +181,14 @@
"none_found": "outputs/recordings 下没有找到录制", "none_found": "outputs/recordings 下没有找到录制",
"frame_dropped": "该帧未被写入视频(记录队列曾溢出)", "frame_dropped": "该帧未被写入视频(记录队列曾溢出)",
"frame_unreadable": "该帧无法从视频中解码", "frame_unreadable": "该帧无法从视频中解码",
"depth_running": "正在重算深度…(在后台线程)",
"depth_busy": "上一次重算尚未完成",
"depth_empty": "该帧没有深度数据",
"depth_failed": "深度重算失败: {detail}",
"depth_sdk_unavailable": "无法重算:SDK 不可用",
"depth_config_missing": "无法重算:录制未记录传感器配置",
"depth_config_file_missing": "无法重算:配置文件的原始路径已不存在",
"depth_no_video": "无法重算:该录制没有视频分段",
"depth_not_recomputed": "当前游标位置尚未重算深度\n(不会沿用其他帧的结果)",
"depth_recomputing": "正在为当前游标位置重算深度…",
"depth_frame_not_recorded": "该帧未录制:数据行在队列溢出时丢失,只有索引记录",
"depth_frame_not_in_video": "该帧未写入视频,无法重算深度",
"depth_state_across_segment": "重算无法跨分段延续算法状态:本段从段首重放",
"depth_state_approximate": "本段缺少归零或预卷历史,算法参考状态为近似重建",
"depth_warnings": "重算提示: {detail}",
"warnings": "回放提示: {detail}", "warnings": "回放提示: {detail}",
"decimated": "该录制较长,曲线为降采样视图(显示 {shown} / 共 {total} 行;区间数据按需从磁盘读取)", "decimated": "该录制较长,曲线为降采样视图(显示 {shown} / 共 {total} 行;区间数据按需从磁盘读取)",
"warn_config_changed": "配置文件与录制时不一致,重算出的深度可能与当时不同", "warn_config_changed": "配置文件与录制时不一致:由视频重算的结果可能与当时不同",
"warn_sdk_changed": "SDK 版本与录制时不一致,重算结果可能不同", "warn_sdk_changed": "SDK 版本与录制时不一致:由视频重算的结果可能不同",
"warn_config_missing": "找不到配置哈希,无法校验重算一致性", "warn_config_missing": "找不到配置哈希,无法校验重算一致性",
"warn_depth_recomputed": "深度由离线重算得到,不是当时保存的数据",
"warn_rows_skipped": "有无法解析的 CSV 行被跳过", "warn_rows_skipped": "有无法解析的 CSV 行被跳过",
"warn_frames_dropped": "有帧在队列溢出时丢失,仅保留索引记录(回放显示为“该帧未录制”)", "warn_frames_dropped": "有帧在队列溢出时丢失,仅保留索引记录(回放显示为“该帧未录制”)",
"warn_out_of_order": "该录制由旧版本写入,行序不是时间序;回放已按时间重排,区间边界可能与当时不同", "warn_out_of_order": "该录制由旧版本写入,行序不是时间序;回放已按时间重排,区间边界可能与当时不同"
"warn_state_across_segment": "重算不能跨分段延续算法状态",
"warn_state_approximate": "算法参考状态为近似重建",
"warn_state_approximate_after_rewind": "为回到更早的帧重新打开了视频,参考状态可能偏离当时",
"warn_state_too_far_back": "归零点过于久远,需要读取的帧数超出上限,已按近似状态重建",
"warn_reset_frame_not_indexed": "归零帧不在帧索引中,无法从该点重建状态",
"warn_reset_frame_not_in_video": "归零帧未写入视频,无法从该点重建状态",
"warn_reset_not_applied": "重放无法重新执行归零,参考状态可能偏离当时"
}, },
"record": { "record": {
"label": "记录", "label": "记录",
+27 -34
View File
@@ -273,37 +273,28 @@ outputs/recordings/<UTC 时间戳>_session<NNNN>/
(`calibration_session_*.json`,`force/calibration_session.py`),并可用「打开标定记录」 (`calibration_session_*.json`,`force/calibration_session.py`),并可用「打开标定记录」
重新载入检查。界面上的降采样 trace 只是预览,不是审计数据。 重新载入检查。界面上的降采样 trace 只是预览,不是审计数据。
### 7.5 深度重算与游标同步(P1-3、P1-4) ### 7.5 深度重算引擎(已删除)
- **不再重开回放位置**:`DepthRecomputer` 记录当前已读到的段内帧号,请求第 2 帧再请求第 3 这一节记录的 `DepthRecomputer` / `depth_backends`(离线把录制视频喂回 SDK 重算深度、按
帧只多读 1 帧(旧实现每次都从头读,2 和 3 会落到 5);往回退才重开视频。 帧号增量读、归零/预卷恢复算法状态、请求绑定帧号)**已整体删除**:用户明确不需要"为回放再
- 每帧都执行**实时同一条链路**:盒子 `compute_deformation + compute_contact`,手指再加上 录/再算一遍深度"这条路径,回放视图只放录制的原始相机图像。要复核算法的当前做法是
`FingerRuntime` 的展开与有效区域(`_BoxReplay` / `_FingerReplay`)。 「视图 → 由视频重算力值」(把录制视频跑一遍采集管线,见 `docs/shell-lifecycle.md` §2.1),
- `config_name` 用**完整解析后的配置路径**(`load_fixed_mode`),不再缩减成父目录名或 它复用实时那条链路,因此不再需要一份离线复现实现。
stem —— 那可能加载同名但不同的配置,而哈希校验还在比对原路径。
- **算法参考状态**:SDK 的跟踪器/滤波是有状态的,离线复现需要历史。两条可用依据: `session.json` 里的深度块保留(它说明"深度没有保存"这一事实),但 `replay` 字段改为
1. 操作归零(`zeroing` 事件,带帧号)会清空跟踪器,因此从该帧重放是**精确**的; `"none"`;`RecordingReader.replay_warnings()` 不再报 `depth_recomputed`。
2. 录制开始前采集线程保留 `PREROLL_FRAMES`(30 帧,且不超过 192 MiB)的预处理帧,
录制开始时写入 `preroll_0001.mp4`,`session.json` 记 `preroll_frames` /
`preroll_segment`,回放先把它们喂给 SDK 再进入首帧。
都不满足时(跨分段、无归零、无预卷)仍然重算,但结果带
`state_approximate` / `state_across_segment` 提示,**不假装与当时一致**。
- 回放游标与深度结果按**请求序号 + 帧号**绑定:等待期间移动游标会作废旧请求,结果回来
只渲染属于当前帧的那一张;否则面板显示「当前游标位置尚未重算深度」,绝不复用旧图。
--- ---
## 7. 同步回放与深度重算 ## 7. 同步回放与深度(不保存深度)
**决定:不保存深度。** 回放时按录像与配置**离线重算**。 **决定:不保存深度。** 理由是体积:30 Hz、320×240 的 float32 深度每帧约 300 KB,
一小时就是 ~16 GB;即使降频到 1 Hz 也要百兆级。所以 `session.json` 里明确写:
理由是体积:30 Hz、320×240 的 float32 深度每帧约 300 KB,一小时就是 ~16 GB;
即使降频到 1 Hz 也要百兆级。所以 `session.json` 里明确写:
```json ```json
"depth": { "depth": {
"stored": false, "stored": false,
"replay": "recompute", "replay": "none",
"quantity": "relative deformation / algorithm response, not a calibrated length", "quantity": "relative deformation / algorithm response, not a calibrated length",
"method": "re-run the SDK over raw_*.mp4 with this config_ref", "method": "re-run the SDK over raw_*.mp4 with this config_ref",
"config_sha256": "...", "config_sha256": "...",
@@ -311,23 +302,26 @@ outputs/recordings/<UTC 时间戳>_session<NNNN>/
} }
``` ```
代价是重算要 SDK 能复现同样结果,因此 `RecordingReader.replay_warnings()` 会主动 因此 `RecordingReader.replay_warnings()` 会比对**当前配置文件哈希**与录制时的哈希、以及
比对**当前配置文件哈希**与录制时的哈希、以及 SDK 版本,回放窗口顶部把这些提示出来: SDK 版本,「由视频重算」前这些提示必须让操作者看到(回放视图里放在来源摘要的 tooltip 与
一条 6 秒通知里):
| 提示 | 含义 | | 提示 | 含义 |
|---|---| |---|---|
| `depth_recomputed` | 深度来自离线重算,不是当时保存的数据(始终提示) | | `config_changed` | 配置文件已变,由视频重算的结果可能与当时不同 |
| `config_changed` | 配置文件已变,重算结果可能与当时不同 |
| `sdk_changed` | SDK 版本已变,重算结果可能不同 | | `sdk_changed` | SDK 版本已变,重算结果可能不同 |
| `config_missing` | 缺少配置哈希,无法校验一致性 | | `config_missing` | 缺少配置哈希,无法校验一致性 |
| `rows_skipped` | 有无法解析的 CSV 行被跳过 | | `rows_skipped` | 有无法解析的 CSV 行被跳过 |
| `frames_dropped` | 有帧在队列溢出时丢失,只留索引行 |
| `out_of_order` | 旧版本写入的行序不是时间序 |
### 回放界面 ### 回放界面
回放**与实时是同一套界面**:`ui/playback_dialog.py` 与 `ui/replay_bar.py` 都已删除。 回放**与实时是同一套界面**:`ui/playback_dialog.py`、`ui/replay_bar.py` 与深度重算引擎都
`replay.py::ReplayController` 只读录制并发信号,主窗口把它接到同一套面板上——曲线表头的 已删除。`replay.py::ReplayController` 只读录制并发信号,主窗口把它接到同一套面板上——
播放按钮就是播放头,来源槽在回放视图里换成「选择录制视频…」。布局、单位、看点与关闭流程 状态轨的主按钮在回放视图里变成「▶ 回放 / ⏸ 暂停」(`toolbar.set_run_role`),来源槽换成
见 `docs/shell-lifecycle.md` §2(含「由视频重算」复核模式)。 「选择录制视频…」。布局、单位、看点与关闭流程见 `docs/shell-lifecycle.md` §2
(含「由视频重算」复核模式)。
- **一个游标驱动三处**:移动位置同时更新视频帧与三张曲线的游标线;并且通过 `cursor_moved` - **一个游标驱动三处**:移动位置同时更新视频帧与三张曲线的游标线;并且通过 `cursor_moved`
信号同步到主界面的曲线面板。 信号同步到主界面的曲线面板。
@@ -339,10 +333,9 @@ outputs/recordings/<UTC 时间戳>_session<NNNN>/
原始计数的曲线旁边会写着实时的 `0.0`,看起来就像"回放根本没打开"。 原始计数的曲线旁边会写着实时的 `0.0`,看起来就像"回放根本没打开"。
实时视图仍是"最新样本就是游标",所以跟随状态下不额外读盘。 实时视图仍是"最新样本就是游标",所以跟随状态下不额外读盘。
- 帧未被写入视频时(记录队列曾溢出)明确提示,而不是显示上一帧冒充当前帧。 - 帧未被写入视频时(记录队列曾溢出)明确提示,而不是显示上一帧冒充当前帧。
- **深度不随录制保存,界面也不再提供深度视图**(用户明确不需要这条回放路径);需要复核 - **解码器常开**(`RecordingReader.video_frame_at`):以前每帧重开文件 + seek,1080p 下
算法时用「由视频重算」,它把录制视频重新跑一遍管线(含深度、形变与接触)。离线深度重算 每帧上百毫秒,30 fps 播放时视频根本不动;现在按分段缓存解码器、顺序帧 `read()` 步进
引擎(`ReplayController.request_depth` / `DepthRecomputer`)保留在 `replay.py` 里并有 (实测 0.5 ms/帧),跳转才 seek。
独立测试,但已没有界面入口。
两个必须知晓的边界: 两个必须知晓的边界:
+16 -10
View File
@@ -14,7 +14,7 @@
| 视图 | 面板 | 曲线 | 比例(3D,2D,曲线,标定) | | 视图 | 面板 | 曲线 | 比例(3D,2D,曲线,标定) |
|---|---|---|---| |---|---|---|---|
| 实时监测 `live` | 2D 图像 + 力曲线(+ 手指三维) | 跟随实时 | `0, 360, 1080, 0` | | 实时监测 `live` | 2D 图像 + 力曲线(+ 手指三维) | 跟随实时 | `0, 360, 1080, 0` |
| 历史回放 `replay` | 2D 图像 + 力曲线 + **回放条** | 读打开的录制;未打开时仍是实时跟随 | `0, 320, 1120, 0` | | 历史回放 `replay` | 2D 图像 + 力曲线(**与实时同布局**) | 读打开的录制;未打开时仍是实时跟随 | `0, 360, 1080, 0` |
| 标定 `calibration` | 标定面板 + 力曲线 | 不变 | `0, 0, 560, 880` | | 标定 `calibration` | 标定面板 + 力曲线 | 不变 | `0, 0, 560, 880` |
| 自由 `custom`(启动默认) | 由视图菜单决定 | 不变 | 用户在分割条上摆的位置 | | 自由 `custom`(启动默认) | 由视图菜单决定 | 不变 | 用户在分割条上摆的位置 |
@@ -33,7 +33,7 @@
|---|---| |---|---|
| 实时监测 + 摄像机 | `摄像机 [编号] [搜索]` | | 实时监测 + 摄像机 | `摄像机 [编号] [搜索]` |
| 实时监测 + 视频文件 | `使用摄像机` 按钮 + 摘要里的 `视频文件: xxx.mp4` | | 实时监测 + 视频文件 | `使用摄像机` 按钮 + 摘要里的 `视频文件: xxx.mp4` |
| 历史回放 | 不占位(回放条里有录制选择与路径) | | 历史回放 | 「选择录制视频…」按钮 + 摘要里的视频名 |
「用视频文件跑一次采集」仍从「文件 → 打开视频…」进入,`source_mode` / `video_path` 「用视频文件跑一次采集」仍从「文件 → 打开视频…」进入,`source_mode` / `video_path`
偏好保留,但不再有一个需要先选「来源」才能用的下拉。 偏好保留,但不再有一个需要先选「来源」才能用的下拉。
@@ -61,10 +61,10 @@
`ui/replay_bar.py`(进度条 + 录制目录选择)与 `PlaybackDialog` 都已删除。 `ui/replay_bar.py`(进度条 + 录制目录选择)与 `PlaybackDialog` 都已删除。
- `apps/studio/src/orisys_studio/replay.py::ReplayController(QObject)`:录制读取、游标、 - `apps/studio/src/orisys_studio/replay.py::ReplayController(QObject)`:录制读取、游标、
离线深度重算引擎(daemon 线程 + 120 ms 轮询、请求绑定 `(_record_serial, seq, frame)`、 回放提示(降采样 / 跨版本)。它不画任何东西,只发信号:`position_changed` /
旧结果一律丢弃;**界面已不提供深度视图**)、深度色标、回放提示(降采样 / 跨版本)。 `frames_changed` / `status_changed` / `recording_changed`。深度重算引擎
它不画任何东西,只发信号:`position_changed` / `frames_changed` / (`DepthRecomputer` / `depth_backends`)已随回放条一起删除——用户不需要这条回放路径,
`status_changed` / `recording_changed`。 要复核算法就用「由视频重算」。
- **选视频即选录制**:状态轨的「选择录制视频…」(`toolbar.btn_replay_video`)与 - **选视频即选录制**:状态轨的「选择录制视频…」(`toolbar.btn_replay_video`)与
「文件 → 打开视频…」都走 `_load_replay_video(path)`——视频所在目录就是录制目录, 「文件 → 打开视频…」都走 `_load_replay_video(path)`——视频所在目录就是录制目录,
于是曲线自动配对同目录的 `forces.csv`。视频与 CSV 仍然同时记录(录制格式不变)。 于是曲线自动配对同目录的 `forces.csv`。视频与 CSV 仍然同时记录(录制格式不变)。
@@ -77,9 +77,15 @@
- 2D 面板:`ArrowPanel.set_source_kind("replay")` 把视图下拉换成「原始图像」一项 - 2D 面板:`ArrowPanel.set_source_kind("replay")` 把视图下拉换成「原始图像」一项
(`REPLAY_VIEW_MODES`;录制里没有光流,也没有存储深度),页脚显示「录制视频(原始相机 (`REPLAY_VIEW_MODES`;录制里没有光流,也没有存储深度),页脚显示「录制视频(原始相机
图像)」并隐藏放大倍率与有效区域。 图像)」并隐藏放大倍率与有效区域。
- **游标就是播放头**:曲线表头的播放按钮(`chart.play` / `chart.pause`,编辑器习惯: - **游标就是播放头,运输控制就是状态轨上那颗主按钮**:回放视图里
空格键同效)以 1× 推进游标,到末尾自动停;拖动游标、框选、滚轮平移或拖总览条都会 `toolbar.set_run_role("playback")` 把它换成「▶ 回放」/「⏸ 暂停」(实时视图仍是「运行」),
**自动暂停**(`ForcePlotsPanel.pause_playback`)。没有独立进度条。 以 1× 推进游标,到末尾自动停;拖动游标、框选、滚轮平移或拖总览条都会**自动暂停**
(`ForcePlotsPanel.timeline_taken_over` → `_pause_playback`)。空格键同效。没有独立进度条,
曲线表头也不再放第二个播放按钮(一个运输控制,不会两处打架)。
- **视频解码器常开**:`RecordingReader.video_frame_at` 以前每帧 `VideoCapture` 重开 + seek
(小片段 10 ms,1080p H.264 上百 ms),30 fps 播放时每一拍都卡在 UI 线程上,视频看起来
根本不动。现在按分段缓存一个解码器,**顺序帧用 `read()` 步进**(实测 0.5 ms/帧),
跳转才 seek,`RecordingReader.close()` 负责释放。
- **数值卡读游标处的那一帧**(`ForcePlotsPanel._readout_at_cursor`)。浏览历史或回放时, - **数值卡读游标处的那一帧**(`ForcePlotsPanel._readout_at_cursor`)。浏览历史或回放时,
三个大数字是播放头所指的样本;跟随实时时"最新样本就是游标",不额外读盘。旧写法在回放 三个大数字是播放头所指的样本;跟随实时时"最新样本就是游标",不额外读盘。旧写法在回放
时显示实时采集的最后一个样本,于是一张 7.5e5 原始计数的曲线旁边写着实时的 `0.0`。 时显示实时采集的最后一个样本,于是一张 7.5e5 原始计数的曲线旁边写着实时的 `0.0`。
@@ -106,7 +112,7 @@
| 用途 | 看当时的记录 | 复核算法版本(mp4 有损,数值与原记录会有差异) | | 用途 | 看当时的记录 | 复核算法版本(mp4 有损,数值与原记录会有差异) |
打开它时会立即开始这一遍(`_on_recompute_toggled` → `force_plots.set_playing(True)` → 打开它时会立即开始这一遍(`_on_recompute_toggled` → `force_plots.set_playing(True)` →
`_on_chart_play_toggled` → `_start_run`,只有一条启动路径,避免按钮与菜单各启动一次); `_on_transport_toggled` → `_start_run`,只有一条启动路径,避免按钮与菜单各启动一次);
关闭时停止该遍并挂回 CSV。`_replay_owns_image()` 在这一遍运行期间为 False,2D 面板交给 关闭时停止该遍并挂回 CSV。`_replay_owns_image()` 在这一遍运行期间为 False,2D 面板交给
管线自己画。 管线自己画。
+3 -3
View File
@@ -180,15 +180,15 @@ flow_ramp = 画布色, #6E4AD0, #FA02F3, #FF8A6B, #D8FF00
| 品牌轨(`QFrame#brandBar`) | 28 px | logo + 字标 + 三个顶层视图(实时监测 / 历史回放 / 标定)+ 右侧产品名 | | 品牌轨(`QFrame#brandBar`) | 28 px | logo + 字标 + 三个顶层视图(实时监测 / 历史回放 / 标定)+ 右侧产品名 |
| 状态轨(`ToolbarWidget` 的 rail) | ~30 px | **传感器摘要按钮**、阶段、**当前视图的来源选择**(摄像机编号 + 搜索 / 使用摄像机)、标定状态、数据有效性、记录状态、记录/运行/重置/设置 | | 状态轨(`ToolbarWidget` 的 rail) | ~30 px | **传感器摘要按钮**、阶段、**当前视图的来源选择**(摄像机编号 + 搜索 / 使用摄像机)、标定状态、数据有效性、记录状态、记录/运行/重置/设置 |
| 设置区(默认收起) | 0 / ~40 px | 单位、记录模式、预览 | | 设置区(默认收起) | 0 / ~40 px | 单位、记录模式、预览 |
| 回放条(仅历史回放视图) | ~50 px | 录制选择、录制信息、进度条、位置、单位、重算深度、提示行 | | ~~回放条~~(已取消) | 0 px | 回放视图与实时同布局:运输控制就是状态轨的主按钮(实时=运行,回放=回放/暂停) |
四点规则: 四点规则:
1. **传感器区是一个摘要按钮,不是两个下拉**。轨上只有 `BOX · box/sensor.json`;类型、配置 1. **传感器区是一个摘要按钮,不是两个下拉**。轨上只有 `BOX · box/sensor.json`;类型、配置
与「当前生效的标定 / 最近导出的系数」都在它打开的 `SensorDialog` 里(见 与「当前生效的标定 / 最近导出的系数」都在它打开的 `SensorDialog` 里(见
`docs/shell-lifecycle.md` §1.2)。摘要按钮属于状态轨,因此文本必须完整显示(见第 5 条)。 `docs/shell-lifecycle.md` §1.2)。摘要按钮属于状态轨,因此文本必须完整显示(见第 5 条)。
2. **来源选择随视图**:实时视图显示摄像机编号 + 搜索;用视频文件采集时显示「使用摄像机」; 2. **来源选择随视图**:实时视图显示摄像机编号 + 搜索;回放视图显示「选择录制视频…」,视频名
回放视图不占位(回放条里有录制选择)。没有「来源」下拉,因此没有"先选来源才能用"的状态。 写在来源摘要里(只在回放视图)。没有「来源」下拉,因此没有"先选来源才能用"的状态。
3. **导航按钮在品牌底上取品牌前景色**。`navButton` 用 `brand_bar_fg_muted`,当前视图用 3. **导航按钮在品牌底上取品牌前景色**。`navButton` 用 `brand_bar_fg_muted`,当前视图用
`brand_bar_fg` + `navActive="true"`;两者都在 §2 的品牌底对比度审计里(7.67 / 12.94), `brand_bar_fg` + `navActive="true"`;两者都在 §2 的品牌底对比度审计里(7.67 / 12.94),
不新增色值。 不新增色值。
+51 -29
View File
@@ -246,13 +246,13 @@ class StudioMainWindow(QMainWindow):
# Playback drives the *chart cursor* (the playhead) instead of a slider # Playback drives the *chart cursor* (the playhead) instead of a slider
# under the panels: the two views share one layout, and the transport is # under the panels: the two views share one layout, and the transport is
# a single button on the chart header. # the rail's primary button — 「运行」while measuring, 「回放」while
# replaying (see ``_on_run_toggled`` / ``toolbar.set_run_role``).
self._play_timer = QTimer(self) self._play_timer = QTimer(self)
self._play_timer.setInterval(PLAYBACK_TICK_MS) self._play_timer.setInterval(PLAYBACK_TICK_MS)
self._play_timer.timeout.connect(self._on_play_tick) self._play_timer.timeout.connect(self._on_play_tick)
self._play_started_at = 0.0 self._play_started_at = 0.0
self._play_from = 0.0 self._play_from = 0.0
self.force_plots.play_toggled.connect(self._on_chart_play_toggled)
self.setStatusBar(_NoticeBar()) self.setStatusBar(_NoticeBar())
# The status bar carries **one-off notices** only (a recording finished, a # The status bar carries **one-off notices** only (a recording finished, a
@@ -526,12 +526,13 @@ class StudioMainWindow(QMainWindow):
"device_sn": self._device_serial(info), "device_sn": self._device_serial(info),
}, },
# Depth is deliberately NOT stored (decided with the team): it is a # Depth is deliberately NOT stored (decided with the team): it is a
# function of the frame and the sensor configuration, so replay # function of the frame and the sensor configuration, and the replay
# recomputes it from raw_*.mp4. The hashes below are what a replay # view shows the recorded frames only. Re-running the pipeline over
# checks before trusting that recomputation. # raw_*.mp4 is the "recompute" mode's job; the hashes below are what
# such a pass (and any other consumer) checks before trusting it.
"depth": { "depth": {
"stored": False, "stored": False,
"replay": "recompute", "replay": "none",
"quantity": "relative deformation / algorithm response, not a calibrated length", "quantity": "relative deformation / algorithm response, not a calibrated length",
"method": "re-run the SDK over raw_*.mp4 with this config_ref", "method": "re-run the SDK over raw_*.mp4 with this config_ref",
"config_sha256": self._config_hash(config_ref), "config_sha256": self._config_hash(config_ref),
@@ -616,6 +617,7 @@ class StudioMainWindow(QMainWindow):
self.replay.recording_changed.connect(self._on_recording_changed) self.replay.recording_changed.connect(self._on_recording_changed)
# Dragging the shared cursor on the charts scrubs the recording too. # Dragging the shared cursor on the charts scrubs the recording too.
self.force_plots.cursor_moved.connect(self._on_cursor_moved) self.force_plots.cursor_moved.connect(self._on_cursor_moved)
self.force_plots.timeline_taken_over.connect(self._pause_playback)
def _on_cursor_moved(self, t_rel_s: float) -> None: def _on_cursor_moved(self, t_rel_s: float) -> None:
if self._recompute_running: if self._recompute_running:
@@ -693,11 +695,12 @@ class StudioMainWindow(QMainWindow):
segments = sorted(reader.segments()) if reader is not None else [] segments = sorted(reader.segments()) if reader is not None else []
if segments: if segments:
self.toolbar.set_video_path(str(segments[0])) self.toolbar.set_video_path(str(segments[0]))
self.force_plots.set_playing(self._play_timer.isActive()) self.toolbar.set_run_checked(self._play_timer.isActive())
# Loading a recording is what makes a recompute pass possible at all. # Loading a recording is what makes a recompute pass possible at all.
self.act_recompute.setEnabled( self.act_recompute.setEnabled(
self._view_preset == "replay" and self.replay.is_loaded() self._view_preset == "replay" and self.replay.is_loaded()
) )
self._update_run_enabled()
self._refresh_status_rail() self._refresh_status_rail()
def _on_replay_position(self, position_s: float) -> None: def _on_replay_position(self, position_s: float) -> None:
@@ -708,8 +711,8 @@ class StudioMainWindow(QMainWindow):
self._set_display_unit("raw" if self.force_plots.unit() == "N" else "N") self._set_display_unit("raw" if self.force_plots.unit() == "N" else "N")
# ------------------------------------------------------------- playback # ------------------------------------------------------------- playback
def _on_chart_play_toggled(self, playing: bool) -> None: def _on_transport_toggled(self, playing: bool) -> None:
"""The transport button on the chart header, like an editor's play/pause.""" """The rail's primary button in the replay view: play or pause."""
if self._recompute_active(): if self._recompute_active():
# A recompute pass is a *run*: play starts the pipeline over the # A recompute pass is a *run*: play starts the pipeline over the
# video, pause stops it. It can only go forward (the chain is # video, pause stops it. It can only go forward (the chain is
@@ -719,7 +722,7 @@ class StudioMainWindow(QMainWindow):
self._start_run() self._start_run()
elif self._recompute_running: elif self._recompute_running:
self._stop_run(reason="user_stop") self._stop_run(reason="user_stop")
self.force_plots.set_playing(bool(playing) and self._running) self.toolbar.set_run_checked(bool(playing) and self._running)
return return
if playing and self.replay.is_loaded(): if playing and self.replay.is_loaded():
self._start_playback() self._start_playback()
@@ -729,7 +732,7 @@ class StudioMainWindow(QMainWindow):
def _start_playback(self) -> None: def _start_playback(self) -> None:
duration = self.replay.duration_s() duration = self.replay.duration_s()
if duration <= 0: if duration <= 0:
self.force_plots.set_playing(False) self.toolbar.set_run_checked(False)
return return
position = self.replay.position() position = self.replay.position()
if position >= duration - 0.02: if position >= duration - 0.02:
@@ -739,12 +742,15 @@ class StudioMainWindow(QMainWindow):
self._play_from = float(position) self._play_from = float(position)
self._play_started_at = time.perf_counter() self._play_started_at = time.perf_counter()
self._play_timer.start() self._play_timer.start()
self.force_plots.set_playing(True) self.toolbar.set_run_checked(True)
self.toolbar.refresh_labels()
def _pause_playback(self) -> None: def _pause_playback(self) -> None:
if self._play_timer.isActive(): if self._play_timer.isActive():
self._play_timer.stop() self._play_timer.stop()
self.force_plots.set_playing(False) if self.toolbar.btn_run.isChecked():
self.toolbar.set_run_checked(False)
self.toolbar.refresh_labels()
def _on_play_tick(self) -> None: def _on_play_tick(self) -> None:
"""Advance the playhead from the wall clock, not from the tick count.""" """Advance the playhead from the wall clock, not from the tick count."""
@@ -809,7 +815,9 @@ class StudioMainWindow(QMainWindow):
self._recompute_running = True self._recompute_running = True
# Playing is what starts the pass: one path in, so the button and the # Playing is what starts the pass: one path in, so the button and the
# menu cannot start it twice. # menu cannot start it twice.
self.force_plots.set_playing(True) self.toolbar.set_run_checked(True)
self.toolbar.refresh_labels()
self._on_transport_toggled(True)
return return
if self._recompute_running: if self._recompute_running:
self._stop_run(reason="user_stop") self._stop_run(reason="user_stop")
@@ -831,18 +839,6 @@ class StudioMainWindow(QMainWindow):
else: else:
self.arrow_panel.show_frame(image) self.arrow_panel.show_frame(image)
def _replay_depth_scale_text(self) -> str:
"""The depth colour scale, latched per recording by the controller.
Kept for the offline depth engine (``ReplayController.depth_image``); the
replay view itself no longer offers a depth pane, because depth is not
stored with a recording.
"""
span = self.replay.depth_range()
if span is None:
return ""
return t("panels.depth_scale", low=f"{span[0]:.3f}", high=f"{span[1]:.3f}")
def _apply_replay_source(self, active: bool) -> None: def _apply_replay_source(self, active: bool) -> None:
"""Point the charts at the recording (or back at the live session).""" """Point the charts at the recording (or back at the live session)."""
if active and self.replay.is_loaded(): if active and self.replay.is_loaded():
@@ -1152,6 +1148,9 @@ class StudioMainWindow(QMainWindow):
# visible difference between the two views. # visible difference between the two views.
self._pause_playback() self._pause_playback()
self.toolbar.set_source_view("replay" if preset == "replay" else "live") self.toolbar.set_source_view("replay" if preset == "replay" else "live")
# The rail's primary button is this view's main action: acquire while
# measuring, play back while replaying.
self.toolbar.set_run_role("playback" if preset == "replay" else "acquire")
self._apply_panel_visibility() self._apply_panel_visibility()
if apply_sizes and preset in VIEW_SPLITTER_SIZES: if apply_sizes and preset in VIEW_SPLITTER_SIZES:
self.content_splitter.setSizes(list(VIEW_SPLITTER_SIZES[preset])) self.content_splitter.setSizes(list(VIEW_SPLITTER_SIZES[preset]))
@@ -1168,6 +1167,7 @@ class StudioMainWindow(QMainWindow):
self._on_recompute_toggled(False) self._on_recompute_toggled(False)
self.brand_bar.set_active_view(preset if preset != "custom" else None) self.brand_bar.set_active_view(preset if preset != "custom" else None)
self._sync_menu_checks() self._sync_menu_checks()
self._update_run_enabled()
self._refresh_status_rail() self._refresh_status_rail()
if persist: if persist:
self._persist_prefs() self._persist_prefs()
@@ -1176,6 +1176,22 @@ class StudioMainWindow(QMainWindow):
return self._view_preset return self._view_preset
def _update_run_enabled(self) -> None: def _update_run_enabled(self) -> None:
if self.toolbar.run_role() == "playback":
# In the replay view the button is a transport: it needs a recording,
# not a runnable sensor. While it plays it stays enabled so it can be
# paused.
loaded = self.replay.is_loaded()
self.toolbar.set_run_tooltip_reason(
"" if loaded else t("toolbar.playback_disabled")
)
self.toolbar.set_run_enabled(loaded or self._play_timer.isActive())
self.toolbar.set_record_enabled(
self._running or self.recorder.is_recording
)
if not self._running:
self.toolbar.refresh_labels()
self._sync_shortcut_actions()
return
sensor_ok = self.sensor_picker.sensor_type() in RUNNABLE_TYPES sensor_ok = self.sensor_picker.sensor_type() in RUNNABLE_TYPES
source_ok = True source_ok = True
reason = "" reason = ""
@@ -1809,9 +1825,9 @@ class StudioMainWindow(QMainWindow):
def _on_play_shortcut(self) -> None: def _on_play_shortcut(self) -> None:
"""Space plays or pauses the recording, if there is one to move through.""" """Space plays or pauses the recording, if there is one to move through."""
if not self.force_plots.btn_play.isEnabled(): if self.toolbar.run_role() != "playback" or not self.toolbar.btn_run.isEnabled():
return return
self.force_plots.btn_play.setChecked(not self.force_plots.btn_play.isChecked()) self.toolbar.btn_run.setChecked(not self.toolbar.btn_run.isChecked())
def _on_run_shortcut(self) -> None: def _on_run_shortcut(self) -> None:
if not self.toolbar.btn_run.isEnabled(): if not self.toolbar.btn_run.isEnabled():
@@ -1953,6 +1969,10 @@ class StudioMainWindow(QMainWindow):
self.arrow_panel.show_frame(_bgr_to_qimage(frame)) self.arrow_panel.show_frame(_bgr_to_qimage(frame))
def _on_run_toggled(self, checked: bool) -> None: def _on_run_toggled(self, checked: bool) -> None:
if self.toolbar.run_role() == "playback":
# The primary button of the replay view is its transport.
self._on_transport_toggled(bool(checked))
return
if checked: if checked:
self._start_run() self._start_run()
else: else:
@@ -2072,7 +2092,9 @@ class StudioMainWindow(QMainWindow):
self._running = False self._running = False
# A recompute pass is over: the transport button goes back to "play". # A recompute pass is over: the transport button goes back to "play".
self._recompute_running = False self._recompute_running = False
self.force_plots.set_playing(False) if self.toolbar.run_role() == "playback" and self.toolbar.btn_run.isChecked():
self.toolbar.set_run_checked(False)
self.toolbar.refresh_labels()
self._set_phase("error" if reason == "failed" else "idle") self._set_phase("error" if reason == "failed" else "idle")
self.cal_panel.set_running(False) self.cal_panel.set_running(False)
self.toolbar.set_run_checked(False) self.toolbar.set_run_checked(False)
+47 -346
View File
@@ -1,32 +1,26 @@
"""Replay engine: one recording, one cursor, and the depth it can recompute. """Replay engine: one recording, one cursor, one frame at a time.
This is the headless half of the old replay window. It owns the recording This is the headless half of the replay view. It owns the recording reader and
reader, the cursor and the asynchronous depth recomputation, and it *shows* the cursor and it *shows* nothing: the main window renders the frame into the 2D
nothing: the main window renders the frames into the 2D panel, the curves read panel, the curves read the recording through
the recording through :class:`~orisys_app_shared.force.source.RecordingCurveSource` :class:`~orisys_app_shared.force.source.RecordingCurveSource`, and the transport
and the progress bar drives :meth:`set_position`. on the chart header drives :meth:`set_position`.
Depth is recomputed on demand rather than read from disk (see There is no depth recomputation here: depth is not stored with a recording, the
:mod:`orisys_app_shared.record.depth`), so the pane states plainly when it is replay view shows the recorded frames, and re-running the pipeline over a video
unavailable instead of showing a stand-in image. is the acquisition path's job (see the studio's "recompute" mode).
""" """
from __future__ import annotations from __future__ import annotations
import logging import logging
import threading
from pathlib import Path
import numpy as np import numpy as np
from PyQt5.QtCore import QObject, QTimer, pyqtSignal from PyQt5.QtCore import QObject, pyqtSignal
from PyQt5.QtGui import QImage from PyQt5.QtGui import QImage
from orisys_app_shared.i18n import t from orisys_app_shared.i18n import t
from orisys_app_shared.record import ( from orisys_app_shared.record import RecordingReader
DepthRecomputer,
DepthResult,
RecordingReader,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -37,14 +31,25 @@ def _bgr_to_qimage(frame: np.ndarray) -> QImage:
return QImage(rgb.data, width, height, channels * width, QImage.Format_RGB888).copy() return QImage(rgb.data, width, height, channels * width, QImage.Format_RGB888).copy()
def _sdk_version() -> str:
"""The SDK version in use, or "" when the SDK is not importable."""
try:
import orisys
return str(getattr(orisys, "__version__", "") or "")
except Exception: # noqa: BLE001 - a missing SDK is a warning, not a crash
logger.debug("Could not read the SDK version", exc_info=True)
return ""
class ReplayController(QObject): class ReplayController(QObject):
"""Open one recording and answer "what is at t?" for every view.""" """Open one recording and answer "what is at t?" for every view."""
#: The cursor moved; the progress bar and the charts' cursor follow. #: The cursor moved; the chart cursor and the 2D panel follow.
position_changed = pyqtSignal(float) position_changed = pyqtSignal(float)
#: The frame under the cursor changed (video and/or depth). #: The frame under the cursor changed.
frames_changed = pyqtSignal() frames_changed = pyqtSignal()
#: Warnings, depth state or the loaded recording changed. #: The notes about this recording, or the loaded recording, changed.
status_changed = pyqtSignal() status_changed = pyqtSignal()
#: A recording was opened (True) or the previous one was dropped (False). #: A recording was opened (True) or the previous one was dropped (False).
recording_changed = pyqtSignal(bool) recording_changed = pyqtSignal(bool)
@@ -52,51 +57,19 @@ class ReplayController(QObject):
def __init__(self, parent=None) -> None: def __init__(self, parent=None) -> None:
super().__init__(parent) super().__init__(parent)
self._reader: RecordingReader | None = None self._reader: RecordingReader | None = None
self._recomputer: DepthRecomputer | None = None
self._unit = "raw" self._unit = "raw"
self._position = 0.0 self._position = 0.0
self._depth_ok = False
# A **daemon** thread, not a ThreadPoolExecutor: the SDK call can block
# indefinitely and an executor's non-daemon worker would then keep the
# whole process alive at exit. A blocked daemon thread cannot be
# cancelled either, so it is left to die with the process.
self._depth_thread: threading.Thread | None = None
self._depth_result = None
self._depth_error: BaseException | None = None
#: Which cursor position each depth answer belongs to. Recomputation is
#: asynchronous, so an answer can arrive after the cursor has moved; the
#: request it answers is remembered and compared, never assumed.
self._depth_request_seq = 0
self._depth_pending_seq: int | None = None
self._depth_pending_position: float | None = None
self._depth_result_seq: int | None = None
#: Which recording each answer belongs to (incremented on every open).
self._record_serial = 0
self._depth_pending_record: int | None = None
self._depth_result_record: int | None = None
self._depth_result_frame = -1
#: Processors whose owning request was still running when it was retired.
self._retired_recomputers: list = []
#: The frame cache: one video image per cursor position. #: The frame cache: one video image per cursor position.
self._video_position: float | None = None self._video_position: float | None = None
self._video_qimage: QImage | None = None self._video_qimage: QImage | None = None
self._video_reason = "" self._video_reason = ""
self._depth_qimage: QImage | None = None
self._depth_message = ""
self._warnings = "" self._warnings = ""
#: The depth colour scale, latched to the first recomputed frame of the
#: recording so two frames stay comparable (same rule as the live view).
self._depth_range: tuple[float, float] | None = None
self._depth_poll = QTimer(self)
self._depth_poll.setInterval(120)
self._depth_poll.timeout.connect(self._poll_depth)
# ------------------------------------------------------------- recording # ------------------------------------------------------------- recording
def is_loaded(self) -> bool: def is_loaded(self) -> bool:
return self._reader is not None return self._reader is not None
def directory(self) -> Path | None: def directory(self):
return self._reader.directory if self._reader is not None else None return self._reader.directory if self._reader is not None else None
def reader(self): def reader(self):
@@ -113,82 +86,45 @@ class ReplayController(QObject):
return RecordingCurveSource(self._reader) return RecordingCurveSource(self._reader)
def open_recording(self, directory: Path) -> bool: def open_recording(self, directory) -> bool:
"""Open a recording. Any depth answer still in flight is invalidated.""" """Open a recording directory."""
# A running request belongs to the *previous* recording: its result must self._release_reader()
# never be pasted into the new one, and its processor must be closed by
# the thread that is using it, not here.
self._retire_recomputer()
self._record_serial += 1
self._depth_request_seq += 1
self._depth_pending_seq = None
self._depth_pending_position = None
self._depth_error = None
try: try:
self._reader = RecordingReader(Path(directory)) self._reader = RecordingReader(directory)
info = self._reader.info() self._reader.info()
except Exception: # noqa: BLE001 - reported through the status text except Exception: # noqa: BLE001 - reported through the status text
logger.exception("Could not open recording %s", directory) logger.exception("Could not open recording %s", directory)
self._reader = None self._reader = None
self._recomputer = None
self._reset_frames() self._reset_frames()
self.recording_changed.emit(False) self.recording_changed.emit(False)
self.status_changed.emit() self.status_changed.emit()
return False return False
self._recomputer = DepthRecomputer(self._reader)
self._depth_ok = False
self._depth_result = None
self._depth_result_seq = None
self._depth_range = None
self._reset_frames() self._reset_frames()
self._position = 0.0 self._position = 0.0
_ = info
self.recording_changed.emit(True) self.recording_changed.emit(True)
self._refresh_warnings() self._refresh_warnings()
self._refresh_depth_pane()
self.set_position(0.0) self.set_position(0.0)
return True return True
def close(self) -> None: def close(self) -> None:
"""Drop the recording and hand every processor to its owning thread.""" """Drop the recording (the decoder is released with it)."""
self._depth_poll.stop() self._release_reader()
for recomputer in list(self._retired_recomputers):
if not self._depth_busy():
self._retired_recomputers.remove(recomputer)
try:
recomputer.close()
except Exception: # noqa: BLE001
logger.exception("Could not close a retired recomputer")
self._retire_recomputer()
self._reader = None
self._reset_frames() self._reset_frames()
self.recording_changed.emit(False) self.recording_changed.emit(False)
self.status_changed.emit() self.status_changed.emit()
def _release_reader(self) -> None:
reader, self._reader = self._reader, None
if reader is not None:
try:
reader.close()
except Exception: # noqa: BLE001 - releasing is best effort
logger.debug("Could not close the recording reader", exc_info=True)
def _reset_frames(self) -> None: def _reset_frames(self) -> None:
self._video_position = None self._video_position = None
self._video_qimage = None self._video_qimage = None
self._video_reason = "" self._video_reason = ""
self._depth_qimage = None
self._depth_message = ""
def _retire_recomputer(self) -> None:
"""Hand the current processor to its worker, or close it here if idle.
Closing a processor that a worker thread is still reading is a use-after-
free; only the thread that owns it may close it, so a busy one is marked
for the worker to close when it finishes.
"""
recomputer, self._recomputer = self._recomputer, None
if recomputer is None:
return
if self._depth_busy():
self._retired_recomputers.append(recomputer)
else:
try:
recomputer.close()
except Exception: # noqa: BLE001 - closing is best effort
logger.exception("Could not close the depth recomputer")
# ---------------------------------------------------------------- cursor # ---------------------------------------------------------------- cursor
def position(self) -> float: def position(self) -> float:
@@ -200,16 +136,12 @@ class ReplayController(QObject):
return float(self._reader.info().duration_s) return float(self._reader.info().duration_s)
def set_position(self, t_rel_s: float) -> None: def set_position(self, t_rel_s: float) -> None:
"""Move the single cursor: video, depth and charts follow it.""" """Move the single cursor: the video frame and the charts follow it."""
if self._reader is None: if self._reader is None:
return return
duration = self.duration_s() duration = self.duration_s()
self._position = max(0.0, min(float(t_rel_s), duration if duration > 0 else 0.0)) self._position = max(0.0, min(float(t_rel_s), duration if duration > 0 else 0.0))
self._refresh_video() self._refresh_video()
if self._depth_ok or self._depth_result is not None:
# The held depth belongs to the frame it was computed for. Moving the
# cursor must not keep showing it as if it were synchronised.
self._refresh_depth_pane()
self.position_changed.emit(self._position) self.position_changed.emit(self._position)
self.frames_changed.emit() self.frames_changed.emit()
@@ -217,20 +149,15 @@ class ReplayController(QObject):
return self._unit return self._unit
def set_unit(self, unit: str) -> str: def set_unit(self, unit: str) -> str:
"""Show the recording in ``unit``; the bar and the charts share one. """Show the recording in ``unit``; the charts and the rail share one.
Opening a recording that carries no calibrated axis in newton mode would Opening a recording that carries no calibrated axis in newton mode would
draw three empty charts, so the caller picks the unit from what the draw three empty charts, so the caller picks the unit from what the
recording actually has and tells both views about it here. recording actually has and tells every view about it here.
""" """
self._unit = "N" if str(unit).upper().startswith("N") else "raw" self._unit = "N" if str(unit).upper().startswith("N") else "raw"
return self._unit return self._unit
def toggle_unit(self) -> str:
self._unit = "N" if self._unit == "raw" else "raw"
self.status_changed.emit()
return self._unit
# ----------------------------------------------------------------- frames # ----------------------------------------------------------------- frames
def video_image(self) -> QImage | None: def video_image(self) -> QImage | None:
self._refresh_video() self._refresh_video()
@@ -261,12 +188,7 @@ class ReplayController(QObject):
self._video_reason = "" self._video_reason = ""
self._video_qimage = _bgr_to_qimage(image) self._video_qimage = _bgr_to_qimage(image)
def depth_image(self) -> QImage | None: # ---------------------------------------------------------------- notes
return self._depth_qimage
def depth_message(self) -> str:
return self._depth_message
def warnings_text(self) -> str: def warnings_text(self) -> str:
return self._warnings return self._warnings
@@ -275,15 +197,12 @@ class ReplayController(QObject):
if self._reader is None: if self._reader is None:
return return
info = self._reader.info() info = self._reader.info()
_ok, sdk_version = DepthRecomputer.sdk_available() codes = self._reader.replay_warnings(sdk_version=_sdk_version())
# The reader compares the recorded config hash against the config file as
# it is now, which is what the depth recomputation would actually use.
codes = self._reader.replay_warnings(sdk_version=sdk_version)
text = "" text = ""
if codes: if codes:
text = t( text = t(
"playback.warnings", "playback.warnings",
detail=", ".join(t(f"playback.warn_{c}") for c in codes), detail=", ".join(t(f"playback.warn_{code}") for code in codes),
) )
if info.decimated: if info.decimated:
# The curves are a view, not every sample: say so rather than let the # The curves are a view, not every sample: say so rather than let the
@@ -295,221 +214,3 @@ class ReplayController(QObject):
) )
text = f"{text} {note}" if text else note text = f"{text} {note}" if text else note
self._warnings = text self._warnings = text
# -------------------------------------------------------------------- depth
def request_depth(self) -> None:
"""Kick off a recomputation on a worker thread.
Opening a video with the SDK and stepping to a frame can block, so it
must never run on the UI thread — that is exactly how the replay would
freeze.
"""
if self._recomputer is None:
return
problem = self._recomputer.preflight()
if problem:
self._depth_ok = False
self._depth_qimage = None
self._depth_message = t(f"playback.depth_{problem}")
self.status_changed.emit()
return
if self._depth_thread is not None and self._depth_thread.is_alive():
self._depth_message = t("playback.depth_busy")
self.status_changed.emit()
return
self._depth_ok = False
self._depth_result = None
self._depth_result_seq = None
self._depth_error = None
self._depth_message = t("playback.depth_running")
self.status_changed.emit()
position = self._position
# Bind the answer to this request: the recording, the frame and a request
# sequence. A result that arrives after the cursor moved — or after
# another recording was opened — is discarded rather than pasted on.
recomputer = self._recomputer
record = self._record_serial
frame = self._current_frame()
self._depth_request_seq += 1
seq = self._depth_request_seq
self._depth_pending_seq = seq
self._depth_pending_position = position
self._depth_pending_record = record
def work() -> None:
# Runs off the UI thread and touches no Qt object. It captures the
# processor it was given: the controller may already be showing
# another recording by the time this returns.
try:
result = recomputer.depth_at(position)
except BaseException as exc: # noqa: BLE001 - reported via the poll
self._depth_error = (record, exc)
return
finally:
if recomputer in self._retired_recomputers:
# Retired while this request was running: the owning thread
# closes it, not the UI thread.
self._retired_recomputers.remove(recomputer)
try:
recomputer.close()
except Exception: # noqa: BLE001
logger.exception("Could not close a retired recomputer")
self._depth_result = result
self._depth_result_seq = seq
self._depth_result_record = record
self._depth_result_frame = getattr(frame, "frame_id", -1)
self._depth_thread = threading.Thread(
target=work, name="depth-replay", daemon=True
)
self._depth_thread.start()
self._depth_poll.start()
def depth_busy(self) -> bool:
return self._depth_thread is not None and self._depth_thread.is_alive()
# The old spelling, kept for the tests and for readability at call sites.
_depth_busy = depth_busy
def _poll_depth(self) -> None:
"""UI-thread poll for the worker result."""
thread = self._depth_thread
if thread is None or thread.is_alive():
return
self._depth_poll.stop()
pending = self._depth_pending_seq
record = self._depth_pending_record
self._depth_pending_seq = None
self._depth_pending_position = None
self._depth_pending_record = None
if self._depth_error is not None:
error_record, exc = self._depth_error
self._depth_error = None
if error_record != self._record_serial:
# The failure belongs to a recording that is no longer open.
return
self._depth_ok = False
self._depth_qimage = None
self._depth_message = t("playback.depth_failed", detail=str(exc))
self.status_changed.emit()
self.frames_changed.emit()
return
if record != self._record_serial:
# Another recording was opened while this was running.
return
result = self._depth_result
if result is None or self._depth_result_seq != pending:
# A newer request (or a moved cursor) owns the pane now.
self._refresh_depth_pane()
self.frames_changed.emit()
return
if not self._depth_matches_cursor(result):
# The cursor moved while the SDK was reading: do not draw an answer
# for another frame, and say the pane needs recomputing.
self._refresh_depth_pane()
self.frames_changed.emit()
return
self._render_depth_result(result)
self.status_changed.emit()
self.frames_changed.emit()
def _current_frame(self):
"""The recorded frame under the cursor, or None without a frame index."""
frame_at = getattr(self._reader, "frame_at", None)
if not callable(frame_at):
return None
try:
return frame_at(self._position)
except Exception: # noqa: BLE001 - a stub reader has no frame index
return None
def _depth_matches_cursor(self, result) -> bool:
"""True when ``result`` was computed for the frame under the cursor."""
frame = self._current_frame()
frame_id = getattr(result, "frame_id", None)
if frame is not None and frame_id is not None and int(frame_id) >= 0:
return int(frame_id) == int(frame.frame_id)
t_rel = getattr(result, "t_rel_s", None)
if t_rel is None:
# Nothing in the result says which frame it belongs to, so it cannot
# be shown to be stale either.
return True
if frame is not None:
return abs(float(t_rel) - float(frame.t_rel_s)) < 1e-6
return abs(float(t_rel) - self._position) < 1e-9
def _render_depth_result(self, result) -> None:
code = str(getattr(result, "error", "") or "")
if code in ("not_recomputed", "recomputing", "stale"):
# A placeholder, not a failure: the pane describes the cursor.
self._depth_ok = False
self._depth_qimage = None
self._depth_message = t(f"playback.depth_{code}")
return
if not result.ok:
self._depth_ok = False
self._depth_qimage = None
self._depth_message = t("playback.depth_failed", detail=result.error)
return
from orisys_app_shared.acquisition.arrows import depth_map_to_bgr
depth = np.asarray(result.depth)
self._latch_depth_range(depth)
bgr = depth_map_to_bgr(depth, value_range=self._depth_range)
if bgr is None:
self._depth_ok = False
self._depth_qimage = None
self._depth_message = t("playback.depth_empty")
return
self._depth_ok = True
self._depth_message = self._depth_warning_text(result)
self._depth_qimage = _bgr_to_qimage(bgr)
def _latch_depth_range(self, depth: np.ndarray) -> None:
if self._depth_range is not None:
return
finite = np.isfinite(depth)
if depth.size == 0 or not finite.any():
return
low = float(np.nanmin(depth[finite]))
high = float(np.nanmax(depth[finite]))
if high <= low:
high = low + 1.0
self._depth_range = (low, high)
def depth_range(self) -> tuple[float, float] | None:
return self._depth_range
@staticmethod
def _depth_warning_text(result) -> str:
"""Say when a replay could not reproduce the live algorithm state."""
codes = tuple(getattr(result, "warnings", ()) or ())
if not codes:
return ""
return t(
"playback.depth_warnings",
detail=", ".join(t(f"playback.warn_{code}") for code in codes),
)
def _refresh_depth_pane(self) -> None:
"""Re-derive the depth pane for the cursor's frame.
Re-drawing the held map here is what used to show a stale depth as though
it were in sync with the cursor.
"""
result = self._depth_result
if result is not None and self._depth_matches_cursor(result):
self._render_depth_result(result)
return
if (
self._depth_pending_seq is not None
and self._depth_pending_position == self._position
and self._depth_busy()
):
self._render_depth_result(
DepthResult(error="recomputing", t_rel_s=self._position)
)
return
self._render_depth_result(
DepthResult(error="not_recomputed", t_rel_s=self._position)
)
@@ -118,8 +118,9 @@ class ForcePlotsPanel(QFrame):
mode_changed = pyqtSignal(str) mode_changed = pyqtSignal(str)
#: The unit chip was clicked: switch between newton and raw. #: The unit chip was clicked: switch between newton and raw.
unit_toggle_requested = pyqtSignal() unit_toggle_requested = pyqtSignal()
#: The play/pause button was pressed (a recording is on screen). #: The operator took the timeline over (dragged the cursor, box-zoomed, panned
play_toggled = pyqtSignal(bool) #: or moved the overview marker): playback stops where it is.
timeline_taken_over = pyqtSignal()
def __init__(self, parent=None, *, theme: str = "dark") -> None: def __init__(self, parent=None, *, theme: str = "dark") -> None:
super().__init__(parent) super().__init__(parent)
@@ -175,15 +176,9 @@ class ForcePlotsPanel(QFrame):
self.mode_label.setObjectName("hintText") self.mode_label.setObjectName("hintText")
header.addWidget(self.mode_label) header.addWidget(self.mode_label)
header.addStretch(1) header.addStretch(1)
# Play/pause leads the header the way a transport leads an editor: the # The transport lives on the rail (the primary button of the replay view);
# cursor is the playhead, and dragging it pauses. # the header keeps the window actions, which are symbols because the
self.btn_play = self._header_button("play") # tooltip explains each one.
self.btn_play.setCheckable(True)
self.btn_play.setEnabled(False)
self.btn_play.toggled.connect(self._on_play_toggled)
header.addWidget(self.btn_play)
# The three window actions are symbols: the tooltip explains each one, and
# the pointer stays inside the chart instead of reading three labels.
self.btn_latest = self._header_button("latest") self.btn_latest = self._header_button("latest")
self.btn_latest.clicked.connect(self.follow_latest) self.btn_latest.clicked.connect(self.follow_latest)
header.addWidget(self.btn_latest) header.addWidget(self.btn_latest)
@@ -334,31 +329,9 @@ class ForcePlotsPanel(QFrame):
button._icon_name = icon_name # noqa: SLF001 - read back by set_theme button._icon_name = icon_name # noqa: SLF001 - read back by set_theme
return button return button
def _on_play_toggled(self, playing: bool) -> None:
"""Draw the transport button for the state it is in, then tell the window."""
self._set_play_icon(bool(playing))
self.play_toggled.emit(bool(playing))
def _set_play_icon(self, playing: bool) -> None:
name = "pause" if playing else "play"
self.btn_play._icon_name = name # noqa: SLF001 - read back by set_theme
self.btn_play.setIcon(icons.icon(name, self._theme))
self.btn_play.setToolTip(t("chart.pause_tip" if playing else "chart.play_tip"))
self.btn_play.setAccessibleName(
t("chart.pause" if playing else "chart.play")
)
def set_playing(self, playing: bool) -> None:
"""Reflect playback that the window started or stopped itself."""
if self.btn_play.isChecked() != bool(playing):
self.btn_play.setChecked(bool(playing))
else:
self._set_play_icon(bool(playing))
def pause_playback(self) -> None: def pause_playback(self) -> None:
"""A manual gesture takes over the timeline: stop advancing it.""" """The operator touched the timeline: playback stops where it is."""
if self.btn_play.isChecked(): self.timeline_taken_over.emit()
self.btn_play.setChecked(False)
def _build_value_card( def _build_value_card(
self, channel: str self, channel: str
@@ -418,7 +391,7 @@ class ForcePlotsPanel(QFrame):
text = color(theme, "text_primary") text = color(theme, "text_primary")
muted = color(theme, "text_muted") muted = color(theme, "text_muted")
# Icons are painted in the theme's own colour, so they follow a switch. # Icons are painted in the theme's own colour, so they follow a switch.
for button in (self.btn_latest, self.btn_overview, self.btn_box_zoom, self.btn_play): for button in (self.btn_latest, self.btn_overview, self.btn_box_zoom):
name = getattr(button, "_icon_name", "") name = getattr(button, "_icon_name", "")
if name: if name:
button.setIcon(icons.icon(name, theme)) button.setIcon(icons.icon(name, theme))
@@ -1079,9 +1052,6 @@ class ForcePlotsPanel(QFrame):
self._mode == "browse" and not self.is_recording_source() self._mode == "browse" and not self.is_recording_source()
) )
self.btn_overview.setEnabled(self._source_ready) self.btn_overview.setEnabled(self._source_ready)
self.btn_play.setEnabled(self.is_recording_source())
if not self.is_recording_source():
self.pause_playback()
def _reload_from_store(self) -> None: def _reload_from_store(self) -> None:
"""Re-populate the visible buffer from the attached source.""" """Re-populate the visible buffer from the attached source."""
@@ -1241,7 +1211,6 @@ class ForcePlotsPanel(QFrame):
name_label.setText(t(f"chart.{key}_name")) name_label.setText(t(f"chart.{key}_name"))
self.btn_latest.setToolTip(t("chart.return_latest_tip")) self.btn_latest.setToolTip(t("chart.return_latest_tip"))
self.btn_latest.setAccessibleName(t("chart.return_latest")) self.btn_latest.setAccessibleName(t("chart.return_latest"))
self._set_play_icon(self.btn_play.isChecked())
self.btn_overview.setToolTip(t("chart.overview_tip")) self.btn_overview.setToolTip(t("chart.overview_tip"))
self.btn_overview.setAccessibleName(t("chart.overview")) self.btn_overview.setAccessibleName(t("chart.overview"))
self.btn_box_zoom.setToolTip(t("chart.box_zoom_tip")) self.btn_box_zoom.setToolTip(t("chart.box_zoom_tip"))
+3 -29
View File
@@ -14,13 +14,13 @@ from __future__ import annotations
from functools import lru_cache from functools import lru_cache
from PyQt5.QtCore import QPointF, QRectF, Qt from PyQt5.QtCore import QRectF, Qt
from PyQt5.QtGui import QColor, QIcon, QPainter, QPen, QPixmap, QPolygonF from PyQt5.QtGui import QColor, QIcon, QPainter, QPen, QPixmap
from orisys_app_shared.theme import color from orisys_app_shared.theme import color
#: Names of the icons this module can paint. #: Names of the icons this module can paint.
NAMES = ("zoom_fit", "box_zoom", "latest", "play", "pause") NAMES = ("zoom_fit", "box_zoom", "latest")
ICON_SIZE = 16 ICON_SIZE = 16
_PEN_WIDTH = 1.4 _PEN_WIDTH = 1.4
@@ -76,36 +76,10 @@ def _paint_latest(painter: QPainter, size: int) -> None:
) )
def _paint_play(painter: QPainter, size: int) -> None:
"""A filled triangle: play the recording from the cursor."""
left = size * 0.30
right = size * 0.76
painter.setBrush(painter.pen().color())
painter.drawPolygon(
QPolygonF(
[
QPointF(left, size * 0.22),
QPointF(right, size * 0.50),
QPointF(left, size * 0.78),
]
)
)
def _paint_pause(painter: QPainter, size: int) -> None:
"""Two bars: hold the recording where it is."""
painter.setBrush(painter.pen().color())
width = size * 0.16
for x in (size * 0.32, size * 0.56):
painter.drawRect(QRectF(x, size * 0.24, width, size * 0.52))
_PAINTERS = { _PAINTERS = {
"zoom_fit": _paint_zoom_fit, "zoom_fit": _paint_zoom_fit,
"box_zoom": _paint_box_zoom, "box_zoom": _paint_box_zoom,
"latest": _paint_latest, "latest": _paint_latest,
"play": _paint_play,
"pause": _paint_pause,
} }
+42 -1
View File
@@ -105,6 +105,8 @@ class ToolbarWidget(QWidget):
self._source_view = "live" self._source_view = "live"
self._source_controls_enabled = True self._source_controls_enabled = True
self._video_path = "" self._video_path = ""
#: ``acquire`` while measuring, ``playback`` while replaying a recording.
self._run_role = "acquire"
outer = QVBoxLayout(self) outer = QVBoxLayout(self)
outer.setContentsMargins(2, 2, 2, 2) outer.setContentsMargins(2, 2, 2, 2)
@@ -369,6 +371,22 @@ class ToolbarWidget(QWidget):
self._source_view = wanted self._source_view = wanted
self._apply_source_mode_ui() self._apply_source_mode_ui()
def set_run_role(self, role: str) -> None:
"""What the primary button does in this view: acquire, or play back."""
wanted = "playback" if str(role) == "playback" else "acquire"
if wanted == self._run_role:
return
self._run_role = wanted
# A view switch never carries the other view's running state across.
if self.btn_run.isChecked():
self.btn_run.blockSignals(True)
self.btn_run.setChecked(False)
self.btn_run.blockSignals(False)
self._refresh_run_label()
def run_role(self) -> str:
return self._run_role
def source_view(self) -> str: def source_view(self) -> str:
return self._source_view return self._source_view
@@ -581,6 +599,15 @@ class ToolbarWidget(QWidget):
self.btn_preview.setText(t("toolbar.preview")) self.btn_preview.setText(t("toolbar.preview"))
def _refresh_run_label(self) -> None: def _refresh_run_label(self) -> None:
if self._run_role == "playback":
# The primary action of the replay view is playing the recording —
# the same button, doing what this view is for.
self.btn_run.setText(
t("toolbar.playback_pause") if self.btn_run.isChecked() else t("toolbar.playback")
)
self._refresh_emphasis()
self._apply_run_tooltip()
return
if self.btn_run.isChecked(): if self.btn_run.isChecked():
# While connecting, cancel is the action the user actually needs — # While connecting, cancel is the action the user actually needs —
# a device open can hang, and this is the way out of it. # a device open can hang, and this is the way out of it.
@@ -608,7 +635,14 @@ class ToolbarWidget(QWidget):
# The phase counts as running even if the button has not been repainted # The phase counts as running even if the button has not been repainted
# yet: the rail must describe the session, not the widget's lag. # yet: the rail must describe the session, not the widget's lag.
running = self.btn_run.isChecked() or self._run_phase in ("connecting", "running") running = self.btn_run.isChecked() or self._run_phase in ("connecting", "running")
if recording: if self._run_role == "playback":
# The transport is the replay view's own action, not a session stop:
# filled while it plays, an outline when idle, and never red.
self.btn_record.setObjectName(
"dangerButton" if recording else "primaryButton"
)
self.btn_run.setObjectName("primaryButton" if running else "ghostButton")
elif recording:
self.btn_record.setObjectName("dangerButton") self.btn_record.setObjectName("dangerButton")
self.btn_run.setObjectName("dangerOutline") self.btn_run.setObjectName("dangerOutline")
else: else:
@@ -624,6 +658,13 @@ class ToolbarWidget(QWidget):
button.style().polish(button) button.style().polish(button)
def _apply_run_tooltip(self) -> None: def _apply_run_tooltip(self) -> None:
if self._run_role == "playback":
key = "toolbar.playback_pause_tip" if self.btn_run.isChecked() else "toolbar.playback_tip"
if not self._run_available and self._run_tooltip_reason:
self.btn_run.setToolTip(self._run_tooltip_reason)
else:
self.btn_run.setToolTip(t(key))
return
if self.btn_run.isChecked(): if self.btn_run.isChecked():
if self._run_phase == "connecting": if self._run_phase == "connecting":
self.btn_run.setToolTip(t("toolbar.cancel_tip")) self.btn_run.setToolTip(t("toolbar.cancel_tip"))
+16 -113
View File
@@ -848,109 +848,6 @@ def test_a_failed_recording_is_consumed_even_when_it_is_not_running(make_window)
_finish(window, stuck) _finish(window, stuck)
def test_the_depth_request_belongs_to_the_recording_it_started_in(make_window, record_root):
"""Opening another recording must invalidate the request still running."""
import threading as _threading
import time as _time
from PyQt5.QtWidgets import QApplication
from orisys_app_shared.record.depth import DepthResult
from orisys_studio.replay import ReplayController
window = _show(make_window(), (1200, 700))
replay = ReplayController()
replay.open_recording(Path(record_root))
first_serial = replay._record_serial
started = _threading.Event()
release = _threading.Event()
closed: list[str] = []
class _Blocking:
def preflight(self) -> str:
return ""
def depth_at(self, _position, **_kwargs):
started.set()
release.wait(10.0)
return DepthResult(
depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0, record_id="old"
)
def close(self) -> None:
closed.append("closed")
blocking = _Blocking()
replay._recomputer = blocking
replay.request_depth()
assert started.wait(5.0), "the depth request never started"
# Another recording is opened while the request is in flight.
replay.open_recording(Path(record_root))
assert replay._record_serial == first_serial + 1
assert replay._recomputer is not blocking, "the processor was not replaced"
assert closed == [], "the UI thread closed a processor a worker was reading"
release.set()
deadline = _time.monotonic() + 10.0
while not closed and _time.monotonic() < deadline:
QApplication.processEvents()
_time.sleep(0.01)
assert closed == ["closed"], "the retired processor was never closed"
assert replay._depth_thread is not None and not replay._depth_thread.is_alive()
# The answer belongs to the previous recording, so it is dropped, not drawn.
replay._poll_depth()
QApplication.processEvents()
assert replay._depth_ok is False
assert replay.depth_image() is None, "an answer for another recording was drawn"
replay.close()
window.close()
def test_a_busy_depth_processor_is_not_closed_by_the_ui_thread(make_window, record_root):
import threading as _threading
import time as _time
from PyQt5.QtWidgets import QApplication
from orisys_app_shared.record.depth import DepthResult
from orisys_studio.replay import ReplayController
window = _show(make_window(), (1200, 700))
replay = ReplayController()
replay.open_recording(Path(record_root))
started = _threading.Event()
release = _threading.Event()
closed: list[str] = []
class _Blocking:
def preflight(self) -> str:
return ""
def depth_at(self, _position, **_kwargs):
started.set()
release.wait(10.0)
return DepthResult(depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0)
def close(self) -> None:
closed.append("closed")
replay._recomputer = _Blocking()
replay.request_depth()
assert started.wait(5.0)
replay.close() # closing must not free a processor in use
QApplication.processEvents()
assert closed == [], "the controller closed a processor that was still reading"
release.set()
deadline = _time.monotonic() + 10.0
while not closed and _time.monotonic() < deadline:
QApplication.processEvents()
_time.sleep(0.01)
assert closed == ["closed"], "the owning thread never closed the processor"
window.close()
def test_a_confirmed_calibration_is_snapshotted_into_the_recording(make_window, record_root): def test_a_confirmed_calibration_is_snapshotted_into_the_recording(make_window, record_root):
"""The recording keeps the coefficients themselves, not only their hash.""" """The recording keeps the coefficients themselves, not only their hash."""
@@ -1691,8 +1588,10 @@ def test_the_replay_view_loads_a_recording_and_scrubs_it(make_window, replay_rec
assert window.arrow_panel.view_modes() == ("recorded",) assert window.arrow_panel.view_modes() == ("recorded",)
assert window.replay.video_image() is not None, "no frame at the cursor" assert window.replay.video_image() is not None, "no frame at the cursor"
assert panel.btn_play.isEnabled(), "the replay has no transport" assert window.toolbar.run_role() == "playback", "the rail lost the transport"
assert not panel.btn_play.isChecked(), "the replay starts out playing" assert window.toolbar.btn_run.isEnabled(), "the replay has no transport"
assert not window.toolbar.btn_run.isChecked(), "the replay starts out playing"
assert window.toolbar.btn_run.text() == t("toolbar.playback")
# The cursor is the playhead: setting it moves the video. # The cursor is the playhead: setting it moves the video.
window.replay.set_position(window.replay.duration_s() / 2) window.replay.set_position(window.replay.duration_s() / 2)
@@ -1701,19 +1600,21 @@ def test_the_replay_view_loads_a_recording_and_scrubs_it(make_window, replay_rec
assert abs(panel.cursors[0].value() - window.replay.position()) < 0.05 assert abs(panel.cursors[0].value() - window.replay.position()) < 0.05
# Dragging the chart cursor scrubs the recording too, and pauses it. # Dragging the chart cursor scrubs the recording too, and pauses it.
panel.btn_play.setChecked(True) window.toolbar.btn_run.setChecked(True)
QApplication.processEvents() QApplication.processEvents()
assert window.toolbar.btn_run.text() == t("toolbar.playback_pause")
panel.cursors[0].setValue(0.25) panel.cursors[0].setValue(0.25)
QApplication.processEvents() QApplication.processEvents()
assert abs(window.replay.position() - 0.25) < 0.05 assert abs(window.replay.position() - 0.25) < 0.05
assert not panel.btn_play.isChecked(), "dragging the cursor did not pause playback" assert not window.toolbar.btn_run.isChecked(), "dragging the cursor did not pause playback"
# Leaving the view returns the charts to the live session. # Leaving the view returns the charts to the live session.
window.brand_bar.btn_live.click() window.brand_bar.btn_live.click()
QApplication.processEvents() QApplication.processEvents()
assert not panel.is_recording_source() assert not panel.is_recording_source()
assert panel.mode() == "follow" assert panel.mode() == "follow"
assert not panel.btn_play.isEnabled(), "the live view still offers the transport" assert window.toolbar.run_role() == "acquire", "the rail kept the transport"
assert window.toolbar.btn_run.text() == t("toolbar.run_label")
assert window.arrow_panel.view_modes() == ("arrows_pure", "arrows_stitched", "depth") assert window.arrow_panel.view_modes() == ("arrows_pure", "arrows_stitched", "depth")
assert "raw_0001" not in window.toolbar.source_summary.text(), ( assert "raw_0001" not in window.toolbar.source_summary.text(), (
"the replay's video is still named in the live view" "the replay's video is still named in the live view"
@@ -2027,8 +1928,8 @@ def test_the_replay_transport_plays_and_pauses(make_window, replay_recording):
QApplication.processEvents() QApplication.processEvents()
panel = window.force_plots panel = window.force_plots
assert panel.btn_play.isEnabled() and not panel.btn_play.isChecked() assert window.toolbar.btn_run.isEnabled() and not window.toolbar.btn_run.isChecked()
panel.btn_play.setChecked(True) window.toolbar.btn_run.setChecked(True)
started = time.perf_counter() started = time.perf_counter()
for _ in range(10): for _ in range(10):
QApplication.processEvents() QApplication.processEvents()
@@ -2045,7 +1946,7 @@ def test_the_replay_transport_plays_and_pauses(make_window, replay_recording):
# Dragging the cursor is a manual gesture: playback pauses where it is. # Dragging the cursor is a manual gesture: playback pauses where it is.
panel.cursors[0].setValue(0.2) panel.cursors[0].setValue(0.2)
QApplication.processEvents() QApplication.processEvents()
assert not panel.btn_play.isChecked(), "dragging the cursor did not pause" assert not window.toolbar.btn_run.isChecked(), "dragging the cursor did not pause"
assert abs(window.replay.position() - 0.2) < 0.05 assert abs(window.replay.position() - 0.2) < 0.05
parked = window.replay.position() parked = window.replay.position()
for _ in range(6): for _ in range(6):
@@ -2082,13 +1983,15 @@ def test_the_recompute_mode_reruns_the_pipeline_over_the_video(
assert not panel.is_recording_source(), "the charts still read the CSV" assert not panel.is_recording_source(), "the charts still read the CSV"
assert window.toolbar.source_mode() == "video" assert window.toolbar.source_mode() == "video"
assert window.toolbar.video_path().endswith(".mp4"), window.toolbar.video_path() assert window.toolbar.video_path().endswith(".mp4"), window.toolbar.video_path()
assert panel.btn_play.isChecked(), "the transport does not show the pass as running" assert window.toolbar.btn_run.isChecked(), (
"the transport does not show the pass as running"
)
window._recompute_running = True window._recompute_running = True
window.act_recompute.setChecked(False) window.act_recompute.setChecked(False)
QApplication.processEvents() QApplication.processEvents()
assert panel.is_recording_source(), "turning the recompute off did not restore the CSV" assert panel.is_recording_source(), "turning the recompute off did not restore the CSV"
assert not panel.btn_play.isChecked(), "the transport stayed checked" assert not window.toolbar.btn_run.isChecked(), "the transport stayed checked"
# ------------------------------------------- selection readout and notices # ------------------------------------------- selection readout and notices
+55 -152
View File
@@ -2864,15 +2864,31 @@ def main() -> int:
) )
# The chart cursor is the playhead, and the transport moves it at 1x. # The chart cursor is the playhead, and the transport moves it at 1x.
check(replay_plots.btn_play.isEnabled(), "the replay has no transport") check(
check(not replay_plots.btn_play.isChecked(), "the replay starts while playing") replay_window.toolbar.run_role() == "playback",
replay_plots.btn_play.setChecked(True) "the rail's primary button is not the replay's transport",
)
check(replay_window.toolbar.btn_run.isEnabled(), "the replay has no transport")
check(
not replay_window.toolbar.btn_run.isChecked(),
"the replay starts while playing",
)
check(
replay_window.toolbar.btn_run.text() == t("toolbar.playback"),
f"the button does not say it plays: {replay_window.toolbar.btn_run.text()!r}",
)
replay_window.toolbar.btn_run.setChecked(True)
app.processEvents()
check(
replay_window.toolbar.btn_run.text() == t("toolbar.playback_pause"),
f"the button does not offer pause: {replay_window.toolbar.btn_run.text()!r}",
)
started = time.perf_counter() started = time.perf_counter()
for _ in range(12): for _ in range(12):
app.processEvents() app.processEvents()
time.sleep(0.03) time.sleep(0.03)
elapsed = time.perf_counter() - started elapsed = time.perf_counter() - started
replay_plots.btn_play.setChecked(False) replay_window.toolbar.btn_run.setChecked(False)
app.processEvents() app.processEvents()
check( check(
replay.position() > 0.05, replay.position() > 0.05,
@@ -2886,14 +2902,17 @@ def main() -> int:
abs(float(replay_plots.cursors[0].value()) - replay.position()) < 0.05, abs(float(replay_plots.cursors[0].value()) - replay.position()) < 0.05,
"the chart cursor did not follow the transport", "the chart cursor did not follow the transport",
) )
check(not replay_plots.btn_play.isChecked(), "the transport stayed checked after pause") check(
not replay_window.toolbar.btn_run.isChecked(),
"the transport stayed checked after pause",
)
# A manual gesture on the timeline takes over: playback pauses. # A manual gesture on the timeline takes over: playback pauses.
replay_plots.btn_play.setChecked(True) replay_window.toolbar.btn_run.setChecked(True)
app.processEvents() app.processEvents()
replay_plots.cursors[0].setValue(0.25) replay_plots.cursors[0].setValue(0.25)
app.processEvents() app.processEvents()
check( check(
not replay_plots.btn_play.isChecked(), not replay_window.toolbar.btn_run.isChecked(),
"dragging the cursor did not pause the playback", "dragging the cursor did not pause the playback",
) )
check( check(
@@ -2934,12 +2953,12 @@ def main() -> int:
f"the recording reports {reader.info().total_rows} rows", f"the recording reports {reader.info().total_rows} rows",
) )
check( check(
t("playback.warn_depth_recomputed").split("{")[0].strip()[:4] t("playback.warn_config_changed").split("{")[0].strip()[:4]
in replay.warnings_text(), in replay.warnings_text(),
f"the replay notes are missing: {replay.warnings_text()!r}", f"the replay notes are missing: {replay.warnings_text()!r}",
) )
check( check(
t("playback.warn_depth_recomputed").split("{")[0].strip()[:4] t("playback.warn_config_changed").split("{")[0].strip()[:4]
in replay_window.toolbar.source_summary.toolTip(), in replay_window.toolbar.source_summary.toolTip(),
f"the replay notes are not reachable from the rail: " f"the replay notes are not reachable from the rail: "
f"{replay_window.toolbar.source_summary.toolTip()!r}", f"{replay_window.toolbar.source_summary.toolTip()!r}",
@@ -2968,157 +2987,41 @@ def main() -> int:
f"the unit chip did not reach the replay: {replay.unit()}", f"the unit chip did not reach the replay: {replay.unit()}",
) )
# The depth recomputation must never run on the UI thread. A stub that blocks # Playing means decoding a frame per tick. Reopening and seeking the file for
# proves the call returns immediately instead of freezing the window. # every frame (10 ms for a small clip, hundreds for 1080p H.264) is what made
import threading as _threading # the video stand still; the reader keeps one decoder open and steps it.
import time as _time span = max(0.1, replay.duration_s() - 0.05)
seeks: list[float] = []
class _Blocking: for fraction in (0.0, 0.25, 0.5, 0.75, 1.0):
released = _threading.Event() moment = fraction * span
started_at = time.perf_counter()
def preflight(self) -> str: frame_image, _frame = replay.reader().video_frame_at(moment)
return "" seeks.append((time.perf_counter() - started_at) * 1000.0)
check(frame_image is not None, f"no frame at {moment:.2f}s")
def depth_at(self, _t):
_Blocking.released.wait(timeout=5.0)
return type("R", (), {"ok": False, "error": "stub", "depth": None})()
def close(self) -> None:
return None
replay._recomputer = _Blocking() # noqa: SLF001 - exercising the contract
started = _time.perf_counter()
replay.request_depth()
elapsed = _time.perf_counter() - started
check(elapsed < 0.5, f"the depth recompute blocked the UI for {elapsed:.2f}s")
check( check(
replay._depth_thread is not None and replay._depth_thread.is_alive(), max(seeks) < 250.0,
"the depth recompute did not run on a worker thread", f"a seek costs {max(seeks):.0f} ms, so playback would stall",
) )
steps: list[float] = []
for index in range(1, 8):
started_at = time.perf_counter()
replay.reader().video_frame_at(index / 30.0)
steps.append((time.perf_counter() - started_at) * 1000.0)
check( check(
replay._depth_thread.daemon, max(steps) < 60.0,
"the depth worker is not a daemon, so a blocked SDK call would keep the " f"stepping one frame costs {max(steps):.0f} ms, so 30 fps cannot hold",
"process alive at exit",
) )
replay.request_depth() # a second request is refused, not queued
check( # There is no depth recomputation any more: the engine was deleted with
replay.depth_message() == t("playback.depth_busy"), # the replay bar (depth is not stored with a recording, and the replay
f"a concurrent recompute was not refused: {replay.depth_message()!r}", # view shows the recorded frames). The arrow panel in the replay view
) # therefore offers the recorded image and nothing else.
_Blocking.released.set()
for _ in range(40):
app.processEvents()
_time.sleep(0.05)
if not replay._depth_busy(): # noqa: SLF001
break
check(not replay._depth_busy(), "the depth worker never finished")
# Deliver the worker result deterministically instead of waiting for the
# 120 ms poll timer.
replay._poll_depth()
app.processEvents()
check(
t("playback.depth_failed").split("{")[0].strip()[:4] in replay.depth_message(),
f"a failed recomputation was not reported: {replay.depth_message()!r}",
)
check(
replay.depth_image() is None,
"a failed recomputation left an image in the depth pane",
)
# The replay view no longer offers a depth pane: depth is not stored with a
# recording, and re-running the pipeline over the video is the recompute
# mode's job (see the `recompute` section).
check( check(
replay_window.arrow_panel.view_modes() == ("recorded",), replay_window.arrow_panel.view_modes() == ("recorded",),
f"the replay still offers a depth pane: " f"the replay still offers a depth pane: "
f"{replay_window.arrow_panel.view_modes()}", f"{replay_window.arrow_panel.view_modes()}",
) )
# The plan must state where to start, which reset it is anchored to and
# whether the state is really recoverable; and a request must belong to the
# recording it started in.
from orisys_app_shared.record.depth import (
STATE_APPROXIMATE,
STATE_PREROLL,
DepthRecomputer,
)
from orisys_app_shared.record.playback import RecordingReader
plan_reader = RecordingReader(recording)
planner = DepthRecomputer(plan_reader)
frames = plan_reader.info().frames
check(bool(frames), "the replay recording has no frame index")
plan = planner.plan_for(frames[10])
check(
plan.state == STATE_APPROXIMATE and plan.anchor == "segment_start",
f"a recording with no reset/pre-roll is not reported as approximate: {plan.state}",
)
check(
plan.steps and plan.steps[0].source == plan_reader.segment_path(frames[10].segment),
"the plan does not name the video to read",
)
check(
"state_approximate" in plan.warnings,
f"the approximate replay is not reported: {plan.warnings}",
)
# A pre-roll makes the start of a recording reproducible.
session_path = recording / "session.json"
session_doc = _json.loads(session_path.read_text(encoding="utf-8"))
prerecord = recording / "preroll_0001.mp4"
prerecord.write_bytes(b"")
session_doc["preroll_frames"] = 30
session_doc["preroll_segment"] = prerecord.name
session_path.write_text(_json.dumps(session_doc), encoding="utf-8")
planner.close()
planner = DepthRecomputer(RecordingReader(recording))
plan = planner.plan_for(planner.reader.info().frames[5])
check(
plan.state == STATE_PREROLL and plan.preroll == prerecord,
f"the pre-roll was not used for the start of the recording: {plan.state}",
)
check(plan.frames_to_read == 30 + 6, f"unexpected replay cost: {plan.frames_to_read}")
planner.close()
# A reset boundary is re-applied, and a request is bound to its recording.
_Blocking.released.clear()
second_controller = ReplayController()
second_controller.open_recording(recording)
serial_before = second_controller._record_serial
blocking = _Blocking()
second_controller._recomputer = blocking
second_controller.request_depth()
check(
second_controller._depth_pending_record == serial_before,
"the request has no recording id",
)
check(
second_controller._depth_pending_seq is not None,
"the request carries no sequence number",
)
# Opening another recording invalidates the running request...
second_controller.open_recording(recording)
check(
second_controller._record_serial == serial_before + 1,
"opening a recording did not invalidate the running request",
)
check(
second_controller._recomputer is not blocking,
"the invalidated processor was not replaced",
)
_Blocking.released.set()
for _ in range(40):
app.processEvents()
_time.sleep(0.05)
if not second_controller._depth_busy():
break
second_controller._poll_depth()
app.processEvents()
check(
second_controller.depth_image() is None,
"an answer for a previous recording was drawn into the new one",
)
second_controller.close()
app.processEvents()
# The recompute mode: off (default) the charts read the recorded CSV; on, the # The recompute mode: off (default) the charts read the recorded CSV; on, the
# recorded video is run through the pipeline instead. The run itself is not # recorded video is run through the pipeline instead. The run itself is not
# started here (that needs the SDK), only the wiring around it. # started here (that needs the SDK), only the wiring around it.
@@ -3169,8 +3072,8 @@ def main() -> int:
"leaving the replay view did not restore the live charts", "leaving the replay view did not restore the live charts",
) )
check( check(
not replay_plots.btn_play.isEnabled(), replay_window.toolbar.run_role() != "playback",
"the live view still offers the replay's transport", "the rail kept the replay's transport in the live view",
) )
check( check(
not replay_window.toolbar.source_summary.text().endswith(".mp4") not replay_window.toolbar.source_summary.text().endswith(".mp4")
@@ -4,7 +4,6 @@ from orisys_app_shared.record.calibration_snapshot import (
list_calibration_snapshots, list_calibration_snapshots,
store_calibration_snapshot, store_calibration_snapshot,
) )
from orisys_app_shared.record.depth import DepthRecomputer, DepthResult
from orisys_app_shared.record.experiment import ( from orisys_app_shared.record.experiment import (
EVENTS_HEADER, EVENTS_HEADER,
FORCES_HEADER, FORCES_HEADER,
@@ -28,8 +27,6 @@ __all__ = [
"FORCES_HEADER", "FORCES_HEADER",
"FRAMES_HEADER", "FRAMES_HEADER",
"RECORD_FORMAT_VERSION", "RECORD_FORMAT_VERSION",
"DepthRecomputer",
"DepthResult",
"ExperimentRecorder", "ExperimentRecorder",
"RecordConfig", "RecordConfig",
"RecordReport", "RecordReport",
@@ -1,548 +0,0 @@
"""Recompute depth for a recording, offline.
Depth is not stored with a recording (a per-frame depth stream would be several
GB per hour). It is a function of the frame and the sensor configuration, so a
replay re-runs the SDK over ``raw_*.mp4`` using the ``config_ref`` recorded in
``session.json``.
The honest failure modes are all surfaced rather than papered over:
* the SDK is not importable, or the recording's sensor config is gone;
* the recording predates the recorded config hash, so the recomputation is not
guaranteed to reproduce what was shown live;
* the segment cannot be decoded.
The caller is expected to show the returned ``error`` and to mark the depth pane
as unavailable instead of substituting some other image.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
import numpy as np
from orisys_app_shared.record.depth_backends import open_replay
from orisys_app_shared.record.playback import RecordingReader
logger = logging.getLogger(__name__)
@dataclass
class DepthResult:
"""Outcome of one recomputation attempt."""
depth: Optional[np.ndarray] = None
error: str = ""
frame_id: int = -1
t_rel_s: float = 0.0
#: What a replay could not reproduce exactly (never silently empty).
warnings: tuple[str, ...] = ()
#: Which recording this answer belongs to (the replay window compares it).
record_id: str = ""
@property
def ok(self) -> bool:
return self.depth is not None and not self.error
#: The replay started at a **reset**: the algorithm's state was cleared there, so
#: the frames since that point determine it exactly.
STATE_EXACT = "exact"
#: The replay is preceded by the recorded pre-roll: the intended reconstruction.
STATE_PREROLL = "preroll"
#: Neither: the state entering the target frame is unknown and may differ from
#: what the live run had.
STATE_APPROXIMATE = "approximate"
@dataclass(frozen=True)
class ReplayStep:
"""One leg of a replay: read ``source`` up to ``to_frame``.
``reset_after`` is the frame at which the algorithm's state was cleared live
(an operator ``zeroing``). The replay reaches that frame, then calls the
processor's ``reset()`` — which is what the live run did — and keeps reading.
"""
source: Path
to_frame: int
reset_after: Optional[int] = None
#: How many SDK reads a faithful replay may cost before it is declared
#: unrecoverable (5 minutes at 30 fps). A replay that re-reads an hour of video
#: to answer one cursor move is not a feature; saying so is.
MAX_REPLAY_FRAMES = 9000
@dataclass(frozen=True)
class ReplayPlan:
"""Where the SDK must start reading to reproduce one frame's state."""
segment: Path
#: The legs of the replay, in order. The last one ends at ``stop_at``.
steps: tuple[ReplayStep, ...] = ()
#: Frame within the final segment the caller asked for (1-based).
stop_at: int = 0
#: Frames to feed from another file before the first step.
preroll: Path | None = None
preroll_frames: int = 0
#: The ``zeroing`` frame the plan is anchored to, when there is one.
reset_frame_id: Optional[int] = None
#: ``reset`` / ``preroll`` / ``segment_start``.
anchor: str = "segment_start"
#: :data:`STATE_EXACT`, :data:`STATE_PREROLL` or :data:`STATE_APPROXIMATE`.
state: str = STATE_APPROXIMATE
warnings: tuple[str, ...] = ()
error: str = ""
@property
def frames_to_read(self) -> int:
"""How many SDK reads a faithful recomputation costs."""
return int(self.preroll_frames) + sum(
max(0, int(step.to_frame)) for step in self.steps
)
class DepthRecomputer:
"""Recompute depth maps for one recording, on demand."""
def __init__(self, reader: RecordingReader) -> None:
self.reader = reader
self._sensor = None
self._segment: Optional[Path] = None
#: How many frames the open sensor has already consumed in ``_segment``.
self._position = 0
#: The depth of ``_position``, so asking for the same frame twice is a
#: cache hit instead of a second (impossible) read.
self._last_depth: Optional[np.ndarray] = None
#: True when an earlier frame had to be re-read without a reset boundary
#: to clean up after it, so the state may not match the live run.
self._rewound_without_anchor = False
self.last_error = ""
self.last_warnings: list[str] = []
# ------------------------------------------------------------------ probing
@staticmethod
def sdk_available() -> tuple[bool, str]:
try:
import orisys # noqa: F401
return True, str(getattr(__import__("orisys"), "__version__", "unknown"))
except Exception as exc: # noqa: BLE001
return False, f"{type(exc).__name__}: {exc}"
def config_ref(self) -> str:
session = self.reader.session()
device = session.get("device") or {}
return str(device.get("config_ref") or "")
def resolved_config(self) -> str:
"""The exact configuration file to open, resolved as the live run did.
Reducing the path to a directory name or a stem could silently load a
*different* file with the same name while the hash check still compared
the recorded path — so the full resolved path is what gets opened.
"""
config_ref = self.config_ref()
try:
from orisys_app_shared.acquisition.sdk_worker import load_fixed_mode
return str(load_fixed_mode(config_ref).config_ref)
except Exception: # noqa: BLE001 - reported by the caller as a failure
return config_ref
def preflight(self) -> str:
"""Return an empty string when recomputation can be attempted."""
ok, detail = self.sdk_available()
if not ok:
return f"sdk_unavailable: {detail}"
config_ref = self.config_ref()
if not config_ref:
return "config_missing"
if not Path(config_ref).is_file():
return "config_file_missing"
if not self.reader.segments():
return "no_video"
return ""
# --------------------------------------------------------------- replay plan
def plan_for(self, frame) -> ReplayPlan:
"""Decide where replay must start so ``frame`` is reproducible.
The SDK's depth is the output of a stateful chain (optical-flow tracker,
temporal filters), so a frame cannot be recomputed in isolation. What the
plan states explicitly:
* **where** to start (``start_at``, plus a pre-roll file when there is one);
* which **reset boundary** the replay is anchored to, if any;
* whether the state is actually recoverable — :data:`STATE_EXACT` (the
replay starts at a reset), :data:`STATE_PREROLL` (the recorded pre-roll
precedes it) or :data:`STATE_APPROXIMATE` (neither: the depth map is
still produced, and the caller is told it may differ from live).
Finding a reset event is **not** by itself a recovered state: the reset
frame has to exist in the index *and* in the video. Every intervening
segment must be read with the same processor to preserve its state.
"""
warnings: list[str] = []
segment_path = self.reader.segment_path(frame.segment)
if segment_path is None:
return ReplayPlan(segment=Path(), error="segment_missing")
stop_at = max(1, int(frame.segment_frame))
reset_frame = self.reader.last_reset_frame(frame.frame_id)
if reset_frame is not None:
origin = self.reader.frame_of(reset_frame)
if origin is None:
warnings.append("reset_frame_not_indexed")
elif origin.dropped or not origin.in_video:
# The boundary frame is not in the video, so the state at that
# point cannot be rebuilt: replay from here is an approximation.
warnings.append("reset_frame_not_in_video")
elif any(
entry.dropped or not entry.in_video
for entry in self.reader.info().frames
if origin.frame_id <= entry.frame_id <= frame.frame_id
):
warnings.append("state_missing_frames")
elif origin.segment == frame.segment:
plan = ReplayPlan(
segment=segment_path,
steps=(
ReplayStep(
source=segment_path,
to_frame=stop_at,
reset_after=int(origin.segment_frame),
),
),
stop_at=stop_at,
reset_frame_id=int(reset_frame),
anchor="reset",
state=STATE_EXACT,
)
if plan.frames_to_read > MAX_REPLAY_FRAMES:
warnings.append("state_too_far_back")
else:
return plan
else:
# The reset is in an earlier segment. The processor survives a
# source swap, so the state *can* be carried: read the tail of
# the reset's segment, then continue in the target's.
steps = []
for segment in range(int(origin.segment), int(frame.segment) + 1):
source = self.reader.segment_path(segment)
last = stop_at if segment == frame.segment else self._last_segment_frame(segment)
if source is None or not source.is_file() or last is None:
steps = []
break
steps.append(ReplayStep(
source=source,
to_frame=int(last),
reset_after=int(origin.segment_frame) if segment == origin.segment else None,
))
if not steps:
warnings.append("state_across_segment")
else:
plan = ReplayPlan(
segment=segment_path,
steps=tuple(steps),
stop_at=stop_at,
reset_frame_id=int(reset_frame),
anchor="reset",
state=STATE_EXACT,
)
if plan.frames_to_read <= MAX_REPLAY_FRAMES:
return plan
# Too expensive: report it rather than freeze the window.
warnings.append("state_too_far_back")
start = self.reader.segment_start_frame(frame.segment)
if start is None:
return ReplayPlan(segment=Path(), error="frame_not_indexed")
preroll = self.reader.preroll_segment()
preroll_frames = self.reader.preroll_frames() if preroll is not None else 0
if preroll is not None and preroll_frames > 0 and frame.segment == 1:
# Recording started mid-stream: the frames before the first recorded
# one are the only history that can rebuild the tracker.
return ReplayPlan(
segment=segment_path,
steps=(ReplayStep(source=segment_path, to_frame=stop_at),),
stop_at=stop_at,
preroll=preroll,
preroll_frames=int(preroll_frames),
anchor="preroll",
state=STATE_PREROLL,
warnings=tuple(warnings),
)
# No reset and no pre-roll: the state entering this frame is unknown.
warnings.append("state_approximate")
return ReplayPlan(
segment=segment_path,
steps=(
ReplayStep(
source=segment_path,
to_frame=stop_at,
),
),
stop_at=stop_at,
anchor="segment_start",
state=STATE_APPROXIMATE,
warnings=tuple(dict.fromkeys(warnings)),
)
def _last_segment_frame(self, segment: int) -> Optional[int]:
"""The highest indexed frame number inside ``segment``."""
frames = [
int(entry.segment_frame)
for entry in self.reader.info().frames
if int(entry.segment) == int(segment)
]
return max(frames) if frames else None
# ----------------------------------------------------------------- recompute
def depth_at(self, t_rel_s: float) -> DepthResult:
"""Recompute the depth map for the frame at ``t_rel_s``."""
problem = self.preflight()
if problem:
self.last_error = problem
return DepthResult(error=problem, t_rel_s=float(t_rel_s))
frame = self.reader.frame_at(t_rel_s)
if frame is None:
return DepthResult(error="no_frame", t_rel_s=float(t_rel_s))
if frame.dropped:
# The row was lost to a full queue; the index row marks the hole.
return DepthResult(
error="frame_not_recorded",
frame_id=frame.frame_id,
t_rel_s=frame.t_rel_s,
)
if not frame.in_video:
# The queue dropped this frame; say so rather than show the previous
# frame as if it belonged to the cursor position.
return DepthResult(
error="frame_not_in_video", frame_id=frame.frame_id, t_rel_s=frame.t_rel_s
)
plan = self.plan_for(frame)
if plan.error:
self.last_error = plan.error
return DepthResult(
error=plan.error, frame_id=frame.frame_id, t_rel_s=frame.t_rel_s
)
try:
depth = self._replay(plan)
except Exception as exc: # noqa: BLE001 - reported to the caller
logger.exception("Depth recomputation failed")
self.last_error = f"{type(exc).__name__}: {exc}"
return DepthResult(
error=self.last_error,
frame_id=frame.frame_id,
t_rel_s=frame.t_rel_s,
warnings=plan.warnings,
record_id=self.record_id(),
)
self.last_warnings = list(plan.warnings)
if self._rewound_without_anchor and "state_approximate" not in self.last_warnings:
# The plan looked recoverable, but the read path had to restart a file
# without a reset boundary to clear the tracker afterwards.
self.last_warnings.append("state_approximate_after_rewind")
warnings = tuple(dict.fromkeys(self.last_warnings))
if depth is None:
self.last_error = "depth_not_available"
return DepthResult(
error=self.last_error,
frame_id=frame.frame_id,
t_rel_s=frame.t_rel_s,
warnings=warnings,
record_id=self.record_id(),
)
self.last_error = ""
return DepthResult(
depth=depth,
frame_id=frame.frame_id,
t_rel_s=frame.t_rel_s,
warnings=warnings,
record_id=self.record_id(),
)
def record_id(self) -> str:
"""Which recording these answers belong to."""
return str(getattr(self.reader, "directory", "") or "")
# ------------------------------------------------------------------ playback
def _replay(self, plan: ReplayPlan) -> Optional[np.ndarray]:
"""Execute the plan's steps, keeping the processor and swapping inputs.
The **processor is kept** across a source change: the SDK's tracker lives
on the object, so disposing of it would dispose of the state the pre-roll
(or the reset boundary) just rebuilt. A reset boundary is re-applied by
calling ``reset()`` where the operator's ``zeroing`` happened.
"""
self._rewound_without_anchor = False
if plan.preroll is not None and plan.preroll_frames > 0:
self._feed(plan.preroll, int(plan.preroll_frames))
depth: Optional[np.ndarray] = None
for step in plan.steps:
depth = self._run_step(plan, step)
return depth
def _run_step(self, plan: ReplayPlan, step: ReplayStep) -> Optional[np.ndarray]:
target = max(1, int(step.to_frame))
if self._sensor is None or self._segment != step.source:
if self._sensor is None or not self._swap_source(step.source):
self._open_segment(step.source)
elif self._position > target:
# Re-reading an earlier frame can only be done from the start of the
# file. Without a reset boundary to clean up after it, the tracker has
# now seen frames it should not have.
if not self._swap_source(step.source):
self._open_segment(step.source)
if plan.anchor != "reset":
self._rewound_without_anchor = True
last: Optional[np.ndarray] = (
self._last_depth if self._position == target else None
)
while self._position < target:
depth = self._read_one()
if depth is None:
raise RuntimeError(f"replay source ended before frame {target}: {step.source}")
last = depth
if step.reset_after is not None and self._position == int(step.reset_after):
# The operator's zeroing: the live run cleared the tracker here.
if not self._reset_processor():
self.last_warnings.append("reset_not_applied")
return last
def _reset_processor(self) -> bool:
replay = self._sensor
reset = getattr(replay, "reset", None)
if not callable(reset):
return False
try:
return bool(reset())
except Exception: # noqa: BLE001 - reported as a replay warning
logger.exception("Could not re-apply the reset boundary")
return False
def _depth_from_segment(
self, segment: Path, segment_frame: int
) -> Optional[np.ndarray]:
"""Recompute the map for ``segment_frame`` of ``segment``.
The SDK's video source only advances, so stepping costs one read per
frame — and the position is tracked, so asking for frame 2 and then
frame 3 reads two frames and then one, instead of two more. Asking twice
for the *same* frame is answered from the cached depth of that frame.
"""
target = max(1, int(segment_frame))
if self._segment != segment:
self._open_segment(segment)
return self._run_step(
ReplayPlan(segment=Path(segment), stop_at=target),
ReplayStep(source=Path(segment), to_frame=target),
)
# --------------------------------------------------------------------- video
def sensor_type(self) -> str:
device = self.reader.session().get("device") or {}
return str(device.get("sensor_type") or "box")
def _open_segment(self, segment: Path) -> None:
"""Open a segment on a **new** processor (the previous one is closed)."""
self._close()
self._sensor = open_replay(
self.sensor_type(),
Path(segment),
self.resolved_config(),
self._finger_export_dir(),
)
self._segment = Path(segment)
self._position = 0
self._last_depth = None
def _swap_source(self, segment: Path) -> bool:
"""Replace only the video source of the open processor.
Returns False when the backend cannot do it, in which case the caller
falls back to a full reopen (and the algorithm state is lost, which the
plan already reports as approximate).
"""
replay = self._sensor
if replay is None:
return False
try:
if not replay.set_source(Path(segment)):
return False
except Exception: # noqa: BLE001 - reported by the plan's warnings
logger.exception("Could not swap the replayed video source")
return False
self._segment = Path(segment)
self._position = 0
self._last_depth = None
return True
def _finger_export_dir(self):
"""The finger pipeline assets, resolved exactly as the live run did."""
try:
from orisys_app_shared.acquisition.finger_worker import (
resolve_finger_export_dir,
)
return resolve_finger_export_dir(self.resolved_config())
except Exception: # noqa: BLE001 - the SDK default is used instead
logger.exception("Could not resolve the finger export directory")
return None
def _feed(self, segment: Path, frames: int) -> None:
"""Consume ``frames`` frames from another file to rebuild state."""
if frames <= 0:
return
if self._sensor is None:
self._open_segment(segment)
elif self._segment != segment:
if not self._swap_source(segment):
self._open_segment(segment)
for _ in range(int(frames)):
if self._read_one() is None:
break
def _read_one(self) -> Optional[np.ndarray]:
"""Read one frame through the **live** processing chain (see ``process_chain``)."""
replay = self._sensor
if replay is None:
return None
depth = replay.read()
if depth is None:
return None
self._position += 1
self._last_depth = depth
return depth
def _close(self) -> None:
replay, self._sensor = self._sensor, None
self._segment = None
self._position = 0
self._last_depth = None
if replay is None:
return
try:
replay.close()
except Exception: # noqa: BLE001
pass
def close(self) -> None:
self._close()
def __enter__(self) -> "DepthRecomputer":
return self
def __exit__(self, *_exc) -> None:
self.close()
@@ -1,194 +0,0 @@
"""Replay backends: the live processing chain, pointed at a video file.
Each backend keeps **one** processor for its lifetime. The tracker state lives on
that object, so a replay must be able to change the *input source* without
throwing the processor away — that is what makes a pre-roll, or a zeroing that
happened in an earlier segment, reproducible at all.
The processing itself is not duplicated here: the reads go through
``acquisition.sdk_worker.process_chain``, the same function the live workers run.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Optional
import numpy as np
logger = logging.getLogger(__name__)
class BoxReplay:
"""The box sensor's live chain, opened on one video file."""
def __init__(self, segment: Path, config_ref: str) -> None:
import orisys
self._config_ref = config_ref
self._sensor = orisys.Sensor(
str(segment),
config_name=config_ref,
verbose=False,
backend="auto",
)
def set_source(self, segment: Path) -> bool:
"""Point the same processor at another file.
The tracker lives on the ``Sensor`` object, so re-opening the sensor would
throw away the state the pre-roll just rebuilt. Only the capture is
replaced, and the tracker's reference frame is preserved.
"""
sensor = self._sensor
open_source = getattr(sensor, "_open_video_source", None)
if not callable(open_source):
return False
initial = getattr(sensor, "initial_img", None)
try:
import orisys
sensor.vid_src = orisys.capture.normalize_source(str(segment))
except Exception: # noqa: BLE001 - fall back to assigning the path
sensor.vid_src = str(segment)
sensor._is_live_camera = False
open_source(False, None)
if initial is not None:
# The tracker's reference frame belongs to the session, not to the
# file: replacing it would silently reset what "no contact" means.
sensor.initial_img = initial
return True
def read(self) -> Optional[np.ndarray]:
from orisys_app_shared.acquisition.sdk_worker import process_chain
image = self._sensor.get_img()
if image is None:
return None
# The same function the acquisition loop runs.
depth, error = process_chain(self._sensor)
if error:
logger.warning("Replay processing chain reported: %s", error)
return depth
def reset(self) -> bool:
"""Clear the tracker, exactly as the operator's zeroing did live."""
reset = getattr(self._sensor, "reset", None)
if not callable(reset):
return False
reset()
return True
def close(self) -> None:
self._sensor.disconnect()
class FingerReplay:
"""The finger's live chain: unroll the source frame, then process the tactile."""
def __init__(self, segment: Path, config_ref: str, export_dir) -> None:
from orisys.finger import FingerRuntime
from orisys_app_shared.acquisition.constants import (
FORCE_SAMPLE_RATE_HZ,
LOWPASS_CUTOFF_HZ,
)
self._config_ref = config_ref
self._export_dir = export_dir
kwargs: dict = {
"config": config_ref,
"video": str(segment),
"verbose": False,
}
if export_dir is not None:
kwargs["export_dir"] = str(export_dir)
self._runtime = FingerRuntime.open(**kwargs)
self._runtime.warmup(color=False)
tactile = getattr(self._runtime, "tactile", None)
if tactile is not None and hasattr(tactile, "configure"):
try:
# The same filter settings the acquisition loop applies.
tactile.configure(
lowpass_cutoff_hz=LOWPASS_CUTOFF_HZ,
lowpass_sample_rate_hz=FORCE_SAMPLE_RATE_HZ,
)
except Exception: # noqa: BLE001 - the defaults still work
logger.exception("Could not configure the replayed tactile sensor")
def set_source(self, segment: Path) -> bool:
"""Point the same runtime at another video, keeping the tactile pipeline."""
try:
from orisys.capture import (
CaptureConfig,
ReconnectPolicy,
is_live_camera,
normalize_source,
open_capture,
)
runtime = self._runtime
video = normalize_source(str(segment))
camera = dict(getattr(runtime.geometry, "config", {}).get("camera", {}))
capture_config = CaptureConfig.from_camera_dict(
video,
camera,
reconnect=ReconnectPolicy(enabled=False),
verbose=False,
background=False,
)
capture_source = open_capture(capture_config, start_background=False)
if not capture_source.is_open():
return False
if is_live_camera(video):
capture_source.start_background()
previous = runtime.capture_source
runtime.capture_source = capture_source
runtime.capture = capture_source.cap
runtime.capture_worker = capture_source
if previous is not None:
try:
previous.close()
except Exception: # noqa: BLE001
logger.exception("Could not close the previous finger source")
return True
except Exception: # noqa: BLE001 - reported as an approximate replay
logger.exception("Could not swap the finger video source")
return False
def read(self) -> Optional[np.ndarray]:
from orisys_app_shared.acquisition.sdk_worker import process_chain
ok, source_frame = self._runtime.read_source()
if not ok or source_frame is None:
return None
unrolled = self._runtime.process_frame(source_frame, color=False)
if unrolled is None:
return None
depth, error = process_chain(self._runtime.tactile)
if error:
logger.warning("Replay processing chain reported: %s", error)
return depth
def reset(self) -> bool:
"""Clear the tactile tracker, as the operator's zeroing did live."""
tactile = getattr(self._runtime, "tactile", None)
target = tactile if callable(getattr(tactile, "reset", None)) else None
if target is None:
sensor = getattr(tactile, "raw_sensor", None)
target = sensor if callable(getattr(sensor, "reset", None)) else None
if target is None:
return False
target.reset()
return True
def close(self) -> None:
self._runtime.close()
def open_replay(sensor_type: str, segment: Path, config_ref: str, export_dir=None):
"""The backend for a recorded sensor type."""
if str(sensor_type) == "finger":
return FingerReplay(segment, config_ref, export_dir)
return BoxReplay(segment, config_ref)
+66 -13
View File
@@ -15,6 +15,7 @@ from __future__ import annotations
import csv import csv
import json import json
import logging import logging
import threading
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Iterator, Optional from typing import Iterator, Optional
@@ -144,6 +145,14 @@ class RecordingReader:
self.max_rows = int(max_rows or self.DEFAULT_MAX_ROWS) self.max_rows = int(max_rows or self.DEFAULT_MAX_ROWS)
self._index = RecordingIndex(self.directory, checkpoint_rows=checkpoint_rows) self._index = RecordingIndex(self.directory, checkpoint_rows=checkpoint_rows)
self._overviews: dict[tuple[str, int], Overview] = {} self._overviews: dict[tuple[str, int], Overview] = {}
#: One open decoder per segment, kept between calls. Opening the file and
#: seeking costs 10 ms for a small clip and hundreds for 1080p H.264, and
#: playback asks for a frame 30 times a second: reopening per frame is
#: what made the video stand still.
self._capture = None
self._capture_segment: int | None = None
self._capture_index = -1
self._capture_lock = threading.Lock()
# ------------------------------------------------------------------ loading # ------------------------------------------------------------------ loading
def info(self) -> RecordingInfo: def info(self) -> RecordingInfo:
@@ -348,24 +357,70 @@ class RecordingReader:
Returns ``(frame_bgr, RecordingFrame)`` or ``(None, frame)`` when the Returns ``(frame_bgr, RecordingFrame)`` or ``(None, frame)`` when the
frame was not written to video (the queue dropped it). frame was not written to video (the queue dropped it).
The decoder stays open and is *stepped* when the requested frame follows
the previous one, which is what playback does: a seek costs a keyframe
decode, a step costs one frame. Seeking is kept for jumps (dragging the
cursor).
""" """
import cv2 import cv2
frame = self.frame_at(t_rel_s) frame = self.frame_at(t_rel_s)
if frame is None or not frame.in_video: if frame is None or not frame.in_video:
return None, frame return None, frame
path = self.segment_path(frame.segment) target = max(0, int(frame.segment_frame) - 1)
if path is None: with self._capture_lock:
return None, frame capture = self._capture_for(frame.segment)
capture = cv2.VideoCapture(str(path)) if capture is None:
if not capture.isOpened():
return None, frame return None, frame
try: try:
capture.set(cv2.CAP_PROP_POS_FRAMES, max(0, frame.segment_frame - 1)) if self._capture_index >= 0 and target == self._capture_index + 1:
ok, image = capture.read() ok, image = capture.read()
else:
capture.set(cv2.CAP_PROP_POS_FRAMES, target)
ok, image = capture.read()
except Exception: # noqa: BLE001 - a broken decode is not a crash
logger.exception("Could not decode frame %s", target)
self._release_capture()
return None, frame
# A failed read leaves the decoder's position unknown: start over next
# time rather than stepping from a frame that was never delivered.
self._capture_index = target if ok else -1
return (image if ok else None), frame return (image if ok else None), frame
finally:
def _capture_for(self, segment: int):
"""The open decoder for ``segment``, opening (or switching) as needed."""
import cv2
if self._capture is not None and self._capture_segment == int(segment):
return self._capture
self._release_capture()
path = self.segment_path(int(segment))
if path is None:
return None
capture = cv2.VideoCapture(str(path))
if not capture.isOpened():
capture.release() capture.release()
return None
self._capture = capture
self._capture_segment = int(segment)
self._capture_index = -1
return capture
def _release_capture(self) -> None:
if self._capture is not None:
try:
self._capture.release()
except Exception: # noqa: BLE001 - releasing is best effort
logger.debug("Could not release the replay decoder", exc_info=True)
self._capture = None
self._capture_segment = None
self._capture_index = -1
def close(self) -> None:
"""Release the decoder. The reader is reusable afterwards."""
with self._capture_lock:
self._release_capture()
def forces_window(self, t0: float, t1: float) -> tuple[np.ndarray, np.ndarray]: def forces_window(self, t0: float, t1: float) -> tuple[np.ndarray, np.ndarray]:
"""``(times, values)`` inside ``[t0, t1]`` for the raw triplet.""" """``(times, values)`` inside ``[t0, t1]`` for the raw triplet."""
@@ -445,16 +500,16 @@ class RecordingReader:
def replay_warnings(self, *, current_config_sha: str = "", sdk_version: str = "") -> list[str]: def replay_warnings(self, *, current_config_sha: str = "", sdk_version: str = "") -> list[str]:
"""What could stop this recording from replaying faithfully. """What could stop this recording from replaying faithfully.
Depth is recomputed rather than stored, so a changed config or SDK means A changed config or SDK means the recorded frames are read by a different
the depth seen on replay may differ from what was shown live. That must pipeline than the one that produced them; that must be stated, not
be stated, not discovered later. discovered later.
""" """
session = self.info().session session = self.info().session
warnings: list[str] = [] warnings: list[str] = []
recorded_sha = str((session.get("device") or {}).get("config_sha256") or "") recorded_sha = str((session.get("device") or {}).get("config_sha256") or "")
if not current_config_sha: if not current_config_sha:
# Compare against the config file as it is now, which is exactly what # Compare against the config file as it is now, which is exactly what
# the recomputation would use. # a recompute over the video would use.
current_config_sha = self.config_sha256(self.recorded_config_ref()) current_config_sha = self.config_sha256(self.recorded_config_ref())
if recorded_sha and current_config_sha and recorded_sha != current_config_sha: if recorded_sha and current_config_sha and recorded_sha != current_config_sha:
warnings.append("config_changed") warnings.append("config_changed")
@@ -464,8 +519,6 @@ class RecordingReader:
warnings.append("sdk_changed") warnings.append("sdk_changed")
if recorded_sha and not current_config_sha: if recorded_sha and not current_config_sha:
warnings.append("config_missing") warnings.append("config_missing")
if not self.depth_is_stored():
warnings.append("depth_recomputed")
if self.info().dropped_rows: if self.info().dropped_rows:
warnings.append("rows_skipped") warnings.append("rows_skipped")
if self.info().frames_dropped: if self.info().frames_dropped:
-424
View File
@@ -1,424 +0,0 @@
"""Offline depth replay: where to start, how far to step, which chain to run.
P1-3 and P1-4: the pane kept showing the previous frame's depth after the cursor
moved, the recomputer re-read ``target`` frames on every call (so requesting
frames 2 and 3 landed on 5), it skipped the live processing chain, and it
reduced the recorded config path to a name that could load a different file.
"""
from __future__ import annotations
import shutil
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
import pytest
from orisys_app_shared.record.depth import (
STATE_APPROXIMATE,
STATE_EXACT,
STATE_PREROLL,
DepthRecomputer,
)
SCRATCH = Path(__file__).resolve().parent / "_scratch"
def _root() -> Path:
shutil.rmtree(SCRATCH, ignore_errors=True)
SCRATCH.mkdir(parents=True, exist_ok=True)
return SCRATCH
class _CountingSensor:
"""A sensor whose reported depth is the number of frames it has read."""
opened: list[dict] = []
def __init__(self, *args, **kwargs):
_CountingSensor.opened.append({"args": args, "kwargs": kwargs})
self.n = 0
self.chain_calls = 0
self.initial_img = "reference-frame"
# The real Sensor opens its source in the constructor, so the first
# source is part of its history too.
self.vid_src = str(args[0]) if args else ""
self.sources: list[str] = [self.vid_src] if self.vid_src else []
self.resets = 0
self.info = SimpleNamespace(DEPTH=1)
def get_img(self):
self.n += 1
return np.ones((2, 2))
def compute_deformation(self, **_kwargs):
self.chain_calls += 1
def compute_contact(self):
self.chain_calls += 1
def reset(self):
self.resets += 1
def read_info(self, _channel):
return np.array([self.n])
def _open_video_source(self, _override, _box):
self.sources.append(str(self.vid_src))
# A real reopen restarts the file from its first frame.
self.n = 0
def disconnect(self):
return None
def _reader(session: dict, *, segments=("raw_0001.mp4",), frames=()):
directory = _root()
for name in segments:
(directory / name).write_bytes(b"")
return SimpleNamespace(
session=lambda: session,
segment_path=lambda segment: directory / f"raw_{segment:04d}.mp4",
segments=lambda: [directory / name for name in segments],
info=lambda: SimpleNamespace(frames=list(frames)),
frame_of=lambda frame_id: next(
(frame for frame in frames if frame.frame_id == frame_id), None
),
frame_at=lambda t: None,
last_reset_frame=lambda frame_id: None,
segment_start_frame=lambda segment: next(
(frame for frame in frames if frame.segment == segment), None
),
preroll_frames=lambda: 0,
preroll_segment=lambda: None,
directory=directory,
)
def _session(config_ref: str) -> dict:
return {"device": {"config_ref": config_ref, "sensor_type": "box"}}
def _config_file() -> str:
"""A config that exists, so ``preflight`` is not what a test measures."""
path = _root() / "sensor.json"
path.write_text("{}", encoding="utf-8")
return str(path)
def _reader_with_config(frames):
"""A reader whose recorded config really exists on disk.
``_reader`` recreates the scratch directory, so the config file is written
*after* the reader is built.
"""
directory = _root()
config = directory / "sensor.json"
reader = _reader(_session(str(config)), frames=frames)
config.write_text("{}", encoding="utf-8")
return reader
def test_stepping_continues_from_the_current_position():
"""The old code re-read ``target`` frames every call: 2 then 3 landed on 5."""
_CountingSensor.opened.clear()
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
with patch("orisys.Sensor", _CountingSensor):
first = computer._depth_from_segment(Path("raw_0001.mp4"), 2)
second = computer._depth_from_segment(Path("raw_0001.mp4"), 3)
assert first.tolist() == [2]
assert second.tolist() == [3]
# One sensor for the whole replay: reopening would reset the algorithm state.
assert len(_CountingSensor.opened) == 1
computer.close()
def test_stepping_backwards_reopens_the_source_but_keeps_the_processor():
_CountingSensor.opened.clear()
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
with patch("orisys.Sensor", _CountingSensor):
assert computer._depth_from_segment(Path("raw_0001.mp4"), 5).tolist() == [5]
# Reading an earlier frame restarts the file; the processor object (and
# therefore the tracker) is not thrown away.
assert computer._depth_from_segment(Path("raw_0001.mp4"), 2).tolist() == [2]
sensor = computer._sensor._sensor
assert len(_CountingSensor.opened) == 1, "the processor was recreated"
assert len(sensor.sources) == 2
assert all(source.endswith("raw_0001.mp4") for source in sensor.sources)
computer.close()
def test_the_live_processing_chain_runs_for_every_replayed_frame():
"""Without it the replay is a different algorithm than the recording."""
_CountingSensor.opened.clear()
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
with patch("orisys.Sensor", _CountingSensor):
computer._depth_from_segment(Path("raw_0001.mp4"), 3)
sensor = computer._sensor._sensor
assert sensor.chain_calls == 6, "one deformation + one contact per frame"
computer.close()
def test_the_exact_config_path_is_opened(tmp_path=None):
"""A reduced name could load a different file with the same name."""
_CountingSensor.opened.clear()
config = Path("D:/custom/box/sensor.json")
computer = DepthRecomputer(_reader(_session(str(config))))
with patch("orisys.Sensor", _CountingSensor):
computer._depth_from_segment(Path("raw_0001.mp4"), 1)
assert _CountingSensor.opened[0]["kwargs"]["config_name"] == str(config)
computer.close()
def test_the_plan_starts_at_the_last_reset_in_the_same_segment():
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1,
segment_frame=index + 1,
in_video=True,
dropped=False,
)
for index in range(10)
]
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
reader.last_reset_frame = lambda frame_id: 6
computer = DepthRecomputer(reader)
plan = computer.plan_for(frames[8])
assert plan.stop_at == 9
assert plan.anchor == "reset"
assert plan.state == STATE_EXACT
assert plan.reset_frame_id == 6
assert len(plan.steps) == 1
step = plan.steps[0]
assert step.source == reader.directory / "raw_0001.mp4"
assert step.to_frame == 9
assert step.reset_after == 7, "the reset boundary is not re-applied by the plan"
assert plan.frames_to_read == 9
assert plan.preroll is None
assert plan.warnings == ()
computer.close()
def test_a_reset_that_is_not_in_the_video_is_not_a_recovered_state():
"""Finding the event is not the same as being able to start there."""
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1,
segment_frame=index + 1,
in_video=index != 6, # the reset frame never reached the video
dropped=False,
)
for index in range(10)
]
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
reader.last_reset_frame = lambda frame_id: 6
computer = DepthRecomputer(reader)
plan = computer.plan_for(frames[8])
assert plan.anchor != "reset"
assert plan.state == STATE_APPROXIMATE
assert "reset_frame_not_in_video" in plan.warnings
assert "state_approximate" in plan.warnings
computer.close()
def test_the_plan_uses_the_pre_roll_for_the_start_of_a_recording():
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1,
segment_frame=index + 1,
in_video=True,
dropped=False,
)
for index in range(5)
]
def reader_factory():
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
reader.preroll_frames = lambda: 30
reader.preroll_segment = lambda: reader.directory / "preroll_0001.mp4"
return reader
reader = reader_factory()
(reader.directory / "preroll_0001.mp4").write_bytes(b"")
computer = DepthRecomputer(reader)
plan = computer.plan_for(frames[3])
assert plan.preroll is not None
assert plan.preroll_frames == 30
assert plan.anchor == "preroll"
assert plan.state == STATE_PREROLL
assert plan.warnings == ()
computer.close()
def test_a_segment_without_history_is_flagged_approximate():
"""Starting mid-recording with no reset and no pre-roll cannot be exact."""
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=2,
segment_frame=index + 1,
in_video=True,
dropped=False,
)
for index in range(5)
]
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
computer = DepthRecomputer(reader)
plan = computer.plan_for(frames[2])
assert plan.state == STATE_APPROXIMATE
assert plan.anchor == "segment_start"
assert "state_approximate" in plan.warnings
computer.close()
def test_a_reset_in_an_earlier_segment_is_carried_across_by_swapping_the_source():
"""The processor survives the swap, so the state *can* cross a segment."""
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1 if index < 3 else 2,
segment_frame=index + 1 if index < 3 else index - 2,
in_video=True,
dropped=False,
)
for index in range(6)
]
reader = _reader(
_session("/cfg/sensor.json"),
segments=("raw_0001.mp4", "raw_0002.mp4"),
frames=frames,
)
reader.last_reset_frame = lambda frame_id: 1
computer = DepthRecomputer(reader)
plan = computer.plan_for(frames[5])
assert plan.anchor == "reset"
assert plan.state == STATE_EXACT
assert "state_across_segment" not in plan.warnings
assert [step.source.name for step in plan.steps] == ["raw_0001.mp4", "raw_0002.mp4"]
assert plan.steps[0].reset_after == 2
assert plan.steps[1].to_frame == 3
computer.close()
def test_a_replay_that_would_read_too_far_back_is_reported_not_attempted():
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1,
segment_frame=index + 1,
in_video=True,
dropped=False,
)
for index in range(20)
]
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
reader.last_reset_frame = lambda frame_id: 1
computer = DepthRecomputer(reader)
with patch("orisys_app_shared.record.depth.MAX_REPLAY_FRAMES", 5):
plan = computer.plan_for(frames[10])
assert "state_too_far_back" in plan.warnings
assert plan.state == STATE_APPROXIMATE
computer.close()
def test_the_same_frame_is_answered_from_the_cache():
"""A second request for the frame just read is a cache hit, not None."""
_CountingSensor.opened.clear()
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
with patch("orisys.Sensor", _CountingSensor):
first = computer._depth_from_segment(Path("raw_0001.mp4"), 3)
second = computer._depth_from_segment(Path("raw_0001.mp4"), 3)
sensor = computer._sensor._sensor
assert first.tolist() == [3]
assert second is not None and second.tolist() == [3]
assert sensor.n == 3, "the cached answer re-read the stream"
computer.close()
def test_the_processor_survives_a_pre_roll_to_segment_switch():
"""Only the video source is replaced; the tracker state must survive."""
_CountingSensor.opened.clear()
reader = _reader(_session("/cfg/sensor.json"))
preroll = reader.directory / "preroll_0001.mp4"
preroll.write_bytes(b"")
computer = DepthRecomputer(reader)
with patch("orisys.Sensor", _CountingSensor):
computer._feed(preroll, 2)
sensor = computer._sensor._sensor
assert sensor.n == 2, "the pre-roll frames were not fed"
# Feed the segment: same processor, new source, position restarted.
computer._feed(reader.directory / "raw_0001.mp4", 1)
assert len(_CountingSensor.opened) == 1, "the processor was recreated"
assert computer._sensor._sensor is sensor
assert len(sensor.sources) == 2
assert sensor.sources[0].endswith("preroll_0001.mp4")
assert sensor.sources[1].endswith("raw_0001.mp4")
assert sensor.initial_img == "reference-frame", "the tracker reference was reset"
computer.close()
def test_a_reset_boundary_is_re_applied_during_the_replay():
"""The live zeroing cleared the tracker; the replay must do the same."""
_CountingSensor.opened.clear()
frames = [
SimpleNamespace(
frame_id=index,
t_rel_s=index / 30.0,
segment=1,
segment_frame=index + 1,
in_video=True,
dropped=False,
)
for index in range(10)
]
reader = _reader_with_config(frames)
reader.last_reset_frame = lambda frame_id: 6
reader.frame_at = lambda _t: frames[8]
computer = DepthRecomputer(reader)
with patch("orisys.Sensor", _CountingSensor):
result = computer.depth_at(frames[8].t_rel_s)
sensor = computer._sensor._sensor
assert result.ok, result.error
assert sensor.resets == 1, "the reset boundary was not re-applied"
assert result.warnings == (), result.warnings
computer.close()
def test_a_dropped_frame_is_reported_not_substituted():
"""The index marks the hole; replay must say so instead of showing a neighbour."""
frames = [
SimpleNamespace(
frame_id=0,
t_rel_s=0.0,
segment=1,
segment_frame=1,
in_video=True,
dropped=False,
),
SimpleNamespace(
frame_id=1,
t_rel_s=1 / 30.0,
segment=1,
segment_frame=0,
in_video=False,
dropped=True,
),
]
reader = _reader_with_config(frames)
reader.frame_at = lambda _t: frames[1]
computer = DepthRecomputer(reader)
result = computer.depth_at(1 / 30.0)
assert not result.ok
assert result.error == "frame_not_recorded"
computer.close()
+2 -7
View File
@@ -92,14 +92,9 @@ def test_the_moved_models_are_the_same_objects_everywhere():
def test_the_recording_pieces_are_separate_modules(): def test_the_recording_pieces_are_separate_modules():
from orisys_app_shared.record import depth, experiment from orisys_app_shared.record import experiment
from orisys_app_shared.record.depth_backends import BoxReplay, FingerReplay
from orisys_app_shared.record.video_writer import VideoSegments from orisys_app_shared.record.video_writer import VideoSegments
# The depth replay no longer defines its backends inline... # The writer delegates video encoding to its own component.
assert not hasattr(depth, "BoxReplay")
assert not hasattr(depth, "_BoxReplay")
# ...and the writer delegates video encoding to its own component.
assert not hasattr(experiment, "_open_segment") assert not hasattr(experiment, "_open_segment")
assert callable(VideoSegments.write) and callable(VideoSegments.close) assert callable(VideoSegments.write) and callable(VideoSegments.close)
assert BoxReplay.__name__ == "BoxReplay" and FingerReplay.__name__ == "FingerReplay"
-49
View File
@@ -24,7 +24,6 @@ from orisys_app_shared.record import (
RecordingReader, RecordingReader,
list_recordings, list_recordings,
) )
from orisys_app_shared.record.depth import DepthRecomputer
SCRATCH = Path(__file__).resolve().parent / "_scratch_playback" SCRATCH = Path(__file__).resolve().parent / "_scratch_playback"
@@ -219,60 +218,12 @@ def test_depth_policy_says_recompute_and_warns_about_drift(recording):
# The config file on disk no longer matches what was recorded. # The config file on disk no longer matches what was recorded.
codes = reader.replay_warnings(sdk_version="0.5.1") codes = reader.replay_warnings(sdk_version="0.5.1")
assert "config_changed" in codes assert "config_changed" in codes
assert "depth_recomputed" in codes
# Same SDK: no version warning. # Same SDK: no version warning.
assert "sdk_changed" not in codes assert "sdk_changed" not in codes
assert "sdk_changed" in reader.replay_warnings(sdk_version="9.9.9") assert "sdk_changed" in reader.replay_warnings(sdk_version="9.9.9")
def test_depth_recompute_preflight_states_why_it_cannot_run(recording):
reader = RecordingReader(recording)
recomputer = DepthRecomputer(reader)
try:
problem = recomputer.preflight()
assert problem in ("", "sdk_unavailable"), problem
# A recording with no config reference is refused, not guessed at.
missing = RecordingReader(recording)
missing._info = None # noqa: SLF001 - force a re-read below
empty = SCRATCH / "empty"
empty.mkdir(exist_ok=True)
(empty / "forces.csv").write_text(
",".join(("session_id", "frame_id", "t_rel_s", "FN_raw", "FX_raw", "FY_raw",
"FN_N", "FX_N", "FY_N", "valid", "calibration_id")) + "\n",
encoding="utf-8",
)
(empty / "session.json").write_text(json.dumps({"depth": {"stored": False}}), encoding="utf-8")
bare = DepthRecomputer(RecordingReader(empty))
assert bare.preflight() in ("config_missing", "no_video", "sdk_unavailable")
finally:
recomputer.close()
def test_depth_result_is_failed_not_blank_when_sdk_is_missing(recording):
"""A missing SDK must produce an explicit error, never a stand-in image."""
saved = sys.modules.get("orisys")
blocker = types.ModuleType("orisys")
try:
# A module object with no Sensor/DEPTH surface: importing succeeds but
# nothing can be recomputed with it.
sys.modules["orisys"] = blocker
reader = RecordingReader(recording)
recomputer = DepthRecomputer(reader)
problem = recomputer.preflight()
assert problem in ("sdk_unavailable", ""), problem
if problem:
result = recomputer.depth_at(1.0)
assert not result.ok
assert result.depth is None
assert result.error
finally:
if saved is None:
sys.modules.pop("orisys", None)
else:
sys.modules["orisys"] = saved
def test_list_recordings_finds_directories_newest_first(recording): def test_list_recordings_finds_directories_newest_first(recording):
second = SCRATCH / "zzz_later" second = SCRATCH / "zzz_later"