Browse a long recording from disk instead of loading it into memory
What
- record/index.py: RecordingIndex streams forces.csv once and keeps only the
small things — one time, one validity flag, one frame id per row plus a byte
offset every 4096 rows (~6.5 MB per 500k rows). row_range() bisects the times,
read_rows() seeks to the nearest checkpoint and scans forward, and overview()
folds every row into its bucket during the same pass to build a per-axis
min/max envelope.
- read_rows(max_points, unit) decimates **while streaming**: each bucket keeps
the extreme row of every axis in the requested unit, so a long interval never
has to exist in memory in full and a shear spike survives a flat normal channel.
- RecordingReader: info() decimates above max_rows (default 200k) and marks
itself decimated, with total_rows and a duration taken from the frame index
rather than from the view; window()/window_all()/overview()/statistics() read
the interval being asked about. list/window semantics unchanged for small
recordings.
- force/source.py: StoreCurveSource (live memory) and RecordingCurveSource (disk
interval + cached envelope) implement one interface — extent, window,
envelope, statistics, duration_s — so ForcePlotsPanel asks a *source* instead
of knowing where the data lives.
- force_plots: reads through the source (window, envelope, statistics, extent),
which also replaces its last direct store accesses.
- playback_dialog: draws the whole-session view through window_all() in the unit
being shown (bounded by CHART_MAX_POINTS), and says so when the view is
decimated ("显示 N / 共 M 行") instead of letting the operator count points.
- experiment._run: a failure that lands while the files are opening can no longer
be overwritten by the writer's own transition to "recording" (found by a flaky
test; the state check now covers STATE_FAILED).
Why
The store releases old blocks above ~4.6 h and the replay loaded every CSV row
into Python lists before drawing anything, so neither the live browser nor the
replay could show an earlier interval of a long recording. The index makes the
disk the data source and memory a cache of what is on screen.
Verification
- shared/tests/test_history_source.py (new, 8 cases) over a 12 000-row recording:
the index reports its rows and reads a bounded interval; a large recording is
marked decimated with its true duration and total rows; a small one is not; an
interval read sees the exact rows including a single peak; a decimated window
keeps the peak; the envelope fits the whole session in ≤200 points, keeps the
spike and is cached; the disk and memory sources agree on extent, values and
statistics; the whole session can be read without holding it all. Shared
suites: 146 tests, 0 failures (three consecutive runs).
- apps/studio/tools/ui_check.py: the replay section now checks the index row
count, that an interval read spans only the interval, the envelope size and
total_rows. 324 checks, 0 failures.
- apps/studio/tests/test_ui_smoke.py: 43 tests, 36 pass; the 7 failures are the
unchanged sandbox/harness artifacts.
- D:/orisys_ws/_review_phase12/reproduce.py: all ten lines unchanged;
compileall, locale and prefs parity clean.
- apps/studio/docs/record-channel.md §8 documents the index, the reader API and
the unified source, and lists what is still hardware work.
Not in this change
- Maximum RSS / backlog / loss measurements at real resolution on the target ARM
board (needs the hardware), and range CSV export.
This commit is contained in:
Binary file not shown.
|
Before Width: | Height: | Size: 2.0 MiB After Width: | Height: | Size: 2.0 MiB |
@@ -115,6 +115,8 @@
|
|||||||
"box_zoom_tip": "Switch to rubber-band zoom (drag a region on the chart)",
|
"box_zoom_tip": "Switch to rubber-band zoom (drag a region on the chart)",
|
||||||
"selection": "Selection {duration}s / {samples} samples — {detail}",
|
"selection": "Selection {duration}s / {samples} samples — {detail}",
|
||||||
"selection_empty": "No valid samples in the selection",
|
"selection_empty": "No valid samples in the selection",
|
||||||
|
"value_unavailable": "n/a",
|
||||||
|
"value_unavailable_tip": "No curve is loaded for {channel}, so it has no value in this unit (raw counts are never shown in its place)",
|
||||||
"browse_only_tip": "Drag or scroll on the chart to browse history"
|
"browse_only_tip": "Drag or scroll on the chart to browse history"
|
||||||
},
|
},
|
||||||
"layout": {
|
"layout": {
|
||||||
@@ -154,14 +156,21 @@
|
|||||||
"depth_state_approximate": "No reset or pre-roll history for this segment: the algorithm state is an approximation",
|
"depth_state_approximate": "No reset or pre-roll history for this segment: the algorithm state is an approximation",
|
||||||
"depth_warnings": "Recomputation notes: {detail}",
|
"depth_warnings": "Recomputation notes: {detail}",
|
||||||
"warnings": "Replay notes: {detail}",
|
"warnings": "Replay notes: {detail}",
|
||||||
|
"decimated": "This recording is long, so the curves are a decimated view ({shown} of {total} rows; intervals are read from disk on demand)",
|
||||||
"warn_config_changed": "the config differs from recording time, so the recomputed depth may differ",
|
"warn_config_changed": "the config differs from recording time, so the recomputed depth may differ",
|
||||||
"warn_sdk_changed": "the SDK version differs from recording time, so the recomputation may differ",
|
"warn_sdk_changed": "the SDK version differs from recording time, so the recomputation may differ",
|
||||||
"warn_config_missing": "no config hash was recorded, so recomputation cannot be verified",
|
"warn_config_missing": "no config hash was recorded, so recomputation cannot be verified",
|
||||||
"warn_depth_recomputed": "depth comes from offline recomputation, not from stored data",
|
"warn_depth_recomputed": "depth comes from offline recomputation, not from stored data",
|
||||||
"warn_rows_skipped": "some unparsable CSV rows were skipped",
|
"warn_rows_skipped": "some unparsable CSV rows were skipped",
|
||||||
"warn_frames_dropped": "frames were lost to a full queue; only their index rows remain (shown as \"not recorded\")",
|
"warn_frames_dropped": "frames were lost to a full queue; only their index rows remain (shown as \"not recorded\")",
|
||||||
|
"warn_out_of_order": "this recording was written by an older version: its rows are not in time order, so the replay re-sorts them and window edges may differ",
|
||||||
"warn_state_across_segment": "the algorithm state cannot be carried across a segment",
|
"warn_state_across_segment": "the algorithm state cannot be carried across a segment",
|
||||||
"warn_state_approximate": "the algorithm state is an approximation"
|
"warn_state_approximate": "the algorithm state is an approximation",
|
||||||
|
"warn_state_approximate_after_rewind": "the video was reopened to reach an earlier frame, so the state may differ",
|
||||||
|
"warn_state_too_far_back": "the zeroing is too far back: reading there would exceed the replay budget, so the state is approximate",
|
||||||
|
"warn_reset_frame_not_indexed": "the zeroing frame is not in the frame index, so the state cannot be rebuilt from it",
|
||||||
|
"warn_reset_frame_not_in_video": "the zeroing frame never reached the video, so the state cannot be rebuilt from it",
|
||||||
|
"warn_reset_not_applied": "the replay could not re-apply the zeroing, so the state may differ"
|
||||||
},
|
},
|
||||||
"record": {
|
"record": {
|
||||||
"label": "Record",
|
"label": "Record",
|
||||||
@@ -177,6 +186,8 @@
|
|||||||
"start_failed": "Could not start recording: {error}",
|
"start_failed": "Could not start recording: {error}",
|
||||||
"finished": "Recording finished — {frames} frames / {segments} segment(s) in {path}",
|
"finished": "Recording finished — {frames} frames / {segments} segment(s) in {path}",
|
||||||
"finished_with_problems": "Recording finished with problems: {detail}",
|
"finished_with_problems": "Recording finished with problems: {detail}",
|
||||||
|
"closing": "Closing the recording files…",
|
||||||
|
"failed_state": "The recording failed (see the summary and the log)",
|
||||||
"alarm": "Recording problem: {detail}"
|
"alarm": "Recording problem: {detail}"
|
||||||
},
|
},
|
||||||
"cal": {
|
"cal": {
|
||||||
@@ -204,8 +215,12 @@
|
|||||||
"open_record_tip": "Reopen a saved calibration session to inspect every point's samples and diagnostics",
|
"open_record_tip": "Reopen a saved calibration session to inspect every point's samples and diagnostics",
|
||||||
"open_record_filter": "Calibration records (*.json)",
|
"open_record_filter": "Calibration records (*.json)",
|
||||||
"open_record_ok": "Loaded {path} ({points} point(s))",
|
"open_record_ok": "Loaded {path} ({points} point(s))",
|
||||||
|
"open_record_device": "Recorded on: {device}",
|
||||||
"open_record_warnings": "Note: {detail}",
|
"open_record_warnings": "Note: {detail}",
|
||||||
"open_record_failed": "Could not open the calibration record: {error}",
|
"open_record_failed": "Could not open the calibration record: {error}",
|
||||||
|
"apply_mismatch_title": "Calibration does not match the target device",
|
||||||
|
"apply_mismatch": "This coefficient file belongs to: {device}\nMismatch: {detail}\n\nCurrent target: {sensor_type} / {config}\n\nApplying it would load coefficients that were not measured on this device.",
|
||||||
|
"apply_mismatch_skipped": "Apply cancelled (the calibration belongs to another device)",
|
||||||
"sn": "SN",
|
"sn": "SN",
|
||||||
"sn_placeholder": "Serial number (optional)",
|
"sn_placeholder": "Serial number (optional)",
|
||||||
"live_raw_label": "Live raw",
|
"live_raw_label": "Live raw",
|
||||||
@@ -255,11 +270,13 @@
|
|||||||
"export_local": "Also wrote {path}",
|
"export_local": "Also wrote {path}",
|
||||||
"export_warnings": "Skipped: {detail}",
|
"export_warnings": "Skipped: {detail}",
|
||||||
"export_failed": "Export failed: {error}",
|
"export_failed": "Export failed: {error}",
|
||||||
|
"export_no_model": "Nothing to export: {detail}",
|
||||||
"apply_title": "Apply calibration",
|
"apply_title": "Apply calibration",
|
||||||
"apply_confirm": "Apply the calibration just exported to the current device?\n\nSensor type: {sensor_type}\nConfig: {config}\n\nChoosing No keeps the exported file and leaves the running device untouched.",
|
"apply_confirm": "Apply the calibration just exported to the current device?\n\nSensor type: {sensor_type}\nConfig: {config}\n\nChoosing No keeps the exported file and leaves the running device untouched.",
|
||||||
"apply_skipped": "Exported; not applied to the device",
|
"apply_skipped": "Exported; not applied to the device",
|
||||||
"apply_pending": "Requested calibration for {sensor_type}; waiting for the device to confirm…",
|
"apply_pending": "Requested calibration for {sensor_type}; waiting for the device to confirm…",
|
||||||
"applied": "Device confirmed the calibration is in effect (version {version})",
|
"applied": "Device confirmed the calibration is in effect (version {version})",
|
||||||
|
"applied_none": "Device confirmed: no calibration is loaded (the column stays empty)",
|
||||||
"export_only": "Exported coefficients to {path} (not applied to the device)",
|
"export_only": "Exported coefficients to {path} (not applied to the device)",
|
||||||
"apply_no_target": "No device to apply to (run an acquisition first)"
|
"apply_no_target": "No device to apply to (run an acquisition first)"
|
||||||
},
|
},
|
||||||
@@ -286,6 +303,7 @@
|
|||||||
"video_ended": "Video ended",
|
"video_ended": "Video ended",
|
||||||
"camera_disconnected": "Camera disconnected",
|
"camera_disconnected": "Camera disconnected",
|
||||||
"reset_ok": "Tracker reset",
|
"reset_ok": "Tracker reset",
|
||||||
|
"reset_pending": "Zeroing requested; waiting for the acquisition thread to confirm…",
|
||||||
"finger_run_unavailable": "Run is not available for this sensor type",
|
"finger_run_unavailable": "Run is not available for this sensor type",
|
||||||
"video_required": "Select a video file first",
|
"video_required": "Select a video file first",
|
||||||
"config_missing": "Sensor config not found for {type}",
|
"config_missing": "Sensor config not found for {type}",
|
||||||
|
|||||||
@@ -115,6 +115,8 @@
|
|||||||
"box_zoom_tip": "切换为框选放大(拖动图表框选区域)",
|
"box_zoom_tip": "切换为框选放大(拖动图表框选区域)",
|
||||||
"selection": "选区 {duration}s / {samples} 点 — {detail}",
|
"selection": "选区 {duration}s / {samples} 点 — {detail}",
|
||||||
"selection_empty": "选区无有效样本",
|
"selection_empty": "选区无有效样本",
|
||||||
|
"value_unavailable": "不可用",
|
||||||
|
"value_unavailable_tip": "{channel} 轴未加载标定曲线,因此没有该单位的数值(不会用原始值冒充)",
|
||||||
"browse_only_tip": "拖动或滚轮即可进入历史浏览"
|
"browse_only_tip": "拖动或滚轮即可进入历史浏览"
|
||||||
},
|
},
|
||||||
"layout": {
|
"layout": {
|
||||||
@@ -154,14 +156,21 @@
|
|||||||
"depth_state_approximate": "本段缺少归零或预卷历史,算法参考状态为近似重建",
|
"depth_state_approximate": "本段缺少归零或预卷历史,算法参考状态为近似重建",
|
||||||
"depth_warnings": "重算提示: {detail}",
|
"depth_warnings": "重算提示: {detail}",
|
||||||
"warnings": "回放提示: {detail}",
|
"warnings": "回放提示: {detail}",
|
||||||
|
"decimated": "该录制较长,曲线为降采样视图(显示 {shown} / 共 {total} 行;区间数据按需从磁盘读取)",
|
||||||
"warn_config_changed": "配置文件与录制时不一致,重算出的深度可能与当时不同",
|
"warn_config_changed": "配置文件与录制时不一致,重算出的深度可能与当时不同",
|
||||||
"warn_sdk_changed": "SDK 版本与录制时不一致,重算结果可能不同",
|
"warn_sdk_changed": "SDK 版本与录制时不一致,重算结果可能不同",
|
||||||
"warn_config_missing": "找不到配置哈希,无法校验重算一致性",
|
"warn_config_missing": "找不到配置哈希,无法校验重算一致性",
|
||||||
"warn_depth_recomputed": "深度由离线重算得到,不是当时保存的数据",
|
"warn_depth_recomputed": "深度由离线重算得到,不是当时保存的数据",
|
||||||
"warn_rows_skipped": "有无法解析的 CSV 行被跳过",
|
"warn_rows_skipped": "有无法解析的 CSV 行被跳过",
|
||||||
"warn_frames_dropped": "有帧在队列溢出时丢失,仅保留索引记录(回放显示为“该帧未录制”)",
|
"warn_frames_dropped": "有帧在队列溢出时丢失,仅保留索引记录(回放显示为“该帧未录制”)",
|
||||||
|
"warn_out_of_order": "该录制由旧版本写入,行序不是时间序;回放已按时间重排,区间边界可能与当时不同",
|
||||||
"warn_state_across_segment": "重算不能跨分段延续算法状态",
|
"warn_state_across_segment": "重算不能跨分段延续算法状态",
|
||||||
"warn_state_approximate": "算法参考状态为近似重建"
|
"warn_state_approximate": "算法参考状态为近似重建",
|
||||||
|
"warn_state_approximate_after_rewind": "为回到更早的帧重新打开了视频,参考状态可能偏离当时",
|
||||||
|
"warn_state_too_far_back": "归零点过于久远,需要读取的帧数超出上限,已按近似状态重建",
|
||||||
|
"warn_reset_frame_not_indexed": "归零帧不在帧索引中,无法从该点重建状态",
|
||||||
|
"warn_reset_frame_not_in_video": "归零帧未写入视频,无法从该点重建状态",
|
||||||
|
"warn_reset_not_applied": "重放无法重新执行归零,参考状态可能偏离当时"
|
||||||
},
|
},
|
||||||
"record": {
|
"record": {
|
||||||
"label": "记录",
|
"label": "记录",
|
||||||
@@ -177,6 +186,8 @@
|
|||||||
"start_failed": "无法开始记录: {error}",
|
"start_failed": "无法开始记录: {error}",
|
||||||
"finished": "记录完成 — {frames} 帧 / {segments} 段,已写入 {path}",
|
"finished": "记录完成 — {frames} 帧 / {segments} 段,已写入 {path}",
|
||||||
"finished_with_problems": "记录完成,但存在问题: {detail}",
|
"finished_with_problems": "记录完成,但存在问题: {detail}",
|
||||||
|
"closing": "正在关闭记录文件…",
|
||||||
|
"failed_state": "记录已失败(详见摘要与日志)",
|
||||||
"alarm": "记录异常: {detail}"
|
"alarm": "记录异常: {detail}"
|
||||||
},
|
},
|
||||||
"cal": {
|
"cal": {
|
||||||
@@ -204,8 +215,12 @@
|
|||||||
"open_record_tip": "重新打开已保存的标定会话,检查每个点的完整采样与诊断状态",
|
"open_record_tip": "重新打开已保存的标定会话,检查每个点的完整采样与诊断状态",
|
||||||
"open_record_filter": "标定记录 (*.json)",
|
"open_record_filter": "标定记录 (*.json)",
|
||||||
"open_record_ok": "已载入 {path}({points} 个点)",
|
"open_record_ok": "已载入 {path}({points} 个点)",
|
||||||
|
"open_record_device": "记录设备: {device}",
|
||||||
"open_record_warnings": "注意: {detail}",
|
"open_record_warnings": "注意: {detail}",
|
||||||
"open_record_failed": "打开标定记录失败: {error}",
|
"open_record_failed": "打开标定记录失败: {error}",
|
||||||
|
"apply_mismatch_title": "标定与目标设备不匹配",
|
||||||
|
"apply_mismatch": "该标定文件属于: {device}\n不匹配项: {detail}\n\n当前目标: {sensor_type} / {config}\n\n继续应用会把不适用于本设备的系数写入运行中的设备。",
|
||||||
|
"apply_mismatch_skipped": "已取消应用(标定身份与目标设备不匹配)",
|
||||||
"sn": "序列号",
|
"sn": "序列号",
|
||||||
"sn_placeholder": "序列号(可选)",
|
"sn_placeholder": "序列号(可选)",
|
||||||
"live_raw_label": "实时原始值",
|
"live_raw_label": "实时原始值",
|
||||||
@@ -255,11 +270,13 @@
|
|||||||
"export_local": "同时写入 {path}",
|
"export_local": "同时写入 {path}",
|
||||||
"export_warnings": "跳过: {detail}",
|
"export_warnings": "跳过: {detail}",
|
||||||
"export_failed": "导出失败: {error}",
|
"export_failed": "导出失败: {error}",
|
||||||
|
"export_no_model": "没有可导出的模型: {detail}",
|
||||||
"apply_title": "应用标定",
|
"apply_title": "应用标定",
|
||||||
"apply_confirm": "将刚导出的标定应用到当前设备?\n\n设备类型: {sensor_type}\n配置: {config}\n\n选择「否」只保留导出文件,不改动运行中的设备。",
|
"apply_confirm": "将刚导出的标定应用到当前设备?\n\n设备类型: {sensor_type}\n配置: {config}\n\n选择「否」只保留导出文件,不改动运行中的设备。",
|
||||||
"apply_skipped": "已导出,未应用到设备",
|
"apply_skipped": "已导出,未应用到设备",
|
||||||
"apply_pending": "已请求把标定应用到 {sensor_type},等待设备确认…",
|
"apply_pending": "已请求把标定应用到 {sensor_type},等待设备确认…",
|
||||||
"applied": "设备已确认标定生效(版本 {version})",
|
"applied": "设备已确认标定生效(版本 {version})",
|
||||||
|
"applied_none": "设备已确认:当前没有加载标定(该项留空)",
|
||||||
"export_only": "已导出系数 {path}(未应用到设备)",
|
"export_only": "已导出系数 {path}(未应用到设备)",
|
||||||
"apply_no_target": "没有可应用的设备(请先运行一次采集)"
|
"apply_no_target": "没有可应用的设备(请先运行一次采集)"
|
||||||
},
|
},
|
||||||
@@ -286,6 +303,7 @@
|
|||||||
"video_ended": "视频已结束",
|
"video_ended": "视频已结束",
|
||||||
"camera_disconnected": "摄像机已断开",
|
"camera_disconnected": "摄像机已断开",
|
||||||
"reset_ok": "追踪器已重置",
|
"reset_ok": "追踪器已重置",
|
||||||
|
"reset_pending": "已请求归零,等待采集线程确认…",
|
||||||
"finger_run_unavailable": "此传感器类型暂不支持运行",
|
"finger_run_unavailable": "此传感器类型暂不支持运行",
|
||||||
"video_required": "请先选择视频文件",
|
"video_required": "请先选择视频文件",
|
||||||
"config_missing": "未找到传感器配置: {type}",
|
"config_missing": "未找到传感器配置: {type}",
|
||||||
|
|||||||
@@ -293,9 +293,44 @@ outputs/recordings/<UTC 时间戳>_session<NNNN>/
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 8. 尚未完成(第三阶段及以后)
|
## 8. 长历史浏览:按需读盘而不是整段载入
|
||||||
|
|
||||||
|
`record/index.py` 的 `RecordingIndex` 是回放的数据源索引:
|
||||||
|
|
||||||
|
- 流式扫一遍 `forces.csv`,只把**小东西**留在内存:每行一个时间(float64)、一个
|
||||||
|
valid 标志、一个帧号,以及每 `CHECKPOINT_ROWS`(4096)行的字节偏移。500k 行约
|
||||||
|
6.5 MB(`SessionStore` 的默认上限正好对应 30 Hz 下约 4.6 小时)。
|
||||||
|
- `row_range(t0, t1)` 用二分定位行区间,`read_rows(start, stop)` 先 `seek` 到最近的
|
||||||
|
检查点再向前扫描——读一个 10 秒区间就是 10 秒的解析量,不是整段。
|
||||||
|
区间会额外带上 `t0` 之前的那一行,绘制时才能画出跨过左边界的线段。
|
||||||
|
- `overview(buckets)` 在**同一次流式扫描**里把每行折进所属桶,得到每桶每轴的
|
||||||
|
min/max/count。全程色带因此不会漏掉某桶里的尖峰(按行抽样会漏)。
|
||||||
|
|
||||||
|
`ReadingReader` 相应变化:
|
||||||
|
|
||||||
|
| 方法 | 作用 |
|
||||||
|
|---|---|
|
||||||
|
| `info()` | 行数超过 `max_rows`(默认 200k)时给出**降采样视图**,并置 `decimated=True`;`duration_s` 始终取自帧索引,不取自视图 |
|
||||||
|
| `window(t0, t1, max_points=…)` | 区间精确读取(可再降采样),尖峰按轴保留 |
|
||||||
|
| `overview(unit, buckets)` | 全程包络,按 (unit, buckets) 缓存 |
|
||||||
|
| `statistics(t0, t1, unit)` | 只算该区间的统计量 |
|
||||||
|
|
||||||
|
`force/source.py` 把两种来源统一成同一接口:`StoreCurveSource`(实时内存块)与
|
||||||
|
`RecordingCurveSource`(磁盘区间 + 缓存包络),都提供
|
||||||
|
`extent() / window() / envelope() / statistics() / duration_s()`;`ForcePlotsPanel`
|
||||||
|
只认这个接口,因此同一套浏览逻辑既能看内存会话也能看磁盘录制。
|
||||||
|
|
||||||
|
回放窗口在曲线被降采样时会明确写出「显示 N / 共 M 行」,而不是让操作员自己数点。
|
||||||
|
|
||||||
|
> 未做的验收项:真实分辨率与目标 ARM 板上的最大 RSS、积压与丢失测试(需要硬件),
|
||||||
|
> 以及选中时间段的 CSV 导出。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 尚未完成(第三阶段及以后)
|
||||||
|
|
||||||
- PyOpenGL 三维显示(`mesh3d_panel.py` 目前仍是占位符,不是已有实现)。
|
- PyOpenGL 三维显示(`mesh3d_panel.py` 目前仍是占位符,不是已有实现)。
|
||||||
- CSV 的「导出选中时间段」(整段导出已随录制落盘;区间导出待做)。
|
- CSV 的「导出选中时间段」(整段导出已随录制落盘;区间导出待做)。
|
||||||
- 标定的稳定后采样(当前按固定时长采样,稳定性以 σ 呈现)。
|
- 标定的稳定后采样(当前按固定时长采样,稳定性以 σ 呈现)。
|
||||||
|
- 在目标 ARM 板上用真实分辨率测长时录制的最大 RSS、队列积压与丢失。
|
||||||
|
|
||||||
|
|||||||
@@ -34,10 +34,11 @@ from orisys_app_shared.acquisition.constants import (
|
|||||||
)
|
)
|
||||||
from orisys_app_shared.force import select_display_forces
|
from orisys_app_shared.force import select_display_forces
|
||||||
from orisys_app_shared.force.calibration_recorder import CalibrationRecorder
|
from orisys_app_shared.force.calibration_recorder import CalibrationRecorder
|
||||||
from orisys_app_shared.force.session import SessionSample, SessionStore
|
from orisys_app_shared.force.session import SessionStore
|
||||||
from orisys_app_shared.paths import recordings_dir
|
from orisys_app_shared.paths import recordings_dir
|
||||||
from orisys_app_shared.record import ExperimentRecorder, RecordConfig
|
from orisys_app_shared.record import RecordConfig
|
||||||
from orisys_app_shared.record.sample import to_record_sample
|
from orisys_app_shared.record.sample import to_record_sample
|
||||||
|
from orisys_studio.recording import RecordingCoordinator, record_config_from_prefs
|
||||||
from orisys_app_shared.i18n import set_locale, t
|
from orisys_app_shared.i18n import set_locale, t
|
||||||
from orisys_app_shared.logging import report_error
|
from orisys_app_shared.logging import report_error
|
||||||
from orisys_app_shared.paths import logo_path, sensor_json_path
|
from orisys_app_shared.paths import logo_path, sensor_json_path
|
||||||
@@ -133,20 +134,20 @@ class StudioMainWindow(QMainWindow):
|
|||||||
on_finished=self._on_calibration_point,
|
on_finished=self._on_calibration_point,
|
||||||
on_cancelled=self._on_calibration_cancelled,
|
on_cancelled=self._on_calibration_cancelled,
|
||||||
)
|
)
|
||||||
self.recorder = ExperimentRecorder()
|
# The record channel's mechanics (recorder lifecycle, metadata, the
|
||||||
self.recorder.set_alarm_handler(self._on_record_alarm)
|
# per-packet fan-out, confirmed events) live in the coordinator; this
|
||||||
self._record_alarms: list[str] = []
|
# window keeps the buttons, the menus and the status messages.
|
||||||
self._record_last_report = None
|
self.recording = RecordingCoordinator(
|
||||||
self._record_fps = 30.0
|
session_store=self.session_store,
|
||||||
|
cal_recorder=self.cal_recorder,
|
||||||
|
)
|
||||||
|
self.recorder = self.recording.recorder
|
||||||
self._session_counter = 0
|
self._session_counter = 0
|
||||||
self._playback = None
|
self._playback = None
|
||||||
self._record_error_count = 0
|
self._record_error_count = 0
|
||||||
self._dropped_reported = 0
|
self._dropped_reported = 0
|
||||||
self._active_calibration_id = ""
|
|
||||||
#: Last coefficients file written by the calibration panel (not applied).
|
#: Last coefficients file written by the calibration panel (not applied).
|
||||||
self._last_calibration_export = ""
|
self._last_calibration_export = ""
|
||||||
#: Confirmed calibration changes, handed over from the acquisition thread.
|
|
||||||
self._pending_calibration_events: list[tuple[str, str, int]] = []
|
|
||||||
self._depth_range_value: tuple[float, float] | None = None
|
self._depth_range_value: tuple[float, float] | None = None
|
||||||
self._last_depth_frame = None
|
self._last_depth_frame = None
|
||||||
|
|
||||||
@@ -249,14 +250,28 @@ class StudioMainWindow(QMainWindow):
|
|||||||
worker.add_sink(self._on_packet_recorded)
|
worker.add_sink(self._on_packet_recorded)
|
||||||
|
|
||||||
# ------------------------------------------------------- experiment record
|
# ------------------------------------------------------- experiment record
|
||||||
def _on_record_alarm(self, message: str) -> None:
|
@property
|
||||||
"""Called from the writer thread. Queue it; never touch Qt here."""
|
def _record_fps(self) -> float:
|
||||||
logger.error("Recording alarm: %s", message)
|
return self.recording.record_fps
|
||||||
self._record_alarms.append(str(message))
|
|
||||||
|
@property
|
||||||
|
def _active_calibration_id(self) -> str:
|
||||||
|
return self.recording.active_calibration
|
||||||
|
|
||||||
|
@_active_calibration_id.setter
|
||||||
|
def _active_calibration_id(self, value: str) -> None:
|
||||||
|
self.recording.set_active_calibration(value)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def _record_last_report(self):
|
||||||
|
return self.recording.last_report
|
||||||
|
|
||||||
|
@property
|
||||||
|
def _record_report_pending(self) -> bool:
|
||||||
|
return self.recording.report_pending
|
||||||
|
|
||||||
def _drain_record_alarms(self) -> None:
|
def _drain_record_alarms(self) -> None:
|
||||||
while self._record_alarms:
|
for message in self.recording.drain_alarms():
|
||||||
message = self._record_alarms.pop(0)
|
|
||||||
self._set_phase("error")
|
self._set_phase("error")
|
||||||
self.statusBar().showMessage(t("record.alarm", detail=message))
|
self.statusBar().showMessage(t("record.alarm", detail=message))
|
||||||
|
|
||||||
@@ -274,16 +289,13 @@ class StudioMainWindow(QMainWindow):
|
|||||||
self.statusBar().showMessage(t("record.requires_run"))
|
self.statusBar().showMessage(t("record.requires_run"))
|
||||||
return
|
return
|
||||||
mode = self.toolbar.record_mode()
|
mode = self.toolbar.record_mode()
|
||||||
config = RecordConfig(
|
config = record_config_from_prefs(
|
||||||
video=(mode == "video_data"),
|
self.prefs, mode=mode, fps=float(self._record_fps)
|
||||||
lossless_frames=bool(self.prefs.get("record_lossless_frames", False)),
|
|
||||||
fps=float(self._record_fps),
|
|
||||||
segment_seconds=float(self.prefs.get("record_segment_seconds", 300.0)),
|
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
directory = self.recorder.start(
|
directory = self.recording.start(
|
||||||
root=recordings_dir(),
|
root=recordings_dir(),
|
||||||
session_id=int(self.session_store.session_id),
|
config=config,
|
||||||
metadata=self._build_session_metadata(config),
|
metadata=self._build_session_metadata(config),
|
||||||
calibration_id=self._active_calibration_id,
|
calibration_id=self._active_calibration_id,
|
||||||
calibration_version=self._active_calibration_id,
|
calibration_version=self._active_calibration_id,
|
||||||
@@ -297,21 +309,41 @@ class StudioMainWindow(QMainWindow):
|
|||||||
self.statusBar().showMessage(t("record.start_failed", error=str(exc)))
|
self.statusBar().showMessage(t("record.start_failed", error=str(exc)))
|
||||||
logger.exception("Could not start recording")
|
logger.exception("Could not start recording")
|
||||||
return
|
return
|
||||||
self._record_last_report = None
|
|
||||||
self.recorder.log_event("record_started", f"mode={mode}")
|
self.recorder.log_event("record_started", f"mode={mode}")
|
||||||
self.toolbar.set_record_status(Path(directory).name)
|
self.toolbar.set_record_status(Path(directory).name)
|
||||||
self.statusBar().showMessage(t("record.started", mode=t(f"record.mode_{mode}")))
|
self.statusBar().showMessage(t("record.started", mode=t(f"record.mode_{mode}")))
|
||||||
logger.info("Recording started: %s (mode=%s)", directory, mode)
|
logger.info("Recording started: %s (mode=%s)", directory, mode)
|
||||||
|
|
||||||
def _stop_recording(self, *, wait: bool = True) -> None:
|
def _stop_recording(self, *, wait: bool = True) -> None:
|
||||||
if not self.recorder.is_recording:
|
"""Stop the recording and publish what actually happened.
|
||||||
self.toolbar.set_record_status("")
|
|
||||||
return
|
A recording that already failed on its own is **not** a no-op: its report
|
||||||
self.recorder.log_event("record_stopped", "")
|
still has to be consumed, or the button keeps claiming a recording is
|
||||||
report = self.recorder.stop(wait=wait)
|
running and the failure is never shown.
|
||||||
self._record_last_report = report
|
|
||||||
|
``wait=False`` returns as soon as the stop is requested; the writer owns
|
||||||
|
the completion and the UI timer publishes it (see ``_poll_record_report``).
|
||||||
|
"""
|
||||||
|
if self.recorder.directory is None and self.recorder.report is None:
|
||||||
|
# Nothing was ever started: keep the widget state consistent anyway.
|
||||||
self.toolbar.set_record_status("")
|
self.toolbar.set_record_status("")
|
||||||
self.toolbar.set_record_checked(False)
|
self.toolbar.set_record_checked(False)
|
||||||
|
return
|
||||||
|
self._publish_record_report(self.recording.stop(wait=wait))
|
||||||
|
|
||||||
|
def _publish_record_report(self, report) -> None:
|
||||||
|
"""Update the button and the summary from a recording report."""
|
||||||
|
self.toolbar.set_record_status("")
|
||||||
|
self.toolbar.set_record_checked(False)
|
||||||
|
if report is None:
|
||||||
|
return
|
||||||
|
if not report.completed:
|
||||||
|
# The writer is still closing its files. Never present that as a
|
||||||
|
# finished experiment; the timer publishes the real report.
|
||||||
|
self.statusBar().showMessage(t("record.closing"))
|
||||||
|
return
|
||||||
|
if report.stats.failed:
|
||||||
|
self._set_phase("error")
|
||||||
if report.problems:
|
if report.problems:
|
||||||
self.statusBar().showMessage(
|
self.statusBar().showMessage(
|
||||||
t("record.finished_with_problems", detail="; ".join(report.problems))
|
t("record.finished_with_problems", detail="; ".join(report.problems))
|
||||||
@@ -326,6 +358,12 @@ class StudioMainWindow(QMainWindow):
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _poll_record_report(self) -> None:
|
||||||
|
"""Publish the terminal report once the writer has closed the files."""
|
||||||
|
report = self.recording.poll()
|
||||||
|
if report is not None:
|
||||||
|
self._publish_record_report(report)
|
||||||
|
|
||||||
def _take_preroll(self) -> list:
|
def _take_preroll(self) -> list:
|
||||||
"""Frame history from the active worker, if it keeps one.
|
"""Frame history from the active worker, if it keeps one.
|
||||||
|
|
||||||
@@ -333,15 +371,17 @@ class StudioMainWindow(QMainWindow):
|
|||||||
recording follows, so a stale history can never be attached to a later
|
recording follows, so a stale history can never be attached to a later
|
||||||
recording.
|
recording.
|
||||||
"""
|
"""
|
||||||
worker = self._active_worker
|
return self.recording.take_preroll(self._active_worker)
|
||||||
take = getattr(worker, "take_preroll", None)
|
|
||||||
if not callable(take):
|
@staticmethod
|
||||||
return []
|
def _device_serial(info) -> str:
|
||||||
try:
|
"""The sensor's own serial number, when the SDK exposes one."""
|
||||||
return list(take())
|
for attribute in ("sn", "serial", "serial_number"):
|
||||||
except Exception: # noqa: BLE001 - the history is optional, not the record
|
value = getattr(info, attribute, None)
|
||||||
logger.exception("Could not take the pre-roll buffer")
|
if value:
|
||||||
return []
|
return str(value)
|
||||||
|
serial = getattr(info, "info", None)
|
||||||
|
return str(getattr(serial, "SN", "") or "")
|
||||||
|
|
||||||
def _build_session_metadata(self, config: RecordConfig) -> dict:
|
def _build_session_metadata(self, config: RecordConfig) -> dict:
|
||||||
from orisys_app_shared import API_VERSION, __version__ as shared_version
|
from orisys_app_shared import API_VERSION, __version__ as shared_version
|
||||||
@@ -379,6 +419,10 @@ class StudioMainWindow(QMainWindow):
|
|||||||
# filled in by the acquisition thread's confirmation, so it can
|
# filled in by the acquisition thread's confirmation, so it can
|
||||||
# be empty here; the CSV's per-row version is authoritative.
|
# be empty here; the CSV's per-row version is authoritative.
|
||||||
"version": self._active_calibration_id,
|
"version": self._active_calibration_id,
|
||||||
|
# The operator-entered serial of the *calibrated* device, as the
|
||||||
|
# calibration panel has it (the device SN when the config has one).
|
||||||
|
"sn": self.cal_panel.sn_edit.text().strip(),
|
||||||
|
"device_sn": self._device_serial(info),
|
||||||
},
|
},
|
||||||
# Depth is deliberately NOT stored (decided with the team): it is a
|
# Depth is deliberately NOT stored (decided with the team): it is a
|
||||||
# function of the frame and the sensor configuration, so replay
|
# function of the frame and the sensor configuration, so replay
|
||||||
@@ -395,7 +439,10 @@ class StudioMainWindow(QMainWindow):
|
|||||||
"conventions": {
|
"conventions": {
|
||||||
"axis_order": ["FN", "FX", "FY"],
|
"axis_order": ["FN", "FX", "FY"],
|
||||||
"force_unit": "N when calibrated, otherwise raw counts",
|
"force_unit": "N when calibrated, otherwise raw counts",
|
||||||
"time_base": "t_rel_s is relative to the first recorded frame",
|
"time_base": (
|
||||||
|
"t_rel_s is relative to the first recorded frame; an event "
|
||||||
|
"that predates it has no relative base and is left empty"
|
||||||
|
),
|
||||||
"frame_id": "monotonic per acquisition session",
|
"frame_id": "monotonic per acquisition session",
|
||||||
},
|
},
|
||||||
"filter": {
|
"filter": {
|
||||||
@@ -432,25 +479,10 @@ class StudioMainWindow(QMainWindow):
|
|||||||
def _on_packet_recorded(self, packet) -> None:
|
def _on_packet_recorded(self, packet) -> None:
|
||||||
"""Runs on the **acquisition thread**. Must stay cheap and Qt-free.
|
"""Runs on the **acquisition thread**. Must stay cheap and Qt-free.
|
||||||
|
|
||||||
The plots are allowed to drop frames; this path is not. It is also the
|
The record channel's mechanics are the coordinator's; all this keeps is
|
||||||
reason calibration sampling no longer depends on the calibration panel
|
the UI's own bookkeeping (how many readouts failed).
|
||||||
being visible.
|
|
||||||
|
|
||||||
The packet is converted exactly once, by the same function the CSV writer
|
|
||||||
uses, so the store and the file can never disagree about whether a value
|
|
||||||
is a measurement.
|
|
||||||
"""
|
"""
|
||||||
record = to_record_sample(
|
record = self.recording.on_packet(packet)
|
||||||
packet, calibration_version=self._active_calibration_id
|
|
||||||
)
|
|
||||||
sample = SessionSample.from_record(record)
|
|
||||||
self.session_store.append(sample)
|
|
||||||
self.cal_recorder.feed(sample)
|
|
||||||
# Video + CSV + metadata share this one record path.
|
|
||||||
self.recorder.on_packet(packet)
|
|
||||||
fps = float(getattr(packet, "fps", 0.0) or 0.0)
|
|
||||||
if fps > 0.0:
|
|
||||||
self._record_fps = fps
|
|
||||||
if not record.valid:
|
if not record.valid:
|
||||||
self._record_error_count += 1
|
self._record_error_count += 1
|
||||||
|
|
||||||
@@ -919,11 +951,64 @@ class StudioMainWindow(QMainWindow):
|
|||||||
self.recorder.log_event("calibration_exported", str(path))
|
self.recorder.log_event("calibration_exported", str(path))
|
||||||
self.statusBar().showMessage(t("cal.export_only", path=Path(path).name))
|
self.statusBar().showMessage(t("cal.export_only", path=Path(path).name))
|
||||||
|
|
||||||
|
def calibration_identity(self, path: str) -> dict:
|
||||||
|
"""What device a coefficient file says it belongs to.
|
||||||
|
|
||||||
|
Read with the SDK's own parser so the check sees the same document the
|
||||||
|
device would load. A file that cannot be read yields no identity, and the
|
||||||
|
caller then applies it without a mismatch claim.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
import orisys.calibration as sdk_calibration
|
||||||
|
|
||||||
|
document = sdk_calibration.load(path, strict=True)
|
||||||
|
except Exception: # noqa: BLE001 - reported by the apply path itself
|
||||||
|
logger.exception("Could not read calibration identity from %s", path)
|
||||||
|
return {}
|
||||||
|
identity = dict(getattr(document, "identity", {}) or {})
|
||||||
|
pipeline = dict(getattr(document, "pipeline", {}) or {})
|
||||||
|
return {
|
||||||
|
"sensor_type": str(identity.get("sensor_type") or ""),
|
||||||
|
"sn": str(identity.get("sn") or ""),
|
||||||
|
"config_ref": str(pipeline.get("config_ref") or ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _same_path(first: str, second: str) -> bool:
|
||||||
|
if not first or not second:
|
||||||
|
return True # nothing to compare: not a mismatch
|
||||||
|
if first == second:
|
||||||
|
return True
|
||||||
|
try:
|
||||||
|
return Path(first).resolve() == Path(second).resolve()
|
||||||
|
except Exception: # noqa: BLE001 - a non-path value simply differs
|
||||||
|
return False
|
||||||
|
|
||||||
|
def calibration_target_mismatch(
|
||||||
|
self, path: str, *, sensor_type: str, config_ref: str
|
||||||
|
) -> list[str]:
|
||||||
|
"""Reasons this calibration does not obviously belong to this device.
|
||||||
|
|
||||||
|
Applying coefficients that were measured on a different sensor silently
|
||||||
|
changes what every recorded newton value means, so the mismatch is stated
|
||||||
|
before the confirmation, not discovered later.
|
||||||
|
"""
|
||||||
|
recorded = self.calibration_identity(path)
|
||||||
|
reasons: list[str] = []
|
||||||
|
recorded_type = recorded.get("sensor_type") or ""
|
||||||
|
if recorded_type and sensor_type and recorded_type != sensor_type:
|
||||||
|
reasons.append(f"sensor_type: {recorded_type} != {sensor_type}")
|
||||||
|
recorded_config = recorded.get("config_ref") or ""
|
||||||
|
if recorded_config and not self._same_path(recorded_config, config_ref):
|
||||||
|
reasons.append(f"config_ref: {recorded_config} != {config_ref}")
|
||||||
|
return reasons
|
||||||
|
|
||||||
def _on_calibration_apply(self, path: str) -> None:
|
def _on_calibration_apply(self, path: str) -> None:
|
||||||
"""Push freshly exported coefficients into the device — on confirmation.
|
"""Push freshly exported coefficients into the device — on confirmation.
|
||||||
|
|
||||||
The user is shown the exact target device first, and only the matching
|
The user is shown the exact target device first, and only the matching
|
||||||
worker is updated.
|
worker is updated. A calibration that names a *different* device or
|
||||||
|
configuration is called out explicitly and defaults to "no".
|
||||||
"""
|
"""
|
||||||
path = str(path or self._last_calibration_export)
|
path = str(path or self._last_calibration_export)
|
||||||
if not path:
|
if not path:
|
||||||
@@ -931,6 +1016,29 @@ class StudioMainWindow(QMainWindow):
|
|||||||
return
|
return
|
||||||
sensor_type = self.sensor_picker.sensor_type()
|
sensor_type = self.sensor_picker.sensor_type()
|
||||||
config_ref = self._resolve_config_ref() or ""
|
config_ref = self._resolve_config_ref() or ""
|
||||||
|
reasons = self.calibration_target_mismatch(
|
||||||
|
path, sensor_type=sensor_type, config_ref=config_ref
|
||||||
|
)
|
||||||
|
if reasons:
|
||||||
|
answer = QMessageBox.question(
|
||||||
|
self,
|
||||||
|
t("cal.apply_mismatch_title"),
|
||||||
|
t(
|
||||||
|
"cal.apply_mismatch",
|
||||||
|
device=self.calibration_identity(path).get("sensor_type") or "?",
|
||||||
|
detail="; ".join(reasons),
|
||||||
|
sensor_type=sensor_type,
|
||||||
|
config=config_ref,
|
||||||
|
),
|
||||||
|
QMessageBox.Yes | QMessageBox.No,
|
||||||
|
QMessageBox.No,
|
||||||
|
)
|
||||||
|
if answer != QMessageBox.Yes:
|
||||||
|
self.recorder.log_event(
|
||||||
|
"calibration_apply_blocked", "; ".join(reasons)
|
||||||
|
)
|
||||||
|
self.statusBar().showMessage(t("cal.apply_mismatch_skipped"))
|
||||||
|
return
|
||||||
answer = QMessageBox.question(
|
answer = QMessageBox.question(
|
||||||
self,
|
self,
|
||||||
t("cal.apply_title"),
|
t("cal.apply_title"),
|
||||||
@@ -952,30 +1060,32 @@ class StudioMainWindow(QMainWindow):
|
|||||||
t("cal.apply_pending", sensor_type=sensor_type)
|
t("cal.apply_pending", sensor_type=sensor_type)
|
||||||
)
|
)
|
||||||
|
|
||||||
def _on_calibration_applied(self, path: str, version: str, frame_id: int) -> None:
|
def _on_calibration_applied(
|
||||||
|
self, path: str, version: str, frame_id: int, mono_t: float, session: int
|
||||||
|
) -> None:
|
||||||
"""Called from the acquisition thread once new coefficients are loaded."""
|
"""Called from the acquisition thread once new coefficients are loaded."""
|
||||||
# The acquisition thread must not touch Qt, so only plain state moves here.
|
# The acquisition thread must not touch Qt, so only plain state moves here.
|
||||||
self._pending_calibration_events.append(
|
self.recording.note_calibration_applied(path, version, frame_id, mono_t, session)
|
||||||
(str(path), str(version), int(frame_id))
|
|
||||||
)
|
def _on_reset_applied(self, frame_id: int, mono_t: float, session: int) -> None:
|
||||||
|
"""Called from the acquisition thread once a zeroing took effect."""
|
||||||
|
self.recording.note_reset(frame_id, mono_t, session)
|
||||||
|
|
||||||
def _drain_calibration_events(self) -> None:
|
def _drain_calibration_events(self) -> None:
|
||||||
"""Publish confirmed calibration changes on the UI thread."""
|
"""Publish confirmed calibration changes on the UI thread."""
|
||||||
while self._pending_calibration_events:
|
for event in self.recording.drain_calibration_events():
|
||||||
path, version, frame_id = self._pending_calibration_events.pop(0)
|
version = event["version"]
|
||||||
# The version is the calibration's content hash: a path can be
|
if version:
|
||||||
# re-exported with different coefficients, a hash cannot.
|
self.statusBar().showMessage(t("cal.applied", version=version[:12]))
|
||||||
self._active_calibration_id = version or path
|
else:
|
||||||
self.session_store.set_calibration_id(self._active_calibration_id)
|
# An empty version is the honest answer for a device with no
|
||||||
self.recorder.set_calibration_version(self._active_calibration_id)
|
# calibration; it must not read as "the last one is still on".
|
||||||
self.recorder.log_event(
|
self.statusBar().showMessage(t("cal.applied_none"))
|
||||||
"calibration_applied",
|
|
||||||
f"{path} ({self._active_calibration_id[:12] or 'unhashed'})",
|
def _drain_reset_events(self) -> None:
|
||||||
frame_id=frame_id,
|
"""Publish confirmed zeroings on the UI thread."""
|
||||||
)
|
for _event in self.recording.drain_reset_events():
|
||||||
self.statusBar().showMessage(
|
self.statusBar().showMessage(t("status.reset_ok"))
|
||||||
t("cal.applied", version=self._active_calibration_id[:12])
|
|
||||||
)
|
|
||||||
|
|
||||||
def _worker_for(self, sensor_type: str):
|
def _worker_for(self, sensor_type: str):
|
||||||
self._ensure_workers()
|
self._ensure_workers()
|
||||||
@@ -1228,7 +1338,6 @@ class StudioMainWindow(QMainWindow):
|
|||||||
# starts empty and is filled from that confirmation — a path is not a
|
# starts empty and is filled from that confirmation — a path is not a
|
||||||
# version, and the same path can hold different numbers.
|
# version, and the same path can hold different numbers.
|
||||||
self._active_calibration_id = ""
|
self._active_calibration_id = ""
|
||||||
self._pending_calibration_events.clear()
|
|
||||||
self._record_error_count = 0
|
self._record_error_count = 0
|
||||||
self._dropped_reported = 0
|
self._dropped_reported = 0
|
||||||
self._depth_range_value = None
|
self._depth_range_value = None
|
||||||
@@ -1318,18 +1427,12 @@ class StudioMainWindow(QMainWindow):
|
|||||||
if self._session_phase != "running" or self._active_worker is None:
|
if self._session_phase != "running" or self._active_worker is None:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
|
# A request, not the zeroing itself: the acquisition thread applies it
|
||||||
|
# between two frames and reports the frame and time it took effect.
|
||||||
self._active_worker.reset()
|
self._active_worker.reset()
|
||||||
self.force_plots.reset()
|
self.force_plots.reset()
|
||||||
self._last_plot_packet_key = None
|
self._last_plot_packet_key = None
|
||||||
# Recorded with the frame it happened at: a reset clears the SDK's
|
self.statusBar().showMessage(t("status.reset_pending"))
|
||||||
# tracker, so an offline depth replay can start here instead of
|
|
||||||
# guessing at the algorithm's earlier state.
|
|
||||||
self.recorder.log_event(
|
|
||||||
"zeroing",
|
|
||||||
"tracker reset",
|
|
||||||
frame_id=self._current_frame_id(),
|
|
||||||
)
|
|
||||||
self.statusBar().showMessage(t("status.reset_ok"))
|
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
self.statusBar().showMessage(t("status.run_failed", error=str(exc)))
|
self.statusBar().showMessage(t("status.run_failed", error=str(exc)))
|
||||||
|
|
||||||
@@ -1351,6 +1454,11 @@ class StudioMainWindow(QMainWindow):
|
|||||||
def _on_run_tick(self) -> None:
|
def _on_run_tick(self) -> None:
|
||||||
if self._active_worker is None:
|
if self._active_worker is None:
|
||||||
return
|
return
|
||||||
|
# Record bookkeeping happens before anything that can return early: a
|
||||||
|
# recording that failed while the device was quiet still has to update
|
||||||
|
# the button and the summary.
|
||||||
|
self._drain_record_alarms()
|
||||||
|
self._poll_record_report()
|
||||||
if self._last_session_error:
|
if self._last_session_error:
|
||||||
err = self._last_session_error
|
err = self._last_session_error
|
||||||
self._last_session_error = ""
|
self._last_session_error = ""
|
||||||
@@ -1405,31 +1513,31 @@ class StudioMainWindow(QMainWindow):
|
|||||||
|
|
||||||
if is_new:
|
if is_new:
|
||||||
self._last_plot_packet_key = packet_key
|
self._last_plot_packet_key = packet_key
|
||||||
# Invalid readouts must not be drawn as if they were data, and they
|
# The live view consumes the *same* standardised measurement as the
|
||||||
# must not be written into the plot buffer: NaN would poison the
|
# CSV and the session store: one conversion decides what a value
|
||||||
# auto-range and a zero would be indistinguishable from a real one.
|
# means, so the chart cannot show something the record does not
|
||||||
if packet.valid:
|
# contain (or vice versa).
|
||||||
|
record = to_record_sample(packet)
|
||||||
|
if record.valid:
|
||||||
display = select_display_forces(
|
display = select_display_forces(
|
||||||
units_pref=self.toolbar.units(),
|
units_pref=self.toolbar.units(),
|
||||||
force_calibrated=packet.force_calibrated,
|
sample=record,
|
||||||
fn=packet.normal_force,
|
|
||||||
fx=packet.shear_x,
|
|
||||||
fy=packet.shear_y,
|
|
||||||
fn_raw=packet.normal_force_raw,
|
|
||||||
fx_raw=packet.shear_x_raw,
|
|
||||||
fy_raw=packet.shear_y_raw,
|
|
||||||
)
|
)
|
||||||
if display.using_raw_fallback:
|
if display.using_raw_fallback:
|
||||||
self.statusBar().showMessage(t("status.units_raw_fallback"))
|
self.statusBar().showMessage(t("status.units_raw_fallback"))
|
||||||
self.force_plots.push(
|
self.force_plots.push(
|
||||||
packet.timestamp,
|
# Session-relative time, the same base the store and
|
||||||
display.fn,
|
# ``forces.csv`` use. The wall clock must never reach the
|
||||||
display.fx,
|
# buffer: it would put the live axis on a different scale.
|
||||||
display.fy,
|
self._session_time(packet),
|
||||||
|
display.values[0],
|
||||||
|
display.values[1],
|
||||||
|
display.values[2],
|
||||||
unit=display.unit,
|
unit=display.unit,
|
||||||
|
available=display.available,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
self._report_invalid_sample(packet.invalid_reason)
|
self._report_invalid_sample(record.invalid_reason)
|
||||||
# The calibration panel is a *view*: sampling is driven by the
|
# The calibration panel is a *view*: sampling is driven by the
|
||||||
# record channel, so hiding the panel can no longer stall it.
|
# record channel, so hiding the panel can no longer stall it.
|
||||||
self.cal_panel.set_live(
|
self.cal_panel.set_live(
|
||||||
@@ -1441,8 +1549,8 @@ class StudioMainWindow(QMainWindow):
|
|||||||
# Finalise calibration sampling on the UI thread once its deadline passes.
|
# Finalise calibration sampling on the UI thread once its deadline passes.
|
||||||
self.cal_recorder.tick()
|
self.cal_recorder.tick()
|
||||||
self.cal_panel.poll_recorder()
|
self.cal_panel.poll_recorder()
|
||||||
self._drain_record_alarms()
|
|
||||||
self._drain_calibration_events()
|
self._drain_calibration_events()
|
||||||
|
self._drain_reset_events()
|
||||||
self._report_dropped_samples()
|
self._report_dropped_samples()
|
||||||
|
|
||||||
mode = self.arrow_panel.view_mode()
|
mode = self.arrow_panel.view_mode()
|
||||||
@@ -1451,6 +1559,23 @@ class StudioMainWindow(QMainWindow):
|
|||||||
if self._show_fps and is_new:
|
if self._show_fps and is_new:
|
||||||
self.arrow_panel.set_fps(packet.fps)
|
self.arrow_panel.set_fps(packet.fps)
|
||||||
|
|
||||||
|
def _session_time(self, packet) -> float:
|
||||||
|
"""Session-relative acquisition time for the live view.
|
||||||
|
|
||||||
|
The session store already defines the origin (its first appended sample),
|
||||||
|
and ``forces.csv`` uses the same base. Deriving the live time from that
|
||||||
|
one origin is what keeps the live buffer, a rebuilt history and the replay
|
||||||
|
on a single X axis.
|
||||||
|
"""
|
||||||
|
origin = self.session_store.origin_time()
|
||||||
|
absolute = getattr(packet, "mono_t", None)
|
||||||
|
if absolute is None:
|
||||||
|
absolute = getattr(packet, "timestamp", 0.0)
|
||||||
|
try:
|
||||||
|
return float(absolute) - float(origin)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return 0.0
|
||||||
|
|
||||||
def _report_invalid_sample(self, reason: str) -> None:
|
def _report_invalid_sample(self, reason: str) -> None:
|
||||||
"""Surface failed readouts instead of letting them pass as data.
|
"""Surface failed readouts instead of letting them pass as data.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,130 @@
|
|||||||
|
"""Calibration service: the panel's operations, without the panel.
|
||||||
|
|
||||||
|
The panel shows a table and emits status messages; everything that touches the
|
||||||
|
file system or a device identity happens here:
|
||||||
|
|
||||||
|
* preparing the session once (store, diagnostics and export share one model);
|
||||||
|
* saving the audit record, reopening one, and exporting the coefficients;
|
||||||
|
* keeping a reopened record's **own** device identity instead of restamping it
|
||||||
|
with whatever device happens to be connected.
|
||||||
|
|
||||||
|
This is deliberately Qt-free so it can be tested without a display.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_io import (
|
||||||
|
CHANNEL_KEYS,
|
||||||
|
CalSessionStore,
|
||||||
|
export_force_calibration_v2,
|
||||||
|
meta_from_sensor_json,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.force.calibration_session import (
|
||||||
|
CalibrationSessionRecord,
|
||||||
|
read_calibration_session,
|
||||||
|
session_filename,
|
||||||
|
write_calibration_session,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.paths import calibration_export_dir
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class CalibrationService:
|
||||||
|
"""Owns the calibration session's files and its prepared model."""
|
||||||
|
|
||||||
|
def __init__(self, *, store: Optional[CalSessionStore] = None) -> None:
|
||||||
|
self.store = store or CalSessionStore()
|
||||||
|
#: The one prepared model the panel displays and exports.
|
||||||
|
self.prepared: Any = None
|
||||||
|
#: Device identity of a reopened record, while it is open.
|
||||||
|
self.record_identity: Optional[dict] = None
|
||||||
|
#: The record currently open, if any.
|
||||||
|
self.open_record: Optional[CalibrationSessionRecord] = None
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- identity
|
||||||
|
def export_meta(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
sensor_type: str,
|
||||||
|
config_ref: str | None,
|
||||||
|
sn: str = "",
|
||||||
|
duration_s: float = 0.0,
|
||||||
|
) -> dict:
|
||||||
|
"""The identity/pipeline metadata an export would carry.
|
||||||
|
|
||||||
|
While a saved record is open its **own** device identity wins: reopening
|
||||||
|
device A's calibration on a machine connected to device B must keep saying
|
||||||
|
A, or the evidence is silently restamped with B's serial number.
|
||||||
|
"""
|
||||||
|
meta = meta_from_sensor_json(config_ref)
|
||||||
|
meta["sensor_type"] = str(sensor_type or "box")
|
||||||
|
meta["sn"] = str(sn or "")
|
||||||
|
meta["record_duration_s"] = float(duration_s or 0.0)
|
||||||
|
identity = self.record_identity
|
||||||
|
if identity:
|
||||||
|
meta["sensor_type"] = identity.get("sensor_type") or meta["sensor_type"]
|
||||||
|
meta["sn"] = identity.get("sn") or meta["sn"]
|
||||||
|
meta["config_ref"] = identity.get("config_ref") or meta.get("config_ref")
|
||||||
|
meta["source_record"] = str(identity.get("path") or "")
|
||||||
|
meta["source_created_at"] = str(identity.get("created_at") or "")
|
||||||
|
return meta
|
||||||
|
|
||||||
|
def clear_record_identity(self) -> None:
|
||||||
|
"""A new run is the live device's context again."""
|
||||||
|
self.record_identity = None
|
||||||
|
self.open_record = None
|
||||||
|
|
||||||
|
# ------------------------------------------------------------ preparation
|
||||||
|
def prepare(self, recorder, meta: dict) -> Any:
|
||||||
|
"""Prepare the session once: store, diagnostics and export share it."""
|
||||||
|
self.prepared = None
|
||||||
|
self.store.clear()
|
||||||
|
if recorder is None:
|
||||||
|
return None
|
||||||
|
prepared = recorder.prepared(meta=meta)
|
||||||
|
self.prepared = prepared
|
||||||
|
for key in CHANNEL_KEYS:
|
||||||
|
self.store.set_channel(key, prepared.channel_samples(key))
|
||||||
|
# The residuals belong to the points, because the table shows them.
|
||||||
|
for point in recorder.all_points():
|
||||||
|
value = prepared.residuals.get(id(point))
|
||||||
|
point.residual_n = float("nan") if value is None else float(value)
|
||||||
|
return prepared
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------- files
|
||||||
|
def save_record(self, recorder, meta: dict) -> Path:
|
||||||
|
"""Write the audit record. Returns the written path."""
|
||||||
|
points = recorder.all_points()
|
||||||
|
path = calibration_export_dir() / session_filename(meta.get("sensor_type"))
|
||||||
|
return write_calibration_session(path, points, meta=meta)
|
||||||
|
|
||||||
|
def load_record(self, path: str | Path) -> CalibrationSessionRecord:
|
||||||
|
"""Reopen a saved session and remember whose record it is."""
|
||||||
|
record = read_calibration_session(path)
|
||||||
|
identity = record.identity()
|
||||||
|
self.record_identity = {**identity, "path": str(record.path)}
|
||||||
|
self.open_record = record
|
||||||
|
return record
|
||||||
|
|
||||||
|
def export_coefficients(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
sensor_type: str,
|
||||||
|
export_dir: str | Path | None = None,
|
||||||
|
local_bundle_dir: str | Path | None = None,
|
||||||
|
) -> tuple[Path, Path | None, list[str]]:
|
||||||
|
"""Serialise the prepared model. Raises ValueError when there is none."""
|
||||||
|
if self.prepared is None or not getattr(self.prepared, "ok", False):
|
||||||
|
warnings = list(getattr(self.prepared, "warnings", []) or [])
|
||||||
|
raise ValueError("; ".join(warnings) or "no complete channels")
|
||||||
|
return export_force_calibration_v2(
|
||||||
|
self.prepared,
|
||||||
|
export_dir=export_dir or calibration_export_dir(),
|
||||||
|
sensor_type=str(sensor_type or "box"),
|
||||||
|
local_bundle_dir=local_bundle_dir,
|
||||||
|
)
|
||||||
@@ -0,0 +1,266 @@
|
|||||||
|
"""The record channel: one experiment directory, one session store, one sampler.
|
||||||
|
|
||||||
|
This is the coordination the main window used to do inline — and it is deliberately
|
||||||
|
free of Qt. Three consumers are fed from the same acquisition packet:
|
||||||
|
|
||||||
|
* the :class:`ExperimentRecorder`, which writes the directory;
|
||||||
|
* the whole-session store the charts and the browser read;
|
||||||
|
* the calibration sampler (``CalibrationRecorder``), whose lifecycle must not
|
||||||
|
depend on the calibration panel being visible.
|
||||||
|
|
||||||
|
The window keeps the buttons, the menus and the status messages; the decisions
|
||||||
|
about *what* is recorded, *how* it is named and *what actually happened* live
|
||||||
|
here, where they can be tested without a display.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable, Optional
|
||||||
|
|
||||||
|
from orisys_app_shared.record import ExperimentRecorder, RecordConfig, RecordReport
|
||||||
|
from orisys_app_shared.record.calibration_snapshot import store_calibration_snapshot
|
||||||
|
from orisys_app_shared.record.sample import to_record_sample
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingCoordinator:
|
||||||
|
"""Owns the recorder and the per-packet fan-out."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
session_store,
|
||||||
|
cal_recorder,
|
||||||
|
recorder: Optional[ExperimentRecorder] = None,
|
||||||
|
) -> None:
|
||||||
|
self.session_store = session_store
|
||||||
|
self.cal_recorder = cal_recorder
|
||||||
|
self.recorder = recorder or ExperimentRecorder()
|
||||||
|
#: Confirmed calibration changes: ``(path, version, frame_id, mono_t, session)``.
|
||||||
|
self._calibration_events: list[tuple[str, str, int, float, int]] = []
|
||||||
|
#: Confirmed zeroings: ``(frame_id, mono_t, session)``.
|
||||||
|
self._reset_events: list[tuple[int, float, int]] = []
|
||||||
|
#: Raised by the writer thread; drained on the UI thread.
|
||||||
|
self._alarms: list[str] = []
|
||||||
|
#: A stop was requested and the writer has not closed its files yet.
|
||||||
|
self.report_pending = False
|
||||||
|
self.last_report: Optional[RecordReport] = None
|
||||||
|
self._active_calibration_id = ""
|
||||||
|
self._record_fps = 30.0
|
||||||
|
self.recorder.set_alarm_handler(self._on_alarm)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------- acquisition
|
||||||
|
def on_packet(self, packet):
|
||||||
|
"""Runs on the **acquisition thread**. Must stay cheap and Qt-free.
|
||||||
|
|
||||||
|
The packet is converted exactly once, by the same function the CSV writer
|
||||||
|
uses, so the store and the file can never disagree about whether a value
|
||||||
|
is a measurement. Returns the converted record, which the caller may use
|
||||||
|
for its own (UI-side) bookkeeping.
|
||||||
|
"""
|
||||||
|
record = to_record_sample(
|
||||||
|
packet, calibration_version=self._active_calibration_id
|
||||||
|
)
|
||||||
|
from orisys_app_shared.force.session import SessionSample
|
||||||
|
|
||||||
|
sample = SessionSample.from_record(record)
|
||||||
|
self.session_store.append(sample)
|
||||||
|
self.cal_recorder.feed(sample)
|
||||||
|
# Video + CSV + metadata share this one record path.
|
||||||
|
self.recorder.on_packet(packet)
|
||||||
|
fps = float(getattr(packet, "fps", 0.0) or 0.0)
|
||||||
|
if fps > 0.0:
|
||||||
|
self._record_fps = fps
|
||||||
|
return record
|
||||||
|
|
||||||
|
def take_preroll(self, worker) -> list:
|
||||||
|
"""Frame history from the active worker, if it keeps one."""
|
||||||
|
take = getattr(worker, "take_preroll", None)
|
||||||
|
if not callable(take):
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
return list(take())
|
||||||
|
except Exception: # noqa: BLE001 - the history is optional, not the record
|
||||||
|
logger.exception("Could not take the pre-roll buffer")
|
||||||
|
return []
|
||||||
|
|
||||||
|
def set_active_calibration(self, version: str) -> None:
|
||||||
|
"""The calibration the record should name until the device says otherwise."""
|
||||||
|
self._active_calibration_id = str(version or "")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def active_calibration(self) -> str:
|
||||||
|
return self._active_calibration_id
|
||||||
|
|
||||||
|
@property
|
||||||
|
def record_fps(self) -> float:
|
||||||
|
return self._record_fps
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ events
|
||||||
|
def note_calibration_applied(
|
||||||
|
self, path: str, version: str, frame_id: int, mono_t: float, session: int
|
||||||
|
) -> None:
|
||||||
|
"""Called from the acquisition thread once coefficients are loaded."""
|
||||||
|
self._calibration_events.append(
|
||||||
|
(str(path), str(version), int(frame_id), float(mono_t), int(session))
|
||||||
|
)
|
||||||
|
|
||||||
|
def note_reset(self, frame_id: int, mono_t: float, session: int) -> None:
|
||||||
|
"""Called from the acquisition thread once a zeroing took effect."""
|
||||||
|
self._reset_events.append((int(frame_id), float(mono_t), int(session)))
|
||||||
|
|
||||||
|
def drain_calibration_events(self) -> list[dict]:
|
||||||
|
"""Publish confirmed calibrations. Returns what the UI should show."""
|
||||||
|
published: list[dict] = []
|
||||||
|
while self._calibration_events:
|
||||||
|
path, version, frame_id, mono_t, session = self._calibration_events.pop(0)
|
||||||
|
# The version is the calibration's content hash: a path can be
|
||||||
|
# re-exported with different coefficients, a hash cannot.
|
||||||
|
self._active_calibration_id = version
|
||||||
|
self.session_store.set_calibration_id(version)
|
||||||
|
self.recorder.set_calibration_version(version)
|
||||||
|
snapshot = self.snapshot_calibration(path, version, mono_t, frame_id)
|
||||||
|
self.recorder.log_event(
|
||||||
|
"calibration_applied",
|
||||||
|
f"{path} ({version[:12] or 'unhashed'})"
|
||||||
|
+ (f" -> {snapshot}" if snapshot else ""),
|
||||||
|
mono_t=mono_t,
|
||||||
|
frame_id=frame_id,
|
||||||
|
)
|
||||||
|
published.append(
|
||||||
|
{
|
||||||
|
"path": path,
|
||||||
|
"version": version,
|
||||||
|
"frame_id": frame_id,
|
||||||
|
"mono_t": mono_t,
|
||||||
|
"session": session,
|
||||||
|
"snapshot": snapshot,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return published
|
||||||
|
|
||||||
|
def drain_reset_events(self) -> list[dict]:
|
||||||
|
"""Publish confirmed zeroings."""
|
||||||
|
published: list[dict] = []
|
||||||
|
while self._reset_events:
|
||||||
|
frame_id, mono_t, session = self._reset_events.pop(0)
|
||||||
|
self.recorder.log_event(
|
||||||
|
"zeroing",
|
||||||
|
"tracker reset (applied by the acquisition thread)",
|
||||||
|
mono_t=mono_t,
|
||||||
|
frame_id=frame_id,
|
||||||
|
)
|
||||||
|
published.append(
|
||||||
|
{"frame_id": frame_id, "mono_t": mono_t, "session": session}
|
||||||
|
)
|
||||||
|
return published
|
||||||
|
|
||||||
|
def snapshot_calibration(
|
||||||
|
self, path: str, version: str, mono_t: float, frame_id: int
|
||||||
|
) -> str:
|
||||||
|
"""Copy the applied coefficients into the recording. Returns its name."""
|
||||||
|
directory = self.recorder.directory
|
||||||
|
if directory is None or not path or not Path(path).is_file():
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
name = store_calibration_snapshot(directory, Path(path), version)
|
||||||
|
except Exception: # noqa: BLE001 - the record continues without it
|
||||||
|
logger.exception("Could not snapshot the applied calibration")
|
||||||
|
return ""
|
||||||
|
self.recorder.note_calibration_snapshot(
|
||||||
|
version=version, name=name, frame_id=frame_id, mono_t=mono_t
|
||||||
|
)
|
||||||
|
return name
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ alarms
|
||||||
|
def _on_alarm(self, message: str) -> None:
|
||||||
|
"""Called from the writer thread. Queue it; never touch Qt here."""
|
||||||
|
logger.error("Recording alarm: %s", message)
|
||||||
|
self._alarms.append(str(message))
|
||||||
|
|
||||||
|
def drain_alarms(self) -> list[str]:
|
||||||
|
drained, self._alarms = self._alarms, []
|
||||||
|
return drained
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- lifecycle
|
||||||
|
def start(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
root: Path,
|
||||||
|
config: RecordConfig,
|
||||||
|
metadata: dict[str, Any],
|
||||||
|
calibration_id: str = "",
|
||||||
|
calibration_version: str = "",
|
||||||
|
preroll=(),
|
||||||
|
) -> Path:
|
||||||
|
"""Start a recording. Raises when the directory cannot be created."""
|
||||||
|
directory = self.recorder.start(
|
||||||
|
root=root,
|
||||||
|
session_id=int(self.session_store.session_id),
|
||||||
|
metadata=metadata,
|
||||||
|
calibration_id=calibration_id,
|
||||||
|
calibration_version=calibration_version,
|
||||||
|
preroll=preroll,
|
||||||
|
)
|
||||||
|
self.last_report = None
|
||||||
|
self.report_pending = False
|
||||||
|
return Path(directory)
|
||||||
|
|
||||||
|
def stop(self, *, wait: bool = True) -> Optional[RecordReport]:
|
||||||
|
"""Stop and return the report, or ``None`` when nothing was started."""
|
||||||
|
if self.recorder.directory is None and self.recorder.report is None:
|
||||||
|
self.report_pending = False
|
||||||
|
return None
|
||||||
|
if self.recorder.is_recording:
|
||||||
|
self.recorder.log_event("record_stopped", "")
|
||||||
|
report = self.recorder.stop(wait=wait)
|
||||||
|
self.publish(report)
|
||||||
|
return report
|
||||||
|
|
||||||
|
def publish(self, report: Optional[RecordReport]) -> None:
|
||||||
|
"""Record a report; a completed one becomes ``last_report``."""
|
||||||
|
if report is None:
|
||||||
|
return
|
||||||
|
if not report.completed:
|
||||||
|
# The writer is still closing; the UI timer publishes the real one.
|
||||||
|
self.report_pending = True
|
||||||
|
return
|
||||||
|
self.report_pending = False
|
||||||
|
self.last_report = report
|
||||||
|
|
||||||
|
def poll(self) -> Optional[RecordReport]:
|
||||||
|
"""The terminal report, once the writer has actually closed the files."""
|
||||||
|
if not self.report_pending:
|
||||||
|
return None
|
||||||
|
report = self.recorder.report
|
||||||
|
if report is None:
|
||||||
|
return None
|
||||||
|
self.publish(report)
|
||||||
|
return report
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_recording(self) -> bool:
|
||||||
|
return self.recorder.is_recording
|
||||||
|
|
||||||
|
@property
|
||||||
|
def directory(self):
|
||||||
|
return self.recorder.directory
|
||||||
|
|
||||||
|
|
||||||
|
def record_config_from_prefs(
|
||||||
|
prefs: dict,
|
||||||
|
*,
|
||||||
|
mode: str,
|
||||||
|
fps: float,
|
||||||
|
on_problem: Optional[Callable[[str], None]] = None,
|
||||||
|
) -> RecordConfig:
|
||||||
|
"""Build the record config the toolbar/prefs describe."""
|
||||||
|
return RecordConfig(
|
||||||
|
video=(mode == "video_data"),
|
||||||
|
lossless_frames=bool(prefs.get("record_lossless_frames", False)),
|
||||||
|
fps=float(fps),
|
||||||
|
segment_seconds=float(prefs.get("record_segment_seconds", 300.0)),
|
||||||
|
)
|
||||||
@@ -30,18 +30,16 @@ from orisys_app_shared.force.calibration_io import (
|
|||||||
CalSample,
|
CalSample,
|
||||||
CalSessionStore,
|
CalSessionStore,
|
||||||
ChannelKey,
|
ChannelKey,
|
||||||
export_force_calibration_v2,
|
|
||||||
meta_from_sensor_json,
|
meta_from_sensor_json,
|
||||||
)
|
)
|
||||||
from orisys_app_shared.force.calibration_session import (
|
from orisys_app_shared.force.calibration_session import (
|
||||||
read_calibration_session,
|
|
||||||
session_filename,
|
session_filename,
|
||||||
write_calibration_session,
|
|
||||||
)
|
)
|
||||||
from orisys_app_shared.i18n import t
|
from orisys_app_shared.i18n import t
|
||||||
from orisys_app_shared.paths import calibration_export_dir
|
from orisys_app_shared.paths import calibration_export_dir
|
||||||
from orisys_app_shared.theme import color
|
from orisys_app_shared.theme import color
|
||||||
|
|
||||||
|
from orisys_studio.calibration_service import CalibrationService
|
||||||
from orisys_studio.ui.cal_point_dialog import CalibrationPointDialog
|
from orisys_studio.ui.cal_point_dialog import CalibrationPointDialog
|
||||||
|
|
||||||
CHANNEL_LABELS = ("fn", "fx", "fy")
|
CHANNEL_LABELS = ("fn", "fx", "fy")
|
||||||
@@ -71,7 +69,10 @@ class CalPanel(QFrame):
|
|||||||
super().__init__(parent)
|
super().__init__(parent)
|
||||||
self.setObjectName("contentPanel")
|
self.setObjectName("contentPanel")
|
||||||
self._theme = theme
|
self._theme = theme
|
||||||
self._store = CalSessionStore()
|
#: The files, the identity and the prepared model live in the service;
|
||||||
|
#: this panel shows them and emits status messages.
|
||||||
|
self._service = CalibrationService()
|
||||||
|
self._store = self._service.store
|
||||||
self._channel: ChannelKey = "fn"
|
self._channel: ChannelKey = "fn"
|
||||||
self._running = False
|
self._running = False
|
||||||
#: Set by :meth:`bind_recorder`. Sampling lives there, not here, so the
|
#: Set by :meth:`bind_recorder`. Sampling lives there, not here, so the
|
||||||
@@ -244,6 +245,14 @@ class CalPanel(QFrame):
|
|||||||
self.retranslate()
|
self.retranslate()
|
||||||
|
|
||||||
# ------------------------------------------------------------------ API
|
# ------------------------------------------------------------------ API
|
||||||
|
@property
|
||||||
|
def _prepared(self):
|
||||||
|
return self._service.prepared
|
||||||
|
|
||||||
|
@property
|
||||||
|
def _record_identity(self):
|
||||||
|
return self._service.record_identity
|
||||||
|
|
||||||
def set_theme(self, theme: str) -> None:
|
def set_theme(self, theme: str) -> None:
|
||||||
self._theme = theme
|
self._theme = theme
|
||||||
self._render_live_label()
|
self._render_live_label()
|
||||||
@@ -266,6 +275,8 @@ class CalPanel(QFrame):
|
|||||||
def set_meta_context(self, *, sensor_type: str, config_ref: str | None) -> None:
|
def set_meta_context(self, *, sensor_type: str, config_ref: str | None) -> None:
|
||||||
self._sensor_type = str(sensor_type or "box")
|
self._sensor_type = str(sensor_type or "box")
|
||||||
self._config_ref = config_ref
|
self._config_ref = config_ref
|
||||||
|
# A new run is the live device's context again.
|
||||||
|
self._service.clear_record_identity()
|
||||||
|
|
||||||
def set_live(self, fn: float, fx: float, fy: float) -> None:
|
def set_live(self, fn: float, fx: float, fy: float) -> None:
|
||||||
"""Update the live readout. Sampling is handled by the recorder."""
|
"""Update the live readout. Sampling is handled by the recorder."""
|
||||||
@@ -519,7 +530,8 @@ class CalPanel(QFrame):
|
|||||||
if self._recorder is None:
|
if self._recorder is None:
|
||||||
self.fit_label.setText("")
|
self.fit_label.setText("")
|
||||||
return
|
return
|
||||||
fit = self._recorder.fit(self._channel)
|
# Same prepared result the store and the export use — never a second one.
|
||||||
|
fit = self._recorder.fit(self._channel, prepared=self._prepared)
|
||||||
if not fit.usable:
|
if not fit.usable:
|
||||||
self.fit_label.setText(t("cal.fit_insufficient", count=fit.n_points))
|
self.fit_label.setText(t("cal.fit_insufficient", count=fit.n_points))
|
||||||
return
|
return
|
||||||
@@ -549,12 +561,30 @@ class CalPanel(QFrame):
|
|||||||
state = t("cal.state_restore") if point.excluded else t("cal.state_exclude")
|
state = t("cal.state_restore") if point.excluded else t("cal.state_exclude")
|
||||||
self.btn_toggle_excluded.setText(state)
|
self.btn_toggle_excluded.setText(state)
|
||||||
|
|
||||||
|
def point_dialog(self, point):
|
||||||
|
"""The diagnostic view for a point, rebuilt from its samples if needed.
|
||||||
|
|
||||||
|
A reopened record carries the full sample series, so the three-axis view
|
||||||
|
can always be produced — waiting for a cached trace would have made
|
||||||
|
exactly the reopened points (the ones worth auditing) unopenable.
|
||||||
|
"""
|
||||||
|
if point is None or not (point.trace or point.samples):
|
||||||
|
return None
|
||||||
|
if not point.trace:
|
||||||
|
from orisys_app_shared.force.calibration_session import (
|
||||||
|
restore_diagnostics,
|
||||||
|
)
|
||||||
|
|
||||||
|
restore_diagnostics(point)
|
||||||
|
if not point.trace:
|
||||||
|
return None
|
||||||
|
return CalibrationPointDialog(point, theme=self._theme, parent=self)
|
||||||
|
|
||||||
def _on_row_activated(self) -> None:
|
def _on_row_activated(self) -> None:
|
||||||
"""Double click a row to inspect that point's three-axis curve."""
|
"""Double click a row to inspect that point's three-axis curve."""
|
||||||
point = self._current_point()
|
dialog = self.point_dialog(self._current_point())
|
||||||
if point is None or not point.trace:
|
if dialog is not None:
|
||||||
return
|
dialog.exec_()
|
||||||
CalibrationPointDialog(point, theme=self._theme, parent=self).exec_()
|
|
||||||
|
|
||||||
def _on_toggle_excluded(self) -> None:
|
def _on_toggle_excluded(self) -> None:
|
||||||
point = self._current_point()
|
point = self._current_point()
|
||||||
@@ -580,33 +610,25 @@ class CalPanel(QFrame):
|
|||||||
t("cal.marked_validation" if point.is_validation else "cal.unmarked_validation")
|
t("cal.marked_validation" if point.is_validation else "cal.unmarked_validation")
|
||||||
)
|
)
|
||||||
|
|
||||||
def _rebuild_store(self) -> None:
|
def _export_meta(self) -> dict:
|
||||||
"""Rebuild the export store from the points that are still in the fit.
|
"""The identity/pipeline metadata the export would carry."""
|
||||||
|
return self._service.export_meta(
|
||||||
Validation points are held out: they exist to check the model, so letting
|
sensor_type=self._sensor_type,
|
||||||
them into the build would both corrupt it and make the "independent"
|
config_ref=self._config_ref,
|
||||||
error meaningless. Repeated reference loads are merged rather than
|
sn=self.sn_edit.text().strip(),
|
||||||
overwritten, so every sample that was actually taken reaches the model.
|
duration_s=float(self.duration_spin.value()),
|
||||||
"""
|
|
||||||
self._store.clear()
|
|
||||||
if self._recorder is None:
|
|
||||||
return
|
|
||||||
for point in self._recorder.all_points():
|
|
||||||
if point.excluded or point.is_validation:
|
|
||||||
continue
|
|
||||||
stats = point.loaded_stats()
|
|
||||||
if not np.isfinite(stats.mean):
|
|
||||||
continue
|
|
||||||
self._store.aggregate(
|
|
||||||
point.channel,
|
|
||||||
CalSample(
|
|
||||||
weight_n=float(point.reference_load_n),
|
|
||||||
raw_mean=float(stats.mean),
|
|
||||||
n=int(stats.n),
|
|
||||||
std=float(stats.std) if np.isfinite(stats.std) else 0.0,
|
|
||||||
),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _rebuild_store(self) -> None:
|
||||||
|
"""Prepare the session once: store, diagnostics and export share it.
|
||||||
|
|
||||||
|
The panel used to filter, aggregate and build its own model while
|
||||||
|
``CalibrationRecorder.fit`` prepared a second one for the table, so the
|
||||||
|
displayed residual and the exported curve could describe different
|
||||||
|
models. Both now read :class:`PreparedCalibration`, built by the service.
|
||||||
|
"""
|
||||||
|
self._service.prepare(self._recorder, self._export_meta())
|
||||||
|
|
||||||
def _on_delete_row(self) -> None:
|
def _on_delete_row(self) -> None:
|
||||||
point = self._current_point()
|
point = self._current_point()
|
||||||
if point is None or self._recorder is None:
|
if point is None or self._recorder is None:
|
||||||
@@ -630,20 +652,24 @@ class CalPanel(QFrame):
|
|||||||
self._refresh_table()
|
self._refresh_table()
|
||||||
|
|
||||||
def _on_export(self) -> None:
|
def _on_export(self) -> None:
|
||||||
meta = meta_from_sensor_json(self._config_ref)
|
# Re-prepare first: the SN and the requested duration may have changed,
|
||||||
meta["sensor_type"] = self._sensor_type
|
# and the file must be exactly the model the panel is showing.
|
||||||
meta["sn"] = self.sn_edit.text().strip()
|
self._rebuild_store()
|
||||||
meta["record_duration_s"] = float(self.duration_spin.value())
|
self._refresh_fit_label()
|
||||||
|
if self._prepared is None or not self._prepared.ok:
|
||||||
|
QMessageBox.warning(
|
||||||
|
self,
|
||||||
|
t("panels.cal"),
|
||||||
|
t("cal.export_no_model", detail="; ".join(self._warnings())),
|
||||||
|
)
|
||||||
|
return
|
||||||
try:
|
try:
|
||||||
# Write into the user's calibration directory only. Overwriting the
|
# Write into the user's calibration directory only. Overwriting the
|
||||||
# device bundle under config/sensors/<type>/ is now an explicit,
|
# device bundle under config/sensors/<type>/ is now an explicit,
|
||||||
# separate step so that exporting cannot silently change what the
|
# separate step so that exporting cannot silently change what the
|
||||||
# next Sensor() will load.
|
# next Sensor() will load.
|
||||||
primary, _local_path, warnings = export_force_calibration_v2(
|
primary, _local_path, warnings = self._service.export_coefficients(
|
||||||
self._store,
|
|
||||||
export_dir=calibration_export_dir(),
|
|
||||||
sensor_type=self._sensor_type,
|
sensor_type=self._sensor_type,
|
||||||
meta=meta,
|
|
||||||
local_bundle_dir=None,
|
local_bundle_dir=None,
|
||||||
)
|
)
|
||||||
except ValueError as exc:
|
except ValueError as exc:
|
||||||
@@ -664,6 +690,9 @@ class CalPanel(QFrame):
|
|||||||
self.calibration_exported.emit(str(primary))
|
self.calibration_exported.emit(str(primary))
|
||||||
QMessageBox.information(self, t("panels.cal"), msg)
|
QMessageBox.information(self, t("panels.cal"), msg)
|
||||||
|
|
||||||
|
def _warnings(self) -> list[str]:
|
||||||
|
return list(getattr(self._prepared, "warnings", []) or [])
|
||||||
|
|
||||||
def _on_apply(self) -> None:
|
def _on_apply(self) -> None:
|
||||||
if not self._last_export_path:
|
if not self._last_export_path:
|
||||||
self.status_message.emit(t("cal.apply_no_export"))
|
self.status_message.emit(t("cal.apply_no_export"))
|
||||||
@@ -678,12 +707,10 @@ class CalPanel(QFrame):
|
|||||||
if not points:
|
if not points:
|
||||||
self.status_message.emit(t("cal.save_record_empty"))
|
self.status_message.emit(t("cal.save_record_empty"))
|
||||||
return
|
return
|
||||||
meta = meta_from_sensor_json(self._config_ref)
|
# ``_export_meta`` keeps a reopened record's own device identity.
|
||||||
meta["sensor_type"] = self._sensor_type
|
meta = self._export_meta()
|
||||||
meta["sn"] = self.sn_edit.text().strip()
|
|
||||||
path = calibration_export_dir() / session_filename(self._sensor_type)
|
|
||||||
try:
|
try:
|
||||||
written = write_calibration_session(path, points, meta=meta)
|
written = self._service.save_record(self._recorder, meta)
|
||||||
except Exception as exc: # noqa: BLE001 - reported to the operator
|
except Exception as exc: # noqa: BLE001 - reported to the operator
|
||||||
QMessageBox.warning(
|
QMessageBox.warning(
|
||||||
self, t("panels.cal"), t("cal.save_record_failed", error=str(exc))
|
self, t("panels.cal"), t("cal.save_record_failed", error=str(exc))
|
||||||
@@ -694,10 +721,11 @@ class CalPanel(QFrame):
|
|||||||
self.status_message.emit(msg)
|
self.status_message.emit(msg)
|
||||||
QMessageBox.information(self, t("panels.cal"), msg)
|
QMessageBox.information(self, t("panels.cal"), msg)
|
||||||
|
|
||||||
def _on_open_record(self) -> None:
|
def _on_open_record(self, path: str | None = None) -> None:
|
||||||
"""Reopen a saved session so it can be inspected again."""
|
"""Reopen a saved session: its points **and** its device identity."""
|
||||||
if self._recorder is None:
|
if self._recorder is None:
|
||||||
return
|
return
|
||||||
|
if path is None:
|
||||||
path, _filter = QFileDialog.getOpenFileName(
|
path, _filter = QFileDialog.getOpenFileName(
|
||||||
self,
|
self,
|
||||||
t("cal.open_record"),
|
t("cal.open_record"),
|
||||||
@@ -707,17 +735,22 @@ class CalPanel(QFrame):
|
|||||||
if not path:
|
if not path:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
record = read_calibration_session(path)
|
record = self._service.load_record(path)
|
||||||
except Exception as exc: # noqa: BLE001 - reported to the operator
|
except Exception as exc: # noqa: BLE001 - reported to the operator
|
||||||
QMessageBox.warning(
|
QMessageBox.warning(
|
||||||
self, t("panels.cal"), t("cal.open_record_failed", error=str(exc))
|
self, t("panels.cal"), t("cal.open_record_failed", error=str(exc))
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
# Show the record's own SN: it is what a later save or export will carry,
|
||||||
|
# so the operator must not be looking at the connected device's number.
|
||||||
|
if record.identity().get("sn"):
|
||||||
|
self.sn_edit.setText(record.identity()["sn"])
|
||||||
self._recorder.load_points(record.points, replace=True)
|
self._recorder.load_points(record.points, replace=True)
|
||||||
self._rebuild_store()
|
self._rebuild_store()
|
||||||
self._refresh_table()
|
self._refresh_table()
|
||||||
self._refresh_enabled()
|
self._refresh_enabled()
|
||||||
msg = t("cal.open_record_ok", path=record.path.name, points=len(record.points))
|
msg = t("cal.open_record_ok", path=record.path.name, points=len(record.points))
|
||||||
|
msg += " " + t("cal.open_record_device", device=record.describe())
|
||||||
if record.warnings:
|
if record.warnings:
|
||||||
msg += "\n" + t("cal.open_record_warnings", detail="; ".join(record.warnings))
|
msg += "\n" + t("cal.open_record_warnings", detail="; ".join(record.warnings))
|
||||||
self.status_message.emit(msg)
|
self.status_message.emit(msg)
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ from PyQt5.QtWidgets import (
|
|||||||
from orisys_app_shared.acquisition.constants import FORCE_SAMPLE_RATE_HZ, LOWPASS_CUTOFF_HZ
|
from orisys_app_shared.acquisition.constants import FORCE_SAMPLE_RATE_HZ, LOWPASS_CUTOFF_HZ
|
||||||
from orisys_app_shared.force import MetricHistory
|
from orisys_app_shared.force import MetricHistory
|
||||||
from orisys_app_shared.force.history import DEFAULT_MAX_FPS
|
from orisys_app_shared.force.history import DEFAULT_MAX_FPS
|
||||||
|
from orisys_app_shared.force.source import StoreCurveSource
|
||||||
from orisys_app_shared.i18n import t
|
from orisys_app_shared.i18n import t
|
||||||
from orisys_app_shared.theme import color
|
from orisys_app_shared.theme import color
|
||||||
|
|
||||||
@@ -100,8 +101,13 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self._fixed_newton = True
|
self._fixed_newton = True
|
||||||
self._unit = "raw"
|
self._unit = "raw"
|
||||||
self._last_values: tuple[float, float, float] = (0.0, 0.0, 0.0)
|
self._last_values: tuple[float, float, float] = (0.0, 0.0, 0.0)
|
||||||
|
#: Per axis: does ``_last_values`` hold a real value in the active unit?
|
||||||
|
self._available: tuple[bool, bool, bool] = (True, True, True)
|
||||||
# -- browsing state -------------------------------------------------
|
# -- browsing state -------------------------------------------------
|
||||||
self._store = None
|
self._store = None
|
||||||
|
#: Where the curves read from: the in-memory session store here, a
|
||||||
|
#: recording on disk in the replay. One interface either way.
|
||||||
|
self._source = StoreCurveSource()
|
||||||
self._mode = "follow"
|
self._mode = "follow"
|
||||||
self._window: tuple[float, float] = (0.0, 0.0)
|
self._window: tuple[float, float] = (0.0, 0.0)
|
||||||
self._cursor_t = 0.0
|
self._cursor_t = 0.0
|
||||||
@@ -313,6 +319,7 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self.history.reset()
|
self.history.reset()
|
||||||
self._clear_curves()
|
self._clear_curves()
|
||||||
self._last_values = (0.0, 0.0, 0.0)
|
self._last_values = (0.0, 0.0, 0.0)
|
||||||
|
self._available = (True, True, True)
|
||||||
self._unit = "raw"
|
self._unit = "raw"
|
||||||
self._mode = "follow"
|
self._mode = "follow"
|
||||||
self._window = (0.0, 0.0)
|
self._window = (0.0, 0.0)
|
||||||
@@ -328,19 +335,33 @@ class ForcePlotsPanel(QFrame):
|
|||||||
|
|
||||||
def push(
|
def push(
|
||||||
self,
|
self,
|
||||||
timestamp: float,
|
t_rel_s: float,
|
||||||
fn: float,
|
fn: float,
|
||||||
fx: float,
|
fx: float,
|
||||||
fy: float,
|
fy: float,
|
||||||
*,
|
*,
|
||||||
unit: str = "raw",
|
unit: str = "raw",
|
||||||
|
available: tuple[bool, bool, bool] | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
|
"""Append one live sample.
|
||||||
|
|
||||||
|
``t_rel_s`` is **session-relative** time — the same base the session store
|
||||||
|
and ``forces.csv`` use. Handing the wall clock in here used to put the live
|
||||||
|
buffer on a different time axis from a rebuilt history, so switching units
|
||||||
|
(or entering browse) made the X axis jump by the size of the epoch.
|
||||||
|
|
||||||
|
``available`` marks axes that have no value in ``unit`` (a device with
|
||||||
|
only its normal-force axis calibrated). Those keep NaN and show as a gap
|
||||||
|
and a dash, never as a raw count wearing a newton label.
|
||||||
|
"""
|
||||||
if unit != self._unit:
|
if unit != self._unit:
|
||||||
# Never mix units in one buffer. Callers switch units through
|
# Never mix units in one buffer. Callers switch units through
|
||||||
# ``set_units``/``set_fixed_newton`` which rebuild from the session
|
# ``set_units``/``set_fixed_newton`` which rebuild from the session
|
||||||
# store; a bare change here would silently reinterpret old samples.
|
# store; a bare change here would silently reinterpret old samples.
|
||||||
self._unit = unit
|
self._unit = unit
|
||||||
self._last_values = (float(fn), float(fx), float(fy))
|
self._last_values = (float(fn), float(fx), float(fy))
|
||||||
|
if available is not None:
|
||||||
|
self._available = tuple(bool(flag) for flag in available)
|
||||||
self._render_value_cards()
|
self._render_value_cards()
|
||||||
if self._mode == "browse":
|
if self._mode == "browse":
|
||||||
# Browsing is a display pause, not a capture pause: the record
|
# Browsing is a display pause, not a capture pause: the record
|
||||||
@@ -351,7 +372,7 @@ class ForcePlotsPanel(QFrame):
|
|||||||
# Store the true value. Clamping here (the old ``max(0.0, fn)``) erased
|
# Store the true value. Clamping here (the old ``max(0.0, fn)``) erased
|
||||||
# negative FN from the buffer and hid zero drift; out-of-range values are
|
# negative FN from the buffer and hid zero drift; out-of-range values are
|
||||||
# now handled by the axis range, which only affects what is drawn.
|
# now handled by the axis range, which only affects what is drawn.
|
||||||
self.history.push(timestamp, float(fn), float(fx), float(fy))
|
self.history.push(t_rel_s, float(fn), float(fx), float(fy))
|
||||||
state = self.history.state()
|
state = self.history.state()
|
||||||
if state is None:
|
if state is None:
|
||||||
return
|
return
|
||||||
@@ -372,7 +393,9 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self.plots[i].setYRange(*state.y_ranges[i], padding=0.0)
|
self.plots[i].setYRange(*state.y_ranges[i], padding=0.0)
|
||||||
finally:
|
finally:
|
||||||
self._syncing = False
|
self._syncing = False
|
||||||
self.set_cursor(float(timestamp) - (self._time_origin() or 0.0))
|
# The buffer already carries session-relative time, so the cursor is the
|
||||||
|
# time itself: subtracting an origin here would double-count it.
|
||||||
|
self.set_cursor(float(t_rel_s))
|
||||||
# The overview is a full-store read, so refresh it on a timer rather
|
# The overview is a full-store read, so refresh it on a timer rather
|
||||||
# than on every frame.
|
# than on every frame.
|
||||||
now = time.perf_counter()
|
now = time.perf_counter()
|
||||||
@@ -381,12 +404,6 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self._last_overview_at = now
|
self._last_overview_at = now
|
||||||
self._sync_overview()
|
self._sync_overview()
|
||||||
|
|
||||||
def _time_origin(self) -> float:
|
|
||||||
"""The store's time origin, so the cursor matches the overview axis."""
|
|
||||||
if self._store is None:
|
|
||||||
return 0.0
|
|
||||||
return self._store.origin_time()
|
|
||||||
|
|
||||||
def set_units(self, unit: str, store=None) -> None:
|
def set_units(self, unit: str, store=None) -> None:
|
||||||
"""Switch display units without mixing units in the curve buffer.
|
"""Switch display units without mixing units in the curve buffer.
|
||||||
|
|
||||||
@@ -398,11 +415,13 @@ class ForcePlotsPanel(QFrame):
|
|||||||
if unit == self._unit and store is None:
|
if unit == self._unit and store is None:
|
||||||
return
|
return
|
||||||
self._unit = unit
|
self._unit = unit
|
||||||
|
# ``clear`` keeps the buffer's time origin on purpose: the rebuilt history
|
||||||
|
# is in the same session-relative base, so the X axis must not restart.
|
||||||
self.history.clear()
|
self.history.clear()
|
||||||
self._clear_curves()
|
self._clear_curves()
|
||||||
self._overview_dirty = True
|
self._overview_dirty = True
|
||||||
if store is not None:
|
if store is not None:
|
||||||
self._store = store
|
self.set_store(store)
|
||||||
if self._mode == "browse":
|
if self._mode == "browse":
|
||||||
self.refresh_from_store()
|
self.refresh_from_store()
|
||||||
else:
|
else:
|
||||||
@@ -414,6 +433,7 @@ class ForcePlotsPanel(QFrame):
|
|||||||
def set_store(self, store) -> None:
|
def set_store(self, store) -> None:
|
||||||
"""Attach the whole-session store the browser reads from."""
|
"""Attach the whole-session store the browser reads from."""
|
||||||
self._store = store
|
self._store = store
|
||||||
|
self._source.set_store(store)
|
||||||
self._overview_dirty = True
|
self._overview_dirty = True
|
||||||
self._refresh_mode_label()
|
self._refresh_mode_label()
|
||||||
|
|
||||||
@@ -437,19 +457,19 @@ class ForcePlotsPanel(QFrame):
|
|||||||
|
|
||||||
def show_all(self) -> None:
|
def show_all(self) -> None:
|
||||||
"""Overview: fit the entire session, in browse mode."""
|
"""Overview: fit the entire session, in browse mode."""
|
||||||
if self._store is None:
|
if self._source is None:
|
||||||
return
|
return
|
||||||
first, last = self._store.extent()
|
first, last = self._source.extent()
|
||||||
if last <= first:
|
if last <= first:
|
||||||
return
|
return
|
||||||
self._set_mode("browse")
|
self._set_mode("browse")
|
||||||
self.set_window(first, last)
|
self.set_window(first, last)
|
||||||
|
|
||||||
def set_window(self, t0: float, t1: float) -> None:
|
def set_window(self, t0: float, t1: float) -> None:
|
||||||
"""Show ``[t0, t1]`` from the whole-session store."""
|
"""Show ``[t0, t1]`` from the whole-session source."""
|
||||||
if self._store is None:
|
if self._source is None:
|
||||||
return
|
return
|
||||||
first, last = self._store.extent()
|
first, last = self._source.extent()
|
||||||
if t1 <= t0:
|
if t1 <= t0:
|
||||||
t1 = t0 + max(0.001, self.history.window_sec)
|
t1 = t0 + max(0.001, self.history.window_sec)
|
||||||
t0 = max(first, float(t0))
|
t0 = max(first, float(t0))
|
||||||
@@ -463,21 +483,23 @@ class ForcePlotsPanel(QFrame):
|
|||||||
if self._store is None:
|
if self._store is None:
|
||||||
return
|
return
|
||||||
t0, t1 = self._window
|
t0, t1 = self._window
|
||||||
slice_ = self._store.visible(
|
# One shared rule for every view, decided **per axis**: valid and finite
|
||||||
|
# for *that* axis. An all-axes rule hid a valid FN whenever the device had
|
||||||
|
# no shear calibration, and a finiteness-only rule let a stale value
|
||||||
|
# reappear after another view excluded it.
|
||||||
|
window = self._source.window(
|
||||||
t0, t1, max_points=self._max_draw_points, unit=self._unit
|
t0, t1, max_points=self._max_draw_points, unit=self._unit
|
||||||
)
|
)
|
||||||
self._syncing = True
|
self._syncing = True
|
||||||
try:
|
try:
|
||||||
if len(slice_) == 0:
|
if len(window) == 0:
|
||||||
for curve in self.curves:
|
for curve in self.curves:
|
||||||
curve.setData([], [])
|
curve.setData([], [])
|
||||||
else:
|
else:
|
||||||
# One shared rule for every view: valid *and* finite. A stale
|
|
||||||
# number left by a failed compute step must not reappear here
|
|
||||||
# after another view already excluded it.
|
|
||||||
times, values, _mask = slice_.plottable(self._unit)
|
|
||||||
for index, curve in enumerate(self.curves):
|
for index, curve in enumerate(self.curves):
|
||||||
curve.setData(times, values[:, index])
|
times, values = window.plottable_axis(index)
|
||||||
|
curve.setData(times, values)
|
||||||
|
self._availability_from_window(window)
|
||||||
for plot in self.plots:
|
for plot in self.plots:
|
||||||
plot.setXRange(t0, t1, padding=0.0)
|
plot.setXRange(t0, t1, padding=0.0)
|
||||||
self._apply_axis_mode()
|
self._apply_axis_mode()
|
||||||
@@ -523,7 +545,7 @@ class ForcePlotsPanel(QFrame):
|
|||||||
x0, x1 = float(x_range[0]), float(x_range[1])
|
x0, x1 = float(x_range[0]), float(x_range[1])
|
||||||
if not (x1 > x0):
|
if not (x1 > x0):
|
||||||
return
|
return
|
||||||
first, last = self._store.extent()
|
first, last = self._source.extent()
|
||||||
if last <= first:
|
if last <= first:
|
||||||
return
|
return
|
||||||
# Keep the window inside the recorded session.
|
# Keep the window inside the recorded session.
|
||||||
@@ -554,7 +576,7 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self.stats_label.setText("")
|
self.stats_label.setText("")
|
||||||
return
|
return
|
||||||
t0, t1 = self._window
|
t0, t1 = self._window
|
||||||
stats = self._store.statistics(t0, t1, unit=self._unit)
|
stats = self._source.statistics(t0, t1, unit=self._unit)
|
||||||
if stats.samples == 0:
|
if stats.samples == 0:
|
||||||
self.stats_label.setText(t("chart.selection_empty"))
|
self.stats_label.setText(t("chart.selection_empty"))
|
||||||
return
|
return
|
||||||
@@ -581,19 +603,19 @@ class ForcePlotsPanel(QFrame):
|
|||||||
def _sync_overview(self) -> None:
|
def _sync_overview(self) -> None:
|
||||||
if self._store is None:
|
if self._store is None:
|
||||||
return
|
return
|
||||||
first, last = self._store.extent()
|
first, last = self._source.extent()
|
||||||
if last <= first:
|
if last <= first:
|
||||||
self.overview_curve.setData([], [])
|
self.overview_curve.setData([], [])
|
||||||
return
|
return
|
||||||
# The overview is a full-store read, so only rebuild it when it can have
|
# The overview is a whole-session read, so only rebuild it when it can
|
||||||
# changed (window/unit/session) rather than on every pan.
|
# have changed (window/unit/session) rather than on every pan.
|
||||||
slice_ = self._store.visible(
|
window = self._source.envelope(
|
||||||
first, last, max_points=self._overview_points(), unit=self._unit
|
buckets=self._overview_points(), unit=self._unit
|
||||||
)
|
)
|
||||||
values = slice_.value_for_unit(self._unit)
|
values = np.asarray(window.values, dtype=float)
|
||||||
column = values[:, 0]
|
column = values[:, 0]
|
||||||
finite = np.isfinite(column)
|
finite = np.isfinite(column)
|
||||||
self.overview_curve.setData(slice_.times[finite], column[finite])
|
self.overview_curve.setData(window.times[finite], column[finite])
|
||||||
self.overview.setXRange(first, last, padding=0.0)
|
self.overview.setXRange(first, last, padding=0.0)
|
||||||
if self._mode == "browse":
|
if self._mode == "browse":
|
||||||
self.overview_region.setRegion(self._window)
|
self.overview_region.setRegion(self._window)
|
||||||
@@ -616,20 +638,25 @@ class ForcePlotsPanel(QFrame):
|
|||||||
self.btn_overview.setEnabled(self._store is not None)
|
self.btn_overview.setEnabled(self._store is not None)
|
||||||
|
|
||||||
def _reload_from_store(self, store) -> None:
|
def _reload_from_store(self, store) -> None:
|
||||||
"""Re-populate the visible buffer from the whole-session store."""
|
"""Re-populate the visible buffer from the session source."""
|
||||||
window = self.history.window_sec
|
window = self.history.window_sec
|
||||||
slice_ = store.tail(window) if window > 0 else store.data()
|
first, last = self._source.extent()
|
||||||
if len(slice_) == 0:
|
span = window if window > 0 else max(0.0, last - first)
|
||||||
|
data = self._source.window(
|
||||||
|
max(first, last - span), last, unit=self._unit
|
||||||
|
)
|
||||||
|
if len(data) == 0:
|
||||||
return
|
return
|
||||||
values = slice_.value_for_unit(self._unit)
|
self._availability_from_window(data)
|
||||||
for index in range(len(slice_)):
|
values = np.asarray(data.values, dtype=float)
|
||||||
if not bool(slice_.valid[index]):
|
for index in range(len(data)):
|
||||||
|
if not bool(data.valid[index]):
|
||||||
continue
|
continue
|
||||||
row = values[index]
|
row = values[index]
|
||||||
if not all(np.isfinite(v) for v in row):
|
if not all(np.isfinite(v) for v in row):
|
||||||
continue
|
continue
|
||||||
self.history.push(
|
self.history.push(
|
||||||
float(slice_.times[index]),
|
float(data.times[index]),
|
||||||
float(row[0]),
|
float(row[0]),
|
||||||
float(row[1]),
|
float(row[1]),
|
||||||
float(row[2]),
|
float(row[2]),
|
||||||
@@ -641,6 +668,19 @@ class ForcePlotsPanel(QFrame):
|
|||||||
curve.setData(state.times, state.values[:, i])
|
curve.setData(state.times, state.values[:, i])
|
||||||
self.plots[i].setXRange(*state.x_range, padding=0.0)
|
self.plots[i].setXRange(*state.x_range, padding=0.0)
|
||||||
|
|
||||||
|
def _availability_from_window(self, window) -> None:
|
||||||
|
"""Mark which axes have a value in the active unit, from a read window.
|
||||||
|
|
||||||
|
Used after a unit switch: the value cards must not keep showing the
|
||||||
|
previous unit's numbers, and an axis with no curve in this unit has to be
|
||||||
|
a dash until a fresh sample arrives.
|
||||||
|
"""
|
||||||
|
if window is None or len(window) == 0:
|
||||||
|
return
|
||||||
|
values = np.asarray(window.values, dtype=float)
|
||||||
|
finite = np.isfinite(values) & np.asarray(window.valid, dtype=bool)[:, None]
|
||||||
|
self._available = tuple(bool(finite[:, index].any()) for index in range(3))
|
||||||
|
|
||||||
def _uses_fixed_y(self) -> bool:
|
def _uses_fixed_y(self) -> bool:
|
||||||
"""Fixed Y whenever we have limits for the active unit.
|
"""Fixed Y whenever we have limits for the active unit.
|
||||||
|
|
||||||
@@ -690,10 +730,21 @@ class ForcePlotsPanel(QFrame):
|
|||||||
|
|
||||||
def _render_value_cards(self) -> None:
|
def _render_value_cards(self) -> None:
|
||||||
fmt = "{value:.3f}" if self._unit == "N" else "{value:.1f}"
|
fmt = "{value:.3f}" if self._unit == "N" else "{value:.1f}"
|
||||||
for value_label, unit_label, value in zip(
|
unavailable = t("chart.value_unavailable")
|
||||||
self.value_labels, self.value_units, self._last_values
|
for index, (value_label, unit_label, value) in enumerate(
|
||||||
|
zip(self.value_labels, self.value_units, self._last_values)
|
||||||
):
|
):
|
||||||
|
if not self._available[index]:
|
||||||
|
# An axis with no curve in this unit: a dash, never a raw count
|
||||||
|
# presented as a newton value.
|
||||||
|
value_label.setText(unavailable)
|
||||||
|
unit_label.setText("")
|
||||||
|
value_label.setToolTip(
|
||||||
|
t("chart.value_unavailable_tip", channel=CHANNEL_LABELS[index])
|
||||||
|
)
|
||||||
|
continue
|
||||||
value_label.setText(fmt.format(value=value))
|
value_label.setText(fmt.format(value=value))
|
||||||
|
value_label.setToolTip("")
|
||||||
unit_label.setText(unit_display(self._unit))
|
unit_label.setText(unit_display(self._unit))
|
||||||
self.unit_badge.setText(unit_display(self._unit))
|
self.unit_badge.setText(unit_display(self._unit))
|
||||||
|
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ video frame, the depth pane and the charts' cursor line together.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
import threading
|
import threading
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -45,6 +46,10 @@ except ImportError as exc: # pragma: no cover
|
|||||||
|
|
||||||
CHANNEL_TOKENS = ("plot_fn", "plot_fx", "plot_fy")
|
CHANNEL_TOKENS = ("plot_fn", "plot_fx", "plot_fy")
|
||||||
SLIDER_STEPS = 2000
|
SLIDER_STEPS = 2000
|
||||||
|
#: Upper bound on the points handed to the replay curves.
|
||||||
|
CHART_MAX_POINTS = 4000
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class PlaybackDialog(QDialog):
|
class PlaybackDialog(QDialog):
|
||||||
@@ -75,6 +80,13 @@ class PlaybackDialog(QDialog):
|
|||||||
self._depth_pending_seq: int | None = None
|
self._depth_pending_seq: int | None = None
|
||||||
self._depth_pending_position: float | None = None
|
self._depth_pending_position: float | None = None
|
||||||
self._depth_result_seq: int | None = None
|
self._depth_result_seq: int | None = None
|
||||||
|
#: Which recording each answer belongs to (incremented on every open).
|
||||||
|
self._record_serial = 0
|
||||||
|
self._depth_pending_record: int | None = None
|
||||||
|
self._depth_result_record: int | None = None
|
||||||
|
self._depth_result_frame = -1
|
||||||
|
#: Processors whose owning request was still running when it was retired.
|
||||||
|
self._retired_recomputers: list = []
|
||||||
self._depth_poll = QTimer(self)
|
self._depth_poll = QTimer(self)
|
||||||
self._depth_poll.setInterval(120)
|
self._depth_poll.setInterval(120)
|
||||||
self._depth_poll.timeout.connect(self._poll_depth)
|
self._depth_poll.timeout.connect(self._poll_depth)
|
||||||
@@ -166,9 +178,16 @@ class PlaybackDialog(QDialog):
|
|||||||
self.warning_label.setText(t("playback.open_failed", error=str(exc)))
|
self.warning_label.setText(t("playback.open_failed", error=str(exc)))
|
||||||
|
|
||||||
def open_recording(self, directory: Path) -> None:
|
def open_recording(self, directory: Path) -> None:
|
||||||
if self._recomputer is not None:
|
"""Open a recording. Any depth answer still in flight is invalidated."""
|
||||||
self._recomputer.close()
|
# A running request belongs to the *previous* recording: its result must
|
||||||
self._recomputer = None
|
# never be pasted into the new one, and its processor must be closed by
|
||||||
|
# the thread that is using it, not here.
|
||||||
|
self._retire_recomputer()
|
||||||
|
self._record_serial += 1
|
||||||
|
self._depth_request_seq += 1
|
||||||
|
self._depth_pending_seq = None
|
||||||
|
self._depth_pending_position = None
|
||||||
|
self._depth_error = None
|
||||||
self._reader = RecordingReader(directory)
|
self._reader = RecordingReader(directory)
|
||||||
self._recomputer = DepthRecomputer(self._reader)
|
self._recomputer = DepthRecomputer(self._reader)
|
||||||
info = self._reader.info()
|
info = self._reader.info()
|
||||||
@@ -188,19 +207,48 @@ class PlaybackDialog(QDialog):
|
|||||||
self._depth_result_seq = None
|
self._depth_result_seq = None
|
||||||
self.set_position(0.0)
|
self.set_position(0.0)
|
||||||
|
|
||||||
|
def _retire_recomputer(self) -> None:
|
||||||
|
"""Hand the current processor to its worker, or close it here if idle.
|
||||||
|
|
||||||
|
Closing a processor that a worker thread is still reading is a use-after-
|
||||||
|
free; only the thread that owns it may close it, so a busy one is marked
|
||||||
|
for the worker to close when it finishes.
|
||||||
|
"""
|
||||||
|
recomputer, self._recomputer = self._recomputer, None
|
||||||
|
if recomputer is None:
|
||||||
|
return
|
||||||
|
if self._depth_busy():
|
||||||
|
self._retired_recomputers.append(recomputer)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
recomputer.close()
|
||||||
|
except Exception: # noqa: BLE001 - closing is best effort
|
||||||
|
logger.exception("Could not close the depth recomputer")
|
||||||
|
|
||||||
def _refresh_warnings(self) -> None:
|
def _refresh_warnings(self) -> None:
|
||||||
if self._reader is None:
|
if self._reader is None:
|
||||||
return
|
return
|
||||||
|
info = self._reader.info()
|
||||||
_ok, sdk_version = DepthRecomputer.sdk_available()
|
_ok, sdk_version = DepthRecomputer.sdk_available()
|
||||||
# The reader compares the recorded config hash against the config file as
|
# The reader compares the recorded config hash against the config file as
|
||||||
# it is now, which is what the depth recomputation would actually use.
|
# it is now, which is what the depth recomputation would actually use.
|
||||||
codes = self._reader.replay_warnings(sdk_version=sdk_version)
|
codes = self._reader.replay_warnings(sdk_version=sdk_version)
|
||||||
if not codes:
|
text = ""
|
||||||
self.warning_label.setText("")
|
if codes:
|
||||||
return
|
text = t(
|
||||||
self.warning_label.setText(
|
"playback.warnings",
|
||||||
t("playback.warnings", detail=", ".join(t(f"playback.warn_{c}") for c in codes))
|
detail=", ".join(t(f"playback.warn_{c}") for c in codes),
|
||||||
)
|
)
|
||||||
|
if info.decimated:
|
||||||
|
# The curves are a view, not every sample: say so rather than let the
|
||||||
|
# operator count points.
|
||||||
|
note = t(
|
||||||
|
"playback.decimated",
|
||||||
|
shown=len(info.times),
|
||||||
|
total=info.total_rows,
|
||||||
|
)
|
||||||
|
text = f"{text} {note}" if text else note
|
||||||
|
self.warning_label.setText(text)
|
||||||
|
|
||||||
# ------------------------------------------------------------------ drawing
|
# ------------------------------------------------------------------ drawing
|
||||||
def _render_curves(self) -> None:
|
def _render_curves(self) -> None:
|
||||||
@@ -211,11 +259,21 @@ class PlaybackDialog(QDialog):
|
|||||||
for curve in self.curves:
|
for curve in self.curves:
|
||||||
curve.setData([], [])
|
curve.setData([], [])
|
||||||
return
|
return
|
||||||
# Same rule as the live charts: invalid or non-finite points are not
|
# Same rule as the live charts, per axis: a point is drawn only where the
|
||||||
# drawn, whichever view is asking.
|
# sample is valid and *that* axis is finite, so a single-axis calibration
|
||||||
times, drawn, _mask = info.plottable(self._unit)
|
# does not hide the axis that is calibrated. For a long recording the
|
||||||
|
# whole-session view is decimated while it is read from disk, in the unit
|
||||||
|
# being shown.
|
||||||
|
window_all = getattr(self._reader, "window_all", None)
|
||||||
|
if callable(window_all):
|
||||||
|
window = window_all(unit=self._unit, max_points=CHART_MAX_POINTS)
|
||||||
for index, curve in enumerate(self.curves):
|
for index, curve in enumerate(self.curves):
|
||||||
curve.setData(times, drawn[:, index])
|
times, values = window.plottable_axis(self._unit, index)
|
||||||
|
curve.setData(times, values)
|
||||||
|
return
|
||||||
|
for index, curve in enumerate(self.curves):
|
||||||
|
times, values = info.plottable_axis(self._unit, index)
|
||||||
|
curve.setData(times, values)
|
||||||
self.plot_widget.ci.layout.setRowStretchFactor(index, 1)
|
self.plot_widget.ci.layout.setRowStretchFactor(index, 1)
|
||||||
self.plot_widget.getItem(index, 0).setXRange(
|
self.plot_widget.getItem(index, 0).setXRange(
|
||||||
float(info.times[0]), float(info.times[-1]), padding=0.02
|
float(info.times[0]), float(info.times[-1]), padding=0.02
|
||||||
@@ -307,22 +365,40 @@ class PlaybackDialog(QDialog):
|
|||||||
self._depth_error = None
|
self._depth_error = None
|
||||||
self.depth_label.setText(t("playback.depth_running"))
|
self.depth_label.setText(t("playback.depth_running"))
|
||||||
position = self._position
|
position = self._position
|
||||||
# Bind the answer to this request: a result that arrives after the cursor
|
# Bind the answer to this request: the recording, the frame and a request
|
||||||
# moved is discarded rather than pasted onto the new position.
|
# sequence. A result that arrives after the cursor moved — or after
|
||||||
|
# another recording was opened — is discarded rather than pasted on.
|
||||||
|
recomputer = self._recomputer
|
||||||
|
record = self._record_serial
|
||||||
|
frame = self._current_frame()
|
||||||
self._depth_request_seq += 1
|
self._depth_request_seq += 1
|
||||||
seq = self._depth_request_seq
|
seq = self._depth_request_seq
|
||||||
self._depth_pending_seq = seq
|
self._depth_pending_seq = seq
|
||||||
self._depth_pending_position = position
|
self._depth_pending_position = position
|
||||||
|
self._depth_pending_record = record
|
||||||
|
|
||||||
def work() -> None:
|
def work() -> None:
|
||||||
# Runs off the UI thread and touches no Qt object.
|
# Runs off the UI thread and touches no Qt object. It captures the
|
||||||
|
# processor it was given: the dialog may already be showing another
|
||||||
|
# recording by the time this returns.
|
||||||
try:
|
try:
|
||||||
result = self._recomputer.depth_at(position)
|
result = recomputer.depth_at(position)
|
||||||
except BaseException as exc: # noqa: BLE001 - reported via the poll
|
except BaseException as exc: # noqa: BLE001 - reported via the poll
|
||||||
self._depth_error = exc
|
self._depth_error = (record, exc)
|
||||||
return
|
return
|
||||||
|
finally:
|
||||||
|
if recomputer in self._retired_recomputers:
|
||||||
|
# Retired while this request was running: the owning thread
|
||||||
|
# closes it, not the UI thread.
|
||||||
|
self._retired_recomputers.remove(recomputer)
|
||||||
|
try:
|
||||||
|
recomputer.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
logger.exception("Could not close a retired recomputer")
|
||||||
self._depth_result = result
|
self._depth_result = result
|
||||||
self._depth_result_seq = seq
|
self._depth_result_seq = seq
|
||||||
|
self._depth_result_record = record
|
||||||
|
self._depth_result_frame = getattr(frame, "frame_id", -1)
|
||||||
|
|
||||||
self._depth_thread = threading.Thread(
|
self._depth_thread = threading.Thread(
|
||||||
target=work, name="depth-replay", daemon=True
|
target=work, name="depth-replay", daemon=True
|
||||||
@@ -337,15 +413,22 @@ class PlaybackDialog(QDialog):
|
|||||||
return
|
return
|
||||||
self._depth_poll.stop()
|
self._depth_poll.stop()
|
||||||
pending = self._depth_pending_seq
|
pending = self._depth_pending_seq
|
||||||
|
record = self._depth_pending_record
|
||||||
self._depth_pending_seq = None
|
self._depth_pending_seq = None
|
||||||
self._depth_pending_position = None
|
self._depth_pending_position = None
|
||||||
|
self._depth_pending_record = None
|
||||||
if self._depth_error is not None:
|
if self._depth_error is not None:
|
||||||
|
error_record, exc = self._depth_error
|
||||||
|
self._depth_error = None
|
||||||
|
if error_record != self._record_serial:
|
||||||
|
# The failure belongs to a recording that is no longer open.
|
||||||
|
return
|
||||||
self._depth_ok = False
|
self._depth_ok = False
|
||||||
self.depth_label.setPixmap(QPixmap())
|
self.depth_label.setPixmap(QPixmap())
|
||||||
self.depth_label.setText(
|
self.depth_label.setText(t("playback.depth_failed", detail=str(exc)))
|
||||||
t("playback.depth_failed", detail=str(self._depth_error))
|
return
|
||||||
)
|
if record != self._record_serial:
|
||||||
self._depth_error = None
|
# Another recording was opened while this was running.
|
||||||
return
|
return
|
||||||
result = self._depth_result
|
result = self._depth_result
|
||||||
if result is None or self._depth_result_seq != pending:
|
if result is None or self._depth_result_seq != pending:
|
||||||
@@ -468,10 +551,18 @@ class PlaybackDialog(QDialog):
|
|||||||
def closeEvent(self, event) -> None: # noqa: N802
|
def closeEvent(self, event) -> None: # noqa: N802
|
||||||
self._depth_poll.stop()
|
self._depth_poll.stop()
|
||||||
# A depth recomputation that is blocked inside the SDK cannot be
|
# A depth recomputation that is blocked inside the SDK cannot be
|
||||||
# cancelled; it is a daemon thread, so it does not keep the process
|
# cancelled; it is a daemon thread, so it does not keep the process alive
|
||||||
# alive and the window always closes.
|
# and the window always closes. The processor is closed by whichever
|
||||||
if self._recomputer is not None and not self._depth_busy():
|
# thread owns it (see ``_retire_recomputer``), never by the UI thread
|
||||||
self._recomputer.close()
|
# while a worker is reading it.
|
||||||
|
for recomputer in list(self._retired_recomputers):
|
||||||
|
if not self._depth_busy():
|
||||||
|
self._retired_recomputers.remove(recomputer)
|
||||||
|
try:
|
||||||
|
recomputer.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
logger.exception("Could not close a retired recomputer")
|
||||||
|
self._retire_recomputer()
|
||||||
super().closeEvent(event)
|
super().closeEvent(event)
|
||||||
|
|
||||||
def _depth_busy(self) -> bool:
|
def _depth_busy(self) -> bool:
|
||||||
|
|||||||
@@ -289,7 +289,7 @@ def test_cal_panel_records_and_lists_samples(make_window):
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40) -> None:
|
def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40):
|
||||||
"""Drive one point through the recorder and into the panel's table."""
|
"""Drive one point through the recorder and into the panel's table."""
|
||||||
from orisys_app_shared.force.session import SessionSample
|
from orisys_app_shared.force.session import SessionSample
|
||||||
|
|
||||||
@@ -305,7 +305,9 @@ def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40
|
|||||||
valid=True,
|
valid=True,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
window.cal_panel.on_record_finished(window.cal_recorder.finish_now())
|
point = window.cal_recorder.finish_now()
|
||||||
|
window.cal_panel.on_record_finished(point)
|
||||||
|
return point
|
||||||
|
|
||||||
|
|
||||||
def test_calibration_diagnostics_drive_the_fit(make_window):
|
def test_calibration_diagnostics_drive_the_fit(make_window):
|
||||||
@@ -357,6 +359,64 @@ def test_calibration_diagnostics_drive_the_fit(make_window):
|
|||||||
model = build_export_model({key: store.channel(key) for key in ("fn", "fx", "fy")})
|
model = build_export_model({key: store.channel(key) for key in ("fn", "fx", "fy")})
|
||||||
assert abs(predict_channel(model, "fn", 400.0) - 2.0) > 0.05
|
assert abs(predict_channel(model, "fn", 400.0) - 2.0) > 0.05
|
||||||
|
|
||||||
|
# The store, the label and the export are the same preparation: the panel
|
||||||
|
# must not derive the model twice and show numbers for one of them.
|
||||||
|
prepared = window.cal_panel._prepared
|
||||||
|
assert prepared is not None and prepared.ok
|
||||||
|
assert list(store.channel("fn")) == list(prepared.samples["fn"])
|
||||||
|
assert prepared.points_in_fit("fn") == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_panel_shows_the_numbers_of_the_file_it_exports(make_window, record_root):
|
||||||
|
"""Displayed diagnostics must match the written coefficients.
|
||||||
|
|
||||||
|
A repeated reference load makes this sharp: the two samples of one load must
|
||||||
|
be merged before the build, so the exported curve and the panel's residual
|
||||||
|
and validation figures all come from that single prepared model.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import orisys.calibration as sdk_calibration
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_io import build_force_calibration_v2
|
||||||
|
|
||||||
|
window = _show(make_window(), (1400, 880))
|
||||||
|
window.cal_panel.set_running(True)
|
||||||
|
window.cal_panel.set_meta_context(sensor_type="box", config_ref=None)
|
||||||
|
_add_calibration_point(window, load=1.0, raw=100.0)
|
||||||
|
_add_calibration_point(window, load=2.0, raw=200.0)
|
||||||
|
_add_calibration_point(window, load=2.0, raw=240.0) # a deliberate repeat
|
||||||
|
_add_calibration_point(window, load=4.0, raw=400.0)
|
||||||
|
_add_calibration_point(window, load=3.0, raw=290.0)
|
||||||
|
|
||||||
|
panel = window.cal_panel
|
||||||
|
panel.table.setCurrentCell(4, 0)
|
||||||
|
panel._on_toggle_validation()
|
||||||
|
panel._rebuild_store()
|
||||||
|
panel._refresh_fit_label()
|
||||||
|
prepared = panel._prepared
|
||||||
|
assert prepared is not None and prepared.ok
|
||||||
|
assert len(prepared.samples["fn"]) == 3, "the repeated load was not merged"
|
||||||
|
|
||||||
|
# Write the document the export would write, load it back with the SDK and
|
||||||
|
# check that its predictions are the prepared ones.
|
||||||
|
document, _warnings = build_force_calibration_v2(prepared)
|
||||||
|
path = Path(record_root) / "force_calibration.json"
|
||||||
|
path.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
loaded = sdk_calibration.load(path, strict=True)
|
||||||
|
|
||||||
|
for point in window.cal_recorder.points:
|
||||||
|
raw = float(point.loaded_stats().mean)
|
||||||
|
predicted = float(loaded.get("fn").convert(raw))
|
||||||
|
assert predicted == pytest.approx(prepared.predict("fn", raw), abs=1e-9)
|
||||||
|
|
||||||
|
# ...and the label describes that same model.
|
||||||
|
label = panel.fit_label.text()
|
||||||
|
assert "分段线性" in label or "piecewise" in label, label
|
||||||
|
assert "3" in label
|
||||||
|
assert prepared.max_validation_error("fn") > 0.0
|
||||||
|
|
||||||
|
|
||||||
def test_calibration_point_dialog_shows_three_axes(make_window):
|
def test_calibration_point_dialog_shows_three_axes(make_window):
|
||||||
window = _show(make_window(), (1400, 880))
|
window = _show(make_window(), (1400, 880))
|
||||||
@@ -373,6 +433,137 @@ def test_calibration_point_dialog_shows_three_axes(make_window):
|
|||||||
dialog.close()
|
dialog.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _write_session_file(root, *, sensor_type, sn, config_ref, points):
|
||||||
|
import json
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_session import build_session_document
|
||||||
|
|
||||||
|
document = build_session_document(
|
||||||
|
points,
|
||||||
|
meta={"sensor_type": sensor_type, "sn": sn, "config_ref": config_ref},
|
||||||
|
)
|
||||||
|
path = Path(root) / f"calibration_session_{sensor_type}.json"
|
||||||
|
path.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def test_opening_device_a_record_keeps_device_a_identity(make_window, record_root):
|
||||||
|
"""Device A's record must not be restamped with device B's identity."""
|
||||||
|
import json
|
||||||
|
from unittest.mock import patch as _patch
|
||||||
|
|
||||||
|
import orisys_studio.ui.cal_panel as cal_panel_module
|
||||||
|
from orisys_app_shared.force.calibration_io import build_force_calibration_v2
|
||||||
|
|
||||||
|
window = _show(make_window(), (1400, 880))
|
||||||
|
panel = window.cal_panel
|
||||||
|
# A record taken on device A, with a full audit series.
|
||||||
|
session_points = [
|
||||||
|
_add_calibration_point(window, load=1.0, raw=100.0),
|
||||||
|
_add_calibration_point(window, load=3.0, raw=300.0),
|
||||||
|
]
|
||||||
|
source = _write_session_file(
|
||||||
|
record_root,
|
||||||
|
sensor_type="box",
|
||||||
|
sn="SN-A",
|
||||||
|
config_ref="D:/lab/deviceA/sensor.json",
|
||||||
|
points=session_points,
|
||||||
|
)
|
||||||
|
window.cal_recorder.clear_points()
|
||||||
|
|
||||||
|
# ...and the live context is device B.
|
||||||
|
panel.set_meta_context(sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json")
|
||||||
|
panel.sn_edit.setText("SN-B")
|
||||||
|
|
||||||
|
panel._on_open_record(str(source))
|
||||||
|
assert panel.sn_edit.text() == "SN-A", "the record's SN was overwritten"
|
||||||
|
meta = panel._export_meta()
|
||||||
|
assert meta["sensor_type"] == "box"
|
||||||
|
assert meta["sn"] == "SN-A"
|
||||||
|
assert meta["config_ref"] == "D:/lab/deviceA/sensor.json"
|
||||||
|
assert meta["source_record"].endswith(source.name)
|
||||||
|
|
||||||
|
# Saving the record again still says A.
|
||||||
|
import orisys_studio.calibration_service as cal_service_module
|
||||||
|
|
||||||
|
with _patch.object(cal_panel_module, "calibration_export_dir", lambda: Path(record_root)):
|
||||||
|
with _patch.object(
|
||||||
|
cal_service_module, "calibration_export_dir", lambda: Path(record_root)
|
||||||
|
):
|
||||||
|
with _patch.object(cal_panel_module.QMessageBox, "information", lambda *a, **k: None):
|
||||||
|
panel._on_save_record()
|
||||||
|
saved = sorted(Path(record_root).glob("calibration_session_box_*.json"))
|
||||||
|
assert saved, "the record was not saved under device A's type"
|
||||||
|
saved_meta = json.loads(saved[-1].read_text(encoding="utf-8"))["meta"]
|
||||||
|
assert saved_meta["sn"] == "SN-A"
|
||||||
|
assert saved_meta["sensor_type"] == "box"
|
||||||
|
assert saved_meta["config_ref"] == "D:/lab/deviceA/sensor.json"
|
||||||
|
|
||||||
|
# ...and so does the exported coefficient document.
|
||||||
|
document, _warnings = build_force_calibration_v2(panel._prepared)
|
||||||
|
assert document["meta"]["sensor_type"] == "box"
|
||||||
|
assert document["meta"]["sn"] == "SN-A"
|
||||||
|
assert document["meta"]["config_ref"] == "D:/lab/deviceA/sensor.json"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_reopened_point_can_be_inspected_from_its_samples(make_window, record_root):
|
||||||
|
"""A record with samples but no cached trace still opens its diagnostics."""
|
||||||
|
window = _show(make_window(), (1400, 880))
|
||||||
|
panel = window.cal_panel
|
||||||
|
points = [
|
||||||
|
_add_calibration_point(window, load=1.0, raw=100.0),
|
||||||
|
_add_calibration_point(window, load=2.0, raw=200.0),
|
||||||
|
]
|
||||||
|
import json
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_session import build_session_document
|
||||||
|
|
||||||
|
document = build_session_document(points, meta={"sensor_type": "box"})
|
||||||
|
for entry in document["points"]:
|
||||||
|
entry["trace"] = {} # only the audit series survived
|
||||||
|
entry["trace_dt_s"] = 0.0
|
||||||
|
source = Path(record_root) / "no_trace.json"
|
||||||
|
source.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
|
||||||
|
panel._on_open_record(str(source))
|
||||||
|
reopened = window.cal_recorder.points[0]
|
||||||
|
assert reopened.samples, "the reopened point lost its samples"
|
||||||
|
dialog = panel.point_dialog(reopened)
|
||||||
|
assert dialog is not None, "the reopened point could not be inspected"
|
||||||
|
assert len(dialog.plot_widget.ci.items) == 3
|
||||||
|
dialog.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_applying_another_devices_calibration_is_flagged(make_window, record_root):
|
||||||
|
"""The apply path states a sensor/config mismatch instead of silently applying."""
|
||||||
|
import json
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_io import build_export_model, write_force_calibration
|
||||||
|
from orisys_app_shared.force.calibration_io import CalSample, CalSessionStore
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
store = CalSessionStore()
|
||||||
|
store.aggregate("fn", CalSample(weight_n=0.0, raw_mean=0.0, n=10))
|
||||||
|
store.aggregate("fn", CalSample(weight_n=5.0, raw_mean=500.0, n=10))
|
||||||
|
model = build_export_model({key: store.channel(key) for key in ("fn", "fx", "fy")})
|
||||||
|
document = model.to_dict()
|
||||||
|
document["identity"] = {"sensor_type": "box", "sn": "SN-A"}
|
||||||
|
document["pipeline"] = {"config_ref": "D:/lab/deviceA/sensor.json"}
|
||||||
|
path = Path(record_root) / "deviceA_calibration.json"
|
||||||
|
path.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
|
||||||
|
# The window is connected to a finger device.
|
||||||
|
mismatch = window.calibration_target_mismatch(
|
||||||
|
str(path), sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json"
|
||||||
|
)
|
||||||
|
assert any("sensor_type" in reason for reason in mismatch), mismatch
|
||||||
|
assert any("config_ref" in reason for reason in mismatch), mismatch
|
||||||
|
assert not window.calibration_target_mismatch(
|
||||||
|
str(path), sensor_type="box", config_ref="D:/lab/deviceA/sensor.json"
|
||||||
|
)
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
def test_force_plots_render_demo_data(make_window):
|
def test_force_plots_render_demo_data(make_window):
|
||||||
window = _show(make_window(), (1440, 880))
|
window = _show(make_window(), (1440, 880))
|
||||||
for i in range(60):
|
for i in range(60):
|
||||||
@@ -502,6 +693,344 @@ def test_history_keeps_negative_fn(make_window):
|
|||||||
_finish(window, stuck)
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_live_axis_is_session_relative_and_survives_a_unit_switch(make_window):
|
||||||
|
"""The live buffer, the store and a rebuilt history share one time base.
|
||||||
|
|
||||||
|
Feeding the wall clock into the plot put the live axis ~1.7e9 s away from the
|
||||||
|
session store's, so a unit switch (which rebuilds from the store) jumped the
|
||||||
|
X axis and the chart looked like it had lost its history.
|
||||||
|
"""
|
||||||
|
wall = 1_700_000_000.0
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
window.toolbar.units_combo.setCurrentIndex(0) # newton
|
||||||
|
window.force_plots.reset()
|
||||||
|
for index in range(90):
|
||||||
|
stuck.emit(
|
||||||
|
ui_check._fake_packet(
|
||||||
|
index, timestamp=wall + index / 30.0, mono_t=index / 30.0
|
||||||
|
)
|
||||||
|
)
|
||||||
|
window._on_run_tick()
|
||||||
|
|
||||||
|
before = window.force_plots.history.state()
|
||||||
|
assert before is not None
|
||||||
|
end_before = float(before.times[-1])
|
||||||
|
assert end_before < 60.0, (
|
||||||
|
f"the live X axis is not session-relative: it ends at {end_before}"
|
||||||
|
)
|
||||||
|
assert abs(end_before - 89 / 30.0) < 0.5, end_before
|
||||||
|
|
||||||
|
for units_index in (1, 0, 1, 0): # raw -> N -> raw -> N
|
||||||
|
window.toolbar.units_combo.setCurrentIndex(units_index)
|
||||||
|
|
||||||
|
after = window.force_plots.history.state()
|
||||||
|
assert after is not None, "the unit switch emptied the rebuilt history"
|
||||||
|
assert abs(float(after.times[-1]) - end_before) < 1.0, (
|
||||||
|
f"the X axis jumped across a unit switch: {end_before} -> {after.times[-1]}"
|
||||||
|
)
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_single_calibrated_axis_is_shown_and_the_others_are_unavailable(make_window):
|
||||||
|
"""Only FN calibrated: FN in newtons, FX/FY explicitly unavailable."""
|
||||||
|
import math
|
||||||
|
|
||||||
|
from orisys_app_shared.i18n import t
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
window.toolbar.units_combo.setCurrentIndex(0) # newton
|
||||||
|
window.force_plots.reset()
|
||||||
|
for index in range(10):
|
||||||
|
stuck.emit(
|
||||||
|
ui_check._fake_packet(
|
||||||
|
index,
|
||||||
|
calibrated_axes=("fn",),
|
||||||
|
shear_n=float("nan"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
window._on_run_tick()
|
||||||
|
|
||||||
|
plots = window.force_plots
|
||||||
|
assert plots._available == (True, False, False), plots._available
|
||||||
|
assert plots.value_labels[0].text() != t("chart.value_unavailable")
|
||||||
|
for index in (1, 2):
|
||||||
|
assert plots.value_labels[index].text() == t("chart.value_unavailable"), (
|
||||||
|
"an axis with no curve must not show a number"
|
||||||
|
)
|
||||||
|
state = plots.history.state()
|
||||||
|
assert math.isfinite(float(state.values[:, 0].max()))
|
||||||
|
assert not any(math.isfinite(v) for v in state.values[:, 1]), (
|
||||||
|
"FX carried a value in newton mode without a curve"
|
||||||
|
)
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_slow_close_is_not_reported_as_a_finished_recording(make_window):
|
||||||
|
"""Stopping must not present an unclosed file as a finished experiment."""
|
||||||
|
import threading as _threading
|
||||||
|
from unittest.mock import patch as _patch
|
||||||
|
|
||||||
|
from orisys_app_shared.i18n import t
|
||||||
|
from orisys_app_shared.record.experiment import _SessionWriter
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
window.toolbar.btn_record.setChecked(True)
|
||||||
|
stuck.emit(ui_check._fake_packet(0))
|
||||||
|
window._on_run_tick()
|
||||||
|
|
||||||
|
started = _threading.Event()
|
||||||
|
release = _threading.Event()
|
||||||
|
original = _SessionWriter.write_sample
|
||||||
|
|
||||||
|
def slow(self, *args):
|
||||||
|
started.set()
|
||||||
|
release.wait(10.0)
|
||||||
|
return original(self, *args)
|
||||||
|
|
||||||
|
with _patch.object(_SessionWriter, "write_sample", slow):
|
||||||
|
stuck.emit(ui_check._fake_packet(1))
|
||||||
|
window._on_run_tick()
|
||||||
|
assert started.wait(5.0), "the writer never picked up the frame"
|
||||||
|
|
||||||
|
window._stop_recording(wait=False)
|
||||||
|
assert window.recorder.report is None, "the stop claimed a finished recording"
|
||||||
|
assert window._record_report_pending, "the pending report was not tracked"
|
||||||
|
assert window._record_last_report is None
|
||||||
|
assert not window.toolbar.btn_record.isChecked(), (
|
||||||
|
"the record button still looks active while the file closes"
|
||||||
|
)
|
||||||
|
assert window.statusBar().currentMessage() == t("record.closing")
|
||||||
|
|
||||||
|
release.set()
|
||||||
|
deadline = __import__("time").monotonic() + 10.0
|
||||||
|
while window.recorder.report is None and __import__("time").monotonic() < deadline:
|
||||||
|
__import__("time").sleep(0.01)
|
||||||
|
window._on_run_tick()
|
||||||
|
|
||||||
|
report = window._record_last_report
|
||||||
|
assert report is not None and report.completed, "the terminal report was never published"
|
||||||
|
assert not window._record_report_pending
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_failed_recording_is_consumed_even_when_it_is_not_running(make_window):
|
||||||
|
"""A recording that died on its own still has a report to publish."""
|
||||||
|
import time as _time
|
||||||
|
from unittest.mock import patch as _patch
|
||||||
|
|
||||||
|
from orisys_app_shared.record.experiment import _SessionWriter
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
window.toolbar.btn_record.setChecked(True)
|
||||||
|
assert window.recorder.is_recording
|
||||||
|
|
||||||
|
with _patch.object(_SessionWriter, "write_sample", side_effect=OSError("disk full")):
|
||||||
|
for index in range(8):
|
||||||
|
stuck.emit(ui_check._fake_packet(index))
|
||||||
|
window._on_run_tick()
|
||||||
|
deadline = _time.monotonic() + 3.0
|
||||||
|
while window.recorder.stats().write_errors < 5 and _time.monotonic() < deadline:
|
||||||
|
_time.sleep(0.01)
|
||||||
|
assert window.recorder.stats().failed
|
||||||
|
assert not window.recorder.is_recording
|
||||||
|
|
||||||
|
# ``is_recording`` is already False: the stop must still consume the report.
|
||||||
|
window._stop_recording()
|
||||||
|
report = window._record_last_report
|
||||||
|
assert report is not None and report.stats.failed
|
||||||
|
assert not window.toolbar.btn_record.isChecked()
|
||||||
|
assert window._session_phase == "error"
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_depth_request_belongs_to_the_recording_it_started_in(make_window, record_root):
|
||||||
|
"""Opening another recording must invalidate the request still running."""
|
||||||
|
import threading as _threading
|
||||||
|
import time as _time
|
||||||
|
|
||||||
|
from PyQt5.QtWidgets import QApplication
|
||||||
|
|
||||||
|
from orisys_app_shared.record.depth import DepthResult
|
||||||
|
from orisys_studio.ui.playback_dialog import PlaybackDialog
|
||||||
|
|
||||||
|
window = _show(make_window(), (1200, 700))
|
||||||
|
dialog = PlaybackDialog(theme="dark", directory=Path(record_root))
|
||||||
|
first_serial = dialog._record_serial
|
||||||
|
|
||||||
|
started = _threading.Event()
|
||||||
|
release = _threading.Event()
|
||||||
|
closed: list[str] = []
|
||||||
|
|
||||||
|
class _Blocking:
|
||||||
|
def preflight(self) -> str:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def depth_at(self, _position, **_kwargs):
|
||||||
|
started.set()
|
||||||
|
release.wait(10.0)
|
||||||
|
return DepthResult(
|
||||||
|
depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0, record_id="old"
|
||||||
|
)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
closed.append("closed")
|
||||||
|
|
||||||
|
blocking = _Blocking()
|
||||||
|
dialog._recomputer = blocking
|
||||||
|
dialog._on_recompute_depth()
|
||||||
|
assert started.wait(5.0), "the depth request never started"
|
||||||
|
|
||||||
|
# Another recording is opened while the request is in flight.
|
||||||
|
dialog.open_recording(Path(record_root))
|
||||||
|
assert dialog._record_serial == first_serial + 1
|
||||||
|
assert dialog._recomputer is not blocking, "the processor was not replaced"
|
||||||
|
assert closed == [], "the UI thread closed a processor a worker was reading"
|
||||||
|
|
||||||
|
release.set()
|
||||||
|
deadline = _time.monotonic() + 10.0
|
||||||
|
while not closed and _time.monotonic() < deadline:
|
||||||
|
QApplication.processEvents()
|
||||||
|
_time.sleep(0.01)
|
||||||
|
assert closed == ["closed"], "the retired processor was never closed"
|
||||||
|
assert dialog._depth_thread is not None and not dialog._depth_thread.is_alive()
|
||||||
|
|
||||||
|
# The answer belongs to the previous recording, so it is dropped, not drawn.
|
||||||
|
dialog._poll_depth()
|
||||||
|
QApplication.processEvents()
|
||||||
|
assert dialog._depth_ok is False
|
||||||
|
pixmap = dialog.depth_label.pixmap()
|
||||||
|
assert pixmap is None or pixmap.isNull(), "an answer for another recording was drawn"
|
||||||
|
dialog.close()
|
||||||
|
window.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_busy_depth_processor_is_not_closed_by_the_ui_thread(make_window, record_root):
|
||||||
|
import threading as _threading
|
||||||
|
import time as _time
|
||||||
|
|
||||||
|
from PyQt5.QtWidgets import QApplication
|
||||||
|
|
||||||
|
from orisys_app_shared.record.depth import DepthResult
|
||||||
|
from orisys_studio.ui.playback_dialog import PlaybackDialog
|
||||||
|
|
||||||
|
window = _show(make_window(), (1200, 700))
|
||||||
|
dialog = PlaybackDialog(theme="dark", directory=Path(record_root))
|
||||||
|
started = _threading.Event()
|
||||||
|
release = _threading.Event()
|
||||||
|
closed: list[str] = []
|
||||||
|
|
||||||
|
class _Blocking:
|
||||||
|
def preflight(self) -> str:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def depth_at(self, _position, **_kwargs):
|
||||||
|
started.set()
|
||||||
|
release.wait(10.0)
|
||||||
|
return DepthResult(depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
closed.append("closed")
|
||||||
|
|
||||||
|
dialog._recomputer = _Blocking()
|
||||||
|
dialog._on_recompute_depth()
|
||||||
|
assert started.wait(5.0)
|
||||||
|
dialog.close() # closing the window must not free a processor in use
|
||||||
|
QApplication.processEvents()
|
||||||
|
assert closed == [], "the window closed a processor that was still reading"
|
||||||
|
release.set()
|
||||||
|
deadline = _time.monotonic() + 10.0
|
||||||
|
while not closed and _time.monotonic() < deadline:
|
||||||
|
QApplication.processEvents()
|
||||||
|
_time.sleep(0.01)
|
||||||
|
assert closed == ["closed"], "the owning thread never closed the processor"
|
||||||
|
window.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_confirmed_calibration_is_snapshotted_into_the_recording(make_window, record_root):
|
||||||
|
"""The recording keeps the coefficients themselves, not only their hash."""
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import orisys_studio.app_window as app_module
|
||||||
|
from orisys_app_shared.i18n import t
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
app_module.recordings_dir = lambda: record_root
|
||||||
|
window.toolbar.btn_record.setChecked(True)
|
||||||
|
assert window.recorder.is_recording
|
||||||
|
|
||||||
|
coefficients = Path(record_root) / "force_calibration_box.json"
|
||||||
|
coefficients.write_text('{"unit": "N", "channels": {}}', encoding="utf-8")
|
||||||
|
version = "deadbeefcafe1234"
|
||||||
|
window._on_calibration_applied(
|
||||||
|
str(coefficients), version, 21, 0.7, int(window.session_store.session_id)
|
||||||
|
)
|
||||||
|
window._drain_calibration_events()
|
||||||
|
|
||||||
|
directory = window.recorder.directory
|
||||||
|
assert directory is not None
|
||||||
|
snapshots = sorted(path.name for path in (directory / "calibration").iterdir())
|
||||||
|
assert snapshots == [f"force_calibration_{version[:12]}.json"], snapshots
|
||||||
|
assert (directory / "calibration" / snapshots[0]).read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
) == '{"unit": "N", "channels": {}}'
|
||||||
|
assert window._active_calibration_id == version
|
||||||
|
assert window.session_store.calibration_id == version
|
||||||
|
|
||||||
|
window._stop_recording()
|
||||||
|
meta = json.loads((directory / "session.json").read_text(encoding="utf-8"))
|
||||||
|
entries = meta["calibration_snapshots"]
|
||||||
|
assert entries and entries[0]["version"] == version
|
||||||
|
assert entries[0]["file"] == f"calibration/{snapshots[0]}"
|
||||||
|
assert entries[0]["frame_id"] == 21
|
||||||
|
event = next(e for e in meta["events"] if e["kind"] == "calibration_applied")
|
||||||
|
assert event["frame_id"] == 21 and event["mono_t"] == pytest.approx(0.7)
|
||||||
|
assert snapshots[0] in event["detail"]
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_confirmed_absence_of_calibration_does_not_keep_the_old_version(make_window):
|
||||||
|
"""An empty version is the honest answer; it must not fall back to the path."""
|
||||||
|
from orisys_app_shared.i18n import t
|
||||||
|
|
||||||
|
window = _show(make_window(), (1400, 880))
|
||||||
|
window._active_calibration_id = "previous-hash"
|
||||||
|
window._on_calibration_applied("/tmp/whatever.json", "", 5, 0.2, 1)
|
||||||
|
window._drain_calibration_events()
|
||||||
|
assert window._active_calibration_id == "", (
|
||||||
|
"an uncalibrated device kept claiming the previous coefficients"
|
||||||
|
)
|
||||||
|
assert window.statusBar().currentMessage() == t("cal.applied_none")
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_zeroing_is_recorded_only_after_the_worker_confirms_it(make_window, record_root):
|
||||||
|
import json
|
||||||
|
|
||||||
|
import orisys_studio.app_window as app_module
|
||||||
|
|
||||||
|
window, stuck = _running_window(make_window)
|
||||||
|
app_module.recordings_dir = lambda: record_root
|
||||||
|
window.toolbar.btn_record.setChecked(True)
|
||||||
|
stuck.emit(ui_check._fake_packet(3))
|
||||||
|
window._on_run_tick()
|
||||||
|
|
||||||
|
window._on_reset()
|
||||||
|
assert stuck.reset_requests == 1, "the zeroing was not requested"
|
||||||
|
# The stub confirms from the acquisition side, as a real device does.
|
||||||
|
stuck.on_reset_applied = window._on_reset_applied
|
||||||
|
window._on_reset()
|
||||||
|
window._on_run_tick()
|
||||||
|
window._stop_recording()
|
||||||
|
|
||||||
|
directory = window.recorder.directory
|
||||||
|
meta = json.loads((directory / "session.json").read_text(encoding="utf-8"))
|
||||||
|
zeroing = [e for e in meta["events"] if e["kind"] == "zeroing"]
|
||||||
|
assert len(zeroing) == 1, f"unexpected zeroing events: {zeroing}"
|
||||||
|
assert zeroing[0]["frame_id"] == 3, zeroing[0]
|
||||||
|
assert zeroing[0]["mono_t"] is not None
|
||||||
|
_finish(window, stuck)
|
||||||
|
|
||||||
|
|
||||||
def test_depth_view_reports_missing_data_and_fixes_its_scale(make_window):
|
def test_depth_view_reports_missing_data_and_fixes_its_scale(make_window):
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
|||||||
@@ -144,6 +144,10 @@ def _fake_packet(
|
|||||||
valid: bool = True,
|
valid: bool = True,
|
||||||
depth=None,
|
depth=None,
|
||||||
reason: str = "",
|
reason: str = "",
|
||||||
|
calibrated_axes=None,
|
||||||
|
timestamp: float | None = None,
|
||||||
|
mono_t: float | None = None,
|
||||||
|
shear_n: float | None = None,
|
||||||
):
|
):
|
||||||
"""A packet shaped like the real worker's, for the record-channel checks."""
|
"""A packet shaped like the real worker's, for the record-channel checks."""
|
||||||
import numpy as np
|
import numpy as np
|
||||||
@@ -157,17 +161,19 @@ def _fake_packet(
|
|||||||
return SdkPacket(
|
return SdkPacket(
|
||||||
raw_frame=blank,
|
raw_frame=blank,
|
||||||
processed_frame=blank,
|
processed_frame=blank,
|
||||||
timestamp=1000.0 + frame_id / 30.0,
|
timestamp=1000.0 + frame_id / 30.0 if timestamp is None else timestamp,
|
||||||
|
mono_t=mono_t,
|
||||||
fps=30.0,
|
fps=30.0,
|
||||||
process_ms=1.0,
|
process_ms=1.0,
|
||||||
normal_force=(2.0 + frame_id * 0.01) if valid else nan,
|
normal_force=(2.0 + frame_id * 0.01) if valid else nan,
|
||||||
shear_x=0.5 if valid else nan,
|
shear_x=(0.5 if shear_n is None else shear_n) if valid else nan,
|
||||||
shear_y=-0.5 if valid else nan,
|
shear_y=-0.5 if valid else nan,
|
||||||
normal_force_raw=(9.0 + frame_id * 0.1) if valid else nan,
|
normal_force_raw=(9.0 + frame_id * 0.1) if valid else nan,
|
||||||
shear_x_raw=0.4 if valid else nan,
|
shear_x_raw=0.4 if valid else nan,
|
||||||
shear_y_raw=-0.4 if valid else nan,
|
shear_y_raw=-0.4 if valid else nan,
|
||||||
force_unit="N",
|
force_unit="N",
|
||||||
force_calibrated=True,
|
force_calibrated=True,
|
||||||
|
calibrated_axes=tuple(calibrated_axes or ()),
|
||||||
session_id=1,
|
session_id=1,
|
||||||
frame_id=frame_id,
|
frame_id=frame_id,
|
||||||
valid=valid,
|
valid=valid,
|
||||||
@@ -194,6 +200,27 @@ class _StuckWorker(RecordSinkMixin):
|
|||||||
self.stop_blocked_for = 0.0
|
self.stop_blocked_for = 0.0
|
||||||
#: Released by ``release()`` so daemon threads do not outlive the run.
|
#: Released by ``release()`` so daemon threads do not outlive the run.
|
||||||
self._release = threading.Event()
|
self._release = threading.Event()
|
||||||
|
#: Calibration requests this worker was asked to apply.
|
||||||
|
self.calibration_requests: list[str] = []
|
||||||
|
#: Zeroing requests, and the confirmation callback the window installs.
|
||||||
|
self.reset_requests = 0
|
||||||
|
self.on_reset_applied = None
|
||||||
|
|
||||||
|
def set_pending_calibration(self, path) -> None:
|
||||||
|
self.calibration_requests.append(str(path))
|
||||||
|
|
||||||
|
def reset(self) -> None:
|
||||||
|
"""Request a zeroing; a real device confirms it from its own thread.
|
||||||
|
|
||||||
|
This stub confirms immediately (a fast device), so the window's event path
|
||||||
|
— request, confirmation, recorded event — is exercised end to end.
|
||||||
|
"""
|
||||||
|
self.reset_requests += 1
|
||||||
|
if callable(self.on_reset_applied):
|
||||||
|
packet = self._latest
|
||||||
|
frame_id = int(getattr(packet, "frame_id", -1))
|
||||||
|
mono_t = float(getattr(packet, "mono_t", 0.0) or 0.0)
|
||||||
|
self.on_reset_applied(frame_id, mono_t, 1)
|
||||||
|
|
||||||
def start(self, _source, *, config_ref=None) -> None:
|
def start(self, _source, *, config_ref=None) -> None:
|
||||||
self._release.clear()
|
self._release.clear()
|
||||||
@@ -229,9 +256,6 @@ class _StuckWorker(RecordSinkMixin):
|
|||||||
reason, self._end_reason = self._end_reason, None
|
reason, self._end_reason = self._end_reason, None
|
||||||
return reason
|
return reason
|
||||||
|
|
||||||
def reset(self) -> None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
def stop(self, *, wait: bool = False) -> None:
|
def stop(self, *, wait: bool = False) -> None:
|
||||||
self._end_reason = self._end_reason or "stopped"
|
self._end_reason = self._end_reason or "stopped"
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
@@ -666,8 +690,10 @@ def main() -> int:
|
|||||||
)
|
)
|
||||||
from orisys_app_shared.force.calibration_io import (
|
from orisys_app_shared.force.calibration_io import (
|
||||||
build_export_model,
|
build_export_model,
|
||||||
|
build_force_calibration_v2 as _build_v2,
|
||||||
predict_channel,
|
predict_channel,
|
||||||
)
|
)
|
||||||
|
import orisys.calibration as _sdk_calibration
|
||||||
|
|
||||||
store = window.cal_panel._store
|
store = window.cal_panel._store
|
||||||
model = build_export_model(
|
model = build_export_model(
|
||||||
@@ -678,6 +704,44 @@ def main() -> int:
|
|||||||
abs(predicted - 2.0) > 0.05,
|
abs(predicted - 2.0) > 0.05,
|
||||||
f"excluding a point left the exported model unchanged: {predicted}",
|
f"excluding a point left the exported model unchanged: {predicted}",
|
||||||
)
|
)
|
||||||
|
# One preparation: the store the panel shows is the prepared model's
|
||||||
|
# input, and the export serialises that same object.
|
||||||
|
prepared = window.cal_panel._prepared
|
||||||
|
check(
|
||||||
|
prepared is not None and prepared.ok,
|
||||||
|
"the panel has no prepared calibration to show or export",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
list(store.channel("fn")) == list(prepared.samples["fn"]),
|
||||||
|
"the panel store is not the prepared model's input",
|
||||||
|
)
|
||||||
|
doc, _warnings = _build_v2(prepared)
|
||||||
|
export_dir = APP_ROOT / "outputs" / "_cal_export_check"
|
||||||
|
if export_dir.exists():
|
||||||
|
for stale in export_dir.iterdir():
|
||||||
|
stale.unlink()
|
||||||
|
export_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
written = export_dir / "exported_calibration.json"
|
||||||
|
import json as _cal_json
|
||||||
|
|
||||||
|
written.write_text(_cal_json.dumps(doc), encoding="utf-8")
|
||||||
|
loaded = _sdk_calibration.load(written, strict=True)
|
||||||
|
for point in window.cal_recorder.points:
|
||||||
|
if point.excluded:
|
||||||
|
continue
|
||||||
|
raw = float(point.loaded_stats().mean)
|
||||||
|
check(
|
||||||
|
abs(
|
||||||
|
float(loaded.get("fn").convert(raw))
|
||||||
|
- prepared.predict("fn", raw)
|
||||||
|
)
|
||||||
|
<= 1e-9,
|
||||||
|
"the exported file and the panel disagree about the model",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
len(prepared.samples["fn"]) == 3,
|
||||||
|
f"the prepared build has {len(prepared.samples['fn'])} points, not 3",
|
||||||
|
)
|
||||||
window.cal_recorder.reset_session()
|
window.cal_recorder.reset_session()
|
||||||
window.cal_panel._refresh_table()
|
window.cal_panel._refresh_table()
|
||||||
|
|
||||||
@@ -961,7 +1025,8 @@ def main() -> int:
|
|||||||
app.processEvents()
|
app.processEvents()
|
||||||
check(window.recorder.is_recording, "the recording did not start")
|
check(window.recorder.is_recording, "the recording did not start")
|
||||||
check(
|
check(
|
||||||
window.toolbar.btn_record.isEnabled() and window.toolbar.btn_record.text() == t("record.stop_label"),
|
window.toolbar.btn_record.isEnabled()
|
||||||
|
and window.toolbar.btn_record.text() == t("record.stop_label"),
|
||||||
"the record button is not a usable stop control",
|
"the record button is not a usable stop control",
|
||||||
)
|
)
|
||||||
check(
|
check(
|
||||||
@@ -974,13 +1039,26 @@ def main() -> int:
|
|||||||
window._on_run_tick()
|
window._on_run_tick()
|
||||||
stuck.emit(_fake_packet(90, valid=False, reason="read failed"))
|
stuck.emit(_fake_packet(90, valid=False, reason="read failed"))
|
||||||
window._on_run_tick()
|
window._on_run_tick()
|
||||||
window._on_reset() # event: zeroing
|
# The zeroing is a *request*; the event is recorded only once the acquisition
|
||||||
|
# thread confirms it took effect (the stub confirms immediately).
|
||||||
|
stuck.on_reset_applied = window._on_reset_applied
|
||||||
|
window._on_reset()
|
||||||
|
window._on_run_tick()
|
||||||
|
app.processEvents()
|
||||||
|
|
||||||
window.toolbar.btn_record.setChecked(False)
|
window.toolbar.btn_record.setChecked(False)
|
||||||
app.processEvents()
|
app.processEvents()
|
||||||
report = window._record_last_report
|
report = window._record_last_report
|
||||||
check(not window.recorder.is_recording, "the recording did not stop")
|
check(not window.recorder.is_recording, "the recording did not stop")
|
||||||
check(report is not None and report.ok, f"recording reported problems: {getattr(report, 'problems', None)}")
|
check(report is not None and report.ok, f"recording reported problems: {getattr(report, 'problems', None)}")
|
||||||
|
check(
|
||||||
|
not window.toolbar.btn_record.isChecked(),
|
||||||
|
"the record button still looks active after the recording stopped",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
window._record_report_pending is False,
|
||||||
|
"a finished recording is still waiting for a report",
|
||||||
|
)
|
||||||
|
|
||||||
import csv as _csv
|
import csv as _csv
|
||||||
import json as _json
|
import json as _json
|
||||||
@@ -1019,6 +1097,388 @@ def main() -> int:
|
|||||||
stuck.release()
|
stuck.release()
|
||||||
_shutil.rmtree(scratch, ignore_errors=True)
|
_shutil.rmtree(scratch, ignore_errors=True)
|
||||||
|
|
||||||
|
section("record failure states")
|
||||||
|
|
||||||
|
from unittest.mock import patch as _patch
|
||||||
|
|
||||||
|
from orisys_app_shared.record.experiment import _SessionWriter as _Writer
|
||||||
|
|
||||||
|
scratch_fail = APP_ROOT / "outputs" / "_record_fail"
|
||||||
|
_shutil.rmtree(scratch_fail, ignore_errors=True)
|
||||||
|
scratch_fail.mkdir(parents=True, exist_ok=True)
|
||||||
|
stuck_fail = _StuckWorker()
|
||||||
|
fail_window = StudioMainWindow(
|
||||||
|
prefs={
|
||||||
|
"language": "zh_CN",
|
||||||
|
"theme": "dark",
|
||||||
|
"sensor_type": "box",
|
||||||
|
"source_mode": "camera",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
fail_window.resize(1440, 880)
|
||||||
|
fail_window.show()
|
||||||
|
fail_window._ensure_workers = lambda: None
|
||||||
|
fail_window._box_worker = stuck_fail
|
||||||
|
fail_window._finger_worker = stuck_fail
|
||||||
|
fail_window._start_run()
|
||||||
|
fail_window._run_timer.stop()
|
||||||
|
fail_window._set_phase("running")
|
||||||
|
|
||||||
|
# A directory that cannot be created is a terminal, retryable failure.
|
||||||
|
blocked = scratch_fail / "blocked"
|
||||||
|
blocked.write_text("", encoding="utf-8")
|
||||||
|
_app_module.recordings_dir = lambda: blocked
|
||||||
|
fail_window.toolbar.btn_record.setChecked(True)
|
||||||
|
app.processEvents()
|
||||||
|
check(
|
||||||
|
not fail_window.recorder.is_recording,
|
||||||
|
"a recording started even though its directory could not be created",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
not fail_window.toolbar.btn_record.isChecked(),
|
||||||
|
"the record button stayed active after the start failed",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
fail_window.recorder.state == "failed",
|
||||||
|
f"a failed start left the recorder in {fail_window.recorder.state!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ...and the operator can simply try again.
|
||||||
|
_app_module.recordings_dir = lambda: scratch_fail
|
||||||
|
fail_window.toolbar.btn_record.setChecked(True)
|
||||||
|
app.processEvents()
|
||||||
|
check(
|
||||||
|
fail_window.recorder.is_recording,
|
||||||
|
"the recorder refused to retry after a failed directory",
|
||||||
|
)
|
||||||
|
|
||||||
|
# A writer that keeps failing must fail the recording, and the window must
|
||||||
|
# consume that report even though ``is_recording`` is already False.
|
||||||
|
with _patch.object(_Writer, "write_sample", side_effect=OSError("disk full")):
|
||||||
|
for index in range(8):
|
||||||
|
stuck_fail.emit(_fake_packet(index))
|
||||||
|
fail_window._on_run_tick()
|
||||||
|
deadline = time.monotonic() + 3.0
|
||||||
|
while (
|
||||||
|
fail_window.recorder.stats().write_errors < 5
|
||||||
|
and time.monotonic() < deadline
|
||||||
|
):
|
||||||
|
app.processEvents()
|
||||||
|
time.sleep(0.01)
|
||||||
|
check(
|
||||||
|
fail_window.recorder.stats().failed,
|
||||||
|
"repeated write failures never failed the recording",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
not fail_window.recorder.is_recording,
|
||||||
|
"a dead writer still claimed to be recording",
|
||||||
|
)
|
||||||
|
fail_window._stop_recording()
|
||||||
|
app.processEvents()
|
||||||
|
failed_report = fail_window._record_last_report
|
||||||
|
check(
|
||||||
|
failed_report is not None and failed_report.stats.failed,
|
||||||
|
"the window did not consume the failure report",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
not fail_window.toolbar.btn_record.isChecked(),
|
||||||
|
"the record button stayed active after the recording failed",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
fail_window._session_phase == "error",
|
||||||
|
f"the session phase is {fail_window._session_phase!r} after a failed recording",
|
||||||
|
)
|
||||||
|
|
||||||
|
fail_window.close()
|
||||||
|
app.processEvents()
|
||||||
|
stuck_fail.release()
|
||||||
|
_app_module.recordings_dir = lambda: scratch
|
||||||
|
_shutil.rmtree(scratch_fail, ignore_errors=True)
|
||||||
|
|
||||||
|
section("calibration record identity")
|
||||||
|
|
||||||
|
import json as _cal_json
|
||||||
|
from unittest.mock import patch as _patch_mismatch
|
||||||
|
|
||||||
|
import orisys_studio.ui.cal_panel as _cal_panel_module
|
||||||
|
import orisys_studio.calibration_service as _cal_service_module
|
||||||
|
from PyQt5.QtWidgets import QMessageBox as _QMessageBox
|
||||||
|
|
||||||
|
from orisys_app_shared.force.calibration_io import (
|
||||||
|
build_force_calibration_v2 as _build_v2b,
|
||||||
|
write_force_calibration as _write_cal,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.force.calibration_session import (
|
||||||
|
build_session_document as _build_session_doc,
|
||||||
|
)
|
||||||
|
|
||||||
|
record_dir = APP_ROOT / "outputs" / "_cal_record_check"
|
||||||
|
if record_dir.exists():
|
||||||
|
for stale in record_dir.iterdir():
|
||||||
|
stale.unlink()
|
||||||
|
record_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
identity_window = StudioMainWindow(
|
||||||
|
prefs={
|
||||||
|
"language": "zh_CN",
|
||||||
|
"theme": "dark",
|
||||||
|
"sensor_type": "box",
|
||||||
|
"source_mode": "camera",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
identity_window.resize(1400, 880)
|
||||||
|
identity_window.show()
|
||||||
|
identity_window.cal_panel.set_running(True)
|
||||||
|
taken_points = []
|
||||||
|
for load, raw in ((1.0, 100.0), (3.0, 300.0)):
|
||||||
|
identity_window.cal_recorder.start(
|
||||||
|
channel="fn", reference_load_n=load, duration_s=1.0
|
||||||
|
)
|
||||||
|
for index in range(20):
|
||||||
|
identity_window.cal_recorder.feed(
|
||||||
|
_calibration_sample(index, fn_raw=raw)
|
||||||
|
)
|
||||||
|
taken_points.append(identity_window.cal_recorder.finish_now())
|
||||||
|
device_a = record_dir / "device_a_session.json"
|
||||||
|
device_a.write_text(
|
||||||
|
_cal_json.dumps(
|
||||||
|
_build_session_doc(
|
||||||
|
taken_points,
|
||||||
|
meta={
|
||||||
|
"sensor_type": "box",
|
||||||
|
"sn": "SN-A",
|
||||||
|
"config_ref": "D:/lab/deviceA/sensor.json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
identity_window.cal_recorder.clear_points()
|
||||||
|
|
||||||
|
# The live device is B; opening A's record must not restamp it.
|
||||||
|
panel = identity_window.cal_panel
|
||||||
|
panel.set_meta_context(sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json")
|
||||||
|
panel.sn_edit.setText("SN-B")
|
||||||
|
panel._on_open_record(str(device_a))
|
||||||
|
app.processEvents()
|
||||||
|
check(
|
||||||
|
panel.sn_edit.text() == "SN-A",
|
||||||
|
f"the reopened record's SN was overwritten: {panel.sn_edit.text()!r}",
|
||||||
|
)
|
||||||
|
meta = panel._export_meta()
|
||||||
|
check(meta["sensor_type"] == "box", "the record's sensor type was overwritten")
|
||||||
|
check(meta["sn"] == "SN-A", "the record's SN was not kept")
|
||||||
|
check(
|
||||||
|
meta["config_ref"] == "D:/lab/deviceA/sensor.json",
|
||||||
|
f"the record's config was overwritten: {meta['config_ref']!r}",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
str(meta.get("source_record", "")).endswith(device_a.name),
|
||||||
|
"the export does not reference the source record",
|
||||||
|
)
|
||||||
|
saved = sorted(record_dir.glob("calibration_session_box_*.json"))
|
||||||
|
with _patch_mismatch.object(
|
||||||
|
_cal_panel_module, "calibration_export_dir", lambda: record_dir
|
||||||
|
):
|
||||||
|
with _patch_mismatch.object(
|
||||||
|
_cal_service_module, "calibration_export_dir", lambda: record_dir
|
||||||
|
):
|
||||||
|
with _patch_mismatch.object(
|
||||||
|
_cal_panel_module.QMessageBox, "information", lambda *a, **k: None
|
||||||
|
):
|
||||||
|
panel._on_save_record()
|
||||||
|
saved = sorted(record_dir.glob("calibration_session_box_*.json"))
|
||||||
|
check(bool(saved), "saving a reopened record produced no file")
|
||||||
|
if saved:
|
||||||
|
saved_meta = _cal_json.loads(saved[-1].read_text(encoding="utf-8"))["meta"]
|
||||||
|
check(
|
||||||
|
saved_meta.get("sn") == "SN-A" and saved_meta.get("sensor_type") == "box",
|
||||||
|
f"the saved record was restamped: {saved_meta.get('sn')!r}",
|
||||||
|
)
|
||||||
|
doc, _warnings = _build_v2b(panel._prepared)
|
||||||
|
check(
|
||||||
|
doc["meta"]["sn"] == "SN-A" and doc["meta"]["sensor_type"] == "box",
|
||||||
|
"the exported coefficients carry the wrong device identity",
|
||||||
|
)
|
||||||
|
|
||||||
|
# A reopened point with only its samples can still be inspected.
|
||||||
|
reopened = identity_window.cal_recorder.points[0]
|
||||||
|
for entry in _build_session_doc([reopened])["points"]:
|
||||||
|
entry["trace"] = {}
|
||||||
|
dialog = panel.point_dialog(reopened)
|
||||||
|
check(dialog is not None, "a reopened point could not be inspected")
|
||||||
|
if dialog is not None:
|
||||||
|
check(
|
||||||
|
len(dialog.plot_widget.ci.items) == 3,
|
||||||
|
"the rebuilt diagnostic is not one trace per axis",
|
||||||
|
)
|
||||||
|
dialog.close()
|
||||||
|
|
||||||
|
# Applying a calibration that names another device is flagged and defaults
|
||||||
|
# to "no".
|
||||||
|
stuck_identity = _StuckWorker()
|
||||||
|
identity_window._ensure_workers = lambda: None
|
||||||
|
identity_window._box_worker = stuck_identity
|
||||||
|
identity_window._finger_worker = stuck_identity
|
||||||
|
coefficients = record_dir / "device_a_coefficients.json"
|
||||||
|
_write_cal(doc, coefficients)
|
||||||
|
check(
|
||||||
|
bool(
|
||||||
|
identity_window.calibration_target_mismatch(
|
||||||
|
str(coefficients),
|
||||||
|
sensor_type="finger",
|
||||||
|
config_ref="D:/lab/B/sensor.json",
|
||||||
|
)
|
||||||
|
),
|
||||||
|
"a calibration for another device was not reported as a mismatch",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
not identity_window.calibration_target_mismatch(
|
||||||
|
str(coefficients),
|
||||||
|
sensor_type="box",
|
||||||
|
config_ref="D:/lab/deviceA/sensor.json",
|
||||||
|
),
|
||||||
|
"matching identities were reported as a mismatch",
|
||||||
|
)
|
||||||
|
with _patch_mismatch.object(
|
||||||
|
_QMessageBox, "question", staticmethod(lambda *a, **k: _QMessageBox.No)
|
||||||
|
):
|
||||||
|
identity_window._on_calibration_apply(str(coefficients))
|
||||||
|
check(
|
||||||
|
not stuck_identity.calibration_requests,
|
||||||
|
"a mismatched calibration was applied after the user declined",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
identity_window.statusBar().currentMessage() == t("cal.apply_mismatch_skipped"),
|
||||||
|
f"the refusal was not reported: {identity_window.statusBar().currentMessage()!r}",
|
||||||
|
)
|
||||||
|
# A matching calibration still goes through the normal confirmation.
|
||||||
|
identity_window.sensor_picker.set_sensor_type("box")
|
||||||
|
with _patch_mismatch.object(
|
||||||
|
_QMessageBox, "question", staticmethod(lambda *a, **k: _QMessageBox.Yes)
|
||||||
|
):
|
||||||
|
identity_window._on_calibration_apply(str(coefficients))
|
||||||
|
check(
|
||||||
|
stuck_identity.calibration_requests == [str(coefficients)],
|
||||||
|
f"a matching calibration was not requested: {stuck_identity.calibration_requests}",
|
||||||
|
)
|
||||||
|
|
||||||
|
identity_window.close()
|
||||||
|
app.processEvents()
|
||||||
|
_shutil.rmtree(record_dir, ignore_errors=True)
|
||||||
|
|
||||||
|
section("display units and time base")
|
||||||
|
import math as _math
|
||||||
|
|
||||||
|
stuck_units = _StuckWorker()
|
||||||
|
units_window = StudioMainWindow(
|
||||||
|
prefs={
|
||||||
|
"language": "zh_CN",
|
||||||
|
"theme": "dark",
|
||||||
|
"sensor_type": "box",
|
||||||
|
"source_mode": "camera",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
units_window.resize(1440, 880)
|
||||||
|
units_window.show()
|
||||||
|
units_window._ensure_workers = lambda: None
|
||||||
|
units_window._box_worker = stuck_units
|
||||||
|
units_window._finger_worker = stuck_units
|
||||||
|
units_window._start_run()
|
||||||
|
units_window._run_timer.stop()
|
||||||
|
units_window._set_phase("running")
|
||||||
|
units_plots = units_window.force_plots
|
||||||
|
units_window.toolbar.units_combo.setCurrentIndex(0) # newton
|
||||||
|
units_plots.reset()
|
||||||
|
wall = 1_700_000_000.0
|
||||||
|
for index in range(90):
|
||||||
|
stuck_units.emit(
|
||||||
|
_fake_packet(index, timestamp=wall + index / 30.0, mono_t=index / 30.0)
|
||||||
|
)
|
||||||
|
units_window._on_run_tick()
|
||||||
|
before = units_plots.history.state()
|
||||||
|
end_before = float(before.times[-1]) if before is not None else -1.0
|
||||||
|
check(
|
||||||
|
before is not None and end_before < 60.0,
|
||||||
|
f"the live X axis is not session-relative (ends at {end_before})",
|
||||||
|
)
|
||||||
|
for units_index in (1, 0, 1, 0):
|
||||||
|
units_window.toolbar.units_combo.setCurrentIndex(units_index)
|
||||||
|
app.processEvents()
|
||||||
|
after = units_plots.history.state()
|
||||||
|
check(
|
||||||
|
after is not None and abs(float(after.times[-1]) - end_before) < 1.0,
|
||||||
|
"the X axis jumped across a raw/N switch: "
|
||||||
|
f"{end_before} -> {None if after is None else after.times[-1]}",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
units_plots.history.sample_count > 0,
|
||||||
|
"the rebuilt history lost every sample",
|
||||||
|
)
|
||||||
|
units_window.close()
|
||||||
|
units_window._box_worker = None
|
||||||
|
stuck_units.release()
|
||||||
|
app.processEvents()
|
||||||
|
|
||||||
|
# A device with only its normal-force axis calibrated: FN in newtons, and the
|
||||||
|
# other two axes explicitly unavailable instead of showing raw counts.
|
||||||
|
stuck_one = _StuckWorker()
|
||||||
|
one_window = StudioMainWindow(
|
||||||
|
prefs={
|
||||||
|
"language": "zh_CN",
|
||||||
|
"theme": "dark",
|
||||||
|
"sensor_type": "box",
|
||||||
|
"source_mode": "camera",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
one_window.resize(1440, 880)
|
||||||
|
one_window.show()
|
||||||
|
one_window._ensure_workers = lambda: None
|
||||||
|
one_window._box_worker = stuck_one
|
||||||
|
one_window._finger_worker = stuck_one
|
||||||
|
one_window._start_run()
|
||||||
|
one_window._run_timer.stop()
|
||||||
|
one_window._set_phase("running")
|
||||||
|
one_window.toolbar.units_combo.setCurrentIndex(0) # newton
|
||||||
|
one_window.force_plots.reset()
|
||||||
|
for index in range(10):
|
||||||
|
stuck_one.emit(
|
||||||
|
_fake_packet(index, calibrated_axes=("fn",), shear_n=float("nan"))
|
||||||
|
)
|
||||||
|
one_window._on_run_tick()
|
||||||
|
app.processEvents()
|
||||||
|
one_plots = one_window.force_plots
|
||||||
|
check(
|
||||||
|
one_plots._available == (True, False, False),
|
||||||
|
f"per-axis availability is wrong: {one_plots._available}",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
one_plots.value_labels[0].text() != t("chart.value_unavailable"),
|
||||||
|
"the calibrated axis was reported as unavailable",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
all(
|
||||||
|
one_plots.value_labels[index].text() == t("chart.value_unavailable")
|
||||||
|
for index in (1, 2)
|
||||||
|
),
|
||||||
|
"an axis without a curve showed a number in newton mode",
|
||||||
|
)
|
||||||
|
one_state = one_plots.history.state()
|
||||||
|
check(
|
||||||
|
one_state is not None
|
||||||
|
and _math.isfinite(float(one_state.values[:, 0].max())),
|
||||||
|
"the calibrated axis was not plotted",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
one_state is not None
|
||||||
|
and not any(_math.isfinite(v) for v in one_state.values[:, 1]),
|
||||||
|
"FX carried a value without a calibration curve",
|
||||||
|
)
|
||||||
|
one_window.close()
|
||||||
|
one_window._box_worker = None
|
||||||
|
stuck_one.release()
|
||||||
|
app.processEvents()
|
||||||
|
|
||||||
section("session browsing")
|
section("session browsing")
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
@@ -1190,6 +1650,31 @@ def main() -> int:
|
|||||||
all(int(curve.getData()[0].size) == 40 for curve in playback.curves),
|
all(int(curve.getData()[0].size) == 40 for curve in playback.curves),
|
||||||
"replay did not draw every force sample",
|
"replay did not draw every force sample",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Long-history browsing: an interval read is bounded by the interval, and the
|
||||||
|
# whole-session envelope is a band rather than every row.
|
||||||
|
from orisys_app_shared.force.source import RecordingCurveSource
|
||||||
|
from orisys_app_shared.record.index import RecordingIndex
|
||||||
|
from orisys_app_shared.record.playback import RecordingReader as _RecordingReader
|
||||||
|
|
||||||
|
reader = _RecordingReader(recording, max_rows=10)
|
||||||
|
source = RecordingCurveSource(reader)
|
||||||
|
check(source.kind == "disk", "the replay source is not the recording")
|
||||||
|
index = reader.index()
|
||||||
|
check(index.rows == 40, f"the index found {index.rows} rows, not 40")
|
||||||
|
start, stop = index.row_range(0.5, 1.0)
|
||||||
|
check(
|
||||||
|
0 <= start < stop <= index.rows and (stop - start) <= 20,
|
||||||
|
f"an interval read spans {stop - start} rows",
|
||||||
|
)
|
||||||
|
interval = source.window(0.5, 1.0, unit="N")
|
||||||
|
check(0 < len(interval) <= 20, f"the interval read returned {len(interval)} rows")
|
||||||
|
envelope = source.envelope(buckets=8, unit="N")
|
||||||
|
check(0 < len(envelope) <= 8, f"the envelope has {len(envelope)} points")
|
||||||
|
check(
|
||||||
|
reader.info().total_rows == 40,
|
||||||
|
f"the recording reports {reader.info().total_rows} rows",
|
||||||
|
)
|
||||||
check(playback.slider.isEnabled(), "the replay position slider is disabled")
|
check(playback.slider.isEnabled(), "the replay position slider is disabled")
|
||||||
check(
|
check(
|
||||||
t("playback.warn_depth_recomputed").split("{")[0].strip()[:4]
|
t("playback.warn_depth_recomputed").split("{")[0].strip()[:4]
|
||||||
@@ -1268,6 +1753,92 @@ def main() -> int:
|
|||||||
|
|
||||||
playback.close()
|
playback.close()
|
||||||
app.processEvents()
|
app.processEvents()
|
||||||
|
|
||||||
|
# The plan must state where to start, which reset it is anchored to and
|
||||||
|
# whether the state is really recoverable; and a request must belong to the
|
||||||
|
# recording it started in.
|
||||||
|
from orisys_app_shared.record.depth import (
|
||||||
|
STATE_APPROXIMATE,
|
||||||
|
STATE_PREROLL,
|
||||||
|
DepthRecomputer,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.record.playback import RecordingReader
|
||||||
|
|
||||||
|
plan_reader = RecordingReader(recording)
|
||||||
|
planner = DepthRecomputer(plan_reader)
|
||||||
|
frames = plan_reader.info().frames
|
||||||
|
check(bool(frames), "the replay recording has no frame index")
|
||||||
|
plan = planner.plan_for(frames[10])
|
||||||
|
check(
|
||||||
|
plan.state == STATE_APPROXIMATE and plan.anchor == "segment_start",
|
||||||
|
f"a recording with no reset/pre-roll is not reported as approximate: {plan.state}",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
plan.steps and plan.steps[0].source == plan_reader.segment_path(frames[10].segment),
|
||||||
|
"the plan does not name the video to read",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
"state_approximate" in plan.warnings,
|
||||||
|
f"the approximate replay is not reported: {plan.warnings}",
|
||||||
|
)
|
||||||
|
# A pre-roll makes the start of a recording reproducible.
|
||||||
|
session_path = recording / "session.json"
|
||||||
|
session_doc = _json.loads(session_path.read_text(encoding="utf-8"))
|
||||||
|
prerecord = recording / "preroll_0001.mp4"
|
||||||
|
prerecord.write_bytes(b"")
|
||||||
|
session_doc["preroll_frames"] = 30
|
||||||
|
session_doc["preroll_segment"] = prerecord.name
|
||||||
|
session_path.write_text(_json.dumps(session_doc), encoding="utf-8")
|
||||||
|
planner.close()
|
||||||
|
planner = DepthRecomputer(RecordingReader(recording))
|
||||||
|
plan = planner.plan_for(planner.reader.info().frames[5])
|
||||||
|
check(
|
||||||
|
plan.state == STATE_PREROLL and plan.preroll == prerecord,
|
||||||
|
f"the pre-roll was not used for the start of the recording: {plan.state}",
|
||||||
|
)
|
||||||
|
check(plan.frames_to_read == 30 + 6, f"unexpected replay cost: {plan.frames_to_read}")
|
||||||
|
planner.close()
|
||||||
|
|
||||||
|
# A reset boundary is re-applied, and a request is bound to its recording.
|
||||||
|
_Blocking.released.clear()
|
||||||
|
second = PlaybackDialog(theme="dark", directory=recording)
|
||||||
|
second.show()
|
||||||
|
app.processEvents()
|
||||||
|
serial_before = second._record_serial
|
||||||
|
blocking = _Blocking()
|
||||||
|
second._recomputer = blocking
|
||||||
|
second._on_recompute_depth()
|
||||||
|
check(second._depth_pending_record == serial_before, "the request has no recording id")
|
||||||
|
check(
|
||||||
|
second._depth_pending_seq is not None,
|
||||||
|
"the request carries no sequence number",
|
||||||
|
)
|
||||||
|
# Opening another recording invalidates the running request...
|
||||||
|
second.open_recording(recording)
|
||||||
|
check(
|
||||||
|
second._record_serial == serial_before + 1,
|
||||||
|
"opening a recording did not invalidate the running request",
|
||||||
|
)
|
||||||
|
check(
|
||||||
|
second._recomputer is not blocking,
|
||||||
|
"the invalidated processor was not replaced",
|
||||||
|
)
|
||||||
|
_Blocking.released.set()
|
||||||
|
for _ in range(40):
|
||||||
|
app.processEvents()
|
||||||
|
_time.sleep(0.05)
|
||||||
|
if not second._depth_busy():
|
||||||
|
break
|
||||||
|
second._poll_depth()
|
||||||
|
app.processEvents()
|
||||||
|
pixmap = second.depth_label.pixmap()
|
||||||
|
check(
|
||||||
|
pixmap is None or pixmap.isNull(),
|
||||||
|
"an answer for a previous recording was drawn into the new one",
|
||||||
|
)
|
||||||
|
second.close()
|
||||||
|
app.processEvents()
|
||||||
|
|
||||||
_shutil.rmtree(replay_root, ignore_errors=True)
|
_shutil.rmtree(replay_root, ignore_errors=True)
|
||||||
|
|
||||||
print(f"\nscenes: {scene_count} checks: {CHECKS} failures: {len(FAILURES)}")
|
print(f"\nscenes: {scene_count} checks: {CHECKS} failures: {len(FAILURES)}")
|
||||||
|
|||||||
@@ -14,20 +14,25 @@ from orisys_app_shared.acquisition.constants import (
|
|||||||
LOWPASS_CUTOFF_HZ,
|
LOWPASS_CUTOFF_HZ,
|
||||||
)
|
)
|
||||||
from orisys_app_shared.acquisition.sinks import RecordSinkMixin
|
from orisys_app_shared.acquisition.sinks import RecordSinkMixin
|
||||||
|
# The packet and the geometry come from ``models``, so the finger worker does not
|
||||||
|
# depend on the box worker; only genuinely shared helpers are imported from it.
|
||||||
|
from orisys_app_shared.acquisition.models import (
|
||||||
|
FixedCameraMode,
|
||||||
|
SdkPacket,
|
||||||
|
ensure_bgr as _ensure_bgr,
|
||||||
|
is_file_source as _is_file_source,
|
||||||
|
load_fixed_mode,
|
||||||
|
orisys_importable,
|
||||||
|
)
|
||||||
from orisys_app_shared.acquisition.sdk_worker import (
|
from orisys_app_shared.acquisition.sdk_worker import (
|
||||||
CAMERA_DISCONNECT_TIMEOUT_S,
|
CAMERA_DISCONNECT_TIMEOUT_S,
|
||||||
HANDOVER_TIMEOUT_S,
|
HANDOVER_TIMEOUT_S,
|
||||||
STOP_JOIN_TIMEOUT_S,
|
STOP_JOIN_TIMEOUT_S,
|
||||||
VIDEO_EOF_TIMEOUT_S,
|
VIDEO_EOF_TIMEOUT_S,
|
||||||
FixedCameraMode,
|
|
||||||
SdkPacket,
|
|
||||||
_ensure_bgr,
|
|
||||||
_is_file_source,
|
|
||||||
calibration_digest,
|
calibration_digest,
|
||||||
load_fixed_mode,
|
|
||||||
loaded_calibration_version,
|
loaded_calibration_version,
|
||||||
loaded_path,
|
loaded_path,
|
||||||
orisys_importable,
|
process_chain,
|
||||||
read_forces_from_sensor,
|
read_forces_from_sensor,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -65,13 +70,17 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
*,
|
*,
|
||||||
on_error: Optional[Callable[[str], None]] = None,
|
on_error: Optional[Callable[[str], None]] = None,
|
||||||
on_status: Optional[Callable[[str], None]] = None,
|
on_status: Optional[Callable[[str], None]] = None,
|
||||||
on_calibration_applied: Optional[Callable[[str, str, int], None]] = None,
|
on_calibration_applied: Optional[Callable[[str, str, int, float, int], None]] = None,
|
||||||
|
on_reset_applied: Optional[Callable[[int, float, int], None]] = None,
|
||||||
):
|
):
|
||||||
self._init_sinks()
|
self._init_sinks()
|
||||||
self._on_error = on_error
|
self._on_error = on_error
|
||||||
self._on_status = on_status
|
self._on_status = on_status
|
||||||
#: Called from the acquisition thread once new coefficients are loaded.
|
#: Called from the acquisition thread once new coefficients are loaded,
|
||||||
|
#: with ``(path, version, frame_id, mono_t, session_id)``.
|
||||||
self._on_calibration_applied = on_calibration_applied
|
self._on_calibration_applied = on_calibration_applied
|
||||||
|
#: Called from the acquisition thread once a queued zeroing took effect.
|
||||||
|
self._on_reset_applied = on_reset_applied
|
||||||
self._thread: Optional[threading.Thread] = None
|
self._thread: Optional[threading.Thread] = None
|
||||||
self._stop = threading.Event()
|
self._stop = threading.Event()
|
||||||
self._lock = threading.Lock()
|
self._lock = threading.Lock()
|
||||||
@@ -86,6 +95,8 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._end_reason: Optional[str] = None
|
self._end_reason: Optional[str] = None
|
||||||
self._phase: str = "idle"
|
self._phase: str = "idle"
|
||||||
self._pending_calibration: Optional[str] = None
|
self._pending_calibration: Optional[str] = None
|
||||||
|
#: A zeroing requested from another thread, applied by this one.
|
||||||
|
self._pending_reset: bool = False
|
||||||
self._mono_t0: float = 0.0
|
self._mono_t0: float = 0.0
|
||||||
self._calibration_version: str = ""
|
self._calibration_version: str = ""
|
||||||
|
|
||||||
@@ -129,6 +140,11 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._latest = None
|
self._latest = None
|
||||||
self._end_reason = None
|
self._end_reason = None
|
||||||
self._phase = "connecting"
|
self._phase = "connecting"
|
||||||
|
# This session starts with no calibration of its own: an uncalibrated
|
||||||
|
# runtime must not inherit the previous session's version, and a stale
|
||||||
|
# zeroing request must not fire into the new session's frames.
|
||||||
|
self._calibration_version = ""
|
||||||
|
self._pending_reset = False
|
||||||
self._session_id += 1
|
self._session_id += 1
|
||||||
session = self._session_id
|
session = self._session_id
|
||||||
self._stop.clear()
|
self._stop.clear()
|
||||||
@@ -171,22 +187,54 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._phase = "idle"
|
self._phase = "idle"
|
||||||
|
|
||||||
def reset(self) -> None:
|
def reset(self) -> None:
|
||||||
|
"""Request a tracker reset, applied by the acquisition thread.
|
||||||
|
|
||||||
|
The runtime's tactile reset from the GUI thread would race the frame the
|
||||||
|
loop is processing, so the zeroing happens at a frame boundary on the
|
||||||
|
acquisition thread, which reports the frame and time it took effect.
|
||||||
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
runtime = self._runtimes.get(self._session_id)
|
if self._session_id <= 0:
|
||||||
if runtime is None:
|
|
||||||
raise RuntimeError("Finger runtime is not running")
|
raise RuntimeError("Finger runtime is not running")
|
||||||
runtime.tactile.reset()
|
self._pending_reset = True
|
||||||
|
|
||||||
|
def _consume_pending_reset(self, runtime, frame_id: int, mono_t: float) -> None:
|
||||||
|
"""Apply a queued reset and report the frame it actually took effect at."""
|
||||||
|
with self._lock:
|
||||||
|
pending = self._pending_reset
|
||||||
|
self._pending_reset = False
|
||||||
|
session = self._session_id
|
||||||
|
if not pending or runtime is None:
|
||||||
|
return
|
||||||
|
tactile = getattr(runtime, "tactile", None)
|
||||||
|
target = tactile if callable(getattr(tactile, "reset", None)) else None
|
||||||
|
if target is None:
|
||||||
|
sensor = getattr(tactile, "raw_sensor", None)
|
||||||
|
target = sensor if callable(getattr(sensor, "reset", None)) else None
|
||||||
|
if target is None:
|
||||||
|
raise RuntimeError("runtime does not support reset")
|
||||||
|
target.reset()
|
||||||
|
if self._on_reset_applied is not None:
|
||||||
|
self._on_reset_applied(int(frame_id), float(mono_t), int(session))
|
||||||
|
|
||||||
def set_pending_calibration(self, path: str | Path | None) -> None:
|
def set_pending_calibration(self, path: str | Path | None) -> None:
|
||||||
"""Queue a force_calibration.json path for the acquisition thread."""
|
"""Queue a force_calibration.json path for the acquisition thread."""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._pending_calibration = str(path) if path is not None else None
|
self._pending_calibration = str(path) if path is not None else None
|
||||||
|
|
||||||
def _consume_pending_calibration(self, sensor, frame_id: int = 0) -> None:
|
def _consume_pending_calibration(
|
||||||
"""Apply queued coefficients and report confirmation, not intent."""
|
self, sensor, frame_id: int = 0, mono_t: float = 0.0
|
||||||
|
) -> None:
|
||||||
|
"""Apply queued coefficients and report confirmation, not intent.
|
||||||
|
|
||||||
|
A rejected document leaves the version alone on purpose: the sensor still
|
||||||
|
has the coefficients it had. What never happens is inventing a version for
|
||||||
|
a device that has no calibration — that is the empty string.
|
||||||
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
path = self._pending_calibration
|
path = self._pending_calibration
|
||||||
self._pending_calibration = None
|
self._pending_calibration = None
|
||||||
|
session = self._session_id
|
||||||
if path is None or sensor is None:
|
if path is None or sensor is None:
|
||||||
return
|
return
|
||||||
if not hasattr(sensor, "set_calibration"):
|
if not hasattr(sensor, "set_calibration"):
|
||||||
@@ -195,7 +243,9 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
version = calibration_digest(path)
|
version = calibration_digest(path)
|
||||||
self._calibration_version = version
|
self._calibration_version = version
|
||||||
if self._on_calibration_applied is not None:
|
if self._on_calibration_applied is not None:
|
||||||
self._on_calibration_applied(str(path), version, int(frame_id))
|
self._on_calibration_applied(
|
||||||
|
str(path), version, int(frame_id), float(mono_t), int(session)
|
||||||
|
)
|
||||||
|
|
||||||
def _set_end_reason(self, reason: str) -> None:
|
def _set_end_reason(self, reason: str) -> None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -287,11 +337,12 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
# name those coefficients from the first frame, not the request path.
|
# name those coefficients from the first frame, not the request path.
|
||||||
tactile = getattr(runtime, "tactile", None)
|
tactile = getattr(runtime, "tactile", None)
|
||||||
loaded = loaded_calibration_version(tactile) if tactile else ""
|
loaded = loaded_calibration_version(tactile) if tactile else ""
|
||||||
if loaded:
|
# Assigned even when empty: an uncalibrated device reports *no*
|
||||||
|
# calibration, never the previous session's version.
|
||||||
self._calibration_version = loaded
|
self._calibration_version = loaded
|
||||||
if self._on_calibration_applied is not None:
|
if loaded and self._on_calibration_applied is not None:
|
||||||
self._on_calibration_applied(
|
self._on_calibration_applied(
|
||||||
str(loaded_path(tactile) or ""), loaded, 0
|
str(loaded_path(tactile) or ""), loaded, 0, 0.0, int(session)
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
self._set_end_reason("error")
|
self._set_end_reason("error")
|
||||||
@@ -341,20 +392,26 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
none_streak_start = None
|
none_streak_start = None
|
||||||
tactile = runtime.tactile
|
tactile = runtime.tactile
|
||||||
compute_error = ""
|
compute_error = ""
|
||||||
|
# The frame's acquisition time, so an event that happens on this
|
||||||
|
# frame is dated when it happened rather than when it was queued.
|
||||||
|
frame_mono = max(0.0, time.perf_counter() - self._mono_t0)
|
||||||
try:
|
try:
|
||||||
self._consume_pending_calibration(tactile, frame_id)
|
self._consume_pending_calibration(tactile, frame_id, frame_mono)
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
message = f"calibration rejected: {type(exc).__name__}: {exc}"
|
message = f"calibration rejected: {type(exc).__name__}: {exc}"
|
||||||
if self._on_error is not None and session == self._session_id:
|
if self._on_error is not None and session == self._session_id:
|
||||||
self._on_error(message)
|
self._on_error(message)
|
||||||
compute_error = message
|
compute_error = message
|
||||||
try:
|
try:
|
||||||
tactile.compute_deformation(decompose=True, check_motion=False)
|
# Applied between two frames on this thread, then reported.
|
||||||
tactile.compute_contact()
|
self._consume_pending_reset(runtime, frame_id, frame_mono)
|
||||||
except Exception as exc: # noqa: BLE001 - reported on the packet
|
except Exception as exc: # noqa: BLE001
|
||||||
compute_error = compute_error or (
|
message = f"reset failed: {type(exc).__name__}: {exc}"
|
||||||
f"compute: {type(exc).__name__}: {exc}"
|
if self._on_error is not None and session == self._session_id:
|
||||||
)
|
self._on_error(message)
|
||||||
|
# The same chain the offline depth replay runs on the tactile.
|
||||||
|
chain_depth, chain_error = process_chain(tactile)
|
||||||
|
compute_error = compute_error or chain_error
|
||||||
t1 = time.perf_counter()
|
t1 = time.perf_counter()
|
||||||
frame_times.append(t1)
|
frame_times.append(t1)
|
||||||
cutoff = t1 - 1.0
|
cutoff = t1 - 1.0
|
||||||
@@ -395,7 +452,11 @@ class FingerAcquisitionWorker(RecordSinkMixin):
|
|||||||
stale_forces=bool(compute_error),
|
stale_forces=bool(compute_error),
|
||||||
raw_available=bool(forces.get("raw_available", True)),
|
raw_available=bool(forces.get("raw_available", True)),
|
||||||
flow=forces["flow"],
|
flow=forces["flow"],
|
||||||
depth=forces.get("depth"),
|
depth=(
|
||||||
|
forces.get("depth")
|
||||||
|
if forces.get("depth") is not None
|
||||||
|
else chain_depth
|
||||||
|
),
|
||||||
valid_mask=runtime.valid_mask,
|
valid_mask=runtime.valid_mask,
|
||||||
)
|
)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
|
|||||||
@@ -0,0 +1,131 @@
|
|||||||
|
"""The data shapes shared by the acquisition workers, the record channel and the UI.
|
||||||
|
|
||||||
|
They live here rather than in one of the workers so that neither worker has to
|
||||||
|
import the other, and so the record channel can describe a packet without
|
||||||
|
importing a device loop. The workers re-export these names for compatibility.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FixedCameraMode:
|
||||||
|
"""The camera geometry a session runs with."""
|
||||||
|
|
||||||
|
width: int
|
||||||
|
height: int
|
||||||
|
fps: float
|
||||||
|
config_ref: str
|
||||||
|
|
||||||
|
def label(self) -> str:
|
||||||
|
return f"{self.width}x{self.height} @{self.fps:g}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SdkPacket:
|
||||||
|
"""One acquired frame: the images, the force readout and its provenance."""
|
||||||
|
|
||||||
|
raw_frame: np.ndarray
|
||||||
|
processed_frame: np.ndarray
|
||||||
|
timestamp: float
|
||||||
|
fps: float
|
||||||
|
process_ms: float
|
||||||
|
normal_force: float = 0.0
|
||||||
|
shear_x: float = 0.0
|
||||||
|
shear_y: float = 0.0
|
||||||
|
normal_force_raw: float = 0.0
|
||||||
|
shear_x_raw: float = 0.0
|
||||||
|
shear_y_raw: float = 0.0
|
||||||
|
force_unit: str = "raw"
|
||||||
|
force_calibrated: bool = False
|
||||||
|
#: Which axes actually have a calibration curve loaded. The sensor-level
|
||||||
|
#: ``force_calibrated`` flag cannot express a partially calibrated device.
|
||||||
|
calibrated_axes: tuple[str, ...] = ()
|
||||||
|
session_id: int = 0
|
||||||
|
#: Monotonic per-session frame index; the join key for video, CSV and
|
||||||
|
#: calibration records.
|
||||||
|
frame_id: int = 0
|
||||||
|
#: Monotonic, session-relative acquisition time. Used for ordering so a wall
|
||||||
|
#: clock that steps backwards cannot reorder history. ``None`` means the
|
||||||
|
#: producer did not measure one, and the wall clock is used instead — a
|
||||||
|
#: default of ``0.0`` would have made every frame look simultaneous.
|
||||||
|
mono_t: Optional[float] = None
|
||||||
|
#: Wall clock at hand-off, for metadata only. ``None`` means unmeasured.
|
||||||
|
wall_t: Optional[float] = None
|
||||||
|
#: False when the force readout failed. Invalid packets carry NaN forces and
|
||||||
|
#: must never be averaged into calibration data or written as zeros.
|
||||||
|
valid: bool = True
|
||||||
|
invalid_reason: str = ""
|
||||||
|
#: True when the compute chain failed for this frame. The SDK's force fields
|
||||||
|
#: then still hold the value computed for an earlier frame, so they must not
|
||||||
|
#: be recorded as this frame's measurement.
|
||||||
|
stale_forces: bool = False
|
||||||
|
#: Content hash of the calibration actually in effect when this frame was
|
||||||
|
#: acquired; changes mid-session when new coefficients are applied.
|
||||||
|
calibration_version: str = ""
|
||||||
|
#: False when the sensor exposes no RAW channels; the raw fields are then
|
||||||
|
#: NaN rather than a copy of the calibrated values.
|
||||||
|
raw_available: bool = True
|
||||||
|
flow: Optional[np.ndarray] = None
|
||||||
|
depth: Optional[np.ndarray] = None
|
||||||
|
valid_mask: Optional[np.ndarray] = None
|
||||||
|
|
||||||
|
|
||||||
|
def load_fixed_mode(config_ref: str | Path) -> FixedCameraMode:
|
||||||
|
"""Resolve a config reference (a file path or a product name) to its geometry."""
|
||||||
|
ref = str(config_ref).strip()
|
||||||
|
path = Path(ref)
|
||||||
|
if path.is_file():
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
resolved = str(path.resolve())
|
||||||
|
else:
|
||||||
|
import orisys
|
||||||
|
|
||||||
|
data = orisys.configs.load_config(ref)
|
||||||
|
resolved = ref
|
||||||
|
camera = data.get("camera") or {}
|
||||||
|
raw = camera.get("resolution_raw") or [1920, 1080]
|
||||||
|
fps = float(camera.get("fps") or 30.0)
|
||||||
|
return FixedCameraMode(
|
||||||
|
width=int(raw[0]),
|
||||||
|
height=int(raw[1]),
|
||||||
|
fps=fps,
|
||||||
|
config_ref=resolved,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_bgr(img: np.ndarray) -> np.ndarray:
|
||||||
|
"""Three-channel BGR view of whatever the sensor hands over."""
|
||||||
|
if img.ndim == 2:
|
||||||
|
return cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
|
||||||
|
if img.ndim == 3 and img.shape[2] == 1:
|
||||||
|
return cv2.cvtColor(img[:, :, 0], cv2.COLOR_GRAY2BGR)
|
||||||
|
if img.ndim == 3 and img.shape[2] == 4:
|
||||||
|
return cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)
|
||||||
|
return img
|
||||||
|
|
||||||
|
|
||||||
|
def is_file_source(video_source: str | int) -> bool:
|
||||||
|
"""True when the source is a file (a replay) rather than a live camera."""
|
||||||
|
return isinstance(video_source, str)
|
||||||
|
|
||||||
|
|
||||||
|
def orisys_importable() -> tuple[bool, str]:
|
||||||
|
"""Whether the SDK can be imported, with the reason when it cannot."""
|
||||||
|
try:
|
||||||
|
import orisys # noqa: F401
|
||||||
|
|
||||||
|
return True, getattr(__import__("orisys"), "__version__", "unknown")
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return False, str(exc)
|
||||||
@@ -19,6 +19,14 @@ from orisys_app_shared.acquisition.constants import (
|
|||||||
FORCE_SAMPLE_RATE_HZ,
|
FORCE_SAMPLE_RATE_HZ,
|
||||||
LOWPASS_CUTOFF_HZ,
|
LOWPASS_CUTOFF_HZ,
|
||||||
)
|
)
|
||||||
|
from orisys_app_shared.acquisition.models import (
|
||||||
|
FixedCameraMode,
|
||||||
|
SdkPacket,
|
||||||
|
ensure_bgr as _ensure_bgr,
|
||||||
|
is_file_source as _is_file_source,
|
||||||
|
load_fixed_mode,
|
||||||
|
orisys_importable,
|
||||||
|
)
|
||||||
from orisys_app_shared.acquisition.sinks import RecordSinkMixin
|
from orisys_app_shared.acquisition.sinks import RecordSinkMixin
|
||||||
|
|
||||||
VIDEO_EOF_TIMEOUT_S = 0.5
|
VIDEO_EOF_TIMEOUT_S = 0.5
|
||||||
@@ -29,105 +37,22 @@ STOP_JOIN_TIMEOUT_S = 5.0
|
|||||||
#: previous session that is still winding down before opening the device.
|
#: previous session that is still winding down before opening the device.
|
||||||
HANDOVER_TIMEOUT_S = 60.0
|
HANDOVER_TIMEOUT_S = 60.0
|
||||||
|
|
||||||
|
# Re-exported so ``from ...sdk_worker import SdkPacket`` keeps working; the
|
||||||
|
# definitions live in ``models`` so neither worker depends on the other.
|
||||||
|
__all__ = [
|
||||||
|
"FixedCameraMode",
|
||||||
|
"SdkPacket",
|
||||||
|
"SdkAcquisitionWorker",
|
||||||
|
"calibration_digest",
|
||||||
|
"load_fixed_mode",
|
||||||
|
"loaded_calibration_version",
|
||||||
|
"loaded_path",
|
||||||
|
"orisys_importable",
|
||||||
|
"process_chain",
|
||||||
|
"read_depth",
|
||||||
|
"read_forces_from_sensor",
|
||||||
|
]
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class FixedCameraMode:
|
|
||||||
width: int
|
|
||||||
height: int
|
|
||||||
fps: float
|
|
||||||
config_ref: str
|
|
||||||
|
|
||||||
def label(self) -> str:
|
|
||||||
return f"{self.width}x{self.height} @{self.fps:g}"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class SdkPacket:
|
|
||||||
raw_frame: np.ndarray
|
|
||||||
processed_frame: np.ndarray
|
|
||||||
timestamp: float
|
|
||||||
fps: float
|
|
||||||
process_ms: float
|
|
||||||
normal_force: float = 0.0
|
|
||||||
shear_x: float = 0.0
|
|
||||||
shear_y: float = 0.0
|
|
||||||
normal_force_raw: float = 0.0
|
|
||||||
shear_x_raw: float = 0.0
|
|
||||||
shear_y_raw: float = 0.0
|
|
||||||
force_unit: str = "raw"
|
|
||||||
force_calibrated: bool = False
|
|
||||||
#: Which axes actually have a calibration curve loaded. The sensor-level
|
|
||||||
#: ``force_calibrated`` flag cannot express a partially calibrated device.
|
|
||||||
calibrated_axes: tuple[str, ...] = ()
|
|
||||||
session_id: int = 0
|
|
||||||
#: Monotonic per-session frame index; the join key for video, CSV and
|
|
||||||
#: calibration records.
|
|
||||||
frame_id: int = 0
|
|
||||||
#: Monotonic, session-relative acquisition time. Used for ordering so a wall
|
|
||||||
#: clock that steps backwards cannot reorder history. ``None`` means the
|
|
||||||
#: producer did not measure one, and the wall clock is used instead — a
|
|
||||||
#: default of ``0.0`` would have made every frame look simultaneous.
|
|
||||||
mono_t: Optional[float] = None
|
|
||||||
#: Wall clock at hand-off, for metadata only. ``None`` means unmeasured.
|
|
||||||
wall_t: Optional[float] = None
|
|
||||||
#: False when the force readout failed. Invalid packets carry NaN forces and
|
|
||||||
#: must never be averaged into calibration data or written as zeros.
|
|
||||||
valid: bool = True
|
|
||||||
invalid_reason: str = ""
|
|
||||||
#: True when the compute chain failed for this frame. The SDK's force fields
|
|
||||||
#: then still hold the value computed for an earlier frame, so they must not
|
|
||||||
#: be recorded as this frame's measurement.
|
|
||||||
stale_forces: bool = False
|
|
||||||
#: Content hash of the calibration actually in effect when this frame was
|
|
||||||
#: acquired; changes mid-session when new coefficients are applied.
|
|
||||||
calibration_version: str = ""
|
|
||||||
#: False when the sensor exposes no RAW channels; the raw fields are then
|
|
||||||
#: NaN rather than a copy of the calibrated values.
|
|
||||||
raw_available: bool = True
|
|
||||||
flow: Optional[np.ndarray] = None
|
|
||||||
depth: Optional[np.ndarray] = None
|
|
||||||
valid_mask: Optional[np.ndarray] = None
|
|
||||||
|
|
||||||
|
|
||||||
def orisys_importable() -> tuple[bool, str]:
|
|
||||||
try:
|
|
||||||
import orisys # noqa: F401
|
|
||||||
|
|
||||||
return True, getattr(__import__("orisys"), "__version__", "unknown")
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
return False, str(exc)
|
|
||||||
|
|
||||||
|
|
||||||
def load_fixed_mode(config_ref: str | Path) -> FixedCameraMode:
|
|
||||||
ref = str(config_ref).strip()
|
|
||||||
path = Path(ref)
|
|
||||||
if path.is_file():
|
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
resolved = str(path.resolve())
|
|
||||||
else:
|
|
||||||
import orisys
|
|
||||||
|
|
||||||
data = orisys.configs.load_config(ref)
|
|
||||||
resolved = ref
|
|
||||||
camera = data.get("camera") or {}
|
|
||||||
raw = camera.get("resolution_raw") or [1920, 1080]
|
|
||||||
fps = float(camera.get("fps") or 30.0)
|
|
||||||
return FixedCameraMode(
|
|
||||||
width=int(raw[0]),
|
|
||||||
height=int(raw[1]),
|
|
||||||
fps=fps,
|
|
||||||
config_ref=resolved,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _ensure_bgr(img: np.ndarray) -> np.ndarray:
|
|
||||||
if img.ndim == 2:
|
|
||||||
return cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
|
|
||||||
if img.ndim == 3 and img.shape[2] == 1:
|
|
||||||
return cv2.cvtColor(img[:, :, 0], cv2.COLOR_GRAY2BGR)
|
|
||||||
if img.ndim == 3 and img.shape[2] == 4:
|
|
||||||
return cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)
|
|
||||||
return img
|
|
||||||
|
|
||||||
|
|
||||||
def _sensor_force_calibrated(sensor, *, config_ref: str | Path | None = None) -> bool:
|
def _sensor_force_calibrated(sensor, *, config_ref: str | Path | None = None) -> bool:
|
||||||
@@ -147,6 +72,43 @@ def _sensor_force_calibrated(sensor, *, config_ref: str | Path | None = None) ->
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def process_chain(sensor) -> tuple[Optional[np.ndarray], str]:
|
||||||
|
"""Run the live processing chain for one already-read frame.
|
||||||
|
|
||||||
|
``get_img`` alone is not the pipeline: the SDK's depth map is produced by
|
||||||
|
``compute_deformation`` + ``compute_contact``. The offline depth replay calls
|
||||||
|
this same function, so it cannot drift from what was recorded live.
|
||||||
|
Returns ``(depth, error)``; the depth is ``None`` when the chain failed or the
|
||||||
|
sensor exposes none.
|
||||||
|
"""
|
||||||
|
compute_error = ""
|
||||||
|
compute_deformation = getattr(sensor, "compute_deformation", None)
|
||||||
|
compute_contact = getattr(sensor, "compute_contact", None)
|
||||||
|
if callable(compute_deformation) and callable(compute_contact):
|
||||||
|
# Same calls and arguments as the acquisition loop.
|
||||||
|
try:
|
||||||
|
compute_deformation(decompose=True, check_motion=False)
|
||||||
|
compute_contact()
|
||||||
|
except Exception as exc: # noqa: BLE001 - reported to the caller
|
||||||
|
compute_error = f"compute: {type(exc).__name__}: {exc}"
|
||||||
|
else:
|
||||||
|
compute_error = f"no processing chain on {type(sensor).__name__}"
|
||||||
|
return read_depth(sensor), compute_error
|
||||||
|
|
||||||
|
|
||||||
|
def read_depth(sensor) -> Optional[np.ndarray]:
|
||||||
|
"""The current depth channel of ``sensor``, or ``None`` when unavailable."""
|
||||||
|
try:
|
||||||
|
info = sensor.info
|
||||||
|
if not hasattr(info, "DEPTH"):
|
||||||
|
return None
|
||||||
|
value = sensor.read_info(info.DEPTH)
|
||||||
|
except Exception: # noqa: BLE001 - reported as "no depth for this frame"
|
||||||
|
logger.exception("Could not read the depth channel")
|
||||||
|
return None
|
||||||
|
return None if value is None else np.asarray(value)
|
||||||
|
|
||||||
|
|
||||||
def read_forces_from_sensor(sensor, *, config_ref: str | Path | None = None):
|
def read_forces_from_sensor(sensor, *, config_ref: str | Path | None = None):
|
||||||
"""Read one force sample.
|
"""Read one force sample.
|
||||||
|
|
||||||
@@ -284,10 +246,6 @@ def _calibrated_axes(sensor) -> tuple[str, ...]:
|
|||||||
return tuple(axes)
|
return tuple(axes)
|
||||||
|
|
||||||
|
|
||||||
def _is_file_source(video_source: str | int) -> bool:
|
|
||||||
return isinstance(video_source, str)
|
|
||||||
|
|
||||||
|
|
||||||
def calibration_digest(path: str | Path) -> str:
|
def calibration_digest(path: str | Path) -> str:
|
||||||
"""Content hash of a calibration document — the calibration's identity.
|
"""Content hash of a calibration document — the calibration's identity.
|
||||||
|
|
||||||
@@ -352,14 +310,20 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
*,
|
*,
|
||||||
on_error: Optional[Callable[[str], None]] = None,
|
on_error: Optional[Callable[[str], None]] = None,
|
||||||
on_status: Optional[Callable[[str], None]] = None,
|
on_status: Optional[Callable[[str], None]] = None,
|
||||||
on_calibration_applied: Optional[Callable[[str, str, int], None]] = None,
|
on_calibration_applied: Optional[Callable[[str, str, int, float, int], None]] = None,
|
||||||
|
on_reset_applied: Optional[Callable[[int, float, int], None]] = None,
|
||||||
):
|
):
|
||||||
self._init_sinks()
|
self._init_sinks()
|
||||||
self._on_error = on_error
|
self._on_error = on_error
|
||||||
self._on_status = on_status
|
self._on_status = on_status
|
||||||
#: Called from the acquisition thread once new coefficients are actually
|
#: Called from the acquisition thread once new coefficients are actually
|
||||||
#: loaded, with ``(path, version, frame_id)``. Confirmation, not intent.
|
#: loaded, with ``(path, version, frame_id, mono_t, session_id)``.
|
||||||
|
#: Confirmation, not intent — and it carries the frame and time so the
|
||||||
|
#: recorded event is dated when it happened.
|
||||||
self._on_calibration_applied = on_calibration_applied
|
self._on_calibration_applied = on_calibration_applied
|
||||||
|
#: Called from the acquisition thread once a queued zeroing took effect,
|
||||||
|
#: with ``(frame_id, mono_t, session_id)``.
|
||||||
|
self._on_reset_applied = on_reset_applied
|
||||||
self._thread: Optional[threading.Thread] = None
|
self._thread: Optional[threading.Thread] = None
|
||||||
self._stop = threading.Event()
|
self._stop = threading.Event()
|
||||||
self._lock = threading.Lock()
|
self._lock = threading.Lock()
|
||||||
@@ -373,6 +337,8 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._end_reason: Optional[str] = None
|
self._end_reason: Optional[str] = None
|
||||||
self._phase: str = "idle"
|
self._phase: str = "idle"
|
||||||
self._pending_calibration: Optional[str] = None
|
self._pending_calibration: Optional[str] = None
|
||||||
|
#: A zeroing requested from another thread, applied by this one.
|
||||||
|
self._pending_reset: bool = False
|
||||||
#: Monotonic origin for ``SdkPacket.mono_t``, reset per session.
|
#: Monotonic origin for ``SdkPacket.mono_t``, reset per session.
|
||||||
self._mono_t0: float = 0.0
|
self._mono_t0: float = 0.0
|
||||||
#: Content hash of the calibration currently in effect.
|
#: Content hash of the calibration currently in effect.
|
||||||
@@ -418,6 +384,11 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._latest = None
|
self._latest = None
|
||||||
self._end_reason = None
|
self._end_reason = None
|
||||||
self._phase = "connecting"
|
self._phase = "connecting"
|
||||||
|
# The previous session's coefficients say nothing about this one: an
|
||||||
|
# uncalibrated device must not inherit a version it never loaded, and
|
||||||
|
# an old reset request must not fire into the new session's frames.
|
||||||
|
self._calibration_version = ""
|
||||||
|
self._pending_reset = False
|
||||||
self._session_id += 1
|
self._session_id += 1
|
||||||
session = self._session_id
|
session = self._session_id
|
||||||
self._stop.clear()
|
self._stop.clear()
|
||||||
@@ -463,28 +434,57 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
self._phase = "idle"
|
self._phase = "idle"
|
||||||
|
|
||||||
def reset(self) -> None:
|
def reset(self) -> None:
|
||||||
|
"""Request a tracker reset, applied by the acquisition thread.
|
||||||
|
|
||||||
|
``Sensor.reset()`` from the GUI thread would race the frame the loop is
|
||||||
|
processing, so the actual zeroing happens at a frame boundary on the
|
||||||
|
acquisition thread — which then reports the frame and the time it took
|
||||||
|
effect. The confirmation callback is ``on_reset_applied``.
|
||||||
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
sensor = self._sensors.get(self._session_id)
|
if self._session_id <= 0:
|
||||||
if sensor is None:
|
|
||||||
raise RuntimeError("Sensor is not running")
|
raise RuntimeError("Sensor is not running")
|
||||||
sensor.reset()
|
self._pending_reset = True
|
||||||
|
|
||||||
|
def _consume_pending_reset(self, sensor, frame_id: int, mono_t: float) -> None:
|
||||||
|
"""Apply a queued reset and report the frame it actually took effect at."""
|
||||||
|
with self._lock:
|
||||||
|
pending = self._pending_reset
|
||||||
|
self._pending_reset = False
|
||||||
|
session = self._session_id
|
||||||
|
if not pending or sensor is None:
|
||||||
|
return
|
||||||
|
reset = getattr(sensor, "reset", None)
|
||||||
|
if not callable(reset):
|
||||||
|
raise RuntimeError("sensor does not support reset")
|
||||||
|
reset()
|
||||||
|
if self._on_reset_applied is not None:
|
||||||
|
self._on_reset_applied(int(frame_id), float(mono_t), int(session))
|
||||||
|
|
||||||
def set_pending_calibration(self, path: str | Path | None) -> None:
|
def set_pending_calibration(self, path: str | Path | None) -> None:
|
||||||
"""Queue a force_calibration.json path for the acquisition thread."""
|
"""Queue a force_calibration.json path for the acquisition thread."""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._pending_calibration = str(path) if path is not None else None
|
self._pending_calibration = str(path) if path is not None else None
|
||||||
|
|
||||||
def _consume_pending_calibration(self, sensor, frame_id: int = 0) -> None:
|
def _consume_pending_calibration(
|
||||||
|
self, sensor, frame_id: int = 0, mono_t: float = 0.0
|
||||||
|
) -> None:
|
||||||
"""Apply queued coefficients and report **confirmation**, not intent.
|
"""Apply queued coefficients and report **confirmation**, not intent.
|
||||||
|
|
||||||
``Sensor.set_calibration`` loads with ``strict=True`` and raises on an
|
``Sensor.set_calibration`` loads with ``strict=True`` and raises on an
|
||||||
invalid document, so success is observable. The old code swallowed that
|
invalid document, so success is observable. The old code swallowed that
|
||||||
exception, which is why the app logged "calibration applied" before
|
exception, which is why the app logged "calibration applied" before
|
||||||
anything had actually been loaded.
|
anything had actually been loaded.
|
||||||
|
|
||||||
|
A rejection leaves the version alone on purpose: the sensor still has the
|
||||||
|
coefficients it had, so the previous version is still the true one. What
|
||||||
|
never happens is inventing a version for a device that has no calibration
|
||||||
|
— that is the empty string, set when the session opened.
|
||||||
"""
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
path = self._pending_calibration
|
path = self._pending_calibration
|
||||||
self._pending_calibration = None
|
self._pending_calibration = None
|
||||||
|
session = self._session_id
|
||||||
if path is None or sensor is None:
|
if path is None or sensor is None:
|
||||||
return
|
return
|
||||||
if not hasattr(sensor, "set_calibration"):
|
if not hasattr(sensor, "set_calibration"):
|
||||||
@@ -493,7 +493,9 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
version = calibration_digest(path)
|
version = calibration_digest(path)
|
||||||
self._calibration_version = version
|
self._calibration_version = version
|
||||||
if self._on_calibration_applied is not None:
|
if self._on_calibration_applied is not None:
|
||||||
self._on_calibration_applied(str(path), version, int(frame_id))
|
self._on_calibration_applied(
|
||||||
|
str(path), version, int(frame_id), float(mono_t), int(session)
|
||||||
|
)
|
||||||
|
|
||||||
def _set_end_reason(self, reason: str) -> None:
|
def _set_end_reason(self, reason: str) -> None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -574,11 +576,12 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
# here. Without this the record would name the path passed on the
|
# here. Without this the record would name the path passed on the
|
||||||
# command line instead of the calibration that was applied.
|
# command line instead of the calibration that was applied.
|
||||||
loaded = loaded_calibration_version(sensor)
|
loaded = loaded_calibration_version(sensor)
|
||||||
if loaded:
|
# Assigned even when empty: an uncalibrated device must report *no*
|
||||||
|
# calibration, never the previous session's version.
|
||||||
self._calibration_version = loaded
|
self._calibration_version = loaded
|
||||||
if self._on_calibration_applied is not None:
|
if loaded and self._on_calibration_applied is not None:
|
||||||
self._on_calibration_applied(
|
self._on_calibration_applied(
|
||||||
str(loaded_path(sensor) or ""), loaded, 0
|
str(loaded_path(sensor) or ""), loaded, 0, 0.0, int(session)
|
||||||
)
|
)
|
||||||
self._status(f"Streaming {mode.label()}")
|
self._status(f"Streaming {mode.label()}")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
@@ -627,23 +630,32 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
|
|
||||||
none_streak_start = None
|
none_streak_start = None
|
||||||
compute_error = ""
|
compute_error = ""
|
||||||
|
# The frame's acquisition time, so an event that happens on this
|
||||||
|
# frame is dated when it happened rather than when it was queued.
|
||||||
|
frame_mono = max(0.0, time.perf_counter() - self._mono_t0)
|
||||||
try:
|
try:
|
||||||
# Applying new coefficients is reported separately from the
|
# Applying new coefficients is reported separately from the
|
||||||
# compute chain: a rejected document is an operator error,
|
# compute chain: a rejected document is an operator error,
|
||||||
# not a numerical failure.
|
# not a numerical failure.
|
||||||
self._consume_pending_calibration(sensor, frame_id)
|
self._consume_pending_calibration(sensor, frame_id, frame_mono)
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
message = f"calibration rejected: {type(exc).__name__}: {exc}"
|
message = f"calibration rejected: {type(exc).__name__}: {exc}"
|
||||||
if self._on_error is not None and session == self._session_id:
|
if self._on_error is not None and session == self._session_id:
|
||||||
self._on_error(message)
|
self._on_error(message)
|
||||||
compute_error = message
|
compute_error = message
|
||||||
try:
|
try:
|
||||||
sensor.compute_deformation(decompose=True, check_motion=False)
|
# The zeroing is applied *here*, between two frames, and only
|
||||||
sensor.compute_contact()
|
# then reported — the same confirmation rule as the
|
||||||
except Exception as exc: # noqa: BLE001 - reported on the packet
|
# calibration.
|
||||||
compute_error = compute_error or (
|
self._consume_pending_reset(sensor, frame_id, frame_mono)
|
||||||
f"compute: {type(exc).__name__}: {exc}"
|
except Exception as exc: # noqa: BLE001
|
||||||
)
|
message = f"reset failed: {type(exc).__name__}: {exc}"
|
||||||
|
if self._on_error is not None and session == self._session_id:
|
||||||
|
self._on_error(message)
|
||||||
|
# The same chain the offline depth replay runs. The depth it
|
||||||
|
# reads is the packet's depth, so the channel is read once.
|
||||||
|
chain_depth, chain_error = process_chain(sensor)
|
||||||
|
compute_error = compute_error or chain_error
|
||||||
t1 = time.perf_counter()
|
t1 = time.perf_counter()
|
||||||
frame_times.append(t1)
|
frame_times.append(t1)
|
||||||
cutoff = t1 - 1.0
|
cutoff = t1 - 1.0
|
||||||
@@ -683,7 +695,7 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
stale_forces=bool(compute_error),
|
stale_forces=bool(compute_error),
|
||||||
raw_available=bool(forces.get("raw_available", True)),
|
raw_available=bool(forces.get("raw_available", True)),
|
||||||
flow=forces["flow"],
|
flow=forces["flow"],
|
||||||
depth=forces.get("depth"),
|
depth=forces.get("depth") if forces.get("depth") is not None else chain_depth,
|
||||||
)
|
)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._latest = packet
|
self._latest = packet
|
||||||
|
|||||||
@@ -8,10 +8,10 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import math
|
import math
|
||||||
from copy import deepcopy
|
from copy import deepcopy
|
||||||
from dataclasses import asdict, dataclass, field
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Literal, Mapping
|
from typing import Any, Iterable, Literal, Mapping
|
||||||
|
|
||||||
from orisys.calibration import (
|
from orisys.calibration import (
|
||||||
CalSample as SdkCalSample,
|
CalSample as SdkCalSample,
|
||||||
@@ -95,8 +95,6 @@ def predict_channel(cal, channel: str, raw: float) -> float:
|
|||||||
triple = [0.0, 0.0, 0.0]
|
triple = [0.0, 0.0, 0.0]
|
||||||
triple[_CONVERT_SLOT[channel]] = float(raw)
|
triple[_CONVERT_SLOT[channel]] = float(raw)
|
||||||
return float(cal.convert(triple[0], triple[1], triple[2])[_CONVERT_SLOT[channel]])
|
return float(cal.convert(triple[0], triple[1], triple[2])[_CONVERT_SLOT[channel]])
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class CalSessionStore:
|
class CalSessionStore:
|
||||||
"""In-memory calibration points for FN / FX / FY."""
|
"""In-memory calibration points for FN / FX / FY."""
|
||||||
@@ -176,12 +174,8 @@ def meta_from_sensor_json(path: str | Path | None) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def build_force_calibration_v2(
|
def _meta_blocks(meta: dict[str, Any] | None) -> tuple[dict, dict, dict, str]:
|
||||||
store: CalSessionStore,
|
"""Split a flat export-meta dict into the SDK's identity/pipeline/extra."""
|
||||||
*,
|
|
||||||
meta: dict[str, Any] | None = None,
|
|
||||||
) -> tuple[dict[str, Any], list[str]]:
|
|
||||||
"""Build schema v2 JSON. Returns (document, skipped_channel_warnings)."""
|
|
||||||
meta = dict(meta or {})
|
meta = dict(meta or {})
|
||||||
identity = {
|
identity = {
|
||||||
"sensor_type": meta.get("sensor_type", ""),
|
"sensor_type": meta.get("sensor_type", ""),
|
||||||
@@ -194,9 +188,9 @@ def build_force_calibration_v2(
|
|||||||
"config_ref": meta.get("config_ref"),
|
"config_ref": meta.get("config_ref"),
|
||||||
}
|
}
|
||||||
extra = {
|
extra = {
|
||||||
k: v
|
key: value
|
||||||
for k, v in meta.items()
|
for key, value in meta.items()
|
||||||
if k
|
if key
|
||||||
not in (
|
not in (
|
||||||
"sensor_type",
|
"sensor_type",
|
||||||
"sn",
|
"sn",
|
||||||
@@ -207,30 +201,255 @@ def build_force_calibration_v2(
|
|||||||
"samples",
|
"samples",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
return identity, pipeline, extra, str(meta.get("origin") or "user")
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_samples(rows: Iterable[CalSample]) -> list[CalSample]:
|
||||||
|
"""n-weighted merge of repeated reference loads, sorted by load.
|
||||||
|
|
||||||
|
Sampling the same load twice is normal and deliberate; the model has to see
|
||||||
|
the combined mean and the scatter *between* the samples, not whichever
|
||||||
|
sample happened to be last.
|
||||||
|
"""
|
||||||
|
merged: list[CalSample] = []
|
||||||
|
for sample in rows:
|
||||||
|
for index, existing in enumerate(merged):
|
||||||
|
if abs(existing.weight_n - sample.weight_n) <= WEIGHT_EPS:
|
||||||
|
merged[index] = merge_samples(existing, sample)
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
merged.append(sample)
|
||||||
|
merged.sort(key=lambda item: item.weight_n)
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
def training_samples(points) -> tuple[dict[str, list[CalSample]], list, list]:
|
||||||
|
"""Split points into the training build and the held-out validation set.
|
||||||
|
|
||||||
|
This is the **single** filter: a point that is excluded, invalid or has no
|
||||||
|
reference load never reaches a model, and a validation point is only ever
|
||||||
|
measured against one. Callers must not re-implement it.
|
||||||
|
"""
|
||||||
|
training: list = []
|
||||||
|
validation: list = []
|
||||||
|
for point in points or ():
|
||||||
|
if bool(getattr(point, "excluded", False)):
|
||||||
|
continue
|
||||||
try:
|
try:
|
||||||
cal = sdk_build(
|
load = float(point.reference_load_n)
|
||||||
store.to_sdk_samples(),
|
raw_mean = float(point.loaded_stats().mean)
|
||||||
|
except (AttributeError, TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
if not math.isfinite(load) or not math.isfinite(raw_mean):
|
||||||
|
continue
|
||||||
|
if bool(getattr(point, "is_validation", False)):
|
||||||
|
validation.append(point)
|
||||||
|
else:
|
||||||
|
training.append(point)
|
||||||
|
|
||||||
|
by_channel: dict[str, list[CalSample]] = {key: [] for key in CHANNEL_KEYS}
|
||||||
|
for point in training:
|
||||||
|
stats = point.loaded_stats()
|
||||||
|
std = float(stats.std)
|
||||||
|
by_channel[str(point.channel)].append(
|
||||||
|
CalSample(
|
||||||
|
weight_n=float(point.reference_load_n),
|
||||||
|
raw_mean=float(stats.mean),
|
||||||
|
n=int(stats.n or 0),
|
||||||
|
std=std if math.isfinite(std) else 0.0,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
{key: aggregate_samples(rows) for key, rows in by_channel.items()},
|
||||||
|
training,
|
||||||
|
validation,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PreparedCalibration:
|
||||||
|
"""One preparation of a calibration session: samples -> model -> diagnostics.
|
||||||
|
|
||||||
|
The diagnostics and the exported document come from *this* object, so the
|
||||||
|
numbers on screen describe exactly the model that is written to disk. Two
|
||||||
|
separate preparations (one for the panel, one for the fit) is how the
|
||||||
|
displayed residual and the applied curve drifted apart in the first place.
|
||||||
|
"""
|
||||||
|
|
||||||
|
model: Any = None
|
||||||
|
samples: dict[str, list[CalSample]] = field(default_factory=dict)
|
||||||
|
training: list = field(default_factory=list)
|
||||||
|
validation: list = field(default_factory=list)
|
||||||
|
#: ``id(point)`` -> prediction error at that point (predicted - reference).
|
||||||
|
residuals: dict[int, float] = field(default_factory=dict)
|
||||||
|
#: ``id(point)`` -> absolute error of a held-out point.
|
||||||
|
validation_errors: dict[int, float] = field(default_factory=dict)
|
||||||
|
warnings: list[str] = field(default_factory=list)
|
||||||
|
meta: dict = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ok(self) -> bool:
|
||||||
|
return self.model is not None
|
||||||
|
|
||||||
|
def channel_samples(self, key: str) -> list[CalSample]:
|
||||||
|
return list(self.samples.get(key) or [])
|
||||||
|
|
||||||
|
def method(self, channel: str):
|
||||||
|
curve = self.curve(channel)
|
||||||
|
return str(getattr(curve, "method", "") or "") if curve is not None else None
|
||||||
|
|
||||||
|
def breakpoints(self, channel: str) -> int:
|
||||||
|
curve = self.curve(channel)
|
||||||
|
return len(getattr(curve, "points", ()) or ()) if curve is not None else 0
|
||||||
|
|
||||||
|
def curve(self, channel: str):
|
||||||
|
if self.model is None:
|
||||||
|
return None
|
||||||
|
return self.model.get(channel)
|
||||||
|
|
||||||
|
def predict(self, channel: str, raw: float) -> float:
|
||||||
|
if self.model is None:
|
||||||
|
return float("nan")
|
||||||
|
return predict_channel(self.model, channel, raw)
|
||||||
|
|
||||||
|
def points_in_fit(self, channel: str) -> int:
|
||||||
|
return len(self.samples.get(channel) or [])
|
||||||
|
|
||||||
|
def validation_points(self, channel: str) -> int:
|
||||||
|
return sum(1 for point in self.validation if point.channel == channel)
|
||||||
|
|
||||||
|
def max_residual(self, channel: str) -> float:
|
||||||
|
values = [
|
||||||
|
abs(value)
|
||||||
|
for point in self.training
|
||||||
|
if point.channel == channel
|
||||||
|
for value in [self.residuals.get(id(point))]
|
||||||
|
if value is not None and math.isfinite(value)
|
||||||
|
]
|
||||||
|
return max(values) if values else float("nan")
|
||||||
|
|
||||||
|
def max_validation_error(self, channel: str) -> float:
|
||||||
|
values = [
|
||||||
|
value
|
||||||
|
for point in self.validation
|
||||||
|
if point.channel == channel
|
||||||
|
for value in [self.validation_errors.get(id(point))]
|
||||||
|
if value is not None and math.isfinite(value)
|
||||||
|
]
|
||||||
|
return max(values) if values else float("nan")
|
||||||
|
|
||||||
|
def to_document(self) -> dict[str, Any]:
|
||||||
|
"""The schema-v2 document for this prepared model."""
|
||||||
|
if self.model is None:
|
||||||
|
raise ValueError(
|
||||||
|
"No complete channels to build (each needs ≥2 points). "
|
||||||
|
+ "; ".join(self.warnings)
|
||||||
|
)
|
||||||
|
return self.model.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_from_samples(
|
||||||
|
samples: dict[str, list[CalSample]],
|
||||||
|
*,
|
||||||
|
meta: dict[str, Any] | None = None,
|
||||||
|
) -> PreparedCalibration:
|
||||||
|
"""Build the model and its diagnostics from prepared training samples.
|
||||||
|
|
||||||
|
Repeated loads are merged here too, so a caller that hands in raw (unmerged)
|
||||||
|
samples still gets the same model as one that hands in points.
|
||||||
|
"""
|
||||||
|
merged = {
|
||||||
|
key: aggregate_samples(samples.get(key) or []) for key in CHANNEL_KEYS
|
||||||
|
}
|
||||||
|
identity, pipeline, extra, origin = _meta_blocks(meta)
|
||||||
|
warnings: list[str] = []
|
||||||
|
model = None
|
||||||
|
try:
|
||||||
|
model = sdk_build(
|
||||||
|
{key: [sample.to_sdk() for sample in rows] for key, rows in merged.items()},
|
||||||
identity=identity,
|
identity=identity,
|
||||||
pipeline=pipeline,
|
pipeline=pipeline,
|
||||||
origin=str(meta.get("origin") or "user"),
|
origin=origin,
|
||||||
extra_meta=extra,
|
extra_meta=extra,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc: # noqa: BLE001 - surfaced as warnings + no model
|
||||||
# Match previous API: raise ValueError with channel warnings when empty.
|
warnings.append(str(exc))
|
||||||
raise ValueError(str(exc)) from exc
|
if model is not None:
|
||||||
|
try:
|
||||||
result = sdk_check(cal)
|
warnings.extend(sdk_check(model).warnings)
|
||||||
warnings = list(result.warnings)
|
except Exception as exc: # noqa: BLE001
|
||||||
|
warnings.append(f"check failed: {exc}")
|
||||||
for key in CHANNEL_KEYS:
|
for key in CHANNEL_KEYS:
|
||||||
if not cal.has(key):
|
if not model.has(key):
|
||||||
rows = store.channel(key)
|
warnings.append(_missing_channel_message(key, merged.get(key) or []))
|
||||||
|
return PreparedCalibration(
|
||||||
|
model=model,
|
||||||
|
samples=merged,
|
||||||
|
warnings=warnings,
|
||||||
|
meta=dict(meta or {}),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _missing_channel_message(key: str, rows: list[CalSample]) -> str:
|
||||||
if rows:
|
if rows:
|
||||||
msg = f"{key}: need at least 2 points (have {len(rows)})"
|
return f"{key}: need at least 2 points (have {len(rows)})"
|
||||||
else:
|
return f"{key}: no samples"
|
||||||
msg = f"{key}: no samples"
|
|
||||||
if msg not in warnings:
|
|
||||||
warnings.append(msg)
|
def prepare_calibration(
|
||||||
return cal.to_dict(), warnings
|
points,
|
||||||
|
*,
|
||||||
|
meta: dict[str, Any] | None = None,
|
||||||
|
) -> PreparedCalibration:
|
||||||
|
"""The one entry point: points -> training samples -> model -> diagnostics."""
|
||||||
|
samples, training, validation = training_samples(points)
|
||||||
|
prepared = prepare_from_samples(samples, meta=meta)
|
||||||
|
residuals: dict[int, float] = {}
|
||||||
|
for point in training:
|
||||||
|
predicted = prepared.predict(str(point.channel), float(point.loaded_stats().mean))
|
||||||
|
if math.isfinite(predicted):
|
||||||
|
residuals[id(point)] = predicted - float(point.reference_load_n)
|
||||||
|
validation_errors: dict[int, float] = {}
|
||||||
|
for point in validation:
|
||||||
|
predicted = prepared.predict(str(point.channel), float(point.loaded_stats().mean))
|
||||||
|
if math.isfinite(predicted):
|
||||||
|
validation_errors[id(point)] = abs(predicted - float(point.reference_load_n))
|
||||||
|
return replace(
|
||||||
|
prepared,
|
||||||
|
training=list(training),
|
||||||
|
validation=list(validation),
|
||||||
|
residuals=residuals,
|
||||||
|
validation_errors=validation_errors,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_export_model(samples: dict[str, list[CalSample]]):
|
||||||
|
"""The SDK model for ``samples`` — the same preparation the panel uses."""
|
||||||
|
return prepare_from_samples(samples).model
|
||||||
|
|
||||||
|
|
||||||
|
def build_force_calibration_v2(
|
||||||
|
source,
|
||||||
|
*,
|
||||||
|
meta: dict[str, Any] | None = None,
|
||||||
|
) -> tuple[dict[str, Any], list[str]]:
|
||||||
|
"""Serialise the prepared model. Returns (document, warnings).
|
||||||
|
|
||||||
|
``source`` is a :class:`PreparedCalibration` (the normal path — the document
|
||||||
|
is exactly the model that was diagnosed) or, for older callers, a
|
||||||
|
:class:`CalSessionStore`, which is prepared here with ``meta``.
|
||||||
|
"""
|
||||||
|
prepared = (
|
||||||
|
source
|
||||||
|
if isinstance(source, PreparedCalibration)
|
||||||
|
else prepare_from_samples(
|
||||||
|
{key: source.channel(key) for key in CHANNEL_KEYS}, meta=meta
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if prepared.model is None:
|
||||||
|
# Match the previous API: raise ValueError carrying the channel warnings.
|
||||||
|
raise ValueError("; ".join(prepared.warnings) or "no complete channels")
|
||||||
|
return prepared.to_document(), list(prepared.warnings)
|
||||||
|
|
||||||
|
|
||||||
def write_force_calibration(
|
def write_force_calibration(
|
||||||
@@ -245,15 +464,18 @@ def write_force_calibration(
|
|||||||
|
|
||||||
|
|
||||||
def export_force_calibration_v2(
|
def export_force_calibration_v2(
|
||||||
store: CalSessionStore,
|
source,
|
||||||
*,
|
*,
|
||||||
export_dir: str | Path,
|
export_dir: str | Path,
|
||||||
sensor_type: str,
|
sensor_type: str,
|
||||||
meta: dict[str, Any] | None = None,
|
meta: dict[str, Any] | None = None,
|
||||||
local_bundle_dir: str | Path | None = None,
|
local_bundle_dir: str | Path | None = None,
|
||||||
) -> tuple[Path, Path | None, list[str]]:
|
) -> tuple[Path, Path | None, list[str]]:
|
||||||
"""Write timestamped export and optional local bundle force_calibration.json."""
|
"""Write timestamped export and optional local bundle force_calibration.json.
|
||||||
doc, warnings = build_force_calibration_v2(store, meta=meta)
|
|
||||||
|
``source`` is the prepared calibration (preferred) or a session store.
|
||||||
|
"""
|
||||||
|
doc, warnings = build_force_calibration_v2(source, meta=meta)
|
||||||
if not doc.get("channels"):
|
if not doc.get("channels"):
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"No complete channels to export (each needs ≥2 points). "
|
"No complete channels to export (each needs ≥2 points). "
|
||||||
|
|||||||
@@ -23,123 +23,39 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import Callable, Literal, Optional
|
from typing import Callable, Optional
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from orisys_app_shared.force.session import NA, SessionSample, summarise
|
from orisys_app_shared.force.models import (
|
||||||
|
CHANNEL_KEYS,
|
||||||
|
MAX_TRACE_POINTS,
|
||||||
|
NA,
|
||||||
|
SAMPLE_FIELDS,
|
||||||
|
STATUS_CANCELLED,
|
||||||
|
STATUS_FEW_SAMPLES,
|
||||||
|
STATUS_OK,
|
||||||
|
AxisStats,
|
||||||
|
CalibrationPoint,
|
||||||
|
ChannelKey,
|
||||||
|
trace_dt,
|
||||||
|
trace_from,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.force.session import SessionSample, summarise
|
||||||
|
|
||||||
try: # pragma: no cover - the module is a plain data definition
|
# Re-exported: the models live in ``force.models`` so the session file format can
|
||||||
from orisys_app_shared.force.calibration_session import SAMPLE_FIELDS
|
# describe a point without importing the sampling state machine.
|
||||||
except Exception: # noqa: BLE001 - a partial install must not break sampling
|
__all__ = [
|
||||||
SAMPLE_FIELDS = (
|
"AxisStats",
|
||||||
"frame_id",
|
"CalibrationPoint",
|
||||||
"t_rel_s",
|
"CalibrationRecorder",
|
||||||
"fn_raw",
|
"FitResult",
|
||||||
"fx_raw",
|
"SAMPLE_FIELDS",
|
||||||
"fy_raw",
|
"trace_dt",
|
||||||
"fn_n",
|
"trace_from",
|
||||||
"fx_n",
|
]
|
||||||
"fy_n",
|
|
||||||
"valid",
|
|
||||||
)
|
|
||||||
|
|
||||||
ChannelKey = Literal["fn", "fx", "fy"]
|
|
||||||
CHANNEL_KEYS: tuple[ChannelKey, ...] = ("fn", "fx", "fy")
|
|
||||||
|
|
||||||
STATUS_OK = "ok"
|
|
||||||
STATUS_FEW_SAMPLES = "few_samples"
|
|
||||||
STATUS_CANCELLED = "cancelled"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class AxisStats:
|
|
||||||
"""Mean / standard deviation / valid count for one axis."""
|
|
||||||
|
|
||||||
mean: float = NA
|
|
||||||
std: float = NA
|
|
||||||
n: int = 0
|
|
||||||
|
|
||||||
def as_dict(self) -> dict:
|
|
||||||
return {"mean": self.mean, "std": self.std, "n": self.n}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class CalibrationPoint:
|
|
||||||
"""One recorded calibration point, with all three axes captured."""
|
|
||||||
|
|
||||||
channel: ChannelKey
|
|
||||||
reference_load_n: float
|
|
||||||
direction: str = "load"
|
|
||||||
round_index: int = 0
|
|
||||||
note: str = ""
|
|
||||||
started_at: str = ""
|
|
||||||
ended_at: str = ""
|
|
||||||
duration_s: float = 0.0
|
|
||||||
requested_duration_s: float = 0.0
|
|
||||||
n_total: int = 0
|
|
||||||
n_valid: int = 0
|
|
||||||
raw: dict[str, AxisStats] = field(default_factory=dict)
|
|
||||||
newtons: dict[str, AxisStats] = field(default_factory=dict)
|
|
||||||
status: str = STATUS_OK
|
|
||||||
#: Downsampled per-axis raw trace, so a row can be inspected after the fact.
|
|
||||||
trace: dict[str, list[float]] = field(default_factory=dict)
|
|
||||||
trace_dt_s: float = 0.0
|
|
||||||
#: **Audit** series: every sample of the point, in
|
|
||||||
#: :data:`~orisys_app_shared.force.calibration_session.SAMPLE_FIELDS` order.
|
|
||||||
#: The trace above is a preview and must never be the only surviving copy.
|
|
||||||
samples: list[tuple] = field(default_factory=list)
|
|
||||||
#: Diagnostics, all operator-controlled. Excluded points leave the fit;
|
|
||||||
#: validation points are held out of it and used to test it.
|
|
||||||
excluded: bool = False
|
|
||||||
is_validation: bool = False
|
|
||||||
#: Filled in by :meth:`CalibrationRecorder.fit`.
|
|
||||||
residual_n: float = NA
|
|
||||||
|
|
||||||
def loaded_stats(self) -> AxisStats:
|
|
||||||
return self.raw.get(self.channel, AxisStats())
|
|
||||||
|
|
||||||
def axis_mean(self, channel: str) -> float:
|
|
||||||
stats = self.raw.get(channel)
|
|
||||||
return stats.mean if stats is not None else NA
|
|
||||||
|
|
||||||
def state_key(self) -> str:
|
|
||||||
if self.excluded:
|
|
||||||
return "cal.state_excluded"
|
|
||||||
if self.is_validation:
|
|
||||||
return "cal.state_validation"
|
|
||||||
return "cal.state_ok"
|
|
||||||
|
|
||||||
def as_dict(self) -> dict:
|
|
||||||
return {
|
|
||||||
"channel": self.channel,
|
|
||||||
"reference_load_n": self.reference_load_n,
|
|
||||||
# Only the loaded axis has a known reference load; the other two are
|
|
||||||
# intentionally absent rather than zero.
|
|
||||||
"direction": self.direction,
|
|
||||||
"round_index": self.round_index,
|
|
||||||
"note": self.note,
|
|
||||||
"started_at": self.started_at,
|
|
||||||
"ended_at": self.ended_at,
|
|
||||||
"duration_s": self.duration_s,
|
|
||||||
"requested_duration_s": self.requested_duration_s,
|
|
||||||
"n_total": self.n_total,
|
|
||||||
"n_valid": self.n_valid,
|
|
||||||
"status": self.status,
|
|
||||||
"excluded": self.excluded,
|
|
||||||
"is_validation": self.is_validation,
|
|
||||||
"residual_n": self.residual_n,
|
|
||||||
"raw": {key: stats.as_dict() for key, stats in self.raw.items()},
|
|
||||||
"newtons": {key: stats.as_dict() for key, stats in self.newtons.items()},
|
|
||||||
# The audit series is part of the point's identity, not a UI extra:
|
|
||||||
# without it the only surviving copy of the samples is the
|
|
||||||
# downsampled trace.
|
|
||||||
"sample_fields": list(SAMPLE_FIELDS),
|
|
||||||
"n_samples": len(self.samples),
|
|
||||||
"samples": [list(row) for row in self.samples],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -177,10 +93,6 @@ class FitResult:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
#: Cap on stored trace samples per axis; peaks are preserved when reducing.
|
|
||||||
MAX_TRACE_POINTS = 400
|
|
||||||
|
|
||||||
|
|
||||||
class CalibrationRecorder:
|
class CalibrationRecorder:
|
||||||
"""Owns the calibration sampling lifecycle."""
|
"""Owns the calibration sampling lifecycle."""
|
||||||
|
|
||||||
@@ -358,8 +270,8 @@ class CalibrationRecorder:
|
|||||||
raw={"fn": axis_stats(2), "fx": axis_stats(3), "fy": axis_stats(4)},
|
raw={"fn": axis_stats(2), "fx": axis_stats(3), "fy": axis_stats(4)},
|
||||||
newtons={"fn": axis_stats(5), "fx": axis_stats(6), "fy": axis_stats(7)},
|
newtons={"fn": axis_stats(5), "fx": axis_stats(6), "fy": axis_stats(7)},
|
||||||
status=STATUS_OK if valid_block.shape[0] >= 2 else STATUS_FEW_SAMPLES,
|
status=STATUS_OK if valid_block.shape[0] >= 2 else STATUS_FEW_SAMPLES,
|
||||||
trace=_trace_from(valid_block),
|
trace=trace_from(valid_block),
|
||||||
trace_dt_s=_trace_dt(valid_block),
|
trace_dt_s=trace_dt(valid_block),
|
||||||
# Every sample is kept, including the invalid ones: a point that was
|
# Every sample is kept, including the invalid ones: a point that was
|
||||||
# mostly invalid is itself a finding. ``samples`` holds the original
|
# mostly invalid is itself a finding. ``samples`` holds the original
|
||||||
# tuples so ``frame_id`` stays an integer.
|
# tuples so ``frame_id`` stays an integer.
|
||||||
@@ -456,94 +368,71 @@ class CalibrationRecorder:
|
|||||||
with self._lock:
|
with self._lock:
|
||||||
return [point for point in self.points if point.channel == channel]
|
return [point for point in self.points if point.channel == channel]
|
||||||
|
|
||||||
def fit(self, channel: ChannelKey) -> FitResult:
|
def prepared(self, *, meta: Optional[dict] = None):
|
||||||
|
"""The session prepared into one model plus its diagnostics.
|
||||||
|
|
||||||
|
Everything that shows or writes a calibration goes through this, so the
|
||||||
|
panel, the fit summary and the exported file cannot disagree about which
|
||||||
|
points were used.
|
||||||
|
"""
|
||||||
|
from orisys_app_shared.force.calibration_io import prepare_calibration
|
||||||
|
|
||||||
|
with self._lock:
|
||||||
|
points = list(self.points)
|
||||||
|
return prepare_calibration(points, meta=meta)
|
||||||
|
|
||||||
|
def fit(self, channel: ChannelKey, prepared=None) -> FitResult:
|
||||||
"""Diagnose the model that export and application would actually use.
|
"""Diagnose the model that export and application would actually use.
|
||||||
|
|
||||||
The residual and validation figures used to come from a private
|
The residual and validation figures used to come from a private
|
||||||
least-squares line while the exported file carried a piecewise curve, so
|
least-squares line while the exported file carried a piecewise curve, so
|
||||||
the numbers on screen did not describe the calibration that would be
|
the numbers on screen did not describe the calibration that would be
|
||||||
applied. Both now come from the exported curve. Excluded points are
|
applied. Both now come from the one prepared model (see
|
||||||
dropped, validation points are held out of the build and only measured.
|
:func:`~orisys_app_shared.force.calibration_io.prepare_calibration`):
|
||||||
|
excluded points are dropped, validation points are held out of the build
|
||||||
|
and only measured, and repeated loads are merged before the build.
|
||||||
"""
|
"""
|
||||||
from orisys_app_shared.force.calibration_io import (
|
if prepared is None:
|
||||||
CalSample,
|
prepared = self.prepared()
|
||||||
build_export_model,
|
|
||||||
predict_channel,
|
|
||||||
)
|
|
||||||
|
|
||||||
points = self.points_for(channel)
|
points = self.points_for(channel)
|
||||||
usable = [
|
|
||||||
point
|
|
||||||
for point in points
|
|
||||||
if not point.excluded and not point.is_validation
|
|
||||||
and np.isfinite(point.reference_load_n)
|
|
||||||
and np.isfinite(point.loaded_stats().mean)
|
|
||||||
]
|
|
||||||
for point in points:
|
for point in points:
|
||||||
point.residual_n = NA
|
value = prepared.residuals.get(id(point))
|
||||||
if len(usable) < 2:
|
point.residual_n = NA if value is None else float(value)
|
||||||
return FitResult(channel=channel, n_points=len(usable))
|
|
||||||
|
|
||||||
samples: dict[str, list[CalSample]] = {key: [] for key in CHANNEL_KEYS}
|
curve = prepared.curve(channel)
|
||||||
for point in usable:
|
n_points = prepared.points_in_fit(channel)
|
||||||
samples[point.channel].append(
|
if curve is None or n_points < 2:
|
||||||
CalSample(
|
return FitResult(channel=channel, n_points=n_points)
|
||||||
weight_n=float(point.reference_load_n),
|
|
||||||
raw_mean=float(point.loaded_stats().mean),
|
raws = np.asarray(
|
||||||
n=int(point.loaded_stats().n),
|
[
|
||||||
std=float(point.loaded_stats().std)
|
float(point.loaded_stats().mean)
|
||||||
if np.isfinite(point.loaded_stats().std)
|
for point in prepared.training
|
||||||
else 0.0,
|
if point.channel == channel
|
||||||
|
],
|
||||||
|
dtype=float,
|
||||||
)
|
)
|
||||||
|
loads = np.asarray(
|
||||||
|
[
|
||||||
|
float(point.reference_load_n)
|
||||||
|
for point in prepared.training
|
||||||
|
if point.channel == channel
|
||||||
|
],
|
||||||
|
dtype=float,
|
||||||
)
|
)
|
||||||
try:
|
|
||||||
model = build_export_model(samples)
|
|
||||||
curve = model.get(channel)
|
|
||||||
except Exception:
|
|
||||||
# Without a buildable model there is nothing honest to report.
|
|
||||||
return FitResult(channel=channel, n_points=len(usable))
|
|
||||||
if curve is None:
|
|
||||||
return FitResult(channel=channel, n_points=len(usable))
|
|
||||||
|
|
||||||
residuals = []
|
|
||||||
for point in usable:
|
|
||||||
predicted = predict_channel(model, point.channel, point.loaded_stats().mean)
|
|
||||||
residual = predicted - float(point.reference_load_n)
|
|
||||||
point.residual_n = float(residual)
|
|
||||||
residuals.append(residual)
|
|
||||||
|
|
||||||
validation = [
|
|
||||||
point
|
|
||||||
for point in points
|
|
||||||
if not point.excluded and point.is_validation
|
|
||||||
and np.isfinite(point.reference_load_n)
|
|
||||||
and np.isfinite(point.loaded_stats().mean)
|
|
||||||
]
|
|
||||||
validation_error = NA
|
|
||||||
if validation:
|
|
||||||
errors = [
|
|
||||||
abs(
|
|
||||||
predict_channel(model, point.channel, point.loaded_stats().mean)
|
|
||||||
- float(point.reference_load_n)
|
|
||||||
)
|
|
||||||
for point in validation
|
|
||||||
]
|
|
||||||
validation_error = float(max(errors))
|
|
||||||
|
|
||||||
raws = np.asarray([point.loaded_stats().mean for point in usable], dtype=float)
|
|
||||||
loads = np.asarray([point.reference_load_n for point in usable], dtype=float)
|
|
||||||
linear_gain = linear_offset = NA
|
linear_gain = linear_offset = NA
|
||||||
if not np.allclose(raws, raws[0]):
|
if raws.size >= 2 and not np.allclose(raws, raws[0]):
|
||||||
linear_gain, linear_offset = (float(v) for v in np.polyfit(raws, loads, 1))
|
linear_gain, linear_offset = (float(v) for v in np.polyfit(raws, loads, 1))
|
||||||
|
|
||||||
return FitResult(
|
return FitResult(
|
||||||
channel=channel,
|
channel=channel,
|
||||||
n_points=len(usable),
|
n_points=n_points,
|
||||||
max_residual_n=float(np.max(np.abs(residuals))),
|
max_residual_n=prepared.max_residual(channel),
|
||||||
validation_error_n=validation_error,
|
validation_error_n=prepared.max_validation_error(channel),
|
||||||
validation_points=len(validation),
|
validation_points=prepared.validation_points(channel),
|
||||||
method=str(getattr(curve, "method", "") or ""),
|
method=str(getattr(curve, "method", "") or ""),
|
||||||
breakpoints=len(getattr(curve, "points", ()) or ()),
|
breakpoints=prepared.breakpoints(channel),
|
||||||
linear_gain=linear_gain,
|
linear_gain=linear_gain,
|
||||||
linear_offset=linear_offset,
|
linear_offset=linear_offset,
|
||||||
)
|
)
|
||||||
@@ -554,37 +443,3 @@ class CalibrationRecorder:
|
|||||||
self._rounds[channel] = index + 1
|
self._rounds[channel] = index + 1
|
||||||
return index
|
return index
|
||||||
|
|
||||||
|
|
||||||
def _trace_from(valid_block: np.ndarray) -> dict[str, list[float]]:
|
|
||||||
"""Downsampled raw trace per axis, bounded by ``MAX_TRACE_POINTS``."""
|
|
||||||
if valid_block.size == 0:
|
|
||||||
return {}
|
|
||||||
traces: dict[str, list[float]] = {}
|
|
||||||
for key, column in (("fn", 2), ("fx", 3), ("fy", 4)):
|
|
||||||
values = valid_block[:, column]
|
|
||||||
values = values[np.isfinite(values)]
|
|
||||||
if values.size == 0:
|
|
||||||
continue
|
|
||||||
if values.size > MAX_TRACE_POINTS:
|
|
||||||
# Keep the shape of the trace, including its extremes.
|
|
||||||
edges = np.linspace(0, values.size, MAX_TRACE_POINTS + 1, dtype=int)
|
|
||||||
reduced = [
|
|
||||||
values[start:max(start + 1, stop)]
|
|
||||||
for start, stop in zip(edges[:-1], edges[1:])
|
|
||||||
]
|
|
||||||
values = np.asarray(
|
|
||||||
[
|
|
||||||
bucket[int(np.argmax(np.abs(bucket)))]
|
|
||||||
for bucket in reduced
|
|
||||||
if bucket.size
|
|
||||||
]
|
|
||||||
)
|
|
||||||
traces[key] = [float(value) for value in values]
|
|
||||||
return traces
|
|
||||||
|
|
||||||
|
|
||||||
def _trace_dt(valid_block: np.ndarray) -> float:
|
|
||||||
if valid_block.shape[0] < 2:
|
|
||||||
return 0.0
|
|
||||||
span = float(valid_block[-1, 1] - valid_block[0, 1])
|
|
||||||
return span / max(1, valid_block.shape[0] - 1)
|
|
||||||
|
|||||||
@@ -23,23 +23,19 @@ from datetime import datetime, timezone
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Iterable, Optional
|
from typing import Any, Iterable, Optional
|
||||||
|
|
||||||
|
from orisys_app_shared.force.models import (
|
||||||
|
SAMPLE_FIELDS,
|
||||||
|
AxisStats,
|
||||||
|
CalibrationPoint,
|
||||||
|
trace_dt,
|
||||||
|
trace_from,
|
||||||
|
)
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
#: Format version of the calibration session document.
|
#: Format version of the calibration session document.
|
||||||
SESSION_FORMAT_VERSION = "1.0"
|
SESSION_FORMAT_VERSION = "1.0"
|
||||||
SESSION_REQUIRED = ("format", "format_version", "created_at", "points")
|
SESSION_REQUIRED = ("format", "format_version", "created_at", "points")
|
||||||
#: Columns of each sample row inside a point, in order.
|
|
||||||
SAMPLE_FIELDS = (
|
|
||||||
"frame_id",
|
|
||||||
"t_rel_s",
|
|
||||||
"fn_raw",
|
|
||||||
"fx_raw",
|
|
||||||
"fy_raw",
|
|
||||||
"fn_n",
|
|
||||||
"fx_n",
|
|
||||||
"fy_n",
|
|
||||||
"valid",
|
|
||||||
)
|
|
||||||
POINT_REQUIRED = ("channel", "reference_load_n")
|
POINT_REQUIRED = ("channel", "reference_load_n")
|
||||||
|
|
||||||
|
|
||||||
@@ -59,6 +55,29 @@ class CalibrationSessionRecord:
|
|||||||
def ok(self) -> bool:
|
def ok(self) -> bool:
|
||||||
return bool(self.points) and not self.warnings
|
return bool(self.points) and not self.warnings
|
||||||
|
|
||||||
|
def identity(self) -> dict:
|
||||||
|
"""The device this session belongs to.
|
||||||
|
|
||||||
|
A calibration record is evidence about *one* device: its sensor type, its
|
||||||
|
serial number and the configuration it was taken with. Reopening it must
|
||||||
|
not restamp it with whatever device happens to be connected now.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"sensor_type": str(self.meta.get("sensor_type") or ""),
|
||||||
|
"sn": str(self.meta.get("sn") or ""),
|
||||||
|
"config_ref": str(self.meta.get("config_ref") or ""),
|
||||||
|
"created_at": self.created_at,
|
||||||
|
}
|
||||||
|
|
||||||
|
def describe(self) -> str:
|
||||||
|
identity = self.identity()
|
||||||
|
parts = [identity["sensor_type"] or "?"]
|
||||||
|
if identity["sn"]:
|
||||||
|
parts.append(f"SN {identity['sn']}")
|
||||||
|
if identity["config_ref"]:
|
||||||
|
parts.append(identity["config_ref"])
|
||||||
|
return " / ".join(parts)
|
||||||
|
|
||||||
|
|
||||||
def _jsonable(value: Any) -> Any:
|
def _jsonable(value: Any) -> Any:
|
||||||
"""NaN/Inf are not valid JSON; they are written as ``null``."""
|
"""NaN/Inf are not valid JSON; they are written as ``null``."""
|
||||||
@@ -147,11 +166,6 @@ def read_calibration_session(path: str | Path) -> CalibrationSessionRecord:
|
|||||||
if version != SESSION_FORMAT_VERSION:
|
if version != SESSION_FORMAT_VERSION:
|
||||||
warnings.append(f"format_version {version or '?'} is not {SESSION_FORMAT_VERSION}")
|
warnings.append(f"format_version {version or '?'} is not {SESSION_FORMAT_VERSION}")
|
||||||
|
|
||||||
from orisys_app_shared.force.calibration_recorder import (
|
|
||||||
CalibrationPoint,
|
|
||||||
AxisStats,
|
|
||||||
)
|
|
||||||
|
|
||||||
points = []
|
points = []
|
||||||
for index, raw in enumerate(data.get("points") or []):
|
for index, raw in enumerate(data.get("points") or []):
|
||||||
if not isinstance(raw, dict):
|
if not isinstance(raw, dict):
|
||||||
@@ -228,9 +242,39 @@ def _point_from_dict(raw: dict, CalibrationPoint, AxisStats):
|
|||||||
values = [float("nan") if value is None else value for value in row]
|
values = [float("nan") if value is None else value for value in row]
|
||||||
rows.append(tuple(values))
|
rows.append(tuple(values))
|
||||||
point.samples = rows
|
point.samples = rows
|
||||||
|
# The full series is the audit data, so everything a diagnostic view needs is
|
||||||
|
# rebuilt from it: the per-axis trace, its sample interval and, when the
|
||||||
|
# stored duration is missing, the real sampling span.
|
||||||
|
restore_diagnostics(point)
|
||||||
return point
|
return point
|
||||||
|
|
||||||
|
|
||||||
|
def restore_diagnostics(point) -> None:
|
||||||
|
"""Rebuild the three-axis trace and the sampling span from the samples."""
|
||||||
|
if not point.samples:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
import numpy as np
|
||||||
|
except Exception: # noqa: BLE001 - a partial install must still reopen points
|
||||||
|
return
|
||||||
|
|
||||||
|
width = max(len(row) for row in point.samples)
|
||||||
|
padded = [tuple(row) + (float("nan"),) * (width - len(row)) for row in point.samples]
|
||||||
|
block = np.asarray(padded, dtype=np.float64)
|
||||||
|
if block.shape[1] < len(SAMPLE_FIELDS):
|
||||||
|
return
|
||||||
|
valid = block[:, 8] > 0.5
|
||||||
|
valid_block = block[valid]
|
||||||
|
if not point.trace and valid_block.size:
|
||||||
|
point.trace = trace_from(valid_block)
|
||||||
|
if not point.trace_dt_s and valid_block.size:
|
||||||
|
point.trace_dt_s = trace_dt(valid_block)
|
||||||
|
if not point.duration_s and block.shape[0] > 1:
|
||||||
|
span = float(block[-1, 1] - block[0, 1])
|
||||||
|
if span == span and span >= 0.0:
|
||||||
|
point.duration_s = span
|
||||||
|
|
||||||
|
|
||||||
def _optional_float(value: Any) -> float:
|
def _optional_float(value: Any) -> float:
|
||||||
if value is None:
|
if value is None:
|
||||||
return float("nan")
|
return float("nan")
|
||||||
|
|||||||
@@ -0,0 +1,170 @@
|
|||||||
|
"""The calibration data shapes, independent of sampling and of file storage.
|
||||||
|
|
||||||
|
They live here so the sampling state machine and the session file format can both
|
||||||
|
describe a calibration point without importing each other. ``calibration_recorder``
|
||||||
|
and ``calibration_session`` re-export these names for compatibility.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
#: Stands in for "this value is not available".
|
||||||
|
NA = float("nan")
|
||||||
|
|
||||||
|
ChannelKey = Literal["fn", "fx", "fy"]
|
||||||
|
CHANNEL_KEYS: tuple[ChannelKey, ...] = ("fn", "fx", "fy")
|
||||||
|
|
||||||
|
STATUS_OK = "ok"
|
||||||
|
STATUS_FEW_SAMPLES = "few_samples"
|
||||||
|
STATUS_CANCELLED = "cancelled"
|
||||||
|
|
||||||
|
#: Columns of one audit sample row, in order. A calibration record is only
|
||||||
|
#: auditable if this layout is fixed: it is what a reopened point is rebuilt from.
|
||||||
|
SAMPLE_FIELDS = (
|
||||||
|
"frame_id",
|
||||||
|
"t_rel_s",
|
||||||
|
"fn_raw",
|
||||||
|
"fx_raw",
|
||||||
|
"fy_raw",
|
||||||
|
"fn_n",
|
||||||
|
"fx_n",
|
||||||
|
"fy_n",
|
||||||
|
"valid",
|
||||||
|
)
|
||||||
|
|
||||||
|
#: Index of each sample column, so callers do not hard-code positions.
|
||||||
|
SAMPLE_INDEX = {name: index for index, name in enumerate(SAMPLE_FIELDS)}
|
||||||
|
|
||||||
|
#: Cap on stored trace samples per axis; peaks are preserved when reducing.
|
||||||
|
MAX_TRACE_POINTS = 400
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AxisStats:
|
||||||
|
"""Mean / standard deviation / valid count for one axis."""
|
||||||
|
|
||||||
|
mean: float = NA
|
||||||
|
std: float = NA
|
||||||
|
n: int = 0
|
||||||
|
|
||||||
|
def as_dict(self) -> dict:
|
||||||
|
return {"mean": self.mean, "std": self.std, "n": self.n}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CalibrationPoint:
|
||||||
|
"""One recorded calibration point, with all three axes captured."""
|
||||||
|
|
||||||
|
channel: ChannelKey
|
||||||
|
reference_load_n: float
|
||||||
|
direction: str = "load"
|
||||||
|
round_index: int = 0
|
||||||
|
note: str = ""
|
||||||
|
started_at: str = ""
|
||||||
|
ended_at: str = ""
|
||||||
|
duration_s: float = 0.0
|
||||||
|
requested_duration_s: float = 0.0
|
||||||
|
n_total: int = 0
|
||||||
|
n_valid: int = 0
|
||||||
|
raw: dict[str, AxisStats] = field(default_factory=dict)
|
||||||
|
newtons: dict[str, AxisStats] = field(default_factory=dict)
|
||||||
|
status: str = STATUS_OK
|
||||||
|
#: Downsampled per-axis raw trace, so a row can be inspected after the fact.
|
||||||
|
trace: dict[str, list[float]] = field(default_factory=dict)
|
||||||
|
trace_dt_s: float = 0.0
|
||||||
|
#: **Audit** series: every sample of the point, in :data:`SAMPLE_FIELDS` order.
|
||||||
|
#: The trace above is a preview and must never be the only surviving copy.
|
||||||
|
samples: list[tuple] = field(default_factory=list)
|
||||||
|
#: Diagnostics, all operator-controlled. Excluded points leave the fit;
|
||||||
|
#: validation points are held out of it and used to test it.
|
||||||
|
excluded: bool = False
|
||||||
|
is_validation: bool = False
|
||||||
|
#: Filled in by :meth:`CalibrationRecorder.fit`.
|
||||||
|
residual_n: float = NA
|
||||||
|
|
||||||
|
def loaded_stats(self) -> AxisStats:
|
||||||
|
return self.raw.get(self.channel, AxisStats())
|
||||||
|
|
||||||
|
def axis_mean(self, channel: str) -> float:
|
||||||
|
stats = self.raw.get(channel)
|
||||||
|
return stats.mean if stats is not None else NA
|
||||||
|
|
||||||
|
def state_key(self) -> str:
|
||||||
|
if self.excluded:
|
||||||
|
return "cal.state_excluded"
|
||||||
|
if self.is_validation:
|
||||||
|
return "cal.state_validation"
|
||||||
|
return "cal.state_ok"
|
||||||
|
|
||||||
|
def as_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"channel": self.channel,
|
||||||
|
"reference_load_n": self.reference_load_n,
|
||||||
|
# Only the loaded axis has a known reference load; the other two are
|
||||||
|
# intentionally absent rather than zero.
|
||||||
|
"direction": self.direction,
|
||||||
|
"round_index": self.round_index,
|
||||||
|
"note": self.note,
|
||||||
|
"started_at": self.started_at,
|
||||||
|
"ended_at": self.ended_at,
|
||||||
|
"duration_s": self.duration_s,
|
||||||
|
"requested_duration_s": self.requested_duration_s,
|
||||||
|
"n_total": self.n_total,
|
||||||
|
"n_valid": self.n_valid,
|
||||||
|
"status": self.status,
|
||||||
|
"excluded": self.excluded,
|
||||||
|
"is_validation": self.is_validation,
|
||||||
|
"residual_n": self.residual_n,
|
||||||
|
"raw": {key: stats.as_dict() for key, stats in self.raw.items()},
|
||||||
|
"newtons": {key: stats.as_dict() for key, stats in self.newtons.items()},
|
||||||
|
# The audit series is part of the point's identity, not a UI extra:
|
||||||
|
# without it the only surviving copy of the samples is the
|
||||||
|
# downsampled trace.
|
||||||
|
"sample_fields": list(SAMPLE_FIELDS),
|
||||||
|
"n_samples": len(self.samples),
|
||||||
|
"samples": [list(row) for row in self.samples],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def trace_from(valid_block: np.ndarray) -> dict[str, list[float]]:
|
||||||
|
"""Downsampled raw trace per axis, bounded by ``MAX_TRACE_POINTS``.
|
||||||
|
|
||||||
|
One downsampling rule, whether the point was just recorded or read back from
|
||||||
|
a saved session.
|
||||||
|
"""
|
||||||
|
if valid_block.size == 0:
|
||||||
|
return {}
|
||||||
|
traces: dict[str, list[float]] = {}
|
||||||
|
for key, column in (("fn", 2), ("fx", 3), ("fy", 4)):
|
||||||
|
values = valid_block[:, column]
|
||||||
|
values = values[np.isfinite(values)]
|
||||||
|
if values.size == 0:
|
||||||
|
continue
|
||||||
|
if values.size > MAX_TRACE_POINTS:
|
||||||
|
# Keep the shape of the trace, including its extremes.
|
||||||
|
edges = np.linspace(0, values.size, MAX_TRACE_POINTS + 1, dtype=int)
|
||||||
|
reduced = [
|
||||||
|
values[start:max(start + 1, stop)]
|
||||||
|
for start, stop in zip(edges[:-1], edges[1:])
|
||||||
|
]
|
||||||
|
values = np.asarray(
|
||||||
|
[
|
||||||
|
bucket[int(np.argmax(np.abs(bucket)))]
|
||||||
|
for bucket in reduced
|
||||||
|
if bucket.size
|
||||||
|
]
|
||||||
|
)
|
||||||
|
traces[key] = [float(value) for value in values]
|
||||||
|
return traces
|
||||||
|
|
||||||
|
|
||||||
|
def trace_dt(valid_block: np.ndarray) -> float:
|
||||||
|
"""The sample interval of a trace, from the block it was taken from."""
|
||||||
|
if valid_block.shape[0] < 2:
|
||||||
|
return 0.0
|
||||||
|
span = float(valid_block[-1, 1] - valid_block[0, 1])
|
||||||
|
return span / max(1, valid_block.shape[0] - 1)
|
||||||
@@ -29,6 +29,8 @@ from typing import Iterable, Iterator, Optional
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from orisys_app_shared.record.sample import plottable_axis_mask
|
||||||
|
|
||||||
#: Stands in for "this value is not available / not valid".
|
#: Stands in for "this value is not available / not valid".
|
||||||
NA = float("nan")
|
NA = float("nan")
|
||||||
|
|
||||||
@@ -127,14 +129,20 @@ class SessionSlice:
|
|||||||
def plottable(self, unit: str) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
def plottable(self, unit: str) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
||||||
"""``(times, values, mask)`` for drawing, under one shared rule.
|
"""``(times, values, mask)`` for drawing, under one shared rule.
|
||||||
|
|
||||||
A point is drawn only when the sample is **valid** *and* the value for the
|
``mask`` is per axis ``(n, 3)``: a slice is drawn only where the sample is
|
||||||
requested unit is finite. Filtering on finiteness alone let a stale
|
**valid** *and that axis* is finite. Filtering on finiteness alone let a
|
||||||
number — left behind by a failed compute step — reappear in one view
|
stale number — left behind by a failed compute step — reappear in one view
|
||||||
after another view had already excluded it.
|
after another view had already excluded it, and an all-axes rule hid a
|
||||||
|
valid FN whenever the device had no shear calibration.
|
||||||
"""
|
"""
|
||||||
values = self.value_for_unit(unit)
|
values = self.value_for_unit(unit)
|
||||||
mask = self.valid & np.isfinite(values).all(axis=1)
|
return self.times, values, plottable_axis_mask(values, self.valid)
|
||||||
return self.times[mask], values[mask], mask
|
|
||||||
|
def plottable_axis(self, unit: str, index: int) -> tuple[np.ndarray, np.ndarray]:
|
||||||
|
"""``(times, values)`` for one channel, with the other axes ignored."""
|
||||||
|
values = self.value_for_unit(unit)
|
||||||
|
mask = plottable_axis_mask(values, self.valid)[:, index]
|
||||||
|
return self.times[mask], values[mask, index]
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
|
|||||||
@@ -0,0 +1,228 @@
|
|||||||
|
"""One interface for reading curve data, whatever it is stored in.
|
||||||
|
|
||||||
|
The live charts read a bounded in-memory session store; the replay reads a
|
||||||
|
recording on disk. Both have to answer the same questions — what is the extent,
|
||||||
|
what is in this interval, what does the whole session look like as an envelope —
|
||||||
|
so the plotting code asks a *source* instead of knowing which one it has.
|
||||||
|
|
||||||
|
A source never hands over more than the caller's point budget, and it always
|
||||||
|
reports what it had to do to fit: ``decimated`` means the values are a view, not
|
||||||
|
every sample.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Protocol
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
from orisys_app_shared.force.session import NA, SessionSlice
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SourceStats:
|
||||||
|
"""What the selection readout shows, for any source."""
|
||||||
|
|
||||||
|
duration_s: float = 0.0
|
||||||
|
samples: int = 0
|
||||||
|
peaks: tuple[float, float, float] = (NA, NA, NA)
|
||||||
|
means: tuple[float, float, float] = (NA, NA, NA)
|
||||||
|
stds: tuple[float, float, float] = (NA, NA, NA)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SourceWindow:
|
||||||
|
"""Rows for one interval, in the shape the charts already consume."""
|
||||||
|
|
||||||
|
times: np.ndarray
|
||||||
|
values: np.ndarray
|
||||||
|
valid: np.ndarray
|
||||||
|
frame_ids: np.ndarray
|
||||||
|
unit: str = "raw"
|
||||||
|
decimated: bool = False
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return int(self.times.size)
|
||||||
|
|
||||||
|
def plottable_axis(self, index: int):
|
||||||
|
from orisys_app_shared.record.sample import plottable_axis_mask
|
||||||
|
|
||||||
|
mask = plottable_axis_mask(self.values, self.valid)[:, index]
|
||||||
|
return self.times[mask], self.values[mask, index]
|
||||||
|
|
||||||
|
def statistics(self) -> SourceStats:
|
||||||
|
mask = np.isfinite(self.values) & self.valid[:, None]
|
||||||
|
peaks: list[float] = []
|
||||||
|
means: list[float] = []
|
||||||
|
stds: list[float] = []
|
||||||
|
for index in range(3):
|
||||||
|
column = self.values[:, index][mask[:, index]]
|
||||||
|
if column.size == 0:
|
||||||
|
peaks.append(NA)
|
||||||
|
means.append(NA)
|
||||||
|
stds.append(NA)
|
||||||
|
continue
|
||||||
|
peaks.append(float(np.nanmax(column)))
|
||||||
|
means.append(float(np.nanmean(column)))
|
||||||
|
stds.append(float(np.nanstd(column)))
|
||||||
|
span = float(self.times[-1] - self.times[0]) if self.times.size > 1 else 0.0
|
||||||
|
return SourceStats(
|
||||||
|
duration_s=span,
|
||||||
|
samples=int(mask.any(axis=1).sum()),
|
||||||
|
peaks=tuple(peaks),
|
||||||
|
means=tuple(means),
|
||||||
|
stds=tuple(stds),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CurveSource(Protocol):
|
||||||
|
"""What the charts need from wherever the data lives."""
|
||||||
|
|
||||||
|
kind: str
|
||||||
|
|
||||||
|
def extent(self) -> tuple[float, float]: ...
|
||||||
|
|
||||||
|
def window(self, t0: float, t1: float, *, max_points: int | None = None) -> SourceWindow: ...
|
||||||
|
|
||||||
|
def envelope(self, *, buckets: int = 1200) -> SourceWindow: ...
|
||||||
|
|
||||||
|
def duration_s(self) -> float: ...
|
||||||
|
|
||||||
|
|
||||||
|
class StoreCurveSource:
|
||||||
|
"""The live session store: already in memory, bounded by its own blocks."""
|
||||||
|
|
||||||
|
kind = "memory"
|
||||||
|
|
||||||
|
def __init__(self, store=None) -> None:
|
||||||
|
self.store = store
|
||||||
|
|
||||||
|
def set_store(self, store) -> None:
|
||||||
|
self.store = store
|
||||||
|
|
||||||
|
def extent(self) -> tuple[float, float]:
|
||||||
|
if self.store is None:
|
||||||
|
return (0.0, 0.0)
|
||||||
|
return self.store.extent()
|
||||||
|
|
||||||
|
def duration_s(self) -> float:
|
||||||
|
if self.store is None:
|
||||||
|
return 0.0
|
||||||
|
first, last = self.store.extent()
|
||||||
|
return max(0.0, last - first)
|
||||||
|
|
||||||
|
def _slice(self, t0: float, t1: float, max_points: int | None, unit: str) -> SessionSlice:
|
||||||
|
if self.store is None:
|
||||||
|
empty = np.empty(0)
|
||||||
|
return SessionSlice(
|
||||||
|
times=empty,
|
||||||
|
raw=np.empty((0, 3)),
|
||||||
|
newtons=np.empty((0, 3)),
|
||||||
|
valid=np.empty(0, dtype=bool),
|
||||||
|
frame_ids=np.empty(0, dtype=np.int64),
|
||||||
|
)
|
||||||
|
return self.store.visible(
|
||||||
|
t0, t1, max_points=max_points or 0, unit=unit
|
||||||
|
)
|
||||||
|
|
||||||
|
def window(
|
||||||
|
self,
|
||||||
|
t0: float,
|
||||||
|
t1: float,
|
||||||
|
*,
|
||||||
|
max_points: int | None = None,
|
||||||
|
unit: str = "raw",
|
||||||
|
) -> SourceWindow:
|
||||||
|
slice_ = self._slice(t0, t1, max_points, unit)
|
||||||
|
return SourceWindow(
|
||||||
|
times=slice_.times,
|
||||||
|
values=slice_.value_for_unit(unit),
|
||||||
|
valid=slice_.valid,
|
||||||
|
frame_ids=slice_.frame_ids,
|
||||||
|
unit=unit,
|
||||||
|
decimated=bool(max_points and slice_.times.size >= int(max_points)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def envelope(self, *, buckets: int = 1200, unit: str = "raw") -> SourceWindow:
|
||||||
|
"""The whole session, decimated — the store's own bounded view."""
|
||||||
|
if self.store is None:
|
||||||
|
return self.window(0.0, 0.0)
|
||||||
|
first, last = self.store.extent()
|
||||||
|
return self.window(first, last, max_points=int(buckets) * 2, unit=unit)
|
||||||
|
|
||||||
|
def statistics(self, t0: float, t1: float, *, unit: str = "raw") -> SourceStats:
|
||||||
|
if self.store is None:
|
||||||
|
return SourceStats()
|
||||||
|
stats = self.store.statistics(t0, t1, unit=unit)
|
||||||
|
return SourceStats(
|
||||||
|
duration_s=stats.duration_s,
|
||||||
|
samples=stats.samples,
|
||||||
|
peaks=stats.peaks,
|
||||||
|
means=stats.means,
|
||||||
|
stds=stats.stds,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingCurveSource:
|
||||||
|
"""A recording on disk: interval reads plus a cached envelope."""
|
||||||
|
|
||||||
|
kind = "disk"
|
||||||
|
|
||||||
|
def __init__(self, reader) -> None:
|
||||||
|
self.reader = reader
|
||||||
|
|
||||||
|
def extent(self) -> tuple[float, float]:
|
||||||
|
return self.reader.index().extent()
|
||||||
|
|
||||||
|
def duration_s(self) -> float:
|
||||||
|
return self.reader.index().duration_s()
|
||||||
|
|
||||||
|
def window(
|
||||||
|
self,
|
||||||
|
t0: float,
|
||||||
|
t1: float,
|
||||||
|
*,
|
||||||
|
max_points: int | None = None,
|
||||||
|
unit: str = "raw",
|
||||||
|
) -> SourceWindow:
|
||||||
|
window = self.reader.window(t0, t1, unit=unit, max_points=max_points)
|
||||||
|
return SourceWindow(
|
||||||
|
times=window.times,
|
||||||
|
values=window.value_for_unit(unit),
|
||||||
|
valid=window.valid,
|
||||||
|
frame_ids=window.frame_ids,
|
||||||
|
unit=unit,
|
||||||
|
decimated=window.decimated,
|
||||||
|
)
|
||||||
|
|
||||||
|
def envelope(self, *, buckets: int = 1200, unit: str = "raw") -> SourceWindow:
|
||||||
|
"""The whole-session envelope, from the index's cached overview."""
|
||||||
|
overview = self.reader.overview(unit=unit, buckets=int(buckets))
|
||||||
|
keep = ~np.isnan(overview.times)
|
||||||
|
times = overview.times[keep]
|
||||||
|
if times.size == 0:
|
||||||
|
return self.window(0.0, 0.0, unit=unit)
|
||||||
|
# The envelope is drawn as a band: the low and high of each bucket. It is
|
||||||
|
# reported as a plain window so callers need only one shape.
|
||||||
|
values = np.where(
|
||||||
|
np.isnan(overview.highs[keep]), overview.lows[keep], overview.highs[keep]
|
||||||
|
)
|
||||||
|
return SourceWindow(
|
||||||
|
times=times,
|
||||||
|
values=values,
|
||||||
|
valid=np.ones(times.size, dtype=bool),
|
||||||
|
frame_ids=np.full(times.size, -1, dtype=np.int64),
|
||||||
|
unit=unit,
|
||||||
|
decimated=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
def statistics(self, t0: float, t1: float, *, unit: str = "raw") -> SourceStats:
|
||||||
|
stats = self.reader.statistics(t0, t1, unit=unit)
|
||||||
|
return SourceStats(
|
||||||
|
duration_s=stats.duration_s,
|
||||||
|
samples=stats.samples,
|
||||||
|
peaks=stats.peaks,
|
||||||
|
means=stats.means,
|
||||||
|
stds=stats.stds,
|
||||||
|
)
|
||||||
@@ -1,46 +1,104 @@
|
|||||||
"""Force unit selection for display."""
|
"""Force unit selection for display.
|
||||||
|
|
||||||
|
The decision is made **per axis** from the standardised measurement
|
||||||
|
(:class:`~orisys_app_shared.record.sample.RecordSample`), not from a device-level
|
||||||
|
``force_calibrated`` flag. A device can legitimately have only its normal-force
|
||||||
|
axis calibrated: the old rule then either showed every axis in raw (losing the
|
||||||
|
one axis that really was in newtons) or, worse, showed raw counts in the newton
|
||||||
|
panels. An axis without a curve is now simply **unavailable** in newton mode and
|
||||||
|
reported as such.
|
||||||
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from orisys_app_shared.record.sample import AXES
|
||||||
|
|
||||||
|
#: Index of each axis inside the triplets, matching ``AXES``.
|
||||||
|
AXIS_INDEX: dict[str, int] = {axis: index for index, axis in enumerate(AXES)}
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class DisplayForces:
|
class DisplayForces:
|
||||||
fn: float
|
"""What the live view should show for one sample."""
|
||||||
fx: float
|
|
||||||
fy: float
|
#: One value per axis, in ``AXES`` order. NaN means "not available".
|
||||||
|
values: tuple[float, float, float]
|
||||||
|
#: The unit the values actually are (``N`` only when they are newtons).
|
||||||
unit: str
|
unit: str
|
||||||
|
#: Per axis: is this value a real measurement in ``unit``?
|
||||||
|
available: tuple[bool, bool, bool]
|
||||||
|
#: The requested unit was newton but no axis had a curve, so raw is shown.
|
||||||
using_raw_fallback: bool
|
using_raw_fallback: bool
|
||||||
|
#: What the operator asked for (``N``/``raw``), for the fallback message.
|
||||||
|
requested_unit: str
|
||||||
|
|
||||||
|
def value(self, axis: str) -> float:
|
||||||
|
return self.values[AXIS_INDEX[axis]]
|
||||||
|
|
||||||
|
def is_available(self, axis: str) -> bool:
|
||||||
|
return self.available[AXIS_INDEX[axis]]
|
||||||
|
|
||||||
|
def unavailable_axes(self) -> tuple[str, ...]:
|
||||||
|
return tuple(
|
||||||
|
axis for axis in AXES if not self.available[AXIS_INDEX[axis]]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def select_display_forces(
|
def _finite(value: float) -> bool:
|
||||||
*,
|
try:
|
||||||
units_pref: str,
|
return value == value and value not in (float("inf"), float("-inf"))
|
||||||
force_calibrated: bool,
|
except TypeError: # pragma: no cover - defensive
|
||||||
fn: float,
|
return False
|
||||||
fx: float,
|
|
||||||
fy: float,
|
|
||||||
fn_raw: float,
|
def select_display_forces(*, units_pref: str, sample) -> DisplayForces:
|
||||||
fx_raw: float,
|
"""Pick the values to display for one standardised measurement.
|
||||||
fy_raw: float,
|
|
||||||
) -> DisplayForces:
|
``sample`` is a
|
||||||
"""Return values for plots/labels given newton/raw preference."""
|
:class:`~orisys_app_shared.record.sample.RecordSample`, so the same conversion
|
||||||
|
that feeds the CSV and the session store also decides what the live charts
|
||||||
|
show: there is no second interpretation of what a value means.
|
||||||
|
"""
|
||||||
want_newton = str(units_pref or "newton").lower().startswith("n")
|
want_newton = str(units_pref or "newton").lower().startswith("n")
|
||||||
if want_newton and force_calibrated:
|
raw = tuple(sample.raw_for(axis) for axis in AXES)
|
||||||
return DisplayForces(fn=fn, fx=fx, fy=fy, unit="N", using_raw_fallback=False)
|
newton = tuple(sample.newton_for(axis) for axis in AXES)
|
||||||
if want_newton and not force_calibrated:
|
requested = "N" if want_newton else "raw"
|
||||||
|
|
||||||
|
if not want_newton:
|
||||||
return DisplayForces(
|
return DisplayForces(
|
||||||
fn=fn_raw,
|
values=raw,
|
||||||
fx=fx_raw,
|
|
||||||
fy=fy_raw,
|
|
||||||
unit="raw",
|
|
||||||
using_raw_fallback=True,
|
|
||||||
)
|
|
||||||
return DisplayForces(
|
|
||||||
fn=fn_raw,
|
|
||||||
fx=fx_raw,
|
|
||||||
fy=fy_raw,
|
|
||||||
unit="raw",
|
unit="raw",
|
||||||
|
available=tuple(_finite(value) for value in raw),
|
||||||
using_raw_fallback=False,
|
using_raw_fallback=False,
|
||||||
|
requested_unit=requested,
|
||||||
|
)
|
||||||
|
|
||||||
|
values: list[float] = []
|
||||||
|
available: list[bool] = []
|
||||||
|
for index, _axis in enumerate(AXES):
|
||||||
|
if _finite(newton[index]):
|
||||||
|
values.append(newton[index])
|
||||||
|
available.append(True)
|
||||||
|
else:
|
||||||
|
# Not "zero" and not the raw count: there is no newton value at all.
|
||||||
|
values.append(float("nan"))
|
||||||
|
available.append(False)
|
||||||
|
|
||||||
|
if not any(available):
|
||||||
|
# Nothing is calibrated: showing raw is honest as long as it is labelled.
|
||||||
|
return DisplayForces(
|
||||||
|
values=raw,
|
||||||
|
unit="raw",
|
||||||
|
available=tuple(_finite(value) for value in raw),
|
||||||
|
using_raw_fallback=True,
|
||||||
|
requested_unit=requested,
|
||||||
|
)
|
||||||
|
return DisplayForces(
|
||||||
|
values=tuple(values),
|
||||||
|
unit="N",
|
||||||
|
available=tuple(available),
|
||||||
|
using_raw_fallback=False,
|
||||||
|
requested_unit=requested,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||
"""Experiment recording (video + data + metadata) and replay."""
|
"""Experiment recording (video + data + metadata) and replay."""
|
||||||
|
|
||||||
|
from orisys_app_shared.record.calibration_snapshot import (
|
||||||
|
list_calibration_snapshots,
|
||||||
|
store_calibration_snapshot,
|
||||||
|
)
|
||||||
from orisys_app_shared.record.depth import DepthRecomputer, DepthResult
|
from orisys_app_shared.record.depth import DepthRecomputer, DepthResult
|
||||||
from orisys_app_shared.record.experiment import (
|
from orisys_app_shared.record.experiment import (
|
||||||
EVENTS_HEADER,
|
EVENTS_HEADER,
|
||||||
@@ -34,5 +38,7 @@ __all__ = [
|
|||||||
"RecordingInfo",
|
"RecordingInfo",
|
||||||
"RecordingReader",
|
"RecordingReader",
|
||||||
"format_value",
|
"format_value",
|
||||||
|
"list_calibration_snapshots",
|
||||||
"list_recordings",
|
"list_recordings",
|
||||||
|
"store_calibration_snapshot",
|
||||||
]
|
]
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
"""Store the coefficients a recording was made with, next to the recording.
|
||||||
|
|
||||||
|
The hash of a calibration tells you *which* coefficients were applied; it does not
|
||||||
|
let you check what they were, and the original export file can be overwritten or
|
||||||
|
deleted afterwards. So the recording keeps its own copy:
|
||||||
|
|
||||||
|
```
|
||||||
|
outputs/recordings/<stamp>_session<NNNN>/
|
||||||
|
calibration/
|
||||||
|
force_calibration_<hash12>.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The copy is content-addressed by the digest, so applying the same calibration
|
||||||
|
twice in one session stores one file, and two different calibrations can never
|
||||||
|
collide.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#: Directory inside a recording that holds the coefficient snapshots.
|
||||||
|
SNAPSHOT_DIRNAME = "calibration"
|
||||||
|
|
||||||
|
|
||||||
|
def snapshot_name(version: str, source: Path) -> str:
|
||||||
|
"""The snapshot's file name: content-addressed, with the original extension."""
|
||||||
|
digest = str(version or "").strip()
|
||||||
|
stem = f"force_calibration_{digest[:12]}" if digest else f"force_calibration_{source.stem}"
|
||||||
|
return f"{stem}{source.suffix or '.json'}"
|
||||||
|
|
||||||
|
|
||||||
|
def store_calibration_snapshot(directory: str | Path, source: str | Path, version: str) -> str:
|
||||||
|
"""Copy ``source`` into ``directory/calibration``. Returns the relative name.
|
||||||
|
|
||||||
|
Raises on I/O failure; the caller reports it and keeps recording.
|
||||||
|
"""
|
||||||
|
base = Path(directory) / SNAPSHOT_DIRNAME
|
||||||
|
base.mkdir(parents=True, exist_ok=True)
|
||||||
|
origin = Path(source)
|
||||||
|
target = base / snapshot_name(version, origin)
|
||||||
|
if not target.is_file() or target.stat().st_size != origin.stat().st_size:
|
||||||
|
# Same content-addressed name means the same coefficients; a size check
|
||||||
|
# is enough to notice a truncated previous copy.
|
||||||
|
shutil.copy2(origin, target)
|
||||||
|
return f"{SNAPSHOT_DIRNAME}/{target.name}"
|
||||||
|
|
||||||
|
|
||||||
|
def list_calibration_snapshots(directory: str | Path) -> list[str]:
|
||||||
|
"""Every snapshot already stored in a recording, sorted."""
|
||||||
|
base = Path(directory) / SNAPSHOT_DIRNAME
|
||||||
|
if not base.is_dir():
|
||||||
|
return []
|
||||||
|
return sorted(
|
||||||
|
f"{SNAPSHOT_DIRNAME}/{path.name}" for path in base.iterdir() if path.is_file()
|
||||||
|
)
|
||||||
@@ -25,6 +25,7 @@ from typing import Optional
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from orisys_app_shared.record.depth_backends import open_replay
|
||||||
from orisys_app_shared.record.playback import RecordingReader
|
from orisys_app_shared.record.playback import RecordingReader
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -40,25 +41,72 @@ class DepthResult:
|
|||||||
t_rel_s: float = 0.0
|
t_rel_s: float = 0.0
|
||||||
#: What a replay could not reproduce exactly (never silently empty).
|
#: What a replay could not reproduce exactly (never silently empty).
|
||||||
warnings: tuple[str, ...] = ()
|
warnings: tuple[str, ...] = ()
|
||||||
|
#: Which recording this answer belongs to (the replay window compares it).
|
||||||
|
record_id: str = ""
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def ok(self) -> bool:
|
def ok(self) -> bool:
|
||||||
return self.depth is not None and not self.error
|
return self.depth is not None and not self.error
|
||||||
|
|
||||||
|
|
||||||
|
#: The replay started at a **reset**: the algorithm's state was cleared there, so
|
||||||
|
#: the frames since that point determine it exactly.
|
||||||
|
STATE_EXACT = "exact"
|
||||||
|
#: The replay is preceded by the recorded pre-roll: the intended reconstruction.
|
||||||
|
STATE_PREROLL = "preroll"
|
||||||
|
#: Neither: the state entering the target frame is unknown and may differ from
|
||||||
|
#: what the live run had.
|
||||||
|
STATE_APPROXIMATE = "approximate"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ReplayStep:
|
||||||
|
"""One leg of a replay: read ``source`` up to ``to_frame``.
|
||||||
|
|
||||||
|
``reset_after`` is the frame at which the algorithm's state was cleared live
|
||||||
|
(an operator ``zeroing``). The replay reaches that frame, then calls the
|
||||||
|
processor's ``reset()`` — which is what the live run did — and keeps reading.
|
||||||
|
"""
|
||||||
|
|
||||||
|
source: Path
|
||||||
|
to_frame: int
|
||||||
|
reset_after: Optional[int] = None
|
||||||
|
|
||||||
|
|
||||||
|
#: How many SDK reads a faithful replay may cost before it is declared
|
||||||
|
#: unrecoverable (5 minutes at 30 fps). A replay that re-reads an hour of video
|
||||||
|
#: to answer one cursor move is not a feature; saying so is.
|
||||||
|
MAX_REPLAY_FRAMES = 9000
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class ReplayPlan:
|
class ReplayPlan:
|
||||||
"""Where the SDK must start reading to reproduce one frame's state."""
|
"""Where the SDK must start reading to reproduce one frame's state."""
|
||||||
|
|
||||||
segment: Path
|
segment: Path
|
||||||
#: Frame within ``segment`` to step to (1-based, as the SDK reads them).
|
#: The legs of the replay, in order. The last one ends at ``stop_at``.
|
||||||
stop_at: int
|
steps: tuple[ReplayStep, ...] = ()
|
||||||
#: Frames to feed before ``stop_at`` to rebuild the algorithm's state.
|
#: Frame within the final segment the caller asked for (1-based).
|
||||||
|
stop_at: int = 0
|
||||||
|
#: Frames to feed from another file before the first step.
|
||||||
preroll: Path | None = None
|
preroll: Path | None = None
|
||||||
preroll_frames: int = 0
|
preroll_frames: int = 0
|
||||||
|
#: The ``zeroing`` frame the plan is anchored to, when there is one.
|
||||||
|
reset_frame_id: Optional[int] = None
|
||||||
|
#: ``reset`` / ``preroll`` / ``segment_start``.
|
||||||
|
anchor: str = "segment_start"
|
||||||
|
#: :data:`STATE_EXACT`, :data:`STATE_PREROLL` or :data:`STATE_APPROXIMATE`.
|
||||||
|
state: str = STATE_APPROXIMATE
|
||||||
warnings: tuple[str, ...] = ()
|
warnings: tuple[str, ...] = ()
|
||||||
error: str = ""
|
error: str = ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def frames_to_read(self) -> int:
|
||||||
|
"""How many SDK reads a faithful recomputation costs."""
|
||||||
|
return int(self.preroll_frames) + sum(
|
||||||
|
max(0, int(step.to_frame)) for step in self.steps
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class DepthRecomputer:
|
class DepthRecomputer:
|
||||||
"""Recompute depth maps for one recording, on demand."""
|
"""Recompute depth maps for one recording, on demand."""
|
||||||
@@ -69,6 +117,12 @@ class DepthRecomputer:
|
|||||||
self._segment: Optional[Path] = None
|
self._segment: Optional[Path] = None
|
||||||
#: How many frames the open sensor has already consumed in ``_segment``.
|
#: How many frames the open sensor has already consumed in ``_segment``.
|
||||||
self._position = 0
|
self._position = 0
|
||||||
|
#: The depth of ``_position``, so asking for the same frame twice is a
|
||||||
|
#: cache hit instead of a second (impossible) read.
|
||||||
|
self._last_depth: Optional[np.ndarray] = None
|
||||||
|
#: True when an earlier frame had to be re-read without a reset boundary
|
||||||
|
#: to clean up after it, so the state may not match the live run.
|
||||||
|
self._rewound_without_anchor = False
|
||||||
self.last_error = ""
|
self.last_error = ""
|
||||||
self.last_warnings: list[str] = []
|
self.last_warnings: list[str] = []
|
||||||
|
|
||||||
@@ -121,46 +175,87 @@ class DepthRecomputer:
|
|||||||
"""Decide where replay must start so ``frame`` is reproducible.
|
"""Decide where replay must start so ``frame`` is reproducible.
|
||||||
|
|
||||||
The SDK's depth is the output of a stateful chain (optical-flow tracker,
|
The SDK's depth is the output of a stateful chain (optical-flow tracker,
|
||||||
temporal filters), so a frame cannot be recomputed in isolation. Two
|
temporal filters), so a frame cannot be recomputed in isolation. What the
|
||||||
things make the state recoverable:
|
plan states explicitly:
|
||||||
|
|
||||||
* an operator **reset** clears the tracker, so replaying from the reset
|
* **where** to start (``start_at``, plus a pre-roll file when there is one);
|
||||||
frame reproduces the state exactly — no earlier history needed;
|
* which **reset boundary** the replay is anchored to, if any;
|
||||||
* the **pre-roll** recorded with the session holds the frames acquired
|
* whether the state is actually recoverable — :data:`STATE_EXACT` (the
|
||||||
just before recording started, which is the only history a reset-free
|
replay starts at a reset), :data:`STATE_PREROLL` (the recorded pre-roll
|
||||||
start has.
|
precedes it) or :data:`STATE_APPROXIMATE` (neither: the depth map is
|
||||||
|
still produced, and the caller is told it may differ from live).
|
||||||
|
|
||||||
Anything else is replayed as faithfully as possible *and reported*: the
|
Finding a reset event is **not** by itself a recovered state: the reset
|
||||||
caller is told the state may differ instead of being shown a depth map
|
frame has to exist in the index *and* in the video, and it has to be in
|
||||||
that pretends to be the live one.
|
the same segment, because the SDK cannot carry its state across a segment
|
||||||
|
boundary.
|
||||||
"""
|
"""
|
||||||
warnings: list[str] = []
|
warnings: list[str] = []
|
||||||
segment_path = self.reader.segment_path(frame.segment)
|
segment_path = self.reader.segment_path(frame.segment)
|
||||||
if segment_path is None:
|
if segment_path is None:
|
||||||
return ReplayPlan(
|
return ReplayPlan(segment=Path(), error="segment_missing")
|
||||||
segment=Path(), stop_at=0, error="segment_missing"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
stop_at = max(1, int(frame.segment_frame))
|
||||||
reset_frame = self.reader.last_reset_frame(frame.frame_id)
|
reset_frame = self.reader.last_reset_frame(frame.frame_id)
|
||||||
if reset_frame is not None:
|
if reset_frame is not None:
|
||||||
origin = self.reader.frame_of(reset_frame)
|
origin = self.reader.frame_of(reset_frame)
|
||||||
if origin is not None and origin.segment == frame.segment:
|
if origin is None:
|
||||||
# Exact: the tracker state at ``frame`` is fully determined by
|
warnings.append("reset_frame_not_indexed")
|
||||||
# the frames since the reset in this same segment.
|
elif origin.dropped or not origin.in_video:
|
||||||
return ReplayPlan(
|
# The boundary frame is not in the video, so the state at that
|
||||||
|
# point cannot be rebuilt: replay from here is an approximation.
|
||||||
|
warnings.append("reset_frame_not_in_video")
|
||||||
|
elif origin.segment == frame.segment:
|
||||||
|
plan = ReplayPlan(
|
||||||
segment=segment_path,
|
segment=segment_path,
|
||||||
stop_at=max(1, int(frame.segment_frame)),
|
steps=(
|
||||||
warnings=(),
|
ReplayStep(
|
||||||
|
source=segment_path,
|
||||||
|
to_frame=stop_at,
|
||||||
|
reset_after=int(origin.segment_frame),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
stop_at=stop_at,
|
||||||
|
reset_frame_id=int(reset_frame),
|
||||||
|
anchor="reset",
|
||||||
|
state=STATE_EXACT,
|
||||||
)
|
)
|
||||||
if origin is not None:
|
if plan.frames_to_read > MAX_REPLAY_FRAMES:
|
||||||
# The reset is in an earlier segment. The SDK opens one video per
|
warnings.append("state_too_far_back")
|
||||||
# sensor, so the state cannot be carried across the boundary; the
|
else:
|
||||||
# replay restarts at this segment and says so.
|
return plan
|
||||||
|
else:
|
||||||
|
# The reset is in an earlier segment. The processor survives a
|
||||||
|
# source swap, so the state *can* be carried: read the tail of
|
||||||
|
# the reset's segment, then continue in the target's.
|
||||||
|
reset_segment = self.reader.segment_path(origin.segment)
|
||||||
|
last_of_reset_segment = self._last_segment_frame(origin.segment)
|
||||||
|
if reset_segment is None or last_of_reset_segment is None:
|
||||||
warnings.append("state_across_segment")
|
warnings.append("state_across_segment")
|
||||||
|
else:
|
||||||
|
plan = ReplayPlan(
|
||||||
|
segment=segment_path,
|
||||||
|
steps=(
|
||||||
|
ReplayStep(
|
||||||
|
source=reset_segment,
|
||||||
|
to_frame=int(last_of_reset_segment),
|
||||||
|
reset_after=int(origin.segment_frame),
|
||||||
|
),
|
||||||
|
ReplayStep(source=segment_path, to_frame=stop_at),
|
||||||
|
),
|
||||||
|
stop_at=stop_at,
|
||||||
|
reset_frame_id=int(reset_frame),
|
||||||
|
anchor="reset",
|
||||||
|
state=STATE_EXACT,
|
||||||
|
)
|
||||||
|
if plan.frames_to_read <= MAX_REPLAY_FRAMES:
|
||||||
|
return plan
|
||||||
|
# Too expensive: report it rather than freeze the window.
|
||||||
|
warnings.append("state_too_far_back")
|
||||||
|
|
||||||
start = self.reader.segment_start_frame(frame.segment)
|
start = self.reader.segment_start_frame(frame.segment)
|
||||||
if start is None:
|
if start is None:
|
||||||
return ReplayPlan(segment=Path(), stop_at=0, error="frame_not_indexed")
|
return ReplayPlan(segment=Path(), error="frame_not_indexed")
|
||||||
|
|
||||||
preroll = self.reader.preroll_segment()
|
preroll = self.reader.preroll_segment()
|
||||||
preroll_frames = self.reader.preroll_frames() if preroll is not None else 0
|
preroll_frames = self.reader.preroll_frames() if preroll is not None else 0
|
||||||
@@ -169,19 +264,39 @@ class DepthRecomputer:
|
|||||||
# one are the only history that can rebuild the tracker.
|
# one are the only history that can rebuild the tracker.
|
||||||
return ReplayPlan(
|
return ReplayPlan(
|
||||||
segment=segment_path,
|
segment=segment_path,
|
||||||
stop_at=max(1, int(frame.segment_frame)),
|
steps=(ReplayStep(source=segment_path, to_frame=stop_at),),
|
||||||
|
stop_at=stop_at,
|
||||||
preroll=preroll,
|
preroll=preroll,
|
||||||
preroll_frames=int(preroll_frames),
|
preroll_frames=int(preroll_frames),
|
||||||
|
anchor="preroll",
|
||||||
|
state=STATE_PREROLL,
|
||||||
warnings=tuple(warnings),
|
warnings=tuple(warnings),
|
||||||
)
|
)
|
||||||
if int(frame.segment_frame) > 1:
|
# No reset and no pre-roll: the state entering this frame is unknown.
|
||||||
warnings.append("state_approximate")
|
warnings.append("state_approximate")
|
||||||
return ReplayPlan(
|
return ReplayPlan(
|
||||||
segment=segment_path,
|
segment=segment_path,
|
||||||
stop_at=max(1, int(frame.segment_frame)),
|
steps=(
|
||||||
warnings=tuple(warnings),
|
ReplayStep(
|
||||||
|
source=segment_path,
|
||||||
|
to_frame=stop_at,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
stop_at=stop_at,
|
||||||
|
anchor="segment_start",
|
||||||
|
state=STATE_APPROXIMATE,
|
||||||
|
warnings=tuple(dict.fromkeys(warnings)),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _last_segment_frame(self, segment: int) -> Optional[int]:
|
||||||
|
"""The highest indexed frame number inside ``segment``."""
|
||||||
|
frames = [
|
||||||
|
int(entry.segment_frame)
|
||||||
|
for entry in self.reader.info().frames
|
||||||
|
if int(entry.segment) == int(segment)
|
||||||
|
]
|
||||||
|
return max(frames) if frames else None
|
||||||
|
|
||||||
# ----------------------------------------------------------------- recompute
|
# ----------------------------------------------------------------- recompute
|
||||||
def depth_at(self, t_rel_s: float) -> DepthResult:
|
def depth_at(self, t_rel_s: float) -> DepthResult:
|
||||||
"""Recompute the depth map for the frame at ``t_rel_s``."""
|
"""Recompute the depth map for the frame at ``t_rel_s``."""
|
||||||
@@ -219,32 +334,95 @@ class DepthRecomputer:
|
|||||||
logger.exception("Depth recomputation failed")
|
logger.exception("Depth recomputation failed")
|
||||||
self.last_error = f"{type(exc).__name__}: {exc}"
|
self.last_error = f"{type(exc).__name__}: {exc}"
|
||||||
return DepthResult(
|
return DepthResult(
|
||||||
error=self.last_error, frame_id=frame.frame_id, t_rel_s=frame.t_rel_s
|
error=self.last_error,
|
||||||
|
frame_id=frame.frame_id,
|
||||||
|
t_rel_s=frame.t_rel_s,
|
||||||
|
warnings=plan.warnings,
|
||||||
|
record_id=self.record_id(),
|
||||||
)
|
)
|
||||||
self.last_warnings = list(plan.warnings)
|
self.last_warnings = list(plan.warnings)
|
||||||
|
if self._rewound_without_anchor and "state_approximate" not in self.last_warnings:
|
||||||
|
# The plan looked recoverable, but the read path had to restart a file
|
||||||
|
# without a reset boundary to clear the tracker afterwards.
|
||||||
|
self.last_warnings.append("state_approximate_after_rewind")
|
||||||
|
warnings = tuple(dict.fromkeys(self.last_warnings))
|
||||||
if depth is None:
|
if depth is None:
|
||||||
self.last_error = "depth_not_available"
|
self.last_error = "depth_not_available"
|
||||||
return DepthResult(
|
return DepthResult(
|
||||||
error=self.last_error,
|
error=self.last_error,
|
||||||
frame_id=frame.frame_id,
|
frame_id=frame.frame_id,
|
||||||
t_rel_s=frame.t_rel_s,
|
t_rel_s=frame.t_rel_s,
|
||||||
warnings=plan.warnings,
|
warnings=warnings,
|
||||||
|
record_id=self.record_id(),
|
||||||
)
|
)
|
||||||
self.last_error = ""
|
self.last_error = ""
|
||||||
return DepthResult(
|
return DepthResult(
|
||||||
depth=depth,
|
depth=depth,
|
||||||
frame_id=frame.frame_id,
|
frame_id=frame.frame_id,
|
||||||
t_rel_s=frame.t_rel_s,
|
t_rel_s=frame.t_rel_s,
|
||||||
warnings=plan.warnings,
|
warnings=warnings,
|
||||||
|
record_id=self.record_id(),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def record_id(self) -> str:
|
||||||
|
"""Which recording these answers belong to."""
|
||||||
|
return str(getattr(self.reader, "directory", "") or "")
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ playback
|
||||||
def _replay(self, plan: ReplayPlan) -> Optional[np.ndarray]:
|
def _replay(self, plan: ReplayPlan) -> Optional[np.ndarray]:
|
||||||
"""Read the pre-roll (if any), then step to the planned frame."""
|
"""Execute the plan's steps, keeping the processor and swapping inputs.
|
||||||
|
|
||||||
|
The **processor is kept** across a source change: the SDK's tracker lives
|
||||||
|
on the object, so disposing of it would dispose of the state the pre-roll
|
||||||
|
(or the reset boundary) just rebuilt. A reset boundary is re-applied by
|
||||||
|
calling ``reset()`` where the operator's ``zeroing`` happened.
|
||||||
|
"""
|
||||||
|
self._rewound_without_anchor = False
|
||||||
if plan.preroll is not None and plan.preroll_frames > 0:
|
if plan.preroll is not None and plan.preroll_frames > 0:
|
||||||
self._feed(plan.preroll, int(plan.preroll_frames))
|
self._feed(plan.preroll, int(plan.preroll_frames))
|
||||||
if self._segment != plan.segment or self._position > plan.stop_at:
|
depth: Optional[np.ndarray] = None
|
||||||
self._reopen_segment(plan.segment)
|
for step in plan.steps:
|
||||||
return self._advance_to(int(plan.stop_at))
|
depth = self._run_step(plan, step)
|
||||||
|
return depth
|
||||||
|
|
||||||
|
def _run_step(self, plan: ReplayPlan, step: ReplayStep) -> Optional[np.ndarray]:
|
||||||
|
target = max(1, int(step.to_frame))
|
||||||
|
if self._sensor is None or self._segment != step.source:
|
||||||
|
if self._sensor is None or not self._swap_source(step.source):
|
||||||
|
self._open_segment(step.source)
|
||||||
|
elif self._position > target:
|
||||||
|
# Re-reading an earlier frame can only be done from the start of the
|
||||||
|
# file. Without a reset boundary to clean up after it, the tracker has
|
||||||
|
# now seen frames it should not have.
|
||||||
|
if not self._swap_source(step.source):
|
||||||
|
self._open_segment(step.source)
|
||||||
|
if plan.anchor != "reset":
|
||||||
|
self._rewound_without_anchor = True
|
||||||
|
|
||||||
|
last: Optional[np.ndarray] = (
|
||||||
|
self._last_depth if self._position == target else None
|
||||||
|
)
|
||||||
|
while self._position < target:
|
||||||
|
depth = self._read_one()
|
||||||
|
if depth is None:
|
||||||
|
break
|
||||||
|
last = depth
|
||||||
|
if step.reset_after is not None and self._position == int(step.reset_after):
|
||||||
|
# The operator's zeroing: the live run cleared the tracker here.
|
||||||
|
if not self._reset_processor():
|
||||||
|
self.last_warnings.append("reset_not_applied")
|
||||||
|
return last
|
||||||
|
|
||||||
|
def _reset_processor(self) -> bool:
|
||||||
|
replay = self._sensor
|
||||||
|
reset = getattr(replay, "reset", None)
|
||||||
|
if not callable(reset):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(reset())
|
||||||
|
except Exception: # noqa: BLE001 - reported as a replay warning
|
||||||
|
logger.exception("Could not re-apply the reset boundary")
|
||||||
|
return False
|
||||||
|
|
||||||
def _depth_from_segment(
|
def _depth_from_segment(
|
||||||
self, segment: Path, segment_frame: int
|
self, segment: Path, segment_frame: int
|
||||||
@@ -253,28 +431,55 @@ class DepthRecomputer:
|
|||||||
|
|
||||||
The SDK's video source only advances, so stepping costs one read per
|
The SDK's video source only advances, so stepping costs one read per
|
||||||
frame — and the position is tracked, so asking for frame 2 and then
|
frame — and the position is tracked, so asking for frame 2 and then
|
||||||
frame 3 reads two frames and then one, instead of two more.
|
frame 3 reads two frames and then one, instead of two more. Asking twice
|
||||||
|
for the *same* frame is answered from the cached depth of that frame.
|
||||||
"""
|
"""
|
||||||
target = max(1, int(segment_frame))
|
target = max(1, int(segment_frame))
|
||||||
if self._segment != segment or self._position > target:
|
if self._segment != segment:
|
||||||
self._reopen_segment(segment)
|
self._open_segment(segment)
|
||||||
return self._advance_to(target)
|
return self._run_step(
|
||||||
|
ReplayPlan(segment=Path(segment), stop_at=target),
|
||||||
|
ReplayStep(source=Path(segment), to_frame=target),
|
||||||
|
)
|
||||||
|
|
||||||
# --------------------------------------------------------------------- video
|
# --------------------------------------------------------------------- video
|
||||||
def sensor_type(self) -> str:
|
def sensor_type(self) -> str:
|
||||||
device = self.reader.session().get("device") or {}
|
device = self.reader.session().get("device") or {}
|
||||||
return str(device.get("sensor_type") or "box")
|
return str(device.get("sensor_type") or "box")
|
||||||
|
|
||||||
def _reopen_segment(self, segment: Path) -> None:
|
def _open_segment(self, segment: Path) -> None:
|
||||||
|
"""Open a segment on a **new** processor (the previous one is closed)."""
|
||||||
self._close()
|
self._close()
|
||||||
if self.sensor_type() == "finger":
|
self._sensor = open_replay(
|
||||||
self._sensor = _FingerReplay(
|
self.sensor_type(),
|
||||||
segment, self.resolved_config(), self._finger_export_dir()
|
Path(segment),
|
||||||
|
self.resolved_config(),
|
||||||
|
self._finger_export_dir(),
|
||||||
)
|
)
|
||||||
else:
|
|
||||||
self._sensor = _BoxReplay(segment, self.resolved_config())
|
|
||||||
self._segment = Path(segment)
|
self._segment = Path(segment)
|
||||||
self._position = 0
|
self._position = 0
|
||||||
|
self._last_depth = None
|
||||||
|
|
||||||
|
def _swap_source(self, segment: Path) -> bool:
|
||||||
|
"""Replace only the video source of the open processor.
|
||||||
|
|
||||||
|
Returns False when the backend cannot do it, in which case the caller
|
||||||
|
falls back to a full reopen (and the algorithm state is lost, which the
|
||||||
|
plan already reports as approximate).
|
||||||
|
"""
|
||||||
|
replay = self._sensor
|
||||||
|
if replay is None:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
if not replay.set_source(Path(segment)):
|
||||||
|
return False
|
||||||
|
except Exception: # noqa: BLE001 - reported by the plan's warnings
|
||||||
|
logger.exception("Could not swap the replayed video source")
|
||||||
|
return False
|
||||||
|
self._segment = Path(segment)
|
||||||
|
self._position = 0
|
||||||
|
self._last_depth = None
|
||||||
|
return True
|
||||||
|
|
||||||
def _finger_export_dir(self):
|
def _finger_export_dir(self):
|
||||||
"""The finger pipeline assets, resolved exactly as the live run did."""
|
"""The finger pipeline assets, resolved exactly as the live run did."""
|
||||||
@@ -292,29 +497,17 @@ class DepthRecomputer:
|
|||||||
"""Consume ``frames`` frames from another file to rebuild state."""
|
"""Consume ``frames`` frames from another file to rebuild state."""
|
||||||
if frames <= 0:
|
if frames <= 0:
|
||||||
return
|
return
|
||||||
if self._sensor is None or self._segment != segment:
|
if self._sensor is None:
|
||||||
self._reopen_segment(segment)
|
self._open_segment(segment)
|
||||||
|
elif self._segment != segment:
|
||||||
|
if not self._swap_source(segment):
|
||||||
|
self._open_segment(segment)
|
||||||
for _ in range(int(frames)):
|
for _ in range(int(frames)):
|
||||||
if self._read_one() is None:
|
if self._read_one() is None:
|
||||||
break
|
break
|
||||||
|
|
||||||
def _advance_to(self, target: int) -> Optional[np.ndarray]:
|
|
||||||
last_depth = None
|
|
||||||
while self._position < int(target):
|
|
||||||
depth = self._read_one()
|
|
||||||
if depth is None:
|
|
||||||
break
|
|
||||||
last_depth = depth
|
|
||||||
return last_depth
|
|
||||||
|
|
||||||
def _read_one(self) -> Optional[np.ndarray]:
|
def _read_one(self) -> Optional[np.ndarray]:
|
||||||
"""Read one frame through the **live** processing chain.
|
"""Read one frame through the **live** processing chain (see ``process_chain``)."""
|
||||||
|
|
||||||
``get_img``/``read_source`` alone is not the live pipeline: the SDK's
|
|
||||||
depth map is produced by ``compute_deformation`` + ``compute_contact``
|
|
||||||
(and, for the finger, the unrolling step), so skipping them would replay
|
|
||||||
a different algorithm than the one that produced the recording.
|
|
||||||
"""
|
|
||||||
replay = self._sensor
|
replay = self._sensor
|
||||||
if replay is None:
|
if replay is None:
|
||||||
return None
|
return None
|
||||||
@@ -322,12 +515,14 @@ class DepthRecomputer:
|
|||||||
if depth is None:
|
if depth is None:
|
||||||
return None
|
return None
|
||||||
self._position += 1
|
self._position += 1
|
||||||
|
self._last_depth = depth
|
||||||
return depth
|
return depth
|
||||||
|
|
||||||
def _close(self) -> None:
|
def _close(self) -> None:
|
||||||
replay, self._sensor = self._sensor, None
|
replay, self._sensor = self._sensor, None
|
||||||
self._segment = None
|
self._segment = None
|
||||||
self._position = 0
|
self._position = 0
|
||||||
|
self._last_depth = None
|
||||||
if replay is None:
|
if replay is None:
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
@@ -343,97 +538,3 @@ class DepthRecomputer:
|
|||||||
|
|
||||||
def __exit__(self, *_exc) -> None:
|
def __exit__(self, *_exc) -> None:
|
||||||
self.close()
|
self.close()
|
||||||
|
|
||||||
|
|
||||||
class _BoxReplay:
|
|
||||||
"""The box sensor's live chain, opened on one video file."""
|
|
||||||
|
|
||||||
def __init__(self, segment: Path, config_ref: str) -> None:
|
|
||||||
import orisys
|
|
||||||
|
|
||||||
self._sensor = orisys.Sensor(
|
|
||||||
str(segment),
|
|
||||||
config_name=config_ref,
|
|
||||||
verbose=False,
|
|
||||||
backend="auto",
|
|
||||||
)
|
|
||||||
|
|
||||||
def read(self) -> Optional[np.ndarray]:
|
|
||||||
image = self._sensor.get_img()
|
|
||||||
if image is None:
|
|
||||||
return None
|
|
||||||
compute_deformation = getattr(self._sensor, "compute_deformation", None)
|
|
||||||
compute_contact = getattr(self._sensor, "compute_contact", None)
|
|
||||||
if callable(compute_deformation) and callable(compute_contact):
|
|
||||||
# Same calls and arguments as the acquisition loop.
|
|
||||||
compute_deformation(decompose=True, check_motion=False)
|
|
||||||
compute_contact()
|
|
||||||
else:
|
|
||||||
logger.warning(
|
|
||||||
"Sensor %r has no processing chain; replay depth is not the live algorithm",
|
|
||||||
type(self._sensor).__name__,
|
|
||||||
)
|
|
||||||
info = self._sensor.info
|
|
||||||
if not hasattr(info, "DEPTH"):
|
|
||||||
return None
|
|
||||||
value = self._sensor.read_info(info.DEPTH)
|
|
||||||
return None if value is None else np.asarray(value)
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
self._sensor.disconnect()
|
|
||||||
|
|
||||||
|
|
||||||
class _FingerReplay:
|
|
||||||
"""The finger's live chain: unroll the source frame, then process the tactile."""
|
|
||||||
|
|
||||||
def __init__(self, segment: Path, config_ref: str, export_dir) -> None:
|
|
||||||
from orisys.finger import FingerRuntime
|
|
||||||
|
|
||||||
from orisys_app_shared.acquisition.constants import (
|
|
||||||
FORCE_SAMPLE_RATE_HZ,
|
|
||||||
LOWPASS_CUTOFF_HZ,
|
|
||||||
)
|
|
||||||
|
|
||||||
kwargs: dict = {
|
|
||||||
"config": config_ref,
|
|
||||||
"video": str(segment),
|
|
||||||
"verbose": False,
|
|
||||||
}
|
|
||||||
if export_dir is not None:
|
|
||||||
kwargs["export_dir"] = str(export_dir)
|
|
||||||
self._runtime = FingerRuntime.open(**kwargs)
|
|
||||||
self._runtime.warmup(color=False)
|
|
||||||
tactile = getattr(self._runtime, "tactile", None)
|
|
||||||
if tactile is not None and hasattr(tactile, "configure"):
|
|
||||||
try:
|
|
||||||
# The same filter settings the acquisition loop applies.
|
|
||||||
tactile.configure(
|
|
||||||
lowpass_cutoff_hz=LOWPASS_CUTOFF_HZ,
|
|
||||||
lowpass_sample_rate_hz=FORCE_SAMPLE_RATE_HZ,
|
|
||||||
)
|
|
||||||
except Exception: # noqa: BLE001 - the defaults still work
|
|
||||||
logger.exception("Could not configure the replayed tactile sensor")
|
|
||||||
|
|
||||||
def read(self) -> Optional[np.ndarray]:
|
|
||||||
ok, source_frame = self._runtime.read_source()
|
|
||||||
if not ok or source_frame is None:
|
|
||||||
return None
|
|
||||||
unrolled = self._runtime.process_frame(source_frame, color=False)
|
|
||||||
if unrolled is None:
|
|
||||||
return None
|
|
||||||
tactile = self._runtime.tactile
|
|
||||||
compute_deformation = getattr(tactile, "compute_deformation", None)
|
|
||||||
compute_contact = getattr(tactile, "compute_contact", None)
|
|
||||||
if callable(compute_deformation) and callable(compute_contact):
|
|
||||||
compute_deformation(decompose=True, check_motion=False)
|
|
||||||
compute_contact()
|
|
||||||
else:
|
|
||||||
logger.warning("Finger tactile sensor has no processing chain")
|
|
||||||
info = tactile.info
|
|
||||||
if not hasattr(info, "DEPTH"):
|
|
||||||
return None
|
|
||||||
value = tactile.read_info(info.DEPTH)
|
|
||||||
return None if value is None else np.asarray(value)
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
self._runtime.close()
|
|
||||||
|
|||||||
@@ -0,0 +1,194 @@
|
|||||||
|
"""Replay backends: the live processing chain, pointed at a video file.
|
||||||
|
|
||||||
|
Each backend keeps **one** processor for its lifetime. The tracker state lives on
|
||||||
|
that object, so a replay must be able to change the *input source* without
|
||||||
|
throwing the processor away — that is what makes a pre-roll, or a zeroing that
|
||||||
|
happened in an earlier segment, reproducible at all.
|
||||||
|
|
||||||
|
The processing itself is not duplicated here: the reads go through
|
||||||
|
``acquisition.sdk_worker.process_chain``, the same function the live workers run.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class BoxReplay:
|
||||||
|
"""The box sensor's live chain, opened on one video file."""
|
||||||
|
|
||||||
|
def __init__(self, segment: Path, config_ref: str) -> None:
|
||||||
|
import orisys
|
||||||
|
|
||||||
|
self._config_ref = config_ref
|
||||||
|
self._sensor = orisys.Sensor(
|
||||||
|
str(segment),
|
||||||
|
config_name=config_ref,
|
||||||
|
verbose=False,
|
||||||
|
backend="auto",
|
||||||
|
)
|
||||||
|
|
||||||
|
def set_source(self, segment: Path) -> bool:
|
||||||
|
"""Point the same processor at another file.
|
||||||
|
|
||||||
|
The tracker lives on the ``Sensor`` object, so re-opening the sensor would
|
||||||
|
throw away the state the pre-roll just rebuilt. Only the capture is
|
||||||
|
replaced, and the tracker's reference frame is preserved.
|
||||||
|
"""
|
||||||
|
sensor = self._sensor
|
||||||
|
open_source = getattr(sensor, "_open_video_source", None)
|
||||||
|
if not callable(open_source):
|
||||||
|
return False
|
||||||
|
initial = getattr(sensor, "initial_img", None)
|
||||||
|
try:
|
||||||
|
import orisys
|
||||||
|
|
||||||
|
sensor.vid_src = orisys.capture.normalize_source(str(segment))
|
||||||
|
except Exception: # noqa: BLE001 - fall back to assigning the path
|
||||||
|
sensor.vid_src = str(segment)
|
||||||
|
sensor._is_live_camera = False
|
||||||
|
open_source(False, None)
|
||||||
|
if initial is not None:
|
||||||
|
# The tracker's reference frame belongs to the session, not to the
|
||||||
|
# file: replacing it would silently reset what "no contact" means.
|
||||||
|
sensor.initial_img = initial
|
||||||
|
return True
|
||||||
|
|
||||||
|
def read(self) -> Optional[np.ndarray]:
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import process_chain
|
||||||
|
|
||||||
|
image = self._sensor.get_img()
|
||||||
|
if image is None:
|
||||||
|
return None
|
||||||
|
# The same function the acquisition loop runs.
|
||||||
|
depth, error = process_chain(self._sensor)
|
||||||
|
if error:
|
||||||
|
logger.warning("Replay processing chain reported: %s", error)
|
||||||
|
return depth
|
||||||
|
|
||||||
|
def reset(self) -> bool:
|
||||||
|
"""Clear the tracker, exactly as the operator's zeroing did live."""
|
||||||
|
reset = getattr(self._sensor, "reset", None)
|
||||||
|
if not callable(reset):
|
||||||
|
return False
|
||||||
|
reset()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._sensor.disconnect()
|
||||||
|
|
||||||
|
|
||||||
|
class FingerReplay:
|
||||||
|
"""The finger's live chain: unroll the source frame, then process the tactile."""
|
||||||
|
|
||||||
|
def __init__(self, segment: Path, config_ref: str, export_dir) -> None:
|
||||||
|
from orisys.finger import FingerRuntime
|
||||||
|
|
||||||
|
from orisys_app_shared.acquisition.constants import (
|
||||||
|
FORCE_SAMPLE_RATE_HZ,
|
||||||
|
LOWPASS_CUTOFF_HZ,
|
||||||
|
)
|
||||||
|
|
||||||
|
self._config_ref = config_ref
|
||||||
|
self._export_dir = export_dir
|
||||||
|
kwargs: dict = {
|
||||||
|
"config": config_ref,
|
||||||
|
"video": str(segment),
|
||||||
|
"verbose": False,
|
||||||
|
}
|
||||||
|
if export_dir is not None:
|
||||||
|
kwargs["export_dir"] = str(export_dir)
|
||||||
|
self._runtime = FingerRuntime.open(**kwargs)
|
||||||
|
self._runtime.warmup(color=False)
|
||||||
|
tactile = getattr(self._runtime, "tactile", None)
|
||||||
|
if tactile is not None and hasattr(tactile, "configure"):
|
||||||
|
try:
|
||||||
|
# The same filter settings the acquisition loop applies.
|
||||||
|
tactile.configure(
|
||||||
|
lowpass_cutoff_hz=LOWPASS_CUTOFF_HZ,
|
||||||
|
lowpass_sample_rate_hz=FORCE_SAMPLE_RATE_HZ,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001 - the defaults still work
|
||||||
|
logger.exception("Could not configure the replayed tactile sensor")
|
||||||
|
|
||||||
|
def set_source(self, segment: Path) -> bool:
|
||||||
|
"""Point the same runtime at another video, keeping the tactile pipeline."""
|
||||||
|
try:
|
||||||
|
from orisys.capture import (
|
||||||
|
CaptureConfig,
|
||||||
|
ReconnectPolicy,
|
||||||
|
is_live_camera,
|
||||||
|
normalize_source,
|
||||||
|
open_capture,
|
||||||
|
)
|
||||||
|
|
||||||
|
runtime = self._runtime
|
||||||
|
video = normalize_source(str(segment))
|
||||||
|
camera = dict(getattr(runtime.geometry, "config", {}).get("camera", {}))
|
||||||
|
capture_config = CaptureConfig.from_camera_dict(
|
||||||
|
video,
|
||||||
|
camera,
|
||||||
|
reconnect=ReconnectPolicy(enabled=False),
|
||||||
|
verbose=False,
|
||||||
|
background=False,
|
||||||
|
)
|
||||||
|
capture_source = open_capture(capture_config, start_background=False)
|
||||||
|
if not capture_source.is_open():
|
||||||
|
return False
|
||||||
|
if is_live_camera(video):
|
||||||
|
capture_source.start_background()
|
||||||
|
previous = runtime.capture_source
|
||||||
|
runtime.capture_source = capture_source
|
||||||
|
runtime.capture = capture_source.cap
|
||||||
|
runtime.capture_worker = capture_source
|
||||||
|
if previous is not None:
|
||||||
|
try:
|
||||||
|
previous.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
logger.exception("Could not close the previous finger source")
|
||||||
|
return True
|
||||||
|
except Exception: # noqa: BLE001 - reported as an approximate replay
|
||||||
|
logger.exception("Could not swap the finger video source")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def read(self) -> Optional[np.ndarray]:
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import process_chain
|
||||||
|
|
||||||
|
ok, source_frame = self._runtime.read_source()
|
||||||
|
if not ok or source_frame is None:
|
||||||
|
return None
|
||||||
|
unrolled = self._runtime.process_frame(source_frame, color=False)
|
||||||
|
if unrolled is None:
|
||||||
|
return None
|
||||||
|
depth, error = process_chain(self._runtime.tactile)
|
||||||
|
if error:
|
||||||
|
logger.warning("Replay processing chain reported: %s", error)
|
||||||
|
return depth
|
||||||
|
|
||||||
|
def reset(self) -> bool:
|
||||||
|
"""Clear the tactile tracker, as the operator's zeroing did live."""
|
||||||
|
tactile = getattr(self._runtime, "tactile", None)
|
||||||
|
target = tactile if callable(getattr(tactile, "reset", None)) else None
|
||||||
|
if target is None:
|
||||||
|
sensor = getattr(tactile, "raw_sensor", None)
|
||||||
|
target = sensor if callable(getattr(sensor, "reset", None)) else None
|
||||||
|
if target is None:
|
||||||
|
return False
|
||||||
|
target.reset()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._runtime.close()
|
||||||
|
|
||||||
|
|
||||||
|
def open_replay(sensor_type: str, segment: Path, config_ref: str, export_dir=None):
|
||||||
|
"""The backend for a recorded sensor type."""
|
||||||
|
if str(sensor_type) == "finger":
|
||||||
|
return FingerReplay(segment, config_ref, export_dir)
|
||||||
|
return BoxReplay(segment, config_ref)
|
||||||
@@ -41,6 +41,7 @@ import cv2
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from orisys_app_shared.record.sample import RecordSample, to_record_sample
|
from orisys_app_shared.record.sample import RecordSample, to_record_sample
|
||||||
|
from orisys_app_shared.record.video_writer import VideoSegments
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -87,7 +88,8 @@ MAX_CONSECUTIVE_ERRORS = 5
|
|||||||
ALARM_EVERY_N_DROPS = 1000
|
ALARM_EVERY_N_DROPS = 1000
|
||||||
#: How many lost-row gaps may be remembered for the frame index.
|
#: How many lost-row gaps may be remembered for the frame index.
|
||||||
GAP_BACKLOG = 4096
|
GAP_BACKLOG = 4096
|
||||||
#: How many data rows may be parked while the item queue is full of video.
|
#: Hard cap on the ordered stream. Past this the producer has to give up a row,
|
||||||
|
#: and it says so by writing a gap marker instead.
|
||||||
ROW_BACKLOG = 8192
|
ROW_BACKLOG = 8192
|
||||||
|
|
||||||
|
|
||||||
@@ -115,6 +117,9 @@ class RecordConfig:
|
|||||||
fps: float = 30.0
|
fps: float = 30.0
|
||||||
codec: str = "mp4v"
|
codec: str = "mp4v"
|
||||||
segment_seconds: float = 300.0
|
segment_seconds: float = 300.0
|
||||||
|
#: Stream depth at which **video** starts being shed. Data rows are not
|
||||||
|
#: dropped by this limit: a row is tiny, and losing one because the encoder
|
||||||
|
#: fell behind is exactly the defect this bound must not reintroduce.
|
||||||
max_queue: int = 2048
|
max_queue: int = 2048
|
||||||
#: Byte budget for the video queue. The item count alone is not a bound: at
|
#: Byte budget for the video queue. The item count alone is not a bound: at
|
||||||
#: 1920x1080 BGR, 2048 queued frame copies would be ~11.9 GiB.
|
#: 1920x1080 BGR, 2048 queued frame copies would be ~11.9 GiB.
|
||||||
@@ -206,14 +211,16 @@ class ExperimentRecorder:
|
|||||||
|
|
||||||
def __init__(self, config: Optional[RecordConfig] = None) -> None:
|
def __init__(self, config: Optional[RecordConfig] = None) -> None:
|
||||||
self.config = config or RecordConfig()
|
self.config = config or RecordConfig()
|
||||||
self._queue: queue.Queue = queue.Queue(maxsize=max(16, self.config.max_queue))
|
#: One ordered stream for rows and events. It is **unbounded** on count:
|
||||||
#: Bounded side channel for frames whose data row was lost to a full
|
#: the producer checks ``ROW_BACKLOG``/``max_queue`` itself, so a slow
|
||||||
#: queue. It is tiny, so it never grows with the recording.
|
#: encoder can no longer make the *queue* quietly delete a data row, and
|
||||||
|
#: rows are written in the order they were acquired.
|
||||||
|
self._queue: queue.Queue = queue.Queue()
|
||||||
|
#: Stream positions of frames whose data row was lost. Kept out of the
|
||||||
|
#: stream because there is no room for them there; flushed in order.
|
||||||
self._gaps: deque = deque(maxlen=GAP_BACKLOG)
|
self._gaps: deque = deque(maxlen=GAP_BACKLOG)
|
||||||
#: Rows that did not fit in the item queue because it was full of video.
|
|
||||||
#: They are tiny, so a small backlog covers any realistic stall.
|
|
||||||
self._row_backlog: deque = deque(maxlen=ROW_BACKLOG)
|
|
||||||
self._queued_bytes = 0
|
self._queued_bytes = 0
|
||||||
|
self._seq = 0
|
||||||
self._thread: Optional[threading.Thread] = None
|
self._thread: Optional[threading.Thread] = None
|
||||||
self._stop = threading.Event()
|
self._stop = threading.Event()
|
||||||
self._lock = threading.Lock()
|
self._lock = threading.Lock()
|
||||||
@@ -242,6 +249,8 @@ class ExperimentRecorder:
|
|||||||
self._preroll_segment: Optional[str] = None
|
self._preroll_segment: Optional[str] = None
|
||||||
#: Problems the writer found while closing, carried into the report.
|
#: Problems the writer found while closing, carried into the report.
|
||||||
self._writer_problems: list[str] = []
|
self._writer_problems: list[str] = []
|
||||||
|
#: Coefficient files stored inside the recording directory.
|
||||||
|
self._calibration_snapshots: list[dict] = []
|
||||||
|
|
||||||
# ------------------------------------------------------------------ state
|
# ------------------------------------------------------------------ state
|
||||||
@property
|
@property
|
||||||
@@ -318,7 +327,20 @@ class ExperimentRecorder:
|
|||||||
self._state = STATE_STARTING
|
self._state = STATE_STARTING
|
||||||
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||||
directory = Path(root) / f"{stamp}_session{session_id:04d}"
|
directory = Path(root) / f"{stamp}_session{session_id:04d}"
|
||||||
|
try:
|
||||||
directory.mkdir(parents=True, exist_ok=True)
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
except Exception as exc: # noqa: BLE001 - reported to the caller
|
||||||
|
# A directory that cannot be created is a *terminal* failure, not a
|
||||||
|
# stuck ``starting``: the operator can fix the path and press record
|
||||||
|
# again, and the recorder must be willing to try.
|
||||||
|
with self._lock:
|
||||||
|
self._stats.failed = True
|
||||||
|
self._stats.last_error = f"could not create {directory}: {exc}"
|
||||||
|
self._state = STATE_FAILED
|
||||||
|
self._accepting = False
|
||||||
|
self._directory = None
|
||||||
|
self._alarm(f"could not create the recording directory: {exc}")
|
||||||
|
raise
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._directory = directory
|
self._directory = directory
|
||||||
self._stats = RecordStats()
|
self._stats = RecordStats()
|
||||||
@@ -335,9 +357,10 @@ class ExperimentRecorder:
|
|||||||
self._preroll_frames = 0
|
self._preroll_frames = 0
|
||||||
self._preroll_segment = None
|
self._preroll_segment = None
|
||||||
self._writer_problems = []
|
self._writer_problems = []
|
||||||
|
self._calibration_snapshots = []
|
||||||
# Stale bookkeeping from a previous run must not leak into this one.
|
# Stale bookkeeping from a previous run must not leak into this one.
|
||||||
self._gaps.clear()
|
self._gaps.clear()
|
||||||
self._row_backlog.clear()
|
self._seq = 0
|
||||||
self._queued_bytes = 0
|
self._queued_bytes = 0
|
||||||
# Accept immediately: see ``_accepts_packets``.
|
# Accept immediately: see ``_accepts_packets``.
|
||||||
self._accepting = True
|
self._accepting = True
|
||||||
@@ -370,26 +393,26 @@ class ExperimentRecorder:
|
|||||||
self._state = STATE_STOPPING
|
self._state = STATE_STOPPING
|
||||||
self._accepting = False
|
self._accepting = False
|
||||||
self._stop.set()
|
self._stop.set()
|
||||||
try:
|
# The stop marker is an ordinary item in the stream: everything acquired
|
||||||
self._queue.put_nowait(("stop", None))
|
# before it is written before it, and the writer stops after it.
|
||||||
except queue.Full: # pragma: no cover - writer will drain and see the flag
|
self._enqueue(("stop", self._next_seq(), None))
|
||||||
pass
|
|
||||||
thread = self._thread
|
thread = self._thread
|
||||||
if wait and thread is not None and thread.is_alive():
|
if wait and thread is not None and thread.is_alive():
|
||||||
thread.join(timeout=timeout)
|
thread.join(timeout=timeout)
|
||||||
if thread is not None and thread.is_alive():
|
if thread is not None and thread.is_alive():
|
||||||
# Still writing: do not claim the experiment was recorded.
|
# Either we were asked not to wait, or the writer missed the deadline.
|
||||||
report = RecordReport(
|
# A finished recording is only ever reported by the thread that closed
|
||||||
|
# the files, so this is explicitly *not* a completion — and it is not
|
||||||
|
# cached as the report either, so a later stop() still sees the real
|
||||||
|
# one (``recorder.report`` stays None until then).
|
||||||
|
return RecordReport(
|
||||||
directory=self._directory,
|
directory=self._directory,
|
||||||
stats=self.stats(),
|
stats=self.stats(),
|
||||||
problems=[
|
problems=[
|
||||||
"the recording writer did not stop within "
|
"the recording writer has not finished closing the files yet"
|
||||||
f"{timeout:g}s; the files are not closed yet"
|
|
||||||
],
|
],
|
||||||
completed=False,
|
completed=False,
|
||||||
)
|
)
|
||||||
self._report = report
|
|
||||||
return report
|
|
||||||
self._thread = None
|
self._thread = None
|
||||||
return self._report or self._finalise()
|
return self._report or self._finalise()
|
||||||
|
|
||||||
@@ -398,13 +421,34 @@ class ExperimentRecorder:
|
|||||||
|
|
||||||
A session can outlive the coefficients it started with, so the version is
|
A session can outlive the coefficients it started with, so the version is
|
||||||
per packet, not per session: the acquisition thread stamps every packet
|
per packet, not per session: the acquisition thread stamps every packet
|
||||||
with the calibration that was loaded when it was acquired.
|
with the calibration that was loaded when it was acquired. An empty
|
||||||
|
version is meaningful — it says the device has no calibration loaded.
|
||||||
"""
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._calibration_version = str(version or "")
|
self._calibration_version = str(version or "")
|
||||||
if self.is_recording:
|
if self.is_recording:
|
||||||
self.log_event("calibration_version", self._calibration_version)
|
self.log_event("calibration_version", self._calibration_version)
|
||||||
|
|
||||||
|
def note_calibration_snapshot(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
version: str,
|
||||||
|
name: str,
|
||||||
|
frame_id: int = -1,
|
||||||
|
mono_t: Optional[float] = None,
|
||||||
|
) -> None:
|
||||||
|
"""Remember a coefficient file stored inside the recording directory."""
|
||||||
|
entry = {
|
||||||
|
"version": str(version or ""),
|
||||||
|
"file": str(name or ""),
|
||||||
|
"frame_id": int(frame_id),
|
||||||
|
"mono_t": None if mono_t is None else round(float(mono_t), 6),
|
||||||
|
"at": datetime.now(timezone.utc).isoformat(timespec="milliseconds"),
|
||||||
|
}
|
||||||
|
with self._lock:
|
||||||
|
self._calibration_snapshots.append(entry)
|
||||||
|
logger.info("Calibration snapshot stored: %s", name)
|
||||||
|
|
||||||
def log_event(
|
def log_event(
|
||||||
self,
|
self,
|
||||||
kind: str,
|
kind: str,
|
||||||
@@ -433,21 +477,29 @@ class ExperimentRecorder:
|
|||||||
"kind": str(kind),
|
"kind": str(kind),
|
||||||
"detail": str(detail or ""),
|
"detail": str(detail or ""),
|
||||||
}
|
}
|
||||||
try:
|
# Events ride the same ordered stream as the rows, so an operator action
|
||||||
self._queue.put_nowait(("event", event))
|
# cannot overtake the samples it was taken between.
|
||||||
except queue.Full:
|
if self._queue.qsize() >= ROW_BACKLOG:
|
||||||
|
# The stream is saturated: an event is the cheapest thing to give up,
|
||||||
|
# but the loss is counted rather than silent.
|
||||||
self._note_drop("event")
|
self._note_drop("event")
|
||||||
|
return
|
||||||
|
self._enqueue(("event", self._next_seq(), event))
|
||||||
|
|
||||||
# ----------------------------------------------------------- sink interface
|
# ----------------------------------------------------------- sink interface
|
||||||
def on_packet(self, packet) -> None:
|
def on_packet(self, packet) -> None:
|
||||||
"""Called on the acquisition thread: enqueue, never do I/O here.
|
"""Called on the acquisition thread: enqueue, never do I/O here.
|
||||||
|
|
||||||
A data row is small and a frame is large, so video is what gives way.
|
A data row is small and a frame is large, so video is what gives way —
|
||||||
When the byte budget is exhausted the **frame** is shed, and when even
|
and the two are decided **separately**:
|
||||||
the item queue is full the frame is shed too and the row is parked in a
|
|
||||||
small backlog. Losing the row because the video could not keep up is what
|
* the rows go into one ordered, unbounded stream (bounded only by
|
||||||
made a video overflow look like a video-only loss while CSV rows quietly
|
``ROW_BACKLOG``, at which point the loss is counted and marked);
|
||||||
disappeared.
|
* the frames are bounded by a byte budget, and a frame that would exceed
|
||||||
|
it is shed *before* it is queued, so nothing has to be rolled back.
|
||||||
|
|
||||||
|
The stream being ordered is what makes the files ordered: rows are written
|
||||||
|
in the order they were acquired, no matter how far behind the encoder is.
|
||||||
"""
|
"""
|
||||||
if not self._accepts_packets():
|
if not self._accepts_packets():
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -462,36 +514,53 @@ class ExperimentRecorder:
|
|||||||
with self._lock:
|
with self._lock:
|
||||||
self._last_t = sample.mono_t
|
self._last_t = sample.mono_t
|
||||||
|
|
||||||
|
# Depth is only consulted to decide the *video*: a row is never refused
|
||||||
|
# because the encoder is behind, only when the stream itself is full.
|
||||||
|
depth = self._queue.qsize()
|
||||||
|
if depth >= ROW_BACKLOG:
|
||||||
|
self._note_gap(sample)
|
||||||
|
return
|
||||||
|
|
||||||
frame = None
|
frame = None
|
||||||
if self.config.video or self.config.lossless_frames:
|
if self.config.video or self.config.lossless_frames:
|
||||||
candidate = self._capture_frame(packet)
|
candidate = self._capture_frame(packet)
|
||||||
size = 0 if candidate is None else int(candidate.nbytes)
|
if candidate is not None:
|
||||||
with self._lock:
|
size = int(candidate.nbytes)
|
||||||
affordable = (self._queued_bytes + size) <= max(
|
if depth >= max(16, int(self.config.max_queue)):
|
||||||
1024, int(self.config.max_queue_bytes)
|
self._shed_video("record queue overflow")
|
||||||
)
|
elif not self._affordable(size):
|
||||||
if candidate is None:
|
self._shed_video("video byte budget reached")
|
||||||
pass
|
else:
|
||||||
elif affordable:
|
|
||||||
frame = candidate
|
frame = candidate
|
||||||
|
# Reserved here and released exactly once, by the writer that
|
||||||
|
# consumes it (see ``_handle_frame``).
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._queued_bytes += size
|
self._queued_bytes += size
|
||||||
else:
|
self._enqueue(("frame", self._next_seq(), sample, frame))
|
||||||
self._shed_video("video byte budget reached")
|
|
||||||
try:
|
def _next_seq(self) -> int:
|
||||||
self._queue.put_nowait(("frame", (sample, frame)))
|
|
||||||
except queue.Full:
|
|
||||||
if frame is not None:
|
|
||||||
# The queue is full of video: the row is tiny and must survive.
|
|
||||||
frame = None
|
|
||||||
self._shed_video("record queue overflow")
|
|
||||||
with self._lock:
|
with self._lock:
|
||||||
room = len(self._row_backlog) < self._row_backlog.maxlen
|
self._seq += 1
|
||||||
if room:
|
return self._seq
|
||||||
self._row_backlog.append(sample)
|
|
||||||
if not room:
|
def _affordable(self, size: int) -> bool:
|
||||||
# Even the row backlog is full: the hole is at least marked.
|
with self._lock:
|
||||||
self._note_gap(sample)
|
return (self._queued_bytes + int(size)) <= max(
|
||||||
|
1024, int(self.config.max_queue_bytes)
|
||||||
|
)
|
||||||
|
|
||||||
|
def _enqueue(self, item) -> bool:
|
||||||
|
"""Append to the ordered stream. False when the stream is full.
|
||||||
|
|
||||||
|
The queue itself is unbounded — its depth is bounded by ``ROW_BACKLOG``
|
||||||
|
and checked by the callers — so a slow writer can never make the producer
|
||||||
|
drop data behind its back.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
self._queue.put_nowait(item)
|
||||||
|
return True
|
||||||
|
except queue.Full: # pragma: no cover - the queue is unbounded
|
||||||
|
return False
|
||||||
|
|
||||||
def _shed_video(self, reason: str) -> None:
|
def _shed_video(self, reason: str) -> None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -505,14 +574,26 @@ class ExperimentRecorder:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def _note_gap(self, sample) -> None:
|
def _note_gap(self, sample) -> None:
|
||||||
"""Remember a frame whose row was lost, so the replay can show the hole."""
|
"""Remember a frame whose row was lost, so the replay can show the hole.
|
||||||
|
|
||||||
|
The gap carries the stream position it happened at. That is what lets the
|
||||||
|
writer put the hole in the *right place* in ``frames.csv`` instead of
|
||||||
|
appending it after newer frames (which would make the index unsorted).
|
||||||
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._stats.dropped_data += 1
|
self._stats.dropped_data += 1
|
||||||
if len(self._gaps) >= self._gaps.maxlen:
|
if len(self._gaps) >= self._gaps.maxlen:
|
||||||
# The gap backlog is bounded, so a long overflow loses the marker.
|
# The gap backlog is bounded, so a long overflow loses the marker.
|
||||||
self._stats.unrecorded_gaps += 1
|
self._stats.unrecorded_gaps += 1
|
||||||
return
|
return
|
||||||
self._gaps.append((sample.frame_id, sample.mono_t, sample.wall_t))
|
self._gaps.append(
|
||||||
|
(
|
||||||
|
self._seq + 1,
|
||||||
|
sample.frame_id,
|
||||||
|
sample.mono_t,
|
||||||
|
sample.wall_t,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _capture_frame(packet):
|
def _capture_frame(packet):
|
||||||
@@ -574,6 +655,7 @@ class ExperimentRecorder:
|
|||||||
session_id=self._session_id,
|
session_id=self._session_id,
|
||||||
calibration_id=self._calibration_id,
|
calibration_id=self._calibration_id,
|
||||||
preroll=self._preroll,
|
preroll=self._preroll,
|
||||||
|
on_problem=self._record_failure,
|
||||||
)
|
)
|
||||||
# Written once the pre-roll file exists, so session.json never names
|
# Written once the pre-roll file exists, so session.json never names
|
||||||
# a file that was not created.
|
# a file that was not created.
|
||||||
@@ -583,26 +665,40 @@ class ExperimentRecorder:
|
|||||||
self._preroll = []
|
self._preroll = []
|
||||||
self._write_session_json(final=False)
|
self._write_session_json(final=False)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
|
# A stop *or a failure* that arrived while the files were opening
|
||||||
|
# must not be undone here: accepting packets again after ``stop``
|
||||||
|
# would make a finished recording look alive, and overwriting
|
||||||
|
# ``failed`` with ``recording`` would hide a dead writer. The
|
||||||
|
# stream is still drained — those rows were accepted.
|
||||||
|
if self._stop.is_set() or self._state in (
|
||||||
|
STATE_STOPPING,
|
||||||
|
STATE_FAILED,
|
||||||
|
):
|
||||||
|
self._accepting = False
|
||||||
|
else:
|
||||||
self._state = STATE_RECORDING
|
self._state = STATE_RECORDING
|
||||||
self._accepting = True
|
self._accepting = True
|
||||||
while True:
|
while True:
|
||||||
# Parked rows and gaps first: a stop request must not strand the
|
# Gaps first, but only those that happened *before* the item about
|
||||||
# data this session already accepted.
|
# to be written: the hole must appear in the frame index where it
|
||||||
self._drain_rows(writer)
|
# belongs, not after newer frames.
|
||||||
self._drain_gaps(writer)
|
|
||||||
if self._stop.is_set() and self._queue.empty():
|
if self._stop.is_set() and self._queue.empty():
|
||||||
|
self._flush_gaps(writer, None)
|
||||||
break
|
break
|
||||||
try:
|
try:
|
||||||
kind, payload = self._queue.get(timeout=0.2)
|
item = self._queue.get(timeout=0.2)
|
||||||
except queue.Empty:
|
except queue.Empty:
|
||||||
writer.maybe_flush()
|
writer.maybe_flush()
|
||||||
continue
|
continue
|
||||||
|
kind = item[0]
|
||||||
|
seq = item[1]
|
||||||
|
self._flush_gaps(writer, seq)
|
||||||
if kind == "stop":
|
if kind == "stop":
|
||||||
break
|
break
|
||||||
if kind == "event":
|
if kind == "event":
|
||||||
self._handle_event(writer, payload)
|
self._handle_event(writer, item[2])
|
||||||
continue
|
continue
|
||||||
sample, frame = payload
|
_kind, _seq, sample, frame = item
|
||||||
self._handle_frame(writer, sample, frame)
|
self._handle_frame(writer, sample, frame)
|
||||||
writer.maybe_flush()
|
writer.maybe_flush()
|
||||||
except Exception as exc: # noqa: BLE001 - reported through stats
|
except Exception as exc: # noqa: BLE001 - reported through stats
|
||||||
@@ -617,32 +713,31 @@ class ExperimentRecorder:
|
|||||||
self._accepting = False
|
self._accepting = False
|
||||||
problems: list[str] = []
|
problems: list[str] = []
|
||||||
if writer is not None:
|
if writer is not None:
|
||||||
# A packet accepted just before the stop flag may have left its
|
# A row or gap accepted just before the stop flag may still be in
|
||||||
# row or gap behind; write those before the file is closed.
|
# the stream; write those before the file is closed.
|
||||||
self._drain_rows(writer)
|
self._flush_gaps(writer, None)
|
||||||
self._drain_gaps(writer)
|
|
||||||
try:
|
try:
|
||||||
problems = writer.close()
|
problems = writer.close()
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
problems = [f"close failed: {type(exc).__name__}: {exc}"]
|
problems = [f"close failed: {type(exc).__name__}: {exc}"]
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._stats.write_errors += writer.write_errors
|
|
||||||
self._stats.segments = writer.segments
|
self._stats.segments = writer.segments
|
||||||
self._stats.video_frames = writer.video_frames
|
self._stats.video_frames = writer.video_frames
|
||||||
self._stats.bytes_written = writer.bytes_written
|
self._stats.bytes_written = writer.bytes_written
|
||||||
|
# Closing problems are failures of the same recording and are counted
|
||||||
|
# exactly once, through the same path as a failed row, flush or event.
|
||||||
|
for problem in problems:
|
||||||
|
self._record_failure(problem)
|
||||||
|
self._alarm(problem)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
if problems or failure:
|
if failure and not self._stats.last_error:
|
||||||
self._stats.failed = True
|
self._stats.last_error = failure
|
||||||
if not self._stats.last_error:
|
|
||||||
self._stats.last_error = problems[0] if problems else failure
|
|
||||||
# The writer's own findings (a flush that failed, a pre-roll that
|
# The writer's own findings (a flush that failed, a pre-roll that
|
||||||
# could not be stored) belong in the report: an operator reading
|
# could not be stored) belong in the report: an operator reading
|
||||||
# only the terminal report must not have to grep the log.
|
# only the terminal report must not have to grep the log.
|
||||||
self._writer_problems = list(problems)
|
self._writer_problems = list(problems)
|
||||||
failed = self._stats.failed
|
failed = self._stats.failed
|
||||||
self._state = STATE_FAILED if failed else STATE_STOPPED
|
self._state = STATE_FAILED if failed else STATE_STOPPED
|
||||||
for problem in problems:
|
|
||||||
self._alarm(problem)
|
|
||||||
# Only now, with the files closed, is the report trustworthy.
|
# Only now, with the files closed, is the report trustworthy.
|
||||||
self._report = self._finalise(completed=True)
|
self._report = self._finalise(completed=True)
|
||||||
|
|
||||||
@@ -657,6 +752,10 @@ class ExperimentRecorder:
|
|||||||
def _handle_frame(self, writer, sample, frame) -> None:
|
def _handle_frame(self, writer, sample, frame) -> None:
|
||||||
try:
|
try:
|
||||||
if frame is not None:
|
if frame is not None:
|
||||||
|
# The one release for the reservation taken in ``on_packet``.
|
||||||
|
# A frame is enqueued once and consumed once, so this cannot
|
||||||
|
# double-count; doing it *before* the write also means a failed
|
||||||
|
# write does not leak the budget for a frame that is gone.
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._queued_bytes = max(0, self._queued_bytes - int(frame.nbytes))
|
self._queued_bytes = max(0, self._queued_bytes - int(frame.nbytes))
|
||||||
writer.write_sample(sample, frame)
|
writer.write_sample(sample, frame)
|
||||||
@@ -668,22 +767,21 @@ class ExperimentRecorder:
|
|||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
self._record_error(exc)
|
self._record_error(exc)
|
||||||
|
|
||||||
def _drain_rows(self, writer) -> None:
|
def _flush_gaps(self, writer, upto_seq) -> None:
|
||||||
"""Write rows parked while the item queue was full of video."""
|
"""Write the frame-index rows for frames whose data row was lost.
|
||||||
while True:
|
|
||||||
with self._lock:
|
|
||||||
if not self._row_backlog:
|
|
||||||
return
|
|
||||||
sample = self._row_backlog.popleft()
|
|
||||||
self._handle_frame(writer, sample, None)
|
|
||||||
|
|
||||||
def _drain_gaps(self, writer) -> None:
|
Gaps are written in stream order and only up to ``upto_seq`` (``None``
|
||||||
"""Write the frame-index rows for frames whose data row was lost."""
|
flushes the rest), so ``frames.csv`` stays sorted by time even when the
|
||||||
|
loss happened while the writer was far behind.
|
||||||
|
"""
|
||||||
while True:
|
while True:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
if not self._gaps:
|
if not self._gaps:
|
||||||
return
|
return
|
||||||
frame_id, mono_t, wall_t = self._gaps.popleft()
|
seq, frame_id, mono_t, wall_t = self._gaps[0]
|
||||||
|
if upto_seq is not None and seq > upto_seq:
|
||||||
|
return
|
||||||
|
self._gaps.popleft()
|
||||||
try:
|
try:
|
||||||
writer.write_gap(frame_id, mono_t, wall_t)
|
writer.write_gap(frame_id, mono_t, wall_t)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -693,18 +791,32 @@ class ExperimentRecorder:
|
|||||||
return
|
return
|
||||||
|
|
||||||
def _record_error(self, exc: Exception) -> None:
|
def _record_error(self, exc: Exception) -> None:
|
||||||
|
self._record_failure(f"{type(exc).__name__}: {exc}")
|
||||||
|
|
||||||
|
def _record_failure(self, message: str, *, fatal: bool = False) -> None:
|
||||||
|
"""One failed write, flush, close or metadata update.
|
||||||
|
|
||||||
|
Every failure enters the state machine here, so the counters, the fatal
|
||||||
|
transition and the report cannot disagree about what happened. A writer
|
||||||
|
that kept its own count (a failed ``maybe_flush`` used to) left the
|
||||||
|
recording looking healthy until the very end.
|
||||||
|
|
||||||
|
``fatal`` marks a failure that ends the recording on its own: a recording
|
||||||
|
whose ``session.json`` could not be written is not a recording anyone can
|
||||||
|
identify afterwards.
|
||||||
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._stats.write_errors += 1
|
self._stats.write_errors += 1
|
||||||
self._stats.last_error = f"{type(exc).__name__}: {exc}"
|
self._stats.last_error = str(message)
|
||||||
self._consecutive_errors += 1
|
self._consecutive_errors += 1
|
||||||
fatal = self._consecutive_errors >= MAX_CONSECUTIVE_ERRORS
|
fatal = fatal or self._consecutive_errors >= MAX_CONSECUTIVE_ERRORS
|
||||||
if fatal:
|
if fatal:
|
||||||
self._stats.failed = True
|
self._stats.failed = True
|
||||||
# Stop accepting immediately: consuming packets that cannot be
|
# Stop accepting immediately: consuming packets that cannot be
|
||||||
# written only hides the loss behind a "recording" state.
|
# written only hides the loss behind a "recording" state.
|
||||||
self._accepting = False
|
self._accepting = False
|
||||||
self._state = STATE_FAILED
|
self._state = STATE_FAILED
|
||||||
self._alarm(f"recording write failed: {exc}")
|
self._alarm(f"recording write failed: {message}")
|
||||||
if fatal:
|
if fatal:
|
||||||
self._stop.set()
|
self._stop.set()
|
||||||
|
|
||||||
@@ -712,6 +824,10 @@ class ExperimentRecorder:
|
|||||||
with self._lock:
|
with self._lock:
|
||||||
if self._t0 is not None and self._last_t is not None:
|
if self._t0 is not None and self._last_t is not None:
|
||||||
self._stats.duration_s = round(float(self._last_t - self._t0), 6)
|
self._stats.duration_s = round(float(self._last_t - self._t0), 6)
|
||||||
|
# Metadata **before** the report is assembled: a session.json that could
|
||||||
|
# not be written is part of what happened, and used to be discovered only
|
||||||
|
# after the report had already been built from the older statistics.
|
||||||
|
self._write_session_json(final=completed)
|
||||||
stats = self.stats()
|
stats = self.stats()
|
||||||
problems: list[str] = []
|
problems: list[str] = []
|
||||||
if stats.dropped_data or stats.dropped_video:
|
if stats.dropped_data or stats.dropped_video:
|
||||||
@@ -734,8 +850,6 @@ class ExperimentRecorder:
|
|||||||
for problem in self._writer_problems:
|
for problem in self._writer_problems:
|
||||||
if problem not in problems:
|
if problem not in problems:
|
||||||
problems.append(problem)
|
problems.append(problem)
|
||||||
# ``ended_at`` is only written once the files are actually closed.
|
|
||||||
self._write_session_json(final=completed)
|
|
||||||
return RecordReport(
|
return RecordReport(
|
||||||
directory=self._directory,
|
directory=self._directory,
|
||||||
stats=stats,
|
stats=stats,
|
||||||
@@ -763,10 +877,11 @@ class ExperimentRecorder:
|
|||||||
)
|
)
|
||||||
temporary.replace(target)
|
temporary.replace(target)
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
with self._lock:
|
# The same single failure path as a write: it is counted, it ends the
|
||||||
self._stats.write_errors += 1
|
# recording, and it reaches the report.
|
||||||
self._stats.last_error = f"session.json: {type(exc).__name__}: {exc}"
|
self._record_failure(
|
||||||
self._alarm(f"could not write session.json: {exc}")
|
f"session.json: {type(exc).__name__}: {exc}", fatal=True
|
||||||
|
)
|
||||||
|
|
||||||
def _build_metadata(self, *, final: bool) -> dict[str, Any]:
|
def _build_metadata(self, *, final: bool) -> dict[str, Any]:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -783,6 +898,9 @@ class ExperimentRecorder:
|
|||||||
"video": self.config.as_dict(),
|
"video": self.config.as_dict(),
|
||||||
"events": events,
|
"events": events,
|
||||||
"stats": stats,
|
"stats": stats,
|
||||||
|
# The coefficients the session actually ran with: the file itself is
|
||||||
|
# copied into the recording, so a re-export cannot rewrite history.
|
||||||
|
"calibration_snapshots": [dict(entry) for entry in self._calibration_snapshots],
|
||||||
# The replay needs these to know whether the algorithm state at the
|
# The replay needs these to know whether the algorithm state at the
|
||||||
# start of the recording can be rebuilt at all.
|
# start of the recording can be rebuilt at all.
|
||||||
"preroll_frames": int(self._preroll_frames),
|
"preroll_frames": int(self._preroll_frames),
|
||||||
@@ -802,16 +920,36 @@ class ExperimentRecorder:
|
|||||||
class _SessionWriter:
|
class _SessionWriter:
|
||||||
"""Owns the file handles. Only ever touched by the writer thread."""
|
"""Owns the file handles. Only ever touched by the writer thread."""
|
||||||
|
|
||||||
|
def _note_problem(self, message: str) -> None:
|
||||||
|
"""Record a failure here *and* hand it to the recorder at once."""
|
||||||
|
self.write_errors += 1
|
||||||
|
if len(self._problems) < 5:
|
||||||
|
self._problems.append(message)
|
||||||
|
if self._on_problem is not None:
|
||||||
|
try:
|
||||||
|
self._on_problem(message)
|
||||||
|
except Exception: # noqa: BLE001 - reporting must not break writing
|
||||||
|
logger.exception("Recording problem handler failed")
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self, *, directory, config, session_id, calibration_id, preroll=()
|
self,
|
||||||
|
*,
|
||||||
|
directory,
|
||||||
|
config,
|
||||||
|
session_id,
|
||||||
|
calibration_id,
|
||||||
|
preroll=(),
|
||||||
|
on_problem=None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self.directory = Path(directory)
|
self.directory = Path(directory)
|
||||||
self.config = config
|
self.config = config
|
||||||
self.session_id = session_id
|
self.session_id = session_id
|
||||||
self.calibration_id = calibration_id
|
self.calibration_id = calibration_id
|
||||||
|
#: ``on_problem(message)`` reports a failure to the recorder's state
|
||||||
|
#: machine *immediately*. A private counter here was invisible until the
|
||||||
|
#: files were closed, so a failing flush looked like a healthy recording.
|
||||||
|
self._on_problem = on_problem
|
||||||
self.write_errors = 0
|
self.write_errors = 0
|
||||||
self.segments = 0
|
|
||||||
self.video_frames = 0
|
|
||||||
self.png_frames = 0
|
self.png_frames = 0
|
||||||
self.gap_rows = 0
|
self.gap_rows = 0
|
||||||
self.bytes_written = 0
|
self.bytes_written = 0
|
||||||
@@ -820,12 +958,14 @@ class _SessionWriter:
|
|||||||
self.preroll_segment: Optional[str] = None
|
self.preroll_segment: Optional[str] = None
|
||||||
self._last_flush = time.perf_counter()
|
self._last_flush = time.perf_counter()
|
||||||
self._t0: Optional[float] = None
|
self._t0: Optional[float] = None
|
||||||
self._segment: Optional[cv2.VideoWriter] = None
|
|
||||||
self._segment_index = 0
|
|
||||||
self._segment_frames = 0
|
|
||||||
self._segment_started = 0.0
|
|
||||||
self._frame_shape: Optional[tuple[int, int]] = None
|
|
||||||
self._problems: list[str] = []
|
self._problems: list[str] = []
|
||||||
|
#: The video side, in its own component: one rolling segment at a time.
|
||||||
|
self._video = VideoSegments(
|
||||||
|
self.directory,
|
||||||
|
codec=self.config.codec,
|
||||||
|
fps=self.config.fps,
|
||||||
|
segment_seconds=self.config.segment_seconds,
|
||||||
|
)
|
||||||
|
|
||||||
self._forces = self._open_csv("forces.csv", FORCES_HEADER)
|
self._forces = self._open_csv("forces.csv", FORCES_HEADER)
|
||||||
self._frames = self._open_csv("frames.csv", FRAMES_HEADER)
|
self._frames = self._open_csv("frames.csv", FRAMES_HEADER)
|
||||||
@@ -863,22 +1003,8 @@ class _SessionWriter:
|
|||||||
kept.append(frame)
|
kept.append(frame)
|
||||||
used += size
|
used += size
|
||||||
kept.reverse()
|
kept.reverse()
|
||||||
self._frame_shape = (int(kept[0].shape[1]), int(kept[0].shape[0]))
|
self.preroll_frames = self._video.write_sequence(kept)
|
||||||
name = "preroll_0001.mp4"
|
self.preroll_segment = "preroll_0001.mp4" if self.preroll_frames else None
|
||||||
path = self.directory / name
|
|
||||||
fourcc = cv2.VideoWriter_fourcc(*self.config.codec)
|
|
||||||
writer = cv2.VideoWriter(
|
|
||||||
str(path), fourcc, max(1.0, float(self.config.fps)), self._frame_shape
|
|
||||||
)
|
|
||||||
if not writer.isOpened():
|
|
||||||
raise OSError(f"could not open video writer for {name}")
|
|
||||||
try:
|
|
||||||
for frame in kept:
|
|
||||||
writer.write(frame)
|
|
||||||
self.preroll_frames += 1
|
|
||||||
finally:
|
|
||||||
writer.release()
|
|
||||||
self.preroll_segment = name if self.preroll_frames else None
|
|
||||||
|
|
||||||
def _open_csv(self, name: str, header):
|
def _open_csv(self, name: str, header):
|
||||||
handle = (self.directory / name).open("w", newline="", encoding="utf-8")
|
handle = (self.directory / name).open("w", newline="", encoding="utf-8")
|
||||||
@@ -957,16 +1083,15 @@ class _SessionWriter:
|
|||||||
self.gap_rows += 1
|
self.gap_rows += 1
|
||||||
|
|
||||||
def _write_video_frame(self, frame) -> tuple[int, int]:
|
def _write_video_frame(self, frame) -> tuple[int, int]:
|
||||||
if self._frame_shape is None:
|
return self._video.write(frame)
|
||||||
self._frame_shape = (int(frame.shape[1]), int(frame.shape[0]))
|
|
||||||
if self._should_roll():
|
@property
|
||||||
self._close_segment()
|
def video_frames(self) -> int:
|
||||||
if self._segment is None:
|
return self._video.video_frames
|
||||||
self._open_segment()
|
|
||||||
self._segment.write(frame)
|
@property
|
||||||
self._segment_frames += 1
|
def segments(self) -> int:
|
||||||
self.video_frames += 1
|
return self._video.segments
|
||||||
return (self._segment_index, self._segment_frames)
|
|
||||||
|
|
||||||
def _write_lossless(self, frame) -> None:
|
def _write_lossless(self, frame) -> None:
|
||||||
self.png_frames += 1
|
self.png_frames += 1
|
||||||
@@ -989,33 +1114,6 @@ class _SessionWriter:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# ------------------------------------------------------------------ segments
|
|
||||||
def _should_roll(self) -> bool:
|
|
||||||
if self._segment is None:
|
|
||||||
return False
|
|
||||||
if self.config.segment_seconds <= 0:
|
|
||||||
return False
|
|
||||||
return (time.perf_counter() - self._segment_started) >= self.config.segment_seconds
|
|
||||||
|
|
||||||
def _open_segment(self) -> None:
|
|
||||||
self._segment_index += 1
|
|
||||||
path = self.directory / f"raw_{self._segment_index:04d}.mp4"
|
|
||||||
fourcc = cv2.VideoWriter_fourcc(*self.config.codec)
|
|
||||||
fps = max(1.0, float(self.config.fps))
|
|
||||||
writer = cv2.VideoWriter(str(path), fourcc, fps, self._frame_shape)
|
|
||||||
if not writer.isOpened():
|
|
||||||
raise OSError(f"could not open video writer for {path.name}")
|
|
||||||
self._segment = writer
|
|
||||||
self._segment_frames = 0
|
|
||||||
self._segment_started = time.perf_counter()
|
|
||||||
self.segments = self._segment_index
|
|
||||||
|
|
||||||
def _close_segment(self) -> None:
|
|
||||||
if self._segment is None:
|
|
||||||
return
|
|
||||||
self._segment.release()
|
|
||||||
self._segment = None
|
|
||||||
|
|
||||||
def maybe_flush(self) -> None:
|
def maybe_flush(self) -> None:
|
||||||
if (time.perf_counter() - self._last_flush) < self.config.flush_interval_s:
|
if (time.perf_counter() - self._last_flush) < self.config.flush_interval_s:
|
||||||
return
|
return
|
||||||
@@ -1024,23 +1122,22 @@ class _SessionWriter:
|
|||||||
try:
|
try:
|
||||||
handle.flush()
|
handle.flush()
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
self.write_errors += 1
|
self._note_problem(f"flush failed: {exc}")
|
||||||
if len(self._problems) < 5:
|
|
||||||
self._problems.append(f"flush failed: {exc}")
|
|
||||||
|
|
||||||
def close(self) -> list[str]:
|
def close(self) -> list[str]:
|
||||||
self._close_segment()
|
try:
|
||||||
|
self._video.close()
|
||||||
|
except Exception as exc: # noqa: BLE001 - the CSVs still have to close
|
||||||
|
self._note_problem(f"video close failed: {exc}")
|
||||||
for handle, _writer in (self._forces, self._frames, self._events):
|
for handle, _writer in (self._forces, self._frames, self._events):
|
||||||
try:
|
try:
|
||||||
handle.flush()
|
handle.flush()
|
||||||
handle.close()
|
handle.close()
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
self.write_errors += 1
|
self._note_problem(f"close failed: {exc}")
|
||||||
if len(self._problems) < 5:
|
|
||||||
self._problems.append(f"close failed: {exc}")
|
|
||||||
try:
|
try:
|
||||||
self.bytes_written = sum(
|
self.bytes_written = sum(
|
||||||
path.stat().st_size for path in self.directory.glob("*") if path.is_file()
|
path.stat().st_size for path in self.directory.rglob("*") if path.is_file()
|
||||||
)
|
)
|
||||||
except Exception: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
self.bytes_written = 0
|
self.bytes_written = 0
|
||||||
|
|||||||
@@ -0,0 +1,453 @@
|
|||||||
|
"""A time index over a recorded ``forces.csv``, so history is read on demand.
|
||||||
|
|
||||||
|
The replay used to load every row of a recording into Python lists and then into
|
||||||
|
arrays. That is fine for a minute and hopeless for the multi-hour recordings the
|
||||||
|
store is designed for, and it made the charts the only place where "the whole
|
||||||
|
session" had to exist in memory at once.
|
||||||
|
|
||||||
|
This index keeps the *small* things in memory — one time per row, one validity
|
||||||
|
flag, one frame id, plus periodic byte offsets — and reads the *values* only for
|
||||||
|
the interval being drawn. It also builds a multi-level overview (per-bucket
|
||||||
|
min/max/count) while streaming the file once, which is what a whole-session strip
|
||||||
|
needs and what it cannot get by sampling a few thousand rows.
|
||||||
|
|
||||||
|
Nothing here touches Qt, and nothing here trusts the file to be in order: a
|
||||||
|
legacy recording is sorted at load time (see ``playback.RecordingReader``).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import logging
|
||||||
|
from bisect import bisect_left, bisect_right
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Iterator, Optional
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#: How many rows to read between two recorded byte offsets. Larger means a
|
||||||
|
#: cheaper interval read but a longer scan to reach the interval.
|
||||||
|
CHECKPOINT_ROWS = 4096
|
||||||
|
#: Default number of overview buckets for the whole-session strip.
|
||||||
|
DEFAULT_BUCKETS = 1200
|
||||||
|
|
||||||
|
_FORCE_COLUMNS = ("FN_raw", "FX_raw", "FY_raw", "FN_N", "FX_N", "FY_N")
|
||||||
|
|
||||||
|
|
||||||
|
def _parse(text: str) -> float:
|
||||||
|
text = (text or "").strip()
|
||||||
|
if not text:
|
||||||
|
# An empty cell means "not available"; it must never become 0.
|
||||||
|
return float("nan")
|
||||||
|
try:
|
||||||
|
return float(text)
|
||||||
|
except ValueError:
|
||||||
|
return float("nan")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ForceWindow:
|
||||||
|
"""Rows inside one interval — the same shape a session slice has."""
|
||||||
|
|
||||||
|
times: np.ndarray
|
||||||
|
raw: np.ndarray
|
||||||
|
newtons: np.ndarray
|
||||||
|
valid: np.ndarray
|
||||||
|
frame_ids: np.ndarray
|
||||||
|
#: True when the rows were decimated to satisfy the caller's point budget.
|
||||||
|
decimated: bool = False
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return int(self.times.size)
|
||||||
|
|
||||||
|
def value_for_unit(self, unit: str) -> np.ndarray:
|
||||||
|
if str(unit).upper().startswith("N"):
|
||||||
|
return self.newtons
|
||||||
|
return self.raw
|
||||||
|
|
||||||
|
def plottable(self, unit: str):
|
||||||
|
"""``(times, values, mask)`` with a per-axis mask — the shared rule."""
|
||||||
|
from orisys_app_shared.record.sample import plottable_axis_mask
|
||||||
|
|
||||||
|
values = self.value_for_unit(unit)
|
||||||
|
return self.times, values, plottable_axis_mask(values, self.valid)
|
||||||
|
|
||||||
|
def plottable_axis(self, unit: str, index: int):
|
||||||
|
from orisys_app_shared.record.sample import plottable_axis_mask
|
||||||
|
|
||||||
|
values = self.value_for_unit(unit)
|
||||||
|
mask = plottable_axis_mask(values, self.valid)[:, index]
|
||||||
|
return self.times[mask], values[mask, index]
|
||||||
|
|
||||||
|
def statistics(self, unit: str) -> "WindowStats":
|
||||||
|
values = self.value_for_unit(unit)
|
||||||
|
mask = np.isfinite(values) & self.valid[:, None]
|
||||||
|
peaks: list[float] = []
|
||||||
|
means: list[float] = []
|
||||||
|
stds: list[float] = []
|
||||||
|
for index in range(3):
|
||||||
|
column = values[:, index][mask[:, index]]
|
||||||
|
if column.size == 0:
|
||||||
|
peaks.append(float("nan"))
|
||||||
|
means.append(float("nan"))
|
||||||
|
stds.append(float("nan"))
|
||||||
|
continue
|
||||||
|
peaks.append(float(np.nanmax(column)))
|
||||||
|
means.append(float(np.nanmean(column)))
|
||||||
|
stds.append(float(np.nanstd(column)))
|
||||||
|
span = 0.0
|
||||||
|
if self.times.size > 1:
|
||||||
|
span = float(self.times[-1] - self.times[0])
|
||||||
|
return WindowStats(
|
||||||
|
duration_s=span,
|
||||||
|
samples=int(mask.any(axis=1).sum()),
|
||||||
|
peaks=tuple(peaks),
|
||||||
|
means=tuple(means),
|
||||||
|
stds=tuple(stds),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WindowStats:
|
||||||
|
duration_s: float = 0.0
|
||||||
|
samples: int = 0
|
||||||
|
peaks: tuple[float, float, float] = (float("nan"),) * 3
|
||||||
|
means: tuple[float, float, float] = (float("nan"),) * 3
|
||||||
|
stds: tuple[float, float, float] = (float("nan"),) * 3
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Overview:
|
||||||
|
"""A whole-session envelope, cheap enough to keep in memory."""
|
||||||
|
|
||||||
|
times: np.ndarray
|
||||||
|
lows: np.ndarray # (buckets, 3)
|
||||||
|
highs: np.ndarray # (buckets, 3)
|
||||||
|
counts: np.ndarray # (buckets,)
|
||||||
|
unit: str = "raw"
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return int(self.times.size)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _Checkpoint:
|
||||||
|
row: int
|
||||||
|
time: float
|
||||||
|
offset: int
|
||||||
|
|
||||||
|
|
||||||
|
class RecordingIndex:
|
||||||
|
"""Index one recording directory's ``forces.csv`` (and its validity flags)."""
|
||||||
|
|
||||||
|
def __init__(self, directory: str | Path, *, checkpoint_rows: int = CHECKPOINT_ROWS):
|
||||||
|
self.directory = Path(directory)
|
||||||
|
self.checkpoint_rows = max(1, int(checkpoint_rows))
|
||||||
|
self.times = np.empty(0, dtype=np.float64)
|
||||||
|
self.valid = np.empty(0, dtype=bool)
|
||||||
|
self.frame_ids = np.empty(0, dtype=np.int64)
|
||||||
|
self._checkpoints: list[_Checkpoint] = []
|
||||||
|
self._rows = 0
|
||||||
|
self._byte_size = 0
|
||||||
|
self._built = False
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ building
|
||||||
|
@property
|
||||||
|
def path(self) -> Path:
|
||||||
|
return self.directory / "forces.csv"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def rows(self) -> int:
|
||||||
|
return self._rows
|
||||||
|
|
||||||
|
@property
|
||||||
|
def byte_size(self) -> int:
|
||||||
|
"""Size of the data file the index describes."""
|
||||||
|
return self._byte_size
|
||||||
|
|
||||||
|
def build(self) -> "RecordingIndex":
|
||||||
|
"""One streaming pass: times, flags, frame ids and byte offsets."""
|
||||||
|
path = self.path
|
||||||
|
times: list[float] = []
|
||||||
|
valid: list[bool] = []
|
||||||
|
frame_ids: list[int] = []
|
||||||
|
checkpoints: list[_Checkpoint] = []
|
||||||
|
if path.is_file():
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
header = handle.readline().decode("utf-8", "replace").strip()
|
||||||
|
columns = {name: index for index, name in enumerate(header.split(","))}
|
||||||
|
time_column = columns.get("t_rel_s", 2)
|
||||||
|
valid_column = columns.get("valid", 9)
|
||||||
|
frame_column = columns.get("frame_id", 1)
|
||||||
|
offset = handle.tell()
|
||||||
|
for row, line in enumerate(handle):
|
||||||
|
next_offset = handle.tell()
|
||||||
|
text = line.decode("utf-8", "replace").rstrip("\r\n")
|
||||||
|
if row % self.checkpoint_rows == 0:
|
||||||
|
checkpoints.append(
|
||||||
|
_Checkpoint(row=row, time=_time_of(text), offset=offset)
|
||||||
|
)
|
||||||
|
fields = next(csv.reader([text])) if text else []
|
||||||
|
if len(fields) > max(time_column, valid_column, frame_column):
|
||||||
|
try:
|
||||||
|
times.append(float(fields[time_column]))
|
||||||
|
valid.append(fields[valid_column].strip() == "1")
|
||||||
|
frame_ids.append(int(fields[frame_column]))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
times.append(float("nan"))
|
||||||
|
valid.append(False)
|
||||||
|
frame_ids.append(-1)
|
||||||
|
offset = next_offset
|
||||||
|
self.times = np.asarray(times, dtype=np.float64)
|
||||||
|
self.valid = np.asarray(valid, dtype=bool)
|
||||||
|
self.frame_ids = np.asarray(frame_ids, dtype=np.int64)
|
||||||
|
self._checkpoints = checkpoints
|
||||||
|
self._rows = int(self.times.size)
|
||||||
|
try:
|
||||||
|
self._byte_size = path.stat().st_size
|
||||||
|
except OSError:
|
||||||
|
self._byte_size = 0
|
||||||
|
self._built = True
|
||||||
|
return self
|
||||||
|
|
||||||
|
def ensure_built(self) -> "RecordingIndex":
|
||||||
|
if not self._built:
|
||||||
|
self.build()
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------- queries
|
||||||
|
def extent(self) -> tuple[float, float]:
|
||||||
|
if self._rows == 0:
|
||||||
|
return (0.0, 0.0)
|
||||||
|
return (float(self.times[0]), float(self.times[-1]))
|
||||||
|
|
||||||
|
def duration_s(self) -> float:
|
||||||
|
first, last = self.extent()
|
||||||
|
return max(0.0, last - first)
|
||||||
|
|
||||||
|
def row_range(self, t0: float, t1: float) -> tuple[int, int]:
|
||||||
|
"""Half-open row range covering ``[t0, t1]``."""
|
||||||
|
self.ensure_built()
|
||||||
|
if self._rows == 0:
|
||||||
|
return (0, 0)
|
||||||
|
if float(t1) < float(t0):
|
||||||
|
t0, t1 = t1, t0
|
||||||
|
start = max(0, bisect_left(self.times.tolist(), float(t0)) - 1)
|
||||||
|
stop = min(self._rows, bisect_right(self.times.tolist(), float(t1)))
|
||||||
|
return (start, max(start, stop))
|
||||||
|
|
||||||
|
def _checkpoint_before(self, row: int) -> _Checkpoint:
|
||||||
|
chosen = _Checkpoint(row=0, time=0.0, offset=0)
|
||||||
|
for checkpoint in self._checkpoints:
|
||||||
|
if checkpoint.row <= row:
|
||||||
|
chosen = checkpoint
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
return chosen
|
||||||
|
|
||||||
|
def read_rows(
|
||||||
|
self,
|
||||||
|
start: int,
|
||||||
|
stop: int,
|
||||||
|
*,
|
||||||
|
max_points: int | None = None,
|
||||||
|
unit: str = "raw",
|
||||||
|
) -> ForceWindow:
|
||||||
|
"""Read ``[start, stop)`` by seeking to the nearest checkpoint.
|
||||||
|
|
||||||
|
With ``max_points`` the rows are decimated **while streaming**: each
|
||||||
|
bucket contributes the extreme row of every axis in ``unit``, so a long
|
||||||
|
interval never has to exist in memory in full. The extremes are taken in
|
||||||
|
both the raw and the requested unit's axis, which is why a shear spike
|
||||||
|
survives a flat normal channel.
|
||||||
|
"""
|
||||||
|
self.ensure_built()
|
||||||
|
start = max(0, min(int(start), self._rows))
|
||||||
|
stop = max(start, min(int(stop), self._rows))
|
||||||
|
if stop <= start:
|
||||||
|
return _empty_window()
|
||||||
|
path = self.path
|
||||||
|
if not path.is_file():
|
||||||
|
return _empty_window()
|
||||||
|
|
||||||
|
newton_unit = str(unit).upper().startswith("N")
|
||||||
|
budget = int(max_points) if max_points else 0
|
||||||
|
total = stop - start
|
||||||
|
bucket = (
|
||||||
|
max(1, total // max(1, budget // 6))
|
||||||
|
if budget and total > budget * 2
|
||||||
|
else 1
|
||||||
|
)
|
||||||
|
|
||||||
|
checkpoint = self._checkpoint_before(start)
|
||||||
|
pending: list[tuple] = []
|
||||||
|
picked: list[tuple] = []
|
||||||
|
|
||||||
|
def flush() -> None:
|
||||||
|
if not pending:
|
||||||
|
return
|
||||||
|
if bucket == 1:
|
||||||
|
picked.extend(pending)
|
||||||
|
else:
|
||||||
|
values = np.asarray(
|
||||||
|
[row[3] if newton_unit else row[2] for row in pending],
|
||||||
|
dtype=np.float64,
|
||||||
|
)
|
||||||
|
for axis in range(3):
|
||||||
|
column = values[:, axis]
|
||||||
|
finite = np.isfinite(column)
|
||||||
|
if not finite.any():
|
||||||
|
continue
|
||||||
|
ordered = np.where(finite, column, -np.inf)
|
||||||
|
for index in {int(np.argmin(ordered)), int(np.argmax(ordered))}:
|
||||||
|
picked.append(pending[index])
|
||||||
|
pending.clear()
|
||||||
|
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
# Every checkpoint offset points at a *data* row: the build records
|
||||||
|
# ``tell()`` after the header, so there is nothing to skip here.
|
||||||
|
handle.seek(checkpoint.offset)
|
||||||
|
for row, line in enumerate(handle, start=checkpoint.row):
|
||||||
|
if row >= stop:
|
||||||
|
break
|
||||||
|
if row < start:
|
||||||
|
continue
|
||||||
|
fields = next(csv.reader([line.decode("utf-8", "replace")]))
|
||||||
|
if len(fields) < 11:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
t_rel = float(fields[2])
|
||||||
|
frame_id = int(fields[1])
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
values = [_parse(fields[index]) for index in (3, 4, 5, 6, 7, 8)]
|
||||||
|
pending.append(
|
||||||
|
(
|
||||||
|
frame_id,
|
||||||
|
t_rel,
|
||||||
|
tuple(values[:3]),
|
||||||
|
tuple(values[3:]),
|
||||||
|
fields[9].strip() == "1",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if len(pending) >= bucket:
|
||||||
|
flush()
|
||||||
|
flush()
|
||||||
|
|
||||||
|
if not picked:
|
||||||
|
return _empty_window()
|
||||||
|
# A row can be the extreme of several axes; keep one copy, in time order.
|
||||||
|
unique = {row[0]: row for row in picked}
|
||||||
|
rows = sorted(unique.values(), key=lambda row: row[1])
|
||||||
|
return ForceWindow(
|
||||||
|
times=np.asarray([row[1] for row in rows], dtype=np.float64),
|
||||||
|
raw=np.asarray([row[2] for row in rows], dtype=np.float64).reshape(-1, 3),
|
||||||
|
newtons=np.asarray([row[3] for row in rows], dtype=np.float64).reshape(-1, 3),
|
||||||
|
valid=np.asarray([row[4] for row in rows], dtype=bool),
|
||||||
|
frame_ids=np.asarray([row[0] for row in rows], dtype=np.int64),
|
||||||
|
decimated=bucket > 1,
|
||||||
|
)
|
||||||
|
|
||||||
|
def iter_rows(self, start: int = 0, stop: Optional[int] = None) -> Iterator[tuple]:
|
||||||
|
"""Yield ``(row, frame_id, t_rel_s, raw3, newtons3, valid)`` in file order."""
|
||||||
|
self.ensure_built()
|
||||||
|
path = self.path
|
||||||
|
if not path.is_file():
|
||||||
|
return
|
||||||
|
stop = self._rows if stop is None else min(int(stop), self._rows)
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
handle.readline() # header
|
||||||
|
for row, line in enumerate(handle):
|
||||||
|
if row >= stop:
|
||||||
|
return
|
||||||
|
if row < start:
|
||||||
|
continue
|
||||||
|
fields = next(csv.reader([line.decode("utf-8", "replace")]))
|
||||||
|
if len(fields) < 11:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
frame_id = int(fields[1])
|
||||||
|
t_rel = float(fields[2])
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
values = [_parse(fields[index]) for index in (3, 4, 5, 6, 7, 8)]
|
||||||
|
yield (
|
||||||
|
row,
|
||||||
|
frame_id,
|
||||||
|
t_rel,
|
||||||
|
tuple(values[:3]),
|
||||||
|
tuple(values[3:]),
|
||||||
|
fields[9].strip() == "1",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ overview
|
||||||
|
def overview(self, *, unit: str = "raw", buckets: int = DEFAULT_BUCKETS) -> Overview:
|
||||||
|
"""A per-bucket envelope of the whole recording, in the active unit.
|
||||||
|
|
||||||
|
Built by streaming the file once and folding each row into its bucket, so
|
||||||
|
a peak in a bucket the strip would otherwise skip is still visible.
|
||||||
|
"""
|
||||||
|
self.ensure_built()
|
||||||
|
buckets = max(1, int(buckets))
|
||||||
|
newton = str(unit).upper().startswith("N")
|
||||||
|
if self._rows == 0:
|
||||||
|
return Overview(
|
||||||
|
times=np.empty(0),
|
||||||
|
lows=np.empty((0, 3)),
|
||||||
|
highs=np.empty((0, 3)),
|
||||||
|
counts=np.empty(0, dtype=np.int64),
|
||||||
|
unit="N" if newton else "raw",
|
||||||
|
)
|
||||||
|
first, last = self.extent()
|
||||||
|
span = max(1e-9, last - first)
|
||||||
|
lows = np.full((buckets, 3), np.nan)
|
||||||
|
highs = np.full((buckets, 3), np.nan)
|
||||||
|
counts = np.zeros(buckets, dtype=np.int64)
|
||||||
|
starts = np.full(buckets, np.nan)
|
||||||
|
for _row, _frame_id, t_rel, raw, newtons, is_valid in self.iter_rows():
|
||||||
|
if not is_valid:
|
||||||
|
continue
|
||||||
|
bucket = int((t_rel - first) / span * (buckets - 1) + 0.5)
|
||||||
|
bucket = max(0, min(buckets - 1, bucket))
|
||||||
|
values = newtons if newton else raw
|
||||||
|
for axis in range(3):
|
||||||
|
value = values[axis]
|
||||||
|
if value != value: # NaN
|
||||||
|
continue
|
||||||
|
current_low = lows[bucket, axis]
|
||||||
|
current_high = highs[bucket, axis]
|
||||||
|
lows[bucket, axis] = value if current_low != current_low else min(current_low, value)
|
||||||
|
highs[bucket, axis] = value if current_high != current_high else max(current_high, value)
|
||||||
|
if starts[bucket] != starts[bucket]:
|
||||||
|
starts[bucket] = t_rel
|
||||||
|
counts[bucket] += 1
|
||||||
|
return Overview(
|
||||||
|
times=starts,
|
||||||
|
lows=lows,
|
||||||
|
highs=highs,
|
||||||
|
counts=counts,
|
||||||
|
unit="N" if newton else "raw",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _time_of(text: str) -> float:
|
||||||
|
fields = next(csv.reader([text])) if text else []
|
||||||
|
if len(fields) > 2:
|
||||||
|
try:
|
||||||
|
return float(fields[2])
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return float("nan")
|
||||||
|
return float("nan")
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_window() -> ForceWindow:
|
||||||
|
return ForceWindow(
|
||||||
|
times=np.empty(0, dtype=np.float64),
|
||||||
|
raw=np.empty((0, 3)),
|
||||||
|
newtons=np.empty((0, 3)),
|
||||||
|
valid=np.empty(0, dtype=bool),
|
||||||
|
frame_ids=np.empty(0, dtype=np.int64),
|
||||||
|
)
|
||||||
@@ -21,6 +21,15 @@ from typing import Iterator, Optional
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from orisys_app_shared.record.index import (
|
||||||
|
CHECKPOINT_ROWS,
|
||||||
|
DEFAULT_BUCKETS,
|
||||||
|
ForceWindow,
|
||||||
|
Overview,
|
||||||
|
RecordingIndex,
|
||||||
|
)
|
||||||
|
from orisys_app_shared.record.sample import plottable_axis_mask
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
#: Column names as written by the recorder (index == position in forces.csv).
|
#: Column names as written by the recorder (index == position in forces.csv).
|
||||||
@@ -65,9 +74,21 @@ class RecordingInfo:
|
|||||||
frame_ids: np.ndarray
|
frame_ids: np.ndarray
|
||||||
dropped_rows: int = 0
|
dropped_rows: int = 0
|
||||||
frames_dropped: int = 0
|
frames_dropped: int = 0
|
||||||
|
#: The index rows were in time order as written (False = a legacy record).
|
||||||
|
frames_out_of_order: bool = True
|
||||||
|
#: The force rows were in time order as written.
|
||||||
|
forces_out_of_order: bool = True
|
||||||
|
#: True when the arrays above are a decimated view of a long recording.
|
||||||
|
decimated: bool = False
|
||||||
|
#: Total rows in ``forces.csv`` (the decimated view has fewer).
|
||||||
|
total_rows: int = 0
|
||||||
|
#: The recording's real duration, from the frame index, never from the view.
|
||||||
|
total_duration_s: float = 0.0
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def duration_s(self) -> float:
|
def duration_s(self) -> float:
|
||||||
|
if self.total_duration_s:
|
||||||
|
return float(self.total_duration_s)
|
||||||
if self.times.size == 0:
|
if self.times.size == 0:
|
||||||
return 0.0
|
return 0.0
|
||||||
return float(self.times[-1])
|
return float(self.times[-1])
|
||||||
@@ -78,26 +99,51 @@ class RecordingInfo:
|
|||||||
return self.raw
|
return self.raw
|
||||||
|
|
||||||
def plottable(self, unit: str) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
def plottable(self, unit: str) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
||||||
"""``(times, values, mask)`` — the same validity rule the live charts use.
|
"""``(times, values, mask)`` — the same rule the live charts use.
|
||||||
|
|
||||||
The replay used to filter on finiteness alone, so a stale value left by a
|
The replay used to filter on finiteness alone, so a stale value left by a
|
||||||
failed compute step could reappear here after another view had already
|
failed compute step could reappear here after another view had already
|
||||||
excluded it.
|
excluded it. ``mask`` is per axis: a single-axis calibration is normal and
|
||||||
|
must not hide the axis that *is* calibrated.
|
||||||
"""
|
"""
|
||||||
values = self.value_for_unit(unit)
|
values = self.value_for_unit(unit)
|
||||||
mask = self.valid & np.isfinite(values).all(axis=1)
|
return self.times, values, plottable_axis_mask(values, self.valid)
|
||||||
return self.times[mask], values[mask], mask
|
|
||||||
|
def plottable_axis(self, unit: str, index: int) -> tuple[np.ndarray, np.ndarray]:
|
||||||
|
"""``(times, values)`` for one channel, with the other axes ignored."""
|
||||||
|
values = self.value_for_unit(unit)
|
||||||
|
mask = plottable_axis_mask(values, self.valid)[:, index]
|
||||||
|
return self.times[mask], values[mask, index]
|
||||||
|
|
||||||
|
|
||||||
class RecordingReader:
|
class RecordingReader:
|
||||||
"""Open one experiment directory and serve replay queries from it."""
|
"""Open one experiment directory and serve replay queries from it.
|
||||||
|
|
||||||
def __init__(self, directory: str | Path) -> None:
|
``max_rows`` bounds what :meth:`info` keeps in memory: a long recording is
|
||||||
|
loaded as a decimated view (peaks preserved) and marked ``decimated``, while
|
||||||
|
:meth:`window` and :meth:`overview` read the interval being drawn from disk
|
||||||
|
through :class:`~orisys_app_shared.record.index.RecordingIndex`. That is what
|
||||||
|
lets the replay show an hour-long recording without holding all of it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#: Above this many rows ``info()`` decimates instead of loading everything.
|
||||||
|
DEFAULT_MAX_ROWS = 200_000
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
directory: str | Path,
|
||||||
|
*,
|
||||||
|
max_rows: int | None = None,
|
||||||
|
checkpoint_rows: int = CHECKPOINT_ROWS,
|
||||||
|
) -> None:
|
||||||
self.directory = Path(directory)
|
self.directory = Path(directory)
|
||||||
if not self.directory.is_dir():
|
if not self.directory.is_dir():
|
||||||
raise FileNotFoundError(f"Not a recording directory: {self.directory}")
|
raise FileNotFoundError(f"Not a recording directory: {self.directory}")
|
||||||
self._info: Optional[RecordingInfo] = None
|
self._info: Optional[RecordingInfo] = None
|
||||||
self._events: Optional[list[dict]] = None
|
self._events: Optional[list[dict]] = None
|
||||||
|
self.max_rows = int(max_rows or self.DEFAULT_MAX_ROWS)
|
||||||
|
self._index = RecordingIndex(self.directory, checkpoint_rows=checkpoint_rows)
|
||||||
|
self._overviews: dict[tuple[str, int], Overview] = {}
|
||||||
|
|
||||||
# ------------------------------------------------------------------ loading
|
# ------------------------------------------------------------------ loading
|
||||||
def info(self) -> RecordingInfo:
|
def info(self) -> RecordingInfo:
|
||||||
@@ -105,6 +151,10 @@ class RecordingReader:
|
|||||||
self._info = self._load()
|
self._info = self._load()
|
||||||
return self._info
|
return self._info
|
||||||
|
|
||||||
|
def index(self) -> RecordingIndex:
|
||||||
|
"""The time index, built once and shared with the windows."""
|
||||||
|
return self._index.ensure_built()
|
||||||
|
|
||||||
def session(self) -> dict:
|
def session(self) -> dict:
|
||||||
path = self.directory / "session.json"
|
path = self.directory / "session.json"
|
||||||
if not path.is_file():
|
if not path.is_file():
|
||||||
@@ -116,8 +166,30 @@ class RecordingReader:
|
|||||||
return {}
|
return {}
|
||||||
|
|
||||||
def _load(self) -> RecordingInfo:
|
def _load(self) -> RecordingInfo:
|
||||||
frames = self._load_frames()
|
frames, frames_ordered = self._load_frames()
|
||||||
times, raw, newtons, valid, frame_ids, dropped = self._load_forces()
|
index = self.index()
|
||||||
|
total_rows = index.rows
|
||||||
|
decimated = total_rows > self.max_rows > 0
|
||||||
|
if decimated:
|
||||||
|
window = self.window_all(
|
||||||
|
max_points=self.max_rows
|
||||||
|
)
|
||||||
|
times = window.times
|
||||||
|
raw = window.raw
|
||||||
|
newtons = window.newtons
|
||||||
|
valid = window.valid
|
||||||
|
frame_ids = window.frame_ids
|
||||||
|
forces_ordered = True
|
||||||
|
dropped = 0
|
||||||
|
else:
|
||||||
|
(times, raw, newtons, valid, frame_ids, dropped, forces_ordered) = (
|
||||||
|
self._load_forces()
|
||||||
|
)
|
||||||
|
total_duration = 0.0
|
||||||
|
if frames:
|
||||||
|
total_duration = float(frames[-1].t_rel_s)
|
||||||
|
elif times.size:
|
||||||
|
total_duration = float(times[-1])
|
||||||
return RecordingInfo(
|
return RecordingInfo(
|
||||||
directory=self.directory,
|
directory=self.directory,
|
||||||
session=self.session(),
|
session=self.session(),
|
||||||
@@ -129,12 +201,26 @@ class RecordingReader:
|
|||||||
frame_ids=frame_ids,
|
frame_ids=frame_ids,
|
||||||
dropped_rows=dropped,
|
dropped_rows=dropped,
|
||||||
frames_dropped=sum(1 for frame in frames if frame.dropped),
|
frames_dropped=sum(1 for frame in frames if frame.dropped),
|
||||||
|
frames_out_of_order=frames_ordered,
|
||||||
|
forces_out_of_order=forces_ordered,
|
||||||
|
decimated=decimated,
|
||||||
|
total_rows=total_rows,
|
||||||
|
total_duration_s=total_duration,
|
||||||
)
|
)
|
||||||
|
|
||||||
def _load_frames(self) -> list[RecordingFrame]:
|
def _load_frames(self) -> tuple[list[RecordingFrame], bool]:
|
||||||
|
"""Load the frame index in time order.
|
||||||
|
|
||||||
|
Recordings made before the ordered-write fix (see
|
||||||
|
``ExperimentRecorder``) can contain a frame index whose rows are not in
|
||||||
|
time order, because the row backlog used to be drained before the queue.
|
||||||
|
``frame_at`` does a binary search over these times, so an unsorted index
|
||||||
|
would silently answer with the wrong frame. The rows are sorted here and
|
||||||
|
the fact is reported as a replay warning.
|
||||||
|
"""
|
||||||
path = self.directory / "frames.csv"
|
path = self.directory / "frames.csv"
|
||||||
if not path.is_file():
|
if not path.is_file():
|
||||||
return []
|
return [], False
|
||||||
rows: list[RecordingFrame] = []
|
rows: list[RecordingFrame] = []
|
||||||
with path.open(encoding="utf-8", newline="") as handle:
|
with path.open(encoding="utf-8", newline="") as handle:
|
||||||
for row in csv.DictReader(handle):
|
for row in csv.DictReader(handle):
|
||||||
@@ -152,13 +238,26 @@ class RecordingReader:
|
|||||||
)
|
)
|
||||||
except (KeyError, TypeError, ValueError):
|
except (KeyError, TypeError, ValueError):
|
||||||
continue
|
continue
|
||||||
return rows
|
ordered = _is_time_ordered([frame.t_rel_s for frame in rows])
|
||||||
|
if not ordered:
|
||||||
|
rows.sort(key=lambda frame: frame.t_rel_s)
|
||||||
|
return rows, ordered
|
||||||
|
|
||||||
def _load_forces(self):
|
def _load_forces(self):
|
||||||
path = self.directory / "forces.csv"
|
path = self.directory / "forces.csv"
|
||||||
if not path.is_file():
|
if not path.is_file():
|
||||||
|
# A directory with no data (an empty or half-written recording) is
|
||||||
|
# readable: it simply has no samples, and nothing is out of order.
|
||||||
empty = np.empty(0, dtype=np.float64)
|
empty = np.empty(0, dtype=np.float64)
|
||||||
return empty, np.empty((0, 3)), np.empty((0, 3)), np.empty(0, bool), np.empty(0, int), 0
|
return (
|
||||||
|
empty,
|
||||||
|
np.empty((0, 3)),
|
||||||
|
np.empty((0, 3)),
|
||||||
|
np.empty(0, bool),
|
||||||
|
np.empty(0, int),
|
||||||
|
0,
|
||||||
|
True,
|
||||||
|
)
|
||||||
|
|
||||||
def parse(text: str) -> float:
|
def parse(text: str) -> float:
|
||||||
text = (text or "").strip()
|
text = (text or "").strip()
|
||||||
@@ -191,23 +290,42 @@ class RecordingReader:
|
|||||||
(parse(row.get("FN_N")), parse(row.get("FX_N")), parse(row.get("FY_N")))
|
(parse(row.get("FN_N")), parse(row.get("FX_N")), parse(row.get("FY_N")))
|
||||||
)
|
)
|
||||||
valid.append((row.get("valid") or "0").strip() == "1")
|
valid.append((row.get("valid") or "0").strip() == "1")
|
||||||
return (
|
times_arr = np.asarray(times, dtype=np.float64)
|
||||||
np.asarray(times, dtype=np.float64),
|
raw_arr = np.asarray(raw, dtype=np.float64).reshape(-1, 3)
|
||||||
np.asarray(raw, dtype=np.float64).reshape(-1, 3),
|
newtons_arr = np.asarray(newtons, dtype=np.float64).reshape(-1, 3)
|
||||||
np.asarray(newtons, dtype=np.float64).reshape(-1, 3),
|
valid_arr = np.asarray(valid, dtype=bool)
|
||||||
np.asarray(valid, dtype=bool),
|
ids_arr = np.asarray(frame_ids, dtype=np.int64)
|
||||||
np.asarray(frame_ids, dtype=np.int64),
|
|
||||||
dropped,
|
# Older recordings can have rows out of time order (the writer used to
|
||||||
)
|
# drain a backlog before the queue). Every window query below assumes
|
||||||
|
# sorted time, so sort once here and report it instead of answering
|
||||||
|
# ``searchsorted`` on unsorted input.
|
||||||
|
ordered = _is_time_ordered(times)
|
||||||
|
if not ordered and times_arr.size:
|
||||||
|
order = np.argsort(times_arr, kind="stable")
|
||||||
|
times_arr = times_arr[order]
|
||||||
|
raw_arr = raw_arr[order]
|
||||||
|
newtons_arr = newtons_arr[order]
|
||||||
|
valid_arr = valid_arr[order]
|
||||||
|
ids_arr = ids_arr[order]
|
||||||
|
return times_arr, raw_arr, newtons_arr, valid_arr, ids_arr, dropped, ordered
|
||||||
|
|
||||||
# ------------------------------------------------------------------ queries
|
# ------------------------------------------------------------------ queries
|
||||||
def frame_at(self, t_rel_s: float) -> Optional[RecordingFrame]:
|
def frame_at(self, t_rel_s: float) -> Optional[RecordingFrame]:
|
||||||
"""The recorded frame at or just before ``t_rel_s``."""
|
"""The recorded frame at or just before ``t_rel_s``.
|
||||||
|
|
||||||
|
The index is sorted at load time, so the binary search is valid; the
|
||||||
|
fallback walk keeps a hand-edited or truncated file from answering with
|
||||||
|
the wrong frame.
|
||||||
|
"""
|
||||||
frames = self.info().frames
|
frames = self.info().frames
|
||||||
if not frames:
|
if not frames:
|
||||||
return None
|
return None
|
||||||
times = [frame.t_rel_s for frame in frames]
|
times = np.asarray([frame.t_rel_s for frame in frames], dtype=np.float64)
|
||||||
index = int(np.searchsorted(np.asarray(times), float(t_rel_s), side="right")) - 1
|
if not _is_time_ordered(times.tolist()):
|
||||||
|
candidates = [frame for frame in frames if frame.t_rel_s <= float(t_rel_s)]
|
||||||
|
return max(candidates, key=lambda frame: frame.t_rel_s) if candidates else frames[0]
|
||||||
|
index = int(np.searchsorted(times, float(t_rel_s), side="right")) - 1
|
||||||
index = max(0, min(index, len(frames) - 1))
|
index = max(0, min(index, len(frames) - 1))
|
||||||
return frames[index]
|
return frames[index]
|
||||||
|
|
||||||
@@ -318,6 +436,47 @@ class RecordingReader:
|
|||||||
mask = (info.times >= float(t0)) & (info.times <= float(t1))
|
mask = (info.times >= float(t0)) & (info.times <= float(t1))
|
||||||
return info.times[mask], info.raw[mask]
|
return info.times[mask], info.raw[mask]
|
||||||
|
|
||||||
|
# ------------------------------------------------------ on-demand history
|
||||||
|
def window(
|
||||||
|
self,
|
||||||
|
t0: float,
|
||||||
|
t1: float,
|
||||||
|
*,
|
||||||
|
unit: str = "raw",
|
||||||
|
max_points: int | None = None,
|
||||||
|
) -> ForceWindow:
|
||||||
|
"""The recorded rows inside ``[t0, t1]``, read from disk.
|
||||||
|
|
||||||
|
This is the interval read: it seeks to the nearest checkpoint in the
|
||||||
|
index and scans forward, so asking for ten seconds of an hour-long
|
||||||
|
recording costs ten seconds of parsing, not an hour. ``max_points``
|
||||||
|
decimates *while* reading, so the interval never has to exist in memory
|
||||||
|
in full.
|
||||||
|
"""
|
||||||
|
start, stop = self.index().row_range(t0, t1)
|
||||||
|
return self.index().read_rows(
|
||||||
|
start, stop, max_points=max_points, unit=unit
|
||||||
|
)
|
||||||
|
|
||||||
|
def window_all(self, *, unit: str = "raw", max_points: int | None = None) -> ForceWindow:
|
||||||
|
"""The whole recording as one window, decimated while streaming."""
|
||||||
|
return self.window(
|
||||||
|
*self.index().extent(), unit=unit, max_points=max_points
|
||||||
|
)
|
||||||
|
|
||||||
|
def overview(self, *, unit: str = "raw", buckets: int = DEFAULT_BUCKETS) -> Overview:
|
||||||
|
"""A whole-session envelope, cached per (unit, bucket count)."""
|
||||||
|
key = (str(unit), int(buckets))
|
||||||
|
if key not in self._overviews:
|
||||||
|
self._overviews[key] = self.index().overview(
|
||||||
|
unit=str(unit), buckets=int(buckets)
|
||||||
|
)
|
||||||
|
return self._overviews[key]
|
||||||
|
|
||||||
|
def statistics(self, t0: float, t1: float, *, unit: str = "raw"):
|
||||||
|
"""Statistics of an interval, computed from the interval's own rows."""
|
||||||
|
return self.window(t0, t1, unit=unit).statistics(unit)
|
||||||
|
|
||||||
# ------------------------------------------------------------------ policy
|
# ------------------------------------------------------------------ policy
|
||||||
def depth_policy(self) -> dict:
|
def depth_policy(self) -> dict:
|
||||||
return dict(self.info().session.get("depth") or {})
|
return dict(self.info().session.get("depth") or {})
|
||||||
@@ -372,6 +531,8 @@ class RecordingReader:
|
|||||||
warnings.append("rows_skipped")
|
warnings.append("rows_skipped")
|
||||||
if self.info().frames_dropped:
|
if self.info().frames_dropped:
|
||||||
warnings.append("frames_dropped")
|
warnings.append("frames_dropped")
|
||||||
|
if not self.info().frames_out_of_order or not self.info().forces_out_of_order:
|
||||||
|
warnings.append("out_of_order")
|
||||||
return warnings
|
return warnings
|
||||||
|
|
||||||
def iter_forces(self, batch: int = 4096) -> Iterator[tuple[np.ndarray, np.ndarray]]:
|
def iter_forces(self, batch: int = 4096) -> Iterator[tuple[np.ndarray, np.ndarray]]:
|
||||||
@@ -400,8 +561,56 @@ class RecordingReader:
|
|||||||
yield np.asarray(times), np.asarray(values).reshape(-1, 6)
|
yield np.asarray(times), np.asarray(values).reshape(-1, 6)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_time_ordered(values) -> bool:
|
||||||
|
"""True when ``values`` never decreases (an empty or single value is ordered)."""
|
||||||
|
previous = None
|
||||||
|
for value in values:
|
||||||
|
if previous is not None and float(value) < float(previous):
|
||||||
|
return False
|
||||||
|
previous = value
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def decimate_window(window: ForceWindow, max_points: int, *, unit: str = "raw") -> ForceWindow:
|
||||||
|
"""Reduce a window to about ``max_points``, keeping every axis' extremes.
|
||||||
|
|
||||||
|
The same rule the live store uses: each bucket contributes the extreme row of
|
||||||
|
**every plotted axis** in the active unit, so a shear spike survives a flat
|
||||||
|
normal channel. The result is marked ``decimated``.
|
||||||
|
"""
|
||||||
|
total = len(window)
|
||||||
|
max_points = max(4, int(max_points))
|
||||||
|
if total <= max_points:
|
||||||
|
return window
|
||||||
|
values = window.value_for_unit(unit)
|
||||||
|
def index_of(row: int) -> int:
|
||||||
|
return int(window.frame_ids[row]) if total else row
|
||||||
|
bucket = max(1, total // max(1, max_points // 6))
|
||||||
|
picked: set[int] = set()
|
||||||
|
for start in range(0, total, bucket):
|
||||||
|
stop = min(total, start + bucket)
|
||||||
|
for axis in range(3):
|
||||||
|
column = values[start:stop, axis]
|
||||||
|
finite = np.isfinite(column)
|
||||||
|
if not finite.any():
|
||||||
|
continue
|
||||||
|
ordered = np.where(finite, column, -np.inf)
|
||||||
|
picked.add(start + int(np.argmin(ordered)))
|
||||||
|
picked.add(start + int(np.argmax(ordered)))
|
||||||
|
if not picked:
|
||||||
|
return window
|
||||||
|
rows = sorted(picked)
|
||||||
|
return ForceWindow(
|
||||||
|
times=window.times[rows],
|
||||||
|
raw=window.raw[rows],
|
||||||
|
newtons=window.newtons[rows],
|
||||||
|
valid=window.valid[rows],
|
||||||
|
frame_ids=window.frame_ids[rows],
|
||||||
|
decimated=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def list_recordings(root: str | Path) -> list[Path]:
|
def list_recordings(root: str | Path) -> list[Path]:
|
||||||
"""Every recording directory under ``root``, newest first."""
|
|
||||||
base = Path(root)
|
base = Path(root)
|
||||||
if not base.is_dir():
|
if not base.is_dir():
|
||||||
return []
|
return []
|
||||||
|
|||||||
@@ -140,11 +140,16 @@ def to_record_sample(packet, *, calibration_version: str = "") -> RecordSample:
|
|||||||
valid = bool(getattr(packet, "valid", True))
|
valid = bool(getattr(packet, "valid", True))
|
||||||
reason = str(getattr(packet, "invalid_reason", "") or "")
|
reason = str(getattr(packet, "invalid_reason", "") or "")
|
||||||
stale = bool(getattr(packet, "stale_forces", False))
|
stale = bool(getattr(packet, "stale_forces", False))
|
||||||
# The packet's own version wins: it was stamped by the acquisition thread at
|
# A packet that *carries* a version owns it, including the empty one: the
|
||||||
# the moment of acquisition, so it is the only value that stays true when a
|
# acquisition thread stamps every frame, and an empty stamp is the device
|
||||||
# calibration is applied in the middle of a session.
|
# saying "no calibration loaded". Falling back to the session default there
|
||||||
version = str(
|
# would resurrect coefficients that are no longer in effect. A packet from an
|
||||||
getattr(packet, "calibration_version", "") or calibration_version or ""
|
# older producer that has no version field at all still gets the default.
|
||||||
|
packet_version = getattr(packet, "calibration_version", None)
|
||||||
|
version = (
|
||||||
|
str(calibration_version or "")
|
||||||
|
if packet_version is None
|
||||||
|
else str(packet_version)
|
||||||
)
|
)
|
||||||
axes = calibrated_axes_of(packet)
|
axes = calibrated_axes_of(packet)
|
||||||
|
|
||||||
@@ -214,3 +219,20 @@ def plottable_mask(values: Iterable[float], valid: Iterable[bool]) -> list[bool]
|
|||||||
bool(is_valid) and math.isfinite(float(value))
|
bool(is_valid) and math.isfinite(float(value))
|
||||||
for value, is_valid in zip(values, valid)
|
for value, is_valid in zip(values, valid)
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def plottable_axis_mask(values, valid) -> "np.ndarray":
|
||||||
|
"""``(n, axes)`` mask: one column per axis, decided for that axis alone.
|
||||||
|
|
||||||
|
The rule has to be per axis. A single-axis calibration is legitimate — the
|
||||||
|
device may have a normal-force curve and no shear curves — and an all-axes
|
||||||
|
rule hid a perfectly good FN because FX and FY had no newton value.
|
||||||
|
"""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
block = np.asarray(values, dtype=np.float64).reshape(-1, 3)
|
||||||
|
validity = np.asarray(valid, dtype=bool).reshape(-1)
|
||||||
|
finite = np.isfinite(block)
|
||||||
|
if validity.size != block.shape[0]:
|
||||||
|
return finite
|
||||||
|
return finite & validity[:, None]
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
"""The video side of a recording: one rolling MP4 segment at a time.
|
||||||
|
|
||||||
|
Kept apart from the CSV writer on purpose. The two have different failure modes —
|
||||||
|
a full disk breaks both, but a codec that refuses to open only breaks this one —
|
||||||
|
and the recorder's byte budget is about *these* frames, not about the rows.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import cv2
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class VideoSegments:
|
||||||
|
"""Writes raw frames into ``raw_NNNN.mp4`` files, rolling on a time budget."""
|
||||||
|
|
||||||
|
def __init__(self, directory: Path, *, codec: str, fps: float, segment_seconds: float):
|
||||||
|
self.directory = Path(directory)
|
||||||
|
self.codec = str(codec)
|
||||||
|
self.fps = max(1.0, float(fps))
|
||||||
|
self.segment_seconds = float(segment_seconds)
|
||||||
|
self.segments = 0
|
||||||
|
self.video_frames = 0
|
||||||
|
self.frame_shape: Optional[tuple[int, int]] = None
|
||||||
|
self._writer: Optional[cv2.VideoWriter] = None
|
||||||
|
self._index = 0
|
||||||
|
self._frames_in_segment = 0
|
||||||
|
self._started = 0.0
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ writing
|
||||||
|
def write(self, frame) -> tuple[int, int]:
|
||||||
|
"""Append one frame. Returns ``(segment, frame_in_segment)``."""
|
||||||
|
if self.frame_shape is None:
|
||||||
|
self.frame_shape = (int(frame.shape[1]), int(frame.shape[0]))
|
||||||
|
if self._should_roll():
|
||||||
|
self._close_segment()
|
||||||
|
if self._writer is None:
|
||||||
|
self._open_segment()
|
||||||
|
self._writer.write(frame)
|
||||||
|
self._frames_in_segment += 1
|
||||||
|
self.video_frames += 1
|
||||||
|
return (self._index, self._frames_in_segment)
|
||||||
|
|
||||||
|
def write_sequence(self, frames) -> int:
|
||||||
|
"""Write a run of frames as one standalone segment. Returns the count."""
|
||||||
|
frames = [frame for frame in frames if frame is not None]
|
||||||
|
if not frames:
|
||||||
|
return 0
|
||||||
|
self.frame_shape = (int(frames[0].shape[1]), int(frames[0].shape[0]))
|
||||||
|
name = "preroll_0001.mp4"
|
||||||
|
writer = self._open_writer(self.directory / name)
|
||||||
|
written = 0
|
||||||
|
try:
|
||||||
|
for frame in frames:
|
||||||
|
writer.write(frame)
|
||||||
|
written += 1
|
||||||
|
finally:
|
||||||
|
writer.release()
|
||||||
|
return written
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------- segments
|
||||||
|
def _should_roll(self) -> bool:
|
||||||
|
if self._writer is None:
|
||||||
|
return False
|
||||||
|
if self.segment_seconds <= 0:
|
||||||
|
return False
|
||||||
|
return (time.perf_counter() - self._started) >= self.segment_seconds
|
||||||
|
|
||||||
|
def _open_writer(self, path: Path) -> cv2.VideoWriter:
|
||||||
|
fourcc = cv2.VideoWriter_fourcc(*self.codec)
|
||||||
|
writer = cv2.VideoWriter(str(path), fourcc, self.fps, self.frame_shape)
|
||||||
|
if not writer.isOpened():
|
||||||
|
raise OSError(f"could not open video writer for {path.name}")
|
||||||
|
return writer
|
||||||
|
|
||||||
|
def _open_segment(self) -> None:
|
||||||
|
self._index += 1
|
||||||
|
path = self.directory / f"raw_{self._index:04d}.mp4"
|
||||||
|
self._writer = self._open_writer(path)
|
||||||
|
self._frames_in_segment = 0
|
||||||
|
self._started = time.perf_counter()
|
||||||
|
self.segments = self._index
|
||||||
|
|
||||||
|
def _close_segment(self) -> None:
|
||||||
|
if self._writer is None:
|
||||||
|
return
|
||||||
|
self._writer.release()
|
||||||
|
self._writer = None
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._close_segment()
|
||||||
@@ -0,0 +1,280 @@
|
|||||||
|
"""Calibration identity and operator events as the acquisition thread sees them.
|
||||||
|
|
||||||
|
The review's P1-7: the application event was written before the device had loaded
|
||||||
|
anything, the recorded ``calibration_id`` never changed mid-session, and an
|
||||||
|
uncalibrated device could keep claiming the coefficients of an earlier session.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import SdkAcquisitionWorker
|
||||||
|
from orisys_app_shared.record import ExperimentRecorder, RecordConfig
|
||||||
|
from orisys_app_shared.record.calibration_snapshot import (
|
||||||
|
list_calibration_snapshots,
|
||||||
|
snapshot_name,
|
||||||
|
store_calibration_snapshot,
|
||||||
|
)
|
||||||
|
|
||||||
|
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
||||||
|
|
||||||
|
|
||||||
|
def _root() -> Path:
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
SCRATCH.mkdir(parents=True, exist_ok=True)
|
||||||
|
return SCRATCH
|
||||||
|
|
||||||
|
|
||||||
|
class _Sensor:
|
||||||
|
"""A device with a swappable calibration, like the SDK's Sensor."""
|
||||||
|
|
||||||
|
def __init__(self, calibration=None, path=None):
|
||||||
|
self.calibration = calibration
|
||||||
|
self.calibration_path = path
|
||||||
|
self.applied: list[str] = []
|
||||||
|
self.resets = 0
|
||||||
|
|
||||||
|
def set_calibration(self, path):
|
||||||
|
if str(path) == "reject-me":
|
||||||
|
raise ValueError("bad document")
|
||||||
|
self.calibration = object()
|
||||||
|
self.calibration_path = Path(path)
|
||||||
|
self.applied.append(str(path))
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.resets += 1
|
||||||
|
|
||||||
|
|
||||||
|
def _calibration_file(root: Path, name: str, body: str = '{"unit": "N"}') -> Path:
|
||||||
|
path = root / name
|
||||||
|
path.write_text(body, encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def _worker(**kwargs) -> SdkAcquisitionWorker:
|
||||||
|
for key in ("on_calibration_applied", "on_reset_applied"):
|
||||||
|
kwargs.setdefault(key, None)
|
||||||
|
return SdkAcquisitionWorker(**kwargs)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ worker state
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_new_session_does_not_inherit_the_previous_version():
|
||||||
|
"""An uncalibrated device must report no calibration, not the last one."""
|
||||||
|
root = _root()
|
||||||
|
worker = _worker()
|
||||||
|
worker._calibration_version = "previous-session-hash"
|
||||||
|
worker._stop.set()
|
||||||
|
worker.start("clip.mp4", config_ref=str(_calibration_file(root, "sensor.json")))
|
||||||
|
assert worker._calibration_version == "", (
|
||||||
|
"the new session kept the previous session's calibration version"
|
||||||
|
)
|
||||||
|
worker.stop(wait=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_applied_version_is_the_content_hash_and_carries_the_frame():
|
||||||
|
root = _root()
|
||||||
|
applied: list[tuple] = []
|
||||||
|
worker = _worker(on_calibration_applied=lambda *args: applied.append(args))
|
||||||
|
path = _calibration_file(root, "force_calibration.json")
|
||||||
|
sensor = _Sensor()
|
||||||
|
worker._session_id = 4
|
||||||
|
worker.set_pending_calibration(path)
|
||||||
|
worker._consume_pending_calibration(sensor, frame_id=17, mono_t=0.5)
|
||||||
|
|
||||||
|
assert len(applied) == 1
|
||||||
|
got_path, version, frame_id, mono_t, session = applied[0]
|
||||||
|
assert got_path == str(path)
|
||||||
|
assert version and version == worker._calibration_version
|
||||||
|
assert version != str(path), "the path was used as the version"
|
||||||
|
assert (frame_id, mono_t, session) == (17, 0.5, 4)
|
||||||
|
assert sensor.applied == [str(path)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_rejected_calibration_keeps_the_version_the_device_still_has():
|
||||||
|
root = _root()
|
||||||
|
applied: list[tuple] = []
|
||||||
|
worker = _worker(on_calibration_applied=lambda *args: applied.append(args))
|
||||||
|
sensor = _Sensor()
|
||||||
|
worker._session_id = 1
|
||||||
|
worker._calibration_version = "loaded-earlier"
|
||||||
|
worker.set_pending_calibration("reject-me")
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
worker._consume_pending_calibration(sensor, frame_id=3, mono_t=0.1)
|
||||||
|
assert not applied, "a rejected document was reported as applied"
|
||||||
|
assert worker._calibration_version == "loaded-earlier", (
|
||||||
|
"the device still has those coefficients, so the version is still true"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_zeroing_is_applied_by_the_worker_and_reported_with_its_frame():
|
||||||
|
root = _root()
|
||||||
|
events: list[tuple] = []
|
||||||
|
worker = _worker(on_reset_applied=lambda *args: events.append(args))
|
||||||
|
sensor = _Sensor()
|
||||||
|
worker._session_id = 2
|
||||||
|
worker.reset() # a request, not the zeroing itself
|
||||||
|
assert sensor.resets == 0, "the zeroing ran on the caller's thread"
|
||||||
|
worker._consume_pending_reset(sensor, frame_id=42, mono_t=1.4)
|
||||||
|
assert sensor.resets == 1
|
||||||
|
assert events == [(42, 1.4, 2)]
|
||||||
|
# The request is consumed once.
|
||||||
|
worker._consume_pending_reset(sensor, frame_id=43, mono_t=1.5)
|
||||||
|
assert events == [(42, 1.4, 2)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_started_session_drops_a_stale_zeroing_request():
|
||||||
|
root = _root()
|
||||||
|
events: list[tuple] = []
|
||||||
|
worker = _worker(on_reset_applied=lambda *args: events.append(args))
|
||||||
|
worker._stop.set()
|
||||||
|
worker.start("clip.mp4", config_ref=str(_calibration_file(root, "sensor.json")))
|
||||||
|
sensor = _Sensor()
|
||||||
|
worker._consume_pending_reset(sensor, frame_id=1, mono_t=0.0)
|
||||||
|
assert sensor.resets == 0, "a reset requested before this session fired into it"
|
||||||
|
assert not events
|
||||||
|
worker.stop(wait=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------- snapshots
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_snapshot_is_content_addressed_and_listed():
|
||||||
|
root = _root()
|
||||||
|
directory = root / "session"
|
||||||
|
source = _calibration_file(root, "force_calibration_box.json", '{"a": 1}')
|
||||||
|
name = store_calibration_snapshot(directory, source, "abcdef1234567890")
|
||||||
|
assert name == "calibration/force_calibration_abcdef123456.json"
|
||||||
|
assert (directory / name).is_file()
|
||||||
|
assert (directory / name).read_text(encoding="utf-8") == '{"a": 1}'
|
||||||
|
# Applying the same coefficients twice stores one file.
|
||||||
|
again = store_calibration_snapshot(directory, source, "abcdef1234567890")
|
||||||
|
assert again == name
|
||||||
|
assert list_calibration_snapshots(directory) == [name]
|
||||||
|
# A different calibration cannot collide with it.
|
||||||
|
other = _calibration_file(root, "force_calibration_box.json", '{"a": 2}')
|
||||||
|
second = store_calibration_snapshot(directory, other, "ffffff000000")
|
||||||
|
assert second != name
|
||||||
|
assert len(list_calibration_snapshots(directory)) == 2
|
||||||
|
assert snapshot_name("", source).startswith("force_calibration_")
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_recording_metadata_names_the_coefficients_it_ran_with():
|
||||||
|
root = _root()
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
directory = recorder.start(root=root, session_id=1, metadata={})
|
||||||
|
source = _calibration_file(root, "force_calibration_box.json", '{"a": 1}')
|
||||||
|
name = store_calibration_snapshot(directory, source, "abcdef1234567890")
|
||||||
|
recorder.note_calibration_snapshot(
|
||||||
|
version="abcdef1234567890", name=name, frame_id=12, mono_t=0.4
|
||||||
|
)
|
||||||
|
recorder.log_event(
|
||||||
|
"calibration_applied", name, mono_t=0.4, frame_id=12
|
||||||
|
)
|
||||||
|
recorder.stop()
|
||||||
|
|
||||||
|
meta = json.loads((Path(directory) / "session.json").read_text(encoding="utf-8"))
|
||||||
|
snapshots = meta["calibration_snapshots"]
|
||||||
|
assert snapshots and snapshots[0]["version"] == "abcdef1234567890"
|
||||||
|
assert snapshots[0]["file"] == name
|
||||||
|
assert snapshots[0]["frame_id"] == 12
|
||||||
|
assert snapshots[0]["mono_t"] == pytest.approx(0.4)
|
||||||
|
event = next(e for e in meta["events"] if e["kind"] == "calibration_applied")
|
||||||
|
assert event["frame_id"] == 12
|
||||||
|
assert event["mono_t"] == pytest.approx(0.4), "the event lost its acquisition time"
|
||||||
|
assert event["at"], "the event lost its wall clock"
|
||||||
|
|
||||||
|
# The CSV carries the same acquisition time in the session's relative base.
|
||||||
|
# An event that predates the first frame has no base yet, so its relative
|
||||||
|
# time is empty — the absolute (session-relative) time is in the metadata.
|
||||||
|
with (Path(directory) / "events.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
import csv as _csv
|
||||||
|
|
||||||
|
rows = list(_csv.DictReader(handle))
|
||||||
|
applied = next(row for row in rows if row["kind"] == "calibration_applied")
|
||||||
|
assert applied["frame_id"] == "12"
|
||||||
|
assert applied["t_rel_s"] == "", "an event before the first frame invented a base"
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_event_during_the_recording_gets_the_session_relative_time():
|
||||||
|
import csv as _csv
|
||||||
|
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import SdkPacket
|
||||||
|
|
||||||
|
root = _root()
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
directory = recorder.start(root=root, session_id=1, metadata={})
|
||||||
|
frame = np.zeros((4, 4, 3), np.uint8)
|
||||||
|
recorder.on_packet(
|
||||||
|
SdkPacket(
|
||||||
|
raw_frame=frame,
|
||||||
|
processed_frame=frame,
|
||||||
|
timestamp=1000.0,
|
||||||
|
mono_t=0.5,
|
||||||
|
fps=30.0,
|
||||||
|
process_ms=1.0,
|
||||||
|
frame_id=7,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
recorder.log_event("zeroing", "tracker reset", mono_t=0.9, frame_id=9)
|
||||||
|
recorder.stop()
|
||||||
|
|
||||||
|
with (Path(directory) / "events.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
rows = list(_csv.DictReader(handle))
|
||||||
|
zeroing = next(row for row in rows if row["kind"] == "zeroing")
|
||||||
|
assert float(zeroing["t_rel_s"]) == pytest.approx(0.4, abs=1e-6)
|
||||||
|
assert zeroing["frame_id"] == "9"
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_event_without_a_mono_time_still_gets_a_wall_clock():
|
||||||
|
root = _root()
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
directory = recorder.start(root=root, session_id=1, metadata={})
|
||||||
|
recorder.log_event("operator_note", "no frame attached")
|
||||||
|
recorder.stop()
|
||||||
|
meta = json.loads((Path(directory) / "session.json").read_text(encoding="utf-8"))
|
||||||
|
event = next(e for e in meta["events"] if e["kind"] == "operator_note")
|
||||||
|
assert event["at"], "the event lost its wall clock"
|
||||||
|
assert event["frame_id"] == -1
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_recorded_row_uses_the_packet_version_not_the_session_default():
|
||||||
|
from orisys_app_shared.record.sample import to_record_sample
|
||||||
|
|
||||||
|
class Packet:
|
||||||
|
frame_id = 3
|
||||||
|
mono_t = 0.2
|
||||||
|
wall_t = 0.2
|
||||||
|
valid = True
|
||||||
|
calibration_version = ""
|
||||||
|
calibrated_axes = ("fn",)
|
||||||
|
force_calibrated = True
|
||||||
|
raw_available = True
|
||||||
|
|
||||||
|
# An empty packet version is the device saying "no calibration": the session
|
||||||
|
# default must not be pasted in behind its back.
|
||||||
|
assert to_record_sample(Packet(), calibration_version="session-default").calibration_version == ""
|
||||||
|
|
||||||
|
class LegacyPacket(Packet):
|
||||||
|
calibration_version = None # an older producer without the field
|
||||||
|
|
||||||
|
# ...but a packet that has no version field at all still gets the default.
|
||||||
|
assert (
|
||||||
|
to_record_sample(LegacyPacket(), calibration_version="session-default").calibration_version
|
||||||
|
== "session-default"
|
||||||
|
)
|
||||||
|
|
||||||
|
class StampedPacket(Packet):
|
||||||
|
calibration_version = "packet-hash"
|
||||||
|
|
||||||
|
assert (
|
||||||
|
to_record_sample(StampedPacket(), calibration_version="session-default").calibration_version
|
||||||
|
== "packet-hash"
|
||||||
|
)
|
||||||
@@ -8,6 +8,9 @@ model.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from orisys_app_shared.force.calibration_io import (
|
from orisys_app_shared.force.calibration_io import (
|
||||||
@@ -17,10 +20,19 @@ from orisys_app_shared.force.calibration_io import (
|
|||||||
build_force_calibration_v2,
|
build_force_calibration_v2,
|
||||||
merge_samples,
|
merge_samples,
|
||||||
predict_channel,
|
predict_channel,
|
||||||
|
write_force_calibration,
|
||||||
)
|
)
|
||||||
from orisys_app_shared.force.calibration_recorder import CalibrationRecorder
|
from orisys_app_shared.force.calibration_recorder import CalibrationRecorder
|
||||||
from orisys_app_shared.force.session import SessionSample
|
from orisys_app_shared.force.session import SessionSample
|
||||||
|
|
||||||
|
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
||||||
|
|
||||||
|
|
||||||
|
def _root() -> Path:
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
SCRATCH.mkdir(parents=True, exist_ok=True)
|
||||||
|
return SCRATCH
|
||||||
|
|
||||||
|
|
||||||
def _point(recorder, load: float, raw: float, *, channel: str = "fn"):
|
def _point(recorder, load: float, raw: float, *, channel: str = "fn"):
|
||||||
recorder.start(channel=channel, reference_load_n=load, duration_s=0.05)
|
recorder.start(channel=channel, reference_load_n=load, duration_s=0.05)
|
||||||
@@ -151,3 +163,84 @@ def test_a_channel_with_one_point_is_not_exported():
|
|||||||
store.aggregate("fn", CalSample(weight_n=1.0, raw_mean=10.0, n=10))
|
store.aggregate("fn", CalSample(weight_n=1.0, raw_mean=10.0, n=10))
|
||||||
with pytest.raises(ValueError):
|
with pytest.raises(ValueError):
|
||||||
build_force_calibration_v2(store)
|
build_force_calibration_v2(store)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- one preparation
|
||||||
|
|
||||||
|
|
||||||
|
def _session_with_duplicates_and_validation():
|
||||||
|
recorder = CalibrationRecorder()
|
||||||
|
points = [
|
||||||
|
_point(recorder, 1.0, 100.0),
|
||||||
|
_point(recorder, 2.0, 200.0),
|
||||||
|
# A deliberate repeat of load 2.0 at a different raw level.
|
||||||
|
_point(recorder, 2.0, 240.0),
|
||||||
|
_point(recorder, 4.0, 400.0),
|
||||||
|
_point(recorder, 3.0, 290.0),
|
||||||
|
]
|
||||||
|
recorder.set_validation(points[4], True)
|
||||||
|
return recorder, points
|
||||||
|
|
||||||
|
|
||||||
|
def test_duplicate_loads_are_merged_before_the_model_is_built():
|
||||||
|
_recorder, _points = _session_with_duplicates_and_validation()
|
||||||
|
prepared = _recorder.prepared()
|
||||||
|
assert prepared.samples["fn"], "no training samples"
|
||||||
|
loads = [sample.weight_n for sample in prepared.samples["fn"]]
|
||||||
|
assert loads == sorted(set(loads)), f"duplicate loads were not merged: {loads}"
|
||||||
|
assert len(loads) == 3
|
||||||
|
merged = next(sample for sample in prepared.samples["fn"] if sample.weight_n == 2.0)
|
||||||
|
assert merged.n == 16, "the repeat's samples were not combined"
|
||||||
|
assert merged.raw_mean == pytest.approx((200.0 + 240.0) / 2.0)
|
||||||
|
assert prepared.points_in_fit("fn") == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_exported_file_predicts_exactly_what_the_panel_shows():
|
||||||
|
"""Read the written document back and compare it with the diagnostics.
|
||||||
|
|
||||||
|
The panel's numbers must describe the file that is actually written, so the
|
||||||
|
test loads the file with the SDK (not with the prepared object) and predicts
|
||||||
|
every training and validation point from it.
|
||||||
|
"""
|
||||||
|
import orisys.calibration as sdk_calibration
|
||||||
|
|
||||||
|
recorder, points = _session_with_duplicates_and_validation()
|
||||||
|
prepared = recorder.prepared(meta={"sensor_type": "box", "sn": "SN9"})
|
||||||
|
assert prepared.ok
|
||||||
|
|
||||||
|
document = prepared.to_document()
|
||||||
|
path = _root() / "force_calibration.json"
|
||||||
|
write_force_calibration(document, path)
|
||||||
|
from_file = sdk_calibration.load(path, strict=True)
|
||||||
|
|
||||||
|
for point in points:
|
||||||
|
raw = float(point.loaded_stats().mean)
|
||||||
|
predicted = float(from_file.get("fn").convert(raw))
|
||||||
|
value = prepared.predict("fn", raw)
|
||||||
|
assert predicted == pytest.approx(value, abs=1e-9)
|
||||||
|
if point.is_validation:
|
||||||
|
assert prepared.validation_errors[id(point)] == pytest.approx(
|
||||||
|
abs(predicted - point.reference_load_n), abs=1e-9
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
assert prepared.residuals[id(point)] == pytest.approx(
|
||||||
|
predicted - point.reference_load_n, abs=1e-9
|
||||||
|
)
|
||||||
|
|
||||||
|
# The validation point is measured, never built in.
|
||||||
|
assert prepared.validation_points("fn") == 1
|
||||||
|
assert prepared.max_validation_error("fn") > 0.0
|
||||||
|
exported_raws = sorted(point[0] for point in document["channels"]["normal"]["points"])
|
||||||
|
assert exported_raws == [100.0, 220.0, 400.0], exported_raws
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_recorder_fit_and_the_preparation_agree():
|
||||||
|
recorder, _points = _session_with_duplicates_and_validation()
|
||||||
|
prepared = recorder.prepared()
|
||||||
|
fit = recorder.fit("fn", prepared=prepared)
|
||||||
|
assert fit.n_points == prepared.points_in_fit("fn")
|
||||||
|
assert fit.breakpoints == prepared.breakpoints("fn")
|
||||||
|
assert fit.max_residual_n == pytest.approx(prepared.max_residual("fn"))
|
||||||
|
assert fit.validation_error_n == pytest.approx(prepared.max_validation_error("fn"))
|
||||||
|
# ...and a fit that prepares on its own gets the same numbers.
|
||||||
|
assert recorder.fit("fn").as_dict() == fit.as_dict()
|
||||||
|
|||||||
@@ -31,15 +31,15 @@ def _root() -> Path:
|
|||||||
return SCRATCH
|
return SCRATCH
|
||||||
|
|
||||||
|
|
||||||
def _recorded_point(index_offset: float = 0.0):
|
def _recorded_point(load: float = 2.0, offset: float = 0.0):
|
||||||
recorder = CalibrationRecorder()
|
recorder = CalibrationRecorder()
|
||||||
recorder.start(channel="fn", reference_load_n=2.0, duration_s=0.05)
|
recorder.start(channel="fn", reference_load_n=load, duration_s=0.05)
|
||||||
for index in range(6):
|
for index in range(6):
|
||||||
recorder.feed(
|
recorder.feed(
|
||||||
SessionSample(
|
SessionSample(
|
||||||
index,
|
index,
|
||||||
index * 1.0 / 30.0,
|
index * 1.0 / 30.0,
|
||||||
fn_raw=float(index) + index_offset,
|
fn_raw=float(index) + offset,
|
||||||
fx_raw=1.0,
|
fx_raw=1.0,
|
||||||
fy_raw=2.0,
|
fy_raw=2.0,
|
||||||
fn_n=float(index) * 10.0,
|
fn_n=float(index) * 10.0,
|
||||||
@@ -126,9 +126,9 @@ def test_a_newer_format_is_loaded_with_a_warning():
|
|||||||
|
|
||||||
def test_the_recorder_can_reopen_a_saved_session():
|
def test_the_recorder_can_reopen_a_saved_session():
|
||||||
points = [
|
points = [
|
||||||
_recorded_point(),
|
_recorded_point(1.0),
|
||||||
_recorded_point(5.0),
|
_recorded_point(2.0, 5.0),
|
||||||
_recorded_point(9.0),
|
_recorded_point(3.0, 9.0),
|
||||||
]
|
]
|
||||||
path = write_calibration_session(_root() / "session.json", points)
|
path = write_calibration_session(_root() / "session.json", points)
|
||||||
restored = read_calibration_session(path).points
|
restored = read_calibration_session(path).points
|
||||||
@@ -141,3 +141,76 @@ def test_the_recorder_can_reopen_a_saved_session():
|
|||||||
fit = recorder.fit("fn")
|
fit = recorder.fit("fn")
|
||||||
assert fit.validation_points == 1
|
assert fit.validation_points == 1
|
||||||
assert fit.n_points == 2
|
assert fit.n_points == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_reopened_points_at_one_load_are_one_calibrated_load():
|
||||||
|
"""Three points at the same reference load are one load, not a curve.
|
||||||
|
|
||||||
|
They are merged (n-weighted) before the build, so a session that only ever
|
||||||
|
sampled one load cannot produce a model — which is the honest answer.
|
||||||
|
"""
|
||||||
|
recorder = CalibrationRecorder()
|
||||||
|
recorder.load_points(
|
||||||
|
[_recorded_point(2.0), _recorded_point(2.0, 5.0), _recorded_point(2.0, 9.0)],
|
||||||
|
replace=True,
|
||||||
|
)
|
||||||
|
prepared = recorder.prepared()
|
||||||
|
assert len(prepared.training) == 3
|
||||||
|
assert len(prepared.samples["fn"]) == 1
|
||||||
|
assert prepared.samples["fn"][0].n == 15, "the valid counts were not summed"
|
||||||
|
assert prepared.model is None, "a single load produced a model"
|
||||||
|
assert recorder.fit("fn").n_points == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------- reopened diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_reopened_record_keeps_its_device_identity():
|
||||||
|
"""Device A's record stays A's, whatever is connected now."""
|
||||||
|
points = [_recorded_point(1.0), _recorded_point(2.0, 5.0)]
|
||||||
|
path = write_calibration_session(
|
||||||
|
_root() / "session.json",
|
||||||
|
points,
|
||||||
|
meta={
|
||||||
|
"sensor_type": "box",
|
||||||
|
"sn": "SN-A",
|
||||||
|
"config_ref": "D:/lab/deviceA/sensor.json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
record = read_calibration_session(path)
|
||||||
|
identity = record.identity()
|
||||||
|
assert identity["sensor_type"] == "box"
|
||||||
|
assert identity["sn"] == "SN-A"
|
||||||
|
assert identity["config_ref"] == "D:/lab/deviceA/sensor.json"
|
||||||
|
assert identity["created_at"] == record.created_at
|
||||||
|
assert "SN-A" in record.describe()
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_three_axis_trace_is_rebuilt_from_the_samples():
|
||||||
|
"""A record whose trace was never written must still be inspectable."""
|
||||||
|
import json
|
||||||
|
|
||||||
|
point = _recorded_point(1.0)
|
||||||
|
document = build_session_document([point])
|
||||||
|
# Simulate a document that only kept the audit series (or an older writer
|
||||||
|
# that stored a truncated trace).
|
||||||
|
document["points"][0]["trace"] = {}
|
||||||
|
document["points"][0]["duration_s"] = 0.0
|
||||||
|
path = _root() / "session.json"
|
||||||
|
path.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
|
||||||
|
restored = read_calibration_session(path).points[0]
|
||||||
|
assert set(restored.trace) == {"fn", "fx", "fy"}, restored.trace
|
||||||
|
assert len(restored.trace["fn"]) == 5 # the valid samples
|
||||||
|
assert restored.trace_dt_s == pytest.approx(1.0 / 30.0, abs=1e-6)
|
||||||
|
assert restored.duration_s == pytest.approx(5.0 / 30.0, abs=1e-6)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_stored_trace_is_not_overwritten_on_reopen():
|
||||||
|
point = _recorded_point(1.0)
|
||||||
|
document = build_session_document([point])
|
||||||
|
document["points"][0]["trace"] = {"fn": [1.0, 2.0]}
|
||||||
|
path = _root() / "session.json"
|
||||||
|
path.write_text(json.dumps(document), encoding="utf-8")
|
||||||
|
restored = read_calibration_session(path).points[0]
|
||||||
|
assert restored.trace["fn"] == [1.0, 2.0]
|
||||||
|
|||||||
@@ -16,7 +16,12 @@ from unittest.mock import patch
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from orisys_app_shared.record.depth import DepthRecomputer
|
from orisys_app_shared.record.depth import (
|
||||||
|
STATE_APPROXIMATE,
|
||||||
|
STATE_EXACT,
|
||||||
|
STATE_PREROLL,
|
||||||
|
DepthRecomputer,
|
||||||
|
)
|
||||||
|
|
||||||
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
||||||
|
|
||||||
@@ -36,6 +41,12 @@ class _CountingSensor:
|
|||||||
_CountingSensor.opened.append({"args": args, "kwargs": kwargs})
|
_CountingSensor.opened.append({"args": args, "kwargs": kwargs})
|
||||||
self.n = 0
|
self.n = 0
|
||||||
self.chain_calls = 0
|
self.chain_calls = 0
|
||||||
|
self.initial_img = "reference-frame"
|
||||||
|
# The real Sensor opens its source in the constructor, so the first
|
||||||
|
# source is part of its history too.
|
||||||
|
self.vid_src = str(args[0]) if args else ""
|
||||||
|
self.sources: list[str] = [self.vid_src] if self.vid_src else []
|
||||||
|
self.resets = 0
|
||||||
self.info = SimpleNamespace(DEPTH=1)
|
self.info = SimpleNamespace(DEPTH=1)
|
||||||
|
|
||||||
def get_img(self):
|
def get_img(self):
|
||||||
@@ -48,9 +59,17 @@ class _CountingSensor:
|
|||||||
def compute_contact(self):
|
def compute_contact(self):
|
||||||
self.chain_calls += 1
|
self.chain_calls += 1
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.resets += 1
|
||||||
|
|
||||||
def read_info(self, _channel):
|
def read_info(self, _channel):
|
||||||
return np.array([self.n])
|
return np.array([self.n])
|
||||||
|
|
||||||
|
def _open_video_source(self, _override, _box):
|
||||||
|
self.sources.append(str(self.vid_src))
|
||||||
|
# A real reopen restarts the file from its first frame.
|
||||||
|
self.n = 0
|
||||||
|
|
||||||
def disconnect(self):
|
def disconnect(self):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -116,13 +135,18 @@ def test_stepping_continues_from_the_current_position():
|
|||||||
computer.close()
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
def test_stepping_backwards_reopens_the_segment():
|
def test_stepping_backwards_reopens_the_source_but_keeps_the_processor():
|
||||||
_CountingSensor.opened.clear()
|
_CountingSensor.opened.clear()
|
||||||
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
|
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
|
||||||
with patch("orisys.Sensor", _CountingSensor):
|
with patch("orisys.Sensor", _CountingSensor):
|
||||||
assert computer._depth_from_segment(Path("raw_0001.mp4"), 5).tolist() == [5]
|
assert computer._depth_from_segment(Path("raw_0001.mp4"), 5).tolist() == [5]
|
||||||
|
# Reading an earlier frame restarts the file; the processor object (and
|
||||||
|
# therefore the tracker) is not thrown away.
|
||||||
assert computer._depth_from_segment(Path("raw_0001.mp4"), 2).tolist() == [2]
|
assert computer._depth_from_segment(Path("raw_0001.mp4"), 2).tolist() == [2]
|
||||||
assert len(_CountingSensor.opened) == 2, "seeking backwards needs a fresh source"
|
sensor = computer._sensor._sensor
|
||||||
|
assert len(_CountingSensor.opened) == 1, "the processor was recreated"
|
||||||
|
assert len(sensor.sources) == 2
|
||||||
|
assert all(source.endswith("raw_0001.mp4") for source in sensor.sources)
|
||||||
computer.close()
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
@@ -165,11 +189,44 @@ def test_the_plan_starts_at_the_last_reset_in_the_same_segment():
|
|||||||
computer = DepthRecomputer(reader)
|
computer = DepthRecomputer(reader)
|
||||||
plan = computer.plan_for(frames[8])
|
plan = computer.plan_for(frames[8])
|
||||||
assert plan.stop_at == 9
|
assert plan.stop_at == 9
|
||||||
|
assert plan.anchor == "reset"
|
||||||
|
assert plan.state == STATE_EXACT
|
||||||
|
assert plan.reset_frame_id == 6
|
||||||
|
assert len(plan.steps) == 1
|
||||||
|
step = plan.steps[0]
|
||||||
|
assert step.source == reader.directory / "raw_0001.mp4"
|
||||||
|
assert step.to_frame == 9
|
||||||
|
assert step.reset_after == 7, "the reset boundary is not re-applied by the plan"
|
||||||
|
assert plan.frames_to_read == 9
|
||||||
assert plan.preroll is None
|
assert plan.preroll is None
|
||||||
assert plan.warnings == ()
|
assert plan.warnings == ()
|
||||||
computer.close()
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_reset_that_is_not_in_the_video_is_not_a_recovered_state():
|
||||||
|
"""Finding the event is not the same as being able to start there."""
|
||||||
|
frames = [
|
||||||
|
SimpleNamespace(
|
||||||
|
frame_id=index,
|
||||||
|
t_rel_s=index / 30.0,
|
||||||
|
segment=1,
|
||||||
|
segment_frame=index + 1,
|
||||||
|
in_video=index != 6, # the reset frame never reached the video
|
||||||
|
dropped=False,
|
||||||
|
)
|
||||||
|
for index in range(10)
|
||||||
|
]
|
||||||
|
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
|
||||||
|
reader.last_reset_frame = lambda frame_id: 6
|
||||||
|
computer = DepthRecomputer(reader)
|
||||||
|
plan = computer.plan_for(frames[8])
|
||||||
|
assert plan.anchor != "reset"
|
||||||
|
assert plan.state == STATE_APPROXIMATE
|
||||||
|
assert "reset_frame_not_in_video" in plan.warnings
|
||||||
|
assert "state_approximate" in plan.warnings
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
def test_the_plan_uses_the_pre_roll_for_the_start_of_a_recording():
|
def test_the_plan_uses_the_pre_roll_for_the_start_of_a_recording():
|
||||||
frames = [
|
frames = [
|
||||||
SimpleNamespace(
|
SimpleNamespace(
|
||||||
@@ -195,6 +252,8 @@ def test_the_plan_uses_the_pre_roll_for_the_start_of_a_recording():
|
|||||||
plan = computer.plan_for(frames[3])
|
plan = computer.plan_for(frames[3])
|
||||||
assert plan.preroll is not None
|
assert plan.preroll is not None
|
||||||
assert plan.preroll_frames == 30
|
assert plan.preroll_frames == 30
|
||||||
|
assert plan.anchor == "preroll"
|
||||||
|
assert plan.state == STATE_PREROLL
|
||||||
assert plan.warnings == ()
|
assert plan.warnings == ()
|
||||||
computer.close()
|
computer.close()
|
||||||
|
|
||||||
@@ -215,10 +274,127 @@ def test_a_segment_without_history_is_flagged_approximate():
|
|||||||
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
|
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
|
||||||
computer = DepthRecomputer(reader)
|
computer = DepthRecomputer(reader)
|
||||||
plan = computer.plan_for(frames[2])
|
plan = computer.plan_for(frames[2])
|
||||||
|
assert plan.state == STATE_APPROXIMATE
|
||||||
|
assert plan.anchor == "segment_start"
|
||||||
assert "state_approximate" in plan.warnings
|
assert "state_approximate" in plan.warnings
|
||||||
computer.close()
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_reset_in_an_earlier_segment_is_carried_across_by_swapping_the_source():
|
||||||
|
"""The processor survives the swap, so the state *can* cross a segment."""
|
||||||
|
frames = [
|
||||||
|
SimpleNamespace(
|
||||||
|
frame_id=index,
|
||||||
|
t_rel_s=index / 30.0,
|
||||||
|
segment=1 if index < 3 else 2,
|
||||||
|
segment_frame=index + 1 if index < 3 else index - 2,
|
||||||
|
in_video=True,
|
||||||
|
dropped=False,
|
||||||
|
)
|
||||||
|
for index in range(6)
|
||||||
|
]
|
||||||
|
reader = _reader(
|
||||||
|
_session("/cfg/sensor.json"),
|
||||||
|
segments=("raw_0001.mp4", "raw_0002.mp4"),
|
||||||
|
frames=frames,
|
||||||
|
)
|
||||||
|
reader.last_reset_frame = lambda frame_id: 1
|
||||||
|
computer = DepthRecomputer(reader)
|
||||||
|
plan = computer.plan_for(frames[5])
|
||||||
|
assert plan.anchor == "reset"
|
||||||
|
assert plan.state == STATE_EXACT
|
||||||
|
assert "state_across_segment" not in plan.warnings
|
||||||
|
assert [step.source.name for step in plan.steps] == ["raw_0001.mp4", "raw_0002.mp4"]
|
||||||
|
assert plan.steps[0].reset_after == 2
|
||||||
|
assert plan.steps[1].to_frame == 3
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_replay_that_would_read_too_far_back_is_reported_not_attempted():
|
||||||
|
frames = [
|
||||||
|
SimpleNamespace(
|
||||||
|
frame_id=index,
|
||||||
|
t_rel_s=index / 30.0,
|
||||||
|
segment=1,
|
||||||
|
segment_frame=index + 1,
|
||||||
|
in_video=True,
|
||||||
|
dropped=False,
|
||||||
|
)
|
||||||
|
for index in range(20)
|
||||||
|
]
|
||||||
|
reader = _reader(_session("/cfg/sensor.json"), frames=frames)
|
||||||
|
reader.last_reset_frame = lambda frame_id: 1
|
||||||
|
computer = DepthRecomputer(reader)
|
||||||
|
with patch("orisys_app_shared.record.depth.MAX_REPLAY_FRAMES", 5):
|
||||||
|
plan = computer.plan_for(frames[10])
|
||||||
|
assert "state_too_far_back" in plan.warnings
|
||||||
|
assert plan.state == STATE_APPROXIMATE
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_same_frame_is_answered_from_the_cache():
|
||||||
|
"""A second request for the frame just read is a cache hit, not None."""
|
||||||
|
_CountingSensor.opened.clear()
|
||||||
|
computer = DepthRecomputer(_reader(_session("/cfg/sensor.json")))
|
||||||
|
with patch("orisys.Sensor", _CountingSensor):
|
||||||
|
first = computer._depth_from_segment(Path("raw_0001.mp4"), 3)
|
||||||
|
second = computer._depth_from_segment(Path("raw_0001.mp4"), 3)
|
||||||
|
sensor = computer._sensor._sensor
|
||||||
|
assert first.tolist() == [3]
|
||||||
|
assert second is not None and second.tolist() == [3]
|
||||||
|
assert sensor.n == 3, "the cached answer re-read the stream"
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_processor_survives_a_pre_roll_to_segment_switch():
|
||||||
|
"""Only the video source is replaced; the tracker state must survive."""
|
||||||
|
_CountingSensor.opened.clear()
|
||||||
|
reader = _reader(_session("/cfg/sensor.json"))
|
||||||
|
preroll = reader.directory / "preroll_0001.mp4"
|
||||||
|
preroll.write_bytes(b"")
|
||||||
|
computer = DepthRecomputer(reader)
|
||||||
|
with patch("orisys.Sensor", _CountingSensor):
|
||||||
|
computer._feed(preroll, 2)
|
||||||
|
sensor = computer._sensor._sensor
|
||||||
|
assert sensor.n == 2, "the pre-roll frames were not fed"
|
||||||
|
# Feed the segment: same processor, new source, position restarted.
|
||||||
|
computer._feed(reader.directory / "raw_0001.mp4", 1)
|
||||||
|
assert len(_CountingSensor.opened) == 1, "the processor was recreated"
|
||||||
|
assert computer._sensor._sensor is sensor
|
||||||
|
assert len(sensor.sources) == 2
|
||||||
|
assert sensor.sources[0].endswith("preroll_0001.mp4")
|
||||||
|
assert sensor.sources[1].endswith("raw_0001.mp4")
|
||||||
|
assert sensor.initial_img == "reference-frame", "the tracker reference was reset"
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_reset_boundary_is_re_applied_during_the_replay():
|
||||||
|
"""The live zeroing cleared the tracker; the replay must do the same."""
|
||||||
|
_CountingSensor.opened.clear()
|
||||||
|
frames = [
|
||||||
|
SimpleNamespace(
|
||||||
|
frame_id=index,
|
||||||
|
t_rel_s=index / 30.0,
|
||||||
|
segment=1,
|
||||||
|
segment_frame=index + 1,
|
||||||
|
in_video=True,
|
||||||
|
dropped=False,
|
||||||
|
)
|
||||||
|
for index in range(10)
|
||||||
|
]
|
||||||
|
reader = _reader_with_config(frames)
|
||||||
|
reader.last_reset_frame = lambda frame_id: 6
|
||||||
|
reader.frame_at = lambda _t: frames[8]
|
||||||
|
computer = DepthRecomputer(reader)
|
||||||
|
with patch("orisys.Sensor", _CountingSensor):
|
||||||
|
result = computer.depth_at(frames[8].t_rel_s)
|
||||||
|
sensor = computer._sensor._sensor
|
||||||
|
assert result.ok, result.error
|
||||||
|
assert sensor.resets == 1, "the reset boundary was not re-applied"
|
||||||
|
assert result.warnings == (), result.warnings
|
||||||
|
computer.close()
|
||||||
|
|
||||||
|
|
||||||
def test_a_dropped_frame_is_reported_not_substituted():
|
def test_a_dropped_frame_is_reported_not_substituted():
|
||||||
"""The index marks the hole; replay must say so instead of showing a neighbour."""
|
"""The index marks the hole; replay must say so instead of showing a neighbour."""
|
||||||
frames = [
|
frames = [
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
"""Display unit selection and the per-axis drawing rule.
|
||||||
|
|
||||||
|
Two defects are covered here:
|
||||||
|
|
||||||
|
* the live charts chose their unit from the device-level ``force_calibrated``
|
||||||
|
flag, so a device with only its normal-force axis calibrated either showed
|
||||||
|
every axis in raw counts or showed raw counts inside newton panels;
|
||||||
|
* the drawing rule asked for **all three** axes to be finite, so a valid FN
|
||||||
|
disappeared from the chart whenever FX/FY had no calibrated value.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import SdkPacket
|
||||||
|
from orisys_app_shared.force import select_display_forces
|
||||||
|
from orisys_app_shared.force.session import SessionSample, SessionStore
|
||||||
|
from orisys_app_shared.record.sample import to_record_sample
|
||||||
|
|
||||||
|
NAN = float("nan")
|
||||||
|
|
||||||
|
|
||||||
|
def _packet(**overrides) -> SdkPacket:
|
||||||
|
frame = np.zeros((4, 4, 3), np.uint8)
|
||||||
|
kwargs = dict(
|
||||||
|
raw_frame=frame,
|
||||||
|
processed_frame=frame,
|
||||||
|
timestamp=1000.0,
|
||||||
|
fps=30.0,
|
||||||
|
process_ms=1.0,
|
||||||
|
normal_force=1.5,
|
||||||
|
shear_x=0.25,
|
||||||
|
shear_y=-0.25,
|
||||||
|
normal_force_raw=123.0,
|
||||||
|
shear_x_raw=2.0,
|
||||||
|
shear_y_raw=-2.0,
|
||||||
|
force_calibrated=True,
|
||||||
|
calibrated_axes=("fn", "fx", "fy"),
|
||||||
|
frame_id=0,
|
||||||
|
)
|
||||||
|
kwargs.update(overrides)
|
||||||
|
return SdkPacket(**kwargs)
|
||||||
|
|
||||||
|
|
||||||
|
def test_newton_mode_marks_uncalibrated_axes_unavailable():
|
||||||
|
sample = to_record_sample(_packet(calibrated_axes=("fn",)))
|
||||||
|
display = select_display_forces(units_pref="newton", sample=sample)
|
||||||
|
assert display.unit == "N"
|
||||||
|
assert display.values[0] == pytest.approx(1.5)
|
||||||
|
assert math.isnan(display.values[1]) and math.isnan(display.values[2])
|
||||||
|
assert display.available == (True, False, False)
|
||||||
|
assert display.unavailable_axes() == ("fx", "fy")
|
||||||
|
assert not display.using_raw_fallback
|
||||||
|
|
||||||
|
|
||||||
|
def test_raw_mode_shows_the_raw_counts_for_every_axis():
|
||||||
|
sample = to_record_sample(_packet(calibrated_axes=("fn",)))
|
||||||
|
display = select_display_forces(units_pref="raw", sample=sample)
|
||||||
|
assert display.unit == "raw"
|
||||||
|
assert display.values == (123.0, 2.0, -2.0)
|
||||||
|
assert display.available == (True, True, True)
|
||||||
|
assert not display.using_raw_fallback
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_uncalibrated_device_falls_back_to_raw_and_says_so():
|
||||||
|
sample = to_record_sample(
|
||||||
|
_packet(force_calibrated=False, calibrated_axes=())
|
||||||
|
)
|
||||||
|
display = select_display_forces(units_pref="newton", sample=sample)
|
||||||
|
assert display.unit == "raw"
|
||||||
|
assert display.using_raw_fallback, "the fallback must be announceable"
|
||||||
|
assert display.requested_unit == "N"
|
||||||
|
assert display.values == (123.0, 2.0, -2.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_failed_readout_has_no_available_axis():
|
||||||
|
sample = to_record_sample(
|
||||||
|
_packet(valid=False, invalid_reason="compute failed", stale_forces=True)
|
||||||
|
)
|
||||||
|
for unit in ("N", "raw"):
|
||||||
|
display = select_display_forces(units_pref=unit, sample=sample)
|
||||||
|
assert display.available == (False, False, False)
|
||||||
|
assert all(math.isnan(value) for value in display.values)
|
||||||
|
|
||||||
|
|
||||||
|
def _store_with_one_calibrated_axis() -> SessionStore:
|
||||||
|
store = SessionStore()
|
||||||
|
for index in range(5):
|
||||||
|
store.append(
|
||||||
|
SessionSample(
|
||||||
|
index,
|
||||||
|
index / 30.0,
|
||||||
|
fn_raw=100.0 + index,
|
||||||
|
fx_raw=2.0,
|
||||||
|
fy_raw=-2.0,
|
||||||
|
fn_n=1.0 + index * 0.1,
|
||||||
|
fx_n=NAN,
|
||||||
|
fy_n=NAN,
|
||||||
|
valid=True,
|
||||||
|
calibrated_axes=("fn",),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return store
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_valid_axis_is_drawn_even_when_the_others_have_no_value():
|
||||||
|
"""The all-axes rule hid a good FN because FX/FY were uncalibrated."""
|
||||||
|
slice_ = _store_with_one_calibrated_axis().data()
|
||||||
|
times, values, mask = slice_.plottable("N")
|
||||||
|
assert mask.shape == (5, 3)
|
||||||
|
assert mask[:, 0].all(), "the calibrated axis was hidden"
|
||||||
|
assert not mask[:, 1].any()
|
||||||
|
assert not mask[:, 2].any()
|
||||||
|
|
||||||
|
fn_times, fn_values = slice_.plottable_axis("N", 0)
|
||||||
|
assert fn_times.size == 5 == fn_values.size
|
||||||
|
assert float(fn_values.max()) == pytest.approx(1.4)
|
||||||
|
fx_times, _fx_values = slice_.plottable_axis("N", 1)
|
||||||
|
assert fx_times.size == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_raw_triplet_is_still_complete_for_the_same_samples():
|
||||||
|
slice_ = _store_with_one_calibrated_axis().data()
|
||||||
|
for index in range(3):
|
||||||
|
times, values = slice_.plottable_axis("raw", index)
|
||||||
|
assert times.size == 5
|
||||||
|
assert np.isfinite(values).all()
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_invalid_sample_is_excluded_from_every_axis():
|
||||||
|
store = _store_with_one_calibrated_axis()
|
||||||
|
store.append(SessionSample(9, 9.0, fn_n=NAN, valid=False))
|
||||||
|
slice_ = store.data()
|
||||||
|
_times, _values, mask = slice_.plottable("N")
|
||||||
|
assert not mask[-1].any(), "an invalid sample reached a drawing path"
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
"""Long-history browsing: interval reads, envelopes and one curve interface.
|
||||||
|
|
||||||
|
The replay used to load every row of a recording before drawing anything, so a
|
||||||
|
multi-hour session could not be shown at all. These tests pin the two things that
|
||||||
|
make it possible — an interval read bounded by the interval, and a whole-session
|
||||||
|
envelope built while streaming the file once — and that the charts can read from
|
||||||
|
either a session store or a recording through the same interface.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from orisys_app_shared.acquisition.sdk_worker import SdkPacket
|
||||||
|
from orisys_app_shared.force.session import SessionSample, SessionStore
|
||||||
|
from orisys_app_shared.force.source import RecordingCurveSource, StoreCurveSource
|
||||||
|
from orisys_app_shared.record import ExperimentRecorder, RecordConfig, RecordingReader
|
||||||
|
from orisys_app_shared.record.index import RecordingIndex
|
||||||
|
|
||||||
|
SCRATCH = Path(__file__).resolve().parent / "_scratch_history"
|
||||||
|
|
||||||
|
ROWS = 12_000
|
||||||
|
SPIKE_ROW = 9_000
|
||||||
|
SPIKE_VALUE = 999.0
|
||||||
|
|
||||||
|
|
||||||
|
def _root() -> Path:
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
SCRATCH.mkdir(parents=True, exist_ok=True)
|
||||||
|
return SCRATCH
|
||||||
|
|
||||||
|
|
||||||
|
def _packet(index: int) -> SdkPacket:
|
||||||
|
frame = np.zeros((4, 4, 3), np.uint8)
|
||||||
|
fx = SPIKE_VALUE if index == SPIKE_ROW else 0.5
|
||||||
|
return SdkPacket(
|
||||||
|
raw_frame=frame,
|
||||||
|
processed_frame=frame,
|
||||||
|
timestamp=1000.0 + index / 30.0,
|
||||||
|
mono_t=index / 30.0,
|
||||||
|
fps=30.0,
|
||||||
|
process_ms=1.0,
|
||||||
|
normal_force=1.0,
|
||||||
|
shear_x=fx,
|
||||||
|
shear_y=-0.5,
|
||||||
|
normal_force_raw=10.0 + (index % 7) * 0.01,
|
||||||
|
shear_x_raw=fx,
|
||||||
|
shear_y_raw=-0.5,
|
||||||
|
force_unit="N",
|
||||||
|
force_calibrated=True,
|
||||||
|
frame_id=index,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _recording() -> Path:
|
||||||
|
"""A recording of ``ROWS`` rows with no drops: the writer is allowed to keep up."""
|
||||||
|
import time
|
||||||
|
|
||||||
|
root = _root()
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
directory = recorder.start(root=root, session_id=1, metadata={})
|
||||||
|
for index in range(ROWS):
|
||||||
|
recorder.on_packet(_packet(index))
|
||||||
|
if index % 256 == 0:
|
||||||
|
# Feeding 12000 rows in a tight loop would (correctly) overflow the
|
||||||
|
# ordered stream and be recorded as gaps; this test is about history
|
||||||
|
# browsing, so let the writer drain.
|
||||||
|
deadline = time.monotonic() + 5.0
|
||||||
|
while recorder._queue.qsize() > 512 and time.monotonic() < deadline:
|
||||||
|
time.sleep(0.001)
|
||||||
|
report = recorder.stop()
|
||||||
|
assert report.stats.dropped_data == 0, report.problems
|
||||||
|
assert report.stats.frames == ROWS
|
||||||
|
return Path(directory)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_index_reads_only_the_interval_it_is_asked_for():
|
||||||
|
directory = _recording()
|
||||||
|
index = RecordingIndex(directory)
|
||||||
|
index.build()
|
||||||
|
assert index.rows == ROWS
|
||||||
|
assert index.duration_s() == pytest.approx((ROWS - 1) / 30.0, abs=1e-6)
|
||||||
|
|
||||||
|
# The whole range keeps every row, including the very first one.
|
||||||
|
whole = index.read_rows(0, ROWS)
|
||||||
|
assert len(whole) == ROWS
|
||||||
|
assert whole.frame_ids[0] == 0
|
||||||
|
assert whole.frame_ids[-1] == ROWS - 1
|
||||||
|
|
||||||
|
start, stop = index.row_range(300.0, 300.5)
|
||||||
|
# One row before the interval is included on purpose: a cursor exactly on a
|
||||||
|
# sample still wants the row it follows.
|
||||||
|
assert 15 <= stop - start <= 20, (start, stop)
|
||||||
|
window = index.read_rows(start, stop)
|
||||||
|
assert 15 <= len(window) <= 20
|
||||||
|
assert window.times[0] >= 300.0 - 1 / 30.0
|
||||||
|
assert window.times[-1] <= 300.5
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_large_recording_is_loaded_as_a_marked_decimated_view():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=500)
|
||||||
|
info = reader.info()
|
||||||
|
assert info.decimated, "a 12000-row recording was loaded in full"
|
||||||
|
assert info.total_rows == ROWS
|
||||||
|
assert len(info.times) <= 500
|
||||||
|
# The duration comes from the frame index, never from the decimated view.
|
||||||
|
assert info.duration_s == pytest.approx((ROWS - 1) / 30.0, abs=0.05)
|
||||||
|
assert len(info.frames) == ROWS
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_small_recording_is_not_decimated():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=50_000)
|
||||||
|
info = reader.info()
|
||||||
|
assert not info.decimated
|
||||||
|
assert len(info.times) == ROWS
|
||||||
|
assert info.total_rows == ROWS
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_interval_read_sees_every_row_including_a_single_peak():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=500)
|
||||||
|
spike_t = SPIKE_ROW / 30.0
|
||||||
|
window = reader.window(spike_t - 0.05, spike_t + 0.05)
|
||||||
|
assert window.times.size >= 3
|
||||||
|
assert float(np.nanmax(window.value_for_unit("N")[:, 1])) == pytest.approx(SPIKE_VALUE)
|
||||||
|
# ...and it is the *exact* interval, not the decimated whole-session view.
|
||||||
|
assert not window.decimated
|
||||||
|
assert window.times[0] >= spike_t - 0.1
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_decimated_window_still_keeps_the_peak():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=500)
|
||||||
|
window = reader.window(0.0, (ROWS - 1) / 30.0, max_points=300, unit="N")
|
||||||
|
assert window.decimated
|
||||||
|
assert len(window) <= 600
|
||||||
|
assert float(np.nanmax(window.value_for_unit("N")[:, 1])) == pytest.approx(SPIKE_VALUE)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_envelope_covers_the_whole_session_in_few_points():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=500)
|
||||||
|
overview = reader.overview(unit="N", buckets=200)
|
||||||
|
assert 0 < len(overview) <= 200
|
||||||
|
assert not np.isnan(overview.highs[:, 1]).all()
|
||||||
|
# The spike is inside a bucket the strip would otherwise skip.
|
||||||
|
assert float(np.nanmax(overview.highs[:, 1])) == pytest.approx(SPIKE_VALUE)
|
||||||
|
assert float(overview.counts.sum()) > 0
|
||||||
|
# Cached: asking again does not re-stream the file.
|
||||||
|
assert reader.overview(unit="N", buckets=200) is overview
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_two_sources_answer_the_same_questions():
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=ROWS + 1)
|
||||||
|
recording = RecordingCurveSource(reader)
|
||||||
|
|
||||||
|
store = SessionStore()
|
||||||
|
for index in range(ROWS):
|
||||||
|
sample = SessionSample(
|
||||||
|
index,
|
||||||
|
index / 30.0,
|
||||||
|
fn_raw=10.0 + (index % 7) * 0.01,
|
||||||
|
fx_raw=SPIKE_VALUE if index == SPIKE_ROW else 0.5,
|
||||||
|
fy_raw=-0.5,
|
||||||
|
fn_n=1.0,
|
||||||
|
fx_n=SPIKE_VALUE if index == SPIKE_ROW else 0.5,
|
||||||
|
fy_n=-0.5,
|
||||||
|
)
|
||||||
|
store.append(sample)
|
||||||
|
memory = StoreCurveSource(store)
|
||||||
|
|
||||||
|
for source in (recording, memory):
|
||||||
|
first, last = source.extent()
|
||||||
|
assert first == pytest.approx(0.0, abs=1e-6)
|
||||||
|
assert last == pytest.approx((ROWS - 1) / 30.0, abs=1e-6)
|
||||||
|
assert source.duration_s() == pytest.approx((ROWS - 1) / 30.0, abs=1e-6)
|
||||||
|
assert recording.kind == "disk" and memory.kind == "memory"
|
||||||
|
|
||||||
|
# The same interval, read from disk and from memory, agrees on the values.
|
||||||
|
# (The disk read also includes the row just before the interval.)
|
||||||
|
disk = recording.window(200.0, 210.0, unit="N")
|
||||||
|
cached = memory.window(200.0, 210.0, unit="N")
|
||||||
|
inside = (disk.times >= 200.0) & (disk.times <= 210.0)
|
||||||
|
assert inside.sum() == len(cached) > 0
|
||||||
|
assert np.allclose(disk.values[inside], cached.values, equal_nan=True)
|
||||||
|
# The CSV keeps six decimals; the in-memory store keeps the raw float.
|
||||||
|
assert np.allclose(disk.times[inside], cached.times, atol=1e-6)
|
||||||
|
|
||||||
|
# ...and so do the statistics of that interval (the disk read may include the
|
||||||
|
# one row that precedes it).
|
||||||
|
a = recording.statistics(200.0, 210.0, unit="N")
|
||||||
|
b = memory.statistics(200.0, 210.0, unit="N")
|
||||||
|
assert abs(a.samples - b.samples) <= 1
|
||||||
|
assert a.peaks[1] == pytest.approx(b.peaks[1])
|
||||||
|
assert a.means[1] == pytest.approx(b.means[1], abs=1e-6)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_whole_session_can_be_read_without_holding_it_all():
|
||||||
|
"""The point of the index: peak value with a bounded number of rows held."""
|
||||||
|
directory = _recording()
|
||||||
|
reader = RecordingReader(directory, max_rows=500)
|
||||||
|
overview = reader.overview(unit="N", buckets=100)
|
||||||
|
# What a whole-session strip needs is one band per bucket, not 12000 rows.
|
||||||
|
assert len(overview) <= 100
|
||||||
|
assert len(reader.info().times) <= 500
|
||||||
|
assert reader.info().total_rows == ROWS
|
||||||
|
# A single sample can still be read exactly, on demand.
|
||||||
|
spike_t = SPIKE_ROW / 30.0
|
||||||
|
exact = reader.window(spike_t, spike_t + 1 / 30.0)
|
||||||
|
assert 1 <= len(exact) <= 3, len(exact)
|
||||||
|
assert float(np.nanmax(exact.newtons[:, 1])) == pytest.approx(SPIKE_VALUE)
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
"""The module boundaries the responsibility split is supposed to create.
|
||||||
|
|
||||||
|
A pure move is only worth anything if the dependency directions actually change:
|
||||||
|
the finger worker must not need the box worker to describe a packet, the session
|
||||||
|
file format must not need the sampling state machine, and the models must not
|
||||||
|
import either. These assertions are cheap and they fail loudly if someone
|
||||||
|
re-introduces the cycle.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ast
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SHARED = Path(__file__).resolve().parents[1] / "src"
|
||||||
|
ACQUISITION = SHARED / "orisys_app_shared" / "acquisition"
|
||||||
|
FORCE = SHARED / "orisys_app_shared" / "force"
|
||||||
|
RECORD = SHARED / "orisys_app_shared" / "record"
|
||||||
|
|
||||||
|
|
||||||
|
def _imports(path: Path) -> set[str]:
|
||||||
|
"""Every module name imported by ``path`` (``from x import y`` counts as x)."""
|
||||||
|
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||||
|
found: set[str] = set()
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, ast.Import):
|
||||||
|
for alias in node.names:
|
||||||
|
found.add(alias.name)
|
||||||
|
elif isinstance(node, ast.ImportFrom):
|
||||||
|
if node.module:
|
||||||
|
found.add(node.module)
|
||||||
|
# ``from . import models`` also names the submodule.
|
||||||
|
for alias in node.names:
|
||||||
|
found.add(f"{node.module}.{alias.name}" if node.module else alias.name)
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
def _imports_any(path: Path, needles) -> bool:
|
||||||
|
found = _imports(path)
|
||||||
|
for needle in needles:
|
||||||
|
if any(entry == needle or entry.endswith(f".{needle}") for entry in found):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_packet_model_does_not_import_a_worker():
|
||||||
|
assert not _imports_any(
|
||||||
|
ACQUISITION / "models.py", {"sdk_worker", "finger_worker"}
|
||||||
|
), "acquisition.models depends on a worker"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_finger_worker_takes_the_packet_from_models():
|
||||||
|
"""It used to import the box worker just to describe a frame."""
|
||||||
|
source = (ACQUISITION / "finger_worker.py").read_text(encoding="utf-8")
|
||||||
|
tree = ast.parse(source)
|
||||||
|
from_sdk_worker: set[str] = set()
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, ast.ImportFrom) and (node.module or "").endswith("sdk_worker"):
|
||||||
|
from_sdk_worker.update(alias.name for alias in node.names)
|
||||||
|
assert "SdkPacket" not in from_sdk_worker, "the finger worker still borrows the packet"
|
||||||
|
assert "FixedCameraMode" not in from_sdk_worker
|
||||||
|
assert "load_fixed_mode" not in from_sdk_worker
|
||||||
|
# ...and the helpers it genuinely shares still come from there.
|
||||||
|
assert "process_chain" in from_sdk_worker
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_calibration_models_do_not_import_the_sampler_or_the_session_file():
|
||||||
|
for path in (FORCE / "models.py",):
|
||||||
|
assert not _imports_any(
|
||||||
|
path, {"calibration_recorder", "calibration_session", "session"}
|
||||||
|
), f"{path.name} depends on a higher layer"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_session_format_does_not_import_the_sampling_state_machine():
|
||||||
|
assert not _imports_any(
|
||||||
|
FORCE / "calibration_session.py", {"calibration_recorder"}
|
||||||
|
), "the session file format still depends on the sampler"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_moved_models_are_the_same_objects_everywhere():
|
||||||
|
from orisys_app_shared.acquisition import finger_worker, models, sdk_worker
|
||||||
|
from orisys_app_shared.force import calibration_recorder, calibration_session, models as force_models
|
||||||
|
|
||||||
|
assert sdk_worker.SdkPacket is models.SdkPacket is finger_worker.SdkPacket
|
||||||
|
assert sdk_worker.FixedCameraMode is models.FixedCameraMode
|
||||||
|
assert calibration_recorder.CalibrationPoint is force_models.CalibrationPoint
|
||||||
|
assert calibration_recorder.AxisStats is force_models.AxisStats
|
||||||
|
assert calibration_session.SAMPLE_FIELDS is force_models.SAMPLE_FIELDS
|
||||||
|
# The recorder still offers the trace helpers it used to own.
|
||||||
|
assert callable(calibration_recorder.trace_from)
|
||||||
|
assert callable(calibration_recorder.trace_dt)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_recording_pieces_are_separate_modules():
|
||||||
|
from orisys_app_shared.record import depth, experiment
|
||||||
|
from orisys_app_shared.record.depth_backends import BoxReplay, FingerReplay
|
||||||
|
from orisys_app_shared.record.video_writer import VideoSegments
|
||||||
|
|
||||||
|
# The depth replay no longer defines its backends inline...
|
||||||
|
assert not hasattr(depth, "BoxReplay")
|
||||||
|
assert not hasattr(depth, "_BoxReplay")
|
||||||
|
# ...and the writer delegates video encoding to its own component.
|
||||||
|
assert not hasattr(experiment, "_open_segment")
|
||||||
|
assert callable(VideoSegments.write) and callable(VideoSegments.close)
|
||||||
|
assert BoxReplay.__name__ == "BoxReplay" and FingerReplay.__name__ == "FingerReplay"
|
||||||
@@ -137,6 +137,78 @@ def test_reader_window_and_streaming_agree(recording):
|
|||||||
assert streamed == 61, "streaming must see every row"
|
assert streamed == 61, "streaming must see every row"
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_empty_recording_directory_is_readable():
|
||||||
|
"""A folder with no CSVs must open (empty), not raise an unpack error."""
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
SCRATCH.mkdir(parents=True, exist_ok=True)
|
||||||
|
try:
|
||||||
|
reader = RecordingReader(SCRATCH)
|
||||||
|
info = reader.info()
|
||||||
|
assert info.frames == []
|
||||||
|
assert info.times.size == 0
|
||||||
|
assert info.duration_s == 0.0
|
||||||
|
assert reader.frame_at(1.0) is None
|
||||||
|
assert reader.replay_warnings(sdk_version="0.5.1")
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_new_recording_is_reported_as_time_ordered(recording):
|
||||||
|
info = RecordingReader(recording).info()
|
||||||
|
assert info.frames_out_of_order, "the writer no longer produces ordered rows"
|
||||||
|
assert info.forces_out_of_order
|
||||||
|
codes = RecordingReader(recording).replay_warnings(sdk_version="0.5.1")
|
||||||
|
assert "out_of_order" not in codes
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_old_out_of_order_recording_is_re_sorted_and_reported():
|
||||||
|
"""Rows written by the pre-fix writer are not in time order.
|
||||||
|
|
||||||
|
``frame_at`` binary-searches these times, so an unsorted file would answer
|
||||||
|
with the wrong frame; the reader sorts and says so instead.
|
||||||
|
"""
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
SCRATCH.mkdir(parents=True, exist_ok=True)
|
||||||
|
(SCRATCH / "forces.csv").write_text(
|
||||||
|
"session_id,frame_id,t_rel_s,FN_raw,FX_raw,FY_raw,FN_N,FX_N,FY_N,valid,calibration_id\n"
|
||||||
|
"1,0,0.0,1.0,0.1,0.1,2.0,0.1,0.1,1,h\n"
|
||||||
|
"1,2,0.066,3.0,0.1,0.1,6.0,0.1,0.1,1,h\n" # written before frame 1
|
||||||
|
"1,1,0.033,2.0,0.1,0.1,4.0,0.1,0.1,1,h\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(SCRATCH / "frames.csv").write_text(
|
||||||
|
"frame_id,t_rel_s,acquired_at,segment,segment_frame,in_video,dropped\n"
|
||||||
|
"0,0.0,2026-01-01T00:00:00.000Z,1,1,1,0\n"
|
||||||
|
"2,0.066,2026-01-01T00:00:00.066Z,1,3,1,0\n"
|
||||||
|
"1,0.033,2026-01-01T00:00:00.033Z,1,2,1,0\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(SCRATCH / "session.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"format_version": "1.1",
|
||||||
|
"device": {"sensor_type": "box", "config_ref": "/cfg/sensor.json"},
|
||||||
|
"versions": {"sdk": "0.5.1"},
|
||||||
|
"depth": {"stored": False},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
reader = RecordingReader(SCRATCH)
|
||||||
|
info = reader.info()
|
||||||
|
assert not info.frames_out_of_order
|
||||||
|
assert not info.forces_out_of_order
|
||||||
|
assert [frame.frame_id for frame in info.frames] == [0, 1, 2]
|
||||||
|
assert info.times.tolist() == pytest.approx([0.0, 0.033, 0.066])
|
||||||
|
# The binary search is only valid because the rows were sorted.
|
||||||
|
assert reader.frame_at(0.05).frame_id == 1
|
||||||
|
assert reader.frame_at(0.07).frame_id == 2
|
||||||
|
assert "out_of_order" in reader.replay_warnings(sdk_version="0.5.1")
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(SCRATCH, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
def test_depth_policy_says_recompute_and_warns_about_drift(recording):
|
def test_depth_policy_says_recompute_and_warns_about_drift(recording):
|
||||||
reader = RecordingReader(recording)
|
reader = RecordingReader(recording)
|
||||||
|
|
||||||
|
|||||||
@@ -10,11 +10,9 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import csv
|
import csv
|
||||||
import json
|
import json
|
||||||
import queue
|
|
||||||
import shutil
|
import shutil
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from collections import deque
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
@@ -23,7 +21,11 @@ import pytest
|
|||||||
|
|
||||||
from orisys_app_shared.acquisition.sdk_worker import SdkPacket
|
from orisys_app_shared.acquisition.sdk_worker import SdkPacket
|
||||||
from orisys_app_shared.record import ExperimentRecorder, RecordConfig
|
from orisys_app_shared.record import ExperimentRecorder, RecordConfig
|
||||||
from orisys_app_shared.record.experiment import STATE_FAILED, STATE_STOPPED, _SessionWriter
|
from orisys_app_shared.record.experiment import (
|
||||||
|
STATE_FAILED,
|
||||||
|
STATE_STOPPED,
|
||||||
|
_SessionWriter,
|
||||||
|
)
|
||||||
|
|
||||||
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
SCRATCH = Path(__file__).resolve().parent / "_scratch"
|
||||||
|
|
||||||
@@ -189,21 +191,28 @@ def test_a_full_queue_keeps_the_rows_and_sheds_the_video():
|
|||||||
|
|
||||||
|
|
||||||
def test_a_lost_row_is_recorded_as_an_index_gap():
|
def test_a_lost_row_is_recorded_as_an_index_gap():
|
||||||
"""A hole in the data must be visible, not silently bridged in replay."""
|
"""A hole in the data must be visible, not silently bridged in replay.
|
||||||
|
|
||||||
|
The row is only given up when the ordered stream itself is full; the gap
|
||||||
|
marker then carries the stream position, so it lands in the index where the
|
||||||
|
hole actually is instead of after newer frames.
|
||||||
|
"""
|
||||||
original = _SessionWriter.write_sample
|
original = _SessionWriter.write_sample
|
||||||
calls = {"count": 0}
|
calls = {"count": 0}
|
||||||
|
|
||||||
def slow(self, *args):
|
def slow(self, *args):
|
||||||
calls["count"] += 1
|
calls["count"] += 1
|
||||||
if calls["count"] <= 2:
|
if calls["count"] <= 2:
|
||||||
# Stall just long enough for the queue and the tiny backlog to fill.
|
# Stall just long enough for the stream to pass the (tiny) cap.
|
||||||
time.sleep(0.3)
|
time.sleep(0.3)
|
||||||
return original(self, *args)
|
return original(self, *args)
|
||||||
|
|
||||||
recorder = ExperimentRecorder(RecordConfig(video=False, max_queue=16))
|
recorder = ExperimentRecorder(RecordConfig(video=False, max_queue=16))
|
||||||
with patch.object(_SessionWriter, "write_sample", slow):
|
with patch.object(_SessionWriter, "write_sample", slow):
|
||||||
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
recorder._row_backlog = deque(recorder._row_backlog, maxlen=2)
|
with patch(
|
||||||
|
"orisys_app_shared.record.experiment.ROW_BACKLOG", 2
|
||||||
|
):
|
||||||
for index in range(40):
|
for index in range(40):
|
||||||
recorder.on_packet(_packet(index))
|
recorder.on_packet(_packet(index))
|
||||||
report = recorder.stop()
|
report = recorder.stop()
|
||||||
@@ -218,6 +227,57 @@ def test_a_lost_row_is_recorded_as_an_index_gap():
|
|||||||
assert report.stats.dropped_gaps > 0, "the loss was not marked in the index"
|
assert report.stats.dropped_gaps > 0, "the loss was not marked in the index"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_frame_index_stays_in_time_order_when_rows_are_lost():
|
||||||
|
"""The old writer drained its backlog before the queue, so the index jumped."""
|
||||||
|
original = _SessionWriter.write_sample
|
||||||
|
calls = {"count": 0}
|
||||||
|
|
||||||
|
def slow(self, *args):
|
||||||
|
calls["count"] += 1
|
||||||
|
if calls["count"] <= 3:
|
||||||
|
time.sleep(0.2)
|
||||||
|
return original(self, *args)
|
||||||
|
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=True, max_queue=4))
|
||||||
|
with patch.object(_SessionWriter, "write_sample", slow):
|
||||||
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
with patch("orisys_app_shared.record.experiment.ROW_BACKLOG", 3):
|
||||||
|
for index in range(60):
|
||||||
|
recorder.on_packet(_packet(index))
|
||||||
|
recorder.stop()
|
||||||
|
|
||||||
|
with (Path(directory) / "frames.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
frame_rows = list(csv.DictReader(handle))
|
||||||
|
times = [float(row["t_rel_s"]) for row in frame_rows]
|
||||||
|
assert times == sorted(times), "frames.csv is not in time order"
|
||||||
|
ids = [int(row["frame_id"]) for row in frame_rows]
|
||||||
|
assert ids == sorted(ids), "forces.csv/frames.csv disagree on order"
|
||||||
|
with (Path(directory) / "forces.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
force_rows = list(csv.DictReader(handle))
|
||||||
|
force_times = [float(row["t_rel_s"]) for row in force_rows]
|
||||||
|
assert force_times == sorted(force_times), "forces.csv is not in time order"
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_video_byte_budget_is_released_when_a_frame_is_shed():
|
||||||
|
"""A shed frame must not leak its reservation for the rest of the session."""
|
||||||
|
recorder = ExperimentRecorder(
|
||||||
|
RecordConfig(video=True, max_queue=16, max_queue_bytes=4096)
|
||||||
|
)
|
||||||
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
# Frames of 8x8x3 = 192 bytes, so the budget holds ~21 of them; without the
|
||||||
|
# release the counter would climb past the budget and stay there.
|
||||||
|
for index in range(200):
|
||||||
|
recorder.on_packet(_packet(index))
|
||||||
|
report = recorder.stop()
|
||||||
|
assert report.stats.frames == 200, "rows were lost to the video budget"
|
||||||
|
assert abs(recorder._queued_bytes) < 4096, (
|
||||||
|
f"the byte budget leaked: {recorder._queued_bytes} bytes still reserved"
|
||||||
|
)
|
||||||
|
with (Path(directory) / "forces.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
rows = list(csv.DictReader(handle))
|
||||||
|
assert len(rows) == 200
|
||||||
|
|
||||||
|
|
||||||
def test_session_json_names_the_pre_roll_when_one_was_written():
|
def test_session_json_names_the_pre_roll_when_one_was_written():
|
||||||
frames = [np.full((8, 8, 3), index, np.uint8) for index in range(3)]
|
frames = [np.full((8, 8, 3), index, np.uint8) for index in range(3)]
|
||||||
recorder = ExperimentRecorder(RecordConfig(video=True, fps=5.0))
|
recorder = ExperimentRecorder(RecordConfig(video=True, fps=5.0))
|
||||||
@@ -241,6 +301,80 @@ def test_a_data_only_recording_says_why_it_has_no_pre_roll():
|
|||||||
assert any("pre-roll" in problem for problem in report.problems)
|
assert any("pre-roll" in problem for problem in report.problems)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_stalled_encoder_leaves_a_readable_recording():
|
||||||
|
"""Order, duration, positioning and the byte counter after a stall.
|
||||||
|
|
||||||
|
Stall the writer, push past the ordered stream's cap, let it recover, then
|
||||||
|
push again and read the directory back the way the replay does. The frames
|
||||||
|
acquired *after* the recovery must all be there: an overflow is a temporary
|
||||||
|
loss, not the end of the recording.
|
||||||
|
"""
|
||||||
|
from orisys_app_shared.record import RecordingReader
|
||||||
|
|
||||||
|
original = _SessionWriter.write_sample
|
||||||
|
started = threading.Event()
|
||||||
|
release = threading.Event()
|
||||||
|
calls = {"count": 0}
|
||||||
|
|
||||||
|
def slow(self, *args):
|
||||||
|
calls["count"] += 1
|
||||||
|
if calls["count"] == 2:
|
||||||
|
started.set()
|
||||||
|
release.wait(10.0)
|
||||||
|
return original(self, *args)
|
||||||
|
|
||||||
|
def drained(recorder, timeout: float = 10.0) -> bool:
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if recorder._queue.qsize() == 0:
|
||||||
|
return True
|
||||||
|
time.sleep(0.01)
|
||||||
|
return False
|
||||||
|
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=True, fps=30.0, max_queue=4))
|
||||||
|
with patch.object(_SessionWriter, "write_sample", slow):
|
||||||
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
with patch("orisys_app_shared.record.experiment.ROW_BACKLOG", 3):
|
||||||
|
for index in range(40):
|
||||||
|
recorder.on_packet(_packet(index))
|
||||||
|
assert started.wait(10.0), "the stall never happened"
|
||||||
|
release.set()
|
||||||
|
assert drained(recorder), "the writer never caught up"
|
||||||
|
before = recorder.stats().frames
|
||||||
|
for index in range(40, 100):
|
||||||
|
recorder.on_packet(_packet(index))
|
||||||
|
time.sleep(0.005)
|
||||||
|
assert drained(recorder), "the recovered writer never caught up"
|
||||||
|
report = recorder.stop()
|
||||||
|
|
||||||
|
assert not report.stats.failed, report.problems
|
||||||
|
assert recorder.stats().frames > before, "recording did not continue after the stall"
|
||||||
|
|
||||||
|
reader = RecordingReader(directory)
|
||||||
|
info = reader.info()
|
||||||
|
times = info.times.tolist()
|
||||||
|
ids = [int(value) for value in info.frame_ids]
|
||||||
|
assert times == sorted(times), "forces.csv is not in time order"
|
||||||
|
assert ids == sorted(ids)
|
||||||
|
frame_times = [frame.t_rel_s for frame in info.frames]
|
||||||
|
assert frame_times == sorted(frame_times), "frames.csv is not in time order"
|
||||||
|
|
||||||
|
# Everything acquired after the recovery is present.
|
||||||
|
assert set(range(40, 100)).issubset(set(ids)), (
|
||||||
|
f"frames lost after the encoder recovered: {sorted(set(range(40, 100)) - set(ids))}"
|
||||||
|
)
|
||||||
|
assert report.stats.duration_s == pytest.approx(99 / 30.0, abs=0.05)
|
||||||
|
assert info.duration_s == pytest.approx(99 / 30.0, abs=0.05)
|
||||||
|
|
||||||
|
# Positioning: the frame at 2 s is the frame acquired at 2 s.
|
||||||
|
frame = reader.frame_at(2.0)
|
||||||
|
assert frame is not None and frame.frame_id == 60, frame
|
||||||
|
assert frame.t_rel_s == pytest.approx(2.0, abs=0.05)
|
||||||
|
|
||||||
|
# The byte budget is back to zero once every frame has been consumed.
|
||||||
|
assert recorder._queued_bytes == 0, recorder._queued_bytes
|
||||||
|
|
||||||
|
|
||||||
def test_stopping_twice_returns_the_same_report():
|
def test_stopping_twice_returns_the_same_report():
|
||||||
recorder = ExperimentRecorder(RecordConfig(video=False))
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
recorder.start(root=_root(), session_id=1, metadata={})
|
recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
@@ -249,3 +383,146 @@ def test_stopping_twice_returns_the_same_report():
|
|||||||
second = recorder.stop()
|
second = recorder.stop()
|
||||||
assert first.directory == second.directory
|
assert first.directory == second.directory
|
||||||
assert second.ok
|
assert second.ok
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------ failure handling
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_directory_that_cannot_be_created_leaves_a_retryable_state():
|
||||||
|
"""A failed ``mkdir`` must not strand the recorder in ``starting``."""
|
||||||
|
root = _root()
|
||||||
|
blocker = root / "not-a-directory"
|
||||||
|
blocker.write_text("", encoding="utf-8")
|
||||||
|
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
with pytest.raises(Exception):
|
||||||
|
recorder.start(root=blocker, session_id=1, metadata={})
|
||||||
|
assert recorder.state == STATE_FAILED
|
||||||
|
assert not recorder.is_recording
|
||||||
|
assert recorder.directory is None
|
||||||
|
assert recorder.stats().last_error
|
||||||
|
|
||||||
|
# Retryable: the operator fixes the path and presses record again.
|
||||||
|
directory = recorder.start(root=root, session_id=2, metadata={})
|
||||||
|
recorder.on_packet(_packet(0))
|
||||||
|
report = recorder.stop()
|
||||||
|
assert report.ok, report.problems
|
||||||
|
assert Path(directory).is_dir()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_stop_during_initialization_never_resumes_recording():
|
||||||
|
"""``stop`` that lands while the files open must not be undone afterwards."""
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
original = ExperimentRecorder._write_session_json
|
||||||
|
writing = threading.Event()
|
||||||
|
release = threading.Event()
|
||||||
|
calls = {"count": 0}
|
||||||
|
|
||||||
|
def gated(self, *, final: bool) -> None:
|
||||||
|
calls["count"] += 1
|
||||||
|
if calls["count"] == 1:
|
||||||
|
# This is the writer thread's first metadata write.
|
||||||
|
writing.set()
|
||||||
|
release.wait(5.0)
|
||||||
|
return original(self, final=final)
|
||||||
|
|
||||||
|
with patch.object(ExperimentRecorder, "_write_session_json", gated):
|
||||||
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
recorder.on_packet(_packet(0))
|
||||||
|
assert writing.wait(5.0)
|
||||||
|
report = recorder.stop(wait=False)
|
||||||
|
release.set()
|
||||||
|
thread = recorder._thread
|
||||||
|
if thread is not None:
|
||||||
|
thread.join(5.0)
|
||||||
|
|
||||||
|
# The row accepted before the stop is still written...
|
||||||
|
with (Path(directory) / "forces.csv").open(encoding="utf-8", newline="") as handle:
|
||||||
|
rows = list(csv.DictReader(handle))
|
||||||
|
assert len(rows) == 1, "rows accepted before the stop were dropped"
|
||||||
|
# ...but the recording never went back to "recording", and it is finished.
|
||||||
|
assert not recorder.is_recording
|
||||||
|
assert recorder.state in (STATE_STOPPED, STATE_FAILED)
|
||||||
|
assert not report.completed or report.directory is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_stop_without_waiting_does_not_claim_completion():
|
||||||
|
started = threading.Event()
|
||||||
|
release = threading.Event()
|
||||||
|
original = _SessionWriter.write_sample
|
||||||
|
|
||||||
|
def slow(self, *args):
|
||||||
|
started.set()
|
||||||
|
release.wait(10.0)
|
||||||
|
return original(self, *args)
|
||||||
|
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
with patch.object(_SessionWriter, "write_sample", slow):
|
||||||
|
recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
recorder.on_packet(_packet(0))
|
||||||
|
assert started.wait(10.0)
|
||||||
|
report = recorder.stop(wait=False)
|
||||||
|
try:
|
||||||
|
assert not report.completed
|
||||||
|
assert not report.ok
|
||||||
|
assert recorder.report is None, (
|
||||||
|
"an unfinished stop was cached as the terminal report"
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
release.set()
|
||||||
|
recorder._thread.join(5.0)
|
||||||
|
assert recorder.report is not None and recorder.report.completed
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_failing_flush_reaches_the_state_machine_at_once():
|
||||||
|
"""A flush failure used to live only in the writer's private counter."""
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False, flush_interval_s=0.0))
|
||||||
|
directory = recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
|
||||||
|
class _BrokenHandle:
|
||||||
|
def flush(self):
|
||||||
|
raise OSError("input/output error")
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
return None
|
||||||
|
|
||||||
|
writer = _SessionWriter(
|
||||||
|
directory=Path(directory),
|
||||||
|
config=recorder.config,
|
||||||
|
session_id=1,
|
||||||
|
calibration_id="",
|
||||||
|
on_problem=recorder._record_failure,
|
||||||
|
)
|
||||||
|
writer._forces = (_BrokenHandle(), None)
|
||||||
|
writer.maybe_flush()
|
||||||
|
assert recorder.stats().write_errors == 1, "the flush failure was not reported"
|
||||||
|
assert "flush failed" in recorder.stats().last_error
|
||||||
|
|
||||||
|
for _ in range(4):
|
||||||
|
writer.maybe_flush()
|
||||||
|
assert recorder.stats().failed, "repeated flush failures never failed the record"
|
||||||
|
assert not recorder.is_recording
|
||||||
|
recorder.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_metadata_write_failure_is_part_of_the_report():
|
||||||
|
"""A session.json that cannot be written is part of what happened."""
|
||||||
|
real_write_text = Path.write_text
|
||||||
|
|
||||||
|
def failing_write_text(self, *args, **kwargs):
|
||||||
|
if self.name.endswith(".tmp"):
|
||||||
|
raise OSError("read-only filesystem")
|
||||||
|
return real_write_text(self, *args, **kwargs)
|
||||||
|
|
||||||
|
recorder = ExperimentRecorder(RecordConfig(video=False))
|
||||||
|
with patch.object(Path, "write_text", failing_write_text):
|
||||||
|
recorder.start(root=_root(), session_id=1, metadata={})
|
||||||
|
recorder.on_packet(_packet(0))
|
||||||
|
report = recorder.stop()
|
||||||
|
|
||||||
|
assert report.stats.write_errors >= 1
|
||||||
|
assert not report.ok, "a missing session.json was reported as a good recording"
|
||||||
|
assert any(
|
||||||
|
"read-only" in problem or "write error" in problem for problem in report.problems
|
||||||
|
), report.problems
|
||||||
|
assert report.stats.failed
|
||||||
|
|||||||
Reference in New Issue
Block a user