Browse a long recording from disk instead of loading it into memory

What
- record/index.py: RecordingIndex streams forces.csv once and keeps only the
  small things — one time, one validity flag, one frame id per row plus a byte
  offset every 4096 rows (~6.5 MB per 500k rows). row_range() bisects the times,
  read_rows() seeks to the nearest checkpoint and scans forward, and overview()
  folds every row into its bucket during the same pass to build a per-axis
  min/max envelope.
- read_rows(max_points, unit) decimates **while streaming**: each bucket keeps
  the extreme row of every axis in the requested unit, so a long interval never
  has to exist in memory in full and a shear spike survives a flat normal channel.
- RecordingReader: info() decimates above max_rows (default 200k) and marks
  itself decimated, with total_rows and a duration taken from the frame index
  rather than from the view; window()/window_all()/overview()/statistics() read
  the interval being asked about. list/window semantics unchanged for small
  recordings.
- force/source.py: StoreCurveSource (live memory) and RecordingCurveSource (disk
  interval + cached envelope) implement one interface — extent, window,
  envelope, statistics, duration_s — so ForcePlotsPanel asks a *source* instead
  of knowing where the data lives.
- force_plots: reads through the source (window, envelope, statistics, extent),
  which also replaces its last direct store accesses.
- playback_dialog: draws the whole-session view through window_all() in the unit
  being shown (bounded by CHART_MAX_POINTS), and says so when the view is
  decimated ("显示 N / 共 M 行") instead of letting the operator count points.
- experiment._run: a failure that lands while the files are opening can no longer
  be overwritten by the writer's own transition to "recording" (found by a flaky
  test; the state check now covers STATE_FAILED).

Why
The store releases old blocks above ~4.6 h and the replay loaded every CSV row
into Python lists before drawing anything, so neither the live browser nor the
replay could show an earlier interval of a long recording. The index makes the
disk the data source and memory a cache of what is on screen.

Verification
- shared/tests/test_history_source.py (new, 8 cases) over a 12 000-row recording:
  the index reports its rows and reads a bounded interval; a large recording is
  marked decimated with its true duration and total rows; a small one is not; an
  interval read sees the exact rows including a single peak; a decimated window
  keeps the peak; the envelope fits the whole session in ≤200 points, keeps the
  spike and is cached; the disk and memory sources agree on extent, values and
  statistics; the whole session can be read without holding it all. Shared
  suites: 146 tests, 0 failures (three consecutive runs).
- apps/studio/tools/ui_check.py: the replay section now checks the index row
  count, that an interval read spans only the interval, the envelope size and
  total_rows. 324 checks, 0 failures.
- apps/studio/tests/test_ui_smoke.py: 43 tests, 36 pass; the 7 failures are the
  unchanged sandbox/harness artifacts.
- D:/orisys_ws/_review_phase12/reproduce.py: all ten lines unchanged;
  compileall, locale and prefs parity clean.
- apps/studio/docs/record-channel.md §8 documents the index, the reader API and
  the unified source, and lists what is still hardware work.

Not in this change
- Maximum RSS / backlog / loss measurements at real resolution on the target ARM
  board (needs the hardware), and range CSV export.
This commit is contained in:
ykcheungab
2026-09-13 16:54:26 +08:00
parent 22cc59c402
commit 73592f085f
40 changed files with 6398 additions and 1070 deletions
Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.0 MiB

After

Width:  |  Height:  |  Size: 2.0 MiB

+19 -1
View File
@@ -115,6 +115,8 @@
"box_zoom_tip": "Switch to rubber-band zoom (drag a region on the chart)",
"selection": "Selection {duration}s / {samples} samples — {detail}",
"selection_empty": "No valid samples in the selection",
"value_unavailable": "n/a",
"value_unavailable_tip": "No curve is loaded for {channel}, so it has no value in this unit (raw counts are never shown in its place)",
"browse_only_tip": "Drag or scroll on the chart to browse history"
},
"layout": {
@@ -154,14 +156,21 @@
"depth_state_approximate": "No reset or pre-roll history for this segment: the algorithm state is an approximation",
"depth_warnings": "Recomputation notes: {detail}",
"warnings": "Replay notes: {detail}",
"decimated": "This recording is long, so the curves are a decimated view ({shown} of {total} rows; intervals are read from disk on demand)",
"warn_config_changed": "the config differs from recording time, so the recomputed depth may differ",
"warn_sdk_changed": "the SDK version differs from recording time, so the recomputation may differ",
"warn_config_missing": "no config hash was recorded, so recomputation cannot be verified",
"warn_depth_recomputed": "depth comes from offline recomputation, not from stored data",
"warn_rows_skipped": "some unparsable CSV rows were skipped",
"warn_frames_dropped": "frames were lost to a full queue; only their index rows remain (shown as \"not recorded\")",
"warn_out_of_order": "this recording was written by an older version: its rows are not in time order, so the replay re-sorts them and window edges may differ",
"warn_state_across_segment": "the algorithm state cannot be carried across a segment",
"warn_state_approximate": "the algorithm state is an approximation"
"warn_state_approximate": "the algorithm state is an approximation",
"warn_state_approximate_after_rewind": "the video was reopened to reach an earlier frame, so the state may differ",
"warn_state_too_far_back": "the zeroing is too far back: reading there would exceed the replay budget, so the state is approximate",
"warn_reset_frame_not_indexed": "the zeroing frame is not in the frame index, so the state cannot be rebuilt from it",
"warn_reset_frame_not_in_video": "the zeroing frame never reached the video, so the state cannot be rebuilt from it",
"warn_reset_not_applied": "the replay could not re-apply the zeroing, so the state may differ"
},
"record": {
"label": "Record",
@@ -177,6 +186,8 @@
"start_failed": "Could not start recording: {error}",
"finished": "Recording finished — {frames} frames / {segments} segment(s) in {path}",
"finished_with_problems": "Recording finished with problems: {detail}",
"closing": "Closing the recording files…",
"failed_state": "The recording failed (see the summary and the log)",
"alarm": "Recording problem: {detail}"
},
"cal": {
@@ -204,8 +215,12 @@
"open_record_tip": "Reopen a saved calibration session to inspect every point's samples and diagnostics",
"open_record_filter": "Calibration records (*.json)",
"open_record_ok": "Loaded {path} ({points} point(s))",
"open_record_device": "Recorded on: {device}",
"open_record_warnings": "Note: {detail}",
"open_record_failed": "Could not open the calibration record: {error}",
"apply_mismatch_title": "Calibration does not match the target device",
"apply_mismatch": "This coefficient file belongs to: {device}\nMismatch: {detail}\n\nCurrent target: {sensor_type} / {config}\n\nApplying it would load coefficients that were not measured on this device.",
"apply_mismatch_skipped": "Apply cancelled (the calibration belongs to another device)",
"sn": "SN",
"sn_placeholder": "Serial number (optional)",
"live_raw_label": "Live raw",
@@ -255,11 +270,13 @@
"export_local": "Also wrote {path}",
"export_warnings": "Skipped: {detail}",
"export_failed": "Export failed: {error}",
"export_no_model": "Nothing to export: {detail}",
"apply_title": "Apply calibration",
"apply_confirm": "Apply the calibration just exported to the current device?\n\nSensor type: {sensor_type}\nConfig: {config}\n\nChoosing No keeps the exported file and leaves the running device untouched.",
"apply_skipped": "Exported; not applied to the device",
"apply_pending": "Requested calibration for {sensor_type}; waiting for the device to confirm…",
"applied": "Device confirmed the calibration is in effect (version {version})",
"applied_none": "Device confirmed: no calibration is loaded (the column stays empty)",
"export_only": "Exported coefficients to {path} (not applied to the device)",
"apply_no_target": "No device to apply to (run an acquisition first)"
},
@@ -286,6 +303,7 @@
"video_ended": "Video ended",
"camera_disconnected": "Camera disconnected",
"reset_ok": "Tracker reset",
"reset_pending": "Zeroing requested; waiting for the acquisition thread to confirm…",
"finger_run_unavailable": "Run is not available for this sensor type",
"video_required": "Select a video file first",
"config_missing": "Sensor config not found for {type}",
+19 -1
View File
@@ -115,6 +115,8 @@
"box_zoom_tip": "切换为框选放大(拖动图表框选区域)",
"selection": "选区 {duration}s / {samples} 点 — {detail}",
"selection_empty": "选区无有效样本",
"value_unavailable": "不可用",
"value_unavailable_tip": "{channel} 轴未加载标定曲线,因此没有该单位的数值(不会用原始值冒充)",
"browse_only_tip": "拖动或滚轮即可进入历史浏览"
},
"layout": {
@@ -154,14 +156,21 @@
"depth_state_approximate": "本段缺少归零或预卷历史,算法参考状态为近似重建",
"depth_warnings": "重算提示: {detail}",
"warnings": "回放提示: {detail}",
"decimated": "该录制较长,曲线为降采样视图(显示 {shown} / 共 {total} 行;区间数据按需从磁盘读取)",
"warn_config_changed": "配置文件与录制时不一致,重算出的深度可能与当时不同",
"warn_sdk_changed": "SDK 版本与录制时不一致,重算结果可能不同",
"warn_config_missing": "找不到配置哈希,无法校验重算一致性",
"warn_depth_recomputed": "深度由离线重算得到,不是当时保存的数据",
"warn_rows_skipped": "有无法解析的 CSV 行被跳过",
"warn_frames_dropped": "有帧在队列溢出时丢失,仅保留索引记录(回放显示为“该帧未录制”)",
"warn_out_of_order": "该录制由旧版本写入,行序不是时间序;回放已按时间重排,区间边界可能与当时不同",
"warn_state_across_segment": "重算不能跨分段延续算法状态",
"warn_state_approximate": "算法参考状态为近似重建"
"warn_state_approximate": "算法参考状态为近似重建",
"warn_state_approximate_after_rewind": "为回到更早的帧重新打开了视频,参考状态可能偏离当时",
"warn_state_too_far_back": "归零点过于久远,需要读取的帧数超出上限,已按近似状态重建",
"warn_reset_frame_not_indexed": "归零帧不在帧索引中,无法从该点重建状态",
"warn_reset_frame_not_in_video": "归零帧未写入视频,无法从该点重建状态",
"warn_reset_not_applied": "重放无法重新执行归零,参考状态可能偏离当时"
},
"record": {
"label": "记录",
@@ -177,6 +186,8 @@
"start_failed": "无法开始记录: {error}",
"finished": "记录完成 — {frames} 帧 / {segments} 段,已写入 {path}",
"finished_with_problems": "记录完成,但存在问题: {detail}",
"closing": "正在关闭记录文件…",
"failed_state": "记录已失败(详见摘要与日志)",
"alarm": "记录异常: {detail}"
},
"cal": {
@@ -204,8 +215,12 @@
"open_record_tip": "重新打开已保存的标定会话,检查每个点的完整采样与诊断状态",
"open_record_filter": "标定记录 (*.json)",
"open_record_ok": "已载入 {path}({points} 个点)",
"open_record_device": "记录设备: {device}",
"open_record_warnings": "注意: {detail}",
"open_record_failed": "打开标定记录失败: {error}",
"apply_mismatch_title": "标定与目标设备不匹配",
"apply_mismatch": "该标定文件属于: {device}\n不匹配项: {detail}\n\n当前目标: {sensor_type} / {config}\n\n继续应用会把不适用于本设备的系数写入运行中的设备。",
"apply_mismatch_skipped": "已取消应用(标定身份与目标设备不匹配)",
"sn": "序列号",
"sn_placeholder": "序列号(可选)",
"live_raw_label": "实时原始值",
@@ -255,11 +270,13 @@
"export_local": "同时写入 {path}",
"export_warnings": "跳过: {detail}",
"export_failed": "导出失败: {error}",
"export_no_model": "没有可导出的模型: {detail}",
"apply_title": "应用标定",
"apply_confirm": "将刚导出的标定应用到当前设备?\n\n设备类型: {sensor_type}\n配置: {config}\n\n选择「否」只保留导出文件,不改动运行中的设备。",
"apply_skipped": "已导出,未应用到设备",
"apply_pending": "已请求把标定应用到 {sensor_type},等待设备确认…",
"applied": "设备已确认标定生效(版本 {version})",
"applied_none": "设备已确认:当前没有加载标定(该项留空)",
"export_only": "已导出系数 {path}(未应用到设备)",
"apply_no_target": "没有可应用的设备(请先运行一次采集)"
},
@@ -286,6 +303,7 @@
"video_ended": "视频已结束",
"camera_disconnected": "摄像机已断开",
"reset_ok": "追踪器已重置",
"reset_pending": "已请求归零,等待采集线程确认…",
"finger_run_unavailable": "此传感器类型暂不支持运行",
"video_required": "请先选择视频文件",
"config_missing": "未找到传感器配置: {type}",
+36 -1
View File
@@ -293,9 +293,44 @@ outputs/recordings/<UTC 时间戳>_session<NNNN>/
---
## 8. 尚未完成(第三阶段及以后)
## 8. 长历史浏览:按需读盘而不是整段载入
`record/index.py` 的 `RecordingIndex` 是回放的数据源索引:
- 流式扫一遍 `forces.csv`,只把**小东西**留在内存:每行一个时间(float64)、一个
valid 标志、一个帧号,以及每 `CHECKPOINT_ROWS`(4096)行的字节偏移。500k 行约
6.5 MB(`SessionStore` 的默认上限正好对应 30 Hz 下约 4.6 小时)。
- `row_range(t0, t1)` 用二分定位行区间,`read_rows(start, stop)` 先 `seek` 到最近的
检查点再向前扫描——读一个 10 秒区间就是 10 秒的解析量,不是整段。
区间会额外带上 `t0` 之前的那一行,绘制时才能画出跨过左边界的线段。
- `overview(buckets)` 在**同一次流式扫描**里把每行折进所属桶,得到每桶每轴的
min/max/count。全程色带因此不会漏掉某桶里的尖峰(按行抽样会漏)。
`ReadingReader` 相应变化:
| 方法 | 作用 |
|---|---|
| `info()` | 行数超过 `max_rows`(默认 200k)时给出**降采样视图**,并置 `decimated=True`;`duration_s` 始终取自帧索引,不取自视图 |
| `window(t0, t1, max_points=…)` | 区间精确读取(可再降采样),尖峰按轴保留 |
| `overview(unit, buckets)` | 全程包络,按 (unit, buckets) 缓存 |
| `statistics(t0, t1, unit)` | 只算该区间的统计量 |
`force/source.py` 把两种来源统一成同一接口:`StoreCurveSource`(实时内存块)与
`RecordingCurveSource`(磁盘区间 + 缓存包络),都提供
`extent() / window() / envelope() / statistics() / duration_s()`;`ForcePlotsPanel`
只认这个接口,因此同一套浏览逻辑既能看内存会话也能看磁盘录制。
回放窗口在曲线被降采样时会明确写出「显示 N / 共 M 行」,而不是让操作员自己数点。
> 未做的验收项:真实分辨率与目标 ARM 板上的最大 RSS、积压与丢失测试(需要硬件),
> 以及选中时间段的 CSV 导出。
---
## 9. 尚未完成(第三阶段及以后)
- PyOpenGL 三维显示(`mesh3d_panel.py` 目前仍是占位符,不是已有实现)。
- CSV 的「导出选中时间段」(整段导出已随录制落盘;区间导出待做)。
- 标定的稳定后采样(当前按固定时长采样,稳定性以 σ 呈现)。
- 在目标 ARM 板上用真实分辨率测长时录制的最大 RSS、队列积压与丢失。
+228 -103
View File
@@ -34,10 +34,11 @@ from orisys_app_shared.acquisition.constants import (
)
from orisys_app_shared.force import select_display_forces
from orisys_app_shared.force.calibration_recorder import CalibrationRecorder
from orisys_app_shared.force.session import SessionSample, SessionStore
from orisys_app_shared.force.session import SessionStore
from orisys_app_shared.paths import recordings_dir
from orisys_app_shared.record import ExperimentRecorder, RecordConfig
from orisys_app_shared.record import RecordConfig
from orisys_app_shared.record.sample import to_record_sample
from orisys_studio.recording import RecordingCoordinator, record_config_from_prefs
from orisys_app_shared.i18n import set_locale, t
from orisys_app_shared.logging import report_error
from orisys_app_shared.paths import logo_path, sensor_json_path
@@ -133,20 +134,20 @@ class StudioMainWindow(QMainWindow):
on_finished=self._on_calibration_point,
on_cancelled=self._on_calibration_cancelled,
)
self.recorder = ExperimentRecorder()
self.recorder.set_alarm_handler(self._on_record_alarm)
self._record_alarms: list[str] = []
self._record_last_report = None
self._record_fps = 30.0
# The record channel's mechanics (recorder lifecycle, metadata, the
# per-packet fan-out, confirmed events) live in the coordinator; this
# window keeps the buttons, the menus and the status messages.
self.recording = RecordingCoordinator(
session_store=self.session_store,
cal_recorder=self.cal_recorder,
)
self.recorder = self.recording.recorder
self._session_counter = 0
self._playback = None
self._record_error_count = 0
self._dropped_reported = 0
self._active_calibration_id = ""
#: Last coefficients file written by the calibration panel (not applied).
self._last_calibration_export = ""
#: Confirmed calibration changes, handed over from the acquisition thread.
self._pending_calibration_events: list[tuple[str, str, int]] = []
self._depth_range_value: tuple[float, float] | None = None
self._last_depth_frame = None
@@ -249,14 +250,28 @@ class StudioMainWindow(QMainWindow):
worker.add_sink(self._on_packet_recorded)
# ------------------------------------------------------- experiment record
def _on_record_alarm(self, message: str) -> None:
"""Called from the writer thread. Queue it; never touch Qt here."""
logger.error("Recording alarm: %s", message)
self._record_alarms.append(str(message))
@property
def _record_fps(self) -> float:
return self.recording.record_fps
@property
def _active_calibration_id(self) -> str:
return self.recording.active_calibration
@_active_calibration_id.setter
def _active_calibration_id(self, value: str) -> None:
self.recording.set_active_calibration(value)
@property
def _record_last_report(self):
return self.recording.last_report
@property
def _record_report_pending(self) -> bool:
return self.recording.report_pending
def _drain_record_alarms(self) -> None:
while self._record_alarms:
message = self._record_alarms.pop(0)
for message in self.recording.drain_alarms():
self._set_phase("error")
self.statusBar().showMessage(t("record.alarm", detail=message))
@@ -274,16 +289,13 @@ class StudioMainWindow(QMainWindow):
self.statusBar().showMessage(t("record.requires_run"))
return
mode = self.toolbar.record_mode()
config = RecordConfig(
video=(mode == "video_data"),
lossless_frames=bool(self.prefs.get("record_lossless_frames", False)),
fps=float(self._record_fps),
segment_seconds=float(self.prefs.get("record_segment_seconds", 300.0)),
config = record_config_from_prefs(
self.prefs, mode=mode, fps=float(self._record_fps)
)
try:
directory = self.recorder.start(
directory = self.recording.start(
root=recordings_dir(),
session_id=int(self.session_store.session_id),
config=config,
metadata=self._build_session_metadata(config),
calibration_id=self._active_calibration_id,
calibration_version=self._active_calibration_id,
@@ -297,21 +309,41 @@ class StudioMainWindow(QMainWindow):
self.statusBar().showMessage(t("record.start_failed", error=str(exc)))
logger.exception("Could not start recording")
return
self._record_last_report = None
self.recorder.log_event("record_started", f"mode={mode}")
self.toolbar.set_record_status(Path(directory).name)
self.statusBar().showMessage(t("record.started", mode=t(f"record.mode_{mode}")))
logger.info("Recording started: %s (mode=%s)", directory, mode)
def _stop_recording(self, *, wait: bool = True) -> None:
if not self.recorder.is_recording:
"""Stop the recording and publish what actually happened.
A recording that already failed on its own is **not** a no-op: its report
still has to be consumed, or the button keeps claiming a recording is
running and the failure is never shown.
``wait=False`` returns as soon as the stop is requested; the writer owns
the completion and the UI timer publishes it (see ``_poll_record_report``).
"""
if self.recorder.directory is None and self.recorder.report is None:
# Nothing was ever started: keep the widget state consistent anyway.
self.toolbar.set_record_status("")
self.toolbar.set_record_checked(False)
return
self.recorder.log_event("record_stopped", "")
report = self.recorder.stop(wait=wait)
self._record_last_report = report
self._publish_record_report(self.recording.stop(wait=wait))
def _publish_record_report(self, report) -> None:
"""Update the button and the summary from a recording report."""
self.toolbar.set_record_status("")
self.toolbar.set_record_checked(False)
if report is None:
return
if not report.completed:
# The writer is still closing its files. Never present that as a
# finished experiment; the timer publishes the real report.
self.statusBar().showMessage(t("record.closing"))
return
if report.stats.failed:
self._set_phase("error")
if report.problems:
self.statusBar().showMessage(
t("record.finished_with_problems", detail="; ".join(report.problems))
@@ -326,6 +358,12 @@ class StudioMainWindow(QMainWindow):
)
)
def _poll_record_report(self) -> None:
"""Publish the terminal report once the writer has closed the files."""
report = self.recording.poll()
if report is not None:
self._publish_record_report(report)
def _take_preroll(self) -> list:
"""Frame history from the active worker, if it keeps one.
@@ -333,15 +371,17 @@ class StudioMainWindow(QMainWindow):
recording follows, so a stale history can never be attached to a later
recording.
"""
worker = self._active_worker
take = getattr(worker, "take_preroll", None)
if not callable(take):
return []
try:
return list(take())
except Exception: # noqa: BLE001 - the history is optional, not the record
logger.exception("Could not take the pre-roll buffer")
return []
return self.recording.take_preroll(self._active_worker)
@staticmethod
def _device_serial(info) -> str:
"""The sensor's own serial number, when the SDK exposes one."""
for attribute in ("sn", "serial", "serial_number"):
value = getattr(info, attribute, None)
if value:
return str(value)
serial = getattr(info, "info", None)
return str(getattr(serial, "SN", "") or "")
def _build_session_metadata(self, config: RecordConfig) -> dict:
from orisys_app_shared import API_VERSION, __version__ as shared_version
@@ -379,6 +419,10 @@ class StudioMainWindow(QMainWindow):
# filled in by the acquisition thread's confirmation, so it can
# be empty here; the CSV's per-row version is authoritative.
"version": self._active_calibration_id,
# The operator-entered serial of the *calibrated* device, as the
# calibration panel has it (the device SN when the config has one).
"sn": self.cal_panel.sn_edit.text().strip(),
"device_sn": self._device_serial(info),
},
# Depth is deliberately NOT stored (decided with the team): it is a
# function of the frame and the sensor configuration, so replay
@@ -395,7 +439,10 @@ class StudioMainWindow(QMainWindow):
"conventions": {
"axis_order": ["FN", "FX", "FY"],
"force_unit": "N when calibrated, otherwise raw counts",
"time_base": "t_rel_s is relative to the first recorded frame",
"time_base": (
"t_rel_s is relative to the first recorded frame; an event "
"that predates it has no relative base and is left empty"
),
"frame_id": "monotonic per acquisition session",
},
"filter": {
@@ -432,25 +479,10 @@ class StudioMainWindow(QMainWindow):
def _on_packet_recorded(self, packet) -> None:
"""Runs on the **acquisition thread**. Must stay cheap and Qt-free.
The plots are allowed to drop frames; this path is not. It is also the
reason calibration sampling no longer depends on the calibration panel
being visible.
The packet is converted exactly once, by the same function the CSV writer
uses, so the store and the file can never disagree about whether a value
is a measurement.
The record channel's mechanics are the coordinator's; all this keeps is
the UI's own bookkeeping (how many readouts failed).
"""
record = to_record_sample(
packet, calibration_version=self._active_calibration_id
)
sample = SessionSample.from_record(record)
self.session_store.append(sample)
self.cal_recorder.feed(sample)
# Video + CSV + metadata share this one record path.
self.recorder.on_packet(packet)
fps = float(getattr(packet, "fps", 0.0) or 0.0)
if fps > 0.0:
self._record_fps = fps
record = self.recording.on_packet(packet)
if not record.valid:
self._record_error_count += 1
@@ -919,11 +951,64 @@ class StudioMainWindow(QMainWindow):
self.recorder.log_event("calibration_exported", str(path))
self.statusBar().showMessage(t("cal.export_only", path=Path(path).name))
def calibration_identity(self, path: str) -> dict:
"""What device a coefficient file says it belongs to.
Read with the SDK's own parser so the check sees the same document the
device would load. A file that cannot be read yields no identity, and the
caller then applies it without a mismatch claim.
"""
try:
import orisys.calibration as sdk_calibration
document = sdk_calibration.load(path, strict=True)
except Exception: # noqa: BLE001 - reported by the apply path itself
logger.exception("Could not read calibration identity from %s", path)
return {}
identity = dict(getattr(document, "identity", {}) or {})
pipeline = dict(getattr(document, "pipeline", {}) or {})
return {
"sensor_type": str(identity.get("sensor_type") or ""),
"sn": str(identity.get("sn") or ""),
"config_ref": str(pipeline.get("config_ref") or ""),
}
@staticmethod
def _same_path(first: str, second: str) -> bool:
if not first or not second:
return True # nothing to compare: not a mismatch
if first == second:
return True
try:
return Path(first).resolve() == Path(second).resolve()
except Exception: # noqa: BLE001 - a non-path value simply differs
return False
def calibration_target_mismatch(
self, path: str, *, sensor_type: str, config_ref: str
) -> list[str]:
"""Reasons this calibration does not obviously belong to this device.
Applying coefficients that were measured on a different sensor silently
changes what every recorded newton value means, so the mismatch is stated
before the confirmation, not discovered later.
"""
recorded = self.calibration_identity(path)
reasons: list[str] = []
recorded_type = recorded.get("sensor_type") or ""
if recorded_type and sensor_type and recorded_type != sensor_type:
reasons.append(f"sensor_type: {recorded_type} != {sensor_type}")
recorded_config = recorded.get("config_ref") or ""
if recorded_config and not self._same_path(recorded_config, config_ref):
reasons.append(f"config_ref: {recorded_config} != {config_ref}")
return reasons
def _on_calibration_apply(self, path: str) -> None:
"""Push freshly exported coefficients into the device — on confirmation.
The user is shown the exact target device first, and only the matching
worker is updated.
worker is updated. A calibration that names a *different* device or
configuration is called out explicitly and defaults to "no".
"""
path = str(path or self._last_calibration_export)
if not path:
@@ -931,6 +1016,29 @@ class StudioMainWindow(QMainWindow):
return
sensor_type = self.sensor_picker.sensor_type()
config_ref = self._resolve_config_ref() or ""
reasons = self.calibration_target_mismatch(
path, sensor_type=sensor_type, config_ref=config_ref
)
if reasons:
answer = QMessageBox.question(
self,
t("cal.apply_mismatch_title"),
t(
"cal.apply_mismatch",
device=self.calibration_identity(path).get("sensor_type") or "?",
detail="; ".join(reasons),
sensor_type=sensor_type,
config=config_ref,
),
QMessageBox.Yes | QMessageBox.No,
QMessageBox.No,
)
if answer != QMessageBox.Yes:
self.recorder.log_event(
"calibration_apply_blocked", "; ".join(reasons)
)
self.statusBar().showMessage(t("cal.apply_mismatch_skipped"))
return
answer = QMessageBox.question(
self,
t("cal.apply_title"),
@@ -952,30 +1060,32 @@ class StudioMainWindow(QMainWindow):
t("cal.apply_pending", sensor_type=sensor_type)
)
def _on_calibration_applied(self, path: str, version: str, frame_id: int) -> None:
def _on_calibration_applied(
self, path: str, version: str, frame_id: int, mono_t: float, session: int
) -> None:
"""Called from the acquisition thread once new coefficients are loaded."""
# The acquisition thread must not touch Qt, so only plain state moves here.
self._pending_calibration_events.append(
(str(path), str(version), int(frame_id))
)
self.recording.note_calibration_applied(path, version, frame_id, mono_t, session)
def _on_reset_applied(self, frame_id: int, mono_t: float, session: int) -> None:
"""Called from the acquisition thread once a zeroing took effect."""
self.recording.note_reset(frame_id, mono_t, session)
def _drain_calibration_events(self) -> None:
"""Publish confirmed calibration changes on the UI thread."""
while self._pending_calibration_events:
path, version, frame_id = self._pending_calibration_events.pop(0)
# The version is the calibration's content hash: a path can be
# re-exported with different coefficients, a hash cannot.
self._active_calibration_id = version or path
self.session_store.set_calibration_id(self._active_calibration_id)
self.recorder.set_calibration_version(self._active_calibration_id)
self.recorder.log_event(
"calibration_applied",
f"{path} ({self._active_calibration_id[:12] or 'unhashed'})",
frame_id=frame_id,
)
self.statusBar().showMessage(
t("cal.applied", version=self._active_calibration_id[:12])
)
for event in self.recording.drain_calibration_events():
version = event["version"]
if version:
self.statusBar().showMessage(t("cal.applied", version=version[:12]))
else:
# An empty version is the honest answer for a device with no
# calibration; it must not read as "the last one is still on".
self.statusBar().showMessage(t("cal.applied_none"))
def _drain_reset_events(self) -> None:
"""Publish confirmed zeroings on the UI thread."""
for _event in self.recording.drain_reset_events():
self.statusBar().showMessage(t("status.reset_ok"))
def _worker_for(self, sensor_type: str):
self._ensure_workers()
@@ -1228,7 +1338,6 @@ class StudioMainWindow(QMainWindow):
# starts empty and is filled from that confirmation — a path is not a
# version, and the same path can hold different numbers.
self._active_calibration_id = ""
self._pending_calibration_events.clear()
self._record_error_count = 0
self._dropped_reported = 0
self._depth_range_value = None
@@ -1318,18 +1427,12 @@ class StudioMainWindow(QMainWindow):
if self._session_phase != "running" or self._active_worker is None:
return
try:
# A request, not the zeroing itself: the acquisition thread applies it
# between two frames and reports the frame and time it took effect.
self._active_worker.reset()
self.force_plots.reset()
self._last_plot_packet_key = None
# Recorded with the frame it happened at: a reset clears the SDK's
# tracker, so an offline depth replay can start here instead of
# guessing at the algorithm's earlier state.
self.recorder.log_event(
"zeroing",
"tracker reset",
frame_id=self._current_frame_id(),
)
self.statusBar().showMessage(t("status.reset_ok"))
self.statusBar().showMessage(t("status.reset_pending"))
except Exception as exc:
self.statusBar().showMessage(t("status.run_failed", error=str(exc)))
@@ -1351,6 +1454,11 @@ class StudioMainWindow(QMainWindow):
def _on_run_tick(self) -> None:
if self._active_worker is None:
return
# Record bookkeeping happens before anything that can return early: a
# recording that failed while the device was quiet still has to update
# the button and the summary.
self._drain_record_alarms()
self._poll_record_report()
if self._last_session_error:
err = self._last_session_error
self._last_session_error = ""
@@ -1405,31 +1513,31 @@ class StudioMainWindow(QMainWindow):
if is_new:
self._last_plot_packet_key = packet_key
# Invalid readouts must not be drawn as if they were data, and they
# must not be written into the plot buffer: NaN would poison the
# auto-range and a zero would be indistinguishable from a real one.
if packet.valid:
# The live view consumes the *same* standardised measurement as the
# CSV and the session store: one conversion decides what a value
# means, so the chart cannot show something the record does not
# contain (or vice versa).
record = to_record_sample(packet)
if record.valid:
display = select_display_forces(
units_pref=self.toolbar.units(),
force_calibrated=packet.force_calibrated,
fn=packet.normal_force,
fx=packet.shear_x,
fy=packet.shear_y,
fn_raw=packet.normal_force_raw,
fx_raw=packet.shear_x_raw,
fy_raw=packet.shear_y_raw,
sample=record,
)
if display.using_raw_fallback:
self.statusBar().showMessage(t("status.units_raw_fallback"))
self.force_plots.push(
packet.timestamp,
display.fn,
display.fx,
display.fy,
# Session-relative time, the same base the store and
# ``forces.csv`` use. The wall clock must never reach the
# buffer: it would put the live axis on a different scale.
self._session_time(packet),
display.values[0],
display.values[1],
display.values[2],
unit=display.unit,
available=display.available,
)
else:
self._report_invalid_sample(packet.invalid_reason)
self._report_invalid_sample(record.invalid_reason)
# The calibration panel is a *view*: sampling is driven by the
# record channel, so hiding the panel can no longer stall it.
self.cal_panel.set_live(
@@ -1441,8 +1549,8 @@ class StudioMainWindow(QMainWindow):
# Finalise calibration sampling on the UI thread once its deadline passes.
self.cal_recorder.tick()
self.cal_panel.poll_recorder()
self._drain_record_alarms()
self._drain_calibration_events()
self._drain_reset_events()
self._report_dropped_samples()
mode = self.arrow_panel.view_mode()
@@ -1451,6 +1559,23 @@ class StudioMainWindow(QMainWindow):
if self._show_fps and is_new:
self.arrow_panel.set_fps(packet.fps)
def _session_time(self, packet) -> float:
"""Session-relative acquisition time for the live view.
The session store already defines the origin (its first appended sample),
and ``forces.csv`` uses the same base. Deriving the live time from that
one origin is what keeps the live buffer, a rebuilt history and the replay
on a single X axis.
"""
origin = self.session_store.origin_time()
absolute = getattr(packet, "mono_t", None)
if absolute is None:
absolute = getattr(packet, "timestamp", 0.0)
try:
return float(absolute) - float(origin)
except (TypeError, ValueError):
return 0.0
def _report_invalid_sample(self, reason: str) -> None:
"""Surface failed readouts instead of letting them pass as data.
@@ -0,0 +1,130 @@
"""Calibration service: the panel's operations, without the panel.
The panel shows a table and emits status messages; everything that touches the
file system or a device identity happens here:
* preparing the session once (store, diagnostics and export share one model);
* saving the audit record, reopening one, and exporting the coefficients;
* keeping a reopened record's **own** device identity instead of restamping it
with whatever device happens to be connected.
This is deliberately Qt-free so it can be tested without a display.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Optional
from orisys_app_shared.force.calibration_io import (
CHANNEL_KEYS,
CalSessionStore,
export_force_calibration_v2,
meta_from_sensor_json,
)
from orisys_app_shared.force.calibration_session import (
CalibrationSessionRecord,
read_calibration_session,
session_filename,
write_calibration_session,
)
from orisys_app_shared.paths import calibration_export_dir
logger = logging.getLogger(__name__)
class CalibrationService:
"""Owns the calibration session's files and its prepared model."""
def __init__(self, *, store: Optional[CalSessionStore] = None) -> None:
self.store = store or CalSessionStore()
#: The one prepared model the panel displays and exports.
self.prepared: Any = None
#: Device identity of a reopened record, while it is open.
self.record_identity: Optional[dict] = None
#: The record currently open, if any.
self.open_record: Optional[CalibrationSessionRecord] = None
# --------------------------------------------------------------- identity
def export_meta(
self,
*,
sensor_type: str,
config_ref: str | None,
sn: str = "",
duration_s: float = 0.0,
) -> dict:
"""The identity/pipeline metadata an export would carry.
While a saved record is open its **own** device identity wins: reopening
device A's calibration on a machine connected to device B must keep saying
A, or the evidence is silently restamped with B's serial number.
"""
meta = meta_from_sensor_json(config_ref)
meta["sensor_type"] = str(sensor_type or "box")
meta["sn"] = str(sn or "")
meta["record_duration_s"] = float(duration_s or 0.0)
identity = self.record_identity
if identity:
meta["sensor_type"] = identity.get("sensor_type") or meta["sensor_type"]
meta["sn"] = identity.get("sn") or meta["sn"]
meta["config_ref"] = identity.get("config_ref") or meta.get("config_ref")
meta["source_record"] = str(identity.get("path") or "")
meta["source_created_at"] = str(identity.get("created_at") or "")
return meta
def clear_record_identity(self) -> None:
"""A new run is the live device's context again."""
self.record_identity = None
self.open_record = None
# ------------------------------------------------------------ preparation
def prepare(self, recorder, meta: dict) -> Any:
"""Prepare the session once: store, diagnostics and export share it."""
self.prepared = None
self.store.clear()
if recorder is None:
return None
prepared = recorder.prepared(meta=meta)
self.prepared = prepared
for key in CHANNEL_KEYS:
self.store.set_channel(key, prepared.channel_samples(key))
# The residuals belong to the points, because the table shows them.
for point in recorder.all_points():
value = prepared.residuals.get(id(point))
point.residual_n = float("nan") if value is None else float(value)
return prepared
# ------------------------------------------------------------------- files
def save_record(self, recorder, meta: dict) -> Path:
"""Write the audit record. Returns the written path."""
points = recorder.all_points()
path = calibration_export_dir() / session_filename(meta.get("sensor_type"))
return write_calibration_session(path, points, meta=meta)
def load_record(self, path: str | Path) -> CalibrationSessionRecord:
"""Reopen a saved session and remember whose record it is."""
record = read_calibration_session(path)
identity = record.identity()
self.record_identity = {**identity, "path": str(record.path)}
self.open_record = record
return record
def export_coefficients(
self,
*,
sensor_type: str,
export_dir: str | Path | None = None,
local_bundle_dir: str | Path | None = None,
) -> tuple[Path, Path | None, list[str]]:
"""Serialise the prepared model. Raises ValueError when there is none."""
if self.prepared is None or not getattr(self.prepared, "ok", False):
warnings = list(getattr(self.prepared, "warnings", []) or [])
raise ValueError("; ".join(warnings) or "no complete channels")
return export_force_calibration_v2(
self.prepared,
export_dir=export_dir or calibration_export_dir(),
sensor_type=str(sensor_type or "box"),
local_bundle_dir=local_bundle_dir,
)
+266
View File
@@ -0,0 +1,266 @@
"""The record channel: one experiment directory, one session store, one sampler.
This is the coordination the main window used to do inline — and it is deliberately
free of Qt. Three consumers are fed from the same acquisition packet:
* the :class:`ExperimentRecorder`, which writes the directory;
* the whole-session store the charts and the browser read;
* the calibration sampler (``CalibrationRecorder``), whose lifecycle must not
depend on the calibration panel being visible.
The window keeps the buttons, the menus and the status messages; the decisions
about *what* is recorded, *how* it is named and *what actually happened* live
here, where they can be tested without a display.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Callable, Optional
from orisys_app_shared.record import ExperimentRecorder, RecordConfig, RecordReport
from orisys_app_shared.record.calibration_snapshot import store_calibration_snapshot
from orisys_app_shared.record.sample import to_record_sample
logger = logging.getLogger(__name__)
class RecordingCoordinator:
"""Owns the recorder and the per-packet fan-out."""
def __init__(
self,
*,
session_store,
cal_recorder,
recorder: Optional[ExperimentRecorder] = None,
) -> None:
self.session_store = session_store
self.cal_recorder = cal_recorder
self.recorder = recorder or ExperimentRecorder()
#: Confirmed calibration changes: ``(path, version, frame_id, mono_t, session)``.
self._calibration_events: list[tuple[str, str, int, float, int]] = []
#: Confirmed zeroings: ``(frame_id, mono_t, session)``.
self._reset_events: list[tuple[int, float, int]] = []
#: Raised by the writer thread; drained on the UI thread.
self._alarms: list[str] = []
#: A stop was requested and the writer has not closed its files yet.
self.report_pending = False
self.last_report: Optional[RecordReport] = None
self._active_calibration_id = ""
self._record_fps = 30.0
self.recorder.set_alarm_handler(self._on_alarm)
# ------------------------------------------------------------- acquisition
def on_packet(self, packet):
"""Runs on the **acquisition thread**. Must stay cheap and Qt-free.
The packet is converted exactly once, by the same function the CSV writer
uses, so the store and the file can never disagree about whether a value
is a measurement. Returns the converted record, which the caller may use
for its own (UI-side) bookkeeping.
"""
record = to_record_sample(
packet, calibration_version=self._active_calibration_id
)
from orisys_app_shared.force.session import SessionSample
sample = SessionSample.from_record(record)
self.session_store.append(sample)
self.cal_recorder.feed(sample)
# Video + CSV + metadata share this one record path.
self.recorder.on_packet(packet)
fps = float(getattr(packet, "fps", 0.0) or 0.0)
if fps > 0.0:
self._record_fps = fps
return record
def take_preroll(self, worker) -> list:
"""Frame history from the active worker, if it keeps one."""
take = getattr(worker, "take_preroll", None)
if not callable(take):
return []
try:
return list(take())
except Exception: # noqa: BLE001 - the history is optional, not the record
logger.exception("Could not take the pre-roll buffer")
return []
def set_active_calibration(self, version: str) -> None:
"""The calibration the record should name until the device says otherwise."""
self._active_calibration_id = str(version or "")
@property
def active_calibration(self) -> str:
return self._active_calibration_id
@property
def record_fps(self) -> float:
return self._record_fps
# ------------------------------------------------------------------ events
def note_calibration_applied(
self, path: str, version: str, frame_id: int, mono_t: float, session: int
) -> None:
"""Called from the acquisition thread once coefficients are loaded."""
self._calibration_events.append(
(str(path), str(version), int(frame_id), float(mono_t), int(session))
)
def note_reset(self, frame_id: int, mono_t: float, session: int) -> None:
"""Called from the acquisition thread once a zeroing took effect."""
self._reset_events.append((int(frame_id), float(mono_t), int(session)))
def drain_calibration_events(self) -> list[dict]:
"""Publish confirmed calibrations. Returns what the UI should show."""
published: list[dict] = []
while self._calibration_events:
path, version, frame_id, mono_t, session = self._calibration_events.pop(0)
# The version is the calibration's content hash: a path can be
# re-exported with different coefficients, a hash cannot.
self._active_calibration_id = version
self.session_store.set_calibration_id(version)
self.recorder.set_calibration_version(version)
snapshot = self.snapshot_calibration(path, version, mono_t, frame_id)
self.recorder.log_event(
"calibration_applied",
f"{path} ({version[:12] or 'unhashed'})"
+ (f" -> {snapshot}" if snapshot else ""),
mono_t=mono_t,
frame_id=frame_id,
)
published.append(
{
"path": path,
"version": version,
"frame_id": frame_id,
"mono_t": mono_t,
"session": session,
"snapshot": snapshot,
}
)
return published
def drain_reset_events(self) -> list[dict]:
"""Publish confirmed zeroings."""
published: list[dict] = []
while self._reset_events:
frame_id, mono_t, session = self._reset_events.pop(0)
self.recorder.log_event(
"zeroing",
"tracker reset (applied by the acquisition thread)",
mono_t=mono_t,
frame_id=frame_id,
)
published.append(
{"frame_id": frame_id, "mono_t": mono_t, "session": session}
)
return published
def snapshot_calibration(
self, path: str, version: str, mono_t: float, frame_id: int
) -> str:
"""Copy the applied coefficients into the recording. Returns its name."""
directory = self.recorder.directory
if directory is None or not path or not Path(path).is_file():
return ""
try:
name = store_calibration_snapshot(directory, Path(path), version)
except Exception: # noqa: BLE001 - the record continues without it
logger.exception("Could not snapshot the applied calibration")
return ""
self.recorder.note_calibration_snapshot(
version=version, name=name, frame_id=frame_id, mono_t=mono_t
)
return name
# ------------------------------------------------------------------ alarms
def _on_alarm(self, message: str) -> None:
"""Called from the writer thread. Queue it; never touch Qt here."""
logger.error("Recording alarm: %s", message)
self._alarms.append(str(message))
def drain_alarms(self) -> list[str]:
drained, self._alarms = self._alarms, []
return drained
# --------------------------------------------------------------- lifecycle
def start(
self,
*,
root: Path,
config: RecordConfig,
metadata: dict[str, Any],
calibration_id: str = "",
calibration_version: str = "",
preroll=(),
) -> Path:
"""Start a recording. Raises when the directory cannot be created."""
directory = self.recorder.start(
root=root,
session_id=int(self.session_store.session_id),
metadata=metadata,
calibration_id=calibration_id,
calibration_version=calibration_version,
preroll=preroll,
)
self.last_report = None
self.report_pending = False
return Path(directory)
def stop(self, *, wait: bool = True) -> Optional[RecordReport]:
"""Stop and return the report, or ``None`` when nothing was started."""
if self.recorder.directory is None and self.recorder.report is None:
self.report_pending = False
return None
if self.recorder.is_recording:
self.recorder.log_event("record_stopped", "")
report = self.recorder.stop(wait=wait)
self.publish(report)
return report
def publish(self, report: Optional[RecordReport]) -> None:
"""Record a report; a completed one becomes ``last_report``."""
if report is None:
return
if not report.completed:
# The writer is still closing; the UI timer publishes the real one.
self.report_pending = True
return
self.report_pending = False
self.last_report = report
def poll(self) -> Optional[RecordReport]:
"""The terminal report, once the writer has actually closed the files."""
if not self.report_pending:
return None
report = self.recorder.report
if report is None:
return None
self.publish(report)
return report
@property
def is_recording(self) -> bool:
return self.recorder.is_recording
@property
def directory(self):
return self.recorder.directory
def record_config_from_prefs(
prefs: dict,
*,
mode: str,
fps: float,
on_problem: Optional[Callable[[str], None]] = None,
) -> RecordConfig:
"""Build the record config the toolbar/prefs describe."""
return RecordConfig(
video=(mode == "video_data"),
lossless_frames=bool(prefs.get("record_lossless_frames", False)),
fps=float(fps),
segment_seconds=float(prefs.get("record_segment_seconds", 300.0)),
)
+88 -55
View File
@@ -30,18 +30,16 @@ from orisys_app_shared.force.calibration_io import (
CalSample,
CalSessionStore,
ChannelKey,
export_force_calibration_v2,
meta_from_sensor_json,
)
from orisys_app_shared.force.calibration_session import (
read_calibration_session,
session_filename,
write_calibration_session,
)
from orisys_app_shared.i18n import t
from orisys_app_shared.paths import calibration_export_dir
from orisys_app_shared.theme import color
from orisys_studio.calibration_service import CalibrationService
from orisys_studio.ui.cal_point_dialog import CalibrationPointDialog
CHANNEL_LABELS = ("fn", "fx", "fy")
@@ -71,7 +69,10 @@ class CalPanel(QFrame):
super().__init__(parent)
self.setObjectName("contentPanel")
self._theme = theme
self._store = CalSessionStore()
#: The files, the identity and the prepared model live in the service;
#: this panel shows them and emits status messages.
self._service = CalibrationService()
self._store = self._service.store
self._channel: ChannelKey = "fn"
self._running = False
#: Set by :meth:`bind_recorder`. Sampling lives there, not here, so the
@@ -244,6 +245,14 @@ class CalPanel(QFrame):
self.retranslate()
# ------------------------------------------------------------------ API
@property
def _prepared(self):
return self._service.prepared
@property
def _record_identity(self):
return self._service.record_identity
def set_theme(self, theme: str) -> None:
self._theme = theme
self._render_live_label()
@@ -266,6 +275,8 @@ class CalPanel(QFrame):
def set_meta_context(self, *, sensor_type: str, config_ref: str | None) -> None:
self._sensor_type = str(sensor_type or "box")
self._config_ref = config_ref
# A new run is the live device's context again.
self._service.clear_record_identity()
def set_live(self, fn: float, fx: float, fy: float) -> None:
"""Update the live readout. Sampling is handled by the recorder."""
@@ -519,7 +530,8 @@ class CalPanel(QFrame):
if self._recorder is None:
self.fit_label.setText("")
return
fit = self._recorder.fit(self._channel)
# Same prepared result the store and the export use — never a second one.
fit = self._recorder.fit(self._channel, prepared=self._prepared)
if not fit.usable:
self.fit_label.setText(t("cal.fit_insufficient", count=fit.n_points))
return
@@ -549,12 +561,30 @@ class CalPanel(QFrame):
state = t("cal.state_restore") if point.excluded else t("cal.state_exclude")
self.btn_toggle_excluded.setText(state)
def point_dialog(self, point):
"""The diagnostic view for a point, rebuilt from its samples if needed.
A reopened record carries the full sample series, so the three-axis view
can always be produced — waiting for a cached trace would have made
exactly the reopened points (the ones worth auditing) unopenable.
"""
if point is None or not (point.trace or point.samples):
return None
if not point.trace:
from orisys_app_shared.force.calibration_session import (
restore_diagnostics,
)
restore_diagnostics(point)
if not point.trace:
return None
return CalibrationPointDialog(point, theme=self._theme, parent=self)
def _on_row_activated(self) -> None:
"""Double click a row to inspect that point's three-axis curve."""
point = self._current_point()
if point is None or not point.trace:
return
CalibrationPointDialog(point, theme=self._theme, parent=self).exec_()
dialog = self.point_dialog(self._current_point())
if dialog is not None:
dialog.exec_()
def _on_toggle_excluded(self) -> None:
point = self._current_point()
@@ -580,32 +610,24 @@ class CalPanel(QFrame):
t("cal.marked_validation" if point.is_validation else "cal.unmarked_validation")
)
def _rebuild_store(self) -> None:
"""Rebuild the export store from the points that are still in the fit.
def _export_meta(self) -> dict:
"""The identity/pipeline metadata the export would carry."""
return self._service.export_meta(
sensor_type=self._sensor_type,
config_ref=self._config_ref,
sn=self.sn_edit.text().strip(),
duration_s=float(self.duration_spin.value()),
)
Validation points are held out: they exist to check the model, so letting
them into the build would both corrupt it and make the "independent"
error meaningless. Repeated reference loads are merged rather than
overwritten, so every sample that was actually taken reaches the model.
def _rebuild_store(self) -> None:
"""Prepare the session once: store, diagnostics and export share it.
The panel used to filter, aggregate and build its own model while
``CalibrationRecorder.fit`` prepared a second one for the table, so the
displayed residual and the exported curve could describe different
models. Both now read :class:`PreparedCalibration`, built by the service.
"""
self._store.clear()
if self._recorder is None:
return
for point in self._recorder.all_points():
if point.excluded or point.is_validation:
continue
stats = point.loaded_stats()
if not np.isfinite(stats.mean):
continue
self._store.aggregate(
point.channel,
CalSample(
weight_n=float(point.reference_load_n),
raw_mean=float(stats.mean),
n=int(stats.n),
std=float(stats.std) if np.isfinite(stats.std) else 0.0,
),
)
self._service.prepare(self._recorder, self._export_meta())
def _on_delete_row(self) -> None:
point = self._current_point()
@@ -630,20 +652,24 @@ class CalPanel(QFrame):
self._refresh_table()
def _on_export(self) -> None:
meta = meta_from_sensor_json(self._config_ref)
meta["sensor_type"] = self._sensor_type
meta["sn"] = self.sn_edit.text().strip()
meta["record_duration_s"] = float(self.duration_spin.value())
# Re-prepare first: the SN and the requested duration may have changed,
# and the file must be exactly the model the panel is showing.
self._rebuild_store()
self._refresh_fit_label()
if self._prepared is None or not self._prepared.ok:
QMessageBox.warning(
self,
t("panels.cal"),
t("cal.export_no_model", detail="; ".join(self._warnings())),
)
return
try:
# Write into the user's calibration directory only. Overwriting the
# device bundle under config/sensors/<type>/ is now an explicit,
# separate step so that exporting cannot silently change what the
# next Sensor() will load.
primary, _local_path, warnings = export_force_calibration_v2(
self._store,
export_dir=calibration_export_dir(),
primary, _local_path, warnings = self._service.export_coefficients(
sensor_type=self._sensor_type,
meta=meta,
local_bundle_dir=None,
)
except ValueError as exc:
@@ -664,6 +690,9 @@ class CalPanel(QFrame):
self.calibration_exported.emit(str(primary))
QMessageBox.information(self, t("panels.cal"), msg)
def _warnings(self) -> list[str]:
return list(getattr(self._prepared, "warnings", []) or [])
def _on_apply(self) -> None:
if not self._last_export_path:
self.status_message.emit(t("cal.apply_no_export"))
@@ -678,12 +707,10 @@ class CalPanel(QFrame):
if not points:
self.status_message.emit(t("cal.save_record_empty"))
return
meta = meta_from_sensor_json(self._config_ref)
meta["sensor_type"] = self._sensor_type
meta["sn"] = self.sn_edit.text().strip()
path = calibration_export_dir() / session_filename(self._sensor_type)
# ``_export_meta`` keeps a reopened record's own device identity.
meta = self._export_meta()
try:
written = write_calibration_session(path, points, meta=meta)
written = self._service.save_record(self._recorder, meta)
except Exception as exc: # noqa: BLE001 - reported to the operator
QMessageBox.warning(
self, t("panels.cal"), t("cal.save_record_failed", error=str(exc))
@@ -694,30 +721,36 @@ class CalPanel(QFrame):
self.status_message.emit(msg)
QMessageBox.information(self, t("panels.cal"), msg)
def _on_open_record(self) -> None:
"""Reopen a saved session so it can be inspected again."""
def _on_open_record(self, path: str | None = None) -> None:
"""Reopen a saved session: its points **and** its device identity."""
if self._recorder is None:
return
path, _filter = QFileDialog.getOpenFileName(
self,
t("cal.open_record"),
str(calibration_export_dir()),
t("cal.open_record_filter"),
)
if path is None:
path, _filter = QFileDialog.getOpenFileName(
self,
t("cal.open_record"),
str(calibration_export_dir()),
t("cal.open_record_filter"),
)
if not path:
return
try:
record = read_calibration_session(path)
record = self._service.load_record(path)
except Exception as exc: # noqa: BLE001 - reported to the operator
QMessageBox.warning(
self, t("panels.cal"), t("cal.open_record_failed", error=str(exc))
)
return
# Show the record's own SN: it is what a later save or export will carry,
# so the operator must not be looking at the connected device's number.
if record.identity().get("sn"):
self.sn_edit.setText(record.identity()["sn"])
self._recorder.load_points(record.points, replace=True)
self._rebuild_store()
self._refresh_table()
self._refresh_enabled()
msg = t("cal.open_record_ok", path=record.path.name, points=len(record.points))
msg += " " + t("cal.open_record_device", device=record.describe())
if record.warnings:
msg += "\n" + t("cal.open_record_warnings", detail="; ".join(record.warnings))
self.status_message.emit(msg)
+91 -40
View File
@@ -18,6 +18,7 @@ from PyQt5.QtWidgets import (
from orisys_app_shared.acquisition.constants import FORCE_SAMPLE_RATE_HZ, LOWPASS_CUTOFF_HZ
from orisys_app_shared.force import MetricHistory
from orisys_app_shared.force.history import DEFAULT_MAX_FPS
from orisys_app_shared.force.source import StoreCurveSource
from orisys_app_shared.i18n import t
from orisys_app_shared.theme import color
@@ -100,8 +101,13 @@ class ForcePlotsPanel(QFrame):
self._fixed_newton = True
self._unit = "raw"
self._last_values: tuple[float, float, float] = (0.0, 0.0, 0.0)
#: Per axis: does ``_last_values`` hold a real value in the active unit?
self._available: tuple[bool, bool, bool] = (True, True, True)
# -- browsing state -------------------------------------------------
self._store = None
#: Where the curves read from: the in-memory session store here, a
#: recording on disk in the replay. One interface either way.
self._source = StoreCurveSource()
self._mode = "follow"
self._window: tuple[float, float] = (0.0, 0.0)
self._cursor_t = 0.0
@@ -313,6 +319,7 @@ class ForcePlotsPanel(QFrame):
self.history.reset()
self._clear_curves()
self._last_values = (0.0, 0.0, 0.0)
self._available = (True, True, True)
self._unit = "raw"
self._mode = "follow"
self._window = (0.0, 0.0)
@@ -328,19 +335,33 @@ class ForcePlotsPanel(QFrame):
def push(
self,
timestamp: float,
t_rel_s: float,
fn: float,
fx: float,
fy: float,
*,
unit: str = "raw",
available: tuple[bool, bool, bool] | None = None,
) -> None:
"""Append one live sample.
``t_rel_s`` is **session-relative** time — the same base the session store
and ``forces.csv`` use. Handing the wall clock in here used to put the live
buffer on a different time axis from a rebuilt history, so switching units
(or entering browse) made the X axis jump by the size of the epoch.
``available`` marks axes that have no value in ``unit`` (a device with
only its normal-force axis calibrated). Those keep NaN and show as a gap
and a dash, never as a raw count wearing a newton label.
"""
if unit != self._unit:
# Never mix units in one buffer. Callers switch units through
# ``set_units``/``set_fixed_newton`` which rebuild from the session
# store; a bare change here would silently reinterpret old samples.
self._unit = unit
self._last_values = (float(fn), float(fx), float(fy))
if available is not None:
self._available = tuple(bool(flag) for flag in available)
self._render_value_cards()
if self._mode == "browse":
# Browsing is a display pause, not a capture pause: the record
@@ -351,7 +372,7 @@ class ForcePlotsPanel(QFrame):
# Store the true value. Clamping here (the old ``max(0.0, fn)``) erased
# negative FN from the buffer and hid zero drift; out-of-range values are
# now handled by the axis range, which only affects what is drawn.
self.history.push(timestamp, float(fn), float(fx), float(fy))
self.history.push(t_rel_s, float(fn), float(fx), float(fy))
state = self.history.state()
if state is None:
return
@@ -372,7 +393,9 @@ class ForcePlotsPanel(QFrame):
self.plots[i].setYRange(*state.y_ranges[i], padding=0.0)
finally:
self._syncing = False
self.set_cursor(float(timestamp) - (self._time_origin() or 0.0))
# The buffer already carries session-relative time, so the cursor is the
# time itself: subtracting an origin here would double-count it.
self.set_cursor(float(t_rel_s))
# The overview is a full-store read, so refresh it on a timer rather
# than on every frame.
now = time.perf_counter()
@@ -381,12 +404,6 @@ class ForcePlotsPanel(QFrame):
self._last_overview_at = now
self._sync_overview()
def _time_origin(self) -> float:
"""The store's time origin, so the cursor matches the overview axis."""
if self._store is None:
return 0.0
return self._store.origin_time()
def set_units(self, unit: str, store=None) -> None:
"""Switch display units without mixing units in the curve buffer.
@@ -398,11 +415,13 @@ class ForcePlotsPanel(QFrame):
if unit == self._unit and store is None:
return
self._unit = unit
# ``clear`` keeps the buffer's time origin on purpose: the rebuilt history
# is in the same session-relative base, so the X axis must not restart.
self.history.clear()
self._clear_curves()
self._overview_dirty = True
if store is not None:
self._store = store
self.set_store(store)
if self._mode == "browse":
self.refresh_from_store()
else:
@@ -414,6 +433,7 @@ class ForcePlotsPanel(QFrame):
def set_store(self, store) -> None:
"""Attach the whole-session store the browser reads from."""
self._store = store
self._source.set_store(store)
self._overview_dirty = True
self._refresh_mode_label()
@@ -437,19 +457,19 @@ class ForcePlotsPanel(QFrame):
def show_all(self) -> None:
"""Overview: fit the entire session, in browse mode."""
if self._store is None:
if self._source is None:
return
first, last = self._store.extent()
first, last = self._source.extent()
if last <= first:
return
self._set_mode("browse")
self.set_window(first, last)
def set_window(self, t0: float, t1: float) -> None:
"""Show ``[t0, t1]`` from the whole-session store."""
if self._store is None:
"""Show ``[t0, t1]`` from the whole-session source."""
if self._source is None:
return
first, last = self._store.extent()
first, last = self._source.extent()
if t1 <= t0:
t1 = t0 + max(0.001, self.history.window_sec)
t0 = max(first, float(t0))
@@ -463,21 +483,23 @@ class ForcePlotsPanel(QFrame):
if self._store is None:
return
t0, t1 = self._window
slice_ = self._store.visible(
# One shared rule for every view, decided **per axis**: valid and finite
# for *that* axis. An all-axes rule hid a valid FN whenever the device had
# no shear calibration, and a finiteness-only rule let a stale value
# reappear after another view excluded it.
window = self._source.window(
t0, t1, max_points=self._max_draw_points, unit=self._unit
)
self._syncing = True
try:
if len(slice_) == 0:
if len(window) == 0:
for curve in self.curves:
curve.setData([], [])
else:
# One shared rule for every view: valid *and* finite. A stale
# number left by a failed compute step must not reappear here
# after another view already excluded it.
times, values, _mask = slice_.plottable(self._unit)
for index, curve in enumerate(self.curves):
curve.setData(times, values[:, index])
times, values = window.plottable_axis(index)
curve.setData(times, values)
self._availability_from_window(window)
for plot in self.plots:
plot.setXRange(t0, t1, padding=0.0)
self._apply_axis_mode()
@@ -523,7 +545,7 @@ class ForcePlotsPanel(QFrame):
x0, x1 = float(x_range[0]), float(x_range[1])
if not (x1 > x0):
return
first, last = self._store.extent()
first, last = self._source.extent()
if last <= first:
return
# Keep the window inside the recorded session.
@@ -554,7 +576,7 @@ class ForcePlotsPanel(QFrame):
self.stats_label.setText("")
return
t0, t1 = self._window
stats = self._store.statistics(t0, t1, unit=self._unit)
stats = self._source.statistics(t0, t1, unit=self._unit)
if stats.samples == 0:
self.stats_label.setText(t("chart.selection_empty"))
return
@@ -581,19 +603,19 @@ class ForcePlotsPanel(QFrame):
def _sync_overview(self) -> None:
if self._store is None:
return
first, last = self._store.extent()
first, last = self._source.extent()
if last <= first:
self.overview_curve.setData([], [])
return
# The overview is a full-store read, so only rebuild it when it can have
# changed (window/unit/session) rather than on every pan.
slice_ = self._store.visible(
first, last, max_points=self._overview_points(), unit=self._unit
# The overview is a whole-session read, so only rebuild it when it can
# have changed (window/unit/session) rather than on every pan.
window = self._source.envelope(
buckets=self._overview_points(), unit=self._unit
)
values = slice_.value_for_unit(self._unit)
values = np.asarray(window.values, dtype=float)
column = values[:, 0]
finite = np.isfinite(column)
self.overview_curve.setData(slice_.times[finite], column[finite])
self.overview_curve.setData(window.times[finite], column[finite])
self.overview.setXRange(first, last, padding=0.0)
if self._mode == "browse":
self.overview_region.setRegion(self._window)
@@ -616,20 +638,25 @@ class ForcePlotsPanel(QFrame):
self.btn_overview.setEnabled(self._store is not None)
def _reload_from_store(self, store) -> None:
"""Re-populate the visible buffer from the whole-session store."""
"""Re-populate the visible buffer from the session source."""
window = self.history.window_sec
slice_ = store.tail(window) if window > 0 else store.data()
if len(slice_) == 0:
first, last = self._source.extent()
span = window if window > 0 else max(0.0, last - first)
data = self._source.window(
max(first, last - span), last, unit=self._unit
)
if len(data) == 0:
return
values = slice_.value_for_unit(self._unit)
for index in range(len(slice_)):
if not bool(slice_.valid[index]):
self._availability_from_window(data)
values = np.asarray(data.values, dtype=float)
for index in range(len(data)):
if not bool(data.valid[index]):
continue
row = values[index]
if not all(np.isfinite(v) for v in row):
continue
self.history.push(
float(slice_.times[index]),
float(data.times[index]),
float(row[0]),
float(row[1]),
float(row[2]),
@@ -641,6 +668,19 @@ class ForcePlotsPanel(QFrame):
curve.setData(state.times, state.values[:, i])
self.plots[i].setXRange(*state.x_range, padding=0.0)
def _availability_from_window(self, window) -> None:
"""Mark which axes have a value in the active unit, from a read window.
Used after a unit switch: the value cards must not keep showing the
previous unit's numbers, and an axis with no curve in this unit has to be
a dash until a fresh sample arrives.
"""
if window is None or len(window) == 0:
return
values = np.asarray(window.values, dtype=float)
finite = np.isfinite(values) & np.asarray(window.valid, dtype=bool)[:, None]
self._available = tuple(bool(finite[:, index].any()) for index in range(3))
def _uses_fixed_y(self) -> bool:
"""Fixed Y whenever we have limits for the active unit.
@@ -690,10 +730,21 @@ class ForcePlotsPanel(QFrame):
def _render_value_cards(self) -> None:
fmt = "{value:.3f}" if self._unit == "N" else "{value:.1f}"
for value_label, unit_label, value in zip(
self.value_labels, self.value_units, self._last_values
unavailable = t("chart.value_unavailable")
for index, (value_label, unit_label, value) in enumerate(
zip(self.value_labels, self.value_units, self._last_values)
):
if not self._available[index]:
# An axis with no curve in this unit: a dash, never a raw count
# presented as a newton value.
value_label.setText(unavailable)
unit_label.setText("")
value_label.setToolTip(
t("chart.value_unavailable_tip", channel=CHANNEL_LABELS[index])
)
continue
value_label.setText(fmt.format(value=value))
value_label.setToolTip("")
unit_label.setText(unit_display(self._unit))
self.unit_badge.setText(unit_display(self._unit))
@@ -10,6 +10,7 @@ video frame, the depth pane and the charts' cursor line together.
from __future__ import annotations
import logging
import threading
from pathlib import Path
@@ -45,6 +46,10 @@ except ImportError as exc: # pragma: no cover
CHANNEL_TOKENS = ("plot_fn", "plot_fx", "plot_fy")
SLIDER_STEPS = 2000
#: Upper bound on the points handed to the replay curves.
CHART_MAX_POINTS = 4000
logger = logging.getLogger(__name__)
class PlaybackDialog(QDialog):
@@ -75,6 +80,13 @@ class PlaybackDialog(QDialog):
self._depth_pending_seq: int | None = None
self._depth_pending_position: float | None = None
self._depth_result_seq: int | None = None
#: Which recording each answer belongs to (incremented on every open).
self._record_serial = 0
self._depth_pending_record: int | None = None
self._depth_result_record: int | None = None
self._depth_result_frame = -1
#: Processors whose owning request was still running when it was retired.
self._retired_recomputers: list = []
self._depth_poll = QTimer(self)
self._depth_poll.setInterval(120)
self._depth_poll.timeout.connect(self._poll_depth)
@@ -166,9 +178,16 @@ class PlaybackDialog(QDialog):
self.warning_label.setText(t("playback.open_failed", error=str(exc)))
def open_recording(self, directory: Path) -> None:
if self._recomputer is not None:
self._recomputer.close()
self._recomputer = None
"""Open a recording. Any depth answer still in flight is invalidated."""
# A running request belongs to the *previous* recording: its result must
# never be pasted into the new one, and its processor must be closed by
# the thread that is using it, not here.
self._retire_recomputer()
self._record_serial += 1
self._depth_request_seq += 1
self._depth_pending_seq = None
self._depth_pending_position = None
self._depth_error = None
self._reader = RecordingReader(directory)
self._recomputer = DepthRecomputer(self._reader)
info = self._reader.info()
@@ -188,19 +207,48 @@ class PlaybackDialog(QDialog):
self._depth_result_seq = None
self.set_position(0.0)
def _retire_recomputer(self) -> None:
"""Hand the current processor to its worker, or close it here if idle.
Closing a processor that a worker thread is still reading is a use-after-
free; only the thread that owns it may close it, so a busy one is marked
for the worker to close when it finishes.
"""
recomputer, self._recomputer = self._recomputer, None
if recomputer is None:
return
if self._depth_busy():
self._retired_recomputers.append(recomputer)
else:
try:
recomputer.close()
except Exception: # noqa: BLE001 - closing is best effort
logger.exception("Could not close the depth recomputer")
def _refresh_warnings(self) -> None:
if self._reader is None:
return
info = self._reader.info()
_ok, sdk_version = DepthRecomputer.sdk_available()
# The reader compares the recorded config hash against the config file as
# it is now, which is what the depth recomputation would actually use.
codes = self._reader.replay_warnings(sdk_version=sdk_version)
if not codes:
self.warning_label.setText("")
return
self.warning_label.setText(
t("playback.warnings", detail=", ".join(t(f"playback.warn_{c}") for c in codes))
)
text = ""
if codes:
text = t(
"playback.warnings",
detail=", ".join(t(f"playback.warn_{c}") for c in codes),
)
if info.decimated:
# The curves are a view, not every sample: say so rather than let the
# operator count points.
note = t(
"playback.decimated",
shown=len(info.times),
total=info.total_rows,
)
text = f"{text} {note}" if text else note
self.warning_label.setText(text)
# ------------------------------------------------------------------ drawing
def _render_curves(self) -> None:
@@ -211,11 +259,21 @@ class PlaybackDialog(QDialog):
for curve in self.curves:
curve.setData([], [])
return
# Same rule as the live charts: invalid or non-finite points are not
# drawn, whichever view is asking.
times, drawn, _mask = info.plottable(self._unit)
# Same rule as the live charts, per axis: a point is drawn only where the
# sample is valid and *that* axis is finite, so a single-axis calibration
# does not hide the axis that is calibrated. For a long recording the
# whole-session view is decimated while it is read from disk, in the unit
# being shown.
window_all = getattr(self._reader, "window_all", None)
if callable(window_all):
window = window_all(unit=self._unit, max_points=CHART_MAX_POINTS)
for index, curve in enumerate(self.curves):
times, values = window.plottable_axis(self._unit, index)
curve.setData(times, values)
return
for index, curve in enumerate(self.curves):
curve.setData(times, drawn[:, index])
times, values = info.plottable_axis(self._unit, index)
curve.setData(times, values)
self.plot_widget.ci.layout.setRowStretchFactor(index, 1)
self.plot_widget.getItem(index, 0).setXRange(
float(info.times[0]), float(info.times[-1]), padding=0.02
@@ -307,22 +365,40 @@ class PlaybackDialog(QDialog):
self._depth_error = None
self.depth_label.setText(t("playback.depth_running"))
position = self._position
# Bind the answer to this request: a result that arrives after the cursor
# moved is discarded rather than pasted onto the new position.
# Bind the answer to this request: the recording, the frame and a request
# sequence. A result that arrives after the cursor moved — or after
# another recording was opened — is discarded rather than pasted on.
recomputer = self._recomputer
record = self._record_serial
frame = self._current_frame()
self._depth_request_seq += 1
seq = self._depth_request_seq
self._depth_pending_seq = seq
self._depth_pending_position = position
self._depth_pending_record = record
def work() -> None:
# Runs off the UI thread and touches no Qt object.
# Runs off the UI thread and touches no Qt object. It captures the
# processor it was given: the dialog may already be showing another
# recording by the time this returns.
try:
result = self._recomputer.depth_at(position)
result = recomputer.depth_at(position)
except BaseException as exc: # noqa: BLE001 - reported via the poll
self._depth_error = exc
self._depth_error = (record, exc)
return
finally:
if recomputer in self._retired_recomputers:
# Retired while this request was running: the owning thread
# closes it, not the UI thread.
self._retired_recomputers.remove(recomputer)
try:
recomputer.close()
except Exception: # noqa: BLE001
logger.exception("Could not close a retired recomputer")
self._depth_result = result
self._depth_result_seq = seq
self._depth_result_record = record
self._depth_result_frame = getattr(frame, "frame_id", -1)
self._depth_thread = threading.Thread(
target=work, name="depth-replay", daemon=True
@@ -337,15 +413,22 @@ class PlaybackDialog(QDialog):
return
self._depth_poll.stop()
pending = self._depth_pending_seq
record = self._depth_pending_record
self._depth_pending_seq = None
self._depth_pending_position = None
self._depth_pending_record = None
if self._depth_error is not None:
error_record, exc = self._depth_error
self._depth_error = None
if error_record != self._record_serial:
# The failure belongs to a recording that is no longer open.
return
self._depth_ok = False
self.depth_label.setPixmap(QPixmap())
self.depth_label.setText(
t("playback.depth_failed", detail=str(self._depth_error))
)
self._depth_error = None
self.depth_label.setText(t("playback.depth_failed", detail=str(exc)))
return
if record != self._record_serial:
# Another recording was opened while this was running.
return
result = self._depth_result
if result is None or self._depth_result_seq != pending:
@@ -468,10 +551,18 @@ class PlaybackDialog(QDialog):
def closeEvent(self, event) -> None: # noqa: N802
self._depth_poll.stop()
# A depth recomputation that is blocked inside the SDK cannot be
# cancelled; it is a daemon thread, so it does not keep the process
# alive and the window always closes.
if self._recomputer is not None and not self._depth_busy():
self._recomputer.close()
# cancelled; it is a daemon thread, so it does not keep the process alive
# and the window always closes. The processor is closed by whichever
# thread owns it (see ``_retire_recomputer``), never by the UI thread
# while a worker is reading it.
for recomputer in list(self._retired_recomputers):
if not self._depth_busy():
self._retired_recomputers.remove(recomputer)
try:
recomputer.close()
except Exception: # noqa: BLE001
logger.exception("Could not close a retired recomputer")
self._retire_recomputer()
super().closeEvent(event)
def _depth_busy(self) -> bool:
+531 -2
View File
@@ -289,7 +289,7 @@ def test_cal_panel_records_and_lists_samples(make_window):
]
def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40) -> None:
def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40):
"""Drive one point through the recorder and into the panel's table."""
from orisys_app_shared.force.session import SessionSample
@@ -305,7 +305,9 @@ def _add_calibration_point(window, *, load: float, raw: float, samples: int = 40
valid=True,
)
)
window.cal_panel.on_record_finished(window.cal_recorder.finish_now())
point = window.cal_recorder.finish_now()
window.cal_panel.on_record_finished(point)
return point
def test_calibration_diagnostics_drive_the_fit(make_window):
@@ -357,6 +359,64 @@ def test_calibration_diagnostics_drive_the_fit(make_window):
model = build_export_model({key: store.channel(key) for key in ("fn", "fx", "fy")})
assert abs(predict_channel(model, "fn", 400.0) - 2.0) > 0.05
# The store, the label and the export are the same preparation: the panel
# must not derive the model twice and show numbers for one of them.
prepared = window.cal_panel._prepared
assert prepared is not None and prepared.ok
assert list(store.channel("fn")) == list(prepared.samples["fn"])
assert prepared.points_in_fit("fn") == 3
def test_the_panel_shows_the_numbers_of_the_file_it_exports(make_window, record_root):
"""Displayed diagnostics must match the written coefficients.
A repeated reference load makes this sharp: the two samples of one load must
be merged before the build, so the exported curve and the panel's residual
and validation figures all come from that single prepared model.
"""
import json
from pathlib import Path
import orisys.calibration as sdk_calibration
from orisys_app_shared.force.calibration_io import build_force_calibration_v2
window = _show(make_window(), (1400, 880))
window.cal_panel.set_running(True)
window.cal_panel.set_meta_context(sensor_type="box", config_ref=None)
_add_calibration_point(window, load=1.0, raw=100.0)
_add_calibration_point(window, load=2.0, raw=200.0)
_add_calibration_point(window, load=2.0, raw=240.0) # a deliberate repeat
_add_calibration_point(window, load=4.0, raw=400.0)
_add_calibration_point(window, load=3.0, raw=290.0)
panel = window.cal_panel
panel.table.setCurrentCell(4, 0)
panel._on_toggle_validation()
panel._rebuild_store()
panel._refresh_fit_label()
prepared = panel._prepared
assert prepared is not None and prepared.ok
assert len(prepared.samples["fn"]) == 3, "the repeated load was not merged"
# Write the document the export would write, load it back with the SDK and
# check that its predictions are the prepared ones.
document, _warnings = build_force_calibration_v2(prepared)
path = Path(record_root) / "force_calibration.json"
path.write_text(json.dumps(document), encoding="utf-8")
loaded = sdk_calibration.load(path, strict=True)
for point in window.cal_recorder.points:
raw = float(point.loaded_stats().mean)
predicted = float(loaded.get("fn").convert(raw))
assert predicted == pytest.approx(prepared.predict("fn", raw), abs=1e-9)
# ...and the label describes that same model.
label = panel.fit_label.text()
assert "分段线性" in label or "piecewise" in label, label
assert "3" in label
assert prepared.max_validation_error("fn") > 0.0
def test_calibration_point_dialog_shows_three_axes(make_window):
window = _show(make_window(), (1400, 880))
@@ -373,6 +433,137 @@ def test_calibration_point_dialog_shows_three_axes(make_window):
dialog.close()
def _write_session_file(root, *, sensor_type, sn, config_ref, points):
import json
from orisys_app_shared.force.calibration_session import build_session_document
document = build_session_document(
points,
meta={"sensor_type": sensor_type, "sn": sn, "config_ref": config_ref},
)
path = Path(root) / f"calibration_session_{sensor_type}.json"
path.write_text(json.dumps(document), encoding="utf-8")
return path
def test_opening_device_a_record_keeps_device_a_identity(make_window, record_root):
"""Device A's record must not be restamped with device B's identity."""
import json
from unittest.mock import patch as _patch
import orisys_studio.ui.cal_panel as cal_panel_module
from orisys_app_shared.force.calibration_io import build_force_calibration_v2
window = _show(make_window(), (1400, 880))
panel = window.cal_panel
# A record taken on device A, with a full audit series.
session_points = [
_add_calibration_point(window, load=1.0, raw=100.0),
_add_calibration_point(window, load=3.0, raw=300.0),
]
source = _write_session_file(
record_root,
sensor_type="box",
sn="SN-A",
config_ref="D:/lab/deviceA/sensor.json",
points=session_points,
)
window.cal_recorder.clear_points()
# ...and the live context is device B.
panel.set_meta_context(sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json")
panel.sn_edit.setText("SN-B")
panel._on_open_record(str(source))
assert panel.sn_edit.text() == "SN-A", "the record's SN was overwritten"
meta = panel._export_meta()
assert meta["sensor_type"] == "box"
assert meta["sn"] == "SN-A"
assert meta["config_ref"] == "D:/lab/deviceA/sensor.json"
assert meta["source_record"].endswith(source.name)
# Saving the record again still says A.
import orisys_studio.calibration_service as cal_service_module
with _patch.object(cal_panel_module, "calibration_export_dir", lambda: Path(record_root)):
with _patch.object(
cal_service_module, "calibration_export_dir", lambda: Path(record_root)
):
with _patch.object(cal_panel_module.QMessageBox, "information", lambda *a, **k: None):
panel._on_save_record()
saved = sorted(Path(record_root).glob("calibration_session_box_*.json"))
assert saved, "the record was not saved under device A's type"
saved_meta = json.loads(saved[-1].read_text(encoding="utf-8"))["meta"]
assert saved_meta["sn"] == "SN-A"
assert saved_meta["sensor_type"] == "box"
assert saved_meta["config_ref"] == "D:/lab/deviceA/sensor.json"
# ...and so does the exported coefficient document.
document, _warnings = build_force_calibration_v2(panel._prepared)
assert document["meta"]["sensor_type"] == "box"
assert document["meta"]["sn"] == "SN-A"
assert document["meta"]["config_ref"] == "D:/lab/deviceA/sensor.json"
def test_a_reopened_point_can_be_inspected_from_its_samples(make_window, record_root):
"""A record with samples but no cached trace still opens its diagnostics."""
window = _show(make_window(), (1400, 880))
panel = window.cal_panel
points = [
_add_calibration_point(window, load=1.0, raw=100.0),
_add_calibration_point(window, load=2.0, raw=200.0),
]
import json
from orisys_app_shared.force.calibration_session import build_session_document
document = build_session_document(points, meta={"sensor_type": "box"})
for entry in document["points"]:
entry["trace"] = {} # only the audit series survived
entry["trace_dt_s"] = 0.0
source = Path(record_root) / "no_trace.json"
source.write_text(json.dumps(document), encoding="utf-8")
panel._on_open_record(str(source))
reopened = window.cal_recorder.points[0]
assert reopened.samples, "the reopened point lost its samples"
dialog = panel.point_dialog(reopened)
assert dialog is not None, "the reopened point could not be inspected"
assert len(dialog.plot_widget.ci.items) == 3
dialog.close()
def test_applying_another_devices_calibration_is_flagged(make_window, record_root):
"""The apply path states a sensor/config mismatch instead of silently applying."""
import json
from orisys_app_shared.force.calibration_io import build_export_model, write_force_calibration
from orisys_app_shared.force.calibration_io import CalSample, CalSessionStore
window, stuck = _running_window(make_window)
store = CalSessionStore()
store.aggregate("fn", CalSample(weight_n=0.0, raw_mean=0.0, n=10))
store.aggregate("fn", CalSample(weight_n=5.0, raw_mean=500.0, n=10))
model = build_export_model({key: store.channel(key) for key in ("fn", "fx", "fy")})
document = model.to_dict()
document["identity"] = {"sensor_type": "box", "sn": "SN-A"}
document["pipeline"] = {"config_ref": "D:/lab/deviceA/sensor.json"}
path = Path(record_root) / "deviceA_calibration.json"
path.write_text(json.dumps(document), encoding="utf-8")
# The window is connected to a finger device.
mismatch = window.calibration_target_mismatch(
str(path), sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json"
)
assert any("sensor_type" in reason for reason in mismatch), mismatch
assert any("config_ref" in reason for reason in mismatch), mismatch
assert not window.calibration_target_mismatch(
str(path), sensor_type="box", config_ref="D:/lab/deviceA/sensor.json"
)
_finish(window, stuck)
def test_force_plots_render_demo_data(make_window):
window = _show(make_window(), (1440, 880))
for i in range(60):
@@ -502,6 +693,344 @@ def test_history_keeps_negative_fn(make_window):
_finish(window, stuck)
def test_the_live_axis_is_session_relative_and_survives_a_unit_switch(make_window):
"""The live buffer, the store and a rebuilt history share one time base.
Feeding the wall clock into the plot put the live axis ~1.7e9 s away from the
session store's, so a unit switch (which rebuilds from the store) jumped the
X axis and the chart looked like it had lost its history.
"""
wall = 1_700_000_000.0
window, stuck = _running_window(make_window)
window.toolbar.units_combo.setCurrentIndex(0) # newton
window.force_plots.reset()
for index in range(90):
stuck.emit(
ui_check._fake_packet(
index, timestamp=wall + index / 30.0, mono_t=index / 30.0
)
)
window._on_run_tick()
before = window.force_plots.history.state()
assert before is not None
end_before = float(before.times[-1])
assert end_before < 60.0, (
f"the live X axis is not session-relative: it ends at {end_before}"
)
assert abs(end_before - 89 / 30.0) < 0.5, end_before
for units_index in (1, 0, 1, 0): # raw -> N -> raw -> N
window.toolbar.units_combo.setCurrentIndex(units_index)
after = window.force_plots.history.state()
assert after is not None, "the unit switch emptied the rebuilt history"
assert abs(float(after.times[-1]) - end_before) < 1.0, (
f"the X axis jumped across a unit switch: {end_before} -> {after.times[-1]}"
)
_finish(window, stuck)
def test_a_single_calibrated_axis_is_shown_and_the_others_are_unavailable(make_window):
"""Only FN calibrated: FN in newtons, FX/FY explicitly unavailable."""
import math
from orisys_app_shared.i18n import t
window, stuck = _running_window(make_window)
window.toolbar.units_combo.setCurrentIndex(0) # newton
window.force_plots.reset()
for index in range(10):
stuck.emit(
ui_check._fake_packet(
index,
calibrated_axes=("fn",),
shear_n=float("nan"),
)
)
window._on_run_tick()
plots = window.force_plots
assert plots._available == (True, False, False), plots._available
assert plots.value_labels[0].text() != t("chart.value_unavailable")
for index in (1, 2):
assert plots.value_labels[index].text() == t("chart.value_unavailable"), (
"an axis with no curve must not show a number"
)
state = plots.history.state()
assert math.isfinite(float(state.values[:, 0].max()))
assert not any(math.isfinite(v) for v in state.values[:, 1]), (
"FX carried a value in newton mode without a curve"
)
_finish(window, stuck)
def test_a_slow_close_is_not_reported_as_a_finished_recording(make_window):
"""Stopping must not present an unclosed file as a finished experiment."""
import threading as _threading
from unittest.mock import patch as _patch
from orisys_app_shared.i18n import t
from orisys_app_shared.record.experiment import _SessionWriter
window, stuck = _running_window(make_window)
window.toolbar.btn_record.setChecked(True)
stuck.emit(ui_check._fake_packet(0))
window._on_run_tick()
started = _threading.Event()
release = _threading.Event()
original = _SessionWriter.write_sample
def slow(self, *args):
started.set()
release.wait(10.0)
return original(self, *args)
with _patch.object(_SessionWriter, "write_sample", slow):
stuck.emit(ui_check._fake_packet(1))
window._on_run_tick()
assert started.wait(5.0), "the writer never picked up the frame"
window._stop_recording(wait=False)
assert window.recorder.report is None, "the stop claimed a finished recording"
assert window._record_report_pending, "the pending report was not tracked"
assert window._record_last_report is None
assert not window.toolbar.btn_record.isChecked(), (
"the record button still looks active while the file closes"
)
assert window.statusBar().currentMessage() == t("record.closing")
release.set()
deadline = __import__("time").monotonic() + 10.0
while window.recorder.report is None and __import__("time").monotonic() < deadline:
__import__("time").sleep(0.01)
window._on_run_tick()
report = window._record_last_report
assert report is not None and report.completed, "the terminal report was never published"
assert not window._record_report_pending
_finish(window, stuck)
def test_a_failed_recording_is_consumed_even_when_it_is_not_running(make_window):
"""A recording that died on its own still has a report to publish."""
import time as _time
from unittest.mock import patch as _patch
from orisys_app_shared.record.experiment import _SessionWriter
window, stuck = _running_window(make_window)
window.toolbar.btn_record.setChecked(True)
assert window.recorder.is_recording
with _patch.object(_SessionWriter, "write_sample", side_effect=OSError("disk full")):
for index in range(8):
stuck.emit(ui_check._fake_packet(index))
window._on_run_tick()
deadline = _time.monotonic() + 3.0
while window.recorder.stats().write_errors < 5 and _time.monotonic() < deadline:
_time.sleep(0.01)
assert window.recorder.stats().failed
assert not window.recorder.is_recording
# ``is_recording`` is already False: the stop must still consume the report.
window._stop_recording()
report = window._record_last_report
assert report is not None and report.stats.failed
assert not window.toolbar.btn_record.isChecked()
assert window._session_phase == "error"
_finish(window, stuck)
def test_the_depth_request_belongs_to_the_recording_it_started_in(make_window, record_root):
"""Opening another recording must invalidate the request still running."""
import threading as _threading
import time as _time
from PyQt5.QtWidgets import QApplication
from orisys_app_shared.record.depth import DepthResult
from orisys_studio.ui.playback_dialog import PlaybackDialog
window = _show(make_window(), (1200, 700))
dialog = PlaybackDialog(theme="dark", directory=Path(record_root))
first_serial = dialog._record_serial
started = _threading.Event()
release = _threading.Event()
closed: list[str] = []
class _Blocking:
def preflight(self) -> str:
return ""
def depth_at(self, _position, **_kwargs):
started.set()
release.wait(10.0)
return DepthResult(
depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0, record_id="old"
)
def close(self) -> None:
closed.append("closed")
blocking = _Blocking()
dialog._recomputer = blocking
dialog._on_recompute_depth()
assert started.wait(5.0), "the depth request never started"
# Another recording is opened while the request is in flight.
dialog.open_recording(Path(record_root))
assert dialog._record_serial == first_serial + 1
assert dialog._recomputer is not blocking, "the processor was not replaced"
assert closed == [], "the UI thread closed a processor a worker was reading"
release.set()
deadline = _time.monotonic() + 10.0
while not closed and _time.monotonic() < deadline:
QApplication.processEvents()
_time.sleep(0.01)
assert closed == ["closed"], "the retired processor was never closed"
assert dialog._depth_thread is not None and not dialog._depth_thread.is_alive()
# The answer belongs to the previous recording, so it is dropped, not drawn.
dialog._poll_depth()
QApplication.processEvents()
assert dialog._depth_ok is False
pixmap = dialog.depth_label.pixmap()
assert pixmap is None or pixmap.isNull(), "an answer for another recording was drawn"
dialog.close()
window.close()
def test_a_busy_depth_processor_is_not_closed_by_the_ui_thread(make_window, record_root):
import threading as _threading
import time as _time
from PyQt5.QtWidgets import QApplication
from orisys_app_shared.record.depth import DepthResult
from orisys_studio.ui.playback_dialog import PlaybackDialog
window = _show(make_window(), (1200, 700))
dialog = PlaybackDialog(theme="dark", directory=Path(record_root))
started = _threading.Event()
release = _threading.Event()
closed: list[str] = []
class _Blocking:
def preflight(self) -> str:
return ""
def depth_at(self, _position, **_kwargs):
started.set()
release.wait(10.0)
return DepthResult(depth=np.ones((2, 2)), frame_id=0, t_rel_s=0.0)
def close(self) -> None:
closed.append("closed")
dialog._recomputer = _Blocking()
dialog._on_recompute_depth()
assert started.wait(5.0)
dialog.close() # closing the window must not free a processor in use
QApplication.processEvents()
assert closed == [], "the window closed a processor that was still reading"
release.set()
deadline = _time.monotonic() + 10.0
while not closed and _time.monotonic() < deadline:
QApplication.processEvents()
_time.sleep(0.01)
assert closed == ["closed"], "the owning thread never closed the processor"
window.close()
def test_a_confirmed_calibration_is_snapshotted_into_the_recording(make_window, record_root):
"""The recording keeps the coefficients themselves, not only their hash."""
import json
from pathlib import Path
import orisys_studio.app_window as app_module
from orisys_app_shared.i18n import t
window, stuck = _running_window(make_window)
app_module.recordings_dir = lambda: record_root
window.toolbar.btn_record.setChecked(True)
assert window.recorder.is_recording
coefficients = Path(record_root) / "force_calibration_box.json"
coefficients.write_text('{"unit": "N", "channels": {}}', encoding="utf-8")
version = "deadbeefcafe1234"
window._on_calibration_applied(
str(coefficients), version, 21, 0.7, int(window.session_store.session_id)
)
window._drain_calibration_events()
directory = window.recorder.directory
assert directory is not None
snapshots = sorted(path.name for path in (directory / "calibration").iterdir())
assert snapshots == [f"force_calibration_{version[:12]}.json"], snapshots
assert (directory / "calibration" / snapshots[0]).read_text(
encoding="utf-8"
) == '{"unit": "N", "channels": {}}'
assert window._active_calibration_id == version
assert window.session_store.calibration_id == version
window._stop_recording()
meta = json.loads((directory / "session.json").read_text(encoding="utf-8"))
entries = meta["calibration_snapshots"]
assert entries and entries[0]["version"] == version
assert entries[0]["file"] == f"calibration/{snapshots[0]}"
assert entries[0]["frame_id"] == 21
event = next(e for e in meta["events"] if e["kind"] == "calibration_applied")
assert event["frame_id"] == 21 and event["mono_t"] == pytest.approx(0.7)
assert snapshots[0] in event["detail"]
_finish(window, stuck)
def test_a_confirmed_absence_of_calibration_does_not_keep_the_old_version(make_window):
"""An empty version is the honest answer; it must not fall back to the path."""
from orisys_app_shared.i18n import t
window = _show(make_window(), (1400, 880))
window._active_calibration_id = "previous-hash"
window._on_calibration_applied("/tmp/whatever.json", "", 5, 0.2, 1)
window._drain_calibration_events()
assert window._active_calibration_id == "", (
"an uncalibrated device kept claiming the previous coefficients"
)
assert window.statusBar().currentMessage() == t("cal.applied_none")
def test_a_zeroing_is_recorded_only_after_the_worker_confirms_it(make_window, record_root):
import json
import orisys_studio.app_window as app_module
window, stuck = _running_window(make_window)
app_module.recordings_dir = lambda: record_root
window.toolbar.btn_record.setChecked(True)
stuck.emit(ui_check._fake_packet(3))
window._on_run_tick()
window._on_reset()
assert stuck.reset_requests == 1, "the zeroing was not requested"
# The stub confirms from the acquisition side, as a real device does.
stuck.on_reset_applied = window._on_reset_applied
window._on_reset()
window._on_run_tick()
window._stop_recording()
directory = window.recorder.directory
meta = json.loads((directory / "session.json").read_text(encoding="utf-8"))
zeroing = [e for e in meta["events"] if e["kind"] == "zeroing"]
assert len(zeroing) == 1, f"unexpected zeroing events: {zeroing}"
assert zeroing[0]["frame_id"] == 3, zeroing[0]
assert zeroing[0]["mono_t"] is not None
_finish(window, stuck)
def test_depth_view_reports_missing_data_and_fixes_its_scale(make_window):
import numpy as np
+578 -7
View File
@@ -144,6 +144,10 @@ def _fake_packet(
valid: bool = True,
depth=None,
reason: str = "",
calibrated_axes=None,
timestamp: float | None = None,
mono_t: float | None = None,
shear_n: float | None = None,
):
"""A packet shaped like the real worker's, for the record-channel checks."""
import numpy as np
@@ -157,17 +161,19 @@ def _fake_packet(
return SdkPacket(
raw_frame=blank,
processed_frame=blank,
timestamp=1000.0 + frame_id / 30.0,
timestamp=1000.0 + frame_id / 30.0 if timestamp is None else timestamp,
mono_t=mono_t,
fps=30.0,
process_ms=1.0,
normal_force=(2.0 + frame_id * 0.01) if valid else nan,
shear_x=0.5 if valid else nan,
shear_x=(0.5 if shear_n is None else shear_n) if valid else nan,
shear_y=-0.5 if valid else nan,
normal_force_raw=(9.0 + frame_id * 0.1) if valid else nan,
shear_x_raw=0.4 if valid else nan,
shear_y_raw=-0.4 if valid else nan,
force_unit="N",
force_calibrated=True,
calibrated_axes=tuple(calibrated_axes or ()),
session_id=1,
frame_id=frame_id,
valid=valid,
@@ -194,6 +200,27 @@ class _StuckWorker(RecordSinkMixin):
self.stop_blocked_for = 0.0
#: Released by ``release()`` so daemon threads do not outlive the run.
self._release = threading.Event()
#: Calibration requests this worker was asked to apply.
self.calibration_requests: list[str] = []
#: Zeroing requests, and the confirmation callback the window installs.
self.reset_requests = 0
self.on_reset_applied = None
def set_pending_calibration(self, path) -> None:
self.calibration_requests.append(str(path))
def reset(self) -> None:
"""Request a zeroing; a real device confirms it from its own thread.
This stub confirms immediately (a fast device), so the window's event path
— request, confirmation, recorded event — is exercised end to end.
"""
self.reset_requests += 1
if callable(self.on_reset_applied):
packet = self._latest
frame_id = int(getattr(packet, "frame_id", -1))
mono_t = float(getattr(packet, "mono_t", 0.0) or 0.0)
self.on_reset_applied(frame_id, mono_t, 1)
def start(self, _source, *, config_ref=None) -> None:
self._release.clear()
@@ -229,9 +256,6 @@ class _StuckWorker(RecordSinkMixin):
reason, self._end_reason = self._end_reason, None
return reason
def reset(self) -> None:
return None
def stop(self, *, wait: bool = False) -> None:
self._end_reason = self._end_reason or "stopped"
started = time.perf_counter()
@@ -666,8 +690,10 @@ def main() -> int:
)
from orisys_app_shared.force.calibration_io import (
build_export_model,
build_force_calibration_v2 as _build_v2,
predict_channel,
)
import orisys.calibration as _sdk_calibration
store = window.cal_panel._store
model = build_export_model(
@@ -678,6 +704,44 @@ def main() -> int:
abs(predicted - 2.0) > 0.05,
f"excluding a point left the exported model unchanged: {predicted}",
)
# One preparation: the store the panel shows is the prepared model's
# input, and the export serialises that same object.
prepared = window.cal_panel._prepared
check(
prepared is not None and prepared.ok,
"the panel has no prepared calibration to show or export",
)
check(
list(store.channel("fn")) == list(prepared.samples["fn"]),
"the panel store is not the prepared model's input",
)
doc, _warnings = _build_v2(prepared)
export_dir = APP_ROOT / "outputs" / "_cal_export_check"
if export_dir.exists():
for stale in export_dir.iterdir():
stale.unlink()
export_dir.mkdir(parents=True, exist_ok=True)
written = export_dir / "exported_calibration.json"
import json as _cal_json
written.write_text(_cal_json.dumps(doc), encoding="utf-8")
loaded = _sdk_calibration.load(written, strict=True)
for point in window.cal_recorder.points:
if point.excluded:
continue
raw = float(point.loaded_stats().mean)
check(
abs(
float(loaded.get("fn").convert(raw))
- prepared.predict("fn", raw)
)
<= 1e-9,
"the exported file and the panel disagree about the model",
)
check(
len(prepared.samples["fn"]) == 3,
f"the prepared build has {len(prepared.samples['fn'])} points, not 3",
)
window.cal_recorder.reset_session()
window.cal_panel._refresh_table()
@@ -961,7 +1025,8 @@ def main() -> int:
app.processEvents()
check(window.recorder.is_recording, "the recording did not start")
check(
window.toolbar.btn_record.isEnabled() and window.toolbar.btn_record.text() == t("record.stop_label"),
window.toolbar.btn_record.isEnabled()
and window.toolbar.btn_record.text() == t("record.stop_label"),
"the record button is not a usable stop control",
)
check(
@@ -974,13 +1039,26 @@ def main() -> int:
window._on_run_tick()
stuck.emit(_fake_packet(90, valid=False, reason="read failed"))
window._on_run_tick()
window._on_reset() # event: zeroing
# The zeroing is a *request*; the event is recorded only once the acquisition
# thread confirms it took effect (the stub confirms immediately).
stuck.on_reset_applied = window._on_reset_applied
window._on_reset()
window._on_run_tick()
app.processEvents()
window.toolbar.btn_record.setChecked(False)
app.processEvents()
report = window._record_last_report
check(not window.recorder.is_recording, "the recording did not stop")
check(report is not None and report.ok, f"recording reported problems: {getattr(report, 'problems', None)}")
check(
not window.toolbar.btn_record.isChecked(),
"the record button still looks active after the recording stopped",
)
check(
window._record_report_pending is False,
"a finished recording is still waiting for a report",
)
import csv as _csv
import json as _json
@@ -1019,6 +1097,388 @@ def main() -> int:
stuck.release()
_shutil.rmtree(scratch, ignore_errors=True)
section("record failure states")
from unittest.mock import patch as _patch
from orisys_app_shared.record.experiment import _SessionWriter as _Writer
scratch_fail = APP_ROOT / "outputs" / "_record_fail"
_shutil.rmtree(scratch_fail, ignore_errors=True)
scratch_fail.mkdir(parents=True, exist_ok=True)
stuck_fail = _StuckWorker()
fail_window = StudioMainWindow(
prefs={
"language": "zh_CN",
"theme": "dark",
"sensor_type": "box",
"source_mode": "camera",
}
)
fail_window.resize(1440, 880)
fail_window.show()
fail_window._ensure_workers = lambda: None
fail_window._box_worker = stuck_fail
fail_window._finger_worker = stuck_fail
fail_window._start_run()
fail_window._run_timer.stop()
fail_window._set_phase("running")
# A directory that cannot be created is a terminal, retryable failure.
blocked = scratch_fail / "blocked"
blocked.write_text("", encoding="utf-8")
_app_module.recordings_dir = lambda: blocked
fail_window.toolbar.btn_record.setChecked(True)
app.processEvents()
check(
not fail_window.recorder.is_recording,
"a recording started even though its directory could not be created",
)
check(
not fail_window.toolbar.btn_record.isChecked(),
"the record button stayed active after the start failed",
)
check(
fail_window.recorder.state == "failed",
f"a failed start left the recorder in {fail_window.recorder.state!r}",
)
# ...and the operator can simply try again.
_app_module.recordings_dir = lambda: scratch_fail
fail_window.toolbar.btn_record.setChecked(True)
app.processEvents()
check(
fail_window.recorder.is_recording,
"the recorder refused to retry after a failed directory",
)
# A writer that keeps failing must fail the recording, and the window must
# consume that report even though ``is_recording`` is already False.
with _patch.object(_Writer, "write_sample", side_effect=OSError("disk full")):
for index in range(8):
stuck_fail.emit(_fake_packet(index))
fail_window._on_run_tick()
deadline = time.monotonic() + 3.0
while (
fail_window.recorder.stats().write_errors < 5
and time.monotonic() < deadline
):
app.processEvents()
time.sleep(0.01)
check(
fail_window.recorder.stats().failed,
"repeated write failures never failed the recording",
)
check(
not fail_window.recorder.is_recording,
"a dead writer still claimed to be recording",
)
fail_window._stop_recording()
app.processEvents()
failed_report = fail_window._record_last_report
check(
failed_report is not None and failed_report.stats.failed,
"the window did not consume the failure report",
)
check(
not fail_window.toolbar.btn_record.isChecked(),
"the record button stayed active after the recording failed",
)
check(
fail_window._session_phase == "error",
f"the session phase is {fail_window._session_phase!r} after a failed recording",
)
fail_window.close()
app.processEvents()
stuck_fail.release()
_app_module.recordings_dir = lambda: scratch
_shutil.rmtree(scratch_fail, ignore_errors=True)
section("calibration record identity")
import json as _cal_json
from unittest.mock import patch as _patch_mismatch
import orisys_studio.ui.cal_panel as _cal_panel_module
import orisys_studio.calibration_service as _cal_service_module
from PyQt5.QtWidgets import QMessageBox as _QMessageBox
from orisys_app_shared.force.calibration_io import (
build_force_calibration_v2 as _build_v2b,
write_force_calibration as _write_cal,
)
from orisys_app_shared.force.calibration_session import (
build_session_document as _build_session_doc,
)
record_dir = APP_ROOT / "outputs" / "_cal_record_check"
if record_dir.exists():
for stale in record_dir.iterdir():
stale.unlink()
record_dir.mkdir(parents=True, exist_ok=True)
identity_window = StudioMainWindow(
prefs={
"language": "zh_CN",
"theme": "dark",
"sensor_type": "box",
"source_mode": "camera",
}
)
identity_window.resize(1400, 880)
identity_window.show()
identity_window.cal_panel.set_running(True)
taken_points = []
for load, raw in ((1.0, 100.0), (3.0, 300.0)):
identity_window.cal_recorder.start(
channel="fn", reference_load_n=load, duration_s=1.0
)
for index in range(20):
identity_window.cal_recorder.feed(
_calibration_sample(index, fn_raw=raw)
)
taken_points.append(identity_window.cal_recorder.finish_now())
device_a = record_dir / "device_a_session.json"
device_a.write_text(
_cal_json.dumps(
_build_session_doc(
taken_points,
meta={
"sensor_type": "box",
"sn": "SN-A",
"config_ref": "D:/lab/deviceA/sensor.json",
},
)
),
encoding="utf-8",
)
identity_window.cal_recorder.clear_points()
# The live device is B; opening A's record must not restamp it.
panel = identity_window.cal_panel
panel.set_meta_context(sensor_type="finger", config_ref="D:/lab/deviceB/sensor.json")
panel.sn_edit.setText("SN-B")
panel._on_open_record(str(device_a))
app.processEvents()
check(
panel.sn_edit.text() == "SN-A",
f"the reopened record's SN was overwritten: {panel.sn_edit.text()!r}",
)
meta = panel._export_meta()
check(meta["sensor_type"] == "box", "the record's sensor type was overwritten")
check(meta["sn"] == "SN-A", "the record's SN was not kept")
check(
meta["config_ref"] == "D:/lab/deviceA/sensor.json",
f"the record's config was overwritten: {meta['config_ref']!r}",
)
check(
str(meta.get("source_record", "")).endswith(device_a.name),
"the export does not reference the source record",
)
saved = sorted(record_dir.glob("calibration_session_box_*.json"))
with _patch_mismatch.object(
_cal_panel_module, "calibration_export_dir", lambda: record_dir
):
with _patch_mismatch.object(
_cal_service_module, "calibration_export_dir", lambda: record_dir
):
with _patch_mismatch.object(
_cal_panel_module.QMessageBox, "information", lambda *a, **k: None
):
panel._on_save_record()
saved = sorted(record_dir.glob("calibration_session_box_*.json"))
check(bool(saved), "saving a reopened record produced no file")
if saved:
saved_meta = _cal_json.loads(saved[-1].read_text(encoding="utf-8"))["meta"]
check(
saved_meta.get("sn") == "SN-A" and saved_meta.get("sensor_type") == "box",
f"the saved record was restamped: {saved_meta.get('sn')!r}",
)
doc, _warnings = _build_v2b(panel._prepared)
check(
doc["meta"]["sn"] == "SN-A" and doc["meta"]["sensor_type"] == "box",
"the exported coefficients carry the wrong device identity",
)
# A reopened point with only its samples can still be inspected.
reopened = identity_window.cal_recorder.points[0]
for entry in _build_session_doc([reopened])["points"]:
entry["trace"] = {}
dialog = panel.point_dialog(reopened)
check(dialog is not None, "a reopened point could not be inspected")
if dialog is not None:
check(
len(dialog.plot_widget.ci.items) == 3,
"the rebuilt diagnostic is not one trace per axis",
)
dialog.close()
# Applying a calibration that names another device is flagged and defaults
# to "no".
stuck_identity = _StuckWorker()
identity_window._ensure_workers = lambda: None
identity_window._box_worker = stuck_identity
identity_window._finger_worker = stuck_identity
coefficients = record_dir / "device_a_coefficients.json"
_write_cal(doc, coefficients)
check(
bool(
identity_window.calibration_target_mismatch(
str(coefficients),
sensor_type="finger",
config_ref="D:/lab/B/sensor.json",
)
),
"a calibration for another device was not reported as a mismatch",
)
check(
not identity_window.calibration_target_mismatch(
str(coefficients),
sensor_type="box",
config_ref="D:/lab/deviceA/sensor.json",
),
"matching identities were reported as a mismatch",
)
with _patch_mismatch.object(
_QMessageBox, "question", staticmethod(lambda *a, **k: _QMessageBox.No)
):
identity_window._on_calibration_apply(str(coefficients))
check(
not stuck_identity.calibration_requests,
"a mismatched calibration was applied after the user declined",
)
check(
identity_window.statusBar().currentMessage() == t("cal.apply_mismatch_skipped"),
f"the refusal was not reported: {identity_window.statusBar().currentMessage()!r}",
)
# A matching calibration still goes through the normal confirmation.
identity_window.sensor_picker.set_sensor_type("box")
with _patch_mismatch.object(
_QMessageBox, "question", staticmethod(lambda *a, **k: _QMessageBox.Yes)
):
identity_window._on_calibration_apply(str(coefficients))
check(
stuck_identity.calibration_requests == [str(coefficients)],
f"a matching calibration was not requested: {stuck_identity.calibration_requests}",
)
identity_window.close()
app.processEvents()
_shutil.rmtree(record_dir, ignore_errors=True)
section("display units and time base")
import math as _math
stuck_units = _StuckWorker()
units_window = StudioMainWindow(
prefs={
"language": "zh_CN",
"theme": "dark",
"sensor_type": "box",
"source_mode": "camera",
}
)
units_window.resize(1440, 880)
units_window.show()
units_window._ensure_workers = lambda: None
units_window._box_worker = stuck_units
units_window._finger_worker = stuck_units
units_window._start_run()
units_window._run_timer.stop()
units_window._set_phase("running")
units_plots = units_window.force_plots
units_window.toolbar.units_combo.setCurrentIndex(0) # newton
units_plots.reset()
wall = 1_700_000_000.0
for index in range(90):
stuck_units.emit(
_fake_packet(index, timestamp=wall + index / 30.0, mono_t=index / 30.0)
)
units_window._on_run_tick()
before = units_plots.history.state()
end_before = float(before.times[-1]) if before is not None else -1.0
check(
before is not None and end_before < 60.0,
f"the live X axis is not session-relative (ends at {end_before})",
)
for units_index in (1, 0, 1, 0):
units_window.toolbar.units_combo.setCurrentIndex(units_index)
app.processEvents()
after = units_plots.history.state()
check(
after is not None and abs(float(after.times[-1]) - end_before) < 1.0,
"the X axis jumped across a raw/N switch: "
f"{end_before} -> {None if after is None else after.times[-1]}",
)
check(
units_plots.history.sample_count > 0,
"the rebuilt history lost every sample",
)
units_window.close()
units_window._box_worker = None
stuck_units.release()
app.processEvents()
# A device with only its normal-force axis calibrated: FN in newtons, and the
# other two axes explicitly unavailable instead of showing raw counts.
stuck_one = _StuckWorker()
one_window = StudioMainWindow(
prefs={
"language": "zh_CN",
"theme": "dark",
"sensor_type": "box",
"source_mode": "camera",
}
)
one_window.resize(1440, 880)
one_window.show()
one_window._ensure_workers = lambda: None
one_window._box_worker = stuck_one
one_window._finger_worker = stuck_one
one_window._start_run()
one_window._run_timer.stop()
one_window._set_phase("running")
one_window.toolbar.units_combo.setCurrentIndex(0) # newton
one_window.force_plots.reset()
for index in range(10):
stuck_one.emit(
_fake_packet(index, calibrated_axes=("fn",), shear_n=float("nan"))
)
one_window._on_run_tick()
app.processEvents()
one_plots = one_window.force_plots
check(
one_plots._available == (True, False, False),
f"per-axis availability is wrong: {one_plots._available}",
)
check(
one_plots.value_labels[0].text() != t("chart.value_unavailable"),
"the calibrated axis was reported as unavailable",
)
check(
all(
one_plots.value_labels[index].text() == t("chart.value_unavailable")
for index in (1, 2)
),
"an axis without a curve showed a number in newton mode",
)
one_state = one_plots.history.state()
check(
one_state is not None
and _math.isfinite(float(one_state.values[:, 0].max())),
"the calibrated axis was not plotted",
)
check(
one_state is not None
and not any(_math.isfinite(v) for v in one_state.values[:, 1]),
"FX carried a value without a calibration curve",
)
one_window.close()
one_window._box_worker = None
stuck_one.release()
app.processEvents()
section("session browsing")
import numpy as np
@@ -1190,6 +1650,31 @@ def main() -> int:
all(int(curve.getData()[0].size) == 40 for curve in playback.curves),
"replay did not draw every force sample",
)
# Long-history browsing: an interval read is bounded by the interval, and the
# whole-session envelope is a band rather than every row.
from orisys_app_shared.force.source import RecordingCurveSource
from orisys_app_shared.record.index import RecordingIndex
from orisys_app_shared.record.playback import RecordingReader as _RecordingReader
reader = _RecordingReader(recording, max_rows=10)
source = RecordingCurveSource(reader)
check(source.kind == "disk", "the replay source is not the recording")
index = reader.index()
check(index.rows == 40, f"the index found {index.rows} rows, not 40")
start, stop = index.row_range(0.5, 1.0)
check(
0 <= start < stop <= index.rows and (stop - start) <= 20,
f"an interval read spans {stop - start} rows",
)
interval = source.window(0.5, 1.0, unit="N")
check(0 < len(interval) <= 20, f"the interval read returned {len(interval)} rows")
envelope = source.envelope(buckets=8, unit="N")
check(0 < len(envelope) <= 8, f"the envelope has {len(envelope)} points")
check(
reader.info().total_rows == 40,
f"the recording reports {reader.info().total_rows} rows",
)
check(playback.slider.isEnabled(), "the replay position slider is disabled")
check(
t("playback.warn_depth_recomputed").split("{")[0].strip()[:4]
@@ -1268,6 +1753,92 @@ def main() -> int:
playback.close()
app.processEvents()
# The plan must state where to start, which reset it is anchored to and
# whether the state is really recoverable; and a request must belong to the
# recording it started in.
from orisys_app_shared.record.depth import (
STATE_APPROXIMATE,
STATE_PREROLL,
DepthRecomputer,
)
from orisys_app_shared.record.playback import RecordingReader
plan_reader = RecordingReader(recording)
planner = DepthRecomputer(plan_reader)
frames = plan_reader.info().frames
check(bool(frames), "the replay recording has no frame index")
plan = planner.plan_for(frames[10])
check(
plan.state == STATE_APPROXIMATE and plan.anchor == "segment_start",
f"a recording with no reset/pre-roll is not reported as approximate: {plan.state}",
)
check(
plan.steps and plan.steps[0].source == plan_reader.segment_path(frames[10].segment),
"the plan does not name the video to read",
)
check(
"state_approximate" in plan.warnings,
f"the approximate replay is not reported: {plan.warnings}",
)
# A pre-roll makes the start of a recording reproducible.
session_path = recording / "session.json"
session_doc = _json.loads(session_path.read_text(encoding="utf-8"))
prerecord = recording / "preroll_0001.mp4"
prerecord.write_bytes(b"")
session_doc["preroll_frames"] = 30
session_doc["preroll_segment"] = prerecord.name
session_path.write_text(_json.dumps(session_doc), encoding="utf-8")
planner.close()
planner = DepthRecomputer(RecordingReader(recording))
plan = planner.plan_for(planner.reader.info().frames[5])
check(
plan.state == STATE_PREROLL and plan.preroll == prerecord,
f"the pre-roll was not used for the start of the recording: {plan.state}",
)
check(plan.frames_to_read == 30 + 6, f"unexpected replay cost: {plan.frames_to_read}")
planner.close()
# A reset boundary is re-applied, and a request is bound to its recording.
_Blocking.released.clear()
second = PlaybackDialog(theme="dark", directory=recording)
second.show()
app.processEvents()
serial_before = second._record_serial
blocking = _Blocking()
second._recomputer = blocking
second._on_recompute_depth()
check(second._depth_pending_record == serial_before, "the request has no recording id")
check(
second._depth_pending_seq is not None,
"the request carries no sequence number",
)
# Opening another recording invalidates the running request...
second.open_recording(recording)
check(
second._record_serial == serial_before + 1,
"opening a recording did not invalidate the running request",
)
check(
second._recomputer is not blocking,
"the invalidated processor was not replaced",
)
_Blocking.released.set()
for _ in range(40):
app.processEvents()
_time.sleep(0.05)
if not second._depth_busy():
break
second._poll_depth()
app.processEvents()
pixmap = second.depth_label.pixmap()
check(
pixmap is None or pixmap.isNull(),
"an answer for a previous recording was drawn into the new one",
)
second.close()
app.processEvents()
_shutil.rmtree(replay_root, ignore_errors=True)
print(f"\nscenes: {scene_count} checks: {CHECKS} failures: {len(FAILURES)}")