调整 Autocal 标定记录格式与目录命名

将人工参考载荷按档位区间保存,重排帧字段并明确界面操作。已通过相关测试、代码和文档检查。
This commit is contained in:
ykcheungab
2026-10-11 16:21:49 +08:00
parent dc7e290489
commit 69c1322616
9 changed files with 233 additions and 46 deletions
+6 -3
View File
@@ -263,6 +263,7 @@
"frame": "Frame {frame} · source {source_frame} · {basis} {time} · dropped {dropped}" "frame": "Frame {frame} · source {source_frame} · {basis} {time} · dropped {dropped}"
}, },
"calibration": { "calibration": {
"sensor_model": "Product model",
"sensor_sn": "Sensor SN", "sensor_sn": "Sensor SN",
"output": "Output directory", "output": "Output directory",
"site_view": "Select site", "site_view": "Select site",
@@ -272,13 +273,15 @@
"levels": "Load levels (N)", "levels": "Load levels (N)",
"hold": "Suggested hold", "hold": "Suggested hold",
"reference": "Actual reference (N)", "reference": "Actual reference (N)",
"start": "Start recording", "start": "Start trial recording",
"confirm": "Confirm level", "confirm": "Confirm current level",
"finish": "Finish trial", "finish": "Finish and save",
"abort": "Abort and keep data", "abort": "Abort and keep data",
"load": "Load", "load": "Load",
"unload": "Unload", "unload": "Unload",
"next_step": "Next: {branch} to {load:g} N", "next_step": "Next: {branch} to {load:g} N",
"before_start": "Start trial recording first",
"ready_to_finish": "All levels confirmed; finish and save",
"no_step": "No step pending", "no_step": "No step pending",
"progress": "Progress {done}/{total} · {status}", "progress": "Progress {done}/{total} · {status}",
"held": " · Current level held {seconds:.1f} s", "held": " · Current level held {seconds:.1f} s",
+6 -3
View File
@@ -263,6 +263,7 @@
"frame": "帧 {frame} · 源帧 {source_frame} · {basis} {time} · 丢帧 {dropped}" "frame": "帧 {frame} · 源帧 {source_frame} · {basis} {time} · 丢帧 {dropped}"
}, },
"calibration": { "calibration": {
"sensor_model": "产品型号",
"sensor_sn": "传感器 SN", "sensor_sn": "传感器 SN",
"output": "保存目录", "output": "保存目录",
"site_view": "选择测点", "site_view": "选择测点",
@@ -272,13 +273,15 @@
"levels": "加载档位 (N)", "levels": "加载档位 (N)",
"hold": "建议保持", "hold": "建议保持",
"reference": "实际参考力 (N)", "reference": "实际参考力 (N)",
"start": "开始记录", "start": "开始试验并录制",
"confirm": "确认到档", "confirm": "确认当前档位",
"finish": "结束试验", "finish": "完成并保存",
"abort": "中止并保留", "abort": "中止并保留",
"load": "加载", "load": "加载",
"unload": "卸载", "unload": "卸载",
"next_step": "下一档:{branch}至 {load:g} N", "next_step": "下一档:{branch}至 {load:g} N",
"before_start": "请先开始试验并录制",
"ready_to_finish": "全部档位已确认,请完成并保存",
"no_step": "无待确认档位", "no_step": "无待确认档位",
"progress": "进度 {done}/{total} · {status}", "progress": "进度 {done}/{total} · {status}",
"held": " · 当前档已保持 {seconds:.1f} s", "held": " · 当前档已保持 {seconds:.1f} s",
+12 -10
View File
@@ -1,16 +1,16 @@
# Autocal 标定操作与数据格式草稿 # Autocal 标定操作与数据格式草稿
状态:待评审,2026-10-11。用途:方案及操作指南。阶段一只针对 finger;单测点、单 trial 手动记录已接通,跨点 session、FT 参考及灵敏度可视化尚未实现。下列目标文件清单不等同于当前写盘格式;当前每 trial 是一个独立目录,含 `raw_0001.mp4`、`session.json`、`frames.csv`、`forces.csv`、`events.csv`。 状态:待评审,2026-10-11。阶段一仅针对 finger;单点手动记录已接通,跨点 session、FT 参考及灵敏度图待实现。每 trial 保存视频、`session.json`、`frames.csv`、`forces.csv`、`reference.csv`、`events.csv`。目录名为 `<UTC时间>_<产品型号>_<SN>_session<采集会话号>_<唯一标识>`;完整型号和 SN 存于 `session.json`。
## 范围与软件现状 ## 范围与软件现状
| 项目 | 当前约定 | | 项目 | 当前约定 |
|---|---| |---|---|
| 标定范围 | 多测点法向力,暂定 0–30 N;剪切及组合载荷后续再做 | | 标定范围 | 多测点法向力,0–30 N;剪切及组合载荷后续再做 |
| 传感器 | 首版固定为 finger;其余类型须另行验证几何、采集和处理契约 | | 传感器 | 首版固定为 finger;其余类型须另行验证几何、采集和处理契约 |
| 压头 | 直径 6 mm、球面半径 3 mm 的半球压头 | | 压头 | 直径 6 mm、球面半径 3 mm 的半球压头 |
| 参考 | 砝码或 50 N/2 N·m FT sensor,首版先接通一种 | | 参考 | 砝码或 50 N/2 N·m FT sensor,首版先接通一种 |
| 图像 | 一个完整测试事件一个连续视频,档位间不切段 | | 图像 | 每测试事件一个连续视频,档位间不切段 |
| 调试输入 | 已有视频与实时相机编号共用 finger worker 和 `SdkPacket`;相机实机尚未验收 | | 调试输入 | 已有视频与实时相机编号共用 finger worker 和 `SdkPacket`;相机实机尚未验收 |
| 交付 | 单个 session 能重新打开、追溯参考载荷,计算并显示不同测点的灵敏度 | | 交付 | 单个 session 能重新打开、追溯参考载荷,计算并显示不同测点的灵敏度 |
@@ -24,7 +24,7 @@
| 步骤 | 操作员动作 | 软件记录/通过条件 | | 步骤 | 操作员动作 | 软件记录/通过条件 |
|---|---|---| |---|---|---|
| 1. 建立会话 | 填写传感器 SN、gel ID、实验台、压头和参考设备;选保存位置 | 创建唯一 `session_id`;保存身份和配置快照 | | 1. 建立会话 | 填写产品型号、传感器 SN、gel ID、实验台、压头和参考设备;选保存位置 | 创建唯一 `session_id`;保存身份和配置快照 |
| 2. 检查设备 | 完成设备设置、回零及既定检查,核对相机和参考输入 | 状态可读;运动边界按既定流程确认 | | 2. 检查设备 | 完成设备设置、回零及既定检查,核对相机和参考输入 | 状态可读;运动边界按既定流程确认 |
| 3. 确认测点 | 在有效 UV 区域选点,核对三维位置和姿态 | 分配 `site_id`,UV→3D 映射有效 | | 3. 确认测点 | 在有效 UV 区域选点,核对三维位置和姿态 | 分配 `site_id`,UV→3D 映射有效 |
| 4. 设置循环 | 定义加载范围、档位、保持时间和重复数,选择手动模式 | 保存计划;自动加载留到下一阶段 | | 4. 设置循环 | 定义加载范围、档位、保持时间和重复数,选择手动模式 | 保存计划;自动加载留到下一阶段 |
@@ -56,16 +56,16 @@
| `session.json` | `format_version`、`session_id`、`sensor_type`、`input_source`、身份、配置快照、坐标定义、文件清单、sites、trials、状态 | 会话建立、事件完成及结束时 | | `session.json` | `format_version`、`session_id`、`sensor_type`、`input_source`、身份、配置快照、坐标定义、文件清单、sites、trials、状态 | 会话建立、事件完成及结束时 |
| `sequence.oryseq` | 实际使用的测点计划;遵循现有格式 | 开始采集前保存快照 | | `sequence.oryseq` | 实际使用的测点计划;遵循现有格式 | 开始采集前保存快照 |
| `video/<trial_id>.<ext>` | 算法入口图像、完整循环及恢复 | 每 trial 开始至结束 | | `video/<trial_id>.<ext>` | 算法入口图像、完整循环及恢复 | 每 trial 开始至结束 |
| `frames.csv` | `session_id`、`trial_id`、`frame_id`、`t_host_mono_ns`、`t_host_utc_ns`、`time_basis`、视频路径、帧号、丢帧标记 | 每帧,时间与帧身份同时绑定 | | `frames.csv` | 当前按帧身份、采集时间、视频位置、状态排序;包含 `frame_id`、`source_frame_id`、`t_rel_s`、`t_host_mono_ns`、`t_host_utc_ns`、`acquired_at`、`time_basis`、`media_time_ms`、`segment`、`segment_frame`、`in_video`、`dropped`、`capture_dropped` | 每帧,时间与帧身份同时绑定;当前与 Studio 共用录制格式 |
| `baseline/` | 基准图像、初始状态或预录引用、`baseline_id` | 建立/重置基准时 | | `baseline/` | 基准图像、初始状态或预录引用、`baseline_id` | 建立/重置基准时 |
| `reference.csv` | `trial_id`、`sample_id`、`t_host_mono_ns`、`clock_id`、力、有效性、来源及延迟估计 | FT/ECAT 每个样本;砝码用事件和有效时段 | | `reference.csv` | 当前为 `step_index`、`branch`、`target_load_n`、区间起止电脑单调时间、`F_ref_normal_N`、预留的 `F_ref_x_N`/`F_ref_y_N`/`F_ref_z_N`、`force_frame`、`source`、`stability_verified` | 当前每个手工确认档位一行;FT/ECAT 逐样本参考尚未接通 |
| `motion.csv` | 时间、目标/反馈位姿、有效性 | 有实际反馈时保存;缺少反馈须注明 | | `motion.csv` | 时间、目标/反馈位姿、有效性 | 有实际反馈时保存;缺少反馈须注明 |
| `events.csv` | `t_host_mono_ns`、trial/step、事件类型、参考有效时段、原因 | 到档确认、阶段切换、归零、异常及中断 | | `events.csv` | `t_host_mono_ns`、trial/step、事件类型、参考有效时段、原因 | 到档确认、阶段切换、归零、异常及中断 |
| `derived/<processing_id>/` | 逐点灵敏度表、加载/卸载曲线、误差、分析窗口和处理配置 | 分析时另存;不覆盖原始文件 | | `derived/<processing_id>/` | 逐点灵敏度表、加载/卸载曲线、误差、分析窗口和处理配置 | 分析时另存;不覆盖原始文件 |
| 参数归属 | 主要参数 | 单位/填写来源 | | 参数归属 | 主要参数 | 单位/填写来源 |
|---|---|---| |---|---|---|
| session 身份 | `sensor_sn`、`gel_id`、`bench_id`、`indenter_id`、`reference_source` | 操作员填写或设备读取 | | session 身份 | `sensor_model`、`sensor_sn`、`gel_id`、`bench_id`、`indenter_id`、`reference_source` | 操作员填写或设备读取;当前型号和 SN 必填 |
| session 采集 | 分辨率、像素格式、编码、版本、滤波与归零配置 | 软件快照;编码待实测确认 | | session 采集 | 分辨率、像素格式、编码、版本、滤波与归零配置 | 软件快照;编码待实测确认 |
| 输入来源 | `input_source.kind` 为 `video` 或 `camera`;`value` 分别为视频路径或相机整数编号;视频另存文件哈希 | 传给同一 `FingerAcquisitionWorker.start`;来源文件与 trial 输出视频不可混淆 | | 输入来源 | `input_source.kind` 为 `video` 或 `camera`;`value` 分别为视频路径或相机整数编号;视频另存文件哈希 | 传给同一 `FingerAcquisitionWorker.start`;来源文件与 trial 输出视频不可混淆 |
| session 时间 | 同进程 `clock_id`、单调时钟原点、UTC 对照、时间来源、延迟及同步误差 | 电脑采集/接收时记录;不能按行号配对 | | session 时间 | 同进程 `clock_id`、单调时钟原点、UTC 对照、时间来源、延迟及同步误差 | 电脑采集/接收时记录;不能按行号配对 |
@@ -77,11 +77,13 @@
| 质量 | `valid`、`invalid_reason`、丢帧/中断状态 | 未知不补零;算法失败不等于图像不可用 | | 质量 | `valid`、`invalid_reason`、丢帧/中断状态 | 未知不补零;算法失败不等于图像不可用 |
| 逐点结果 | `site_id`、`gain_n_per_raw`、`sensitivity_raw_per_n`、有效档位数、误差及滞后 | 仅对有效独立参考与原始法向信号拟合;缺数据标无效 | | 逐点结果 | `site_id`、`gain_n_per_raw`、`sensitivity_raw_per_n`、有效档位数、误差及滞后 | 仅对有效独立参考与原始法向信号拟合;缺数据标无效 |
实时采集以电脑收到**该帧**时的单调时间为主时间。采集线程需把 `t_host_mono_ns` 与帧序号、图像原子配对;记录器再保存 UTC 对照,处理结束时间不能代替采集时间。FT/ECAT 样本也在电脑收到时打同源时间戳,按时间关联,并记录可测得的延迟与失同步;低延迟不等于零延迟。砝码的人工到档时刻只定义有效区间,稳定窗口仍需检查。 实时采集以电脑收到**该帧**时的单调时间为主时间。采集线程需把 `t_host_mono_ns` 与帧序号、图像原子配对;记录器再保存 UTC 对照,处理结束时间不能代替采集时间。FT/ECAT 样本也在电脑收到时打同源时间戳,按时间关联,并记录可测得的延迟与失同步;低延迟不等于零延迟。砝码的人工到档时刻只定义候选区间,稳定窗口仍需检查。
旧视频没有原始逐帧电脑时间时,重放取得的是本次解码时间,`time_basis=host_replay`;它可验证处理链,不能还原原实验与 FT 的同步。若视频附有原始帧时间则保留 `time_basis=original_recorded` 和原时钟身份。finger worker 已停止按配置帧率推算 `mono_t`,并在包中保留主机采集时间、源帧编号及可用的媒体时间;正式记录器仍须将这些字段写入 `frames.csv`。manifest 保留来源路径、哈希、配置、媒体帧率及各时间基准。 当前 `reference.csv` 的起点是操作员确认时刻,终点是下一次确认或结束时刻。`F_ref_normal_N` 是人工声明的正向压缩载荷;未测量的 `F_ref_x_N`、`F_ref_y_N`、`F_ref_z_N` 及对应 `force_frame` 留空,不写零,也不将法向值直接当作平台 Z 分量。`stability_verified=0` 表示整个声明区间尚未验证稳定,后续分析必须另选稳定窗口。`forces.csv` 只保存 finger 图像处理输出,不含独立参考力。
事件边界如下。`events.csv` 每行至少保存 `event_id`、`trial_id`、可为空的 `step_id`、`event_type`、`t_host_mono_ns`、`clock_id` 和 `reason`。操作员的确认时间和设备反馈时间分别标明来源;不得把按钮时刻冒充设备实际到档。所有流用 `trial_id` 与同源单调时间关联。 旧视频没有原始逐帧电脑时间时,重放取得的是本次解码时间,`time_basis=host_replay`;它可验证处理链,不能还原原实验与 FT 的同步。finger worker 和录制器已保存主机采集时间、源帧编号及可用的媒体时间;来源路径、哈希、配置和媒体帧率仍需在 manifest 中核对。
目标事件需标明 trial、step、事件时刻及来源;按钮时刻不能冒充设备实际到档。当前 `events.csv` 的 `detail` 保存 `t_host_mono_ns` 和档位信息。
| 事件 | 触发与状态 | | 事件 | 触发与状态 |
|---|---| |---|---|
+2 -2
View File
@@ -33,9 +33,9 @@
M1 的视频与相机输入现共用带时间的数据包;M2 记录器与 M3 手动 UI 可在接口固定后并行。M4 可先用合成 session 开发,但阶段一只有在真实记录上重新打开并计算逐测点结果才算完成。Studio 的 `SessionController`、widget 和 `RecordingCoordinator` 保留在 Studio;Autocal 直接复用 shared `FingerAcquisitionWorker`、`SdkPacket` 和 sink,不跨应用导入私有模块。只有出现两端实际共用的逻辑时才移到 shared;SDK 不依赖 UI。 M1 的视频与相机输入现共用带时间的数据包;M2 记录器与 M3 手动 UI 可在接口固定后并行。M4 可先用合成 session 开发,但阶段一只有在真实记录上重新打开并计算逐测点结果才算完成。Studio 的 `SessionController`、widget 和 `RecordingCoordinator` 保留在 Studio;Autocal 直接复用 shared `FingerAcquisitionWorker`、`SdkPacket` 和 sink,不跨应用导入私有模块。只有出现两端实际共用的逻辑时才移到 shared;SDK 不依赖 UI。
界面复核:原先测点、计划和输入分散在“手指调试”“测试序列”“传感器”页。现在“标定采集”页串起单次 trial;全局“运行”按钮尚未负责标定,页面内有独立的“开始记录”。 界面复核:原先测点、计划和输入分散在“手指调试”“测试序列”“传感器”页。现在“标定采集”页串起单次 trial;全局“运行”按钮尚未负责标定,页面内有独立的“开始试验并录制”。
已实现的最小闭环在“标定采集”页:点击 UV 图粗选、以 U/V 整数像素精确修改、切换到实时二维 UV 图像,逐档填写实际参考力并确认。每个 trial 单独使用 shared `ExperimentRecorder` 生成目录、视频、帧索引、力数据和事件;`frames.csv` 附加主机采集时间及来源帧编号。当前人工参考只记录到档事件,尚无可自动判定的稳定窗口;未实现 FT 同步、跨点 session 汇总、灵敏度拟合和实体相机验收。手动 Z 控制仍使用既有设备界面,专用弹窗需先确定操作边界。 已实现的最小闭环在“标定采集”页:点击 UV 图粗选、以 U/V 整数像素精确修改、切换到实时二维 UV 图像,逐档填写实际参考力并确认。每个 trial 单独使用 shared `ExperimentRecorder` 生成目录、视频、帧索引、力数据和事件;人工参考另存 `reference.csv`,仅声明档位区间,尚无可自动判定的稳定窗口。未实现 FT 同步、跨点 session 汇总、灵敏度拟合和实体相机验收。手动 Z 控制仍使用既有设备界面,专用弹窗需先确定操作边界。
主代理负责接口、M3 和集成;一名子代理负责 M1/M2,另一名负责 M4 与合成会话验证。建议总计主代理加两名子代理,约 4–6 个工程工作日并留设备联调时间;单人约 6–9 日。估算不包括设备、FT 协议或硬件等待。当前计划不启动子代理。 主代理负责接口、M3 和集成;一名子代理负责 M1/M2,另一名负责 M4 与合成会话验证。建议总计主代理加两名子代理,约 4–6 个工程工作日并留设备联调时间;单人约 6–9 日。估算不包括设备、FT 协议或硬件等待。当前计划不启动子代理。
+115 -5
View File
@@ -6,8 +6,10 @@ module owns only the site, planned steps and operator-confirmed event order.
from __future__ import annotations from __future__ import annotations
import csv
import hashlib import hashlib
import json import json
import re
import time import time
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -16,6 +18,26 @@ import numpy as np
from orisys_app_shared.record import ExperimentRecorder, RecordConfig from orisys_app_shared.record import ExperimentRecorder, RecordConfig
REFERENCE_HEADER = (
"step_index",
"branch",
"target_load_n",
"t_start_host_mono_ns",
"t_end_host_mono_ns",
"F_ref_normal_N",
"F_ref_x_N",
"F_ref_y_N",
"F_ref_z_N",
"force_frame",
"source",
"stability_verified",
)
def _folder_part(value: str) -> str:
return re.sub(r"[^\w-]+", "-", value.strip()).strip("-_ ")[:48]
@dataclass(frozen=True) @dataclass(frozen=True)
class Site: class Site:
u_px: int u_px: int
@@ -84,6 +106,10 @@ class ManualTrial:
self.aborted = False self.aborted = False
self.reference_alignment_possible = False self.reference_alignment_possible = False
self._last_confirm_ns: int | None = None self._last_confirm_ns: int | None = None
self._reference_file = None
self._reference_writer = None
self._active_reference: tuple[int, str, float, float, int] | None = None
self._reference_error = ""
def start( def start(
self, self,
@@ -92,6 +118,7 @@ class ManualTrial:
site: Site, site: Site,
levels: list[float], levels: list[float],
hold_s: float, hold_s: float,
sensor_model: str,
sensor_sn: str, sensor_sn: str,
bundle_path: str, bundle_path: str,
source: dict, source: dict,
@@ -104,6 +131,15 @@ class ManualTrial:
raise RuntimeError("start the finger input and wait for its first frame") raise RuntimeError("start the finger input and wait for its first frame")
if not sensor_sn.strip(): if not sensor_sn.strip():
raise ValueError("sensor SN is required") raise ValueError("sensor SN is required")
if not sensor_model.strip():
raise ValueError("sensor model is required")
model_part = _folder_part(sensor_model)
sn_part = _folder_part(sensor_sn)
if not model_part or not sn_part:
raise ValueError(
"sensor model and SN need letters or numbers for the folder"
)
label = f"{model_part}_{sn_part}"
if not float(fps) > 0.0: if not float(fps) > 0.0:
raise ValueError("input frame rate must be positive") raise ValueError("input frame rate must be positive")
self.steps = load_path(levels) self.steps = load_path(levels)
@@ -122,6 +158,7 @@ class ManualTrial:
metadata = { metadata = {
"purpose": "autocal_finger_manual_trial", "purpose": "autocal_finger_manual_trial",
"sensor_sn": sensor_sn.strip(), "sensor_sn": sensor_sn.strip(),
"sensor_model": sensor_model.strip(),
"site": { "site": {
"uv_xy_px": [site.u_px, site.v_px], "uv_xy_px": [site.u_px, site.v_px],
"position_sensor_mm": list(site.position_sensor_mm), "position_sensor_mm": list(site.position_sensor_mm),
@@ -138,15 +175,36 @@ class ManualTrial:
"time_basis": packet.time_basis, "time_basis": packet.time_basis,
"reference_alignment_possible": self.reference_alignment_possible, "reference_alignment_possible": self.reference_alignment_possible,
"reference_source": "manual_operator_confirmation", "reference_source": "manual_operator_confirmation",
"reference_file": "reference.csv",
} }
recorder = ExperimentRecorder( recorder = ExperimentRecorder(
RecordConfig(fps=float(fps), segment_seconds=24 * 3600.0) RecordConfig(fps=float(fps), segment_seconds=24 * 3600.0)
) )
directory = recorder.start( directory = recorder.start(
root=Path(root), session_id=packet.session_id, metadata=metadata root=Path(root),
session_id=packet.session_id,
metadata=metadata,
directory_label=label,
) )
reference_file = None
try:
reference_file = (directory / "reference.csv").open(
"w", newline="", encoding="utf-8"
)
reference_writer = csv.writer(reference_file)
reference_writer.writerow(REFERENCE_HEADER)
reference_file.flush()
except OSError:
if reference_file is not None:
reference_file.close()
recorder.stop(wait=True)
raise
self.recorder = recorder self.recorder = recorder
self.directory = directory self.directory = directory
self._reference_file = reference_file
self._reference_writer = reference_writer
self._active_reference = None
self._reference_error = ""
self.step_index = 0 self.step_index = 0
self.aborted = False self.aborted = False
self._last_confirm_ns = None self._last_confirm_ns = None
@@ -161,11 +219,11 @@ class ManualTrial:
self.steps[self.step_index] if self.step_index < len(self.steps) else None self.steps[self.step_index] if self.step_index < len(self.steps) else None
) )
def _event(self, kind: str, detail: dict) -> None: def _event(self, kind: str, detail: dict, *, now_ns: int | None = None) -> None:
recorder = self.recorder recorder = self.recorder
if recorder is None: if recorder is None:
return return
now_mono = time.perf_counter_ns() now_mono = now_ns if now_ns is not None else time.perf_counter_ns()
packet = self.worker.latest() packet = self.worker.latest()
if packet is not None and packet.capture_mono_ns is not None: if packet is not None and packet.capture_mono_ns is not None:
event_mono_t = ( event_mono_t = (
@@ -185,6 +243,33 @@ class ManualTrial:
frame_id=frame_id, frame_id=frame_id,
) )
def _end_reference(self, end_ns: int) -> None:
if self._active_reference is None:
return
step_index, branch, target, reference, start_ns = self._active_reference
try:
self._reference_writer.writerow(
(
step_index,
branch,
target,
start_ns,
end_ns,
reference,
"",
"",
"",
"",
"manual_operator_confirmation",
0,
)
)
self._reference_file.flush()
except OSError as exc:
self._reference_error = str(exc)
finally:
self._active_reference = None
def confirm(self, reference_n: float) -> tuple[str, float]: def confirm(self, reference_n: float) -> tuple[str, float]:
if self.status != "recording" or self.next_step is None: if self.status != "recording" or self.next_step is None:
raise RuntimeError("no load step is waiting for confirmation") raise RuntimeError("no load step is waiting for confirmation")
@@ -194,12 +279,20 @@ class ManualTrial:
raise ValueError("reference load must be between 0 and 30 N") raise ValueError("reference load must be between 0 and 30 N")
now_ns = time.perf_counter_ns() now_ns = time.perf_counter_ns()
if self._last_confirm_ns is not None: if self._last_confirm_ns is not None:
self._end_reference(now_ns)
if self._reference_error:
self.finish(
aborted=True,
reason=f"reference write failed: {self._reference_error}",
)
raise OSError(self._reference_error)
self._event( self._event(
"step_ended", "step_ended",
{ {
"step_index": self.step_index - 1, "step_index": self.step_index - 1,
"held_s": (now_ns - self._last_confirm_ns) / 1e9, "held_s": (now_ns - self._last_confirm_ns) / 1e9,
}, },
now_ns=now_ns,
) )
self._event( self._event(
"step_reached", "step_reached",
@@ -210,7 +303,9 @@ class ManualTrial:
"reference_load_n": reference, "reference_load_n": reference,
"reference_kind": "manual", "reference_kind": "manual",
}, },
now_ns=now_ns,
) )
self._active_reference = (self.step_index, branch, target, reference, now_ns)
self.step_index += 1 self.step_index += 1
self._last_confirm_ns = now_ns self._last_confirm_ns = now_ns
return branch, target return branch, target
@@ -226,13 +321,28 @@ class ManualTrial:
raise RuntimeError("no active trial") raise RuntimeError("no active trial")
if not aborted and self.next_step is not None: if not aborted and self.next_step is not None:
raise RuntimeError("complete the load/unload path before finishing") raise RuntimeError("complete the load/unload path before finishing")
now_ns = time.perf_counter_ns()
if self._last_confirm_ns is not None: if self._last_confirm_ns is not None:
self._end_reference(now_ns)
self._event( self._event(
"step_ended", "step_ended",
{"step_index": self.step_index - 1, "held_s": self.held_s}, {
"step_index": self.step_index - 1,
"held_s": (now_ns - self._last_confirm_ns) / 1e9,
},
now_ns=now_ns,
) )
try:
self._reference_file.close()
except OSError as exc:
self._reference_error = str(exc)
if self._reference_error:
aborted = True
reason = f"reference write failed: {self._reference_error}"
self._event( self._event(
"trial_aborted" if aborted else "trial_finished", {"reason": reason} "trial_aborted" if aborted else "trial_finished",
{"reason": reason},
now_ns=now_ns,
) )
self.aborted = aborted self.aborted = aborted
self.worker.remove_sink(self.recorder.on_packet) self.worker.remove_sink(self.recorder.on_packet)
@@ -43,6 +43,9 @@ class CalibrationPanel(QWidget):
outer = QVBoxLayout(self) outer = QVBoxLayout(self)
header = QHBoxLayout() header = QHBoxLayout()
self.model_label = QLabel(self)
self.sensor_model = QLineEdit(self)
self.sensor_model.setObjectName("calibrationSensorModel")
self.sn_label = QLabel(self) self.sn_label = QLabel(self)
self.sensor_sn = QLineEdit(self) self.sensor_sn = QLineEdit(self)
self.sensor_sn.setObjectName("calibrationSensorSn") self.sensor_sn.setObjectName("calibrationSensorSn")
@@ -50,6 +53,8 @@ class CalibrationPanel(QWidget):
self.output_path = QLineEdit(str(Path(outputs_dir()) / "calibrations"), self) self.output_path = QLineEdit(str(Path(outputs_dir()) / "calibrations"), self)
self.output_path.setObjectName("calibrationOutputPath") self.output_path.setObjectName("calibrationOutputPath")
self.output_browse = QPushButton(self) self.output_browse = QPushButton(self)
header.addWidget(self.model_label)
header.addWidget(self.sensor_model)
header.addWidget(self.sn_label) header.addWidget(self.sn_label)
header.addWidget(self.sensor_sn) header.addWidget(self.sensor_sn)
header.addWidget(self.output_label) header.addWidget(self.output_label)
@@ -214,6 +219,7 @@ class CalibrationPanel(QWidget):
site=self._site, site=self._site,
levels=levels, levels=levels,
hold_s=self.hold.value(), hold_s=self.hold.value(),
sensor_model=self.sensor_model.text(),
sensor_sn=self.sensor_sn.text(), sensor_sn=self.sensor_sn.text(),
bundle_path=self._bundle_path, bundle_path=self._bundle_path,
source=source, source=source,
@@ -239,21 +245,21 @@ class CalibrationPanel(QWidget):
raise ValueError(t("calibration.reference_required")) raise ValueError(t("calibration.reference_required"))
self.trial.confirm(float(self.reference.text())) self.trial.confirm(float(self.reference.text()))
self.reference.clear() self.reference.clear()
except (RuntimeError, ValueError) as exc: except (OSError, RuntimeError, ValueError) as exc:
self.result_label.setText(t("calibration.error").format(detail=exc)) self.result_label.setText(t("calibration.error").format(detail=exc))
self._sync() self._sync()
def _finish(self) -> None: def _finish(self) -> None:
try: try:
self.trial.finish() self.trial.finish()
except RuntimeError as exc: except (OSError, RuntimeError) as exc:
self.result_label.setText(t("calibration.error").format(detail=exc)) self.result_label.setText(t("calibration.error").format(detail=exc))
self._sync() self._sync()
def _abort(self) -> None: def _abort(self) -> None:
try: try:
self.trial.finish(aborted=True, reason="operator aborted") self.trial.finish(aborted=True, reason="operator aborted")
except RuntimeError as exc: except (OSError, RuntimeError) as exc:
self.result_label.setText(t("calibration.error").format(detail=exc)) self.result_label.setText(t("calibration.error").format(detail=exc))
self._sync() self._sync()
@@ -289,6 +295,7 @@ class CalibrationPanel(QWidget):
self.levels, self.levels,
self.hold, self.hold,
self.sensor_sn, self.sensor_sn,
self.sensor_model,
self.output_path, self.output_path,
self.output_browse, self.output_browse,
self.bundle_button, self.bundle_button,
@@ -299,13 +306,17 @@ class CalibrationPanel(QWidget):
self.finish_button.setEnabled(recording and self.trial.next_step is None) self.finish_button.setEnabled(recording and self.trial.next_step is None)
self.abort_button.setEnabled(recording) self.abort_button.setEnabled(recording)
step = self.trial.next_step step = self.trial.next_step
self.target_label.setText( if recording and step is not None:
t("calibration.next_step").format( target_text = t("calibration.next_step").format(
branch=t(f"calibration.{step[0]}"), load=step[1] branch=t(f"calibration.{step[0]}"), load=step[1]
) )
if recording and step is not None elif recording:
else t("calibration.no_step") target_text = t("calibration.ready_to_finish")
) elif self.trial.status == "idle":
target_text = t("calibration.before_start")
else:
target_text = t("calibration.no_step")
self.target_label.setText(target_text)
self.progress_label.setText( self.progress_label.setText(
t("calibration.progress").format( t("calibration.progress").format(
done=self.trial.step_index, done=self.trial.step_index,
@@ -333,6 +344,7 @@ class CalibrationPanel(QWidget):
) )
def retranslate(self) -> None: def retranslate(self) -> None:
self.model_label.setText(t("calibration.sensor_model"))
self.sn_label.setText(t("calibration.sensor_sn")) self.sn_label.setText(t("calibration.sensor_sn"))
self.output_label.setText(t("calibration.output")) self.output_label.setText(t("calibration.output"))
self.output_browse.setText(t("source.browse")) self.output_browse.setText(t("source.browse"))
+38 -1
View File
@@ -10,7 +10,13 @@ from types import SimpleNamespace
import numpy as np import numpy as np
import pytest import pytest
from orisys_app_shared.acquisition.models import SdkPacket from orisys_app_shared.acquisition.models import SdkPacket
from orisys_autocal.trial import ManualTrial, load_path, site_at, uv_origin from orisys_autocal.trial import (
REFERENCE_HEADER,
ManualTrial,
load_path,
site_at,
uv_origin,
)
class Worker: class Worker:
@@ -95,6 +101,7 @@ def test_manual_trial_records_video_stamps_and_load_unload_events(tmp_dir):
site=site, site=site,
levels=[5, 10], levels=[5, 10],
hold_s=20, hold_s=20,
sensor_model="Finger D40",
sensor_sn="F-001", sensor_sn="F-001",
bundle_path=str(bundle), bundle_path=str(bundle),
source={"kind": "camera", "value": 0}, source={"kind": "camera", "value": 0},
@@ -108,6 +115,7 @@ def test_manual_trial_records_video_stamps_and_load_unload_events(tmp_dir):
report = trial.recorder.stop(wait=True) report = trial.recorder.stop(wait=True)
assert report.ok assert report.ok
assert trial.poll() == "complete" assert trial.poll() == "complete"
assert "Finger-D40_F-001" in directory.name
assert len(list(directory.glob("raw_*.mp4"))) == 1 assert len(list(directory.glob("raw_*.mp4"))) == 1
with (directory / "frames.csv").open(encoding="utf-8", newline="") as handle: with (directory / "frames.csv").open(encoding="utf-8", newline="") as handle:
row = next(csv.DictReader(handle)) row = next(csv.DictReader(handle))
@@ -115,6 +123,30 @@ def test_manual_trial_records_video_stamps_and_load_unload_events(tmp_dir):
assert row["source_frame_id"] == "7" assert row["source_frame_id"] == "7"
assert row["time_basis"] == "host_camera" assert row["time_basis"] == "host_camera"
assert row["capture_dropped"] == "0" assert row["capture_dropped"] == "0"
with (directory / "reference.csv").open(encoding="utf-8", newline="") as handle:
reader = csv.DictReader(handle)
assert tuple(reader.fieldnames) == REFERENCE_HEADER
references = list(reader)
assert [row["branch"] for row in references] == ["load", "load", "unload", "unload"]
assert [row["F_ref_normal_N"] for row in references] == [
"5.0",
"10.0",
"5.0",
"0.0",
]
assert all(
row["F_ref_x_N"] == row["F_ref_y_N"] == row["F_ref_z_N"] == ""
for row in references
)
assert all(row["stability_verified"] == "0" for row in references)
assert all(
int(row["t_start_host_mono_ns"]) < int(row["t_end_host_mono_ns"])
for row in references
)
assert all(
previous["t_end_host_mono_ns"] == following["t_start_host_mono_ns"]
for previous, following in zip(references, references[1:])
)
with (directory / "events.csv").open(encoding="utf-8", newline="") as handle: with (directory / "events.csv").open(encoding="utf-8", newline="") as handle:
events = list(csv.DictReader(handle)) events = list(csv.DictReader(handle))
reached = [event for event in events if event["kind"] == "step_reached"] reached = [event for event in events if event["kind"] == "step_reached"]
@@ -128,12 +160,15 @@ def test_manual_trial_records_video_stamps_and_load_unload_events(tmp_dir):
with (directory / "session.json").open(encoding="utf-8") as handle: with (directory / "session.json").open(encoding="utf-8") as handle:
session = json.load(handle) session = json.load(handle)
assert session["site"]["uv_xy_px"] == [0, 1] assert session["site"]["uv_xy_px"] == [0, 1]
assert session["sensor_model"] == "Finger D40"
assert session["reference_file"] == "reference.csv"
assert session["site"]["mesh_sha256"] assert session["site"]["mesh_sha256"]
def test_calibration_page_keeps_precise_uv_site_when_switching_to_live_view( def test_calibration_page_keeps_precise_uv_site_when_switching_to_live_view(
make_window, make_window,
): ):
from orisys_app_shared.i18n import t
from orisys_autocal.sensor.uv_flat import FlatSurface from orisys_autocal.sensor.uv_flat import FlatSurface
window = make_window() window = make_window()
@@ -151,6 +186,8 @@ def test_calibration_page_keeps_precise_uv_site_when_switching_to_live_view(
uv_max=(1, 1), uv_max=(1, 1),
) )
panel = window.calibration_panel panel = window.calibration_panel
assert panel.start_button.text() == t("calibration.start")
assert panel.target_label.text() == t("calibration.before_start")
panel.set_flat(flat, "bundle") panel.set_flat(flat, "bundle")
panel._picked(2, 1) panel._picked(2, 1)
assert (panel.u_spin.value(), panel.v_spin.value()) == (-1, 2) assert (panel.u_spin.value(), panel.v_spin.value()) == (-1, 2)
@@ -46,7 +46,7 @@ from orisys_app_shared.record.video_writer import VideoSegments
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
RECORD_FORMAT_VERSION = "1.3" RECORD_FORMAT_VERSION = "1.4"
# Lifecycle states. ``failed`` and ``stopped`` are terminal; a recorder that has # Lifecycle states. ``failed`` and ``stopped`` are terminal; a recorder that has
# not reached a terminal state must not be reused. # not reached a terminal state must not be reused.
@@ -80,17 +80,17 @@ FORCES_HEADER = (
) )
FRAMES_HEADER = ( FRAMES_HEADER = (
"frame_id", "frame_id",
"source_frame_id",
"t_rel_s", "t_rel_s",
"t_host_mono_ns",
"t_host_utc_ns",
"acquired_at", "acquired_at",
"time_basis",
"media_time_ms",
"segment", "segment",
"segment_frame", "segment_frame",
"in_video", "in_video",
"dropped", "dropped",
"t_host_mono_ns",
"t_host_utc_ns",
"source_frame_id",
"time_basis",
"media_time_ms",
"capture_dropped", "capture_dropped",
) )
#: ``frame_id`` was added so an operator action can be tied to a frame position. #: ``frame_id`` was added so an operator action can be tied to a frame position.
@@ -345,8 +345,14 @@ class ExperimentRecorder:
calibration_id: str = "", calibration_id: str = "",
calibration_version: str = "", calibration_version: str = "",
preroll=(), preroll=(),
directory_label: str = "",
) -> Path: ) -> Path:
"""Create the experiment directory and start the writer thread.""" """Create the experiment directory and start the writer thread."""
if directory_label and (
directory_label in {".", ".."}
or not all(char.isalnum() or char in "-_" for char in directory_label)
):
raise ValueError("directory label must contain only letters, numbers, - or _")
# A previous writer that has not reached a terminal state still owns the # A previous writer that has not reached a terminal state still owns the
# files and the statistics; reusing it would silently mix two sessions. # files and the statistics; reusing it would silently mix two sessions.
if self._thread is not None and self._thread.is_alive(): if self._thread is not None and self._thread.is_alive():
@@ -363,7 +369,8 @@ class ExperimentRecorder:
# lifecycle checks, so caller mutations cannot change this run. # lifecycle checks, so caller mutations cannot change this run.
self.config = replace(config if config is not None else self.config) self.config = replace(config if config is not None else self.config)
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
directory = Path(root) / f"{stamp}_session{session_id:04d}_{uuid4().hex}" label = f"_{directory_label}" if directory_label else ""
directory = Path(root) / f"{stamp}{label}_session{session_id:04d}_{uuid4().hex}"
try: try:
directory.mkdir(parents=True, exist_ok=False) directory.mkdir(parents=True, exist_ok=False)
except Exception as exc: # noqa: BLE001 - reported to the caller except Exception as exc: # noqa: BLE001 - reported to the caller
@@ -1141,17 +1148,17 @@ class _SessionWriter:
self._frames[1].writerow( self._frames[1].writerow(
( (
sample.frame_id, sample.frame_id,
sample.source_frame_id if sample.source_frame_id is not None else "",
repr(t_rel), repr(t_rel),
sample.capture_mono_ns or "",
sample.capture_utc_ns or "",
acquired, acquired,
sample.time_basis,
format_value(sample.media_time_ms),
segment, segment,
segment_frame, segment_frame,
1 if segment else 0, 1 if segment else 0,
0, 0,
sample.capture_mono_ns or "",
sample.capture_utc_ns or "",
sample.source_frame_id if sample.source_frame_id is not None else "",
sample.time_basis,
format_value(sample.media_time_ms),
sample.capture_dropped, sample.capture_dropped,
) )
) )
@@ -1170,7 +1177,7 @@ class _SessionWriter:
wall_t or time.time(), timezone.utc wall_t or time.time(), timezone.utc
).isoformat(timespec="milliseconds") ).isoformat(timespec="milliseconds")
self._frames[1].writerow( self._frames[1].writerow(
(frame_id, repr(t_rel), acquired, 0, 0, 0, 1, "", "", "", "", "", "") (frame_id, "", repr(t_rel), "", "", acquired, "", "", 0, 0, 0, 1, "")
) )
self.gap_rows += 1 self.gap_rows += 1
+14 -1
View File
@@ -92,6 +92,19 @@ def test_directory_layout_and_headers(root):
assert tuple(next(csv.reader(handle))) == header, name assert tuple(next(csv.reader(handle))) == header, name
def test_frame_columns_are_grouped_by_identity_time_video_and_status(root):
directory, report = _record(root, [3], config=RecordConfig(video=False))
assert report.ok
with (directory / "frames.csv").open(encoding="utf-8", newline="") as handle:
reader = csv.reader(handle)
assert tuple(next(reader)) == (
"frame_id", "source_frame_id", "t_rel_s", "t_host_mono_ns",
"t_host_utc_ns", "acquired_at", "time_basis", "media_time_ms",
"segment", "segment_frame", "in_video", "dropped", "capture_dropped",
)
assert len(next(reader)) == len(FRAMES_HEADER)
def test_unavailable_values_are_empty_not_zero(root): def test_unavailable_values_are_empty_not_zero(root):
recorder = ExperimentRecorder(RecordConfig(video=False)) recorder = ExperimentRecorder(RecordConfig(video=False))
directory = recorder.start(root=root, session_id=1, metadata={}, calibration_id="cal-1") directory = recorder.start(root=root, session_id=1, metadata={}, calibration_id="cal-1")
@@ -180,7 +193,7 @@ def test_queue_overflow_is_counted_and_alarmed(root):
# A queue that holds one item, with a writer we deliberately stall. # A queue that holds one item, with a writer we deliberately stall.
recorder = ExperimentRecorder(RecordConfig(video=True, max_queue=16)) recorder = ExperimentRecorder(RecordConfig(video=True, max_queue=16))
recorder.set_alarm_handler(alarms.append) recorder.set_alarm_handler(alarms.append)
directory = recorder.start(root=root, session_id=1, metadata={}) recorder.start(root=root, session_id=1, metadata={})
# Push far more than the queue can hold without letting the writer catch up. # Push far more than the queue can hold without letting the writer catch up.
for index in range(4000): for index in range(4000):