Compare commits

..

5 Commits

Author SHA1 Message Date
ykcheungab cf86bd0702 Merge branch 'main' of ssh://892887cvmk08.aicp.io:24928/orisys-tech/orisys-app 2026-10-10 14:16:54 +08:00
ykcheungab f5ba64e01a Merge branch 'main' of ssh://892887cvmk08.aicp.io:24928/orisys-tech/orisys-app 2026-10-10 14:15:35 +08:00
ykcheungab 768eeb2040 补充 Autocal 最小标定流程实施计划
划分工作包、依赖和子代理职责,估算工作量及设备接入增量。文档门禁通过。
2026-10-10 14:05:12 +08:00
ykcheungab d2122e60e1 整理 Autocal 操作步骤与数据流表格
汇总文件与参数,区分现有软件链路和待实现流程。文档门禁通过。
2026-10-10 13:15:14 +08:00
ykcheungab b15842ec00 补充 Autocal 最小标定数据流程草稿
明确像素到三维映射、逐级加载卸载及每事件独立视频,压缩坐标说明。文档门禁通过,流程尚未实现。
2026-10-10 13:10:45 +08:00
17 changed files with 331 additions and 4 deletions
@@ -0,0 +1,115 @@
# Autocal 标定操作与数据格式草稿
状态:待评审,2026-10-10。用途:方案及操作指南。近期目标是跑通多测点法向采集、保存和离线重算;下列操作是首版目标流程,尚非全部可执行的软件说明。
## 范围与软件现状
| 项目 | 当前约定 |
|---|---|
| 标定范围 | 多测点法向力,暂定 0–30 N;剪切及组合载荷后续再做 |
| 压头 | 直径 6 mm、球面半径 3 mm 的半球压头 |
| 参考 | 砝码或 50 N/2 N·m FT sensor,首版先接通一种 |
| 图像 | 一个完整测试事件一个连续视频,档位间不切段 |
| 交付 | 单个 session 能重新打开、追溯参考载荷并离线重算 |
[LF606 规格书](references/lf606-d40-h18-4x-spec.pdf)量程表包含 50 N/2 N·m 档位,但图签为 200 N/8 N·m;实机以铭牌及标定证书核对。
| 功能 | 源码现状 | 首版需要补齐 |
|---|---|---|
| 设备设置、轴操作、测点姿态预览 | Autocal 已有对应模块 | 与采集事件关联 |
| `.oryseq` 导入导出 | 已有;见 [sequence-format.md](sequence-format.md) | 载荷循环由 trial 的 steps 表达,不假定旧序列已支持 |
| 自动执行、传感器及标定点页面 | 运行入口提示未实现,相关页面占位 | 最小事件控制和采集入口 |
| 图像、读数、事件记录 | Studio/shared 已有记录链 | 接入 Autocal,增加 trial/step、参考流及独立事件视频 |
| 新格式回放、拟合及导出 | 本文定义目标契约 | 关联校验与最小离线处理 |
## 操作员逐步流程
设备操作沿用 [workflow.md](workflow.md) 的准备、运动边界及停机流程。以下不新增自动运动授权;尚未接通的步骤不能靠占位按钮完成。
| 步骤 | 操作员动作 | 软件记录/通过条件 |
|---|---|---|
| 1. 建立会话 | 填写传感器 SN、gel ID、实验台、压头和参考设备;选保存位置 | 创建唯一 `session_id`;保存身份和配置快照 |
| 2. 检查设备 | 完成设备设置、回零及既定检查,核对相机和参考输入 | 状态可读;运动边界按既定流程确认 |
| 3. 确认测点 | 在有效 UV 区域选点,核对三维位置和姿态 | 分配 `site_id`,UV→3D 映射有效 |
| 4. 设置循环 | 选择载荷档位、保持时间及重复数;预览正反路径 | 保存 `load_levels_n`、`hold_s`、`repeats`;档位不超本次确认范围 |
| 5. 建立基准 | 无接触时检查零载荷,保存参考图像和必要历史 | 保存归零/基准事件及 `baseline_id` |
| 6. 开始事件 | 到达确认的测点姿态后,在加载前启动记录 | 新建 `trial_id` 和视频;确认视频与参考记录正常后加载 |
| 7. 逐级加载 | 按路径到各目标档位并保持,标记实际到达时间 | 每档有 `step_id`、目标及实际参考力、分支和时间范围 |
| 8. 逐级卸载 | 按相同档位下降到零,保留恢复过程 | 连续记录卸载分支;循环内不重新归零或重置图像基准 |
| 9. 结束/重复 | 检查恢复与记录状态,停止事件;重复或移到下一点 | 完成写入后标记状态;每次重复使用新 trial 和视频 |
| 10. 复核会话 | 重新打开文件,回放至少一个完整事件,检查档位对应关系 | 缺失、丢帧和中断可见;保留完整 trial 作独立验证 |
推荐路径为 `0 → F1 → F2 → Fmax → F2 → F1 → 0`,具体档位待定。逐级卸载提供与加载分支相同载荷的对照;各档采用相同保持规则。保持时间先支持 10–30 s 配置,人工选择分析窗口并记录原因;到时不自动等于稳定。异常时按既定流程停止,保留未完成记录,重测用新 trial。
## 坐标与关联
| 对象 | 保存内容 | 约定 |
|---|---|---|
| 测点 | `uv_xy`、`position_sensor_mm` | UV 单位 pixel,可为亚像素值;直接映射到 mesh 的 mm 位置 |
| UV 定义 | `uv_origin_px`、图像尺寸、轴正方向 | 原点在有效 mask 内底部中间;允许负 Y |
| 几何 | `geometry_id`、mesh/mask/映射文件及哈希 | 固定映射版本,二维与三维值可相互核对 |
| 姿态 | `T_platform_from_sensor`、`pose_source`、激励方向 | 记录传感器→平台变换,区分实测和指令;normal 不必单独保存 |
| 关联层级 | `session → site → trial → step → samples` | trial 是完整循环,step 是循环内档位;帧与参考流按时间关联 |
## 文件及内部参数
以下为新格式建议,不是现有共享记录器文件的原样清单。`sites`、`trials` 和 `steps` 先内嵌于 `session.json`。
| 文件 | 主要字段/数据 | 写入时机 |
|---|---|---|
| `session.json` | `format_version`、`session_id`、身份、配置快照、坐标定义、文件清单、sites、trials、状态 | 会话建立、事件完成及结束时 |
| `sequence.oryseq` | 实际使用的测点计划;遵循现有格式 | 开始采集前保存快照 |
| `video/<trial_id>.<ext>` | 算法入口图像、完整循环及恢复 | 每 trial 开始至结束 |
| `frames.csv` | `session_id`、`trial_id`、`frame_id`、`t_rel_s`、视频路径、段内帧号、丢帧标记 | 每帧 |
| `baseline/` | 基准图像、初始状态或预录引用、`baseline_id` | 建立/重置基准时 |
| `reference.csv` | `trial_id`、`t_rel_s`、`clock_id`、实测力分量、有效性 | FT 自身采样频率;砝码方式可不生成此流 |
| `motion.csv` | 时间、目标/反馈位姿、有效性 | 有实际反馈时保存;缺少反馈须注明 |
| `events.csv` | 时间、trial/step、事件类型、原因 | 到档、阶段切换、归零、异常及中断 |
| `derived/<processing_id>/` | 对齐样本、特征、拟合结果、验证指标和处理配置 | 离线处理;不覆盖原始文件 |
| 参数归属 | 主要参数 | 单位/填写来源 |
|---|---|---|
| session 身份 | `sensor_sn`、`gel_id`、`bench_id`、`indenter_id`、`reference_source` | 操作员填写或设备读取 |
| session 采集 | 分辨率、像素格式、编码、版本、滤波与归零配置 | 软件快照;编码待实测确认 |
| session 时间 | 各流 `clock_id`、时间戳含义、偏移及同步误差 | 软件记录;不能按行号配对 |
| site | `site_id`、UV、三维位置、几何关联 | pixel/mm;软件映射 |
| trial | `trial_id`、`site_id`、重复编号、姿态、`baseline_id`、视频、起止时间、状态 | 软件分配并记录实际执行 |
| trial 计划 | `load_levels_n`、`hold_s`、`repeats` | N/s/次数;操作员设置 |
| step | `step_id`、`target_load_n`、`branch`、起止时间、`analysis_window_s` | N/会话相对秒;branch 为 load/unload/zero |
| step 砝码参考 | 质量、重力加速度、传力方向、有效时段及 `reference_load_n` | kg/m/s²/N;仅在传力条件成立时使用 m×g |
| 质量 | `valid`、`invalid_reason`、丢帧/中断状态 | 未知不补零;算法失败不等于图像不可用 |
## 软件数据流与责任
### 当前实际链路
| 链路 | 代码位置/输出 | 对 Autocal 的含义 |
|---|---|---|
| 运行请求 → 未实现提示 | [window.py](../src/orisys_autocal/ui/window.py) 的 `_on_run_requested` | 尚无自动采集循环 |
| 页面装配 → 占位页面 | [workspace.py](../src/orisys_autocal/ui/workspace.py) | 传感器、表面和标定点页尚未接通 |
| Studio 采集包 → 记录通道 | [recording.py](../../studio/src/orisys_studio/recording.py) 的 `RecordingChannel.on_packet` | 将采集包分发给会话存储、标定采样器及文件记录器 |
| 采集包 → 有界队列 → 写入线程 | [experiment.py](../../../shared/src/orisys_app_shared/record/experiment.py) | 当前写 session、forces、frames、events、分段 MP4 或 PNG;不能直接视为新格式已完成 |
### 首版待接通链路
| 顺序 | 输入 → 处理 → 输出 | 负责模块 |
|---|---|---|
| 1 | 操作员设置 + `.oryseq` → sites/载荷路径 → session 快照 | Autocal 会话与事件控制 |
| 2 | UV + 固定映射 → mesh 位置 + 平台姿态 → site/trial | 几何及现有姿态计算能力 |
| 3 | 相机 → SDK 采集包 → 视频、帧索引 | 复用 shared 采集与记录能力 |
| 4 | FT/砝码 + 平台反馈 + 档位事件 → 带时间的数据流 | Autocal 参考输入及事件控制;与图像并行 |
| 5 | 事件结束 → 完成写入 + 完整性检查 → trial 状态 | 文件记录器及会话控制 |
| 6 | 视频解码 + 基准/配置 → SDK 特征重算;时间对齐参考 → 拟合/验证 | 离线处理,结果存 derived |
保留现有算法通道值时可沿用 `forces.csv`,但它不是独立参考力。业务层组织试验,SDK 负责图像算法,共享记录器负责文件写入;不让 SDK 依赖 Autocal UI。
## 首版验收与待定事项
| 验收 | 通过条件 |
|---|---|
| 数据关联 | 每个视频可追溯到测点、姿态、基准、加载/卸载档位及参考力 |
| 可回放 | 重新打开后能解码完整 trial,恢复处理状态并重算选定窗口 |
| 有效性 | 丢帧、参考缺失、中断及失败显式可见;原始记录不被重处理覆盖 |
| 最小验证 | 至少留出完整 trial,分别比较加载/卸载的法向误差;阈值待定 |
开跑前仍须确定载荷档位、轴方向及精确原点、参考采集与同步、视频编码和恢复窗口。先实测单 trial 的容量及丢帧,再估算整场存储。视频空档不承诺连续回放。空间模型、实时集成及长期可靠性验证后续再做;本流程不证明剪切或多接触精度。
@@ -0,0 +1,98 @@
# Autocal 最小标定流程实施计划
状态:待评审,2026-10-10。用途:方案。建议使用两个实施子代理,主代理负责接口、UI 和集成;本计划不启动代理、不实施设备动作。需求以[操作与数据格式草稿](calibration-data-format.zh.md)为准。
## 首版边界
| 交付 | 首版选择 |
|---|---|
| 采集对象 | 先支持一种实际传感器配置及相机后端,不承诺所有型号 |
| 测点 | 复用现有选择和姿态能力;UV pixel→3D mm,保存映射身份 |
| 参考输入 | 人工确认砝码档位,记录质量、传力条件及有效时段;FT 自动采集另估 |
| 载荷循环 | 操作员逐级加载卸载,软件记录到档、保持及恢复;暂不做闭环力控制 |
| 视频 | 每 trial 一个独立连续视频,含必要预录/基准;编码必须通过本机回放验证 |
| 复核 | 重新打开 session,按 step 选择窗口,重算特征并对照参考载荷 |
| 验证 | 独立 trial 的法向拟合误差及加载卸载差异;不承诺未经验证的实时精度 |
人工砝码方案依赖现有装夹能够实现有效法向传力;若实际只能用 FT,自动接口须提前纳入关键路径。0–30 N 是需求范围,不是已批准的设备动作上限。自动测点遍历、自动压入、复杂空间模型、实时部署及新安装包均不在首版。
## 现有基础及缺口
基线为 app `d2122e60e1ed2468fb72cb351a07b0182166a09f`。以下是源码审阅结论,尚未进行设备或吞吐测试。
| 位置 | 可复用 | 尚缺 |
|---|---|---|
| `apps/autocal/src/orisys_autocal/sequence_io.py` | `.oryseq` 校验与保存 | trial/step 载荷路径 |
| `sensor/export_bundle.py`、`sensor/uv_flat.py`、`motion/kinematics.py` | 几何、UV 与姿态能力;bundle 已有物理尺度处理 | 验证计算管线 UV 与预览映射、坐标尺度一致 |
| `shared/src/orisys_app_shared/acquisition/` | 采集 worker、帧号、时间、sink | Autocal 生命周期与最小预览接入 |
| `shared/.../record/experiment.py`、`video_writer.py` | 有界队列、后台写入、丢帧及错误处理 | 当前为滚动 MP4;事件视频、编码和新 manifest 不可直接视作已支持 |
| `apps/studio/src/orisys_studio/recording.py`、`replay_analysis.py` | 已有记录组织和回放经验 | Autocal 不应直接依赖 Studio 私有模块;需要时仅提取共同核心到 shared |
| Autocal `ui/window.py`、`ui/workspace.py` | 设备和页面装配 | 运行入口未实现,采集及标定点页占位 |
原始 mesh 不应直接当作 mm;沿用 `SensorBundle.vertices_mm` 等经过验证的物理输出。SDK 只负责图像算法,Autocal 组织试验,shared 提供实际共用的采集/存储能力。
## 工作包与估算
单位为工程人时,包含针对性测试和代码审查,不是代理运行时承诺。按现有环境可用、无设备通信等待、首版采用人工档位估算;各行不重复计算集成工作。
| 包 | 内容及可观察交付 | 负责人/写入范围 | 依赖 | 人时 |
|---|---|---|---|---:|
| P0 | 固定 schema、事件 API、时间原点、状态及示例 session;验证实际相机和编码可用性 | 主代理:契约、共享模型声明 | 无 | 3–4 |
| P1 | 相机开关、基准/预录、trial 视频、CSV/manifest 写入、排空与失败状态 | A:shared acquisition/record 及对应测试;不改 UI | P0 | 10–14 |
| P2 | session/site/trial/step 控制、人工载荷确认、阶段时间及状态约束 | B:Autocal 无 Qt 的标定控制及测试 | P0 | 8–12 |
| P3 | 最小会话表单、测点表、预览、开始/到档/结束/中止、状态显示 | 主代理:Autocal UI、配置及集成入口 | P0 后可用桩;实接 P1/P2 | 8–12 |
| P4 | 新格式读取、逐事件回放、基准恢复、窗口特征重算、最小拟合与误差表 | B:离线处理及测试;共享提取先协调 | P2;集成依赖 P1 样本 | 6–10 |
| P5 | 故障、旧消费者回归、GUI 流程、实际采集样本验收与交接 | 主代理集成,A/B 修复各自模块 | P1–P4 | 5–8 |
| 合计 | 最小可采集、可回放并离线复核 | 主代理 + A + B | 按下表排程 | 40–60 |
## 并行安排与代理数量
| 配置 | 预期软件交付窗口 | 取舍 |
|---|---|---|
| 主代理单独执行 | 约 5–8 个工作日 | 接口协调少,所有开发串行 |
| 主代理 + 1 子代理 | 约 4–6 个工作日 | 主代理承担控制、UI 和回放,容易成为瓶颈 |
| 主代理 + 2 子代理 | 约 3–4 个工作日,建议预留到第 5 天联调 | 推荐;采集存储与事件控制并行,主代理保持集成连续 |
| 主代理 + 3 子代理 | 通常仍约 3–4 个工作日 | 第三个仅在后期做独立审查/故障验证;不建议再拆一条实施线 |
按每日 8 小时有效工程时间估算;AI 编码可能更快,硬件、人工采样和联调不会按代理数等比加速。最短两三天可以优先完成采集里程碑,但不能把未完成的重算验收称为完整交付。
| 阶段 | 主代理 | A | B | 出口 |
|---|---|---|---|---|
| 第 1 天 | P0,随后做 UI 骨架 | P1 编码及采集验证 | P2 控制及合成事件 | 契约冻结,单 trial 合成数据能保存 |
| 第 2 天 | P3 接通控制与记录 | P1 收尾及错误路径 | P2 收尾,开始 P4 | G1:人工档位的多事件采集可用 |
| 第 3–4 天 | P5 操作流程及集成 | 修复记录与性能问题 | P4,留出 trial 验证 | G2:重新打开、重算及对照通过 |
| 预留第 5 天 | 实际设备联调和交接 | 定向修复 | 定向修复 | 关闭实机问题或明确未验证项 |
依赖为 `P0 → (P1 ∥ P2 ∥ P3骨架) → P3实接`,随后 P4/P5 完成闭环。具体设备可用时间会改变日程,不能把等待藏进人时。
## 并行接口与文件所有权
P0 先提供一份被各包共用的模型和合成数据,避免三个代理自行定义不同 schema。函数名称在实施时确定,不为假想插件建立框架。
| 边界 | 必须固定的输入/输出 |
|---|---|
| 控制 → 记录 | session/trial/step 身份、阶段事件、姿态和参考有效时段;记录器返回写入状态 |
| 采集 → 记录 | `SdkPacket`、帧身份、采集时刻、图像及有效性;sink 不执行 UI 或阻塞磁盘写入 |
| 记录 → 回放 | manifest、视频逐帧索引、基准、配置快照及缺失标记 |
| UI → 控制 | 操作请求;控制检查状态后返回结果,UI 不直接改 manifest |
A 独占新格式的文件写入,B 产生业务事件;主代理独占 `window.py`、`workspace.py` 及契约公共声明。修改 shared 行为必须保持 Studio 默认格式和生命周期。原记录器若不能满足事件视频路径,通过兼容的公共入口扩展,不能以 UI 层移动录像文件来补救。
## 风险、增量与验收
| 项目 | 处理及增量 |
|---|---|
| FT 自动输入 | 未找到现成 LF606 接口;规格书不含采集器协议。确认接口、样例和设备后暂估另加 8–20 人时;协议开发或采购等待须重估 |
| 编码不可用或有损 | P0/P1 做实际编码解码检查;不能悄悄用普通 MP4替代无损要求。可先交演示录像,但 G2 重算精度仍待验证 |
| 有状态算法回放 | P1 保存基准及初始化历史,P4 独立比较重算结果;失败时定位状态缺口,不只比较导出系数 |
| 自动加载/运动 | 暂不纳入;需要控制策略、传力验证和独立运动验收,不能只增加一个循环按钮 |
| 检查 | 通过条件 |
|---|---|
| G1 采集 | 至少两个测点、每点两个 trial;每 trial 包含正反载荷档位,视频连续且身份唯一 |
| G2 重算 | 全新进程读取文件,恢复基准/历史,重算窗口并对齐独立参考;完整 trial 留出验证 |
| 故障 | 中止、磁盘写失败、丢帧、参考缺失、相机关闭可见;未写完不标完成 |
| 回归 | Autocal 现有测试、shared 记录/生命周期/回放测试、Studio 相关消费者通过;GUI 与硬件证据分别报告 |
| 完成 | 文档和 lint 门禁通过;实际编码吞吐、容量及误差有证据;缺失设备不报通过 |
首版通过 G1/G2 后停止扩展。实施开始前先确认所用 sensor_type、参考方式及设备可用性,决定是否把 FT 增量纳入关键路径。
+3 -1
View File
@@ -527,6 +527,8 @@
"contact_arrow_unavailable": "This scene cannot draw the contact arrow.", "contact_arrow_unavailable": "This scene cannot draw the contact arrow.",
"flow_threshold": "Arrow threshold", "flow_threshold": "Arrow threshold",
"flow_threshold_tip": "Arrows below this displacement (px) are not drawn; shared by 2D and 3D.", "flow_threshold_tip": "Arrows below this displacement (px) are not drawn; shared by 2D and 3D.",
"depth_threshold": "Depth display threshold",
"depth_threshold_tip": "Values below this relative depth response use the base ramp colour; 0 disables it. Shared by 2D and 3D without changing measurements.",
"quantity_flow_magnitude": "Optical flow magnitude", "quantity_flow_magnitude": "Optical flow magnitude",
"quantity_depth_response": "Depth response", "quantity_depth_response": "Depth response",
"preset_default": "Default view", "preset_default": "Default view",
@@ -544,7 +546,7 @@
"max_fps_tip": "How many frames per second the 3D view draws. A display budget only: the camera and the recording are unaffected, and a dropped frame here is never a dropped sample.", "max_fps_tip": "How many frames per second the 3D view draws. A display budget only: the camera and the recording are unaffected, and a dropped frame here is never a dropped sample.",
"reset_view": "Reset view", "reset_view": "Reset view",
"settings_title": "3D display settings", "settings_title": "3D display settings",
"settings_hint": "Per-frame adaptive applies to both 2D and 3D.", "settings_hint": "Depth display threshold and per-frame adaptive apply to both 2D and 3D.",
"legend": "{quantity} [{unit}] · {quality}", "legend": "{quantity} [{unit}] · {quality}",
"legend_none": "No measurement on screen — the surface model is shown uncoloured", "legend_none": "No measurement on screen — the surface model is shown uncoloured",
"quality_calibrated": "calibrated", "quality_calibrated": "calibrated",
+3 -1
View File
@@ -527,6 +527,8 @@
"contact_arrow_unavailable": "当前场景画不了接触箭头。", "contact_arrow_unavailable": "当前场景画不了接触箭头。",
"flow_threshold": "箭头阈值", "flow_threshold": "箭头阈值",
"flow_threshold_tip": "低于此位移(像素)不画箭头;二维与三维共用。", "flow_threshold_tip": "低于此位移(像素)不画箭头;二维与三维共用。",
"depth_threshold": "深度显示阈值",
"depth_threshold_tip": "低于此相对深度响应的区域显示为色标基色;0 表示关闭。二维深度图与三维表面共用,不改变测量数据。",
"quantity_flow_magnitude": "光流幅值", "quantity_flow_magnitude": "光流幅值",
"quantity_depth_response": "深度响应", "quantity_depth_response": "深度响应",
"preset_default": "默认视角", "preset_default": "默认视角",
@@ -544,7 +546,7 @@
"max_fps_tip": "三维视图每秒绘制多少帧。这只是显示预算:摄像头与记录不受影响,这里丢的帧永远不会变成丢的样本。", "max_fps_tip": "三维视图每秒绘制多少帧。这只是显示预算:摄像头与记录不受影响,这里丢的帧永远不会变成丢的样本。",
"reset_view": "复位视角", "reset_view": "复位视角",
"settings_title": "三维显示设置", "settings_title": "三维显示设置",
"settings_hint": "每帧自适应同时作用于二维和三维。", "settings_hint": "深度显示阈值与每帧自适应同时作用于二维深度图和三维表面。",
"legend": "{quantity} [{unit}] · {quality}", "legend": "{quantity} [{unit}] · {quality}",
"legend_none": "屏幕上没有测量值 — 仅显示不着色的表面模型", "legend_none": "屏幕上没有测量值 — 仅显示不着色的表面模型",
"quality_calibrated": "已标定", "quality_calibrated": "已标定",
@@ -251,6 +251,9 @@ class FrameView:
theme=theme, theme=theme,
draw_style=draw_style, draw_style=draw_style,
depth_range=span, depth_range=span,
depth_threshold=float(
getattr(self.depth_display_settings(), "depth_threshold", 0.0)
),
canvas_bgr=hex_to_bgr(color(theme, "arrow_canvas", "#0B1830")), canvas_bgr=hex_to_bgr(color(theme, "arrow_canvas", "#0B1830")),
domain_mask=boundary, domain_mask=boundary,
# A frame read from a stored analysis result says which policy its *recovered* # A frame read from a stored analysis result says which policy its *recovered*
@@ -164,6 +164,7 @@ uniform sampler2D u_valid;
uniform vec2 u_inv_domain_size; uniform vec2 u_inv_domain_size;
uniform float u_lower; uniform float u_lower;
uniform float u_upper; uniform float u_upper;
uniform float u_depth_threshold;
uniform vec4 u_invalid_color; uniform vec4 u_invalid_color;
out vec4 frag_color; out vec4 frag_color;
@@ -181,7 +182,11 @@ void main() {
return; return;
} }
float span = max(u_upper - u_lower, 1e-12); float span = max(u_upper - u_lower, 1e-12);
float t = clamp((texture(u_field, texel).r - u_lower) / span, 0.0, 1.0); float value = texture(u_field, texel).r;
float t = clamp((value - u_lower) / span, 0.0, 1.0);
if (u_depth_threshold > 0.0 && value < u_depth_threshold) {
t = 0.0;
}
frag_color = vec4(texture(u_lut, vec2(t, 0.5)).rgb, 1.0); frag_color = vec4(texture(u_lut, vec2(t, 0.5)).rgb, 1.0);
} }
""" """
@@ -717,6 +722,7 @@ class GLRenderer:
#: Range of the valid samples of the field currently uploaded, for the #: Range of the valid samples of the field currently uploaded, for the
#: automatic colour scale. ``None`` until a frame carries valid data. #: automatic colour scale. ``None`` until a frame carries valid data.
self._scalar_range: tuple[float, float] | None = None self._scalar_range: tuple[float, float] | None = None
self._field_quantity = ""
#: The 2D canvas's latched depth ``(lo, hi)``, when the window has one. #: The 2D canvas's latched depth ``(lo, hi)``, when the window has one.
#: Depth automatic colouring uses this so both legends name one interval. #: Depth automatic colouring uses this so both legends name one interval.
self._shared_depth_scale: tuple[float, float] | None = None self._shared_depth_scale: tuple[float, float] | None = None
@@ -771,6 +777,7 @@ class GLRenderer:
# longer on screen: keeping it made the first automatic draw of the new scene use # longer on screen: keeping it made the first automatic draw of the new scene use
# the previous scene's bounds. # the previous scene's bounds.
self._scalar_range = None self._scalar_range = None
self._field_quantity = ""
self._last_scale = None self._last_scale = None
self._last_scale_mode = "" self._last_scale_mode = ""
uvs = np.asarray(assets.binding.uvs, dtype=np.float32) uvs = np.asarray(assets.binding.uvs, dtype=np.float32)
@@ -847,6 +854,7 @@ class GLRenderer:
previous=self._scalar_range, previous=self._scalar_range,
shared_depth_scale=self._shared_depth_scale, shared_depth_scale=self._shared_depth_scale,
) )
self._field_quantity = str(getattr(field.descriptor, "quantity", ""))
self._upload_overlays(frame) self._upload_overlays(frame)
def _upload_field_textures(self, field: Any) -> None: def _upload_field_textures(self, field: Any) -> None:
@@ -1070,6 +1078,13 @@ class GLRenderer:
if part.role == "sensing": if part.role == "sensing":
_set_float(GL, locations.get("u_lower"), lower) _set_float(GL, locations.get("u_lower"), lower)
_set_float(GL, locations.get("u_upper"), upper) _set_float(GL, locations.get("u_upper"), upper)
_set_float(
GL,
locations.get("u_depth_threshold"),
settings.depth_threshold
if self._field_quantity == "depth_response"
else 0.0,
)
_set_vec4(GL, locations.get("u_invalid_color"), INVALID_COLOR) _set_vec4(GL, locations.get("u_invalid_color"), INVALID_COLOR)
self._bind_field_textures(GL, locations) self._bind_field_textures(GL, locations)
else: else:
@@ -1203,6 +1218,7 @@ class GLRenderer:
self._initialized = False self._initialized = False
self._estimated_gpu_bytes = 0 self._estimated_gpu_bytes = 0
self._scalar_range = None self._scalar_range = None
self._field_quantity = ""
self._last_scale = None self._last_scale = None
self._last_scale_mode = "" self._last_scale_mode = ""
self._paint_calls = 0 self._paint_calls = 0
@@ -1339,7 +1355,8 @@ def _compile_program(GL: Any, profile: Profile, name: str) -> int:
def _uniform_locations(GL: Any, program: int) -> dict[str, int]: def _uniform_locations(GL: Any, program: int) -> dict[str, int]:
names = ( names = (
"u_view", "u_projection", "u_lower", "u_upper", "u_view", "u_projection", "u_lower", "u_upper",
"u_invalid_color", "u_lut", "u_field", "u_valid", "u_inv_domain_size", "u_invalid_color", "u_depth_threshold", "u_lut", "u_field",
"u_valid", "u_inv_domain_size",
"u_light_direction", "u_base_color", "u_light_direction", "u_base_color",
) )
out: dict[str, int] = {} out: dict[str, int] = {}
@@ -265,6 +265,16 @@ class ThreeDPanel(QFrame):
self.flow_threshold_spin.valueChanged.connect(self._emit_settings) self.flow_threshold_spin.valueChanged.connect(self._emit_settings)
form.addRow(self.flow_threshold_label, self.flow_threshold_spin) form.addRow(self.flow_threshold_label, self.flow_threshold_spin)
self.depth_threshold_label = QLabel()
self.depth_threshold_spin = QDoubleSpinBox()
self.depth_threshold_spin.setRange(0.0, 1000.0)
self.depth_threshold_spin.setDecimals(4)
self.depth_threshold_spin.setSingleStep(0.005)
self.depth_threshold_spin.setValue(0.0)
self.depth_threshold_spin.setMaximumWidth(SPIN_WIDTH)
self.depth_threshold_spin.valueChanged.connect(self._emit_settings)
form.addRow(self.depth_threshold_label, self.depth_threshold_spin)
self.scale_label = QLabel() self.scale_label = QLabel()
self.scale_combo = QComboBox() self.scale_combo = QComboBox()
self.scale_combo.addItem("fixed", "fixed") self.scale_combo.addItem("fixed", "fixed")
@@ -701,6 +711,7 @@ class ThreeDPanel(QFrame):
self.flow_arrows_check, self.flow_arrows_check,
self.contact_arrow_check, self.contact_arrow_check,
self.flow_threshold_spin, self.flow_threshold_spin,
self.depth_threshold_spin,
self.scale_combo, self.scale_combo,
self.fps_combo, self.fps_combo,
self.reset_button, self.reset_button,
@@ -735,6 +746,7 @@ class ThreeDPanel(QFrame):
self.flow_arrows_check.setChecked(bool(settings.flow_arrows_enabled)) self.flow_arrows_check.setChecked(bool(settings.flow_arrows_enabled))
self.contact_arrow_check.setChecked(bool(settings.contact_arrow_enabled)) self.contact_arrow_check.setChecked(bool(settings.contact_arrow_enabled))
self.flow_threshold_spin.setValue(float(settings.flow_on_px)) self.flow_threshold_spin.setValue(float(settings.flow_on_px))
self.depth_threshold_spin.setValue(float(settings.depth_threshold))
self._flow_grid_spacing = max(1, int(settings.flow_grid_spacing)) self._flow_grid_spacing = max(1, int(settings.flow_grid_spacing))
self._max_arrows = max(1, int(settings.max_arrows)) self._max_arrows = max(1, int(settings.max_arrows))
self.depth_adaptive_check.setChecked(bool(settings.depth_scale_adaptive)) self.depth_adaptive_check.setChecked(bool(settings.depth_scale_adaptive))
@@ -791,6 +803,7 @@ class ThreeDPanel(QFrame):
flow_grid_spacing=int(self._flow_grid_spacing), flow_grid_spacing=int(self._flow_grid_spacing),
max_arrows=int(self._max_arrows), max_arrows=int(self._max_arrows),
depth_scale_adaptive=bool(self.depth_adaptive_check.isChecked()), depth_scale_adaptive=bool(self.depth_adaptive_check.isChecked()),
depth_threshold=float(self.depth_threshold_spin.value()),
) )
# ------------------------------------------------------------------ signals # ------------------------------------------------------------------ signals
@@ -863,6 +876,9 @@ class ThreeDPanel(QFrame):
self.flow_threshold_label.setText(t("scene3d.flow_threshold")) self.flow_threshold_label.setText(t("scene3d.flow_threshold"))
self.flow_threshold_label.setToolTip(t("scene3d.flow_threshold_tip")) self.flow_threshold_label.setToolTip(t("scene3d.flow_threshold_tip"))
self.flow_threshold_spin.setToolTip(t("scene3d.flow_threshold_tip")) self.flow_threshold_spin.setToolTip(t("scene3d.flow_threshold_tip"))
self.depth_threshold_label.setText(t("scene3d.depth_threshold"))
self.depth_threshold_label.setToolTip(t("scene3d.depth_threshold_tip"))
self.depth_threshold_spin.setToolTip(t("scene3d.depth_threshold_tip"))
self.scale_label.setText(t("scene3d.scale")) self.scale_label.setText(t("scene3d.scale"))
self.range_label.setText(t("scene3d.scale_range")) self.range_label.setText(t("scene3d.scale_range"))
self.range_label.setToolTip(t("scene3d.scale_range_tip")) self.range_label.setToolTip(t("scene3d.scale_range_tip"))
@@ -89,6 +89,7 @@ def settings_to_prefs(settings: DisplaySettings) -> dict[str, Any]:
"flow_grid_spacing": int(settings.flow_grid_spacing), "flow_grid_spacing": int(settings.flow_grid_spacing),
"max_arrows": int(settings.max_arrows), "max_arrows": int(settings.max_arrows),
"depth_scale_adaptive": bool(settings.depth_scale_adaptive), "depth_scale_adaptive": bool(settings.depth_scale_adaptive),
"depth_threshold": float(settings.depth_threshold),
} }
if scale is not None: if scale is not None:
block["color_scale"] = { block["color_scale"] = {
@@ -171,4 +172,5 @@ def settings_from_prefs(
flow_grid_spacing=spacing, flow_grid_spacing=spacing,
max_arrows=max_arrows, max_arrows=max_arrows,
depth_scale_adaptive=_flag(raw.get("depth_scale_adaptive"), False), depth_scale_adaptive=_flag(raw.get("depth_scale_adaptive"), False),
depth_threshold=min(max(_number(raw.get("depth_threshold"), 0.0), 0.0), 1000.0),
) )
@@ -280,6 +280,38 @@ def _plate_mode_rgb(pixels: np.ndarray) -> np.ndarray:
return values[int(np.argmax(counts))].astype(float) return values[int(np.argmax(counts))].astype(float)
def test_depth_threshold_only_changes_depth_colours(gl_widget, qt_app):
from orisys_studio.scene3d.gl_backend import lut_rgba
widget, _, _ = gl_widget
assets = synthetic.poc_scene()
values = np.full(assets.shape_hw, 0.05, dtype=np.float32)
depth = FieldDescriptor("depth_response", "1", "approximate", False)
flow = FieldDescriptor("flow_magnitude", "px", "approximate", False)
def settings(descriptor, threshold):
return DisplaySettings(
housing_visible=False,
depth_threshold=threshold,
color_scale=ColorScale(
descriptor=descriptor, lower=0.0, upper=1.0, lut_id="sequential"
),
)
widget.set_settings(settings(depth, 0.0))
widget.present(_frame(assets, values, descriptor=depth, frame_id=30))
unfiltered = _plate_mean_rgb(_render(widget, qt_app))
widget.set_settings(settings(depth, 0.1))
filtered = _plate_mean_rgb(_render(widget, qt_app))
base = lut_rgba("sequential")[0, 0, :3].astype(float)
assert np.abs(filtered - base).max() < 8.0
assert np.abs(unfiltered - filtered).max() > 10.0
widget.set_settings(settings(flow, 0.1))
widget.present(_frame(assets, values, descriptor=flow, frame_id=31))
assert np.abs(_plate_mean_rgb(_render(widget, qt_app)) - unfiltered).max() < 8.0
def test_a_press_does_not_move_the_surfaces_base_colour(gl_widget, qt_app): def test_a_press_does_not_move_the_surfaces_base_colour(gl_widget, qt_app):
"""The base colour follows the field, not the frame's current response. """The base colour follows the field, not the frame's current response.
@@ -57,6 +57,7 @@ def test_the_pane_holds_a_legend_and_no_settings(panel):
panel.deform_check, panel.deform_check,
panel.gain_spin, panel.gain_spin,
panel.flow_threshold_spin, panel.flow_threshold_spin,
panel.depth_threshold_spin,
panel.scale_combo, panel.scale_combo,
panel.depth_adaptive_check, panel.depth_adaptive_check,
panel.fps_combo, panel.fps_combo,
@@ -107,6 +108,14 @@ def test_adaptive_depth_scale_round_trips_through_the_dialog(panel):
assert panel.settings().depth_scale_adaptive is False assert panel.settings().depth_scale_adaptive is False
def test_depth_threshold_round_trips_through_the_dialog(panel):
from orisys_app_shared.scene3d.contracts import DisplaySettings
panel.apply_settings(DisplaySettings(depth_threshold=0.025))
assert panel.depth_threshold_spin.value() == pytest.approx(0.025)
assert panel.settings().depth_threshold == pytest.approx(0.025)
def test_arrow_threshold_round_trips_and_feeds_the_shared_policy(panel): def test_arrow_threshold_round_trips_and_feeds_the_shared_policy(panel):
"""The threshold the dialog writes is the one both views sample with.""" """The threshold the dialog writes is the one both views sample with."""
from orisys_app_shared.scene3d.contracts import DisplaySettings from orisys_app_shared.scene3d.contracts import DisplaySettings
+1
View File
@@ -261,6 +261,7 @@ def test_shader_sources_are_two_dialects_of_one_body():
len(shader_header("gles30")): len(shader_header("gles30")):
] ]
assert "texture(u_lut" in desktop_fragment and "texture(u_lut" in es_fragment assert "texture(u_lut" in desktop_fragment and "texture(u_lut" in es_fragment
assert "value < u_depth_threshold" in desktop_fragment
# The surface reads *two textures* and decides validity from one of them. There is # The surface reads *two textures* and decides validity from one of them. There is
# no per-vertex scalar or validity attribute, and no ``isnan`` rule: the NaN that # no per-vertex scalar or validity attribute, and no ``isnan`` rule: the NaN that
# rule used to catch was produced by the CPU gather this path replaced. # rule used to catch was produced by the CPU gather this path replaced.
+3
View File
@@ -474,6 +474,7 @@ def test_the_display_settings_round_trip():
flow_grid_spacing=12, flow_grid_spacing=12,
max_arrows=250, max_arrows=250,
depth_scale_adaptive=True, depth_scale_adaptive=True,
depth_threshold=0.025,
color_scale=ColorScale( color_scale=ColorScale(
descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False), descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
lower=0.25, lower=0.25,
@@ -497,7 +498,9 @@ def test_the_display_settings_round_trip():
assert (restored.color_scale.lower, restored.color_scale.upper) == (0.25, 4.5) assert (restored.color_scale.lower, restored.color_scale.upper) == (0.25, 4.5)
assert restored.color_scale.mode == "fixed" assert restored.color_scale.mode == "fixed"
assert restored.depth_scale_adaptive is True assert restored.depth_scale_adaptive is True
assert restored.depth_threshold == 0.025
assert scene3d_prefs.settings_from_prefs({}).depth_scale_adaptive is False assert scene3d_prefs.settings_from_prefs({}).depth_scale_adaptive is False
assert scene3d_prefs.settings_from_prefs({"depth_threshold": -1}).depth_threshold == 0.0
def test_a_saved_descriptor_is_never_restored_as_a_measurement(): def test_a_saved_descriptor_is_never_restored_as_a_measurement():
@@ -598,6 +598,7 @@ def depth_map_to_bgr(
*, *,
value_range: tuple[float, float] | None = None, value_range: tuple[float, float] | None = None,
invalid_mask: np.ndarray | None = None, invalid_mask: np.ndarray | None = None,
threshold: float = 0.0,
) -> np.ndarray | None: ) -> np.ndarray | None:
"""Convert a float depth/divergence map to a colormap BGR image. """Convert a float depth/divergence map to a colormap BGR image.
@@ -608,6 +609,9 @@ def depth_map_to_bgr(
``invalid_mask`` marks samples that carry no depth. They are painted with a ``invalid_mask`` marks samples that carry no depth. They are painted with a
neutral grey so "no data" can never be mistaken for "zero depth". neutral grey so "no data" can never be mistaken for "zero depth".
``threshold`` only changes valid pixels' colours: below it they use the
first ramp colour, while the source depth and the colour span stay intact.
""" """
if depth is None: if depth is None:
return None return None
@@ -628,6 +632,8 @@ def depth_map_to_bgr(
scaled = np.zeros(arr.shape, dtype=np.float64) scaled = np.zeros(arr.shape, dtype=np.float64)
scaled = np.clip(scaled, 0.0, 1.0) scaled = np.clip(scaled, 0.0, 1.0)
normalized = (scaled * 255.0).astype(np.uint8) normalized = (scaled * 255.0).astype(np.uint8)
if threshold > 0.0:
normalized[finite & (arr < threshold)] = 0
image = cv2.applyColorMap( image = cv2.applyColorMap(
normalized, cv2.COLORMAP_JET if colormap is None else colormap normalized, cv2.COLORMAP_JET if colormap is None else colormap
) )
@@ -418,6 +418,7 @@ def render_image(
theme: str = "dark", theme: str = "dark",
draw_style: str = "box", draw_style: str = "box",
depth_range: tuple[float, float] | None = None, depth_range: tuple[float, float] | None = None,
depth_threshold: float = 0.0,
canvas_bgr: Sequence[int] | None = None, canvas_bgr: Sequence[int] | None = None,
domain_mask: np.ndarray | None = None, domain_mask: np.ndarray | None = None,
arrow_policy=None, arrow_policy=None,
@@ -441,6 +442,9 @@ def render_image(
with, so both figures trigger at the same strength. ``None`` means the draw style's with, so both figures trigger at the same strength. ``None`` means the draw style's
own defaults. own defaults.
``depth_threshold`` changes depth colouring only. It never changes the frame's
measured depth, its valid mask or the arrows drawn over it.
``orientation`` is a **display** convention for the domain-grid views: the picture is ``orientation`` is a **display** convention for the domain-grid views: the picture is
turned or mirrored once it has been drawn, so the frame's own ``flow``, ``valid_mask`` turned or mirrored once it has been drawn, so the frame's own ``flow``, ``valid_mask``
and ``image_bgr`` are untouched — a recorded session's flow is stored in the domain's and ``image_bgr`` are untouched — a recorded session's flow is stored in the domain's
@@ -460,6 +464,7 @@ def render_image(
frame.depth, frame.depth,
flow_colormap(theme), flow_colormap(theme),
value_range=depth_range, value_range=depth_range,
threshold=depth_threshold,
invalid_mask=boundary, invalid_mask=boundary,
) )
if wanted == DEPTH: if wanted == DEPTH:
@@ -650,6 +650,8 @@ class DisplaySettings:
#: refits both views to the current frame's finite minimum and maximum — the #: refits both views to the current frame's finite minimum and maximum — the
#: earlier per-frame colour bar. A fixed colour scale still wins over either. #: earlier per-frame colour bar. A fixed colour scale still wins over either.
depth_scale_adaptive: bool = False depth_scale_adaptive: bool = False
#: Paint depth response below this relative value with the base ramp colour.
depth_threshold: float = 0.0
@dataclass(frozen=True) @dataclass(frozen=True)
+14
View File
@@ -52,6 +52,20 @@ def test_depth_arrows_draws_the_flow_on_the_depth_picture():
assert not np.array_equal(over, plain) assert not np.array_equal(over, plain)
def test_depth_display_threshold_uses_base_colour_without_changing_source():
depth = np.array([[0.1, 0.4, 0.8]], dtype=np.float32)
original = depth.copy()
frame = backend.ImageFrame(_image(1, 3), depth=depth)
plain = backend.render_image(frame, mode="depth", depth_range=(0.0, 1.0))
tuned = backend.render_image(
frame, mode="depth", depth_range=(0.0, 1.0), depth_threshold=0.5
)
assert plain is not None and tuned is not None
assert np.array_equal(tuned[0, 0], tuned[0, 1])
assert np.array_equal(tuned[0, 2], plain[0, 2])
assert np.array_equal(depth, original)
def test_a_recorded_frame_is_raw_and_says_so(): def test_a_recorded_frame_is_raw_and_says_so():
"""No flow and no depth: the frame *is* the picture, and the note explains it.""" """No flow and no depth: the frame *is* the picture, and the note explains it."""
plan = backend.plan_image(backend.ImageFrame(_image(), source=backend.SOURCE_REPLAY)) plan = backend.plan_image(backend.ImageFrame(_image(), source=backend.SOURCE_REPLAY))