feat(studio-3d): 历史回放里的三维面板 —— 由「重算」通道喂它，并标注来源
- What: `VIEW_SHOWS_3D` 加入 `replay`；`app_window` 新增 `_shows_3d()`（可见性与布局共用一个判据） 与 `_splitter_sizes_for()`（预设表里的 3D 槽是"二维布局"的描述，面板打开时从最宽的面板借 `DEFAULT_3D_PANE_WIDTH=340`）；`ThreeDPanel` 新增第三个来源标签（实时采集 / 回放：录像不含三维场 / 重算通道：由录像重算，非记录值）与 `set_source_kind()`；`ThreeDCoordinator.show_rest()` 把参考模型 放回屏幕并停止主张测量值（重算停止时调用，否则最后一帧会留在屏幕上）；`Mesh3dPanel` 转发这两者， 并新增"加载中"状态的修正：切换传感器再切回来时若该包已加载，直接显示已就绪的画面而不是一条永远 不会被替换的"正在加载"。 - Why: 录像里没有三维场（raw_*.mp4 是原始相机图像，`depth.stored=false`，CSV 只有力值），所以纯回放只能 显示参考模型；但「从视频重算力值」本来就把同一个运行路径指向录像的原始视频，而采集线程上的三维 sink 早已注册——因此重算一开始面板就会收到真实的形变帧。这两件事必须由界面区分：一个是"没有测量"， 一个是"当前算法对这段录像会说什么"，把后者当成记录值展示就是另一种 `display_illustrative`。 - Verification: apps/studio 全量通过（test_scene3d_view.py 32 项，含"回放里面板可见且标注正确""重算时 标注切换""停止重算后清回参考模型""实时视图标为实时"；test_scene3d_coordinator.py 15 项，含 show_rest 与"停止时在飞的帧不会随后被显示"）；ui_check 570 项 0 失败（新增"已安装包必须被加载并标注来源"）； locale_check 0 缺失；仓库内工程包经 validate_scene 全部 ok。 - Not in this change: 逐帧保存展开场/深度（记录格式变更，设计文档明确否掉）；重算通道本身（已存在，本次 只是把它的帧接到面板上）；示意位移（仍需 SDK lift + 已确认尺度）。
This commit is contained in:
ykcheungab
@@ -0,0 +1,126 @@
|
||||
# Scene asset tools
|
||||
|
||||
Offline tools for the three-dimensional view. They never run in the application:
|
||||
the app loads a **compiled** package (`.oryscene`) and nothing else.
|
||||
|
||||
| tool | what it is for |
|
||||
|---|---|
|
||||
| `compile_scene.py` | a Blender export (+ optional housing GLB) → one `.oryscene` package |
|
||||
| `validate_scene.py` | the checks the loader and the compiler must agree on; the release gate |
|
||||
| `probe_gl.py` | what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context) |
|
||||
| `bench_scene.py` | per-stage p50/p95 of the running view (`--cpu-only` for the pure CPU cost) |
|
||||
| `soak_scene.py` | long run: RSS trend and the queue bound |
|
||||
|
||||
Run them with the repository's own interpreter, from `orisys-app`:
|
||||
|
||||
```powershell
|
||||
$py = '.venv-build\Scripts\python.exe'
|
||||
```
|
||||
|
||||
## Compiling the shipped finger sensor
|
||||
|
||||
The **runtime** asset is `sensors/finger/assets/30fps-0703`: it is what
|
||||
`sensors/finger/manifest.json` names (`assets_dir`), and its unroll map
|
||||
(`unroll_map_uv_s0.4.npz`, out size 300×280, `source_scale` 0.4) matches
|
||||
`camera.resolution_stitch` in `sensor.json`, which is the grid the live optical flow
|
||||
is computed on.
|
||||
|
||||
Do **not** compile `assets/visualization/flatten_remap.npz`: it is a 1024×1024
|
||||
domain, so every live frame would be refused with `field_domain_mismatch`
|
||||
(300x280 vs 1024x1024). That map belongs to offline visualisation.
|
||||
|
||||
```powershell
|
||||
$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703' # or the copy in apps/studio/config
|
||||
|
||||
# 1) state the unit (the importer never guesses it) and read what it measured
|
||||
& $py tools\scene_assets\validate_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz"
|
||||
|
||||
# 2) compile
|
||||
& $py tools\scene_assets\compile_scene.py `
|
||||
--legacy-dir $sdk `
|
||||
--map "$sdk\unroll_map_uv_s0.4.npz" `
|
||||
--meters-per-source-unit 0.001 `
|
||||
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
|
||||
# 3) verify
|
||||
& $py tools\scene_assets\validate_scene.py apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
```
|
||||
|
||||
**`--meters-per-source-unit` is a physical statement about the sensor, not a
|
||||
formatting choice.** The measured mesh extent is 15.81 × 15.0 × 7.5 source units
|
||||
(1032 vertices, 1974 triangles); the shipped finger surface is about 15.8 mm across,
|
||||
so one source unit is one millimetre and the value is `0.001`. If the mechanical
|
||||
drawing says otherwise, state that number instead — nothing downstream can tell the
|
||||
difference, and a wrong factor looks plausible.
|
||||
|
||||
Expected result: `1032 vertices`, `1974 triangles`, `geometry_id` starting
|
||||
`1a2e0e92…`, a 56 KB package, and `validate_scene.py` reporting every check `ok`.
|
||||
|
||||
## Where the application looks for a package
|
||||
|
||||
1. `preference.json` → `"scene3d_package": "<path>"` (an absolute path; also the way
|
||||
to point a developer build at a freshly compiled package);
|
||||
2. `<config>/sensors/<sensor>/scene/*.oryscene` — for the studio app that is
|
||||
`apps/studio/config/sensors/finger/scene/`;
|
||||
3. nothing: the pane reports the directory it searched and the compile command above,
|
||||
and the two-dimensional views keep working.
|
||||
|
||||
The SDK's own `sensors/` tree is **not** searched (the app prefers its local
|
||||
`config/sensors` bundles). Compile into the app's `scene/` directory, or set
|
||||
`scene3d_package`.
|
||||
|
||||
## Release packages (signature, optional encryption, platform lock)
|
||||
|
||||
All three are optional and independent of the plain package above:
|
||||
|
||||
```powershell
|
||||
& $py tools\scene_assets\compile_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz" `
|
||||
--meters-per-source-unit 0.001 `
|
||||
--sign-key keys\build-ed25519.pem --sign-key-id build-2024 `
|
||||
--write-verify-key keys\build-ed25519.pub.pem `
|
||||
--encrypt-key-id finger-display --key-file keys\finger-display.key `
|
||||
--lock-from-probe docs\architecture\gl-report-windows-dev.json `
|
||||
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
|
||||
```
|
||||
|
||||
- `--sign-key` signs the manifest (and therefore every entry hash, the compatibility
|
||||
information and the platform lock). The private key stays on the build machine; the
|
||||
client ships only `--write-verify-key`'s public half.
|
||||
- `--encrypt-key-id` + `--key-file` additionally seal the content with AES-256-GCM.
|
||||
Encryption never replaces signing. Reading a sealed package needs the
|
||||
`protected-assets` extra (`cryptography`); a plain package does not.
|
||||
- `--lock-from-probe` takes the platform lock (OS, architecture, GL profile, vendor,
|
||||
renderer, minimum texture size, "no software rasteriser") from a `probe_gl.py`
|
||||
report. A machine the lock excludes loses the 3D pane, with the reason, and keeps
|
||||
everything else.
|
||||
- `--encrypt-key-id` without `--key-file`, or with no crypto backend installed, is
|
||||
**refused** rather than silently writing a plain package that looks protected.
|
||||
|
||||
Release gate:
|
||||
|
||||
```powershell
|
||||
& $py tools\scene_assets\validate_scene.py <package> `
|
||||
--key-file keys\finger-display.key --key-id finger-display `
|
||||
--verify-key keys\build-ed25519.pub.pem --require-signature
|
||||
```
|
||||
|
||||
## Performance and long runs
|
||||
|
||||
```powershell
|
||||
# per-stage p50/p95 of the running view (needs a real window system)
|
||||
$env:QT_QPA_PLATFORM = 'windows'
|
||||
& $py tools\scene_assets\bench_scene.py --frames 600 --output perf.json
|
||||
& $py tools\scene_assets\bench_scene.py --cpu-only --frames 500 --vertices 20000 --output perf-cpu.json
|
||||
& $py tools\scene_assets\bench_scene.py --package <package> --key-file keys\finger-display.key `
|
||||
--key-id finger-display --verify-key keys\build-ed25519.pub.pem --output perf-real.json
|
||||
|
||||
# long run: the release gate is two hours in the GUI mode (it is the stricter one)
|
||||
& $py tools\scene_assets\soak_scene.py --minutes 120 --fps 30 --output soak.json
|
||||
& $py tools\scene_assets\soak_scene.py --no-window --minutes 10 --fps 60 --output soak-cpu.json
|
||||
```
|
||||
|
||||
The committed reports for the developer machine are
|
||||
`docs/architecture/perf-report-windows-dev.json` and
|
||||
`docs/architecture/soak-report-windows-dev.json`; the stage-by-stage acceptance, the
|
||||
failure samples and the compatibility notes are in
|
||||
`docs/architecture/studio-3d-stages.md`.
|
||||
Reference in New Issue
Block a user