orisys-app/tools/scene_assets/README.md

# Scene asset tools

Offline tools for the three-dimensional view. They never run in the application:
the app loads a **compiled** package (`.oryscene`) and nothing else.

| tool | what it is for |
|---|---|
| `compile_scene.py` | a legacy NPZ export (+ optional housing npz) → one `.oryscene` package |
| `align_housing.py` | a housing exported in another frame → a sensor-frame `housing.npz`, with the fit checked |
| `fit_design_frame.py` | the design export's *(unit scale, rigid transform)* pair, split out of the fitted similarity |
| `validate_scene.py` | the checks the loader and the compiler must agree on; the release gate |
| `probe_gl.py` | what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context) |
| `bench_scene.py` | per-stage p50/p95 of the running view (`--cpu-only` for the pure CPU cost) |
| `soak_scene.py` | long run: RSS trend and the queue bound |

Run them with the repository's own interpreter, from `orisys-app`:

```powershell
$py = '.venv-build\Scripts\python.exe'
```

## Compiling the shipped finger sensor

The **runtime** asset is `sensors/finger/assets/30fps-0703`: it is what
`sensors/finger/manifest.json` names (`assets_dir`), and its unroll map
(`unroll_map_uv_s0.4.npz`, out size 300×280, `source_scale` 0.4) matches
`camera.resolution_stitch` in `sensor.json`, which is the grid the live optical flow
is computed on.

Do **not** compile `assets/visualization/flatten_remap.npz`: it is a 1024×1024
domain, so every live frame would be refused with `field_domain_mismatch`
(300x280 vs 1024x1024). That map belongs to offline visualisation.

```powershell
$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703'   # or the copy in apps/studio/config

# 1) state the unit (the importer never guesses it) and read what it measured
& $py tools\scene_assets\validate_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz"

# 2) compile
& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $sdk `
  --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene

# 3) verify
& $py tools\scene_assets\validate_scene.py apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```

**`--meters-per-source-unit` is a physical statement about the sensor, not a
formatting choice.** The measured mesh extent is 15.81 × 15.0 × 7.5 source units
(1032 vertices, 1974 triangles); the shipped finger surface is about 15.8 mm across,
so one source unit is one millimetre and the value is `0.001`. If the mechanical
drawing says otherwise, state that number instead — nothing downstream can tell the
difference, and a wrong factor looks plausible.

Expected result: `1032 vertices`, `1974 triangles`, `geometry_id` starting
`1a2e0e92…`, a 56 KB package, and `validate_scene.py` reporting every check `ok`.

The package also records a **unit-free** `source_geometry_id` (`46a03a5f…` for the
shipped bundle). It is the only geometry identity the acquisition side can reproduce
without the unit statement above, so it is what lets the running application verify
that the scene it is drawing was built from the same surface — and the same mapping,
via `field_domain_id` — that the device is actually running. A package that predates
this record is still accepted, and reported as *unverified* rather than as a match.

## The housing (the mounting frame)

Non-sensing geometry is a **separate part** with `part_id="housing"`, `role="housing"`: it is
drawn in a flat graphite grey (`unlit`-free, never coloured by the measurement) and the operator
can hide it. It ships **hidden by default** (`DisplaySettings.housing_visible`), because on the
finger sensor the frame is ~33 % wider than the sensing surface and dominated the picture.

`compile_scene.py --housing <npz>` takes a `vertices`/`faces` file **in the same source frame and
units as the export's `mesh.npz`**. Omitted, `<legacy-dir>/housing.npz` is used when present.

The finger sensor's design export lives in a **different frame**: it is at
`orisys-sensor-design/build/export/finger/r01/compute` and holds both `mesh.npz` and
`housing.npz` (`part_names = ['frame']`) in a camera/world frame. That export's mesh and the SDK
asset's mesh are the same surface with **identical `faces`**, so the frame relationship can be
fitted from the 1032 shared vertices instead of transcribed by hand:

```powershell
$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
& $py tools\scene_assets\align_housing.py `
  --design $design `
  --sensor $sdk `
  --output tools\scene_assets\housing\finger_r01_housing.npz `
  --matrix tools\scene_assets\housing\finger_r01_design_to_sensor.npy

& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $sdk `
  --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --housing tools\scene_assets\housing\finger_r01_housing.npz `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```

`align_housing.py` **refuses to write** when the fitted transform does not hold to
`--tolerance` of the mesh diagonal, so a housing from an unrelated mesh cannot be dropped in
silently (a wrong transform still produces a plausible-looking picture). Re-running it also
regenerates `finger_r01_design_to_sensor.npy`, the 4×4 (`sensor = 0.018388452·R·design + t`)
that the fit found.

`geometry_id` and `source_geometry_id` cover the **sensing surface only** — the housing is in the
asset hash but not in the identities the acquisition side verifies, so adding a frame does not
make every device look unverified.

## Rebuilding the shipped package from the design export (r01)

The committed `apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene` is compiled
from the **design export** — the authoritative surface — and from the unroll map that export
produces (`unroll_map_uv_s0.42.npz`, coverage 39.211 %, domain `dfac267f…`). The SDK asset is
neither the source of it nor promoted into it; the *filename* stays `1.0.0` because
`asset_source.COMPILE_HINT`, `shared/tests/test_scene3d_device_geometry.py` and the stage
documents all name that exact path, and because `find_package()` takes
`sorted(*.oryscene)[-1]` — a second, higher-named package would shadow this one while the
hard-coded references kept validating the stale one. Bumping `asset_version` is a release
decision, not a rebuild side effect.

The export is in the design repo's camera/world frame and in *design* units, so the pair the
compiler needs is **derived**, never transcribed:

```powershell
$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
$sdk    = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703'

& $py tools\scene_assets\fit_design_frame.py `
  --design $design --sensor $sdk `
  --rigid-output tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
  --report       tools\scene_assets\housing\finger_r01_design_frame.json

& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $design `
  --map "$design\unroll_map_uv_s0.42.npz" `
  --meters-per-source-unit 1.8388451511463653e-05 `
  --transform tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
  --housing "$design\housing.npz" `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```

- The fit over the 1032 shared vertices (identical `faces`/`loop_to_vertex`, so vertex *i* is
  vertex *i*) is exact to float noise: rms **5.97e-07 sensor-mm**, 2.59e-08 of the 23.048-mm
  diagonal. One design unit is therefore 1/54.38195812 of a sensor millimetre, and the unit
  statement is `0.001 × 0.0183884515115`.
- `--transform` must be **rigid** (`compile_scene.py` validates the rotation block) and must be
  in the compiler's **column** convention, `p_sensor = T @ [p; 1]`. `align_housing.fit_similarity`
  fits the **row** form `p_sensor = s·(p @ R) + t`, so the block is `R.T` and the translation is
  `0.001·t` metres. Feeding `housing/finger_r01_design_to_sensor.npy` straight in — it already
  carries the scale *and* is in the row convention — is refused outright:
  `geometry_mismatch: transform is not a rigid rotation`.
- Hence the trap the two files exist to prevent: the unit statement already contains the fit
  scale, so a matrix that contains it too would apply it twice. Calling a design unit a
  millimetre (`0.001`) with the correct rigid transform is worse, because it *compiles* and is
  54.38196× too large.
- The housing comes from the export's own `housing.npz` (`part_names = ['frame', 'rubber-b']`,
  2716 vertices / 5424 triangles). It is already in the frame and units `mesh.npz` is stated in,
  which is exactly what `--housing` documents, so the sensor-frame
  `housing/finger_r01_housing.npz` that `align_housing.py` regenerates is provenance and an
  independent cross-check (the two routes agree to 9.5e-10 m), not a compile input.
- `housing_visible` keeps its shipped default `False` (`DisplaySettings.housing_visible`); there
  is no compile flag for it.

The identities this package records are **not** the SDK-route ones, and that is the honest
result rather than a defect to paper over: `geometry_id` and `source_geometry_id` are digests of
positions and of the map's `world_xyz` surface, which the design frame and its differing UVs
state differently, and `field_domain.id` is the new map's. Consequence to expect: the device
description the app derives from `config/sensors/finger/assets/30fps-0703` (its `s0.4` map,
domain `feb7f71e…`) no longer matches, so the pane refuses the package with
`field_domain_mismatch` until that bundle carries the matching `unroll_map_uv_s0.42.npz` and
`camera.source_scale` `0.42`.

## Where the application looks for a package

1. `preference.json` → `"scene3d_package": "<path>"` (an absolute path; also the way
   to point a developer build at a freshly compiled package);
2. `<config>/sensors/<sensor>/scene/*.oryscene` — for the studio app that is
   `apps/studio/config/sensors/finger/scene/`;
3. nothing: the pane reports the directory it searched and the compile command above,
   and the two-dimensional views keep working.

The SDK's own `sensors/` tree is **not** searched (the app prefers its local
`config/sensors` bundles). Compile into the app's `scene/` directory, or set
`scene3d_package`.

## Release packages (signature, optional encryption, platform lock)

All three are optional and independent of the plain package above:

```powershell
& $py tools\scene_assets\compile_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --sign-key keys\build-ed25519.pem --sign-key-id build-2024 `
  --write-verify-key keys\build-ed25519.pub.pem `
  --encrypt-key-id finger-display --key-file keys\finger-display.key `
  --lock-from-probe docs\architecture\gl-report-windows-dev.json `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```

- `--sign-key` signs the manifest (and therefore every entry hash, the compatibility
  information and the platform lock). The private key stays on the build machine; the
  client ships only `--write-verify-key`'s public half.
- `--encrypt-key-id` + `--key-file` additionally seal the content with AES-256-GCM.
  Encryption never replaces signing. Reading a sealed package needs the
  `protected-assets` extra (`cryptography`); a plain package does not.
- `--lock-from-probe` takes the platform lock (OS, architecture, GL profile, vendor,
  renderer, minimum texture size, "no software rasteriser") from a `probe_gl.py`
  report. A machine the lock excludes loses the 3D pane, with the reason, and keeps
  everything else.
- `--encrypt-key-id` without `--key-file`, or with no crypto backend installed, is
  **refused** rather than silently writing a plain package that looks protected.

Release gate:

```powershell
& $py tools\scene_assets\validate_scene.py <package> `
  --key-file keys\finger-display.key --key-id finger-display `
  --verify-key keys\build-ed25519.pub.pem --require-signature
```

### What the Studio application needs for a protected package

The application reads the same material from its preferences file (paths only — the
private half never leaves the build machine):

```json
{
  "scene3d_verify_key": "C:\\keys\\build-ed25519.pub.pem",
  "scene3d_content_key_file": "C:\\keys\\finger-display.key",
  "scene3d_content_key_id": "finger-display",
  "scene3d_require_signature": true
}
```

- `scene3d_verify_key` is the **public** key `--write-verify-key` produced. Without it a
  signed package is refused by the container ("a verification key is required"), which is
  why the application could not open a protected release at all before this existed.
- `scene3d_content_key_file` + `scene3d_content_key_id` are only needed for a **sealed**
  (`aes-gcm`) package. The file may hold the key as hex, base64 or raw bytes.
- `scene3d_require_signature` is the application's own release gate: with it on, an
  unsigned package is refused (`asset_auth_failed`) instead of being quietly drawn.
  Leave it off for engineering builds — the shipped `finger-display-1.0.0.oryscene` is
  deliberately plain.
- A missing or unreadable file is *not* a crash: the pane reports why, and the
  two-dimensional application keeps running. `session.json` records which of these were
  configured (booleans and paths, never key material).

## Performance and long runs

```powershell
# per-stage p50/p95 of the running view (needs a real window system)
$env:QT_QPA_PLATFORM = 'windows'
& $py tools\scene_assets\bench_scene.py --frames 600 --output perf.json
& $py tools\scene_assets\bench_scene.py --cpu-only --frames 500 --vertices 20000 --output perf-cpu.json
& $py tools\scene_assets\bench_scene.py --package <package> --key-file keys\finger-display.key `
    --key-id finger-display --verify-key keys\build-ed25519.pub.pem --output perf-real.json

# long run: the release gate is two hours in the GUI mode (it is the stricter one)
& $py tools\scene_assets\soak_scene.py --minutes 120 --fps 30 --output soak.json
& $py tools\scene_assets\soak_scene.py --no-window --minutes 10 --fps 60 --output soak-cpu.json
```

The committed reports for the developer machine are
`docs/architecture/perf-report-windows-dev.json` and
`docs/architecture/soak-report-windows-dev.json`; the stage-by-stage acceptance, the
failure samples and the compatibility notes are in
`docs/architecture/studio-3d-stages.md`.