orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a225905b495617d29184840409cd6338a042e566
orisys-app/tools/scene_assets/README.md
T
ykcheungab 12781854ca fixed commetn from songxiao
2026-09-25 17:46:37 +08:00

14 KiB
Raw Blame History

Scene asset tools

Offline tools for the three-dimensional view. They never run in the application: the app loads a compiled package (.oryscene) and nothing else.

tool what it is for
compile_scene.py a legacy NPZ export (+ optional housing npz) → one .oryscene package
align_housing.py a housing exported in another frame → a sensor-frame housing.npz, with the fit checked
fit_design_frame.py the design export's (unit scale, rigid transform) pair, split out of the fitted similarity
validate_scene.py the checks the loader and the compiler must agree on; the release gate
probe_gl.py what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context)
bench_scene.py per-stage p50/p95 of the running view (--cpu-only for the pure CPU cost)
soak_scene.py long run: RSS trend and the queue bound

Run them with the repository's own interpreter, from orisys-app:

$py = '.venv-build\Scripts\python.exe'

Compiling the shipped finger sensor

The runtime asset is sensors/finger/assets/30fps-0703: it is what sensors/finger/manifest.json names (assets_dir), and its unroll map (unroll_map_uv_s0.4.npz, out size 300×280, source_scale 0.4) matches camera.resolution_stitch in sensor.json, which is the grid the live optical flow is computed on.

Do not compile assets/visualization/flatten_remap.npz: it is a 1024×1024 domain, so every live frame would be refused with field_domain_mismatch (300x280 vs 1024x1024). That map belongs to offline visualisation.

$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703'   # or the copy in apps/studio/config

# 1) state the unit (the importer never guesses it) and read what it measured
& $py tools\scene_assets\validate_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz"

# 2) compile
& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $sdk `
  --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene

# 3) verify
& $py tools\scene_assets\validate_scene.py apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene

--meters-per-source-unit is a physical statement about the sensor, not a formatting choice. The measured mesh extent is 15.81 × 15.0 × 7.5 source units (1032 vertices, 1974 triangles); the shipped finger surface is about 15.8 mm across, so one source unit is one millimetre and the value is 0.001. If the mechanical drawing says otherwise, state that number instead — nothing downstream can tell the difference, and a wrong factor looks plausible.

Expected result: 1032 vertices, 1974 triangles, geometry_id starting 1a2e0e92…, a 56 KB package, and validate_scene.py reporting every check ok.

The package also records a unit-free source_geometry_id (46a03a5f… for the shipped bundle). It is the only geometry identity the acquisition side can reproduce without the unit statement above, so it is what lets the running application verify that the scene it is drawing was built from the same surface — and the same mapping, via field_domain_id — that the device is actually running. A package that predates this record is still accepted, and reported as unverified rather than as a match.

The housing (the mounting frame)

Non-sensing geometry is a separate part with part_id="housing", role="housing": it is drawn in a flat graphite grey (unlit-free, never coloured by the measurement) and the operator can hide it. It ships hidden by default (DisplaySettings.housing_visible), because on the finger sensor the frame is ~33 % wider than the sensing surface and dominated the picture.

compile_scene.py --housing <npz> takes a vertices/faces file in the same source frame and units as the export's mesh.npz. Omitted, <legacy-dir>/housing.npz is used when present.

The finger sensor's design export lives in a different frame: it is at orisys-sensor-design/build/export/finger/r01/compute and holds both mesh.npz and housing.npz (part_names = ['frame']) in a camera/world frame. That export's mesh and the SDK asset's mesh are the same surface with identical faces, so the frame relationship can be fitted from the 1032 shared vertices instead of transcribed by hand:

$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
& $py tools\scene_assets\align_housing.py `
  --design $design `
  --sensor $sdk `
  --output tools\scene_assets\housing\finger_r01_housing.npz `
  --matrix tools\scene_assets\housing\finger_r01_design_to_sensor.npy

& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $sdk `
  --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --housing tools\scene_assets\housing\finger_r01_housing.npz `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene

align_housing.py refuses to write when the fitted transform does not hold to --tolerance of the mesh diagonal, so a housing from an unrelated mesh cannot be dropped in silently (a wrong transform still produces a plausible-looking picture). Re-running it also regenerates finger_r01_design_to_sensor.npy, the 4×4 (sensor = 0.018388452·R·design + t) that the fit found.

geometry_id and source_geometry_id cover the sensing surface only — the housing is in the asset hash but not in the identities the acquisition side verifies, so adding a frame does not make every device look unverified.

Rebuilding the shipped package from the design export (r01)

The committed apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene is compiled from the design export — the authoritative surface — and from the unroll map that export produces (unroll_map_uv_s0.42.npz, coverage 39.211 %, domain dfac267f…). The SDK asset is neither the source of it nor promoted into it; the filename stays 1.0.0 because asset_source.COMPILE_HINT, shared/tests/test_scene3d_device_geometry.py and the stage documents all name that exact path, and because find_package() takes sorted(*.oryscene)[-1] — a second, higher-named package would shadow this one while the hard-coded references kept validating the stale one. Bumping asset_version is a release decision, not a rebuild side effect.

The export is in the design repo's camera/world frame and in design units, so the pair the compiler needs is derived, never transcribed:

$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
$sdk    = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703'

& $py tools\scene_assets\fit_design_frame.py `
  --design $design --sensor $sdk `
  --rigid-output tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
  --report       tools\scene_assets\housing\finger_r01_design_frame.json

& $py tools\scene_assets\compile_scene.py `
  --legacy-dir $design `
  --map "$design\unroll_map_uv_s0.42.npz" `
  --meters-per-source-unit 1.8388451511463653e-05 `
  --transform tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
  --housing "$design\housing.npz" `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
  • The fit over the 1032 shared vertices (identical faces/loop_to_vertex, so vertex i is vertex i) is exact to float noise: rms 5.97e-07 sensor-mm, 2.59e-08 of the 23.048-mm diagonal. One design unit is therefore 1/54.38195812 of a sensor millimetre, and the unit statement is 0.001 × 0.0183884515115.
  • --transform must be rigid (compile_scene.py validates the rotation block) and must be in the compiler's column convention, p_sensor = T @ [p; 1]. align_housing.fit_similarity fits the row form p_sensor = s·(p @ R) + t, so the block is R.T and the translation is 0.001·t metres. Feeding housing/finger_r01_design_to_sensor.npy straight in — it already carries the scale and is in the row convention — is refused outright: geometry_mismatch: transform is not a rigid rotation.
  • Hence the trap the two files exist to prevent: the unit statement already contains the fit scale, so a matrix that contains it too would apply it twice. Calling a design unit a millimetre (0.001) with the correct rigid transform is worse, because it compiles and is 54.38196× too large.
  • The housing comes from the export's own housing.npz (part_names = ['frame', 'rubber-b'], 2716 vertices / 5424 triangles). It is already in the frame and units mesh.npz is stated in, which is exactly what --housing documents, so the sensor-frame housing/finger_r01_housing.npz that align_housing.py regenerates is provenance and an independent cross-check (the two routes agree to 9.5e-10 m), not a compile input.
  • housing_visible keeps its shipped default False (DisplaySettings.housing_visible); there is no compile flag for it.

The identities this package records are not the SDK-route ones, and that is the honest result rather than a defect to paper over: geometry_id and source_geometry_id are digests of positions and of the map's world_xyz surface, which the design frame and its differing UVs state differently, and field_domain.id is the new map's. Consequence to expect: the device description the app derives from config/sensors/finger/assets/30fps-0703 (its s0.4 map, domain feb7f71e…) no longer matches, so the pane refuses the package with field_domain_mismatch until that bundle carries the matching unroll_map_uv_s0.42.npz and camera.source_scale 0.42.

Where the application looks for a package

  1. preference.json → "scene3d_package": "<path>" (an absolute path; also the way to point a developer build at a freshly compiled package);
  2. <config>/sensors/<sensor>/scene/*.oryscene — for the studio app that is apps/studio/config/sensors/finger/scene/;
  3. nothing: the pane reports the directory it searched and the compile command above, and the two-dimensional views keep working.

The SDK's own sensors/ tree is not searched (the app prefers its local config/sensors bundles). Compile into the app's scene/ directory, or set scene3d_package.

Release packages (signature, optional encryption, platform lock)

All three are optional and independent of the plain package above:

& $py tools\scene_assets\compile_scene.py --legacy-dir $sdk --map "$sdk\unroll_map_uv_s0.4.npz" `
  --meters-per-source-unit 0.001 `
  --sign-key keys\build-ed25519.pem --sign-key-id build-2024 `
  --write-verify-key keys\build-ed25519.pub.pem `
  --encrypt-key-id finger-display --key-file keys\finger-display.key `
  --lock-from-probe docs\architecture\gl-report-windows-dev.json `
  --output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
  • --sign-key signs the manifest (and therefore every entry hash, the compatibility information and the platform lock). The private key stays on the build machine; the client ships only --write-verify-key's public half.
  • --encrypt-key-id + --key-file additionally seal the content with AES-256-GCM. Encryption never replaces signing. Reading a sealed package needs the protected-assets extra (cryptography); a plain package does not.
  • --lock-from-probe takes the platform lock (OS, architecture, GL profile, vendor, renderer, minimum texture size, "no software rasteriser") from a probe_gl.py report. A machine the lock excludes loses the 3D pane, with the reason, and keeps everything else.
  • --encrypt-key-id without --key-file, or with no crypto backend installed, is refused rather than silently writing a plain package that looks protected.

Release gate:

& $py tools\scene_assets\validate_scene.py <package> `
  --key-file keys\finger-display.key --key-id finger-display `
  --verify-key keys\build-ed25519.pub.pem --require-signature

What the Studio application needs for a protected package

The application reads the same material from its preferences file (paths only — the private half never leaves the build machine):

{
  "scene3d_verify_key": "C:\\keys\\build-ed25519.pub.pem",
  "scene3d_content_key_file": "C:\\keys\\finger-display.key",
  "scene3d_content_key_id": "finger-display",
  "scene3d_require_signature": true
}
  • scene3d_verify_key is the public key --write-verify-key produced. Without it a signed package is refused by the container ("a verification key is required"), which is why the application could not open a protected release at all before this existed.
  • scene3d_content_key_file + scene3d_content_key_id are only needed for a sealed (aes-gcm) package. The file may hold the key as hex, base64 or raw bytes.
  • scene3d_require_signature is the application's own release gate: with it on, an unsigned package is refused (asset_auth_failed) instead of being quietly drawn. Leave it off for engineering builds — the shipped finger-display-1.0.0.oryscene is deliberately plain.
  • A missing or unreadable file is not a crash: the pane reports why, and the two-dimensional application keeps running. session.json records which of these were configured (booleans and paths, never key material).

Performance and long runs

# per-stage p50/p95 of the running view (needs a real window system)
$env:QT_QPA_PLATFORM = 'windows'
& $py tools\scene_assets\bench_scene.py --frames 600 --output perf.json
& $py tools\scene_assets\bench_scene.py --cpu-only --frames 500 --vertices 20000 --output perf-cpu.json
& $py tools\scene_assets\bench_scene.py --package <package> --key-file keys\finger-display.key `
    --key-id finger-display --verify-key keys\build-ed25519.pub.pem --output perf-real.json

# long run: the release gate is two hours in the GUI mode (it is the stricter one)
& $py tools\scene_assets\soak_scene.py --minutes 120 --fps 30 --output soak.json
& $py tools\scene_assets\soak_scene.py --no-window --minutes 10 --fps 60 --output soak-cpu.json

The committed reports for the developer machine are docs/architecture/perf-report-windows-dev.json and docs/architecture/soak-report-windows-dev.json; the stage-by-stage acceptance, the failure samples and the compatibility notes are in docs/architecture/studio-3d-stages.md.

Reference in New Issue View Git Blame Copy Permalink