orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

fixed commetn from songxiao

Browse Source
This commit is contained in:
ykcheungab
2026-09-25 17:46:37 +08:00
parent 6effe74172
commit 12781854ca
56 changed files with 4935 additions and 419 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/scene_assets/README.md
+109 -1
View File
@@ -5,7 +5,9 @@ the app loads a **compiled** package (`.oryscene`) and nothing else.
| tool | what it is for |
|---|---|
| `compile_scene.py` | a Blender export (+ optional housing GLB) → one `.oryscene` package |
| `compile_scene.py` | a legacy NPZ export (+ optional housing npz) → one `.oryscene` package |
| `align_housing.py` | a housing exported in another frame → a sensor-frame `housing.npz`, with the fit checked |
| `fit_design_frame.py` | the design export's *(unit scale, rigid transform)* pair, split out of the fitted similarity |
| `validate_scene.py` | the checks the loader and the compiler must agree on; the release gate |
| `probe_gl.py` | what this machine's OpenGL actually is (exit 0 GPU, 3 software, 4 no context) |
| `bench_scene.py` | per-stage p50/p95 of the running view (`--cpu-only` for the pure CPU cost) |
@@ -63,6 +65,112 @@ that the scene it is drawing was built from the same surface — and the same ma
via `field_domain_id` — that the device is actually running. A package that predates
this record is still accepted, and reported as *unverified* rather than as a match.
## The housing (the mounting frame)
Non-sensing geometry is a **separate part** with `part_id="housing"`, `role="housing"`: it is
drawn in a flat graphite grey (`unlit`-free, never coloured by the measurement) and the operator
can hide it. It ships **hidden by default** (`DisplaySettings.housing_visible`), because on the
finger sensor the frame is ~33 % wider than the sensing surface and dominated the picture.
`compile_scene.py --housing <npz>` takes a `vertices`/`faces` file **in the same source frame and
units as the export's `mesh.npz`**. Omitted, `<legacy-dir>/housing.npz` is used when present.
The finger sensor's design export lives in a **different frame**: it is at
`orisys-sensor-design/build/export/finger/r01/compute` and holds both `mesh.npz` and
`housing.npz` (`part_names = ['frame']`) in a camera/world frame. That export's mesh and the SDK
asset's mesh are the same surface with **identical `faces`**, so the frame relationship can be
fitted from the 1032 shared vertices instead of transcribed by hand:
```powershell
$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
& $py tools\scene_assets\align_housing.py `
--design $design `
--sensor $sdk `
--output tools\scene_assets\housing\finger_r01_housing.npz `
--matrix tools\scene_assets\housing\finger_r01_design_to_sensor.npy
& $py tools\scene_assets\compile_scene.py `
--legacy-dir $sdk `
--map "$sdk\unroll_map_uv_s0.4.npz" `
--meters-per-source-unit 0.001 `
--housing tools\scene_assets\housing\finger_r01_housing.npz `
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```
`align_housing.py` **refuses to write** when the fitted transform does not hold to
`--tolerance` of the mesh diagonal, so a housing from an unrelated mesh cannot be dropped in
silently (a wrong transform still produces a plausible-looking picture). Re-running it also
regenerates `finger_r01_design_to_sensor.npy`, the 4×4 (`sensor = 0.018388452·R·design + t`)
that the fit found.
`geometry_id` and `source_geometry_id` cover the **sensing surface only** — the housing is in the
asset hash but not in the identities the acquisition side verifies, so adding a frame does not
make every device look unverified.
## Rebuilding the shipped package from the design export (r01)
The committed `apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene` is compiled
from the **design export** — the authoritative surface — and from the unroll map that export
produces (`unroll_map_uv_s0.42.npz`, coverage 39.211 %, domain `dfac267f…`). The SDK asset is
neither the source of it nor promoted into it; the *filename* stays `1.0.0` because
`asset_source.COMPILE_HINT`, `shared/tests/test_scene3d_device_geometry.py` and the stage
documents all name that exact path, and because `find_package()` takes
`sorted(*.oryscene)[-1]` — a second, higher-named package would shadow this one while the
hard-coded references kept validating the stale one. Bumping `asset_version` is a release
decision, not a rebuild side effect.
The export is in the design repo's camera/world frame and in *design* units, so the pair the
compiler needs is **derived**, never transcribed:
```powershell
$design = 'D:\orisys_ws\orisys-sensor-design\build\export\finger\r01\compute'
$sdk = 'D:\orisys_ws\orisys-sdk\sensors\finger\assets\30fps-0703'
& $py tools\scene_assets\fit_design_frame.py `
--design $design --sensor $sdk `
--rigid-output tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
--report tools\scene_assets\housing\finger_r01_design_frame.json
& $py tools\scene_assets\compile_scene.py `
--legacy-dir $design `
--map "$design\unroll_map_uv_s0.42.npz" `
--meters-per-source-unit 1.8388451511463653e-05 `
--transform tools\scene_assets\housing\finger_r01_design_to_sensor_rigid_m.npy `
--housing "$design\housing.npz" `
--output apps\studio\config\sensors\finger\scene\finger-display-1.0.0.oryscene
```
- The fit over the 1032 shared vertices (identical `faces`/`loop_to_vertex`, so vertex *i* is
vertex *i*) is exact to float noise: rms **5.97e-07 sensor-mm**, 2.59e-08 of the 23.048-mm
diagonal. One design unit is therefore 1/54.38195812 of a sensor millimetre, and the unit
statement is `0.001 × 0.0183884515115`.
- `--transform` must be **rigid** (`compile_scene.py` validates the rotation block) and must be
in the compiler's **column** convention, `p_sensor = T @ [p; 1]`. `align_housing.fit_similarity`
fits the **row** form `p_sensor = s·(p @ R) + t`, so the block is `R.T` and the translation is
`0.001·t` metres. Feeding `housing/finger_r01_design_to_sensor.npy` straight in — it already
carries the scale *and* is in the row convention — is refused outright:
`geometry_mismatch: transform is not a rigid rotation`.
- Hence the trap the two files exist to prevent: the unit statement already contains the fit
scale, so a matrix that contains it too would apply it twice. Calling a design unit a
millimetre (`0.001`) with the correct rigid transform is worse, because it *compiles* and is
54.38196× too large.
- The housing comes from the export's own `housing.npz` (`part_names = ['frame', 'rubber-b']`,
2716 vertices / 5424 triangles). It is already in the frame and units `mesh.npz` is stated in,
which is exactly what `--housing` documents, so the sensor-frame
`housing/finger_r01_housing.npz` that `align_housing.py` regenerates is provenance and an
independent cross-check (the two routes agree to 9.5e-10 m), not a compile input.
- `housing_visible` keeps its shipped default `False` (`DisplaySettings.housing_visible`); there
is no compile flag for it.
The identities this package records are **not** the SDK-route ones, and that is the honest
result rather than a defect to paper over: `geometry_id` and `source_geometry_id` are digests of
positions and of the map's `world_xyz` surface, which the design frame and its differing UVs
state differently, and `field_domain.id` is the new map's. Consequence to expect: the device
description the app derives from `config/sensors/finger/assets/30fps-0703` (its `s0.4` map,
domain `feb7f71e…`) no longer matches, so the pane refuses the package with
`field_domain_mismatch` until that bundle carries the matching `unroll_map_uv_s0.42.npz` and
`camera.source_scale` `0.42`.
## Where the application looks for a package
1. `preference.json` → `"scene3d_package": "<path>"` (an absolute path; also the way
tools/scene_assets/align_housing.py
+163
View File
@@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""Put a housing exported in *another* frame into the sensor frame, once, reproducibly.
The finger sensor's design export (``orisys-sensor-design/build/export/finger/r01/compute``)
holds ``mesh.npz`` and ``housing.npz`` (the mounting frame, ``part_names = ['frame']``) in a
**camera/world frame**, while ``compile_scene.py`` compiles from the SDK's asset directory,
whose ``mesh.npz`` is already in the sensor frame (millimetres, the unit stated by
``--meters-per-source-unit``).
The two meshes are the same surface, and their ``faces`` arrays are identical — so vertex *i*
corresponds to vertex *i* and the frame relationship is a similarity transform that can be
**fitted** rather than guessed. This tool:
1. loads both ``mesh.npz`` files,
2. fits the similarity (rotation + uniform scale + translation) from design to sensor vertices,
3. refuses to write anything if the fit does not hold to ``--tolerance`` of the mesh diagonal,
4. applies that transform to the housing vertices and writes a sensor-frame ``housing.npz``
(millimetres, same as the SDK's mesh) ready for ``compile_scene.py --housing``.
Why fit instead of hand-transcribing a matrix: a wrong matrix still produces a plausible
picture, and this one is checked against 1032 known correspondences every time it runs.
python tools/scene_assets/align_housing.py \
--design D:/orisys_ws/orisys-sensor-design/build/export/finger/r01/compute \
--sensor D:/orisys_ws/orisys-sdk/sensors/finger/assets/30fps-0703 \
--output tools/scene_assets/housing/finger_r01_housing.npz
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
import numpy as np
#: Residual budget, as a fraction of the sensor-frame mesh diagonal. The fit is exact in
#: practice (float noise); anything above this means the two meshes are not the same shape and
#: the housing must not be transformed as if they were.
DEFAULT_TOLERANCE = 1e-4
def load_vertices(path: Path) -> np.ndarray:
with np.load(path, allow_pickle=False) as data:
if "vertices" not in data.files:
raise SystemExit(f"{path} has no 'vertices' array (found {list(data.files)})")
return np.asarray(data["vertices"], dtype=np.float64)
def load_faces(path: Path) -> np.ndarray:
with np.load(path, allow_pickle=False) as data:
return np.asarray(data["faces"])
def fit_similarity(source: np.ndarray, target: np.ndarray) -> tuple[np.ndarray, np.ndarray, float]:
"""Rotation, translation and uniform scale mapping ``source`` onto ``target``.
``target = scale * (source @ rotation) + translation``. Uses the closed-form Kabsch
solution with a scale factor; a reflection is never the answer here.
"""
source_centroid = source.mean(axis=0)
target_centroid = target.mean(axis=0)
source_centered = source - source_centroid
target_centered = target - target_centroid
u, singular, vt = np.linalg.svd(source_centered.T @ target_centered)
rotation = u @ vt
if np.linalg.det(rotation) < 0.0:
u = u.copy()
u[:, -1] *= -1.0
rotation = u @ vt
scale = float(np.sum(singular) / np.sum(source_centered**2))
translation = target_centroid - scale * (source_centroid @ rotation)
return rotation, translation, scale
def apply_similarity(
points: np.ndarray, rotation: np.ndarray, translation: np.ndarray, scale: float
) -> np.ndarray:
return scale * (points @ rotation) + translation
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--design", type=Path, required=True, help="export dir with mesh+housing")
parser.add_argument("--sensor", type=Path, required=True, help="export dir whose mesh is sensor-frame")
parser.add_argument("--output", type=Path, required=True, help="sensor-frame housing npz to write")
parser.add_argument("--tolerance", type=float, default=DEFAULT_TOLERANCE)
parser.add_argument(
"--matrix",
type=Path,
default=None,
help="also write the fitted 4x4 (source -> sensor) as a .npy, for the record",
)
args = parser.parse_args(argv)
design_mesh = args.design / "mesh.npz"
sensor_mesh = args.sensor / "mesh.npz"
housing_path = args.design / "housing.npz"
for path in (design_mesh, sensor_mesh, housing_path):
if not path.is_file():
raise SystemExit(f"missing {path}")
source = load_vertices(design_mesh)
target = load_vertices(sensor_mesh)
if source.shape != target.shape:
raise SystemExit(
f"the meshes disagree: {source.shape} vs {target.shape} — the correspondence the "
"fit relies on does not exist"
)
if not np.array_equal(load_faces(design_mesh), load_faces(sensor_mesh)):
raise SystemExit(
"the faces arrays differ, so vertex i is not vertex i; refusing to fit a transform"
)
rotation, translation, scale = fit_similarity(source, target)
residual = apply_similarity(source, rotation, translation, scale) - target
rms = float(np.sqrt((residual**2).sum(axis=1).mean()))
diagonal = float(np.linalg.norm(target.max(axis=0) - target.min(axis=0)))
relative = rms / max(diagonal, 1e-12)
print(f"fit over {len(source)} vertices: scale={scale:.8g} rms={rms:.3g} "
f"({relative:.3g} of the {diagonal:.3f}-unit diagonal)")
if relative > float(args.tolerance):
raise SystemExit(
f"the fit is off by {relative:.3g} of the mesh diagonal (budget {args.tolerance:g}): "
"these two meshes are not the same shape, so transforming the housing with this "
"matrix would put it in the wrong place"
)
with np.load(housing_path, allow_pickle=False) as data:
housing_vertices = np.asarray(data["vertices"], dtype=np.float64)
housing_faces = np.asarray(data["faces"])
part_names = data["part_names"] if "part_names" in data.files else None
aligned = apply_similarity(housing_vertices, rotation, translation, scale)
args.output.parent.mkdir(parents=True, exist_ok=True)
np.savez(
args.output,
vertices=aligned.astype(np.float32),
faces=housing_faces.astype(np.int32),
# Provenance, so a later reader can see this was fitted rather than exported.
source_frame=np.array(str(args.design)),
sensor_frame=np.array(str(args.sensor)),
fitted_scale=np.array(scale, dtype=np.float64),
fitted_rms=np.array(rms, dtype=np.float64),
part_names=(part_names if part_names is not None else np.array(["housing"])),
)
print(f"wrote {args.output}")
print(f" housing in the sensor frame: low={np.round(aligned.min(axis=0), 3)} "
f"high={np.round(aligned.max(axis=0), 3)}"
f" ({len(housing_faces)} triangles)")
if args.matrix is not None:
matrix = np.eye(4)
matrix[:3, :3] = scale * rotation
matrix[:3, 3] = translation
args.matrix.parent.mkdir(parents=True, exist_ok=True)
np.save(args.matrix, matrix)
print(f"wrote {args.matrix}")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/scene_assets/compare_uv_map.py
+247
View File
@@ -0,0 +1,247 @@
#!/usr/bin/env python3
"""Measure a UV/unroll map — and compare it with the shipped one and the built package.
This is the tool that answers "is the ~40 % of the finger mesh that renders grey *supposed* to be
non-sensing?", with numbers rather than by eye:
* the island's coverage and its bounding box in UV pixels;
* the island in **sensor units** (from ``world_xyz``), so it can be compared with the mesh extent
the app reports;
* how much of the compiled package's surface has no mapping (that is the grey the operator sees);
* the map's ``field_domain_id`` against the one recorded in the package — a mismatch means the pane
would refuse live frames (``coordinator.py`` drops a frame whose domain differs from the scene);
* with ``--against``, a second map (normally the shipped one): coverage/bbox deltas and the share of
pixels whose validity differs.
Offline only; it never runs in the application. Needs only numpy plus the app's own shared package
(the same ``load_legacy_domain`` the compiler uses, so the id is the compiler's id):
python tools/scene_assets/compare_uv_map.py --map <map.npz> \
--against <sdk>/sensors/finger/assets/30fps-0703/unroll_map_uv_s0.4.npz \
--package apps/studio/config/sensors/finger/scene/finger-display-1.0.0.oryscene
"""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
import zipfile
from pathlib import Path
import numpy as np
_MM_PER_SOURCE_UNIT = 1.0 # the finger export states 1 source unit = 1 mm
def _sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1 << 20), b""):
digest.update(chunk)
return digest.hexdigest()
def load_map(path: Path) -> dict:
with np.load(path, allow_pickle=False) as data:
arrays = {key: data[key] for key in data.files}
return arrays
def _scalar(arrays: dict, key: str):
value = arrays.get(key)
if value is None:
return None
return np.asarray(value).reshape(-1)[0].item() if np.asarray(value).size else None
def describe_map(path: Path, arrays: dict) -> dict:
"""Everything worth knowing about one map, as plain JSON-able values."""
mask = np.asarray(arrays["valid_mask"]).astype(bool)
height, width = mask.shape
rows, cols = np.nonzero(mask)
report: dict = {
"path": str(path),
"sha256": _sha256(path),
"shape_hw": [int(height), int(width)],
"valid_pixels": int(mask.sum()),
"coverage_pct": round(float(mask.mean()) * 100.0, 3),
"recorded": {
key: _scalar(arrays, key)
for key in ("out_width", "out_height", "source_width", "source_height",
"flip_v", "edge_band", "edge_expand", "source_scale", "method")
},
}
if rows.size:
report["island_bbox_px"] = {
"rows": [int(rows.min()), int(rows.max())],
"cols": [int(cols.min()), int(cols.max())],
}
else:
report["island_bbox_px"] = None
world = arrays.get("world_xyz")
if world is not None and rows.size:
points = np.asarray(world)[mask]
finite = np.isfinite(points).all(axis=1)
if finite.any():
points = points[finite] * _MM_PER_SOURCE_UNIT
low, high = points.min(axis=0), points.max(axis=0)
report["island_sensor_units"] = {
"low": [round(float(v), 4) for v in low],
"high": [round(float(v), 4) for v in high],
"size": [round(float(v), 4) for v in (high - low)],
"note": "source units; the finger export states 1 unit = 1 mm",
}
return report
def field_domain_id(map_path: Path) -> str:
"""The compiler's own domain identity for this map (same code path as the compile)."""
from orisys_app_shared.scene3d.legacy import load_legacy_domain
return str(load_legacy_domain(map_path).domain_id)
def package_facts(package: Path) -> dict:
"""What the built package says about the same surface."""
facts: dict = {"path": str(package)}
with zipfile.ZipFile(package) as archive:
names = set(archive.namelist())
manifest = json.loads(archive.read("manifest.json").decode("utf-8"))
facts["manifest_keys"] = sorted(manifest.keys())
for key, value in manifest.items():
if isinstance(value, dict) and "id" in value:
facts[f"manifest.{key}.id"] = value["id"]
elif isinstance(value, str) and "domain" in key:
facts[f"manifest.{key}"] = value
for member in ("sensing/surface_valid.npy", "sensing/surface_positions.npy"):
if member not in names:
continue
with archive.open(member) as handle:
array = np.load(handle, allow_pickle=False)
facts[member] = list(array.shape)
if member.endswith("surface_valid.npy"):
valid = np.asarray(array).astype(bool)
facts["surface_valid_true"] = int(valid.sum())
facts["surface_valid_false"] = int((~valid).sum())
facts["surface_unmapped_pct"] = round(
float((~valid).mean()) * 100.0, 3
)
return facts
def compare(left: dict, right: dict) -> dict:
"""How two maps differ: coverage, island movement, and which pixels changed."""
left_mask = np.asarray(left["valid_mask"]).astype(bool)
right_mask = np.asarray(right["valid_mask"]).astype(bool)
result: dict = {}
if left_mask.shape != right_mask.shape:
result["shape_mismatch"] = [list(left_mask.shape), list(right_mask.shape)]
return result
changed = left_mask ^ right_mask
result["coverage_pct"] = {
"left": round(float(left_mask.mean()) * 100.0, 3),
"right": round(float(right_mask.mean()) * 100.0, 3),
"delta": round(float((left_mask.mean() - right_mask.mean()) * 100.0), 3),
}
result["pixels_only_in_left"] = int((left_mask & ~right_mask).sum())
result["pixels_only_in_right"] = int((right_mask & ~left_mask).sum())
result["pixels_differing_pct"] = round(float(changed.mean()) * 100.0, 4)
for name, mask in (("left", left_mask), ("right", right_mask)):
rows, cols = np.nonzero(mask)
result[f"{name}_bbox_px"] = (
[int(rows.min()), int(rows.max()), int(cols.min()), int(cols.max())]
if rows.size
else None
)
result["identical"] = bool(not changed.any())
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--map", type=Path, required=True, help="the map to measure")
parser.add_argument("--against", type=Path, default=None, help="a second map to diff against")
parser.add_argument("--package", type=Path, default=None, help="the built .oryscene to compare")
parser.add_argument("--json", action="store_true", help="machine-readable output")
args = parser.parse_args(argv)
if not args.map.is_file():
print(f"no such map: {args.map}", file=sys.stderr)
return 2
report: dict = {}
arrays = load_map(args.map)
report["map"] = describe_map(args.map, arrays)
print(f"map {args.map}")
info = report["map"]
print(f" shape {info['shape_hw'][1]} x {info['shape_hw'][0]} (W x H)")
print(f" coverage {info['coverage_pct']} % of the domain "
f"({info['valid_pixels']} pixels)")
if info["island_bbox_px"]:
rows, cols = info["island_bbox_px"]["rows"], info["island_bbox_px"]["cols"]
print(f" island bbox rows {rows[0]}..{rows[1]}, cols {cols[0]}..{cols[1]}")
if "island_sensor_units" in info:
print(f" island size {info['island_sensor_units']['size']} (source units)")
print(f" island span {info['island_sensor_units']['low']} .. "
f"{info['island_sensor_units']['high']}")
recorded = {k: v for k, v in info["recorded"].items() if v is not None}
print(f" recorded {recorded}")
try:
report["field_domain_id"] = field_domain_id(args.map)
print(f" domain id {report['field_domain_id']}")
except Exception as exc: # noqa: BLE001 - the tool reports, it does not crash
report["field_domain_id_error"] = f"{type(exc).__name__}: {exc}"
print(f" domain id (could not compute: {type(exc).__name__}: {exc})")
if args.against is not None:
if not args.against.is_file():
print(f"no such map to compare against: {args.against}", file=sys.stderr)
return 2
other = load_map(args.against)
report["against"] = describe_map(args.against, other)
report["comparison"] = compare(arrays, other)
print(f"\nagainst {args.against}")
print(f" coverage {report['against']['coverage_pct']} %")
diff = report["comparison"]
if "shape_mismatch" in diff:
print(f" SHAPE MISMATCH {diff['shape_mismatch']}")
else:
print(f" delta {diff['coverage_pct']['delta']:+.3f} pp, "
f"{diff['pixels_differing_pct']} % of pixels differ")
print(f" only here {diff['pixels_only_in_left']} px; "
f"only there {diff['pixels_only_in_right']} px")
print(f" verdict {'IDENTICAL island' if diff['identical'] else 'different island'}")
if args.package is not None:
if not args.package.is_file():
print(f"no such package: {args.package}", file=sys.stderr)
return 2
facts = package_facts(args.package)
report["package"] = facts
print(f"\npackage {args.package}")
if "surface_unmapped_pct" in facts:
print(f" surface {facts['surface_valid_false']} of "
f"{facts['surface_valid_true'] + facts['surface_valid_false']} render vertices "
f"have no mapping ({facts['surface_unmapped_pct']} %)")
recorded_ids = {k: v for k, v in facts.items() if k.endswith(".id")}
if recorded_ids:
print(f" recorded ids {recorded_ids}")
if report.get("field_domain_id"):
matches = [
key for key, value in recorded_ids.items()
if str(value) == report["field_domain_id"]
]
report["package_domain_matches_map"] = bool(matches)
print(f" domain match {'yes' if matches else 'NO — the pane would refuse live frames'}")
if args.json:
print(json.dumps(report, indent=2, default=str))
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/scene_assets/compile_scene.py
+12
View File
@@ -144,6 +144,17 @@ def main(argv: list[str] | None = None) -> int:
default=None,
help="optional 4x4 rigid transform (npy) from source to sensor coordinates",
)
parser.add_argument(
"--housing",
type=Path,
default=None,
help=(
"optional housing/frame npz (vertices, faces) in the same source frame and "
"units as the export's mesh.npz; omitted, <legacy-dir>/housing.npz is used when "
"present. A housing exported in a different frame must be aligned first with "
"tools/scene_assets/align_housing.py"
),
)
protection = parser.add_argument_group(
"release protection (stage 5)", "all optional; a plain package is the default"
)
@@ -205,6 +216,7 @@ def main(argv: list[str] | None = None) -> int:
asset_id=args.asset_id,
asset_version=args.version,
lod_id=args.lod_id,
housing_path=args.housing,
)
if lock is not None:
from dataclasses import replace
tools/scene_assets/fit_design_frame.py
+326
View File
@@ -0,0 +1,326 @@
#!/usr/bin/env python3
"""Derive the *(unit scale, rigid transform)* pair that compiles the **design export**.
``compile_scene.py`` splits two jobs that must not be confused (design document §8.1):
* ``--meters-per-source-unit`` changes *lengths*: one stated source unit, in metres;
* ``--transform`` is **rigid** (orthonormal rotation + translation, no scale), and the
importer validates that before it compiles anything.
The finger design export (``orisys-sensor-design/build/export/finger/r01/compute``) is the
same surface as the SDK asset (``orisys-sdk/sensors/finger/assets/30fps-0703``) — identical
``faces`` and ``loop_to_vertex``, so vertex *i* is vertex *i* — but it lives in the design
repo's camera/world frame and in *design* units. ``align_housing.py`` fits that relationship
as a similarity ``p_sensor_mm = s·R·p_design + t``; the matrix it writes
(``housing/finger_r01_design_to_sensor.npy``) therefore **already carries the scale**: it is
not a legal ``--transform``, and pairing it with ``--meters-per-source-unit 1.8388e-5`` would
apply the scale twice.
This tool does the split, once and reproducibly:
design frame, design units --(R, t_m)--> sensor frame, metres
p_sensor_m = (0.001·s)·(R·p_design) + 0.001·t
so the compiler can be given ``--meters-per-source-unit 0.001·s`` and a rigid 4x4 whose
rotation block is exactly ``R`` and whose translation is ``0.001·t`` (metres). The metre
translation matters: ``t`` from the fit is in the sensor frame's *millimetres*, because that is
the frame the SDK asset is stated in.
It also checks, numerically, the wrong pairings — all three of which are live mistakes here:
* ``--transform`` = the scale-bearing matrix is **refused** by the rigid-transform validator
(it is not a legal rigid transform at all, whatever the unit says), so the "scale applied
twice" trap cannot reach the compiler through that door;
* what the double scale *would* produce is computed directly from the importer's own
``_apply_rigid``, so the number is on the record rather than asserted;
* ``--meters-per-source-unit 0.001`` (calling a design unit a millimetre) with the correct
rigid transform does compile, and is 54.38x too large — a package that looks entirely
plausible.
python tools/scene_assets/fit_design_frame.py \\
--design D:/orisys_ws/orisys-sensor-design/build/export/finger/r01/compute \\
--sensor D:/orisys_ws/orisys-sdk/sensors/finger/assets/30fps-0703 \\
--rigid-output tools/scene_assets/housing/finger_r01_design_to_sensor_rigid_m.npy \\
--report tools/scene_assets/housing/finger_r01_design_frame.json
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
import numpy as np
def _bootstrap_paths() -> None:
here = Path(__file__).resolve()
repo = here.parents[2]
for path in (repo / "apps" / "studio" / "src", repo / "shared" / "src"):
if path.is_dir() and str(path) not in sys.path:
sys.path.insert(0, str(path))
if str(here.parent) not in sys.path:
# The fit itself stays in one place: ``align_housing.fit_similarity``, which
# checks the 1032 correspondences before it writes anything.
sys.path.insert(0, str(here.parent))
#: The SDK asset's frame is millimetres; the compiled package is metres.
MM_PER_SENSOR_UNIT = 0.001
def derive(design_dir: Path, sensor_dir: Path, *, tolerance: float) -> dict:
from align_housing import apply_similarity, fit_similarity, load_faces, load_vertices
design_mesh = design_dir / "mesh.npz"
sensor_mesh = sensor_dir / "mesh.npz"
source = load_vertices(design_mesh)
target = load_vertices(sensor_mesh)
if source.shape != target.shape:
raise SystemExit(
f"the meshes disagree: {source.shape} vs {target.shape} — the correspondence the "
"fit relies on does not exist"
)
if not np.array_equal(load_faces(design_mesh), load_faces(sensor_mesh)):
raise SystemExit(
"the faces arrays differ, so vertex i is not vertex i; refusing to fit a transform"
)
rotation, translation_mm, scale_mm = fit_similarity(source, target)
residual_mm = apply_similarity(source, rotation, translation_mm, scale_mm) - target
rms_mm = float(np.sqrt((residual_mm**2).sum(axis=1).mean()))
diagonal_mm = float(np.linalg.norm(target.max(axis=0) - target.min(axis=0)))
relative = rms_mm / max(diagonal_mm, 1e-12)
if relative > float(tolerance):
raise SystemExit(
f"the fit is off by {relative:.3g} of the mesh diagonal (budget {tolerance:g}): "
"these two meshes are not the same shape"
)
# The split. ``scale_mm`` maps design units -> sensor millimetres, so the unit
# statement is that many millimetres, and the translation is carried into metres
# with the same factor the rotation gets.
#
# **Matrix convention.** ``align_housing.fit_similarity`` fits the *row-vector* form
# ``p_sensor = s·(p_design @ R) + t``, while ``compile_scene.py`` hands ``--transform``
# to ``legacy._apply_rigid``, which is the *column* form ``p_sensor = T @ [p_design; 1]``.
# So the block the compiler needs is ``R.T``, not ``R``: pasting ``R`` in would rotate the
# surface the wrong way by twice the fitted angle while leaving every extent plausible
# (a 90-degree rotation's transpose is its inverse). ``verify()`` is what proves this.
meters_per_source_unit = MM_PER_SENSOR_UNIT * scale_mm
rigid = np.eye(4, dtype=np.float64)
rigid[:3, :3] = rotation.T
rigid[:3, 3] = MM_PER_SENSOR_UNIT * translation_mm
scale_bearing = np.eye(4, dtype=np.float64)
scale_bearing[:3, :3] = scale_mm * rotation
scale_bearing[:3, 3] = translation_mm
extent_design = np.ptp(source, axis=0)
extent_sensor_mm = np.ptp(target, axis=0)
return {
"design_dir": str(design_dir),
"sensor_dir": str(sensor_dir),
"vertices": int(len(source)),
"fit": {
"scale_design_unit_to_sensor_mm": scale_mm,
"rotation_row_convention": rotation.tolist(),
"rotation_transposed_column_convention": rotation.T.tolist(),
"translation_sensor_mm": translation_mm.tolist(),
"rms_sensor_mm": rms_mm,
"rms_sensor_m": rms_mm * MM_PER_SENSOR_UNIT,
"diagonal_sensor_mm": diagonal_mm,
"relative_of_diagonal": relative,
"tolerance": float(tolerance),
},
"unit_statement": {
"meters_per_source_unit": meters_per_source_unit,
"one_design_unit_is_mm": scale_mm,
"one_sensor_mm_is_design_units": 1.0 / scale_mm,
},
"rigid_transform_sensor_from_design_m": rigid.tolist(),
"scale_bearing_matrix_sensor_mm_from_design": scale_bearing.tolist(),
"extent_design_units": extent_design.tolist(),
"extent_sensor_mm": extent_sensor_mm.tolist(),
"extent_compiled_m": (extent_sensor_mm * MM_PER_SENSOR_UNIT).tolist(),
"rotation_orthonormality_error": float(
np.abs(rotation @ rotation.T - np.eye(3)).max()
),
"rotation_determinant": float(np.linalg.det(rotation)),
}
def verify(design_dir: Path, sensor_dir: Path, rigid: np.ndarray, units: float) -> dict:
"""Compile in memory through the real importer and compare with the SDK-frame route.
The reference is the *shipped* route: the SDK asset with ``0.001`` and no transform.
If the design route is right, both land on the same sensor-metre positions — order
included, because identical ``faces``/``loop_to_vertex`` and no UV seam means the
render vertices come out in the same order.
"""
from orisys_app_shared.scene3d.contracts import ThreeDError
from orisys_app_shared.scene3d.legacy import LegacyFingerImporter, validate_rigid_transform
design_map = design_dir / "unroll_map_uv_s0.42.npz"
sensor_map = sensor_dir / "unroll_map_uv_s0.4.npz"
importer = LegacyFingerImporter()
reference = importer.convert(
sensor_dir, sensor_map,
transform_sensor_from_source=np.eye(4),
meters_per_source_unit=MM_PER_SENSOR_UNIT,
).sensing_part().positions_sensor_m
result: dict = {}
compiled = importer.convert(
design_dir, design_map,
transform_sensor_from_source=rigid,
meters_per_source_unit=units,
).sensing_part().positions_sensor_m
difference = np.asarray(compiled, dtype=np.float64) - np.asarray(reference, dtype=np.float64)
result["design_route"] = {
"extent_m": np.ptp(compiled, axis=0).tolist(),
"low_m": compiled.min(axis=0).tolist(),
"high_m": compiled.max(axis=0).tolist(),
"max_abs_difference_from_sdk_route_m": float(np.abs(difference).max()),
"rms_difference_from_sdk_route_m": float(np.sqrt((difference**2).sum(axis=1).mean())),
"bitwise_identical_to_sdk_route": bool(
np.array_equal(np.asarray(compiled, dtype=np.float32),
np.asarray(reference, dtype=np.float32))
),
}
result["sdk_route_extent_m"] = np.ptp(reference, axis=0).tolist()
from orisys_app_shared.scene3d.legacy import _apply_rigid, split_at_uv_seams
stored = np.load(
Path(__file__).resolve().parent / "housing" / "finger_r01_design_to_sensor.npy"
)
# Trap 1: the scale-bearing matrix as ``--transform``. Refused, whatever the unit is.
try:
validate_rigid_transform(stored)
result["trap_stored_matrix_as_transform"] = "ACCEPTED (the validator missed it)"
except ThreeDError as exc:
result["trap_stored_matrix_as_transform"] = (
f"refused: {exc.diagnostic.code}: {exc.diagnostic.message}"
)
# Trap 2: what the double scale *would* do, computed through the importer's own
# rigid application — the stored matrix already holds 1/54.38, and the derived unit
# holds it too, so the rotation block scales twice and ``t`` (sensor-mm) is added to
# a metre-scale result.
with np.load(design_dir / "mesh.npz", allow_pickle=False) as handle:
positions, _, _, _ = split_at_uv_seams(
handle["vertices"], handle["faces"], handle["uv_data"], handle["loop_to_vertex"]
)
doubled = _apply_rigid(positions * units, stored)
result["trap_double_scale_if_forced"] = {
"extent_m": np.ptp(doubled, axis=0).tolist(),
"low_m": doubled.min(axis=0).tolist(),
"high_m": doubled.max(axis=0).tolist(),
"note": (
"the stated unit already carries the 1/54.38 fit scale and the matrix carries it "
"again; the translation is sensor-mm added to a metre-scale result, so the extent "
"collapses to the fit residual and the surface sits ~4.5 m from the origin"
),
}
# Trap 3: a *legal* rigid transform with the wrong unit statement (a design unit is not
# a millimetre). This one compiles; nothing downstream can tell it is 54.38x too big.
milli = importer.convert(
design_dir, design_map,
transform_sensor_from_source=rigid,
meters_per_source_unit=MM_PER_SENSOR_UNIT,
).sensing_part().positions_sensor_m
result["trap_design_unit_called_a_millimetre"] = {
"extent_m": np.ptp(milli, axis=0).tolist(),
"low_m": milli.min(axis=0).tolist(),
"high_m": milli.max(axis=0).tolist(),
"factor_too_large": float(
np.ptp(milli, axis=0).max() / max(np.ptp(compiled, axis=0).max(), 1e-30)
),
"note": "compiles cleanly and is wrong by the fit scale; the validator cannot see it",
}
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--design", type=Path, required=True, help="design export dir")
parser.add_argument("--sensor", type=Path, required=True, help="sensor-frame export dir")
parser.add_argument("--rigid-output", type=Path, required=True, help="rigid 4x4 to write")
parser.add_argument("--report", type=Path, default=None, help="JSON record of the derivation")
parser.add_argument("--tolerance", type=float, default=1e-4)
parser.add_argument(
"--no-verify",
action="store_true",
help="skip the in-memory compile checks (they need the app's shared package)",
)
args = parser.parse_args(argv)
_bootstrap_paths()
record = derive(args.design, args.sensor, tolerance=args.tolerance)
rigid = np.asarray(record["rigid_transform_sensor_from_design_m"], dtype=np.float64)
units = float(record["unit_statement"]["meters_per_source_unit"])
fit = record["fit"]
print(f"fit over {record['vertices']} design->sensor correspondences "
f"(identical faces, so vertex i is vertex i)")
print(f" scale {fit['scale_design_unit_to_sensor_mm']:.12g} "
f"sensor-mm per design unit (= 1/{(1.0 / fit['scale_design_unit_to_sensor_mm']):.8f})")
print(f" residual rms {fit['rms_sensor_mm']:.3g} mm "
f"({fit['rms_sensor_m']:.3g} m) = {fit['relative_of_diagonal']:.3g} of the "
f"{fit['diagonal_sensor_mm']:.3f}-mm diagonal (budget {fit['tolerance']:g})")
print(f" rotation orthonormality error {record['rotation_orthonormality_error']:.3g}, "
f"det {record['rotation_determinant']:.12f}")
print(f" translation {[round(v, 12) for v in fit['translation_sensor_mm']]} sensor-mm")
print()
print("the split the compiler needs:")
print(f" --meters-per-source-unit {units:.12g} (one design unit is "
f"{(1.0 / fit['scale_design_unit_to_sensor_mm']):.8f} sensor-mm)")
print(f" --transform <rigid 4x4> rotation = R, translation = 0.001*t = "
f"{[round(float(v), 12) for v in rigid[:3, 3]]} m")
print(f" design extent {np.round(record['extent_design_units'], 4)} -> sensor "
f"{np.round(record['extent_sensor_mm'], 4)} mm = "
f"{np.round(record['extent_compiled_m'], 6)} m")
if not args.no_verify:
print()
print("in-memory compile checks through the real importer:")
checks = verify(args.design, args.sensor, rigid, units)
design = checks["design_route"]
print(f" design route extent {np.round(design['extent_m'], 8)} m "
f"low {np.round(design['low_m'], 8)} high {np.round(design['high_m'], 8)}")
print(f" sdk route extent {np.round(checks['sdk_route_extent_m'], 8)} m")
print(f" agreement max|diff| {design['max_abs_difference_from_sdk_route_m']:.3g} m, "
f"rms {design['rms_difference_from_sdk_route_m']:.3g} m, "
f"bitwise identical {design['bitwise_identical_to_sdk_route']}")
print(f" trap (stored scale-bearing matrix as --transform): "
f"{checks['trap_stored_matrix_as_transform']}")
forced = checks["trap_double_scale_if_forced"]
print(f" trap (derived unit + scale-bearing matrix, forced): extent "
f"{np.round(forced['extent_m'], 9)} m, low {np.round(forced['low_m'], 6)}, "
f"high {np.round(forced['high_m'], 6)}")
milli = checks["trap_design_unit_called_a_millimetre"]
print(f" trap (design unit called a millimetre): extent "
f"{np.round(milli['extent_m'], 6)} m — "
f"{milli['factor_too_large']:.5f}x too large, and accepted")
record["verification"] = checks
args.rigid_output.parent.mkdir(parents=True, exist_ok=True)
np.save(args.rigid_output, rigid)
print()
print(f"wrote {args.rigid_output}")
if args.report is not None:
args.report.parent.mkdir(parents=True, exist_ok=True)
args.report.write_text(
json.dumps(record, indent=2, ensure_ascii=False) + "\n", encoding="utf-8"
)
print(f"wrote {args.report}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/scene_assets/housing/finger_r01_design_frame.json
+191
View File
@@ -0,0 +1,191 @@
{
"design_dir": "D:\\orisys_ws\\orisys-sensor-design\\build\\export\\finger\\r01\\compute",
"sensor_dir": "D:\\orisys_ws\\orisys-sdk\\sensors\\finger\\assets\\30fps-0703",
"vertices": 1032,
"fit": {
"scale_design_unit_to_sensor_mm": 0.01838845151146365,
"rotation_row_convention": [
[
8.005154792182337e-08,
0.9999999999999964,
-4.310626667257856e-08
],
[
4.169177018082366e-08,
-4.310626975245822e-08,
-0.999999999999999
],
[
-0.9999999999999958,
8.005154632977046e-08,
-4.169177398834113e-08
]
],
"rotation_transposed_column_convention": [
[
8.005154792182337e-08,
4.169177018082366e-08,
-0.9999999999999958
],
[
0.9999999999999964,
-4.310626975245822e-08,
8.005154632977046e-08
],
[
-4.310626667257856e-08,
-0.999999999999999,
-4.169177398834113e-08
]
],
"translation_sensor_mm": [
4.555480057606443,
-0.18626088406372004,
8.49151750616984
],
"rms_sensor_mm": 5.97442703083736e-07,
"rms_sensor_m": 5.97442703083736e-10,
"diagonal_sensor_mm": 23.047642302706198,
"relative_of_diagonal": 2.5922074598215443e-08,
"tolerance": 0.0001
},
"unit_statement": {
"meters_per_source_unit": 1.8388451511463648e-05,
"one_design_unit_is_mm": 0.01838845151146365,
"one_sensor_mm_is_design_units": 54.381958120648946
},
"rigid_transform_sensor_from_design_m": [
[
8.005154792182337e-08,
4.169177018082366e-08,
-0.9999999999999958,
0.004555480057606444
],
[
0.9999999999999964,
-4.310626975245822e-08,
8.005154632977046e-08,
-0.00018626088406372004
],
[
-4.310626667257856e-08,
-0.999999999999999,
-4.169177398834113e-08,
0.008491517506169841
],
[
0.0,
0.0,
0.0,
1.0
]
],
"scale_bearing_matrix_sensor_mm_from_design": [
[
1.4720240073780576e-09,
0.018388451511463583,
-7.926574945489324e-10,
4.555480057606443
],
[
7.66647094397162e-10,
-7.926575511831501e-10,
-0.018388451511463632,
-0.18626088406372004
],
[
-0.018388451511463573,
1.47202397810267e-09,
-7.666471644115123e-10,
8.49151750616984
],
[
0.0,
0.0,
0.0,
1.0
]
],
"extent_design_units": [
815.7294616699219,
407.8647003173828,
859.7575836181641
],
"extent_sensor_mm": [
15.809610366821289,
15.000001430511475,
7.499999523162842
],
"extent_compiled_m": [
0.01580961036682129,
0.015000001430511474,
0.007499999523162842
],
"rotation_orthonormality_error": 1.5543122344752192e-15,
"rotation_determinant": 1.0000000000000013,
"verification": {
"design_route": {
"extent_m": [
0.01580961048603058,
0.015000001527369022,
0.007499999832361937
],
"low_m": [
-0.01580961048603058,
-0.007500000763684511,
-2.6377949824407665e-10
],
"high_m": [
-1.61737203491219e-11,
0.007500000763684511,
0.007499999366700649
],
"max_abs_difference_from_sdk_route_m": 2.7939677238464355e-09,
"rms_difference_from_sdk_route_m": 7.126150500418507e-10,
"bitwise_identical_to_sdk_route": false
},
"sdk_route_extent_m": [
0.01580961048603058,
0.015000000596046448,
0.007499999366700649
],
"trap_stored_matrix_as_transform": "refused: geometry_mismatch: transform is not a rigid rotation (its rows are not orthonormal): it would scale the geometry as well",
"trap_double_scale_if_forced": {
"extent_m": [
0.00013791339792046386,
0.0002907142510321925,
0.00027582680075788346
],
"low_m": [
4.5554982900546195,
-0.1866353665546318,
8.49137616772656
],
"high_m": [
4.55563620345254,
-0.18634465230359962,
8.491651994527318
],
"note": "the stated unit already carries the 1/54.38 fit scale and the matrix carries it again; the translation is sensor-mm added to a metre-scale result, so the extent collapses to the fit residual and the surface sits ~4.5 m from the origin"
},
"trap_design_unit_called_a_millimetre": {
"extent_m": [
0.8597576022148132,
0.8157294988632202,
0.40786468982696533
],
"low_m": [
-1.1029380559921265,
-0.39792177081108093,
-0.4532938599586487
],
"high_m": [
-0.24318045377731323,
0.4178076982498169,
-0.04542917385697365
],
"factor_too_large": 54.3819580078125,
"note": "compiles cleanly and is wrong by the fit scale; the validator cannot see it"
}
}
}
tools/scene_assets/housing/finger_r01_design_to_sensor.npy
BIN
View File
Binary file not shown.
tools/scene_assets/housing/finger_r01_design_to_sensor_rigid_m.npy
BIN
View File
Binary file not shown.
tools/scene_assets/housing/finger_r01_housing.npz
BIN
View File
Binary file not shown.