阶段 2：资产合同 —— 旧 NPZ 导入、坐标/单位、绑定、包容器与工具 - What: shared/scene3d 新增四个模块与两个工具的真实现。digest.py：规范化内容摘要（C 连续小端字节 + 排序 JSON；DIGEST_SCHEMA 版本化），geometry_id（只看 part 角色/顶点/三角形，故意不含法线材质与 LOD 名——改颜色不得让标定失效）、binding_id、field_domain_id（形状、flip_v、source_scale、原点/索引约定、映射内容）、asset_hash（条目摘要 + 去自引用字段的 manifest）。legacy.py：LegacyFingerImporter 按"先单位、后刚体"两步转换（meters_per_source_unit 必填且不得猜、validate_rigid_transform 拒绝含缩放/镜像/非仿射的 4x4）；split_at_uv_seams 按 (源顶点, 量化 UV) 拆点，接缝处不平均 UV；vertex_normals 面积加权并给无面顶点有限回退；uv_to_pixel 采用实测约定 x=u*(W-1)、y=(1-v)*(H-1)（flip_v 时）；build_binding 生成四点双线性采样表（域内裁剪、零权 tap 丢弃并重分配、未用项 -1/0）；uv_correspondence_residual 用 Procrustes 残差把"UV 是否对得上 world_xyz"变成可断言的事实；estimate_meters_per_source_unit 只报告不应用。glb.py：glTF 2.0 子集读取（GLB 头/chunk 校验、accessor/bufferView/stride、节点 TRS 与矩阵烘焙进顶点、法线用逆转置再归一化、缺法线则计算），明确拒绝 Draco/meshopt/量化扩展、稀疏 accessor、外部 buffer、骨骼、形态目标、非 TRIANGLES、越界索引、非有限坐标与超预算三角形；with_housing 把外壳作为独立 part 加入且不动感知几何 id。package.py：.oryscene（zip：envelope.json + manifest.json + sensing/housing *.npy，无 pickle 无代码执行），写入原子（先写 .partial 再替换）、读取逐一校验 magic/envelope 版本/payload 状态/条目路径/manifest 自哈希/逐条目 sha256/dtype/shape/**从包内数组重算 geometry_id 与 binding_id**，加密封套明确报"本版本只开明文包"。tools/scene_assets/validate_scene.py 与 compile_scene.py 真实现（含 --json 报告与退出码：0 通过、2 检查失败、1 参数/文件问题；未给单位时停在 needs_units 并打印实测候选尺度）。测试：shared/tests/test_scene3d_assets.py 33 项（真实 1032 导出与 1005 对比资产的输入样例、接缝/法线/绑定/单位/变换/摘要的失败样例、GLB 往返与拒绝样例）、test_scene3d_package.py 24 项（往返、外壳条目、以及篡改条目/篡改 manifest 哈希/自洽但错误的几何/非有限坐标/多余条目/缺条目/路径穿越/dtype 与 shape 不符/未来版本/加密封套/key provider 的拒绝）。docs/architecture/studio-3d-stages.md 阶段 2 记为完成并附验收结果。 - Why: 阶段 2 要回答的是"这份几何到底是什么、单位是什么、它和哪套 UV 域配套、以及这份包有没有被动过"，这些一旦错了会以"三维看起来有点怪"的形式出现且事后无法追溯。因此把 UV 约定做成可测量的事实（真实资产残差 0.0064 vs 错误翻转 0.59）、把单位做成必填参数而不是默认值、把 identity 做成内容摘要并**在读取时重算**，同时用真实资产当输入样例、用篡改后的包当失败样例。 - Verification: 真实资产导入：1032 顶点/1974 三角形/域 300×280/尺寸 15.81×15.0×7.5 mm，validate_assets 通过；UV 残差 0.0064（错翻转 0.59）；两套资产混用被拒（geometry_mismatch）；缩放/镜像/非仿射变换被拒；compile→validate 端到端 ok（包内条目哈希与 manifest 自哈希全部通过）；shared 全量 244 passed；studio 118 passed；ui_check 554 项 0 失败；.repro/verify_assets.py 22 项 0 失败。过程中修掉 5 个真实缺陷：np.load 需要真实文件对象（手写内存 stub 抛空消息异常）、包可以"自洽却错误"（改数组+改哈希后无人重算几何 id，现已重算并比对）、binding 行数不匹配被误报为域错误、np.ptp 平铺比较无意义（换算/不变性断言改为按轴）、以及合成 PoC 场景自己用字符串当 geometry_id（所有人都会抄它，等于绕过检查）。 - Not in this change: 签名与 AES-GCM（阶段 5）；LOD 简化；calibration.py 侧的资产哈希登记；三维视图仍未被应用导入（阶段 4 接线）。

2026-09-14 15:08:29 +08:00
parent cad795c890
commit d0dcac743f
11 changed files with 3065 additions and 58 deletions
@@ -1,43 +1,111 @@
-"""Compile a validated scene, and optionally a housing GLB, into a runtime package.
+"""Compile a legacy export into a runtime package (stage 2 of the 3D plan).

-Stage 2 and stage 5 deliverables (``docs/architecture/studio-3d-stages.md``).
+Offline only: this is the tool that bakes the coordinate transform, the unit
+scale, the UV sampling table and the identity hashes into one file, so the runtime
+never repeats that work.

-Offline only: this tool is what turns a Blender export and the legacy NPZ geometry
-into the ``.oryscene`` payload the runtime loads. It bakes the coordinate
-transform, the sampling table and the LOD once, so the runtime never repeats that
-work; it signs with a key from the build environment, so no signing key ever
-reaches a customer runtime.
+Units are an argument, never a default. The legacy mesh is 15.8 x 15.0 x 7.5 source
+units; whether one of those is a millimetre is a physical claim about the sensor,
+and ``validate_scene.py`` prints the measured candidates so a person can decide.

-Until stage 2 lands, no package is produced and the exit code is non-zero, so a
-release script cannot ship a package that was never compiled.
+Example::
+
+    python tools/scene_assets/compile_scene.py \\
+        --legacy-dir sensors/finger/assets/30fps-0703 \\
+        --map       sensors/finger/assets/30fps-0703/unroll_map_uv_s0.4.npz \\
+        --meters-per-source-unit 0.001 \\
+        --output    build/finger-display-1.0.0.oryscene
 """

 from __future__ import annotations

 import argparse
+import json
 import sys
 from pathlib import Path

-_STAGE = "stage 2 (asset contracts) and stage 5 (release protection)"
+
+def _bootstrap_paths() -> None:
+    here = Path(__file__).resolve()
+    repo = here.parents[2]
+    for path in (repo / "apps" / "studio" / "src", repo / "shared" / "src"):
+        if path.is_dir() and str(path) not in sys.path:
+            sys.path.insert(0, str(path))


 def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
-    parser.add_argument("--legacy-dir", type=Path, default=None, help="legacy NPZ export")
-    parser.add_argument("--map", type=Path, default=None, help="run-time UV mapping (npz)")
-    parser.add_argument("--housing", type=Path, default=None, help="housing GLB")
+    parser.add_argument("--legacy-dir", type=Path, required=True, help="legacy NPZ export")
+    parser.add_argument("--map", type=Path, required=True, help="field map (npz)")
+    parser.add_argument(
+        "--meters-per-source-unit",
+        type=float,
+        required=True,
+        help="stated unit scale (the importer never guesses it)",
+    )
    parser.add_argument("--output", type=Path, required=True, help="package to write")
    parser.add_argument("--asset-id", default="finger-display")
-    parser.add_argument("--version", default="0.0.0")
+    parser.add_argument("--version", default="1.0.0")
+    parser.add_argument("--lod-id", default="lod0")
+    parser.add_argument("--sensor-model", default="finger")
+    parser.add_argument(
+        "--transform",
+        type=Path,
+        default=None,
+        help="optional 4x4 rigid transform (npy) from source to sensor coordinates",
+    )
+    parser.add_argument("--json", action="store_true")
    args = parser.parse_args(argv)

-    print(
-        "not_implemented: "
-        f"{_STAGE}. Would write {args.output} from "
-        f"legacy={args.legacy_dir} map={args.map} housing={args.housing} "
-        f"as {args.asset_id} {args.version}."
+    _bootstrap_paths()
+    import numpy as np
+
+    from orisys_app_shared.scene3d.contracts import ThreeDError
+    from orisys_app_shared.scene3d.legacy import LegacyFingerImporter
+    from orisys_app_shared.scene3d.package import write_package
+
+    transform = (
+        np.load(args.transform, allow_pickle=False)
+        if args.transform is not None
+        else np.eye(4)
    )
-    return 2
+    try:
+        assets = LegacyFingerImporter().convert(
+            args.legacy_dir,
+            args.map,
+            transform_sensor_from_source=transform,
+            meters_per_source_unit=args.meters_per_source_unit,
+            asset_id=args.asset_id,
+            asset_version=args.version,
+            lod_id=args.lod_id,
+        )
+        report = write_package(
+            assets, args.output,
+            sensor_model=args.sensor_model,
+            hardware_revisions=("*",),
+        )
+    except ThreeDError as exc:
+        print(f"rejected: {exc.diagnostic.code}: {exc.diagnostic.message}", file=sys.stderr)
+        return 2
+    summary = {
+        "status": "ok",
+        "output": str(report.output),
+        "asset_hash": report.asset_hash,
+        "vertices": report.part_vertex_counts,
+        "triangles": report.part_triangle_counts,
+        "geometry_id": assets.geometry_id,
+        "field_domain_id": assets.field_domain_id,
+    }
+    if args.json:
+        print(json.dumps(summary, indent=2, ensure_ascii=False))
+    else:
+        print(
+            f"wrote {report.output}  vertices={report.part_vertex_counts} "
+            f"triangles={report.part_triangle_counts}"
+        )
+        print(f"  geometry_id={assets.geometry_id}")
+        print(f"  asset_hash={report.asset_hash}")
+    return 0


 if __name__ == "__main__":
@@ -1,10 +1,21 @@
-"""Validate a scene package (or a legacy import) without a GPU.
+"""Validate a scene package, or import a legacy export and validate that.

 Stage 2 deliverable (``docs/architecture/studio-3d-stages.md``): the checks the
 loader and the compiler must agree on, runnable on a build machine and in CI.

-Until stage 2 lands, importing an asset is not implemented; the argument parsing
-and the exit contract exist so the CI job can be wired now.
+Exit codes: 0 = every check passed, 2 = a check failed (with the reason printed),
+1 = the arguments or the files were wrong.
+
+Examples::
+
+    # a package produced by compile_scene.py
+    python tools/scene_assets/validate_scene.py build/finger.oryscene
+
+    # the legacy export, before packaging (units are stated, never guessed)
+    python tools/scene_assets/validate_scene.py \\
+        --legacy-dir sensors/finger/assets/30fps-0703 \\
+        --map       sensors/finger/assets/30fps-0703/unroll_map_uv_s0.4.npz \\
+        --meters-per-source-unit 0.001
 """

 from __future__ import annotations
@@ -14,31 +25,208 @@ import json
 import sys
 from pathlib import Path

-_STAGE = "stage 2 (asset contracts)"
+
+def _bootstrap_paths() -> None:
+    here = Path(__file__).resolve()
+    repo = here.parents[2]
+    for path in (repo / "apps" / "studio" / "src", repo / "shared" / "src"):
+        if path.is_dir() and str(path) not in sys.path:
+            sys.path.insert(0, str(path))
+
+
+def _legacy_report(args) -> tuple[int, dict]:
+    import numpy as np
+
+    from orisys_app_shared.scene3d.contracts import ExpectedGeometry, ThreeDError, validate_assets
+    from orisys_app_shared.scene3d.legacy import (
+        LegacyFingerImporter,
+        estimate_meters_per_source_unit,
+        load_legacy_domain,
+        split_at_uv_seams,
+        uv_correspondence_residual,
+    )
+
+    report: dict = {"source": "legacy", "dir": str(args.legacy_dir), "checks": []}
+
+    def record(name: str, ok: bool, detail: str = "") -> None:
+        report["checks"].append({"check": name, "ok": bool(ok), "detail": detail})
+
+    map_path = Path(args.map)
+    domain = load_legacy_domain(map_path)
+    report["field_domain"] = {
+        "shape_hw": list(domain.shape_hw),
+        "flip_v": domain.flip_v,
+        "source_scale": domain.source_scale,
+        "domain_id": domain.domain_id,
+        "valid_ratio": round(domain.valid_ratio, 4),
+    }
+    record("field domain reads", True, f"{domain.shape_hw} flip_v={domain.flip_v}")
+
+    with np.load(Path(args.legacy_dir) / "mesh.npz", allow_pickle=False) as handle:
+        positions, uvs, _, _ = split_at_uv_seams(
+            handle["vertices"], handle["faces"], handle["uv_data"], handle["loop_to_vertex"]
+        )
+    report["mesh"] = {
+        "render_vertices": int(len(positions)),
+        "source_vertices": int(len(np.load(Path(args.legacy_dir) / "mesh.npz")["vertices"])),
+        "extent_source_units": [float(v) for v in np.ptp(positions, axis=0)],
+    }
+    if domain.world_xyz is not None:
+        residual, samples = uv_correspondence_residual(positions, uvs, domain)
+        report["uv_correspondence"] = {
+            "relative_residual": round(residual, 4),
+            "samples": int(samples),
+        }
+        record(
+            "UV convention agrees with world_xyz",
+            residual < 0.05,
+            f"residual {residual:.4f} over {samples} samples",
+        )
+    measured = estimate_meters_per_source_unit(positions)
+    report["scale"] = {
+        "meters_per_source_unit_if_span_is_1m": round(measured, 6),
+        "stated": args.meters_per_source_unit,
+        "extent_m": (
+            [float(v) * args.meters_per_source_unit for v in np.ptp(positions, axis=0)]
+            if args.meters_per_source_unit
+            else None
+        ),
+    }
+
+    if not args.meters_per_source_unit:
+        report["status"] = "needs_units"
+        report["note"] = (
+            "state --meters-per-source-unit: the importer never guesses what a source "
+            "unit is. Compare 'extent_source_units' with the sensor's real size; a "
+            "15.8-unit span is 15.8 mm on the shipped finger sensor, i.e. 0.001."
+        )
+        return 1, report
+
+    try:
+        assets = LegacyFingerImporter().convert(
+            Path(args.legacy_dir),
+            map_path,
+            transform_sensor_from_source=np.eye(4),
+            meters_per_source_unit=float(args.meters_per_source_unit),
+        )
+    except ThreeDError as exc:
+        record("import", False, f"{exc.diagnostic.code}: {exc.diagnostic.message}")
+        report["status"] = "rejected"
+        return 2, report
+    record("import", True, f"{assets.sensing_part().positions_sensor_m.shape[0]} vertices")
+    try:
+        validate_assets(assets)
+        record("contracts", True)
+    except ThreeDError as exc:
+        record("contracts", False, f"{exc.diagnostic.code}: {exc.diagnostic.message}")
+    if args.sensor_model:
+        expected = ExpectedGeometry(
+            sensor_model=args.sensor_model,
+            hardware_revision=args.hardware_revision,
+            geometry_id=assets.geometry_id,
+            field_domain_id=assets.field_domain_id,
+            shape_hw=assets.shape_hw,
+        )
+        try:
+            validate_assets(assets, expected)
+            record("device description", True, args.sensor_model)
+        except ThreeDError as exc:
+            record("device description", False, exc.diagnostic.code)
+    report["geometry_id"] = assets.geometry_id
+    report["asset_hash"] = assets.asset_hash
+    report["binding"] = {
+        "rows": int(assets.binding.sample_indices.shape[0]),
+        "weight_sum_min": float(np.asarray(assets.binding.sample_weights).sum(axis=1).min()),
+        "weight_sum_max": float(np.asarray(assets.binding.sample_weights).sum(axis=1).max()),
+    }
+    report["status"] = "ok" if all(item["ok"] for item in report["checks"]) else "failed"
+    return (0 if report["status"] == "ok" else 2), report
+
+
+def _package_report(args) -> tuple[int, dict]:
+    from orisys_app_shared.scene3d.contracts import ExpectedGeometry, ThreeDError
+    from orisys_app_shared.scene3d.package import (
+        describe_package,
+        read_package,
+        scene_from_contents,
+    )
+
+    report: dict = {"source": "package", "path": str(args.asset), "checks": []}
+
+    def record(name: str, ok: bool, detail: str = "") -> None:
+        report["checks"].append({"check": name, "ok": bool(ok), "detail": detail})
+
+    try:
+        contents = read_package(Path(args.asset))
+        record("envelope, manifest and every entry hash", True)
+    except ThreeDError as exc:
+        record("envelope, manifest and every entry hash", False,
+               f"{exc.diagnostic.code}: {exc.diagnostic.message}")
+        report["status"] = "rejected"
+        return 2, report
+    report["manifest"] = describe_package(Path(args.asset))
+    try:
+        expected = (
+            ExpectedGeometry(
+                args.sensor_model, args.hardware_revision,
+                str(contents.manifest.get("geometry_id")),
+                str((contents.manifest.get("field_domain") or {}).get("id")),
+                tuple(int(v) for v in (contents.manifest.get("field_domain") or {}).get("shape_hw", (0, 0))),
+            )
+            if args.sensor_model
+            else None
+        )
+        assets = scene_from_contents(contents, expected)
+        record("scene builds and validates", True,
+               f"{assets.sensing_part().positions_sensor_m.shape[0]} vertices")
+    except ThreeDError as exc:
+        record("scene builds and validates", False,
+               f"{exc.diagnostic.code}: {exc.diagnostic.message}")
+    report["status"] = "ok" if all(item["ok"] for item in report["checks"]) else "failed"
+    return (0 if report["status"] == "ok" else 2), report


 def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
-    parser.add_argument("asset", type=Path, help="scene package or legacy asset directory")
-    parser.add_argument("--housing", type=Path, default=None, help="housing GLB to check")
+    parser.add_argument("asset", nargs="?", type=Path, help="scene package to validate")
+    parser.add_argument("--legacy-dir", type=Path, default=None, help="legacy NPZ export")
+    parser.add_argument("--map", type=Path, default=None, help="legacy field map (npz)")
+    parser.add_argument(
+        "--meters-per-source-unit",
+        type=float,
+        default=None,
+        help="stated unit scale; without it the legacy report stops at 'needs_units'",
+    )
+    parser.add_argument("--sensor-model", default="finger")
+    parser.add_argument("--hardware-revision", default="*")
    parser.add_argument("--json", action="store_true", help="machine-readable output")
    args = parser.parse_args(argv)

-    report = {
-        "status": "not_implemented",
-        "asset": str(args.asset),
-        "stage": _STAGE,
-        "note": (
-            "Checks: manifest/schema, signature and entry hashes, coordinate frame "
-            "and units, part roles, index ranges, binding weights and domain, "
-            "geometry/domain match against the device description."
-        ),
-    }
-    if args.json:
-        print(json.dumps(report, indent=2, ensure_ascii=False))
+    _bootstrap_paths()
+    if args.legacy_dir is not None:
+        if args.map is None:
+            print("--legacy-dir also needs --map", file=sys.stderr)
+            return 1
+        code, report = _legacy_report(args)
+    elif args.asset is not None:
+        code, report = _package_report(args)
    else:
-        print(f"{report['status']}: {report['note']}")
-    return 2
+        parser.print_help()
+        return 1
+
+    if args.json:
+        print(json.dumps(report, indent=2, ensure_ascii=False, default=str))
+    else:
+        print(f"{report['status']}: {report.get('path') or report.get('dir')}")
+        for item in report.get("checks", []):
+            mark = "ok  " if item["ok"] else "FAIL"
+            print(f"  {mark} {item['check']}  {item['detail']}")
+        for key in ("field_domain", "mesh", "uv_correspondence", "scale", "binding"):
+            if key in report:
+                print(f"  {key}: {json.dumps(report[key], ensure_ascii=False, default=str)}")
+        if "note" in report:
+            print(f"  note: {report['note']}")
+    return code


 if __name__ == "__main__":