feat(studio-3d): 阶段 5 —— 发布保护（签名/可选加密/平台锁定）与性能长时报告
- What: 新增 `scene3d/protect.py`（封套 v2：Ed25519 签名 + 可选 AES-256-GCM，nonce/salt 每包唯一、 元数据作 AAD、明文摘要进签名、KeyProvider 只问 get_content_key）与 `record/visualization3d.py` （session.json 的 visualization3d 块：写入版本化、读取时缺块=不支持）；`package.py` 支持 v2 （payload=plain 仅签名 / aes-gcm 签名+加密）并继续读 v1，新增解压比上限与"写入后自读回"校验； `contracts.py` 新增 PlatformLock/PlatformInfo/check_platform_lock、Sealing、platform_mismatch 诊断码； `Scene3DView` 在装配前查 OS/架构、拿到 capabilities 后查图形半区，违反即撤下场景并在面板显示原因； 工具 `compile_scene.py` 新增 --sign-key/--write-verify-key/--encrypt-key-id/--key-file/--lock-from-probe， `validate_scene.py` 新增 --key-file/--key-id/--verify-key/--require-signature（发布门禁）， 新增 `bench_scene.py`（p50/p95 分阶段 + --cpu-only + --vertices N + --package）与 `soak_scene.py`（RSS 四分位速率 + 队列上界 + 抽稀采样）；两份报告落到 docs/architecture/。 - Why: 规格 §5.2 的保护目标是"提高复制成本"而不是"防提取"，所以要的是签名（来源）+ 可选认证加密（机密性） + 密钥隔离，而不是自制加密或把 ZIP 当加密；§12 的"包保护/性能/长时"三行需要可复现的数字与判定标准， 而不是一次性的观感。平台锁定还必须只降级三维：锁定不通过时二维照常。 - Verification: shared 全量通过（新增 test_scene3d_protect.py 35 项：封套往返/篡改/密钥/锁定/记录块/ compile→validate 端到端发布流程；test_scene3d_package.py 20 项含 v1↔v2 兼容）；apps/studio 全量通过 （test_scene3d_view.py 30 项含锁定拒绝与 session.json 块）；ui_check 566 项 0 失败；locale_check 0 缺失； 真实资产端到端：1032 顶点编译出 signed+sealed+locked 包（56 927 B 密文），validate --require-signature 退出 0， 改一个字节退出 2；性能报告（perf-report-windows-dev.json）：1032 顶点 compose p50 0.50 ms（30 fps 上限内）， 20022 顶点/39480 三角形 compose p50 6.15 ms（约 33 ms 帧的 18%），采集线程 submit p95 < 0.4 ms（合成器被人为放慢到 50 ms）； 长时报告（soak-report-windows-dev.json）：无窗口 10 min/60 fps 尾段 0.354 MB/min，GUI 4 min/30 fps 尾段 0.809 MB/min， 队列上界（mailbox=1、单飞行合成）两者均成立。过程中修掉：GUI 进程里的合成器计时被 GIL/vsync 污染（98.8 ms vs 真值 6.1 ms）、 跨线程读 last_stats 撕裂、key id 无 provider 时会静默写明文包、soak 工具自身记时间列表会无界增长、RSS 读取失败伪装成零增长、 平台锁定不是终态诊断。 - Not in this change: ARM 板卡的能力报告与 2 小时门禁运行（按用户要求暂缓；命令与判定标准已写入阶段 5 文档）； GPU 显存随时间的测量（无可移植 API，报告说明渲染器缓冲估计恒定且更新路径不新建 GPU 对象）； `cryptography` 在此环境无法联网安装，本机验收用的是本机已有的 cp310 构建（pyproject 里已声明为可选 extra `protected-assets`）。
This commit is contained in:
ykcheungab
@@ -0,0 +1,667 @@
|
||||
"""Measure the 3D view's cost on this machine, per stage and as a percentile.
|
||||
|
||||
Stage 5 deliverable (``docs/architecture/studio-3d-stages.md``): the design
|
||||
document's performance row asks for p50/p95 of the *compose*, the *upload*, the
|
||||
*draw* and the total response latency, reported together with the resolution, the
|
||||
mesh and the GL context they were measured on. A single average would hide exactly
|
||||
the thing that matters here — the tail is where a display pipeline either keeps its
|
||||
frame rate or visibly stutters.
|
||||
|
||||
What is measured, and why each one is separate:
|
||||
|
||||
``submit_ms`` what the **acquisition thread** pays per frame. This is the only
|
||||
number that can damage the measurement, so it is measured with the
|
||||
composer deliberately slowed down: the display slowing must not
|
||||
move this number at all.
|
||||
``compose_ms`` the CPU worker: sample the UV field onto the vertices, deform,
|
||||
recompute normals.
|
||||
``upload_ms`` the GPU buffer update for the sensing surface.
|
||||
``paint_ms`` Qt's paint, including the widget's own bookkeeping.
|
||||
``latency_ms`` from ``submit`` to the frame being on screen — the number an
|
||||
operator actually perceives, and the one that includes every
|
||||
queue in between.
|
||||
|
||||
Run it with a real platform (offscreen cannot create a GL context)::
|
||||
|
||||
QT_QPA_PLATFORM=windows python tools/scene_assets/bench_scene.py \\
|
||||
--frames 600 --output docs/architecture/perf-report-windows-dev.json
|
||||
|
||||
Exit codes: 0 = measured on a GPU, 3 = software renderer, 4 = no usable context.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _bootstrap_paths() -> None:
|
||||
here = Path(__file__).resolve()
|
||||
repo = here.parents[2]
|
||||
for path in (repo / "apps" / "studio" / "src", repo / "shared" / "src"):
|
||||
if path.is_dir() and str(path) not in sys.path:
|
||||
sys.path.insert(0, str(path))
|
||||
|
||||
|
||||
def _percentiles(values: list[float]) -> dict[str, float]:
|
||||
if not values:
|
||||
return {"p50": 0.0, "p95": 0.0, "max": 0.0, "mean": 0.0, "n": 0}
|
||||
ordered = sorted(values)
|
||||
|
||||
def pick(fraction: float) -> float:
|
||||
index = max(0, min(len(ordered) - 1, int(round(fraction * len(ordered))) - 1))
|
||||
return float(ordered[index])
|
||||
|
||||
return {
|
||||
"p50": round(pick(0.50), 4),
|
||||
"p95": round(pick(0.95), 4),
|
||||
"max": round(float(ordered[-1]), 4),
|
||||
"mean": round(sum(ordered) / len(ordered), 4),
|
||||
"n": len(ordered),
|
||||
}
|
||||
|
||||
|
||||
def _grid_scene(target_vertices: int):
|
||||
"""A plane grid with a bilinear binding, for the design document's budget.
|
||||
|
||||
The design's performance row is written against "about 20k vertices and 40k
|
||||
triangles" for the sensing surface, and the shipped finger export is 1032 — so a
|
||||
report that only measured the PoC scene would answer a question nobody asked. This
|
||||
builds a grid of roughly ``target_vertices`` vertices whose UV domain *is* the
|
||||
grid, with the same measured pixel convention the real binding uses
|
||||
(``x = u(W-1)``, ``y = (1-v)(H-1)``) and a genuine four-tap bilinear table, so the
|
||||
sampling cost in the report is the real one rather than a nearest-neighbour
|
||||
stand-in.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraPreset,
|
||||
Material,
|
||||
MeshPart,
|
||||
SceneAssets,
|
||||
SurfaceBinding,
|
||||
freeze,
|
||||
)
|
||||
from orisys_app_shared.scene3d.digest import (
|
||||
field_domain_id as compute_domain_id,
|
||||
geometry_id as compute_geometry_id,
|
||||
)
|
||||
from orisys_app_shared.scene3d.legacy import vertex_normals
|
||||
|
||||
columns = max(2, int(round(target_vertices ** 0.5)))
|
||||
rows = max(2, int(round(target_vertices / columns)))
|
||||
count = rows * columns
|
||||
xs = np.linspace(-0.008, 0.008, columns, dtype=np.float64)
|
||||
ys = np.linspace(-0.008, 0.008, rows, dtype=np.float64)
|
||||
grid_x, grid_y = np.meshgrid(xs, ys)
|
||||
# A gentle dome, so the normals are not all identical and the shading path is used.
|
||||
radius = np.sqrt(grid_x**2 + grid_y**2)
|
||||
z = 0.001 * np.cos(np.clip(radius / 0.008, 0.0, 1.0) * np.pi / 2.0)
|
||||
positions = np.stack([grid_x, grid_y, z], axis=-1).reshape(-1, 3).astype(np.float32)
|
||||
|
||||
faces: list[tuple[int, int, int]] = []
|
||||
for i in range(rows - 1):
|
||||
for j in range(columns - 1):
|
||||
a = i * columns + j
|
||||
b = a + 1
|
||||
c = a + columns
|
||||
d = c + 1
|
||||
faces.append((a, c, b))
|
||||
faces.append((b, c, d))
|
||||
triangles = np.asarray(faces, dtype=np.uint32)
|
||||
normals = vertex_normals(positions.astype(np.float64), triangles.astype(np.int64))
|
||||
|
||||
height, width = rows, columns
|
||||
index = np.arange(count, dtype=np.int64)
|
||||
row_of = index // columns
|
||||
column_of = index % columns
|
||||
# Vertex (row, column) sits *between* pixels, so every vertex needs four taps.
|
||||
u = (column_of + 0.5) / columns
|
||||
v = 1.0 - (row_of + 0.5) / rows
|
||||
x = u * (width - 1)
|
||||
y = v * (height - 1)
|
||||
x0 = np.clip(np.floor(x).astype(np.int64), 0, width - 2)
|
||||
y0 = np.clip(np.floor(y).astype(np.int64), 0, height - 2)
|
||||
x1, y1 = x0 + 1, y0 + 1
|
||||
fx = (x - x0).astype(np.float32)
|
||||
fy = (y - y0).astype(np.float32)
|
||||
indices = np.stack(
|
||||
[
|
||||
y0 * width + x0,
|
||||
y0 * width + x1,
|
||||
y1 * width + x0,
|
||||
y1 * width + x1,
|
||||
],
|
||||
axis=-1,
|
||||
).astype(np.int32)
|
||||
weights = np.stack(
|
||||
[
|
||||
(1.0 - fx) * (1.0 - fy),
|
||||
fx * (1.0 - fy),
|
||||
(1.0 - fx) * fy,
|
||||
fx * fy,
|
||||
],
|
||||
axis=-1,
|
||||
).astype(np.float32)
|
||||
|
||||
domain_id = compute_domain_id(shape_hw=(height, width), flip_v=True, source_scale=1.0)
|
||||
preset = CameraPreset(
|
||||
name="bench",
|
||||
eye_sensor_m=(0.02, -0.02, 0.03),
|
||||
target_sensor_m=(0.0, 0.0, 0.0),
|
||||
up_sensor=(0.0, 0.0, 1.0),
|
||||
vertical_fov_rad=0.7,
|
||||
)
|
||||
part = MeshPart(
|
||||
part_id="sensing_surface",
|
||||
role="sensing",
|
||||
positions_sensor_m=freeze(positions, dtype=np.float32),
|
||||
normals_sensor=freeze(normals, dtype=np.float32),
|
||||
triangles=freeze(triangles, dtype=np.uint32),
|
||||
material=Material((0.85, 0.85, 0.88, 1.0), "unlit_scalar", True),
|
||||
lod_id="lod0",
|
||||
)
|
||||
geometry = compute_geometry_id((part,))
|
||||
binding = SurfaceBinding(
|
||||
part_id=part.part_id,
|
||||
geometry_id=geometry,
|
||||
field_domain_id=domain_id,
|
||||
shape_hw=(height, width),
|
||||
sample_indices=freeze(indices, dtype=np.int32),
|
||||
sample_weights=freeze(weights, dtype=np.float32),
|
||||
)
|
||||
bounds = np.stack([positions.min(axis=0), positions.max(axis=0)]).astype(np.float32)
|
||||
return SceneAssets(
|
||||
asset_id="bench-grid",
|
||||
asset_version="0",
|
||||
asset_hash=domain_id,
|
||||
geometry_id=geometry,
|
||||
parts=(part,),
|
||||
binding=binding,
|
||||
camera_presets=(preset,),
|
||||
bounds_sensor_m=freeze(bounds, dtype=np.float32),
|
||||
field_domain_id=domain_id,
|
||||
shape_hw=(height, width),
|
||||
)
|
||||
|
||||
|
||||
def measure_cpu(
|
||||
*,
|
||||
frames: int,
|
||||
vertices: int = 0,
|
||||
package: Path | None = None,
|
||||
key_file: Path | None = None,
|
||||
key_id: str = "",
|
||||
verify_key: Path | None = None,
|
||||
) -> tuple[dict, int]:
|
||||
"""The composer's own cost, with no window in the process.
|
||||
|
||||
This is the mode to quote for "how long does one frame take": in the GUI run the
|
||||
worker thread and the GUI thread compete for the GIL and for a vsync-throttled
|
||||
swap, so the composer's *wall* timers pick up 15-17 ms of waiting that has nothing
|
||||
to do with the arithmetic. Removing Qt from the process removes the contention, and
|
||||
what is left is the number an LOD decision should be made on.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.composer import SceneComposer
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
DisplaySettings,
|
||||
FieldDescriptor,
|
||||
FrameIdentity,
|
||||
SurfaceFieldFrame,
|
||||
freeze,
|
||||
)
|
||||
|
||||
report: dict = {
|
||||
"mode": "cpu",
|
||||
"frames_requested": int(frames),
|
||||
"scene": (
|
||||
f"package:{package}"
|
||||
if package is not None
|
||||
else (f"grid:{vertices}" if vertices else "synthetic-poc")
|
||||
),
|
||||
}
|
||||
if package is not None:
|
||||
from orisys_app_shared.scene3d import protect
|
||||
from orisys_app_shared.scene3d.package import PackagedAssetLoader
|
||||
|
||||
provider = (
|
||||
protect.load_key_file(Path(key_file), key_id=key_id)
|
||||
if key_file is not None
|
||||
else None
|
||||
)
|
||||
public = Path(verify_key).read_bytes() if verify_key is not None else None
|
||||
try:
|
||||
assets = PackagedAssetLoader().load(
|
||||
Path(package), None, provider, verify_key_pem=public
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
report["verdict"] = "asset_rejected"
|
||||
report["reason"] = str(exc)
|
||||
return report, 4
|
||||
elif vertices:
|
||||
assets = _grid_scene(int(vertices))
|
||||
else:
|
||||
assets = synthetic.poc_scene()
|
||||
report["mesh"] = {
|
||||
"sensing_vertices": int(assets.sensing_part().positions_sensor_m.shape[0]),
|
||||
"sensing_triangles": int(assets.sensing_part().triangles.shape[0]),
|
||||
"field_domain_hw": [int(assets.shape_hw[0]), int(assets.shape_hw[1])],
|
||||
}
|
||||
shape = tuple(assets.shape_hw)
|
||||
rng = np.random.default_rng(12345)
|
||||
scalar = freeze(
|
||||
(rng.random(shape) * 4.0).astype(np.float32), dtype=np.float32
|
||||
)
|
||||
valid = freeze(np.ones(shape, dtype=bool), dtype=np.bool_)
|
||||
frame = SurfaceFieldFrame(
|
||||
identity=FrameIdentity("bench", 1, 1, 0.0, generation=1),
|
||||
geometry_id=assets.geometry_id,
|
||||
field_domain_id=assets.field_domain_id,
|
||||
scalar=scalar,
|
||||
scalar_descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
|
||||
valid=valid,
|
||||
)
|
||||
composer = SceneComposer()
|
||||
settings = DisplaySettings()
|
||||
for _ in range(5):
|
||||
composer.compose(assets, frame, settings)
|
||||
totals: list[float] = []
|
||||
samples: list[float] = []
|
||||
normals: list[float] = []
|
||||
walls: list[float] = []
|
||||
for _ in range(int(frames)):
|
||||
started = time.perf_counter()
|
||||
composer.compose(assets, frame, settings)
|
||||
walls.append((time.perf_counter() - started) * 1000.0)
|
||||
stats = composer.last_stats
|
||||
totals.append(float(stats.total_ms))
|
||||
samples.append(float(stats.sample_ms))
|
||||
normals.append(float(stats.normals_ms))
|
||||
report["timings_ms"] = {
|
||||
"compose": _percentiles(totals),
|
||||
"compose_sample": _percentiles(samples),
|
||||
"compose_normals": _percentiles(normals),
|
||||
"compose_wall": _percentiles(walls),
|
||||
}
|
||||
report["verdict"] = "ok"
|
||||
return report, 0
|
||||
|
||||
|
||||
def measure(
|
||||
*,
|
||||
frames: int,
|
||||
profile: str,
|
||||
width: int,
|
||||
height: int,
|
||||
slow_compose_ms: float,
|
||||
max_fps: int,
|
||||
vertices: int = 0,
|
||||
package: Path | None = None,
|
||||
key_file: Path | None = None,
|
||||
key_id: str = "",
|
||||
verify_key: Path | None = None,
|
||||
) -> tuple[dict, int]:
|
||||
"""Run the assembled view and report what each stage cost."""
|
||||
import numpy as np
|
||||
from PyQt5.QtWidgets import QApplication
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.contracts import DisplaySettings
|
||||
from orisys_studio.scene3d.gl_backend import configure_surface_format
|
||||
from orisys_studio.scene3d.view import Scene3DView
|
||||
|
||||
configure_surface_format(profile) # type: ignore[arg-type]
|
||||
app = QApplication.instance() or QApplication(sys.argv)
|
||||
report: dict = {
|
||||
"profile_requested": profile,
|
||||
"frames_requested": int(frames),
|
||||
"viewport_px": [int(width), int(height)],
|
||||
"slow_compose_ms": float(slow_compose_ms),
|
||||
"max_fps_requested": int(max_fps),
|
||||
"scene": (
|
||||
f"package:{package}"
|
||||
if package is not None
|
||||
else (f"grid:{vertices}" if vertices else "synthetic-poc")
|
||||
),
|
||||
}
|
||||
view = Scene3DView()
|
||||
if not view.available:
|
||||
report["verdict"] = "no_view"
|
||||
report["reason"] = view.reason
|
||||
return report, 4
|
||||
if package is not None:
|
||||
from orisys_app_shared.scene3d.package import PackagedAssetLoader
|
||||
from orisys_app_shared.scene3d import protect
|
||||
|
||||
provider = (
|
||||
protect.load_key_file(Path(key_file), key_id=key_id)
|
||||
if key_file is not None
|
||||
else None
|
||||
)
|
||||
public = Path(verify_key).read_bytes() if verify_key is not None else None
|
||||
try:
|
||||
assets = PackagedAssetLoader().load(
|
||||
Path(package), None, provider, verify_key_pem=public
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
report["verdict"] = "asset_rejected"
|
||||
report["reason"] = str(exc)
|
||||
view.close()
|
||||
return report, 4
|
||||
elif vertices:
|
||||
assets = _grid_scene(int(vertices))
|
||||
else:
|
||||
assets = synthetic.poc_scene()
|
||||
if not view.use_scene(assets, source_id="bench"):
|
||||
report["verdict"] = "asset_rejected"
|
||||
report["reason"] = str(view.last_diagnostic)
|
||||
return report, 4
|
||||
view.resize(width, height)
|
||||
view.show()
|
||||
view.set_enabled(True)
|
||||
widget = view.gl_widget
|
||||
renderer = widget._renderer
|
||||
|
||||
deadline = time.perf_counter() + 10.0
|
||||
while time.perf_counter() < deadline and not renderer.part_ids:
|
||||
widget.repaint()
|
||||
app.processEvents()
|
||||
time.sleep(0.005)
|
||||
capabilities = widget.capabilities()
|
||||
if not renderer.part_ids or capabilities is None:
|
||||
report["verdict"] = "no_context"
|
||||
report["reason"] = str(widget.diagnostic())
|
||||
view.close()
|
||||
return report, 4
|
||||
report["context"] = {
|
||||
"profile_used": capabilities.profile,
|
||||
"vendor": capabilities.vendor,
|
||||
"renderer": capabilities.renderer,
|
||||
"gl_version": capabilities.gl_version,
|
||||
"glsl_version": capabilities.glsl_version,
|
||||
"software_renderer": bool(capabilities.software_renderer),
|
||||
"max_texture_size": int(capabilities.max_texture_size),
|
||||
"device_pixel_ratio": float(widget.devicePixelRatioF()),
|
||||
"qt_platform": capabilities.qt_platform,
|
||||
}
|
||||
report["mesh"] = {
|
||||
"sensing_vertices": int(assets.sensing_part().positions_sensor_m.shape[0]),
|
||||
"sensing_triangles": int(assets.sensing_part().triangles.shape[0]),
|
||||
"housing_triangles": int(
|
||||
assets.part("housing").triangles.shape[0] if assets.part("housing") else 0
|
||||
),
|
||||
"field_domain_hw": [int(assets.shape_hw[0]), int(assets.shape_hw[1])],
|
||||
"estimated_gpu_bytes": int(getattr(renderer, "_estimated_gpu_bytes", 0) or 0),
|
||||
}
|
||||
|
||||
coordinator = view.coordinator
|
||||
real_composer = coordinator._composer
|
||||
|
||||
compose_records: list[tuple[float, float, float, float]] = []
|
||||
compose_lock = __import__("threading").Lock()
|
||||
|
||||
class Slow:
|
||||
"""The composer, delayed: the point is that ``submit`` does not notice.
|
||||
|
||||
The statistics are collected *here*, on the worker thread that produces them,
|
||||
immediately after ``compose`` returns. Reading ``last_stats`` from the GUI loop
|
||||
instead produces torn samples — the worker writes ``sample_ms``, then
|
||||
``normals_ms``, then ``total_ms``, so a reader can see one frame's sampling next
|
||||
to the previous frame's normals, which reads as a plausible and completely wrong
|
||||
breakdown.
|
||||
"""
|
||||
|
||||
def __init__(self, delay_s: float, inner) -> None:
|
||||
self._delay = delay_s
|
||||
self._inner = inner
|
||||
self.last_stats = getattr(inner, "last_stats", None)
|
||||
|
||||
def compose(self, *args, **kwargs):
|
||||
started = time.perf_counter()
|
||||
result = self._inner.compose(*args, **kwargs)
|
||||
stats = getattr(self._inner, "last_stats", None)
|
||||
record = (
|
||||
float(getattr(stats, "total_ms", 0.0)),
|
||||
float(getattr(stats, "sample_ms", 0.0)),
|
||||
float(getattr(stats, "normals_ms", 0.0)),
|
||||
(time.perf_counter() - started) * 1000.0,
|
||||
)
|
||||
with compose_lock:
|
||||
compose_records.append(record)
|
||||
if self._delay:
|
||||
time.sleep(self._delay)
|
||||
return result
|
||||
|
||||
def rest_frame(self, *args, **kwargs):
|
||||
return self._inner.rest_frame(*args, **kwargs)
|
||||
|
||||
coordinator._composer = Slow(slow_compose_ms / 1000.0, real_composer)
|
||||
view.restore_settings(DisplaySettings(max_fps=max_fps))
|
||||
shape = tuple(assets.shape_hw)
|
||||
ramp = np.linspace(0.0, 1.0, int(shape[0]) * int(shape[1]), dtype=np.float32)
|
||||
flow = np.stack([ramp, ramp], axis=-1).reshape(int(shape[0]), int(shape[1]), 2)
|
||||
|
||||
class Packet:
|
||||
"""A packet shaped like the finger adapter expects."""
|
||||
|
||||
def __init__(self, index: int) -> None:
|
||||
self.session_id = 1
|
||||
self.frame_id = int(index)
|
||||
self.mono_t = float(index) / 30.0
|
||||
self.flow = flow + (index % 7) * 0.01
|
||||
self.raw_frame = np.zeros((4, 4, 3), dtype=np.uint8)
|
||||
|
||||
# Measured by wrapping the two places that define the timeline: `present` is the
|
||||
# moment a composed frame becomes the one that should be on screen, and `paintGL`
|
||||
# is the moment it actually is. Anything measured by polling the widget's own
|
||||
# statistics would report the *last published* sample (they are rate-limited to one
|
||||
# per 500 ms), which is how a benchmark ends up reporting a constant p50 for a
|
||||
# stage it never sampled.
|
||||
submits: list[float] = []
|
||||
latencies: list[float] = []
|
||||
composes: list[float] = []
|
||||
samples: list[float] = []
|
||||
normals: list[float] = []
|
||||
uploads: list[float] = []
|
||||
paints: list[float] = []
|
||||
submitted_at: dict[int, float] = {}
|
||||
presented_at: dict[int, float] = {}
|
||||
original_present = widget.present
|
||||
original_paint = widget.paintGL
|
||||
|
||||
def present(frame):
|
||||
presented_at[int(frame.identity.frame_id)] = time.perf_counter()
|
||||
original_present(frame)
|
||||
|
||||
def paint():
|
||||
frame = widget._frame # the frame this paint is about to consume
|
||||
started = time.perf_counter()
|
||||
original_paint()
|
||||
paints.append((time.perf_counter() - started) * 1000.0)
|
||||
if frame is not None:
|
||||
key = int(frame.identity.frame_id)
|
||||
if key in submitted_at:
|
||||
latencies.append((time.perf_counter() - submitted_at[key]) * 1000.0)
|
||||
|
||||
widget.present = present
|
||||
widget.paintGL = paint
|
||||
try:
|
||||
for index in range(int(frames)):
|
||||
started = time.perf_counter()
|
||||
view.submit_packet(Packet(index))
|
||||
submits.append((time.perf_counter() - started) * 1000.0)
|
||||
submitted_at[index] = started
|
||||
view.tick()
|
||||
app.processEvents()
|
||||
# Synchronous paint, the way the GL tests drive the widget: a queued
|
||||
# `update()` may not be delivered by `processEvents()` for a GL widget,
|
||||
# which would leave every paint-stage number at zero.
|
||||
widget.repaint()
|
||||
uploads.append(float(getattr(renderer, "_last_upload_ms", 0.0) or 0.0))
|
||||
finally:
|
||||
widget.present = original_present
|
||||
widget.paintGL = original_paint
|
||||
coordinator._composer = real_composer
|
||||
view.close()
|
||||
app.processEvents()
|
||||
|
||||
from orisys_studio.scene3d.coordinator import MAX_INFLIGHT_COMPOSES
|
||||
|
||||
with compose_lock:
|
||||
records = list(compose_records)
|
||||
composes = [item[0] for item in records]
|
||||
samples = [item[1] for item in records]
|
||||
normals = [item[2] for item in records]
|
||||
compose_wall = [item[3] for item in records]
|
||||
|
||||
coordinator_stats = {
|
||||
"superseded_display_frames": int(coordinator.superseded_frames),
|
||||
"stale_frames": int(coordinator.stale_frames),
|
||||
"invalid_frames": int(coordinator.invalid_frames),
|
||||
"max_inflight_composes": int(MAX_INFLIGHT_COMPOSES),
|
||||
"mailbox_capacity": 1,
|
||||
"frames_presented": len(presented_at),
|
||||
}
|
||||
report["timings_ms"] = {
|
||||
"submit": _percentiles(submits),
|
||||
"compose": _percentiles(composes),
|
||||
# Which half of the compose: on a large sensing surface one of the two
|
||||
# dominates, and "compose" alone cannot say which.
|
||||
"compose_sample": _percentiles(samples),
|
||||
"compose_normals": _percentiles(normals),
|
||||
# Wall clock inside the composer, which includes the artificial delay: the
|
||||
# difference between this and ``compose`` is the delay the caller asked for.
|
||||
"compose_wall": _percentiles(compose_wall),
|
||||
"upload": _percentiles(uploads),
|
||||
"paint": _percentiles(paints),
|
||||
"latency": _percentiles(latencies),
|
||||
}
|
||||
report["compose_count"] = len(records)
|
||||
report["compose_note"] = (
|
||||
"compose/compose_sample/compose_normals are the composer's own timers, "
|
||||
"collected on the worker thread; compose_wall adds the requested delay. "
|
||||
"latency is submit-to-paint and therefore includes the frame-rate limiter."
|
||||
)
|
||||
report["display"] = coordinator_stats
|
||||
report["crypto_backend"] = _crypto_version()
|
||||
if capabilities.software_renderer:
|
||||
report["verdict"] = "software_renderer"
|
||||
return report, 3
|
||||
report["verdict"] = "ok"
|
||||
return report, 0
|
||||
|
||||
|
||||
def _crypto_version() -> str:
|
||||
try:
|
||||
from orisys_app_shared.scene3d.package import backend_version
|
||||
|
||||
return str(backend_version())
|
||||
except Exception: # noqa: BLE001
|
||||
return "unavailable"
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--frames", type=int, default=600, help="frames to submit")
|
||||
parser.add_argument("--profile", default="desktop33", choices=("desktop33", "gles30"))
|
||||
parser.add_argument("--size", default="900x700", help="WxH of the 3D widget")
|
||||
parser.add_argument(
|
||||
"--slow-compose-ms",
|
||||
type=float,
|
||||
default=50.0,
|
||||
help="artificial composer delay: the submit budget must not move with it",
|
||||
)
|
||||
parser.add_argument("--max-fps", type=int, default=30)
|
||||
parser.add_argument(
|
||||
"--vertices",
|
||||
type=int,
|
||||
default=0,
|
||||
help="benchmark a generated grid of about this many sensing vertices "
|
||||
"(the design document's budget is 20k); 0 = the synthetic PoC scene",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--package",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="benchmark a released .oryscene package instead of a synthetic scene",
|
||||
)
|
||||
parser.add_argument("--key-file", type=Path, default=None, help="key for a sealed package")
|
||||
parser.add_argument("--key-id", default="", help="key id that --key-file holds")
|
||||
parser.add_argument("--verify-key", type=Path, default=None, help="public key for a signed package")
|
||||
parser.add_argument("--output", type=Path, default=None, help="write the report here")
|
||||
parser.add_argument(
|
||||
"--cpu-only",
|
||||
action="store_true",
|
||||
help="measure the composer with no window in the process (no vsync, no GIL "
|
||||
"contention): the number to quote for the per-frame CPU cost",
|
||||
)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
_bootstrap_paths()
|
||||
width, _, height = args.size.partition("x")
|
||||
if args.cpu_only:
|
||||
report, code = measure_cpu(
|
||||
frames=args.frames,
|
||||
vertices=args.vertices,
|
||||
package=args.package,
|
||||
key_file=args.key_file,
|
||||
key_id=args.key_id,
|
||||
verify_key=args.verify_key,
|
||||
)
|
||||
else:
|
||||
report, code = measure(
|
||||
frames=args.frames,
|
||||
profile=args.profile,
|
||||
width=int(width),
|
||||
height=int(height or 700),
|
||||
slow_compose_ms=args.slow_compose_ms,
|
||||
max_fps=args.max_fps,
|
||||
vertices=args.vertices,
|
||||
package=args.package,
|
||||
key_file=args.key_file,
|
||||
key_id=args.key_id,
|
||||
verify_key=args.verify_key,
|
||||
)
|
||||
report["exit_code"] = int(code)
|
||||
text = json.dumps(report, indent=2, ensure_ascii=False, default=str)
|
||||
if args.output is not None:
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(text + "\n", encoding="utf-8")
|
||||
if args.json or args.output is None:
|
||||
print(text)
|
||||
else:
|
||||
timings = report.get("timings_ms", {})
|
||||
print(
|
||||
f"verdict={report['verdict']} frames={report['frames_requested']} "
|
||||
f"renderer={(report.get('context') or {}).get('renderer', '')}"
|
||||
)
|
||||
for stage in (
|
||||
"submit",
|
||||
"compose",
|
||||
"compose_sample",
|
||||
"compose_normals",
|
||||
"compose_wall",
|
||||
"upload",
|
||||
"paint",
|
||||
"latency",
|
||||
):
|
||||
if stage in timings:
|
||||
item = timings[stage]
|
||||
print(
|
||||
f" {stage:8s} p50={item['p50']:8.3f} p95={item['p95']:8.3f} "
|
||||
f"max={item['max']:8.3f} n={item['n']}"
|
||||
)
|
||||
print(f" wrote {args.output}")
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,20 +1,39 @@
|
||||
"""Compile a legacy export into a runtime package (stage 2 of the 3D plan).
|
||||
"""Compile a legacy export into a runtime package (stages 2 and 5 of the 3D plan).
|
||||
|
||||
Offline only: this is the tool that bakes the coordinate transform, the unit
|
||||
scale, the UV sampling table and the identity hashes into one file, so the runtime
|
||||
never repeats that work.
|
||||
never repeats that work. Stage 5 adds the three release protections the design
|
||||
document asks for, and each is optional:
|
||||
|
||||
* ``--sign-key``: sign the package with an Ed25519 private key that stays on the
|
||||
build machine. The client carries only the matching public key.
|
||||
* ``--encrypt-key-id`` + ``--key-file``: additionally seal the content with
|
||||
AES-256-GCM. Encryption never replaces signing; it is added to it.
|
||||
* ``--lock-*`` (or ``--lock-from-probe``): record where the package may be
|
||||
displayed. A release compiled for a GPU must not silently render on llvmpipe.
|
||||
|
||||
Units are an argument, never a default. The legacy mesh is 15.8 x 15.0 x 7.5 source
|
||||
units; whether one of those is a millimetre is a physical claim about the sensor,
|
||||
and ``validate_scene.py`` prints the measured candidates so a person can decide.
|
||||
|
||||
Example::
|
||||
Examples::
|
||||
|
||||
# an engineering package: plain, unsigned, unlocked
|
||||
python tools/scene_assets/compile_scene.py \\
|
||||
--legacy-dir sensors/finger/assets/30fps-0703 \\
|
||||
--map sensors/finger/assets/30fps-0703/unroll_map_uv_s0.4.npz \\
|
||||
--meters-per-source-unit 0.001 \\
|
||||
--output build/finger-display-1.0.0.oryscene
|
||||
|
||||
# a release package: signed, sealed, locked to the machine we probed
|
||||
python tools/scene_assets/compile_scene.py \\
|
||||
--legacy-dir sensors/finger/assets/30fps-0703 \\
|
||||
--map sensors/finger/assets/30fps-0703/unroll_map_uv_s0.4.npz \\
|
||||
--meters-per-source-unit 0.001 \\
|
||||
--sign-key keys/build-ed25519.pem --sign-key-id build-2024 \\
|
||||
--encrypt-key-id finger-display --key-file keys/finger-display.key \\
|
||||
--lock-from-probe docs/architecture/gl-report-windows-dev.json \\
|
||||
--output build/finger-display-1.0.0.oryscene
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -33,6 +52,77 @@ def _bootstrap_paths() -> None:
|
||||
sys.path.insert(0, str(path))
|
||||
|
||||
|
||||
def _platform_lock_from_args(args):
|
||||
"""The lock the command line describes, from a probe report or from flags."""
|
||||
from orisys_app_shared.scene3d.contracts import PlatformLock
|
||||
|
||||
if args.lock_from_probe is not None:
|
||||
report = json.loads(Path(args.lock_from_probe).read_text(encoding="utf-8"))
|
||||
context = report.get("context") or {}
|
||||
host = report.get("host") or {}
|
||||
os_family = host.get("os_family") or host.get("system") or ""
|
||||
machine = host.get("machine") or ""
|
||||
return PlatformLock(
|
||||
os_family=(str(os_family).lower(),) if os_family else (),
|
||||
machine=(str(machine).lower(),) if machine else (),
|
||||
gl_profile=(
|
||||
str(context.get("profile_used")) if context.get("profile_used") else None
|
||||
),
|
||||
allowed_vendors=(str(context["vendor"]),) if context.get("vendor") else (),
|
||||
allowed_renderers=(str(context["renderer"]),) if context.get("renderer") else (),
|
||||
forbid_software_renderer=not args.allow_software_renderer,
|
||||
min_texture_size=int(context.get("max_texture_size") or 0),
|
||||
note=str(args.lock_note or f"qualified from {Path(args.lock_from_probe).name}"),
|
||||
)
|
||||
lock = PlatformLock(
|
||||
os_family=tuple(v.lower() for v in (args.lock_os or ())),
|
||||
machine=tuple(v.lower() for v in (args.lock_machine or ())),
|
||||
gl_profile=args.lock_gl_profile,
|
||||
allowed_vendors=tuple(args.lock_vendor or ()),
|
||||
allowed_renderers=tuple(args.lock_renderer or ()),
|
||||
forbid_software_renderer=not args.allow_software_renderer,
|
||||
min_texture_size=int(args.lock_min_texture or 0),
|
||||
note=str(args.lock_note or ""),
|
||||
)
|
||||
return None if lock.is_unrestricted() and not args.lock_note else lock
|
||||
|
||||
|
||||
def _sealing_from_args(args):
|
||||
"""The key material the command line points at, or ``None`` for a plain package."""
|
||||
from orisys_app_shared.scene3d.contracts import Sealing
|
||||
from orisys_app_shared.scene3d import protect
|
||||
|
||||
signing_pem = None
|
||||
if args.sign_key is not None:
|
||||
path = Path(args.sign_key)
|
||||
if not path.is_file():
|
||||
raise SystemExit(f"the signing key is missing: {path}")
|
||||
signing_pem = path.read_bytes()
|
||||
# Prove the key is usable before spending a compile on it.
|
||||
protect.public_key_pem(signing_pem)
|
||||
provider = None
|
||||
if args.encrypt_key_id:
|
||||
if args.key_file is None:
|
||||
raise SystemExit("--encrypt-key-id also needs --key-file")
|
||||
provider = protect.load_key_file(Path(args.key_file), key_id=args.encrypt_key_id)
|
||||
if signing_pem is None and provider is None:
|
||||
return None
|
||||
verify = protect.public_key_pem(signing_pem) if signing_pem is not None else None
|
||||
if verify is not None and args.write_verify_key is not None:
|
||||
# The client's half of the pair. Publishing it is part of the release, and
|
||||
# writing it here keeps it from being re-derived differently somewhere else.
|
||||
target = Path(args.write_verify_key)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(verify)
|
||||
return Sealing(
|
||||
key_id=str(args.encrypt_key_id or ""),
|
||||
provider=provider,
|
||||
signing_key_pem=signing_pem,
|
||||
signing_key_reference=str(args.sign_key_id or "build-key"),
|
||||
verify_key_pem=verify,
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--legacy-dir", type=Path, required=True, help="legacy NPZ export")
|
||||
@@ -54,6 +144,37 @@ def main(argv: list[str] | None = None) -> int:
|
||||
default=None,
|
||||
help="optional 4x4 rigid transform (npy) from source to sensor coordinates",
|
||||
)
|
||||
protection = parser.add_argument_group(
|
||||
"release protection (stage 5)", "all optional; a plain package is the default"
|
||||
)
|
||||
protection.add_argument("--sign-key", type=Path, default=None, help="Ed25519 private key (PEM)")
|
||||
protection.add_argument("--sign-key-id", default="", help="name recorded in the envelope")
|
||||
protection.add_argument(
|
||||
"--write-verify-key",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="write the public half of --sign-key here (this is what the client ships)",
|
||||
)
|
||||
protection.add_argument("--encrypt-key-id", default="", help="content key id to seal with")
|
||||
protection.add_argument("--key-file", type=Path, default=None, help="key for --encrypt-key-id")
|
||||
protection.add_argument(
|
||||
"--lock-from-probe",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="take the platform lock from a probe_gl.py report",
|
||||
)
|
||||
protection.add_argument("--lock-os", action="append", default=None, help="repeatable")
|
||||
protection.add_argument("--lock-machine", action="append", default=None, help="repeatable")
|
||||
protection.add_argument("--lock-gl-profile", default=None, choices=(None, "desktop33", "gles30"))
|
||||
protection.add_argument("--lock-vendor", action="append", default=None, help="repeatable")
|
||||
protection.add_argument("--lock-renderer", action="append", default=None, help="repeatable")
|
||||
protection.add_argument("--lock-min-texture", type=int, default=0)
|
||||
protection.add_argument("--lock-note", default="")
|
||||
protection.add_argument(
|
||||
"--allow-software-renderer",
|
||||
action="store_true",
|
||||
help="do not forbid a software rasteriser (the design document says do)",
|
||||
)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
@@ -62,13 +183,19 @@ def main(argv: list[str] | None = None) -> int:
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import ThreeDError
|
||||
from orisys_app_shared.scene3d.legacy import LegacyFingerImporter
|
||||
from orisys_app_shared.scene3d.package import write_package
|
||||
from orisys_app_shared.scene3d.package import describe_envelope, write_package
|
||||
|
||||
transform = (
|
||||
np.load(args.transform, allow_pickle=False)
|
||||
if args.transform is not None
|
||||
else np.eye(4)
|
||||
)
|
||||
try:
|
||||
sealing = _sealing_from_args(args)
|
||||
lock = _platform_lock_from_args(args)
|
||||
except ThreeDError as exc:
|
||||
print(f"rejected: {exc.diagnostic.code}: {exc.diagnostic.message}", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
assets = LegacyFingerImporter().convert(
|
||||
args.legacy_dir,
|
||||
@@ -79,14 +206,21 @@ def main(argv: list[str] | None = None) -> int:
|
||||
asset_version=args.version,
|
||||
lod_id=args.lod_id,
|
||||
)
|
||||
if lock is not None:
|
||||
from dataclasses import replace
|
||||
|
||||
# Part of the asset, so it travels into the manifest and the signature.
|
||||
assets = replace(assets, platform_lock=lock)
|
||||
report = write_package(
|
||||
assets, args.output,
|
||||
sensor_model=args.sensor_model,
|
||||
hardware_revisions=("*",),
|
||||
sealing=sealing,
|
||||
)
|
||||
except ThreeDError as exc:
|
||||
print(f"rejected: {exc.diagnostic.code}: {exc.diagnostic.message}", file=sys.stderr)
|
||||
return 2
|
||||
envelope = describe_envelope(report.output)
|
||||
summary = {
|
||||
"status": "ok",
|
||||
"output": str(report.output),
|
||||
@@ -95,6 +229,10 @@ def main(argv: list[str] | None = None) -> int:
|
||||
"triangles": report.part_triangle_counts,
|
||||
"geometry_id": assets.geometry_id,
|
||||
"field_domain_id": assets.field_domain_id,
|
||||
"envelope": envelope,
|
||||
"platform_lock": (
|
||||
assets.platform_lock.to_dict() if assets.platform_lock is not None else None
|
||||
),
|
||||
}
|
||||
if args.json:
|
||||
print(json.dumps(summary, indent=2, ensure_ascii=False))
|
||||
@@ -105,6 +243,13 @@ def main(argv: list[str] | None = None) -> int:
|
||||
)
|
||||
print(f" geometry_id={assets.geometry_id}")
|
||||
print(f" asset_hash={report.asset_hash}")
|
||||
print(
|
||||
f" envelope=v{envelope['envelope_version']} {envelope['payload']}"
|
||||
f"{' signed(' + str(envelope['signing_key']) + ')' if envelope['signed'] else ' unsigned'}"
|
||||
+ (f" sealed for key {envelope['key_id']}" if envelope["sealed"] else "")
|
||||
)
|
||||
if summary["platform_lock"]:
|
||||
print(f" platform_lock={json.dumps(summary['platform_lock'], ensure_ascii=False)}")
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
@@ -64,6 +64,14 @@ def collect(profile: str, *, keep_open_s: float) -> dict:
|
||||
"python": sys.version.split()[0],
|
||||
"os": f"{platform.system()} {platform.release()}",
|
||||
},
|
||||
# The facts a platform lock is built from (`compile_scene.py
|
||||
# --lock-from-probe`), named as `PlatformInfo` names them so the two cannot
|
||||
# drift: changing one of these keys changes what a release can be locked to.
|
||||
"host": {
|
||||
"os_family": platform.system().lower(),
|
||||
"machine": (platform.machine() or "unknown").lower(),
|
||||
"os": f"{platform.system()} {platform.release()}",
|
||||
},
|
||||
}
|
||||
try:
|
||||
from OpenGL import __version__ as pyopengl_version
|
||||
|
||||
@@ -0,0 +1,560 @@
|
||||
"""Long-run check: does the 3D pipeline leak, and is the queue bounded?
|
||||
|
||||
Stage 5 deliverable (``docs/architecture/studio-3d-stages.md``). The design
|
||||
document asks for a two-hour run of acquisition + recording + 3D with a report of the
|
||||
maximum RSS and the queue bound, and for memory that does not *keep* growing.
|
||||
|
||||
Two things make this more than "run it and look at Task Manager":
|
||||
|
||||
* **the pipeline is the real one.** Frames are produced on a worker thread, adapted by
|
||||
the same adapter the acquisition thread uses, offered to the same capacity-one
|
||||
mailbox, composed on the same executor and dropped by the same counters. What is
|
||||
synthetic is the *sensor*: a monotonic ramp instead of a camera, because a soak test
|
||||
that needs hardware cannot run in CI or overnight.
|
||||
* **the question is the trend, not the number.** A process that allocates 200 MB in
|
||||
its first minute and then holds steady is healthy; one that grows 1 MB a minute for
|
||||
two hours is not. So the report keeps every sample and compares the *first* and
|
||||
*last* quarter means, which is why the verdict can be a fact rather than an
|
||||
impression.
|
||||
|
||||
Run it for the release gate's two hours, or for a few minutes as a smoke test::
|
||||
|
||||
python tools/scene_assets/soak_scene.py --minutes 120 \\
|
||||
--output docs/architecture/soak-report-windows-dev.json
|
||||
python tools/scene_assets/soak_scene.py --minutes 3 --fps 30 --output short.json
|
||||
|
||||
Exit codes: 0 = no growth and the queue stayed bounded, 2 = a check failed,
|
||||
1 = the arguments were wrong.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ctypes
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _bootstrap_paths() -> None:
|
||||
here = Path(__file__).resolve()
|
||||
repo = here.parents[2]
|
||||
for path in (repo / "apps" / "studio" / "src", repo / "shared" / "src"):
|
||||
if path.is_dir() and str(path) not in sys.path:
|
||||
sys.path.insert(0, str(path))
|
||||
|
||||
|
||||
def resident_bytes() -> int:
|
||||
"""This process's resident set, without a third-party dependency.
|
||||
|
||||
Windows reports it through ``GetProcessMemoryInfo``; Linux through ``/proc``. A
|
||||
missing measurement returns 0 rather than raising: a soak run that cannot read its
|
||||
own memory is still a soak run, and the report says which was available.
|
||||
"""
|
||||
if sys.platform == "win32":
|
||||
class PROCESS_MEMORY_COUNTERS(ctypes.Structure):
|
||||
_fields_ = [
|
||||
("cb", ctypes.c_ulong),
|
||||
("PageFaultCount", ctypes.c_ulong),
|
||||
("PeakWorkingSetSize", ctypes.c_size_t),
|
||||
("WorkingSetSize", ctypes.c_size_t),
|
||||
("QuotaPeakPagedPoolUsage", ctypes.c_size_t),
|
||||
("QuotaPagedPoolUsage", ctypes.c_size_t),
|
||||
("QuotaPeakNonPagedPoolUsage", ctypes.c_size_t),
|
||||
("QuotaNonPagedPoolUsage", ctypes.c_size_t),
|
||||
("PagefileUsage", ctypes.c_size_t),
|
||||
("PeakPagefileUsage", ctypes.c_size_t),
|
||||
]
|
||||
|
||||
counters = PROCESS_MEMORY_COUNTERS()
|
||||
counters.cb = ctypes.sizeof(counters)
|
||||
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
psapi = ctypes.WinDLL("psapi", use_last_error=True)
|
||||
# Both signatures have to be declared: the current-process handle is a
|
||||
# pseudo-handle (-1), and letting ctypes guess a 32-bit return type turns it
|
||||
# into an invalid handle whose call fails silently — which is how a memory
|
||||
# report ends up full of zeros that look like a healthy flat line.
|
||||
kernel32.GetCurrentProcess.restype = ctypes.c_void_p
|
||||
psapi.GetProcessMemoryInfo.argtypes = [
|
||||
ctypes.c_void_p,
|
||||
ctypes.POINTER(PROCESS_MEMORY_COUNTERS),
|
||||
ctypes.c_ulong,
|
||||
]
|
||||
psapi.GetProcessMemoryInfo.restype = ctypes.c_bool
|
||||
handle = kernel32.GetCurrentProcess()
|
||||
if psapi.GetProcessMemoryInfo(handle, ctypes.byref(counters), counters.cb):
|
||||
return int(counters.WorkingSetSize)
|
||||
return 0
|
||||
try:
|
||||
with open("/proc/self/statm", encoding="ascii") as handle:
|
||||
pages = int(handle.read().split()[1])
|
||||
return pages * 4096
|
||||
except Exception: # noqa: BLE001
|
||||
return 0
|
||||
|
||||
|
||||
def _quarter_means(samples: list[float]) -> tuple[float, float]:
|
||||
if len(samples) < 8:
|
||||
return (samples[0] if samples else 0.0, samples[-1] if samples else 0.0)
|
||||
quarter = max(1, len(samples) // 4)
|
||||
first = sum(samples[:quarter]) / quarter
|
||||
last = sum(samples[-quarter:]) / quarter
|
||||
return first, last
|
||||
|
||||
|
||||
def run(
|
||||
*,
|
||||
minutes: float,
|
||||
fps: float,
|
||||
profile: str,
|
||||
use_gui: bool,
|
||||
mode: str,
|
||||
growth_limit_mb: float,
|
||||
late_rate_limit_mb_per_min: float,
|
||||
) -> tuple[dict, int]:
|
||||
"""Produce frames through the real pipeline and watch what it costs over time."""
|
||||
import numpy as np
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.composer import SceneComposer
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
DisplaySettings,
|
||||
FrameIdentity,
|
||||
validate_assets,
|
||||
)
|
||||
|
||||
report: dict = {
|
||||
"kind": "orisys-scene-soak-report",
|
||||
"minutes_requested": float(minutes),
|
||||
"fps_requested": float(fps),
|
||||
"profile": profile,
|
||||
"window": bool(use_gui),
|
||||
"adapter": "finger",
|
||||
"scene": mode,
|
||||
"rss_available": resident_bytes() > 0,
|
||||
}
|
||||
app = None
|
||||
view = None
|
||||
widget = None
|
||||
if use_gui:
|
||||
from PyQt5.QtWidgets import QApplication
|
||||
|
||||
from orisys_studio.scene3d.gl_backend import configure_surface_format
|
||||
from orisys_studio.scene3d.view import Scene3DView
|
||||
|
||||
configure_surface_format(profile) # type: ignore[arg-type]
|
||||
app = QApplication.instance() or QApplication(sys.argv)
|
||||
view = Scene3DView()
|
||||
if not view.available:
|
||||
report["verdict"] = "no_view"
|
||||
report["reason"] = view.reason
|
||||
return report, 2
|
||||
assets = synthetic.poc_scene()
|
||||
if not view.use_scene(assets, source_id="soak"):
|
||||
report["verdict"] = "asset_rejected"
|
||||
report["reason"] = str(view.last_diagnostic)
|
||||
return report, 2
|
||||
view.resize(640, 480)
|
||||
view.show()
|
||||
view.set_enabled(True)
|
||||
widget = view.gl_widget
|
||||
deadline = time.perf_counter() + 10.0
|
||||
while time.perf_counter() < deadline and not widget._renderer.part_ids:
|
||||
widget.repaint()
|
||||
app.processEvents()
|
||||
time.sleep(0.005)
|
||||
report["context"] = {
|
||||
"renderer": getattr(widget.capabilities(), "renderer", ""),
|
||||
"software_renderer": bool(
|
||||
getattr(widget.capabilities(), "software_renderer", False)
|
||||
),
|
||||
}
|
||||
else:
|
||||
from orisys_studio.scene3d.adapters import FingerFrameAdapter
|
||||
|
||||
assets = synthetic.poc_scene()
|
||||
validate_assets(assets)
|
||||
report["mesh"] = {
|
||||
"sensing_vertices": int(assets.sensing_part().positions_sensor_m.shape[0]),
|
||||
"field_domain_hw": [int(assets.shape_hw[0]), int(assets.shape_hw[1])],
|
||||
}
|
||||
|
||||
# The producer is its own thread, exactly like acquisition: the mailbox has to
|
||||
# absorb a producer that does not care what the display is doing.
|
||||
shape = tuple(assets.shape_hw)
|
||||
ramp = np.linspace(0.0, 1.0, int(shape[0]) * int(shape[1]), dtype=np.float32)
|
||||
flow = np.stack([ramp, ramp], axis=-1).reshape(int(shape[0]), int(shape[1]), 2)
|
||||
|
||||
class Packet:
|
||||
def __init__(self, index: int) -> None:
|
||||
self.session_id = 1
|
||||
self.frame_id = int(index)
|
||||
self.mono_t = float(index) / 30.0
|
||||
self.flow = flow
|
||||
self.raw_frame = np.zeros((4, 4, 3), dtype=np.uint8)
|
||||
|
||||
executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="soak")
|
||||
stop = threading.Event()
|
||||
produced = 0
|
||||
add_produce, produce_samples, produce_seen = _bounded_collector()
|
||||
produce_lock = threading.Lock()
|
||||
|
||||
if not use_gui:
|
||||
# Without a window, drive the same adapter + mailbox + compose chain by hand so
|
||||
# the soak still covers the queue bound (the GUI path is covered by `use_gui`).
|
||||
from orisys_app_shared.scene3d.contracts import LatestFrameMailbox
|
||||
|
||||
adapter = FingerFrameAdapter(
|
||||
geometry_id=assets.geometry_id,
|
||||
field_domain_id=assets.field_domain_id,
|
||||
shape_hw=shape,
|
||||
)
|
||||
mailbox: LatestFrameMailbox = LatestFrameMailbox()
|
||||
composer = SceneComposer()
|
||||
settings = DisplaySettings()
|
||||
composed = {"frames": 0}
|
||||
|
||||
def work(frame) -> None:
|
||||
composer.compose(assets, frame, settings)
|
||||
with produce_lock:
|
||||
composed["frames"] += 1
|
||||
|
||||
def producer() -> None:
|
||||
nonlocal produced
|
||||
period = 1.0 / max(fps, 1.0)
|
||||
next_at = time.perf_counter()
|
||||
while not stop.is_set():
|
||||
started = time.perf_counter()
|
||||
packet = Packet(produced)
|
||||
identity = FrameIdentity(
|
||||
source_id="soak", session_id=1, frame_id=produced,
|
||||
t_rel_s=packet.mono_t, generation=1,
|
||||
)
|
||||
frame = adapter.snapshot(packet, None, identity)
|
||||
if frame is not None:
|
||||
mailbox.offer(frame)
|
||||
produced += 1
|
||||
with produce_lock:
|
||||
add_produce((time.perf_counter() - started) * 1000.0)
|
||||
next_at += period
|
||||
delay = next_at - time.perf_counter()
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
else:
|
||||
next_at = time.perf_counter()
|
||||
|
||||
thread = threading.Thread(target=producer, name="soak-producer", daemon=True)
|
||||
thread.start()
|
||||
samples = _sample_loop(
|
||||
stop=stop,
|
||||
executor=executor,
|
||||
mailbox=mailbox,
|
||||
work=work,
|
||||
produced=lambda: produced,
|
||||
report=report,
|
||||
minutes=minutes,
|
||||
)
|
||||
stop.set()
|
||||
thread.join(timeout=5.0)
|
||||
executor.shutdown(wait=False)
|
||||
inbox = mailbox
|
||||
else:
|
||||
def producer() -> None:
|
||||
nonlocal produced
|
||||
period = 1.0 / max(fps, 1.0)
|
||||
next_at = time.perf_counter()
|
||||
while not stop.is_set():
|
||||
started = time.perf_counter()
|
||||
view.submit_packet(Packet(produced))
|
||||
produced += 1
|
||||
with produce_lock:
|
||||
add_produce((time.perf_counter() - started) * 1000.0)
|
||||
next_at += period
|
||||
delay = next_at - time.perf_counter()
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
else:
|
||||
next_at = time.perf_counter()
|
||||
|
||||
thread = threading.Thread(target=producer, name="soak-producer", daemon=True)
|
||||
thread.start()
|
||||
samples = _sample_gui_loop(
|
||||
stop=stop, view=view, app=app, widget=widget, report=report, minutes=minutes
|
||||
)
|
||||
stop.set()
|
||||
thread.join(timeout=5.0)
|
||||
view.close()
|
||||
app.processEvents()
|
||||
inbox = view.coordinator._incoming if view is not None else None
|
||||
|
||||
submitted = produce_samples()
|
||||
report["frames"] = {
|
||||
"produced": int(produced),
|
||||
# ``sampled`` says how many timings the percentiles were computed from: the
|
||||
# collector decimates instead of growing, and a report has to be explicit about
|
||||
# which of the two numbers it is quoting.
|
||||
"producer_ms": _percentiles(submitted),
|
||||
"producer_ms_sampled": len(submitted),
|
||||
"producer_ms_total": int(produce_seen()),
|
||||
}
|
||||
if inbox is not None:
|
||||
report["queue"] = {
|
||||
"mailbox_capacity": 1,
|
||||
"superseded_by_mailbox": int(getattr(inbox, "superseded_frames", 0)),
|
||||
"max_inflight_composes": 1,
|
||||
}
|
||||
if view is not None and view.coordinator is not None:
|
||||
report["queue"].update(
|
||||
{
|
||||
"superseded_display_frames": int(view.coordinator.superseded_frames),
|
||||
"stale_frames": int(view.coordinator.stale_frames),
|
||||
"invalid_frames": int(view.coordinator.invalid_frames),
|
||||
}
|
||||
)
|
||||
report["rss_mb"] = _rss_summary(samples, seconds_per_sample=1.0)
|
||||
report["late_rate_limit_mb_per_min"] = float(late_rate_limit_mb_per_min)
|
||||
ok, reason = _judge(report, growth_limit_mb)
|
||||
report["verdict"] = "ok" if ok else "failed"
|
||||
if not ok:
|
||||
report["problems"] = [reason]
|
||||
return report, (0 if ok else 2)
|
||||
|
||||
|
||||
def _sample_loop(*, stop, executor, mailbox, work, produced, report, minutes):
|
||||
"""CPU mode: tick the mailbox, compose in the background, sample RSS."""
|
||||
from concurrent.futures import Future
|
||||
|
||||
samples: list[float] = []
|
||||
inflight: list[Future] = []
|
||||
deadline = time.perf_counter() + minutes * 60.0
|
||||
next_sample = time.perf_counter()
|
||||
composed = 0
|
||||
while time.perf_counter() < deadline:
|
||||
frame = mailbox.take()
|
||||
if frame is not None and len(inflight) < 1:
|
||||
inflight.append(executor.submit(work, frame))
|
||||
inflight = [item for item in inflight if not item.done()]
|
||||
composed += 1
|
||||
now = time.perf_counter()
|
||||
if now >= next_sample:
|
||||
samples.append(resident_bytes() / (1024 * 1024))
|
||||
next_sample = now + 1.0
|
||||
time.sleep(0.002)
|
||||
stop.set()
|
||||
samples.append(resident_bytes() / (1024 * 1024))
|
||||
report["ticks"] = composed
|
||||
return samples
|
||||
|
||||
|
||||
def _sample_gui_loop(*, stop, view, app, widget, report, minutes):
|
||||
samples: list[float] = []
|
||||
deadline = time.perf_counter() + minutes * 60.0
|
||||
next_sample = time.perf_counter()
|
||||
ticks = 0
|
||||
while time.perf_counter() < deadline:
|
||||
view.tick()
|
||||
app.processEvents()
|
||||
ticks += 1
|
||||
now = time.perf_counter()
|
||||
if now >= next_sample:
|
||||
samples.append(resident_bytes() / (1024 * 1024))
|
||||
next_sample = now + 1.0
|
||||
time.sleep(0.004)
|
||||
stop.set()
|
||||
samples.append(resident_bytes() / (1024 * 1024))
|
||||
report["ticks"] = ticks
|
||||
return samples
|
||||
|
||||
|
||||
def _bounded_collector(capacity: int = 20000):
|
||||
"""A percentile sample that cannot itself grow with the run.
|
||||
|
||||
The first version appended every frame's timing to a list, which over a two-hour
|
||||
run is 400k floats — a soak test whose own bookkeeping grows linearly reports a
|
||||
leak that is its own. Samples are taken at a fixed stride once ``capacity`` is
|
||||
reached, so the memory the measurement costs is bounded by construction.
|
||||
"""
|
||||
values: list[float] = []
|
||||
seen = 0
|
||||
|
||||
def add(value: float) -> None:
|
||||
nonlocal seen
|
||||
seen += 1
|
||||
if len(values) < capacity:
|
||||
values.append(value)
|
||||
return
|
||||
# Decimate in place: drop every other retained sample and double the stride.
|
||||
del values[1::2]
|
||||
values.append(value)
|
||||
|
||||
return add, lambda: values, lambda: seen
|
||||
|
||||
|
||||
def _percentiles(values: list[float]) -> dict[str, float]:
|
||||
if not values:
|
||||
return {"p50": 0.0, "p95": 0.0, "max": 0.0, "n": 0}
|
||||
ordered = sorted(values)
|
||||
index = max(0, min(len(ordered) - 1, int(round(0.95 * len(ordered))) - 1))
|
||||
return {
|
||||
"p50": round(ordered[len(ordered) // 2], 4),
|
||||
"p95": round(ordered[index], 4),
|
||||
"max": round(ordered[-1], 4),
|
||||
"n": len(ordered),
|
||||
}
|
||||
|
||||
|
||||
def _rss_summary(samples: list[float], *, seconds_per_sample: float = 1.0) -> dict:
|
||||
if not samples or max(samples) <= 0.0:
|
||||
# All zeros means the platform call failed, not that the process uses no
|
||||
# memory: reporting a flat line at 0 MB would be a false clean bill of health.
|
||||
return {"available": False}
|
||||
first, last = _quarter_means(samples)
|
||||
quarter = max(1, len(samples) // 4)
|
||||
# The rate that matters is the one *after* the process has stopped warming up: a
|
||||
# run that allocates 5 MB in its first seconds and then holds is healthy, and
|
||||
# comparing the first minute with the last would call it a leak. The warm-up is
|
||||
# the first tenth of the run (at least a minute), and the late rate is measured
|
||||
# over its last quarter.
|
||||
warmup = max(1, min(len(samples) // 4, max(int(60 / seconds_per_sample), len(samples) // 10)))
|
||||
late_window = samples[-quarter:]
|
||||
late_rate = (
|
||||
(late_window[-1] - late_window[0]) / max(len(late_window) * seconds_per_sample / 60.0, 1e-9)
|
||||
)
|
||||
# The rate of each quarter, because the question a soak test really answers is
|
||||
# "is it still climbing when the run ends": a rate that falls quarter by quarter is
|
||||
# an allocator settling; the same rate in every quarter is a leak.
|
||||
quarter_rates = []
|
||||
for index in range(4):
|
||||
start = index * quarter
|
||||
end = min(len(samples), (index + 1) * quarter)
|
||||
if end - start < 2:
|
||||
continue
|
||||
minutes = (end - start) * seconds_per_sample / 60.0
|
||||
quarter_rates.append(round((samples[end - 1] - samples[start]) / max(minutes, 1e-9), 4))
|
||||
return {
|
||||
"available": True,
|
||||
"samples": len(samples),
|
||||
"seconds_per_sample": round(float(seconds_per_sample), 3),
|
||||
"max_mb": round(max(samples), 2),
|
||||
"min_mb": round(min(samples), 2),
|
||||
"first_quarter_mean_mb": round(first, 2),
|
||||
"last_quarter_mean_mb": round(last, 2),
|
||||
"growth_mb": round(last - first, 2),
|
||||
"after_warmup_growth_mb": round(samples[-1] - samples[warmup - 1], 2),
|
||||
"late_rate_mb_per_min": round(late_rate, 4),
|
||||
"quarter_rates_mb_per_min": quarter_rates,
|
||||
"series_mb": [round(value, 2) for value in samples],
|
||||
}
|
||||
|
||||
|
||||
def _judge(report: dict, growth_limit_mb: float) -> tuple[bool, str]:
|
||||
"""The verdict: bounded growth after warm-up, and a queue that cannot grow at all.
|
||||
|
||||
Two memory questions, both asked because either alone can be fooled:
|
||||
|
||||
* absolute growth from the end of the warm-up to the end of the run, so a slow
|
||||
climb is caught even when the per-minute rate is small;
|
||||
* the *rate over the last quarter*, so a run that is still climbing when it stops
|
||||
is not called flat just because the total is under the limit.
|
||||
"""
|
||||
rss = report.get("rss_mb") or {}
|
||||
if not rss.get("available"):
|
||||
return False, "resident memory could not be read on this platform"
|
||||
growth = float(rss.get("growth_mb", 0.0))
|
||||
if growth > growth_limit_mb:
|
||||
return False, (
|
||||
f"resident memory grew {growth:.1f} MB between the first and last quarter "
|
||||
f"(limit {growth_limit_mb:.1f} MB): the pipeline is holding on to frames"
|
||||
)
|
||||
rate = float(rss.get("late_rate_mb_per_min", 0.0))
|
||||
rate_limit = float(report.get("late_rate_limit_mb_per_min", 0.5))
|
||||
if rate > rate_limit:
|
||||
return False, (
|
||||
f"resident memory is still climbing at {rate:.3f} MB/min in the last "
|
||||
f"quarter (limit {rate_limit:.3f}): this is continuous growth, not "
|
||||
"allocator noise"
|
||||
)
|
||||
queue = report.get("queue") or {}
|
||||
if int(queue.get("mailbox_capacity", 0)) != 1:
|
||||
return False, "the mailbox is not capacity one: the queue can grow"
|
||||
if int(queue.get("max_inflight_composes", 0)) > 1:
|
||||
return False, "more than one compose may be in flight: the queue can grow"
|
||||
return True, ""
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--minutes", type=float, default=120.0, help="run length")
|
||||
parser.add_argument("--fps", type=float, default=30.0, help="producer rate")
|
||||
parser.add_argument("--profile", default="desktop33", choices=("desktop33", "gles30"))
|
||||
parser.add_argument(
|
||||
"--no-window",
|
||||
action="store_true",
|
||||
help="drive the adapter/mailbox/compose chain without a GL window (runs anywhere)",
|
||||
)
|
||||
parser.add_argument("--scene", default="synthetic-poc")
|
||||
parser.add_argument(
|
||||
"--growth-limit-mb",
|
||||
type=float,
|
||||
default=64.0,
|
||||
help="allowed growth between the first and last quarter (allocator noise)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--late-rate-limit",
|
||||
type=float,
|
||||
default=1.0,
|
||||
help=(
|
||||
"allowed MB/min in the last quarter: a run still climbing when it stops is "
|
||||
"not flat, however small the total. The default is set from measurement, not "
|
||||
"from hope: on the developer machine this pipeline shows about 0.35 MB/min "
|
||||
"in its tail after warm-up (see docs/architecture/soak-report-windows-dev."
|
||||
"json), which the stage-by-stage probe attributes to allocator/page "
|
||||
"behaviour rather than to retained objects. Tighten it on a platform that "
|
||||
"is quieter."
|
||||
),
|
||||
)
|
||||
parser.add_argument("--output", type=Path, default=None)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
_bootstrap_paths()
|
||||
report, code = run(
|
||||
minutes=args.minutes,
|
||||
fps=args.fps,
|
||||
profile=args.profile,
|
||||
use_gui=not args.no_window,
|
||||
mode=args.scene,
|
||||
growth_limit_mb=args.growth_limit_mb,
|
||||
late_rate_limit_mb_per_min=args.late_rate_limit,
|
||||
)
|
||||
report["exit_code"] = int(code)
|
||||
text = json.dumps(report, indent=2, ensure_ascii=False, default=str)
|
||||
if args.output is not None:
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_text(text + "\n", encoding="utf-8")
|
||||
if args.json or args.output is None:
|
||||
print(text)
|
||||
else:
|
||||
rss = report.get("rss_mb", {})
|
||||
print(
|
||||
f"verdict={report['verdict']} minutes={report['minutes_requested']} "
|
||||
f"frames={report['frames']['produced']} ticks={report.get('ticks')}"
|
||||
)
|
||||
print(
|
||||
f" rss first={rss.get('first_quarter_mean_mb')} MB "
|
||||
f"last={rss.get('last_quarter_mean_mb')} MB max={rss.get('max_mb')} MB "
|
||||
f"growth={rss.get('growth_mb')} MB"
|
||||
)
|
||||
print(f" queue {json.dumps(report.get('queue', {}), ensure_ascii=False)}")
|
||||
if report.get("problems"):
|
||||
print(f" problems: {report['problems']}")
|
||||
print(f" wrote {args.output}")
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -143,9 +143,26 @@ def _legacy_report(args) -> tuple[int, dict]:
|
||||
return (0 if report["status"] == "ok" else 2), report
|
||||
|
||||
|
||||
def _load_keys(args):
|
||||
"""The key provider and verification key the arguments describe, if any."""
|
||||
from orisys_app_shared.scene3d import protect
|
||||
|
||||
provider = None
|
||||
if args.key_file is not None:
|
||||
provider = protect.load_key_file(Path(args.key_file), key_id=args.key_id)
|
||||
verify = None
|
||||
if args.verify_key is not None:
|
||||
path = Path(args.verify_key)
|
||||
if not path.is_file():
|
||||
raise SystemExit(f"the verification key is missing: {path}")
|
||||
verify = path.read_bytes()
|
||||
return provider, verify
|
||||
|
||||
|
||||
def _package_report(args) -> tuple[int, dict]:
|
||||
from orisys_app_shared.scene3d.contracts import ExpectedGeometry, ThreeDError
|
||||
from orisys_app_shared.scene3d.package import (
|
||||
describe_envelope,
|
||||
describe_package,
|
||||
read_package,
|
||||
scene_from_contents,
|
||||
@@ -157,14 +174,64 @@ def _package_report(args) -> tuple[int, dict]:
|
||||
report["checks"].append({"check": name, "ok": bool(ok), "detail": detail})
|
||||
|
||||
try:
|
||||
contents = read_package(Path(args.asset))
|
||||
record("envelope, manifest and every entry hash", True)
|
||||
except ThreeDError as exc:
|
||||
record("envelope, manifest and every entry hash", False,
|
||||
f"{exc.diagnostic.code}: {exc.diagnostic.message}")
|
||||
provider, verify = _load_keys(args)
|
||||
except SystemExit as exc:
|
||||
record("key material", False, str(exc))
|
||||
report["status"] = "rejected"
|
||||
return 2, report
|
||||
report["manifest"] = describe_package(Path(args.asset))
|
||||
|
||||
# What the container says about itself, *without* a key: a support engineer can
|
||||
# report "sealed for key id X, signed by Y" without being able to read it.
|
||||
try:
|
||||
report["envelope"] = describe_envelope(Path(args.asset))
|
||||
record(
|
||||
"envelope reads",
|
||||
True,
|
||||
f"v{report['envelope'].get('envelope_version')} "
|
||||
f"{report['envelope'].get('payload')}"
|
||||
f"{' signed' if report['envelope'].get('signed') else ' unsigned'}",
|
||||
)
|
||||
except ThreeDError as exc:
|
||||
record("envelope reads", False, f"{exc.diagnostic.code}: {exc.diagnostic.message}")
|
||||
report["status"] = "rejected"
|
||||
return 2, report
|
||||
if report["envelope"].get("signed"):
|
||||
record("package is signed", True, str(report["envelope"].get("signing_key") or ""))
|
||||
else:
|
||||
# A warning by default, a hard failure when the caller says this is a release:
|
||||
# an engineering package is legitimately plain, and a release package that is
|
||||
# plain has no origin, which is the design document's step 2.
|
||||
record(
|
||||
"package is signed",
|
||||
not args.require_signature,
|
||||
"unsigned: integrity only, no origin (use --require-signature for a release "
|
||||
"build) "
|
||||
if not args.require_signature
|
||||
else "unsigned: a release package must be signed",
|
||||
)
|
||||
|
||||
try:
|
||||
contents = read_package(Path(args.asset), keys=provider, verify_key_pem=verify)
|
||||
record("signature, envelope, manifest and every entry hash", True)
|
||||
except ThreeDError as exc:
|
||||
record(
|
||||
"signature, envelope, manifest and every entry hash",
|
||||
False,
|
||||
f"{exc.diagnostic.code}: {exc.diagnostic.message}",
|
||||
)
|
||||
report["status"] = "rejected"
|
||||
return 2, report
|
||||
report["manifest"] = describe_package(
|
||||
Path(args.asset), keys=provider, verify_key_pem=verify
|
||||
)
|
||||
lock = report["manifest"].get("platform_lock")
|
||||
if lock:
|
||||
record(
|
||||
"platform lock of this package",
|
||||
True,
|
||||
f"{lock.get('os_family') or 'any OS'} / {lock.get('machine') or 'any arch'}"
|
||||
f"{' / ' + str(lock.get('gl_profile')) if lock.get('gl_profile') else ''}",
|
||||
)
|
||||
try:
|
||||
expected = (
|
||||
ExpectedGeometry(
|
||||
@@ -199,6 +266,24 @@ def main(argv: list[str] | None = None) -> int:
|
||||
)
|
||||
parser.add_argument("--sensor-model", default="finger")
|
||||
parser.add_argument("--hardware-revision", default="*")
|
||||
parser.add_argument(
|
||||
"--key-file",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="content key for a sealed package (hex, base64 or raw)",
|
||||
)
|
||||
parser.add_argument("--key-id", default="", help="key id that --key-file holds")
|
||||
parser.add_argument(
|
||||
"--verify-key",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Ed25519 public key (PEM) a signed package must verify against",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--require-signature",
|
||||
action="store_true",
|
||||
help="treat an unsigned package as a failure: the release gate",
|
||||
)
|
||||
parser.add_argument("--json", action="store_true", help="machine-readable output")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
@@ -221,7 +306,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
for item in report.get("checks", []):
|
||||
mark = "ok " if item["ok"] else "FAIL"
|
||||
print(f" {mark} {item['check']} {item['detail']}")
|
||||
for key in ("field_domain", "mesh", "uv_correspondence", "scale", "binding"):
|
||||
for key in ("envelope", "field_domain", "mesh", "uv_correspondence", "scale", "binding"):
|
||||
if key in report:
|
||||
print(f" {key}: {json.dumps(report[key], ensure_ascii=False, default=str)}")
|
||||
if "note" in report:
|
||||
|
||||
Reference in New Issue
Block a user