阶段 3：实时数据 —— 采样、合成、最新帧调度与适配器 - What: sampling.py 实现 SurfaceSampler（向量化四点双线性：无效 tap 不计入也不致无效、任一**正权** tap 无效即整顶点无效且值为 NaN、域/形状不符先拒；采样表只在加载时构建）与 remap_to_domain（按 unroll 映射 gather，空洞保持 NaN 且 valid=False，保留尾随轴以支持矢量场）。composer.py 实现 SceneComposer.compose（域校验 → 采样 → 位移 → **从形变后位置重算法线**；scalar 绝不成以 display_gain；approximate 位移产生 display_illustrative(info) 诊断、calibrated 不产生、无位移则着色并报 field_unavailable）与 build_rest_frame（参考帧：scalar 全 NaN、valid 全假、descriptor=unavailable/display_only），gl_backend.rest_surface 改为委托它，避免两处定义。coordinator.py 实现完整调度：容量 1 双 mailbox、generation/来源/会话三重过期拒绝并计数、**最多一次合成在飞行**、可见性挂起/恢复、设置变更（倍率变化重合成**屏幕上那一帧**）、close 幂等；submit 只做身份与形状的廉价校验，绝不等显示。adapters.py 实现 Finger/Box 适配器：|flow| 或 depth 响应、可选经 unroll 映射 remap、设备 valid_mask 优先、域不符时**发布空**并报出所见的两个形状；一律不造位移（未标定的毫米不如没有），并新增 display_illustrative 到诊断码表（info）。assets.py 从"桩类堆"改成门面：re-export 真正实现 + SceneCompiler.compile（外壳 GLB 与预算检查；加密请求明确拒绝并指向阶段 5）。测试：shared/tests/test_scene3d_live.py 25 项（采样/无效语义/倍率只影响位置/示意形变标注/法线重算/remap 空洞/参考帧/只读数组）、apps/studio/tests/test_scene3d_coordinator.py 13 项（状态机、过期、替换计数、单飞行、submit p99<1ms、切换场景丢弃在飞结果、隐藏挂起、倍率重合成、close 幂等）、test_scene3d_adapters.py 13 项；并把三处"桩仍未实现"的过时断言改成对已实现行为的断言。docs/architecture/studio-3d-stages.md 阶段 3 记为完成。 - Why: 阶段 3 决定"屏幕上的颜色和力值到底对应什么"。三条最容易错且肉眼看不出来的规则必须由代码而不是约定来保证：无效点不是 0（NaN 与 valid 分离，否则缺口会被画成一个测量值）、显示倍率只动位置不动测量（否则色标悄悄变了含义）、以及显示永远不能给采集施加背压（容量 1 mailbox + 单飞行合成 + 廉价 submit）。 - Verification: shared 全量通过（scene3d 相关 103 项：contracts 26、assets 33、package 19、live 25）；studio 全量通过（scene3d 相关 54 项：stage1 24、skeleton 4、coordinator 13、adapters 13；GL 冒烟 10 项在真实 context 下另行通过）；ui_check 554 项 0 失败。过程中修掉 3 个真实缺陷：倍率变化时用 incoming.peek() 取不到"屏幕上那一帧"（已被消费）导致改了倍率画面不动——改为保留 _last_field_frame；display_illustrative 不在码表里被默认成 error；assets.py 里与 package/legacy 同名的重复桩类（会误导集成者，已改为门面）。另修正两处我自己的错误断言（对称凸起顶点法线本就竖直；pytest.approx 不能直接 .all()）。 - Not in this change: 手势与面板（阶段 4）；示意位移的真实来源（需要 SDK lift + 已确认尺度，现阶段 adapter 一��� displacement=None）；app_window 接线与记录隔离（阶段 4）。
This commit is contained in:
ykcheungab
@@ -1,11 +1,13 @@
|
||||
"""Assets: loading a validated scene, importing the legacy finger export.
|
||||
"""Assets: the one place a caller should look for loading and compiling a scene.
|
||||
|
||||
Stage 2 in ``docs/architecture/studio-3d-stages.md``. The signatures are fixed
|
||||
here so the rest of the skeleton (and the compiler tool) can be written against
|
||||
them; the bodies arrive with the stage.
|
||||
Stage 2 in ``docs/architecture/studio-3d-stages.md``. The implementations live in
|
||||
:mod:`~orisys_app_shared.scene3d.legacy` (the Blender export),
|
||||
:mod:`~orisys_app_shared.scene3d.glb` (the housing model) and
|
||||
:mod:`~orisys_app_shared.scene3d.package` (the ``.oryscene`` container); this module
|
||||
is their public face, so an integrator does not have to know which file holds what.
|
||||
|
||||
Two rules the implementation must keep, because violating them is invisible until
|
||||
a customer sees the wrong part move:
|
||||
Two rules the implementation keeps, because violating them is invisible until a
|
||||
customer sees the wrong part move:
|
||||
|
||||
* the algorithm mesh and the display mesh are **different topologies** (1032 vs
|
||||
1005 vertices) and are never joined by array index;
|
||||
@@ -17,69 +19,56 @@ a customer sees the wrong part move:
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from typing import Any, Optional
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CompileOptions,
|
||||
CompileReport,
|
||||
Diagnostic,
|
||||
ExpectedGeometry,
|
||||
F64,
|
||||
KeyProvider,
|
||||
SceneAssets,
|
||||
error,
|
||||
validate_assets,
|
||||
)
|
||||
from orisys_app_shared.scene3d.glb import load_housing_glb, with_housing
|
||||
from orisys_app_shared.scene3d.legacy import LegacyFingerImporter
|
||||
from orisys_app_shared.scene3d.package import (
|
||||
ENVELOPE_VERSION,
|
||||
MAGIC,
|
||||
SCHEMA_VERSION,
|
||||
PackagedAssetLoader,
|
||||
PackageContents,
|
||||
build_manifest,
|
||||
describe_package,
|
||||
read_package,
|
||||
scene_from_contents,
|
||||
write_package,
|
||||
)
|
||||
|
||||
_STAGE = "stage 2 (asset contracts)"
|
||||
|
||||
|
||||
class PackagedAssetLoader:
|
||||
"""Load and authenticate a ``.oryscene`` runtime package.
|
||||
|
||||
CPU/IO thread. Verification order: envelope signature, then entry hashes, then
|
||||
geometry/domain/unit checks against ``expected``. There is no plaintext
|
||||
fallback: a package that fails authentication is refused, never retried.
|
||||
"""
|
||||
|
||||
def load(
|
||||
self,
|
||||
path: Path,
|
||||
expected: ExpectedGeometry,
|
||||
keys: Optional[KeyProvider] = None,
|
||||
) -> SceneAssets:
|
||||
raise NotImplementedError(f"{_STAGE}: load a signed .oryscene package")
|
||||
|
||||
def validate(
|
||||
self, assets: SceneAssets, expected: ExpectedGeometry
|
||||
) -> tuple[Diagnostic, ...]:
|
||||
raise NotImplementedError(f"{_STAGE}: validate a loaded scene")
|
||||
|
||||
|
||||
class LegacyFingerImporter:
|
||||
"""Convert ``sensors/finger/assets/30fps-0703`` into a scene (offline only).
|
||||
|
||||
``meters_per_source_unit`` scales source lengths into metres first;
|
||||
``transform_sensor_from_source`` is then rigid (rotation and a translation in
|
||||
metres, no extra scale). Normals receive the rotation only. Missing coordinate
|
||||
evidence is a refusal, not a guess from a filename.
|
||||
"""
|
||||
|
||||
def convert(
|
||||
self,
|
||||
export_dir: Path,
|
||||
map_path: Path,
|
||||
*,
|
||||
transform_sensor_from_source: F64,
|
||||
meters_per_source_unit: float,
|
||||
) -> SceneAssets:
|
||||
raise NotImplementedError(f"{_STAGE}: import the legacy NPZ export")
|
||||
#: Bumped when the container layout changes incompatibly.
|
||||
__all__ = [
|
||||
"ENVELOPE_VERSION",
|
||||
"MAGIC",
|
||||
"SCHEMA_VERSION",
|
||||
"LegacyFingerImporter",
|
||||
"PackagedAssetLoader",
|
||||
"PackageContents",
|
||||
"SceneCompiler",
|
||||
"build_manifest",
|
||||
"describe_package",
|
||||
"read_package",
|
||||
"scene_from_contents",
|
||||
"write_package",
|
||||
]
|
||||
|
||||
|
||||
class SceneCompiler:
|
||||
"""Package a scene (and an optional housing GLB) for release (offline).
|
||||
"""Package a scene (and an optional housing GLB) for release. Offline only.
|
||||
|
||||
Signing happens with a key from the build environment; the private key is
|
||||
never part of the client runtime. Output is written to a temporary file and
|
||||
atomically renamed, so a failed build cannot leave a half package behind.
|
||||
The signing and encryption that :class:`CompileOptions` describes are stage 5:
|
||||
until then a request for an encrypted package is *refused*, rather than
|
||||
quietly producing a plain one that a release script would ship as protected.
|
||||
"""
|
||||
|
||||
def compile(
|
||||
@@ -89,4 +78,77 @@ class SceneCompiler:
|
||||
output: Path,
|
||||
options: CompileOptions,
|
||||
) -> CompileReport:
|
||||
raise NotImplementedError(f"{_STAGE}: compile a runtime package")
|
||||
if options.encryption_key_id:
|
||||
raise error(
|
||||
"asset_auth_failed",
|
||||
"encrypted packages are stage 5; refusing rather than writing a "
|
||||
"plain package that claims to be protected",
|
||||
key_id=str(options.encryption_key_id),
|
||||
)
|
||||
scene = surface
|
||||
if housing_glb is not None:
|
||||
from orisys_app_shared.scene3d.glb import HousingLimits
|
||||
|
||||
housing = load_housing_glb(
|
||||
Path(housing_glb),
|
||||
limits=HousingLimits(
|
||||
max_triangles=int(options.max_housing_triangles),
|
||||
max_vertices=max(int(options.max_surface_vertices), 1),
|
||||
),
|
||||
)
|
||||
scene = with_housing(surface, housing)
|
||||
if int(options.max_surface_vertices) > 0:
|
||||
sensing = scene.sensing_part()
|
||||
count = int(sensing.positions_sensor_m.shape[0])
|
||||
if count > int(options.max_surface_vertices):
|
||||
raise error(
|
||||
"gpu_budget_exceeded",
|
||||
f"the surface has {count} vertices (limit "
|
||||
f"{int(options.max_surface_vertices)}); export a lower LOD",
|
||||
)
|
||||
validate_assets(scene)
|
||||
report = write_package(
|
||||
scene,
|
||||
Path(output),
|
||||
sensor_model=options.expected.sensor_model,
|
||||
hardware_revisions=(options.expected.hardware_revision,),
|
||||
)
|
||||
return report
|
||||
|
||||
def validate_report(
|
||||
self, assets: SceneAssets, expected: ExpectedGeometry
|
||||
) -> tuple[Diagnostic, ...]:
|
||||
return validate_assets(assets, expected)
|
||||
|
||||
|
||||
def load_or_import(
|
||||
path: Path,
|
||||
*,
|
||||
expected: ExpectedGeometry,
|
||||
keys: Optional[KeyProvider] = None,
|
||||
legacy_map: Optional[Path] = None,
|
||||
transform_source_from_sensor: Any = None,
|
||||
meters_per_source_unit: Optional[float] = None,
|
||||
) -> SceneAssets:
|
||||
"""Load a package, or import a legacy directory, behind one entry point.
|
||||
|
||||
Keeping both paths reachable from one function is what lets the rest of the app
|
||||
stop caring which era an asset came from — and the legacy path still refuses to
|
||||
guess its units.
|
||||
"""
|
||||
path = Path(path)
|
||||
if path.is_file():
|
||||
return PackagedAssetLoader().load(path, expected, keys)
|
||||
if legacy_map is None or meters_per_source_unit is None or transform_source_from_sensor is None:
|
||||
raise error(
|
||||
"asset_rejected",
|
||||
"a legacy directory needs --map, --meters-per-source-unit and a transform",
|
||||
)
|
||||
assets = LegacyFingerImporter().convert(
|
||||
path,
|
||||
Path(legacy_map),
|
||||
transform_sensor_from_source=transform_source_from_sensor,
|
||||
meters_per_source_unit=float(meters_per_source_unit),
|
||||
)
|
||||
validate_assets(assets, expected)
|
||||
return assets
|
||||
|
||||
@@ -2,50 +2,187 @@
|
||||
|
||||
Stage 3 in ``docs/architecture/studio-3d-stages.md``.
|
||||
|
||||
This is the module that decides what a colour and a vertex position *mean*, so the
|
||||
rules from §3.2 of the design document are its reason to exist:
|
||||
This module decides what a colour and a vertex position *mean*, so the rules from
|
||||
§3.2 of the design document are its reason to exist:
|
||||
|
||||
* ``scalar`` is never multiplied by the display gain — the colour scale describes
|
||||
the measurement, and the gain describes only how far the drawing is exaggerated;
|
||||
* a displacement whose quality is ``approximate`` is drawn as "relative
|
||||
deformation, ×N" and is never labelled in millimetres;
|
||||
* without a trustworthy displacement the surface is coloured in place, and the
|
||||
total force is never turned into a per-vertex height.
|
||||
* ``scalar`` is never multiplied by the display gain. The colour scale describes the
|
||||
measurement; the gain describes only how far the drawing is exaggerated. A gain
|
||||
that changed the numbers would quietly change what the colour bar means.
|
||||
* a displacement whose quality is ``approximate`` is drawn as "relative deformation,
|
||||
×N" and is never labelled in millimetres — the composer says so in a diagnostic
|
||||
instead of leaving the UI to guess.
|
||||
* without a trustworthy displacement the surface is coloured in place, and the total
|
||||
force is never turned into a per-vertex height.
|
||||
* a reference frame measures nothing: every validity flag is false and the scalar is
|
||||
absent, so the UI hides the colour bar rather than leaving the previous one on
|
||||
screen.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
Diagnostic,
|
||||
DisplaySettings,
|
||||
FieldDescriptor,
|
||||
FrameIdentity,
|
||||
Quality,
|
||||
RenderFrame,
|
||||
SceneAssets,
|
||||
SurfaceFieldFrame,
|
||||
diagnostic,
|
||||
error,
|
||||
freeze,
|
||||
rest_descriptor,
|
||||
validate_render_frame,
|
||||
validate_surface_frame,
|
||||
)
|
||||
from orisys_app_shared.scene3d.legacy import vertex_normals
|
||||
from orisys_app_shared.scene3d.sampling import SurfaceSampler
|
||||
|
||||
_STAGE = "stage 3 (live data)"
|
||||
#: Reported when a display gain is applied to a displacement that is only
|
||||
#: approximate. Informational: the picture is legitimate as an illustration, and
|
||||
#: must not be read as a measured displacement.
|
||||
DISPLAY_ILLUSTRATIVE = "display_illustrative"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ComposeStats:
|
||||
"""What one compose cost, for the coordinator's statistics."""
|
||||
|
||||
sample_ms: float = 0.0
|
||||
normals_ms: float = 0.0
|
||||
total_ms: float = 0.0
|
||||
|
||||
|
||||
class SceneComposer:
|
||||
"""CPU worker: one field frame in, one render frame out.
|
||||
|
||||
Runs off the GUI thread and never touches Qt or GL. Recomputes normals from the
|
||||
deformed positions rather than transforming the rest normals, because the
|
||||
difference is exactly the deformation the operator is looking at.
|
||||
Runs off the GUI thread and never touches Qt or GL. Normals are recomputed from
|
||||
the deformed positions rather than transformed from the rest normals, because
|
||||
the difference between those two is exactly the deformation the operator is
|
||||
looking at.
|
||||
"""
|
||||
|
||||
def __init__(self, sampler: Optional[SurfaceSampler] = None) -> None:
|
||||
self._sampler = sampler or SurfaceSampler()
|
||||
self.last_stats = ComposeStats()
|
||||
|
||||
def compose(
|
||||
self,
|
||||
assets: SceneAssets,
|
||||
frame: SurfaceFieldFrame,
|
||||
settings: DisplaySettings,
|
||||
) -> RenderFrame:
|
||||
raise NotImplementedError(f"{_STAGE}: compose a render frame")
|
||||
import time
|
||||
|
||||
started = time.perf_counter()
|
||||
self._check(assets, frame)
|
||||
for item in validate_surface_frame(frame):
|
||||
# A frame with, say, non-finite scalars marked valid is still usable:
|
||||
# the vertex ends up invalid. Reported rather than refused.
|
||||
if item.severity == "error":
|
||||
raise error(item.code, item.message, **dict(item.details))
|
||||
part = assets.sensing_part()
|
||||
sampling_started = time.perf_counter()
|
||||
fields = self._sampler.sample(frame, assets.binding)
|
||||
self.last_stats.sample_ms = (time.perf_counter() - sampling_started) * 1000.0
|
||||
rest_positions = np.asarray(part.positions_sensor_m, dtype=np.float64)
|
||||
diagnostics: list[Diagnostic] = []
|
||||
|
||||
positions = rest_positions
|
||||
quality = frame.displacement_quality
|
||||
if settings.deform_enabled and fields.displacement_sensor_m is not None:
|
||||
gain = float(settings.display_gain)
|
||||
displacement = np.asarray(fields.displacement_sensor_m, dtype=np.float64)
|
||||
positions = rest_positions + gain * displacement
|
||||
if quality != "calibrated":
|
||||
diagnostics.append(
|
||||
diagnostic(
|
||||
DISPLAY_ILLUSTRATIVE,
|
||||
"the displayed deformation is an illustration (×"
|
||||
f"{gain:g}), not a measured displacement",
|
||||
quality=str(quality or "unspecified"),
|
||||
gain=f"{gain:g}",
|
||||
)
|
||||
)
|
||||
elif settings.deform_enabled and fields.displacement_sensor_m is None:
|
||||
diagnostics.append(
|
||||
diagnostic(
|
||||
"field_unavailable",
|
||||
"no displacement is available for this frame: colouring in place",
|
||||
)
|
||||
)
|
||||
normals_started = time.perf_counter()
|
||||
normals = vertex_normals(positions, np.asarray(part.triangles, dtype=np.int64))
|
||||
self.last_stats.normals_ms = (time.perf_counter() - normals_started) * 1000.0
|
||||
|
||||
result = RenderFrame(
|
||||
identity=frame.identity,
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=part.part_id,
|
||||
positions_sensor_m=freeze(positions, dtype=np.float32),
|
||||
normals_sensor=freeze(normals, dtype=np.float32),
|
||||
# The measurement, untouched by the display gain.
|
||||
scalar=fields.scalar,
|
||||
valid=fields.valid,
|
||||
scalar_descriptor=frame.scalar_descriptor,
|
||||
diagnostics=tuple(diagnostics),
|
||||
)
|
||||
validate_render_frame(result, vertices=int(rest_positions.shape[0]))
|
||||
self.last_stats.total_ms = (time.perf_counter() - started) * 1000.0
|
||||
return result
|
||||
|
||||
def rest_frame(self, assets: SceneAssets, identity: FrameIdentity) -> RenderFrame:
|
||||
"""The static reference: nothing measured, so nothing coloured.
|
||||
"""The static reference: nothing measured, so nothing coloured."""
|
||||
return build_rest_frame(assets, identity)
|
||||
|
||||
The scalar is all-zero with an ``unavailable`` descriptor; the UI hides the
|
||||
measurement colour bar instead of leaving the previous one on screen.
|
||||
"""
|
||||
raise NotImplementedError(f"{_STAGE}: compose the reference frame")
|
||||
@staticmethod
|
||||
def _check(assets: SceneAssets, frame: SurfaceFieldFrame) -> None:
|
||||
if assets.geometry_id and frame.geometry_id != assets.geometry_id:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
"the frame belongs to a different geometry than the scene",
|
||||
frame=frame.geometry_id,
|
||||
scene=assets.geometry_id,
|
||||
)
|
||||
if assets.field_domain_id and frame.field_domain_id != assets.field_domain_id:
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"the frame is on a different field domain than the scene",
|
||||
frame=frame.field_domain_id,
|
||||
scene=assets.field_domain_id,
|
||||
)
|
||||
if tuple(frame.scalar.shape) != tuple(assets.binding.shape_hw):
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"the frame's shape does not match the scene's UV domain",
|
||||
frame=f"{frame.scalar.shape[0]}x{frame.scalar.shape[1]}",
|
||||
scene=f"{assets.shape_hw[0]}x{assets.shape_hw[1]}",
|
||||
)
|
||||
|
||||
|
||||
def build_rest_frame(assets: SceneAssets, identity: FrameIdentity) -> RenderFrame:
|
||||
"""The asset's own geometry at rest, with no measurement attached.
|
||||
|
||||
Shared by the composer and by the GL layer, which needs it to *clear* a stale
|
||||
measurement after a refused frame or a disconnect.
|
||||
"""
|
||||
part = assets.sensing_part()
|
||||
count = int(part.positions_sensor_m.shape[0])
|
||||
return RenderFrame(
|
||||
identity=identity,
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=part.part_id,
|
||||
positions_sensor_m=part.positions_sensor_m,
|
||||
normals_sensor=part.normals_sensor,
|
||||
scalar=freeze(np.full(count, np.nan, dtype=np.float32), dtype=np.float32),
|
||||
valid=freeze(np.zeros(count, dtype=bool), dtype=np.bool_),
|
||||
scalar_descriptor=rest_descriptor(),
|
||||
)
|
||||
|
||||
@@ -60,6 +60,9 @@ DIAGNOSTIC_SEVERITY: Mapping[str, Severity] = {
|
||||
"gpu_budget_exceeded": "error",
|
||||
"asset_rejected": "error",
|
||||
"no_data": "info",
|
||||
#: An illustrative display deformation was applied. Informational by design:
|
||||
#: the picture is legitimate, and the point is that it is *labelled*.
|
||||
"display_illustrative": "info",
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -74,6 +74,21 @@ class LegacyFieldDomain:
|
||||
index_order: str = "row-major"
|
||||
mapping_digest: str = ""
|
||||
world_xyz: Optional[np.ndarray] = None # float32 [H,W,3], for verification only
|
||||
#: The unroll map itself: for every output pixel, the source pixel it came from
|
||||
#: (``-1`` where there is none). The adapters need it to bring a live field into
|
||||
#: this domain, and the digest above already covers it.
|
||||
map_x: Optional[np.ndarray] = None
|
||||
map_y: Optional[np.ndarray] = None
|
||||
#: The grid the map samples from, as the file states it (0 when unstated).
|
||||
source_width: int = 0
|
||||
source_height: int = 0
|
||||
|
||||
@property
|
||||
def source_hw(self) -> Optional[tuple[int, int]]:
|
||||
"""The grid the map samples from, when the file states it."""
|
||||
if self.source_width <= 0 or self.source_height <= 0:
|
||||
return None
|
||||
return (int(self.source_height), int(self.source_width))
|
||||
|
||||
@property
|
||||
def valid_ratio(self) -> float:
|
||||
@@ -137,6 +152,20 @@ def load_legacy_domain(map_path: Path) -> LegacyFieldDomain:
|
||||
source_scale=source_scale,
|
||||
mapping_digest=mapping_digest,
|
||||
)
|
||||
|
||||
def _optional_array(name: str) -> Optional[np.ndarray]:
|
||||
value = data.get(name)
|
||||
return None if value is None else np.asarray(value)
|
||||
|
||||
def _optional_int(name: str) -> int:
|
||||
value = data.get(name)
|
||||
if value is None:
|
||||
return 0
|
||||
try:
|
||||
return int(np.asarray(value).reshape(-1)[0])
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
return LegacyFieldDomain(
|
||||
shape_hw=(shape[0], shape[1]),
|
||||
flip_v=flip_v,
|
||||
@@ -149,6 +178,12 @@ def load_legacy_domain(map_path: Path) -> LegacyFieldDomain:
|
||||
if world is not None
|
||||
else None
|
||||
),
|
||||
# Kept for the adapters: this is the map that brings a live field into this
|
||||
# domain, and the domain digest above already covers its content.
|
||||
map_x=_optional_array("map_x"),
|
||||
map_y=_optional_array("map_y"),
|
||||
source_width=_optional_int("source_width"),
|
||||
source_height=_optional_int("source_height"),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -2,30 +2,172 @@
|
||||
|
||||
Stage 3 in ``docs/architecture/studio-3d-stages.md``.
|
||||
|
||||
The sampling table is built once, at load time, from the UV field's own domain
|
||||
(shape, crop, scale, flip). It is never rebuilt per frame and never repaired by
|
||||
searching for the nearest triangle: a vertex whose samples are not all valid is
|
||||
invalid, and the mask is not crossed.
|
||||
The sampling table is built once, at load time, from the field's own domain. It is
|
||||
never rebuilt per frame, and a vertex is never repaired by searching for the nearest
|
||||
triangle: if any tap that carries weight is invalid, the vertex is invalid.
|
||||
|
||||
Three details that decide whether a measurement is honest:
|
||||
|
||||
* **an invalid tap is not zero.** Weighting a gap with weight 0 would still poison
|
||||
the sum, because ``0 * NaN`` is ``NaN``; invalid taps are therefore excluded from
|
||||
the sum and the *vertex* is marked invalid, which is what the shader draws as the
|
||||
neutral colour.
|
||||
* **validity is a separate array from the value.** A vertex whose scalar happens to
|
||||
be 0.0 is a measurement of zero; a vertex that is invalid has no value at all, and
|
||||
the value is left as ``NaN`` so nobody can mistake one for the other.
|
||||
* **the domain is checked, not assumed.** A frame whose domain or shape does not
|
||||
match the binding is refused with ``field_domain_mismatch`` rather than sampled
|
||||
into a plausible-looking but wrong picture.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
SurfaceBinding,
|
||||
SurfaceFieldFrame,
|
||||
VertexFields,
|
||||
error,
|
||||
freeze,
|
||||
)
|
||||
|
||||
_STAGE = "stage 3 (live data)"
|
||||
#: A weight below this is treated as "not used": the binding builder never emits
|
||||
#: one, and a rounding-level weight must not be able to invalidate a vertex.
|
||||
WEIGHT_EPS = 1e-7
|
||||
|
||||
|
||||
class SurfaceSampler:
|
||||
"""Bilinear sampling through a precomputed binding.
|
||||
|
||||
Vectorised over vertices: gather ``[N,4]`` taps, weight them, and treat any
|
||||
positive-weight invalid tap as a reason to invalidate the whole vertex. A
|
||||
scalar that is not finite is "no measurement", not a zero.
|
||||
"""
|
||||
"""Vectorised bilinear sampling through a precomputed binding."""
|
||||
|
||||
def sample(self, frame: SurfaceFieldFrame, binding: SurfaceBinding) -> VertexFields:
|
||||
raise NotImplementedError(f"{_STAGE}: sample a UV field onto the vertices")
|
||||
self._check_domains(frame, binding)
|
||||
indices = np.asarray(binding.sample_indices, dtype=np.int64) # [N,4]
|
||||
weights = np.asarray(binding.sample_weights, dtype=np.float64) # [N,4]
|
||||
used = indices >= 0
|
||||
if not used.any() and len(indices):
|
||||
raise error("field_domain_mismatch", "the binding samples nothing at all")
|
||||
# ``-1`` -> index 0 for the gather; the weight there is 0 by contract and the
|
||||
# ``used`` mask keeps it out of both the sum and the validity test.
|
||||
safe = np.where(used, indices, 0)
|
||||
height, width = frame.scalar.shape
|
||||
flat_scalar = np.asarray(frame.scalar, dtype=np.float64).reshape(-1)
|
||||
flat_valid = np.asarray(frame.valid, dtype=bool).reshape(-1)
|
||||
if safe.size and int(safe.max()) >= flat_scalar.size:
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"a binding index falls outside the frame's field",
|
||||
index=int(safe.max()),
|
||||
size=int(flat_scalar.size),
|
||||
)
|
||||
gathered_scalar = flat_scalar[safe]
|
||||
gathered_valid = flat_valid[safe]
|
||||
finite = np.isfinite(gathered_scalar)
|
||||
# A tap counts only where it is used, in range, finite and valid.
|
||||
contributed = used & gathered_valid & finite
|
||||
valid = contributed | ~used
|
||||
valid = valid.all(axis=1)
|
||||
contribution = np.where(contributed, gathered_scalar, 0.0) * weights
|
||||
scalar = contribution.sum(axis=1)
|
||||
# No measurement: the value is absent, not zero.
|
||||
scalar = np.where(valid, scalar, np.nan)
|
||||
|
||||
displacement = None
|
||||
if frame.displacement_sensor_m is not None:
|
||||
flat_displacement = np.asarray(
|
||||
frame.displacement_sensor_m, dtype=np.float64
|
||||
).reshape(-1, 3)
|
||||
if flat_displacement.shape[0] != flat_scalar.shape[0]:
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"the displacement field does not match the scalar field",
|
||||
)
|
||||
gathered = flat_displacement[safe] # [N,4,3]
|
||||
finite_disp = np.isfinite(gathered).all(axis=2)
|
||||
contributed_disp = used & gathered_valid & finite_disp
|
||||
displacement = np.where(contributed_disp[:, :, None], gathered, 0.0)
|
||||
displacement = (displacement * weights[:, :, None]).sum(axis=1)
|
||||
displacement = np.where(valid[:, None], displacement, np.nan)
|
||||
displacement = freeze(displacement, dtype=np.float32)
|
||||
del height, width
|
||||
return VertexFields(
|
||||
scalar=freeze(scalar, dtype=np.float32),
|
||||
valid=freeze(valid, dtype=np.bool_),
|
||||
displacement_sensor_m=displacement,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _check_domains(frame: SurfaceFieldFrame, binding: SurfaceBinding) -> None:
|
||||
if binding.geometry_id and frame.geometry_id != binding.geometry_id:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
"the frame belongs to a different geometry than the binding",
|
||||
frame=frame.geometry_id,
|
||||
binding=binding.geometry_id,
|
||||
)
|
||||
if binding.field_domain_id and frame.field_domain_id != binding.field_domain_id:
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"the frame is on a different field domain than the binding",
|
||||
frame=frame.field_domain_id,
|
||||
binding=binding.field_domain_id,
|
||||
)
|
||||
if tuple(frame.scalar.shape) != tuple(binding.shape_hw):
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
"the frame's shape does not match the binding's domain",
|
||||
frame=f"{frame.scalar.shape[0]}x{frame.scalar.shape[1]}",
|
||||
binding=f"{binding.shape_hw[0]}x{binding.shape_hw[1]}",
|
||||
)
|
||||
if frame.scalar.shape != frame.valid.shape:
|
||||
raise error(
|
||||
"field_domain_mismatch", "the frame's scalar and validity disagree"
|
||||
)
|
||||
|
||||
|
||||
def sample_field(frame: SurfaceFieldFrame, binding: SurfaceBinding) -> VertexFields:
|
||||
"""Convenience wrapper: one sampler, no state to keep."""
|
||||
return SurfaceSampler().sample(frame, binding)
|
||||
|
||||
|
||||
def remap_to_domain(
|
||||
field: np.ndarray,
|
||||
*,
|
||||
map_x: np.ndarray,
|
||||
map_y: np.ndarray,
|
||||
shape_hw: tuple[int, int],
|
||||
) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Resample a field from its own grid onto the scene's UV domain.
|
||||
|
||||
The legacy unroll map is exactly this: for every output pixel it stores the
|
||||
source pixel it came from (``-1`` where there is none). Remapping is therefore
|
||||
a gather, not an interpolation — the map already chose the sample.
|
||||
|
||||
Returns ``(values, valid)`` where ``values`` keeps the trailing axes of the
|
||||
input (so ``[H,W,2]`` flow becomes ``[h,w,2]``) and ``values`` is ``NaN``
|
||||
wherever the map has no source pixel. Nothing is invented for a gap: a remap
|
||||
that filled holes would put measurements where the sensor has no surface.
|
||||
"""
|
||||
values = np.asarray(field)
|
||||
if values.ndim < 2:
|
||||
raise error("field_domain_mismatch", "a field to remap must be at least 2-D")
|
||||
height, width = int(shape_hw[0]), int(shape_hw[1])
|
||||
xs = np.asarray(map_x)
|
||||
ys = np.asarray(map_y)
|
||||
if xs.shape != (height, width) or ys.shape != (height, width):
|
||||
raise error(
|
||||
"field_domain_mismatch",
|
||||
f"the remap is {xs.shape} but the domain is {height}x{width}",
|
||||
)
|
||||
source_height, source_width = values.shape[0], values.shape[1]
|
||||
usable = (xs >= 0) & (ys >= 0) & (xs < source_width) & (ys < source_height)
|
||||
safe_x = np.where(usable, xs, 0).astype(np.int64)
|
||||
safe_y = np.where(usable, ys, 0).astype(np.int64)
|
||||
gathered = values[safe_y, safe_x] # [h,w] or [h,w,...]
|
||||
finite = np.isfinite(np.asarray(gathered, dtype=np.float64))
|
||||
if gathered.ndim > 2:
|
||||
finite = finite.all(axis=tuple(range(2, gathered.ndim)))
|
||||
valid = usable & finite
|
||||
out = np.array(gathered, dtype=np.float32, copy=True)
|
||||
out[~valid] = np.nan
|
||||
return freeze(out, dtype=np.float32), freeze(valid, dtype=np.bool_)
|
||||
|
||||
Reference in New Issue
Block a user