orisys-tech/orisys-sensor-design
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
59f48d6d06ab28e07174573430022ed1a7814a94
orisys-sensor-design/docs/probe-pipeline.md
T
ykcheungab c377c14474 1. added blender deformation probe for marker size picking and optimization 
2. added unwarp map export in current cli, don't need to generate by precompute in sensor running
2026-09-25 15:44:44 +08:00

35 KiB
Raw Blame History

Probe sensitivity pipeline — frozen interfaces

This document is the interface freeze for the probe sensitivity bench. Four workstreams code against it in parallel. Treat every schema, path, and name here as a contract: if it must change, the Lead changes this file first and tells everyone.

Scope: deform the sensing surface through a known displacement, render what cam_sensor sees, measure optical flow against the zero-displacement reference, and report a ranked, auditable sensitivity score per marker pattern.

Measured ground truth for sensors/finger-xynova/r01 (from geometry.blend, read-only; the tool never writes the .blend):

Fact Value
sensing_surface 2508 verts, 4834 polys, UV maps compute, produce; no modifiers, no shape keys
probe vertex group 6 verts [215, 216, 218, 219, 327, 329], weight 1.0
Probe patch 0.755 x 1.018 mm, flat to 7.6 um, all 6 normals within 2.6 deg of their mean
Authoring unit 1 Blender unit = 1 mm
Camera cam_sensor PERSP, lens 0.5469, sensor_width 1.4, fit HORIZONTAL
Render 500x500, CYCLES, PNG RGBA
Projection mesh spans x 8.13..491.88, y 33.13..397.59 (fully inside)
Scale 13.430 px/mm (p50) -> 0.01 mm = 0.134 px, 0.1 mm = 1.34 px
Mesh edges min 0.0504, p50 0.6285, p90 0.991, max 1.816 mm
Falloff at r=0.5 mm only 5 of 2502 non-probe verts (0.285 x1, 0.4161..0.4168 x4; next 0.796)
DIS patch (8 px) 0.5957 mm at 500x500; an 80x80 marker cell is 0.2313 mm = 3.11 px
Blend sha256 1C8A050B8490B6832D3A9F269DFEF3DAFBF94DBFE3E943B3956BE1104A50D9C9

Pipeline

design.yaml [probe:]
   |
   +-- deform.py  (host, py310, no bpy)  -> _deform_job.json
   |        probe_deform.py (Blender)    -> probe_frames.npz + deform_report.json
   |
   +-- render.py  (host, no bpy)         -> _render_job.json
   |        probe_render.py (Blender)    -> frames/*.png + render_manifest.json
   |
   +-- flow.py    (host, cv2)            -> _track_job.json, flow/, flow_report.json
   |        (or is called in-process by cli.py)
   |
   +-- sensitivity.py  -> sensitivity_report.json
   +-- optimise.py     -> optimizer_report.json

deform.frame_plan() is the single ordering authority. Every job carries the resolved frame list so render and analysis cannot disagree about frame order.

All artifacts live under:

build/probe/<sensor>/<revision>/

resolved the same way texture.py resolves build/textures/... (see texture_revision_rel; reuse it, do not re-invent it).

Frame plan

Positive-only sweep, axes in the order probe.axes (default x, y, z), steps step_mm, 2*step_mm, ... up to and including max_mm.

index name axis step_mm
0 000_reference null 0.0
1 001_x_+0.01mm x 0.01
... ... ... ...
10 010_x_+0.10mm x 0.10
11..20 0NN_y_+0.0Xmm y ...
21..30 0NN_z_+0.0Xmm z ...

F = 1 + len(axes) * round(max_mm / step_mm) = 31 for the default settings. PNG for frame i is frames/<name>.png.

The displacement for frame i is the frame's unit axis of the probe-average basis (frozen below) scaled by step_mm. All 6 probe vertices move by exactly that vector; every other vertex moves by weight * vector.

The probe-average basis

Built from the area-independent average of the 6 probe vertex normals, then made orthonormal so the three sweeps are exactly orthogonal and comparable:

z = normalize(sum(normal_i for i in probe))     # surface normal
x = normalize(cross(Vector((0,0,1)), z))        # fall back to (1,0,0) if degenerate
y = cross(z, x)
origin = mean(co_i for i in probe)

Measured for Xynova (orthonormal to 1e-10):

z = (-0.000027, -0.061749,  0.998092)
x = ( 1.000000, -0.000431,  0.000000)
y = ( 0.000430,  0.998092,  0.061749)
origin = (74.76961, 16.21550, 120.80202)

An axis vector for displacement is {'x': x, 'y': y, 'z': z}[axis] * step_mm in the mesh object's local space. Since sensing_surface.matrix_world.to_scale() == (1,1,1) and meters_per_source_unit == 0.001, local units are mm and the vector is in mm.

Proportional falloff

We deliberately do not use bpy.ops.transform.translate(use_proportional_edit=True): that operator is viewport-modal and its proportional_size is in screen pixels relative to a 3D-view region, which does not exist under blender -b and would not be reproducible. The falloff is computed explicitly, in mm, in a bpy-free module.

Semantics match Blender's proportional editing: the falloff distance is measured from the original positions of the selected (probe) vertices, so the weight set is constant across all frames.

t = d / radius_mm                       # d = min distance to any probe vertex
smooth:   w = 1 - (3t^2 - 2t^3)         # Blender's "Smooth" curve; 1 at t=0, 0 at t>=1
linear:   w = 1 - t
sharp:    w = (1 - t)^2
constant: w = 1
w = 0 whenever t >= 1, and w = 1 for probe members

design.yaml probe: block

probe:
  mesh: sensing_surface
  camera: cam_sensor
  group: probe
  uv_map: compute
  texture: patch_texture_compute.png
  radius_mm: 0.5
  falloff: smooth
  step_mm: 0.01
  max_mm: 0.10
  axes: [x, y, z]
  resolution: [500, 500]
  process_resolution: [250, 250]
  tracker:
    algorithm: dis
    dis_preset: medium
    dis_mode: 1
  render:
    engine: CYCLES
    samples: 16
    denoise: false
    view_transform: Standard
    interpolate: Closest
    seed: 0

Built-in fallbacks live in deform.DEFAULT_PROBE and are merged DEFAULT_PROBE <- design.yaml probe <- CLI overrides, exactly like texture.merge_texture_patch. Unknown keys are an error. Validation returns one message per problem and raises ProbeConfigError.

Job files (stdlib JSON; Blender never reads YAML)

_deform_job.json

{
  "schema": "orisys-sensor-design/probe-deform-job-v1",
  "revision_dir": "<abs>",
  "source_blend": "<abs>",
  "output_dir": "<abs>",
  "objects": {"mesh": "sensing_surface", "group": "probe"},
  "settings": {
    "radius_mm": 0.5,
    "falloff": "smooth",
    "step_mm": 0.01,
    "max_mm": 0.10,
    "axes": ["x", "y", "z"]
  },
  "frames": [
    {"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0},
    {"index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01}
  ],
  "artifacts": {"frames": "<abs>/probe_frames.npz", "report": "<abs>/deform_report.json"}
}

_render_job.json

{
  "schema": "orisys-sensor-design/probe-render-job-v1",
  "revision_dir": "<abs>",
  "source_blend": "<abs>",
  "output_dir": "<abs>",
  "objects": {"mesh": "sensing_surface", "camera": "cam_sensor"},
  "settings": {
    "uv_map": "compute",
    "texture": "<abs path to the marker PNG>",
    "width": 500, "height": 500,
    "radius_mm": 0.5,
    "engine": "CYCLES", "samples": 16,
    "denoise": false, "view_transform": "Standard",
    "interpolate": "Closest", "seed": 0
  },
  "frames": ["<same frames array as the deform job>"],
  "null_frame": {"name": "reference_null", "seed": 1},
  "artifacts": {"frames_npz": "<abs>/probe_frames.npz", "report": "<abs>/render_manifest.json"}
}

null_frame is a second render of the reference at a different Cycles seed. It is never passed to the tracker: it exists only to measure the estimator's zero-input floor.

_track_job.json

{
  "schema": "orisys-sensor-design/probe-track-job-v1",
  "output_dir": "<abs>",
  "settings": {
    "axes": ["x", "y", "z"],
    "step_mm": 0.01, "max_mm": 0.10,
    "radius_mm": 0.5,
    "process_resolution": [250, 250],
    "tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1}
  },
  "scales": [
    {"name": "render", "width": 500, "height": 500, "resample": false},
    {"name": "process", "width": 250, "height": 250, "resample": true}
  ],
  "artifacts": {"manifest": "<abs>/render_manifest.json", "strategy": "<abs>/strategy_probe.json"}
}

probe_frames.npz

The contract between the deformation and rendering workstreams.

Key Shape / dtype Meaning
base_vertices (V, 3) float64 original object-local coords, all V vertices
vertices (F, V, 3) float64 deformed coords per frame; index 0 = reference
frame_names (F,) <U... frame name, matching the plan
frame_axes (F,) <U4 null for the reference, else x/y/z
frame_steps_mm (F,) float64 applied displacement magnitude in mm
weights (V,) float64 falloff weight per vertex (constant across frames)
origin (3,) float64 probe centroid, object-local
basis (3, 3) float64 rows are x, y, z unit axes, object-local
probe_indices (P,) int32 member vertex indices

probe_render.py must assert that len(vertices) == len(frame_names) and that the plan in _render_job.json matches frame_names element-wise, then fail with exit 1 if not. It writes each frame to the mesh and renders; it never re-derives weights.

deform_report.json

{
  "schema": "orisys-sensor-design/probe-deform-report-v1",
  "generated_at": "<iso8601 utc>",
  "source_blend": {"path": "<abs>", "sha256": "..."},
  "mesh": "sensing_surface",
  "group": "probe",
  "probe_indices": [215, 216, 218, 219, 327, 329],
  "origin": [74.76961, 16.2155, 120.80202],
  "basis": [[1.0, -0.000431, 0.0], [0.00043, 0.998092, 0.061749],
            [-0.000027, -0.061749, 0.998092]],
  "basis_orthonormal": true,
  "normal_angle_max_deg": 2.553,
  "flatness_mm": 0.007611,
  "radius_mm": 0.5,
  "falloff": "smooth",
  "vertex_count": 2508,
  "edge_p50_mm": 0.628501,
  "weights": {"min": 0.0, "max": 1.0, "members_at_one": 6,
              "partial": 5, "unaffected": 2497},
  "affect_bands": {"1.0": 6, "0.3-0.5": 1, "0.05-0.1": 4},
  "frames": [{"index": 0, "name": "000_reference", "axis": null,
              "step_mm": 0.0, "max_delta_mm": 0.0, "moved_vertices": 0}],
  "warnings": ["radius 0.5 mm reaches only 5 vertices outside the probe group ..."]
}

moved_vertices is the number of vertices whose displacement is non-zero in that frame: 0 for the reference, and 6 + partial (= 11 on Xynova) for any non-zero step. It is not the probe member count, and it is not the same for every frame — reporting 6 on the reference would contradict max_delta_mm: 0.0. The weights block remains the authority for how many vertices are partially weighted.

warnings is the honesty channel. Emit one whenever affected (partial) < 4 * len(probe_indices), or radius_mm < edge_p50_mm.

render_manifest.json

{
  "schema": "orisys-sensor-design/probe-render-manifest-v1",
  "generated_at": "<iso8601 utc>",
  "source_blend": {"path": "<abs>", "sha256": "..."},
  "texture": {"path": "<abs>", "sha256": "..."},
  "uv_map": "compute",
  "width": 500, "height": 500,
  "settings_applied": {"engine": "CYCLES", "samples": 16, "denoise": false,
                       "view_transform": "Standard", "look": "None",
                       "exposure": 0.0, "interpolate": "Closest", "seed": 0},
  "mesh_px_per_mm": {"min": 10.499, "p05": 10.946, "p50": 13.430,
                     "p95": 22.886, "max": 38.594,
                     "basis": "per-mesh-edge projected px / 3D mm, all 7341 edges"},
  "probe_px_per_mm": {"p50": 13.248, "x": 13.248, "y": 17.670, "z": 1.519,
                      "basis": "probe-local: |centroid_px(step) - centroid_px(0)| / step_mm, per axis at its largest step"},
  "probe_roi_px": {"u": 0.0, "v": 0.0, "radius": 0.0},
  "reference_index": 0,
  "reference_deterministic": true,
  "frames": [{"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0,
              "png": "frames/000_reference.png", "sha256": "...",
              "width": 500, "height": 500,
              "centroid_px": [249.61, 214.38]}],
  "null": {"name": "reference_null", "seed": 1,
           "png": "frames/reference_null.png", "sha256": "..."}
}

reference_deterministic reports the result of rendering frame 0 twice in the same session with the same seed and comparing the saved bytes. It must be true; if it is not, the render is not reproducible and the CLI must fail.

Compare decoded pixels, not file bytes, and allow a 1-LSB tolerance. This was established the hard way: with literal whole-file byte equality the check rejected candidate nx=36 on the real run, and PNG chunk analysis showed why — the two files had identical chunk structure (the only tEXt chunk was the deterministic cycles.ViewLayer.samples) yet their decompressed pixel data differed in 2 bytes of 250,250, at two scanline offsets. That is Cycles CPU thread-order floating-point nondeterminism tipping a boundary pixel by one 8-bit step, not a metadata artefact and not a defect in the render.

A gate that fails on a 1-LSB difference is not measuring reproducibility; it is measuring thread scheduling, and because it is content-dependent it silently disqualifies some patterns and biases a pattern search. So:

  • Strip only the known-volatile tEXt chunks as before (Date, Time, RenderTime, cycles.ViewLayer.*_time); keep image chunks byte-for-byte.
  • Decode both PNGs and require the same shape, max |delta| <= 1 per 8-bit channel, and a differing-pixel fraction at or below 0.05%. Record the measured max_abs_delta and differing_fraction in the manifest either way.
  • Fail loudly when a tolerance is exceeded, so real nondeterminism still stops the run.
  • Never compare raw file bytes: PNG compression and metadata are not the measurement.

probe_roi_px is the probe centroid projected to pixels plus the radius in pixels that corresponds to radius_mm (radius = radius_mm * probe_px_per_mm.p50), all measured from the actual projection (not assumed). The tracker uses it to build its ROI, and the ROI is fixed at the reference centroid for every frame — a moving ROI would bias the ROI mean as the probe moves, which is exactly the signal we are trying to measure.

Two different pixel scales, and why they must not be confused

  • mesh_px_per_mm is a mesh-wide statistic: for every mesh edge, its projected length in pixels divided by its 3D length in mm, summarised over all 7341 edges. It describes how the whole surface is sampled in the camera. It is not the gain to convert the probe's measured flow into millimetres.
  • probe_px_per_mm is the probe-local scale, and it is the one the scorer must use: |centroid_px(step) - centroid_px(0)| / step_mm, per axis, i.e. how many image pixels the probe actually moves per millimetre applied. It is anisotropic, and z (motion along the surface normal) is expected to be much smaller than x and y because normal motion is largely out of the image plane. That is a real physical property of the sensor, not a defect.

Because the z scale can approach zero, dividing a measured response by it is ill-conditioned. The scorer therefore reports gain in px per mm as the primary quantity and only derives mm per mm when the axis scale clears sensitivity.min_local_scale_px_per_mm (default 1.0; a module constant in sensitivity.py, deliberately not a design.yaml key so it cannot be tuned into silence). Never divide by a scale below that; report null and warn instead. A large "mm/mm" gain produced by dividing by a near-zero scale would be an artefact, not a sensitivity.

The default of 1.0 is validated by real data rather than chosen by feel. Measured per-axis probe scales: x 13.248, y 17.670, z 1.519 px/mm at the render scale, and x 6.624, y 8.835, z 0.760 px/mm at the process scale. So 1.0 discriminates exactly where it should on this revision: render-scale z (1.519) is admitted and yields a derived gain_mm_per_mm of 0.445 backed by 7 steps above the floor at snr 4.2, while process-scale z (0.760) is correctly gated to null.

Note what this implies about the earlier worry that z would be unmeasurable: at the process scale a 0.1 mm normal displacement is only 0.076 px against a 0.19 px floor, so it genuinely is unmeasurable there; at the render scale it is 0.152 px against a 0.028 px floor, so it is measurable from about 0.04 mm upward. The gate is a guard against dividing by a degenerate scale, not a substitute for the measured snr, points_above_floor and resolvable_step_mm, which are the quantities that decide whether an axis is useful. Report all of them; do not suppress a number that the data supports.

What actually limits the measurement (measured, not assumed)

Do not attribute the response deficit to aliasing, noise, or ROI dilution. The render-scale deficit is spatial resolution of the measurement region, established by these measurements on the real frames:

  • The real x +0.10 mm flow radial profile is essentially uniform inside the ROI (-0.9162, -0.9096, -0.8725 px at r 0-2, 2-4, 4-6.6), with |flow| p5..p100 of 0.81..0.93 px. So it is a coherent translation, not noise and not a diluted average: the motion is simply shrunk to about 0.67x of the applied 1.3248 px.
  • Injecting a synthetic local rigid translation of 1.3248 px into the reference and measuring it over the same ROI recovers only 17% at a patch radius of 6.6 px (equal to the ROI), then 33% at r 8, 51% at r 10, 63% at r 13, 81% at r 20 and 95% at r 40.
  • Therefore the attenuation is caused by the moving probe patch plus its falloff taper spanning only about 13 px, i.e. 1-2 DIS patches (8 px), so DIS regularizes toward the static background. At the process scale the same region is about 6.5 px, under one patch, and the response collapses while the forward-backward error grows from 1.6 to 8.8 (x) and 15.8 (y).

Two consequences the report must state rather than bury:

  1. The measured gain is confounded with the ROI size. It grows strongly with radius_mm because a larger radius gives DIS more moving texture to lock onto. Relative ranking of marker patterns is valid only while radius_mm, the ROI and the scale are held fixed across candidates — which is why run_optimise varies nx/ny/seed and never the radius.
  2. "Below one pixel" and "below the estimator floor" are different statements and must not be conflated. The measured zero-input floor is 0.0292 px at the render scale, so a 0.01 mm step (0.1325 px) is about 4.5x above the floor: the ROI-averaged response genuinely resolves single steps at 500x500, with monotone per-step responses (0.0810, 0.1316, 0.1971, ... 0.8859 px). The earlier expectation that the sub-pixel step would be lost in the floor is not supported by the measurement; what is lost is per-pixel precision, not the ROI mean. Report the floor and the step in pixels and let the numbers speak.

Each frames[] entry also carries centroid_px: the probe centroid projected to pixels for that frame's deformation, as full-precision floats (not rounded). This is the authoritative source for the image-space displacement direction the sensitivity stage needs. It is a small number of pixels (1.34 px across the whole 0.1 mm sweep at 500x500), which is why it is recorded as a float and why the scorer regresses over all steps rather than differencing a single pair.

flow_report.json

{
  "schema": "orisys-sensor-design/flow-report-v1",
  "generated_at": "<iso8601 utc>",
  "tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1,
              "finest_scale": 1, "source": "design.yaml probe.tracker"},
  "known_answer": {"passed": true, "cases": [
      {"dx": 2, "dy": 0, "measured_u": 2.0, "measured_v": 0.0, "tol": 0.25},
      {"dx": 0, "dy": 2, "measured_u": 0.0, "measured_v": 2.0, "tol": 0.25}]},
  "reference_texture_known_answer": {
      "passed": false, "gating": false,
      "note": "a known shift applied to the real rendered marker; a periodic "
              "pattern legitimately fails this when its cell approaches the DIS "
              "patch, so it is a diagnostic, not a plumbing gate",
      "cases": [{"dx": 2, "dy": 0, "measured_u": 8.17, "expected_u": 2.0}]},
  "reference_self": {"max_abs_flow_px": 0.0},
  "scales": [{
    "name": "render", "width": 500, "height": 500, "resample": false,
    "px_per_mm": 13.43, "mm_per_px": 0.07446,
    "frames": [{
      "index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01,
      "roi": {"u": 249.6, "v": 214.4, "radius": 6.7, "pixels": 141},
      "flow": {"mean_u": 0.02, "mean_v": -0.01,
               "mean_mag": 0.03, "p50_mag": 0.02, "p95_mag": 0.08, "max_mag": 0.31,
               "coverage": 0.41},
      "response_proj_px": 0.02,
      "response_mm": 0.0015,
      "forward_backward_error": 1.87}
    ],
    "null": {"floor_px": 0.05, "floor_source": "reference_null"}
  }]
}

Per-frame flow arrays are also written to flow/<scale>/<name>.npy as float32 (H, W, 2) so the sensitivity stage and any later inspection can reuse them without re-running DIS.

response_mm and response_proj_px are informational. Pixels are authoritative (response_proj_px); response_mm is derived through a single isotropic scalar scale (px_per_mm, the probe-local p50) and therefore understates or overstates the true conversion on an anisotropic axis — notably z. The scorer must use response_proj_px plus the per-axis local scale from render_manifest.json and must not consume response_mm. Each scale records px_per_mm_source naming the exact manifest field it came from, so the provenance of the scalar is auditable.

sensitivity_report.json

{
  "schema": "orisys-sensor-design/sensitivity-report-v1",
  "generated_at": "<iso8601 utc>",
  "revision": "sensors/finger-xynova/r01",
  "texture": {"path": "<abs>", "sha256": "...", "grid": [80, 80],
              "cell_mm": [0.23125, 0.205]},
  "probe": {"group": "probe", "origin": [...], "basis": [...],
            "radius_mm": 0.5, "falloff": "smooth", "affected_vertices": 5},
  "tracker": {"dis_preset": "medium", "dis_mode": 1, "patch_px": 8},
  "scales": [{
    "name": "render", "px_per_mm": 13.43,
    "displacement_px_per_step": 0.1343,
    "cell_px": 3.11,
    "cell_over_patch_px": 0.389,
    "null_floor_px": 0.05,
    "null_floor_basis": "render_seed_reproducibility",
    "axes": [{
      "axis": "x", "direction_px": [0.999, -0.031],
      "direction_source": "centroid_regression",
      "points": [{"step_mm": 0.01, "applied_px": 0.134, "response_px": 0.02,
                  "above_floor": false}],
      "summary": {"gain_px_per_mm": 6.2, "gain_mm_per_mm": 0.46,
                  "scale_px_per_mm": 13.42, "snr": 0.42, "spearman": 0.95,
                  "resolvable_step_mm": null, "saturation_mm": null,
                  "cross_axis_leak": 0.18}
    }],
    "score": 0.11,
    "verdict": "unresolvable"
  }],
  "overall": {"score": 0.11, "verdict": "unresolvable",
              "best_scale": "render", "best_axis": "z"},
  "warnings": ["0.01 mm is 0.134 px at 500x500; a single step is below the estimator floor"],
  "notes": []
}

Scoring (every input measured; no assumed constants)

For one scale and one axis:

  1. d_hat_px — unit image-space direction of the applied displacement, from render_manifest.json's per-frame centroid_px. Regress centroid_px on step_mm over every frame of that axis and normalize the slope vector (du/dstep, dv/dstep); this is preferred over a single 0 -> +max_mm finite difference because the whole sweep is only ~1.3 px. Fall back to the two-point finite difference when the regression is degenerate, and to the ROI-mean flow at the +max_mm frame when centroid_px is absent. Record which was used as direction_source per axis.
  2. response_i = mean over ROI pixels of dot(flow_i, d_hat_px), in px. Sign kept.
  3. floor_px = p95 of |response| measured on reference_null, projected on the same d_hat_px. Reported per axis.
  4. gain_px_per_mm = OLS slope of response_px vs applied_mm, over the steps with |response_i| > floor_px. Requires >= 3 such points, else null. This is the primary sensitivity: measured pixels of probe response per millimetre applied.
  5. scale_px_per_mm = probe_px_per_mm[axis]. Then gain_mm_per_mm = gain_px_per_mm / scale_px_per_mm, reported only when scale_px_per_mm >= min_local_scale_px_per_mm (default 1.0); otherwise null plus a warning. Never divide by a near-zero scale: for the z axis that would manufacture an enormous gain no camera could observe.
  6. snr = |gain_px_per_mm| * max_mm / max(floor_px, eps) — both terms in px, so it is dimensionally consistent and comparable across patterns and axes.
  7. spearman = Spearman rank correlation of response_i vs applied_mm over all steps; clamped to [0, 1].
  8. cross_axis_leak = mean |dot(flow, d_perp_px)| / mean |dot(flow, d_hat_px)| over the ROI and steps, clamped to [0, 1].
  9. score = snr * spearman * (1 - min(leak, 0.5)).
  10. resolvable_step_mm = the smallest step_mm with |response_i| > floor_px, else null.
  11. saturation_mm = the first step where response stops increasing (monotonically decreasing after a rise), else null.

verdict, in precedence order:

  • ambiguous — the tracker's reference_texture_known_answer failed for this scale: a known shift applied to the actual rendered marker was not recovered. For a periodic marker this is expected once the cell approaches the DIS patch size — DIS locks onto the period and reports the shift modulo that period — and it means the measured direction and magnitude are not trustworthy, so no score can promote this pattern. This takes precedence over everything below.
  • resolvable — gain_px_per_mm is not null and snr >= 3.
  • marginal — gain_px_per_mm is not null and snr < 3.
  • unresolvable — gain_px_per_mm is null.

overall.score = mean of score over scales and axes. overall.verdict = resolvable only if every scale and axis is resolvable; ambiguous if any scale is ambiguous; otherwise the worst remaining verdict.

Measured evidence for why ambiguous must exist (estimator, 500x500, probe scale 13.43 px/mm, DIS medium, dis_mode=1): a 2 px known shift measured 8.17 px on a 3.11 px checker cell (aliased onto the 6.22 px period), 0.0 px on a 1 px cell, -19.8 px on 3.11 px grid lines, but exactly 2.000 px on a 12.4 px cell and exactly 2.000 px on a broadband random texture. A periodic pattern aliases a sub-period shift onto the wrong period; a broadband one does not. That is the whole reason the anisotropy and periodicity of the marker matter, and it is measured, not assumed.

The floor is not a noise model

flow_report.json's scales[].null.floor_px is measured as reference vs a re-render at render seed + 1: Cycles antialiasing jitter, ~5 grey levels RMS on half the pixels and scaling with edge contrast. It is labelled floor_basis: "render_seed_reproducibility" and must not be used to rank patterns. Measured on this bench:

  • its ordering reverses under additive noise (render-seed: nx56 < nx80 < nx31; σ=2: nx31 < nx56 < nx80);
  • it understates the real idle floor, coarse patterns most.

The calibrated replacement is:

  • design.yaml probe.noise: idle_footage (real no-contact recording), roi_fraction, max_frames, sigmas, draws, seed, models, scales.
  • floorcal.measure_idle_floor: runs the same FixedReferenceFlow on idle footage and reports floor_px_short (consecutive frames, noise), floor_px_long (vs frame 0, noise + drift), dominant_period_px, floor_per_period_short|long and drift_ratio. A ratio near 1 means the clip is idle and the floor is frame-independent white noise. On data/dd02-0703/190.mp4 (1280x720): 0.2267 px, period 25.5 px, 0.0089 floor per period, ratio 0.98.
  • flow.noise_model_pass: additive white Gaussian applied to both the reference and the displaced frames, so the signal and the floor come from one model. Written to each scale as noise_models (basis, draws, sigmas, per-axis gain_px_per_mm, floor_px, floor_samples).
  • sensitivity.attach_noise_models: adds response_retention (gain(σ)/gain(clean)) and snr_noise (same-model floor) per axis, and emits a model_disagreement note when the response moves more than 15%.
  • sensitivity.noise_calibration: top-level noise_calibration block with the real numbers, cached as idle_floor.json keyed by the source's size and mtime. Skip with --no-calibration.

Schema: flow-report-v2 and sensitivity-report-v3. All additions are additive; a report without noise_model is still valid and means the render-seed model only.

Consequence for the ranking

Under the calibrated model at the render scale, radius_mm=0.5, dis_mode=1, nx=80 has the highest clean response on both axes (x 9.10 vs 4.74 px/mm; y 11.69 vs 10.90) and the best retention (99%/84% at σ=4/16 against nx=31's 72%/59%). The earlier "nx=31 wins" and "the cell must match the effective window" conclusions were artefacts of the render-seed floor and are retracted; the window ratio remains a useful diagnostic, not a validated law. The process scale and the radius_mm=1.5 block are not yet re-ranked.

Out-of-scope / next hardware ask

The idle clip gives only one pattern, so it validates the floor's magnitude and mechanism, not its pattern trend. A deliberate no-contact recording at the shipped configuration would let the trend itself be measured, and is the natural next ask.

optimizer_report.json

{
  "schema": "orisys-sensor-design/optimizer-report-v1",
  "generated_at": "<iso8601 utc>",
  "search": {"grid": [15, 20, 24, 28, 31, 36, 42, 48, 56, 64, 80],
             "seeds": [7], "render_resolution": [250, 250],
             "candidates": 11, "objective": "sensitivity_report.overall.score"},
  "candidates": [{"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
                  "verdict": "marginal", "cell_px": 8.01,
                  "report": "<abs>", "texture_sha256": "..."}],
  "ranked": ["31@7", "36@7"],
  "winner": {"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
             "confirmed_at_resolution": [500, 500], "confirmed_score": 0.39,
             "report": "<abs>"},
  "notes": ["Ranked by the measured score under a fixed estimator and camera; this is not a global optimum."]
}

The optimizer searches (nx, ny, seed). Candidates are enumerated in a deterministic sorted order so two runs produce byte-identical rankings. Each candidate regenerates the marker texture, then runs deform -> render -> track -> sensitivity at the search resolution. Only the winner is re-run at the full probe.resolution and its confirmed score recorded.

The search resolution must be the resolution you care about

A reduced search resolution answers a different question and must not be used to rank patterns. Measured on the real run, searching at 250x250 and confirming at 500x500 produced a winner that the real resolution disagrees with:

pattern search score @250 render score @500 overall
nx=48 (search winner) 8.94 20.95 15.39
nx=80 (shipped) 1.77 (ranked 10th of 10) 36.77 22.22

At 250x250 the nx=80 cell is 1.53 px, below the DIS-patch Nyquist limit, so it scored near the bottom; at 500x500 the same cell is 3.06 px and it scored top. The shipped pattern is therefore the best of the grid at the real resolution and one of the worst at the processed one — and a 250-based search excludes it. That is physics (the two scales genuinely resolve different cells), not a bug, but it means the default search_resolution must be probe.resolution, not a reduced size. Rendering 31 frames at 500x500 costs about 14 s, so a full-resolution search over the default grid is only a few minutes and there is no reason to trade correctness for it.

Two further consequences for the report:

  • Record search_resolution and confirm_resolution, and mark the winner's confirmed_score, so a reader can see whether the search and the confirm measured the same thing. A large gap between them means the search resolution was wrong.
  • Rank and report candidates per scale as well as by the mean. "Best" is scale-dependent here: nx=80 wins the render scale while nx=48 balances both. overall.score is the mean over scales, which is a deliberate modelling choice, not a neutral one, and the notes must say so.

CLI surface

orisys-design deform      <revision_dir> [--output-dir DIR] [--radius-mm F]
                          [--falloff MODE] [--step-mm F] [--max-mm F]
                          [--group NAME] [--mesh NAME] [--axes x,y,z]
orisys-design render      <revision_dir> [--output-dir DIR] [--resolution WxH]
                          [--samples N] [--uv-map NAME] [--texture PATH] [--seed N]
orisys-design track       <revision_dir> [--output-dir DIR]
                          [--process-resolution WxH] [--no-process-scale]
orisys-design sensitivity <revision_dir> [--output-dir DIR] [--skip-deform]
                          [--skip-render] [--skip-track] [--radius-mm F] [...]
orisys-design optimise    <revision_dir> [--output-dir DIR] [--grid LIST]
                          [--seeds LIST] [--search-resolution N] [--top N] [--dry-run]

Default --output-dir is build/probe/<sensor>/<revision>.

All argparse wiring lives in cli.py, owned by the Lead. The feature modules expose only plain entry points and must not add their own argument helpers or duplicate cmd_* functions:

  • deform.py: resolve_probe, probe_overrides_from_args, frame_plan, build_deform_job, write_deform_job, resolve_probe_output_dir
  • render.py: resolve_marker_texture, build_render_job, write_render_job, render_manifest_path
  • flow.py: build_track_job, write_track_job, run_track_job
  • sensitivity.py: run_sensitivity(settings, output_dir, *, manifest_path=None, flow_report_path=None, progress=print) — scoring only; it must not re-run the deform/render/track stages, because cli.py already chains them (and --skip-* exists precisely to reuse their artifacts)
  • optimise.py: run_optimise(paths, settings, *, grid, seeds, search_resolution, top, dry_run=False, runner=None, progress=print). The per-candidate runner is an injected callable; cli.py supplies one that runs the real stages, and tests supply a fake. Do not resolve stage functions by inspect or by guessing module layout.

Exit codes, matching the existing convention:

Code Meaning
0 success
1 scene / metadata / geometry error
2 config error, or a missing optional dependency
3 a required artifact is missing or empty
4 (reserved; not used by this pipeline)

Failure modes that must fail loudly

  • Probe group missing, empty, or not on the requested mesh.
  • The basis is not orthonormal within 1e-6.
  • A displacement vector cannot be expressed in the basis.
  • The marker texture PNG is missing or empty.
  • view_transform is not Standard (AgX or Filmic silently destroys marker values).
  • Frame count/order in frames/ disagrees with the plan, or a PNG is missing/empty.
  • reference_deterministic is false.
  • The tracker's known-answer test fails.
  • cv2 is unavailable -> exit 2 with python -m pip install -e .[probe].
  • The probe ROI falls outside the frame at the process scale -> warn and truncate, do not silently report a full-ROI number.

Non-goals

Large-deformation robustness beyond max_mm; signed/bidirectional sweeps; per-vertex normal indentation; refraction through shell/rubber-1; wiring the SDK processed image pipeline (resolution_stitch, source_scale) into the estimator; absolute force calibration. Sensitivity here is a relative ranking under a fixed estimator and camera, not a force measurement.

Reference in New Issue View Git Blame Copy Permalink