orisys-tech/orisys-sensor-design
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
59f48d6d06ab28e07174573430022ed1a7814a94
orisys-sensor-design/docs/probe-pipeline.md
T
ykcheungab c377c14474 1. added blender deformation probe for marker size picking and optimization 
2. added unwarp map export in current cli, don't need to generate by precompute in sensor running
2026-09-25 15:44:44 +08:00

764 lines
35 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Probe sensitivity pipeline — frozen interfaces
This document is the **interface freeze** for the probe sensitivity bench. Four
workstreams code against it in parallel. Treat every schema, path, and name here as
a contract: if it must change, the Lead changes this file first and tells everyone.
Scope: deform the sensing surface through a known displacement, render what
`cam_sensor` sees, measure optical flow against the zero-displacement reference,
and report a ranked, auditable sensitivity score per marker pattern.
Measured ground truth for `sensors/finger-xynova/r01` (from `geometry.blend`,
read-only; the tool never writes the `.blend`):
| Fact | Value |
|---|---|
| `sensing_surface` | 2508 verts, 4834 polys, UV maps `compute`, `produce`; no modifiers, no shape keys |
| `probe` vertex group | 6 verts `[215, 216, 218, 219, 327, 329]`, weight 1.0 |
| Probe patch | 0.755 x 1.018 mm, flat to 7.6 um, all 6 normals within 2.6 deg of their mean |
| Authoring unit | 1 Blender unit = 1 mm |
| Camera | `cam_sensor` PERSP, lens 0.5469, sensor_width 1.4, fit HORIZONTAL |
| Render | 500x500, CYCLES, PNG RGBA |
| Projection | mesh spans x 8.13..491.88, y 33.13..397.59 (fully inside) |
| Scale | **13.430 px/mm** (p50) -> 0.01 mm = **0.134 px**, 0.1 mm = **1.34 px** |
| Mesh edges | min 0.0504, p50 0.6285, p90 0.991, max 1.816 mm |
| Falloff at r=0.5 mm | **only 5 of 2502** non-probe verts (0.285 x1, 0.4161..0.4168 x4; next 0.796) |
| DIS patch (8 px) | 0.5957 mm at 500x500; an 80x80 marker cell is 0.2313 mm = 3.11 px |
| Blend sha256 | `1C8A050B8490B6832D3A9F269DFEF3DAFBF94DBFE3E943B3956BE1104A50D9C9` |
## Pipeline
```
design.yaml [probe:]
|
+-- deform.py (host, py310, no bpy) -> _deform_job.json
| probe_deform.py (Blender) -> probe_frames.npz + deform_report.json
|
+-- render.py (host, no bpy) -> _render_job.json
| probe_render.py (Blender) -> frames/*.png + render_manifest.json
|
+-- flow.py (host, cv2) -> _track_job.json, flow/, flow_report.json
| (or is called in-process by cli.py)
|
+-- sensitivity.py -> sensitivity_report.json
+-- optimise.py -> optimizer_report.json
```
`deform.frame_plan()` is the **single ordering authority**. Every job carries the
resolved frame list so render and analysis cannot disagree about frame order.
All artifacts live under:
```
build/probe/<sensor>/<revision>/
```
resolved the same way `texture.py` resolves `build/textures/...` (see
`texture_revision_rel`; reuse it, do not re-invent it).
## Frame plan
Positive-only sweep, axes in the order `probe.axes` (default `x, y, z`),
steps `step_mm`, `2*step_mm`, ... up to and including `max_mm`.
| index | name | axis | step_mm |
|---|---|---|---|
| 0 | `000_reference` | `null` | 0.0 |
| 1 | `001_x_+0.01mm` | `x` | 0.01 |
| ... | ... | ... | ... |
| 10 | `010_x_+0.10mm` | `x` | 0.10 |
| 11..20 | `0NN_y_+0.0Xmm` | `y` | ... |
| 21..30 | `0NN_z_+0.0Xmm` | `z` | ... |
`F = 1 + len(axes) * round(max_mm / step_mm)` = **31** for the default settings.
PNG for frame `i` is `frames/<name>.png`.
The displacement for frame `i` is the frame's unit axis of the **probe-average
basis** (frozen below) scaled by `step_mm`. All 6 probe vertices move by exactly
that vector; every other vertex moves by `weight * vector`.
## The probe-average basis
Built from the area-independent average of the 6 probe vertex normals, then made
orthonormal so the three sweeps are exactly orthogonal and comparable:
```
z = normalize(sum(normal_i for i in probe)) # surface normal
x = normalize(cross(Vector((0,0,1)), z)) # fall back to (1,0,0) if degenerate
y = cross(z, x)
origin = mean(co_i for i in probe)
```
Measured for Xynova (orthonormal to 1e-10):
```
z = (-0.000027, -0.061749, 0.998092)
x = ( 1.000000, -0.000431, 0.000000)
y = ( 0.000430, 0.998092, 0.061749)
origin = (74.76961, 16.21550, 120.80202)
```
An axis vector for displacement is `{'x': x, 'y': y, 'z': z}[axis] * step_mm`
in the **mesh object's local space**. Since
`sensing_surface.matrix_world.to_scale() == (1,1,1)` and
`meters_per_source_unit == 0.001`, local units are mm and the vector is in mm.
## Proportional falloff
We deliberately do **not** use `bpy.ops.transform.translate(use_proportional_edit=True)`:
that operator is viewport-modal and its `proportional_size` is in screen pixels
relative to a 3D-view region, which does not exist under `blender -b` and would not
be reproducible. The falloff is computed explicitly, in mm, in a bpy-free module.
Semantics match Blender's proportional editing: the falloff distance is measured
from the **original** positions of the selected (probe) vertices, so the weight set
is constant across all frames.
```
t = d / radius_mm # d = min distance to any probe vertex
smooth: w = 1 - (3t^2 - 2t^3) # Blender's "Smooth" curve; 1 at t=0, 0 at t>=1
linear: w = 1 - t
sharp: w = (1 - t)^2
constant: w = 1
w = 0 whenever t >= 1, and w = 1 for probe members
```
## `design.yaml` `probe:` block
```yaml
probe:
mesh: sensing_surface
camera: cam_sensor
group: probe
uv_map: compute
texture: patch_texture_compute.png
radius_mm: 0.5
falloff: smooth
step_mm: 0.01
max_mm: 0.10
axes: [x, y, z]
resolution: [500, 500]
process_resolution: [250, 250]
tracker:
algorithm: dis
dis_preset: medium
dis_mode: 1
render:
engine: CYCLES
samples: 16
denoise: false
view_transform: Standard
interpolate: Closest
seed: 0
```
Built-in fallbacks live in `deform.DEFAULT_PROBE` and are merged
`DEFAULT_PROBE <- design.yaml probe <- CLI overrides`, exactly like
`texture.merge_texture_patch`. Unknown keys are an error. Validation returns one
message per problem and raises `ProbeConfigError`.
## Job files (stdlib JSON; Blender never reads YAML)
### `_deform_job.json`
```json
{
"schema": "orisys-sensor-design/probe-deform-job-v1",
"revision_dir": "<abs>",
"source_blend": "<abs>",
"output_dir": "<abs>",
"objects": {"mesh": "sensing_surface", "group": "probe"},
"settings": {
"radius_mm": 0.5,
"falloff": "smooth",
"step_mm": 0.01,
"max_mm": 0.10,
"axes": ["x", "y", "z"]
},
"frames": [
{"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0},
{"index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01}
],
"artifacts": {"frames": "<abs>/probe_frames.npz", "report": "<abs>/deform_report.json"}
}
```
### `_render_job.json`
```json
{
"schema": "orisys-sensor-design/probe-render-job-v1",
"revision_dir": "<abs>",
"source_blend": "<abs>",
"output_dir": "<abs>",
"objects": {"mesh": "sensing_surface", "camera": "cam_sensor"},
"settings": {
"uv_map": "compute",
"texture": "<abs path to the marker PNG>",
"width": 500, "height": 500,
"radius_mm": 0.5,
"engine": "CYCLES", "samples": 16,
"denoise": false, "view_transform": "Standard",
"interpolate": "Closest", "seed": 0
},
"frames": ["<same frames array as the deform job>"],
"null_frame": {"name": "reference_null", "seed": 1},
"artifacts": {"frames_npz": "<abs>/probe_frames.npz", "report": "<abs>/render_manifest.json"}
}
```
`null_frame` is a second render of the reference at a different Cycles seed. It is
never passed to the tracker: it exists only to measure the estimator's zero-input
floor.
### `_track_job.json`
```json
{
"schema": "orisys-sensor-design/probe-track-job-v1",
"output_dir": "<abs>",
"settings": {
"axes": ["x", "y", "z"],
"step_mm": 0.01, "max_mm": 0.10,
"radius_mm": 0.5,
"process_resolution": [250, 250],
"tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1}
},
"scales": [
{"name": "render", "width": 500, "height": 500, "resample": false},
{"name": "process", "width": 250, "height": 250, "resample": true}
],
"artifacts": {"manifest": "<abs>/render_manifest.json", "strategy": "<abs>/strategy_probe.json"}
}
```
## `probe_frames.npz`
The contract between the deformation and rendering workstreams.
| Key | Shape / dtype | Meaning |
|---|---|---|
| `base_vertices` | `(V, 3) float64` | original object-local coords, all V vertices |
| `vertices` | `(F, V, 3) float64` | deformed coords per frame; index 0 = reference |
| `frame_names` | `(F,) <U...` | frame name, matching the plan |
| `frame_axes` | `(F,) <U4` | `null` for the reference, else `x`/`y`/`z` |
| `frame_steps_mm` | `(F,) float64` | applied displacement magnitude in mm |
| `weights` | `(V,) float64` | falloff weight per vertex (constant across frames) |
| `origin` | `(3,) float64` | probe centroid, object-local |
| `basis` | `(3, 3) float64` | rows are `x`, `y`, `z` unit axes, object-local |
| `probe_indices` | `(P,) int32` | member vertex indices |
`probe_render.py` must assert that `len(vertices) == len(frame_names)` and that the
plan in `_render_job.json` matches `frame_names` element-wise, then fail with exit 1
if not. It writes each frame to the mesh and renders; it never re-derives weights.
## `deform_report.json`
```json
{
"schema": "orisys-sensor-design/probe-deform-report-v1",
"generated_at": "<iso8601 utc>",
"source_blend": {"path": "<abs>", "sha256": "..."},
"mesh": "sensing_surface",
"group": "probe",
"probe_indices": [215, 216, 218, 219, 327, 329],
"origin": [74.76961, 16.2155, 120.80202],
"basis": [[1.0, -0.000431, 0.0], [0.00043, 0.998092, 0.061749],
[-0.000027, -0.061749, 0.998092]],
"basis_orthonormal": true,
"normal_angle_max_deg": 2.553,
"flatness_mm": 0.007611,
"radius_mm": 0.5,
"falloff": "smooth",
"vertex_count": 2508,
"edge_p50_mm": 0.628501,
"weights": {"min": 0.0, "max": 1.0, "members_at_one": 6,
"partial": 5, "unaffected": 2497},
"affect_bands": {"1.0": 6, "0.3-0.5": 1, "0.05-0.1": 4},
"frames": [{"index": 0, "name": "000_reference", "axis": null,
"step_mm": 0.0, "max_delta_mm": 0.0, "moved_vertices": 0}],
"warnings": ["radius 0.5 mm reaches only 5 vertices outside the probe group ..."]
}
```
`moved_vertices` is the number of vertices whose displacement is non-zero **in that
frame**: `0` for the reference, and `6 + partial` (= 11 on Xynova) for any non-zero
step. It is not the probe member count, and it is not the same for every frame —
reporting 6 on the reference would contradict `max_delta_mm: 0.0`. The `weights`
block remains the authority for how many vertices are partially weighted.
`warnings` is the honesty channel. Emit one whenever
`affected (partial) < 4 * len(probe_indices)`, or `radius_mm < edge_p50_mm`.
## `render_manifest.json`
```json
{
"schema": "orisys-sensor-design/probe-render-manifest-v1",
"generated_at": "<iso8601 utc>",
"source_blend": {"path": "<abs>", "sha256": "..."},
"texture": {"path": "<abs>", "sha256": "..."},
"uv_map": "compute",
"width": 500, "height": 500,
"settings_applied": {"engine": "CYCLES", "samples": 16, "denoise": false,
"view_transform": "Standard", "look": "None",
"exposure": 0.0, "interpolate": "Closest", "seed": 0},
"mesh_px_per_mm": {"min": 10.499, "p05": 10.946, "p50": 13.430,
"p95": 22.886, "max": 38.594,
"basis": "per-mesh-edge projected px / 3D mm, all 7341 edges"},
"probe_px_per_mm": {"p50": 13.248, "x": 13.248, "y": 17.670, "z": 1.519,
"basis": "probe-local: |centroid_px(step) - centroid_px(0)| / step_mm, per axis at its largest step"},
"probe_roi_px": {"u": 0.0, "v": 0.0, "radius": 0.0},
"reference_index": 0,
"reference_deterministic": true,
"frames": [{"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0,
"png": "frames/000_reference.png", "sha256": "...",
"width": 500, "height": 500,
"centroid_px": [249.61, 214.38]}],
"null": {"name": "reference_null", "seed": 1,
"png": "frames/reference_null.png", "sha256": "..."}
}
```
`reference_deterministic` reports the result of rendering frame 0 twice in the same
session with the same seed and comparing the saved bytes. It must be `true`; if it
is not, the render is not reproducible and the CLI must fail.
**Compare decoded pixels, not file bytes, and allow a 1-LSB tolerance.** This was
established the hard way: with literal whole-file byte equality the check rejected
candidate `nx=36` on the real run, and PNG chunk analysis showed why — the two files
had **identical chunk structure** (the only `tEXt` chunk was the deterministic
`cycles.ViewLayer.samples`) yet their decompressed pixel data differed in **2 bytes of
250,250**, at two scanline offsets. That is Cycles CPU thread-order floating-point
nondeterminism tipping a boundary pixel by one 8-bit step, not a metadata artefact and
not a defect in the render.
A gate that fails on a 1-LSB difference is not measuring reproducibility; it is
measuring thread scheduling, and because it is **content-dependent** it silently
disqualifies some patterns and biases a pattern search. So:
- Strip only the known-volatile `tEXt` chunks as before (`Date`, `Time`, `RenderTime`,
`cycles.ViewLayer.*_time`); keep image chunks byte-for-byte.
- Decode both PNGs and require the same shape, `max |delta| <= 1` per 8-bit channel,
and a differing-pixel fraction at or below **0.05%**. Record the measured
`max_abs_delta` and `differing_fraction` in the manifest either way.
- Fail loudly when a tolerance is exceeded, so real nondeterminism still stops the run.
- Never compare raw file bytes: PNG compression and metadata are not the measurement.
`probe_roi_px` is the probe centroid projected to pixels plus the radius in pixels
that corresponds to `radius_mm` (`radius = radius_mm * probe_px_per_mm.p50`), all
measured from the actual projection (not assumed). The tracker uses it to build its
ROI, and the ROI is **fixed at the reference centroid** for every frame — a moving
ROI would bias the ROI mean as the probe moves, which is exactly the signal we are
trying to measure.
### Two different pixel scales, and why they must not be confused
- `mesh_px_per_mm` is a **mesh-wide** statistic: for every mesh edge, its projected
length in pixels divided by its 3D length in mm, summarised over all 7341 edges.
It describes how the whole surface is sampled in the camera. It is **not** the
gain to convert the probe's measured flow into millimetres.
- `probe_px_per_mm` is the **probe-local** scale, and it is the one the scorer must
use: `|centroid_px(step) - centroid_px(0)| / step_mm`, per axis, i.e. how many
image pixels the probe actually moves per millimetre applied. It is anisotropic,
and `z` (motion along the surface normal) is expected to be **much smaller** than
x and y because normal motion is largely out of the image plane. That is a real
physical property of the sensor, not a defect.
Because the `z` scale can approach zero, dividing a measured response by it is
ill-conditioned. The scorer therefore reports gain in **px per mm** as the primary
quantity and only derives mm per mm when the axis scale clears
`sensitivity.min_local_scale_px_per_mm` (default **1.0**; a module constant in
`sensitivity.py`, deliberately not a `design.yaml` key so it cannot be tuned into
silence). Never divide by a scale below that; report `null` and warn instead. A large
"mm/mm" gain produced by dividing by a near-zero scale would be an artefact, not a
sensitivity.
The default of 1.0 is validated by real data rather than chosen by feel. Measured
per-axis probe scales: **x 13.248, y 17.670, z 1.519 px/mm at the render scale**, and
**x 6.624, y 8.835, z 0.760 px/mm at the process scale**. So 1.0 discriminates exactly
where it should on this revision: render-scale `z` (1.519) is admitted and yields a
derived `gain_mm_per_mm` of 0.445 backed by 7 steps above the floor at `snr` 4.2,
while process-scale `z` (0.760) is correctly gated to `null`.
Note what this implies about the earlier worry that `z` would be unmeasurable: at the
**process** scale a 0.1 mm normal displacement is only 0.076 px against a 0.19 px
floor, so it genuinely is unmeasurable there; at the **render** scale it is 0.152 px
against a 0.028 px floor, so it is measurable from about `0.04 mm` upward. The gate is
a guard against dividing by a degenerate scale, **not** a substitute for the measured
`snr`, `points_above_floor` and `resolvable_step_mm`, which are the quantities that
decide whether an axis is useful. Report all of them; do not suppress a number that
the data supports.
### What actually limits the measurement (measured, not assumed)
Do **not** attribute the response deficit to aliasing, noise, or ROI dilution. The
render-scale deficit is **spatial resolution of the measurement region**, established
by these measurements on the real frames:
- The real `x +0.10 mm` flow radial profile is essentially uniform inside the ROI
(`-0.9162`, `-0.9096`, `-0.8725` px at r 0-2, 2-4, 4-6.6), with `|flow|` p5..p100 of
`0.81..0.93` px. So it *is* a coherent translation, not noise and not a diluted
average: the motion is simply shrunk to about `0.67x` of the applied `1.3248 px`.
- Injecting a synthetic **local rigid translation** of `1.3248 px` into the reference
and measuring it over the same ROI recovers only **17%** at a patch radius of
6.6 px (equal to the ROI), then **33%** at r 8, **51%** at r 10, **63%** at r 13,
**81%** at r 20 and **95%** at r 40.
- Therefore the attenuation is caused by the moving probe patch plus its falloff taper
spanning only about **13 px, i.e. 1-2 DIS patches (8 px)**, so DIS regularizes
toward the static background. At the process scale the same region is about 6.5 px,
under one patch, and the response collapses while the forward-backward error grows
from 1.6 to 8.8 (x) and 15.8 (y).
Two consequences the report must state rather than bury:
1. **The measured gain is confounded with the ROI size.** It grows strongly with
`radius_mm` because a larger radius gives DIS more moving texture to lock onto.
Relative ranking of marker patterns is valid **only** while `radius_mm`, the ROI and
the scale are held fixed across candidates — which is why `run_optimise` varies
`nx`/`ny`/`seed` and never the radius.
2. **"Below one pixel" and "below the estimator floor" are different statements and
must not be conflated.** The measured zero-input floor is **0.0292 px at the render
scale**, so a `0.01 mm` step (`0.1325 px`) is about **4.5x above the floor**: the
ROI-averaged response genuinely resolves single steps at 500x500, with monotone
per-step responses (`0.0810, 0.1316, 0.1971, ... 0.8859` px). The earlier
expectation that the sub-pixel step would be lost in the floor is **not** supported
by the measurement; what is lost is *per-pixel* precision, not the ROI mean. Report
the floor and the step in pixels and let the numbers speak.
Each `frames[]` entry also carries `centroid_px`: the probe centroid projected to
pixels **for that frame's deformation**, as full-precision floats (not rounded).
This is the authoritative source for the image-space displacement direction the
sensitivity stage needs. It is a small number of pixels (1.34 px across the whole
0.1 mm sweep at 500x500), which is why it is recorded as a float and why the scorer
regresses over all steps rather than differencing a single pair.
## `flow_report.json`
```json
{
"schema": "orisys-sensor-design/flow-report-v1",
"generated_at": "<iso8601 utc>",
"tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1,
"finest_scale": 1, "source": "design.yaml probe.tracker"},
"known_answer": {"passed": true, "cases": [
{"dx": 2, "dy": 0, "measured_u": 2.0, "measured_v": 0.0, "tol": 0.25},
{"dx": 0, "dy": 2, "measured_u": 0.0, "measured_v": 2.0, "tol": 0.25}]},
"reference_texture_known_answer": {
"passed": false, "gating": false,
"note": "a known shift applied to the real rendered marker; a periodic "
"pattern legitimately fails this when its cell approaches the DIS "
"patch, so it is a diagnostic, not a plumbing gate",
"cases": [{"dx": 2, "dy": 0, "measured_u": 8.17, "expected_u": 2.0}]},
"reference_self": {"max_abs_flow_px": 0.0},
"scales": [{
"name": "render", "width": 500, "height": 500, "resample": false,
"px_per_mm": 13.43, "mm_per_px": 0.07446,
"frames": [{
"index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01,
"roi": {"u": 249.6, "v": 214.4, "radius": 6.7, "pixels": 141},
"flow": {"mean_u": 0.02, "mean_v": -0.01,
"mean_mag": 0.03, "p50_mag": 0.02, "p95_mag": 0.08, "max_mag": 0.31,
"coverage": 0.41},
"response_proj_px": 0.02,
"response_mm": 0.0015,
"forward_backward_error": 1.87}
],
"null": {"floor_px": 0.05, "floor_source": "reference_null"}
}]
}
```
Per-frame flow arrays are also written to `flow/<scale>/<name>.npy` as
`float32 (H, W, 2)` so the sensitivity stage and any later inspection can reuse them
without re-running DIS.
`response_mm` and `response_proj_px` are informational. **Pixels are authoritative**
(`response_proj_px`); `response_mm` is derived through a single isotropic scalar
scale (`px_per_mm`, the probe-local p50) and therefore *understates or overstates*
the true conversion on an anisotropic axis — notably `z`. The scorer must use
`response_proj_px` plus the per-axis local scale from `render_manifest.json` and must
not consume `response_mm`. Each scale records `px_per_mm_source` naming the exact
manifest field it came from, so the provenance of the scalar is auditable.
## `sensitivity_report.json`
```json
{
"schema": "orisys-sensor-design/sensitivity-report-v1",
"generated_at": "<iso8601 utc>",
"revision": "sensors/finger-xynova/r01",
"texture": {"path": "<abs>", "sha256": "...", "grid": [80, 80],
"cell_mm": [0.23125, 0.205]},
"probe": {"group": "probe", "origin": [...], "basis": [...],
"radius_mm": 0.5, "falloff": "smooth", "affected_vertices": 5},
"tracker": {"dis_preset": "medium", "dis_mode": 1, "patch_px": 8},
"scales": [{
"name": "render", "px_per_mm": 13.43,
"displacement_px_per_step": 0.1343,
"cell_px": 3.11,
"cell_over_patch_px": 0.389,
"null_floor_px": 0.05,
"null_floor_basis": "render_seed_reproducibility",
"axes": [{
"axis": "x", "direction_px": [0.999, -0.031],
"direction_source": "centroid_regression",
"points": [{"step_mm": 0.01, "applied_px": 0.134, "response_px": 0.02,
"above_floor": false}],
"summary": {"gain_px_per_mm": 6.2, "gain_mm_per_mm": 0.46,
"scale_px_per_mm": 13.42, "snr": 0.42, "spearman": 0.95,
"resolvable_step_mm": null, "saturation_mm": null,
"cross_axis_leak": 0.18}
}],
"score": 0.11,
"verdict": "unresolvable"
}],
"overall": {"score": 0.11, "verdict": "unresolvable",
"best_scale": "render", "best_axis": "z"},
"warnings": ["0.01 mm is 0.134 px at 500x500; a single step is below the estimator floor"],
"notes": []
}
```
### Scoring (every input measured; no assumed constants)
For one scale and one axis:
1. `d_hat_px` — unit image-space direction of the applied displacement, from
`render_manifest.json`'s per-frame `centroid_px`. Regress `centroid_px` on
`step_mm` over every frame of that axis and normalize the slope vector
`(du/dstep, dv/dstep)`; this is preferred over a single 0 -> `+max_mm` finite
difference because the whole sweep is only ~1.3 px. Fall back to the two-point
finite difference when the regression is degenerate, and to the ROI-mean flow at
the `+max_mm` frame when `centroid_px` is absent. Record which was used as
`direction_source` per axis.
2. `response_i` = mean over ROI pixels of `dot(flow_i, d_hat_px)`, in px. Sign kept.
3. `floor_px` = p95 of `|response|` measured on `reference_null`, projected on the
same `d_hat_px`. Reported per axis.
4. `gain_px_per_mm` = OLS slope of `response_px` vs `applied_mm`, over the steps with
`|response_i| > floor_px`. Requires >= 3 such points, else `null`. **This is the
primary sensitivity**: measured pixels of probe response per millimetre applied.
5. `scale_px_per_mm` = `probe_px_per_mm[axis]`. Then `gain_mm_per_mm` =
`gain_px_per_mm / scale_px_per_mm`, reported **only** when
`scale_px_per_mm >= min_local_scale_px_per_mm` (default 1.0); otherwise `null`
plus a warning. Never divide by a near-zero scale: for the `z` axis that would
manufacture an enormous gain no camera could observe.
6. `snr` = `|gain_px_per_mm| * max_mm / max(floor_px, eps)` — both terms in px, so it
is dimensionally consistent and comparable across patterns and axes.
7. `spearman` = Spearman rank correlation of `response_i` vs `applied_mm` over all
steps; clamped to `[0, 1]`.
8. `cross_axis_leak` = mean `|dot(flow, d_perp_px)|` / mean `|dot(flow, d_hat_px)|`
over the ROI and steps, clamped to `[0, 1]`.
9. `score` = `snr * spearman * (1 - min(leak, 0.5))`.
10. `resolvable_step_mm` = the smallest `step_mm` with `|response_i| > floor_px`, else
`null`.
11. `saturation_mm` = the first step where `response` stops increasing
(monotonically decreasing after a rise), else `null`.
`verdict`, in precedence order:
- **`ambiguous`** — the tracker's `reference_texture_known_answer` failed for this
scale: a known shift applied to the *actual rendered marker* was not recovered.
For a **periodic** marker this is expected once the cell approaches the DIS patch
size — DIS locks onto the period and reports the shift modulo that period — and it
means the measured direction and magnitude are not trustworthy, so no score can
promote this pattern. This takes precedence over everything below.
- `resolvable` — `gain_px_per_mm` is not null and `snr >= 3`.
- `marginal` — `gain_px_per_mm` is not null and `snr < 3`.
- `unresolvable` — `gain_px_per_mm` is null.
`overall.score` = mean of `score` over scales and axes. `overall.verdict` =
`resolvable` only if every scale and axis is `resolvable`; `ambiguous` if any scale is
`ambiguous`; otherwise the worst remaining verdict.
Measured evidence for why `ambiguous` must exist (estimator, 500x500, probe scale
13.43 px/mm, DIS medium, `dis_mode=1`): a **2 px** known shift measured
**8.17 px** on a 3.11 px checker cell (aliased onto the 6.22 px period), **0.0 px** on
a 1 px cell, **-19.8 px** on 3.11 px grid lines, but **exactly 2.000 px** on a
12.4 px cell and **exactly 2.000 px** on a broadband random texture. A periodic
pattern aliases a sub-period shift onto the wrong period; a broadband one does not.
That is the whole reason the anisotropy and periodicity of the marker matter, and it
is measured, not assumed.
## The floor is not a noise model
`flow_report.json`'s `scales[].null.floor_px` is measured as `reference` vs a re-render at
**render seed + 1**: Cycles antialiasing jitter, ~5 grey levels RMS on half the pixels and
scaling with edge contrast. It is labelled
`floor_basis: "render_seed_reproducibility"` and **must not be used to rank patterns**.
Measured on this bench:
- its ordering **reverses** under additive noise (render-seed: nx56 < nx80 < nx31;
σ=2: nx31 < nx56 < nx80);
- it **understates** the real idle floor, coarse patterns most.
The calibrated replacement is:
- **`design.yaml` `probe.noise`**: `idle_footage` (real no-contact recording),
`roi_fraction`, `max_frames`, `sigmas`, `draws`, `seed`, `models`, `scales`.
- **`floorcal.measure_idle_floor`**: runs the *same* `FixedReferenceFlow` on idle footage
and reports `floor_px_short` (consecutive frames, noise), `floor_px_long` (vs frame 0,
noise + drift), `dominant_period_px`, `floor_per_period_short|long` and `drift_ratio`.
A ratio near 1 means the clip is idle and the floor is frame-independent white noise.
On `data/dd02-0703/190.mp4` (1280x720): **0.2267 px**, period **25.5 px**,
**0.0089 floor per period**, ratio **0.98**.
- **`flow.noise_model_pass`**: additive white Gaussian applied to **both** the reference
and the displaced frames, so the signal and the floor come from one model. Written to
each scale as `noise_models` (`basis`, `draws`, `sigmas`, per-axis
`gain_px_per_mm`, `floor_px`, `floor_samples`).
- **`sensitivity.attach_noise_models`**: adds `response_retention`
(`gain(σ)/gain(clean)`) and `snr_noise` (same-model floor) per axis, and emits a
`model_disagreement` note when the response moves more than 15%.
- **`sensitivity.noise_calibration`**: top-level `noise_calibration` block with the real
numbers, cached as `idle_floor.json` keyed by the source's size and mtime. Skip with
`--no-calibration`.
Schema: `flow-report-v2` and `sensitivity-report-v3`. All additions are additive; a report
without `noise_model` is still valid and means the render-seed model only.
### Consequence for the ranking
Under the calibrated model at the render scale, `radius_mm=0.5`, `dis_mode=1`, nx=80 has
the **highest clean response on both axes** (x 9.10 vs 4.74 px/mm; y 11.69 vs 10.90) and
the **best retention** (99%/84% at σ=4/16 against nx=31's 72%/59%). The earlier
"nx=31 wins" and "the cell must match the effective window" conclusions were artefacts of
the render-seed floor and are retracted; the window ratio remains a useful diagnostic, not
a validated law. The process scale and the `radius_mm=1.5` block are not yet re-ranked.
## Out-of-scope / next hardware ask
The idle clip gives only one pattern, so it validates the floor's **magnitude** and
**mechanism**, not its pattern trend. A deliberate no-contact recording at the shipped
configuration would let the trend itself be measured, and is the natural next ask.
## `optimizer_report.json`
```json
{
"schema": "orisys-sensor-design/optimizer-report-v1",
"generated_at": "<iso8601 utc>",
"search": {"grid": [15, 20, 24, 28, 31, 36, 42, 48, 56, 64, 80],
"seeds": [7], "render_resolution": [250, 250],
"candidates": 11, "objective": "sensitivity_report.overall.score"},
"candidates": [{"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
"verdict": "marginal", "cell_px": 8.01,
"report": "<abs>", "texture_sha256": "..."}],
"ranked": ["31@7", "36@7"],
"winner": {"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
"confirmed_at_resolution": [500, 500], "confirmed_score": 0.39,
"report": "<abs>"},
"notes": ["Ranked by the measured score under a fixed estimator and camera; this is not a global optimum."]
}
```
The optimizer searches `(nx, ny, seed)`. Candidates are enumerated in a
**deterministic sorted order** so two runs produce byte-identical rankings. Each
candidate regenerates the marker texture, then runs deform -> render -> track ->
sensitivity at the search resolution. Only the winner is re-run at the full
`probe.resolution` and its confirmed score recorded.
### The search resolution must be the resolution you care about
A reduced search resolution **answers a different question** and must not be used to
rank patterns. Measured on the real run, searching at 250x250 and confirming at
500x500 produced a winner that the real resolution disagrees with:
| pattern | search score @250 | render score @500 | overall |
|---|---|---|---|
| `nx=48` (search winner) | **8.94** | 20.95 | 15.39 |
| `nx=80` (shipped) | **1.77** (ranked 10th of 10) | **36.77** | **22.22** |
At 250x250 the `nx=80` cell is 1.53 px, below the DIS-patch Nyquist limit, so it
scored near the bottom; at 500x500 the same cell is 3.06 px and it scored *top*. The
shipped pattern is therefore the best of the grid at the real resolution and one of
the worst at the processed one — and a 250-based search excludes it. That is physics
(the two scales genuinely resolve different cells), not a bug, but it means the
default `search_resolution` must be `probe.resolution`, not a reduced size. Rendering
31 frames at 500x500 costs about 14 s, so a full-resolution search over the default
grid is only a few minutes and there is no reason to trade correctness for it.
Two further consequences for the report:
- Record **`search_resolution` and `confirm_resolution`**, and mark the winner's
`confirmed_score`, so a reader can see whether the search and the confirm measured
the same thing. A large gap between them means the search resolution was wrong.
- Rank and report candidates **per scale** as well as by the mean. "Best" is
scale-dependent here: `nx=80` wins the render scale while `nx=48` balances both.
`overall.score` is the mean over scales, which is a deliberate modelling choice, not
a neutral one, and the notes must say so.
## CLI surface
```
orisys-design deform <revision_dir> [--output-dir DIR] [--radius-mm F]
[--falloff MODE] [--step-mm F] [--max-mm F]
[--group NAME] [--mesh NAME] [--axes x,y,z]
orisys-design render <revision_dir> [--output-dir DIR] [--resolution WxH]
[--samples N] [--uv-map NAME] [--texture PATH] [--seed N]
orisys-design track <revision_dir> [--output-dir DIR]
[--process-resolution WxH] [--no-process-scale]
orisys-design sensitivity <revision_dir> [--output-dir DIR] [--skip-deform]
[--skip-render] [--skip-track] [--radius-mm F] [...]
orisys-design optimise <revision_dir> [--output-dir DIR] [--grid LIST]
[--seeds LIST] [--search-resolution N] [--top N] [--dry-run]
```
Default `--output-dir` is `build/probe/<sensor>/<revision>`.
**All argparse wiring lives in `cli.py`, owned by the Lead.** The feature modules
expose only plain entry points and must not add their own argument helpers or
duplicate `cmd_*` functions:
- `deform.py`: `resolve_probe`, `probe_overrides_from_args`, `frame_plan`,
`build_deform_job`, `write_deform_job`, `resolve_probe_output_dir`
- `render.py`: `resolve_marker_texture`, `build_render_job`, `write_render_job`,
`render_manifest_path`
- `flow.py`: `build_track_job`, `write_track_job`, `run_track_job`
- `sensitivity.py`: `run_sensitivity(settings, output_dir, *, manifest_path=None,
flow_report_path=None, progress=print)` — **scoring only**; it must not re-run the
deform/render/track stages, because `cli.py` already chains them (and `--skip-*`
exists precisely to reuse their artifacts)
- `optimise.py`: `run_optimise(paths, settings, *, grid, seeds, search_resolution,
top, dry_run=False, runner=None, progress=print)`. The per-candidate `runner` is an
injected callable; `cli.py` supplies one that runs the real stages, and tests supply
a fake. Do not resolve stage functions by `inspect` or by guessing module layout.
Exit codes, matching the existing convention:
| Code | Meaning |
|---|---|
| 0 | success |
| 1 | scene / metadata / geometry error |
| 2 | config error, or a missing optional dependency |
| 3 | a required artifact is missing or empty |
| 4 | (reserved; not used by this pipeline) |
## Failure modes that must fail loudly
- Probe group missing, empty, or not on the requested mesh.
- The basis is not orthonormal within 1e-6.
- A displacement vector cannot be expressed in the basis.
- The marker texture PNG is missing or empty.
- `view_transform` is not `Standard` (AgX or Filmic silently destroys marker values).
- Frame count/order in `frames/` disagrees with the plan, or a PNG is missing/empty.
- `reference_deterministic` is false.
- The tracker's known-answer test fails.
- `cv2` is unavailable -> exit 2 with `python -m pip install -e .[probe]`.
- The probe ROI falls outside the frame at the process scale -> warn and truncate, do
not silently report a full-ROI number.
## Non-goals
Large-deformation robustness beyond `max_mm`; signed/bidirectional sweeps; per-vertex
normal indentation; refraction through `shell`/`rubber-1`; wiring the SDK processed
image pipeline (`resolution_stitch`, `source_scale`) into the estimator; absolute
force calibration. Sensitivity here is a **relative ranking under a fixed estimator
and camera**, not a force measurement.
Reference in New Issue View Git Blame Copy Permalink