ykcheungab
c377c14474
2. added unwarp map export in current cli, don't need to generate by precompute in sensor running
764 lines
35 KiB
Markdown
764 lines
35 KiB
Markdown
# Probe sensitivity pipeline — frozen interfaces
|
||
|
||
This document is the **interface freeze** for the probe sensitivity bench. Four
|
||
workstreams code against it in parallel. Treat every schema, path, and name here as
|
||
a contract: if it must change, the Lead changes this file first and tells everyone.
|
||
|
||
Scope: deform the sensing surface through a known displacement, render what
|
||
`cam_sensor` sees, measure optical flow against the zero-displacement reference,
|
||
and report a ranked, auditable sensitivity score per marker pattern.
|
||
|
||
Measured ground truth for `sensors/finger-xynova/r01` (from `geometry.blend`,
|
||
read-only; the tool never writes the `.blend`):
|
||
|
||
| Fact | Value |
|
||
|---|---|
|
||
| `sensing_surface` | 2508 verts, 4834 polys, UV maps `compute`, `produce`; no modifiers, no shape keys |
|
||
| `probe` vertex group | 6 verts `[215, 216, 218, 219, 327, 329]`, weight 1.0 |
|
||
| Probe patch | 0.755 x 1.018 mm, flat to 7.6 um, all 6 normals within 2.6 deg of their mean |
|
||
| Authoring unit | 1 Blender unit = 1 mm |
|
||
| Camera | `cam_sensor` PERSP, lens 0.5469, sensor_width 1.4, fit HORIZONTAL |
|
||
| Render | 500x500, CYCLES, PNG RGBA |
|
||
| Projection | mesh spans x 8.13..491.88, y 33.13..397.59 (fully inside) |
|
||
| Scale | **13.430 px/mm** (p50) -> 0.01 mm = **0.134 px**, 0.1 mm = **1.34 px** |
|
||
| Mesh edges | min 0.0504, p50 0.6285, p90 0.991, max 1.816 mm |
|
||
| Falloff at r=0.5 mm | **only 5 of 2502** non-probe verts (0.285 x1, 0.4161..0.4168 x4; next 0.796) |
|
||
| DIS patch (8 px) | 0.5957 mm at 500x500; an 80x80 marker cell is 0.2313 mm = 3.11 px |
|
||
| Blend sha256 | `1C8A050B8490B6832D3A9F269DFEF3DAFBF94DBFE3E943B3956BE1104A50D9C9` |
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
design.yaml [probe:]
|
||
|
|
||
+-- deform.py (host, py310, no bpy) -> _deform_job.json
|
||
| probe_deform.py (Blender) -> probe_frames.npz + deform_report.json
|
||
|
|
||
+-- render.py (host, no bpy) -> _render_job.json
|
||
| probe_render.py (Blender) -> frames/*.png + render_manifest.json
|
||
|
|
||
+-- flow.py (host, cv2) -> _track_job.json, flow/, flow_report.json
|
||
| (or is called in-process by cli.py)
|
||
|
|
||
+-- sensitivity.py -> sensitivity_report.json
|
||
+-- optimise.py -> optimizer_report.json
|
||
```
|
||
|
||
`deform.frame_plan()` is the **single ordering authority**. Every job carries the
|
||
resolved frame list so render and analysis cannot disagree about frame order.
|
||
|
||
All artifacts live under:
|
||
|
||
```
|
||
build/probe/<sensor>/<revision>/
|
||
```
|
||
|
||
resolved the same way `texture.py` resolves `build/textures/...` (see
|
||
`texture_revision_rel`; reuse it, do not re-invent it).
|
||
|
||
## Frame plan
|
||
|
||
Positive-only sweep, axes in the order `probe.axes` (default `x, y, z`),
|
||
steps `step_mm`, `2*step_mm`, ... up to and including `max_mm`.
|
||
|
||
| index | name | axis | step_mm |
|
||
|---|---|---|---|
|
||
| 0 | `000_reference` | `null` | 0.0 |
|
||
| 1 | `001_x_+0.01mm` | `x` | 0.01 |
|
||
| ... | ... | ... | ... |
|
||
| 10 | `010_x_+0.10mm` | `x` | 0.10 |
|
||
| 11..20 | `0NN_y_+0.0Xmm` | `y` | ... |
|
||
| 21..30 | `0NN_z_+0.0Xmm` | `z` | ... |
|
||
|
||
`F = 1 + len(axes) * round(max_mm / step_mm)` = **31** for the default settings.
|
||
PNG for frame `i` is `frames/<name>.png`.
|
||
|
||
The displacement for frame `i` is the frame's unit axis of the **probe-average
|
||
basis** (frozen below) scaled by `step_mm`. All 6 probe vertices move by exactly
|
||
that vector; every other vertex moves by `weight * vector`.
|
||
|
||
## The probe-average basis
|
||
|
||
Built from the area-independent average of the 6 probe vertex normals, then made
|
||
orthonormal so the three sweeps are exactly orthogonal and comparable:
|
||
|
||
```
|
||
z = normalize(sum(normal_i for i in probe)) # surface normal
|
||
x = normalize(cross(Vector((0,0,1)), z)) # fall back to (1,0,0) if degenerate
|
||
y = cross(z, x)
|
||
origin = mean(co_i for i in probe)
|
||
```
|
||
|
||
Measured for Xynova (orthonormal to 1e-10):
|
||
|
||
```
|
||
z = (-0.000027, -0.061749, 0.998092)
|
||
x = ( 1.000000, -0.000431, 0.000000)
|
||
y = ( 0.000430, 0.998092, 0.061749)
|
||
origin = (74.76961, 16.21550, 120.80202)
|
||
```
|
||
|
||
An axis vector for displacement is `{'x': x, 'y': y, 'z': z}[axis] * step_mm`
|
||
in the **mesh object's local space**. Since
|
||
`sensing_surface.matrix_world.to_scale() == (1,1,1)` and
|
||
`meters_per_source_unit == 0.001`, local units are mm and the vector is in mm.
|
||
|
||
## Proportional falloff
|
||
|
||
We deliberately do **not** use `bpy.ops.transform.translate(use_proportional_edit=True)`:
|
||
that operator is viewport-modal and its `proportional_size` is in screen pixels
|
||
relative to a 3D-view region, which does not exist under `blender -b` and would not
|
||
be reproducible. The falloff is computed explicitly, in mm, in a bpy-free module.
|
||
|
||
Semantics match Blender's proportional editing: the falloff distance is measured
|
||
from the **original** positions of the selected (probe) vertices, so the weight set
|
||
is constant across all frames.
|
||
|
||
```
|
||
t = d / radius_mm # d = min distance to any probe vertex
|
||
smooth: w = 1 - (3t^2 - 2t^3) # Blender's "Smooth" curve; 1 at t=0, 0 at t>=1
|
||
linear: w = 1 - t
|
||
sharp: w = (1 - t)^2
|
||
constant: w = 1
|
||
w = 0 whenever t >= 1, and w = 1 for probe members
|
||
```
|
||
|
||
## `design.yaml` `probe:` block
|
||
|
||
```yaml
|
||
probe:
|
||
mesh: sensing_surface
|
||
camera: cam_sensor
|
||
group: probe
|
||
uv_map: compute
|
||
texture: patch_texture_compute.png
|
||
radius_mm: 0.5
|
||
falloff: smooth
|
||
step_mm: 0.01
|
||
max_mm: 0.10
|
||
axes: [x, y, z]
|
||
resolution: [500, 500]
|
||
process_resolution: [250, 250]
|
||
tracker:
|
||
algorithm: dis
|
||
dis_preset: medium
|
||
dis_mode: 1
|
||
render:
|
||
engine: CYCLES
|
||
samples: 16
|
||
denoise: false
|
||
view_transform: Standard
|
||
interpolate: Closest
|
||
seed: 0
|
||
```
|
||
|
||
Built-in fallbacks live in `deform.DEFAULT_PROBE` and are merged
|
||
`DEFAULT_PROBE <- design.yaml probe <- CLI overrides`, exactly like
|
||
`texture.merge_texture_patch`. Unknown keys are an error. Validation returns one
|
||
message per problem and raises `ProbeConfigError`.
|
||
|
||
## Job files (stdlib JSON; Blender never reads YAML)
|
||
|
||
### `_deform_job.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/probe-deform-job-v1",
|
||
"revision_dir": "<abs>",
|
||
"source_blend": "<abs>",
|
||
"output_dir": "<abs>",
|
||
"objects": {"mesh": "sensing_surface", "group": "probe"},
|
||
"settings": {
|
||
"radius_mm": 0.5,
|
||
"falloff": "smooth",
|
||
"step_mm": 0.01,
|
||
"max_mm": 0.10,
|
||
"axes": ["x", "y", "z"]
|
||
},
|
||
"frames": [
|
||
{"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0},
|
||
{"index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01}
|
||
],
|
||
"artifacts": {"frames": "<abs>/probe_frames.npz", "report": "<abs>/deform_report.json"}
|
||
}
|
||
```
|
||
|
||
### `_render_job.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/probe-render-job-v1",
|
||
"revision_dir": "<abs>",
|
||
"source_blend": "<abs>",
|
||
"output_dir": "<abs>",
|
||
"objects": {"mesh": "sensing_surface", "camera": "cam_sensor"},
|
||
"settings": {
|
||
"uv_map": "compute",
|
||
"texture": "<abs path to the marker PNG>",
|
||
"width": 500, "height": 500,
|
||
"radius_mm": 0.5,
|
||
"engine": "CYCLES", "samples": 16,
|
||
"denoise": false, "view_transform": "Standard",
|
||
"interpolate": "Closest", "seed": 0
|
||
},
|
||
"frames": ["<same frames array as the deform job>"],
|
||
"null_frame": {"name": "reference_null", "seed": 1},
|
||
"artifacts": {"frames_npz": "<abs>/probe_frames.npz", "report": "<abs>/render_manifest.json"}
|
||
}
|
||
```
|
||
|
||
`null_frame` is a second render of the reference at a different Cycles seed. It is
|
||
never passed to the tracker: it exists only to measure the estimator's zero-input
|
||
floor.
|
||
|
||
### `_track_job.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/probe-track-job-v1",
|
||
"output_dir": "<abs>",
|
||
"settings": {
|
||
"axes": ["x", "y", "z"],
|
||
"step_mm": 0.01, "max_mm": 0.10,
|
||
"radius_mm": 0.5,
|
||
"process_resolution": [250, 250],
|
||
"tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1}
|
||
},
|
||
"scales": [
|
||
{"name": "render", "width": 500, "height": 500, "resample": false},
|
||
{"name": "process", "width": 250, "height": 250, "resample": true}
|
||
],
|
||
"artifacts": {"manifest": "<abs>/render_manifest.json", "strategy": "<abs>/strategy_probe.json"}
|
||
}
|
||
```
|
||
|
||
## `probe_frames.npz`
|
||
|
||
The contract between the deformation and rendering workstreams.
|
||
|
||
| Key | Shape / dtype | Meaning |
|
||
|---|---|---|
|
||
| `base_vertices` | `(V, 3) float64` | original object-local coords, all V vertices |
|
||
| `vertices` | `(F, V, 3) float64` | deformed coords per frame; index 0 = reference |
|
||
| `frame_names` | `(F,) <U...` | frame name, matching the plan |
|
||
| `frame_axes` | `(F,) <U4` | `null` for the reference, else `x`/`y`/`z` |
|
||
| `frame_steps_mm` | `(F,) float64` | applied displacement magnitude in mm |
|
||
| `weights` | `(V,) float64` | falloff weight per vertex (constant across frames) |
|
||
| `origin` | `(3,) float64` | probe centroid, object-local |
|
||
| `basis` | `(3, 3) float64` | rows are `x`, `y`, `z` unit axes, object-local |
|
||
| `probe_indices` | `(P,) int32` | member vertex indices |
|
||
|
||
`probe_render.py` must assert that `len(vertices) == len(frame_names)` and that the
|
||
plan in `_render_job.json` matches `frame_names` element-wise, then fail with exit 1
|
||
if not. It writes each frame to the mesh and renders; it never re-derives weights.
|
||
|
||
## `deform_report.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/probe-deform-report-v1",
|
||
"generated_at": "<iso8601 utc>",
|
||
"source_blend": {"path": "<abs>", "sha256": "..."},
|
||
"mesh": "sensing_surface",
|
||
"group": "probe",
|
||
"probe_indices": [215, 216, 218, 219, 327, 329],
|
||
"origin": [74.76961, 16.2155, 120.80202],
|
||
"basis": [[1.0, -0.000431, 0.0], [0.00043, 0.998092, 0.061749],
|
||
[-0.000027, -0.061749, 0.998092]],
|
||
"basis_orthonormal": true,
|
||
"normal_angle_max_deg": 2.553,
|
||
"flatness_mm": 0.007611,
|
||
"radius_mm": 0.5,
|
||
"falloff": "smooth",
|
||
"vertex_count": 2508,
|
||
"edge_p50_mm": 0.628501,
|
||
"weights": {"min": 0.0, "max": 1.0, "members_at_one": 6,
|
||
"partial": 5, "unaffected": 2497},
|
||
"affect_bands": {"1.0": 6, "0.3-0.5": 1, "0.05-0.1": 4},
|
||
"frames": [{"index": 0, "name": "000_reference", "axis": null,
|
||
"step_mm": 0.0, "max_delta_mm": 0.0, "moved_vertices": 0}],
|
||
"warnings": ["radius 0.5 mm reaches only 5 vertices outside the probe group ..."]
|
||
}
|
||
```
|
||
|
||
`moved_vertices` is the number of vertices whose displacement is non-zero **in that
|
||
frame**: `0` for the reference, and `6 + partial` (= 11 on Xynova) for any non-zero
|
||
step. It is not the probe member count, and it is not the same for every frame —
|
||
reporting 6 on the reference would contradict `max_delta_mm: 0.0`. The `weights`
|
||
block remains the authority for how many vertices are partially weighted.
|
||
|
||
`warnings` is the honesty channel. Emit one whenever
|
||
`affected (partial) < 4 * len(probe_indices)`, or `radius_mm < edge_p50_mm`.
|
||
|
||
## `render_manifest.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/probe-render-manifest-v1",
|
||
"generated_at": "<iso8601 utc>",
|
||
"source_blend": {"path": "<abs>", "sha256": "..."},
|
||
"texture": {"path": "<abs>", "sha256": "..."},
|
||
"uv_map": "compute",
|
||
"width": 500, "height": 500,
|
||
"settings_applied": {"engine": "CYCLES", "samples": 16, "denoise": false,
|
||
"view_transform": "Standard", "look": "None",
|
||
"exposure": 0.0, "interpolate": "Closest", "seed": 0},
|
||
"mesh_px_per_mm": {"min": 10.499, "p05": 10.946, "p50": 13.430,
|
||
"p95": 22.886, "max": 38.594,
|
||
"basis": "per-mesh-edge projected px / 3D mm, all 7341 edges"},
|
||
"probe_px_per_mm": {"p50": 13.248, "x": 13.248, "y": 17.670, "z": 1.519,
|
||
"basis": "probe-local: |centroid_px(step) - centroid_px(0)| / step_mm, per axis at its largest step"},
|
||
"probe_roi_px": {"u": 0.0, "v": 0.0, "radius": 0.0},
|
||
"reference_index": 0,
|
||
"reference_deterministic": true,
|
||
"frames": [{"index": 0, "name": "000_reference", "axis": null, "step_mm": 0.0,
|
||
"png": "frames/000_reference.png", "sha256": "...",
|
||
"width": 500, "height": 500,
|
||
"centroid_px": [249.61, 214.38]}],
|
||
"null": {"name": "reference_null", "seed": 1,
|
||
"png": "frames/reference_null.png", "sha256": "..."}
|
||
}
|
||
```
|
||
|
||
`reference_deterministic` reports the result of rendering frame 0 twice in the same
|
||
session with the same seed and comparing the saved bytes. It must be `true`; if it
|
||
is not, the render is not reproducible and the CLI must fail.
|
||
|
||
**Compare decoded pixels, not file bytes, and allow a 1-LSB tolerance.** This was
|
||
established the hard way: with literal whole-file byte equality the check rejected
|
||
candidate `nx=36` on the real run, and PNG chunk analysis showed why — the two files
|
||
had **identical chunk structure** (the only `tEXt` chunk was the deterministic
|
||
`cycles.ViewLayer.samples`) yet their decompressed pixel data differed in **2 bytes of
|
||
250,250**, at two scanline offsets. That is Cycles CPU thread-order floating-point
|
||
nondeterminism tipping a boundary pixel by one 8-bit step, not a metadata artefact and
|
||
not a defect in the render.
|
||
|
||
A gate that fails on a 1-LSB difference is not measuring reproducibility; it is
|
||
measuring thread scheduling, and because it is **content-dependent** it silently
|
||
disqualifies some patterns and biases a pattern search. So:
|
||
|
||
- Strip only the known-volatile `tEXt` chunks as before (`Date`, `Time`, `RenderTime`,
|
||
`cycles.ViewLayer.*_time`); keep image chunks byte-for-byte.
|
||
- Decode both PNGs and require the same shape, `max |delta| <= 1` per 8-bit channel,
|
||
and a differing-pixel fraction at or below **0.05%**. Record the measured
|
||
`max_abs_delta` and `differing_fraction` in the manifest either way.
|
||
- Fail loudly when a tolerance is exceeded, so real nondeterminism still stops the run.
|
||
- Never compare raw file bytes: PNG compression and metadata are not the measurement.
|
||
|
||
|
||
`probe_roi_px` is the probe centroid projected to pixels plus the radius in pixels
|
||
that corresponds to `radius_mm` (`radius = radius_mm * probe_px_per_mm.p50`), all
|
||
measured from the actual projection (not assumed). The tracker uses it to build its
|
||
ROI, and the ROI is **fixed at the reference centroid** for every frame — a moving
|
||
ROI would bias the ROI mean as the probe moves, which is exactly the signal we are
|
||
trying to measure.
|
||
|
||
### Two different pixel scales, and why they must not be confused
|
||
|
||
- `mesh_px_per_mm` is a **mesh-wide** statistic: for every mesh edge, its projected
|
||
length in pixels divided by its 3D length in mm, summarised over all 7341 edges.
|
||
It describes how the whole surface is sampled in the camera. It is **not** the
|
||
gain to convert the probe's measured flow into millimetres.
|
||
- `probe_px_per_mm` is the **probe-local** scale, and it is the one the scorer must
|
||
use: `|centroid_px(step) - centroid_px(0)| / step_mm`, per axis, i.e. how many
|
||
image pixels the probe actually moves per millimetre applied. It is anisotropic,
|
||
and `z` (motion along the surface normal) is expected to be **much smaller** than
|
||
x and y because normal motion is largely out of the image plane. That is a real
|
||
physical property of the sensor, not a defect.
|
||
|
||
Because the `z` scale can approach zero, dividing a measured response by it is
|
||
ill-conditioned. The scorer therefore reports gain in **px per mm** as the primary
|
||
quantity and only derives mm per mm when the axis scale clears
|
||
`sensitivity.min_local_scale_px_per_mm` (default **1.0**; a module constant in
|
||
`sensitivity.py`, deliberately not a `design.yaml` key so it cannot be tuned into
|
||
silence). Never divide by a scale below that; report `null` and warn instead. A large
|
||
"mm/mm" gain produced by dividing by a near-zero scale would be an artefact, not a
|
||
sensitivity.
|
||
|
||
The default of 1.0 is validated by real data rather than chosen by feel. Measured
|
||
per-axis probe scales: **x 13.248, y 17.670, z 1.519 px/mm at the render scale**, and
|
||
**x 6.624, y 8.835, z 0.760 px/mm at the process scale**. So 1.0 discriminates exactly
|
||
where it should on this revision: render-scale `z` (1.519) is admitted and yields a
|
||
derived `gain_mm_per_mm` of 0.445 backed by 7 steps above the floor at `snr` 4.2,
|
||
while process-scale `z` (0.760) is correctly gated to `null`.
|
||
|
||
Note what this implies about the earlier worry that `z` would be unmeasurable: at the
|
||
**process** scale a 0.1 mm normal displacement is only 0.076 px against a 0.19 px
|
||
floor, so it genuinely is unmeasurable there; at the **render** scale it is 0.152 px
|
||
against a 0.028 px floor, so it is measurable from about `0.04 mm` upward. The gate is
|
||
a guard against dividing by a degenerate scale, **not** a substitute for the measured
|
||
`snr`, `points_above_floor` and `resolvable_step_mm`, which are the quantities that
|
||
decide whether an axis is useful. Report all of them; do not suppress a number that
|
||
the data supports.
|
||
|
||
### What actually limits the measurement (measured, not assumed)
|
||
|
||
Do **not** attribute the response deficit to aliasing, noise, or ROI dilution. The
|
||
render-scale deficit is **spatial resolution of the measurement region**, established
|
||
by these measurements on the real frames:
|
||
|
||
- The real `x +0.10 mm` flow radial profile is essentially uniform inside the ROI
|
||
(`-0.9162`, `-0.9096`, `-0.8725` px at r 0-2, 2-4, 4-6.6), with `|flow|` p5..p100 of
|
||
`0.81..0.93` px. So it *is* a coherent translation, not noise and not a diluted
|
||
average: the motion is simply shrunk to about `0.67x` of the applied `1.3248 px`.
|
||
- Injecting a synthetic **local rigid translation** of `1.3248 px` into the reference
|
||
and measuring it over the same ROI recovers only **17%** at a patch radius of
|
||
6.6 px (equal to the ROI), then **33%** at r 8, **51%** at r 10, **63%** at r 13,
|
||
**81%** at r 20 and **95%** at r 40.
|
||
- Therefore the attenuation is caused by the moving probe patch plus its falloff taper
|
||
spanning only about **13 px, i.e. 1-2 DIS patches (8 px)**, so DIS regularizes
|
||
toward the static background. At the process scale the same region is about 6.5 px,
|
||
under one patch, and the response collapses while the forward-backward error grows
|
||
from 1.6 to 8.8 (x) and 15.8 (y).
|
||
|
||
Two consequences the report must state rather than bury:
|
||
|
||
1. **The measured gain is confounded with the ROI size.** It grows strongly with
|
||
`radius_mm` because a larger radius gives DIS more moving texture to lock onto.
|
||
Relative ranking of marker patterns is valid **only** while `radius_mm`, the ROI and
|
||
the scale are held fixed across candidates — which is why `run_optimise` varies
|
||
`nx`/`ny`/`seed` and never the radius.
|
||
2. **"Below one pixel" and "below the estimator floor" are different statements and
|
||
must not be conflated.** The measured zero-input floor is **0.0292 px at the render
|
||
scale**, so a `0.01 mm` step (`0.1325 px`) is about **4.5x above the floor**: the
|
||
ROI-averaged response genuinely resolves single steps at 500x500, with monotone
|
||
per-step responses (`0.0810, 0.1316, 0.1971, ... 0.8859` px). The earlier
|
||
expectation that the sub-pixel step would be lost in the floor is **not** supported
|
||
by the measurement; what is lost is *per-pixel* precision, not the ROI mean. Report
|
||
the floor and the step in pixels and let the numbers speak.
|
||
|
||
|
||
|
||
|
||
Each `frames[]` entry also carries `centroid_px`: the probe centroid projected to
|
||
pixels **for that frame's deformation**, as full-precision floats (not rounded).
|
||
This is the authoritative source for the image-space displacement direction the
|
||
sensitivity stage needs. It is a small number of pixels (1.34 px across the whole
|
||
0.1 mm sweep at 500x500), which is why it is recorded as a float and why the scorer
|
||
regresses over all steps rather than differencing a single pair.
|
||
|
||
## `flow_report.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/flow-report-v1",
|
||
"generated_at": "<iso8601 utc>",
|
||
"tracker": {"algorithm": "dis", "dis_preset": "medium", "dis_mode": 1,
|
||
"finest_scale": 1, "source": "design.yaml probe.tracker"},
|
||
"known_answer": {"passed": true, "cases": [
|
||
{"dx": 2, "dy": 0, "measured_u": 2.0, "measured_v": 0.0, "tol": 0.25},
|
||
{"dx": 0, "dy": 2, "measured_u": 0.0, "measured_v": 2.0, "tol": 0.25}]},
|
||
"reference_texture_known_answer": {
|
||
"passed": false, "gating": false,
|
||
"note": "a known shift applied to the real rendered marker; a periodic "
|
||
"pattern legitimately fails this when its cell approaches the DIS "
|
||
"patch, so it is a diagnostic, not a plumbing gate",
|
||
"cases": [{"dx": 2, "dy": 0, "measured_u": 8.17, "expected_u": 2.0}]},
|
||
"reference_self": {"max_abs_flow_px": 0.0},
|
||
"scales": [{
|
||
"name": "render", "width": 500, "height": 500, "resample": false,
|
||
"px_per_mm": 13.43, "mm_per_px": 0.07446,
|
||
"frames": [{
|
||
"index": 1, "name": "001_x_+0.01mm", "axis": "x", "step_mm": 0.01,
|
||
"roi": {"u": 249.6, "v": 214.4, "radius": 6.7, "pixels": 141},
|
||
"flow": {"mean_u": 0.02, "mean_v": -0.01,
|
||
"mean_mag": 0.03, "p50_mag": 0.02, "p95_mag": 0.08, "max_mag": 0.31,
|
||
"coverage": 0.41},
|
||
"response_proj_px": 0.02,
|
||
"response_mm": 0.0015,
|
||
"forward_backward_error": 1.87}
|
||
],
|
||
"null": {"floor_px": 0.05, "floor_source": "reference_null"}
|
||
}]
|
||
}
|
||
```
|
||
|
||
Per-frame flow arrays are also written to `flow/<scale>/<name>.npy` as
|
||
`float32 (H, W, 2)` so the sensitivity stage and any later inspection can reuse them
|
||
without re-running DIS.
|
||
|
||
`response_mm` and `response_proj_px` are informational. **Pixels are authoritative**
|
||
(`response_proj_px`); `response_mm` is derived through a single isotropic scalar
|
||
scale (`px_per_mm`, the probe-local p50) and therefore *understates or overstates*
|
||
the true conversion on an anisotropic axis — notably `z`. The scorer must use
|
||
`response_proj_px` plus the per-axis local scale from `render_manifest.json` and must
|
||
not consume `response_mm`. Each scale records `px_per_mm_source` naming the exact
|
||
manifest field it came from, so the provenance of the scalar is auditable.
|
||
|
||
## `sensitivity_report.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/sensitivity-report-v1",
|
||
"generated_at": "<iso8601 utc>",
|
||
"revision": "sensors/finger-xynova/r01",
|
||
"texture": {"path": "<abs>", "sha256": "...", "grid": [80, 80],
|
||
"cell_mm": [0.23125, 0.205]},
|
||
"probe": {"group": "probe", "origin": [...], "basis": [...],
|
||
"radius_mm": 0.5, "falloff": "smooth", "affected_vertices": 5},
|
||
"tracker": {"dis_preset": "medium", "dis_mode": 1, "patch_px": 8},
|
||
"scales": [{
|
||
"name": "render", "px_per_mm": 13.43,
|
||
"displacement_px_per_step": 0.1343,
|
||
"cell_px": 3.11,
|
||
"cell_over_patch_px": 0.389,
|
||
"null_floor_px": 0.05,
|
||
"null_floor_basis": "render_seed_reproducibility",
|
||
"axes": [{
|
||
"axis": "x", "direction_px": [0.999, -0.031],
|
||
"direction_source": "centroid_regression",
|
||
"points": [{"step_mm": 0.01, "applied_px": 0.134, "response_px": 0.02,
|
||
"above_floor": false}],
|
||
"summary": {"gain_px_per_mm": 6.2, "gain_mm_per_mm": 0.46,
|
||
"scale_px_per_mm": 13.42, "snr": 0.42, "spearman": 0.95,
|
||
"resolvable_step_mm": null, "saturation_mm": null,
|
||
"cross_axis_leak": 0.18}
|
||
}],
|
||
"score": 0.11,
|
||
"verdict": "unresolvable"
|
||
}],
|
||
"overall": {"score": 0.11, "verdict": "unresolvable",
|
||
"best_scale": "render", "best_axis": "z"},
|
||
"warnings": ["0.01 mm is 0.134 px at 500x500; a single step is below the estimator floor"],
|
||
"notes": []
|
||
}
|
||
```
|
||
|
||
### Scoring (every input measured; no assumed constants)
|
||
|
||
For one scale and one axis:
|
||
|
||
1. `d_hat_px` — unit image-space direction of the applied displacement, from
|
||
`render_manifest.json`'s per-frame `centroid_px`. Regress `centroid_px` on
|
||
`step_mm` over every frame of that axis and normalize the slope vector
|
||
`(du/dstep, dv/dstep)`; this is preferred over a single 0 -> `+max_mm` finite
|
||
difference because the whole sweep is only ~1.3 px. Fall back to the two-point
|
||
finite difference when the regression is degenerate, and to the ROI-mean flow at
|
||
the `+max_mm` frame when `centroid_px` is absent. Record which was used as
|
||
`direction_source` per axis.
|
||
2. `response_i` = mean over ROI pixels of `dot(flow_i, d_hat_px)`, in px. Sign kept.
|
||
3. `floor_px` = p95 of `|response|` measured on `reference_null`, projected on the
|
||
same `d_hat_px`. Reported per axis.
|
||
4. `gain_px_per_mm` = OLS slope of `response_px` vs `applied_mm`, over the steps with
|
||
`|response_i| > floor_px`. Requires >= 3 such points, else `null`. **This is the
|
||
primary sensitivity**: measured pixels of probe response per millimetre applied.
|
||
5. `scale_px_per_mm` = `probe_px_per_mm[axis]`. Then `gain_mm_per_mm` =
|
||
`gain_px_per_mm / scale_px_per_mm`, reported **only** when
|
||
`scale_px_per_mm >= min_local_scale_px_per_mm` (default 1.0); otherwise `null`
|
||
plus a warning. Never divide by a near-zero scale: for the `z` axis that would
|
||
manufacture an enormous gain no camera could observe.
|
||
6. `snr` = `|gain_px_per_mm| * max_mm / max(floor_px, eps)` — both terms in px, so it
|
||
is dimensionally consistent and comparable across patterns and axes.
|
||
7. `spearman` = Spearman rank correlation of `response_i` vs `applied_mm` over all
|
||
steps; clamped to `[0, 1]`.
|
||
8. `cross_axis_leak` = mean `|dot(flow, d_perp_px)|` / mean `|dot(flow, d_hat_px)|`
|
||
over the ROI and steps, clamped to `[0, 1]`.
|
||
9. `score` = `snr * spearman * (1 - min(leak, 0.5))`.
|
||
10. `resolvable_step_mm` = the smallest `step_mm` with `|response_i| > floor_px`, else
|
||
`null`.
|
||
11. `saturation_mm` = the first step where `response` stops increasing
|
||
(monotonically decreasing after a rise), else `null`.
|
||
|
||
`verdict`, in precedence order:
|
||
|
||
- **`ambiguous`** — the tracker's `reference_texture_known_answer` failed for this
|
||
scale: a known shift applied to the *actual rendered marker* was not recovered.
|
||
For a **periodic** marker this is expected once the cell approaches the DIS patch
|
||
size — DIS locks onto the period and reports the shift modulo that period — and it
|
||
means the measured direction and magnitude are not trustworthy, so no score can
|
||
promote this pattern. This takes precedence over everything below.
|
||
- `resolvable` — `gain_px_per_mm` is not null and `snr >= 3`.
|
||
- `marginal` — `gain_px_per_mm` is not null and `snr < 3`.
|
||
- `unresolvable` — `gain_px_per_mm` is null.
|
||
|
||
`overall.score` = mean of `score` over scales and axes. `overall.verdict` =
|
||
`resolvable` only if every scale and axis is `resolvable`; `ambiguous` if any scale is
|
||
`ambiguous`; otherwise the worst remaining verdict.
|
||
|
||
Measured evidence for why `ambiguous` must exist (estimator, 500x500, probe scale
|
||
13.43 px/mm, DIS medium, `dis_mode=1`): a **2 px** known shift measured
|
||
**8.17 px** on a 3.11 px checker cell (aliased onto the 6.22 px period), **0.0 px** on
|
||
a 1 px cell, **-19.8 px** on 3.11 px grid lines, but **exactly 2.000 px** on a
|
||
12.4 px cell and **exactly 2.000 px** on a broadband random texture. A periodic
|
||
pattern aliases a sub-period shift onto the wrong period; a broadband one does not.
|
||
That is the whole reason the anisotropy and periodicity of the marker matter, and it
|
||
is measured, not assumed.
|
||
|
||
## The floor is not a noise model
|
||
|
||
`flow_report.json`'s `scales[].null.floor_px` is measured as `reference` vs a re-render at
|
||
**render seed + 1**: Cycles antialiasing jitter, ~5 grey levels RMS on half the pixels and
|
||
scaling with edge contrast. It is labelled
|
||
`floor_basis: "render_seed_reproducibility"` and **must not be used to rank patterns**.
|
||
Measured on this bench:
|
||
|
||
- its ordering **reverses** under additive noise (render-seed: nx56 < nx80 < nx31;
|
||
σ=2: nx31 < nx56 < nx80);
|
||
- it **understates** the real idle floor, coarse patterns most.
|
||
|
||
The calibrated replacement is:
|
||
|
||
- **`design.yaml` `probe.noise`**: `idle_footage` (real no-contact recording),
|
||
`roi_fraction`, `max_frames`, `sigmas`, `draws`, `seed`, `models`, `scales`.
|
||
- **`floorcal.measure_idle_floor`**: runs the *same* `FixedReferenceFlow` on idle footage
|
||
and reports `floor_px_short` (consecutive frames, noise), `floor_px_long` (vs frame 0,
|
||
noise + drift), `dominant_period_px`, `floor_per_period_short|long` and `drift_ratio`.
|
||
A ratio near 1 means the clip is idle and the floor is frame-independent white noise.
|
||
On `data/dd02-0703/190.mp4` (1280x720): **0.2267 px**, period **25.5 px**,
|
||
**0.0089 floor per period**, ratio **0.98**.
|
||
- **`flow.noise_model_pass`**: additive white Gaussian applied to **both** the reference
|
||
and the displaced frames, so the signal and the floor come from one model. Written to
|
||
each scale as `noise_models` (`basis`, `draws`, `sigmas`, per-axis
|
||
`gain_px_per_mm`, `floor_px`, `floor_samples`).
|
||
- **`sensitivity.attach_noise_models`**: adds `response_retention`
|
||
(`gain(σ)/gain(clean)`) and `snr_noise` (same-model floor) per axis, and emits a
|
||
`model_disagreement` note when the response moves more than 15%.
|
||
- **`sensitivity.noise_calibration`**: top-level `noise_calibration` block with the real
|
||
numbers, cached as `idle_floor.json` keyed by the source's size and mtime. Skip with
|
||
`--no-calibration`.
|
||
|
||
Schema: `flow-report-v2` and `sensitivity-report-v3`. All additions are additive; a report
|
||
without `noise_model` is still valid and means the render-seed model only.
|
||
|
||
### Consequence for the ranking
|
||
|
||
Under the calibrated model at the render scale, `radius_mm=0.5`, `dis_mode=1`, nx=80 has
|
||
the **highest clean response on both axes** (x 9.10 vs 4.74 px/mm; y 11.69 vs 10.90) and
|
||
the **best retention** (99%/84% at σ=4/16 against nx=31's 72%/59%). The earlier
|
||
"nx=31 wins" and "the cell must match the effective window" conclusions were artefacts of
|
||
the render-seed floor and are retracted; the window ratio remains a useful diagnostic, not
|
||
a validated law. The process scale and the `radius_mm=1.5` block are not yet re-ranked.
|
||
|
||
## Out-of-scope / next hardware ask
|
||
|
||
The idle clip gives only one pattern, so it validates the floor's **magnitude** and
|
||
**mechanism**, not its pattern trend. A deliberate no-contact recording at the shipped
|
||
configuration would let the trend itself be measured, and is the natural next ask.
|
||
|
||
## `optimizer_report.json`
|
||
|
||
```json
|
||
{
|
||
"schema": "orisys-sensor-design/optimizer-report-v1",
|
||
"generated_at": "<iso8601 utc>",
|
||
"search": {"grid": [15, 20, 24, 28, 31, 36, 42, 48, 56, 64, 80],
|
||
"seeds": [7], "render_resolution": [250, 250],
|
||
"candidates": 11, "objective": "sensitivity_report.overall.score"},
|
||
"candidates": [{"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
|
||
"verdict": "marginal", "cell_px": 8.01,
|
||
"report": "<abs>", "texture_sha256": "..."}],
|
||
"ranked": ["31@7", "36@7"],
|
||
"winner": {"nx": 31, "ny": 31, "seed": 7, "score": 0.44,
|
||
"confirmed_at_resolution": [500, 500], "confirmed_score": 0.39,
|
||
"report": "<abs>"},
|
||
"notes": ["Ranked by the measured score under a fixed estimator and camera; this is not a global optimum."]
|
||
}
|
||
```
|
||
|
||
The optimizer searches `(nx, ny, seed)`. Candidates are enumerated in a
|
||
**deterministic sorted order** so two runs produce byte-identical rankings. Each
|
||
candidate regenerates the marker texture, then runs deform -> render -> track ->
|
||
sensitivity at the search resolution. Only the winner is re-run at the full
|
||
`probe.resolution` and its confirmed score recorded.
|
||
|
||
### The search resolution must be the resolution you care about
|
||
|
||
A reduced search resolution **answers a different question** and must not be used to
|
||
rank patterns. Measured on the real run, searching at 250x250 and confirming at
|
||
500x500 produced a winner that the real resolution disagrees with:
|
||
|
||
| pattern | search score @250 | render score @500 | overall |
|
||
|---|---|---|---|
|
||
| `nx=48` (search winner) | **8.94** | 20.95 | 15.39 |
|
||
| `nx=80` (shipped) | **1.77** (ranked 10th of 10) | **36.77** | **22.22** |
|
||
|
||
At 250x250 the `nx=80` cell is 1.53 px, below the DIS-patch Nyquist limit, so it
|
||
scored near the bottom; at 500x500 the same cell is 3.06 px and it scored *top*. The
|
||
shipped pattern is therefore the best of the grid at the real resolution and one of
|
||
the worst at the processed one — and a 250-based search excludes it. That is physics
|
||
(the two scales genuinely resolve different cells), not a bug, but it means the
|
||
default `search_resolution` must be `probe.resolution`, not a reduced size. Rendering
|
||
31 frames at 500x500 costs about 14 s, so a full-resolution search over the default
|
||
grid is only a few minutes and there is no reason to trade correctness for it.
|
||
|
||
Two further consequences for the report:
|
||
|
||
- Record **`search_resolution` and `confirm_resolution`**, and mark the winner's
|
||
`confirmed_score`, so a reader can see whether the search and the confirm measured
|
||
the same thing. A large gap between them means the search resolution was wrong.
|
||
- Rank and report candidates **per scale** as well as by the mean. "Best" is
|
||
scale-dependent here: `nx=80` wins the render scale while `nx=48` balances both.
|
||
`overall.score` is the mean over scales, which is a deliberate modelling choice, not
|
||
a neutral one, and the notes must say so.
|
||
|
||
|
||
## CLI surface
|
||
|
||
```
|
||
orisys-design deform <revision_dir> [--output-dir DIR] [--radius-mm F]
|
||
[--falloff MODE] [--step-mm F] [--max-mm F]
|
||
[--group NAME] [--mesh NAME] [--axes x,y,z]
|
||
orisys-design render <revision_dir> [--output-dir DIR] [--resolution WxH]
|
||
[--samples N] [--uv-map NAME] [--texture PATH] [--seed N]
|
||
orisys-design track <revision_dir> [--output-dir DIR]
|
||
[--process-resolution WxH] [--no-process-scale]
|
||
orisys-design sensitivity <revision_dir> [--output-dir DIR] [--skip-deform]
|
||
[--skip-render] [--skip-track] [--radius-mm F] [...]
|
||
orisys-design optimise <revision_dir> [--output-dir DIR] [--grid LIST]
|
||
[--seeds LIST] [--search-resolution N] [--top N] [--dry-run]
|
||
```
|
||
|
||
Default `--output-dir` is `build/probe/<sensor>/<revision>`.
|
||
|
||
**All argparse wiring lives in `cli.py`, owned by the Lead.** The feature modules
|
||
expose only plain entry points and must not add their own argument helpers or
|
||
duplicate `cmd_*` functions:
|
||
|
||
- `deform.py`: `resolve_probe`, `probe_overrides_from_args`, `frame_plan`,
|
||
`build_deform_job`, `write_deform_job`, `resolve_probe_output_dir`
|
||
- `render.py`: `resolve_marker_texture`, `build_render_job`, `write_render_job`,
|
||
`render_manifest_path`
|
||
- `flow.py`: `build_track_job`, `write_track_job`, `run_track_job`
|
||
- `sensitivity.py`: `run_sensitivity(settings, output_dir, *, manifest_path=None,
|
||
flow_report_path=None, progress=print)` — **scoring only**; it must not re-run the
|
||
deform/render/track stages, because `cli.py` already chains them (and `--skip-*`
|
||
exists precisely to reuse their artifacts)
|
||
- `optimise.py`: `run_optimise(paths, settings, *, grid, seeds, search_resolution,
|
||
top, dry_run=False, runner=None, progress=print)`. The per-candidate `runner` is an
|
||
injected callable; `cli.py` supplies one that runs the real stages, and tests supply
|
||
a fake. Do not resolve stage functions by `inspect` or by guessing module layout.
|
||
|
||
|
||
|
||
Exit codes, matching the existing convention:
|
||
|
||
| Code | Meaning |
|
||
|---|---|
|
||
| 0 | success |
|
||
| 1 | scene / metadata / geometry error |
|
||
| 2 | config error, or a missing optional dependency |
|
||
| 3 | a required artifact is missing or empty |
|
||
| 4 | (reserved; not used by this pipeline) |
|
||
|
||
## Failure modes that must fail loudly
|
||
|
||
- Probe group missing, empty, or not on the requested mesh.
|
||
- The basis is not orthonormal within 1e-6.
|
||
- A displacement vector cannot be expressed in the basis.
|
||
- The marker texture PNG is missing or empty.
|
||
- `view_transform` is not `Standard` (AgX or Filmic silently destroys marker values).
|
||
- Frame count/order in `frames/` disagrees with the plan, or a PNG is missing/empty.
|
||
- `reference_deterministic` is false.
|
||
- The tracker's known-answer test fails.
|
||
- `cv2` is unavailable -> exit 2 with `python -m pip install -e .[probe]`.
|
||
- The probe ROI falls outside the frame at the process scale -> warn and truncate, do
|
||
not silently report a full-ROI number.
|
||
|
||
## Non-goals
|
||
|
||
Large-deformation robustness beyond `max_mm`; signed/bidirectional sweeps; per-vertex
|
||
normal indentation; refraction through `shell`/`rubber-1`; wiring the SDK processed
|
||
image pipeline (`resolution_stitch`, `source_scale`) into the estimator; absolute
|
||
force calibration. Sensitivity here is a **relative ranking under a fixed estimator
|
||
and camera**, not a force measurement.
|