orisys-sensor-design/docs/probe-findings.md

# Probe bench: findings and recommendations

Short, decision-oriented summary of what the sensitivity bench has established. The
mechanics are in `docs/probe-pipeline.md` (contract) and `docs/probe-method.md` (operator
guide). This file is the *conclusion* — read it first.

Last updated after the nx 60-120 sweep at 500x500 and the 250x250 cross-check.

---

## 1. Retraction: the noise floor was not a noise model

An earlier version of these docs claimed **nx=31 is the best pattern** and **"the marker
cell must match the estimator's effective window"**. Both are **retracted**.

`floor_px` was measured as `reference` versus a re-render at **render seed + 1** — Cycles
antialiasing jitter, about 5 grey levels RMS on half the pixels, scaling with edge
contrast. It is not sensor noise, and it cannot rank patterns:

1. **Its trend reverses** under additive noise. p95 |flow| over the ROI, same DIS config:

   | pattern | render-seed | injected sigma=2 | sigma=4 |
   |---|---|---|---|
   | nx=80 | 0.0292 | **0.0161** | 0.0278 |
   | nx=31 | 0.0376 | **0.0084** | 0.0182 |
   | nx=56 | 0.0134 | 0.0102 | 0.0203 |

2. **It understates the real floor, coarse patterns most.** Measured on idle footage
   (`data/dd02-0703/190.mp4`, 1280x720) with the same estimator: **0.2267 px**,
   dominant marker period **25.5 px**, i.e. **0.0089 floor per period**; short baseline
   equals long baseline (ratio **0.98**), so it is frame-independent white noise and the
   clip is genuinely idle. Simulated floors per cell sit about **2x** (nx=80), **4x**
   (nx=31) and **6x** (nx=56) below it — the bias that made the coarse patterns win.

3. **Signal is not noise-invariant either.** Retention `gain(sigma)/gain(clean)` at
   +0.10 mm, x axis, render scale: nx=80 dm1 **99%** at sigma=4 and **84%** at sigma=16,
   against nx=31 dm1 **72%** and **59%**.

**What replaced it.** The render-seed floor is kept only as a reproducibility diagnostic
and is labelled `floor_basis: "render_seed_reproducibility"`. Signal and noise are now
measured under **one** injected-noise model (`noise_models` per axis: gain, floor,
`response_retention`, `snr_noise`), calibrated against the real idle footage. The real
number is cached as `idle_floor.json`; skip with `--no-calibration`.

---

## 2. What actually governs the measurement: the estimator window vs the moving region

This is the most useful result, and it explains every resolution effect seen. Earlier notes
called it "ROI versus effective window"; that was the right *variable* but the wrong
*mechanism*, and measuring it changed the fix. See `build/probe/finger-xynova/r01/roi_regime.png`.

### The two regions, concretely

- **The moving region.** `radius_mm` (0.5) is the proportional-editing falloff radius for
  the deformation, so the surface that actually moves is roughly a 1 mm disc — about
  **13.25 px across at 500x500** and **6.62 px at 250x250**. It is a fixed *physical* size,
  so its pixel size is set by the render resolution.
- **The measurement ROI.** A disc of radius `radius_mm x px_per_mm` centred on the projected
  probe centroid. The response is the **mean of `dot(flow, d_hat)` over that disc**.
  At 500x500 it is 6.62 px radius / 139 pixels; at 250x250, 3.31 px radius / **37 pixels**.
- **The estimator window.** DIS matches an 8 px patch, and `dis_mode` is OpenCV's
  `setFinestScale`, so the flow is computed at `2**dis_mode` lower resolution and upscaled.
  The effective support is therefore `8 x 2**dis_mode` **input** pixels: **16 px at
  `dis_mode=1`, 8 px at `dis_mode=0`**.

### The mechanism is attenuation inside the estimator, not dilution in the average

Both were plausible. They are distinguished by comparing the flow **at the probe centroid**
against the **mean over the ROI**, and by sweeping the ROI radius over one fixed flow field:

| config | window | applied | flow @ centroid | mean over ROI | recovery |
|---|---|---|---|---|---|
| 500 dm1 | 16 px | 1.3248 px | -0.928 | -0.897 | **70%** |
| 500 dm0 | 8 px | 1.3248 px | -1.170 | -1.134 | **88%** |
| 250 dm1 | 16 px | 0.6624 px | **-0.166** | -0.161 | **25%** |
| 250 dm0 | 8 px | 0.6624 px | -0.482 | -0.429 | **73%** |

The centroid value and the ROI mean agree closely **in every case**, and sweeping the ROI
radius from 1 px to 30 px shows the response is **flat across the whole moving region** and
only falls off once the disc grows past it. So the ROI is well placed in all four cases and
the average is not diluting anything. The attenuation is already present in the flow field
that DIS produces.

**Why DIS attenuates it.** DIS minimises a photometric term plus a smoothness term over a
patch-and-propagation neighbourhood of about the window size. When the true displacement is
nonzero only inside a region *smaller* than that neighbourhood, most of the pixels the fit
sees are static, and both terms pull the estimate toward zero: the photometric term is best
satisfied by the static majority, and the smoothness term penalises a localised spike.
The attenuation therefore depends on how much of the window actually moves.

### The law

Recovery is a function of **`window / moving-region diameter`**, not of resolution:

| config | window / moving diameter | recovery |
|---|---|---|
| 500 dm0 | 8 / 13.25 = **0.60** | 88% |
| 500 dm1 | 16 / 13.25 = **1.21** | 70% |
| 250 dm0 | 8 / 6.62 = **1.21** | 73% |
| 250 dm1 | 16 / 6.62 = **2.42** | 25% |

The two configurations at ratio 1.21 differ by a **factor of two in resolution** and agree
to **3 percentage points**. That is the whole story: the governing quantity is the ratio of
the estimator's support to the physical size of the thing you are measuring.

### Which mode for a spatially resolved deformation map?

`dis_mode` changes **two** things, and only one is obvious:

1. the **window** (`8 x 2**dis_mode` px), which sets the attenuation; and
2. the **internal solve grid**. `setFinestScale(n)` solves the flow at `2**n` lower
   resolution and bilinearly upscales to the output size. So:

| mode | output | actually solved at | window | recovers | background error | tail @1.88 mm |
|---|---|---|---|---|---|---|
| **500 dm0** | 500x500 | **500x500** | 8 px | **86%** | **0.74 um** | **1.4%** |
| 500 dm1 | 500x500 | 250x250 | 16 px | 68% | 1.77 um | 4.8% |
| 250 dm0 | 250x250 | 250x250 | 8 px | 66% | 1.56 um | 4.6% |
| 250 dm1 | 250x250 | 125x125 | 16 px | 25% | 4.24 um | **26.7%** |

("tail" is the residual flow at 1.88 mm from the probe, where the surface is undeformed,
as a fraction of each mode's own plateau; "background error" is the mean spurious
displacement in the 1.5-3.0 mm annulus, in micrometres. Figure:
`build/probe/finger-xynova/r01/edge_quality.png`.)

**So 250x250 `dm0` is *not* sharper than 500x500.** In physical units the edge width is the
same (`r50` = 1.12 mm for 500 dm0, 500 dm1 and 250 dm0) and the normalised profiles of
**500 dm1 and 250 dm0 nearly coincide** — because they are the *same computation*: both
solve on a 250x250 grid with an 8 px window there. A half-resolution round trip through the
500 dm1 field leaves a residual of only **0.019** of its RMS, confirming it carries no detail
above 250x250; it is a 250x250 field upsampled onto 4x as many pixels.

**The sharpest mode is 500x500 `dm0`**, the only one that solves at its full output
resolution. It recovers 86% of the true displacement (vs 66%), has a 2x cleaner background,
and a **3x smaller edge tail** (1.4% vs 4.6%).

An apparent "sharper falloff" for 250 dm0 is a **unit artefact**: on the ROI-radius sweep
plotted in *pixels* the 250 curve looks steeper only because the whole scene is half the
size in pixels. Plotted in millimetres the two profiles overlay (0.287 vs 0.280 of plateau
at 1.12 mm).

**250 dm1 is unusable for a map at any ROI size**: 25% recovery, and the profile never
returns to zero (26.7% of the plateau at 1.88 mm, still 7.3% at 2.88 mm).

**Important qualification: the bench does not currently produce a map.** It reports one
scalar per probe position — the mean of `dot(flow, d_hat)` over a small disc (139 px at
500, 37 px at 250). The profiles above describe the *footprint of a single probe*, measured
by evaluating the full flow field. The far-field numbers do show that field returning to
~0 outside the deformation (0.74-1.56 um), so a genuine spatial map is feasible — but it
would be a new output, and it should be built on 500 dm0.

### Consequences

1. **Keep `window / moving diameter` at or below ~1.2.** Above ~2 the estimator is mostly
   measuring its own smoothness prior. This is why 250x250 with the shipped `dis_mode=1`
   (ratio 2.42) loses ~3x of its signal, and why `dis_mode=0` fixes it.
2. **A larger ROI does not help.** The sweep is monotone *decreasing* past the moving
   region, so widening the ROI to "get more signal" only adds static background. Three
   levers exist instead: raise the render resolution, reduce the window (`dis_mode`), or
   make the deformation physically larger (`radius_mm` / a bigger probe).
3. **Even at ratio ~1.2 you only recover ~70%.** Every absolute sensitivity number in this
   bench is attenuated by 12-75% depending on configuration. Relative *pattern* rankings are
   still valid because the geometry is held fixed, but do not read `gain_px_per_mm` as the
   true displacement sensitivity.
4. **At 250 dm1 the field is distorted, not merely scaled.** `max|du|` reaches 121% of the
   applied displacement somewhere in the frame while the probe region sits at 25% — spurious
   high-flow pixels plus an attenuated centre. At 250 dm0 the peak (73%) matches the ROI
   recovery, i.e. a clean uniform scale factor. So the 250/dm1 response is not a reliable
   fraction of the truth at any ROI size.


---

## 3. Pattern choice at 500x500: nx = 80-90

13 candidates, nx 60-120, `radius_mm=0.5`, `dis_mode=1`, render scale, 5 noise draws.
Gain is px per mm of applied displacement at 0.10 mm, measured **under the calibrated
sigma=4 noise model** (clean gain is within 5% of these, so the noise term is not what
separates the candidates).

| nx | cell (mm) | cell (px) | x gain | y gain | retention x/y |
|---|---|---|---|---|---|
| 60 | 0.308 | 8.33 | 5.81 | 11.65 | 93% / 93% |
| 65 | 0.285 | 7.69 | 7.97 | 12.04 | 98% / 98% |
| 70 | 0.264 | 7.14 | 7.81 | 9.55 | 96% / 98% |
| 75 | 0.247 | 6.67 | 7.02 | 11.37 | 96% / 98% |
| **80** | **0.231** | **6.25** | **9.08** | 11.68 | **100% / 100%** |
| **85** | **0.218** | **5.88** | 8.68 | **12.88** | **101% / 99%** |
| **90** | **0.206** | **5.56** | 8.57 | 12.15 | 100% / 98% |
| 100 | 0.185 | 5.00 | 7.52 | 10.03 | 99% / 100% |
| 105 | 0.176 | 4.76 | 8.37 | 12.95 | 101% / 100% |
| 120 | 0.154 | 4.17 | 7.11 | 11.25 | 100% / 98% |

- **There is no sharp optimum — there is a broad plateau across nx 65-105.**
- **nx=85** has the best combined clean gain (21.6) with 101%/99% retention. **nx=80**
  (the shipped value) is statistically indistinguishable (20.8, best x gain at 9.08,
  100%/100% retention).
- **Coarsening past ~70 costs sensitivity**: nx=60 loses 36% of x gain (9.08 -> 5.81) and
  its retention drops to 93%/93%. Its ROI holds only ~1.6 cells.
- **Going finer than ~105 does not pay**: nx=110-120 decline on both gain and retention.
- Retention is flat (96-101%) across 65-115, so noise robustness does **not**
  discriminate in this range; it only penalises the coarsest candidate.

**On manufacturing:** at 500x500 the cell runs 154-308 um against a 50 um production
minimum, i.e. every option is 3-6x the process capability. **Manufacturability is not
binding at 500x500**, and coarsening past nx~70 buys nothing.

---

## 4. Pattern choice at 250x250

At 250x250 the camera-space cell halves, so the same *cell in pixels* needs half the nx
(the analogue of nx=80 at 500 is nx=40 at 250). 13 candidates, nx 30-120, `radius_mm=0.5`,
`dis_mode=0`, render scale, 5 noise draws. Gain is px per mm at 0.10 mm under the
calibrated sigma=4 model.

| nx | cell (mm) | cell (px) | x gain | y gain | retention x/y | mfg margin |
|---|---|---|---|---|---|---|
| 30 | 0.617 | 8.33 | 3.47 | 4.09 | 78% / 84% | 12.3x |
| 35 | 0.529 | 7.14 | 3.14 | 5.70 | 83% / 87% | 10.6x |
| **40** | **0.463** | **6.25** | 3.86 | 4.89 | 90% / 87% | **9.2x** |
| 45 | 0.411 | 5.56 | 4.58 | 4.40 | 90% / 84% | 8.2x |
| 50 | 0.370 | 5.00 | 2.22 | 5.73 | 74% / 93% | 7.4x |
| 55 | 0.336 | 4.55 | 3.04 | 3.62 | 82% / 85% | 6.7x |
| 60 | 0.308 | 4.17 | 3.02 | 5.29 | 87% / 90% | 6.2x |
| **65** | **0.285** | **3.85** | 4.32 | 5.32 | 95% / 98% | **5.7x** |
| 70 | 0.264 | 3.57 | 4.06 | 4.74 | 98% / 96% | 5.3x |
| 80 | 0.231 | 3.12 | 4.44 | 5.63 | 97% / 97% | 4.6x |
| **90** | **0.206** | **2.78** | 4.46 | 6.00 | 98% / 99% | **4.1x** |
| 105 | 0.176 | 2.38 | 4.29 | 6.11 | 98% / 97% | 3.5x |
| 120 | 0.154 | 2.08 | 3.44 | 5.07 | 88% / 91% | 3.1x |

**`dis_mode` must be 0 at 250x250.** With the shipped `dis_mode=1` the window is 2.4x the
moving region (window/moving = 2.42, section 2) and every candidate is weak or
unresolvable (nx=40: 1.42 px/mm instead of
4.30, and 16.8 instead of 25.0 on the render-scale score).

**Recommended at 250: nx = 40-65.**
- **nx=40** (cell 0.463 mm, **9.2x** the production minimum) if manufacturability dominates:
  gain 3.86/4.89 px/mm, retention 90%/87%.
- **nx=65** (cell 0.285 mm, still **5.7x** the minimum) if you want the sensitivity and
  noise robustness back: gain 4.32/5.32, retention 95%/98%.
- **Do not go finer than ~65** for sensitivity reasons: gain plateaus at ~4.3-4.5 px/mm
  across nx 65-105 while the cell keeps shrinking, so extra nx buys no signal. nx=120 is
  past the point where it helps and its retention drops.
- **Do not go coarser than ~40**: at nx=30-35 retention falls to 78-87%, because the ROI
  holds fewer than two marker cells.

**Caveat — the 250 ranking is less discriminating than the 500 one.** The ROI is only
6.6 px across, so the response is comparatively flat and some of the ordering is
regulariser behaviour rather than pattern quality. Treat the 250 optimum as the band
nx 40-65, not as a sharp point.

**Two further 250 limitations, measured on the end-to-end run at nx=80, `dis_mode=0`:**

- **The ROI holds only 37 pixels** (radius 3.31 px), against 139 at 500x500. The response
  is an ROI mean over 37 samples, so its own sampling error is much larger. This is the
  main reason the 250 sweep looks flat.
- **A 0.01 mm step is only 0.066 px** at 250 (0.133 px at 500). The ROI mean still resolves
  motion, but the per-step verdict machinery is markedly more conservative: the shipped
  nx=80 at 250/dm0 scores overall `unresolvable` even though the gain is a healthy
  4.21/5.57 px/mm. **If the application needs 0.01 mm steps resolved, 500x500 is the safer
  operating point**; 250 is better suited to coarser displacement resolution.
- At sigma=16 the small z response at 250 goes **negative** (-0.05 px/mm), i.e. noise can
  flip the sign of an axis whose signal is near the floor.

**Trade-off summary.** 250x250 forces `dis_mode=0` and roughly halves the absolute gain in
px/mm (4.30 vs 9.10 on x at the matched cell), but it buys a **2x coarser cell for the same
sensitivity**: 0.463 mm at nx=40 versus 0.231 mm at nx=80. Against a 50 um production
minimum that is 9.2x versus 4.6x of margin.


---

## 5. Which mode to run

| | 250x250 dm0 | 500x500 dm1 | 500x500 dm0 |
|---|---|---|---|
| output / solved at | 250 / **250** | 500 / 250 (upscaled) | 500 / **500** |
| window (`8*2**dm`) | 8 px | 16 px | 8 px |
| window / moving region | 1.21 | 1.21 | **0.60** |
| recovers of the true displacement | 66% | 68% | **86%** |
| background error (truth = 0) | 1.56 um | 1.77 um | **0.74 um** |
| edge tail at 1.88 mm | 4.6% | 4.8% | **1.4%** |
| ROI pixels | **37** | 139 | 139 |
| 0.01 mm step in px | **0.066** | 0.133 | 0.133 |
| DIS cost (medium, 500 px input) | 7.4 ms | 7.9 ms | **19.8 ms** |
| clean gain at nx=85 (x+y) | not comparable | 21.7 px/mm | **28.3 px/mm** |
| mirrors the deployed SDK config | no | **yes** | no |

Clean-gain units differ between resolutions (px per mm scales with the render width), so
compare 500 dm0 with 500 dm1 directly. At matched nx the clean gain is **1.31-1.35x higher
under dm0** (nx=80: 26.7 vs 20.8; nx=85: 28.3 vs 21.7; nx=90: 28.4 vs 21.0; nx=105: 28.1 vs
21.2 px/mm combined), i.e. **about a third more signal**. Compare 250 dm0 against the 500
modes only in *recovery* terms.

**Recommendation: 500x500 `dm0`**, with two conditions that override it.

1. **If the camera is physically 250x250, run 250x250 `dm0`** — resolution is set by
   hardware, not a free choice, and `dm1` there is broken (25% recovery, the field never
   returns to zero). 250 dm0 is informationally identical to 500 dm1 (same 250x250 solve,
   same window, matching profiles), so nothing is lost against an upsampled 500 field.
2. **If the deployed tracker is fixed at `dis_mode=1`**, run 500x500 `dm1` to predict
   deployment honestly — but know it is the pessimistic configuration: it discards 18
   percentage points of recovery by solving at half resolution.

**The single most actionable item: set `dis_mode=0` if the frame budget allows.** At
500x500 it is a ~30-35% sensitivity gain for 2.5x tracker time (19.8 ms vs 7.9 ms per frame;
40% of a 20 fps budget vs 16%), with no hardware change. That is a much larger effect than
any pattern choice in the plateau, which spans only ~5%.

**The pattern optimum survives the mode change.** Under both `dm0` and `dm1` at 500x500,
clean gain is flat within ~5% across nx 75-105, with nx 80-90 in the top group of both, and
nx <= 70 and nx >= 115 declining. So nx 80-90 is a safe choice either way; the dm0 sweep's
top candidates were nx 65/105/85/90/100, the dm1 sweep's were nx 85/105/90/80.

## 6. Recommendations

1. **Pick the mode first** (section 5), because it moves the instrument by ~30%.
   Then pick the pattern.

   - **Camera 500x500 and the tracker can run `dm0` → 500x500 `dm0`**, pattern
     **nx 80-90** (cell 0.231-0.206 mm, 4.6x-4.1x the production minimum). Best on every
     measured axis.
   - **Camera 500x500 but the tracker is fixed at `dm1` → 500x500 `dm1`**, pattern
     **nx 80-90**. Honest for deployment prediction; 18 percentage points of recovery are
     discarded by solving at half resolution.
   - **Camera 250x250 → 250x250 `dm0`** (mandatory), pattern **nx 40-65**
     (cell 0.463-0.285 mm, 9.2x-5.7x the production minimum).
   - **Never 250x250 with `dm1`**: window/moving-region = 2.42, roughly 3x of signal lost
     and the field is distorted rather than uniformly scaled.
2. **Set `dis_mode=0` wherever the frame budget allows.** At 500x500 that is ~30-35% more
   signal for 2.5x tracker time (19.8 ms vs 7.9 ms per frame). It is a software change with
   no hardware impact, and it is a far bigger effect than the ~5% spread across the pattern
   plateau.
3. **Do not treat the second scale as evidence** unless its window-to-moving-region ratio is
   checked. At 125x125 the ROI radius is 1.66 px and the moving region only ~3.3 px, against
   an 8 px window — ratio 2.4, the same broken regime as 250/dm1.
4. **Before committing to a final nx, average 3-5 seeds per nx.** The y gain is
   non-monotonic at both resolutions (9.55 at nx=70 vs 12.88 at nx=85 at 500 dm1), and with
   a single seed per nx part of that is *which* pattern rather than the grid size.
5. **Re-run `optimise` under the noise-aware objective** before trusting any automated
   winner. The stored optimizer result (nx=31) came from the retracted render-seed score
   and has been removed.


---

## 7. What is not established

- **Absolute force calibration.** Sensitivity is a relative ranking under a fixed estimator
  and camera, not newtons.
- **The process scale** (250x250 at 500 native) and the `radius_mm=1.5` block have not been
  re-ranked under the calibrated model.
- **The real floor's pattern trend.** The idle clip gives one pattern, so it validates the
  floor's magnitude and mechanism but not how it varies with nx. A deliberate no-contact
  recording at the shipped configuration would settle it.
- **The 500x500 optimum's dependence on the random draw.** One seed per nx.
- **Multi-seed null rendering** — now diagnostic only, not part of the score.

---

## 8. Defects found and fixed while producing this

Recorded because each one silently corrupted results at the time:

- **`build_track_job` omitted the `noise` block**, so `design.yaml`'s sigmas/draws/scales
  and every `--noise-*` flag were ignored and the pass ran on built-in defaults that
  happened to match. Caught when a 5-draw run reproduced 1-draw numbers exactly.
- **The injected floor used a signed p95** of the projected flow, so it came out negative
  or near-zero and the objective divided by ~0 — scores of ~1e7. Now `p95(|projection|)`,
  guarded below by the render-seed reproducibility limit.
- **Re-running `track --output-dir` on a variant silently rebuilt the job from
  `design.yaml`**, overwriting a `dis_mode=0` run with `dis_mode=1`. Now the existing job is
  reused unless `--rebuild-job` is passed.
- **Manual re-scoring without the candidate pattern** reset every candidate's `cell_px` to
  the `design.yaml` grid. `compare` does this correctly; ad-hoc re-scoring must pass
  `pattern=` explicitly.
- **The response figure's label omitted the radius**, so `nx80@r0.5@dm0` and
  `nx80@r1.5@dm0` produced two identical legend entries, and one global floor band belonged
  to no plotted curve. Both fixed.