# Probe bench: findings and recommendations Short, decision-oriented summary of what the sensitivity bench has established. The mechanics are in `docs/probe-pipeline.md` (contract) and `docs/probe-method.md` (operator guide). This file is the *conclusion* — read it first. Last updated after the nx 60-120 sweep at 500x500 and the 250x250 cross-check. --- ## 1. Retraction: the noise floor was not a noise model An earlier version of these docs claimed **nx=31 is the best pattern** and **"the marker cell must match the estimator's effective window"**. Both are **retracted**. `floor_px` was measured as `reference` versus a re-render at **render seed + 1** — Cycles antialiasing jitter, about 5 grey levels RMS on half the pixels, scaling with edge contrast. It is not sensor noise, and it cannot rank patterns: 1. **Its trend reverses** under additive noise. p95 |flow| over the ROI, same DIS config: | pattern | render-seed | injected sigma=2 | sigma=4 | |---|---|---|---| | nx=80 | 0.0292 | **0.0161** | 0.0278 | | nx=31 | 0.0376 | **0.0084** | 0.0182 | | nx=56 | 0.0134 | 0.0102 | 0.0203 | 2. **It understates the real floor, coarse patterns most.** Measured on idle footage (`data/dd02-0703/190.mp4`, 1280x720) with the same estimator: **0.2267 px**, dominant marker period **25.5 px**, i.e. **0.0089 floor per period**; short baseline equals long baseline (ratio **0.98**), so it is frame-independent white noise and the clip is genuinely idle. Simulated floors per cell sit about **2x** (nx=80), **4x** (nx=31) and **6x** (nx=56) below it — the bias that made the coarse patterns win. 3. **Signal is not noise-invariant either.** Retention `gain(sigma)/gain(clean)` at +0.10 mm, x axis, render scale: nx=80 dm1 **99%** at sigma=4 and **84%** at sigma=16, against nx=31 dm1 **72%** and **59%**. **What replaced it.** The render-seed floor is kept only as a reproducibility diagnostic and is labelled `floor_basis: "render_seed_reproducibility"`. Signal and noise are now measured under **one** injected-noise model (`noise_models` per axis: gain, floor, `response_retention`, `snr_noise`), calibrated against the real idle footage. The real number is cached as `idle_floor.json`; skip with `--no-calibration`. --- ## 2. What actually governs the measurement: the estimator window vs the moving region This is the most useful result, and it explains every resolution effect seen. Earlier notes called it "ROI versus effective window"; that was the right *variable* but the wrong *mechanism*, and measuring it changed the fix. See `build/probe/finger-xynova/r01/roi_regime.png`. ### The two regions, concretely - **The moving region.** `radius_mm` (0.5) is the proportional-editing falloff radius for the deformation, so the surface that actually moves is roughly a 1 mm disc — about **13.25 px across at 500x500** and **6.62 px at 250x250**. It is a fixed *physical* size, so its pixel size is set by the render resolution. - **The measurement ROI.** A disc of radius `radius_mm x px_per_mm` centred on the projected probe centroid. The response is the **mean of `dot(flow, d_hat)` over that disc**. At 500x500 it is 6.62 px radius / 139 pixels; at 250x250, 3.31 px radius / **37 pixels**. - **The estimator window.** DIS matches an 8 px patch, and `dis_mode` is OpenCV's `setFinestScale`, so the flow is computed at `2**dis_mode` lower resolution and upscaled. The effective support is therefore `8 x 2**dis_mode` **input** pixels: **16 px at `dis_mode=1`, 8 px at `dis_mode=0`**. ### The mechanism is attenuation inside the estimator, not dilution in the average Both were plausible. They are distinguished by comparing the flow **at the probe centroid** against the **mean over the ROI**, and by sweeping the ROI radius over one fixed flow field: | config | window | applied | flow @ centroid | mean over ROI | recovery | |---|---|---|---|---|---| | 500 dm1 | 16 px | 1.3248 px | -0.928 | -0.897 | **70%** | | 500 dm0 | 8 px | 1.3248 px | -1.170 | -1.134 | **88%** | | 250 dm1 | 16 px | 0.6624 px | **-0.166** | -0.161 | **25%** | | 250 dm0 | 8 px | 0.6624 px | -0.482 | -0.429 | **73%** | The centroid value and the ROI mean agree closely **in every case**, and sweeping the ROI radius from 1 px to 30 px shows the response is **flat across the whole moving region** and only falls off once the disc grows past it. So the ROI is well placed in all four cases and the average is not diluting anything. The attenuation is already present in the flow field that DIS produces. **Why DIS attenuates it.** DIS minimises a photometric term plus a smoothness term over a patch-and-propagation neighbourhood of about the window size. When the true displacement is nonzero only inside a region *smaller* than that neighbourhood, most of the pixels the fit sees are static, and both terms pull the estimate toward zero: the photometric term is best satisfied by the static majority, and the smoothness term penalises a localised spike. The attenuation therefore depends on how much of the window actually moves. ### The law Recovery is a function of **`window / moving-region diameter`**, not of resolution: | config | window / moving diameter | recovery | |---|---|---| | 500 dm0 | 8 / 13.25 = **0.60** | 88% | | 500 dm1 | 16 / 13.25 = **1.21** | 70% | | 250 dm0 | 8 / 6.62 = **1.21** | 73% | | 250 dm1 | 16 / 6.62 = **2.42** | 25% | The two configurations at ratio 1.21 differ by a **factor of two in resolution** and agree to **3 percentage points**. That is the whole story: the governing quantity is the ratio of the estimator's support to the physical size of the thing you are measuring. ### Which mode for a spatially resolved deformation map? `dis_mode` changes **two** things, and only one is obvious: 1. the **window** (`8 x 2**dis_mode` px), which sets the attenuation; and 2. the **internal solve grid**. `setFinestScale(n)` solves the flow at `2**n` lower resolution and bilinearly upscales to the output size. So: | mode | output | actually solved at | window | recovers | background error | tail @1.88 mm | |---|---|---|---|---|---|---| | **500 dm0** | 500x500 | **500x500** | 8 px | **86%** | **0.74 um** | **1.4%** | | 500 dm1 | 500x500 | 250x250 | 16 px | 68% | 1.77 um | 4.8% | | 250 dm0 | 250x250 | 250x250 | 8 px | 66% | 1.56 um | 4.6% | | 250 dm1 | 250x250 | 125x125 | 16 px | 25% | 4.24 um | **26.7%** | ("tail" is the residual flow at 1.88 mm from the probe, where the surface is undeformed, as a fraction of each mode's own plateau; "background error" is the mean spurious displacement in the 1.5-3.0 mm annulus, in micrometres. Figure: `build/probe/finger-xynova/r01/edge_quality.png`.) **So 250x250 `dm0` is *not* sharper than 500x500.** In physical units the edge width is the same (`r50` = 1.12 mm for 500 dm0, 500 dm1 and 250 dm0) and the normalised profiles of **500 dm1 and 250 dm0 nearly coincide** — because they are the *same computation*: both solve on a 250x250 grid with an 8 px window there. A half-resolution round trip through the 500 dm1 field leaves a residual of only **0.019** of its RMS, confirming it carries no detail above 250x250; it is a 250x250 field upsampled onto 4x as many pixels. **The sharpest mode is 500x500 `dm0`**, the only one that solves at its full output resolution. It recovers 86% of the true displacement (vs 66%), has a 2x cleaner background, and a **3x smaller edge tail** (1.4% vs 4.6%). An apparent "sharper falloff" for 250 dm0 is a **unit artefact**: on the ROI-radius sweep plotted in *pixels* the 250 curve looks steeper only because the whole scene is half the size in pixels. Plotted in millimetres the two profiles overlay (0.287 vs 0.280 of plateau at 1.12 mm). **250 dm1 is unusable for a map at any ROI size**: 25% recovery, and the profile never returns to zero (26.7% of the plateau at 1.88 mm, still 7.3% at 2.88 mm). **Important qualification: the bench does not currently produce a map.** It reports one scalar per probe position — the mean of `dot(flow, d_hat)` over a small disc (139 px at 500, 37 px at 250). The profiles above describe the *footprint of a single probe*, measured by evaluating the full flow field. The far-field numbers do show that field returning to ~0 outside the deformation (0.74-1.56 um), so a genuine spatial map is feasible — but it would be a new output, and it should be built on 500 dm0. ### Consequences 1. **Keep `window / moving diameter` at or below ~1.2.** Above ~2 the estimator is mostly measuring its own smoothness prior. This is why 250x250 with the shipped `dis_mode=1` (ratio 2.42) loses ~3x of its signal, and why `dis_mode=0` fixes it. 2. **A larger ROI does not help.** The sweep is monotone *decreasing* past the moving region, so widening the ROI to "get more signal" only adds static background. Three levers exist instead: raise the render resolution, reduce the window (`dis_mode`), or make the deformation physically larger (`radius_mm` / a bigger probe). 3. **Even at ratio ~1.2 you only recover ~70%.** Every absolute sensitivity number in this bench is attenuated by 12-75% depending on configuration. Relative *pattern* rankings are still valid because the geometry is held fixed, but do not read `gain_px_per_mm` as the true displacement sensitivity. 4. **At 250 dm1 the field is distorted, not merely scaled.** `max|du|` reaches 121% of the applied displacement somewhere in the frame while the probe region sits at 25% — spurious high-flow pixels plus an attenuated centre. At 250 dm0 the peak (73%) matches the ROI recovery, i.e. a clean uniform scale factor. So the 250/dm1 response is not a reliable fraction of the truth at any ROI size. --- ## 3. Pattern choice at 500x500: nx = 80-90 13 candidates, nx 60-120, `radius_mm=0.5`, `dis_mode=1`, render scale, 5 noise draws. Gain is px per mm of applied displacement at 0.10 mm, measured **under the calibrated sigma=4 noise model** (clean gain is within 5% of these, so the noise term is not what separates the candidates). | nx | cell (mm) | cell (px) | x gain | y gain | retention x/y | |---|---|---|---|---|---| | 60 | 0.308 | 8.33 | 5.81 | 11.65 | 93% / 93% | | 65 | 0.285 | 7.69 | 7.97 | 12.04 | 98% / 98% | | 70 | 0.264 | 7.14 | 7.81 | 9.55 | 96% / 98% | | 75 | 0.247 | 6.67 | 7.02 | 11.37 | 96% / 98% | | **80** | **0.231** | **6.25** | **9.08** | 11.68 | **100% / 100%** | | **85** | **0.218** | **5.88** | 8.68 | **12.88** | **101% / 99%** | | **90** | **0.206** | **5.56** | 8.57 | 12.15 | 100% / 98% | | 100 | 0.185 | 5.00 | 7.52 | 10.03 | 99% / 100% | | 105 | 0.176 | 4.76 | 8.37 | 12.95 | 101% / 100% | | 120 | 0.154 | 4.17 | 7.11 | 11.25 | 100% / 98% | - **There is no sharp optimum — there is a broad plateau across nx 65-105.** - **nx=85** has the best combined clean gain (21.6) with 101%/99% retention. **nx=80** (the shipped value) is statistically indistinguishable (20.8, best x gain at 9.08, 100%/100% retention). - **Coarsening past ~70 costs sensitivity**: nx=60 loses 36% of x gain (9.08 -> 5.81) and its retention drops to 93%/93%. Its ROI holds only ~1.6 cells. - **Going finer than ~105 does not pay**: nx=110-120 decline on both gain and retention. - Retention is flat (96-101%) across 65-115, so noise robustness does **not** discriminate in this range; it only penalises the coarsest candidate. **On manufacturing:** at 500x500 the cell runs 154-308 um against a 50 um production minimum, i.e. every option is 3-6x the process capability. **Manufacturability is not binding at 500x500**, and coarsening past nx~70 buys nothing. --- ## 4. Pattern choice at 250x250 At 250x250 the camera-space cell halves, so the same *cell in pixels* needs half the nx (the analogue of nx=80 at 500 is nx=40 at 250). 13 candidates, nx 30-120, `radius_mm=0.5`, `dis_mode=0`, render scale, 5 noise draws. Gain is px per mm at 0.10 mm under the calibrated sigma=4 model. | nx | cell (mm) | cell (px) | x gain | y gain | retention x/y | mfg margin | |---|---|---|---|---|---|---| | 30 | 0.617 | 8.33 | 3.47 | 4.09 | 78% / 84% | 12.3x | | 35 | 0.529 | 7.14 | 3.14 | 5.70 | 83% / 87% | 10.6x | | **40** | **0.463** | **6.25** | 3.86 | 4.89 | 90% / 87% | **9.2x** | | 45 | 0.411 | 5.56 | 4.58 | 4.40 | 90% / 84% | 8.2x | | 50 | 0.370 | 5.00 | 2.22 | 5.73 | 74% / 93% | 7.4x | | 55 | 0.336 | 4.55 | 3.04 | 3.62 | 82% / 85% | 6.7x | | 60 | 0.308 | 4.17 | 3.02 | 5.29 | 87% / 90% | 6.2x | | **65** | **0.285** | **3.85** | 4.32 | 5.32 | 95% / 98% | **5.7x** | | 70 | 0.264 | 3.57 | 4.06 | 4.74 | 98% / 96% | 5.3x | | 80 | 0.231 | 3.12 | 4.44 | 5.63 | 97% / 97% | 4.6x | | **90** | **0.206** | **2.78** | 4.46 | 6.00 | 98% / 99% | **4.1x** | | 105 | 0.176 | 2.38 | 4.29 | 6.11 | 98% / 97% | 3.5x | | 120 | 0.154 | 2.08 | 3.44 | 5.07 | 88% / 91% | 3.1x | **`dis_mode` must be 0 at 250x250.** With the shipped `dis_mode=1` the window is 2.4x the moving region (window/moving = 2.42, section 2) and every candidate is weak or unresolvable (nx=40: 1.42 px/mm instead of 4.30, and 16.8 instead of 25.0 on the render-scale score). **Recommended at 250: nx = 40-65.** - **nx=40** (cell 0.463 mm, **9.2x** the production minimum) if manufacturability dominates: gain 3.86/4.89 px/mm, retention 90%/87%. - **nx=65** (cell 0.285 mm, still **5.7x** the minimum) if you want the sensitivity and noise robustness back: gain 4.32/5.32, retention 95%/98%. - **Do not go finer than ~65** for sensitivity reasons: gain plateaus at ~4.3-4.5 px/mm across nx 65-105 while the cell keeps shrinking, so extra nx buys no signal. nx=120 is past the point where it helps and its retention drops. - **Do not go coarser than ~40**: at nx=30-35 retention falls to 78-87%, because the ROI holds fewer than two marker cells. **Caveat — the 250 ranking is less discriminating than the 500 one.** The ROI is only 6.6 px across, so the response is comparatively flat and some of the ordering is regulariser behaviour rather than pattern quality. Treat the 250 optimum as the band nx 40-65, not as a sharp point. **Two further 250 limitations, measured on the end-to-end run at nx=80, `dis_mode=0`:** - **The ROI holds only 37 pixels** (radius 3.31 px), against 139 at 500x500. The response is an ROI mean over 37 samples, so its own sampling error is much larger. This is the main reason the 250 sweep looks flat. - **A 0.01 mm step is only 0.066 px** at 250 (0.133 px at 500). The ROI mean still resolves motion, but the per-step verdict machinery is markedly more conservative: the shipped nx=80 at 250/dm0 scores overall `unresolvable` even though the gain is a healthy 4.21/5.57 px/mm. **If the application needs 0.01 mm steps resolved, 500x500 is the safer operating point**; 250 is better suited to coarser displacement resolution. - At sigma=16 the small z response at 250 goes **negative** (-0.05 px/mm), i.e. noise can flip the sign of an axis whose signal is near the floor. **Trade-off summary.** 250x250 forces `dis_mode=0` and roughly halves the absolute gain in px/mm (4.30 vs 9.10 on x at the matched cell), but it buys a **2x coarser cell for the same sensitivity**: 0.463 mm at nx=40 versus 0.231 mm at nx=80. Against a 50 um production minimum that is 9.2x versus 4.6x of margin. --- ## 5. Which mode to run | | 250x250 dm0 | 500x500 dm1 | 500x500 dm0 | |---|---|---|---| | output / solved at | 250 / **250** | 500 / 250 (upscaled) | 500 / **500** | | window (`8*2**dm`) | 8 px | 16 px | 8 px | | window / moving region | 1.21 | 1.21 | **0.60** | | recovers of the true displacement | 66% | 68% | **86%** | | background error (truth = 0) | 1.56 um | 1.77 um | **0.74 um** | | edge tail at 1.88 mm | 4.6% | 4.8% | **1.4%** | | ROI pixels | **37** | 139 | 139 | | 0.01 mm step in px | **0.066** | 0.133 | 0.133 | | DIS cost (medium, 500 px input) | 7.4 ms | 7.9 ms | **19.8 ms** | | clean gain at nx=85 (x+y) | not comparable | 21.7 px/mm | **28.3 px/mm** | | mirrors the deployed SDK config | no | **yes** | no | Clean-gain units differ between resolutions (px per mm scales with the render width), so compare 500 dm0 with 500 dm1 directly. At matched nx the clean gain is **1.31-1.35x higher under dm0** (nx=80: 26.7 vs 20.8; nx=85: 28.3 vs 21.7; nx=90: 28.4 vs 21.0; nx=105: 28.1 vs 21.2 px/mm combined), i.e. **about a third more signal**. Compare 250 dm0 against the 500 modes only in *recovery* terms. **Recommendation: 500x500 `dm0`**, with two conditions that override it. 1. **If the camera is physically 250x250, run 250x250 `dm0`** — resolution is set by hardware, not a free choice, and `dm1` there is broken (25% recovery, the field never returns to zero). 250 dm0 is informationally identical to 500 dm1 (same 250x250 solve, same window, matching profiles), so nothing is lost against an upsampled 500 field. 2. **If the deployed tracker is fixed at `dis_mode=1`**, run 500x500 `dm1` to predict deployment honestly — but know it is the pessimistic configuration: it discards 18 percentage points of recovery by solving at half resolution. **The single most actionable item: set `dis_mode=0` if the frame budget allows.** At 500x500 it is a ~30-35% sensitivity gain for 2.5x tracker time (19.8 ms vs 7.9 ms per frame; 40% of a 20 fps budget vs 16%), with no hardware change. That is a much larger effect than any pattern choice in the plateau, which spans only ~5%. **The pattern optimum survives the mode change.** Under both `dm0` and `dm1` at 500x500, clean gain is flat within ~5% across nx 75-105, with nx 80-90 in the top group of both, and nx <= 70 and nx >= 115 declining. So nx 80-90 is a safe choice either way; the dm0 sweep's top candidates were nx 65/105/85/90/100, the dm1 sweep's were nx 85/105/90/80. ## 6. Recommendations 1. **Pick the mode first** (section 5), because it moves the instrument by ~30%. Then pick the pattern. - **Camera 500x500 and the tracker can run `dm0` → 500x500 `dm0`**, pattern **nx 80-90** (cell 0.231-0.206 mm, 4.6x-4.1x the production minimum). Best on every measured axis. - **Camera 500x500 but the tracker is fixed at `dm1` → 500x500 `dm1`**, pattern **nx 80-90**. Honest for deployment prediction; 18 percentage points of recovery are discarded by solving at half resolution. - **Camera 250x250 → 250x250 `dm0`** (mandatory), pattern **nx 40-65** (cell 0.463-0.285 mm, 9.2x-5.7x the production minimum). - **Never 250x250 with `dm1`**: window/moving-region = 2.42, roughly 3x of signal lost and the field is distorted rather than uniformly scaled. 2. **Set `dis_mode=0` wherever the frame budget allows.** At 500x500 that is ~30-35% more signal for 2.5x tracker time (19.8 ms vs 7.9 ms per frame). It is a software change with no hardware impact, and it is a far bigger effect than the ~5% spread across the pattern plateau. 3. **Do not treat the second scale as evidence** unless its window-to-moving-region ratio is checked. At 125x125 the ROI radius is 1.66 px and the moving region only ~3.3 px, against an 8 px window — ratio 2.4, the same broken regime as 250/dm1. 4. **Before committing to a final nx, average 3-5 seeds per nx.** The y gain is non-monotonic at both resolutions (9.55 at nx=70 vs 12.88 at nx=85 at 500 dm1), and with a single seed per nx part of that is *which* pattern rather than the grid size. 5. **Re-run `optimise` under the noise-aware objective** before trusting any automated winner. The stored optimizer result (nx=31) came from the retracted render-seed score and has been removed. --- ## 7. What is not established - **Absolute force calibration.** Sensitivity is a relative ranking under a fixed estimator and camera, not newtons. - **The process scale** (250x250 at 500 native) and the `radius_mm=1.5` block have not been re-ranked under the calibrated model. - **The real floor's pattern trend.** The idle clip gives one pattern, so it validates the floor's magnitude and mechanism but not how it varies with nx. A deliberate no-contact recording at the shipped configuration would settle it. - **The 500x500 optimum's dependence on the random draw.** One seed per nx. - **Multi-seed null rendering** — now diagnostic only, not part of the score. --- ## 8. Defects found and fixed while producing this Recorded because each one silently corrupted results at the time: - **`build_track_job` omitted the `noise` block**, so `design.yaml`'s sigmas/draws/scales and every `--noise-*` flag were ignored and the pass ran on built-in defaults that happened to match. Caught when a 5-draw run reproduced 1-draw numbers exactly. - **The injected floor used a signed p95** of the projected flow, so it came out negative or near-zero and the objective divided by ~0 — scores of ~1e7. Now `p95(|projection|)`, guarded below by the render-seed reproducibility limit. - **Re-running `track --output-dir` on a variant silently rebuilt the job from `design.yaml`**, overwriting a `dis_mode=0` run with `dis_mode=1`. Now the existing job is reused unless `--rebuild-job` is passed. - **Manual re-scoring without the candidate pattern** reset every candidate's `cell_px` to the `design.yaml` grid. `compare` does this correctly; ad-hoc re-scoring must pass `pattern=` explicitly. - **The response figure's label omitted the radius**, so `nx80@r0.5@dm0` and `nx80@r1.5@dm0` produced two identical legend entries, and one global floor band belonged to no plotted curve. Both fixed.