orisys-sensor-design/docs/probe-findings.md

Probe bench: findings and recommendations

Short, decision-oriented summary of what the sensitivity bench has established. The mechanics are in docs/probe-pipeline.md (contract) and docs/probe-method.md (operator guide). This file is the conclusion — read it first.

Last updated after the nx 60-120 sweep at 500x500 and the 250x250 cross-check.

---

1. Retraction: the noise floor was not a noise model

An earlier version of these docs claimed nx=31 is the best pattern and "the marker cell must match the estimator's effective window". Both are retracted.

floor_px was measured as reference versus a re-render at render seed + 1 — Cycles antialiasing jitter, about 5 grey levels RMS on half the pixels, scaling with edge contrast. It is not sensor noise, and it cannot rank patterns:

1. Its trend reverses under additive noise. p95 |flow| over the ROI, same DIS config:

   pattern	render-seed	injected sigma=2	sigma=4
   nx=80	0.0292	0.0161	0.0278
   nx=31	0.0376	0.0084	0.0182
   nx=56	0.0134	0.0102	0.0203

2. It understates the real floor, coarse patterns most. Measured on idle footage (data/dd02-0703/190.mp4, 1280x720) with the same estimator: 0.2267 px, dominant marker period 25.5 px, i.e. 0.0089 floor per period; short baseline equals long baseline (ratio 0.98), so it is frame-independent white noise and the clip is genuinely idle. Simulated floors per cell sit about 2x (nx=80), 4x (nx=31) and 6x (nx=56) below it — the bias that made the coarse patterns win.

3. Signal is not noise-invariant either. Retention gain(sigma)/gain(clean) at +0.10 mm, x axis, render scale: nx=80 dm1 99% at sigma=4 and 84% at sigma=16, against nx=31 dm1 72% and 59%.

What replaced it. The render-seed floor is kept only as a reproducibility diagnostic and is labelled floor_basis: "render_seed_reproducibility". Signal and noise are now measured under one injected-noise model (noise_models per axis: gain, floor, response_retention, snr_noise), calibrated against the real idle footage. The real number is cached as idle_floor.json; skip with --no-calibration.

---

2. What actually governs the measurement: the estimator window vs the moving region

This is the most useful result, and it explains every resolution effect seen. Earlier notes called it "ROI versus effective window"; that was the right variable but the wrong mechanism, and measuring it changed the fix. See build/probe/finger-xynova/r01/roi_regime.png.

The two regions, concretely

• The moving region. radius_mm (0.5) is the proportional-editing falloff radius for the deformation, so the surface that actually moves is roughly a 1 mm disc — about 13.25 px across at 500x500 and 6.62 px at 250x250. It is a fixed physical size, so its pixel size is set by the render resolution.
• The measurement ROI. A disc of radius radius_mm x px_per_mm centred on the projected probe centroid. The response is the mean of dot(flow, d_hat) over that disc. At 500x500 it is 6.62 px radius / 139 pixels; at 250x250, 3.31 px radius / 37 pixels.
• The estimator window. DIS matches an 8 px patch, and dis_mode is OpenCV's setFinestScale, so the flow is computed at 2**dis_mode lower resolution and upscaled. The effective support is therefore 8 x 2**dis_mode input pixels: 16 px at dis_mode=1, 8 px at dis_mode=0.

The mechanism is attenuation inside the estimator, not dilution in the average

Both were plausible. They are distinguished by comparing the flow at the probe centroid against the mean over the ROI, and by sweeping the ROI radius over one fixed flow field:

config	window	applied	flow @ centroid	mean over ROI	recovery
500 dm1	16 px	1.3248 px	-0.928	-0.897	70%
500 dm0	8 px	1.3248 px	-1.170	-1.134	88%
250 dm1	16 px	0.6624 px	-0.166	-0.161	25%
250 dm0	8 px	0.6624 px	-0.482	-0.429	73%

The centroid value and the ROI mean agree closely in every case, and sweeping the ROI radius from 1 px to 30 px shows the response is flat across the whole moving region and only falls off once the disc grows past it. So the ROI is well placed in all four cases and the average is not diluting anything. The attenuation is already present in the flow field that DIS produces.

Why DIS attenuates it. DIS minimises a photometric term plus a smoothness term over a patch-and-propagation neighbourhood of about the window size. When the true displacement is nonzero only inside a region smaller than that neighbourhood, most of the pixels the fit sees are static, and both terms pull the estimate toward zero: the photometric term is best satisfied by the static majority, and the smoothness term penalises a localised spike. The attenuation therefore depends on how much of the window actually moves.

The law

Recovery is a function of window / moving-region diameter, not of resolution:

config	window / moving diameter	recovery
500 dm0	8 / 13.25 = 0.60	88%
500 dm1	16 / 13.25 = 1.21	70%
250 dm0	8 / 6.62 = 1.21	73%
250 dm1	16 / 6.62 = 2.42	25%

The two configurations at ratio 1.21 differ by a factor of two in resolution and agree to 3 percentage points. That is the whole story: the governing quantity is the ratio of the estimator's support to the physical size of the thing you are measuring.

Which mode for a spatially resolved deformation map?

dis_mode changes two things, and only one is obvious:

1. the window (8 x 2**dis_mode px), which sets the attenuation; and
2. the internal solve grid. setFinestScale(n) solves the flow at 2**n lower resolution and bilinearly upscales to the output size. So:

mode	output	actually solved at	window	recovers	background error	tail @1.88 mm
500 dm0	500x500	500x500	8 px	86%	0.74 um	1.4%
500 dm1	500x500	250x250	16 px	68%	1.77 um	4.8%
250 dm0	250x250	250x250	8 px	66%	1.56 um	4.6%
250 dm1	250x250	125x125	16 px	25%	4.24 um	26.7%

("tail" is the residual flow at 1.88 mm from the probe, where the surface is undeformed, as a fraction of each mode's own plateau; "background error" is the mean spurious displacement in the 1.5-3.0 mm annulus, in micrometres. Figure: build/probe/finger-xynova/r01/edge_quality.png.)

So 250x250 dm0 is not sharper than 500x500. In physical units the edge width is the same (r50 = 1.12 mm for 500 dm0, 500 dm1 and 250 dm0) and the normalised profiles of 500 dm1 and 250 dm0 nearly coincide — because they are the same computation: both solve on a 250x250 grid with an 8 px window there. A half-resolution round trip through the 500 dm1 field leaves a residual of only 0.019 of its RMS, confirming it carries no detail above 250x250; it is a 250x250 field upsampled onto 4x as many pixels.

The sharpest mode is 500x500 dm0, the only one that solves at its full output resolution. It recovers 86% of the true displacement (vs 66%), has a 2x cleaner background, and a 3x smaller edge tail (1.4% vs 4.6%).

An apparent "sharper falloff" for 250 dm0 is a unit artefact: on the ROI-radius sweep plotted in pixels the 250 curve looks steeper only because the whole scene is half the size in pixels. Plotted in millimetres the two profiles overlay (0.287 vs 0.280 of plateau at 1.12 mm).

250 dm1 is unusable for a map at any ROI size: 25% recovery, and the profile never returns to zero (26.7% of the plateau at 1.88 mm, still 7.3% at 2.88 mm).

Important qualification: the bench does not currently produce a map. It reports one scalar per probe position — the mean of dot(flow, d_hat) over a small disc (139 px at 500, 37 px at 250). The profiles above describe the footprint of a single probe, measured by evaluating the full flow field. The far-field numbers do show that field returning to ~0 outside the deformation (0.74-1.56 um), so a genuine spatial map is feasible — but it would be a new output, and it should be built on 500 dm0.

Consequences

1. Keep window / moving diameter at or below ~1.2. Above ~2 the estimator is mostly measuring its own smoothness prior. This is why 250x250 with the shipped dis_mode=1 (ratio 2.42) loses ~3x of its signal, and why dis_mode=0 fixes it.
2. A larger ROI does not help. The sweep is monotone decreasing past the moving region, so widening the ROI to "get more signal" only adds static background. Three levers exist instead: raise the render resolution, reduce the window (dis_mode), or make the deformation physically larger (radius_mm / a bigger probe).
3. Even at ratio ~1.2 you only recover ~70%. Every absolute sensitivity number in this bench is attenuated by 12-75% depending on configuration. Relative pattern rankings are still valid because the geometry is held fixed, but do not read gain_px_per_mm as the true displacement sensitivity.
4. At 250 dm1 the field is distorted, not merely scaled. max|du| reaches 121% of the applied displacement somewhere in the frame while the probe region sits at 25% — spurious high-flow pixels plus an attenuated centre. At 250 dm0 the peak (73%) matches the ROI recovery, i.e. a clean uniform scale factor. So the 250/dm1 response is not a reliable fraction of the truth at any ROI size.

---

3. Pattern choice at 500x500: nx = 80-90

13 candidates, nx 60-120, radius_mm=0.5, dis_mode=1, render scale, 5 noise draws. Gain is px per mm of applied displacement at 0.10 mm, measured under the calibrated sigma=4 noise model (clean gain is within 5% of these, so the noise term is not what separates the candidates).

nx	cell (mm)	cell (px)	x gain	y gain	retention x/y
60	0.308	8.33	5.81	11.65	93% / 93%
65	0.285	7.69	7.97	12.04	98% / 98%
70	0.264	7.14	7.81	9.55	96% / 98%
75	0.247	6.67	7.02	11.37	96% / 98%
80	0.231	6.25	9.08	11.68	100% / 100%
85	0.218	5.88	8.68	12.88	101% / 99%
90	0.206	5.56	8.57	12.15	100% / 98%
100	0.185	5.00	7.52	10.03	99% / 100%
105	0.176	4.76	8.37	12.95	101% / 100%
120	0.154	4.17	7.11	11.25	100% / 98%

• There is no sharp optimum — there is a broad plateau across nx 65-105.
• nx=85 has the best combined clean gain (21.6) with 101%/99% retention. nx=80 (the shipped value) is statistically indistinguishable (20.8, best x gain at 9.08, 100%/100% retention).
• Coarsening past ~70 costs sensitivity: nx=60 loses 36% of x gain (9.08 -> 5.81) and its retention drops to 93%/93%. Its ROI holds only ~1.6 cells.
• Going finer than ~105 does not pay: nx=110-120 decline on both gain and retention.
• Retention is flat (96-101%) across 65-115, so noise robustness does not discriminate in this range; it only penalises the coarsest candidate.

On manufacturing: at 500x500 the cell runs 154-308 um against a 50 um production minimum, i.e. every option is 3-6x the process capability. Manufacturability is not binding at 500x500, and coarsening past nx~70 buys nothing.

---

4. Pattern choice at 250x250

At 250x250 the camera-space cell halves, so the same cell in pixels needs half the nx (the analogue of nx=80 at 500 is nx=40 at 250). 13 candidates, nx 30-120, radius_mm=0.5, dis_mode=0, render scale, 5 noise draws. Gain is px per mm at 0.10 mm under the calibrated sigma=4 model.

nx	cell (mm)	cell (px)	x gain	y gain	retention x/y	mfg margin
30	0.617	8.33	3.47	4.09	78% / 84%	12.3x
35	0.529	7.14	3.14	5.70	83% / 87%	10.6x
40	0.463	6.25	3.86	4.89	90% / 87%	9.2x
45	0.411	5.56	4.58	4.40	90% / 84%	8.2x
50	0.370	5.00	2.22	5.73	74% / 93%	7.4x
55	0.336	4.55	3.04	3.62	82% / 85%	6.7x
60	0.308	4.17	3.02	5.29	87% / 90%	6.2x
65	0.285	3.85	4.32	5.32	95% / 98%	5.7x
70	0.264	3.57	4.06	4.74	98% / 96%	5.3x
80	0.231	3.12	4.44	5.63	97% / 97%	4.6x
90	0.206	2.78	4.46	6.00	98% / 99%	4.1x
105	0.176	2.38	4.29	6.11	98% / 97%	3.5x
120	0.154	2.08	3.44	5.07	88% / 91%	3.1x

dis_mode must be 0 at 250x250. With the shipped dis_mode=1 the window is 2.4x the moving region (window/moving = 2.42, section 2) and every candidate is weak or unresolvable (nx=40: 1.42 px/mm instead of 4.30, and 16.8 instead of 25.0 on the render-scale score).

Recommended at 250: nx = 40-65.

• nx=40 (cell 0.463 mm, 9.2x the production minimum) if manufacturability dominates: gain 3.86/4.89 px/mm, retention 90%/87%.
• nx=65 (cell 0.285 mm, still 5.7x the minimum) if you want the sensitivity and noise robustness back: gain 4.32/5.32, retention 95%/98%.
• Do not go finer than ~65 for sensitivity reasons: gain plateaus at ~4.3-4.5 px/mm across nx 65-105 while the cell keeps shrinking, so extra nx buys no signal. nx=120 is past the point where it helps and its retention drops.
• Do not go coarser than ~40: at nx=30-35 retention falls to 78-87%, because the ROI holds fewer than two marker cells.

Caveat — the 250 ranking is less discriminating than the 500 one. The ROI is only 6.6 px across, so the response is comparatively flat and some of the ordering is regulariser behaviour rather than pattern quality. Treat the 250 optimum as the band nx 40-65, not as a sharp point.

Two further 250 limitations, measured on the end-to-end run at nx=80, dis_mode=0:

• The ROI holds only 37 pixels (radius 3.31 px), against 139 at 500x500. The response is an ROI mean over 37 samples, so its own sampling error is much larger. This is the main reason the 250 sweep looks flat.
• A 0.01 mm step is only 0.066 px at 250 (0.133 px at 500). The ROI mean still resolves motion, but the per-step verdict machinery is markedly more conservative: the shipped nx=80 at 250/dm0 scores overall unresolvable even though the gain is a healthy 4.21/5.57 px/mm. If the application needs 0.01 mm steps resolved, 500x500 is the safer operating point; 250 is better suited to coarser displacement resolution.
• At sigma=16 the small z response at 250 goes negative (-0.05 px/mm), i.e. noise can flip the sign of an axis whose signal is near the floor.

Trade-off summary. 250x250 forces dis_mode=0 and roughly halves the absolute gain in px/mm (4.30 vs 9.10 on x at the matched cell), but it buys a 2x coarser cell for the same sensitivity: 0.463 mm at nx=40 versus 0.231 mm at nx=80. Against a 50 um production minimum that is 9.2x versus 4.6x of margin.

---

5. Which mode to run

	250x250 dm0	500x500 dm1	500x500 dm0
output / solved at	250 / 250	500 / 250 (upscaled)	500 / 500
window (8*2**dm)	8 px	16 px	8 px
window / moving region	1.21	1.21	0.60
recovers of the true displacement	66%	68%	86%
background error (truth = 0)	1.56 um	1.77 um	0.74 um
edge tail at 1.88 mm	4.6%	4.8%	1.4%
ROI pixels	37	139	139
0.01 mm step in px	0.066	0.133	0.133
DIS cost (medium, 500 px input)	7.4 ms	7.9 ms	19.8 ms
clean gain at nx=85 (x+y)	not comparable	21.7 px/mm	28.3 px/mm
mirrors the deployed SDK config	no	yes	no

Clean-gain units differ between resolutions (px per mm scales with the render width), so compare 500 dm0 with 500 dm1 directly. At matched nx the clean gain is 1.31-1.35x higher under dm0 (nx=80: 26.7 vs 20.8; nx=85: 28.3 vs 21.7; nx=90: 28.4 vs 21.0; nx=105: 28.1 vs 21.2 px/mm combined), i.e. about a third more signal. Compare 250 dm0 against the 500 modes only in recovery terms.

Recommendation: 500x500 dm0, with two conditions that override it.

1. If the camera is physically 250x250, run 250x250 dm0 — resolution is set by hardware, not a free choice, and dm1 there is broken (25% recovery, the field never returns to zero). 250 dm0 is informationally identical to 500 dm1 (same 250x250 solve, same window, matching profiles), so nothing is lost against an upsampled 500 field.
2. If the deployed tracker is fixed at dis_mode=1, run 500x500 dm1 to predict deployment honestly — but know it is the pessimistic configuration: it discards 18 percentage points of recovery by solving at half resolution.

The single most actionable item: set dis_mode=0 if the frame budget allows. At 500x500 it is a ~30-35% sensitivity gain for 2.5x tracker time (19.8 ms vs 7.9 ms per frame; 40% of a 20 fps budget vs 16%), with no hardware change. That is a much larger effect than any pattern choice in the plateau, which spans only ~5%.

The pattern optimum survives the mode change. Under both dm0 and dm1 at 500x500, clean gain is flat within ~5% across nx 75-105, with nx 80-90 in the top group of both, and nx <= 70 and nx >= 115 declining. So nx 80-90 is a safe choice either way; the dm0 sweep's top candidates were nx 65/105/85/90/100, the dm1 sweep's were nx 85/105/90/80.

6. Recommendations

1. Pick the mode first (section 5), because it moves the instrument by ~30%. Then pick the pattern.

   • Camera 500x500 and the tracker can run dm0 → 500x500 dm0, pattern nx 80-90 (cell 0.231-0.206 mm, 4.6x-4.1x the production minimum). Best on every measured axis.
   • Camera 500x500 but the tracker is fixed at dm1 → 500x500 dm1, pattern nx 80-90. Honest for deployment prediction; 18 percentage points of recovery are discarded by solving at half resolution.
   • Camera 250x250 → 250x250 dm0 (mandatory), pattern nx 40-65 (cell 0.463-0.285 mm, 9.2x-5.7x the production minimum).
   • Never 250x250 with dm1: window/moving-region = 2.42, roughly 3x of signal lost and the field is distorted rather than uniformly scaled.

2. Set dis_mode=0 wherever the frame budget allows. At 500x500 that is ~30-35% more signal for 2.5x tracker time (19.8 ms vs 7.9 ms per frame). It is a software change with no hardware impact, and it is a far bigger effect than the ~5% spread across the pattern plateau.

3. Do not treat the second scale as evidence unless its window-to-moving-region ratio is checked. At 125x125 the ROI radius is 1.66 px and the moving region only ~3.3 px, against an 8 px window — ratio 2.4, the same broken regime as 250/dm1.

4. Before committing to a final nx, average 3-5 seeds per nx. The y gain is non-monotonic at both resolutions (9.55 at nx=70 vs 12.88 at nx=85 at 500 dm1), and with a single seed per nx part of that is which pattern rather than the grid size.

5. Re-run optimise under the noise-aware objective before trusting any automated winner. The stored optimizer result (nx=31) came from the retracted render-seed score and has been removed.

---

7. What is not established

• Absolute force calibration. Sensitivity is a relative ranking under a fixed estimator and camera, not newtons.
• The process scale (250x250 at 500 native) and the radius_mm=1.5 block have not been re-ranked under the calibrated model.
• The real floor's pattern trend. The idle clip gives one pattern, so it validates the floor's magnitude and mechanism but not how it varies with nx. A deliberate no-contact recording at the shipped configuration would settle it.
• The 500x500 optimum's dependence on the random draw. One seed per nx.
• Multi-seed null rendering — now diagnostic only, not part of the score.

---

8. Defects found and fixed while producing this

Recorded because each one silently corrupted results at the time:

• build_track_job omitted the noise block, so design.yaml's sigmas/draws/scales and every --noise-* flag were ignored and the pass ran on built-in defaults that happened to match. Caught when a 5-draw run reproduced 1-draw numbers exactly.
• The injected floor used a signed p95 of the projected flow, so it came out negative or near-zero and the objective divided by ~0 — scores of ~1e7. Now p95(|projection|), guarded below by the render-seed reproducibility limit.
• Re-running track --output-dir on a variant silently rebuilt the job from design.yaml, overwriting a dis_mode=0 run with dis_mode=1. Now the existing job is reused unless --rebuild-job is passed.
• Manual re-scoring without the candidate pattern reset every candidate's cell_px to the design.yaml grid. compare does this correctly; ad-hoc re-scoring must pass pattern= explicitly.
• The response figure's label omitted the radius, so nx80@r0.5@dm0 and nx80@r1.5@dm0 produced two identical legend entries, and one global floor band belonged to no plotted curve. Both fixed.