1. added blender deformation probe for marker size picking and optimization
2. added unwarp map export in current cli, don't need to generate by precompute in sensor running
This commit is contained in:
@@ -367,6 +367,199 @@ Collection check:
|
||||
blender -b sensors/finger/r01/source/geometry.blend -P blender/scripts/validate.py -- sensors/finger/r01
|
||||
```
|
||||
|
||||
## Probe sensitivity bench
|
||||
|
||||
`orisys-design deform|render|track|sensitivity|optimise` answers *"which marker
|
||||
pattern gives the best tracking sensitivity on this sensor?"* It deforms the
|
||||
sensing surface through a **known** displacement, renders what `cam_sensor`
|
||||
actually sees, measures DIS optical flow against the zero-displacement reference,
|
||||
and reports a ranked, auditable score. The full interface contract is in
|
||||
[`docs/probe-pipeline.md`](docs/probe-pipeline.md).
|
||||
|
||||
```powershell
|
||||
python -m pip install -e .[probe] # numpy + opencv-python
|
||||
|
||||
# 1) deform 31 frames (1 reference + 3 axes x 10 steps)
|
||||
orisys-design deform sensors/finger-xynova/r01
|
||||
# 2) render what the camera sees for each frame
|
||||
orisys-design render sensors/finger-xynova/r01
|
||||
# 3) DIS flow against the reference
|
||||
orisys-design track sensors/finger-xynova/r01
|
||||
# 4) score and rank (writes sensitivity_report.json)
|
||||
orisys-design sensitivity sensors/finger-xynova/r01 --skip-deform --skip-render --skip-track
|
||||
# 5) search patch-pattern parameters
|
||||
orisys-design optimise sensors/finger-xynova/r01 --dry-run
|
||||
# 6) compare different settings and plot the measured optical flow
|
||||
orisys-design compare sensors/finger-xynova/r01
|
||||
```
|
||||
|
||||
### Visualising the flow and comparing settings
|
||||
|
||||
`orisys-design compare` runs one marker pattern under several estimator settings and
|
||||
writes three figures plus `settings_comparison.json` into
|
||||
`build/probe/<sensor>/<revision>/comparison/`:
|
||||
|
||||
| figure | what it shows |
|
||||
|---|---|
|
||||
| `settings_flow.png` | one panel per setting: the rendered marker around the probe with the measured flow as arrows, the ROI, the cell grid and the **effective window** drawn on it, and the applied vs measured displacement. Below each panel, the measured `\|flow\|` field on a shared colour scale. |
|
||||
| `settings_tuning.png` | score vs the camera-space cell, and vs the cell/window ratio, one series per setting. |
|
||||
| `settings_response.png` | two panels. **Signal**: response vs applied displacement per setting, with each curve's *own* noise floor drawn as a line and the ideal 1:1 line. **Retention**: `gain(σ)/gain(clean)` vs injected noise σ, so a pattern that looks strong but collapses under noise is visible at a glance. |
|
||||
|
||||
Only the settings that need re-running do: `nx` regenerates the texture, `radius_mm`
|
||||
regenerates the deformation and frames, and `dis_mode` reuses the *same frames* (the
|
||||
variant directory holds a manifest with absolute frame paths), so the decisive
|
||||
estimator comparison costs seconds rather than a re-render.
|
||||
|
||||
```powershell
|
||||
# the shipped setting vs the full-resolution estimator, on identical frames
|
||||
orisys-design compare sensors/finger-xynova/r01 --grid 31,56,80 --dis-modes 1,0
|
||||
# a different rubber radius changes the optimum, so compare for stability
|
||||
orisys-design compare sensors/finger-xynova/r01 --radii 0.5,1.0,1.5
|
||||
```
|
||||
|
||||
`orisys-design sensitivity` chains all of steps 1–4 itself, so a single run is
|
||||
usually enough; the `--skip-*` flags let you re-score without re-rendering. All
|
||||
artifacts land in `build/probe/<sensor>/<revision>/`.
|
||||
|
||||
### How the deformation is defined
|
||||
|
||||
The vertices in the `probe` vertex group move rigidly by a displacement expressed
|
||||
in a **probe-average orthonormal frame** — x and y are shear, z is the surface
|
||||
normal, i.e. normal force. The 0.01 mm steps run from 0 to 0.10 mm per axis.
|
||||
Vertices outside the group follow Blender's proportional-editing falloff
|
||||
(`smooth`/`linear`/`sharp`/`constant`) out to `radius_mm`, representing the outer
|
||||
rubber.
|
||||
|
||||
The falloff is computed explicitly in millimetres rather than through
|
||||
`bpy.ops.transform.translate(use_proportional_edit=True)`: that operator is
|
||||
viewport-modal and its radius is screen-space, so it is neither available nor
|
||||
reproducible under `blender -b`. The `.blend` is never written — the whole
|
||||
pipeline is read-only towards it.
|
||||
|
||||
`deform_report.json` reports how many vertices actually moved. That matters: on
|
||||
the Xynova revision the median mesh edge is 0.63 mm, so the default 0.5 mm radius
|
||||
reaches only 5 of 2502 vertices outside the group, producing a sharp 6-vertex
|
||||
island rather than a smooth bulge. Raise `--radius-mm` to ~1.0 mm (17 vertices)
|
||||
or 1.5 mm (30 vertices) for a rounded indentation.
|
||||
|
||||
### Reading the report
|
||||
|
||||
`render_manifest.json` records the render settings actually applied. Two are
|
||||
forced for correctness: the view transform must be `Standard` (the scene ships
|
||||
with AgX, a filmic tonemapper that silently rewrites every marker value) and
|
||||
denoising is off (it smears the pattern the tracker is meant to measure). The
|
||||
render also asserts that re-rendering the reference frame produces identical
|
||||
bytes, because a non-reproducible render invalidates every later number.
|
||||
|
||||
`sensitivity_report.json` carries, per axis and per scale, the measured `gain`
|
||||
(measured mm of flow per mm of applied displacement), the `snr` against the
|
||||
null-frame floor, cross-axis leakage, and `resolvable_step_mm`.
|
||||
|
||||
**Be careful reading small steps.** The Xynova design camera renders 500x500, which
|
||||
is **13.25 px/mm** at the probe, so 0.01 mm is only **0.13 px**. That is a *per-pixel*
|
||||
precision limit, not an estimator limit: because the response is averaged over the
|
||||
probe ROI (~139 px), the measured zero-input floor is only **0.029 px** and a single
|
||||
0.01 mm step (0.13 px) clears it about 4.5x over. The report therefore states
|
||||
`resolvable_step_mm` per axis, reports the floor and the step separately in pixels,
|
||||
and evaluates the process scale (250x250 by default) alongside the render scale so you
|
||||
can see what the runtime can actually resolve. Do not assume a sub-pixel step is
|
||||
unmeasurable — check the floor.
|
||||
|
||||
Sensitivity here is a **relative ranking under a fixed estimator and camera**, not a
|
||||
force calibration, and it must be read under a **stated noise model** — see the section
|
||||
above. The optimizer's winner is the best of a bounded deterministic candidate grid, and
|
||||
is not a proven global optimum.
|
||||
|
||||
### The measured gain is confounded with the ROI size
|
||||
|
||||
This is the most important caveat in the tool, and it is measured rather than
|
||||
theoretical. On Xynova the probe patch plus its falloff taper spans only about
|
||||
**13 px**, i.e. 1-2 DIS patches (8 px), so DIS regularizes toward the static
|
||||
background and the coherent motion is measured at roughly **0.67x** of the applied
|
||||
displacement. Injecting a synthetic rigid translation of the same 1.3248 px recovers
|
||||
**17%** at a patch radius of 6.6 px (equal to the ROI), **51%** at r 10, **81%** at
|
||||
r 20 and **95%** at r 40. So the absolute gain depends strongly on `radius_mm`:
|
||||
compare patterns only while the radius, ROI and scale are held fixed.
|
||||
`orisys-design optimise` varies `nx`/`ny`/`seed` and never the radius for exactly this
|
||||
reason.
|
||||
|
||||
Attribution matters too: the deficit is **not** aliasing, noise or a diluted average.
|
||||
The real flow's radial profile is flat inside the ROI, and the texture-level
|
||||
known-answer recovers a 2 px shift as 2.0000 px. Only strongly *periodic* synthetic
|
||||
textures alias (a 3.11 px checker turns a 2 px shift into 8.17 px); the shipped random
|
||||
per-cell pattern does not.
|
||||
|
||||
### The noise floor was wrong, and it changed the answer
|
||||
|
||||
**Read [`docs/probe-findings.md`](docs/probe-findings.md) for the conclusions and
|
||||
recommendations.** The short version is below.
|
||||
|
||||
**This section replaces an earlier claim that nx=31 is the best pattern and that "the
|
||||
marker cell must match the estimator's effective window". That claim is retracted.** It
|
||||
was driven by the noise floor, and the floor was not a valid noise model.
|
||||
|
||||
`floor_px` is measured as `reference` vs a re-render at **seed + 1** — Cycles
|
||||
antialiasing jitter. On this data it differs by ~5 grey levels RMS on half the pixels and
|
||||
scales with edge contrast. It is not sensor noise, and three checks show it cannot rank
|
||||
patterns:
|
||||
|
||||
1. **Its trend reverses** under additive noise. p95 |flow| over the ROI, same DIS config:
|
||||
|
||||
| pattern | render-seed | injected σ=2 | σ=4 |
|
||||
|---|---|---|---|
|
||||
| nx=80 | 0.0292 | **0.0161** | 0.0278 |
|
||||
| nx=31 | 0.0376 | **0.0084** | 0.0182 |
|
||||
| nx=56 | 0.0134 | 0.0102 | 0.0203 |
|
||||
|
||||
2. **It understates the real floor.** Measured on idle dd02 footage
|
||||
(`data/dd02-0703/190.mp4`, 1280x720) with the same estimator:
|
||||
**0.2267 px** p95 |flow| over the ROI, dominant marker period **25.5 px**, i.e.
|
||||
**0.0089 floor per period**. Short baseline (consecutive frames) equals long baseline
|
||||
(vs frame 0) — ratio 0.98 — so this is frame-independent white noise, not drift, and
|
||||
the clip is genuinely idle. Every simulated floor per cell is well below it, and the
|
||||
**coarse patterns are understated most** (nx=56 by ~6x, nx=31 by ~4x, nx=80 by ~2x) —
|
||||
exactly the bias that made nx=31 and nx=56 look best.
|
||||
|
||||
3. **Signal is not noise-invariant either**, so gain must be quoted under a stated model.
|
||||
At +0.10 mm on the x axis, retention is `gain(σ)/gain(clean)`:
|
||||
|
||||
| pattern | σ=4 | σ=16 |
|
||||
|---|---|---|
|
||||
| nx=80 dm1 | **99%** | **84%** |
|
||||
| nx=56 dm1 | 96% | 71% |
|
||||
| nx=31 dm1 | 72% | 59% |
|
||||
|
||||
**What the calibrated model supports:** at the render scale, `radius_mm=0.5`, `dis_mode=1`,
|
||||
nx=80 has the **highest clean response on both axes** (x 9.10 vs 4.74 px/mm; y 11.69 vs
|
||||
10.90) and the **best noise retention** (84–99% vs 59–73%), so on this evidence the
|
||||
shipped 80x80 is competitive or better, not worse. What is *not* yet re-ranked: the
|
||||
process (250x250) scale and the `radius_mm=1.5` block, where the score ordering differs.
|
||||
|
||||
Every report now carries `noise_models` (per-axis gain, floor and `response_retention` per
|
||||
σ, measured under one model) and `noise_calibration` (the real idle numbers above), and
|
||||
`settings_response.png` has a retention panel so a pattern that looks strong but collapses
|
||||
under noise is visible.
|
||||
|
||||
### What the bench measures
|
||||
|
||||
The marker grid is defined **across the camera image**, so a cell is
|
||||
`render_width / nx` pixels, not a millimetre quantity: at 500x500 the shipped 80x80 grid
|
||||
has a **6.25 px** cell and nx=31 has **16.13 px**. `dis_mode=1` is OpenCV's
|
||||
`setFinestScale(1)`, so DIS computes flow at half resolution and upscales, giving an
|
||||
effective window of `patch_px * 2**dis_mode` = **16 px**. Those are the right quantities to
|
||||
watch — but the earlier reading of them as *the* explanation of the ranking was an artefact
|
||||
of the floor above, so treat the ratio as a useful diagnostic, not a validated law.
|
||||
|
||||
The measured gain is also **confounded with the ROI size**: the probe patch plus its
|
||||
falloff taper spans only ~13 px (1-2 DIS patches), so DIS regularizes toward the static
|
||||
background. Injecting a synthetic rigid translation of the same 1.3248 px recovers ~17% at
|
||||
a patch radius of 6.6 px (equal to the ROI), 51% at r 10, 81% at r 20 and 95% at r 40.
|
||||
Compare patterns only with the radius, ROI and scale held fixed — which is why
|
||||
`orisys-design optimise` varies `nx`/`ny`/`seed` and never the radius.
|
||||
|
||||
|
||||
|
||||
|
||||
## Profiles
|
||||
|
||||
- **`compute`** — `sensing_surface` + `cam_sensor` relative to `origin_sensor`. Writes
|
||||
|
||||
Reference in New Issue
Block a user