orisys-sdk/docs/sensor-bundles.md

# Sensor bundles and the processing contract

A sensor bundle is a folder that describes one processing profile. This document owns the bundle
descriptor, the resolver, the backend registry and the fingerprints. `sensor.json` keeps its
existing meaning: it is the processing configuration, and its `type` field is the image-processing
arrangement (`compound_eye`, `single_camera`, `uv_unroll`) - never a product family.

## Why the descriptor exists

"Which pipeline processes these frames" used to be inferred from a product name. Those are four
different things, and collapsing them is what made Mini work live but be refused in replay:

| Concept | Meaning | Example |
|---|---|---|
| Product family | What the operator calls it | Box, Mini, Finger |
| Processing backend | Which SDK pipeline processes the images | `compound_eye`, `uv_unroll` |
| Configuration profile | Camera, geometry and algorithm settings for one design revision | `orisys.mini.prototype-r1` |
| Device instance | The physical unit | Serial number, its calibration |

Several families can share one backend, one family can have several profiles, and a recorded clip
has no device instance at all - only a profile and, optionally, a compatible calibration.

## Descriptor

`manifest.json` in the bundle root. Every field except the five identifying keys is optional, so a
bundle with no manifest still resolves through the compatibility adapter.

```json
{
  "schema_version": 1,
  "profile_id": "orisys.mini.prototype-r1",
  "display_name": "Mini prototype R1",
  "product_family": "mini",
  "processing_backend": "compound_eye",
  "sensor_json": "sensor.json",
  "default_calibration": "force_calibration.json",
  "computational_assets": { "unroll_map": "assets/30fps-0703/unroll_map_uv.npz" },
  "assets": { "scene": "scene/finger-display-1.0.0.oryscene" }
}
```

- **`computational_assets`** are required to process at all. A missing one fails the profile.
- **`assets`** are visualization only. A missing one is reported and never blocks processing - a
  missing 3D scene is not a missing deformation field.
- Paths resolve relative to the bundle root, so moving or renaming the folder changes nothing.
  Absolute paths and `..` escapes are rejected at parse time.
- An unsupported `processing_backend` or `schema_version` raises `DescriptorError`. The supported
  descriptor schema version is 1.

## Compatibility for existing bundles

`resolve_bundle` reads a new-shape descriptor when `schema_version` is present. Otherwise - no
manifest, or the older `name`/`sensor_json`/`assets_dir`/`viz_pose` shape - the adapter describes
it. That is the only place a folder name is consulted, and only for the built-in set:

| Folder | Product family | Backend (when the config does not decide) |
|---|---|---|
| `box` | box | `compound_eye` |
| `mini` | mini | `compound_eye` |
| `finger` | finger | `uv_unroll` |
| `finger-mems` | finger-mems | `compound_eye` |

For a bundle without a descriptor the backend follows the configuration, using the rule
`Sensor._load_config` already applies: `type` when present, else `compound_eye` when the config has
a `stitcher` block, else `single_camera`. Only `finger` declares `type` today; `box`, `mini` and
`finger-mems` are decided by the stitcher. A folder that is neither built-in nor carries a
descriptor is rejected - never routed to `compound_eye` because box works that way.

## Resolver

```python
from orisys import resolve_bundle, validate_profile, create_processor, CalibrationSelection

profile = resolve_bundle("sensors/mini")                      # or any bundle folder
profile = resolve_bundle("sensors/mini", calibration_selection=CalibrationSelection.NONE)
session = validate_profile(profile)                           # re-check against the filesystem now
rebuilt = create_processor(profile, source=0)                 # live camera
rebuilt = create_processor(profile, source="clip.mp4")        # recorded frames, same semantics
```

`resolve_bundle(bundle_path, *, calibration_selection=None) -> ResolvedSensorProfile`. The profile
is plain data - no Qt, no application preferences, no device handle - so the same object drives a
camera and a recorded clip. `_set_calibration` on a device affects only its own calibration
selection.

`validate_profile(profile) -> ValidationReport` re-reads the filesystem. It reports
`fingerprint.stale` when a computational asset changed since the profile was resolved.

`create_processor(profile, *, sensor=None, source=None, include=..., contact=...)` returns a
`ProcessingSession` with `start`/`wait`/`latest`/`reset`/`pause`/`resume`/`stop`. It raises
`ProcessorCreationError` naming what is missing rather than returning a session that does nothing.

### Backends

| Identifier | Requires | Produces |
|---|---|---|
| `compound_eye` | `camera`, `stitcher`, `tracker` config | stitched frame, flow, force |
| `single_camera` | `camera` config | frames only - no deformation field, so no force |
| `uv_unroll` | `camera` config, the mapped computational asset | unrolled frame, flow, force |

`known_backends()` and `spec_for(...)` are the registry. `uv_unroll` is not yet materialised from a
profile alone, because it needs geometry, a UV domain, a tactile processor and a capture object
assembled together - that is `orisys.finger.FingerRuntime.open`'s shape. `create_processor` says so
instead of pretending.

## Capabilities

Each capability is decided by the facts it depends on, not by the presence of a file:

| Capability | Unavailable when |
|---|---|
| Video processing | the configuration cannot be loaded |
| Optical flow | no `tracker` block |
| Raw force | the backend produces no deformation field |
| Calibrated force | per axis: the calibration has no curve for it |
| Deformation | the backend computes no deformation field |
| 3D visualization | no scene asset is declared, or the declared one is missing |

Deformation is relative unless a calibration says otherwise; it is reported with its meaning rather
than as a bare "available". `ValidationReport.state` is `ok`, `degraded` (usable, something
optional is missing) or `failed`.

## Calibration

`CalibrationSelection` is `BUNDLE_DEFAULT` (the default), `NONE`, or an explicit path.

- `NONE` is a choice, not a failure: raw force stays available and the calibration fingerprint
  records `none`.
- An explicitly selected calibration that is missing or unparseable is an **error**. It is never
  replaced by another file.
- Applicability is `declared` when the document carries `meta.sensor_type`, `unknown` when legacy
  metadata is absent, or `none`. `unknown` means it applies but its fit is unverified - which is
  what the shipped `box` calibration reports.

| Read | Meaning |
|---|---|
| `FrameSnapshot.normal_force_raw` / `shear_x_raw` / `shear_y_raw` | device counts, pre-calibration |
| `FrameSnapshot.normal_force` / `shear_x` / `shear_y` | newtons when a curve applies, counts otherwise |
| `FrameSnapshot.depth`, `centroid`, `contour`, `vnormal` | relative quantities, not newtons |

`Sensor.force_triplets()` and `_force_raw` remain the raw and unfiltered view.

## Fingerprints

`processing_fingerprint` hashes the effective backend, the paired data, and each computational asset's role name with
its content hash. It excludes the bundle location, `display_name`, and every visualization asset: a
relocated or cosmetically renamed bundle keeps its fingerprint, and a missing scene never changes it.
It includes the config minus `name` and `version`.

`calibration_fingerprint` hashes the selected document's content, or the constant `none`.

`profile.inventory()` returns the reproducible snapshot: bundle-relative paths, asset hashes and
both fingerprints. Compare an inventory, not a path, to decide whether two installations would
compute the same numbers.

## Resetting the reference

Both backends process externally supplied frames through the same profile. After a gap in the
input - a pause, a seek, a scene change - call `session.reset()`. It is queued to the stage that
owns the tracker, so the reference frame is rebased on the frame about to be computed rather than
on a buffer the pipeline is still reading.