Add a sensor bundle contract separating family, backend and profile
Product name decided the pipeline: Mini worked live but replay refused it. A bundle descriptor now declares profile_id, product_family and processing_backend, with computational and visualization assets kept apart, and one resolver serves live and recorded input. Legacy bundles load through a documented adapter that never guesses a backend from a folder name.
This commit is contained in:
ykcheungab
@@ -0,0 +1,161 @@
|
||||
# Sensor bundles and the processing contract
|
||||
|
||||
A sensor bundle is a folder that describes one processing profile. This document owns the bundle
|
||||
descriptor, the resolver, the backend registry and the fingerprints. `sensor.json` keeps its
|
||||
existing meaning: it is the processing configuration, and its `type` field is the image-processing
|
||||
arrangement (`compound_eye`, `single_camera`, `uv_unroll`) - never a product family.
|
||||
|
||||
## Why the descriptor exists
|
||||
|
||||
"Which pipeline processes these frames" used to be inferred from a product name. Those are four
|
||||
different things, and collapsing them is what made Mini work live but be refused in replay:
|
||||
|
||||
| Concept | Meaning | Example |
|
||||
|---|---|---|
|
||||
| Product family | What the operator calls it | Box, Mini, Finger |
|
||||
| Processing backend | Which SDK pipeline processes the images | `compound_eye`, `uv_unroll` |
|
||||
| Configuration profile | Camera, geometry and algorithm settings for one design revision | `orisys.mini.prototype-r1` |
|
||||
| Device instance | The physical unit | Serial number, its calibration |
|
||||
|
||||
Several families can share one backend, one family can have several profiles, and a recorded clip
|
||||
has no device instance at all - only a profile and, optionally, a compatible calibration.
|
||||
|
||||
## Descriptor
|
||||
|
||||
`manifest.json` in the bundle root. Every field except the five identifying keys is optional, so a
|
||||
bundle with no manifest still resolves through the compatibility adapter.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"profile_id": "orisys.mini.prototype-r1",
|
||||
"display_name": "Mini prototype R1",
|
||||
"product_family": "mini",
|
||||
"processing_backend": "compound_eye",
|
||||
"sensor_json": "sensor.json",
|
||||
"default_calibration": "force_calibration.json",
|
||||
"computational_assets": { "unroll_map": "assets/30fps-0703/unroll_map_uv.npz" },
|
||||
"assets": { "scene": "scene/finger-display-1.0.0.oryscene" }
|
||||
}
|
||||
```
|
||||
|
||||
- **`computational_assets`** are required to process at all. A missing one fails the profile.
|
||||
- **`assets`** are visualization only. A missing one is reported and never blocks processing - a
|
||||
missing 3D scene is not a missing deformation field.
|
||||
- Paths resolve relative to the bundle root, so moving or renaming the folder changes nothing.
|
||||
Absolute paths and `..` escapes are rejected at parse time.
|
||||
- An unsupported `processing_backend` or `schema_version` raises `DescriptorError`. The supported
|
||||
descriptor schema version is 1.
|
||||
|
||||
## Compatibility for existing bundles
|
||||
|
||||
`resolve_bundle` reads a new-shape descriptor when `schema_version` is present. Otherwise - no
|
||||
manifest, or the older `name`/`sensor_json`/`assets_dir`/`viz_pose` shape - the adapter describes
|
||||
it. That is the only place a folder name is consulted, and only for the built-in set:
|
||||
|
||||
| Folder | Product family | Backend (when the config does not decide) |
|
||||
|---|---|---|
|
||||
| `box` | box | `compound_eye` |
|
||||
| `mini` | mini | `compound_eye` |
|
||||
| `finger` | finger | `uv_unroll` |
|
||||
| `finger-mems` | finger-mems | `compound_eye` |
|
||||
|
||||
For a bundle without a descriptor the backend follows the configuration, using the rule
|
||||
`Sensor._load_config` already applies: `type` when present, else `compound_eye` when the config has
|
||||
a `stitcher` block, else `single_camera`. Only `finger` declares `type` today; `box`, `mini` and
|
||||
`finger-mems` are decided by the stitcher. A folder that is neither built-in nor carries a
|
||||
descriptor is rejected - never routed to `compound_eye` because box works that way.
|
||||
|
||||
## Resolver
|
||||
|
||||
```python
|
||||
from orisys import resolve_bundle, validate_profile, create_processor, CalibrationSelection
|
||||
|
||||
profile = resolve_bundle("sensors/mini") # or any bundle folder
|
||||
profile = resolve_bundle("sensors/mini", calibration_selection=CalibrationSelection.NONE)
|
||||
session = validate_profile(profile) # re-check against the filesystem now
|
||||
rebuilt = create_processor(profile, source=0) # live camera
|
||||
rebuilt = create_processor(profile, source="clip.mp4") # recorded frames, same semantics
|
||||
```
|
||||
|
||||
`resolve_bundle(bundle_path, *, calibration_selection=None) -> ResolvedSensorProfile`. The profile
|
||||
is plain data - no Qt, no application preferences, no device handle - so the same object drives a
|
||||
camera and a recorded clip. `_set_calibration` on a device affects only its own calibration
|
||||
selection.
|
||||
|
||||
`validate_profile(profile) -> ValidationReport` re-reads the filesystem. It reports
|
||||
`fingerprint.stale` when a computational asset changed since the profile was resolved.
|
||||
|
||||
`create_processor(profile, *, sensor=None, source=None, include=..., contact=...)` returns a
|
||||
`ProcessingSession` with `start`/`wait`/`latest`/`reset`/`pause`/`resume`/`stop`. It raises
|
||||
`ProcessorCreationError` naming what is missing rather than returning a session that does nothing.
|
||||
|
||||
### Backends
|
||||
|
||||
| Identifier | Requires | Produces |
|
||||
|---|---|---|
|
||||
| `compound_eye` | `camera`, `stitcher`, `tracker` config | stitched frame, flow, force |
|
||||
| `single_camera` | `camera` config | frames only - no deformation field, so no force |
|
||||
| `uv_unroll` | `camera` config, the mapped computational asset | unrolled frame, flow, force |
|
||||
|
||||
`known_backends()` and `spec_for(...)` are the registry. `uv_unroll` is not yet materialised from a
|
||||
profile alone, because it needs geometry, a UV domain, a tactile processor and a capture object
|
||||
assembled together - that is `orisys.finger.FingerRuntime.open`'s shape. `create_processor` says so
|
||||
instead of pretending.
|
||||
|
||||
## Capabilities
|
||||
|
||||
Each capability is decided by the facts it depends on, not by the presence of a file:
|
||||
|
||||
| Capability | Unavailable when |
|
||||
|---|---|
|
||||
| Video processing | the configuration cannot be loaded |
|
||||
| Optical flow | no `tracker` block |
|
||||
| Raw force | the backend produces no deformation field |
|
||||
| Calibrated force | per axis: the calibration has no curve for it |
|
||||
| Deformation | the backend computes no deformation field |
|
||||
| 3D visualization | no scene asset is declared, or the declared one is missing |
|
||||
|
||||
Deformation is relative unless a calibration says otherwise; it is reported with its meaning rather
|
||||
than as a bare "available". `ValidationReport.state` is `ok`, `degraded` (usable, something
|
||||
optional is missing) or `failed`.
|
||||
|
||||
## Calibration
|
||||
|
||||
`CalibrationSelection` is `BUNDLE_DEFAULT` (the default), `NONE`, or an explicit path.
|
||||
|
||||
- `NONE` is a choice, not a failure: raw force stays available and the calibration fingerprint
|
||||
records `none`.
|
||||
- An explicitly selected calibration that is missing or unparseable is an **error**. It is never
|
||||
replaced by another file.
|
||||
- Applicability is `declared` when the document carries `meta.sensor_type`, `unknown` when legacy
|
||||
metadata is absent, or `none`. `unknown` means it applies but its fit is unverified - which is
|
||||
what the shipped `box` calibration reports.
|
||||
|
||||
| Read | Meaning |
|
||||
|---|---|
|
||||
| `FrameSnapshot.normal_force_raw` / `shear_x_raw` / `shear_y_raw` | device counts, pre-calibration |
|
||||
| `FrameSnapshot.normal_force` / `shear_x` / `shear_y` | newtons when a curve applies, counts otherwise |
|
||||
| `FrameSnapshot.depth`, `centroid`, `contour`, `vnormal` | relative quantities, not newtons |
|
||||
|
||||
`Sensor.force_triplets()` and `_force_raw` remain the raw and unfiltered view.
|
||||
|
||||
## Fingerprints
|
||||
|
||||
`processing_fingerprint` hashes the effective backend, the paired data, and each computational asset's role name with
|
||||
its content hash. It excludes the bundle location, `display_name`, and every visualization asset: a
|
||||
relocated or cosmetically renamed bundle keeps its fingerprint, and a missing scene never changes it.
|
||||
It includes the config minus `name` and `version`.
|
||||
|
||||
`calibration_fingerprint` hashes the selected document's content, or the constant `none`.
|
||||
|
||||
`profile.inventory()` returns the reproducible snapshot: bundle-relative paths, asset hashes and
|
||||
both fingerprints. Compare an inventory, not a path, to decide whether two installations would
|
||||
compute the same numbers.
|
||||
|
||||
## Resetting the reference
|
||||
|
||||
Both backends process externally supplied frames through the same profile. After a gap in the
|
||||
input - a pause, a seek, a scene change - call `session.reset()`. It is queued to the stage that
|
||||
owns the tracker, so the reference frame is rebased on the frame about to be computed rather than
|
||||
on a buffer the pipeline is still reading.
|
||||
Reference in New Issue
Block a user