orisys-sdk/docs/sensor-bundles.md

Sensor bundles and the processing contract

A sensor bundle is a folder that describes one processing profile. This document owns the bundle descriptor, the resolver, the backend registry and the fingerprints. sensor.json keeps its existing meaning: it is the processing configuration, and its type field is the image-processing arrangement (compound_eye, single_camera, uv_unroll) - never a product family.

Why the descriptor exists

"Which pipeline processes these frames" used to be inferred from a product name. Those are four different things, and collapsing them is what made Mini work live but be refused in replay:

Concept	Meaning	Example
Product family	What the operator calls it	Box, Mini, Finger
Processing backend	Which SDK pipeline processes the images	compound_eye, uv_unroll
Configuration profile	Camera, geometry and algorithm settings for one design revision	orisys.mini.prototype-r1
Device instance	The physical unit	Serial number, its calibration

Several families can share one backend, one family can have several profiles, and a recorded clip has no device instance at all - only a profile and, optionally, a compatible calibration.

Descriptor

manifest.json in the bundle root. Every field except the five identifying keys is optional, so a bundle with no manifest still resolves through the compatibility adapter.

{
  "schema_version": 1,
  "profile_id": "orisys.mini.prototype-r1",
  "display_name": "Mini prototype R1",
  "product_family": "mini",
  "processing_backend": "compound_eye",
  "sensor_json": "sensor.json",
  "default_calibration": "force_calibration.json",
  "computational_assets": { "unroll_map": "assets/30fps-0703/unroll_map_uv.npz" },
  "assets": { "scene": "scene/finger-display-1.0.0.oryscene" }
}

• computational_assets are required to process at all. A missing one fails the profile.
• assets are visualization only. A missing one is reported and never blocks processing - a missing 3D scene is not a missing deformation field.
• Paths resolve relative to the bundle root, so moving or renaming the folder changes nothing. Absolute paths and .. escapes are rejected at parse time.
• An unsupported processing_backend or schema_version raises DescriptorError. The supported descriptor schema version is 1.

Compatibility for existing bundles

The shipped Finger compute assets under sensors/finger/assets/30fps-0703/ mirror the Studio calibration baseline. Synchronize the complete mesh, camera intrinsics, camera pose and UV cache together; the current profile uses source_scale=0.42. The SDK retains its own copy and has no runtime dependency on a Studio checkout. The obsolete 0.4 cache from the previous geometry is not shipped. tests/test_finger_asset_identity.py pins the verified calibration geometry and domain identity. Changing SHA fields alone does not migrate a calibration; regenerate and validate its coordinate artifact against the selected bundle before using its gain map.

resolve_bundle reads a new-shape descriptor when schema_version is present. Otherwise - no manifest, or the older name/sensor_json/assets_dir/viz_pose shape - the adapter describes it. That is the only place a folder name is consulted, and only for the built-in set:

Folder	Product family	Backend (when the config does not decide)
box	box	compound_eye
mini	mini	compound_eye
finger	finger	uv_unroll
finger-mems	finger-mems	compound_eye

For a bundle without a descriptor the backend follows the configuration, using the rule Sensor._load_config already applies: type when present, else compound_eye when the config has a stitcher block, else single_camera. Only finger declares type today; box, mini and finger-mems are decided by the stitcher. A folder that is neither built-in nor carries a descriptor is rejected - never routed to compound_eye because box works that way.

Resolver

from orisys import resolve_bundle, validate_profile, create_processor, CalibrationSelection

profile = resolve_bundle("sensors/mini")                      # or any bundle folder
profile = resolve_bundle("sensors/mini", calibration_selection=CalibrationSelection.NONE)
session = validate_profile(profile)                           # re-check against the filesystem now
rebuilt = create_processor(profile, source=0)                 # live camera
rebuilt = create_processor(profile, source="clip.mp4")        # recorded frames, same semantics

resolve_bundle(bundle_path, *, calibration_selection=None) -> ResolvedSensorProfile. The profile is plain data - no Qt, no application preferences, no device handle - so the same object drives a camera and a recorded clip. _set_calibration on a device affects only its own calibration selection.

validate_profile(profile) -> ValidationReport re-reads the filesystem. It reports fingerprint.stale when a computational asset changed since the profile was resolved.

create_processor(profile, *, sensor=None, source=None, include=..., contact=...) returns a ProcessingSession with start/wait/latest/reset/pause/resume/stop. It raises ProcessorCreationError naming what is missing rather than returning a session that does nothing.

Backends

Identifier	Requires	Produces
compound_eye	camera, stitcher, tracker config	stitched frame, flow, force
single_camera	camera config	frames only - no deformation field, so no force
uv_unroll	camera config, the mapped computational asset	unrolled frame, flow, force

Tile detection margin

compound_eye detects n_boxes_target tiles (15 by default) when it opens. A module mounted off centre can show an extra column at one edge, and detection then never settles on 15. stitcher.exclude_x is [left, right] in raw-frame pixels: a detected region whose centre lies within that distance of the edge is not counted. It changes only which tiles are chosen; the frames used afterwards are untouched. The box bundle ships [0, 240], which drops an extra right column (about 170 px from the edge) and keeps the outer column of an aligned module (about 320 px). A module whose extra column is on the left needs [240, 0] in its own copy of sensor.json.

known_backends() and spec_for(...) are the registry. uv_unroll is not yet materialised from a profile alone, because it needs geometry, a UV domain, a tactile processor and a capture object assembled together - that is orisys.finger.FingerRuntime.open's shape. create_processor says so instead of pretending.

Capabilities

Each capability is decided by the facts it depends on, not by the presence of a file:

Capability	Unavailable when
Video processing	the configuration cannot be loaded
Optical flow	no tracker block
Raw force	the backend produces no deformation field
Calibrated force	per axis: the calibration has no curve for it
Deformation	the backend computes no deformation field
3D visualization	no scene asset is declared, or the declared one is missing

Deformation is relative unless a calibration says otherwise; it is reported with its meaning rather than as a bare "available". ValidationReport.state is ok, degraded (usable, something optional is missing) or failed.

Calibration

CalibrationSelection is BUNDLE_DEFAULT (the default), NONE, or an explicit path.

• NONE is a choice, not a failure: raw force stays available and the calibration fingerprint records none.
• An explicitly selected calibration that is missing or unparseable is an error. It is never replaced by another file.
• Applicability is declared when the document carries meta.sensor_type, unknown when legacy metadata is absent, or none. unknown means it applies but its fit is unverified - which is what the shipped box calibration reports.

Read	Meaning
FrameSnapshot.normal_force_raw / shear_x_raw / shear_y_raw	device counts, pre-calibration
FrameSnapshot.normal_force / shear_x / shear_y	newtons when a curve applies, counts otherwise
FrameSnapshot.depth, centroid, contour, vnormal	relative quantities, not newtons

Sensor.force_triplets() and _force_raw remain the raw and unfiltered view.

Fingerprints

processing_fingerprint hashes the effective backend, the paired data, and each computational asset's role name with its content hash. It excludes the bundle location, display_name, and every visualization asset: a relocated or cosmetically renamed bundle keeps its fingerprint, and a missing scene never changes it. It includes the config minus name and version.

calibration_fingerprint hashes the selected document's content, or the constant none.

profile.inventory() returns the reproducible snapshot: bundle-relative paths, asset hashes and both fingerprints. Compare an inventory, not a path, to decide whether two installations would compute the same numbers.

Resetting the reference

Both backends process externally supplied frames through the same profile. After a gap in the input - a pause, a seek, a scene change - call session.reset(). It is queued to the stages that own the sensor state, so the reference frame is rebased on the frame about to be computed rather than on a buffer the pipeline is still reading: the flow stage resets the image/tracker state and bumps the reference version, and the HHD stage resets the force/readout state (filters, contact flags, calibration session) on the first frame carrying the new version, before decomposing it. FrameSnapshot.tracker_reset_seq is the version that frame was measured with.

When driving StreamPipeline directly, call target = pipeline.request_reset() to obtain the future reference version reserved by that request. Confirm it only when a snapshot carries tracker_reset_seq >= target; the current output frame can trail an earlier reset and cannot identify a new request. Requests share a target until flow claims it; a later request gets a new target even while the earlier reset is still being applied. The existing reset() returns None. Regression coverage lives in tests/test_pipeline.py (request coalescing and reset-in-progress).