接管 Studio 流水线迁移记录

将历史比较放回产品仓库并标明现行实现所有者。文档与仓库门禁通过，原记录正文保留。
2026-10-19 10:15:45 +08:00
parent 570daae68f
commit 8a027bfcb9
1 changed files with 184 additions and 0 deletions
@@ -0,0 +1,184 @@
 # Studio acquisition on the SDK's concurrent pipeline
 Historical migration record, 2026-10-01; moved to its product owner on 2026-10-19.
 Numbers, failures and proposed SDK changes below describe that run, not current acceptance.
 The original scope changed App workers and read the SDK; no SDK change was part of it.
 Current implementation owners are the [SDK worker](../../shared/src/orisys_app_shared/acquisition/sdk_worker.py),
 [finger worker](../../shared/src/orisys_app_shared/acquisition/finger_worker.py) and
 [packet mapping](../../shared/src/orisys_app_shared/acquisition/models.py).
 The open contact-detection and live-camera questions were not revalidated by this move.
 ## What changed
 The box/mini worker (`shared/src/orisys_app_shared/acquisition/sdk_worker.py`) replaced
 its `get_img()` → `process_chain()` → `read_forces_from_sensor()` loop with
 `StreamPipeline(sensor, producer, contact=…, include=…)` and now consumes the finished
 `FrameSnapshot`. The finger worker (`.../acquisition/finger_worker.py`) replaced
 `read_source()` → `process_frame()` → `process_chain(tactile)` with a pipeline over
 `runtime.tactile.raw_sensor`, built the way `runtime.pipeline()` builds it;
 `runtime.warmup(color=False)` still runs first.
 The worker's public contract is unchanged: `pause()/resume()/paused`, `latest()`,
 `reset()`, `stop()`, `consume_end_reason()`, `phase`, `capture_status()`,
 `filter_state()`, the record-channel sinks, and every `SdkPacket` field the UI reads.
 The record channel still sees every published frame: the pipeline's `on_frame` hands
 each snapshot to the worker, which builds one packet and emits it as the serial loop
 did. `SdkPacket` construction now lives in one place,
 `models.from_snapshot(snapshot, sensor, …)`, rather than being spelled out field by
 field in each worker.
 | Snapshot / app fact | Packet field |
 |---|---|
 | `seq` | `frame_id` |
 | `timestamp`, collect time (`timestamp − age_ms`) | `timestamp`, `wall_t`, `mono_t` |
 | `fps` | `fps` |
 | `normal_force`, `shear_x`, `shear_y`, `*_raw` | same |
 | `unit` | `force_unit` |
 | `image` → BGR | `processed_frame` |
 | `flow`, `depth` | same |
 | `centroid` + sensor `centroid_found` | `contact` (`ContactObservation`) |
 | stage times | `process_ms` |
 | app | `session_id`, `valid`, `invalid_reason`, `stale_forces`, `force_calibrated`, `calibrated_axes`, `calibration_version`, `raw_available`, `capture_fps`, `capture_dropped` |
 `mono_t` is the collect time for a live source and `frame_id × frame period` for a file,
 so a replay keeps the video's own timeline. `include=` asks only for what the app can
 display; flow and depth are both carried because the 3D pane lets the operator switch
 between them at runtime.
 ## The filter decision: the SDK filters, the app does not
 Studio already pushed its filter choice into the device (`FilterSettings.apply_to` →
 `Sensor.configure`) and read the filtered values back; no app-side force filter exists.
 The migration consumes the SDK's fields as they are: `snapshot.normal_force`/`shear_*`
 (filtered and calibrated) and `snapshot.normal_force_raw`/`shear_*_raw` (the unfiltered
 HHD integrator output). `ForcePresentation` (offsets, deadband, tare) is untouched and
 still display-only.
 The packet diff decided this, not inspection. With the SDK filter chain off, the two
 paths agree **exactly** on all six force fields — only possible if neither side filters
 separately. With `median(3) + lowpass(10 Hz)` pushed into the device, the *raw* six
 still agree exactly while the filtered three do not — and running the **same path
 twice** on the same clip reproduces that divergence (max |Δnormal_force| ≈ 0.010 N,
 0 at the raw fields). The SDK's `_update_force_sample_rate` re-estimates the low-pass
 sample rate from wall-clock cadence, so a *filtered* value on a file source is a
 function of throughput as well as of the frames. A filtered force on a replay is
 reproducible on neither path, and the honest comparison is the raw one.
 ## Packet diff
 Same clip through both paths, one `SdkPacket` per frame, every field compared.
 | Run | Frames | Fields | Mismatches |
 |---|---|---|---|
 | box | 26 | 27 | none; `contact` only differs because the serial loop stated none |
 | finger | 32 | 29 | one frame's contact detection |
 | box, 264-frame clip | 252 | 27 | none; the same `contact` difference |
 | finger, 264-frame clip | 263 | 29 | 23 frames' contact detection |
 Tolerances: forces and `*_raw` at 1e-6, arrays at 1e-6 max absolute difference,
 ids/units/flags/provenance exactly equal. `fps_present`, `valid_mask_matches_field` and
 `contact_frame_id` are consistency checks. The only real difference is the `contact`
 observation the serial loop never stated: on 23 of 263 finger frames the new path finds
 a contact (a centre around (150, 250) in UV) where the serial path finds none. The
 serial path computes optical flow **twice per frame** (`compute_deformation` calls
 `compute_flow` after `get_img` already did), so its expansion mask and the pipeline's
 are not the same measurement. Which is right is unresolved.
 ## Performance, measured
 264-frame 1920×1080 box clip and the same frames through the finger runtime, two
 repetitions each, this machine.
 | Path | Wall (median) | Settled fps | `process_ms` |
 |---|---|---|---|
 | box, serial (252 frames) | 9.42 s | 51.9 | 16.5 |
 | box, pipeline | 7.40 s | 93.9 | 24.7 |
 | finger, serial (263 frames) | 6.31 s | 47.9 | 14.1 |
 | finger, pipeline | 2.56 s | 130.5 | 18.6 |
 Read the columns separately. The rate is what the pipeline buys — **1.8×** for the box,
 **2.7×** for the finger — and it is the number the rail shows. The wall-clock gain is
 smaller (1.27× box, 2.47× finger) because a file pass also pays a fixed cost for
 opening the device and filling the SDK's caches; on the 33-frame clip that cost
 dominated completely (2.25 s vs 2.27 s) while the rate still went 66.6 → 112.3 fps.
 `process_ms` rises because it is now the **stage sum** the SDK measures. A live camera
 cannot be exercised here; its gain should be the pipeline's own.
 ## Behavioural checks
 - **Pause keeps the device open and publishes nothing.** `pause()` holds the pipeline at
  `PauseLevel.CAPTURE_ONLY`: the read continues (exposure, white balance and frame rate
  stay converged) while flow and HHD are skipped and no snapshot is published. A frame
  already in flight is dropped rather than handed over. `resume(reset=True)` rebases
  the tracker, which keeps a `CAPTURE_ONLY` hold out of the force reading.
 - **A recording session is structurally unchanged.** Packets are still built in order,
  one per published frame, from the acquisition thread; `frame_id`, `mono_t` and the
  per-frame provenance flags mean what they did.
 - **No orphaned threads.** `stop()` joins all three pipeline threads. A new test
  (`shared/tests/test_stream_pipeline_acquisition.py`) asserts that
  `orisys-pipeline-produce/-flow/-hhd` are gone after `stop()`, that a stop/start cycle
  runs a second session, and that the packet mapping carries every UI field.
 - **End of input.** A file's end is reported as `ended` exactly as before, and a source
  that yields nothing reports an error instead of waiting out the ten-second deadline.
 ## Verification actually run
 | Gate | Result | Before |
 |---|---|---|
 | `shared/tests` | 389: 386 passed, 1 skipped, 2 failed | 383: 382 passed, 1 skipped, 2 failed |
 | `apps/studio/tests` | 375: 358 passed, 17 skipped, 0 failed | same |
 | `apps/autocal/tests` | 358 passed, 0 failed | same |
 | `apps/studio/tools/ui_check.py` | 600 checks, 0 failures | same |
 | real-GL smoke | 14: 13 passed, 1 failed | same |
 | `tools/lint_gate.py`, changed files | 3 findings, all at HEAD (S110 ×3) | same |
 The two shared failures are the documented wall-clock pair in
 `test_worker_frame_rate.py`: the fake camera delivers ~21.5 fps on this machine against
 the 30 fps its thresholds assume, and the serial path measures 21.5 fps in the same
 conditions, so both are limited by the fake device's own thread. The GL failure is the
 pre-existing `test_an_automatic_colour_scale_follows_the_frame`.
 ## What was kept, and what was decided alone
 The serial chain is **kept** behind `use_stream_pipeline=False` on both workers; it is
 the reference the packet diff runs against, not the path a session uses. Decided here,
 not in the brief:
 1. **The finger pipeline is constructed directly**, not through `runtime.pipeline()`,
   which hardcodes `grab_and_process` as the producer and so cannot see the source
   frame the record channel's pre-roll history wants, nor skip a repeated frame.
 2. **The producer holds the read until the capture source has published another frame**,
   for a live camera. The serial loop did this through the source's sequence counter,
   and without it a 30 fps device is recorded at the polling rate with every
   measurement duplicated.
 3. **A file pass is paced by the declared frame period.** Without it a replay runs as
   fast as the CPU allows and stamps 122 s of video time in 0.4 s of wall clock.
 4. **The SDK's contact `centroid_found` decides whether there is a contact**, not the
   centre itself: the SDK leaves `centroid` at `(0, 0)` when it found nothing, and
   reporting that would put a marker in the frame's corner that nothing measured.
 5. **`capture_dropped` prefers the capture source's own counter**, falling back to the
   pipeline's produced-minus-published backlog, which is all a file source can answer.
 6. **The packet's `fps` is measured at the collection times**, the same basis the serial
   loop used, so a replay's reported rate means the same thing on both paths.
 Left open: the finger contact-detection difference, and a live-camera measurement of the
 throughput gain.
 ## Proposed SDK changes
 The migration needed no SDK change, but three would remove workarounds the app carries.
 1. **Let a caller supply the producer to `runtime.pipeline()`** (or accept a
   `raw_frame=True` / frame-sink option), so a finger caller gets the pre-unroll source
   frame and repeated-frame suppression without building the `StreamPipeline` itself.
   `for_finger_runtime` hardcodes `grab_and_process` today.
 2. **Pin the force low-pass sample rate for file sources** instead of re-estimating it
   from wall-clock cadence (`_update_force_sample_rate`, `sensor.py`:814-844).
   `_lowpass_rate_auto` is already off for external-frame sources; the same for an
   opened video would make a filtered force reproducible, which is now false on both
   acquisition paths.
 3. **Expose the frames a pipeline has produced and published**, and a documented way to
   wait for the next delivered frame. The app reaches into `StreamPipeline._done` and
   re-implements the repeat check around `CaptureSource`, because `wait()` is
   latest-wins and the only every-frame hook is `on_frame`, which the app must then
   drain itself for ordering and backpressure.