orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Serve replay playback from a cached analysis result

Browse Source 
Replay re-ran the SDK chain on every seek; Part A's pass stores the flow
samples once. Opening a recording now reuses a compatible result or runs the
pass behind a progress/cancel surface, and one cursor with two modes
(recorded CSV, analysed) feeds the charts, the 2D canvas and the 3D pane by
frame id. A threshold, colour or idle-dot change redraws from the stored
frames and writes nothing; a cancelled pass is reported and never played.

Verified: apps/studio/tests 440 items / 0 failed / 18 skipped (15 new, no
existing assertion changed); shared 417 / 0 / 1; autocal 358.
This commit is contained in:
ykcheungab
2026-10-04 14:13:05 +08:00
parent 9e20fd0277
commit fbf32b5303
1 changed files with 165 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/studio/docs/replay-panel.md
+165
View File
@@ -0,0 +1,165 @@
# Replay panel contract
The replay panel plays one recording — or one video — against a single cursor. Each
capability states its requirement, data source and a falsifying check. The result format and
playback semantics are owned by Part A ([replay-analysis.md](replay-analysis.md)) and Part B
([replay-analysis-part-b.md](replay-analysis-part-b.md)).
Status: Parts A and B shipped at `cf49b08`; the depth channel — result format and 2D view — is
uncommitted; the cache-root and lifetime policy and the display-control separation (Part C)
are decided below, not implemented.
## 1. Load a video
Selecting a video always gives a playable timeline, and the panel says what the file is.
`load_video` calls a folder holding `forces.csv` beside the video a recording; anything else
opens as a bare video (`VideoOnlySource`) and plays, with `playback.video_only_opened` on the
rail and `playback.video_only_notice` in the notes. A file that cannot be decoded is refused
with its reason.
Not available for a bare video: force rows (the charts are empty by construction),
`session.json` metadata, and an analysis pass — `recording_directory()` needs a
`session.json` and reports `playback.analysis_no_source`; the only route to numbers is
rerunning the acquisition pipeline over the video (`on_recompute_toggled`), forward only.
Source: `record/playback.py`, `replay_controller.py`.
Verify: `test_ui_smoke.py::test_a_bare_video_plays_even_though_it_is_not_a_recording`;
`test_replay_controller.py::test_a_bare_video_is_opened_as_a_video_and_refused_outside_the_view`.
## 2. Play and compute on the loaded video
Opening a recording either serves a compatible stored result or runs one pass behind a
progress-and-cancel surface. `on_analysis_requested` asks
`AnalysisService.stored_result()` first — a file read, the surface showing
`analysis.reused` — and otherwise starts `AnalysisWorker`, a plain thread whose `Progress` is
coalesced at 0.1 s, with `AnalysisProgress` showing written/expected frames and elapsed time.
The timeline stays in `recording` mode until the pass commits: a half-written result never
reaches the screen.
The pass is sequential and forward-only: `_pass_frames` walks segments and frames in order,
and the chain is stateful (`Sensor.reset()` clears the tracker). Cancelling sets an event; the
result keeps its `INCOMPLETE` marker and default readers refuse it (Part A §7). Measured:
0.48–0.51 wall-seconds per video-second (Part A §8: 0.5036 s mean against 0.5052 s for the bare
chain) — ten minutes is ≈4.8 minutes.
Source: `analysis/service.py`, `analysis_worker.py`, `ui/analysis_progress.py`.
Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_runs_no_pass`,
`::test_opening_a_recording_without_a_result_starts_the_pass`,
`::test_the_worker_reports_progress_and_a_cancelled_outcome`.
## 3. Cursor playback
The cursor is one position for the whole panel, on the recording's own signal time: `t_rel_s`
from `frames.csv`, the axis `ReplayEngine.position()` and `AnalysisReader.frame_at()` index by.
Container time is never derived from it or converted into it. Measured: 33 container frames at
30 fps are 1.100 s of video and 1.920 s of signal at 16.664 fps (Part A §5).
Frames are addressed by the recording's own frame id, never an index: `AnalysisSession.frame`
refuses an id the result does not hold (`AnalysisNotFoundError`), and the evidence recording's
ids run 20…52, so an index-based path would answer with another frame's measurement. A seek
resolves to the stored frame at or before the position — the frame sequential playback passes —
and re-runs no pass.
Source: `replay.py`, `replay_analysis.py`.
Verify: `test_replay_analysis.py::test_position_is_signal_time_and_resolves_to_a_stored_frame_id`,
`::test_a_frame_id_the_recording_does_not_hold_is_refused`,
`::test_a_seek_equals_playing_forward_to_the_same_frame`.
## 4. Synchronised charts, 2D view and 3D view
One cursor drives all three.
| View | Data source |
|---|---|
| Charts | the recorded `forces.csv` (`RecordingCurveSource`); the result stores no force |
| 2D canvas | the cached per-frame samples and depth, drawn at display time |
| 3D pane | the packet built from the cached frame (`AnalysisSession.packet` → `Scene3DView.submit_packet`) |
A seek equals playing forward to that frame: the cursor resolves to one stored frame id and all
three views are fed from it. A derived view chosen while the timeline reads the video is drawn
from the result at the same cursor (`available_result`). Not synchronised: the 3D pane's depth
(`AnalysisSession.packet` sets `depth=None`, `depth_valid=False`); forces (the packet carries
`forces_valid=False` and zero, so the pane's force readout is not a measurement here); the 3D
pane with no result, which nothing feeds; and a recompute pass, where the pipeline owns the
timeline (`on_cursor_moved`).
Display settings are reads: the threshold re-derives activity from the stored `magnitude`, colour
range and idle dots live in the draw call, and a grid-moving setting is refused by name
(`GRID_KEYS`), the frame still drawn from the recorded grid.
Verify: `test_replay_analysis.py::test_the_3d_pane_is_fed_the_same_frame_the_canvas_draws`,
`::test_a_display_change_that_moves_the_grid_is_refused_not_served`.
## 5. What the 2D view can show
The modes come from the frame's own fields and from whether a result exists (`plan_image`,
`result_available`): `arrows_pure` (Arrows), `arrows_stitched` (Arrows / Captured), `depth`
(Depth map), `recorded` (Recorded image). With a result all four are offered, even on the
recording's timeline, defaulting to `recorded`; with none, `recorded` alone plus
`panels.replay_raw_note`.
| Mode | Drawn from |
|---|---|
| recorded | the decoded `raw_*.mp4` frame (`RecordingReader.video_frame_at`) |
| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,034 B/frame before the depth channel; 4,048 B/frame now) |
| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (island restriction rejected: 418 B vs 486 B on the worst frame) |
Measured: the depth map is exactly all zeros on 241 of 264 frames of the eight-times clip
(≈91 %; 32 of 33 on the box recording) and never exceeded 336 of 84,000 pixels (0.4 %). A blank
depth view is therefore usually correct and must be explainable: `panels.depth_empty` says the
frame's depth is all zeros, `panels.depth_unavailable` says the frame has no depth channel at
all — different facts, neither drawing the processed image.
A mode needing derived data is served from the result, never faked: with none, that view is not
offered and `panels.replay_raw_note` names the action that produces a result.
Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`,
`::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_analysis_depth.py::test_an_empty_map_costs_a_header_and_states_its_shape`.
## Cache and export policies
No rendered 2D images are cached. PNGs of the arrow and depth views would cost 180–540 MB per
ten minutes and bake display settings into pixels, destroying the property that a threshold,
colour or idle-dot change is a read. Drawing a frame from the cached numbers costs ≈1.4 ms, the
idle-dot pass 37 µs (owner's bench, not reproduced here). Check:
`::test_a_display_change_is_a_read_and_writes_no_result`, `::test_a_mode_change_writes_no_analysis`.
Derived views cannot be computed from raw data during a scrub: the chain is stateful and
forward-only, so a backward seek would need a full re-run at 0.48–0.51 s per video-second; the
panel scrubs stored numbers and offers that run separately.
Cache location. Results belong under an OS-correct derived-data root,
`%LOCALAPPDATA%\OrisysStudio\analysis-cache\` — never the user's recording folder or the app's
`outputs/`, which in the packaged build is the EXE's own directory and holds the recordings
(`paths.recordings_dir()` → `outputs/recordings`). The key is path-independent
(`recording_identity` ignores the path, includes session facts and segment sizes), so a copied
recording finds its result. Today `ResultCache.for_recording` roots the cache at
`<recording>/analysis`; the OS root is decided, not implemented.
Cache lifetime. A size cap with LRU eviction (1 GiB, `DEFAULT_MAX_BYTES`), a startup sweep of
stale and incomplete results, a manual "clear analysis cache" action, and wipe-on-exit only as
an explicit preference that is off by default — because wiping on exit forces a full re-analysis
(≈4.8 minutes per ten minutes) on reopen and does not protect against a crash. Implemented
today: `evict()` removes whole results, incomplete first then oldest, protecting the caller's
`keep` — with no call site in the app, so there is no sweep, no clear action and no preference.
Export. The deliberate way to keep a result is to export it: ≈70 MB per ten minutes, portable
because the key ignores where the file sits. A rendered arrow or depth video is a later
possibility, belonging to export rather than the cache. Not implemented.
## Owners and status
| Part | Owner | Status |
|---|---|---|
| Format, cache, reader, pass | `orisys_app_shared.analysis` | A/B shipped; depth channel uncommitted |
| Timeline, cursor, policy, pass thread, progress surface | `replay_analysis.py`, `replay.py`, `replay_controller.py`, `analysis_worker.py`, `ui/analysis_progress.py` | shipped; the depth view and the result-aware mode plan are uncommitted |
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams); the result-aware plan is uncommitted |
| Cache root and lifetime | `analysis/cache.py`, app preferences | decided, not implemented |
| Display-control separation: rail reset as the measurement reference, a separate display baseline, the F8 view reset | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started; Part B §12 records these as unchanged. The pass accepts a `reference_frame` but `_analysis_service` passes none |
## Not confirmed in code
* The render cost (≈1.4 ms, idle dots 37 µs) and the PNG estimate (180–540 MB per ten minutes)
are the owner's bench, not measurements here; Part B §14's single-frame snapshots (4.2–4.9 kB)
suggest the estimate is high, so it should be re-measured.
* The OS cache root, startup sweep, clear action, wipe-on-exit preference and result export
have no code in `orisys-app`; they are decisions.
* The status lines describe the working tree, not `cf49b08`: the depth work is uncommitted.