ykcheungab
184 lines
12 KiB
Markdown
184 lines
12 KiB
Markdown
# Replay panel contract
|
||
|
||
The replay panel plays one recording or video against one cursor. Part A
|
||
([replay-analysis.md](replay-analysis.md)) and Part B
|
||
([replay-analysis-part-b.md](replay-analysis-part-b.md)) own format and playback semantics.
|
||
|
||
Status: Parts A and B shipped at `cf49b08`, the depth channel at `9e20fd0`, the result-aware mode
|
||
plan at `1aa4543`. The cache root and the lifetime sweep are implemented; the display-control
|
||
separation (Part C) is not.
|
||
|
||
## 1. Load a video
|
||
|
||
`load_video` treats video beside `forces.csv` as a recording; otherwise it opens a bare video
|
||
(`VideoOnlySource`), with `playback.video_only_opened` on the rail and
|
||
`playback.video_only_notice` in the notes. Undecodable files are refused with their reason.
|
||
Bare video lacks device-measured force rows and `session.json` metadata.
|
||
|
||
**A bare video can be analysed, and is asked how.** The two files the pass reads are *generated*
|
||
for the video in an app-owned workspace and every later step is the existing recording path; the
|
||
workspace, its frame index and the synthetic marker are Part A's (§1.1). One prompt asks what a
|
||
video cannot carry: the **profile** (from the catalog, each row naming what it can do — flow /
|
||
raw force / calibrated axes / depth / 3D), an **optional calibration** defaulting to the profile's
|
||
declared `default_calibration`, and a **reference frame** defaulting to the first. A profile the
|
||
SDK rejects is listed, disabled and refused with the catalog's own `start_blocked_reason()`.
|
||
Declining keeps the old behaviour (`playback.analysis_setup_declined`), and **more than one file
|
||
is refused** with its reason.
|
||
|
||
Source: `analysis/bare_video.py`, `ui/video_analysis_dialog.py`, `replay_controller.py`.
|
||
Verify: `test_bare_video_analysis.py`, `test_bare_video.py`,
|
||
`test_replay_controller.py::test_a_bare_video_is_opened_as_a_video_and_refused_outside_the_view`.
|
||
|
||
## 2. Play and compute on the loaded video
|
||
|
||
Opening a recording either serves a compatible stored result or runs one pass behind a
|
||
progress-and-cancel surface. `on_analysis_requested` asks `AnalysisService.stored_result()`
|
||
first — a file read, with `AnalysisProgress.reused` filling the bar — and otherwise starts
|
||
`AnalysisWorker`, a plain thread whose `Progress` is coalesced at 0.1 s, with
|
||
`AnalysisProgress` showing written/expected frames and elapsed time. The timeline stays in
|
||
`recording` mode until the pass commits: a half-written result never reaches the screen.
|
||
|
||
The pass is forward-only (`_pass_frames` walks segments and frames in order; the chain is
|
||
stateful). Cancelling sets an event; the result keeps its `INCOMPLETE` marker and default
|
||
readers refuse it (Part A §7). Measured: 0.48–0.51 wall-seconds per video-second (Part A §8).
|
||
|
||
Source: `analysis/service.py`, `analysis_worker.py`, `ui/analysis_progress.py`.
|
||
Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_runs_no_pass`,
|
||
`::test_opening_a_recording_without_a_result_starts_the_pass`,
|
||
`::test_the_worker_reports_progress_and_a_cancelled_outcome`.
|
||
|
||
## 3. Cursor playback
|
||
|
||
The cursor is one position for the whole panel, on the recording's own signal time: `t_rel_s`
|
||
from `frames.csv`, the axis `ReplayEngine.position()` and `AnalysisReader.frame_at()` index by.
|
||
Container time is never derived from it (Part A §5).
|
||
|
||
Frames are addressed by the recording's own frame id, never an index: `AnalysisSession.frame`
|
||
refuses an id the result does not hold, and the evidence recording's ids run 20…52, so an
|
||
index-based path would answer with another frame's measurement. A seek resolves to the stored
|
||
frame at or before the position — the frame sequential playback passes — and re-runs no pass.
|
||
|
||
Source: `replay.py`, `replay_analysis.py`.
|
||
Verify: `test_replay_analysis.py::test_position_is_signal_time_and_resolves_to_a_stored_frame_id`,
|
||
`::test_a_frame_id_the_recording_does_not_hold_is_refused`,
|
||
`::test_a_seek_equals_playing_forward_to_the_same_frame`.
|
||
|
||
## 4. Synchronised charts, 2D view and 3D view
|
||
|
||
One cursor drives all three.
|
||
|
||
| View | Data source |
|
||
|---|---|
|
||
| Charts | recorded `forces.csv` (`RecordingCurveSource`), or a recomputed trace for bare videos and reconfigured recordings (`AnalysisCurveSource`) |
|
||
| 2D canvas | the cached per-frame samples and depth, drawn at display time |
|
||
| 3D pane | the packet built from the cached frame (`AnalysisSession.packet` → `Scene3DView.submit_packet`) |
|
||
|
||
The charts read one of two sources, and the header names which: `chart.mode_replay` for a
|
||
recording's own `forces.csv` (*the* measurement — a device was in the loop) and
|
||
`chart.mode_recomputed` for the recomputed trace
|
||
(`FRAME_FLAG_FORCE`, 28 B/frame; `provenance.sources["forces"]` says which). With neither —
|
||
nothing analysed yet, or a profile whose chain produced no force for this footage — the header
|
||
states why (`chart.mode_no_trace`) rather than leaving an empty plot.
|
||
|
||
A seek equals playing forward to that frame: the cursor resolves to one stored frame id and all
|
||
three views are fed from it. A derived view chosen while the timeline reads the video is drawn
|
||
from the result at the same cursor (`available_result`). Not synchronised: the 3D pane's depth
|
||
(`AnalysisSession.packet` sets `depth=None`, `depth_valid=False`); forces (the packet carries
|
||
`forces_valid=False` and zero); and the 3D pane with no result. 「从视频重算」 shares
|
||
`on_analysis_requested` with opening a recording — no separate acquisition chain owns the
|
||
timeline.
|
||
|
||
Display settings are reads: the threshold re-derives activity from the stored `magnitude`, colour
|
||
range and idle dots live in the draw call, and a grid-moving setting is refused by name
|
||
(`GRID_KEYS`), the frame still drawn from the recorded grid. The recovered field can only be
|
||
sampled on the pass's own grid, so the frame carries that policy (`ImageFrame.arrow_policy`).
|
||
Verify: `test_replay_analysis.py::test_the_3d_pane_is_fed_the_same_frame_the_canvas_draws`,
|
||
`::test_a_display_change_that_moves_the_grid_is_refused_not_served`.
|
||
|
||
### Calibration choices and curve retention
|
||
|
||
Recording setups scope calibration, confirmed device, unloaded and tracker references to one
|
||
directory; the profile stays locked. References use recorded IDs/times. Chosen calibration content and force-channel choice enter
|
||
the cache key. A complete `sources.forces="recomputed"` result replaces the displayed CSV trace,
|
||
labelled `chart.mode_recomputed`; original files remain unchanged. Failed/cancelled passes
|
||
replace no trace. Without an override, recorded force retains priority.
|
||
|
||
Display controls preserve measurements, session time and partial axes, including FN-only.
|
||
New live runs remain independent; manual graph reset may clear the graph. Storage remains
|
||
bounded and reports drops. See the [SDK calibration contract](../../../../orisys-sdk/docs/calibration-model-contract.zh.md).
|
||
Verify: `test_recording_recalibration.py`, `test_curve_retention.py`, `test_force_history.py`.
|
||
|
||
## 5. What the 2D view can show
|
||
|
||
The modes come from the frame's own fields and from whether a result exists (`plan_image`,
|
||
`result_available`): `arrows_pure`, `arrows_stitched`, `depth`, `recorded`. With a result all
|
||
four are offered, even on the recording's timeline, defaulting to `recorded`; with none,
|
||
`recorded` alone plus one line naming *which* absence it is (`result_state`): no result yet
|
||
(`panels.replay_raw_note`), a pass running (`panels.replay_analysis_running`), a pass failed or
|
||
refused with its own reason (`panels.replay_analysis_failed`), or no pass possible
|
||
(`panels.replay_analysis_unavailable`).
|
||
|
||
| Mode | Drawn from |
|
||
|---|---|
|
||
| recorded | the decoded `raw_*.mp4` frame (`RecordingReader.video_frame_at`) |
|
||
| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,048 B/frame with the depth channel) |
|
||
| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (Part A §1) |
|
||
|
||
A blank depth view is usually correct and must be explainable: `panels.depth_empty` says the
|
||
frame's depth is all zeros, `panels.depth_unavailable` says the frame has no depth channel at
|
||
all — different facts. Measured: the map is exactly all zeros on 32 of the box recording's 33
|
||
frames (Part A §1 has the rest).
|
||
**A box replay's arrow canvas is painted, by the shared painter.** The box style draws through
|
||
the SDK's `orisys.util.draw_arrows`, which sweeps the dense sampling grid; a replay hands it a
|
||
*recovered* field, finite only at the stored candidate cells. Measured on one stored box frame:
|
||
a dense field through the SDK painter paints 24,440 px, that field in its recovered form 0 px,
|
||
the shared painter 1,486 px (1,486–1,964 px over the result's 25 frames with arrows). So the
|
||
painter follows **where the frame came from** — a frame read from a stored result
|
||
(`SOURCE_ANALYSIS`/`SOURCE_REPLAY`) is drawn per sample, a live or recomputed one keeps the SDK
|
||
path (`draw_arrow_display(shared_painter=True)`) — and the box style still owns the
|
||
magnification and the idle-dot tint.
|
||
|
||
A mode needing derived data is served from the result, never faked: with none, that view is not
|
||
offered and the state line names the action. The cursor stays on the recording's own timeline
|
||
while the derived views are read from the result — including while the session that recorded it
|
||
is still live, because reading a result opens no chain.
|
||
Verify: `test_image_backend.py` (the plan and the painter routing),
|
||
`test_bare_video_analysis.py::test_the_box_arrow_canvas_is_painted_by_the_shared_painter`,
|
||
`test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`,
|
||
`::test_the_four_no_result_states_are_four_different_lines`,
|
||
`::test_the_empty_depth_note_is_not_the_missing_depth_note`,
|
||
`test_replay_wiring.py::test_the_operator_sees_the_cached_views_over_the_recording`,
|
||
`::test_a_running_session_does_not_hide_a_stored_result`.
|
||
|
||
## Cache and export policies
|
||
|
||
No rendered 2D images are cached: a PNG would bake display settings into pixels, destroying the
|
||
property that a threshold, colour or idle-dot change is a read. Check:
|
||
`::test_a_display_change_is_a_read_and_writes_no_result`,
|
||
`::test_a_mode_change_writes_no_analysis`. A backward seek cannot re-derive a frame either: the
|
||
chain is forward-only, so it would cost a full re-run (Part A §8).
|
||
|
||
Results live under `%LOCALAPPDATA%\OrisysStudio\analysis-cache\`
|
||
(`paths.analysis_cache_dir()`) — never the user's recording folder or the app's `outputs/` (the
|
||
EXE's own directory in a packaged build). The key is path-independent
|
||
(`recording_identity`), so a copied recording finds its result; a result left under the old
|
||
`<recording>/analysis` root is ignored, neither read nor deleted. A bare video's generated
|
||
workspace lives under `…\video-workspaces\<video digest>\`. Implemented: a 1 GiB cap with LRU
|
||
eviction and `ResultCache.sweep()` at startup. Not implemented: a manual clear action, the opt-in
|
||
wipe-on-exit preference and result export (≈70 MB per ten minutes).
|
||
|
||
## Owners and status
|
||
|
||
| Part | Owner | Status |
|
||
|---|---|---|
|
||
| Format, cache, reader, pass | `orisys_app_shared.analysis` | shipped (`9e20fd0`, cache root and sweep now) |
|
||
| Timeline, cursor, policy, pass thread, progress surface | `replay_analysis.py`, `replay.py`, `replay_controller.py`, `analysis_worker.py`, `ui/analysis_progress.py` | shipped; the no-result states are part of the panel's line |
|
||
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12); the painter is chosen by the frame's source |
|
||
| A bare video: workspace, setup prompt, per-frame force | `analysis/bare_video.py`, `ui/video_analysis_dialog.py`, `analysis/{model,service,reader}.py`, `force/source.py` | shipped (appended `FRAME_FLAG_FORCE`, format version unchanged) |
|
||
| Display-control separation (Part C) | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started (Part B §12) |
|
||
|
||
## Not confirmed in code
|
||
|
||
* The render cost (≈1.4 ms) and the PNG estimate (180–540 MB per ten minutes) are the owner's
|
||
bench; Part B §14's snapshots (4.2–4.9 kB each) suggest the estimate is high.
|