orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

把 Studio 主机协议改成单向端口，并让「从视频重算」走分析入口

Browse Source 
控制器经 AcquireGate / ImageSink / ShellNotes 与信号接线，采集 tick 用 acquire_presenter 上屏；三维测试证明分析结果已喂 3D 后，重算并入 on_analysis_requested。
验证：apps/studio 下相关 pytest 与 tools/ui_check.py（634 checks）通过。
This commit is contained in:
ykcheungab
2026-10-05 14:00:24 +08:00
parent 25ccffd814
commit 28e955a817
27 changed files with 1764 additions and 745 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/studio/docs/replay-panel.md
+88 -87
View File
@@ -6,8 +6,8 @@ playback semantics are owned by Part A ([replay-analysis.md](replay-analysis.md)
([replay-analysis-part-b.md](replay-analysis-part-b.md)).
Status: Parts A and B shipped at `cf49b08`, the depth channel at `9e20fd0`, the result-aware mode
plan and the wiring that reaches it at `1aa4543`. The cache root and the lifetime sweep are
implemented (§Cache and export policies); the display-control separation (Part C) is not.
plan at `1aa4543`. The cache root and the lifetime sweep are implemented; the display-control
separation (Part C) is not.
## 1. Load a video
@@ -15,31 +15,35 @@ Selecting a video always gives a playable timeline, and the panel says what the
`load_video` calls a folder holding `forces.csv` beside the video a recording; anything else
opens as a bare video (`VideoOnlySource`) and plays, with `playback.video_only_opened` on the
rail and `playback.video_only_notice` in the notes. A file that cannot be decoded is refused
with its reason.
with its reason. What a bare video lacks is force rows *measured by a device*, and
`session.json` metadata.
Not available for a bare video: force rows (the charts are empty by construction), `session.json`
metadata, and an analysis pass — `recording_directory()` needs a `session.json` and reports
`playback.analysis_no_source`; the only route to numbers is rerunning the acquisition pipeline over
the video (`on_recompute_toggled`), forward only.
**A bare video can be analysed, and is asked how.** The two files the pass reads are *generated*
for the video in an app-owned workspace and every later step is the existing recording path; the
workspace, its frame index and the synthetic marker are Part A's (§1.1). One prompt asks what a
video cannot carry: the **profile** (from the catalog, each row naming what it can do — flow /
raw force / calibrated axes / depth / 3D), an **optional calibration** defaulting to the profile's
declared `default_calibration`, and a **reference frame** defaulting to the first. A profile the
SDK rejects is listed, disabled and refused with the catalog's own `start_blocked_reason()`.
Declining keeps the old behaviour (`playback.analysis_setup_declined`), and **more than one file
is refused** with its reason.
Source: `record/playback.py`, `replay_controller.py`.
Verify: `test_ui_smoke.py::test_a_bare_video_plays_even_though_it_is_not_a_recording`;
Source: `analysis/bare_video.py`, `ui/video_analysis_dialog.py`, `replay_controller.py`.
Verify: `test_bare_video_analysis.py`, `test_bare_video.py`,
`test_replay_controller.py::test_a_bare_video_is_opened_as_a_video_and_refused_outside_the_view`.
## 2. Play and compute on the loaded video
Opening a recording either serves a compatible stored result or runs one pass behind a
progress-and-cancel surface. `on_analysis_requested` asks
`AnalysisService.stored_result()` first — a file read, the surface showing
`analysis.reused` — and otherwise starts `AnalysisWorker`, a plain thread whose `Progress` is
coalesced at 0.1 s, with `AnalysisProgress` showing written/expected frames and elapsed time.
The timeline stays in `recording` mode until the pass commits: a half-written result never
reaches the screen.
progress-and-cancel surface. `on_analysis_requested` asks `AnalysisService.stored_result()`
first — a file read, the surface showing `analysis.reused` — and otherwise starts
`AnalysisWorker`, a plain thread whose `Progress` is coalesced at 0.1 s, with
`AnalysisProgress` showing written/expected frames and elapsed time. The timeline stays in
`recording` mode until the pass commits: a half-written result never reaches the screen.
The pass is sequential and forward-only: `_pass_frames` walks segments and frames in order, and the
chain is stateful (`Sensor.reset()` clears the tracker). Cancelling sets an event; the result keeps
its `INCOMPLETE` marker and default readers refuse it (Part A §7). Measured: 0.48–0.51
wall-seconds per video-second (Part A §8) — ten minutes is ≈4.8 minutes.
The pass is forward-only (`_pass_frames` walks segments and frames in order; the chain is
stateful). Cancelling sets an event; the result keeps its `INCOMPLETE` marker and default
readers refuse it (Part A §7). Measured: 0.48–0.51 wall-seconds per video-second (Part A §8).
Source: `analysis/service.py`, `analysis_worker.py`, `ui/analysis_progress.py`.
Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_runs_no_pass`,
@@ -50,14 +54,12 @@ Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_
The cursor is one position for the whole panel, on the recording's own signal time: `t_rel_s`
from `frames.csv`, the axis `ReplayEngine.position()` and `AnalysisReader.frame_at()` index by.
Container time is never derived from it or converted into it. Measured: 33 container frames at
30 fps are 1.100 s of video and 1.920 s of signal at 16.664 fps (Part A §5).
Container time is never derived from it (Part A §5).
Frames are addressed by the recording's own frame id, never an index: `AnalysisSession.frame`
refuses an id the result does not hold (`AnalysisNotFoundError`), and the evidence recording's
ids run 20…52, so an index-based path would answer with another frame's measurement. A seek
resolves to the stored frame at or before the position — the frame sequential playback passes —
and re-runs no pass.
refuses an id the result does not hold, and the evidence recording's ids run 20…52, so an
index-based path would answer with another frame's measurement. A seek resolves to the stored
frame at or before the position — the frame sequential playback passes — and re-runs no pass.
Source: `replay.py`, `replay_analysis.py`.
Verify: `test_replay_analysis.py::test_position_is_signal_time_and_resolves_to_a_stored_frame_id`,
@@ -70,58 +72,69 @@ One cursor drives all three.
| View | Data source |
|---|---|
| Charts | the recorded `forces.csv` (`RecordingCurveSource`); the result stores no force |
| Charts | the recorded `forces.csv` (`RecordingCurveSource`), or — over a bare video — the result's own recomputed trace (`AnalysisCurveSource`) |
| 2D canvas | the cached per-frame samples and depth, drawn at display time |
| 3D pane | the packet built from the cached frame (`AnalysisSession.packet` → `Scene3DView.submit_packet`) |
The charts read one of two sources, and the header names which: `chart.mode_replay` for a
recording's own `forces.csv` (*the* measurement — a device was in the loop) and
`chart.mode_recomputed` for the trace the pass recomputed from a bare video
(`FRAME_FLAG_FORCE`, 28 B/frame; `provenance.sources["forces"]` says which). With neither —
nothing analysed yet, or a profile whose chain produced no force for this footage — the header
states why (`chart.mode_no_trace`) rather than leaving an empty plot.
A seek equals playing forward to that frame: the cursor resolves to one stored frame id and all
three views are fed from it. A derived view chosen while the timeline reads the video is drawn
from the result at the same cursor (`available_result`). Not synchronised: the 3D pane's depth
(`AnalysisSession.packet` sets `depth=None`, `depth_valid=False`); forces (the packet carries
`forces_valid=False` and zero, so the pane's force readout is not a measurement here); the 3D
pane with no result, which nothing feeds; and a recompute pass, where the pipeline owns the
timeline (`on_cursor_moved`).
`forces_valid=False` and zero); and the 3D pane with no result. 「从视频重算」 shares
`on_analysis_requested` with opening a recording — no separate acquisition chain owns the
timeline.
Display settings are reads: the threshold re-derives activity from the stored `magnitude`, colour
range and idle dots live in the draw call, and a grid-moving setting is refused by name
(`GRID_KEYS`), the frame still drawn from the recorded grid.
(`GRID_KEYS`), the frame still drawn from the recorded grid. The recovered field can only be
sampled on the pass's own grid, so the frame carries that policy (`ImageFrame.arrow_policy`).
Verify: `test_replay_analysis.py::test_the_3d_pane_is_fed_the_same_frame_the_canvas_draws`,
`::test_a_display_change_that_moves_the_grid_is_refused_not_served`.
## 5. What the 2D view can show
The modes come from the frame's own fields and from whether a result exists (`plan_image`,
`result_available`): `arrows_pure` (Arrows), `arrows_stitched` (Arrows / Captured), `depth`
(Depth map), `recorded` (Recorded image). With a result all four are offered, even on the
recording's timeline, defaulting to `recorded`; with none, `recorded` alone plus one line naming
*which* absence it is (`result_state`): no result yet (`panels.replay_raw_note`), a pass running
(`panels.replay_analysis_running`), a pass failed or refused with its own reason
(`panels.replay_analysis_failed`), or no pass possible at all — a bare video
`result_available`): `arrows_pure`, `arrows_stitched`, `depth`, `recorded`. With a result all
four are offered, even on the recording's timeline, defaulting to `recorded`; with none,
`recorded` alone plus one line naming *which* absence it is (`result_state`): no result yet
(`panels.replay_raw_note`), a pass running (`panels.replay_analysis_running`), a pass failed or
refused with its own reason (`panels.replay_analysis_failed`), or no pass possible
(`panels.replay_analysis_unavailable`).
| Mode | Drawn from |
|---|---|
| recorded | the decoded `raw_*.mp4` frame (`RecordingReader.video_frame_at`) |
| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,034 B/frame before the depth channel; 4,048 B/frame now) |
| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (island restriction rejected: 418 B vs 486 B on the worst frame) |
| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,048 B/frame with the depth channel) |
| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (Part A §1) |
Measured: the depth map is exactly all zeros on 241 of 264 frames of the eight-times clip
(32 of 33 on the box recording) and never exceeded 336 of 84,000 pixels (0.4 %). A blank depth view
is therefore usually correct and must be explainable: `panels.depth_empty` says the frame's depth is
all zeros, `panels.depth_unavailable` says the frame has no depth channel at all — different facts,
neither drawing the processed image.
**A box replay's arrow canvas is empty** (pre-existing): the box style draws through the SDK's
`orisys.util.draw_arrows`, while a replay hands it the recovered field, finite only at the stored
candidate cells — one dense field paints 24,440 px, its recovered form 0 px, the shared painter
1,486 px.
A blank depth view is usually correct and must be explainable: `panels.depth_empty` says the
frame's depth is all zeros, `panels.depth_unavailable` says the frame has no depth channel at
all — different facts. Measured: the map is exactly all zeros on 32 of the box recording's 33
frames (Part A §1 has the rest).
**A box replay's arrow canvas is painted, by the shared painter.** The box style draws through
the SDK's `orisys.util.draw_arrows`, which sweeps the dense sampling grid; a replay hands it a
*recovered* field, finite only at the stored candidate cells. Measured on one stored box frame:
a dense field through the SDK painter paints 24,440 px, that field in its recovered form 0 px,
the shared painter 1,486 px (1,486–1,964 px over the result's 25 frames with arrows). So the
painter follows **where the frame came from** — a frame read from a stored result
(`SOURCE_ANALYSIS`/`SOURCE_REPLAY`) is drawn per sample, a live or recomputed one keeps the SDK
path (`draw_arrow_display(shared_painter=True)`) — and the box style still owns the
magnification and the idle-dot tint.
A mode needing derived data is served from the result, never faked: with none, that view is not
offered and the state line names the action or the reason. The state above is what an operator
reaches by opening a recording that has a result: the cursor stays on the recording's own timeline
and the panel opens on a derived view, the recorded one still offered — including while the
session that recorded it is still live, because reading a result opens no chain.
Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`,
offered and the state line names the action. The cursor stays on the recording's own timeline
while the derived views are read from the result — including while the session that recorded it
is still live, because reading a result opens no chain.
Verify: `test_image_backend.py` (the plan and the painter routing),
`test_bare_video_analysis.py::test_the_box_arrow_canvas_is_painted_by_the_shared_painter`,
`test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`,
`::test_the_four_no_result_states_are_four_different_lines`,
`::test_the_empty_depth_note_is_not_the_missing_depth_note`,
`test_replay_wiring.py::test_the_operator_sees_the_cached_views_over_the_recording`,
@@ -129,35 +142,20 @@ Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_der
## Cache and export policies
No rendered 2D images are cached. PNGs of the arrow and depth views would cost 180–540 MB per
ten minutes and bake display settings into pixels, destroying the property that a threshold,
colour or idle-dot change is a read. Drawing a frame from the cached numbers costs ≈1.4 ms, the
idle-dot pass 37 µs (owner's bench, not reproduced here). Check:
`::test_a_display_change_is_a_read_and_writes_no_result`, `::test_a_mode_change_writes_no_analysis`.
No rendered 2D images are cached: a PNG would bake display settings into pixels, destroying the
property that a threshold, colour or idle-dot change is a read. Check:
`::test_a_display_change_is_a_read_and_writes_no_result`,
`::test_a_mode_change_writes_no_analysis`. A backward seek cannot re-derive a frame either: the
chain is forward-only, so it would cost a full re-run (Part A §8).
Derived views cannot be computed from raw data during a scrub: the chain is stateful and
forward-only, so a backward seek would need a full re-run at 0.48–0.51 s per video-second; the
panel scrubs stored numbers and offers that run separately.
Cache location. Results live under the OS-correct derived-data root,
`%LOCALAPPDATA%\OrisysStudio\analysis-cache\` (`paths.analysis_cache_dir()`) — never the user's
recording folder or the app's `outputs/`, which in the packaged build is the EXE's own directory
and holds the recordings. The key is path-independent (`recording_identity`), so a copied
recording finds its result; a result left under the old `<recording>/analysis` root is ignored —
neither read nor deleted (`test_replay_analysis.py::test_a_result_under_the_old_root_is_ignored_not_migrated`).
Cache lifetime. A size cap with LRU eviction (1 GiB, `DEFAULT_MAX_BYTES`), a startup sweep of
stale and incomplete results, a manual "clear analysis cache" action, and wipe-on-exit only as an
explicit preference that is off by default — because wiping on exit forces a full re-analysis
(≈4.8 minutes per ten minutes) on reopen and does not protect against a crash. Implemented:
`evict()` (incomplete first, then oldest, protecting `keep`) and `ResultCache.sweep()`, which
clears incomplete results older than an hour and then applies the cap, called once at startup
(`ReplayController.sweep_cache`). Not implemented: the clear action and the wipe-on-exit
preference.
Export. The deliberate way to keep a result is to export it: ≈70 MB per ten minutes, portable
because the key ignores where the file sits. A rendered arrow or depth video belongs to export
rather than the cache. Not implemented.
Results live under `%LOCALAPPDATA%\OrisysStudio\analysis-cache\`
(`paths.analysis_cache_dir()`) — never the user's recording folder or the app's `outputs/` (the
EXE's own directory in a packaged build). The key is path-independent
(`recording_identity`), so a copied recording finds its result; a result left under the old
`<recording>/analysis` root is ignored, neither read nor deleted. A bare video's generated
workspace lives under `…\video-workspaces\<video digest>\`. Implemented: a 1 GiB cap with LRU
eviction and `ResultCache.sweep()` at startup. Not implemented: a manual clear action, the opt-in
wipe-on-exit preference and result export (≈70 MB per ten minutes).
## Owners and status
@@ -165,14 +163,17 @@ rather than the cache. Not implemented.
|---|---|---|
| Format, cache, reader, pass | `orisys_app_shared.analysis` | shipped (`9e20fd0`, cache root and sweep now) |
| Timeline, cursor, policy, pass thread, progress surface | `replay_analysis.py`, `replay.py`, `replay_controller.py`, `analysis_worker.py`, `ui/analysis_progress.py` | shipped; the no-result states are part of the panel's line |
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12 names the seams) |
| Cache root and lifetime | `analysis/cache.py`, `paths.py`, `app_window.py` | root and startup sweep implemented; clear action and wipe-on-exit preference are not |
| Display-control separation: rail reset as the measurement reference, a separate display baseline, the F8 view reset | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started; Part B §12 records these as unchanged. The pass accepts a `reference_frame` but `_analysis_service` passes none |
| Mode plan, drawing, window wiring | `image_backend.py`, `acquisition/arrows.py`, `app_window.py` | shipped (Part B §12); the painter is chosen by the frame's source |
| A bare video: workspace, setup prompt, per-frame force | `analysis/bare_video.py`, `ui/video_analysis_dialog.py`, `analysis/{model,service,reader}.py`, `force/source.py` | shipped (appended `FRAME_FLAG_FORCE`, format version unchanged) |
| Display-control separation (Part C) | `toolbar.py`, `display_prefs.py`, `app_window.py` | not started (Part B §12) |
## Not confirmed in code
* The render cost (≈1.4 ms, idle dots 37 µs) and the PNG estimate (180–540 MB per ten minutes)
are the owner's bench, not measurements here; Part B §14's single-frame snapshots (4.2–4.9 kB)
suggest the estimate is high, so it should be re-measured.
* The render cost (≈1.4 ms) and the PNG estimate (180–540 MB per ten minutes) are the owner's
bench; Part B §14's snapshots (4.2–4.9 kB each) suggest the estimate is high.
* The manual clear action, the wipe-on-exit preference and result export have no code in
`orisys-app`; they are decisions.
* A chosen calibration can be applied to the **box** chain only (`Sensor(cal_path=…)`). The
finger runtime opens its sensor without one, so a non-default curve for a finger profile is
refused rather than silently ignored — the SDK has no entry point for it (reported, not
worked around).