diff --git a/apps/studio/config/locales/en_US.json b/apps/studio/config/locales/en_US.json index 22a5f78..f888d0b 100644 --- a/apps/studio/config/locales/en_US.json +++ b/apps/studio/config/locales/en_US.json @@ -181,7 +181,10 @@ "export_identity": "file identity type {sensor_type} · SN {sn} · config {config}", "thumb_placeholder": "Product model", "family_unidentified": "Unidentified", - "catalog_profile": "Profile {profile_id} · family {family} · from {origin} · {state}", + "catalog_profile": "Profile {profile_id} · {shape} · from {origin} · SDK validation: {state}", + "catalog_shape_descriptor": "descriptor bundle", + "catalog_shape_compatibility": "compatibility bundle, no descriptor", + "catalog_shape_unreadable": "no readable bundle", "catalog_blocked": "Cannot start: {reason}" }, "panels": { diff --git a/apps/studio/config/locales/zh_CN.json b/apps/studio/config/locales/zh_CN.json index ef73faf..3cb3711 100644 --- a/apps/studio/config/locales/zh_CN.json +++ b/apps/studio/config/locales/zh_CN.json @@ -181,7 +181,10 @@ "export_identity": "文件身份 类型 {sensor_type} · SN {sn} · 配置 {config}", "thumb_placeholder": "产品型号", "family_unidentified": "未识别", - "catalog_profile": "配置档案 {profile_id} · 家族 {family} · 来源 {origin} · 状态 {state}", + "catalog_profile": "配置档案 {profile_id} · {shape} · 来源 {origin} · SDK 校验:{state}", + "catalog_shape_descriptor": "描述符配置包", + "catalog_shape_compatibility": "兼容配置包(无描述符)", + "catalog_shape_unreadable": "没有可读取的配置包", "catalog_blocked": "无法启动:{reason}" }, "panels": { diff --git a/apps/studio/config/sensors/finger/manifest.json b/apps/studio/config/sensors/finger/manifest.json index 27f92c8..6f87d2f 100644 --- a/apps/studio/config/sensors/finger/manifest.json +++ b/apps/studio/config/sensors/finger/manifest.json @@ -1,6 +1,17 @@ { - "name": "finger", + "schema_version": 1, + "profile_id": "orisys.finger.r01", + "display_name": "Finger R01", + "product_family": "finger", + "processing_backend": "uv_unroll", "sensor_json": "sensor.json", + "computational_assets": { + "unroll_map": "assets/30fps-0703/unroll_map_uv_s0.42.npz" + }, + "assets": { + "scene": "scene/finger-display-1.0.0.oryscene" + }, + "name": "finger", "assets_dir": "assets/30fps-0703", "viz_pose": "assets/visualization/world_to_cam.npy" } diff --git a/apps/studio/docs/replay-analysis-part-b.md b/apps/studio/docs/replay-analysis-part-b.md index 264cf00..d6698b2 100644 --- a/apps/studio/docs/replay-analysis-part-b.md +++ b/apps/studio/docs/replay-analysis-part-b.md @@ -19,8 +19,8 @@ (连"原始图像"一起),派生模式那些帧一律从结果读、按窗口唯一的 2D 出口画。 图表、二维画布、三维面板都读这一个游标,模式由 `ReplayEngine.mode()` 回答;「从视频重算」 -因此不再是第三种来源,而是**显式重跑**这一遍的开关(第 10 节)。时间基准是**信号时间** -(Part A 第 5 节),进度、位置、单步与跳转都说它。 +与打开录像共用 `ReplayController.on_analysis_requested()`(第 10 节),不再另开一条采集链。 +时间基准是**信号时间**(Part A 第 5 节),进度、位置、单步与跳转都说它。 **应用里的时间轴永远是录像的。** 引擎留在 `recording`:游标、帧 id 与时长都是录像自己的, 结果挂在游标后面当缓存(`available_result()`),`_adopt_session()` 调用 `use_recording()` 保证 @@ -58,10 +58,14 @@ | 视图 | 读什么 | 怎么读 | |---|---|---| -| 图表 | `forces.csv`(记录值) | `charts.set_cursor(t)` 用同一游标;结果里不存力 | +| 图表 | `forces.csv`(记录值),**或结果里重算的力**(裸视频) | `charts.set_cursor(t)` 用同一游标;两条来源在标题上分开命名(`chart.mode_replay` / `chart.mode_recomputed`) | | 二维画布 | 结果的 `flow`、`depth` 与 `valid_mask` | `ImageFrame` 走窗口唯一的 2D 出口 `_show_image_frame`(录像时间轴上的派生视图带 `source="replay"` 与 `result_available=True`,因此"原始图像"仍在选择器里),箭头、暗点、配色、域边界与深度视图与实时/重算**同一套代码** | | 三维面板 | 同一帧的 `SdkPacket` | `session.packet(frame)` → `Scene3DView.submit_packet`,走同一个 adapter、域掩码 latch 与有效性求交 | +**裸视频的力是重算的,而且只有这一条。** 没有 `forces.csv` 时,pass 把链路对每一帧算出的力按帧 +存进结果(`FRAME_FLAG_FORCE`;Part A 第 1.1 节),曲线读它,标题写 `chart.mode_recomputed`。没有 +任何力可画时标题说明原因。 + **没有结果时那一行必须说清是四种里的哪一种。** `plan_image()` 只拿到 `ImageFrame`,四种状态 因此由生产者写进帧里(`result_state`/`result_reason`),`ReplayController.result_state()` 是唯一 的判断处;有结果时这一行根本不出现: @@ -142,18 +146,17 @@ on_progress)`(`chain_spec_for` 从录像自己的 `session.json` 取链路, ## 15. 已知限制 -* 结果里没有力:力曲线始终读记录的 `forces.csv`。深度有了(第 1、11 节),但**三维面板仍只 - 拿光流**:`session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False` —— 接进 - 三维会改变三维图形的构成,不在这一片里。 -* **box 的箭头画布在回放里是空的**(本片实测,非本次引入):box 的绘制路径吃不下只有候选格 - 有限的恢复场,同一帧密流场画 24,440 px、恢复场 0 px;见 [replay-panel.md](replay-panel.md) 第 5 节。 +* **录像**的结果里没有力:力曲线读那份录像自己的 `forces.csv`。裸视频是另一回事(Part A 第 1.1 + 节):pass 存下链路重算的力,曲线读它。深度有了(第 1、11 节),但**三维面板仍只拿光流**: + `session.packet()` 的 `depth` 依旧是 `None`、`depth_valid` 依旧是 `False` —— 接进三维会改变 + 三维图形的构成,不在这一片里。 * 结果覆盖的帧是 pass 走过的那些:**结果的第一帧可能晚于录像的起点**(box 链路构造时吃掉前 7 个解码帧,这份结果从 t=0.4162 s 开始),最后一批帧也可能早于录像的终点。游标落在这两段里 - 时,派生视图按 `frame_id_at()` 的"该时刻或之前、越界夹到端点"给帧:起点之前画的是最早那一帧 - 测量,终点之后画的是最后一帧。两段没有各自的提示,"不会静默重复最后一帧"在本片没有兑现。 -* 栅格类显示设置不生效(第 12 节);换栅格就是重跑一遍,那是**另一个键**。 + 时,派生视图按 `frame_id_at()` 的"该时刻或之前、越界夹到端点"给帧,两段没有各自的提示。 +* 栅格类显示设置不生效(第 12 节);换栅格就是重跑一遍,那是**另一个键**。被拒的栅格**仍按 + 结果自己的网格绘制**(帧携带 `arrow_policy`)。 * 一个进程同时只有一遍分析在跑;采集会话运行期间不启动分析 —— 但**读结果不受这条限制**(第 10 - 节):会话开着时面板照画缓存,只是新的那一遍要等会话停下、「从视频重算」再点一次。 + 节):会话开着时面板照画缓存,新的那一遍要等会话停下、「从视频重算」再点一次。 * "有没有结果"是录像的属性、不是游标的属性:`available_result()` 查的是本录像在 `ResultLibrary` 里的那一份,第 9 节的接线之后这是常态。 * 面板的 `failed` 把"跑挂"和"被拒跑"合成一句(原因与动作都不同:一个要修链路,一个要停会话); diff --git a/apps/studio/docs/replay-panel.md b/apps/studio/docs/replay-panel.md index c033eef..9d6f7c3 100644 --- a/apps/studio/docs/replay-panel.md +++ b/apps/studio/docs/replay-panel.md @@ -6,8 +6,8 @@ playback semantics are owned by Part A ([replay-analysis.md](replay-analysis.md) ([replay-analysis-part-b.md](replay-analysis-part-b.md)). Status: Parts A and B shipped at `cf49b08`, the depth channel at `9e20fd0`, the result-aware mode -plan and the wiring that reaches it at `1aa4543`. The cache root and the lifetime sweep are -implemented (§Cache and export policies); the display-control separation (Part C) is not. +plan at `1aa4543`. The cache root and the lifetime sweep are implemented; the display-control +separation (Part C) is not. ## 1. Load a video @@ -15,31 +15,35 @@ Selecting a video always gives a playable timeline, and the panel says what the `load_video` calls a folder holding `forces.csv` beside the video a recording; anything else opens as a bare video (`VideoOnlySource`) and plays, with `playback.video_only_opened` on the rail and `playback.video_only_notice` in the notes. A file that cannot be decoded is refused -with its reason. +with its reason. What a bare video lacks is force rows *measured by a device*, and +`session.json` metadata. -Not available for a bare video: force rows (the charts are empty by construction), `session.json` -metadata, and an analysis pass — `recording_directory()` needs a `session.json` and reports -`playback.analysis_no_source`; the only route to numbers is rerunning the acquisition pipeline over -the video (`on_recompute_toggled`), forward only. +**A bare video can be analysed, and is asked how.** The two files the pass reads are *generated* +for the video in an app-owned workspace and every later step is the existing recording path; the +workspace, its frame index and the synthetic marker are Part A's (§1.1). One prompt asks what a +video cannot carry: the **profile** (from the catalog, each row naming what it can do — flow / +raw force / calibrated axes / depth / 3D), an **optional calibration** defaulting to the profile's +declared `default_calibration`, and a **reference frame** defaulting to the first. A profile the +SDK rejects is listed, disabled and refused with the catalog's own `start_blocked_reason()`. +Declining keeps the old behaviour (`playback.analysis_setup_declined`), and **more than one file +is refused** with its reason. -Source: `record/playback.py`, `replay_controller.py`. -Verify: `test_ui_smoke.py::test_a_bare_video_plays_even_though_it_is_not_a_recording`; +Source: `analysis/bare_video.py`, `ui/video_analysis_dialog.py`, `replay_controller.py`. +Verify: `test_bare_video_analysis.py`, `test_bare_video.py`, `test_replay_controller.py::test_a_bare_video_is_opened_as_a_video_and_refused_outside_the_view`. ## 2. Play and compute on the loaded video Opening a recording either serves a compatible stored result or runs one pass behind a -progress-and-cancel surface. `on_analysis_requested` asks -`AnalysisService.stored_result()` first — a file read, the surface showing -`analysis.reused` — and otherwise starts `AnalysisWorker`, a plain thread whose `Progress` is -coalesced at 0.1 s, with `AnalysisProgress` showing written/expected frames and elapsed time. -The timeline stays in `recording` mode until the pass commits: a half-written result never -reaches the screen. +progress-and-cancel surface. `on_analysis_requested` asks `AnalysisService.stored_result()` +first — a file read, the surface showing `analysis.reused` — and otherwise starts +`AnalysisWorker`, a plain thread whose `Progress` is coalesced at 0.1 s, with +`AnalysisProgress` showing written/expected frames and elapsed time. The timeline stays in +`recording` mode until the pass commits: a half-written result never reaches the screen. -The pass is sequential and forward-only: `_pass_frames` walks segments and frames in order, and the -chain is stateful (`Sensor.reset()` clears the tracker). Cancelling sets an event; the result keeps -its `INCOMPLETE` marker and default readers refuse it (Part A §7). Measured: 0.48–0.51 -wall-seconds per video-second (Part A §8) — ten minutes is ≈4.8 minutes. +The pass is forward-only (`_pass_frames` walks segments and frames in order; the chain is +stateful). Cancelling sets an event; the result keeps its `INCOMPLETE` marker and default +readers refuse it (Part A §7). Measured: 0.48–0.51 wall-seconds per video-second (Part A §8). Source: `analysis/service.py`, `analysis_worker.py`, `ui/analysis_progress.py`. Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_runs_no_pass`, @@ -50,14 +54,12 @@ Verify: `test_replay_analysis.py::test_opening_a_recording_with_a_stored_result_ The cursor is one position for the whole panel, on the recording's own signal time: `t_rel_s` from `frames.csv`, the axis `ReplayEngine.position()` and `AnalysisReader.frame_at()` index by. -Container time is never derived from it or converted into it. Measured: 33 container frames at -30 fps are 1.100 s of video and 1.920 s of signal at 16.664 fps (Part A §5). +Container time is never derived from it (Part A §5). Frames are addressed by the recording's own frame id, never an index: `AnalysisSession.frame` -refuses an id the result does not hold (`AnalysisNotFoundError`), and the evidence recording's -ids run 20…52, so an index-based path would answer with another frame's measurement. A seek -resolves to the stored frame at or before the position — the frame sequential playback passes — -and re-runs no pass. +refuses an id the result does not hold, and the evidence recording's ids run 20…52, so an +index-based path would answer with another frame's measurement. A seek resolves to the stored +frame at or before the position — the frame sequential playback passes — and re-runs no pass. Source: `replay.py`, `replay_analysis.py`. Verify: `test_replay_analysis.py::test_position_is_signal_time_and_resolves_to_a_stored_frame_id`, @@ -70,58 +72,69 @@ One cursor drives all three. | View | Data source | |---|---| -| Charts | the recorded `forces.csv` (`RecordingCurveSource`); the result stores no force | +| Charts | the recorded `forces.csv` (`RecordingCurveSource`), or — over a bare video — the result's own recomputed trace (`AnalysisCurveSource`) | | 2D canvas | the cached per-frame samples and depth, drawn at display time | | 3D pane | the packet built from the cached frame (`AnalysisSession.packet` → `Scene3DView.submit_packet`) | +The charts read one of two sources, and the header names which: `chart.mode_replay` for a +recording's own `forces.csv` (*the* measurement — a device was in the loop) and +`chart.mode_recomputed` for the trace the pass recomputed from a bare video +(`FRAME_FLAG_FORCE`, 28 B/frame; `provenance.sources["forces"]` says which). With neither — +nothing analysed yet, or a profile whose chain produced no force for this footage — the header +states why (`chart.mode_no_trace`) rather than leaving an empty plot. + A seek equals playing forward to that frame: the cursor resolves to one stored frame id and all three views are fed from it. A derived view chosen while the timeline reads the video is drawn from the result at the same cursor (`available_result`). Not synchronised: the 3D pane's depth (`AnalysisSession.packet` sets `depth=None`, `depth_valid=False`); forces (the packet carries -`forces_valid=False` and zero, so the pane's force readout is not a measurement here); the 3D -pane with no result, which nothing feeds; and a recompute pass, where the pipeline owns the -timeline (`on_cursor_moved`). +`forces_valid=False` and zero); and the 3D pane with no result. 「从视频重算」 shares +`on_analysis_requested` with opening a recording — no separate acquisition chain owns the +timeline. Display settings are reads: the threshold re-derives activity from the stored `magnitude`, colour range and idle dots live in the draw call, and a grid-moving setting is refused by name -(`GRID_KEYS`), the frame still drawn from the recorded grid. +(`GRID_KEYS`), the frame still drawn from the recorded grid. The recovered field can only be +sampled on the pass's own grid, so the frame carries that policy (`ImageFrame.arrow_policy`). Verify: `test_replay_analysis.py::test_the_3d_pane_is_fed_the_same_frame_the_canvas_draws`, `::test_a_display_change_that_moves_the_grid_is_refused_not_served`. ## 5. What the 2D view can show The modes come from the frame's own fields and from whether a result exists (`plan_image`, -`result_available`): `arrows_pure` (Arrows), `arrows_stitched` (Arrows / Captured), `depth` -(Depth map), `recorded` (Recorded image). With a result all four are offered, even on the -recording's timeline, defaulting to `recorded`; with none, `recorded` alone plus one line naming -*which* absence it is (`result_state`): no result yet (`panels.replay_raw_note`), a pass running -(`panels.replay_analysis_running`), a pass failed or refused with its own reason -(`panels.replay_analysis_failed`), or no pass possible at all — a bare video +`result_available`): `arrows_pure`, `arrows_stitched`, `depth`, `recorded`. With a result all +four are offered, even on the recording's timeline, defaulting to `recorded`; with none, +`recorded` alone plus one line naming *which* absence it is (`result_state`): no result yet +(`panels.replay_raw_note`), a pass running (`panels.replay_analysis_running`), a pass failed or +refused with its own reason (`panels.replay_analysis_failed`), or no pass possible (`panels.replay_analysis_unavailable`). | Mode | Drawn from | |---|---| | recorded | the decoded `raw_*.mp4` frame (`RecordingReader.video_frame_at`) | -| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,034 B/frame before the depth channel; 4,048 B/frame now) | -| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (island restriction rejected: 418 B vs 486 B on the worst frame) | +| arrows / arrows-captured | the cached samples: the grid keyframe's locations plus this frame's float32 `dx`/`dy`/`magnitude` and bit-packed `active` (4,048 B/frame with the depth channel) | +| depth | a per-frame depth flag, a non-zero count and min/max; only a non-empty map adds a uint8 plane quantised over that frame's range and zlib'd at level 6 (Part A §1) | -Measured: the depth map is exactly all zeros on 241 of 264 frames of the eight-times clip -(32 of 33 on the box recording) and never exceeded 336 of 84,000 pixels (0.4 %). A blank depth view -is therefore usually correct and must be explainable: `panels.depth_empty` says the frame's depth is -all zeros, `panels.depth_unavailable` says the frame has no depth channel at all — different facts, -neither drawing the processed image. - -**A box replay's arrow canvas is empty** (pre-existing): the box style draws through the SDK's -`orisys.util.draw_arrows`, while a replay hands it the recovered field, finite only at the stored -candidate cells — one dense field paints 24,440 px, its recovered form 0 px, the shared painter -1,486 px. +A blank depth view is usually correct and must be explainable: `panels.depth_empty` says the +frame's depth is all zeros, `panels.depth_unavailable` says the frame has no depth channel at +all — different facts. Measured: the map is exactly all zeros on 32 of the box recording's 33 +frames (Part A §1 has the rest). +**A box replay's arrow canvas is painted, by the shared painter.** The box style draws through +the SDK's `orisys.util.draw_arrows`, which sweeps the dense sampling grid; a replay hands it a +*recovered* field, finite only at the stored candidate cells. Measured on one stored box frame: +a dense field through the SDK painter paints 24,440 px, that field in its recovered form 0 px, +the shared painter 1,486 px (1,486–1,964 px over the result's 25 frames with arrows). So the +painter follows **where the frame came from** — a frame read from a stored result +(`SOURCE_ANALYSIS`/`SOURCE_REPLAY`) is drawn per sample, a live or recomputed one keeps the SDK +path (`draw_arrow_display(shared_painter=True)`) — and the box style still owns the +magnification and the idle-dot tint. A mode needing derived data is served from the result, never faked: with none, that view is not -offered and the state line names the action or the reason. The state above is what an operator -reaches by opening a recording that has a result: the cursor stays on the recording's own timeline -and the panel opens on a derived view, the recorded one still offered — including while the -session that recorded it is still live, because reading a result opens no chain. -Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`, +offered and the state line names the action. The cursor stays on the recording's own timeline +while the derived views are read from the result — including while the session that recorded it +is still live, because reading a result opens no chain. +Verify: `test_image_backend.py` (the plan and the painter routing), +`test_bare_video_analysis.py::test_the_box_arrow_canvas_is_painted_by_the_shared_painter`, +`test_replay_analysis.py::test_a_derived_mode_is_served_from_the_result_even_on_the_recording_timeline`, `::test_the_four_no_result_states_are_four_different_lines`, `::test_the_empty_depth_note_is_not_the_missing_depth_note`, `test_replay_wiring.py::test_the_operator_sees_the_cached_views_over_the_recording`, @@ -129,35 +142,20 @@ Verify: `test_image_backend.py` (the plan), `test_replay_analysis.py::test_a_der ## Cache and export policies -No rendered 2D images are cached. PNGs of the arrow and depth views would cost 180–540 MB per -ten minutes and bake display settings into pixels, destroying the property that a threshold, -colour or idle-dot change is a read. Drawing a frame from the cached numbers costs ≈1.4 ms, the -idle-dot pass 37 µs (owner's bench, not reproduced here). Check: -`::test_a_display_change_is_a_read_and_writes_no_result`, `::test_a_mode_change_writes_no_analysis`. +No rendered 2D images are cached: a PNG would bake display settings into pixels, destroying the +property that a threshold, colour or idle-dot change is a read. Check: +`::test_a_display_change_is_a_read_and_writes_no_result`, +`::test_a_mode_change_writes_no_analysis`. A backward seek cannot re-derive a frame either: the +chain is forward-only, so it would cost a full re-run (Part A §8). -Derived views cannot be computed from raw data during a scrub: the chain is stateful and -forward-only, so a backward seek would need a full re-run at 0.48–0.51 s per video-second; the -panel scrubs stored numbers and offers that run separately. - -Cache location. Results live under the OS-correct derived-data root, -`%LOCALAPPDATA%\OrisysStudio\analysis-cache\` (`paths.analysis_cache_dir()`) — never the user's -recording folder or the app's `outputs/`, which in the packaged build is the EXE's own directory -and holds the recordings. The key is path-independent (`recording_identity`), so a copied -recording finds its result; a result left under the old `/analysis` root is ignored — -neither read nor deleted (`test_replay_analysis.py::test_a_result_under_the_old_root_is_ignored_not_migrated`). - -Cache lifetime. A size cap with LRU eviction (1 GiB, `DEFAULT_MAX_BYTES`), a startup sweep of -stale and incomplete results, a manual "clear analysis cache" action, and wipe-on-exit only as an -explicit preference that is off by default — because wiping on exit forces a full re-analysis -(≈4.8 minutes per ten minutes) on reopen and does not protect against a crash. Implemented: -`evict()` (incomplete first, then oldest, protecting `keep`) and `ResultCache.sweep()`, which -clears incomplete results older than an hour and then applies the cap, called once at startup -(`ReplayController.sweep_cache`). Not implemented: the clear action and the wipe-on-exit -preference. - -Export. The deliberate way to keep a result is to export it: ≈70 MB per ten minutes, portable -because the key ignores where the file sits. A rendered arrow or depth video belongs to export -rather than the cache. Not implemented. +Results live under `%LOCALAPPDATA%\OrisysStudio\analysis-cache\` +(`paths.analysis_cache_dir()`) — never the user's recording folder or the app's `outputs/` (the +EXE's own directory in a packaged build). The key is path-independent +(`recording_identity`), so a copied recording finds its result; a result left under the old +`/analysis` root is ignored, neither read nor deleted. A bare video's generated +workspace lives under `…\video-workspaces\