orisys-app

orisys-tech/orisys-app

Fork 0

Commit Graph

Author	SHA1	Message	Date
ykcheungab	fa2703968e	added ruff test	2026-09-27 19:20:43 +08:00
ykcheungab	73592f085f	Browse a long recording from disk instead of loading it into memory What - record/index.py: RecordingIndex streams forces.csv once and keeps only the small things — one time, one validity flag, one frame id per row plus a byte offset every 4096 rows (~6.5 MB per 500k rows). row_range() bisects the times, read_rows() seeks to the nearest checkpoint and scans forward, and overview() folds every row into its bucket during the same pass to build a per-axis min/max envelope. - read_rows(max_points, unit) decimates while streaming: each bucket keeps the extreme row of every axis in the requested unit, so a long interval never has to exist in memory in full and a shear spike survives a flat normal channel. - RecordingReader: info() decimates above max_rows (default 200k) and marks itself decimated, with total_rows and a duration taken from the frame index rather than from the view; window()/window_all()/overview()/statistics() read the interval being asked about. list/window semantics unchanged for small recordings. - force/source.py: StoreCurveSource (live memory) and RecordingCurveSource (disk interval + cached envelope) implement one interface — extent, window, envelope, statistics, duration_s — so ForcePlotsPanel asks a source instead of knowing where the data lives. - force_plots: reads through the source (window, envelope, statistics, extent), which also replaces its last direct store accesses. - playback_dialog: draws the whole-session view through window_all() in the unit being shown (bounded by CHART_MAX_POINTS), and says so when the view is decimated ("显示 N / 共 M 行") instead of letting the operator count points. - experiment._run: a failure that lands while the files are opening can no longer be overwritten by the writer's own transition to "recording" (found by a flaky test; the state check now covers STATE_FAILED). Why The store releases old blocks above ~4.6 h and the replay loaded every CSV row into Python lists before drawing anything, so neither the live browser nor the replay could show an earlier interval of a long recording. The index makes the disk the data source and memory a cache of what is on screen. Verification - shared/tests/test_history_source.py (new, 8 cases) over a 12 000-row recording: the index reports its rows and reads a bounded interval; a large recording is marked decimated with its true duration and total rows; a small one is not; an interval read sees the exact rows including a single peak; a decimated window keeps the peak; the envelope fits the whole session in ≤200 points, keeps the spike and is cached; the disk and memory sources agree on extent, values and statistics; the whole session can be read without holding it all. Shared suites: 146 tests, 0 failures (three consecutive runs). - apps/studio/tools/ui_check.py: the replay section now checks the index row count, that an interval read spans only the interval, the envelope size and total_rows. 324 checks, 0 failures. - apps/studio/tests/test_ui_smoke.py: 43 tests, 36 pass; the 7 failures are the unchanged sandbox/harness artifacts. - D:/orisys_ws/_review_phase12/reproduce.py: all ten lines unchanged; compileall, locale and prefs parity clean. - apps/studio/docs/record-channel.md §8 documents the index, the reader API and the unified source, and lists what is still hardware work. Not in this change - Maximum RSS / backlog / loss measurements at real resolution on the target ARM board (needs the hardware), and range CSV export.	2026-09-13 16:54:26 +08:00
ykcheungab	22cc59c402	Fix the phase-1/2 review findings in recording, calibration and depth replay What - record: one packet->row conversion (record/sample.py). A failed compute chain records no force value at all, a newton value is written only for an axis that really has a calibration curve, and mono_t (not the wall clock) orders history with a per-row calibration hash. - record lifecycle: explicit starting/recording/stopping/stopped/failed states owned by the writer thread; stop() reports completed=False on timeout and does not write ended_at; consecutive write failures stop acceptance immediately. - record capacity: byte budget for the video queue (512 MiB default); a full queue sheds the frame and parks the tiny data row, so a video overflow no longer deletes CSV rows. Lost rows leave a dropped=1 index row and are counted as dropped_gaps / unrecorded_gaps. - calibration identity: sha256 content hash instead of a path, minted from the calibration the sensor actually loaded (reported before the first frame) and confirmed by the acquisition thread; events are timestamped when they happen. - calibration diagnostics: the fit summary, residual and validation error now come from the model that export/apply would use (piecewise_linear); validation points are held out of the build; repeated reference loads are merged with an n-weighted mean and pooled std. - calibration record: every point keeps its full three-axis sample series (frame id, time, raw, newtons, valid) plus diagnostics, written to an independent calibration_session_*.json that can be reopened for inspection; save record / export coefficients / apply to device are three separate actions. - depth replay: the cursor's frame is bound to the depth result by request sequence and frame id, so a moved cursor never shows another frame's map; the SDK is stepped from its current position instead of re-reading from the start; the live chain (box deformation+contact, finger unroll+tactile) is replayed with the exact resolved config path; algorithm state is rebuilt from a reset marker or from the recorded pre-roll, and says so when it cannot be. Why The review (D:/orisys_ws/_review_phase12/review.md) showed the app writing raw counts into the newton columns, recording a stale value after a failed compute, losing CSV rows when the video queue filled, reporting success while a writer was still running, diagnosing a model nobody applies, applying a calibration before the device confirmed it, dropping the samples behind a calibration point, showing a stale depth map under a moved cursor and recomputing depth of the wrong frame. Verification - D:/orisys_ws/_review_phase12/reproduce.py: all ten acceptance lines flipped (log saved as _review_phase12/reproduce_after.log). Including 25/25 rows kept with 8 video frames shed, frames [2]->[3], depth 8.0/8.0, fit 2/2 points. - apps/studio/tools/ui_check.py: 249 checks, 0 failures (twice). - shared suites: 91 tests, 0 failures (new: test_record_semantics, test_record_lifecycle, test_calibration_model, test_calibration_record, test_depth_replay). - apps/studio/tests/test_ui_smoke.py: 23/30; the 7 remaining are harness artifacts of the sandbox (4 tmp_path PermissionError, 3 parametrized tests the shim cannot drive), unchanged from before this change. - compileall clean; locale parity zh/en + t() key coverage: 0 failures. Not in this change - P2#10 disk-based browsing of a long recording and range CSV export (next round, per your choice). - Stability-triggered calibration sampling, replay fixed colour scale, PyOpenGL three-dimensional view. - Hardware verification: no camera attached, so the SDK's real processing chain and the pre-roll's effect on a live tracker were exercised with fakes only.	2026-09-13 14:55:53 +08:00

Author

SHA1

Message

Date

ykcheungab

fa2703968e

added ruff test

2026-09-27 19:20:43 +08:00

ykcheungab

73592f085f

Browse a long recording from disk instead of loading it into memory

What
- record/index.py: RecordingIndex streams forces.csv once and keeps only the
  small things — one time, one validity flag, one frame id per row plus a byte
  offset every 4096 rows (~6.5 MB per 500k rows). row_range() bisects the times,
  read_rows() seeks to the nearest checkpoint and scans forward, and overview()
  folds every row into its bucket during the same pass to build a per-axis
  min/max envelope.
- read_rows(max_points, unit) decimates **while streaming**: each bucket keeps
  the extreme row of every axis in the requested unit, so a long interval never
  has to exist in memory in full and a shear spike survives a flat normal channel.
- RecordingReader: info() decimates above max_rows (default 200k) and marks
  itself decimated, with total_rows and a duration taken from the frame index
  rather than from the view; window()/window_all()/overview()/statistics() read
  the interval being asked about. list/window semantics unchanged for small
  recordings.
- force/source.py: StoreCurveSource (live memory) and RecordingCurveSource (disk
  interval + cached envelope) implement one interface — extent, window,
  envelope, statistics, duration_s — so ForcePlotsPanel asks a *source* instead
  of knowing where the data lives.
- force_plots: reads through the source (window, envelope, statistics, extent),
  which also replaces its last direct store accesses.
- playback_dialog: draws the whole-session view through window_all() in the unit
  being shown (bounded by CHART_MAX_POINTS), and says so when the view is
  decimated ("显示 N / 共 M 行") instead of letting the operator count points.
- experiment._run: a failure that lands while the files are opening can no longer
  be overwritten by the writer's own transition to "recording" (found by a flaky
  test; the state check now covers STATE_FAILED).

Why
The store releases old blocks above ~4.6 h and the replay loaded every CSV row
into Python lists before drawing anything, so neither the live browser nor the
replay could show an earlier interval of a long recording. The index makes the
disk the data source and memory a cache of what is on screen.

Verification
- shared/tests/test_history_source.py (new, 8 cases) over a 12 000-row recording:
  the index reports its rows and reads a bounded interval; a large recording is
  marked decimated with its true duration and total rows; a small one is not; an
  interval read sees the exact rows including a single peak; a decimated window
  keeps the peak; the envelope fits the whole session in ≤200 points, keeps the
  spike and is cached; the disk and memory sources agree on extent, values and
  statistics; the whole session can be read without holding it all. Shared
  suites: 146 tests, 0 failures (three consecutive runs).
- apps/studio/tools/ui_check.py: the replay section now checks the index row
  count, that an interval read spans only the interval, the envelope size and
  total_rows. 324 checks, 0 failures.
- apps/studio/tests/test_ui_smoke.py: 43 tests, 36 pass; the 7 failures are the
  unchanged sandbox/harness artifacts.
- D:/orisys_ws/_review_phase12/reproduce.py: all ten lines unchanged;
  compileall, locale and prefs parity clean.
- apps/studio/docs/record-channel.md §8 documents the index, the reader API and
  the unified source, and lists what is still hardware work.

Not in this change
- Maximum RSS / backlog / loss measurements at real resolution on the target ARM
  board (needs the hardware), and range CSV export.

2026-09-13 16:54:26 +08:00

ykcheungab

22cc59c402

Fix the phase-1/2 review findings in recording, calibration and depth replay

What
- record: one packet->row conversion (record/sample.py). A failed compute chain
  records no force value at all, a newton value is written only for an axis that
  really has a calibration curve, and mono_t (not the wall clock) orders history
  with a per-row calibration hash.
- record lifecycle: explicit starting/recording/stopping/stopped/failed states
  owned by the writer thread; stop() reports completed=False on timeout and does
  not write ended_at; consecutive write failures stop acceptance immediately.
- record capacity: byte budget for the video queue (512 MiB default); a full
  queue sheds the *frame* and parks the tiny data row, so a video overflow no
  longer deletes CSV rows. Lost rows leave a dropped=1 index row and are counted
  as dropped_gaps / unrecorded_gaps.
- calibration identity: sha256 content hash instead of a path, minted from the
  calibration the sensor actually loaded (reported before the first frame) and
  confirmed by the acquisition thread; events are timestamped when they happen.
- calibration diagnostics: the fit summary, residual and validation error now
  come from the model that export/apply would use (piecewise_linear); validation
  points are held out of the build; repeated reference loads are merged with an
  n-weighted mean and pooled std.
- calibration record: every point keeps its full three-axis sample series
  (frame id, time, raw, newtons, valid) plus diagnostics, written to an
  independent calibration_session_*.json that can be reopened for inspection;
  save record / export coefficients / apply to device are three separate actions.
- depth replay: the cursor's frame is bound to the depth result by request
  sequence and frame id, so a moved cursor never shows another frame's map; the
  SDK is stepped from its current position instead of re-reading from the start;
  the live chain (box deformation+contact, finger unroll+tactile) is replayed
  with the exact resolved config path; algorithm state is rebuilt from a reset
  marker or from the recorded pre-roll, and says so when it cannot be.

Why
The review (D:/orisys_ws/_review_phase12/review.md) showed the app writing raw
counts into the newton columns, recording a stale value after a failed compute,
losing CSV rows when the video queue filled, reporting success while a writer was
still running, diagnosing a model nobody applies, applying a calibration before
the device confirmed it, dropping the samples behind a calibration point, showing
a stale depth map under a moved cursor and recomputing depth of the wrong frame.

Verification
- D:/orisys_ws/_review_phase12/reproduce.py: all ten acceptance lines flipped
  (log saved as _review_phase12/reproduce_after.log). Including 25/25 rows kept
  with 8 video frames shed, frames [2]->[3], depth 8.0/8.0, fit 2/2 points.
- apps/studio/tools/ui_check.py: 249 checks, 0 failures (twice).
- shared suites: 91 tests, 0 failures (new: test_record_semantics,
  test_record_lifecycle, test_calibration_model, test_calibration_record,
  test_depth_replay).
- apps/studio/tests/test_ui_smoke.py: 23/30; the 7 remaining are harness
  artifacts of the sandbox (4 tmp_path PermissionError, 3 parametrized tests the
  shim cannot drive), unchanged from before this change.
- compileall clean; locale parity zh/en + t() key coverage: 0 failures.

Not in this change
- P2#10 disk-based browsing of a long recording and range CSV export (next
  round, per your choice).
- Stability-triggered calibration sampling, replay fixed colour scale, PyOpenGL
  three-dimensional view.
- Hardware verification: no camera attached, so the SDK's real processing chain
  and the pre-roll's effect on a live tracker were exercised with fakes only.

2026-09-13 14:55:53 +08:00

3 Commits