回放与实时统一为一套界面；修复 30 fps 设备被以 16.7 fps 消费的计时器缺陷 - What: 1) 删除 ui/replay_bar.py（进度条、录制目录选择、重算深度按钮、提示行）与回放条的全部接线：历史回放现在与实时监测同布局（分割比例同为 0,360,1080,0）、同一条状态轨、同一套曲线手势，唯一差别是来源槽——实时是摄像机，回放是「选择录制视频…」（toolbar.btn_replay_video，与文件菜单「打开视频…」同一条 _load_replay_video 路径：视频所在目录即录制目录，曲线自动配对同目录 forces.csv）。时间轴改为曲线游标本身：表头新增播放/暂停图标按钮（icons.play/pause，空格键同效）以 1× 推进（按墙钟计时，漏拍不变慢），到末尾自动停；拖动游标、框选、滚轮平移或拖总览条一律自动暂停（ForcePlotsPanel.pause_playback）。回放的降采样/跨版本提示移入来源摘要 tooltip，打开录制时另发一条 6 秒通知。修复视频文件名残留在实时视图的问题：_refresh_status_rail 只在 _view_preset == "replay" 时写视频名。2) 录制格式不变：raw_*.mp4 与 forces.csv/frames.csv/session.json 继续一起写。按「两者都要」新增视图 → 由视频重算力值（act_recompute）：默认曲线读录制的 CSV，打开后把录制视频重新跑一遍视觉管线（进 SessionStore、2D 面板交给管线绘制、只能顺序前进，播放按钮即开始/暂停，关闭即挂回 CSV），启动路径唯一（set_playing → _on_chart_play_toggled → _start_run），并让文件源的时间轴改为视频自身时间（mono_t = 帧号 × 1/fps），否则重算结果会落在与 CSV 不同的横轴上。回放 2D 面板不再提供深度视图（REPLAY_VIEW_MODES=("recorded",)），离线深度重算引擎保留但无界面入口。3) 采集只有 16.67 fps 的根因是上一轮引入的等待：Windows 默认计时器分辨率 15.6 ms，time.sleep(0.002) 实测睡 15.5 ms，重复帧等两拍即跨过 30 fps 的 33 ms 周期，30 fps 设备被以 ~16.7 fps 消费并隔帧丢弃（配置本身正确：mini/sensor.json 写 camera.fps=30；离线实测整套管线 15.2 ms/帧≈66 fps，余量充足）。新增 _high_resolution_timer() 在会话期间把分辨率提到 1 ms（结束还原），并把重复帧等待改为 min(距下一帧时间, 半个周期)、到期只 sleep(0)。 - Why: 用户评审 3 点——回放界面与实时不一致且拖拽条效果一般、不想为回放录制深度信息、想同时保留视频与 CSV 记录、以及采集帧率只有 16.67 fps（应为 30）。 - Verification: ui_check.py 562 项 0 失败（新增两视图同布局/无 bar、来源槽随视图切换且无残留、播放暂停与手动手势暂停、重算模式切换与视频源、回放提示在 tooltip）；test_ui_smoke.py 73 项 0 失败（新增视图统一、传输控制、重算模式）；drive_shared.py 150 项 0 失败（新增 shared/tests/test_worker_frame_rate.py::test_a_file_source_runs_on_the_video_s_own_timeline，并把 30 fps 用例改成必须落在 27–31.5 fps；同文件实测无 1 ms 计时器 21.6 fps、有则 30.0 fps，同一帧返回 3 次时 11317:39）；locale_studio.py 266 键齐全、zh==en；reproduce.py 退出 0；ui_snapshot.py 三张复核图（统一后的回放、实时、选区中英）。 - Not in this change: 录制文件格式与 CSV 列序不变；离线深度重算引擎（ReplayController.request_depth / DepthRecomputer）保留并有独立测试，但界面已无入口，可下一步删除；重算模式用有损 mp4，数值与原 CSV 会有差异（这正是它要显式呈现的）。

2026-09-13 22:04:01 +08:00
parent e6a1e77a5b
commit 9b3c5fe46c
17 changed files with 896 additions and 387 deletions
@@ -2,8 +2,11 @@

 from __future__ import annotations

+import contextlib
+import ctypes
 import json
 import logging
+import sys
 import threading
 import time
 from dataclasses import dataclass
@@ -39,9 +42,43 @@ HANDOVER_TIMEOUT_S = 60.0
 #: Window over which the delivered frame rate is measured.
 FPS_WINDOW_S = 1.0
 #: How long to wait before asking again when the capture source has not produced
-#: a new frame yet. Long enough not to spin, short enough not to add latency.
+#: a new frame yet. Long enough not to spin, short enough not to add latency —
+#: but only if the platform can actually sleep this finely: see
+#: ``_high_resolution_timer``.
 CAPTURE_REPEAT_POLL_S = 0.002

+
+@contextlib.contextmanager
+def _high_resolution_timer():
+    """Ask Windows for a 1 ms timer for the duration of the block.
+
+    The default Windows timer resolution is 15.6 ms, so ``time.sleep(0.002)``
+    sleeps 15.6 ms. The acquisition loop waits for the *next* frame between
+    frames, and two of those ticks overshoot a 33 ms frame period: a 30 fps
+    sensor is then consumed at ~16.7 fps with every other frame dropped. Raising
+    the resolution to 1 ms makes the wait behave as written. It is restored on
+    exit, so idle power behaviour is unchanged, and on other platforms this does
+    nothing.
+    """
+    if sys.platform != "win32":  # pragma: no cover - platform specific
+        yield
+        return
+    winmm = None
+    try:
+        winmm = ctypes.WinDLL("winmm")
+        winmm.timeBeginPeriod(1)
+    except Exception:  # noqa: BLE001 - the loop still works, only coarser
+        logger.debug("Could not raise the timer resolution", exc_info=True)
+        winmm = None
+    try:
+        yield
+    finally:
+        if winmm is not None:
+            try:
+                winmm.timeEndPeriod(1)
+            except Exception:  # noqa: BLE001
+                pass
+
 # Re-exported so ``from ...sdk_worker import SdkPacket`` keeps working; the
 # definitions live in ``models`` so neither worker depends on the other.
 __all__ = [
@@ -557,7 +594,11 @@ class SdkAcquisitionWorker(RecordSinkMixin):

    def _run(self, session: int, previous: Optional[threading.Thread] = None) -> None:
        try:
-            self._run_session(session, previous)
+            # A fine timer for the whole session: the loop waits for the next
+            # frame between frames, and on Windows a sub-tick wait otherwise
+            # costs a full 15.6 ms (see ``_high_resolution_timer``).
+            with _high_resolution_timer():
+                self._run_session(session, previous)
        finally:
            # The finishing thread owns the teardown — this is what lets
            # ``stop()`` return immediately without leaking the device.
@@ -626,6 +667,10 @@ class SdkAcquisitionWorker(RecordSinkMixin):
        #: The capture sequence of the last frame that was actually *processed*:
        #: an unchanged counter means the same frame came back again.
        last_delivered: Optional[int] = None
+        #: When that frame arrived, and how far apart this device's frames are
+        #: meant to be: the wait below never steps over the next frame's arrival.
+        last_frame_at = 0.0
+        frame_period = 1.0 / max(1.0, float(mode.fps or 30.0))
        # Per-session monotonic frame index: the join key shared by the video,
        # the CSV and the calibration records.
        frame_id = 0
@@ -655,7 +700,15 @@ class SdkAcquisitionWorker(RecordSinkMixin):
                # per call, so it has no repeat marker and is left alone.
                delivered = _capture_sequence(sensor)
                if delivered is not None and delivered == last_delivered:
-                    time.sleep(CAPTURE_REPEAT_POLL_S)
+                    # Wait for the next frame, but never *past* the moment it is
+                    # due: sleeping a whole timer tick before looking again is how
+                    # a 30 fps sensor gets consumed at 16.7 fps. When the frame is
+                    # already due, yield and look again instead.
+                    remaining = frame_period - (now - last_frame_at)
+                    if remaining > CAPTURE_REPEAT_POLL_S:
+                        time.sleep(min(remaining, frame_period * 0.5))
+                    else:
+                        time.sleep(0)
                    continue
                last_delivered = delivered
                processed = sensor.get_img()
@@ -721,7 +774,15 @@ class SdkAcquisitionWorker(RecordSinkMixin):
                    # Wall clock for metadata; monotonic for ordering.
                    timestamp=time.time(),
                    wall_t=time.time(),
-                    mono_t=max(0.0, t1 - self._mono_t0),
+                    # A file has the timeline of the *video*, not of how long the
+                    # pass took: recomputing a 5 s recording at 60 fps would
+                    # otherwise land on a 0–2 s axis and could not be compared
+                    # with the forces.csv it is being checked against.
+                    mono_t=(
+                        frame_id * frame_period
+                        if is_file
+                        else max(0.0, t1 - self._mono_t0)
+                    ),
                    fps=fps,
                    process_ms=(t1 - t0) * 1000.0,
                    normal_force=forces["normal_force"],
@@ -753,6 +814,7 @@ class SdkAcquisitionWorker(RecordSinkMixin):
                frame_id += 1
                saw_frame = True
                last_frame_mono = t1
+                last_frame_at = t1
            except Exception as exc:
                self._set_end_reason("error")
                if self._on_error is not None and session == self._session_id:
@@ -18,8 +18,9 @@ from pathlib import Path
 APP_ROOT = Path(__file__).resolve().parents[2]
 BOX_CONFIG = APP_ROOT / "apps" / "studio" / "config" / "sensors" / "box" / "sensor.json"

-#: The fake device's own frame period: ~100 frames per second.
-FRAME_PERIOD_S = 0.01
+#: The fake device's own frame period: 30 frames per second, the rate the studio
+#: sensor configs declare.
+FRAME_PERIOD_S = 1.0 / 30.0
 #: How many times the SDK hands the same frame back before the next one arrives.
 REPEATS_PER_FRAME = 3

@@ -153,7 +154,9 @@ def _collect(duration_s: float = 1.2):

    try:
        worker.add_sink(sink)
-        worker.start("0", config_ref=BOX_CONFIG)
+        # Camera id 0, not "0": a string source is a file, and a file runs on the
+        # video's timeline rather than the wall clock (see the file test below).
+        worker.start(0, config_ref=BOX_CONFIG)
        time.sleep(duration_s)
        worker.stop(wait=True)
        delivered = CREATED[-1].capture_source.status.last_seq if CREATED else 0
@@ -181,6 +184,22 @@ def test_the_same_frame_is_not_counted_twice():
    )


+def test_a_30_fps_device_is_consumed_at_30_fps():
+    """The loop must not add a timer tick of latency per frame.
+
+    ``time.sleep(0.002)`` costs a full 15.6 ms on Windows, so a loop that sleeps
+    before looking for the next frame spends ~47 ms per frame on a 30 fps device
+    and records 16.7 fps with half the frames dropped.
+    """
+    packets, delivered = _collect(duration_s=1.5)
+    assert packets, "the loop emitted nothing at all"
+    rate = _rate(packets)
+    assert 27.0 <= rate <= 31.5, (
+        f"a 30 fps device was consumed at {rate:.1f} fps "
+        f"({len(packets)} packets for {delivered} delivered frames)"
+    )
+
+
 def test_the_reported_rate_is_the_delivered_rate():
    """``packet.fps`` describes the frames, not how often the loop asked."""
    packets, delivered = _collect()
@@ -192,6 +211,40 @@ def test_the_reported_rate_is_the_delivered_rate():
    assert abs(settled[-1] - measured) < 0.25 * measured, (settled[-1], measured)


+def test_a_file_source_runs_on_the_video_s_own_timeline():
+    """A pass over a video is stamped with the video's time, not the wall clock."""
+    from orisys_app_shared.acquisition.sdk_worker import SdkAcquisitionWorker
+
+    CREATED.clear()
+    saved = _install()
+    worker = SdkAcquisitionWorker()
+    packets: list = []
+    lock = threading.Lock()
+
+    def sink(packet) -> None:
+        with lock:
+            packets.append(packet)
+
+    try:
+        worker.add_sink(sink)
+        worker.start("recorded.mp4", config_ref=BOX_CONFIG)
+        time.sleep(0.4)
+        worker.stop(wait=True)
+    finally:
+        _restore(saved)
+    assert len(packets) > 3, f"the pass produced {len(packets)} packets"
+    steps = [
+        round(packets[index + 1].mono_t - packets[index].mono_t, 6)
+        for index in range(min(5, len(packets) - 1))
+    ]
+    assert packets[0].mono_t == 0.0, packets[0].mono_t
+    assert all(abs(step - 1.0 / 30.0) < 1e-6 for step in steps), steps
+    assert packets[-1].mono_t < 0.4 + 1.0, (
+        f"the pass ran on the wall clock: {packets[-1].mono_t} s of video in "
+        f"{0.4} s of real time"
+    )
+
+
 if __name__ == "__main__":  # pragma: no cover - manual runner
    failures = 0
    for name, function in sorted(globals().items()):