界面显示每帧耗时与摄像头自身帧率,让低帧率可归因 - What: SdkPacket 新增 capture_fps(采集源自报的发布速率,与 fps 同一窗口)与 capture_dropped(SDK 后台读取线程累计的"发布了但没人取走"的帧数);sdk_worker 的循环记录 (t0, 已发布序号) 并在每帧算出摄像头自身的发布速率,同时读 dropped_frames;新增 _capture_source()/_capture_dropped() 两个取值助手(原 _capture_sequence 里的取值逻辑抽出来复用)。app_window 在 视图▸显示帧率 打开时把这一段一起显示在轨道摘要上:16.7 FPS · 12 ms · 摄像头 30.0 (-13)——速率是我们消费的、耗时是每帧全部开销、摄像头速率是采集源自己发布的;两个速率不一致时后面的 (-N) 就是"摄像头有、我们没接住"的帧数。新增 locale 键 status.rail_camera_fps(中英)。tests/test_worker_frame_rate.py:假设备的采集源现在也像 orisys.capture.source 一样统计"发布时上一帧还没被取走"的丢帧,并支持给 get_img 加一个可配置的每帧耗时(PROCESS_COST_S);新增 test_the_camera_s_own_rate_is_reported_next_to_ours:循环跟得上时摄像头速率≈30、丢帧 0;把流水线放到 60 ms/帧后我们的速率掉到 22 以下、**摄像头速率仍报 30**、丢帧 >5。 - Why: 用户报"上方显示 16.7 fps 而不是实际 30",并且给出了关键数据:16.7 fps 但每帧只耗 12 ms。12 ms 的工作量塞进 60 ms 的周期里,说明循环大部分时间在等——而"等"有两种完全不同的原因:(a) 摄像头/驱动只发布 16.7(USB 带宽、MJPG/YUYV、格式协商),(b) 摄像头发布 30 而我们只取到一半(节拍与轮询问题)。原来的读数只有"我们消费了多少",两种情形长得一模一样,无法判断,所以这不是显示 bug 而是**读数缺一半**。SDK 的 status.last_seq 与 dropped_frames 正好提供另一半(last_seq 每发布一帧就自增,与消费者无关;dropped_frames 在发布时若上一帧仍在缓存里就 +1,即消费者的亏欠),因此把它读出来即可判定。 - Verification: shared 全量 160 passed(新增 2 项,含上面的归因测试);studio 90 passed;ui_check 554 项检查 0 失败;locale_check 287 字面键全部命中、中英键集一致。过程中自查出一个我自己引入的问题:最初把 (t0, 序号) 的记录与按时间裁剪放在**每次轮询**里,而等待期间该分支每秒会进几万次,列表推导把 GIL 抢光,假摄像头的发布速率从 30 掉到 8(test_the_same_frame_is_not_counted_twice 立刻变红)——改为"仅在该序号发生变化时记录",即为每发布帧 O(1) 而非每轮询 O(n)。 - Not in this change: 未改节拍/等待逻辑(假设备上 30 fps 与 60 ms 慢流水线两种情形的消费速率都符合预期,未发现节拍 bug);frames.csv 仍未记录 process_ms/capture_fps(下次改记录格式时一并加)。
This commit is contained in:
@@ -356,6 +356,7 @@
|
|||||||
"samples_dropped": "Record buffer is full; the oldest {count} samples were released (the full session is no longer complete)",
|
"samples_dropped": "Record buffer is full; the oldest {count} samples were released (the full session is no longer complete)",
|
||||||
"run_failed": "Run failed: {error}",
|
"run_failed": "Run failed: {error}",
|
||||||
"filters_saved": "Filter settings saved: {describe} (applies to the next session)",
|
"filters_saved": "Filter settings saved: {describe} (applies to the next session)",
|
||||||
|
"rail_camera_fps": "camera {fps}",
|
||||||
"video_ended": "Video ended",
|
"video_ended": "Video ended",
|
||||||
"camera_disconnected": "Camera disconnected",
|
"camera_disconnected": "Camera disconnected",
|
||||||
"reset_ok": "Tracker reset",
|
"reset_ok": "Tracker reset",
|
||||||
|
|||||||
@@ -356,6 +356,7 @@
|
|||||||
"samples_dropped": "记录缓冲已达上限,最早的 {count} 个样本已被释放(全程记录不再完整)",
|
"samples_dropped": "记录缓冲已达上限,最早的 {count} 个样本已被释放(全程记录不再完整)",
|
||||||
"run_failed": "启动失败: {error}",
|
"run_failed": "启动失败: {error}",
|
||||||
"filters_saved": "滤波设置已保存:{describe}(下次采集生效)",
|
"filters_saved": "滤波设置已保存:{describe}(下次采集生效)",
|
||||||
|
"rail_camera_fps": "摄像头 {fps}",
|
||||||
"video_ended": "视频已结束",
|
"video_ended": "视频已结束",
|
||||||
"camera_disconnected": "摄像机已断开",
|
"camera_disconnected": "摄像机已断开",
|
||||||
"reset_ok": "追踪器已重置",
|
"reset_ok": "追踪器已重置",
|
||||||
|
|||||||
@@ -200,6 +200,10 @@ class StudioMainWindow(QMainWindow):
|
|||||||
#: What the last frame cost to acquire and process, in milliseconds — the
|
#: What the last frame cost to acquire and process, in milliseconds — the
|
||||||
#: other half of the frame-rate reading.
|
#: other half of the frame-rate reading.
|
||||||
self._latest_process_ms: float | None = None
|
self._latest_process_ms: float | None = None
|
||||||
|
#: The capture source's own published rate and shortfall, over the same
|
||||||
|
#: window: together with the above they say who is the slow part.
|
||||||
|
self._latest_capture_fps: float = 0.0
|
||||||
|
self._latest_capture_dropped: int = 0
|
||||||
#: Last coefficients file written by the calibration panel (not applied).
|
#: Last coefficients file written by the calibration panel (not applied).
|
||||||
self._last_calibration_export = ""
|
self._last_calibration_export = ""
|
||||||
self._depth_range_value: tuple[float, float] | None = None
|
self._depth_range_value: tuple[float, float] | None = None
|
||||||
@@ -1365,6 +1369,13 @@ class StudioMainWindow(QMainWindow):
|
|||||||
# cannot keep up, and only this number tells the two apart.
|
# cannot keep up, and only this number tells the two apart.
|
||||||
if self._latest_process_ms:
|
if self._latest_process_ms:
|
||||||
fps = f"{fps} · {self._latest_process_ms:.0f} ms"
|
fps = f"{fps} · {self._latest_process_ms:.0f} ms"
|
||||||
|
# ...and the camera's own rate settles it: a capture rate well above
|
||||||
|
# what we consumed means the frames were there and we did not take
|
||||||
|
# them (the drop counter says how many).
|
||||||
|
if self._latest_capture_fps:
|
||||||
|
fps = f"{fps} · {t('status.rail_camera_fps', fps=f'{self._latest_capture_fps:.1f}')}"
|
||||||
|
if self._latest_capture_dropped:
|
||||||
|
fps = f"{fps} (-{self._latest_capture_dropped})"
|
||||||
detail = f"{detail} · {fps}" if detail else fps
|
detail = f"{detail} · {fps}" if detail else fps
|
||||||
self.toolbar.set_source_summary(detail)
|
self.toolbar.set_source_summary(detail)
|
||||||
# The full path and the replay's notes do not fit on a 30 px rail, so
|
# The full path and the replay's notes do not fit on a 30 px rail, so
|
||||||
@@ -2120,6 +2131,8 @@ class StudioMainWindow(QMainWindow):
|
|||||||
self._last_sample_valid = None
|
self._last_sample_valid = None
|
||||||
self._latest_fps = None
|
self._latest_fps = None
|
||||||
self._latest_process_ms = None
|
self._latest_process_ms = None
|
||||||
|
self._latest_capture_fps = 0.0
|
||||||
|
self._latest_capture_dropped = 0
|
||||||
self._depth_range_value = None
|
self._depth_range_value = None
|
||||||
self._last_depth_frame = None
|
self._last_depth_frame = None
|
||||||
# The new session's store is cleared *before* the charts are attached to
|
# The new session's store is cleared *before* the charts are attached to
|
||||||
@@ -2207,6 +2220,8 @@ class StudioMainWindow(QMainWindow):
|
|||||||
self.arrow_panel.clear_preview(t("panels.arrow_empty"))
|
self.arrow_panel.clear_preview(t("panels.arrow_empty"))
|
||||||
self._latest_fps = None
|
self._latest_fps = None
|
||||||
self._latest_process_ms = None
|
self._latest_process_ms = None
|
||||||
|
self._latest_capture_fps = 0.0
|
||||||
|
self._latest_capture_dropped = 0
|
||||||
self._last_sample_valid = None
|
self._last_sample_valid = None
|
||||||
self._last_display_fallback = None
|
self._last_display_fallback = None
|
||||||
self._last_display_available = None
|
self._last_display_available = None
|
||||||
@@ -2376,6 +2391,10 @@ class StudioMainWindow(QMainWindow):
|
|||||||
if is_new:
|
if is_new:
|
||||||
self._latest_fps = getattr(packet, "fps", None)
|
self._latest_fps = getattr(packet, "fps", None)
|
||||||
self._latest_process_ms = getattr(packet, "process_ms", None)
|
self._latest_process_ms = getattr(packet, "process_ms", None)
|
||||||
|
self._latest_capture_fps = float(getattr(packet, "capture_fps", 0.0) or 0.0)
|
||||||
|
self._latest_capture_dropped = int(
|
||||||
|
getattr(packet, "capture_dropped", 0) or 0
|
||||||
|
)
|
||||||
# The rail carries the live telemetry (fps, validity, calibration) and
|
# The rail carries the live telemetry (fps, validity, calibration) and
|
||||||
# the record strip its own numbers. Both compare before repainting, so
|
# the record strip its own numbers. Both compare before repainting, so
|
||||||
# this is safe on a 16 ms timer.
|
# this is safe on a 16 ms timer.
|
||||||
|
|||||||
@@ -41,6 +41,12 @@ class SdkPacket:
|
|||||||
timestamp: float
|
timestamp: float
|
||||||
fps: float
|
fps: float
|
||||||
process_ms: float
|
process_ms: float
|
||||||
|
#: The rate the *capture source* published frames at, over the same window as
|
||||||
|
#: ``fps``. ``fps`` is what this loop consumed; when the two disagree, the
|
||||||
|
#: difference is frames the camera had and the loop did not take.
|
||||||
|
capture_fps: float = 0.0
|
||||||
|
#: How many published frames were replaced before anyone consumed them.
|
||||||
|
capture_dropped: int = 0
|
||||||
normal_force: float = 0.0
|
normal_force: float = 0.0
|
||||||
shear_x: float = 0.0
|
shear_x: float = 0.0
|
||||||
shear_y: float = 0.0
|
shear_y: float = 0.0
|
||||||
|
|||||||
@@ -123,9 +123,7 @@ def _capture_sequence(sensor) -> Optional[int]:
|
|||||||
every call decodes the next frame, so its counter says nothing about repeats
|
every call decodes the next frame, so its counter says nothing about repeats
|
||||||
and is ignored (``None``).
|
and is ignored (``None``).
|
||||||
"""
|
"""
|
||||||
source = getattr(sensor, "capture_source", None)
|
source = _capture_source(sensor)
|
||||||
if source is None:
|
|
||||||
source = getattr(sensor, "_capture_source", None)
|
|
||||||
if source is None or not bool(getattr(source, "running", False)):
|
if source is None or not bool(getattr(source, "running", False)):
|
||||||
return None
|
return None
|
||||||
status = getattr(source, "status", None)
|
status = getattr(source, "status", None)
|
||||||
@@ -135,6 +133,28 @@ def _capture_sequence(sensor) -> Optional[int]:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _capture_dropped(sensor) -> Optional[int]:
|
||||||
|
"""Frames the capture source published that nobody took in time.
|
||||||
|
|
||||||
|
The SDK's background reader counts one of these every time it replaces a
|
||||||
|
frame that was never consumed, so this is the consumer's shortfall — the
|
||||||
|
number that says "the camera had it and we did not take it".
|
||||||
|
"""
|
||||||
|
source = _capture_source(sensor)
|
||||||
|
status = getattr(source, "status", None) if source is not None else None
|
||||||
|
try:
|
||||||
|
return int(getattr(status, "dropped_frames"))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _capture_source(sensor):
|
||||||
|
source = getattr(sensor, "capture_source", None)
|
||||||
|
if source is None:
|
||||||
|
source = getattr(sensor, "_capture_source", None)
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
def process_chain(sensor) -> tuple[Optional[np.ndarray], str]:
|
def process_chain(sensor) -> tuple[Optional[np.ndarray], str]:
|
||||||
"""Run the live processing chain for one already-read frame.
|
"""Run the live processing chain for one already-read frame.
|
||||||
|
|
||||||
@@ -714,6 +734,13 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
#: When that frame arrived, and how far apart this device's frames are
|
#: When that frame arrived, and how far apart this device's frames are
|
||||||
#: meant to be: the wait below never steps over the next frame's arrival.
|
#: meant to be: the wait below never steps over the next frame's arrival.
|
||||||
last_frame_at = 0.0
|
last_frame_at = 0.0
|
||||||
|
#: ``(collection time, published sequence)`` over the same window as
|
||||||
|
#: ``frame_times``. The sequence advances whenever the *capture source*
|
||||||
|
#: publishes, whether or not this loop takes the frame, so comparing the
|
||||||
|
#: two rates says whether a low frame rate is the camera's or ours.
|
||||||
|
source_times: list[tuple[float, int]] = []
|
||||||
|
source_dropped = 0
|
||||||
|
last_source_seq: Optional[int] = None
|
||||||
frame_period = 1.0 / max(1.0, float(mode.fps or 30.0))
|
frame_period = 1.0 / max(1.0, float(mode.fps or 30.0))
|
||||||
# Per-session monotonic frame index: the join key shared by the video,
|
# Per-session monotonic frame index: the join key shared by the video,
|
||||||
# the CSV and the calibration records.
|
# the CSV and the calibration records.
|
||||||
@@ -748,6 +775,17 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
# without a background reader (a video file) decodes a new frame
|
# without a background reader (a video file) decodes a new frame
|
||||||
# per call, so it has no repeat marker and is left alone.
|
# per call, so it has no repeat marker and is left alone.
|
||||||
delivered = _capture_sequence(sensor)
|
delivered = _capture_sequence(sensor)
|
||||||
|
if delivered is not None and delivered != last_source_seq:
|
||||||
|
# Bookkeeping per *published* frame, not per poll: this branch
|
||||||
|
# is entered many times while waiting, and a list comprehension
|
||||||
|
# on every one of them starves the capture thread of the GIL
|
||||||
|
# (measured: the fake camera's own 30 fps dropped to 8).
|
||||||
|
last_source_seq = delivered
|
||||||
|
source_times.append((t0, delivered))
|
||||||
|
source_cutoff = t0 - FPS_WINDOW_S
|
||||||
|
source_times = [
|
||||||
|
item for item in source_times if item[0] >= source_cutoff
|
||||||
|
]
|
||||||
if delivered is not None and delivered == last_delivered:
|
if delivered is not None and delivered == last_delivered:
|
||||||
# Wait for the next frame, but never *past* the moment it is
|
# Wait for the next frame, but never *past* the moment it is
|
||||||
# due: sleeping a whole timer tick before looking again is how
|
# due: sleeping a whole timer tick before looking again is how
|
||||||
@@ -820,6 +858,17 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
# number that says whether the loop can still keep up with the
|
# number that says whether the loop can still keep up with the
|
||||||
# device, which the frame rate alone cannot.
|
# device, which the frame rate alone cannot.
|
||||||
t_done = time.perf_counter()
|
t_done = time.perf_counter()
|
||||||
|
# What the capture source published over the window, so a rate
|
||||||
|
# below the camera's own can be attributed instead of guessed.
|
||||||
|
capture_fps = 0.0
|
||||||
|
if len(source_times) > 1:
|
||||||
|
source_span = source_times[-1][0] - source_times[0][0]
|
||||||
|
published = source_times[-1][1] - source_times[0][1]
|
||||||
|
if source_span > 0.0:
|
||||||
|
capture_fps = max(0.0, published / source_span)
|
||||||
|
taken = _capture_dropped(sensor)
|
||||||
|
if taken is not None:
|
||||||
|
source_dropped = taken
|
||||||
invalid_reason = forces.get("invalid_reason") or compute_error
|
invalid_reason = forces.get("invalid_reason") or compute_error
|
||||||
raw_frame = _ensure_bgr(raw)
|
raw_frame = _ensure_bgr(raw)
|
||||||
# The frame the chain above actually consumed, kept as the
|
# The frame the chain above actually consumed, kept as the
|
||||||
@@ -842,6 +891,8 @@ class SdkAcquisitionWorker(RecordSinkMixin):
|
|||||||
),
|
),
|
||||||
fps=fps,
|
fps=fps,
|
||||||
process_ms=(t_done - t0) * 1000.0,
|
process_ms=(t_done - t0) * 1000.0,
|
||||||
|
capture_fps=capture_fps,
|
||||||
|
capture_dropped=source_dropped,
|
||||||
normal_force=forces["normal_force"],
|
normal_force=forces["normal_force"],
|
||||||
shear_x=forces["shear_x"],
|
shear_x=forces["shear_x"],
|
||||||
shear_y=forces["shear_y"],
|
shear_y=forces["shear_y"],
|
||||||
|
|||||||
@@ -38,7 +38,9 @@ class _CaptureSource:
|
|||||||
|
|
||||||
The real one is a thread too: the sequence advances whether or not the
|
The real one is a thread too: the sequence advances whether or not the
|
||||||
consumer asks for a frame, which is exactly why an unchanged sequence means
|
consumer asks for a frame, which is exactly why an unchanged sequence means
|
||||||
"no new frame" rather than "the consumer has not called yet".
|
"no new frame" rather than "the consumer has not called yet". It also counts
|
||||||
|
a drop whenever it replaces a frame nobody took, like
|
||||||
|
``orisys.capture.source.CaptureSource._background_loop`` does.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, period_s: float = FRAME_PERIOD_S) -> None:
|
def __init__(self, period_s: float = FRAME_PERIOD_S) -> None:
|
||||||
@@ -46,6 +48,7 @@ class _CaptureSource:
|
|||||||
self.running = True
|
self.running = True
|
||||||
self._period = float(period_s)
|
self._period = float(period_s)
|
||||||
self._stop = threading.Event()
|
self._stop = threading.Event()
|
||||||
|
self._taken_seq = 0
|
||||||
self._thread = threading.Thread(target=self._publish, daemon=True)
|
self._thread = threading.Thread(target=self._publish, daemon=True)
|
||||||
self._thread.start()
|
self._thread.start()
|
||||||
|
|
||||||
@@ -53,14 +56,25 @@ class _CaptureSource:
|
|||||||
while not self._stop.is_set():
|
while not self._stop.is_set():
|
||||||
if self._stop.wait(timeout=self._period):
|
if self._stop.wait(timeout=self._period):
|
||||||
return
|
return
|
||||||
|
# Replacing a frame that was never consumed is a drop on our side.
|
||||||
|
if self.status.last_seq > self._taken_seq:
|
||||||
|
self.status.dropped_frames += 1
|
||||||
self.status.last_seq += 1
|
self.status.last_seq += 1
|
||||||
|
|
||||||
|
def take(self) -> None:
|
||||||
|
self._taken_seq = self.status.last_seq
|
||||||
|
|
||||||
def close(self) -> None:
|
def close(self) -> None:
|
||||||
self.running = False
|
self.running = False
|
||||||
self._stop.set()
|
self._stop.set()
|
||||||
self._thread.join(timeout=1.0)
|
self._thread.join(timeout=1.0)
|
||||||
|
|
||||||
|
|
||||||
|
#: Per-frame processing cost the fake device charges, so a test can make the
|
||||||
|
#: pipeline slower than the camera (the real box chain costs ~10 ms at 480p).
|
||||||
|
PROCESS_COST_S = 0.0
|
||||||
|
|
||||||
|
|
||||||
class _Info:
|
class _Info:
|
||||||
"""The channel names the worker reads off ``sensor.info``."""
|
"""The channel names the worker reads off ``sensor.info``."""
|
||||||
|
|
||||||
@@ -105,7 +119,11 @@ class _Sensor:
|
|||||||
|
|
||||||
def get_img(self):
|
def get_img(self):
|
||||||
# ``read_latest``: hand back the newest frame immediately, the same one
|
# ``read_latest``: hand back the newest frame immediately, the same one
|
||||||
# again when nothing new has arrived.
|
# again when nothing new has arrived. A test can make this cost time, so
|
||||||
|
# the loop can be put under a pipeline slower than the camera.
|
||||||
|
if PROCESS_COST_S > 0:
|
||||||
|
time.sleep(PROCESS_COST_S)
|
||||||
|
self.capture_source.take()
|
||||||
return self.frame
|
return self.frame
|
||||||
|
|
||||||
def disconnect(self):
|
def disconnect(self):
|
||||||
@@ -211,6 +229,42 @@ def test_the_reported_rate_is_the_delivered_rate():
|
|||||||
assert abs(settled[-1] - measured) < 0.25 * measured, (settled[-1], measured)
|
assert abs(settled[-1] - measured) < 0.25 * measured, (settled[-1], measured)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_camera_s_own_rate_is_reported_next_to_ours(monkeypatch):
|
||||||
|
"""A low frame rate must be attributable, not just visible.
|
||||||
|
|
||||||
|
With a pipeline that fits the frame period the two rates agree. With one that
|
||||||
|
does not, the *camera's* rate stays at 30 while ours falls and the source
|
||||||
|
counts the frames nobody took — which is the whole difference between "the
|
||||||
|
camera is slow" and "we are slow".
|
||||||
|
"""
|
||||||
|
packets, _ = _collect(duration_s=1.5)
|
||||||
|
assert packets, "the loop emitted nothing at all"
|
||||||
|
camera = [float(packet.capture_fps) for packet in packets if packet.capture_fps]
|
||||||
|
assert camera, "no packet carried the capture source's own rate"
|
||||||
|
assert 28.0 <= camera[-1] <= 31.0, f"the fake camera publishes at 30 fps, got {camera[-1]}"
|
||||||
|
assert packets[-1].capture_dropped == 0, (
|
||||||
|
f"the loop dropped {packets[-1].capture_dropped} frames while keeping up"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Now make the pipeline three times slower than the camera's own period.
|
||||||
|
monkeypatch.setattr(sys.modules[__name__], "PROCESS_COST_S", 0.06)
|
||||||
|
slow, _ = _collect(duration_s=2.0)
|
||||||
|
try:
|
||||||
|
assert slow, "the loop emitted nothing at all"
|
||||||
|
ours = _rate(slow)
|
||||||
|
assert ours < 22.0, f"a 60 ms pipeline still kept up at {ours:.1f} fps"
|
||||||
|
assert 28.0 <= float(slow[-1].capture_fps) <= 31.0, (
|
||||||
|
"the camera's rate was reported as slow too: "
|
||||||
|
f"{slow[-1].capture_fps}"
|
||||||
|
)
|
||||||
|
assert slow[-1].capture_dropped > 5, (
|
||||||
|
"the loop fell behind without recording a single dropped frame: "
|
||||||
|
f"{slow[-1].capture_dropped}"
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
monkeypatch.setattr(sys.modules[__name__], "PROCESS_COST_S", 0.0)
|
||||||
|
|
||||||
|
|
||||||
def test_a_file_source_runs_on_the_video_s_own_timeline():
|
def test_a_file_source_runs_on_the_video_s_own_timeline():
|
||||||
"""A pass over a video is stamped with the video's time, not the wall clock."""
|
"""A pass over a video is stamped with the video's time, not the wall clock."""
|
||||||
from orisys_app_shared.acquisition.sdk_worker import SdkAcquisitionWorker
|
from orisys_app_shared.acquisition.sdk_worker import SdkAcquisitionWorker
|
||||||
|
|||||||
Reference in New Issue
Block a user