达不到帧率的判据改用实测速率(DSHOW 回读值会说谎),探针加帧率扫描 - What: short_capture_fallacy() 改为以**实测**速率为准(新增 measured_fps 关键字参数),消息改成"it delivers about 15 fps while the config asks for 30 (1920x1080 reported 30, MJPG/DSHOW)"——把驱动自报值降级为"矛盾的一部分"而不是判据;app_window 的 _check_capture_negotiation() 相应改为用 packet 的 capture_fps(实测)判据、实测为 0 时不下结论(等窗口填满)。tools/camera_probe.py 重写:probe() 返回结构体、format_row() 把 reported 与 measured 分列并在达不到 90% 时标 "<-- misses the request";新增 --samples 与**帧率扫描**(同一格式依次申请 60/30/20/15/10 fps,逐行给实测值与"跟随申请 / 只有申请的 x 倍"的判断,最后给"本次看到的ceiling")。tests/test_worker_frame_rate.py 的归因测试同步更新,并显式覆盖"驱动自报 30 而实测 15 必须判为不达标"与"没有实测值时不说话"。 - Why: 用户跑探针的结果推翻了我上一轮的判据:MJPG 1920x1080@30 -> MJPG 1920x1080 @ 30.0 measured 15.0。DSHOW 后端下 CAP_PROP_FPS 只是回读你设的值,所以 SDK 的 status.fps 同样不可信 —— 我原来比较"申请 vs 协商"等于拿申请跟自己比,那个提示永远不会出现(用户实际看到的只有轨道上的实测 16.7)。这也解释了为什么这件事必须靠实测:真实速率只能测出来。另外探针显示 1280x720 MJPG 实测也是 15、YUYV 只有 1 fps,说明换分辨率不解决问题,需要区分"摄像头本身的上限"与"我们申请多少就被砍半"。 - Verification: shared 全量 161 passed(含新增/更新的 3 项断言);studio 91 passed;ui_check 554 项检查 0 失败;探针在本机(无摄像头)仍返回 1 并给出解释,format_row/_decode 单测通过("reported 30.0 measured 15.0 ... <-- misses the request")。 - Not in this change: 未改 SDK 采集申请顺序/后端;未做自动降级;frames.csv 仍未记录 process_ms/capture_fps。

This commit is contained in:
ykcheungab
2026-09-14 13:38:57 +08:00
parent 3db55b2077
commit a44eb2d65d
4 changed files with 154 additions and 54 deletions
+10 -8
View File
@@ -630,12 +630,13 @@ class StudioMainWindow(QMainWindow):
return {} return {}
def _check_capture_negotiation(self) -> None: def _check_capture_negotiation(self) -> None:
"""Say it out loud when the camera cannot do what the config asks. """Say it out loud when the camera delivers less than the config asks.
A camera that only offers a lower rate than the config requests is not The evidence is the *measured* capture rate, not the driver's reported
something the pipeline can fix, and it is not something the operator one: OpenCV's DSHOW backend echoes ``CAP_PROP_FPS`` back as whatever was
should have to infer from a small number on the rail. Said once per set, so a camera advertising 30 and delivering 15 reports "30.0" (measured
session, and only once the driver has confirmed a rate. on this project's own hardware). Said once per session, and only once a
measured rate exists.
""" """
if self._capture_checked: if self._capture_checked:
return return
@@ -649,10 +650,11 @@ class StudioMainWindow(QMainWindow):
logger.exception("Could not read the capture status") logger.exception("Could not read the capture status")
self._capture_checked = True self._capture_checked = True
return return
if not ((status.get("negotiated") or {}).get("fps")): measured = float(self._latest_capture_fps or 0.0)
return # the driver has not reported a rate yet: ask again next frame if measured <= 0.0:
return # no measured rate yet: ask again on the next frame
self._capture_checked = True self._capture_checked = True
detail = short_capture_fallacy(status) detail = short_capture_fallacy(status, measured_fps=measured)
if detail: if detail:
self._notify(t("status.capture_slow", detail=detail)) self._notify(t("status.capture_slow", detail=detail))
+109 -30
View File
@@ -1,24 +1,25 @@
"""Ask each camera what it can actually deliver, without the app in the way. """Ask each camera what it can actually deliver, without the app in the way.
The studio's sensor configs pin the capture at 1920x1080 @ 30 fps (the tile grid The studio's sensor configs pin the capture at 1920x1080 @ 30 fps (the tile grid
depends on it), and nothing downstream can raise a frame rate the driver never is 5x3 tiles of 384x360, so the size is structural), and nothing downstream can
offered. When a session runs at 16.7 fps, the question is which of these it is: raise a frame rate the driver never offered. When a session runs at ~15-17 fps,
there are three questions to separate, and this script answers all three:
* the camera cannot do 1080p30 at all — then the config has to change, and the 1. what does the driver *report* for a given request?
tile geometry with it; 2. what does it actually **deliver**? (Measured from 60 reads. On DSHOW the
* it can, but only in a particular pixel format or backend — then the SDK's reported figure is a liar: ``CAP_PROP_FPS`` echoes back whatever was set, so a
request sequence/format is what to fix; camera that advertises 30 and delivers 15 reports "30.0".)
* it can, and the rate really is ~30 — then the frames are being lost after the 3. does the delivered rate follow the request? The sweep at the end asks for
capture, and the rail's camera rate will say so. 60/30/20/15/10 fps at one format. A rate that *halves* with the request means
the request path is wrong; a rate that sticks at the same number whatever is
asked means that number is the hardware's ceiling.
Run it with the venv interpreter, from anywhere:: Run it with the venv interpreter, with the app closed (two processes cannot own
the same camera)::
python apps/studio/tools/camera_probe.py # every camera, every combo python apps/studio/tools/camera_probe.py # every camera
python apps/studio/tools/camera_probe.py --index 0 # just one python apps/studio/tools/camera_probe.py --index 0 # just one
python apps/studio/tools/camera_probe.py --samples 120 # longer measurement
Each row is what OpenCV/the driver negotiated for that request, plus the rate
measured from 60 grabs (so "negotiated 30" and "actually delivers 30" are two
separate facts).
""" """
from __future__ import annotations from __future__ import annotations
@@ -30,7 +31,7 @@ import time
import cv2 import cv2
#: What the sensor bundles ask for, and the alternatives worth trying when the #: What the sensor bundles ask for, and the alternatives worth trying when the
#: negotiated rate falls short of the requested one. #: delivered rate falls short of the requested one.
REQUESTS: tuple[tuple[str, int, int, float], ...] = ( REQUESTS: tuple[tuple[str, int, int, float], ...] = (
("MJPG", 1920, 1080, 30.0), ("MJPG", 1920, 1080, 30.0),
("MJPG", 1920, 1080, 0.0), # "whatever the driver prefers at this size" ("MJPG", 1920, 1080, 0.0), # "whatever the driver prefers at this size"
@@ -44,7 +45,13 @@ BACKENDS: tuple[tuple[str, int], ...] = (
("MSMF", cv2.CAP_MSMF), ("MSMF", cv2.CAP_MSMF),
) )
#: Frame rates asked for at one fixed format, to see whether the delivered rate
#: follows the request or ignores it.
SWEEP_RATES: tuple[float, ...] = (60.0, 30.0, 20.0, 15.0, 10.0)
SAMPLES = 60 SAMPLES = 60
#: A request is "missed" when the measured rate is below this fraction of it.
TOLERANCE = 0.9
def _fourcc_code(text: str) -> int: def _fourcc_code(text: str) -> int:
@@ -59,7 +66,15 @@ def _decode(code: float) -> str:
return letters if letters.isprintable() else f"0x{value:08x}" return letters if letters.isprintable() else f"0x{value:08x}"
def probe(index: int, backend_name: str, backend: int, request) -> str | None: def probe(
index: int,
backend_name: str,
backend: int,
request: tuple[str, int, int, float],
*,
samples: int = SAMPLES,
) -> dict | None:
"""Open, ask for one format, and measure what actually comes out."""
fourcc, width, height, fps = request fourcc, width, height, fps = request
cap = cv2.VideoCapture(index, backend) cap = cv2.VideoCapture(index, backend)
if not cap.isOpened(): if not cap.isOpened():
@@ -76,47 +91,116 @@ def probe(index: int, backend_name: str, backend: int, request) -> str | None:
cap.grab() cap.grab()
started = time.perf_counter() started = time.perf_counter()
taken = 0 taken = 0
for _ in range(SAMPLES): for _ in range(samples):
ok, frame = cap.read() ok, frame = cap.read()
if not ok or frame is None: if not ok or frame is None:
break break
taken += 1 taken += 1
span = time.perf_counter() - started span = time.perf_counter() - started
rate = taken / span if span > 0 else 0.0
got_w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) got_w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
got_h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) got_h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
got_fps = cap.get(cv2.CAP_PROP_FPS) reported = cap.get(cv2.CAP_PROP_FPS)
got_cc = _decode(cap.get(cv2.CAP_PROP_FOURCC)) got_cc = _decode(cap.get(cv2.CAP_PROP_FOURCC))
finally: finally:
cap.release() cap.release()
return {
"asked": (fourcc, width, height, fps),
"backend": backend_name,
"fourcc": got_cc,
"width": got_w,
"height": got_h,
"reported_fps": reported,
"measured_fps": (taken / span) if span > 0 else 0.0,
"taken": taken,
"samples": samples,
}
def format_row(result: dict) -> str:
fourcc, width, height, fps = result["asked"]
asked = f"{fourcc} {width}x{height}" + (f"@{fps:g}" if fps else "@auto") asked = f"{fourcc} {width}x{height}" + (f"@{fps:g}" if fps else "@auto")
measured = result["measured_fps"]
missed = ""
if fps > 0 and measured < fps * TOLERANCE:
missed = " <-- misses the request"
return ( return (
f" {asked:<20} -> {got_cc} {got_w}x{got_h} @{got_fps:5.1f} " f" {asked:<20} -> {result['fourcc']} {result['width']}x{result['height']} "
f"measured {rate:5.1f} fps ({taken}/{SAMPLES}) [{backend_name}]" f"reported {result['reported_fps']:5.1f} measured {measured:5.1f} fps "
f"({result['taken']}/{result['samples']}) [{result['backend']}]{missed}"
) )
def sweep(index: int, backend_name: str, backend: int, size: tuple[int, int],
*, samples: int = SAMPLES) -> list[dict]:
"""Ask for a range of rates at one format: does the delivery follow?"""
width, height = size
rows = []
for rate in SWEEP_RATES:
result = probe(
index, backend_name, backend, ("MJPG", width, height, rate),
samples=samples,
)
if result is not None:
rows.append(result)
return rows
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--index", type=int, default=None, help="camera index to probe") parser.add_argument("--index", type=int, default=None, help="camera index to probe")
parser.add_argument("--max-index", type=int, default=4, help="highest index to try") parser.add_argument("--max-index", type=int, default=4, help="highest index to try")
parser.add_argument("--samples", type=int, default=SAMPLES, help="frames per row")
args = parser.parse_args(argv) args = parser.parse_args(argv)
indices = [args.index] if args.index is not None else list(range(args.max_index + 1)) indices = [args.index] if args.index is not None else list(range(args.max_index + 1))
found = 0 found = 0
for index in indices: for index in indices:
header = False header = False
first_size: tuple[int, int] | None = None
first_backend: tuple[str, int] | None = None
for backend_name, backend in BACKENDS: for backend_name, backend in BACKENDS:
for request in REQUESTS: for request in REQUESTS:
line = probe(index, backend_name, backend, request) result = probe(index, backend_name, backend, request, samples=args.samples)
if line is None: if result is None:
continue continue
if not header: if not header:
print(f"\ncamera {index}:") print(f"\ncamera {index}:")
header = True header = True
print(line) print(format_row(result))
if first_size is None and request[0] == "MJPG" and request[3] > 0:
first_size = (request[1], request[2])
first_backend = (backend_name, backend)
if header and first_size is not None and first_backend is not None:
print(
f"\n sweep at MJPG {first_size[0]}x{first_size[1]} "
f"[{first_backend[0]}] -- does the delivered rate follow the request?"
)
rows = sweep(index, first_backend[0], first_backend[1], first_size,
samples=args.samples)
for result in rows:
asked = result["asked"][3]
measured = result["measured_fps"]
verdict = ""
if asked > 0 and measured > 0:
ratio = measured / asked
if ratio < 0.35:
verdict = f" -> {ratio:.2f}x the request"
elif ratio > 0.9:
verdict = " -> follows the request"
print(
f" asked {asked:5.1f} fps -> measured {measured:5.1f} fps"
f"{verdict}"
)
rates = [r["measured_fps"] for r in rows if r["measured_fps"] > 0]
if rates:
print(
f" ceiling seen here: {max(rates):.1f} fps "
f"(a ceiling that does not move with the request is the "
f"hardware's, not the request's)"
)
if header: if header:
found += 1 found += 1
if not found: if not found:
print("No camera answered on indices 0..%d." % args.max_index) print("No camera answered on indices 0..%d." % args.max_index)
print( print(
@@ -125,11 +209,6 @@ def main(argv: list[str] | None = None) -> int:
"the same camera." "the same camera."
) )
return 1 return 1
print(
"\nRead it this way: a row whose measured rate tracks its negotiated rate "
"is the truth about that camera; a row that negotiates 30 and measures ~17 "
"is dropping frames, and the request sequence is what to fix."
)
return 0 return 0
@@ -195,24 +195,33 @@ def capture_status_of(sensor) -> dict:
} }
def short_capture_fallacy(status: Mapping[str, Any] | None) -> str: def short_capture_fallacy(
"""A one-line description of a capture negotiation that fell short. status: Mapping[str, Any] | None, *, measured_fps: float = 0.0
) -> str:
"""A one-line description of a capture that falls short of the request.
Empty when the driver gave what was asked for (or when either half is The rate to compare against is the **measured** one. OpenCV's DSHOW backend
unknown): this exists to be *said*, not to fill a field. reports ``CAP_PROP_FPS`` back as whatever was set — a camera that advertises
30 and delivers 15 says "30.0" — so the reported figure cannot be the evidence
(it is included only to show the contradiction).
Empty when nothing measurable fell short, or when either half is unknown: this
exists to be *said*, not to fill a field.
""" """
if not isinstance(status, Mapping): if not isinstance(status, Mapping):
return "" return ""
requested = dict(status.get("requested") or {}) requested = dict(status.get("requested") or {})
negotiated = dict(status.get("negotiated") or {}) negotiated = dict(status.get("negotiated") or {})
wanted = requested.get("fps")
got = negotiated.get("fps")
try: try:
wanted_value = float(wanted) wanted_value = float(requested.get("fps"))
got_value = float(got)
except (TypeError, ValueError): except (TypeError, ValueError):
return "" return ""
if wanted_value <= 0 or got_value <= 0 or got_value >= wanted_value * 0.9: if wanted_value <= 0:
return ""
measured = float(measured_fps or 0.0)
if measured <= 0.0:
return ""
if measured >= wanted_value * 0.9:
return "" return ""
size = "" size = ""
width, height = negotiated.get("width"), negotiated.get("height") width, height = negotiated.get("width"), negotiated.get("height")
@@ -220,11 +229,13 @@ def short_capture_fallacy(status: Mapping[str, Any] | None) -> str:
size = f"{int(width)}x{int(height)} " size = f"{int(width)}x{int(height)} "
fourcc = str(negotiated.get("fourcc") or "").strip() fourcc = str(negotiated.get("fourcc") or "").strip()
backend = str(negotiated.get("backend") or "").strip() backend = str(negotiated.get("backend") or "").strip()
reported = negotiated.get("fps")
detail = "/".join(part for part in (fourcc, backend) if part) detail = "/".join(part for part in (fourcc, backend) if part)
return ( return (
f"the camera negotiated {size}@{got_value:g} fps " f"it delivers about {measured:g} fps while the config asks for "
f"(the config asks for {wanted_value:g})" f"{wanted_value:g}"
+ (f" [{detail}]" if detail else "") + (f" ({size}reported {float(reported):g}" if reported else f" ({size}")
+ (f", {detail})" if detail else ")")
) )
+12 -4
View File
@@ -312,11 +312,19 @@ def test_a_low_camera_rate_is_recorded_as_a_negotiation(monkeypatch):
assert status["negotiated"]["fps"] == 16.67, status assert status["negotiated"]["fps"] == 16.67, status
assert status["negotiated"]["fourcc"] == "MJPG" assert status["negotiated"]["fourcc"] == "MJPG"
assert status["negotiated"]["backend"] == "MSMF" assert status["negotiated"]["backend"] == "MSMF"
detail = short_capture_fallacy(status) # The *measured* rate is the evidence: DSHOW echoes the requested fps back
assert "1920x1080" in detail and "16.67" in detail and "30" in detail, detail # as the "negotiated" one, so trusting that number would hide a camera
# A driver that gives what was asked for says nothing at all. # that advertises 30 and delivers 15 (measured on this project's camera).
detail = short_capture_fallacy(status, measured_fps=16.67)
assert "16.67" in detail and "30" in detail and "1920x1080" in detail, detail
assert short_capture_fallacy(status, measured_fps=29.5) == ""
# A driver reporting 30 while delivering half of it is the case that
# matters, and it must not slip through.
source.status.fps = 30.0 source.status.fps = 30.0
assert short_capture_fallacy(worker.capture_status()) == "" echoing = worker.capture_status()
assert short_capture_fallacy(echoing, measured_fps=15.0) != ""
# No measured rate yet: nothing to say.
assert short_capture_fallacy(echoing) == ""
finally: finally:
worker.stop(wait=True) worker.stop(wait=True)
_restore(saved) _restore(saved)