refactor(studio-2d): 二维图像单一后端 —— 视图模式由帧内容决定,四条路径共用一个渲染入口
- What:
1. 新增 `shared/.../acquisition/image_backend.py`(纯 NumPy,无 Qt):`ImageFrame`(底图 + flow/depth/
valid_mask + source/frame_id/t_rel_s/stale)、`ImagePlan`、`plan_image()`、`render_image()`,以及模式
常量(`arrows_pure`/`arrows_stitched`/`depth`/`recorded`)与来源常量(live/recompute/replay/preview)。
`plan_image()` 分两个问题回答:**可选项**由来源决定(live/recompute = 三种派生视图;录像/摄像头预览
= 只有 `recorded`),**这一帧能不能画**由 `missing` 说明(某帧缺 flow/depth 不会悄悄改掉操作者选的
视图,而是在页脚写一行)。`render_image()` 是唯一渲染入口:内部复用 `draw_arrow_display`/
`depth_map_to_bgr`/`flow_colormap`(公共签名不变),深度图**改用主题色带**(与面板色条同一张表——
原来窗口没传 colormap,深度图走 OpenCV JET 而色条画品牌色带,图例与图片两张表)。
2. `ui/arrow_panel.py`:新增 `bgr_to_qimage()`(全应用唯一一份 BGR→QImage,`app_window._bgr_to_qimage`
成为它的别名、`replay.py` 的重复实现删除);新增 `set_plan(plan)`/`plan_modes()`——模式集来自 plan,
`view_modes()` 在无 plan 时回退旧的预设式列表(`set_source_kind` 保留"命名/文案"职责,并支持
`recompute` 状态与 `panels.recompute_pipeline`);页脚新增一行 `note_label` 说明 plan(如
"录像只保存原始相机图像…"),箭头放大/有效区注解改由"本帧是否支持箭头视图"决定;
`set_view_mode()` 与 `_populate_view_combo()` 按当前模式集校验(不再只认三个常量,否则 `recorded`
会被强行改回 `arrows_pure`)。
3. `app_window.py`:新增唯一入口 `_show_image_frame(ImageFrame)`(plan → 渲染 → 一个 QImage 给面板),
**四条生产者全部改走它**:实时/重算(`_render_run_image`,来源按 `_recompute_running` 区分)、回放
(`_render_replay_frame` 用 `replay.video_frame_bgr()`,不再自己转 QImage)、摄像头预览
(`_on_preview_tick`)。`_render_run_image` 里那套按 `mode` 分支画箭头的代码整体删除。
4. `replay.py`:新增 `video_frame_bgr()`(按光标缓存 BGR),`video_image()` 保留为兼容薄封装;删除
本模块的 `_bgr_to_qimage`。
5. locale:新增 `panels.replay_raw_note`(录像只有原始图像的说明 + 指路「由视频重算」)、
`panels.frame_missing_note`(`{fields}` 参数化)、`panels.field_flow`/`panels.field_depth`、
`panels.recompute_pipeline(_tip)`(中英各一套)。
- Why: 用户报的两个 bug 之一——「历史回放面板的 2 维图只能显示原始采集图像,没有进行任何处理」,并建议
"每一个 2D 图的后端保持一致"。核对后确认根因是架构而非漏了一步渲染:二维有三个(实际四个)互不一致的
生产者——实时/重算走 `draw_arrow_display`/`depth_map_to_bgr`,回放与预览把 BGR 直接交给面板;模式集与
注解由**视图预设**决定(`arrow_panel.set_source_kind("replay")` 才切成 `recorded`),于是同一段视频在
"回放"和"重算"下是两个成像栈,且 `_bgr_to_qimage` 有两份实现。用户已选定"统一后端 + 一键重算(不改记录
格式)",本提交做前半段;"一键重算"的入口在下一提交。
- Verification:
· 新增 `shared/tests/test_image_backend.py` 15 项:plan 的三类来源(live/recompute 三种派生视图、
replay/preview 只有 recorded)、`missing`/`note_key`/`note_fields`、模式回退、**与旧渲染逐像素一致**
(arrows_pure/arrows_stitched 对 `draw_arrow_display`、depth 对 `depth_map_to_bgr` 带主题色带)、
原始帧不做缩放/不替图、缺 depth → None、缺 flow → 底图、灰度帧只在这里加宽一次。
· `apps/studio/tests/test_ui_smoke.py` +2:同一段视频在回放路径只出 `recorded` 且有说明行、在重算/实时
路径出三种派生视图且都经过同一个 `render_image`(用 spy 断言来源与模式)、面板 `note_label` 不是裸键、
combo 项数与模式集一致;以及"全应用只有一处 BGR→QImage"的结构断言。
· 既有 `test_depth_view_reports_missing_data_and_fixes_its_scale`(缺深度必须报"本帧没有深度数据"而不是
回退显示普通形变图)继续通过——新的"缺字段说明 + 不回退替图"语义与它一致。
· 全量:apps/studio 236 项 0 失败 14 跳过;shared 331 项 0 失败 1 跳过;ui_check 587 项 0 失败;
locale_check 0 缺失。
- Not in this change: 「一键重算」按钮与"实时界面禁止回放入口"(7.2/7.3);深度图改用主题色带改变了 depth
视图的颜色(这是修正:色条一直画的就是主题色带);三维(scene3d)未动;录像格式未动(仍只存原始相机图像)。
This commit is contained in:
@@ -0,0 +1,192 @@
|
||||
"""Acceptance for the single 2D image backend.
|
||||
|
||||
The point of this module is that live, recomputed, replayed and previewed frames go
|
||||
through *one* renderer and that the view modes come from what the frame carries. These
|
||||
tests pin both halves: a plan derived from fields (not from a view preset), and
|
||||
pixel-for-pixel agreement with the primitives the application used before.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from orisys_app_shared.acquisition import image_backend as backend
|
||||
from orisys_app_shared.acquisition.arrows import (
|
||||
depth_map_to_bgr,
|
||||
draw_arrow_display,
|
||||
flow_colormap,
|
||||
)
|
||||
|
||||
|
||||
def _image(h: int = 12, w: int = 16) -> np.ndarray:
|
||||
return np.arange(h * w * 3, dtype=np.uint8).reshape(h, w, 3)
|
||||
|
||||
|
||||
def _flow(h: int = 12, w: int = 16) -> np.ndarray:
|
||||
flow = np.zeros((h, w, 2), dtype=np.float32)
|
||||
flow[..., 0] = 3.0
|
||||
flow[..., 1] = -1.5
|
||||
return flow
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------- plans
|
||||
def test_a_live_frame_offers_the_derived_modes_and_needs_no_note():
|
||||
plan = backend.plan_image(
|
||||
backend.ImageFrame(_image(), flow=_flow(), depth=np.ones((12, 16)))
|
||||
)
|
||||
assert plan.available_modes == ("arrows_pure", "arrows_stitched", "depth")
|
||||
assert plan.default_mode == "arrows_pure"
|
||||
assert plan.missing == ()
|
||||
assert plan.note_key == ""
|
||||
|
||||
|
||||
def test_a_recorded_frame_is_raw_and_says_so():
|
||||
"""No flow and no depth: the frame *is* the picture, and the note explains it."""
|
||||
plan = backend.plan_image(backend.ImageFrame(_image(), source=backend.SOURCE_REPLAY))
|
||||
assert plan.available_modes == ("recorded",)
|
||||
assert plan.default_mode == "recorded"
|
||||
assert plan.missing == ("flow", "depth")
|
||||
assert plan.note_key == backend.RAW_NOTE_KEY
|
||||
assert plan.source == "replay"
|
||||
|
||||
|
||||
def test_a_derived_source_keeps_every_mode_and_reports_what_is_missing():
|
||||
"""A dropped frame must not silently switch the operator's view.
|
||||
|
||||
Live and recomputed frames may be missing a field for one frame; the mode stays
|
||||
selectable and ``missing`` says which field is absent, so the panel can state it
|
||||
instead of the view looking broken.
|
||||
"""
|
||||
flow_only = backend.plan_image(backend.ImageFrame(_image(), flow=_flow()))
|
||||
assert flow_only.available_modes == backend.DERIVED_MODES
|
||||
assert flow_only.missing == ("depth",)
|
||||
assert flow_only.default_mode == "arrows_pure"
|
||||
assert flow_only.note_key == backend.MISSING_NOTE_KEY
|
||||
assert flow_only.note_fields == ("depth",)
|
||||
assert flow_only.needs("depth") == "depth"
|
||||
assert flow_only.needs("arrows_pure") is None
|
||||
|
||||
depth_only = backend.plan_image(backend.ImageFrame(_image(), depth=np.ones((12, 16))))
|
||||
assert depth_only.available_modes == backend.DERIVED_MODES
|
||||
assert depth_only.missing == ("flow",)
|
||||
assert depth_only.default_mode == "depth"
|
||||
assert depth_only.needs("arrows_pure") == "flow"
|
||||
|
||||
complete = backend.plan_image(
|
||||
backend.ImageFrame(_image(), flow=_flow(), depth=np.ones((12, 16)))
|
||||
)
|
||||
assert complete.missing == ()
|
||||
assert complete.note_key == ""
|
||||
|
||||
|
||||
def test_a_raw_source_cannot_offer_a_derived_view_at_all():
|
||||
"""A recording and a camera preview have no derived field, ever."""
|
||||
for source in (backend.SOURCE_REPLAY, backend.SOURCE_PREVIEW):
|
||||
plan = backend.plan_image(backend.ImageFrame(_image(), source=source))
|
||||
assert plan.available_modes == ("recorded",)
|
||||
assert plan.note_key == backend.RAW_NOTE_KEY
|
||||
assert plan.note_fields == ()
|
||||
|
||||
|
||||
def test_the_plan_carries_the_frame_identity_for_the_panel():
|
||||
plan = backend.plan_image(
|
||||
backend.ImageFrame(
|
||||
_image(), source=backend.SOURCE_RECOMPUTE, frame_id=42, t_rel_s=1.25, stale=True
|
||||
)
|
||||
)
|
||||
assert (plan.source, plan.frame_id, plan.t_rel_s, plan.stale) == ("recompute", 42, 1.25, True)
|
||||
|
||||
|
||||
def test_a_mode_the_plan_does_not_offer_falls_back_to_the_default():
|
||||
"""Never draw a view the source cannot have at all."""
|
||||
recorded = backend.plan_image(
|
||||
backend.ImageFrame(_image(), source=backend.SOURCE_REPLAY)
|
||||
)
|
||||
assert recorded.resolve("arrows_pure") == "recorded"
|
||||
assert recorded.resolve("nonsense") == "recorded"
|
||||
live = backend.plan_image(backend.ImageFrame(_image(), flow=_flow()))
|
||||
assert live.resolve("recorded") == "arrows_pure", "a derived source has no raw mode"
|
||||
assert live.resolve("arrows_stitched") == "arrows_stitched"
|
||||
|
||||
|
||||
# --------------------------------------------------------------- one renderer
|
||||
def test_the_backend_renders_what_the_primitives_rendered_before():
|
||||
"""Live went through these two calls; the backend must not change a pixel."""
|
||||
image = _image()
|
||||
flow = _flow()
|
||||
mask = np.ones((12, 16), dtype=bool)
|
||||
mask[0, 0] = False
|
||||
|
||||
frame = backend.ImageFrame(image, flow=flow, valid_mask=mask)
|
||||
plan = backend.plan_image(frame)
|
||||
for mode in ("arrows_pure", "arrows_stitched"):
|
||||
expected = draw_arrow_display(
|
||||
image,
|
||||
flow,
|
||||
background="captured" if mode == "arrows_stitched" else "pure_color",
|
||||
draw_style="finger",
|
||||
valid_mask=mask,
|
||||
)
|
||||
got = backend.render_image(frame, plan, mode=mode, draw_style="finger")
|
||||
assert np.array_equal(got, expected), mode
|
||||
|
||||
depth = np.linspace(0.0, 1.0, 12 * 16, dtype=np.float32).reshape(12, 16)
|
||||
depth_frame = backend.ImageFrame(image, depth=depth, valid_mask=mask)
|
||||
# The depth picture uses the *theme's* ramp — the same table the panel's colour bar
|
||||
# paints. The window used to leave this argument out, so the map was drawn with
|
||||
# OpenCV's JET while the legend under it showed the brand ramp: two colour tables
|
||||
# for one picture, and the legend claiming they were the same.
|
||||
expected_depth = depth_map_to_bgr(
|
||||
depth, flow_colormap("dark"), value_range=(0.0, 1.0), invalid_mask=mask
|
||||
)
|
||||
got_depth = backend.render_image(
|
||||
depth_frame, mode="depth", depth_range=(0.0, 1.0)
|
||||
)
|
||||
assert np.array_equal(got_depth, expected_depth)
|
||||
|
||||
|
||||
def test_a_recorded_frame_is_passed_through_and_never_resized_to_fit():
|
||||
image = _image(9, 7)
|
||||
got = backend.render_image(
|
||||
backend.ImageFrame(image, source=backend.SOURCE_REPLAY), mode="recorded"
|
||||
)
|
||||
assert got is not None and got.shape == image.shape
|
||||
assert np.array_equal(got, image)
|
||||
|
||||
|
||||
def test_nothing_to_draw_is_none_not_a_blank_picture():
|
||||
assert backend.render_image(backend.ImageFrame(), mode="recorded") is None
|
||||
assert backend.render_image(
|
||||
backend.ImageFrame(None, flow=_flow()), mode="arrows_pure"
|
||||
) is None
|
||||
assert backend.render_image(backend.ImageFrame(None, depth=None), mode="depth") is None
|
||||
|
||||
|
||||
def test_a_grey_base_frame_is_widened_once_by_the_backend():
|
||||
"""A 2-D frame must reach the shader as BGR, and only one place may do it."""
|
||||
grey = np.zeros((8, 6), dtype=np.uint8)
|
||||
got = backend.render_image(backend.ImageFrame(grey), mode="recorded")
|
||||
assert got is not None and got.shape == (8, 6, 3)
|
||||
assert np.array_equal(got, np.stack([grey] * 3, axis=-1))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", list(backend.ALL_MODES))
|
||||
def test_every_offered_mode_renders_something_or_says_it_cannot(mode):
|
||||
"""A mode the plan offers must never come back empty."""
|
||||
frame = backend.ImageFrame(_image(), flow=_flow(), depth=np.ones((12, 16)))
|
||||
plan = backend.plan_image(frame)
|
||||
if not plan.accepts(mode):
|
||||
assert plan.resolve(mode) in plan.available_modes
|
||||
return
|
||||
assert backend.render_image(frame, plan, mode=mode) is not None
|
||||
|
||||
|
||||
def test_a_missing_field_is_reported_not_invented():
|
||||
"""Depth with no depth says so; arrows with no flow show the base picture."""
|
||||
no_depth = backend.ImageFrame(_image(), flow=_flow())
|
||||
assert backend.render_image(no_depth, mode="depth") is None
|
||||
no_flow = backend.ImageFrame(_image(), depth=np.ones((12, 16)))
|
||||
base = backend.render_image(no_flow, mode="arrows_pure")
|
||||
assert base is not None
|
||||
assert np.array_equal(base, _image()), "the base picture must not be redrawn"
|
||||
Reference in New Issue
Block a user