refactor(studio-2d): 二维图像单一后端 —— 视图模式由帧内容决定，四条路径共用一个渲染入口
- What:
1. 新增 `shared/.../acquisition/image_backend.py`（纯 NumPy，无 Qt）：`ImageFrame`（底图 + flow/depth/
valid_mask + source/frame_id/t_rel_s/stale）、`ImagePlan`、`plan_image()`、`render_image()`，以及模式
常量（`arrows_pure`/`arrows_stitched`/`depth`/`recorded`）与来源常量（live/recompute/replay/preview）。
`plan_image()` 分两个问题回答：**可选项**由来源决定（live/recompute = 三种派生视图；录像/摄像头预览
= 只有 `recorded`），**这一帧能不能画**由 `missing` 说明（某帧缺 flow/depth 不会悄悄改掉操作者选的
视图，而是在页脚写一行）。`render_image()` 是唯一渲染入口：内部复用 `draw_arrow_display`/
`depth_map_to_bgr`/`flow_colormap`（公共签名不变），深度图**改用主题色带**（与面板色条同一张表——
原来窗口没传 colormap，深度图走 OpenCV JET 而色条画品牌色带，图例与图片两张表）。
2. `ui/arrow_panel.py`：新增 `bgr_to_qimage()`（全应用唯一一份 BGR→QImage，`app_window._bgr_to_qimage`
成为它的别名、`replay.py` 的重复实现删除）；新增 `set_plan(plan)`/`plan_modes()`——模式集来自 plan，
`view_modes()` 在无 plan 时回退旧的预设式列表（`set_source_kind` 保留"命名/文案"职责，并支持
`recompute` 状态与 `panels.recompute_pipeline`）；页脚新增一行 `note_label` 说明 plan（如
"录像只保存原始相机图像…"），箭头放大/有效区注解改由"本帧是否支持箭头视图"决定；
`set_view_mode()` 与 `_populate_view_combo()` 按当前模式集校验（不再只认三个常量，否则 `recorded`
会被强行改回 `arrows_pure`）。
3. `app_window.py`：新增唯一入口 `_show_image_frame(ImageFrame)`（plan → 渲染 → 一个 QImage 给面板），
**四条生产者全部改走它**：实时/重算（`_render_run_image`，来源按 `_recompute_running` 区分）、回放
（`_render_replay_frame` 用 `replay.video_frame_bgr()`，不再自己转 QImage）、摄像头预览
（`_on_preview_tick`）。`_render_run_image` 里那套按 `mode` 分支画箭头的代码整体删除。
4. `replay.py`：新增 `video_frame_bgr()`（按光标缓存 BGR），`video_image()` 保留为兼容薄封装；删除
本模块的 `_bgr_to_qimage`。
5. locale：新增 `panels.replay_raw_note`（录像只有原始图像的说明 + 指路「由视频重算」）、
`panels.frame_missing_note`（`{fields}` 参数化）、`panels.field_flow`/`panels.field_depth`、
`panels.recompute_pipeline(_tip)`（中英各一套）。
- Why: 用户报的两个 bug 之一——「历史回放面板的 2 维图只能显示原始采集图像，没有进行任何处理」，并建议
"每一个 2D 图的后端保持一致"。核对后确认根因是架构而非漏了一步渲染：二维有三个（实际四个）互不一致的
生产者——实时/重算走 `draw_arrow_display`/`depth_map_to_bgr`，回放与预览把 BGR 直接交给面板；模式集与
注解由**视图预设**决定（`arrow_panel.set_source_kind("replay")` 才切成 `recorded`），于是同一段视频在
"回放"和"重算"下是两个成像栈，且 `_bgr_to_qimage` 有两份实现。用户已选定"统一后端 + 一键重算（不改记录
格式）"，本提交做前半段；"一键重算"的入口在下一提交。
- Verification:
· 新增 `shared/tests/test_image_backend.py` 15 项：plan 的三类来源（live/recompute 三种派生视图、
replay/preview 只有 recorded）、`missing`/`note_key`/`note_fields`、模式回退、**与旧渲染逐像素一致**
（arrows_pure/arrows_stitched 对 `draw_arrow_display`、depth 对 `depth_map_to_bgr` 带主题色带）、
原始帧不做缩放/不替图、缺 depth → None、缺 flow → 底图、灰度帧只在这里加宽一次。
· `apps/studio/tests/test_ui_smoke.py` +2：同一段视频在回放路径只出 `recorded` 且有说明行、在重算/实时
路径出三种派生视图且都经过同一个 `render_image`（用 spy 断言来源与模式）、面板 `note_label` 不是裸键、
combo 项数与模式集一致；以及"全应用只有一处 BGR→QImage"的结构断言。
· 既有 `test_depth_view_reports_missing_data_and_fixes_its_scale`（缺深度必须报"本帧没有深度数据"而不是
回退显示普通形变图）继续通过——新的"缺字段说明 + 不回退替图"语义与它一致。
· 全量：apps/studio 236 项 0 失败 14 跳过；shared 331 项 0 失败 1 跳过；ui_check 587 项 0 失败；
locale_check 0 缺失。
- Not in this change: 「一键重算」按钮与"实时界面禁止回放入口"（7.2/7.3）；深度图改用主题色带改变了 depth
视图的颜色（这是修正：色条一直画的就是主题色带）；三维（scene3d）未动；录像格式未动（仍只存原始相机图像）。
This commit is contained in:
ykcheungab
@@ -0,0 +1,220 @@
|
||||
"""One place that turns a frame into the picture the 2D panel shows.
|
||||
|
||||
The application used to have four producers of that picture and two different imaging
|
||||
stacks: live and recomputed frames went through ``draw_arrow_display``/
|
||||
``depth_map_to_bgr`` in the window, the preview and the replay path handed a raw BGR
|
||||
frame straight to the panel, and the panel decided which view modes existed from the
|
||||
*view preset* (``set_source_kind("replay")``) rather than from what the frame actually
|
||||
carried. The same video therefore behaved differently in "replay" and "recompute", and
|
||||
a mode that needed data nobody had could still be offered.
|
||||
|
||||
This module is the single backend. Two rules make the difference:
|
||||
|
||||
* **the frame says what it has** (:class:`ImageFrame`), and the plan is derived from
|
||||
that — not from which view is on screen. A frame with no flow has no arrow modes; a
|
||||
frame with no depth has no depth mode; a frame with neither is a raw picture;
|
||||
* **rendering is one function** (:func:`render_image`), so live, recompute, replay and
|
||||
preview cannot drift apart. A mode the frame cannot support falls back to the plan's
|
||||
default instead of drawing something plausible.
|
||||
|
||||
NumPy only: the Qt conversion belongs to the UI layer (``ui/arrow_panel.bgr_to_qimage``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, Sequence
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.acquisition.arrows import (
|
||||
depth_map_to_bgr,
|
||||
draw_arrow_display,
|
||||
flow_colormap,
|
||||
)
|
||||
from orisys_app_shared.acquisition.models import ensure_bgr
|
||||
|
||||
#: The view modes, in the order a panel should offer them. ``recorded`` is not a
|
||||
#: choice: it is what is left when the frame carries no derived field at all.
|
||||
ARROW_PURE = "arrows_pure"
|
||||
ARROW_STITCHED = "arrows_stitched"
|
||||
DEPTH = "depth"
|
||||
RECORDED = "recorded"
|
||||
|
||||
DERIVED_MODES = (ARROW_PURE, ARROW_STITCHED, DEPTH)
|
||||
ALL_MODES = (*DERIVED_MODES, RECORDED)
|
||||
|
||||
#: Where a frame came from. Only used for wording and for the recording; the modes
|
||||
#: come from the frame's own fields.
|
||||
SOURCE_LIVE = "live"
|
||||
SOURCE_RECOMPUTE = "recompute"
|
||||
SOURCE_REPLAY = "replay"
|
||||
SOURCE_PREVIEW = "preview"
|
||||
|
||||
#: Shown when the only thing a frame can be is a raw picture.
|
||||
RAW_NOTE_KEY = "panels.replay_raw_note"
|
||||
#: Shown when a frame of a *derived* source is missing a field (a dropped frame, a
|
||||
#: failed tracker): the mode stays selectable — that is the operator's choice — and the
|
||||
#: line says which field is absent instead of the view looking broken.
|
||||
MISSING_NOTE_KEY = "panels.frame_missing_note"
|
||||
|
||||
#: Sources whose frames carry derived fields.
|
||||
DERIVED_SOURCES = (SOURCE_LIVE, SOURCE_RECOMPUTE)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImageFrame:
|
||||
"""A picture to draw, plus whatever derived fields came with it.
|
||||
|
||||
``image_bgr`` is the *base* picture (the processed/unrolled frame for a live or
|
||||
recomputed source, the recorded or camera frame otherwise). ``flow``/``depth``/
|
||||
``valid_mask`` are the same arrays the rest of the pipeline uses; ``None`` means the
|
||||
frame does not have them, and the plan will say so rather than offer a mode that
|
||||
would have to invent them.
|
||||
"""
|
||||
|
||||
image_bgr: Optional[np.ndarray] = None
|
||||
flow: Optional[np.ndarray] = None
|
||||
depth: Optional[np.ndarray] = None
|
||||
valid_mask: Optional[np.ndarray] = None
|
||||
source: str = SOURCE_LIVE
|
||||
frame_id: int = 0
|
||||
t_rel_s: float = 0.0
|
||||
#: True when the producer knows this frame's measurement is not current (the
|
||||
#: coordinator's "stale" state); the panel can say so under the picture.
|
||||
stale: bool = False
|
||||
|
||||
def has_flow(self) -> bool:
|
||||
return self.flow is not None
|
||||
|
||||
def has_depth(self) -> bool:
|
||||
return self.depth is not None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImagePlan:
|
||||
"""What this frame can be shown as, and what is missing from it."""
|
||||
|
||||
available_modes: tuple[str, ...]
|
||||
default_mode: str
|
||||
missing: tuple[str, ...]
|
||||
source: str
|
||||
frame_id: int
|
||||
t_rel_s: float
|
||||
stale: bool
|
||||
#: An i18n key for the one line that explains the plan, or ``""``. When
|
||||
#: :attr:`note_fields` is non-empty the line takes it as ``fields``.
|
||||
note_key: str = ""
|
||||
note_fields: tuple[str, ...] = ()
|
||||
|
||||
def accepts(self, mode: str) -> bool:
|
||||
return str(mode) in self.available_modes
|
||||
|
||||
def resolve(self, mode: str) -> str:
|
||||
"""The mode to draw: the request, or the default when it cannot be drawn."""
|
||||
wanted = str(mode or "")
|
||||
return wanted if self.accepts(wanted) else self.default_mode
|
||||
|
||||
def needs(self, mode: str) -> Optional[str]:
|
||||
"""The field a mode needs that this frame does not have, if any."""
|
||||
wanted = str(mode or "")
|
||||
if wanted in (ARROW_PURE, ARROW_STITCHED) and not self.has_flow():
|
||||
return "flow"
|
||||
if wanted == DEPTH and not self.has_depth():
|
||||
return "depth"
|
||||
return None
|
||||
|
||||
def has_flow(self) -> bool:
|
||||
return "flow" not in self.missing
|
||||
|
||||
def has_depth(self) -> bool:
|
||||
return "depth" not in self.missing
|
||||
|
||||
|
||||
def plan_image(frame: ImageFrame) -> ImagePlan:
|
||||
"""Derive the modes from the frame's *source* and its own fields.
|
||||
|
||||
Two different questions, deliberately answered in two different places:
|
||||
|
||||
* **which views may the operator choose?** For a source that carries derived fields
|
||||
(live, recompute) all of them — a single dropped frame must not silently switch
|
||||
the operator's view. For a source that never does (a recording, a camera preview)
|
||||
exactly one, because offering the others would be offering a lie;
|
||||
* **what can this frame actually be drawn as?** ``missing`` says which field is
|
||||
absent, and the panel states it in one line.
|
||||
"""
|
||||
derived = str(frame.source or SOURCE_LIVE) in DERIVED_SOURCES
|
||||
missing: list[str] = []
|
||||
if not frame.has_flow():
|
||||
missing.append("flow")
|
||||
if not frame.has_depth():
|
||||
missing.append("depth")
|
||||
if not derived:
|
||||
return ImagePlan(
|
||||
available_modes=(RECORDED,),
|
||||
default_mode=RECORDED,
|
||||
missing=("flow", "depth"),
|
||||
source=str(frame.source or SOURCE_LIVE),
|
||||
frame_id=int(frame.frame_id),
|
||||
t_rel_s=float(frame.t_rel_s),
|
||||
stale=bool(frame.stale),
|
||||
note_key=RAW_NOTE_KEY,
|
||||
)
|
||||
default = ARROW_PURE if frame.has_flow() else (DEPTH if frame.has_depth() else ARROW_PURE)
|
||||
return ImagePlan(
|
||||
available_modes=DERIVED_MODES,
|
||||
default_mode=default,
|
||||
missing=tuple(missing),
|
||||
source=str(frame.source or SOURCE_LIVE),
|
||||
frame_id=int(frame.frame_id),
|
||||
t_rel_s=float(frame.t_rel_s),
|
||||
stale=bool(frame.stale),
|
||||
note_key=MISSING_NOTE_KEY if missing else "",
|
||||
note_fields=tuple(missing),
|
||||
)
|
||||
|
||||
|
||||
def render_image(
|
||||
frame: ImageFrame,
|
||||
plan: Optional[ImagePlan] = None,
|
||||
*,
|
||||
mode: str = "",
|
||||
theme: str = "dark",
|
||||
draw_style: str = "box",
|
||||
depth_range: Optional[tuple[float, float]] = None,
|
||||
canvas_bgr: Optional[Sequence[int]] = None,
|
||||
) -> Optional[np.ndarray]:
|
||||
"""Draw ``frame`` in ``mode``; ``None`` when there is nothing to draw.
|
||||
|
||||
The mode is resolved through the plan first, so a caller that asks for a view the
|
||||
frame cannot support gets the plan's default rather than a picture assembled from
|
||||
an array that is not there.
|
||||
"""
|
||||
plan = plan or plan_image(frame)
|
||||
wanted = plan.resolve(mode)
|
||||
if wanted == DEPTH:
|
||||
# Never substitute another picture for a missing depth map: that made "no
|
||||
# depth" look like a depth reading. The panel says so instead.
|
||||
return depth_map_to_bgr(
|
||||
frame.depth,
|
||||
flow_colormap(theme),
|
||||
value_range=depth_range,
|
||||
invalid_mask=frame.valid_mask,
|
||||
)
|
||||
if frame.image_bgr is None:
|
||||
return None
|
||||
if wanted in (ARROW_PURE, ARROW_STITCHED):
|
||||
if frame.flow is None:
|
||||
# The base picture with no arrows, and the plan's ``missing`` says why.
|
||||
return ensure_bgr(np.asarray(frame.image_bgr))
|
||||
return draw_arrow_display(
|
||||
frame.image_bgr,
|
||||
frame.flow,
|
||||
background="captured" if wanted == ARROW_STITCHED else "pure_color",
|
||||
canvas_bgr=canvas_bgr,
|
||||
draw_style=draw_style,
|
||||
valid_mask=frame.valid_mask,
|
||||
colormap=flow_colormap(theme),
|
||||
)
|
||||
# ``recorded``: the frame is the picture, with nothing drawn over it.
|
||||
return ensure_bgr(np.asarray(frame.image_bgr))
|
||||
Reference in New Issue
Block a user