阶段 1：平台 PoC —— 真实 GL 上跑通静态场景，并修掉它抓出的相机/上下文错误 - What: 按 docs/architecture/studio-3d-stages.md 阶段 1 落地图形栈。OrbitCamera 全部实现（target/distance/yaw/pitch 表示；fit 按包围球与较窄的那个视场轴定距离；orbit/pan/zoom 有界；set_preset 从 eye/target 反解角度使拖动可延续；matrices 输出只读 float32 view/projection；另加 depth_range_m() 便于断言）。gl_backend：GLDispatch.bind_current_context 真正绑定 PyOpenGL 并读 vendor/renderer/GL/GLSL/max texture、判软件渲染、profile 不达标时抛 ThreeDUnsupported 并带回能力；GLRenderer 实现 initialize（两个 program，desktop33/gles30 共用同一份着色器主体、只换头部）、upload_scene（每 part 一套 VAO/VBO/EBO，交错 layout position/normal/scalar/valid = 8 float）、update_surface（只对感知面 glBufferSubData，绝不重建索引）、draw（绑 Qt 的 defaultFramebufferObject、深度测试、按 role 选 program、housing 隐藏即跳过、scalar 经 LUT 纹理与 range uniform 上色，Python 不生成 RGB 数组）、release（幂等）；纯函数 pack_part/pack_scene_part/estimate_gpu_bytes/rest_surface/shader_source/shader_header/lut_rgba/is_software_renderer/profile_from_renderable_type 都可在无 GPU 环境测试。Sensor3DWidget 实现 initializeGL/paintGL/resizeGL/release_resources/aboutToBeDestroyed/showEvent/hideEvent 与鼠标事件；加载场景时先采用资产自带的默认视角再 fit；一帧几何不匹配只"拒绝该帧"（计数 + 诊断 + 用 rest_surface 清掉上次测量）而不是把整个视图打成不可用。shared/scene3d/synthetic.py 提供 PoC 与测试共用的已知场景（20mm 感知板 + 外壳盒，米、sensor 坐标系、带默认三视角预设）。tools/scene_assets/probe_gl.py 真实现：真实窗口取 context 并输出 JSON 报告（平台/Qt/PyOpenGL/vendor/renderer/GL/GLSL/DPR/framebuffer/是否软件渲染），退出码即门禁（0 通过、3 软件渲染、4 无上下文）；apps/studio/pyproject.toml 增加 [three-d] 可选依赖（仅 PyOpenGL；Open3D 不进运行时）。测试：apps/studio/tests/test_scene3d_stage1.py 24 项（相机数学、PoC 场景合规、打包布局、着色器两方言共用主体、LUT、软件渲染识别、renderableType 映射、GPU 预算、参考帧语义，全部无需 GPU）＋ test_scene3d_gl_smoke.py 8 项（真实 context：能力、两个 part 都画出来、外壳可隐藏、参考帧中性 vs 色图有差异、动态更新不动 housing 缓冲、坏几何帧被拒且视图存活、resize 跟随、release 幂等且新 context 能重传场景、探针如实报告）；offscreen 下后者自动 skip。docs/architecture/gl-report-windows-dev.json 为本机平台报告。 - Why: 用户要求"从骨架搭起→计划阶段→逐阶段实现与验收"，并明确 ARM 暂缓。阶段 1 的产出本身就是一份能回答"这块板的图形栈能不能用"的报告，所以先把它做实、把 profile 冻结点定在真实 context 上，而不是先堆屏幕上的东西。 - Verification: 真实 context（QT_QPA_PLATFORM=windows）下 8 项 GL 验收全过；offscreen 下 24 项 CPU 项全过、GL 项按设计 skip；shared 全量 187 passed；studio 全量 116 passed（含新 32 项）；ui_check 554 项 0 失败；probe_gl 输出 verdict=ok（NVIDIA RTX 4080 SUPER、GL 3.3 Core、GLSL 3.30、非软件渲染）。过程中这套验收抓出并修掉了 7 个真实缺陷：Qt renderableType 映射反了（NVIDIA 兼容 ES 着色器所以本机看不出，Intel/ARM 会直接失败）、观察相机是左手基导致整模型镜像（det=-1）、近平面过近使 NDC z 全挤在 0.99（深度精度归零、外壳与感知面会 z-fighting）、手动 makeCurrent 早于 Qt 首次绘制会把 GPU 对象建在被丢弃的 context 里（glBindVertexArray 报 INVALID_OPERATION）、context 重建后不重传场景、一帧几何不匹配把整个视图打死、以及 grabFramebuffer 是 BGRA+stride 导致颜色断言全部读错通道。 - Not in this change: 未接入 app_window/main.py（三维对现有应用仍完全惰性，故"开/关三维 CSV 行序一致"这条验收随阶段 4 做）；阶段 2 的资产导入/坐标/包格式；ARM 平台实测（probe 已可在板上直接运行）。
This commit is contained in:
ykcheungab
@@ -16,6 +16,18 @@ dependencies = [
|
||||
"pyqtgraph>=0.13",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
# The three-dimensional view (docs/architecture/studio-3d-design.md §11.1).
|
||||
# PyOpenGL is the GL binding; PyOpenGL_accelerate is an optional speed-up and is
|
||||
# deliberately not required. Open3D stays out: it is an offline tool, and its ARM
|
||||
# support is experimental and its GUI needs a desktop GL profile.
|
||||
#
|
||||
# The binding does not ship a driver. On a board, do not let pip or a system
|
||||
# package replace the BSP's EGL/GLES libraries.
|
||||
three-d = [
|
||||
"PyOpenGL>=3.1.6",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
orisys-studio = "orisys_studio.main:main"
|
||||
|
||||
|
||||
@@ -1,50 +1,238 @@
|
||||
"""The observation camera: orbit, pan, zoom, presets. Pure maths.
|
||||
"""The observation camera: orbit, pan, zoom, presets. Pure maths, no Qt.
|
||||
|
||||
Stage 1 gets a minimal ``fit``/``orbit`` so a static scene can be looked at; the
|
||||
full interaction (pan, zoom limits, presets, reset) is stage 4 in
|
||||
``docs/architecture/studio-3d-stages.md``.
|
||||
It changes ``view`` and ``projection`` and nothing else: the pixel→surface
|
||||
mapping, the calibration and the force axes are not reachable from here. The
|
||||
default view is a ``CameraPreset`` that ships with the geometry, so "what the part
|
||||
looks like" is versioned with the asset rather than with somebody's laptop.
|
||||
|
||||
It is deliberately *not* the algorithm camera. It changes ``view`` and
|
||||
``projection`` only: ``window_to_world`` for the pixel→surface mapping, the
|
||||
calibration and the force axes are untouched by anything a user does here. The
|
||||
default view comes from the asset's ``CameraPreset``, so it is versioned with the
|
||||
geometry rather than with a user's preference file.
|
||||
Conventions (design document §3.1): sensor frame, right-handed, metres; column
|
||||
vectors, so ``p_camera = view @ p_sensor`` and ``p_clip = projection @ p_camera``.
|
||||
The view matrix is built from a target, a distance and two angles, which is what
|
||||
makes orbit/pan/zoom trivial and keeps "reset" exact rather than approximate.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraMatrices,
|
||||
CameraPreset,
|
||||
F32,
|
||||
freeze,
|
||||
)
|
||||
|
||||
_STAGE = "stage 1 (fit/orbit) and stage 4 (full interaction)"
|
||||
#: Distance limits, in metres: close enough to inspect a 20 mm part, far enough
|
||||
#: not to lose a 10 cm one. The far plane follows the distance, so precision stays
|
||||
#: usable at both ends instead of being decided once.
|
||||
MIN_DISTANCE_M = 0.005
|
||||
MAX_DISTANCE_M = 5.0
|
||||
#: Radians per pixel of drag, and the pitch limit that keeps the up vector sane.
|
||||
ORBIT_RAD_PER_PX = 0.008
|
||||
MAX_PITCH_RAD = math.radians(89.0)
|
||||
#: Zoom per wheel step.
|
||||
ZOOM_PER_STEP = 0.12
|
||||
|
||||
|
||||
@dataclass
|
||||
class OrbitCamera:
|
||||
"""A target, a distance and two angles. Everything else is derived."""
|
||||
"""A target, a distance and two angles; everything else is derived."""
|
||||
|
||||
target_sensor_m: np.ndarray = field(
|
||||
default_factory=lambda: np.zeros(3, dtype=np.float64)
|
||||
)
|
||||
distance_m: float = 0.1
|
||||
yaw_rad: float = 0.0
|
||||
pitch_rad: float = 0.0
|
||||
up_sensor: np.ndarray = field(
|
||||
default_factory=lambda: np.array([0.0, 0.0, 1.0], dtype=np.float64)
|
||||
)
|
||||
vertical_fov_rad: float = 0.6
|
||||
aspect: float = 1.0
|
||||
_fitted_radius_m: float = 0.05
|
||||
_preset_name: Optional[str] = None
|
||||
|
||||
# ------------------------------------------------------------------- setup
|
||||
def fit(self, bounds_sensor_m: F32, aspect: float) -> None:
|
||||
"""Frame the whole scene: set the target, distance, near and far planes."""
|
||||
raise NotImplementedError(f"{_STAGE}: fit the camera to the scene bounds")
|
||||
"""Frame the whole scene from the bounds, without changing the angles.
|
||||
|
||||
The distance is derived from the bounding sphere and the *narrower* of the
|
||||
two field-of-view axes, so a wide window does not clip the model
|
||||
vertically while a tall one does not clip it horizontally.
|
||||
"""
|
||||
bounds = np.asarray(bounds_sensor_m, dtype=np.float64)
|
||||
if bounds.shape != (2, 3):
|
||||
raise ValueError(f"bounds must be [2,3], got {bounds.shape}")
|
||||
low, high = bounds[0], bounds[1]
|
||||
self.target_sensor_m = (low + high) * 0.5
|
||||
radius = float(np.linalg.norm(high - low) * 0.5)
|
||||
self._fitted_radius_m = max(radius, 1e-6)
|
||||
self.aspect = max(float(aspect), 1e-6)
|
||||
self.vertical_fov_rad = _clamp(self.vertical_fov_rad, 0.1, 2.6)
|
||||
vertical_half = 0.5 * self.vertical_fov_rad
|
||||
horizontal_half = math.atan(math.tan(vertical_half) * self.aspect)
|
||||
half = min(vertical_half, horizontal_half)
|
||||
# A little margin, so the silhouette is not flush with the edge.
|
||||
self.distance_m = _clamp(
|
||||
self._fitted_radius_m / max(math.sin(half), 1e-6) * 1.15,
|
||||
MIN_DISTANCE_M,
|
||||
MAX_DISTANCE_M,
|
||||
)
|
||||
|
||||
def set_preset(self, preset: CameraPreset) -> None:
|
||||
"""Point the camera where the asset says its default view is.
|
||||
|
||||
The preset's ``eye`` is turned into yaw/pitch/distance around its target,
|
||||
so a later drag continues from the preset instead of jumping.
|
||||
"""
|
||||
eye = np.asarray(preset.eye_sensor_m, dtype=np.float64)
|
||||
target = np.asarray(preset.target_sensor_m, dtype=np.float64)
|
||||
up = np.asarray(preset.up_sensor, dtype=np.float64)
|
||||
offset = eye - target
|
||||
distance = float(np.linalg.norm(offset))
|
||||
if distance <= 0.0:
|
||||
raise ValueError(f"preset {preset.name!r} puts the eye on the target")
|
||||
self.target_sensor_m = target
|
||||
self.distance_m = _clamp(distance, MIN_DISTANCE_M, MAX_DISTANCE_M)
|
||||
self.up_sensor = up / max(float(np.linalg.norm(up)), 1e-9)
|
||||
direction = offset / distance
|
||||
self.pitch_rad = _clamp(math.asin(_clamp(direction[2], -1.0, 1.0)),
|
||||
-MAX_PITCH_RAD, MAX_PITCH_RAD)
|
||||
self.yaw_rad = math.atan2(direction[0], -direction[1])
|
||||
self.vertical_fov_rad = _clamp(float(preset.vertical_fov_rad), 0.1, 2.6)
|
||||
self._preset_name = preset.name
|
||||
|
||||
# -------------------------------------------------------------- interaction
|
||||
def orbit(self, delta_px: tuple[float, float], viewport: tuple[int, int]) -> None:
|
||||
"""Rotate about the target. The model data does not move."""
|
||||
raise NotImplementedError(f"{_STAGE}: orbit")
|
||||
dx, dy = (float(delta_px[0]), float(delta_px[1]))
|
||||
self.yaw_rad = (self.yaw_rad + dx * ORBIT_RAD_PER_PX) % (2.0 * math.pi)
|
||||
self.pitch_rad = _clamp(
|
||||
self.pitch_rad + dy * ORBIT_RAD_PER_PX, -MAX_PITCH_RAD, MAX_PITCH_RAD
|
||||
)
|
||||
|
||||
def pan(self, delta_px: tuple[float, float], viewport: tuple[int, int]) -> None:
|
||||
"""Slide the target in the view plane, scaled by distance."""
|
||||
raise NotImplementedError(f"{_STAGE}: pan")
|
||||
"""Slide the target in the view plane, scaled so it tracks the cursor."""
|
||||
width = max(int(viewport[0]), 1)
|
||||
height = max(int(viewport[1]), 1)
|
||||
dx, dy = float(delta_px[0]), float(delta_px[1])
|
||||
world_per_px = (
|
||||
2.0 * self.distance_m * math.tan(0.5 * self.vertical_fov_rad) / height
|
||||
)
|
||||
right, up = self._basis()
|
||||
self.target_sensor_m = (
|
||||
self.target_sensor_m - right * (dx * world_per_px) + up * (dy * world_per_px)
|
||||
)
|
||||
|
||||
def zoom(self, wheel_steps: float) -> None:
|
||||
"""Exponential zoom, clamped so the surface cannot be entered or lost."""
|
||||
raise NotImplementedError(f"{_STAGE}: zoom")
|
||||
factor = math.pow(1.0 - ZOOM_PER_STEP, float(wheel_steps))
|
||||
self.distance_m = _clamp(
|
||||
self.distance_m * factor, MIN_DISTANCE_M, MAX_DISTANCE_M
|
||||
)
|
||||
|
||||
def set_preset(self, preset: CameraPreset) -> None:
|
||||
"""Front / side / top / isometric / asset default."""
|
||||
raise NotImplementedError(f"{_STAGE}: apply a camera preset")
|
||||
def reset(self, bounds_sensor_m: F32) -> None:
|
||||
"""Back to the framed default view."""
|
||||
self.yaw_rad = 0.0
|
||||
self.pitch_rad = 0.0
|
||||
self.fit(bounds_sensor_m, self.aspect)
|
||||
|
||||
# ------------------------------------------------------------------ matrices
|
||||
def eye_sensor_m(self) -> np.ndarray:
|
||||
return self.target_sensor_m + self._direction() * self.distance_m
|
||||
|
||||
def matrices(self, viewport: tuple[int, int]) -> CameraMatrices:
|
||||
"""``p_camera = view @ p_sensor``, ``p_clip = projection @ p_camera``."""
|
||||
raise NotImplementedError(f"{_STAGE}: build the view and projection matrices")
|
||||
"""``view`` and ``projection`` for the given physical viewport size."""
|
||||
width = max(int(viewport[0]), 1)
|
||||
height = max(int(viewport[1]), 1)
|
||||
aspect = width / height
|
||||
self.aspect = aspect
|
||||
eye = self.eye_sensor_m()
|
||||
# ``_direction()`` points from the target towards the eye, i.e. it is the
|
||||
# camera's *backward* axis. The camera looks along ``-backward``, and the
|
||||
# right vector is ``cross(-backward, up)`` — building it as
|
||||
# ``cross(backward, up)`` yields a left-handed basis (det = -1), which
|
||||
# mirrors the whole model.
|
||||
backward = self._direction()
|
||||
up = np.asarray(self.up_sensor, dtype=np.float64)
|
||||
right = np.cross(up, backward)
|
||||
norm = float(np.linalg.norm(right))
|
||||
if norm < 1e-8:
|
||||
# Looking straight down the up axis: pick any stable right vector.
|
||||
right = np.array([1.0, 0.0, 0.0])
|
||||
norm = 1.0
|
||||
right = right / norm
|
||||
true_up = np.cross(backward, right)
|
||||
view = np.eye(4, dtype=np.float64)
|
||||
view[0, :3], view[1, :3], view[2, :3] = right, true_up, backward
|
||||
view[0, 3] = -float(np.dot(right, eye))
|
||||
view[1, 3] = -float(np.dot(true_up, eye))
|
||||
view[2, 3] = -float(np.dot(backward, eye))
|
||||
near, far = self.depth_range_m()
|
||||
projection = _perspective(self.vertical_fov_rad, aspect, near, far)
|
||||
return CameraMatrices(
|
||||
view=freeze(view, dtype=np.float32),
|
||||
projection=freeze(projection, dtype=np.float32),
|
||||
)
|
||||
|
||||
def depth_range_m(self) -> tuple[float, float]:
|
||||
"""The frustum's near and far planes, for tests and diagnostics.
|
||||
|
||||
The near plane sits just in front of the scene rather than at a fraction of
|
||||
the distance: a near plane orders of magnitude closer than the model throws
|
||||
away every bit of depth resolution — everything lands at NDC z ~ 0.99 and
|
||||
the housing z-fights the plate.
|
||||
"""
|
||||
near = max(
|
||||
self.distance_m - self._fitted_radius_m * 1.5,
|
||||
self.distance_m * 0.05,
|
||||
MIN_DISTANCE_M * 0.05,
|
||||
)
|
||||
far = max(self.distance_m + self._fitted_radius_m * 4.0, near * 2.0)
|
||||
return (near, far)
|
||||
|
||||
# ------------------------------------------------------------------- helpers
|
||||
def _direction(self) -> np.ndarray:
|
||||
"""Unit vector from the target towards the eye."""
|
||||
cos_pitch = math.cos(self.pitch_rad)
|
||||
return np.array(
|
||||
[
|
||||
cos_pitch * math.sin(self.yaw_rad),
|
||||
-cos_pitch * math.cos(self.yaw_rad),
|
||||
math.sin(self.pitch_rad),
|
||||
],
|
||||
dtype=np.float64,
|
||||
)
|
||||
|
||||
def _basis(self) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""``(right, up)`` of the camera, in the same handedness as ``matrices``."""
|
||||
backward = self._direction()
|
||||
up = np.asarray(self.up_sensor, dtype=np.float64)
|
||||
right = np.cross(up, backward)
|
||||
norm = float(np.linalg.norm(right))
|
||||
if norm < 1e-8:
|
||||
right = np.array([1.0, 0.0, 0.0])
|
||||
norm = 1.0
|
||||
right = right / norm
|
||||
true_up = np.cross(backward, right)
|
||||
return right, true_up / max(float(np.linalg.norm(true_up)), 1e-9)
|
||||
|
||||
|
||||
def _clamp(value: float, low: float, high: float) -> float:
|
||||
return max(low, min(high, float(value)))
|
||||
|
||||
|
||||
def _perspective(fov_rad: float, aspect: float, near: float, far: float) -> np.ndarray:
|
||||
"""Right-handed perspective, OpenGL clip convention (z in [-1, 1])."""
|
||||
f = 1.0 / math.tan(0.5 * fov_rad)
|
||||
matrix = np.zeros((4, 4), dtype=np.float64)
|
||||
matrix[0, 0] = f / aspect
|
||||
matrix[1, 1] = f
|
||||
matrix[2, 2] = (far + near) / (near - far)
|
||||
matrix[2, 3] = (2.0 * far * near) / (near - far)
|
||||
matrix[3, 2] = -1.0
|
||||
return matrix
|
||||
|
||||
@@ -9,29 +9,57 @@ Import discipline, and why it is not negotiable:
|
||||
* the PyOpenGL import happens inside :meth:`GLDispatch.bind_current_context`,
|
||||
after Qt has made a real context current, so the binding is chosen for the
|
||||
platform Qt actually used (GLX or EGL) instead of being guessed;
|
||||
* ``GLRenderer.release`` is idempotent and requires the owning context to be
|
||||
* :meth:`GLRenderer.release` is idempotent and requires the owning context to be
|
||||
current — resources are freed by the context that made them.
|
||||
|
||||
The probe is what decides the profile on a board. A missing GL, a software
|
||||
renderer, or a failed shader compile means the three-dimensional view reports
|
||||
itself unavailable and the two-dimensional application carries on unchanged.
|
||||
Two parts, two programs, one draw call per part. The sensing surface is re-uploaded
|
||||
per frame and its colour comes from ``scalar`` through a LUT texture with the range
|
||||
as uniforms, so Python never builds an RGB array per frame; the housing is uploaded
|
||||
once and never touched again. A frame with no measurement (``valid == 0``) is drawn
|
||||
in the neutral colour rather than in colour zero — the reference model is not a
|
||||
measurement and must not look like one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
import math
|
||||
import time
|
||||
from typing import Any, Optional
|
||||
|
||||
import numpy as np
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraMatrices,
|
||||
DisplaySettings,
|
||||
GLCapabilities,
|
||||
MeshPart,
|
||||
Profile,
|
||||
RenderFrame,
|
||||
RenderStats,
|
||||
SceneAssets,
|
||||
diagnostic,
|
||||
error,
|
||||
freeze,
|
||||
)
|
||||
from orisys_app_shared.scene3d.contracts import Diagnostic
|
||||
|
||||
#: Renderer names that are not a GPU. A board must not be signed off on one of
|
||||
#: these, and neither must a laptop with a broken driver.
|
||||
SOFTWARE_RENDERER_MARKERS = (
|
||||
"llvmpipe",
|
||||
"softpipe",
|
||||
"swrast",
|
||||
"swiftshader",
|
||||
"microsoft basic render",
|
||||
"gdi generic",
|
||||
"mesa offscreen",
|
||||
)
|
||||
|
||||
_STAGE = "stage 1 (platform PoC)"
|
||||
#: The neutral colour an invalid or unmeasured vertex gets (linear, dark grey).
|
||||
INVALID_COLOR = (0.18, 0.18, 0.20, 1.0)
|
||||
|
||||
#: LUT size. 256 is enough for a sequential ramp and cheap to rebuild.
|
||||
LUT_SIZE = 256
|
||||
|
||||
|
||||
def configure_surface_format(profile: Profile = "desktop33") -> None:
|
||||
@@ -63,28 +91,474 @@ def configure_surface_format(profile: Profile = "desktop33") -> None:
|
||||
QSurfaceFormat.setDefaultFormat(fmt)
|
||||
|
||||
|
||||
def is_software_renderer(vendor: str, renderer: str) -> bool:
|
||||
"""Whether these GL strings describe a rasteriser rather than a GPU."""
|
||||
haystack = f"{vendor} {renderer}".lower()
|
||||
return any(marker in haystack for marker in SOFTWARE_RENDERER_MARKERS)
|
||||
|
||||
|
||||
#: ``QSurfaceFormat.RenderableType``. Spelled out because guessing it wrong is
|
||||
#: invisible on a driver that accepts both shader dialects (NVIDIA does, through
|
||||
#: ``ARB_ES3_compatibility``) and fails on one that does not.
|
||||
RENDERABLE_OPENGL = 1
|
||||
RENDERABLE_OPENGL_ES = 2
|
||||
|
||||
|
||||
def profile_from_renderable_type(value: int) -> Profile:
|
||||
"""Map Qt's renderable type to the shader dialect that must be compiled."""
|
||||
return "gles30" if int(value) == RENDERABLE_OPENGL_ES else "desktop33"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- shaders
|
||||
#: Shared bodies: the version header is the only difference, so the maths cannot
|
||||
#: drift between the desktop and the embedded build.
|
||||
_VERTEX_BODY = """
|
||||
layout(location = 0) in vec3 a_position;
|
||||
layout(location = 1) in vec3 a_normal;
|
||||
layout(location = 2) in float a_scalar;
|
||||
layout(location = 3) in float a_valid;
|
||||
|
||||
uniform mat4 u_view;
|
||||
uniform mat4 u_projection;
|
||||
|
||||
out vec3 v_normal;
|
||||
out float v_scalar;
|
||||
out float v_valid;
|
||||
|
||||
void main() {
|
||||
v_normal = a_normal;
|
||||
v_scalar = a_scalar;
|
||||
v_valid = a_valid;
|
||||
gl_Position = u_projection * u_view * vec4(a_position, 1.0);
|
||||
}
|
||||
"""
|
||||
|
||||
_SCALAR_FRAGMENT_BODY = """
|
||||
in vec3 v_normal;
|
||||
in float v_scalar;
|
||||
in float v_valid;
|
||||
|
||||
uniform sampler2D u_lut;
|
||||
uniform float u_lower;
|
||||
uniform float u_upper;
|
||||
uniform vec4 u_invalid_color;
|
||||
|
||||
out vec4 frag_color;
|
||||
|
||||
void main() {
|
||||
if (v_valid < 0.5 || isnan(v_scalar)) {
|
||||
frag_color = u_invalid_color;
|
||||
return;
|
||||
}
|
||||
float span = max(u_upper - u_lower, 1e-12);
|
||||
float t = clamp((v_scalar - u_lower) / span, 0.0, 1.0);
|
||||
frag_color = vec4(texture(u_lut, vec2(t, 0.5)).rgb, 1.0);
|
||||
}
|
||||
"""
|
||||
|
||||
_DIFFUSE_FRAGMENT_BODY = """
|
||||
in vec3 v_normal;
|
||||
in float v_scalar;
|
||||
in float v_valid;
|
||||
|
||||
uniform vec3 u_light_direction;
|
||||
uniform vec4 u_base_color;
|
||||
|
||||
out vec4 frag_color;
|
||||
|
||||
void main() {
|
||||
vec3 n = normalize(v_normal);
|
||||
if (!gl_FrontFacing) {
|
||||
n = -n;
|
||||
}
|
||||
float lambert = max(dot(n, normalize(u_light_direction)), 0.0);
|
||||
frag_color = vec4(u_base_color.rgb * (0.35 + 0.65 * lambert), u_base_color.a);
|
||||
}
|
||||
"""
|
||||
|
||||
_HEADERS: dict[Profile, str] = {
|
||||
"desktop33": "#version 330 core\n",
|
||||
"gles30": "#version 300 es\nprecision highp float;\nprecision highp int;\n",
|
||||
}
|
||||
|
||||
|
||||
def shader_header(profile: Profile) -> str:
|
||||
"""The dialect header for a profile. Public so a test can compare bodies."""
|
||||
try:
|
||||
return _HEADERS[profile]
|
||||
except KeyError: # pragma: no cover - guarded by configure_surface_format
|
||||
raise ValueError(f"unknown GL profile: {profile!r}") from None
|
||||
|
||||
|
||||
def shader_source(profile: Profile, program: str) -> tuple[str, str]:
|
||||
"""``(vertex, fragment)`` source for a profile and a program name.
|
||||
|
||||
Pure text: testable without a context, so "the ES build really is ES" is an
|
||||
assertion rather than a hope.
|
||||
"""
|
||||
header = shader_header(profile)
|
||||
if program == "scalar":
|
||||
body = _SCALAR_FRAGMENT_BODY
|
||||
elif program == "diffuse":
|
||||
body = _DIFFUSE_FRAGMENT_BODY
|
||||
else:
|
||||
raise ValueError(f"unknown shader program: {program!r}")
|
||||
return header + _VERTEX_BODY, header + body
|
||||
|
||||
|
||||
def lut_rgba(lut_id: str = "sequential") -> np.ndarray:
|
||||
"""A 256x1 RGBA8 lookup table.
|
||||
|
||||
Hand-rolled rather than imported: the runtime must not grow a plotting
|
||||
dependency for one texture, and the ramp is part of what the colours *mean*.
|
||||
``sequential`` is perceptually increasing in lightness (safe for magnitude);
|
||||
``diverging`` is the same ramp mirrored through a neutral midpoint (safe for a
|
||||
signed quantity, and never used for an unsigned one).
|
||||
"""
|
||||
stops = np.array(
|
||||
[
|
||||
[0.267, 0.005, 0.329],
|
||||
[0.283, 0.141, 0.458],
|
||||
[0.254, 0.265, 0.530],
|
||||
[0.207, 0.372, 0.553],
|
||||
[0.164, 0.471, 0.558],
|
||||
[0.128, 0.567, 0.551],
|
||||
[0.135, 0.659, 0.518],
|
||||
[0.267, 0.749, 0.441],
|
||||
[0.478, 0.821, 0.318],
|
||||
[0.741, 0.873, 0.150],
|
||||
[0.993, 0.906, 0.144],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
if lut_id == "diverging":
|
||||
stops = np.concatenate([stops[::-1], stops], axis=0)
|
||||
positions = np.linspace(0.0, 1.0, stops.shape[0], dtype=np.float32)
|
||||
grid = np.linspace(0.0, 1.0, LUT_SIZE, dtype=np.float32)
|
||||
rgba = np.empty((LUT_SIZE, 1, 4), dtype=np.uint8)
|
||||
for channel in range(3):
|
||||
rgba[:, 0, channel] = np.clip(
|
||||
np.interp(grid, positions, stops[:, channel]) * 255.0, 0, 255
|
||||
).astype(np.uint8)
|
||||
rgba[:, 0, 3] = 255
|
||||
return rgba
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- vertex packing
|
||||
#: Floats per vertex in the interleaved buffer: position(3) + normal(3) + scalar(1)
|
||||
#: + valid(1). One buffer per part, so a per-frame upload is one glBufferSubData.
|
||||
STRIDE_FLOATS = 8
|
||||
|
||||
|
||||
def pack_part(part: MeshPart, *, scalar: Optional[np.ndarray] = None,
|
||||
valid: Optional[np.ndarray] = None) -> np.ndarray:
|
||||
"""Interleave one part into the layout the shaders read.
|
||||
|
||||
The static case (no scalar yet) marks every vertex invalid, which the shader
|
||||
draws as the neutral colour: a scene that has never received a measurement
|
||||
must not be coloured as if it had.
|
||||
"""
|
||||
count = int(part.positions_sensor_m.shape[0])
|
||||
rows = np.empty((count, STRIDE_FLOATS), dtype=np.float32)
|
||||
rows[:, 0:3] = np.asarray(part.positions_sensor_m, dtype=np.float32)
|
||||
rows[:, 3:6] = np.asarray(part.normals_sensor, dtype=np.float32)
|
||||
if scalar is None:
|
||||
rows[:, 6] = np.nan
|
||||
else:
|
||||
values = np.asarray(scalar, dtype=np.float32).reshape(-1)
|
||||
if values.size != count:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
f"{part.part_id}: {values.size} scalars for {count} vertices",
|
||||
)
|
||||
rows[:, 6] = values
|
||||
if valid is None:
|
||||
rows[:, 7] = 0.0
|
||||
else:
|
||||
flags = np.asarray(valid, dtype=bool).reshape(-1)
|
||||
if flags.size != count:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
f"{part.part_id}: {flags.size} validity flags for {count} vertices",
|
||||
)
|
||||
rows[:, 7] = flags.astype(np.float32)
|
||||
return freeze(rows, dtype=np.float32)
|
||||
|
||||
|
||||
def pack_scene_part(part: MeshPart) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""``(vertices, indices)`` as the GPU wants them, both read-only."""
|
||||
indices = np.asarray(part.triangles, dtype=np.uint32).reshape(-1)
|
||||
return pack_part(part), freeze(indices, dtype=np.uint32)
|
||||
|
||||
|
||||
def estimate_gpu_bytes(assets: SceneAssets, *, dynamic_part_id: str) -> int:
|
||||
"""A rough static-plus-one-dynamic budget, for the stats line."""
|
||||
total = 0
|
||||
for part in assets.parts:
|
||||
vertices = int(part.positions_sensor_m.shape[0])
|
||||
indices = int(part.triangles.size)
|
||||
total += vertices * STRIDE_FLOATS * 4 + indices * 4
|
||||
if part.part_id == dynamic_part_id:
|
||||
# A second copy for the frame in flight.
|
||||
total += vertices * STRIDE_FLOATS * 4
|
||||
total += LUT_SIZE * 4
|
||||
return total
|
||||
|
||||
|
||||
def rest_surface(assets: SceneAssets, identity: Any) -> RenderFrame:
|
||||
"""The asset's own geometry, with nothing measured.
|
||||
|
||||
Used to *clear* a measurement — after a refused frame, on connect, or on a
|
||||
disconnect — so the screen cannot keep showing the previous acquisition's
|
||||
colours as if they were current. ``quality="display_only"`` and an
|
||||
``unavailable`` quantity are what tell the UI to hide the measurement bar.
|
||||
"""
|
||||
from orisys_app_shared.scene3d.contracts import rest_descriptor
|
||||
|
||||
part = assets.sensing_part()
|
||||
count = int(part.positions_sensor_m.shape[0])
|
||||
return RenderFrame(
|
||||
identity=identity,
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=part.part_id,
|
||||
positions_sensor_m=part.positions_sensor_m,
|
||||
normals_sensor=part.normals_sensor,
|
||||
scalar=freeze(np.full(count, np.nan, dtype=np.float32), dtype=np.float32),
|
||||
valid=freeze(np.zeros(count, dtype=bool), dtype=np.bool_),
|
||||
scalar_descriptor=rest_descriptor(),
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- dispatch
|
||||
class GLDispatch:
|
||||
"""Bind the GL entry points for the context Qt actually created."""
|
||||
|
||||
def bind_current_context(self, context: Any) -> GLCapabilities:
|
||||
raise NotImplementedError(f"{_STAGE}: probe the context and bind PyOpenGL")
|
||||
"""Read the real context. Raises ``gl_unsupported`` when there is none.
|
||||
|
||||
Nothing here creates a second context or an EGL surface: Qt owns the
|
||||
window and the context, and this only asks it what it got.
|
||||
"""
|
||||
try:
|
||||
from OpenGL import GL
|
||||
except Exception as exc: # noqa: BLE001 - the binding may be missing
|
||||
raise error(
|
||||
"gl_unsupported",
|
||||
f"the OpenGL binding could not be imported: {type(exc).__name__}",
|
||||
) from exc
|
||||
if context is None or not context.isValid():
|
||||
raise error("gl_unsupported", "no current OpenGL context")
|
||||
fmt = context.format()
|
||||
profile = profile_from_renderable_type(int(fmt.renderableType()))
|
||||
try:
|
||||
vendor = _gl_string(GL, GL.GL_VENDOR)
|
||||
renderer = _gl_string(GL, GL.GL_RENDERER)
|
||||
gl_version = _gl_string(GL, GL.GL_VERSION)
|
||||
glsl_version = _gl_string(GL, GL.GL_SHADING_LANGUAGE_VERSION)
|
||||
max_texture = int(GL.glGetIntegerv(GL.GL_MAX_TEXTURE_SIZE))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise error(
|
||||
"gl_unsupported",
|
||||
f"the context answered no queries: {type(exc).__name__}: {exc}",
|
||||
) from exc
|
||||
from PyQt5.QtGui import QGuiApplication
|
||||
|
||||
platform_name = ""
|
||||
try:
|
||||
app = QGuiApplication.instance()
|
||||
platform_name = app.platformName() if app is not None else ""
|
||||
except Exception: # noqa: BLE001
|
||||
platform_name = ""
|
||||
software = is_software_renderer(vendor, renderer)
|
||||
diagnostics: tuple[Diagnostic, ...] = ()
|
||||
if software:
|
||||
diagnostics = (
|
||||
diagnostic(
|
||||
"gl_unsupported",
|
||||
f"{renderer or vendor} is a software rasteriser, not a GPU",
|
||||
vendor=vendor,
|
||||
renderer=renderer,
|
||||
),
|
||||
)
|
||||
elif gl_version:
|
||||
major = _leading_int(gl_version)
|
||||
if major and major < 3:
|
||||
diagnostics = (
|
||||
diagnostic(
|
||||
"gl_unsupported",
|
||||
f"OpenGL {gl_version} is older than the 3.3 the shaders need",
|
||||
version=gl_version,
|
||||
),
|
||||
)
|
||||
result = GLCapabilities(
|
||||
profile=profile,
|
||||
vendor=vendor,
|
||||
renderer=renderer,
|
||||
gl_version=gl_version,
|
||||
glsl_version=glsl_version,
|
||||
qt_platform=platform_name,
|
||||
software_renderer=software,
|
||||
max_texture_size=max_texture,
|
||||
diagnostics=diagnostics,
|
||||
)
|
||||
if any(item.severity == "error" for item in diagnostics):
|
||||
raise ThreeDUnsupported(result)
|
||||
return result
|
||||
|
||||
|
||||
class ThreeDUnsupported(Exception):
|
||||
"""Raised with the capabilities that were obtained, so they can be shown."""
|
||||
|
||||
def __init__(self, capabilities: GLCapabilities) -> None:
|
||||
super().__init__("; ".join(item.message for item in capabilities.diagnostics))
|
||||
self.capabilities = capabilities
|
||||
self.diagnostic = (
|
||||
capabilities.diagnostics[0]
|
||||
if capabilities.diagnostics
|
||||
else diagnostic("gl_unsupported", "the context is not usable")
|
||||
)
|
||||
|
||||
|
||||
def _gl_string(GL: Any, name: int) -> str:
|
||||
value = GL.glGetString(name)
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, bytes):
|
||||
return value.decode("utf-8", "replace")
|
||||
return str(value)
|
||||
|
||||
|
||||
def _leading_int(text: str) -> int:
|
||||
digits = ""
|
||||
for char in str(text):
|
||||
if char.isdigit() or (char == "." and digits):
|
||||
if char == ".":
|
||||
break
|
||||
digits += char
|
||||
elif digits:
|
||||
break
|
||||
try:
|
||||
return int(digits) if digits else 0
|
||||
except ValueError: # pragma: no cover - digits only
|
||||
return 0
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- renderer
|
||||
class _PartBuffers:
|
||||
"""One part's GPU state. Kept dumb on purpose: no Qt, no ownership games."""
|
||||
|
||||
__slots__ = ("part_id", "role", "vao", "vbo", "ebo", "triangles", "vertices")
|
||||
|
||||
def __init__(self, part: MeshPart, vao: int, vbo: int, ebo: int) -> None:
|
||||
self.part_id = part.part_id
|
||||
self.role = part.role
|
||||
self.vao = vao
|
||||
self.vbo = vbo
|
||||
self.ebo = ebo
|
||||
self.triangles = int(part.triangles.shape[0])
|
||||
self.vertices = int(part.positions_sensor_m.shape[0])
|
||||
|
||||
|
||||
class GLRenderer:
|
||||
"""Shaders, buffers and draw calls for one scene.
|
||||
"""Shaders, buffers and draw calls for one scene."""
|
||||
|
||||
Two parts, two materials, one draw call per part: the sensing surface is
|
||||
re-uploaded per frame, the housing is uploaded once and then left alone.
|
||||
"""
|
||||
def __init__(self) -> None:
|
||||
self._capabilities: Optional[GLCapabilities] = None
|
||||
self._programs: dict[str, int] = {}
|
||||
self._uniforms: dict[str, dict[str, int]] = {}
|
||||
self._parts: dict[str, _PartBuffers] = {}
|
||||
self._lut: int = 0
|
||||
self._lut_id: str = ""
|
||||
self._sensing_part_id: str = ""
|
||||
self._estimated_gpu_bytes: int = 0
|
||||
self._initialized = False
|
||||
self._displayed_frames = 0
|
||||
self._last_upload_ms = 0.0
|
||||
|
||||
# ------------------------------------------------------------------ lifetime
|
||||
def initialize(self, capabilities: GLCapabilities) -> None:
|
||||
raise NotImplementedError(f"{_STAGE}: compile the shaders for this profile")
|
||||
"""Compile the two programs for this profile. Requires a current context."""
|
||||
from OpenGL import GL
|
||||
|
||||
if self._initialized:
|
||||
# Qt may initialize the widget and then have it re-parented into a new
|
||||
# context; releasing first is what keeps that from leaking a program
|
||||
# per rebuild.
|
||||
self.release()
|
||||
self._capabilities = capabilities
|
||||
try:
|
||||
self._programs["scalar"] = _compile_program(GL, capabilities.profile, "scalar")
|
||||
self._programs["diffuse"] = _compile_program(GL, capabilities.profile, "diffuse")
|
||||
except Exception as exc: # noqa: BLE001
|
||||
self.release()
|
||||
raise error(
|
||||
"shader_compile_failed",
|
||||
f"{type(exc).__name__}: {exc}",
|
||||
profile=capabilities.profile,
|
||||
) from exc
|
||||
self._uniforms = {
|
||||
name: _uniform_locations(GL, program) for name, program in self._programs.items()
|
||||
}
|
||||
self._initialized = True
|
||||
|
||||
def upload_scene(self, assets: SceneAssets) -> None:
|
||||
raise NotImplementedError(f"{_STAGE}: upload static buffers and the LUT")
|
||||
"""Upload every part once. The housing is never uploaded again."""
|
||||
from OpenGL import GL
|
||||
|
||||
self._require_initialized()
|
||||
self.release_parts()
|
||||
self._sensing_part_id = assets.sensing_part().part_id
|
||||
for part in assets.parts:
|
||||
vertices, indices = pack_scene_part(part)
|
||||
vao = int(GL.glGenVertexArrays(1))
|
||||
vbo = int(GL.glGenBuffers(1))
|
||||
ebo = int(GL.glGenBuffers(1))
|
||||
GL.glBindVertexArray(vao)
|
||||
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, vbo)
|
||||
GL.glBufferData(
|
||||
GL.GL_ARRAY_BUFFER, vertices.nbytes, vertices, GL.GL_STATIC_DRAW
|
||||
)
|
||||
GL.glBindBuffer(GL.GL_ELEMENT_ARRAY_BUFFER, ebo)
|
||||
GL.glBufferData(
|
||||
GL.GL_ELEMENT_ARRAY_BUFFER, indices.nbytes, indices, GL.GL_STATIC_DRAW
|
||||
)
|
||||
_bind_attributes(GL)
|
||||
GL.glBindVertexArray(0)
|
||||
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, 0)
|
||||
GL.glBindBuffer(GL.GL_ELEMENT_ARRAY_BUFFER, 0)
|
||||
self._parts[part.part_id] = _PartBuffers(part, vao, vbo, ebo)
|
||||
self._estimated_gpu_bytes = estimate_gpu_bytes(
|
||||
assets, dynamic_part_id=self._sensing_part_id
|
||||
)
|
||||
|
||||
def update_surface(self, frame: RenderFrame) -> None:
|
||||
raise NotImplementedError(f"{_STAGE}: update the dynamic surface buffers")
|
||||
"""Update the sensing part's vertices in place: no index rebuild, ever."""
|
||||
from OpenGL import GL
|
||||
|
||||
self._require_initialized()
|
||||
buffers = self._parts.get(frame.sensing_part_id)
|
||||
if buffers is None:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
f"the scene has no part {frame.sensing_part_id!r} to update",
|
||||
)
|
||||
rows = pack_part(
|
||||
_PartLayout(frame.sensing_part_id, frame.positions_sensor_m, frame.normals_sensor),
|
||||
scalar=frame.scalar,
|
||||
valid=frame.valid,
|
||||
)
|
||||
if int(rows.shape[0]) != buffers.vertices:
|
||||
raise error(
|
||||
"geometry_mismatch",
|
||||
f"{buffers.part_id}: {rows.shape[0]} vertices for "
|
||||
f"{buffers.vertices} uploaded",
|
||||
)
|
||||
started = time.perf_counter()
|
||||
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, buffers.vbo)
|
||||
GL.glBufferSubData(GL.GL_ARRAY_BUFFER, 0, rows.nbytes, rows)
|
||||
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, 0)
|
||||
self._last_upload_ms = (time.perf_counter() - started) * 1000.0
|
||||
|
||||
def draw(
|
||||
self,
|
||||
@@ -93,7 +567,240 @@ class GLRenderer:
|
||||
framebuffer: int,
|
||||
viewport: tuple[int, int],
|
||||
) -> RenderStats:
|
||||
raise NotImplementedError(f"{_STAGE}: draw one frame")
|
||||
"""One scene draw into the framebuffer Qt owns (never target 0)."""
|
||||
from OpenGL import GL
|
||||
|
||||
self._require_initialized()
|
||||
started = time.perf_counter()
|
||||
width = max(int(viewport[0]), 1)
|
||||
height = max(int(viewport[1]), 1)
|
||||
GL.glBindFramebuffer(GL.GL_FRAMEBUFFER, int(framebuffer))
|
||||
GL.glViewport(0, 0, width, height)
|
||||
GL.glEnable(GL.GL_DEPTH_TEST)
|
||||
GL.glClearColor(0.06, 0.07, 0.09, 1.0)
|
||||
GL.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT)
|
||||
view = np.asarray(camera.view, dtype=np.float32).reshape(4, 4)
|
||||
projection = np.asarray(camera.projection, dtype=np.float32).reshape(4, 4)
|
||||
scale = settings.color_scale
|
||||
lower = float(scale.lower) if scale is not None else 0.0
|
||||
upper = float(scale.upper) if scale is not None else 1.0
|
||||
if upper <= lower:
|
||||
upper = lower + 1.0
|
||||
self._ensure_lut("diverging" if (scale and scale.descriptor.signed) else "sequential")
|
||||
for part in self._parts.values():
|
||||
if part.role == "housing" and not settings.housing_visible:
|
||||
continue
|
||||
program = self._programs["scalar" if part.role == "sensing" else "diffuse"]
|
||||
locations = self._uniforms["scalar" if part.role == "sensing" else "diffuse"]
|
||||
GL.glUseProgram(program)
|
||||
_set_mat4(GL, locations.get("u_view"), view)
|
||||
_set_mat4(GL, locations.get("u_projection"), projection)
|
||||
if part.role == "sensing":
|
||||
_set_float(GL, locations.get("u_lower"), lower)
|
||||
_set_float(GL, locations.get("u_upper"), upper)
|
||||
_set_vec4(GL, locations.get("u_invalid_color"), INVALID_COLOR)
|
||||
if locations.get("u_lut") is not None:
|
||||
GL.glActiveTexture(GL.GL_TEXTURE0)
|
||||
GL.glBindTexture(GL.GL_TEXTURE_2D, self._lut)
|
||||
GL.glUniform1i(locations["u_lut"], 0)
|
||||
else:
|
||||
_set_vec3(GL, locations.get("u_light_direction"), (0.3, -0.5, 0.8))
|
||||
_set_vec4(GL, locations.get("u_base_color"), _base_color(part.part_id))
|
||||
GL.glBindVertexArray(part.vao)
|
||||
GL.glDrawElements(
|
||||
GL.GL_TRIANGLES, part.triangles * 3, GL.GL_UNSIGNED_INT, None
|
||||
)
|
||||
GL.glBindVertexArray(0)
|
||||
GL.glUseProgram(0)
|
||||
self._displayed_frames += 1
|
||||
return RenderStats(
|
||||
displayed_frames=self._displayed_frames,
|
||||
compose_ms=0.0,
|
||||
upload_ms=self._last_upload_ms,
|
||||
paint_ms=(time.perf_counter() - started) * 1000.0,
|
||||
estimated_gpu_bytes=self._estimated_gpu_bytes,
|
||||
)
|
||||
|
||||
def release(self) -> None:
|
||||
raise NotImplementedError(f"{_STAGE}: release GPU resources")
|
||||
"""Delete every GPU resource. Idempotent; the owning context must be current."""
|
||||
from OpenGL import GL
|
||||
|
||||
self.release_parts()
|
||||
for program in self._programs.values():
|
||||
try:
|
||||
GL.glDeleteProgram(int(program))
|
||||
except Exception: # noqa: BLE001 - nothing left to delete
|
||||
pass
|
||||
self._programs = {}
|
||||
self._uniforms = {}
|
||||
if self._lut:
|
||||
try:
|
||||
GL.glDeleteTextures([int(self._lut)])
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
self._lut = 0
|
||||
self._lut_id = ""
|
||||
self._initialized = False
|
||||
self._estimated_gpu_bytes = 0
|
||||
self._displayed_frames = 0
|
||||
|
||||
def release_parts(self) -> None:
|
||||
from OpenGL import GL
|
||||
|
||||
for part in self._parts.values():
|
||||
try:
|
||||
GL.glDeleteVertexArrays(1, [int(part.vao)])
|
||||
GL.glDeleteBuffers(1, [int(part.vbo)])
|
||||
GL.glDeleteBuffers(1, [int(part.ebo)])
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
self._parts = {}
|
||||
|
||||
# ------------------------------------------------------------------- helpers
|
||||
def _require_initialized(self) -> None:
|
||||
if not self._initialized:
|
||||
raise error("gl_unsupported", "the renderer was not initialized")
|
||||
|
||||
def _ensure_lut(self, lut_id: str) -> None:
|
||||
from OpenGL import GL
|
||||
|
||||
if self._lut and self._lut_id == lut_id:
|
||||
return
|
||||
if self._lut:
|
||||
GL.glDeleteTextures([int(self._lut)])
|
||||
self._lut = 0
|
||||
data = lut_rgba(lut_id)
|
||||
self._lut = int(GL.glGenTextures(1))
|
||||
GL.glBindTexture(GL.GL_TEXTURE_2D, self._lut)
|
||||
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR)
|
||||
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR)
|
||||
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP_TO_EDGE)
|
||||
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP_TO_EDGE)
|
||||
GL.glTexImage2D(
|
||||
GL.GL_TEXTURE_2D, 0, GL.GL_RGBA8, LUT_SIZE, 1, 0, GL.GL_RGBA,
|
||||
GL.GL_UNSIGNED_BYTE, data,
|
||||
)
|
||||
GL.glBindTexture(GL.GL_TEXTURE_2D, 0)
|
||||
self._lut_id = lut_id
|
||||
|
||||
def capabilities(self) -> Optional[GLCapabilities]:
|
||||
return self._capabilities
|
||||
|
||||
@property
|
||||
def part_ids(self) -> tuple[str, ...]:
|
||||
return tuple(self._parts)
|
||||
|
||||
@property
|
||||
def is_initialized(self) -> bool:
|
||||
return self._initialized
|
||||
|
||||
|
||||
class _PartLayout:
|
||||
"""A minimal stand-in for :class:`MeshPart` when updating an existing buffer."""
|
||||
|
||||
__slots__ = ("part_id", "positions_sensor_m", "normals_sensor")
|
||||
|
||||
def __init__(self, part_id: str, positions: np.ndarray, normals: np.ndarray) -> None:
|
||||
self.part_id = part_id
|
||||
self.positions_sensor_m = positions
|
||||
self.normals_sensor = normals
|
||||
|
||||
|
||||
#: Base colours for non-sensing parts, by part id: the housing is grey/graphite so
|
||||
#: it never competes with the measurement colour scale.
|
||||
_PART_COLORS: dict[str, tuple[float, float, float, float]] = {
|
||||
"housing": (0.32, 0.33, 0.35, 1.0),
|
||||
}
|
||||
|
||||
|
||||
def _base_color(part_id: str) -> tuple[float, float, float, float]:
|
||||
return _PART_COLORS.get(part_id, (0.45, 0.45, 0.47, 1.0))
|
||||
|
||||
|
||||
def _bind_attributes(GL: Any) -> None:
|
||||
stride = STRIDE_FLOATS * 4
|
||||
GL.glEnableVertexAttribArray(0)
|
||||
GL.glVertexAttribPointer(0, 3, GL.GL_FLOAT, GL.GL_FALSE, stride, None)
|
||||
GL.glEnableVertexAttribArray(1)
|
||||
GL.glVertexAttribPointer(1, 3, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(3))
|
||||
GL.glEnableVertexAttribArray(2)
|
||||
GL.glVertexAttribPointer(2, 1, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(6))
|
||||
GL.glEnableVertexAttribArray(3)
|
||||
GL.glVertexAttribPointer(3, 1, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(7))
|
||||
|
||||
|
||||
def _offset(floats: int) -> Any:
|
||||
from OpenGL import GL
|
||||
|
||||
return GL.ctypes.c_void_p(floats * 4)
|
||||
|
||||
|
||||
def _compile_program(GL: Any, profile: Profile, name: str) -> int:
|
||||
vertex_source, fragment_source = shader_source(profile, name)
|
||||
program = int(GL.glCreateProgram())
|
||||
shaders = []
|
||||
for kind, source in ((GL.GL_VERTEX_SHADER, vertex_source), (GL.GL_FRAGMENT_SHADER, fragment_source)):
|
||||
shader = int(GL.glCreateShader(kind))
|
||||
GL.glShaderSource(shader, source)
|
||||
GL.glCompileShader(shader)
|
||||
if not GL.glGetShaderiv(shader, GL.GL_COMPILE_STATUS):
|
||||
log = GL.glGetShaderInfoLog(shader)
|
||||
GL.glDeleteShader(shader)
|
||||
raise RuntimeError(f"{name} shader: {_decode(log)}")
|
||||
GL.glAttachShader(program, shader)
|
||||
shaders.append(shader)
|
||||
GL.glLinkProgram(program)
|
||||
if not GL.glGetProgramiv(program, GL.GL_LINK_STATUS):
|
||||
log = GL.glGetProgramInfoLog(program)
|
||||
for shader in shaders:
|
||||
GL.glDeleteShader(shader)
|
||||
GL.glDeleteProgram(program)
|
||||
raise RuntimeError(f"{name} program: {_decode(log)}")
|
||||
for shader in shaders:
|
||||
GL.glDetachShader(program, shader)
|
||||
GL.glDeleteShader(shader)
|
||||
return program
|
||||
|
||||
|
||||
def _uniform_locations(GL: Any, program: int) -> dict[str, int]:
|
||||
names = (
|
||||
"u_view", "u_projection", "u_scalar", "u_lower", "u_upper",
|
||||
"u_invalid_color", "u_lut", "u_light_direction", "u_base_color",
|
||||
)
|
||||
out: dict[str, int] = {}
|
||||
for name in names:
|
||||
location = GL.glGetUniformLocation(program, name)
|
||||
if location is not None and int(location) >= 0:
|
||||
out[name] = int(location)
|
||||
return out
|
||||
|
||||
|
||||
def _decode(value: Any) -> str:
|
||||
if isinstance(value, bytes):
|
||||
return value.decode("utf-8", "replace").strip()
|
||||
return str(value).strip()
|
||||
|
||||
|
||||
def _set_mat4(GL: Any, location: Optional[int], values: np.ndarray) -> None:
|
||||
if location is None:
|
||||
return
|
||||
# Column-major upload, explicitly: the layout of the NumPy array must not
|
||||
# decide what the coordinates mean (design document §3.1).
|
||||
GL.glUniformMatrix4fv(location, 1, GL.GL_FALSE, np.ascontiguousarray(values.T))
|
||||
|
||||
|
||||
def _set_float(GL: Any, location: Optional[int], value: float) -> None:
|
||||
if location is not None:
|
||||
GL.glUniform1f(location, float(value))
|
||||
|
||||
|
||||
def _set_vec3(GL: Any, location: Optional[int], values: tuple[float, float, float]) -> None:
|
||||
if location is not None:
|
||||
GL.glUniform3f(location, float(values[0]), float(values[1]), float(values[2]))
|
||||
|
||||
|
||||
def _set_vec4(GL: Any, location: Optional[int], values: tuple[float, float, float, float]) -> None:
|
||||
if location is not None:
|
||||
GL.glUniform4f(
|
||||
location, float(values[0]), float(values[1]), float(values[2]), float(values[3])
|
||||
)
|
||||
|
||||
@@ -3,18 +3,24 @@
|
||||
Stage 1 (a static scene, resize, close) and stage 4 (product integration) in
|
||||
``docs/architecture/studio-3d-stages.md``.
|
||||
|
||||
Qt owns the context and the framebuffer, so the widget must never bind target 0:
|
||||
it draws into the FBO Qt has already set up for the current ``paintGL`` call. The
|
||||
widget holds CPU-side data (the latest :class:`RenderFrame`) and schedules the
|
||||
upload; it does no composing, no asset IO and no SDK access, so a slow render can
|
||||
only drop *display* frames.
|
||||
Qt owns the context and the framebuffer, so the widget never binds target 0: it
|
||||
draws into ``defaultFramebufferObject()``, which is the FBO of the current
|
||||
``paintGL`` call. The widget holds CPU-side data (the latest ``RenderFrame``) and
|
||||
schedules the upload; it does no composing, no asset IO and no SDK access, so a
|
||||
slow render can only drop *display* frames.
|
||||
|
||||
Failure is a state, not an exception: if the context is unusable (no GL, software
|
||||
rasteriser, shader failure) the widget reports it once and stays empty, while the
|
||||
two-dimensional application carries on.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from PyQt5.QtCore import pyqtSignal
|
||||
from PyQt5.QtCore import QEvent, Qt, pyqtSignal
|
||||
from PyQt5.QtGui import QMouseEvent, QWheelEvent
|
||||
from PyQt5.QtWidgets import QOpenGLWidget, QWidget
|
||||
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
@@ -24,11 +30,19 @@ from orisys_app_shared.scene3d.contracts import (
|
||||
RenderFrame,
|
||||
RenderStats,
|
||||
SceneAssets,
|
||||
diagnostic,
|
||||
)
|
||||
from orisys_studio.scene3d.camera import OrbitCamera
|
||||
from orisys_studio.scene3d.gl_backend import GLRenderer
|
||||
from orisys_studio.scene3d.gl_backend import (
|
||||
GLDispatch,
|
||||
GLRenderer,
|
||||
ThreeDUnsupported,
|
||||
rest_surface,
|
||||
)
|
||||
|
||||
_STAGE = "stage 1 (static scene) and stage 4 (product integration)"
|
||||
#: How often render statistics are published. The numbers are for a person to
|
||||
#: read, and a signal per frame would cost more than the drawing.
|
||||
STATS_INTERVAL_S = 0.5
|
||||
|
||||
|
||||
class Sensor3DWidget(QOpenGLWidget):
|
||||
@@ -43,21 +57,36 @@ class Sensor3DWidget(QOpenGLWidget):
|
||||
renderer: GLRenderer,
|
||||
camera: OrbitCamera,
|
||||
parent: Optional[QWidget] = None,
|
||||
*,
|
||||
dispatch: Optional[GLDispatch] = None,
|
||||
) -> None:
|
||||
super().__init__(parent)
|
||||
self._renderer = renderer
|
||||
self._camera = camera
|
||||
self._dispatch = dispatch or GLDispatch()
|
||||
self._assets: Optional[SceneAssets] = None
|
||||
self._frame: Optional[RenderFrame] = None
|
||||
self._settings = DisplaySettings()
|
||||
self._capabilities: Optional[GLCapabilities] = None
|
||||
self._diagnostic: Optional[Diagnostic] = None
|
||||
self._needs_scene_upload = False
|
||||
self._needs_surface_upload = False
|
||||
self._last_stats_at = 0.0
|
||||
self._last_press: Optional[tuple[float, float]] = None
|
||||
self._unavailable = False
|
||||
#: Frames the display never showed, set by the coordinator. Never a
|
||||
#: recording figure.
|
||||
self._superseded_frames = 0
|
||||
#: Frames refused because they did not belong to this scene.
|
||||
self._rejected_frames = 0
|
||||
|
||||
# ------------------------------------------------------------- CPU-side API
|
||||
def set_scene(self, assets: SceneAssets) -> None:
|
||||
"""Retain the asset and schedule its upload for the next current context."""
|
||||
self._assets = assets
|
||||
self._frame = None
|
||||
self._needs_scene_upload = True
|
||||
self._unavailable = False
|
||||
self.update()
|
||||
|
||||
def present(self, frame: RenderFrame) -> None:
|
||||
@@ -70,15 +99,217 @@ class Sensor3DWidget(QOpenGLWidget):
|
||||
self._settings = settings
|
||||
self.update()
|
||||
|
||||
def camera(self) -> OrbitCamera:
|
||||
return self._camera
|
||||
|
||||
def capabilities(self) -> Optional[GLCapabilities]:
|
||||
return self._capabilities
|
||||
|
||||
def diagnostic(self) -> Optional[Diagnostic]:
|
||||
return self._diagnostic
|
||||
|
||||
@property
|
||||
def unavailable(self) -> bool:
|
||||
return self._unavailable
|
||||
|
||||
def viewport_pixels(self) -> tuple[int, int]:
|
||||
"""The physical framebuffer size: device pixels, not logical ones."""
|
||||
ratio = float(self.devicePixelRatioF() or 1.0)
|
||||
return (
|
||||
max(int(round(self.width() * ratio)), 1),
|
||||
max(int(round(self.height() * ratio)), 1),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ Qt overrides
|
||||
def initializeGL(self) -> None: # pragma: no cover - needs a real context
|
||||
raise NotImplementedError(f"{_STAGE}: bind the backend and connect teardown")
|
||||
try:
|
||||
from OpenGL import GL
|
||||
|
||||
# Both materials are double-sided by design (a thin plate seen from
|
||||
# behind must not vanish), so culling stays off.
|
||||
GL.glDisable(GL.GL_CULL_FACE)
|
||||
capabilities = self._dispatch.bind_current_context(self.context())
|
||||
self._renderer.initialize(capabilities)
|
||||
except ThreeDUnsupported as exc:
|
||||
self._fail(exc.diagnostic)
|
||||
self._capabilities = exc.capabilities
|
||||
self.capabilitiesChanged.emit(exc.capabilities)
|
||||
return
|
||||
except Exception as exc: # noqa: BLE001 - any GL failure is a state, not a crash
|
||||
self._fail(
|
||||
getattr(exc, "diagnostic", None)
|
||||
or diagnostic("gl_unsupported", f"{type(exc).__name__}: {exc}")
|
||||
)
|
||||
return
|
||||
self._capabilities = capabilities
|
||||
self._diagnostic = None
|
||||
self._unavailable = False
|
||||
# A fresh context holds nothing, whatever the CPU still has: re-upload
|
||||
# from the retained assets. Re-parenting a window rebuilds the context,
|
||||
# and without this the scene would simply vanish.
|
||||
self._needs_scene_upload = self._assets is not None
|
||||
self._needs_surface_upload = self._frame is not None
|
||||
context = self.context()
|
||||
if context is not None:
|
||||
context.aboutToBeDestroyed.connect(self.release_resources)
|
||||
self.capabilitiesChanged.emit(capabilities)
|
||||
|
||||
def paintGL(self) -> None: # pragma: no cover - needs a real context
|
||||
raise NotImplementedError(f"{_STAGE}: upload what changed and draw")
|
||||
if self._unavailable or not self._renderer.is_initialized:
|
||||
return
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
if self._needs_scene_upload and self._assets is not None:
|
||||
self._renderer.upload_scene(self._assets)
|
||||
self._needs_scene_upload = False
|
||||
self._needs_surface_upload = self._frame is not None
|
||||
# The default view belongs to the asset, not to a user preference:
|
||||
# adopt it, then frame the whole scene from that orientation. Going
|
||||
# straight to fit() would keep whatever angles the camera happened
|
||||
# to start with (looking along the sensor's long axis, which shows
|
||||
# a flat surface edge-on).
|
||||
presets = self._assets.camera_presets
|
||||
if presets:
|
||||
try:
|
||||
self._camera.set_preset(presets[0])
|
||||
except ValueError:
|
||||
pass
|
||||
self._camera.fit(self._assets.bounds_sensor_m, self._aspect())
|
||||
if self._needs_surface_upload and self._frame is not None:
|
||||
frame, self._frame = self._frame, None
|
||||
self._needs_surface_upload = False
|
||||
if self._assets is not None and frame.geometry_id != self._assets.geometry_id:
|
||||
# A frame from another geometry is *rejected*, not fatal: the
|
||||
# scene stays usable, and leaving the old measurement on screen
|
||||
# would be claiming it is the current one.
|
||||
self._reject(
|
||||
diagnostic(
|
||||
"geometry_mismatch",
|
||||
"the frame belongs to a different geometry than the scene",
|
||||
frame=frame.geometry_id,
|
||||
scene=self._assets.geometry_id,
|
||||
)
|
||||
)
|
||||
if self._assets is not None:
|
||||
self._renderer.update_surface(
|
||||
rest_surface(self._assets, frame.identity)
|
||||
)
|
||||
else:
|
||||
self._renderer.update_surface(frame)
|
||||
stats = self._renderer.draw(
|
||||
self._camera.matrices(self.viewport_pixels()),
|
||||
self._settings,
|
||||
int(self.defaultFramebufferObject()),
|
||||
self.viewport_pixels(),
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# A failure while drawing is the GL stack being unusable, not a shader
|
||||
# that failed to compile (that case raises its own diagnostic above).
|
||||
self._fail(
|
||||
getattr(exc, "diagnostic", None)
|
||||
or diagnostic("gl_unsupported", f"{type(exc).__name__}: {exc}")
|
||||
)
|
||||
return
|
||||
now = time.perf_counter()
|
||||
if now - self._last_stats_at >= STATS_INTERVAL_S:
|
||||
self._last_stats_at = now
|
||||
self.renderStatsChanged.emit(
|
||||
RenderStats(
|
||||
displayed_frames=stats.displayed_frames,
|
||||
superseded_frames=self._superseded,
|
||||
compose_ms=stats.compose_ms,
|
||||
upload_ms=stats.upload_ms,
|
||||
paint_ms=(now - started) * 1000.0,
|
||||
estimated_gpu_bytes=stats.estimated_gpu_bytes,
|
||||
)
|
||||
)
|
||||
|
||||
def resizeGL(self, w: int, h: int) -> None: # pragma: no cover
|
||||
raise NotImplementedError(f"{_STAGE}: calibrate DPR and update the projection")
|
||||
def resizeGL(self, w: int, h: int) -> None: # pragma: no cover - needs a context
|
||||
"""Nothing to cache: the projection is rebuilt from the live viewport."""
|
||||
self._camera.aspect = max(float(w), 1.0) / max(float(h), 1.0)
|
||||
|
||||
def release_resources(self) -> None: # pragma: no cover - needs a real context
|
||||
raise NotImplementedError(f"{_STAGE}: makeCurrent, release, doneCurrent")
|
||||
def release_resources(self) -> None: # pragma: no cover - needs a context
|
||||
"""Free GPU resources from this widget's own context. Safe to repeat."""
|
||||
if not self._renderer.is_initialized:
|
||||
return
|
||||
try:
|
||||
self.makeCurrent()
|
||||
self._renderer.release()
|
||||
except Exception: # noqa: BLE001 - teardown must not raise on the way out
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
self.doneCurrent()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
def hideEvent(self, event: QEvent) -> None: # pragma: no cover - Qt event
|
||||
super().hideEvent(event)
|
||||
# Hiding suspends drawing; the coordinator stops composing on this signal.
|
||||
self._needs_surface_upload = False
|
||||
|
||||
def showEvent(self, event: QEvent) -> None: # pragma: no cover - Qt event
|
||||
super().showEvent(event)
|
||||
self._needs_surface_upload = self._frame is not None
|
||||
self.update()
|
||||
|
||||
# --------------------------------------------------------------- interaction
|
||||
def mousePressEvent(self, event: QMouseEvent) -> None: # pragma: no cover
|
||||
self._last_press = (float(event.x()), float(event.y()))
|
||||
super().mousePressEvent(event)
|
||||
|
||||
def mouseReleaseEvent(self, event: QMouseEvent) -> None: # pragma: no cover
|
||||
self._last_press = None
|
||||
super().mouseReleaseEvent(event)
|
||||
|
||||
def mouseMoveEvent(self, event: QMouseEvent) -> None: # pragma: no cover
|
||||
if self._last_press is None:
|
||||
super().mouseMoveEvent(event)
|
||||
return
|
||||
x, y = float(event.x()), float(event.y())
|
||||
delta = (x - self._last_press[0], y - self._last_press[1])
|
||||
self._last_press = (x, y)
|
||||
viewport = (max(self.width(), 1), max(self.height(), 1))
|
||||
buttons = event.buttons()
|
||||
panning = bool(buttons & Qt.MiddleButton) or bool(
|
||||
buttons & Qt.LeftButton and event.modifiers() & Qt.ShiftModifier
|
||||
)
|
||||
if panning:
|
||||
self._camera.pan(delta, viewport)
|
||||
elif buttons & Qt.LeftButton:
|
||||
self._camera.orbit(delta, viewport)
|
||||
self.update()
|
||||
|
||||
def wheelEvent(self, event: QWheelEvent) -> None: # pragma: no cover
|
||||
steps = float(event.angleDelta().y()) / 120.0
|
||||
self._camera.zoom(steps)
|
||||
self.update()
|
||||
|
||||
# ------------------------------------------------------------------ internal
|
||||
@property
|
||||
def _superseded(self) -> int:
|
||||
return self._superseded_frames
|
||||
|
||||
def set_superseded_frames(self, count: int) -> None:
|
||||
"""Set by the coordinator: display drops, never recording drops."""
|
||||
self._superseded_frames = int(count)
|
||||
|
||||
def _aspect(self) -> float:
|
||||
width, height = self.viewport_pixels()
|
||||
return width / height
|
||||
|
||||
def _fail(self, item: Diagnostic) -> None:
|
||||
"""The whole view is out of action (no context, no shaders, GL error)."""
|
||||
self._unavailable = True
|
||||
self._diagnostic = item
|
||||
self.diagnosticRaised.emit(item)
|
||||
|
||||
def _reject(self, item: Diagnostic) -> None:
|
||||
"""One frame was refused. Reported, counted, and otherwise ignored."""
|
||||
self._rejected_frames += 1
|
||||
self._diagnostic = item
|
||||
self.diagnosticRaised.emit(item)
|
||||
|
||||
@property
|
||||
def rejected_frames(self) -> int:
|
||||
return self._rejected_frames
|
||||
|
||||
@@ -0,0 +1,316 @@
|
||||
"""Stage 1 acceptance: a real GL context draws the scene, and survives its life.
|
||||
|
||||
This file deliberately does **not** force ``QT_QPA_PLATFORM=offscreen``: the
|
||||
offscreen plugin cannot create a platform GL context at all ("this plugin does not
|
||||
support createPlatformOpenGLContext"), and the design document is explicit that an
|
||||
offscreen UI test cannot stand in for a GPU. Every test here skips itself when no
|
||||
real context can be obtained, and asserts on pixels otherwise.
|
||||
|
||||
Run it with a real platform, e.g.::
|
||||
|
||||
QT_QPA_PLATFORM=windows python -m pytest tests/test_scene3d_gl_smoke.py
|
||||
|
||||
Two rules the tests themselves have to respect, learned the hard way here: the
|
||||
widget is driven the way the application drives it (``show``/``repaint``, so Qt
|
||||
owns the context), and pixels are read through Qt's own ``grabFramebuffer`` —
|
||||
calling ``makeCurrent``/``initializeGL`` by hand creates the context *before* Qt
|
||||
does, and the GPU objects then belong to a context that is thrown away.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
ColorScale,
|
||||
DisplaySettings,
|
||||
FieldDescriptor,
|
||||
FrameIdentity,
|
||||
RenderFrame,
|
||||
freeze,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
os.environ.get("QT_QPA_PLATFORM", "").lower() in ("offscreen", "minimal"),
|
||||
reason="the offscreen/minimal plugin cannot create a platform GL context",
|
||||
)
|
||||
|
||||
#: The renderer's clear colour in 8-bit, so "background" can be excluded from a
|
||||
#: pixel assertion instead of dominating it.
|
||||
BACKGROUND_RGB = np.array([15, 18, 23])
|
||||
|
||||
|
||||
def _pump(qt_app, widget, predicate, *, timeout_s: float = 5.0) -> bool:
|
||||
"""Let Qt paint until ``predicate`` holds. Returns whether it did."""
|
||||
import time
|
||||
|
||||
deadline = time.perf_counter() + timeout_s
|
||||
while time.perf_counter() < deadline:
|
||||
widget.repaint()
|
||||
qt_app.processEvents()
|
||||
if predicate():
|
||||
return True
|
||||
time.sleep(0.01)
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def gl_widget(qt_app):
|
||||
"""A real, shown QOpenGLWidget with the synthetic scene, driven by Qt."""
|
||||
from orisys_studio.scene3d.camera import OrbitCamera
|
||||
from orisys_studio.scene3d.gl_backend import GLDispatch, GLRenderer, configure_surface_format
|
||||
from orisys_studio.scene3d.widget import Sensor3DWidget
|
||||
|
||||
configure_surface_format("desktop33")
|
||||
renderer = GLRenderer()
|
||||
widget = Sensor3DWidget(renderer, OrbitCamera(), dispatch=GLDispatch())
|
||||
failures: list = []
|
||||
widget.diagnosticRaised.connect(failures.append)
|
||||
widget.resize(240, 180)
|
||||
widget.show()
|
||||
widget.set_scene(synthetic.poc_scene())
|
||||
painted = _pump(qt_app, widget, lambda: bool(renderer.part_ids), timeout_s=8.0)
|
||||
if not painted:
|
||||
widget.deleteLater()
|
||||
pytest.skip(
|
||||
"no usable GL context here: "
|
||||
f"{widget.diagnostic() or 'the widget never initialized'}"
|
||||
)
|
||||
yield widget, renderer, failures
|
||||
widget.release_resources()
|
||||
widget.close()
|
||||
widget.deleteLater()
|
||||
qt_app.processEvents()
|
||||
|
||||
|
||||
def _render(widget, qt_app) -> np.ndarray:
|
||||
"""One Qt-driven paint, read back through Qt's own framebuffer grab.
|
||||
|
||||
``grabFramebuffer`` hands back a ``Format_RGB32`` image, whose bytes are BGRA
|
||||
on a little-endian machine, with a stride that can exceed the width — so the
|
||||
conversion and the stride trim are both load-bearing.
|
||||
"""
|
||||
from PyQt5.QtGui import QImage
|
||||
|
||||
widget.repaint()
|
||||
qt_app.processEvents()
|
||||
image = widget.grabFramebuffer()
|
||||
assert not image.isNull(), "Qt returned no framebuffer"
|
||||
image = image.convertToFormat(QImage.Format_RGB888)
|
||||
width, height = image.width(), image.height()
|
||||
buffer = image.constBits().asarray(image.byteCount())
|
||||
stride = image.bytesPerLine()
|
||||
rows = np.frombuffer(buffer, dtype=np.uint8).reshape(height, stride)
|
||||
return rows[:, : width * 3].reshape(height, width, 3).copy()
|
||||
|
||||
|
||||
def _drawn(pixels: np.ndarray) -> np.ndarray:
|
||||
"""Pixels that are not the clear colour, as ints."""
|
||||
flat = pixels.reshape(-1, 3).astype(int)
|
||||
background = np.abs(flat - BACKGROUND_RGB).max(axis=1) <= 3
|
||||
return flat[~background]
|
||||
|
||||
|
||||
def _plate_is_grey(pixels: np.ndarray) -> float:
|
||||
"""Mean |r-g| of the drawn pixels. The neutral path is grey, the LUT is not."""
|
||||
drawn = _drawn(pixels)
|
||||
assert drawn.size, "nothing was drawn"
|
||||
return float(np.abs(drawn[:, 0] - drawn[:, 1]).mean())
|
||||
|
||||
|
||||
def test_the_capabilities_come_from_a_real_context(gl_widget):
|
||||
widget, _, _ = gl_widget
|
||||
capabilities = widget.capabilities()
|
||||
assert capabilities is not None
|
||||
assert capabilities.vendor and capabilities.renderer
|
||||
assert capabilities.profile == "desktop33", "a desktop context must compile desktop shaders"
|
||||
assert not capabilities.software_renderer, "the dev machine must be driven by a GPU"
|
||||
assert capabilities.max_texture_size >= 1024
|
||||
assert widget.viewport_pixels() == (240, 180)
|
||||
|
||||
|
||||
def test_both_parts_are_drawn_and_the_housing_can_be_hidden(gl_widget, qt_app):
|
||||
widget, renderer, _ = gl_widget
|
||||
assert set(renderer.part_ids) == {"sensing_surface", "housing"}
|
||||
|
||||
with_housing = _drawn(_render(widget, qt_app))
|
||||
assert with_housing.size, "nothing was drawn at all"
|
||||
widget.set_settings(DisplaySettings(housing_visible=False))
|
||||
without_housing = _drawn(_render(widget, qt_app))
|
||||
assert 0 < without_housing.size < with_housing.size, (
|
||||
"hiding the housing did not reduce what is on screen "
|
||||
f"({without_housing.size} vs {with_housing.size} pixels)"
|
||||
)
|
||||
widget.set_settings(DisplaySettings(housing_visible=True))
|
||||
|
||||
|
||||
def test_the_rest_frame_is_drawn_neutral_not_as_measurement_zero(gl_widget, qt_app):
|
||||
"""No measurement yet: the plate must not be coloured as if it had one.
|
||||
|
||||
Checked as a difference: the neutral path is grey (r≈g), the colour map's low
|
||||
end is not. Counting "grey pixels" over the whole frame would just be counting
|
||||
the background.
|
||||
"""
|
||||
widget, _, _ = gl_widget
|
||||
assets = synthetic.poc_scene()
|
||||
count = int(assets.sensing_part().positions_sensor_m.shape[0])
|
||||
widget.set_settings(DisplaySettings(housing_visible=False))
|
||||
|
||||
rest_grey = _plate_is_grey(_render(widget, qt_app))
|
||||
|
||||
frame = RenderFrame(
|
||||
identity=FrameIdentity("poc", 1, 5, 0.5, generation=1),
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=assets.sensing_part().part_id,
|
||||
positions_sensor_m=assets.sensing_part().positions_sensor_m,
|
||||
normals_sensor=assets.sensing_part().normals_sensor,
|
||||
scalar=freeze(np.full(count, 0.05, dtype=np.float32), dtype=np.float32),
|
||||
valid=freeze(np.ones(count, dtype=np.bool_), dtype=np.bool_),
|
||||
scalar_descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
|
||||
)
|
||||
widget.set_settings(
|
||||
DisplaySettings(
|
||||
housing_visible=False,
|
||||
color_scale=ColorScale(
|
||||
descriptor=frame.scalar_descriptor,
|
||||
lower=0.0,
|
||||
upper=1.0,
|
||||
lut_id="sequential",
|
||||
),
|
||||
)
|
||||
)
|
||||
widget.present(frame)
|
||||
measured_grey = _plate_is_grey(_render(widget, qt_app))
|
||||
|
||||
assert rest_grey < 6.0, f"the unmeasured surface is not neutral (r-g={rest_grey:.1f})"
|
||||
assert measured_grey > rest_grey + 10.0, (
|
||||
"the colour map made no difference: the invalid path is not being taken "
|
||||
f"(rest r-g={rest_grey:.1f}, measured r-g={measured_grey:.1f})"
|
||||
)
|
||||
widget.set_settings(DisplaySettings(housing_visible=True))
|
||||
|
||||
|
||||
def test_a_frame_updates_only_the_sensing_buffers(gl_widget, qt_app):
|
||||
widget, renderer, _ = gl_widget
|
||||
assets = synthetic.poc_scene()
|
||||
count = int(assets.sensing_part().positions_sensor_m.shape[0])
|
||||
positions = np.asarray(assets.sensing_part().positions_sensor_m).copy()
|
||||
positions[:, 2] = 0.004 # a visible bump
|
||||
frame = RenderFrame(
|
||||
identity=FrameIdentity("poc", 1, 6, 0.6, generation=1),
|
||||
asset_hash=assets.asset_hash,
|
||||
geometry_id=assets.geometry_id,
|
||||
sensing_part_id=assets.sensing_part().part_id,
|
||||
positions_sensor_m=freeze(positions, dtype=np.float32),
|
||||
normals_sensor=assets.sensing_part().normals_sensor,
|
||||
scalar=freeze(np.linspace(0.0, 1.0, count).astype(np.float32), dtype=np.float32),
|
||||
valid=freeze(np.ones(count, dtype=np.bool_), dtype=np.bool_),
|
||||
scalar_descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
|
||||
)
|
||||
widget.set_settings(
|
||||
DisplaySettings(
|
||||
color_scale=ColorScale(
|
||||
descriptor=frame.scalar_descriptor, lower=0.0, upper=1.0, lut_id="sequential"
|
||||
)
|
||||
)
|
||||
)
|
||||
# The housing's GPU buffers must be untouched by a surface update.
|
||||
before = _buffer_names(renderer, "housing")
|
||||
widget.present(frame)
|
||||
assert _plate_is_grey(_render(widget, qt_app)) > 2.0, "the scalar was not colour-mapped"
|
||||
assert _buffer_names(renderer, "housing") == before, (
|
||||
"a dynamic surface update rebuilt the static housing buffers"
|
||||
)
|
||||
|
||||
|
||||
def _buffer_names(renderer, part_id: str) -> tuple[int, int, int]:
|
||||
part = renderer._parts[part_id]
|
||||
return (part.vao, part.vbo, part.ebo)
|
||||
|
||||
|
||||
def test_a_frame_from_another_geometry_is_refused_without_losing_the_view(gl_widget, qt_app):
|
||||
"""A bad frame is refused; the scene stays usable and stops showing it."""
|
||||
widget, _, failures = gl_widget
|
||||
count = 9
|
||||
before = widget.rejected_frames
|
||||
frame = RenderFrame(
|
||||
identity=FrameIdentity("poc", 1, 2, 0.2, generation=1),
|
||||
asset_hash="other",
|
||||
geometry_id="some-other-geometry",
|
||||
sensing_part_id="sensing_surface",
|
||||
positions_sensor_m=freeze(np.zeros((count, 3), dtype=np.float32), dtype=np.float32),
|
||||
normals_sensor=freeze(
|
||||
np.tile([0.0, 0.0, 1.0], (count, 1)).astype(np.float32), dtype=np.float32
|
||||
),
|
||||
scalar=freeze(np.zeros(count, dtype=np.float32), dtype=np.float32),
|
||||
valid=freeze(np.zeros(count, dtype=np.bool_), dtype=np.bool_),
|
||||
scalar_descriptor=FieldDescriptor("flow_magnitude", "px", "approximate", False),
|
||||
)
|
||||
widget.present(frame)
|
||||
assert _drawn(_render(widget, qt_app)).size, "the view died with a refused frame"
|
||||
assert widget.rejected_frames == before + 1
|
||||
assert any(item.code == "geometry_mismatch" for item in failures), (
|
||||
"a frame from another geometry was accepted"
|
||||
)
|
||||
assert not widget.unavailable, "one bad frame must not take the whole view down"
|
||||
|
||||
|
||||
def test_resize_is_followed(gl_widget, qt_app):
|
||||
widget, _, _ = gl_widget
|
||||
widget.resize(320, 240)
|
||||
image = _render(widget, qt_app)
|
||||
assert (image.shape[1], image.shape[0]) == (320, 240), image.shape
|
||||
widget.resize(120, 90)
|
||||
image = _render(widget, qt_app)
|
||||
assert (image.shape[1], image.shape[0]) == (120, 90), image.shape
|
||||
widget.resize(240, 180)
|
||||
assert _drawn(_render(widget, qt_app)).size, "nothing was drawn after resizing back"
|
||||
|
||||
|
||||
def test_release_is_idempotent_and_a_new_context_gets_the_scene_back(gl_widget, qt_app):
|
||||
"""Closing must free everything; a rebuilt context must get the scene again.
|
||||
|
||||
A context rebuild is the case that matters: the GPU objects are gone while the
|
||||
CPU asset is still here, so "the parts are still uploaded" would be a lie and
|
||||
the window would come back empty. Qt calls ``initializeGL`` for a new context —
|
||||
re-parenting does *not* rebuild one on Qt 5.15 (measured), so the test drives
|
||||
that one call explicitly rather than pretending re-parenting did it.
|
||||
"""
|
||||
widget, renderer, _ = gl_widget
|
||||
widget.release_resources()
|
||||
assert not renderer.is_initialized
|
||||
widget.release_resources() # idempotent: no crash, still released
|
||||
assert not renderer.is_initialized
|
||||
assert not renderer.part_ids, "release left GPU buffers behind"
|
||||
|
||||
# What Qt does when it hands the widget a new context.
|
||||
widget.makeCurrent()
|
||||
widget.initializeGL()
|
||||
widget.doneCurrent()
|
||||
recovered = _pump(qt_app, widget, lambda: bool(renderer.part_ids), timeout_s=8.0)
|
||||
assert recovered, "the scene did not come back for the new context"
|
||||
assert renderer.is_initialized
|
||||
assert _drawn(_render(widget, qt_app)).size, "the scene is not on screen again"
|
||||
|
||||
|
||||
def test_the_probe_reports_this_machine_truthfully():
|
||||
"""The probe's contract: exit 0 only for a GPU-driven context."""
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
path = Path(__file__).resolve().parents[3] / "tools" / "scene_assets" / "probe_gl.py"
|
||||
spec = importlib.util.spec_from_file_location("probe_gl_tool", path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
|
||||
report = module.collect("desktop33", keep_open_s=0.3)
|
||||
context = report.get("context")
|
||||
assert context and context["valid"], report
|
||||
assert context["profile_used"] == "desktop33"
|
||||
assert not context["software_renderer"], "the dev machine has a real GPU"
|
||||
assert report["renderer_initialized"] and report["released"]
|
||||
@@ -89,53 +89,60 @@ def test_the_widget_holds_cpu_state_and_touches_no_context(qt_app):
|
||||
|
||||
|
||||
def test_the_stubs_name_the_stage_that_fills_them_in(qt_app):
|
||||
"""What is implemented is exercised elsewhere; what is not says so.
|
||||
|
||||
Stage 1 landed the camera, the dispatcher and the renderer (see
|
||||
``test_scene3d_stage1.py`` and ``test_scene3d_gl_smoke.py``). The remaining
|
||||
placeholders must still fail loudly with the stage that owns them rather than
|
||||
quietly doing nothing.
|
||||
"""
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraPreset,
|
||||
DisplaySettings,
|
||||
ExpectedGeometry,
|
||||
FrameIdentity,
|
||||
GLCapabilities,
|
||||
RenderFrame,
|
||||
SceneAssets,
|
||||
)
|
||||
from orisys_studio.scene3d.adapters import BoxFrameAdapter, FingerFrameAdapter
|
||||
from orisys_studio.scene3d.camera import OrbitCamera
|
||||
from orisys_studio.scene3d.coordinator import ThreeDCoordinator
|
||||
from orisys_studio.scene3d.gl_backend import GLDispatch, GLRenderer
|
||||
from orisys_studio.scene3d.gl_backend import GLRenderer
|
||||
from orisys_studio.scene3d.panel import ThreeDPanel
|
||||
from orisys_studio.scene3d.widget import Sensor3DWidget
|
||||
|
||||
camera = OrbitCamera()
|
||||
renderer = GLRenderer()
|
||||
widget = Sensor3DWidget(renderer, camera)
|
||||
coordinator = ThreeDCoordinator(widget, composer=object(), executor=None, max_fps=30) # type: ignore[arg-type]
|
||||
panel = ThreeDPanel(coordinator, widget)
|
||||
try:
|
||||
assert coordinator.state == "disabled"
|
||||
assert coordinator.generation == 0
|
||||
assert coordinator.superseded_frames == 0 # the mailbox is real, not a stub
|
||||
|
||||
with pytest.raises(NotImplementedError) as camera_error:
|
||||
camera.set_preset(CameraPreset("default", (0.0, 0.0, 1.0), (0.0, 0.0, 0.0), (0.0, 1.0, 0.0), 0.7))
|
||||
assert "stage 1" in str(camera_error.value)
|
||||
# Stage 1 is real: the camera works with no context at all.
|
||||
camera.set_preset(
|
||||
CameraPreset("default", (0.0, 0.0, 1.0), (0.0, 0.0, 0.0), (0.0, 1.0, 0.0), 0.7)
|
||||
)
|
||||
assert camera.matrices((100, 100)).view.shape == (4, 4)
|
||||
|
||||
with pytest.raises(NotImplementedError) as gl_error:
|
||||
GLDispatch().bind_current_context(object())
|
||||
assert "stage 1" in str(gl_error.value)
|
||||
|
||||
with pytest.raises(NotImplementedError) as activate_error:
|
||||
coordinator.activate(SceneAssets, "live-A") # type: ignore[arg-type]
|
||||
assert "stage 3" in str(activate_error.value)
|
||||
|
||||
with pytest.raises(NotImplementedError) as settings_error:
|
||||
coordinator.set_settings(DisplaySettings())
|
||||
assert "stage 3" in str(settings_error.value)
|
||||
|
||||
with pytest.raises(NotImplementedError) as initialize_error:
|
||||
renderer.initialize(GLCapabilities("desktop33", "", "", "", "", "", False, 0))
|
||||
assert "stage 1" in str(initialize_error.value)
|
||||
|
||||
with pytest.raises(NotImplementedError) as update_error:
|
||||
renderer.update_surface(None) # type: ignore[arg-type]
|
||||
assert "stage 1" in str(update_error.value)
|
||||
|
||||
assert FrameIdentity("live-A", 1, 0, 0.0, 1).generation == 1
|
||||
assert RenderFrame is not None and SceneAssets is not None
|
||||
for call, stage in (
|
||||
(lambda: coordinator.activate(None, "live-A"), "stage 3"),
|
||||
(lambda: coordinator.set_settings(DisplaySettings()), "stage 3"),
|
||||
(lambda: coordinator.set_visible(True), "stage 3"),
|
||||
(lambda: coordinator.close(), "stage 3"),
|
||||
(lambda: panel.set_state("ready"), "stage 4"),
|
||||
(lambda: BoxFrameAdapter().snapshot(None, FrameIdentity("a", 1, 0, 0.0, 1), None),
|
||||
"stage 3"),
|
||||
(
|
||||
lambda: FingerFrameAdapter().snapshot(
|
||||
None, None, FrameIdentity("a", 1, 0, 0.0, 1)
|
||||
),
|
||||
"stage 3",
|
||||
),
|
||||
):
|
||||
with pytest.raises(NotImplementedError) as excinfo:
|
||||
call()
|
||||
assert stage in str(excinfo.value), (stage, str(excinfo.value))
|
||||
del ExpectedGeometry
|
||||
finally:
|
||||
widget.deleteLater()
|
||||
|
||||
@@ -0,0 +1,370 @@
|
||||
"""Stage 1 interface tests: the camera, the synthetic scene, the CPU-side GL logic.
|
||||
|
||||
Everything here runs without a GPU — that is the point of splitting it out. The
|
||||
parts that need a real context are in ``test_scene3d_gl_smoke.py``, which skips
|
||||
itself on a platform that cannot create one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from orisys_app_shared.scene3d import synthetic
|
||||
from orisys_app_shared.scene3d.contracts import (
|
||||
CameraPreset,
|
||||
FrameIdentity,
|
||||
SceneAssets,
|
||||
validate_assets,
|
||||
)
|
||||
from orisys_studio.scene3d.camera import (
|
||||
MAX_DISTANCE_M,
|
||||
MAX_PITCH_RAD,
|
||||
MIN_DISTANCE_M,
|
||||
OrbitCamera,
|
||||
)
|
||||
from orisys_studio.scene3d.gl_backend import (
|
||||
LUT_SIZE,
|
||||
STRIDE_FLOATS,
|
||||
estimate_gpu_bytes,
|
||||
is_software_renderer,
|
||||
lut_rgba,
|
||||
pack_part,
|
||||
pack_scene_part,
|
||||
profile_from_renderable_type,
|
||||
rest_surface,
|
||||
shader_source,
|
||||
)
|
||||
|
||||
|
||||
def _project(camera: OrbitCamera, points: np.ndarray, viewport=(240, 180)) -> np.ndarray:
|
||||
"""NDC coordinates of sensor-space points, through the real matrices."""
|
||||
matrices = camera.matrices(viewport)
|
||||
view = np.asarray(matrices.view, dtype=np.float64)
|
||||
projection = np.asarray(matrices.projection, dtype=np.float64)
|
||||
homogeneous = np.concatenate(
|
||||
[np.asarray(points, dtype=np.float64), np.ones((len(points), 1))], axis=1
|
||||
)
|
||||
camera_space = (view @ homogeneous.T).T
|
||||
clip = (projection @ camera_space.T).T
|
||||
return clip[:, :3] / clip[:, 3:4]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- camera
|
||||
def test_fit_frames_the_whole_scene_inside_the_viewport():
|
||||
assets = synthetic.poc_scene()
|
||||
camera = OrbitCamera()
|
||||
# What the widget does on load: the asset's default view, then framed.
|
||||
camera.set_preset(assets.camera_presets[0])
|
||||
camera.fit(assets.bounds_sensor_m, 240 / 180)
|
||||
for part in assets.parts:
|
||||
ndc = _project(camera, np.asarray(part.positions_sensor_m))
|
||||
assert np.all(np.abs(ndc[:, :2]) <= 1.0), (
|
||||
f"{part.part_id} is outside the viewport: {ndc[:, :2].min(axis=0)} .. "
|
||||
f"{ndc[:, :2].max(axis=0)}"
|
||||
)
|
||||
assert np.all(ndc[:, 2] <= 1.0), "the part is beyond the far plane"
|
||||
# ...and not so far away that it is a dot. The default view is a three-quarter
|
||||
# one, so the plate's face fills the frame in both axes.
|
||||
plate = np.asarray(assets.sensing_part().positions_sensor_m)
|
||||
span = np.ptp(_project(camera, plate)[:, :2], axis=0)
|
||||
assert span.min() > 0.25, f"the fitted scene is tiny on screen: {span}"
|
||||
# The scene sits inside the frustum, not against a plane: at NDC z ~ 1 there
|
||||
# is no usable depth resolution and the housing z-fights the plate.
|
||||
depth = _project(camera, plate)[:, 2]
|
||||
assert np.all(np.abs(depth) < 0.9), f"the scene is against a frustum plane: {depth}"
|
||||
|
||||
|
||||
def test_fit_respects_a_wide_and_a_tall_viewport():
|
||||
"""A wide window must not clip vertically, a tall one not horizontally."""
|
||||
assets = synthetic.poc_scene()
|
||||
for aspect in (0.5, 1.0, 2.5):
|
||||
camera = OrbitCamera()
|
||||
camera.fit(assets.bounds_sensor_m, aspect)
|
||||
points = np.concatenate([np.asarray(p.positions_sensor_m) for p in assets.parts])
|
||||
ndc = _project(camera, points, viewport=(int(200 * aspect), 200))
|
||||
assert np.all(np.abs(ndc[:, :2]) <= 1.0), (aspect, np.ptp(ndc[:, :2], axis=0))
|
||||
|
||||
|
||||
def test_orbit_keeps_the_target_and_the_distance():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
target = camera.target_sensor_m.copy()
|
||||
distance = camera.distance_m
|
||||
camera.orbit((40.0, -20.0), (240, 180))
|
||||
assert np.allclose(camera.target_sensor_m, target), "orbiting moved the target"
|
||||
assert camera.distance_m == pytest.approx(distance)
|
||||
assert camera.yaw_rad != 0.0 and camera.pitch_rad != 0.0
|
||||
|
||||
|
||||
def test_orbit_clamps_the_pitch_instead_of_flipping():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
camera.orbit((0.0, 100000.0), (240, 180))
|
||||
assert camera.pitch_rad <= MAX_PITCH_RAD + 1e-9
|
||||
camera.orbit((0.0, -100000.0), (240, 180))
|
||||
assert camera.pitch_rad >= -MAX_PITCH_RAD - 1e-9
|
||||
# The view stays well formed: an up vector that never collapses.
|
||||
up = camera.matrices((240, 180))
|
||||
assert np.isfinite(np.asarray(up.view)).all()
|
||||
|
||||
|
||||
def test_zoom_is_exponential_and_clamped():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
start = camera.distance_m
|
||||
camera.zoom(1.0)
|
||||
assert camera.distance_m < start
|
||||
inward = camera.distance_m
|
||||
camera.zoom(-1.0)
|
||||
assert camera.distance_m == pytest.approx(start, rel=1e-6), "zoom in then out must return"
|
||||
for _ in range(200):
|
||||
camera.zoom(10.0)
|
||||
assert camera.distance_m == pytest.approx(MIN_DISTANCE_M)
|
||||
for _ in range(400):
|
||||
camera.zoom(-10.0)
|
||||
assert camera.distance_m == pytest.approx(MAX_DISTANCE_M)
|
||||
del inward
|
||||
|
||||
|
||||
def test_pan_moves_the_target_in_the_view_plane():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
before = camera.target_sensor_m.copy()
|
||||
eye_before = camera.eye_sensor_m()
|
||||
camera.pan((10.0, 0.0), (240, 180))
|
||||
moved = camera.target_sensor_m - before
|
||||
assert float(np.linalg.norm(moved)) > 0.0, "panning did nothing"
|
||||
# The eye follows the target, so the model does not appear to rotate.
|
||||
assert np.allclose(camera.eye_sensor_m() - eye_before, moved, atol=1e-9)
|
||||
# A pan of half the viewport height moves the target by half the visible height.
|
||||
camera2 = OrbitCamera()
|
||||
camera2.fit(synthetic.poc_scene().bounds_sensor_m, 240 / 180)
|
||||
visible_height = 2.0 * camera2.distance_m * np.tan(0.5 * camera2.vertical_fov_rad)
|
||||
before2 = camera2.target_sensor_m.copy()
|
||||
camera2.pan((0.0, 90.0), (240, 180))
|
||||
moved2 = camera2.target_sensor_m - before2
|
||||
assert float(np.linalg.norm(moved2)) == pytest.approx(visible_height * 0.5, rel=1e-6), (
|
||||
"a half-viewport drag must move the target by half the visible height"
|
||||
)
|
||||
# ...and along the camera's up axis: a screen-vertical drag is not world-Z.
|
||||
direction = moved2 / float(np.linalg.norm(moved2))
|
||||
assert abs(float(np.dot(direction, camera2._direction()))) < 1e-6
|
||||
|
||||
|
||||
def test_presets_reproduce_their_eye_and_target():
|
||||
camera = OrbitCamera()
|
||||
preset = CameraPreset(
|
||||
"asset-default", (0.0, -0.05, 0.05), (0.0, 0.0, 0.0), (0.0, 0.0, 1.0), 0.6
|
||||
)
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
camera.set_preset(preset)
|
||||
assert np.allclose(camera.target_sensor_m, preset.target_sensor_m)
|
||||
assert np.allclose(camera.eye_sensor_m(), preset.eye_sensor_m, atol=1e-9)
|
||||
assert camera.vertical_fov_rad == pytest.approx(preset.vertical_fov_rad)
|
||||
# A drag continues from the preset rather than jumping.
|
||||
eye = camera.eye_sensor_m()
|
||||
camera.orbit((5.0, 2.0), (240, 180))
|
||||
assert not np.allclose(camera.eye_sensor_m(), eye)
|
||||
|
||||
|
||||
def test_reset_returns_to_the_framed_view():
|
||||
camera = OrbitCamera()
|
||||
assets = synthetic.poc_scene()
|
||||
camera.fit(assets.bounds_sensor_m, 1.0)
|
||||
target = camera.target_sensor_m.copy()
|
||||
distance = camera.distance_m
|
||||
camera.orbit((30.0, 10.0), (240, 180))
|
||||
camera.zoom(3.0)
|
||||
camera.reset(assets.bounds_sensor_m)
|
||||
assert np.allclose(camera.target_sensor_m, target)
|
||||
assert camera.distance_m == pytest.approx(distance)
|
||||
assert camera.yaw_rad == 0.0 and camera.pitch_rad == 0.0
|
||||
|
||||
|
||||
def test_the_view_matrix_is_a_rigid_transform():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.3)
|
||||
view = np.asarray(camera.matrices((320, 240)).view, dtype=np.float64)
|
||||
rotation = view[:3, :3]
|
||||
assert np.allclose(rotation @ rotation.T, np.eye(3), atol=1e-5), "the view is not rigid"
|
||||
assert np.linalg.det(rotation) == pytest.approx(1.0, abs=1e-5)
|
||||
# The camera sits in front of the scene: visible points have negative z.
|
||||
target = camera.target_sensor_m
|
||||
homogeneous = np.append(target, 1.0)
|
||||
assert float((view @ homogeneous)[2]) < 0.0
|
||||
|
||||
|
||||
def test_matrices_are_read_only_float32():
|
||||
camera = OrbitCamera()
|
||||
camera.fit(synthetic.poc_scene().bounds_sensor_m, 1.0)
|
||||
matrices = camera.matrices((240, 180))
|
||||
for name, value in (("view", matrices.view), ("projection", matrices.projection)):
|
||||
assert value.dtype == np.float32 and value.shape == (4, 4)
|
||||
assert not value.flags["WRITEABLE"], f"{name} is writable"
|
||||
assert value.flags["C_CONTIGUOUS"]
|
||||
|
||||
|
||||
def test_fit_refuses_bounds_that_are_not_two_by_three():
|
||||
camera = OrbitCamera()
|
||||
with pytest.raises(ValueError):
|
||||
camera.fit(np.zeros((3, 3), dtype=np.float32), 1.0)
|
||||
|
||||
|
||||
def test_a_preset_with_the_eye_on_the_target_is_refused():
|
||||
camera = OrbitCamera()
|
||||
with pytest.raises(ValueError):
|
||||
camera.set_preset(
|
||||
CameraPreset("degenerate", (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 1.0), 0.6)
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------- the POC scene
|
||||
def test_the_synthetic_scene_obeys_the_contracts():
|
||||
assets = synthetic.poc_scene()
|
||||
assert isinstance(assets, SceneAssets)
|
||||
assert validate_assets(assets, synthetic.poc_expected()) == ()
|
||||
roles = {part.role for part in assets.parts}
|
||||
assert roles == {"sensing", "housing"}, "a scene needs one of each for the PoC"
|
||||
# The plate faces +Z and is flat, so a normal assertion has an answer.
|
||||
plate = assets.sensing_part()
|
||||
assert np.allclose(np.asarray(plate.normals_sensor), [0.0, 0.0, 1.0])
|
||||
assert float(np.asarray(plate.positions_sensor_m)[:, 2].max()) == 0.0
|
||||
assert assets.bounds_sensor_m.shape == (2, 3)
|
||||
assert [preset.name for preset in assets.camera_presets][0] == "default"
|
||||
|
||||
|
||||
# ------------------------------------------------------------ GL CPU helpers
|
||||
def test_shader_sources_are_two_dialects_of_one_body():
|
||||
for profile, marker in (("desktop33", "#version 330 core"), ("gles30", "#version 300 es")):
|
||||
for program in ("scalar", "diffuse"):
|
||||
vertex, fragment = shader_source(profile, program)
|
||||
assert vertex.startswith(marker), (profile, program, vertex[:40])
|
||||
assert fragment.startswith(marker)
|
||||
if profile == "gles30":
|
||||
# ES requires explicit precision for floats.
|
||||
assert "precision highp float;" in fragment
|
||||
# No fixed pipeline, ever.
|
||||
for banned in ("glBegin", "gl_End", "glMatrixMode", "gl_FragColor", "GL_QUADS"):
|
||||
assert banned not in vertex and banned not in fragment
|
||||
# The maths is shared: the bodies differ only by their header.
|
||||
from orisys_studio.scene3d.gl_backend import shader_header
|
||||
|
||||
desktop_vertex, desktop_fragment = shader_source("desktop33", "scalar")
|
||||
es_vertex, es_fragment = shader_source("gles30", "scalar")
|
||||
desktop_body = desktop_vertex[len(shader_header("desktop33")):]
|
||||
es_body = es_vertex[len(shader_header("gles30")):]
|
||||
assert desktop_body == es_body, "the vertex maths drifted between dialects"
|
||||
assert desktop_fragment[len(shader_header("desktop33")):] == es_fragment[
|
||||
len(shader_header("gles30")):
|
||||
]
|
||||
assert "texture(u_lut" in desktop_fragment and "texture(u_lut" in es_fragment
|
||||
assert "isnan" in desktop_fragment, "an unmeasured value must not be colour-mapped"
|
||||
|
||||
|
||||
def test_an_unknown_profile_or_program_is_refused():
|
||||
with pytest.raises(ValueError):
|
||||
shader_source("vulkan", "scalar") # type: ignore[arg-type]
|
||||
with pytest.raises(ValueError):
|
||||
shader_source("desktop33", "wireframe")
|
||||
|
||||
|
||||
def test_the_lut_is_an_ordered_ramp_and_the_diverging_one_is_centred():
|
||||
sequential = lut_rgba("sequential")
|
||||
assert sequential.shape == (LUT_SIZE, 1, 4) and sequential.dtype == np.uint8
|
||||
assert (sequential[:, 0, 3] == 255).all(), "the LUT must be opaque"
|
||||
luminance = sequential[:, 0, :3].astype(float).mean(axis=1)
|
||||
assert luminance[0] < luminance[-1], "a sequential ramp must get lighter"
|
||||
assert (np.diff(luminance) >= -6.0).all(), "the ramp is not monotone"
|
||||
diverging = lut_rgba("diverging")
|
||||
assert diverging.shape == sequential.shape
|
||||
assert not np.array_equal(diverging, sequential)
|
||||
|
||||
|
||||
def test_software_renderers_are_recognised():
|
||||
assert is_software_renderer("Mesa", "llvmpipe (LLVM 15.0.7, 256 bits)")
|
||||
assert is_software_renderer("Microsoft", "Microsoft Basic Render Driver")
|
||||
assert is_software_renderer("", "swrast")
|
||||
assert not is_software_renderer("NVIDIA Corporation", "NVIDIA GeForce RTX 4080 SUPER/PCIe/SSE2")
|
||||
assert not is_software_renderer("Intel", "Intel(R) UHD Graphics 620")
|
||||
|
||||
|
||||
def test_the_renderable_type_mapping_is_not_inverted():
|
||||
"""Qt: OpenGL = 1, OpenGLES = 2. Getting this backwards is invisible on a
|
||||
driver that accepts both dialects and fatal on one that does not."""
|
||||
assert profile_from_renderable_type(1) == "desktop33"
|
||||
assert profile_from_renderable_type(2) == "gles30"
|
||||
assert profile_from_renderable_type(0) == "desktop33", "the default is desktop"
|
||||
|
||||
|
||||
def _plate(**kwargs):
|
||||
return synthetic.poc_scene().sensing_part()
|
||||
|
||||
|
||||
def test_packing_lays_out_position_normal_scalar_valid():
|
||||
part = _plate()
|
||||
count = int(part.positions_sensor_m.shape[0])
|
||||
scalar = np.linspace(0.0, 1.0, count, dtype=np.float32)
|
||||
valid = np.ones(count, dtype=bool)
|
||||
rows = pack_part(part, scalar=scalar, valid=valid)
|
||||
assert rows.shape == (count, STRIDE_FLOATS)
|
||||
assert rows.dtype == np.float32 and rows.flags["C_CONTIGUOUS"]
|
||||
assert not rows.flags["WRITEABLE"], "the packed buffer must be read-only"
|
||||
assert np.allclose(rows[:, 0:3], part.positions_sensor_m)
|
||||
assert np.allclose(rows[:, 3:6], part.normals_sensor)
|
||||
assert np.allclose(rows[:, 6], scalar)
|
||||
assert np.allclose(rows[:, 7], 1.0)
|
||||
|
||||
|
||||
def test_packing_a_scene_without_a_measurement_marks_everything_invalid():
|
||||
rows = pack_part(_plate())
|
||||
assert np.isnan(rows[:, 6]).all(), "an unmeasured scalar must be NaN, not zero"
|
||||
assert np.allclose(rows[:, 7], 0.0), "an unmeasured vertex must be invalid"
|
||||
|
||||
|
||||
def test_packing_refuses_a_mismatched_length():
|
||||
part = _plate()
|
||||
with pytest.raises(Exception) as scalar_error:
|
||||
pack_part(part, scalar=np.zeros(3, dtype=np.float32))
|
||||
assert "scalars for" in str(scalar_error.value)
|
||||
with pytest.raises(Exception) as valid_error:
|
||||
pack_part(part, valid=np.zeros(2, dtype=bool))
|
||||
assert "validity flags for" in str(valid_error.value)
|
||||
|
||||
|
||||
def test_scene_part_indices_are_uint32_and_read_only():
|
||||
vertices, indices = pack_scene_part(_plate())
|
||||
assert indices.dtype == np.uint32 and indices.ndim == 1
|
||||
assert not indices.flags["WRITEABLE"] and not vertices.flags["WRITEABLE"]
|
||||
assert int(indices.max()) < vertices.shape[0]
|
||||
|
||||
|
||||
def test_the_gpu_budget_counts_both_parts_and_one_dynamic_copy():
|
||||
assets = synthetic.poc_scene()
|
||||
budget = estimate_gpu_bytes(assets, dynamic_part_id="sensing_surface")
|
||||
plate = assets.sensing_part()
|
||||
housing = assets.part("housing")
|
||||
assert housing is not None
|
||||
expected = (
|
||||
plate.positions_sensor_m.size // 3 * STRIDE_FLOATS * 4
|
||||
+ plate.triangles.size * 4
|
||||
+ plate.positions_sensor_m.size // 3 * STRIDE_FLOATS * 4 # the in-flight copy
|
||||
+ housing.positions_sensor_m.size // 3 * STRIDE_FLOATS * 4
|
||||
+ housing.triangles.size * 4
|
||||
+ LUT_SIZE * 4
|
||||
)
|
||||
assert budget == expected
|
||||
assert budget < 1024 * 1024, "the PoC scene should be tiny"
|
||||
|
||||
|
||||
def test_the_rest_surface_claims_no_measurement():
|
||||
assets = synthetic.poc_scene()
|
||||
frame = rest_surface(assets, FrameIdentity("poc", 1, 0, 0.0, 1))
|
||||
count = int(assets.sensing_part().positions_sensor_m.shape[0])
|
||||
assert frame.scalar.shape == (count,) and frame.valid.shape == (count,)
|
||||
assert not frame.valid.any(), "a reference frame measures nothing"
|
||||
assert np.isnan(frame.scalar).all(), "an unavailable scalar is NaN, not zero"
|
||||
assert frame.scalar_descriptor.quantity == "unavailable"
|
||||
assert frame.scalar_descriptor.quality == "display_only"
|
||||
assert frame.geometry_id == assets.geometry_id
|
||||
assert frame.sensing_part_id == assets.sensing_part().part_id
|
||||
Reference in New Issue
Block a user