orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

阶段 1：平台 PoC —— 真实 GL 上跑通静态场景，并修掉它抓出的相机/上下文错误 - What: 按 docs/architecture/studio-3d-stages.md 阶段 1 落地图形栈。OrbitCamera 全部实现（target/distance/yaw/pitch 表示；fit 按包围球与较窄的那个视场轴定距离；orbit/pan/zoom 有界；set_preset 从 eye/target 反解角度使拖动可延续；matrices 输出只读 float32 view/projection；另加 depth_range_m() 便于断言）。gl_backend：GLDispatch.bind_current_context 真正绑定 PyOpenGL 并读 vendor/renderer/GL/GLSL/max texture、判软件渲染、profile 不达标时抛 ThreeDUnsupported 并带回能力；GLRenderer 实现 initialize（两个 program，desktop33/gles30 共用同一份着色器主体、只换头部）、upload_scene（每 part 一套 VAO/VBO/EBO，交错 layout position/normal/scalar/valid = 8 float）、update_surface（只对感知面 glBufferSubData，绝不重建索引）、draw（绑 Qt 的 defaultFramebufferObject、深度测试、按 role 选 program、housing 隐藏即跳过、scalar 经 LUT 纹理与 range uniform 上色，Python 不生成 RGB 数组）、release（幂等）；纯函数 pack_part/pack_scene_part/estimate_gpu_bytes/rest_surface/shader_source/shader_header/lut_rgba/is_software_renderer/profile_from_renderable_type 都可在无 GPU 环境测试。Sensor3DWidget 实现 initializeGL/paintGL/resizeGL/release_resources/aboutToBeDestroyed/showEvent/hideEvent 与鼠标事件；加载场景时先采用资产自带的默认视角再 fit；一帧几何不匹配只"拒绝该帧"（计数 + 诊断 + 用 rest_surface 清掉上次测量）而不是把整个视图打成不可用。shared/scene3d/synthetic.py 提供 PoC 与测试共用的已知场景（20mm 感知板 + 外壳盒，米、sensor 坐标系、带默认三视角预设）。tools/scene_assets/probe_gl.py 真实现：真实窗口取 context 并输出 JSON 报告（平台/Qt/PyOpenGL/vendor/renderer/GL/GLSL/DPR/framebuffer/是否软件渲染），退出码即门禁（0 通过、3 软件渲染、4 无上下文）；apps/studio/pyproject.toml 增加 [three-d] 可选依赖（仅 PyOpenGL；Open3D 不进运行时）。测试：apps/studio/tests/test_scene3d_stage1.py 24 项（相机数学、PoC 场景合规、打包布局、着色器两方言共用主体、LUT、软件渲染识别、renderableType 映射、GPU 预算、参考帧语义，全部无需 GPU）＋ test_scene3d_gl_smoke.py 8 项（真实 context：能力、两个 part 都画出来、外壳可隐藏、参考帧中性 vs 色图有差异、动态更新不动 housing 缓冲、坏几何帧被拒且视图存活、resize 跟随、release 幂等且新 context 能重传场景、探针如实报告）；offscreen 下后者自动 skip。docs/architecture/gl-report-windows-dev.json 为本机平台报告。 - Why: 用户要求"从骨架搭起→计划阶段→逐阶段实现与验收"，并明确 ARM 暂缓。阶段 1 的产出本身就是一份能回答"这块板的图形栈能不能用"的报告，所以先把它做实、把 profile 冻结点定在真实 context 上，而不是先堆屏幕上的东西。 - Verification: 真实 context（QT_QPA_PLATFORM=windows）下 8 项 GL 验收全过；offscreen 下 24 项 CPU 项全过、GL 项按设计 skip；shared 全量 187 passed；studio 全量 116 passed（含新 32 项）；ui_check 554 项 0 失败；probe_gl 输出 verdict=ok（NVIDIA RTX 4080 SUPER、GL 3.3 Core、GLSL 3.30、非软件渲染）。过程中这套验收抓出并修掉了 7 个真实缺陷：Qt renderableType 映射反了（NVIDIA 兼容 ES 着色器所以本机看不出，Intel/ARM 会直接失败）、观察相机是左手基导致整模型镜像（det=-1）、近平面过近使 NDC z 全挤在 0.99（深度精度归零、外壳与感知面会 z-fighting）、手动 makeCurrent 早于 Qt 首次绘制会把 GPU 对象建在被丢弃的 context 里（glBindVertexArray 报 INVALID_OPERATION）、context 重建后不重传场景、一帧几何不匹配把整个视图打死、以及 grabFramebuffer 是 BGRA+stride 导致颜色断言全部读错通道。 - Not in this change: 未接入 app_window/main.py（三维对现有应用仍完全惰性，故"开/关三维 CSV 行序一致"这条验收随阶段 4 做）；阶段 2 的资产导入/坐标/包格式；ARM 平台实测（probe 已可在板上直接运行）。

Browse Source
This commit is contained in:
ykcheungab
2026-09-14 14:55:45 +08:00
parent 0b309e7edf
commit 5767b4bf11
11 changed files with 2324 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/studio/src/orisys_studio/scene3d/camera.py
+210 -22
View File
@@ -1,50 +1,238 @@
"""The observation camera: orbit, pan, zoom, presets. Pure maths.
"""The observation camera: orbit, pan, zoom, presets. Pure maths, no Qt.
Stage 1 gets a minimal ``fit``/``orbit`` so a static scene can be looked at; the
full interaction (pan, zoom limits, presets, reset) is stage 4 in
``docs/architecture/studio-3d-stages.md``.
It changes ``view`` and ``projection`` and nothing else: the pixel→surface
mapping, the calibration and the force axes are not reachable from here. The
default view is a ``CameraPreset`` that ships with the geometry, so "what the part
looks like" is versioned with the asset rather than with somebody's laptop.
It is deliberately *not* the algorithm camera. It changes ``view`` and
``projection`` only: ``window_to_world`` for the pixel→surface mapping, the
calibration and the force axes are untouched by anything a user does here. The
default view comes from the asset's ``CameraPreset``, so it is versioned with the
geometry rather than with a user's preference file.
Conventions (design document §3.1): sensor frame, right-handed, metres; column
vectors, so ``p_camera = view @ p_sensor`` and ``p_clip = projection @ p_camera``.
The view matrix is built from a target, a distance and two angles, which is what
makes orbit/pan/zoom trivial and keeps "reset" exact rather than approximate.
"""
from __future__ import annotations
import math
from dataclasses import dataclass, field
from typing import Optional
import numpy as np
from orisys_app_shared.scene3d.contracts import (
CameraMatrices,
CameraPreset,
F32,
freeze,
)
_STAGE = "stage 1 (fit/orbit) and stage 4 (full interaction)"
#: Distance limits, in metres: close enough to inspect a 20 mm part, far enough
#: not to lose a 10 cm one. The far plane follows the distance, so precision stays
#: usable at both ends instead of being decided once.
MIN_DISTANCE_M = 0.005
MAX_DISTANCE_M = 5.0
#: Radians per pixel of drag, and the pitch limit that keeps the up vector sane.
ORBIT_RAD_PER_PX = 0.008
MAX_PITCH_RAD = math.radians(89.0)
#: Zoom per wheel step.
ZOOM_PER_STEP = 0.12
@dataclass
class OrbitCamera:
"""A target, a distance and two angles. Everything else is derived."""
"""A target, a distance and two angles; everything else is derived."""
target_sensor_m: np.ndarray = field(
default_factory=lambda: np.zeros(3, dtype=np.float64)
)
distance_m: float = 0.1
yaw_rad: float = 0.0
pitch_rad: float = 0.0
up_sensor: np.ndarray = field(
default_factory=lambda: np.array([0.0, 0.0, 1.0], dtype=np.float64)
)
vertical_fov_rad: float = 0.6
aspect: float = 1.0
_fitted_radius_m: float = 0.05
_preset_name: Optional[str] = None
# ------------------------------------------------------------------- setup
def fit(self, bounds_sensor_m: F32, aspect: float) -> None:
"""Frame the whole scene: set the target, distance, near and far planes."""
raise NotImplementedError(f"{_STAGE}: fit the camera to the scene bounds")
"""Frame the whole scene from the bounds, without changing the angles.
The distance is derived from the bounding sphere and the *narrower* of the
two field-of-view axes, so a wide window does not clip the model
vertically while a tall one does not clip it horizontally.
"""
bounds = np.asarray(bounds_sensor_m, dtype=np.float64)
if bounds.shape != (2, 3):
raise ValueError(f"bounds must be [2,3], got {bounds.shape}")
low, high = bounds[0], bounds[1]
self.target_sensor_m = (low + high) * 0.5
radius = float(np.linalg.norm(high - low) * 0.5)
self._fitted_radius_m = max(radius, 1e-6)
self.aspect = max(float(aspect), 1e-6)
self.vertical_fov_rad = _clamp(self.vertical_fov_rad, 0.1, 2.6)
vertical_half = 0.5 * self.vertical_fov_rad
horizontal_half = math.atan(math.tan(vertical_half) * self.aspect)
half = min(vertical_half, horizontal_half)
# A little margin, so the silhouette is not flush with the edge.
self.distance_m = _clamp(
self._fitted_radius_m / max(math.sin(half), 1e-6) * 1.15,
MIN_DISTANCE_M,
MAX_DISTANCE_M,
)
def set_preset(self, preset: CameraPreset) -> None:
"""Point the camera where the asset says its default view is.
The preset's ``eye`` is turned into yaw/pitch/distance around its target,
so a later drag continues from the preset instead of jumping.
"""
eye = np.asarray(preset.eye_sensor_m, dtype=np.float64)
target = np.asarray(preset.target_sensor_m, dtype=np.float64)
up = np.asarray(preset.up_sensor, dtype=np.float64)
offset = eye - target
distance = float(np.linalg.norm(offset))
if distance <= 0.0:
raise ValueError(f"preset {preset.name!r} puts the eye on the target")
self.target_sensor_m = target
self.distance_m = _clamp(distance, MIN_DISTANCE_M, MAX_DISTANCE_M)
self.up_sensor = up / max(float(np.linalg.norm(up)), 1e-9)
direction = offset / distance
self.pitch_rad = _clamp(math.asin(_clamp(direction[2], -1.0, 1.0)),
-MAX_PITCH_RAD, MAX_PITCH_RAD)
self.yaw_rad = math.atan2(direction[0], -direction[1])
self.vertical_fov_rad = _clamp(float(preset.vertical_fov_rad), 0.1, 2.6)
self._preset_name = preset.name
# -------------------------------------------------------------- interaction
def orbit(self, delta_px: tuple[float, float], viewport: tuple[int, int]) -> None:
"""Rotate about the target. The model data does not move."""
raise NotImplementedError(f"{_STAGE}: orbit")
dx, dy = (float(delta_px[0]), float(delta_px[1]))
self.yaw_rad = (self.yaw_rad + dx * ORBIT_RAD_PER_PX) % (2.0 * math.pi)
self.pitch_rad = _clamp(
self.pitch_rad + dy * ORBIT_RAD_PER_PX, -MAX_PITCH_RAD, MAX_PITCH_RAD
)
def pan(self, delta_px: tuple[float, float], viewport: tuple[int, int]) -> None:
"""Slide the target in the view plane, scaled by distance."""
raise NotImplementedError(f"{_STAGE}: pan")
"""Slide the target in the view plane, scaled so it tracks the cursor."""
width = max(int(viewport[0]), 1)
height = max(int(viewport[1]), 1)
dx, dy = float(delta_px[0]), float(delta_px[1])
world_per_px = (
2.0 * self.distance_m * math.tan(0.5 * self.vertical_fov_rad) / height
)
right, up = self._basis()
self.target_sensor_m = (
self.target_sensor_m - right * (dx * world_per_px) + up * (dy * world_per_px)
)
def zoom(self, wheel_steps: float) -> None:
"""Exponential zoom, clamped so the surface cannot be entered or lost."""
raise NotImplementedError(f"{_STAGE}: zoom")
factor = math.pow(1.0 - ZOOM_PER_STEP, float(wheel_steps))
self.distance_m = _clamp(
self.distance_m * factor, MIN_DISTANCE_M, MAX_DISTANCE_M
)
def set_preset(self, preset: CameraPreset) -> None:
"""Front / side / top / isometric / asset default."""
raise NotImplementedError(f"{_STAGE}: apply a camera preset")
def reset(self, bounds_sensor_m: F32) -> None:
"""Back to the framed default view."""
self.yaw_rad = 0.0
self.pitch_rad = 0.0
self.fit(bounds_sensor_m, self.aspect)
# ------------------------------------------------------------------ matrices
def eye_sensor_m(self) -> np.ndarray:
return self.target_sensor_m + self._direction() * self.distance_m
def matrices(self, viewport: tuple[int, int]) -> CameraMatrices:
"""``p_camera = view @ p_sensor``, ``p_clip = projection @ p_camera``."""
raise NotImplementedError(f"{_STAGE}: build the view and projection matrices")
"""``view`` and ``projection`` for the given physical viewport size."""
width = max(int(viewport[0]), 1)
height = max(int(viewport[1]), 1)
aspect = width / height
self.aspect = aspect
eye = self.eye_sensor_m()
# ``_direction()`` points from the target towards the eye, i.e. it is the
# camera's *backward* axis. The camera looks along ``-backward``, and the
# right vector is ``cross(-backward, up)`` — building it as
# ``cross(backward, up)`` yields a left-handed basis (det = -1), which
# mirrors the whole model.
backward = self._direction()
up = np.asarray(self.up_sensor, dtype=np.float64)
right = np.cross(up, backward)
norm = float(np.linalg.norm(right))
if norm < 1e-8:
# Looking straight down the up axis: pick any stable right vector.
right = np.array([1.0, 0.0, 0.0])
norm = 1.0
right = right / norm
true_up = np.cross(backward, right)
view = np.eye(4, dtype=np.float64)
view[0, :3], view[1, :3], view[2, :3] = right, true_up, backward
view[0, 3] = -float(np.dot(right, eye))
view[1, 3] = -float(np.dot(true_up, eye))
view[2, 3] = -float(np.dot(backward, eye))
near, far = self.depth_range_m()
projection = _perspective(self.vertical_fov_rad, aspect, near, far)
return CameraMatrices(
view=freeze(view, dtype=np.float32),
projection=freeze(projection, dtype=np.float32),
)
def depth_range_m(self) -> tuple[float, float]:
"""The frustum's near and far planes, for tests and diagnostics.
The near plane sits just in front of the scene rather than at a fraction of
the distance: a near plane orders of magnitude closer than the model throws
away every bit of depth resolution — everything lands at NDC z ~ 0.99 and
the housing z-fights the plate.
"""
near = max(
self.distance_m - self._fitted_radius_m * 1.5,
self.distance_m * 0.05,
MIN_DISTANCE_M * 0.05,
)
far = max(self.distance_m + self._fitted_radius_m * 4.0, near * 2.0)
return (near, far)
# ------------------------------------------------------------------- helpers
def _direction(self) -> np.ndarray:
"""Unit vector from the target towards the eye."""
cos_pitch = math.cos(self.pitch_rad)
return np.array(
[
cos_pitch * math.sin(self.yaw_rad),
-cos_pitch * math.cos(self.yaw_rad),
math.sin(self.pitch_rad),
],
dtype=np.float64,
)
def _basis(self) -> tuple[np.ndarray, np.ndarray]:
"""``(right, up)`` of the camera, in the same handedness as ``matrices``."""
backward = self._direction()
up = np.asarray(self.up_sensor, dtype=np.float64)
right = np.cross(up, backward)
norm = float(np.linalg.norm(right))
if norm < 1e-8:
right = np.array([1.0, 0.0, 0.0])
norm = 1.0
right = right / norm
true_up = np.cross(backward, right)
return right, true_up / max(float(np.linalg.norm(true_up)), 1e-9)
def _clamp(value: float, low: float, high: float) -> float:
return max(low, min(high, float(value)))
def _perspective(fov_rad: float, aspect: float, near: float, far: float) -> np.ndarray:
"""Right-handed perspective, OpenGL clip convention (z in [-1, 1])."""
f = 1.0 / math.tan(0.5 * fov_rad)
matrix = np.zeros((4, 4), dtype=np.float64)
matrix[0, 0] = f / aspect
matrix[1, 1] = f
matrix[2, 2] = (far + near) / (near - far)
matrix[2, 3] = (2.0 * far * near) / (near - far)
matrix[3, 2] = -1.0
return matrix
apps/studio/src/orisys_studio/scene3d/gl_backend.py
+723 -16
View File
@@ -9,29 +9,57 @@ Import discipline, and why it is not negotiable:
* the PyOpenGL import happens inside :meth:`GLDispatch.bind_current_context`,
after Qt has made a real context current, so the binding is chosen for the
platform Qt actually used (GLX or EGL) instead of being guessed;
* ``GLRenderer.release`` is idempotent and requires the owning context to be
* :meth:`GLRenderer.release` is idempotent and requires the owning context to be
current — resources are freed by the context that made them.
The probe is what decides the profile on a board. A missing GL, a software
renderer, or a failed shader compile means the three-dimensional view reports
itself unavailable and the two-dimensional application carries on unchanged.
Two parts, two programs, one draw call per part. The sensing surface is re-uploaded
per frame and its colour comes from ``scalar`` through a LUT texture with the range
as uniforms, so Python never builds an RGB array per frame; the housing is uploaded
once and never touched again. A frame with no measurement (``valid == 0``) is drawn
in the neutral colour rather than in colour zero — the reference model is not a
measurement and must not look like one.
"""
from __future__ import annotations
from typing import Any
import math
import time
from typing import Any, Optional
import numpy as np
from orisys_app_shared.scene3d.contracts import (
CameraMatrices,
DisplaySettings,
GLCapabilities,
MeshPart,
Profile,
RenderFrame,
RenderStats,
SceneAssets,
diagnostic,
error,
freeze,
)
from orisys_app_shared.scene3d.contracts import Diagnostic
#: Renderer names that are not a GPU. A board must not be signed off on one of
#: these, and neither must a laptop with a broken driver.
SOFTWARE_RENDERER_MARKERS = (
"llvmpipe",
"softpipe",
"swrast",
"swiftshader",
"microsoft basic render",
"gdi generic",
"mesa offscreen",
)
_STAGE = "stage 1 (platform PoC)"
#: The neutral colour an invalid or unmeasured vertex gets (linear, dark grey).
INVALID_COLOR = (0.18, 0.18, 0.20, 1.0)
#: LUT size. 256 is enough for a sequential ramp and cheap to rebuild.
LUT_SIZE = 256
def configure_surface_format(profile: Profile = "desktop33") -> None:
@@ -63,28 +91,474 @@ def configure_surface_format(profile: Profile = "desktop33") -> None:
QSurfaceFormat.setDefaultFormat(fmt)
def is_software_renderer(vendor: str, renderer: str) -> bool:
"""Whether these GL strings describe a rasteriser rather than a GPU."""
haystack = f"{vendor} {renderer}".lower()
return any(marker in haystack for marker in SOFTWARE_RENDERER_MARKERS)
#: ``QSurfaceFormat.RenderableType``. Spelled out because guessing it wrong is
#: invisible on a driver that accepts both shader dialects (NVIDIA does, through
#: ``ARB_ES3_compatibility``) and fails on one that does not.
RENDERABLE_OPENGL = 1
RENDERABLE_OPENGL_ES = 2
def profile_from_renderable_type(value: int) -> Profile:
"""Map Qt's renderable type to the shader dialect that must be compiled."""
return "gles30" if int(value) == RENDERABLE_OPENGL_ES else "desktop33"
# --------------------------------------------------------------------- shaders
#: Shared bodies: the version header is the only difference, so the maths cannot
#: drift between the desktop and the embedded build.
_VERTEX_BODY = """
layout(location = 0) in vec3 a_position;
layout(location = 1) in vec3 a_normal;
layout(location = 2) in float a_scalar;
layout(location = 3) in float a_valid;
uniform mat4 u_view;
uniform mat4 u_projection;
out vec3 v_normal;
out float v_scalar;
out float v_valid;
void main() {
v_normal = a_normal;
v_scalar = a_scalar;
v_valid = a_valid;
gl_Position = u_projection * u_view * vec4(a_position, 1.0);
}
"""
_SCALAR_FRAGMENT_BODY = """
in vec3 v_normal;
in float v_scalar;
in float v_valid;
uniform sampler2D u_lut;
uniform float u_lower;
uniform float u_upper;
uniform vec4 u_invalid_color;
out vec4 frag_color;
void main() {
if (v_valid < 0.5 || isnan(v_scalar)) {
frag_color = u_invalid_color;
return;
}
float span = max(u_upper - u_lower, 1e-12);
float t = clamp((v_scalar - u_lower) / span, 0.0, 1.0);
frag_color = vec4(texture(u_lut, vec2(t, 0.5)).rgb, 1.0);
}
"""
_DIFFUSE_FRAGMENT_BODY = """
in vec3 v_normal;
in float v_scalar;
in float v_valid;
uniform vec3 u_light_direction;
uniform vec4 u_base_color;
out vec4 frag_color;
void main() {
vec3 n = normalize(v_normal);
if (!gl_FrontFacing) {
n = -n;
}
float lambert = max(dot(n, normalize(u_light_direction)), 0.0);
frag_color = vec4(u_base_color.rgb * (0.35 + 0.65 * lambert), u_base_color.a);
}
"""
_HEADERS: dict[Profile, str] = {
"desktop33": "#version 330 core\n",
"gles30": "#version 300 es\nprecision highp float;\nprecision highp int;\n",
}
def shader_header(profile: Profile) -> str:
"""The dialect header for a profile. Public so a test can compare bodies."""
try:
return _HEADERS[profile]
except KeyError: # pragma: no cover - guarded by configure_surface_format
raise ValueError(f"unknown GL profile: {profile!r}") from None
def shader_source(profile: Profile, program: str) -> tuple[str, str]:
"""``(vertex, fragment)`` source for a profile and a program name.
Pure text: testable without a context, so "the ES build really is ES" is an
assertion rather than a hope.
"""
header = shader_header(profile)
if program == "scalar":
body = _SCALAR_FRAGMENT_BODY
elif program == "diffuse":
body = _DIFFUSE_FRAGMENT_BODY
else:
raise ValueError(f"unknown shader program: {program!r}")
return header + _VERTEX_BODY, header + body
def lut_rgba(lut_id: str = "sequential") -> np.ndarray:
"""A 256x1 RGBA8 lookup table.
Hand-rolled rather than imported: the runtime must not grow a plotting
dependency for one texture, and the ramp is part of what the colours *mean*.
``sequential`` is perceptually increasing in lightness (safe for magnitude);
``diverging`` is the same ramp mirrored through a neutral midpoint (safe for a
signed quantity, and never used for an unsigned one).
"""
stops = np.array(
[
[0.267, 0.005, 0.329],
[0.283, 0.141, 0.458],
[0.254, 0.265, 0.530],
[0.207, 0.372, 0.553],
[0.164, 0.471, 0.558],
[0.128, 0.567, 0.551],
[0.135, 0.659, 0.518],
[0.267, 0.749, 0.441],
[0.478, 0.821, 0.318],
[0.741, 0.873, 0.150],
[0.993, 0.906, 0.144],
],
dtype=np.float32,
)
if lut_id == "diverging":
stops = np.concatenate([stops[::-1], stops], axis=0)
positions = np.linspace(0.0, 1.0, stops.shape[0], dtype=np.float32)
grid = np.linspace(0.0, 1.0, LUT_SIZE, dtype=np.float32)
rgba = np.empty((LUT_SIZE, 1, 4), dtype=np.uint8)
for channel in range(3):
rgba[:, 0, channel] = np.clip(
np.interp(grid, positions, stops[:, channel]) * 255.0, 0, 255
).astype(np.uint8)
rgba[:, 0, 3] = 255
return rgba
# ---------------------------------------------------------------- vertex packing
#: Floats per vertex in the interleaved buffer: position(3) + normal(3) + scalar(1)
#: + valid(1). One buffer per part, so a per-frame upload is one glBufferSubData.
STRIDE_FLOATS = 8
def pack_part(part: MeshPart, *, scalar: Optional[np.ndarray] = None,
valid: Optional[np.ndarray] = None) -> np.ndarray:
"""Interleave one part into the layout the shaders read.
The static case (no scalar yet) marks every vertex invalid, which the shader
draws as the neutral colour: a scene that has never received a measurement
must not be coloured as if it had.
"""
count = int(part.positions_sensor_m.shape[0])
rows = np.empty((count, STRIDE_FLOATS), dtype=np.float32)
rows[:, 0:3] = np.asarray(part.positions_sensor_m, dtype=np.float32)
rows[:, 3:6] = np.asarray(part.normals_sensor, dtype=np.float32)
if scalar is None:
rows[:, 6] = np.nan
else:
values = np.asarray(scalar, dtype=np.float32).reshape(-1)
if values.size != count:
raise error(
"geometry_mismatch",
f"{part.part_id}: {values.size} scalars for {count} vertices",
)
rows[:, 6] = values
if valid is None:
rows[:, 7] = 0.0
else:
flags = np.asarray(valid, dtype=bool).reshape(-1)
if flags.size != count:
raise error(
"geometry_mismatch",
f"{part.part_id}: {flags.size} validity flags for {count} vertices",
)
rows[:, 7] = flags.astype(np.float32)
return freeze(rows, dtype=np.float32)
def pack_scene_part(part: MeshPart) -> tuple[np.ndarray, np.ndarray]:
"""``(vertices, indices)`` as the GPU wants them, both read-only."""
indices = np.asarray(part.triangles, dtype=np.uint32).reshape(-1)
return pack_part(part), freeze(indices, dtype=np.uint32)
def estimate_gpu_bytes(assets: SceneAssets, *, dynamic_part_id: str) -> int:
"""A rough static-plus-one-dynamic budget, for the stats line."""
total = 0
for part in assets.parts:
vertices = int(part.positions_sensor_m.shape[0])
indices = int(part.triangles.size)
total += vertices * STRIDE_FLOATS * 4 + indices * 4
if part.part_id == dynamic_part_id:
# A second copy for the frame in flight.
total += vertices * STRIDE_FLOATS * 4
total += LUT_SIZE * 4
return total
def rest_surface(assets: SceneAssets, identity: Any) -> RenderFrame:
"""The asset's own geometry, with nothing measured.
Used to *clear* a measurement — after a refused frame, on connect, or on a
disconnect — so the screen cannot keep showing the previous acquisition's
colours as if they were current. ``quality="display_only"`` and an
``unavailable`` quantity are what tell the UI to hide the measurement bar.
"""
from orisys_app_shared.scene3d.contracts import rest_descriptor
part = assets.sensing_part()
count = int(part.positions_sensor_m.shape[0])
return RenderFrame(
identity=identity,
asset_hash=assets.asset_hash,
geometry_id=assets.geometry_id,
sensing_part_id=part.part_id,
positions_sensor_m=part.positions_sensor_m,
normals_sensor=part.normals_sensor,
scalar=freeze(np.full(count, np.nan, dtype=np.float32), dtype=np.float32),
valid=freeze(np.zeros(count, dtype=bool), dtype=np.bool_),
scalar_descriptor=rest_descriptor(),
)
# ------------------------------------------------------------------- dispatch
class GLDispatch:
"""Bind the GL entry points for the context Qt actually created."""
def bind_current_context(self, context: Any) -> GLCapabilities:
raise NotImplementedError(f"{_STAGE}: probe the context and bind PyOpenGL")
"""Read the real context. Raises ``gl_unsupported`` when there is none.
Nothing here creates a second context or an EGL surface: Qt owns the
window and the context, and this only asks it what it got.
"""
try:
from OpenGL import GL
except Exception as exc: # noqa: BLE001 - the binding may be missing
raise error(
"gl_unsupported",
f"the OpenGL binding could not be imported: {type(exc).__name__}",
) from exc
if context is None or not context.isValid():
raise error("gl_unsupported", "no current OpenGL context")
fmt = context.format()
profile = profile_from_renderable_type(int(fmt.renderableType()))
try:
vendor = _gl_string(GL, GL.GL_VENDOR)
renderer = _gl_string(GL, GL.GL_RENDERER)
gl_version = _gl_string(GL, GL.GL_VERSION)
glsl_version = _gl_string(GL, GL.GL_SHADING_LANGUAGE_VERSION)
max_texture = int(GL.glGetIntegerv(GL.GL_MAX_TEXTURE_SIZE))
except Exception as exc: # noqa: BLE001
raise error(
"gl_unsupported",
f"the context answered no queries: {type(exc).__name__}: {exc}",
) from exc
from PyQt5.QtGui import QGuiApplication
platform_name = ""
try:
app = QGuiApplication.instance()
platform_name = app.platformName() if app is not None else ""
except Exception: # noqa: BLE001
platform_name = ""
software = is_software_renderer(vendor, renderer)
diagnostics: tuple[Diagnostic, ...] = ()
if software:
diagnostics = (
diagnostic(
"gl_unsupported",
f"{renderer or vendor} is a software rasteriser, not a GPU",
vendor=vendor,
renderer=renderer,
),
)
elif gl_version:
major = _leading_int(gl_version)
if major and major < 3:
diagnostics = (
diagnostic(
"gl_unsupported",
f"OpenGL {gl_version} is older than the 3.3 the shaders need",
version=gl_version,
),
)
result = GLCapabilities(
profile=profile,
vendor=vendor,
renderer=renderer,
gl_version=gl_version,
glsl_version=glsl_version,
qt_platform=platform_name,
software_renderer=software,
max_texture_size=max_texture,
diagnostics=diagnostics,
)
if any(item.severity == "error" for item in diagnostics):
raise ThreeDUnsupported(result)
return result
class ThreeDUnsupported(Exception):
"""Raised with the capabilities that were obtained, so they can be shown."""
def __init__(self, capabilities: GLCapabilities) -> None:
super().__init__("; ".join(item.message for item in capabilities.diagnostics))
self.capabilities = capabilities
self.diagnostic = (
capabilities.diagnostics[0]
if capabilities.diagnostics
else diagnostic("gl_unsupported", "the context is not usable")
)
def _gl_string(GL: Any, name: int) -> str:
value = GL.glGetString(name)
if value is None:
return ""
if isinstance(value, bytes):
return value.decode("utf-8", "replace")
return str(value)
def _leading_int(text: str) -> int:
digits = ""
for char in str(text):
if char.isdigit() or (char == "." and digits):
if char == ".":
break
digits += char
elif digits:
break
try:
return int(digits) if digits else 0
except ValueError: # pragma: no cover - digits only
return 0
# ------------------------------------------------------------------- renderer
class _PartBuffers:
"""One part's GPU state. Kept dumb on purpose: no Qt, no ownership games."""
__slots__ = ("part_id", "role", "vao", "vbo", "ebo", "triangles", "vertices")
def __init__(self, part: MeshPart, vao: int, vbo: int, ebo: int) -> None:
self.part_id = part.part_id
self.role = part.role
self.vao = vao
self.vbo = vbo
self.ebo = ebo
self.triangles = int(part.triangles.shape[0])
self.vertices = int(part.positions_sensor_m.shape[0])
class GLRenderer:
"""Shaders, buffers and draw calls for one scene.
"""Shaders, buffers and draw calls for one scene."""
Two parts, two materials, one draw call per part: the sensing surface is
re-uploaded per frame, the housing is uploaded once and then left alone.
"""
def __init__(self) -> None:
self._capabilities: Optional[GLCapabilities] = None
self._programs: dict[str, int] = {}
self._uniforms: dict[str, dict[str, int]] = {}
self._parts: dict[str, _PartBuffers] = {}
self._lut: int = 0
self._lut_id: str = ""
self._sensing_part_id: str = ""
self._estimated_gpu_bytes: int = 0
self._initialized = False
self._displayed_frames = 0
self._last_upload_ms = 0.0
# ------------------------------------------------------------------ lifetime
def initialize(self, capabilities: GLCapabilities) -> None:
raise NotImplementedError(f"{_STAGE}: compile the shaders for this profile")
"""Compile the two programs for this profile. Requires a current context."""
from OpenGL import GL
if self._initialized:
# Qt may initialize the widget and then have it re-parented into a new
# context; releasing first is what keeps that from leaking a program
# per rebuild.
self.release()
self._capabilities = capabilities
try:
self._programs["scalar"] = _compile_program(GL, capabilities.profile, "scalar")
self._programs["diffuse"] = _compile_program(GL, capabilities.profile, "diffuse")
except Exception as exc: # noqa: BLE001
self.release()
raise error(
"shader_compile_failed",
f"{type(exc).__name__}: {exc}",
profile=capabilities.profile,
) from exc
self._uniforms = {
name: _uniform_locations(GL, program) for name, program in self._programs.items()
}
self._initialized = True
def upload_scene(self, assets: SceneAssets) -> None:
raise NotImplementedError(f"{_STAGE}: upload static buffers and the LUT")
"""Upload every part once. The housing is never uploaded again."""
from OpenGL import GL
self._require_initialized()
self.release_parts()
self._sensing_part_id = assets.sensing_part().part_id
for part in assets.parts:
vertices, indices = pack_scene_part(part)
vao = int(GL.glGenVertexArrays(1))
vbo = int(GL.glGenBuffers(1))
ebo = int(GL.glGenBuffers(1))
GL.glBindVertexArray(vao)
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, vbo)
GL.glBufferData(
GL.GL_ARRAY_BUFFER, vertices.nbytes, vertices, GL.GL_STATIC_DRAW
)
GL.glBindBuffer(GL.GL_ELEMENT_ARRAY_BUFFER, ebo)
GL.glBufferData(
GL.GL_ELEMENT_ARRAY_BUFFER, indices.nbytes, indices, GL.GL_STATIC_DRAW
)
_bind_attributes(GL)
GL.glBindVertexArray(0)
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, 0)
GL.glBindBuffer(GL.GL_ELEMENT_ARRAY_BUFFER, 0)
self._parts[part.part_id] = _PartBuffers(part, vao, vbo, ebo)
self._estimated_gpu_bytes = estimate_gpu_bytes(
assets, dynamic_part_id=self._sensing_part_id
)
def update_surface(self, frame: RenderFrame) -> None:
raise NotImplementedError(f"{_STAGE}: update the dynamic surface buffers")
"""Update the sensing part's vertices in place: no index rebuild, ever."""
from OpenGL import GL
self._require_initialized()
buffers = self._parts.get(frame.sensing_part_id)
if buffers is None:
raise error(
"geometry_mismatch",
f"the scene has no part {frame.sensing_part_id!r} to update",
)
rows = pack_part(
_PartLayout(frame.sensing_part_id, frame.positions_sensor_m, frame.normals_sensor),
scalar=frame.scalar,
valid=frame.valid,
)
if int(rows.shape[0]) != buffers.vertices:
raise error(
"geometry_mismatch",
f"{buffers.part_id}: {rows.shape[0]} vertices for "
f"{buffers.vertices} uploaded",
)
started = time.perf_counter()
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, buffers.vbo)
GL.glBufferSubData(GL.GL_ARRAY_BUFFER, 0, rows.nbytes, rows)
GL.glBindBuffer(GL.GL_ARRAY_BUFFER, 0)
self._last_upload_ms = (time.perf_counter() - started) * 1000.0
def draw(
self,
@@ -93,7 +567,240 @@ class GLRenderer:
framebuffer: int,
viewport: tuple[int, int],
) -> RenderStats:
raise NotImplementedError(f"{_STAGE}: draw one frame")
"""One scene draw into the framebuffer Qt owns (never target 0)."""
from OpenGL import GL
self._require_initialized()
started = time.perf_counter()
width = max(int(viewport[0]), 1)
height = max(int(viewport[1]), 1)
GL.glBindFramebuffer(GL.GL_FRAMEBUFFER, int(framebuffer))
GL.glViewport(0, 0, width, height)
GL.glEnable(GL.GL_DEPTH_TEST)
GL.glClearColor(0.06, 0.07, 0.09, 1.0)
GL.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT)
view = np.asarray(camera.view, dtype=np.float32).reshape(4, 4)
projection = np.asarray(camera.projection, dtype=np.float32).reshape(4, 4)
scale = settings.color_scale
lower = float(scale.lower) if scale is not None else 0.0
upper = float(scale.upper) if scale is not None else 1.0
if upper <= lower:
upper = lower + 1.0
self._ensure_lut("diverging" if (scale and scale.descriptor.signed) else "sequential")
for part in self._parts.values():
if part.role == "housing" and not settings.housing_visible:
continue
program = self._programs["scalar" if part.role == "sensing" else "diffuse"]
locations = self._uniforms["scalar" if part.role == "sensing" else "diffuse"]
GL.glUseProgram(program)
_set_mat4(GL, locations.get("u_view"), view)
_set_mat4(GL, locations.get("u_projection"), projection)
if part.role == "sensing":
_set_float(GL, locations.get("u_lower"), lower)
_set_float(GL, locations.get("u_upper"), upper)
_set_vec4(GL, locations.get("u_invalid_color"), INVALID_COLOR)
if locations.get("u_lut") is not None:
GL.glActiveTexture(GL.GL_TEXTURE0)
GL.glBindTexture(GL.GL_TEXTURE_2D, self._lut)
GL.glUniform1i(locations["u_lut"], 0)
else:
_set_vec3(GL, locations.get("u_light_direction"), (0.3, -0.5, 0.8))
_set_vec4(GL, locations.get("u_base_color"), _base_color(part.part_id))
GL.glBindVertexArray(part.vao)
GL.glDrawElements(
GL.GL_TRIANGLES, part.triangles * 3, GL.GL_UNSIGNED_INT, None
)
GL.glBindVertexArray(0)
GL.glUseProgram(0)
self._displayed_frames += 1
return RenderStats(
displayed_frames=self._displayed_frames,
compose_ms=0.0,
upload_ms=self._last_upload_ms,
paint_ms=(time.perf_counter() - started) * 1000.0,
estimated_gpu_bytes=self._estimated_gpu_bytes,
)
def release(self) -> None:
raise NotImplementedError(f"{_STAGE}: release GPU resources")
"""Delete every GPU resource. Idempotent; the owning context must be current."""
from OpenGL import GL
self.release_parts()
for program in self._programs.values():
try:
GL.glDeleteProgram(int(program))
except Exception: # noqa: BLE001 - nothing left to delete
pass
self._programs = {}
self._uniforms = {}
if self._lut:
try:
GL.glDeleteTextures([int(self._lut)])
except Exception: # noqa: BLE001
pass
self._lut = 0
self._lut_id = ""
self._initialized = False
self._estimated_gpu_bytes = 0
self._displayed_frames = 0
def release_parts(self) -> None:
from OpenGL import GL
for part in self._parts.values():
try:
GL.glDeleteVertexArrays(1, [int(part.vao)])
GL.glDeleteBuffers(1, [int(part.vbo)])
GL.glDeleteBuffers(1, [int(part.ebo)])
except Exception: # noqa: BLE001
pass
self._parts = {}
# ------------------------------------------------------------------- helpers
def _require_initialized(self) -> None:
if not self._initialized:
raise error("gl_unsupported", "the renderer was not initialized")
def _ensure_lut(self, lut_id: str) -> None:
from OpenGL import GL
if self._lut and self._lut_id == lut_id:
return
if self._lut:
GL.glDeleteTextures([int(self._lut)])
self._lut = 0
data = lut_rgba(lut_id)
self._lut = int(GL.glGenTextures(1))
GL.glBindTexture(GL.GL_TEXTURE_2D, self._lut)
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MIN_FILTER, GL.GL_LINEAR)
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_MAG_FILTER, GL.GL_LINEAR)
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP_TO_EDGE)
GL.glTexParameteri(GL.GL_TEXTURE_2D, GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP_TO_EDGE)
GL.glTexImage2D(
GL.GL_TEXTURE_2D, 0, GL.GL_RGBA8, LUT_SIZE, 1, 0, GL.GL_RGBA,
GL.GL_UNSIGNED_BYTE, data,
)
GL.glBindTexture(GL.GL_TEXTURE_2D, 0)
self._lut_id = lut_id
def capabilities(self) -> Optional[GLCapabilities]:
return self._capabilities
@property
def part_ids(self) -> tuple[str, ...]:
return tuple(self._parts)
@property
def is_initialized(self) -> bool:
return self._initialized
class _PartLayout:
"""A minimal stand-in for :class:`MeshPart` when updating an existing buffer."""
__slots__ = ("part_id", "positions_sensor_m", "normals_sensor")
def __init__(self, part_id: str, positions: np.ndarray, normals: np.ndarray) -> None:
self.part_id = part_id
self.positions_sensor_m = positions
self.normals_sensor = normals
#: Base colours for non-sensing parts, by part id: the housing is grey/graphite so
#: it never competes with the measurement colour scale.
_PART_COLORS: dict[str, tuple[float, float, float, float]] = {
"housing": (0.32, 0.33, 0.35, 1.0),
}
def _base_color(part_id: str) -> tuple[float, float, float, float]:
return _PART_COLORS.get(part_id, (0.45, 0.45, 0.47, 1.0))
def _bind_attributes(GL: Any) -> None:
stride = STRIDE_FLOATS * 4
GL.glEnableVertexAttribArray(0)
GL.glVertexAttribPointer(0, 3, GL.GL_FLOAT, GL.GL_FALSE, stride, None)
GL.glEnableVertexAttribArray(1)
GL.glVertexAttribPointer(1, 3, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(3))
GL.glEnableVertexAttribArray(2)
GL.glVertexAttribPointer(2, 1, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(6))
GL.glEnableVertexAttribArray(3)
GL.glVertexAttribPointer(3, 1, GL.GL_FLOAT, GL.GL_FALSE, stride, _offset(7))
def _offset(floats: int) -> Any:
from OpenGL import GL
return GL.ctypes.c_void_p(floats * 4)
def _compile_program(GL: Any, profile: Profile, name: str) -> int:
vertex_source, fragment_source = shader_source(profile, name)
program = int(GL.glCreateProgram())
shaders = []
for kind, source in ((GL.GL_VERTEX_SHADER, vertex_source), (GL.GL_FRAGMENT_SHADER, fragment_source)):
shader = int(GL.glCreateShader(kind))
GL.glShaderSource(shader, source)
GL.glCompileShader(shader)
if not GL.glGetShaderiv(shader, GL.GL_COMPILE_STATUS):
log = GL.glGetShaderInfoLog(shader)
GL.glDeleteShader(shader)
raise RuntimeError(f"{name} shader: {_decode(log)}")
GL.glAttachShader(program, shader)
shaders.append(shader)
GL.glLinkProgram(program)
if not GL.glGetProgramiv(program, GL.GL_LINK_STATUS):
log = GL.glGetProgramInfoLog(program)
for shader in shaders:
GL.glDeleteShader(shader)
GL.glDeleteProgram(program)
raise RuntimeError(f"{name} program: {_decode(log)}")
for shader in shaders:
GL.glDetachShader(program, shader)
GL.glDeleteShader(shader)
return program
def _uniform_locations(GL: Any, program: int) -> dict[str, int]:
names = (
"u_view", "u_projection", "u_scalar", "u_lower", "u_upper",
"u_invalid_color", "u_lut", "u_light_direction", "u_base_color",
)
out: dict[str, int] = {}
for name in names:
location = GL.glGetUniformLocation(program, name)
if location is not None and int(location) >= 0:
out[name] = int(location)
return out
def _decode(value: Any) -> str:
if isinstance(value, bytes):
return value.decode("utf-8", "replace").strip()
return str(value).strip()
def _set_mat4(GL: Any, location: Optional[int], values: np.ndarray) -> None:
if location is None:
return
# Column-major upload, explicitly: the layout of the NumPy array must not
# decide what the coordinates mean (design document §3.1).
GL.glUniformMatrix4fv(location, 1, GL.GL_FALSE, np.ascontiguousarray(values.T))
def _set_float(GL: Any, location: Optional[int], value: float) -> None:
if location is not None:
GL.glUniform1f(location, float(value))
def _set_vec3(GL: Any, location: Optional[int], values: tuple[float, float, float]) -> None:
if location is not None:
GL.glUniform3f(location, float(values[0]), float(values[1]), float(values[2]))
def _set_vec4(GL: Any, location: Optional[int], values: tuple[float, float, float, float]) -> None:
if location is not None:
GL.glUniform4f(
location, float(values[0]), float(values[1]), float(values[2]), float(values[3])
)
apps/studio/src/orisys_studio/scene3d/widget.py
+245 -14
View File
@@ -3,18 +3,24 @@
Stage 1 (a static scene, resize, close) and stage 4 (product integration) in
``docs/architecture/studio-3d-stages.md``.
Qt owns the context and the framebuffer, so the widget must never bind target 0:
it draws into the FBO Qt has already set up for the current ``paintGL`` call. The
widget holds CPU-side data (the latest :class:`RenderFrame`) and schedules the
upload; it does no composing, no asset IO and no SDK access, so a slow render can
only drop *display* frames.
Qt owns the context and the framebuffer, so the widget never binds target 0: it
draws into ``defaultFramebufferObject()``, which is the FBO of the current
``paintGL`` call. The widget holds CPU-side data (the latest ``RenderFrame``) and
schedules the upload; it does no composing, no asset IO and no SDK access, so a
slow render can only drop *display* frames.
Failure is a state, not an exception: if the context is unusable (no GL, software
rasteriser, shader failure) the widget reports it once and stays empty, while the
two-dimensional application carries on.
"""
from __future__ import annotations
import time
from typing import Optional
from PyQt5.QtCore import pyqtSignal
from PyQt5.QtCore import QEvent, Qt, pyqtSignal
from PyQt5.QtGui import QMouseEvent, QWheelEvent
from PyQt5.QtWidgets import QOpenGLWidget, QWidget
from orisys_app_shared.scene3d.contracts import (
@@ -24,11 +30,19 @@ from orisys_app_shared.scene3d.contracts import (
RenderFrame,
RenderStats,
SceneAssets,
diagnostic,
)
from orisys_studio.scene3d.camera import OrbitCamera
from orisys_studio.scene3d.gl_backend import GLRenderer
from orisys_studio.scene3d.gl_backend import (
GLDispatch,
GLRenderer,
ThreeDUnsupported,
rest_surface,
)
_STAGE = "stage 1 (static scene) and stage 4 (product integration)"
#: How often render statistics are published. The numbers are for a person to
#: read, and a signal per frame would cost more than the drawing.
STATS_INTERVAL_S = 0.5
class Sensor3DWidget(QOpenGLWidget):
@@ -43,21 +57,36 @@ class Sensor3DWidget(QOpenGLWidget):
renderer: GLRenderer,
camera: OrbitCamera,
parent: Optional[QWidget] = None,
*,
dispatch: Optional[GLDispatch] = None,
) -> None:
super().__init__(parent)
self._renderer = renderer
self._camera = camera
self._dispatch = dispatch or GLDispatch()
self._assets: Optional[SceneAssets] = None
self._frame: Optional[RenderFrame] = None
self._settings = DisplaySettings()
self._capabilities: Optional[GLCapabilities] = None
self._diagnostic: Optional[Diagnostic] = None
self._needs_scene_upload = False
self._needs_surface_upload = False
self._last_stats_at = 0.0
self._last_press: Optional[tuple[float, float]] = None
self._unavailable = False
#: Frames the display never showed, set by the coordinator. Never a
#: recording figure.
self._superseded_frames = 0
#: Frames refused because they did not belong to this scene.
self._rejected_frames = 0
# ------------------------------------------------------------- CPU-side API
def set_scene(self, assets: SceneAssets) -> None:
"""Retain the asset and schedule its upload for the next current context."""
self._assets = assets
self._frame = None
self._needs_scene_upload = True
self._unavailable = False
self.update()
def present(self, frame: RenderFrame) -> None:
@@ -70,15 +99,217 @@ class Sensor3DWidget(QOpenGLWidget):
self._settings = settings
self.update()
def camera(self) -> OrbitCamera:
return self._camera
def capabilities(self) -> Optional[GLCapabilities]:
return self._capabilities
def diagnostic(self) -> Optional[Diagnostic]:
return self._diagnostic
@property
def unavailable(self) -> bool:
return self._unavailable
def viewport_pixels(self) -> tuple[int, int]:
"""The physical framebuffer size: device pixels, not logical ones."""
ratio = float(self.devicePixelRatioF() or 1.0)
return (
max(int(round(self.width() * ratio)), 1),
max(int(round(self.height() * ratio)), 1),
)
# ------------------------------------------------------------ Qt overrides
def initializeGL(self) -> None: # pragma: no cover - needs a real context
raise NotImplementedError(f"{_STAGE}: bind the backend and connect teardown")
try:
from OpenGL import GL
# Both materials are double-sided by design (a thin plate seen from
# behind must not vanish), so culling stays off.
GL.glDisable(GL.GL_CULL_FACE)
capabilities = self._dispatch.bind_current_context(self.context())
self._renderer.initialize(capabilities)
except ThreeDUnsupported as exc:
self._fail(exc.diagnostic)
self._capabilities = exc.capabilities
self.capabilitiesChanged.emit(exc.capabilities)
return
except Exception as exc: # noqa: BLE001 - any GL failure is a state, not a crash
self._fail(
getattr(exc, "diagnostic", None)
or diagnostic("gl_unsupported", f"{type(exc).__name__}: {exc}")
)
return
self._capabilities = capabilities
self._diagnostic = None
self._unavailable = False
# A fresh context holds nothing, whatever the CPU still has: re-upload
# from the retained assets. Re-parenting a window rebuilds the context,
# and without this the scene would simply vanish.
self._needs_scene_upload = self._assets is not None
self._needs_surface_upload = self._frame is not None
context = self.context()
if context is not None:
context.aboutToBeDestroyed.connect(self.release_resources)
self.capabilitiesChanged.emit(capabilities)
def paintGL(self) -> None: # pragma: no cover - needs a real context
raise NotImplementedError(f"{_STAGE}: upload what changed and draw")
if self._unavailable or not self._renderer.is_initialized:
return
started = time.perf_counter()
try:
if self._needs_scene_upload and self._assets is not None:
self._renderer.upload_scene(self._assets)
self._needs_scene_upload = False
self._needs_surface_upload = self._frame is not None
# The default view belongs to the asset, not to a user preference:
# adopt it, then frame the whole scene from that orientation. Going
# straight to fit() would keep whatever angles the camera happened
# to start with (looking along the sensor's long axis, which shows
# a flat surface edge-on).
presets = self._assets.camera_presets
if presets:
try:
self._camera.set_preset(presets[0])
except ValueError:
pass
self._camera.fit(self._assets.bounds_sensor_m, self._aspect())
if self._needs_surface_upload and self._frame is not None:
frame, self._frame = self._frame, None
self._needs_surface_upload = False
if self._assets is not None and frame.geometry_id != self._assets.geometry_id:
# A frame from another geometry is *rejected*, not fatal: the
# scene stays usable, and leaving the old measurement on screen
# would be claiming it is the current one.
self._reject(
diagnostic(
"geometry_mismatch",
"the frame belongs to a different geometry than the scene",
frame=frame.geometry_id,
scene=self._assets.geometry_id,
)
)
if self._assets is not None:
self._renderer.update_surface(
rest_surface(self._assets, frame.identity)
)
else:
self._renderer.update_surface(frame)
stats = self._renderer.draw(
self._camera.matrices(self.viewport_pixels()),
self._settings,
int(self.defaultFramebufferObject()),
self.viewport_pixels(),
)
except Exception as exc: # noqa: BLE001
# A failure while drawing is the GL stack being unusable, not a shader
# that failed to compile (that case raises its own diagnostic above).
self._fail(
getattr(exc, "diagnostic", None)
or diagnostic("gl_unsupported", f"{type(exc).__name__}: {exc}")
)
return
now = time.perf_counter()
if now - self._last_stats_at >= STATS_INTERVAL_S:
self._last_stats_at = now
self.renderStatsChanged.emit(
RenderStats(
displayed_frames=stats.displayed_frames,
superseded_frames=self._superseded,
compose_ms=stats.compose_ms,
upload_ms=stats.upload_ms,
paint_ms=(now - started) * 1000.0,
estimated_gpu_bytes=stats.estimated_gpu_bytes,
)
)
def resizeGL(self, w: int, h: int) -> None: # pragma: no cover
raise NotImplementedError(f"{_STAGE}: calibrate DPR and update the projection")
def resizeGL(self, w: int, h: int) -> None: # pragma: no cover - needs a context
"""Nothing to cache: the projection is rebuilt from the live viewport."""
self._camera.aspect = max(float(w), 1.0) / max(float(h), 1.0)
def release_resources(self) -> None: # pragma: no cover - needs a real context
raise NotImplementedError(f"{_STAGE}: makeCurrent, release, doneCurrent")
def release_resources(self) -> None: # pragma: no cover - needs a context
"""Free GPU resources from this widget's own context. Safe to repeat."""
if not self._renderer.is_initialized:
return
try:
self.makeCurrent()
self._renderer.release()
except Exception: # noqa: BLE001 - teardown must not raise on the way out
pass
finally:
try:
self.doneCurrent()
except Exception: # noqa: BLE001
pass
def hideEvent(self, event: QEvent) -> None: # pragma: no cover - Qt event
super().hideEvent(event)
# Hiding suspends drawing; the coordinator stops composing on this signal.
self._needs_surface_upload = False
def showEvent(self, event: QEvent) -> None: # pragma: no cover - Qt event
super().showEvent(event)
self._needs_surface_upload = self._frame is not None
self.update()
# --------------------------------------------------------------- interaction
def mousePressEvent(self, event: QMouseEvent) -> None: # pragma: no cover
self._last_press = (float(event.x()), float(event.y()))
super().mousePressEvent(event)
def mouseReleaseEvent(self, event: QMouseEvent) -> None: # pragma: no cover
self._last_press = None
super().mouseReleaseEvent(event)
def mouseMoveEvent(self, event: QMouseEvent) -> None: # pragma: no cover
if self._last_press is None:
super().mouseMoveEvent(event)
return
x, y = float(event.x()), float(event.y())
delta = (x - self._last_press[0], y - self._last_press[1])
self._last_press = (x, y)
viewport = (max(self.width(), 1), max(self.height(), 1))
buttons = event.buttons()
panning = bool(buttons & Qt.MiddleButton) or bool(
buttons & Qt.LeftButton and event.modifiers() & Qt.ShiftModifier
)
if panning:
self._camera.pan(delta, viewport)
elif buttons & Qt.LeftButton:
self._camera.orbit(delta, viewport)
self.update()
def wheelEvent(self, event: QWheelEvent) -> None: # pragma: no cover
steps = float(event.angleDelta().y()) / 120.0
self._camera.zoom(steps)
self.update()
# ------------------------------------------------------------------ internal
@property
def _superseded(self) -> int:
return self._superseded_frames
def set_superseded_frames(self, count: int) -> None:
"""Set by the coordinator: display drops, never recording drops."""
self._superseded_frames = int(count)
def _aspect(self) -> float:
width, height = self.viewport_pixels()
return width / height
def _fail(self, item: Diagnostic) -> None:
"""The whole view is out of action (no context, no shaders, GL error)."""
self._unavailable = True
self._diagnostic = item
self.diagnosticRaised.emit(item)
def _reject(self, item: Diagnostic) -> None:
"""One frame was refused. Reported, counted, and otherwise ignored."""
self._rejected_frames += 1
self._diagnostic = item
self.diagnosticRaised.emit(item)
@property
def rejected_frames(self) -> int:
return self._rejected_frames