Compare commits
110 Commits
28f4663a5f
...
dev
| Author | SHA1 | Date | |
|---|---|---|---|
| d1f168c9c7 | |||
| e9710e140b | |||
| ff7614a6a0 | |||
| f7f4682ff9 | |||
| 9c951e9d3c | |||
| bb15b750fb | |||
| 3953747f17 | |||
| ae64c65c45 | |||
| 120b8c2cf6 | |||
| 0784eacf51 | |||
| 68edb0dc4c | |||
| 4c3d0f8864 | |||
| ceb06f5702 | |||
| 8c147f03aa | |||
| 34cb83d4a4 | |||
| 38d3e67b00 | |||
| 80f7da5a20 | |||
| 230b04a79d | |||
| 6e3d509239 | |||
| 333fa8fcff | |||
| f700102067 | |||
| b5d7533e24 | |||
| 4c1ab5c27e | |||
| cde832eaca | |||
| 73b1fa7fa4 | |||
| d5b938738e | |||
| e2ac36269c | |||
| 07da372204 | |||
| 0e903792bd | |||
| 153f51df11 | |||
| 2e7b65298b | |||
| e2b906a640 | |||
| 45419a59c4 | |||
| 05b2354d0d | |||
| 1cf78c9d71 | |||
| 14fbfec62e | |||
| 11962cdc34 | |||
| ae5bf0c208 | |||
| 7f8e5d0975 | |||
| 35190562e5 | |||
| b956d071ea | |||
| 0beb36a1af | |||
| 1adb1c30f0 | |||
| 514874e2f4 | |||
| 0d5d8ff70d | |||
| 5f78115429 | |||
| 488c8b0bc1 | |||
| 94f38bc389 | |||
| 174bceabc4 | |||
| 388f34ca85 | |||
| 35af74f639 | |||
| 57bef85e32 | |||
| 9537cf91a1 | |||
| 9c23ba3d50 | |||
| 96f4428230 | |||
| 9cbba08d68 | |||
| ad680b77dc | |||
| 75fa98eff0 | |||
| bc059e3539 | |||
| 5be2d9759a | |||
| 32823a7666 | |||
| 26168852ef | |||
| 36e0c22de6 | |||
| a387b8b693 | |||
| 8a4be5f8f2 | |||
| 68910e63c1 | |||
| 577e3fe528 | |||
| 32d4164471 | |||
| d59e1cb7fb | |||
| 2cb152b76e | |||
| ab97efe039 | |||
| 135857b6de | |||
| 8a7760f12b | |||
| 4583dc3a40 | |||
| 1ceb0d9ca4 | |||
| 44bcc19426 | |||
| 2e09c9360e | |||
| 93b4517ebf | |||
| 9a69f01278 | |||
| 4acdce4456 | |||
| d7c5845646 | |||
| 3117ddd5b9 | |||
| 8b2f76c76f | |||
| 8d1772300f | |||
| 044fe02107 | |||
| b154a3acbb | |||
| 2e3d0a0609 | |||
| 28e6c7c8a9 | |||
| ee3ef3e961 | |||
| 62bc8dbff8 | |||
| 0e7fd2eeaf | |||
| 11a747a771 | |||
| 7b12d17595 | |||
| 87ac152907 | |||
| 3d1509df9d | |||
| 9ee6ab9470 | |||
| a9c26fd1fb | |||
| 30f9deb5b2 | |||
| fe6c880473 | |||
| d58fa1caa2 | |||
| 4b38dbb6b6 | |||
| 3d1039b52c | |||
| c2fb0b12b5 | |||
| c77a64ffe7 | |||
| 8c5a20b5cb | |||
| 8706e405d7 | |||
| 68b6dadff4 | |||
| edc6e42191 | |||
| 63f9622efb | |||
| 42e2264f06 |
+15
-2
@@ -5,8 +5,13 @@ __pycache__/
|
||||
|
||||
# Packaging / build outputs
|
||||
/build/nuitka/
|
||||
/build/lib/orisys/
|
||||
/build/build/
|
||||
/build/bdist.*/
|
||||
/build/dist/
|
||||
/build/lib/
|
||||
/build/pyinstaller_work/
|
||||
/build/release/reports/
|
||||
/build/*.spec
|
||||
/dist/
|
||||
/dist_obfuscated/
|
||||
/dist_pyd/
|
||||
@@ -19,12 +24,14 @@ __pycache__/
|
||||
# Local tools / editor state
|
||||
/.cursor/
|
||||
/.vscode/
|
||||
/_perf/
|
||||
|
||||
# Local data / models
|
||||
/models/
|
||||
/data/*
|
||||
!/data/sample/
|
||||
!/data/sample/**
|
||||
!/data/calibrations/
|
||||
|
||||
# Generated docs / tooling outputs
|
||||
/docs/.quarto/
|
||||
@@ -37,4 +44,10 @@ __pycache__/
|
||||
|
||||
# Legacy / scratch folders
|
||||
/old_orisys/
|
||||
/tools/calibration/
|
||||
/tools/calibration/
|
||||
|
||||
# Scratch output of examples/ (multi_stream.py's default --out-dir)
|
||||
/outputs/
|
||||
|
||||
# Agent scratch: one directory per task, see D:\orisys_ws\AGENTS.md
|
||||
.scratch/
|
||||
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
# Non-Python files needed for sdist -> wheel builds from an unpacked sdist.
|
||||
include pyproject.toml
|
||||
include setup.py
|
||||
include README.md
|
||||
# Build-time input for the PyPI long description (read from the checkout).
|
||||
include docs/pypi_readme.md
|
||||
recursive-include config *.json
|
||||
recursive-include sensors *
|
||||
recursive-include build/common *.py
|
||||
recursive-include build/wheel *.py
|
||||
@@ -4,11 +4,14 @@ Orisys SDK is a Python toolkit for optical tactile sensing. It provides image ca
|
||||
stitching, dense optical flow, Helmholtz-Hodge decomposition, contact/depth estimation,
|
||||
and several runnable examples for camera and video-based workflows.
|
||||
|
||||
PyPI: [https://pypi.org/project/orisys/](https://pypi.org/project/orisys/)
|
||||
|
||||
## Repository Layout
|
||||
|
||||
- `src/orisys/`: installable SDK package
|
||||
- `config/`: default and lab/runtime JSON configurations
|
||||
- `examples/`: user-facing example scripts
|
||||
- `sensors/`: product bundles containing the current `sensor.json` configurations and assets
|
||||
- `config/`: legacy JSON configurations retained for compatibility
|
||||
- `examples/`: runnable demos (`basic_pipeline.py`, `mini_pipeline.py`, `finger_pipeline*.py`) plus shared `gui/`, `pipeline/`, `viz/`
|
||||
- `scripts/`: maintenance, calibration, packaging, and demo scripts
|
||||
- `tools/`: non-package tooling such as patch-generation utilities
|
||||
- `tests/`: automated pytest suite
|
||||
@@ -49,38 +52,95 @@ pytest
|
||||
|
||||
### 4. Run an example
|
||||
|
||||
ORY-BOX:
|
||||
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c ./config/ddjx01.json
|
||||
python examples/basic_pipeline.py -v 0 -c box
|
||||
```
|
||||
|
||||
ORY-MINI:
|
||||
|
||||
```bash
|
||||
python examples/mini_pipeline.py -v 0 -c mini
|
||||
```
|
||||
|
||||
ORY-FINGER:
|
||||
|
||||
```bash
|
||||
python examples/finger_pipeline.py --camera_id 0 --config finger
|
||||
```
|
||||
|
||||
Run commands from the repository root.
|
||||
|
||||
## Configuration
|
||||
|
||||
The canonical configuration files now live in the repo-level `config/` folder.
|
||||
Product configurations live in `sensors/<name>/sensor.json`. The repo-level `config/`
|
||||
folder remains available for older configurations and aliases.
|
||||
The package loader `orisys.configs` resolves configs from:
|
||||
|
||||
1. an explicit JSON file path
|
||||
2. a config file stem under `config/`
|
||||
3. a backward-compatible JSON `"name"` match, such as `"prototype"`
|
||||
2. `sensors/<name>/sensor.json` (for example `box`, `mini`, `finger`)
|
||||
3. a file stem under the legacy `config/` folder, including aliases such as `dd02` -> `dd02-ov`
|
||||
4. a backward-compatible JSON `"name"` match, such as `"prototype"`
|
||||
|
||||
Examples:
|
||||
|
||||
```python
|
||||
import orisys
|
||||
|
||||
sensor = orisys.Sensor(0, config_name="ddjx01")
|
||||
cfg = orisys.export_config("ddjx01", "./config/exported_config.json")
|
||||
sensor = orisys.Sensor(0, config_name="box")
|
||||
cfg = orisys.export_config("box", "./config/exported_config.json")
|
||||
```
|
||||
|
||||
For multiple physical sensors, keep calibration files in a user-selected directory:
|
||||
`<calibration_dir>/<product>/<device_id>.json` (for example,
|
||||
`data/calibrations/box/BOX-001.json`). Each file uses the existing calibration JSON
|
||||
format and must contain matching `identity.sensor_type` and `identity.sn`, Newton
|
||||
curves for FN/FX/FY, and a compatible measurement pipeline. Enter the device ID
|
||||
manually until hardware identification is available:
|
||||
|
||||
The source checkout provides `data/calibrations/{box,mini,finger}/` folders; device
|
||||
files placed there are ignored by Git. This location is optional: pass
|
||||
`--calibration-dir data/calibrations`
|
||||
and `--device-id BOX-001` to `examples/basic_pipeline.py` to select a file there.
|
||||
Installed wheels do not create or search this directory automatically.
|
||||
|
||||
```python
|
||||
sensor = orisys.Sensor(0, config_name="box", calibration_dir="data/calibrations", device_id="BOX-001")
|
||||
```
|
||||
|
||||
If that ID has no file, the sensor reports raw force; it never falls back to the
|
||||
product's default. An invalid or mismatched file raises `CalibrationError`.
|
||||
`FingerRuntime.open` accepts the same `calibration_dir` and `device_id` arguments.
|
||||
Existing calls without a device ID still use the product bundle default, and
|
||||
explicit `cal_path` remains available for one-off use. `cal_path` and
|
||||
`calibration_dir` cannot be combined. Pass `device_id` with `cal_path` to validate
|
||||
that explicit file against the selected device.
|
||||
|
||||
## Examples and Tools
|
||||
|
||||
- Example entry points: `examples/README.md`
|
||||
- User documentation: `docs/get_started.md`
|
||||
- Packaging helper: `build/build_nuitka.py` (compiled SDK wheel)
|
||||
- GUI standalone folder: `build/build_gui_win.bat` → `dist/OrisysDashboard/`
|
||||
- Contributor start: [docs/get_started.en.md](docs/get_started.en.md)
|
||||
- Chinese installation guide: [docs/get_started.md](docs/get_started.md)
|
||||
- Build launcher: `python build/run.py` (see `build/README.md`)
|
||||
- Build and publish guide: [docs/publishing.en.md](docs/publishing.en.md)
|
||||
- Chinese release runbook: [docs/publishing.md](docs/publishing.md)
|
||||
- Specialized utilities: `tools/`
|
||||
|
||||
## Publishing (Linux manylinux)
|
||||
|
||||
To build PyPI-compliant CPython 3.10/3.11/3.12 Linux wheels from Windows Docker Desktop:
|
||||
|
||||
```powershell
|
||||
python build/run.py manylinux
|
||||
```
|
||||
|
||||
Upload candidates land in `dist/repaired/`. Full TestPyPI → PyPI steps:
|
||||
`docs/publishing.md`.
|
||||
|
||||
## Notes for Collaborators
|
||||
|
||||
- Generated build outputs such as `build/`, `dist/`, and `SDK/` are intentionally ignored.
|
||||
- Large or generated tooling outputs should stay under ignored output folders rather than be committed.
|
||||
- If you add new images or arrays that should be versioned, review `.gitignore` first.
|
||||
- Never commit PyPI / TestPyPI API tokens.
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
|
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 48 KiB |
@@ -1,44 +0,0 @@
|
||||
# -*- mode: python ; coding: utf-8 -*-
|
||||
|
||||
|
||||
a = Analysis(
|
||||
['D:\\repo\\orisyssdk_git\\examples\\finger_pipeline_3dview.py'],
|
||||
pathex=[],
|
||||
binaries=[],
|
||||
datas=[('D:\\repo\\orisyssdk_git\\config', 'config'), ('D:\\repo\\orisyssdk_git\\assets', 'assets')],
|
||||
hiddenimports=['PyQt5', 'pyqtgraph', 'open3d', 'cv2', 'orisys', 'gui', 'pipeline', 'viz', 'app_paths', 'plotly', '_plotly_utils', 'open3d.cpu.pybind'],
|
||||
hookspath=[],
|
||||
hooksconfig={},
|
||||
runtime_hooks=[],
|
||||
excludes=['cupy', 'cupyx', 'pyopencl', 'tkinter', 'pip', 'wheel', 'matplotlib'],
|
||||
noarchive=False,
|
||||
optimize=0,
|
||||
)
|
||||
pyz = PYZ(a.pure)
|
||||
|
||||
exe = EXE(
|
||||
pyz,
|
||||
a.scripts,
|
||||
[],
|
||||
exclude_binaries=True,
|
||||
name='OrisysDashboard',
|
||||
debug=False,
|
||||
bootloader_ignore_signals=False,
|
||||
strip=False,
|
||||
upx=True,
|
||||
console=False,
|
||||
disable_windowed_traceback=False,
|
||||
argv_emulation=False,
|
||||
target_arch=None,
|
||||
codesign_identity=None,
|
||||
entitlements_file=None,
|
||||
)
|
||||
coll = COLLECT(
|
||||
exe,
|
||||
a.binaries,
|
||||
a.datas,
|
||||
strip=False,
|
||||
upx=True,
|
||||
upx_exclude=[],
|
||||
name='OrisysDashboard',
|
||||
)
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
# Build tooling
|
||||
|
||||
Use one launcher from the repo root:
|
||||
|
||||
```bash
|
||||
python build/run.py <command>
|
||||
```
|
||||
|
||||
```text
|
||||
build/
|
||||
run.py << single entrypoint
|
||||
README.md
|
||||
common/ shared helpers (paths, pyproject, verify)
|
||||
wheel/ SDK .whl (Nuitka + manylinux Docker)
|
||||
release/ local release gate (preflight / audit)
|
||||
gui/
|
||||
common/ shared PyInstaller driver
|
||||
ory_finger/ ORY-FINGER dashboard
|
||||
ory_mini/ ORY-MINI internal GUI (scaffold)
|
||||
ory_box/ ORY-BOX internal GUI (scaffold)
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### SDK wheel
|
||||
|
||||
```bash
|
||||
python build/run.py wheel-win
|
||||
python build/run.py wheel-manylinux
|
||||
python build/run.py wheel-manylinux -Architecture aarch64 -PythonVersions 3.10
|
||||
|
||||
# Single Version
|
||||
python build/run.py wheel
|
||||
# Inside Docker
|
||||
python build/run.py wheel --target pypi-linux --verify
|
||||
```
|
||||
|
||||
Direct module paths (same behavior):
|
||||
|
||||
```bash
|
||||
python build/wheel/build.py
|
||||
build\wheel\win.bat
|
||||
.\build\wheel\docker\run.ps1 -SkipImageBuild # reuse the image when only sources changed
|
||||
```
|
||||
|
||||
`wheel-manylinux` defaults to `x86_64`; `-Architecture aarch64` builds ARM64 Linux
|
||||
wheels in `quay.io/pypa/manylinux_2_28_aarch64`. Docker Desktop must use Linux
|
||||
containers and support ARM64 emulation on an Intel/AMD host. This runs the ARM64
|
||||
compiler under QEMU, so compilation can be much slower than on native ARM64.
|
||||
The default Python versions remain 3.10, 3.11 and 3.12; `-PythonVersions` selects
|
||||
which ABIs to compile. The image still prepares tooling for all three ABIs.
|
||||
|
||||
The default image names are `orisys-manylinux` and `orisys-manylinux-aarch64`.
|
||||
`-ImageName` overrides the name; a reused image must match the requested
|
||||
architecture. For example, reuse the ARM64 image after source-only changes:
|
||||
|
||||
```powershell
|
||||
.\build\wheel\docker\run.ps1 -Architecture aarch64 -PythonVersions 3.10 -SkipImageBuild
|
||||
```
|
||||
|
||||
ARM64 wheels target glibc Linux, including Q6A and Jetson systems with compatible
|
||||
Python ABIs and glibc versions (2.28 or newer for this build image). They do not
|
||||
target Android. The build requires binary ARM64 wheels for the runtime
|
||||
dependencies and performs the existing isolated import check. Validate on each
|
||||
board before delivery; CUDA, JetPack and Qualcomm GPU/NPU runtimes need separate
|
||||
compatibility checks. Builds share staging directories, so run architectures
|
||||
and Python ABIs sequentially.
|
||||
|
||||
The image bakes in per-ABI build tooling and a per-ABI wheelhouse of the runtime
|
||||
deps, so the entrypoint no longer re-resolves them over the network. See
|
||||
`docs/publishing.md` §6 for build-time details, why `-SkipImageBuild` matters,
|
||||
and what to change before attempting to parallelize the per-ABI builds.
|
||||
|
||||
### Release gate
|
||||
|
||||
Local only: never publishes. `--check-pypi` probes the index; native Windows
|
||||
venvs may download dependencies. Linux verification uses local Docker images
|
||||
and their offline wheelhouses. See `docs/publishing.en.md` for the full flow.
|
||||
|
||||
```bash
|
||||
python build/run.py release preflight --version <version>
|
||||
python build/run.py release preflight --all-wheel-pythons --check-pypi
|
||||
python build/run.py release audit SDK\<version>\orisys-<version>-cp312-cp312-win_amd64.whl
|
||||
```
|
||||
|
||||
`preflight` runs: repo state → version consistency → version-string scan →
|
||||
duplicate guard (fails only when the version is already released: a local git
|
||||
tag, or PyPI with `--check-pypi`) → pytest → bundle precheck → per-wheel static
|
||||
content/metadata assertions → isolated runtime verification in one venv per wheel
|
||||
(imports + bundled assets + real frames, no camera). A populated `SDK/<version>/`
|
||||
is the expected input, so build first:
|
||||
|
||||
```bash
|
||||
python build/run.py wheel-win
|
||||
python build/run.py wheel-manylinux
|
||||
python build/run.py wheel-manylinux -Architecture aarch64
|
||||
python build/run.py release preflight --all-wheel-pythons --check-pypi
|
||||
```
|
||||
|
||||
Reports land in `build/release/reports/<version>/`.
|
||||
|
||||
### Publish
|
||||
|
||||
Both default to a dry run; `--apply` is required for anything irreversible.
|
||||
|
||||
```bash
|
||||
python build/run.py release auth # credential status
|
||||
python build/run.py release auth --set-token pypi # hidden prompt, writes ~/.pypirc
|
||||
python build/run.py release github --version <version> # preview
|
||||
python build/run.py release github --version <version> --apply # sync + tag + push + Release
|
||||
python build/run.py release pypi --version <version> --apply # TestPyPI only
|
||||
python build/run.py release pypi --version <version> --confirm-pypi --apply # + PyPI
|
||||
python build/run.py release pypi --version <version> --pypi-only --apply # skip TestPyPI
|
||||
```
|
||||
|
||||
Both refuse to act unless `release preflight` passed on a clean commit with no
|
||||
waived options, and every wheel still hashes to what that report recorded.
|
||||
Runtime evidence must cover every wheel hash. `--all-wheel-pythons` verifies all
|
||||
staged platforms and ABIs, using conda for Windows and matching Docker images for
|
||||
Linux. GitHub attaches all certified wheels; PyPI verifies every uploaded
|
||||
Python/platform wheel by downloading it and comparing its hash.
|
||||
Tokens are only ever read from a hidden prompt or stdin, never from argv.
|
||||
See `docs/publishing.md` §3.
|
||||
|
||||
### GUI
|
||||
|
||||
```bash
|
||||
python build/run.py gui finger
|
||||
python build/run.py gui mini # scaffold / NotImplemented
|
||||
python build/run.py gui box # scaffold / NotImplemented
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
python build/run.py verify path\to\orisys.whl
|
||||
python build/run.py verify path\to\orisys.whl --contents --assets --runtime
|
||||
python build/run.py verify --gui-dist --product ory_finger
|
||||
```
|
||||
|
||||
## Outputs
|
||||
|
||||
- Wheels: `SDK/<version>/`, `dist/repaired/`
|
||||
- Preflight reports: `build/release/reports/<version>/`
|
||||
- Finger GUI: `dist/OrisysDashboard/`
|
||||
- Future mini/box: `dist/OryMini/`, `dist/OryBox/`
|
||||
|
||||
Generated work files under `build/nuitka/`, `build/release/reports/` and
|
||||
`build/pyinstaller_work/` are gitignored.
|
||||
@@ -1,194 +0,0 @@
|
||||
"""Build a CPU-only standalone Windows folder for Orisys 3D dashboard using PyInstaller.
|
||||
|
||||
Usage:
|
||||
conda activate py310 # or a clean virtualenv with only required packages
|
||||
pip install pyinstaller
|
||||
python build/build_gui_win_pyinstaller.py
|
||||
|
||||
Output:
|
||||
dist/OrisysDashboard/OrisysDashboard.exe (+ bundled Python, DLLs, config/, assets/)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# ---------- Configuration ----------
|
||||
ENTRY_SCRIPT = Path("examples/finger_pipeline_3dview.py").resolve()
|
||||
EXE_NAME = "OrisysDashboard.exe"
|
||||
DIST_NAME = "OrisysDashboard" # final folder name
|
||||
repo_root = Path(__file__).parent.parent
|
||||
|
||||
# Packages you want to force-include (PyInstaller often auto-detects them, but explicit avoids misses)
|
||||
HIDDEN_IMPORTS = [
|
||||
"PyQt5",
|
||||
"pyqtgraph",
|
||||
"open3d",
|
||||
"cv2",
|
||||
"orisys",
|
||||
"gui",
|
||||
"pipeline",
|
||||
"viz",
|
||||
"app_paths",
|
||||
"plotly",
|
||||
"_plotly_utils",
|
||||
# sometimes needed for open3d:
|
||||
"open3d.cpu.pybind", # adjust to actual module if needed
|
||||
]
|
||||
|
||||
# Modules to completely exclude (they won't be imported anyway if absent in venv)
|
||||
EXCLUDE_MODULES = [
|
||||
"cupy",
|
||||
"cupyx",
|
||||
"pyopencl",
|
||||
"tkinter",
|
||||
# "unittest",
|
||||
# "email",
|
||||
# "http",
|
||||
# "xmlrpc",
|
||||
# "pkg_resources",
|
||||
# "setuptools",
|
||||
"pip",
|
||||
"wheel",
|
||||
"matplotlib", # unless you use it
|
||||
# "scipy", # unless you use it
|
||||
]
|
||||
|
||||
# Data directories: (source relative to repo root, destination in bundle)
|
||||
DATA_DIRS = [
|
||||
("config", "config"),
|
||||
("assets", "assets"),
|
||||
]
|
||||
|
||||
# Optional: UPX directory (download from https://upx.github.io)
|
||||
UPX_DIR = r"C:\tools\upx" # set to None if not available
|
||||
# ---------- End Configuration ----------
|
||||
|
||||
|
||||
def clean_dist(dist_path: Path) -> None:
|
||||
if dist_path.exists():
|
||||
shutil.rmtree(dist_path)
|
||||
dist_path.mkdir(parents=True)
|
||||
|
||||
|
||||
def build_with_pyinstaller():
|
||||
# Ensure we are in a clean, minimal venv
|
||||
print("Using Python:", sys.executable)
|
||||
|
||||
# Output folder for PyInstaller work files
|
||||
work_dir = Path("build/pyinstaller_work")
|
||||
dist_dir = Path("dist") / DIST_NAME
|
||||
clean_dist(work_dir)
|
||||
clean_dist(dist_dir)
|
||||
|
||||
# Build PyInstaller command
|
||||
cmd = [
|
||||
sys.executable, "-m", "PyInstaller",
|
||||
"--onedir", # produce a folder, not a single exe
|
||||
"--name", DIST_NAME,
|
||||
f"--distpath={dist_dir.parent}",
|
||||
f"--workpath={work_dir}",
|
||||
f"--specpath=build",
|
||||
"--noconfirm",
|
||||
"--clean",
|
||||
"--windowed", # or "--console" if you need terminal
|
||||
]
|
||||
|
||||
# Add data files
|
||||
for src_rel, dest_rel in DATA_DIRS:
|
||||
src = Path(src_rel).resolve()
|
||||
if not src.is_dir():
|
||||
raise FileNotFoundError(f"Data directory missing: {src}")
|
||||
cmd.append(f"--add-data={src};{dest_rel}")
|
||||
|
||||
# Logo file if exists
|
||||
logo = Path("logo.png")
|
||||
if logo.exists():
|
||||
cmd.append(f"--add-data={logo.resolve()};.")
|
||||
|
||||
# Hidden imports
|
||||
for mod in HIDDEN_IMPORTS:
|
||||
cmd.append(f"--hidden-import={mod}")
|
||||
|
||||
# Exclude modules
|
||||
for mod in EXCLUDE_MODULES:
|
||||
cmd.append(f"--exclude-module={mod}")
|
||||
|
||||
# UPX if available
|
||||
if UPX_DIR and Path(UPX_DIR).is_dir():
|
||||
cmd.append(f"--upx-dir={UPX_DIR}")
|
||||
|
||||
# Entry point
|
||||
cmd.append(str(ENTRY_SCRIPT))
|
||||
|
||||
# Environment (same as Nuitka script)
|
||||
env = os.environ.copy()
|
||||
env["PYTHONPATH"] = str(Path("src").resolve()) + os.pathsep + str(Path("examples").resolve()) + os.pathsep + env.get("PYTHONPATH", "")
|
||||
|
||||
print("\n=== Running PyInstaller ===")
|
||||
print("Command:", " ".join(cmd))
|
||||
subprocess.check_call(cmd, env=env)
|
||||
|
||||
# After build, copy data directories explicitly
|
||||
# PyInstaller output folder
|
||||
dist_dir = Path("dist") / DIST_NAME
|
||||
final_exe = dist_dir / EXE_NAME
|
||||
if not final_exe.is_file():
|
||||
raise RuntimeError(f"Executable not found at {final_exe}")
|
||||
print("\n=== Copying data directories ===")
|
||||
for src_rel, dest_rel in DATA_DIRS:
|
||||
src = repo_root / src_rel # use an absolute reference to repo root
|
||||
dest = dist_dir / dest_rel
|
||||
if not src.is_dir():
|
||||
raise FileNotFoundError(f"Data directory not found: {src}")
|
||||
if dest.exists():
|
||||
shutil.rmtree(dest)
|
||||
shutil.copytree(src, dest, dirs_exist_ok=True) # Python 3.8+ use dirs_exist_ok=True
|
||||
print(f" {src} -> {dest}")
|
||||
|
||||
# Copy logo
|
||||
logo_src = repo_root / "logo.png"
|
||||
if logo_src.is_file():
|
||||
shutil.copy2(logo_src, dist_dir / "logo.png")
|
||||
print(f" Logo copied")
|
||||
|
||||
# Write README (same as Nuitka script)
|
||||
readme = dist_dir / "README.txt"
|
||||
readme.write_text(
|
||||
"\n".join([
|
||||
"Orisys Finger Dashboard (standalone)",
|
||||
"",
|
||||
"Run:",
|
||||
f" {EXE_NAME}",
|
||||
f" {EXE_NAME} --video path\\to\\video.mp4",
|
||||
f" {EXE_NAME} --config config\\dd02-ov.json",
|
||||
"",
|
||||
"Bundled folders:",
|
||||
" config/ sensor JSON configs",
|
||||
" assets/ finger mesh and visualization data",
|
||||
"",
|
||||
"CPU-only build (no CUDA/CuPy).",
|
||||
]),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print(f"\n[SUCCESS] Standalone GUI folder: {dist_dir}")
|
||||
print(f"Executable: {final_exe}")
|
||||
|
||||
# Optional smoke test (same as Nuitka script)
|
||||
try:
|
||||
from verify import verify_gui_dist
|
||||
print("\n=== Smoke test ===")
|
||||
verify_gui_dist(dist_dir)
|
||||
except Exception as exc:
|
||||
print(f"\n[WARNING] Smoke test failed: {exc}")
|
||||
|
||||
return dist_dir
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
build_with_pyinstaller()
|
||||
@@ -1,89 +0,0 @@
|
||||
"""Build a Nuitka-compiled orisys wheel and publish it to SDK/{version}/.
|
||||
|
||||
Pipeline:
|
||||
1. Compile src/orisys → .pyd
|
||||
2. Stage wheel tree (compiled module + bundled files from pyproject.toml)
|
||||
3. Run bdist_wheel and move output to SDK/{version}/
|
||||
4. Smoke-test imports in a temporary venv
|
||||
|
||||
Run from repo root: python build/build_nuitka.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from nuitka_compile import compile_orisys, copy_compiled_orisys, install_nuitka
|
||||
from paths import NUITKA_COMPILE_DIR, NUITKA_WHEEL_DIR, REPO_ROOT, repo_path
|
||||
from pyproject_utils import bundle_items, extras_require, get_version, install_requires
|
||||
from verify import verify_built_wheel
|
||||
from wheel_build import publish_wheel_to_sdk, run_bdist_wheel, write_wheel_setup
|
||||
from wheel_layout import copy_bundle_items, package_data_from_bundle
|
||||
|
||||
|
||||
def _reset_dir(path) -> None:
|
||||
"""Remove and recreate a build output directory."""
|
||||
if path.exists():
|
||||
shutil.rmtree(path)
|
||||
path.mkdir(parents=True)
|
||||
|
||||
|
||||
def build() -> None:
|
||||
# --- 1. Compile ---
|
||||
install_nuitka()
|
||||
|
||||
version = get_version(REPO_ROOT)
|
||||
_reset_dir(NUITKA_COMPILE_DIR)
|
||||
_reset_dir(NUITKA_WHEEL_DIR)
|
||||
|
||||
print("\n=== Compiling with Nuitka ===")
|
||||
env = os.environ.copy()
|
||||
src_dir = repo_path("src")
|
||||
env["PYTHONPATH"] = src_dir.as_posix() + os.pathsep + env.get("PYTHONPATH", "")
|
||||
ext_file = compile_orisys(NUITKA_COMPILE_DIR, env)
|
||||
|
||||
# --- 2. Stage wheel contents ---
|
||||
print("\n=== Preparing Wheel Structure ===")
|
||||
pkg_dir = NUITKA_WHEEL_DIR / "wheel_root" / "orisys"
|
||||
pkg_dir.mkdir(parents=True)
|
||||
|
||||
ext_suffix = copy_compiled_orisys(pkg_dir, ext_file)
|
||||
|
||||
# Bundle rules live in pyproject.toml → [[tool.orisys.bundle.items]]
|
||||
bundle = bundle_items(REPO_ROOT)
|
||||
copied_counts = copy_bundle_items(REPO_ROOT, pkg_dir, bundle)
|
||||
print(f"Copied {copied_counts['configs']} JSON config files")
|
||||
print(f"Copied {copied_counts['examples']} example files to orisys/examples/")
|
||||
print(f"Copied {copied_counts['docs']} doc files to orisys/docs/")
|
||||
print(f"Copied {copied_counts['assets']} finger asset files to orisys/assets/")
|
||||
|
||||
write_wheel_setup(
|
||||
NUITKA_WHEEL_DIR,
|
||||
version=version,
|
||||
package_data=package_data_from_bundle(ext_suffix, bundle),
|
||||
install_requires=install_requires(REPO_ROOT),
|
||||
extras_require=extras_require(REPO_ROOT),
|
||||
)
|
||||
|
||||
# --- 3. Build wheel and publish to SDK/ ---
|
||||
wheel_path = run_bdist_wheel(NUITKA_WHEEL_DIR)
|
||||
if wheel_path is None:
|
||||
print("\n[ERROR] Wheel not found.")
|
||||
return
|
||||
|
||||
final_wheel = publish_wheel_to_sdk(wheel_path, version)
|
||||
print(f"\n[SUCCESS] Wheel created and moved to: {final_wheel}")
|
||||
|
||||
# # --- 4. Verify (isolated venv; does not touch the active environment) ---
|
||||
# print("\n=== Verifying finger mesh/viewer imports ===")
|
||||
# try:
|
||||
# verify_built_wheel(final_wheel)
|
||||
# except subprocess.CalledProcessError as exc:
|
||||
# print(f"\n[WARNING] Wheel verify failed: {exc}")
|
||||
# print("The wheel was built successfully; verification runs in an isolated temp venv.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
build()
|
||||
@@ -0,0 +1 @@
|
||||
"""Shared build utilities used by wheel and GUI packaging targets."""
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Ensure ``build/`` is importable as the root for ``common``, ``wheel``, and ``gui``."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def ensure_build_root_on_path() -> Path:
|
||||
root = str(_BUILD_ROOT)
|
||||
if root not in sys.path:
|
||||
sys.path.insert(0, root)
|
||||
return _BUILD_ROOT
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Repository and packaging output paths."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
# build/common/paths.py → parents: common, build, repo
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
NUITKA_COMPILE_DIR = REPO_ROOT / "build" / "nuitka" / "compile"
|
||||
NUITKA_WHEEL_DIR = REPO_ROOT / "build" / "nuitka" / "wheel"
|
||||
NUITKA_GUI_DIR = REPO_ROOT / "build" / "nuitka" / "gui"
|
||||
GUI_DIST_DIR = REPO_ROOT / "dist" / "OrisysDashboard"
|
||||
PYINSTALLER_WORK_DIR = REPO_ROOT / "build" / "pyinstaller_work"
|
||||
|
||||
|
||||
def repo_path(*parts: str) -> Path:
|
||||
return REPO_ROOT.joinpath(*parts)
|
||||
|
||||
|
||||
def sdk_dir(version: str) -> Path:
|
||||
return REPO_ROOT / "SDK" / version
|
||||
|
||||
|
||||
def repaired_wheel_dir() -> Path:
|
||||
"""Directory for auditwheel-repaired wheels under the Nuitka wheel build tree."""
|
||||
return NUITKA_WHEEL_DIR / "repaired"
|
||||
|
||||
|
||||
def gui_dist_dir(dist_name: str) -> Path:
|
||||
return REPO_ROOT / "dist" / dist_name
|
||||
@@ -2,11 +2,25 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import ast
|
||||
import importlib.util
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def _load_wheel_layout_module():
|
||||
layout_path = _BUILD_ROOT / "wheel" / "layout.py"
|
||||
spec = importlib.util.spec_from_file_location("_orisys_build_wheel_layout", layout_path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(f"Cannot load wheel layout helper from {layout_path}")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _load_toml(text: str) -> dict:
|
||||
if sys.version_info >= (3, 11):
|
||||
@@ -41,17 +55,29 @@ def extras_require(repo_root: Path) -> dict[str, list[str]]:
|
||||
|
||||
def _resolve_version_attr(repo_root: Path, attr: str) -> str:
|
||||
"""Resolve a setuptools dynamic version attr (e.g. orisys.__version__)."""
|
||||
src_dir = str(repo_root / "src")
|
||||
if src_dir not in sys.path:
|
||||
sys.path.insert(0, src_dir)
|
||||
module_name, _, attr_name = attr.partition(".")
|
||||
if not attr_name:
|
||||
raise RuntimeError(f"Invalid dynamic version attr: {attr!r}")
|
||||
module = importlib.import_module(module_name)
|
||||
value = module
|
||||
for name in attr_name.split("."):
|
||||
value = getattr(value, name)
|
||||
return str(value)
|
||||
|
||||
# Avoid importing package modules here: __init__ may import heavy runtime deps
|
||||
# (e.g., cv2/numpy) that are not required just to read __version__.
|
||||
rel_module = Path(*module_name.split("."))
|
||||
module_file = repo_root / "src" / rel_module.with_suffix(".py")
|
||||
if not module_file.is_file():
|
||||
module_file = repo_root / "src" / rel_module / "__init__.py"
|
||||
if not module_file.is_file():
|
||||
raise RuntimeError(f"Cannot resolve module file for dynamic version attr: {attr!r}")
|
||||
|
||||
tree = ast.parse(module_file.read_text(encoding="utf-8"), filename=str(module_file))
|
||||
target_name = attr_name.split(".")[0]
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Assign):
|
||||
for target in node.targets:
|
||||
if isinstance(target, ast.Name) and target.id == target_name:
|
||||
value = ast.literal_eval(node.value)
|
||||
return str(value)
|
||||
|
||||
raise RuntimeError(f"Attribute {target_name!r} not found in {module_file}")
|
||||
|
||||
|
||||
def get_version(repo_root: Path) -> str:
|
||||
@@ -77,8 +103,12 @@ def bundle_items(repo_root: Path) -> list[dict]:
|
||||
|
||||
|
||||
def bundle_data_files(repo_root: Path) -> list[tuple[str, list[str]]]:
|
||||
"""Map bundle items marked dev_install=true to setuptools data_files entries."""
|
||||
from wheel_layout import resolve_bundle_source_files
|
||||
"""Map bundle items marked ``dev_install=true`` to install destinations.
|
||||
|
||||
Kept for tests / tooling. Prefer :func:`stage_dev_bundle_into` for pip
|
||||
installs so files land inside ``site-packages/orisys/``.
|
||||
"""
|
||||
resolve_bundle_source_files = _load_wheel_layout_module().resolve_bundle_source_files
|
||||
|
||||
grouped: dict[str, list[str]] = {}
|
||||
for item in bundle_items(repo_root):
|
||||
@@ -98,3 +128,33 @@ def bundle_data_files(repo_root: Path) -> list[tuple[str, list[str]]]:
|
||||
grouped.setdefault(dest, []).append(rel_src)
|
||||
|
||||
return [(dest, sorted(files)) for dest, files in sorted(grouped.items())]
|
||||
|
||||
|
||||
def stage_dev_bundle_into(repo_root: Path, pkg_dir: Path) -> list[str]:
|
||||
"""Copy ``dev_install`` bundle files under ``pkg_dir`` (the orisys package).
|
||||
|
||||
Returns package-relative paths that were written (e.g. ``sensors/box/...``).
|
||||
"""
|
||||
resolve_bundle_source_files = _load_wheel_layout_module().resolve_bundle_source_files
|
||||
packaged: list[str] = []
|
||||
|
||||
for item in bundle_items(repo_root):
|
||||
if not item.get("dev_install", False):
|
||||
continue
|
||||
|
||||
src_dir = repo_root / item["source"]
|
||||
dst_dir = pkg_dir / item["target"]
|
||||
dst_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for src_file in resolve_bundle_source_files(repo_root, item):
|
||||
if item.get("preserve_tree", False):
|
||||
rel = src_file.relative_to(src_dir)
|
||||
dst_file = dst_dir / rel
|
||||
else:
|
||||
rel = Path(src_file.name)
|
||||
dst_file = dst_dir / src_file.name
|
||||
dst_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(src_file, dst_file)
|
||||
packaged.append(f"{item['target']}/{rel.as_posix()}")
|
||||
|
||||
return packaged
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Post-build smoke tests for wheels and product GUI folders."""
|
||||
|
||||
from common.verify.gui import GUI_PRODUCTS, verify_gui_dist
|
||||
from common.verify.wheel import (
|
||||
verify_built_wheel,
|
||||
verify_wheel,
|
||||
verify_wheel_assets,
|
||||
verify_wheel_runtime,
|
||||
)
|
||||
from common.verify.wheel_contents import WheelCheckResult, inspect_wheel
|
||||
|
||||
__all__ = [
|
||||
"GUI_PRODUCTS",
|
||||
"WheelCheckResult",
|
||||
"inspect_wheel",
|
||||
"verify_built_wheel",
|
||||
"verify_gui_dist",
|
||||
"verify_wheel",
|
||||
"verify_wheel_assets",
|
||||
"verify_wheel_runtime",
|
||||
]
|
||||
@@ -0,0 +1,141 @@
|
||||
"""CLI for packaged artifact verification."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
from common.verify import ( # noqa: E402
|
||||
GUI_PRODUCTS,
|
||||
inspect_wheel,
|
||||
verify_built_wheel,
|
||||
verify_gui_dist,
|
||||
verify_wheel,
|
||||
verify_wheel_assets,
|
||||
verify_wheel_runtime,
|
||||
)
|
||||
from common.verify.wheel import ( # noqa: E402
|
||||
DEFAULT_RUNTIME_CONFIG,
|
||||
default_verification_video,
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Verify packaged Orisys artifacts.")
|
||||
parser.add_argument("--gui-dist", action="store_true", help="Verify a GUI dist folder")
|
||||
parser.add_argument(
|
||||
"--product",
|
||||
choices=sorted(GUI_PRODUCTS),
|
||||
default="ory_finger",
|
||||
help="GUI product profile (default: ory_finger)",
|
||||
)
|
||||
parser.add_argument("--dist-dir", type=Path, default=None, help="Override GUI dist folder")
|
||||
parser.add_argument(
|
||||
"--contents",
|
||||
action="store_true",
|
||||
help="Run static metadata/content assertions on the wheel",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--assets",
|
||||
action="store_true",
|
||||
help="Install the wheel in a temp venv and assert bundled assets load",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--runtime",
|
||||
action="store_true",
|
||||
help="Install the wheel and process real frames (no camera required)",
|
||||
)
|
||||
parser.add_argument("--expected-version", default=None, help="Assert the wheel version")
|
||||
parser.add_argument(
|
||||
"--strict-bundle",
|
||||
action="store_true",
|
||||
help="Treat bundled-file drift from the source tree as a failure",
|
||||
)
|
||||
parser.add_argument("--video", type=Path, default=None, help="Clip for --runtime")
|
||||
parser.add_argument("--config", default=DEFAULT_RUNTIME_CONFIG, help="Config for --runtime")
|
||||
parser.add_argument("--frames", type=int, default=40, help="Frames for --runtime")
|
||||
parser.add_argument("--offline", action="store_true", help="Reuse parent-env dependencies")
|
||||
parser.add_argument("wheel", nargs="?", type=Path, help="Wheel path for import smoke test")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.gui_dist:
|
||||
verify_gui_dist(args.dist_dir, product=args.product)
|
||||
return 0
|
||||
if args.wheel is None:
|
||||
parser.error("Provide a wheel path or pass --gui-dist")
|
||||
|
||||
exit_code = 0
|
||||
|
||||
if args.contents:
|
||||
result = inspect_wheel(
|
||||
args.wheel,
|
||||
expected_version=args.expected_version,
|
||||
repo_root=REPO_ROOT,
|
||||
strict_bundle=args.strict_bundle,
|
||||
)
|
||||
print(result.render())
|
||||
if not result.ok:
|
||||
exit_code = 1
|
||||
|
||||
if args.assets and args.runtime:
|
||||
video = args.video or default_verification_video(REPO_ROOT)
|
||||
if video is None:
|
||||
print("[FAIL] runtime: no clip found; pass --video or set ORISYS_VERIFY_VIDEO")
|
||||
return 1
|
||||
try:
|
||||
record = verify_wheel(
|
||||
args.wheel,
|
||||
video=video,
|
||||
config_name=args.config,
|
||||
frames=args.frames,
|
||||
offline=args.offline,
|
||||
)
|
||||
print(f"[ok ] {record['imports']}")
|
||||
print(f"[ok ] {record['assets']}")
|
||||
print(f"[ok ] runtime: frames={record['frames']} flow={record['flow_shape']}")
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] isolated verification: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
return exit_code
|
||||
|
||||
if args.assets:
|
||||
try:
|
||||
print(verify_wheel_assets(args.wheel))
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] bundled assets: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
|
||||
if args.runtime:
|
||||
video = args.video or default_verification_video(REPO_ROOT)
|
||||
if video is None:
|
||||
print("[FAIL] runtime: no clip found; pass --video or set ORISYS_VERIFY_VIDEO")
|
||||
return 1
|
||||
try:
|
||||
info = verify_wheel_runtime(
|
||||
args.wheel,
|
||||
video=video,
|
||||
config_name=args.config,
|
||||
frames=args.frames,
|
||||
offline=args.offline,
|
||||
)
|
||||
print(f"[ok ] runtime: {info}")
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] runtime: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
|
||||
if args.contents or args.assets or args.runtime:
|
||||
return exit_code
|
||||
|
||||
# Default behaviour is unchanged: import smoke test in an isolated venv.
|
||||
verify_built_wheel(args.wheel)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Smoke tests for standalone GUI product folders."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import GUI_DIST_DIR, gui_dist_dir
|
||||
|
||||
# Product profiles for GUI packaging verification.
|
||||
GUI_PRODUCTS: dict[str, dict] = {
|
||||
"ory_finger": {
|
||||
"dist_name": "OrisysDashboard",
|
||||
"exe_name": "OrisysDashboard.exe",
|
||||
"required_relpaths": [
|
||||
"config/dd02-ov.json",
|
||||
"sensors/finger/assets/30fps-0703/mesh.npz",
|
||||
"sensors/finger/assets/30fps-0703/world_to_cam.npy",
|
||||
"sensors/finger/assets/visualization/world_to_cam.npy",
|
||||
],
|
||||
},
|
||||
"ory_mini": {
|
||||
"dist_name": "OryMini",
|
||||
"exe_name": "OryMini.exe",
|
||||
"required_relpaths": [
|
||||
"config/mini.json",
|
||||
],
|
||||
},
|
||||
"ory_box": {
|
||||
"dist_name": "OryBox",
|
||||
"exe_name": "OryBox.exe",
|
||||
"required_relpaths": [
|
||||
"config/ddjx01.json",
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def verify_gui_dist(
|
||||
dist_dir: Path | None = None,
|
||||
*,
|
||||
product: str = "ory_finger",
|
||||
) -> None:
|
||||
"""Smoke-test a standalone GUI folder for the given product."""
|
||||
if product not in GUI_PRODUCTS:
|
||||
raise ValueError(f"Unknown GUI product {product!r}; choose from {sorted(GUI_PRODUCTS)}")
|
||||
|
||||
profile = GUI_PRODUCTS[product]
|
||||
if dist_dir is None:
|
||||
dist_dir = gui_dist_dir(profile["dist_name"]) if product != "ory_finger" else GUI_DIST_DIR
|
||||
dist_dir = dist_dir.resolve()
|
||||
|
||||
exe = dist_dir / profile["exe_name"]
|
||||
if not exe.is_file():
|
||||
raise FileNotFoundError(f"GUI executable not found: {exe}")
|
||||
|
||||
required = [dist_dir / rel for rel in profile["required_relpaths"]]
|
||||
missing = [str(path.relative_to(dist_dir)) for path in required if not path.is_file()]
|
||||
if missing:
|
||||
raise FileNotFoundError(f"Missing bundled files in {dist_dir}: {', '.join(missing)}")
|
||||
|
||||
subprocess.check_call([str(exe), "--help"], cwd=dist_dir)
|
||||
print(f"GUI dist smoke test OK ({product}): {exe}")
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Isolated venv smoke tests for packaged orisys wheels.
|
||||
|
||||
Three levels of confidence, all in a throwaway virtualenv so the active
|
||||
environment is never touched:
|
||||
|
||||
* :func:`verify_built_wheel` - the public API imports (fast, original check);
|
||||
* :func:`verify_wheel_assets` - bundled configs/sensors resolve and load via
|
||||
``importlib.resources``;
|
||||
* :func:`verify_wheel_runtime` - the compiled wheel actually processes real
|
||||
frames through the public pipeline with no camera attached.
|
||||
|
||||
Every level can be bootstrapped from another interpreter so each supported
|
||||
CPython ABI can be verified against its own wheel.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import venv
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
# Public SDK surface that must import from a released wheel.
|
||||
_SMOKE_IMPORT_SCRIPT = """
|
||||
import orisys
|
||||
from orisys import Sensor, list_config, export_config, util, finger
|
||||
from orisys.finger import UVMeshViewer, build_uv_surface_mesh, FingerRuntime
|
||||
from orisys.finger.mesh import UVSurfaceMesh
|
||||
from orisys.finger.viewer import init_uv_mesh_viewer
|
||||
from orisys.hhd import nHHD
|
||||
from orisys.tracker import Tracker
|
||||
from orisys import capture
|
||||
from orisys.Stitcher import Stitcher
|
||||
|
||||
assert orisys.__version__
|
||||
assert Sensor is not None
|
||||
assert callable(list_config)
|
||||
assert callable(export_config)
|
||||
assert util is not None
|
||||
assert finger is not None
|
||||
assert UVMeshViewer is not None
|
||||
assert build_uv_surface_mesh is not None
|
||||
assert FingerRuntime is not None
|
||||
assert UVSurfaceMesh is not None
|
||||
assert init_uv_mesh_viewer is not None
|
||||
assert nHHD is not None
|
||||
assert Tracker is not None
|
||||
assert capture is not None
|
||||
assert Stitcher is not None
|
||||
print(f"orisys {orisys.__version__} public API imports OK")
|
||||
"""
|
||||
|
||||
# Bundled data that must be resolvable and loadable from inside site-packages.
|
||||
# The wheel ships the compiled package, so these paths only exist if the bundle
|
||||
# step actually ran for this build.
|
||||
_ASSET_SCRIPT = r"""
|
||||
import sys
|
||||
from importlib.metadata import distribution
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import orisys
|
||||
|
||||
# Locate bundled data through the installed distribution, NOT through
|
||||
# orisys.__file__ / importlib.resources. Nuitka compiles the package into one
|
||||
# extension module whose __init__ installs nuitka_module_loader, which reports a
|
||||
# synthetic __file__ (e.g. ...\site-packages\orisys\orisys\__init__.py) and a
|
||||
# synthetic __path__ ending in '\not_existing'. Neither points at the real data
|
||||
# directory, so importlib.resources silently finds nothing in a Nuitka wheel.
|
||||
dist = distribution("orisys")
|
||||
dist_root = Path(dist.locate_file("")).resolve()
|
||||
package_root = Path(dist.locate_file("orisys")).resolve()
|
||||
|
||||
_diagnosis = (
|
||||
f"orisys distribution resolved to {dist_root} (sys.prefix={sys.prefix!r}, "
|
||||
f"sys.executable={sys.executable!r}), outside the verification venv. A stale "
|
||||
"orisys in the parent environment is visible through --system-site-packages; "
|
||||
"run `pip uninstall -y orisys` there, or re-run without --offline."
|
||||
)
|
||||
assert str(Path(sys.prefix).resolve()) in str(dist_root), _diagnosis
|
||||
assert package_root.is_dir(), f"package directory not found: {package_root}"
|
||||
|
||||
required = [
|
||||
"configs/box.json",
|
||||
"configs/finger.json",
|
||||
"sensors/box/force_calibration.json",
|
||||
"sensors/box/sensor.json",
|
||||
"sensors/finger/manifest.json",
|
||||
"sensors/finger/sensor.json",
|
||||
"sensors/mini/sensor.json",
|
||||
"sensors/finger/assets/30fps-0703/mesh.npz",
|
||||
"sensors/finger/assets/30fps-0703/camera_intrinsics.npz",
|
||||
"sensors/finger/assets/30fps-0703/world_to_cam.npy",
|
||||
"sensors/finger/assets/visualization/mesh.npz",
|
||||
"sensors/finger/assets/visualization/world_to_cam.npy",
|
||||
]
|
||||
|
||||
missing = [name for name in required if not (package_root / name).is_file()]
|
||||
assert not missing, (
|
||||
f"bundled assets missing from the wheel: {missing}\n"
|
||||
f"package root: {package_root}\ninstalled tree: "
|
||||
f"{sorted(p.name for p in package_root.iterdir()) if package_root.is_dir() else '<missing>'}"
|
||||
)
|
||||
|
||||
for name in required:
|
||||
path = package_root / name
|
||||
if name.endswith(".npz"):
|
||||
data = np.load(str(path))
|
||||
assert len(data.files) > 0, f"{name} is an empty npz"
|
||||
elif name.endswith(".npy"):
|
||||
arr = np.load(str(path))
|
||||
assert arr.size > 0, f"{name} is an empty npy"
|
||||
|
||||
configs = orisys.list_config()
|
||||
assert configs, "orisys.list_config() returned nothing"
|
||||
|
||||
print(f"bundled assets OK ({len(required)} files, {len(configs)} configs) "
|
||||
f"from {package_root}")
|
||||
"""
|
||||
|
||||
# End-to-end pipeline run against a video file: no camera, no GUI, no imshow.
|
||||
_RUNTIME_SCRIPT = r"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from importlib.metadata import distribution
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
os.environ.setdefault("OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS", "0")
|
||||
|
||||
import orisys
|
||||
|
||||
# Provenance via the installed distribution: a Nuitka-compiled package reports a
|
||||
# synthetic orisys.__file__ (see the asset check) and a stale orisys visible
|
||||
# through --system-site-packages would otherwise satisfy this check.
|
||||
dist = distribution("orisys")
|
||||
dist_root = Path(dist.locate_file("")).resolve()
|
||||
assert str(Path(sys.prefix).resolve()) in str(dist_root), (
|
||||
f"the installed orisys came from {dist_root}, outside the verification venv "
|
||||
f"{sys.prefix!r}; run `pip uninstall -y orisys` in the parent environment or "
|
||||
"re-run without --offline."
|
||||
)
|
||||
|
||||
video, config_name, frames = sys.argv[1], sys.argv[2], int(sys.argv[3])
|
||||
|
||||
sensor = orisys.Sensor(
|
||||
vid_src=video,
|
||||
config_name=config_name,
|
||||
backend="cpu",
|
||||
verbose=False,
|
||||
)
|
||||
processed = 0
|
||||
try:
|
||||
for _ in range(frames):
|
||||
img = sensor.get_img()
|
||||
assert img is not None, "get_img returned None"
|
||||
assert isinstance(img, np.ndarray) and img.ndim in (2, 3)
|
||||
|
||||
sensor.compute_deformation(check_motion=False)
|
||||
flow, vnormal, vshear = sensor.read_info(
|
||||
sensor.info.VRAW, sensor.info.VNORMAL, sensor.info.VSHEAR
|
||||
)
|
||||
for name, array in (("flow", flow), ("vnormal", vnormal), ("vshear", vshear)):
|
||||
assert np.isfinite(array).all(), f"{name} contains NaN/Inf"
|
||||
|
||||
contact = sensor.compute_contact()
|
||||
assert isinstance(contact, dict) and "centroid_found" in contact
|
||||
|
||||
fnormal, fsx, fsy = sensor.read_info(
|
||||
sensor.info.FNORMAL, sensor.info.FSHEARX, sensor.info.FSHEARY
|
||||
)
|
||||
assert np.isfinite([fnormal, fsx, fsy]).all(), "force output contains NaN/Inf"
|
||||
processed += 1
|
||||
finally:
|
||||
sensor.disconnect()
|
||||
|
||||
assert processed == frames, f"only processed {processed}/{frames} frames"
|
||||
print(json.dumps({
|
||||
"version": orisys.__version__,
|
||||
"frames": processed,
|
||||
"flow_shape": list(flow.shape),
|
||||
"video": video,
|
||||
"config": config_name,
|
||||
"package_root": str(Path(dist.locate_file("orisys")).resolve()),
|
||||
}))
|
||||
"""
|
||||
|
||||
|
||||
# Default runtime-verification inputs. Overridable by CLI flags or environment.
|
||||
DEFAULT_RUNTIME_CONFIG = os.getenv("ORISYS_VERIFY_CONFIG", "box")
|
||||
# Clips tracked in the repository. ``data/sample/s1.mp4`` is the sanctioned
|
||||
# verification input; the larger 720p clip is deliberately not listed, so a
|
||||
# missing s1.mp4 fails loudly instead of silently verifying against it.
|
||||
DEFAULT_VIDEO_CANDIDATES = (
|
||||
Path("data") / "sample" / "s1.mp4",
|
||||
)
|
||||
|
||||
|
||||
def default_verification_video(repo_root: Path) -> Path | None:
|
||||
"""Resolve the clip used for the end-to-end runtime check, if any."""
|
||||
env_video = os.getenv("ORISYS_VERIFY_VIDEO")
|
||||
repo_root = Path(repo_root)
|
||||
if env_video:
|
||||
candidate = Path(env_video)
|
||||
return candidate if candidate.is_absolute() else repo_root / candidate
|
||||
for relative in DEFAULT_VIDEO_CANDIDATES:
|
||||
candidate = repo_root / relative
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def _venv_python(venv_dir: Path) -> Path:
|
||||
if sys.platform == "win32":
|
||||
return venv_dir / "Scripts" / "python.exe"
|
||||
return venv_dir / "bin" / "python"
|
||||
|
||||
|
||||
def _hermetic_env() -> dict:
|
||||
"""Environment for the verification venv with PYTHONPATH removed.
|
||||
|
||||
An inherited PYTHONPATH=src would let the checkout satisfy the checks
|
||||
instead of the installed wheel, which is precisely the failure mode the
|
||||
provenance assertions in the scripts exist to catch.
|
||||
"""
|
||||
return {key: value for key, value in os.environ.items() if key != "PYTHONPATH"}
|
||||
|
||||
|
||||
def _create_venv(
|
||||
venv_dir: Path,
|
||||
*,
|
||||
python_executable: Path | None = None,
|
||||
system_site_packages: bool = False,
|
||||
) -> Path:
|
||||
"""Create a throwaway venv, optionally from a different interpreter."""
|
||||
if python_executable is None:
|
||||
venv.create(venv_dir, with_pip=True, system_site_packages=system_site_packages)
|
||||
else:
|
||||
command = [str(python_executable), "-m", "venv"]
|
||||
if system_site_packages:
|
||||
command.append("--system-site-packages")
|
||||
command.append(str(venv_dir))
|
||||
subprocess.check_call(command, env=_hermetic_env())
|
||||
return _venv_python(venv_dir)
|
||||
|
||||
|
||||
def _install_wheel(
|
||||
python: Path, wheel_path: Path, *, offline: bool = False, env: dict | None = None
|
||||
) -> None:
|
||||
"""Install the wheel under test, guaranteeing it lands in the venv.
|
||||
|
||||
``--force-reinstall --no-deps`` is not optional: pip reports a same-version
|
||||
requirement as "already satisfied" and skips the file, which is fatal here
|
||||
because ``--offline`` also exposes the parent environment's site-packages.
|
||||
Without it the venv check can silently validate a *different* orisys than
|
||||
the one passed in (the scripts additionally assert provenance).
|
||||
"""
|
||||
if offline:
|
||||
# Dependencies come from the parent environment via --system-site-packages.
|
||||
commands = [
|
||||
[
|
||||
str(python),
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"-q",
|
||||
"--no-index",
|
||||
"--no-deps",
|
||||
"--force-reinstall",
|
||||
str(wheel_path),
|
||||
]
|
||||
]
|
||||
else:
|
||||
commands = [
|
||||
[str(python), "-m", "pip", "install", "-q", "--upgrade", "pip"],
|
||||
[str(python), "-m", "pip", "install", "-q", str(wheel_path)],
|
||||
[
|
||||
str(python),
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"-q",
|
||||
"--force-reinstall",
|
||||
"--no-deps",
|
||||
str(wheel_path),
|
||||
],
|
||||
]
|
||||
|
||||
for command in commands:
|
||||
subprocess.check_call(command, stdout=subprocess.DEVNULL, env=env)
|
||||
|
||||
|
||||
class WheelVerificationError(RuntimeError):
|
||||
"""A verification step failed inside the throwaway venv.
|
||||
|
||||
Carries the child's stdout/stderr so the real traceback is visible instead of
|
||||
a bare non-zero exit code.
|
||||
"""
|
||||
|
||||
|
||||
def _run_script(
|
||||
python: Path, script: str, args: list[str] | None = None, *, env: dict
|
||||
) -> str:
|
||||
"""Run a verification script in an already-prepared venv."""
|
||||
completed = subprocess.run(
|
||||
[str(python), "-c", script, *(args or [])],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
check=False,
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
command = " ".join(str(part) for part in (args or []))
|
||||
raise WheelVerificationError(
|
||||
f"verification script failed in the venv (exit {completed.returncode}"
|
||||
+ (f", args: {command}" if command else "")
|
||||
+ ").\n--- stdout ---\n"
|
||||
+ (completed.stdout.strip() or "<empty>")
|
||||
+ "\n--- stderr ---\n"
|
||||
+ (completed.stderr.strip() or "<empty>")
|
||||
)
|
||||
return completed.stdout.strip()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _installed_venv(
|
||||
wheel_path: Path,
|
||||
*,
|
||||
python_executable: Path | None = None,
|
||||
offline: bool = False,
|
||||
prefix: str = "orisys-wheel-verify-",
|
||||
):
|
||||
"""Create a venv with the wheel installed, for one or more checks.
|
||||
|
||||
Creating a venv and installing the runtime dependencies is by far the most
|
||||
expensive part of verification (tens of seconds plus a large download when
|
||||
not using ``--offline``), so callers that run several checks must share one
|
||||
venv instead of building one per check.
|
||||
"""
|
||||
wheel_path = wheel_path.resolve()
|
||||
env = _hermetic_env()
|
||||
with tempfile.TemporaryDirectory(prefix=prefix) as tmp:
|
||||
venv_dir = Path(tmp) / "venv"
|
||||
interpreter = python_executable or Path(sys.executable)
|
||||
print(
|
||||
f" [{wheel_path.name}] creating venv from {interpreter.name}"
|
||||
+ ("" if offline else " and installing dependencies (first run downloads packages)")
|
||||
+ " ...",
|
||||
flush=True,
|
||||
)
|
||||
python = _create_venv(
|
||||
venv_dir,
|
||||
python_executable=python_executable,
|
||||
system_site_packages=offline,
|
||||
)
|
||||
_install_wheel(python, wheel_path, offline=offline, env=env)
|
||||
yield python, env
|
||||
|
||||
|
||||
def verify_built_wheel(wheel_path: Path, *, python_executable: Path | None = None) -> None:
|
||||
"""Install the wheel in a temporary venv and smoke-test public imports."""
|
||||
with _installed_venv(wheel_path, python_executable=python_executable) as (python, env):
|
||||
_run_script(python, _SMOKE_IMPORT_SCRIPT, env=env)
|
||||
|
||||
|
||||
def verify_wheel_assets(wheel_path: Path, *, python_executable: Path | None = None) -> str:
|
||||
"""Assert the wheel carries loadable bundled configs and sensor assets."""
|
||||
with _installed_venv(wheel_path, python_executable=python_executable) as (python, env):
|
||||
return _run_script(python, _ASSET_SCRIPT, env=env)
|
||||
|
||||
|
||||
def verify_wheel_runtime(
|
||||
wheel_path: Path,
|
||||
*,
|
||||
video: Path,
|
||||
config_name: str,
|
||||
frames: int = 40,
|
||||
python_executable: Path | None = None,
|
||||
offline: bool = False,
|
||||
) -> dict:
|
||||
"""Run real frames through the installed wheel with no camera attached.
|
||||
|
||||
This is the only check that proves the compiled extension plus the bundled
|
||||
sensor assets actually produce finite flow/force output end to end.
|
||||
"""
|
||||
with _installed_venv(
|
||||
wheel_path, python_executable=python_executable, offline=offline
|
||||
) as (python, env):
|
||||
return _runtime_check(python, env, video=video, config_name=config_name, frames=frames)
|
||||
|
||||
|
||||
def _runtime_check(
|
||||
python: Path, env: dict, *, video: Path, config_name: str, frames: int
|
||||
) -> dict:
|
||||
video = Path(video).resolve()
|
||||
if not video.is_file():
|
||||
raise FileNotFoundError(f"verification video not found: {video}")
|
||||
if frames < 1:
|
||||
raise ValueError("frames must be >= 1")
|
||||
|
||||
print(f" processing {frames} frame(s) with config {config_name!r} ...", flush=True)
|
||||
output = _run_script(
|
||||
python, _RUNTIME_SCRIPT, [str(video), config_name, str(frames)], env=env
|
||||
)
|
||||
import json
|
||||
|
||||
return json.loads(output.splitlines()[-1])
|
||||
|
||||
|
||||
def verify_wheel(
|
||||
wheel_path: Path,
|
||||
*,
|
||||
video: Path | None = None,
|
||||
config_name: str = DEFAULT_RUNTIME_CONFIG,
|
||||
frames: int = 40,
|
||||
python_executable: Path | None = None,
|
||||
offline: bool = False,
|
||||
check_assets: bool = True,
|
||||
) -> dict:
|
||||
"""Run every isolated-check level for one wheel inside a single venv.
|
||||
|
||||
Use this instead of calling the individual ``verify_wheel_*`` helpers: it
|
||||
pays the venv/dependency cost once and returns the runtime record plus the
|
||||
asset and import summaries.
|
||||
"""
|
||||
with _installed_venv(
|
||||
wheel_path, python_executable=python_executable, offline=offline
|
||||
) as (python, env):
|
||||
imports = _run_script(python, _SMOKE_IMPORT_SCRIPT, env=env)
|
||||
record: dict = {"wheel": Path(wheel_path).name, "imports": imports}
|
||||
if check_assets:
|
||||
record["assets"] = _run_script(python, _ASSET_SCRIPT, env=env)
|
||||
else:
|
||||
record["assets"] = "asset check skipped (--no-assets)"
|
||||
if video is not None:
|
||||
record.update(
|
||||
_runtime_check(
|
||||
python, env, video=video, config_name=config_name, frames=frames
|
||||
)
|
||||
)
|
||||
return record
|
||||
@@ -0,0 +1,502 @@
|
||||
"""Static content and metadata validation for built orisys wheels.
|
||||
|
||||
Everything here is pure standard library: the wheel is opened as a zip and
|
||||
inspected in place. No installation, no network, no hardware.
|
||||
|
||||
The checks encode the guarantees a Nuitka-compiled release must satisfy:
|
||||
|
||||
* the filename, ``METADATA`` and ``RECORD`` agree on name/version/tags;
|
||||
* PyPI-visible metadata (``Requires-Python``, classifiers, project URLs,
|
||||
license) is present instead of silently dropped by the hand-written
|
||||
``setup.py`` template in :mod:`wheel.package`;
|
||||
* no Python sources ship inside ``orisys/`` (IP containment) while the compiled
|
||||
extension is present;
|
||||
* every file declared by ``[[tool.orisys.bundle.items]]`` is present and
|
||||
byte-identical to the source tree;
|
||||
* no caches, build leftovers or scratch directories leak in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import re
|
||||
import sys
|
||||
import zipfile
|
||||
from dataclasses import dataclass, field
|
||||
from email.parser import Parser
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from wheel.layout import resolve_bundle_source_files # noqa: E402
|
||||
from wheel.package import is_compliant_manylinux_wheel # noqa: E402
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
from common.pyproject_utils import ( # noqa: E402
|
||||
bundle_items,
|
||||
extras_require,
|
||||
install_requires,
|
||||
)
|
||||
|
||||
# PyPI's default per-project file limit is 100 MB; keep headroom.
|
||||
MAX_WHEEL_BYTES = 90 * 1024 * 1024
|
||||
|
||||
_FILENAME_RE = re.compile(
|
||||
r"^(?P<name>[^-]+)-(?P<version>[^-]+)-(?P<python>[^-]+)-(?P<abi>[^-]+)-(?P<platform>.+)\.whl$"
|
||||
)
|
||||
_COMPILED_EXT_RE = re.compile(r"^orisys/__init__\..+\.(?:pyd|so)$")
|
||||
_FORBIDDEN_PREFIXES = (
|
||||
"test/",
|
||||
"tests/",
|
||||
"data/",
|
||||
"build/",
|
||||
"dist/",
|
||||
"sdk/",
|
||||
".git",
|
||||
)
|
||||
_FORBIDDEN_SUFFIXES = (".pyc", ".pyo", ".c", ".pdb", ".map", ".spec")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Check:
|
||||
name: str
|
||||
ok: bool
|
||||
detail: str = ""
|
||||
|
||||
def render(self) -> str:
|
||||
return f"[{'ok' if self.ok else 'FAIL'}] {self.name}: {self.detail}"
|
||||
|
||||
|
||||
@dataclass
|
||||
class WheelCheckResult:
|
||||
wheel: Path
|
||||
version: str | None = None
|
||||
name: str | None = None
|
||||
platform_tag: str | None = None
|
||||
entry_count: int = 0
|
||||
uncompressed_bytes: int = 0
|
||||
file_bytes: int = 0
|
||||
checks: list[Check] = field(default_factory=list)
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return all(check.ok for check in self.checks)
|
||||
|
||||
@property
|
||||
def failures(self) -> list[Check]:
|
||||
return [check for check in self.checks if not check.ok]
|
||||
|
||||
def add(self, name: str, ok: bool, detail: str = "") -> bool:
|
||||
self.checks.append(Check(name=name, ok=ok, detail=detail))
|
||||
return ok
|
||||
|
||||
def warn(self, message: str) -> None:
|
||||
self.warnings.append(message)
|
||||
|
||||
def render(self) -> str:
|
||||
lines = [
|
||||
f"wheel: {self.wheel.name}",
|
||||
f" version={self.version} platform={self.platform_tag} "
|
||||
f"entries={self.entry_count} file_size={self.file_bytes} "
|
||||
f"uncompressed={self.uncompressed_bytes}",
|
||||
]
|
||||
lines.extend(f" {check.render()}" for check in self.checks)
|
||||
lines.extend(f" [warn] {message}" for message in self.warnings)
|
||||
lines.append(f" RESULT: {'PASS' if self.ok else 'FAIL'}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _sha256(data: bytes) -> str:
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def _requirement_name(spec: str) -> str:
|
||||
return re.split(r"[<>=!~\[; ]", spec, maxsplit=1)[0].strip().lower().replace("_", "-")
|
||||
|
||||
|
||||
def _metadata_message(zf: zipfile.ZipFile, result: WheelCheckResult):
|
||||
candidates = [name for name in zf.namelist() if name.endswith(".dist-info/METADATA")]
|
||||
if not candidates:
|
||||
result.add("metadata-present", False, "no *.dist-info/METADATA entry in the wheel")
|
||||
return None
|
||||
raw = zf.read(candidates[0]).decode("utf-8", errors="replace")
|
||||
message = Parser().parsestr(raw)
|
||||
result.add("metadata-present", True, candidates[0])
|
||||
return message
|
||||
|
||||
|
||||
def _check_filename(result: WheelCheckResult, expected_version: str | None) -> None:
|
||||
match = _FILENAME_RE.match(result.wheel.name)
|
||||
if match is None:
|
||||
result.add("filename", False, f"unparsable wheel filename: {result.wheel.name}")
|
||||
return
|
||||
result.name = match.group("name")
|
||||
result.version = match.group("version")
|
||||
result.platform_tag = match.group("platform")
|
||||
|
||||
problems = []
|
||||
if result.name != "orisys":
|
||||
problems.append(f"distribution name is {result.name!r}, expected 'orisys'")
|
||||
if expected_version is not None and result.version != expected_version:
|
||||
problems.append(f"filename version {result.version!r} != expected {expected_version!r}")
|
||||
result.add("filename", not problems, "; ".join(problems) or result.wheel.name)
|
||||
|
||||
|
||||
def _check_metadata(
|
||||
result: WheelCheckResult,
|
||||
message,
|
||||
expected_version: str | None,
|
||||
repo_root: Path | None,
|
||||
) -> None:
|
||||
if message is None:
|
||||
return
|
||||
|
||||
result.add(
|
||||
"metadata-name",
|
||||
message.get("Name") == "orisys",
|
||||
f"Name={message.get('Name')!r}",
|
||||
)
|
||||
|
||||
declared = message.get("Version")
|
||||
expected = expected_version or result.version
|
||||
if expected is not None and declared != expected:
|
||||
result.add(
|
||||
"metadata-version",
|
||||
False,
|
||||
f"METADATA Version={declared!r} != expected {expected!r}",
|
||||
)
|
||||
else:
|
||||
result.add("metadata-version", True, f"Version={declared!r}")
|
||||
|
||||
requires_python = (message.get("Requires-Python") or "").strip()
|
||||
result.add(
|
||||
"metadata-requires-python",
|
||||
bool(requires_python),
|
||||
f"Requires-Python={requires_python!r}" if requires_python else "Requires-Python is missing",
|
||||
)
|
||||
|
||||
classifiers = message.get_all("Classifier") or []
|
||||
result.add(
|
||||
"metadata-classifiers",
|
||||
bool(classifiers),
|
||||
f"{len(classifiers)} classifier(s)" if classifiers else "no Classifier fields",
|
||||
)
|
||||
|
||||
project_urls = message.get_all("Project-URL") or []
|
||||
result.add(
|
||||
"metadata-project-urls",
|
||||
bool(project_urls),
|
||||
f"{len(project_urls)} Project-URL(s)" if project_urls else "no Project-URL fields",
|
||||
)
|
||||
|
||||
license_values = [
|
||||
(message.get("License") or "").strip(),
|
||||
(message.get("License-Expression") or "").strip(),
|
||||
]
|
||||
has_license = any(license_values) or any(
|
||||
value.startswith("License ::") for value in classifiers
|
||||
)
|
||||
result.add(
|
||||
"metadata-license",
|
||||
has_license,
|
||||
f"License={license_values[0]!r} License-Expression={license_values[1]!r}"
|
||||
if has_license
|
||||
else "no License / License-Expression / 'License ::' classifier",
|
||||
)
|
||||
|
||||
dynamic = message.get_all("Dynamic") or []
|
||||
result.add(
|
||||
"metadata-no-dynamic-version",
|
||||
"version" not in dynamic,
|
||||
f"Dynamic={dynamic!r}" if dynamic else "no Dynamic fields",
|
||||
)
|
||||
if dynamic:
|
||||
result.warn(f"METADATA still declares Dynamic fields: {dynamic!r}")
|
||||
|
||||
if repo_root is not None and (repo_root / "pyproject.toml").is_file():
|
||||
requires_dist = message.get_all("Requires-Dist") or []
|
||||
provided = {_requirement_name(entry) for entry in requires_dist}
|
||||
missing_core = [
|
||||
spec for spec in install_requires(repo_root) if _requirement_name(spec) not in provided
|
||||
]
|
||||
result.add(
|
||||
"metadata-requires-dist",
|
||||
not missing_core,
|
||||
f"missing core Requires-Dist entries: {missing_core!r}"
|
||||
if missing_core
|
||||
else f"{len(requires_dist)} Requires-Dist entries",
|
||||
)
|
||||
|
||||
extras = message.get_all("Provides-Extra") or []
|
||||
missing_extras = sorted(set(extras_require(repo_root)) - set(extras))
|
||||
result.add(
|
||||
"metadata-extras",
|
||||
not missing_extras,
|
||||
f"missing Provides-Extra: {missing_extras!r}"
|
||||
if missing_extras
|
||||
else f"Provides-Extra={sorted(extras)!r}",
|
||||
)
|
||||
|
||||
|
||||
def _check_compiled_only(result: WheelCheckResult, names: list[str]) -> None:
|
||||
compiled = [name for name in names if _COMPILED_EXT_RE.match(name)]
|
||||
result.add(
|
||||
"compiled-extension",
|
||||
bool(compiled),
|
||||
f"{compiled[0]}" if compiled else "no orisys/__init__.<abi>.pyd|.so entry",
|
||||
)
|
||||
|
||||
# orisys/examples/** is shipped as package data on purpose; every other
|
||||
# orisys/**/*.py means source code leaked past the Nuitka compile step.
|
||||
sources = [
|
||||
name
|
||||
for name in names
|
||||
if name.startswith("orisys/")
|
||||
and name.endswith(".py")
|
||||
and not name.startswith("orisys/examples/")
|
||||
]
|
||||
result.add(
|
||||
"no-python-sources",
|
||||
not sources,
|
||||
f"python sources leaked into the wheel: {sources[:5]!r}"
|
||||
if sources
|
||||
else "compiled package ships no python sources (examples excluded by design)",
|
||||
)
|
||||
|
||||
blend = [name for name in names if name.startswith("orisys_blend/")]
|
||||
result.add(
|
||||
"orisys-blend-present",
|
||||
"orisys_blend/__init__.py" in blend and "orisys_blend/kernel.py" in blend,
|
||||
f"{sorted(blend)!r}",
|
||||
)
|
||||
|
||||
|
||||
def _check_bundle_fidelity(
|
||||
result: WheelCheckResult,
|
||||
zf: zipfile.ZipFile,
|
||||
names: set[str],
|
||||
repo_root: Path | None,
|
||||
strict: bool,
|
||||
) -> None:
|
||||
if repo_root is None or not (repo_root / "pyproject.toml").is_file():
|
||||
result.warn("pyproject.toml not found; bundle fidelity checks skipped")
|
||||
return
|
||||
|
||||
missing: list[str] = []
|
||||
mismatched: list[str] = []
|
||||
checked = 0
|
||||
|
||||
for item in bundle_items(repo_root):
|
||||
target = str(item["target"]).replace("\\", "/")
|
||||
source_dir = repo_root / item["source"]
|
||||
for source_file in resolve_bundle_source_files(repo_root, item):
|
||||
if item.get("preserve_tree", False):
|
||||
relative = source_file.relative_to(source_dir).as_posix()
|
||||
entry = f"orisys/{target}/{relative}"
|
||||
else:
|
||||
entry = f"orisys/{target}/{source_file.name}"
|
||||
|
||||
if entry not in names:
|
||||
missing.append(entry)
|
||||
continue
|
||||
checked += 1
|
||||
if _sha256(zf.read(entry)) != _sha256(source_file.read_bytes()):
|
||||
mismatched.append(entry)
|
||||
|
||||
# Both entries here mean "the wheel does not match this revision of the
|
||||
# source tree". Against a fresh build that is fatal; against an already
|
||||
# released wheel it is expected, so ``strict`` (set by preflight, unset by
|
||||
# audit) decides whether it fails or warns.
|
||||
if missing:
|
||||
detail = (
|
||||
f"{len(missing)} declared file(s) absent from the wheel (added to the "
|
||||
f"source tree after this build?): {missing[:5]!r}"
|
||||
)
|
||||
result.add("bundle-completeness", not strict, detail)
|
||||
if not strict:
|
||||
result.warn(detail)
|
||||
else:
|
||||
result.add(
|
||||
"bundle-completeness", True, f"{checked} declared bundled file(s) present"
|
||||
)
|
||||
|
||||
if mismatched:
|
||||
detail = (
|
||||
f"{len(mismatched)} of {checked} bundled file(s) differ from the source "
|
||||
f"tree (wheel predates or postdates it): {mismatched[:5]!r}"
|
||||
)
|
||||
result.add("bundle-fidelity", not strict, detail)
|
||||
if not strict:
|
||||
result.warn(detail)
|
||||
else:
|
||||
result.add(
|
||||
"bundle-fidelity",
|
||||
True,
|
||||
f"{checked} bundled file(s) byte-identical to source",
|
||||
)
|
||||
|
||||
|
||||
def _check_forbidden_entries(result: WheelCheckResult, names: list[str]) -> None:
|
||||
offenders = []
|
||||
for name in names:
|
||||
lowered = name.lower()
|
||||
if lowered.startswith(_FORBIDDEN_PREFIXES) or lowered.endswith(_FORBIDDEN_SUFFIXES) or "__pycache__" in lowered or ".egg-info" in lowered:
|
||||
offenders.append(name)
|
||||
result.add(
|
||||
"no-forbidden-entries",
|
||||
not offenders,
|
||||
f"unexpected entries: {offenders[:5]!r}" if offenders else "no caches/leftovers/scratch paths",
|
||||
)
|
||||
|
||||
|
||||
def _check_archive_paths(result: WheelCheckResult, names: list[str]) -> None:
|
||||
unsafe = [
|
||||
name
|
||||
for name in names
|
||||
if name.startswith(("/", "\\")) or ".." in Path(name.replace("\\", "/")).parts
|
||||
]
|
||||
result.add(
|
||||
"archive-paths",
|
||||
not unsafe,
|
||||
f"absolute or traversing entries: {unsafe[:5]!r}" if unsafe else "all entries are relative",
|
||||
)
|
||||
|
||||
|
||||
def _check_record(result: WheelCheckResult, zf: zipfile.ZipFile, names: list[str]) -> None:
|
||||
records = [name for name in names if name.endswith(".dist-info/RECORD")]
|
||||
if not records:
|
||||
result.add("record", False, "no *.dist-info/RECORD entry")
|
||||
return
|
||||
|
||||
bad_paths: list[str] = []
|
||||
bad_hashes: list[str] = []
|
||||
bad_sizes: list[str] = []
|
||||
present = set(names)
|
||||
|
||||
for line in zf.read(records[0]).decode("utf-8").splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
parts = line.split(",")
|
||||
while len(parts) < 3:
|
||||
parts.append("")
|
||||
path, hash_field, size_field = parts[0], parts[1], parts[2]
|
||||
|
||||
if path not in present:
|
||||
bad_paths.append(path)
|
||||
continue
|
||||
if not hash_field.startswith("sha256="):
|
||||
continue
|
||||
|
||||
expected = hash_field[len("sha256=") :]
|
||||
actual = base64.urlsafe_b64encode(
|
||||
hashlib.sha256(zf.read(path)).digest()
|
||||
).rstrip(b"=").decode("ascii")
|
||||
if actual != expected:
|
||||
bad_hashes.append(path)
|
||||
if size_field and size_field.isdigit():
|
||||
if int(size_field) != len(zf.read(path)):
|
||||
bad_sizes.append(path)
|
||||
|
||||
problems = []
|
||||
if bad_paths:
|
||||
problems.append(f"paths not in archive: {bad_paths[:5]!r}")
|
||||
if bad_hashes:
|
||||
problems.append(f"sha256 mismatch: {bad_hashes[:5]!r}")
|
||||
if bad_sizes:
|
||||
problems.append(f"size mismatch: {bad_sizes[:5]!r}")
|
||||
result.add(
|
||||
"record",
|
||||
not problems,
|
||||
"; ".join(problems) if problems else f"{records[0]} verifies",
|
||||
)
|
||||
|
||||
|
||||
def _check_platform_tag(result: WheelCheckResult) -> None:
|
||||
tag = result.platform_tag
|
||||
if tag is None:
|
||||
return
|
||||
if "win" in tag:
|
||||
ok = tag == "win_amd64"
|
||||
result.add("platform-tag", ok, f"{tag!r} (expected 'win_amd64')")
|
||||
return
|
||||
ok = is_compliant_manylinux_wheel(result.wheel)
|
||||
result.add(
|
||||
"platform-tag",
|
||||
ok,
|
||||
f"{tag!r}" if ok else f"{tag!r} is not a supported manylinux x86_64 or aarch64 tag",
|
||||
)
|
||||
|
||||
|
||||
def _check_size(result: WheelCheckResult, max_bytes: int) -> None:
|
||||
result.add(
|
||||
"size-limit",
|
||||
result.file_bytes <= max_bytes,
|
||||
f"{result.file_bytes} bytes (limit {max_bytes})",
|
||||
)
|
||||
|
||||
|
||||
def inspect_wheel(
|
||||
wheel_path: Path,
|
||||
*,
|
||||
expected_version: str | None = None,
|
||||
repo_root: Path | None = None,
|
||||
max_bytes: int = MAX_WHEEL_BYTES,
|
||||
strict_bundle: bool = True,
|
||||
) -> WheelCheckResult:
|
||||
"""Validate one built wheel without installing it.
|
||||
|
||||
``strict_bundle`` controls how the wheel's bundled files are compared with the
|
||||
current source tree: ``True`` (gate a fresh build, used by
|
||||
``release preflight``) fails on any missing or changed declared file;
|
||||
``False`` (audit an already-released wheel) reports the same findings as
|
||||
warnings.
|
||||
"""
|
||||
wheel_path = Path(wheel_path).resolve()
|
||||
if repo_root is None:
|
||||
repo_root = REPO_ROOT if (REPO_ROOT / "pyproject.toml").is_file() else None
|
||||
else:
|
||||
repo_root = Path(repo_root).resolve()
|
||||
|
||||
result = WheelCheckResult(wheel=wheel_path)
|
||||
if not wheel_path.is_file():
|
||||
result.add("wheel-exists", False, f"not a file: {wheel_path}")
|
||||
return result
|
||||
|
||||
result.file_bytes = wheel_path.stat().st_size
|
||||
result.add("wheel-exists", True, str(wheel_path))
|
||||
_check_filename(result, expected_version)
|
||||
_check_platform_tag(result)
|
||||
_check_size(result, max_bytes)
|
||||
|
||||
try:
|
||||
zf = zipfile.ZipFile(wheel_path)
|
||||
except zipfile.BadZipFile as exc:
|
||||
result.add("zip-integrity", False, f"not a valid zip: {exc}")
|
||||
return result
|
||||
|
||||
with zf:
|
||||
corrupt = zf.testzip()
|
||||
result.add(
|
||||
"zip-integrity",
|
||||
corrupt is None,
|
||||
"CRC check passed" if corrupt is None else f"corrupt entry: {corrupt}",
|
||||
)
|
||||
|
||||
names = zf.namelist()
|
||||
name_set = set(names)
|
||||
result.entry_count = len(names)
|
||||
result.uncompressed_bytes = sum(info.file_size for info in zf.infolist())
|
||||
|
||||
_check_archive_paths(result, names)
|
||||
_check_forbidden_entries(result, names)
|
||||
message = _metadata_message(zf, result)
|
||||
_check_metadata(result, message, expected_version, repo_root)
|
||||
_check_compiled_only(result, names)
|
||||
_check_bundle_fidelity(result, zf, name_set, repo_root, strict_bundle)
|
||||
_check_record(result, zf, names)
|
||||
|
||||
return result
|
||||
@@ -0,0 +1 @@
|
||||
"""Standalone GUI packaging targets (PyInstaller)."""
|
||||
@@ -0,0 +1 @@
|
||||
"""Shared helpers for product GUI builds."""
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Shared PyInstaller driver for product GUI standalone folders."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import PYINSTALLER_WORK_DIR, REPO_ROOT, gui_dist_dir
|
||||
|
||||
|
||||
@dataclass
|
||||
class GuiBuildConfig:
|
||||
product: str
|
||||
entry_script: Path
|
||||
exe_name: str
|
||||
dist_name: str
|
||||
hidden_imports: list[str] = field(default_factory=list)
|
||||
exclude_modules: list[str] = field(default_factory=list)
|
||||
data_dirs: list[tuple[str, str]] = field(default_factory=list)
|
||||
readme_lines: list[str] = field(default_factory=list)
|
||||
windowed: bool = True
|
||||
upx_dir: str | None = r"C:\tools\upx"
|
||||
logo_name: str = "logo.png"
|
||||
work_dir: Path = PYINSTALLER_WORK_DIR
|
||||
spec_dir: Path = REPO_ROOT / "build" / "pyinstaller_work"
|
||||
|
||||
|
||||
def clean_dir(path: Path) -> None:
|
||||
if path.exists():
|
||||
shutil.rmtree(path)
|
||||
path.mkdir(parents=True)
|
||||
|
||||
|
||||
def build_with_pyinstaller(
|
||||
config: GuiBuildConfig,
|
||||
*,
|
||||
verify_fn: Callable[[Path], None] | None = None,
|
||||
) -> Path:
|
||||
"""Run PyInstaller for one GUI product and copy bundled data folders."""
|
||||
print("Using Python:", sys.executable)
|
||||
print(f"Building GUI product: {config.product}")
|
||||
|
||||
dist_dir = gui_dist_dir(config.dist_name)
|
||||
clean_dir(config.work_dir)
|
||||
clean_dir(dist_dir)
|
||||
config.spec_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
entry = config.entry_script
|
||||
if not entry.is_file():
|
||||
raise FileNotFoundError(f"GUI entry script not found: {entry}")
|
||||
|
||||
cmd = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"PyInstaller",
|
||||
"--onedir",
|
||||
"--name",
|
||||
config.dist_name,
|
||||
f"--distpath={dist_dir.parent}",
|
||||
f"--workpath={config.work_dir}",
|
||||
f"--specpath={config.spec_dir}",
|
||||
"--noconfirm",
|
||||
"--clean",
|
||||
"--windowed" if config.windowed else "--console",
|
||||
]
|
||||
|
||||
for src_rel, dest_rel in config.data_dirs:
|
||||
src = (REPO_ROOT / src_rel).resolve()
|
||||
if not src.is_dir():
|
||||
raise FileNotFoundError(f"Data directory missing: {src}")
|
||||
cmd.append(f"--add-data={src};{dest_rel}")
|
||||
|
||||
logo = REPO_ROOT / config.logo_name
|
||||
if logo.is_file():
|
||||
cmd.append(f"--add-data={logo.resolve()};.")
|
||||
|
||||
for mod in config.hidden_imports:
|
||||
cmd.append(f"--hidden-import={mod}")
|
||||
for mod in config.exclude_modules:
|
||||
cmd.append(f"--exclude-module={mod}")
|
||||
|
||||
if config.upx_dir and Path(config.upx_dir).is_dir():
|
||||
cmd.append(f"--upx-dir={config.upx_dir}")
|
||||
|
||||
cmd.append(str(entry.resolve()))
|
||||
|
||||
env = os.environ.copy()
|
||||
env["PYTHONPATH"] = (
|
||||
str((REPO_ROOT / "src").resolve())
|
||||
+ os.pathsep
|
||||
+ str((REPO_ROOT / "examples").resolve())
|
||||
+ os.pathsep
|
||||
+ env.get("PYTHONPATH", "")
|
||||
)
|
||||
|
||||
print("\n=== Running PyInstaller ===")
|
||||
print("Command:", " ".join(cmd))
|
||||
subprocess.check_call(cmd, env=env, cwd=REPO_ROOT)
|
||||
|
||||
final_exe = dist_dir / config.exe_name
|
||||
if not final_exe.is_file():
|
||||
raise RuntimeError(f"Executable not found at {final_exe}")
|
||||
|
||||
print("\n=== Copying data directories ===")
|
||||
for src_rel, dest_rel in config.data_dirs:
|
||||
src = REPO_ROOT / src_rel
|
||||
dest = dist_dir / dest_rel
|
||||
if not src.is_dir():
|
||||
raise FileNotFoundError(f"Data directory not found: {src}")
|
||||
if dest.exists():
|
||||
shutil.rmtree(dest)
|
||||
shutil.copytree(src, dest, dirs_exist_ok=True)
|
||||
print(f" {src} -> {dest}")
|
||||
|
||||
if logo.is_file():
|
||||
shutil.copy2(logo, dist_dir / config.logo_name)
|
||||
print(" Logo copied")
|
||||
|
||||
if config.readme_lines:
|
||||
(dist_dir / "README.txt").write_text("\n".join(config.readme_lines) + "\n", encoding="utf-8")
|
||||
|
||||
print(f"\n[SUCCESS] Standalone GUI folder: {dist_dir}")
|
||||
print(f"Executable: {final_exe}")
|
||||
|
||||
if verify_fn is not None:
|
||||
try:
|
||||
print("\n=== Smoke test ===")
|
||||
verify_fn(dist_dir)
|
||||
except Exception as exc:
|
||||
print(f"\n[WARNING] Smoke test failed: {exc}")
|
||||
|
||||
return dist_dir
|
||||
@@ -0,0 +1 @@
|
||||
"""ORY-BOX internal calibration/testing GUI packaging (scaffold)."""
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Scaffold for ORY-BOX internal GUI packaging.
|
||||
|
||||
Not implemented yet. When ready, wire a dedicated calibration/testing entry
|
||||
script and call ``gui.common.pyinstaller_driver.build_with_pyinstaller``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from gui.ory_box.config import DIST_NAME, ENTRY_SCRIPT, make_config
|
||||
|
||||
|
||||
def main() -> None:
|
||||
config = make_config()
|
||||
raise NotImplementedError(
|
||||
"ORY-BOX GUI packaging is a scaffold only. "
|
||||
f"Planned dist={DIST_NAME!r}, entry={ENTRY_SCRIPT}. "
|
||||
"Implement the internal calibration/testing UI entry, then call "
|
||||
"gui.common.pyinstaller_driver.build_with_pyinstaller(config)."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Product configuration scaffold for the ORY-BOX internal GUI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT
|
||||
|
||||
from gui.common.pyinstaller_driver import GuiBuildConfig
|
||||
|
||||
ENTRY_SCRIPT = REPO_ROOT / "examples" / "basic_pipeline.py"
|
||||
EXE_NAME = "OryBox.exe"
|
||||
DIST_NAME = "OryBox"
|
||||
|
||||
HIDDEN_IMPORTS = [
|
||||
"PyQt5",
|
||||
"pyqtgraph",
|
||||
"cv2",
|
||||
"orisys",
|
||||
"gui",
|
||||
"pipeline",
|
||||
"viz",
|
||||
]
|
||||
|
||||
EXCLUDE_MODULES = [
|
||||
"cupy",
|
||||
"cupyx",
|
||||
"pyopencl",
|
||||
"open3d",
|
||||
"tkinter",
|
||||
"pip",
|
||||
"wheel",
|
||||
]
|
||||
|
||||
DATA_DIRS = [
|
||||
("config", "config"),
|
||||
]
|
||||
|
||||
README_LINES = [
|
||||
"ORY-BOX internal calibration/testing GUI (scaffold)",
|
||||
"",
|
||||
"This product packaging target is reserved for a future internal tool.",
|
||||
"Entry point placeholder:",
|
||||
f" {ENTRY_SCRIPT.as_posix()}",
|
||||
]
|
||||
|
||||
|
||||
def make_config() -> GuiBuildConfig:
|
||||
return GuiBuildConfig(
|
||||
product="ory_box",
|
||||
entry_script=ENTRY_SCRIPT,
|
||||
exe_name=EXE_NAME,
|
||||
dist_name=DIST_NAME,
|
||||
hidden_imports=list(HIDDEN_IMPORTS),
|
||||
exclude_modules=list(EXCLUDE_MODULES),
|
||||
data_dirs=list(DATA_DIRS),
|
||||
readme_lines=list(README_LINES),
|
||||
windowed=False,
|
||||
)
|
||||
@@ -0,0 +1 @@
|
||||
"""ORY-FINGER standalone GUI packaging."""
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Build the ORY-FINGER standalone Windows GUI folder with PyInstaller.
|
||||
|
||||
Usage:
|
||||
python build/run.py gui finger
|
||||
python build/gui/ory_finger/build.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.verify.gui import verify_gui_dist
|
||||
|
||||
from gui.common.pyinstaller_driver import build_with_pyinstaller
|
||||
from gui.ory_finger.config import make_config
|
||||
|
||||
|
||||
def main() -> None:
|
||||
config = make_config()
|
||||
build_with_pyinstaller(
|
||||
config,
|
||||
verify_fn=lambda dist_dir: verify_gui_dist(dist_dir, product="ory_finger"),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Product configuration for the ORY-FINGER Windows dashboard."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT
|
||||
|
||||
from gui.common.pyinstaller_driver import GuiBuildConfig
|
||||
|
||||
ENTRY_SCRIPT = REPO_ROOT / "examples" / "finger_pipeline_3dview.py"
|
||||
EXE_NAME = "OrisysDashboard.exe"
|
||||
DIST_NAME = "OrisysDashboard"
|
||||
|
||||
HIDDEN_IMPORTS = [
|
||||
"PyQt5",
|
||||
"pyqtgraph",
|
||||
"open3d",
|
||||
"cv2",
|
||||
"orisys",
|
||||
"gui",
|
||||
"pipeline",
|
||||
"viz",
|
||||
"gui.app_paths",
|
||||
"plotly",
|
||||
"_plotly_utils",
|
||||
"open3d.cpu.pybind",
|
||||
]
|
||||
|
||||
EXCLUDE_MODULES = [
|
||||
"cupy",
|
||||
"cupyx",
|
||||
"pyopencl",
|
||||
"tkinter",
|
||||
"pip",
|
||||
"wheel",
|
||||
"matplotlib",
|
||||
]
|
||||
|
||||
DATA_DIRS = [
|
||||
("config", "config"),
|
||||
("sensors", "sensors"),
|
||||
]
|
||||
|
||||
README_LINES = [
|
||||
"Orisys Finger Dashboard (standalone)",
|
||||
"",
|
||||
"Run:",
|
||||
f" {EXE_NAME}",
|
||||
f" {EXE_NAME} --video path\\to\\video.mp4",
|
||||
f" {EXE_NAME} --config config\\dd02-ov.json",
|
||||
"",
|
||||
"Bundled folders:",
|
||||
" config/ sensor JSON configs",
|
||||
" sensors/ per-sensor bundles (finger mesh and visualization data)",
|
||||
"",
|
||||
"CPU-only build (no CUDA/CuPy).",
|
||||
]
|
||||
|
||||
|
||||
def make_config() -> GuiBuildConfig:
|
||||
return GuiBuildConfig(
|
||||
product="ory_finger",
|
||||
entry_script=ENTRY_SCRIPT,
|
||||
exe_name=EXE_NAME,
|
||||
dist_name=DIST_NAME,
|
||||
hidden_imports=list(HIDDEN_IMPORTS),
|
||||
exclude_modules=list(EXCLUDE_MODULES),
|
||||
data_dirs=list(DATA_DIRS),
|
||||
readme_lines=list(README_LINES),
|
||||
)
|
||||
@@ -0,0 +1 @@
|
||||
"""ORY-MINI internal calibration/testing GUI packaging (scaffold)."""
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Scaffold for ORY-MINI internal GUI packaging.
|
||||
|
||||
Not implemented yet. When ready, wire a dedicated calibration/testing entry
|
||||
script and call ``gui.common.pyinstaller_driver.build_with_pyinstaller``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from gui.ory_mini.config import DIST_NAME, ENTRY_SCRIPT, make_config
|
||||
|
||||
|
||||
def main() -> None:
|
||||
config = make_config()
|
||||
raise NotImplementedError(
|
||||
"ORY-MINI GUI packaging is a scaffold only. "
|
||||
f"Planned dist={DIST_NAME!r}, entry={ENTRY_SCRIPT}. "
|
||||
"Implement the internal calibration/testing UI entry, then call "
|
||||
"gui.common.pyinstaller_driver.build_with_pyinstaller(config)."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Product configuration scaffold for the ORY-MINI internal GUI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT
|
||||
|
||||
from gui.common.pyinstaller_driver import GuiBuildConfig
|
||||
|
||||
ENTRY_SCRIPT = REPO_ROOT / "examples" / "mini_pipeline.py"
|
||||
EXE_NAME = "OryMini.exe"
|
||||
DIST_NAME = "OryMini"
|
||||
|
||||
HIDDEN_IMPORTS = [
|
||||
"PyQt5",
|
||||
"pyqtgraph",
|
||||
"cv2",
|
||||
"orisys",
|
||||
"gui",
|
||||
"pipeline",
|
||||
"viz",
|
||||
]
|
||||
|
||||
EXCLUDE_MODULES = [
|
||||
"cupy",
|
||||
"cupyx",
|
||||
"pyopencl",
|
||||
"open3d",
|
||||
"tkinter",
|
||||
"pip",
|
||||
"wheel",
|
||||
]
|
||||
|
||||
DATA_DIRS = [
|
||||
("config", "config"),
|
||||
]
|
||||
|
||||
README_LINES = [
|
||||
"ORY-MINI internal calibration/testing GUI (scaffold)",
|
||||
"",
|
||||
"This product packaging target is reserved for a future internal tool.",
|
||||
"Entry point placeholder:",
|
||||
f" {ENTRY_SCRIPT.as_posix()}",
|
||||
]
|
||||
|
||||
|
||||
def make_config() -> GuiBuildConfig:
|
||||
return GuiBuildConfig(
|
||||
product="ory_mini",
|
||||
entry_script=ENTRY_SCRIPT,
|
||||
exe_name=EXE_NAME,
|
||||
dist_name=DIST_NAME,
|
||||
hidden_imports=list(HIDDEN_IMPORTS),
|
||||
exclude_modules=list(EXCLUDE_MODULES),
|
||||
data_dirs=list(DATA_DIRS),
|
||||
readme_lines=list(README_LINES),
|
||||
windowed=False,
|
||||
)
|
||||
@@ -1,20 +0,0 @@
|
||||
"""Repository and Nuitka build output paths."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
NUITKA_COMPILE_DIR = REPO_ROOT / "build" / "nuitka" / "compile"
|
||||
NUITKA_WHEEL_DIR = REPO_ROOT / "build" / "nuitka" / "wheel"
|
||||
NUITKA_GUI_DIR = REPO_ROOT / "build" / "nuitka" / "gui"
|
||||
GUI_DIST_DIR = REPO_ROOT / "dist" / "OrisysDashboard"
|
||||
|
||||
|
||||
def repo_path(*parts: str) -> Path:
|
||||
return REPO_ROOT.joinpath(*parts)
|
||||
|
||||
|
||||
def sdk_dir(version: str) -> Path:
|
||||
return REPO_ROOT / "SDK" / version
|
||||
@@ -0,0 +1,7 @@
|
||||
"""Local release tooling: preflight gate and per-wheel audit.
|
||||
|
||||
Entry point is ``python build/run.py release <command>``; see
|
||||
:mod:`release.preflight` for the gate steps and :mod:`release.cli` for the CLI.
|
||||
Imports are intentionally not re-exported here so importing a submodule never
|
||||
triggers the whole subsystem.
|
||||
"""
|
||||
@@ -0,0 +1,398 @@
|
||||
"""PyPI credential setup: locate, inspect and write ``~/.pypirc``.
|
||||
|
||||
Editing the config from a command rather than by hand avoids the two mistakes
|
||||
that actually happen: putting something other than ``__token__`` in ``username``,
|
||||
and pasting a TestPyPI token into the ``[pypi]`` section (the two indexes reject
|
||||
each other's tokens with 403, and both token kinds start with ``pypi-``).
|
||||
|
||||
The writer does targeted text edits rather than rewriting through configparser,
|
||||
so the explanatory comments in the file survive.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import configparser
|
||||
import getpass
|
||||
import os
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
INDEXES: dict[str, dict[str, str | None]] = {
|
||||
"pypi": {
|
||||
"upload_url": "https://upload.pypi.org/legacy/",
|
||||
"json_url": "https://pypi.org/pypi/orisys/json",
|
||||
"simple_url": "https://pypi.org/simple/",
|
||||
"token_url": "https://pypi.org/manage/account/token/",
|
||||
"register_url": None,
|
||||
},
|
||||
"testpypi": {
|
||||
"upload_url": "https://test.pypi.org/legacy/",
|
||||
"json_url": "https://test.pypi.org/pypi/orisys/json",
|
||||
"simple_url": "https://test.pypi.org/simple/",
|
||||
"token_url": "https://test.pypi.org/manage/account/token/",
|
||||
"register_url": "https://test.pypi.org/account/register/",
|
||||
},
|
||||
}
|
||||
|
||||
# Substrings that mark a token slot that was never filled in.
|
||||
PLACEHOLDER_MARKERS = ("PASTE", "REPLACE", "TOKEN-HERE", "YOUR-TOKEN", "XXXX")
|
||||
|
||||
DEFAULT_TEMPLATE = """\
|
||||
# PyPI / TestPyPI credentials for twine.
|
||||
#
|
||||
# Keep this file out of any git repository: a token committed once stays in the
|
||||
# history even after you delete it.
|
||||
#
|
||||
# * username is literally "__token__" - never an account name or email
|
||||
# * PyPI and TestPyPI tokens both start with "pypi-" but are NOT
|
||||
# interchangeable: each index rejects the other's with 403
|
||||
# * TestPyPI is a separate service with a separate account
|
||||
|
||||
[distutils]
|
||||
index-servers =
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class CredentialState:
|
||||
index: str
|
||||
configured: bool
|
||||
source: str | None = None
|
||||
problem: str | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class TwineLookup:
|
||||
"""What twine's own resolver says about one index."""
|
||||
|
||||
available: bool
|
||||
known: bool
|
||||
config: dict = field(default_factory=dict)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
def pypirc_path() -> Path:
|
||||
"""Where twine looks by default: ``~/.pypirc`` (same expansion twine uses)."""
|
||||
return Path(os.path.expanduser("~")) / ".pypirc"
|
||||
|
||||
|
||||
def looks_like_placeholder(token: str) -> bool:
|
||||
upper = token.upper()
|
||||
return any(marker in upper for marker in PLACEHOLDER_MARKERS)
|
||||
|
||||
|
||||
def parse_pypirc(text: str) -> dict[str, dict[str, str]]:
|
||||
"""Section -> key -> value, tolerating comments and continuation lines."""
|
||||
sections: dict[str, dict[str, str]] = {}
|
||||
current: str | None = None
|
||||
for raw in text.splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith(("#", ";", "%")):
|
||||
continue
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
current = line[1:-1].strip()
|
||||
sections.setdefault(current, {})
|
||||
continue
|
||||
if current is not None and "=" in line:
|
||||
key, _, value = line.partition("=")
|
||||
sections[current][key.strip()] = value.strip()
|
||||
return sections
|
||||
|
||||
|
||||
def credential_state(
|
||||
index: str,
|
||||
*,
|
||||
path: Path | None = None,
|
||||
env: dict | None = None,
|
||||
) -> CredentialState:
|
||||
"""Is ``index`` usable by twine, and why not when it is not.
|
||||
|
||||
Delegates to ``twine.utils.get_config`` when twine is importable, because the
|
||||
only definition that matters is twine's. The important subtlety that a naive
|
||||
check misses: **twine ignores any section not listed in
|
||||
``[distutils] index-servers``** (with a fallback of ``pypi testpypi`` when the
|
||||
key is absent). A ``[testpypi]`` section without that listing looks configured
|
||||
but twine reports "Missing 'testpypi' section".
|
||||
"""
|
||||
if index not in INDEXES:
|
||||
raise ValueError(f"unknown index {index!r}; expected one of {sorted(INDEXES)}")
|
||||
|
||||
env = os.environ if env is None else env
|
||||
if env.get("TWINE_PASSWORD"):
|
||||
return CredentialState(index, True, "environment variables")
|
||||
|
||||
path = path or pypirc_path()
|
||||
if not path.is_file():
|
||||
return CredentialState(index, False, None, f"{path} does not exist yet")
|
||||
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||
except OSError as exc:
|
||||
return CredentialState(index, False, None, f"cannot read {path}: {exc}")
|
||||
|
||||
sections = parse_pypirc(text)
|
||||
if index not in sections and index not in index_servers(text):
|
||||
# Do not blame index-servers for a section that simply is not there.
|
||||
return CredentialState(index, False, None, f"{path} has no [{index}] section")
|
||||
|
||||
lookup = _twine_lookup(index, path)
|
||||
if lookup.error:
|
||||
return CredentialState(index, False, None, f"twine cannot read {path}: {lookup.error}")
|
||||
|
||||
if lookup.available:
|
||||
if not lookup.known:
|
||||
return CredentialState(index, False, None, _ignored_section(index, path, text, sections))
|
||||
username = str(lookup.config.get("username") or "").strip()
|
||||
password = str(lookup.config.get("password") or "").strip()
|
||||
if not password:
|
||||
# twine knows the index but resolved no password for it: almost always
|
||||
# a section that index-servers does not list.
|
||||
detail = _ignored_section(index, path, text, sections)
|
||||
return CredentialState(index, False, None, detail)
|
||||
else:
|
||||
# twine is not installed: replicate its rule locally.
|
||||
if index not in index_servers(text):
|
||||
return CredentialState(index, False, None, _ignored_section(index, path, text, sections))
|
||||
section = sections[index]
|
||||
username = section.get("username", "").strip()
|
||||
password = section.get("password", "").strip()
|
||||
|
||||
if looks_like_placeholder(password):
|
||||
return CredentialState(
|
||||
index, False, str(path), f"[{index}] still contains a placeholder token"
|
||||
)
|
||||
if username != "__token__":
|
||||
return CredentialState(
|
||||
index,
|
||||
False,
|
||||
str(path),
|
||||
f"[{index}] username is {username!r}; API tokens need the literal '__token__'",
|
||||
)
|
||||
return CredentialState(index, True, str(path))
|
||||
|
||||
|
||||
def _ignored_section(
|
||||
index: str, path: Path, text: str, sections: dict[str, dict[str, str]]
|
||||
) -> str:
|
||||
"""Name the real cause: absent section, or present but not listed by twine."""
|
||||
if index not in sections:
|
||||
return f"{path} has no [{index}] section"
|
||||
if index not in index_servers(text):
|
||||
return (
|
||||
f"[{index}] exists in {path} but is not listed in "
|
||||
"[distutils] index-servers, so twine ignores it entirely; "
|
||||
"fix with `release auth --repair`"
|
||||
)
|
||||
return f"twine resolves no password for [{index}] from {path}"
|
||||
|
||||
|
||||
def _twine_lookup(index: str, path: Path) -> TwineLookup:
|
||||
"""Ask twine itself; ``available=False`` when twine is not installed.
|
||||
|
||||
``get_config`` is not enough on its own: it always contains a ``pypi`` entry
|
||||
(and a ``testpypi`` one when listed) even when it never copied any
|
||||
credentials, so the resolved username/password are what matter.
|
||||
"""
|
||||
try:
|
||||
from twine import utils
|
||||
except ImportError:
|
||||
return TwineLookup(available=False, known=False)
|
||||
try:
|
||||
config = utils.get_config(str(path))
|
||||
except Exception as exc: # twine reports malformed files as exceptions
|
||||
return TwineLookup(available=True, known=False, error=f"{type(exc).__name__}: {exc}")
|
||||
section = config.get(index)
|
||||
if section is None:
|
||||
return TwineLookup(available=True, known=False)
|
||||
return TwineLookup(available=True, known=True, config=dict(section))
|
||||
|
||||
|
||||
def index_servers(text: str) -> set[str]:
|
||||
"""Names twine honours from ``[distutils] index-servers``.
|
||||
|
||||
Replicates ``twine.utils.get_config``, including its surprising asymmetry:
|
||||
|
||||
* key **absent** -> falls back to ``pypi testpypi``
|
||||
* key **present but empty** -> an empty list, so *no* section is honoured
|
||||
(the fallback does not apply), and every section looks configured while
|
||||
twine silently ignores it
|
||||
* continuation lines are part of the value, so ``index-servers =`` followed
|
||||
by indented names is the normal form
|
||||
"""
|
||||
parser = configparser.RawConfigParser()
|
||||
try:
|
||||
parser.read_string(text)
|
||||
except configparser.Error:
|
||||
return set()
|
||||
raw = parser.get("distutils", "index-servers", fallback="pypi testpypi")
|
||||
return set(raw.split())
|
||||
|
||||
|
||||
def validate_token(token: str) -> list[str]:
|
||||
"""Problems with a pasted token. Empty and whitespace are fatal; the rest warn."""
|
||||
return [problem for problem in _token_problems(token) if problem.startswith("ERROR")]
|
||||
|
||||
|
||||
def token_warnings(token: str) -> list[str]:
|
||||
return [problem for problem in _token_problems(token) if problem.startswith("WARN")]
|
||||
|
||||
|
||||
def _token_problems(token: str) -> list[str]:
|
||||
problems: list[str] = []
|
||||
if not token:
|
||||
problems.append("ERROR the token is empty")
|
||||
return problems
|
||||
if any(char.isspace() for char in token):
|
||||
problems.append("ERROR the token contains whitespace (copy/paste artefact?)")
|
||||
if not token.startswith("pypi-"):
|
||||
problems.append(
|
||||
"WARN PyPI API tokens start with 'pypi-'; a legacy account password also "
|
||||
"works but needs username != '__token__'"
|
||||
)
|
||||
if len(token) < 20:
|
||||
problems.append("WARN the token looks too short; check the paste was complete")
|
||||
return problems
|
||||
|
||||
|
||||
def unlisted_sections(path: Path | None = None) -> list[str]:
|
||||
"""Known indexes whose section exists but is missing from ``index-servers``."""
|
||||
path = path or pypirc_path()
|
||||
if not path.is_file():
|
||||
return []
|
||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||
present = set(parse_pypirc(text))
|
||||
listed = index_servers(text)
|
||||
return [index for index in sorted(INDEXES) if index in present and index not in listed]
|
||||
|
||||
|
||||
def repair_index_servers(path: Path | None = None) -> list[str]:
|
||||
"""List sections that exist but are not in ``index-servers``.
|
||||
|
||||
This is the state a hand-edited file lands in: the section is present, so it
|
||||
looks configured, but twine ignores it completely. Returns the indexes added.
|
||||
"""
|
||||
path = path or pypirc_path()
|
||||
if not path.is_file():
|
||||
return []
|
||||
text = path.read_text(encoding="utf-8")
|
||||
present = set(parse_pypirc(text))
|
||||
added: list[str] = []
|
||||
for index in sorted(INDEXES):
|
||||
if index in present and index not in index_servers(text):
|
||||
text = ensure_index_server(text, index)
|
||||
added.append(index)
|
||||
if added:
|
||||
path.write_text(text, encoding="utf-8", newline="")
|
||||
_restrict_permissions(path)
|
||||
return added
|
||||
|
||||
|
||||
def upsert_token(text: str, index: str, token: str) -> str:
|
||||
"""Set ``password`` inside ``[index]``, creating the section if needed."""
|
||||
lines = text.splitlines()
|
||||
start = next((i for i, line in enumerate(lines) if line.strip() == f"[{index}]"), None)
|
||||
|
||||
if start is None:
|
||||
if lines and lines[-1].strip():
|
||||
lines.append("")
|
||||
lines.extend(
|
||||
[
|
||||
f"[{index}]",
|
||||
f"repository = {INDEXES[index]['upload_url']}",
|
||||
"username = __token__",
|
||||
f"password = {token}",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
end = len(lines)
|
||||
for offset in range(start + 1, len(lines)):
|
||||
stripped = lines[offset].strip()
|
||||
if stripped.startswith("[") and stripped.endswith("]"):
|
||||
end = offset
|
||||
break
|
||||
|
||||
insert_at = end
|
||||
for offset in range(start + 1, end):
|
||||
stripped = lines[offset].strip()
|
||||
if stripped.startswith("password"):
|
||||
lines[offset] = f"password = {token}"
|
||||
return "\n".join(lines) + "\n"
|
||||
if stripped.startswith(("username", "repository")):
|
||||
insert_at = offset + 1
|
||||
lines.insert(insert_at, f"password = {token}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def ensure_index_server(text: str, index: str) -> str:
|
||||
"""Make sure ``[distutils] index-servers`` lists ``index``."""
|
||||
if index in index_servers(text):
|
||||
return text
|
||||
|
||||
lines = text.splitlines()
|
||||
if "[distutils]" not in (line.strip() for line in lines):
|
||||
block = ["[distutils]", "index-servers =", f" {index}", ""]
|
||||
return "\n".join(block + lines) + "\n"
|
||||
|
||||
start = next(i for i, line in enumerate(lines) if line.strip() == "[distutils]")
|
||||
for offset in range(start + 1, len(lines)):
|
||||
if lines[offset].strip().startswith("index-servers"):
|
||||
lines.insert(offset + 1, f" {index}")
|
||||
return "\n".join(lines) + "\n"
|
||||
lines.insert(start + 1, f" {index}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _restrict_permissions(path: Path) -> None:
|
||||
"""Best effort: owner-only. Windows ACLs need icacls, not chmod."""
|
||||
try:
|
||||
os.chmod(path, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def write_token(index: str, token: str, *, path: Path | None = None) -> Path:
|
||||
"""Write ``token`` into ``[index]`` of the pypirc file, preserving comments."""
|
||||
if index not in INDEXES:
|
||||
raise ValueError(f"unknown index {index!r}; expected one of {sorted(INDEXES)}")
|
||||
|
||||
path = path or pypirc_path()
|
||||
if path.is_file():
|
||||
text = path.read_text(encoding="utf-8")
|
||||
else:
|
||||
text = DEFAULT_TEMPLATE
|
||||
text = upsert_token(text, index, token)
|
||||
text = ensure_index_server(text, index)
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text, encoding="utf-8", newline="")
|
||||
_restrict_permissions(path)
|
||||
return path
|
||||
|
||||
|
||||
def prompt_for_token(index: str, *, prompt_fn=None, out=None) -> str | None:
|
||||
"""Ask for a token with hidden input. Returns ``None`` if input is unavailable."""
|
||||
out = out or sys.stdout
|
||||
meta = INDEXES[index]
|
||||
print(f"PyPI API token for [{index}]", file=out)
|
||||
print(f" create one at: {meta['token_url']}", file=out)
|
||||
if meta.get("register_url"):
|
||||
print(f" needs an account first: {meta['register_url']}", file=out)
|
||||
print(" input is hidden and is written only to your pypirc file", file=out)
|
||||
|
||||
ask = prompt_fn or getpass.getpass
|
||||
try:
|
||||
return ask("token: ").strip()
|
||||
except (EOFError, KeyboardInterrupt, OSError):
|
||||
# OSError covers environments where stdin cannot be read at all (pytest
|
||||
# capture, a detached CI stdin): treat that as "cannot prompt", never as a
|
||||
# crash.
|
||||
print("", file=out)
|
||||
print("aborted: no token entered", file=out)
|
||||
return None
|
||||
|
||||
|
||||
def read_token_from_stdin() -> str:
|
||||
"""For CI: read the token from stdin instead of a prompt or the command line."""
|
||||
return (sys.stdin.readline() or "").strip()
|
||||
@@ -0,0 +1,421 @@
|
||||
"""Command line for ``python build/run.py release``.
|
||||
|
||||
Commands: ``preflight`` (local gate), ``audit`` (single wheel), ``auth`` (PyPI
|
||||
credentials in ~/.pypirc), ``github`` (sync the public repo + publish a GitHub
|
||||
Release), ``pypi`` (TestPyPI then PyPI). The publish commands are dry-run unless
|
||||
``--apply`` is passed, and tokens are never accepted as command-line arguments.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
from common.verify.wheel import ( # noqa: E402
|
||||
DEFAULT_RUNTIME_CONFIG,
|
||||
default_verification_video,
|
||||
verify_wheel,
|
||||
verify_wheel_assets,
|
||||
verify_wheel_runtime,
|
||||
)
|
||||
from common.verify.wheel_contents import inspect_wheel # noqa: E402
|
||||
|
||||
from release import auth, github, pypi # noqa: E402
|
||||
from release.preflight import PreflightOptions, run_preflight # noqa: E402
|
||||
|
||||
|
||||
def _cmd_preflight(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="build/run.py release preflight",
|
||||
description="Gate a release locally: version, tests, wheel content, runtime.",
|
||||
)
|
||||
parser.add_argument("--version", help="release version (default: pyproject's dynamic version)")
|
||||
parser.add_argument("--wheel-dir", type=Path, help="directory to inspect (default: SDK/<version>)")
|
||||
parser.add_argument("--video", type=Path, help="clip for the end-to-end runtime check")
|
||||
parser.add_argument("--config", default=DEFAULT_RUNTIME_CONFIG, help="config name for that check")
|
||||
parser.add_argument("--frames", type=int, default=40, help="frames to process (default: 40)")
|
||||
parser.add_argument("--skip-tests", action="store_true", help="skip pytest (recorded in the report)")
|
||||
parser.add_argument("--longrun", action="store_true", help="include the slow long-run stability test")
|
||||
parser.add_argument("--allow-dirty", action="store_true", help="allow a dirty working tree")
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="accept a version that a git tag already marks as released",
|
||||
)
|
||||
parser.add_argument("--check-pypi", action="store_true", help="read PyPI and fail if the version exists")
|
||||
parser.add_argument(
|
||||
"--all-wheel-pythons",
|
||||
action="store_true",
|
||||
help="verify every staged wheel (Windows conda environments and Linux Docker images)",
|
||||
)
|
||||
parser.add_argument("--offline", action="store_true", help="reuse parent-env deps instead of PyPI")
|
||||
parser.add_argument("--no-runtime", action="store_true", help="skip the isolated runtime check")
|
||||
parser.add_argument("--no-assets", action="store_true", help="skip the bundled-asset check")
|
||||
parser.add_argument("--keep-going", action="store_true", help="collect every failure instead of stopping")
|
||||
parser.add_argument("--report-dir", type=Path, help="override build/release/reports")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
options = PreflightOptions(
|
||||
version=args.version,
|
||||
wheel_dir=args.wheel_dir,
|
||||
video=args.video,
|
||||
config_name=args.config,
|
||||
frames=args.frames,
|
||||
skip_tests=args.skip_tests,
|
||||
longrun=args.longrun,
|
||||
allow_dirty=args.allow_dirty,
|
||||
force=args.force,
|
||||
check_pypi=args.check_pypi,
|
||||
all_wheel_pythons=args.all_wheel_pythons,
|
||||
offline=args.offline,
|
||||
keep_going=args.keep_going,
|
||||
report_dir=args.report_dir,
|
||||
no_runtime=args.no_runtime,
|
||||
no_assets=args.no_assets,
|
||||
)
|
||||
return run_preflight(options, repo_root=REPO_ROOT)
|
||||
|
||||
|
||||
def _cmd_audit(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="build/run.py release audit",
|
||||
description="Inspect one built wheel without a release context.",
|
||||
)
|
||||
parser.add_argument("wheel", type=Path, help="path to an .whl")
|
||||
parser.add_argument("--expected-version", help="assert the wheel reports this version")
|
||||
parser.add_argument(
|
||||
"--strict-bundle",
|
||||
action="store_true",
|
||||
help="treat bundled-file drift from the current source tree as a failure",
|
||||
)
|
||||
parser.add_argument("--runtime", action="store_true", help="also install and run the wheel")
|
||||
parser.add_argument("--assets", action="store_true", help="also assert bundled assets load")
|
||||
parser.add_argument("--video", type=Path, help="clip for --runtime")
|
||||
parser.add_argument("--config", default=DEFAULT_RUNTIME_CONFIG)
|
||||
parser.add_argument("--frames", type=int, default=40)
|
||||
parser.add_argument("--offline", action="store_true")
|
||||
parser.add_argument(
|
||||
"--python",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="interpreter to build the verification venv from (default: the running one)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
wheel = args.wheel
|
||||
if not wheel.is_file():
|
||||
print(f"[ERROR] wheel not found: {wheel}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
exit_code = 0
|
||||
|
||||
result = inspect_wheel(
|
||||
wheel,
|
||||
expected_version=args.expected_version,
|
||||
repo_root=REPO_ROOT,
|
||||
strict_bundle=args.strict_bundle,
|
||||
)
|
||||
print(result.render())
|
||||
if not result.ok:
|
||||
exit_code = 1
|
||||
|
||||
if args.assets and args.runtime:
|
||||
# One venv for both checks: building it (and downloading dependencies in
|
||||
# online mode) is the expensive part.
|
||||
video = args.video or default_verification_video(REPO_ROOT)
|
||||
if video is None:
|
||||
print("[FAIL] runtime: no verification clip; pass --video or set ORISYS_VERIFY_VIDEO")
|
||||
return 1
|
||||
try:
|
||||
record = verify_wheel(
|
||||
wheel,
|
||||
video=video,
|
||||
config_name=args.config,
|
||||
frames=args.frames,
|
||||
python_executable=args.python,
|
||||
offline=args.offline,
|
||||
)
|
||||
print(f"[ok ] {record['imports']}")
|
||||
print(f"[ok ] {record['assets']}")
|
||||
print(f"[ok ] runtime: frames={record['frames']} flow={record['flow_shape']}")
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] isolated verification: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
return exit_code
|
||||
|
||||
if args.assets:
|
||||
try:
|
||||
print(verify_wheel_assets(wheel, python_executable=args.python))
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] bundled assets: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
|
||||
if args.runtime:
|
||||
video = args.video or default_verification_video(REPO_ROOT)
|
||||
if video is None:
|
||||
print("[FAIL] runtime: no verification clip; pass --video or set ORISYS_VERIFY_VIDEO")
|
||||
return 1
|
||||
try:
|
||||
info = verify_wheel_runtime(
|
||||
wheel,
|
||||
video=video,
|
||||
config_name=args.config,
|
||||
frames=args.frames,
|
||||
python_executable=args.python,
|
||||
offline=args.offline,
|
||||
)
|
||||
print(f"[ok ] runtime: {info}")
|
||||
except Exception as exc:
|
||||
print(f"[FAIL] runtime: {type(exc).__name__}: {exc}")
|
||||
exit_code = 1
|
||||
|
||||
return exit_code
|
||||
|
||||
|
||||
def _cmd_github(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="build/run.py release github",
|
||||
description="Sync orisys-sdk-release, tag, and publish a GitHub Release.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--version", required=True, help="release version, e.g. <major>.<minor>.<patch>"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--apply",
|
||||
action="store_true",
|
||||
help="actually mutate the release repo, push, and create the release",
|
||||
)
|
||||
parser.add_argument("--notes-file", type=Path, default=None, help="release notes markdown")
|
||||
parser.add_argument(
|
||||
"--attach-preflight",
|
||||
action="store_true",
|
||||
help="also upload the preflight report as a release asset (contains local paths)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--allow-dirty-release",
|
||||
action="store_true",
|
||||
help="proceed even if the release repo has uncommitted changes",
|
||||
)
|
||||
parser.add_argument("--skip-fetch", action="store_true", help="skip `git fetch --tags`")
|
||||
parser.add_argument(
|
||||
"--skip-checks", action="store_true", help="skip the manifest's downstream checks"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-release-upload",
|
||||
action="store_true",
|
||||
help="stop after pushing the tag and print the `gh release create` command",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
return github.publish(
|
||||
github.Options(
|
||||
version=args.version,
|
||||
apply=args.apply,
|
||||
notes_file=args.notes_file,
|
||||
attach_preflight=args.attach_preflight,
|
||||
allow_dirty_release=args.allow_dirty_release,
|
||||
skip_fetch=args.skip_fetch,
|
||||
skip_checks=args.skip_checks,
|
||||
skip_release_upload=args.skip_release_upload,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _cmd_auth(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="build/run.py release auth",
|
||||
description=(
|
||||
"Inspect or set up PyPI credentials in ~/.pypirc. "
|
||||
"Tokens are read from a hidden prompt or stdin, never from argv."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--set-token",
|
||||
choices=sorted(auth.INDEXES),
|
||||
default=None,
|
||||
help="store a token for this index (prompts unless --token-stdin)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--token-stdin",
|
||||
action="store_true",
|
||||
help="read the token from stdin instead of prompting (for scripts/CI)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--repair",
|
||||
action="store_true",
|
||||
help=(
|
||||
"add existing sections to [distutils] index-servers; twine ignores "
|
||||
"sections that are not listed there"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--index",
|
||||
choices=sorted(auth.INDEXES),
|
||||
default="pypi",
|
||||
help="index to report on when not setting a token (default: pypi)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.repair:
|
||||
path = auth.pypirc_path()
|
||||
added = auth.repair_index_servers(path)
|
||||
if added:
|
||||
print(f"[ok ] added {added} to [distutils] index-servers in {path}")
|
||||
else:
|
||||
print(f"[ok ] nothing to repair in {path}")
|
||||
for index in sorted(auth.INDEXES):
|
||||
state = auth.credential_state(index)
|
||||
flag = "[ok ]" if state.configured else "[-- ]"
|
||||
detail = state.source if state.configured else state.problem
|
||||
print(f"{flag} [{index}] {detail}")
|
||||
return 0 if auth.credential_state(args.index).configured else 1
|
||||
|
||||
if args.set_token:
|
||||
if args.token_stdin:
|
||||
token = auth.read_token_from_stdin()
|
||||
else:
|
||||
token = auth.prompt_for_token(args.set_token)
|
||||
if not token:
|
||||
print("[FAIL] no token entered")
|
||||
return 1
|
||||
for problem in auth.validate_token(token):
|
||||
print(f"[FAIL] {problem.removeprefix('ERROR ')}")
|
||||
return 1
|
||||
for warning in auth.token_warnings(token):
|
||||
print(f"[warn] {warning.removeprefix('WARN ')}")
|
||||
path = auth.write_token(args.set_token, token)
|
||||
print(f"[ok ] wrote [{args.set_token}] credentials to {path}")
|
||||
state = auth.credential_state(args.set_token)
|
||||
if not state.configured:
|
||||
print(f"[FAIL] the file still does not validate: {state.problem}")
|
||||
return 1
|
||||
print(f"[ok ] [{args.set_token}] credentials verified ({state.source})")
|
||||
return 0
|
||||
|
||||
path = auth.pypirc_path()
|
||||
print(f"config file: {path} ({'exists' if path.is_file() else 'missing'})")
|
||||
for index in sorted(auth.INDEXES):
|
||||
state = auth.credential_state(index)
|
||||
if state.configured:
|
||||
print(f"[ok ] [{index}] configured ({state.source})")
|
||||
else:
|
||||
print(f"[-- ] [{index}] not usable: {state.problem}")
|
||||
|
||||
unlisted = auth.unlisted_sections(path)
|
||||
if unlisted:
|
||||
print()
|
||||
print(
|
||||
f"[warn] {unlisted} have sections that [distutils] index-servers does not "
|
||||
"list; twine ignores those entirely"
|
||||
)
|
||||
print(" fix with: python build/run.py release auth --repair")
|
||||
|
||||
print()
|
||||
print("set a token with:")
|
||||
for index in sorted(auth.INDEXES):
|
||||
if not auth.credential_state(index).configured:
|
||||
print(f" python build/run.py release auth --set-token {index}")
|
||||
print(" (TestPyPI needs its own account; if you only hold a production token,")
|
||||
print(" skip it and publish with `release pypi --pypi-only`)")
|
||||
|
||||
return 0 if auth.credential_state(args.index).configured else 1
|
||||
|
||||
|
||||
def _cmd_pypi(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="build/run.py release pypi",
|
||||
description="Upload the certified wheels to TestPyPI and then PyPI.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--version", required=True, help="release version, e.g. <major>.<minor>.<patch>"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--apply",
|
||||
action="store_true",
|
||||
help="actually upload (uploads are permanent and cannot be overwritten)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--confirm-pypi",
|
||||
action="store_true",
|
||||
help="after TestPyPI, also publish to the real PyPI",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pypi-only",
|
||||
action="store_true",
|
||||
help=(
|
||||
"upload straight to PyPI, skipping the TestPyPI rehearsal "
|
||||
"(for when you only hold a production token)"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prompt-credentials",
|
||||
action="store_true",
|
||||
help="if a token is missing, prompt for it and store it in ~/.pypirc",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--token-stdin",
|
||||
action="store_true",
|
||||
help="read a missing token from stdin instead of prompting (scripts/CI)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-install-verify",
|
||||
action="store_true",
|
||||
help="skip fetching the artifact back out of the index",
|
||||
)
|
||||
parser.add_argument("--skip-check", action="store_true", help="skip `twine check`")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.pypi_only and args.confirm_pypi:
|
||||
parser.error("--pypi-only and --confirm-pypi are mutually exclusive")
|
||||
|
||||
if args.token_stdin and not args.prompt_credentials:
|
||||
parser.error("--token-stdin requires --prompt-credentials")
|
||||
|
||||
token = auth.read_token_from_stdin() if args.token_stdin else None
|
||||
|
||||
return pypi.publish(
|
||||
pypi.Options(
|
||||
version=args.version,
|
||||
apply=args.apply,
|
||||
confirm_pypi=args.confirm_pypi,
|
||||
verify_install=not args.skip_install_verify,
|
||||
skip_check=args.skip_check,
|
||||
pypi_only=args.pypi_only,
|
||||
prompt_credentials=args.prompt_credentials,
|
||||
token=token,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
argv = list(sys.argv[1:] if argv is None else argv)
|
||||
if not argv or argv[0] in ("-h", "--help"):
|
||||
print(__doc__)
|
||||
print("Commands: preflight | audit | auth | github | pypi")
|
||||
return 0
|
||||
|
||||
command, rest = argv[0], argv[1:]
|
||||
if command == "preflight":
|
||||
return _cmd_preflight(rest)
|
||||
if command == "audit":
|
||||
return _cmd_audit(rest)
|
||||
if command == "auth":
|
||||
return _cmd_auth(rest)
|
||||
if command == "github":
|
||||
return _cmd_github(rest)
|
||||
if command == "pypi":
|
||||
return _cmd_pypi(rest)
|
||||
|
||||
print(f"Unknown release command: {command!r}", file=sys.stderr)
|
||||
print("Choose from: preflight, audit, auth, github, pypi", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Sync the public release repo and publish a GitHub Release.
|
||||
|
||||
``python build/run.py release github`` runs, in order:
|
||||
|
||||
1. certified-wheels gate (a passing ``release preflight`` report must exist and
|
||||
every wheel must still hash to what it recorded)
|
||||
2. refuse if the version is already tagged or already has a GitHub Release
|
||||
3. sync the open-source subset of this checkout into ``orisys-sdk-release``
|
||||
(``tools/sync_downstreams.py`` + the manifest's ``checks``)
|
||||
4. stage ``SDK/<version>/*.whl`` into ``orisys-sdk-release/release/<version>/``
|
||||
5. commit, tag ``v<version>``, push branch + tag
|
||||
6. create the GitHub Release with the wheels attached, then verify the assets
|
||||
|
||||
Nothing mutates anything unless ``--apply`` is passed: the default is a dry run
|
||||
that prints every command it would run.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
|
||||
from release.publish import GateResult, PublishGateError, load_gate # noqa: E402
|
||||
from release.runtime import PLATFORM_LABELS, wheel_target # noqa: E402
|
||||
|
||||
MANIFEST_PATH = REPO_ROOT / "tools" / "downstream_sync_manifest.json"
|
||||
SYNC_TOOL = REPO_ROOT / "tools" / "sync_downstreams.py"
|
||||
RELEASE_TARGET = "orisys-sdk-release"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Options:
|
||||
version: str
|
||||
apply: bool = False
|
||||
notes_file: Path | None = None
|
||||
attach_preflight: bool = False
|
||||
allow_dirty_release: bool = False
|
||||
skip_fetch: bool = False
|
||||
skip_checks: bool = False
|
||||
skip_release_upload: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class CommandStep:
|
||||
description: str
|
||||
argv: list[str]
|
||||
cwd: Path | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class Plan:
|
||||
release_repo: Path
|
||||
branch: str
|
||||
tag: str
|
||||
assets: list[Path] = field(default_factory=list)
|
||||
steps: list[CommandStep] = field(default_factory=list)
|
||||
|
||||
|
||||
def manifest_entry() -> dict:
|
||||
"""The manifest entry for the public release repo."""
|
||||
payload = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
|
||||
for item in payload.get("downstreams", []):
|
||||
if item.get("name") == RELEASE_TARGET:
|
||||
return item
|
||||
raise PublishGateError(f"{MANIFEST_PATH} has no downstream named {RELEASE_TARGET!r}")
|
||||
|
||||
|
||||
def release_repo_path() -> Path:
|
||||
return (REPO_ROOT / manifest_entry()["path"]).resolve()
|
||||
|
||||
|
||||
def _git(repo: Path, *args: str, check: bool = True) -> str:
|
||||
completed = subprocess.run(
|
||||
["git", "-C", str(repo), *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if check and completed.returncode != 0:
|
||||
raise PublishGateError(
|
||||
f"git {' '.join(args)} failed in {repo}:\n{completed.stderr.strip()}"
|
||||
)
|
||||
return (completed.stdout or "").strip()
|
||||
|
||||
|
||||
def build_release_notes(version: str, wheels: list[Path], git_commit: str | None) -> str:
|
||||
platforms = sorted({PLATFORM_LABELS[wheel_target(wheel)[1]] for wheel in wheels})
|
||||
lines = [
|
||||
f"orisys {version}",
|
||||
"",
|
||||
f"Wheels ({', '.join(platforms)}; Python ABI is shown in each filename):",
|
||||
]
|
||||
lines.extend(f"- `{wheel.name}`" for wheel in wheels)
|
||||
lines.append("")
|
||||
if git_commit:
|
||||
lines.append(
|
||||
f"Built from `{git_commit[:12]}` and certified by `release preflight`."
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def build_github_steps(
|
||||
*,
|
||||
version: str,
|
||||
branch: str,
|
||||
tag: str,
|
||||
assets: list[Path],
|
||||
notes_file: Path | None,
|
||||
create_tag: bool = True,
|
||||
title: str | None = None,
|
||||
) -> list[CommandStep]:
|
||||
"""The mutating commands, as pure data so they can be asserted in tests.
|
||||
|
||||
``create_tag=False`` is the resume path: the tag already exists (a previous
|
||||
run pushed it before failing), so only the remaining steps are emitted.
|
||||
"""
|
||||
release_argv = [
|
||||
"gh",
|
||||
"release",
|
||||
"create",
|
||||
tag,
|
||||
"--verify-tag",
|
||||
"--title",
|
||||
title or f"v{version}",
|
||||
"--latest",
|
||||
]
|
||||
if notes_file is not None:
|
||||
release_argv += ["--notes-file", str(notes_file)]
|
||||
release_argv += [str(asset) for asset in assets]
|
||||
|
||||
steps: list[CommandStep] = []
|
||||
if create_tag:
|
||||
steps.append(CommandStep("create tag", ["git", "tag", tag]))
|
||||
steps.append(CommandStep("push branch", ["git", "push", "origin", branch]))
|
||||
steps.append(CommandStep("push tag", ["git", "push", "origin", tag]))
|
||||
steps.append(CommandStep("create GitHub Release with assets", release_argv))
|
||||
return steps
|
||||
|
||||
|
||||
def non_github_remote_problem(repo: Path) -> str | None:
|
||||
"""Why ``gh`` cannot target this repo, or ``None`` if the remote is fine.
|
||||
|
||||
``gh`` resolves its repository from the working directory. Running it from
|
||||
this checkout instead of the release repo is a real failure mode: the SDK
|
||||
repo's origin is an internal host, and ``gh`` reports the confusing "none of
|
||||
the git remotes ... point to a known GitHub host".
|
||||
"""
|
||||
remote = _git(repo, "remote", "get-url", "origin", check=False)
|
||||
if "github.com" in remote:
|
||||
return None
|
||||
return (
|
||||
f"the release repo's origin is {remote!r}, which is not a GitHub host. "
|
||||
"`gh` resolves the repository from the working directory, so point origin "
|
||||
"at github.com before publishing a release"
|
||||
)
|
||||
|
||||
|
||||
def _check_gh(repo: Path) -> list[str]:
|
||||
"""Problems that would stop the release step, reported before any mutation."""
|
||||
problems: list[str] = []
|
||||
if shutil.which("gh") is None:
|
||||
problems.append(
|
||||
"the `gh` CLI is not on PATH. Install it (`winget install --id GitHub.cli`) "
|
||||
"and authenticate (`gh auth login`), or use --skip-release-upload"
|
||||
)
|
||||
return problems
|
||||
|
||||
remote_problem = non_github_remote_problem(repo)
|
||||
if remote_problem:
|
||||
problems.append(remote_problem)
|
||||
|
||||
completed = subprocess.run(
|
||||
["gh", "auth", "status"], capture_output=True, text=True, check=False
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
problems.append("`gh auth status` failed; run `gh auth login` first")
|
||||
return problems
|
||||
|
||||
|
||||
def _existing_release_tags(repo: Path, version: str) -> list[str]:
|
||||
wanted = {version, f"v{version}"}
|
||||
tags = set(_git(repo, "tag", "--list").splitlines())
|
||||
return sorted(wanted & tags)
|
||||
|
||||
|
||||
def _release_exists(repo: Path, tag: str) -> bool | None:
|
||||
"""Is there already a GitHub Release for ``tag``?
|
||||
|
||||
Returns ``None`` when that cannot be determined (``gh`` unavailable), so the
|
||||
caller can distinguish "not published yet" from "unknown" instead of treating
|
||||
a missing tool as proof that nothing was published.
|
||||
|
||||
Must run with ``cwd`` set to the release repo: `gh` resolves the target
|
||||
repository from the working directory, and this checkout's own remote is an
|
||||
internal host.
|
||||
"""
|
||||
if shutil.which("gh") is None:
|
||||
return None
|
||||
completed = subprocess.run(
|
||||
["gh", "release", "view", tag, "--json", "tagName"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
cwd=str(repo),
|
||||
)
|
||||
return completed.returncode == 0
|
||||
|
||||
|
||||
def _sync(*, run_checks: bool) -> None:
|
||||
"""Copy the public subset into the release repo and run its checks."""
|
||||
python = sys.executable
|
||||
for command in (
|
||||
[python, str(SYNC_TOOL), "apply", "--target", RELEASE_TARGET],
|
||||
*(
|
||||
[[python, str(SYNC_TOOL), "run-checks", "--target", RELEASE_TARGET]]
|
||||
if run_checks
|
||||
else []
|
||||
),
|
||||
):
|
||||
completed = subprocess.run(command, cwd=REPO_ROOT, check=False)
|
||||
if completed.returncode != 0:
|
||||
raise PublishGateError(
|
||||
f"`{' '.join(command[1:])}` failed with exit {completed.returncode}"
|
||||
)
|
||||
|
||||
|
||||
def _stage_wheels(wheels: list[Path], asset_dir: Path) -> None:
|
||||
asset_dir.mkdir(parents=True, exist_ok=True)
|
||||
for wheel in wheels:
|
||||
shutil.copy2(wheel, asset_dir / wheel.name)
|
||||
|
||||
|
||||
def publish(options: Options) -> int:
|
||||
version = options.version
|
||||
mode = "APPLY" if options.apply else "DRY RUN"
|
||||
print(f"=== release github: orisys {version} ({mode}) ===")
|
||||
|
||||
# --- 1. gate -------------------------------------------------------
|
||||
try:
|
||||
gate: GateResult = load_gate(REPO_ROOT, version)
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
print(f"[ok ] preflight report: {gate.report_path}")
|
||||
print(f" certified commit: {gate.git_commit}")
|
||||
for wheel in gate.wheels:
|
||||
print(f" [ok] {wheel.name} sha256={wheel.sha256[:12]} {wheel.size} bytes")
|
||||
for warning in gate.warnings:
|
||||
print(f"[warn] {warning}")
|
||||
|
||||
# --- 2. target repo ------------------------------------------------
|
||||
try:
|
||||
repo = release_repo_path()
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
if not (repo / ".git").exists():
|
||||
print(f"[FAIL] {repo} is not a git repository")
|
||||
return 1
|
||||
|
||||
try:
|
||||
if not options.skip_fetch:
|
||||
_git(repo, "fetch", "--tags", "--quiet")
|
||||
print(f"[ok ] git fetch --tags in {repo}")
|
||||
branch = _git(repo, "rev-parse", "--abbrev-ref", "HEAD")
|
||||
dirty = _git(repo, "status", "--porcelain")
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
|
||||
if dirty and not options.allow_dirty_release:
|
||||
preview = "\n".join(dirty.splitlines()[:10])
|
||||
print(
|
||||
f"[FAIL] {repo} has uncommitted changes; commit or stash them first "
|
||||
f"(or pass --allow-dirty-release):\n{preview}"
|
||||
)
|
||||
return 1
|
||||
print(f"[ok ] release repo: {repo} ({branch}) clean={not dirty}")
|
||||
|
||||
tag = f"v{version}"
|
||||
existing = _existing_release_tags(repo, version)
|
||||
# A tag that exists without a GitHub Release means a previous run got as far
|
||||
# as pushing the tag and then failed (or the upload was left manual). That is
|
||||
# exactly the resumable state, so continue instead of refusing.
|
||||
create_tag = not existing
|
||||
if existing:
|
||||
print(
|
||||
f"[warn] tag(s) {existing} already exist in {repo}; skipping tag creation "
|
||||
"and continuing to the release upload"
|
||||
)
|
||||
release_state = _release_exists(repo, tag)
|
||||
if release_state is True:
|
||||
print(f"[FAIL] GitHub Release {tag} already exists; nothing to publish")
|
||||
return 1
|
||||
if release_state is None:
|
||||
print(
|
||||
"[warn] cannot tell whether a GitHub Release for "
|
||||
f"{tag} already exists (`gh` not installed); --apply will stop at the gh check"
|
||||
)
|
||||
if create_tag:
|
||||
print(f"[ok ] tag {tag} does not exist in {repo}")
|
||||
elif create_tag:
|
||||
print(f"[ok ] no existing tag or GitHub Release for {tag}")
|
||||
else:
|
||||
print(f"[ok ] no GitHub Release for {tag} yet")
|
||||
|
||||
# A dry run is a preview of what would happen, so missing tooling is a warning
|
||||
# there and a hard stop only when we are about to mutate something.
|
||||
if options.skip_release_upload:
|
||||
print("[warn] --skip-release-upload: the GitHub Release will be left to you")
|
||||
else:
|
||||
gh_problems = _check_gh(repo)
|
||||
if gh_problems:
|
||||
for problem in gh_problems:
|
||||
print(f"[{'FAIL' if options.apply else 'warn'}] {problem}")
|
||||
if options.apply:
|
||||
return 1
|
||||
else:
|
||||
print("[ok ] gh CLI is installed and authenticated")
|
||||
|
||||
# --- 3. plan the mutating steps ------------------------------------
|
||||
wheels = [REPO_ROOT / "SDK" / version / record.name for record in gate.wheels]
|
||||
missing = [str(wheel) for wheel in wheels if not wheel.is_file()]
|
||||
if missing:
|
||||
print(f"[FAIL] wheels missing from disk: {missing}")
|
||||
return 1
|
||||
|
||||
asset_dir = repo / "release" / version
|
||||
assets = [asset_dir / wheel.name for wheel in wheels]
|
||||
notes_file = options.notes_file or gate.report_path.parent / "github-release-notes.md"
|
||||
generated_notes = build_release_notes(version, wheels, gate.git_commit) if options.notes_file is None else None
|
||||
steps = build_github_steps(
|
||||
version=version,
|
||||
branch=branch,
|
||||
tag=tag,
|
||||
assets=assets,
|
||||
notes_file=notes_file,
|
||||
create_tag=create_tag,
|
||||
)
|
||||
|
||||
print()
|
||||
print(f"[plan] sync {len(manifest_entry().get('sync', []))} entry(ies) into {repo}")
|
||||
print(f"[plan] stage {len(wheels)} wheel(s) into {asset_dir}")
|
||||
if create_tag:
|
||||
print(f"[plan] commit {repo} and create tag {tag}")
|
||||
else:
|
||||
print(f"[plan] commit {repo} (tag {tag} already exists)")
|
||||
for step in steps:
|
||||
manual = options.skip_release_upload and step.description.startswith(
|
||||
"create GitHub Release"
|
||||
)
|
||||
prefix = "[plan] (left manual)" if manual else "[plan]"
|
||||
print(f"{prefix} {step.description}: {' '.join(step.argv)}")
|
||||
print()
|
||||
|
||||
if not options.apply:
|
||||
if generated_notes:
|
||||
print(generated_notes)
|
||||
print("dry run: nothing was changed. Re-run with --apply to execute.")
|
||||
return 0
|
||||
|
||||
# --- 4. execute ----------------------------------------------------
|
||||
try:
|
||||
if generated_notes:
|
||||
notes_file.write_text(generated_notes, encoding="utf-8")
|
||||
commit_before = _git(repo, "rev-parse", "HEAD")
|
||||
_sync(run_checks=not options.skip_checks)
|
||||
_stage_wheels(wheels, asset_dir)
|
||||
|
||||
if options.attach_preflight:
|
||||
shutil.copy2(gate.report_path, asset_dir / f"{version}-preflight.json")
|
||||
|
||||
_git(repo, "add", "-A")
|
||||
pending = _git(repo, "status", "--porcelain")
|
||||
if pending:
|
||||
_git(repo, "commit", "-m", f"release {version}")
|
||||
print(f"[ok ] committed {len(pending.splitlines())} change(s) in {repo}")
|
||||
else:
|
||||
print("[ok ] nothing new to commit in the release repo")
|
||||
|
||||
if create_tag:
|
||||
_git(repo, "tag", tag)
|
||||
# Pushing an already-up-to-date ref is a no-op, so resuming is safe.
|
||||
_git(repo, "push", "origin", branch)
|
||||
_git(repo, "push", "origin", tag)
|
||||
print(f"[ok ] pushed {branch} and {tag}")
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
|
||||
release_argv = next(
|
||||
step.argv for step in steps if step.description.startswith("create GitHub Release")
|
||||
)
|
||||
|
||||
if options.skip_release_upload:
|
||||
print()
|
||||
print("[todo] publish the release yourself:")
|
||||
print(" " + " ".join(release_argv))
|
||||
print()
|
||||
print("=== release github: repo synced and tagged; release upload left manual ===")
|
||||
return 0
|
||||
|
||||
# `gh` resolves its target repository from the working directory, so it must
|
||||
# run inside the release repo - this checkout's own origin is an internal host.
|
||||
completed = subprocess.run(release_argv, check=False, cwd=str(repo))
|
||||
if completed.returncode != 0:
|
||||
print(f"[FAIL] gh release create exited {completed.returncode}")
|
||||
return 1
|
||||
print(f"[ok ] GitHub Release {tag} created")
|
||||
|
||||
# --- 5. verify the published assets --------------------------------
|
||||
completed = subprocess.run(
|
||||
["gh", "release", "view", tag, "--json", "assets,tagName"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
cwd=str(repo),
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
print("[FAIL] could not read back the release to verify assets")
|
||||
return 1
|
||||
payload = json.loads(completed.stdout or "{}")
|
||||
published = {asset["name"] for asset in payload.get("assets", [])}
|
||||
expected = {asset.name for asset in assets}
|
||||
if options.attach_preflight:
|
||||
expected.add(f"{version}-preflight.json")
|
||||
if published != expected:
|
||||
print(
|
||||
f"[FAIL] release asset mismatch:\n expected {sorted(expected)}\n"
|
||||
f" published {sorted(published)}"
|
||||
)
|
||||
return 1
|
||||
print(f"[ok ] verified {len(published)} asset(s) on the release")
|
||||
print()
|
||||
print(f"=== release github PASSED (commit {commit_before[:12]} -> {_git(repo, 'rev-parse', 'HEAD')[:12]}) ===")
|
||||
return 0
|
||||
@@ -0,0 +1,580 @@
|
||||
"""Local release gate: everything that must hold before publishing a wheel.
|
||||
|
||||
``release preflight`` answers one question with an exit code: is this checkout,
|
||||
this version and these wheels safe to publish? It never publishes anything;
|
||||
native verification environments may download dependencies from PyPI.
|
||||
|
||||
Steps, in order:
|
||||
|
||||
1. ``repo-state`` - git repository, clean working tree, recorded commit
|
||||
2. ``version-consistency`` - __version__ == pyproject dynamic attr == wheel names
|
||||
3. ``version-strings`` - no stale version literal left in tracked prose
|
||||
4. ``duplicate-guard`` - already-released version (git tag, optionally PyPI)
|
||||
5. ``tests`` - pytest (slow/hardware markers excluded)
|
||||
6. ``bundle-precheck`` - every bundle item resolves; required assets present
|
||||
7. ``wheel-inspection`` - static metadata/content assertions per wheel
|
||||
8. ``wheel-runtime`` - one isolated venv per wheel: bundled assets load and
|
||||
real frames flow through the public pipeline
|
||||
|
||||
Output lands in ``build/release/reports/<version>/preflight.{json,txt}``, and is
|
||||
the contract the publish commands (``release github`` / ``release pypi``) consume:
|
||||
they refuse to ship anything this run did not certify.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import platform
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
from common.pyproject_utils import bundle_items, get_version # noqa: E402
|
||||
from common.verify.wheel import ( # noqa: E402
|
||||
DEFAULT_RUNTIME_CONFIG,
|
||||
default_verification_video,
|
||||
verify_wheel,
|
||||
)
|
||||
from common.verify.wheel_contents import inspect_wheel # noqa: E402
|
||||
from wheel.layout import ( # noqa: E402
|
||||
resolve_bundle_source_files,
|
||||
verify_required_assets,
|
||||
)
|
||||
|
||||
from release import report as report_mod # noqa: E402
|
||||
from release import version as version_mod # noqa: E402
|
||||
from release.runtime import verify_manylinux_wheel, wheel_target # noqa: E402
|
||||
|
||||
|
||||
@dataclass
|
||||
class Step:
|
||||
name: str
|
||||
ok: bool
|
||||
detail: str = ""
|
||||
duration_s: float = 0.0
|
||||
data: dict = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PreflightOptions:
|
||||
version: str | None = None
|
||||
wheel_dir: Path | None = None
|
||||
video: Path | None = None
|
||||
config_name: str = DEFAULT_RUNTIME_CONFIG
|
||||
frames: int = 40
|
||||
skip_tests: bool = False
|
||||
longrun: bool = False
|
||||
allow_dirty: bool = False
|
||||
force: bool = False
|
||||
check_pypi: bool = False
|
||||
all_wheel_pythons: bool = False
|
||||
offline: bool = False
|
||||
keep_going: bool = False
|
||||
report_dir: Path | None = None
|
||||
no_runtime: bool = False
|
||||
no_assets: bool = False
|
||||
|
||||
|
||||
def _sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with Path(path).open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
|
||||
|
||||
class Preflight:
|
||||
def __init__(self, options: PreflightOptions, repo_root: Path | None = None) -> None:
|
||||
self.options = options
|
||||
self.repo_root = Path(repo_root or REPO_ROOT)
|
||||
self.steps: list[Step] = []
|
||||
self.git: dict = {"commit": None, "branch": None, "dirty": None}
|
||||
self.wheels: list[Path] = []
|
||||
self.wheel_dir: Path | None = None
|
||||
self.runtime_records: list[dict] = []
|
||||
self.aborted = False
|
||||
self.ok = False
|
||||
|
||||
if options.version:
|
||||
self.version = options.version
|
||||
self.version_source = "explicit --version"
|
||||
elif (self.repo_root / "pyproject.toml").is_file():
|
||||
self.version = get_version(self.repo_root)
|
||||
self.version_source = "pyproject.toml dynamic attr"
|
||||
else:
|
||||
# Not version-shaped on purpose: this is a sentinel for "unresolved",
|
||||
# and it must not look like a release version to the version-string scan.
|
||||
self.version = "unknown"
|
||||
self.version_source = "unresolved"
|
||||
|
||||
self.report_dir = report_mod.reports_root(self.repo_root, options.report_dir) / self.version
|
||||
|
||||
# -- helpers ---------------------------------------------------------
|
||||
|
||||
def _git(self, *args: str) -> str:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
["git", *args],
|
||||
cwd=self.repo_root,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
except OSError:
|
||||
return ""
|
||||
return (completed.stdout or "").strip()
|
||||
|
||||
def _execute(self, step_fn) -> None:
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
step = step_fn()
|
||||
except Exception as exc: # a crashed step is a failed step, never a silent pass
|
||||
step = Step(name=getattr(step_fn, "__name__", "step").lstrip("_").replace("_", "-"),
|
||||
ok=False, detail=f"crashed: {type(exc).__name__}: {exc}")
|
||||
step.duration_s = round(time.perf_counter() - started, 3)
|
||||
self.steps.append(step)
|
||||
|
||||
flag = "ok " if step.ok else "FAIL"
|
||||
print(f"[{flag}] {step.name} ({step.duration_s:.2f}s)")
|
||||
for line in str(step.detail).splitlines():
|
||||
print(f" {line}")
|
||||
|
||||
if not step.ok and not self.options.keep_going:
|
||||
self.aborted = True
|
||||
print(" stopping: pass --keep-going to collect every failure")
|
||||
|
||||
# -- steps -----------------------------------------------------------
|
||||
|
||||
def _repo_state(self) -> Step:
|
||||
if not (self.repo_root / ".git").exists():
|
||||
return Step("repo-state", False, f"{self.repo_root} is not a git repository")
|
||||
|
||||
commit = self._git("rev-parse", "HEAD")
|
||||
branch = self._git("rev-parse", "--abbrev-ref", "HEAD")
|
||||
porcelain = self._git("status", "--porcelain")
|
||||
dirty = bool(porcelain)
|
||||
self.git = {"commit": commit, "branch": branch, "dirty": dirty}
|
||||
|
||||
if dirty and not self.options.allow_dirty:
|
||||
preview = "\n".join(porcelain.splitlines()[:20])
|
||||
return Step(
|
||||
"repo-state",
|
||||
False,
|
||||
f"working tree is dirty; commit, stash or pass --allow-dirty:\n{preview}",
|
||||
data=self.git,
|
||||
)
|
||||
|
||||
detail = f"{branch} @ {commit[:12]} dirty={dirty} version={self.version} ({self.version_source})"
|
||||
if dirty:
|
||||
detail += " (accepted via --allow-dirty)"
|
||||
return Step("repo-state", True, detail, data=self.git)
|
||||
|
||||
def _version_consistency(self) -> Step:
|
||||
problems: list[str] = []
|
||||
data: dict = {"version": self.version, "source": self.version_source}
|
||||
|
||||
attr = version_mod.dynamic_version_attr(self.repo_root)
|
||||
data["dynamic_attr"] = attr
|
||||
if attr is None:
|
||||
problems.append(
|
||||
"pyproject.toml no longer declares tool.setuptools.dynamic.version.attr"
|
||||
)
|
||||
|
||||
if (self.repo_root / "pyproject.toml").is_file():
|
||||
resolved = get_version(self.repo_root)
|
||||
data["resolved_version"] = resolved
|
||||
if resolved != self.version:
|
||||
problems.append(
|
||||
f"src/orisys/__init__.py __version__ is {resolved!r} but the release "
|
||||
f"version is {self.version!r}"
|
||||
)
|
||||
|
||||
sdk_dirs = version_mod.sdk_versions(self.repo_root)
|
||||
data["sdk_versions"] = sdk_dirs
|
||||
for wheel in version_mod.sdk_wheels(self.repo_root, self.version):
|
||||
found = version_mod.wheel_version(wheel)
|
||||
if found != self.version:
|
||||
problems.append(f"{wheel.name} carries version {found!r}, expected {self.version!r}")
|
||||
|
||||
detail = (
|
||||
f"version={self.version} ({self.version_source}) dynamic_attr={attr}\n"
|
||||
f"SDK/ versions present: {sdk_dirs or 'none'}"
|
||||
)
|
||||
if problems:
|
||||
detail += "\n" + "\n".join(problems)
|
||||
return Step("version-consistency", not problems, detail, data=data)
|
||||
|
||||
def _duplicate_guard(self) -> Step:
|
||||
"""Fail only when this version has already been *released*.
|
||||
|
||||
A populated ``SDK/<version>/`` is the normal input to preflight (it is
|
||||
what gets verified), so it is reported, not blocked. What must not happen
|
||||
silently is re-releasing a version that is already out: that is detected
|
||||
offline through git tags and, opt-in, against PyPI.
|
||||
"""
|
||||
problems: list[str] = []
|
||||
lines: list[str] = []
|
||||
|
||||
existing = version_mod.sdk_wheels(self.repo_root, self.version)
|
||||
if existing:
|
||||
lines.append(
|
||||
f"SDK/{self.version}/ holds {len(existing)} staged wheel(s) to verify: "
|
||||
f"{[w.name for w in existing]}"
|
||||
)
|
||||
else:
|
||||
lines.append(
|
||||
f"SDK/{self.version}/ is empty; nothing is staged yet "
|
||||
"(build with `python build/run.py wheel-win` / `manylinux`)"
|
||||
)
|
||||
|
||||
tags = version_mod.git_tags(self.repo_root)
|
||||
tag_hits = version_mod.released_tags_for(self.version, tags)
|
||||
if tag_hits:
|
||||
message = f"git tag(s) {tag_hits} already mark {self.version} as released"
|
||||
if self.options.force:
|
||||
lines.append(message + " (accepted via --force)")
|
||||
else:
|
||||
problems.append(
|
||||
message + "; pass --force to re-verify or intentionally re-release"
|
||||
)
|
||||
else:
|
||||
lines.append(
|
||||
f"no local git tag marks {self.version} as released "
|
||||
f"({len(tags)} tag(s) checked in this repo and {version_mod.RELEASE_REPO_DIRNAME})"
|
||||
)
|
||||
|
||||
if self.options.check_pypi:
|
||||
versions, error = version_mod.pypi_released_versions()
|
||||
if error:
|
||||
problems.append(f"PyPI lookup failed: {error}")
|
||||
elif versions is None:
|
||||
problems.append("PyPI lookup returned no version list")
|
||||
elif self.version in versions:
|
||||
problems.append(
|
||||
f"orisys {self.version} is already published on PyPI; a PyPI upload "
|
||||
"cannot be undone. Published: " + ", ".join(versions)
|
||||
)
|
||||
else:
|
||||
lines.append(
|
||||
f"PyPI has no {self.version}; published: " + (", ".join(versions) or "none")
|
||||
)
|
||||
else:
|
||||
lines.append("PyPI existence check skipped (pass --check-pypi to enable)")
|
||||
|
||||
detail = "\n".join(lines)
|
||||
if problems:
|
||||
detail += "\n" + "\n".join(problems)
|
||||
return Step("duplicate-guard", not problems, detail)
|
||||
|
||||
def _version_strings(self) -> Step:
|
||||
"""Keep the version out of prose: one source, everything else derived.
|
||||
|
||||
Scans tracked text for version-looking literals that disagree with the
|
||||
release version. Dependency constraints are ignored by design; deliberate
|
||||
exceptions live in ``version.VERSION_STRING_ALLOWLIST``.
|
||||
"""
|
||||
findings = version_mod.find_version_literals(self.repo_root, self.version)
|
||||
if not findings:
|
||||
scanned = len(version_mod.tracked_text_files(self.repo_root))
|
||||
return Step(
|
||||
"version-strings",
|
||||
True,
|
||||
f"no stale version literal in {scanned} scanned file(s); "
|
||||
f"version {self.version} lives only where it is derived from",
|
||||
)
|
||||
|
||||
by_file: dict[str, list[str]] = {}
|
||||
for path, lineno, literal in findings:
|
||||
by_file.setdefault(path, []).append(f"{path}:{lineno} -> {literal}")
|
||||
detail = (
|
||||
f"{len(findings)} version literal(s) disagree with {self.version}. "
|
||||
"Update them, or make the file version-agnostic, or add a documented "
|
||||
"entry to build/release/version.py::VERSION_STRING_ALLOWLIST:\n"
|
||||
+ "\n".join(line for lines in by_file.values() for line in lines)
|
||||
)
|
||||
return Step("version-strings", False, detail, data={"count": len(findings)})
|
||||
|
||||
def _tests(self) -> Step:
|
||||
if self.options.skip_tests:
|
||||
return Step(
|
||||
"tests",
|
||||
True,
|
||||
"WARNING: test gate SKIPPED via --skip-tests; this run cannot certify a release",
|
||||
data={"status": "skipped"},
|
||||
)
|
||||
|
||||
marker = "not hardware" if self.options.longrun else "not slow and not hardware"
|
||||
self.report_dir.mkdir(parents=True, exist_ok=True)
|
||||
junit = self.report_dir / "junit.xml"
|
||||
command = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pytest",
|
||||
"-q",
|
||||
"-m",
|
||||
marker,
|
||||
f"--junitxml={junit}",
|
||||
]
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
command, cwd=self.repo_root, capture_output=True, text=True, check=False
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
return Step("tests", False, f"cannot run pytest: {exc}")
|
||||
|
||||
ok = completed.returncode == 0
|
||||
tail = ((completed.stdout or "") + (completed.stderr or "")).strip().splitlines()[-12:]
|
||||
detail = f"pytest -m {marker!r} in {sys.executable}\nexit code {completed.returncode}"
|
||||
if tail:
|
||||
detail += "\n" + "\n".join(tail)
|
||||
return Step("tests", ok, detail, data={"marker": marker, "junit": str(junit)})
|
||||
|
||||
def _bundle_precheck(self) -> Step:
|
||||
if not (self.repo_root / "pyproject.toml").is_file():
|
||||
return Step("bundle-precheck", False, "pyproject.toml not found")
|
||||
|
||||
items = bundle_items(self.repo_root)
|
||||
problems: list[str] = []
|
||||
counts: dict[str, int] = {}
|
||||
for item in items:
|
||||
files = resolve_bundle_source_files(self.repo_root, item)
|
||||
counts[str(item.get("name"))] = len(files)
|
||||
if not files:
|
||||
problems.append(
|
||||
f"bundle item {item.get('name')!r} resolved 0 files; its patterns match nothing"
|
||||
)
|
||||
|
||||
try:
|
||||
verify_required_assets(self.repo_root, items)
|
||||
except FileNotFoundError as exc:
|
||||
problems.append(str(exc))
|
||||
|
||||
detail = "resolved files per bundle item: " + json.dumps(counts)
|
||||
if problems:
|
||||
detail += "\n" + "\n".join(problems)
|
||||
return Step("bundle-precheck", not problems, detail, data={"counts": counts})
|
||||
|
||||
def _discover_wheels(self) -> str | None:
|
||||
"""Populate self.wheels; return an error message when nothing was found."""
|
||||
directory = (
|
||||
Path(self.options.wheel_dir)
|
||||
if self.options.wheel_dir
|
||||
else self.repo_root / "SDK" / self.version
|
||||
)
|
||||
self.wheel_dir = directory
|
||||
if not directory.is_dir():
|
||||
return (
|
||||
f"{directory} does not exist. Build first "
|
||||
"(`python build/run.py wheel`) or pass --wheel-dir."
|
||||
)
|
||||
|
||||
self.wheels = sorted(directory.glob("*.whl"))
|
||||
if not self.wheels:
|
||||
return f"no *.whl found in {directory}. Build first or pass --wheel-dir."
|
||||
return None
|
||||
|
||||
def _wheel_inspection(self) -> Step:
|
||||
problem = self._discover_wheels()
|
||||
if problem:
|
||||
return Step("wheel-inspection", False, problem)
|
||||
|
||||
problems: list[str] = []
|
||||
blocks: list[str] = []
|
||||
for wheel in self.wheels:
|
||||
result = inspect_wheel(
|
||||
wheel,
|
||||
expected_version=self.version,
|
||||
repo_root=self.repo_root,
|
||||
strict_bundle=True,
|
||||
)
|
||||
blocks.append(result.render())
|
||||
if not result.ok:
|
||||
problems.append(
|
||||
f"{wheel.name}: " + ", ".join(check.name for check in result.failures)
|
||||
)
|
||||
|
||||
detail = f"{len(self.wheels)} wheel(s) in {self.wheel_dir}\n" + "\n".join(blocks)
|
||||
if problems:
|
||||
detail += "\nfailed checks:\n" + "\n".join(problems)
|
||||
return Step("wheel-inspection", not problems, detail, data={"count": len(self.wheels)})
|
||||
|
||||
def _platform_token(self) -> str:
|
||||
return "win_amd64" if sys.platform == "win32" else platform.machine().lower()
|
||||
|
||||
def _runtime_targets(self) -> list[tuple[str, Path, Path | None]]:
|
||||
token = self._platform_token()
|
||||
if self.options.all_wheel_pythons:
|
||||
targets: list[tuple[str, Path, Path | None]] = []
|
||||
for wheel in self.wheels:
|
||||
tag, target = wheel_target(wheel)
|
||||
executable = None
|
||||
if target == "win_amd64":
|
||||
if sys.platform != "win32":
|
||||
raise RuntimeError("Verify Windows wheels from a Windows host")
|
||||
executable = version_mod.python_for_tag(tag)
|
||||
if target == "win_amd64" and executable is None:
|
||||
raise RuntimeError(
|
||||
f"cannot resolve an interpreter for {tag}; create the conda env "
|
||||
f"py{tag[2:]} or set CONDA_ENVS_ROOT"
|
||||
)
|
||||
targets.append((tag, wheel, executable))
|
||||
return targets
|
||||
|
||||
tag = f"cp{sys.version_info.major}{sys.version_info.minor}"
|
||||
match = next((w for w in self.wheels if wheel_target(w) == (tag, token)), None)
|
||||
if match is None:
|
||||
raise RuntimeError(
|
||||
f"no {tag} {token} wheel among {[w.name for w in self.wheels]}; pass "
|
||||
"--all-wheel-pythons to verify every ABI in its own environment, or run from "
|
||||
"a matching interpreter"
|
||||
)
|
||||
return [(tag, match, None)]
|
||||
|
||||
def _wheel_runtime(self) -> Step:
|
||||
if self.options.no_runtime:
|
||||
return Step(
|
||||
"wheel-runtime",
|
||||
True,
|
||||
"WARNING: runtime verification SKIPPED via --no-runtime",
|
||||
data={"status": "skipped"},
|
||||
)
|
||||
|
||||
video = self.options.video or default_verification_video(self.repo_root)
|
||||
if video is None:
|
||||
return Step(
|
||||
"wheel-runtime",
|
||||
False,
|
||||
"no verification clip found. Pass --video, set ORISYS_VERIFY_VIDEO, or commit a "
|
||||
"clip under data/sample/. The runtime check is never silently skipped.",
|
||||
)
|
||||
video = Path(video)
|
||||
if not video.is_file():
|
||||
return Step("wheel-runtime", False, f"verification clip not found: {video}")
|
||||
|
||||
try:
|
||||
targets = self._runtime_targets()
|
||||
except RuntimeError as exc:
|
||||
return Step("wheel-runtime", False, str(exc))
|
||||
|
||||
problems: list[str] = []
|
||||
lines: list[str] = [f"clip: {video}", f"config: {self.options.config_name}"]
|
||||
for tag, wheel, executable in targets:
|
||||
label = f"{tag} {wheel.name}"
|
||||
try:
|
||||
digest = _sha256_file(wheel)
|
||||
# One venv per wheel: it runs the import smoke test, the bundled
|
||||
# asset check and the frame pipeline together.
|
||||
verifier = verify_wheel
|
||||
extra = {"python_executable": executable, "offline": self.options.offline}
|
||||
if wheel_target(wheel)[1] != "win_amd64":
|
||||
verifier = verify_manylinux_wheel
|
||||
extra = {"repo_root": self.repo_root}
|
||||
record = verifier(
|
||||
wheel,
|
||||
video=video,
|
||||
config_name=self.options.config_name,
|
||||
frames=self.options.frames,
|
||||
check_assets=not self.options.no_assets,
|
||||
**extra,
|
||||
)
|
||||
if _sha256_file(wheel) != digest:
|
||||
raise RuntimeError(f"{wheel.name} changed during runtime verification")
|
||||
except subprocess.CalledProcessError as exc:
|
||||
problems.append(f"{label}: command failed ({exc})")
|
||||
continue
|
||||
except Exception as exc:
|
||||
problems.append(f"{label}: {type(exc).__name__}: {exc}")
|
||||
continue
|
||||
|
||||
lines.append(
|
||||
f"{label}: {record['assets']} | frames={record['frames']} "
|
||||
f"flow={record['flow_shape']}"
|
||||
)
|
||||
self.runtime_records.append({"tag": tag, **record, "sha256": digest})
|
||||
|
||||
detail = "\n".join(lines)
|
||||
if problems:
|
||||
detail += "\nfailures:\n" + "\n".join(problems)
|
||||
return Step("wheel-runtime", not problems, detail, data={"records": self.runtime_records})
|
||||
|
||||
# -- driver ----------------------------------------------------------
|
||||
|
||||
def run(self) -> int:
|
||||
started_at = _now()
|
||||
started = time.perf_counter()
|
||||
print(f"=== orisys release preflight: version {self.version} ({self.version_source}) ===")
|
||||
print(f"repo: {self.repo_root}")
|
||||
print(f"report: {self.report_dir}")
|
||||
print()
|
||||
|
||||
try:
|
||||
for step_fn in (
|
||||
self._repo_state,
|
||||
self._version_consistency,
|
||||
self._version_strings,
|
||||
self._duplicate_guard,
|
||||
self._tests,
|
||||
self._bundle_precheck,
|
||||
self._wheel_inspection,
|
||||
self._wheel_runtime,
|
||||
):
|
||||
self._execute(step_fn)
|
||||
if self.aborted:
|
||||
break
|
||||
finally:
|
||||
self.ok = bool(self.steps) and all(step.ok for step in self.steps) and not self.aborted
|
||||
duration = round(time.perf_counter() - started, 3)
|
||||
payload = {
|
||||
"version": self.version,
|
||||
"version_source": self.version_source,
|
||||
"ok": self.ok,
|
||||
"aborted": self.aborted,
|
||||
"started_at": started_at,
|
||||
"finished_at": _now(),
|
||||
"duration_s": duration,
|
||||
"git": self.git,
|
||||
"interpreter": {
|
||||
"executable": sys.executable,
|
||||
"version": platform.python_version(),
|
||||
"platform": sys.platform,
|
||||
},
|
||||
"options": {key: value for key, value in asdict(self.options).items()
|
||||
if not isinstance(value, Path)},
|
||||
"steps": [asdict(step) for step in self.steps],
|
||||
"wheels": [
|
||||
{
|
||||
"name": wheel.name,
|
||||
"size": wheel.stat().st_size,
|
||||
"sha256": _sha256_file(wheel),
|
||||
}
|
||||
for wheel in self.wheels
|
||||
],
|
||||
}
|
||||
payload["options"]["wheel_dir"] = str(self.options.wheel_dir) if self.options.wheel_dir else None
|
||||
payload["options"]["video"] = str(self.options.video) if self.options.video else None
|
||||
|
||||
json_path, text_path = report_mod.write_report(payload, self.report_dir)
|
||||
print()
|
||||
print(f"report: {json_path}")
|
||||
print(f"report: {text_path}")
|
||||
|
||||
print()
|
||||
print(f"=== preflight {'PASSED' if self.ok else 'FAILED'} ({duration}s) ===")
|
||||
return 0 if self.ok else 1
|
||||
|
||||
|
||||
def run_preflight(options: PreflightOptions, repo_root: Path | None = None) -> int:
|
||||
return Preflight(options, repo_root=repo_root).run()
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Shared publish gate: certify a release from its preflight report.
|
||||
|
||||
Both publish commands (``release github`` and ``release pypi``) refuse to act
|
||||
unless a passing ``release preflight`` report exists for the exact version and
|
||||
every wheel still hashes to what that report recorded. The report is the contract
|
||||
between "we verified this" and "we shipped this"; re-deriving the checks here
|
||||
would just be a second, weaker opinion.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
# Options that weaken the gate. A release must not be certified by a run that
|
||||
# waived any of them: re-run preflight on a clean commit instead.
|
||||
WAIVED_GUARD_KEYS = ("skip_tests", "allow_dirty", "force", "no_runtime", "no_assets")
|
||||
|
||||
|
||||
class PublishGateError(RuntimeError):
|
||||
"""The preflight report does not certify these wheels."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class WheelRecord:
|
||||
name: str
|
||||
sha256: str
|
||||
size: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class GateResult:
|
||||
version: str
|
||||
git_commit: str | None
|
||||
report_path: Path
|
||||
wheels: list[WheelRecord] = field(default_factory=list)
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with Path(path).open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def evaluate_gate(
|
||||
payload: dict,
|
||||
version: str,
|
||||
wheels: dict[str, Path],
|
||||
*,
|
||||
head_commit: str | None = None,
|
||||
) -> tuple[list[str], list[str]]:
|
||||
"""Return ``(problems, warnings)`` for a report payload and the wheels on disk.
|
||||
|
||||
Pure function: no I/O beyond hashing the wheels, so it is cheap to test.
|
||||
"""
|
||||
problems: list[str] = []
|
||||
warnings: list[str] = []
|
||||
|
||||
if str(payload.get("version")) != version:
|
||||
problems.append(
|
||||
f"report is for version {payload.get('version')!r}, not {version!r}"
|
||||
)
|
||||
if not payload.get("ok"):
|
||||
problems.append(
|
||||
"report result is FAIL - run `release preflight --keep-going` and fix "
|
||||
"everything before publishing"
|
||||
)
|
||||
if payload.get("aborted"):
|
||||
problems.append("report stopped early (aborted); it certifies nothing")
|
||||
|
||||
options = payload.get("options") or {}
|
||||
waived = sorted(key for key in WAIVED_GUARD_KEYS if options.get(key))
|
||||
if waived:
|
||||
problems.append(
|
||||
"report waived gate options "
|
||||
+ ", ".join(f"--{key.replace('_', '-')}" for key in waived)
|
||||
+ "; commit the tree and re-run `release preflight` without them"
|
||||
)
|
||||
|
||||
recorded = {item.get("name"): item.get("sha256") for item in payload.get("wheels") or []}
|
||||
if not recorded:
|
||||
problems.append("report lists no wheels")
|
||||
|
||||
runtime_records = {}
|
||||
for step in payload.get("steps") or []:
|
||||
if step.get("name") == "wheel-runtime" and step.get("ok"):
|
||||
runtime_records.update({item.get("wheel"): item.get("sha256")
|
||||
for item in (step.get("data") or {}).get("records", [])})
|
||||
uncovered = sorted(name for name, digest in recorded.items()
|
||||
if runtime_records.get(name) != digest)
|
||||
if uncovered:
|
||||
problems.append(f"wheels lack matching runtime verification: {uncovered}; "
|
||||
"re-run preflight --all-wheel-pythons")
|
||||
|
||||
missing = sorted(set(recorded) - set(wheels))
|
||||
if missing:
|
||||
problems.append(f"wheels in the report are missing from disk: {missing}")
|
||||
extra = sorted(set(wheels) - set(recorded))
|
||||
if extra:
|
||||
problems.append(f"wheels on disk are not covered by the report: {extra}")
|
||||
|
||||
for name in sorted(set(recorded) & set(wheels)):
|
||||
actual = sha256_file(wheels[name])
|
||||
if actual != recorded[name]:
|
||||
problems.append(
|
||||
f"{name} changed after preflight: report {recorded[name][:12]}, "
|
||||
f"disk {actual[:12]} - rebuild and re-run preflight"
|
||||
)
|
||||
|
||||
commit = payload.get("git", {}).get("commit") if isinstance(payload.get("git"), dict) else None
|
||||
if head_commit and commit and commit != head_commit:
|
||||
warnings.append(
|
||||
f"report certifies commit {commit[:12]} but HEAD is {head_commit[:12]}; "
|
||||
"the wheels are unchanged, so this is usually just a later commit"
|
||||
)
|
||||
return problems, warnings
|
||||
|
||||
|
||||
def load_gate(repo_root: Path, version: str) -> GateResult:
|
||||
"""Load and enforce the preflight report for ``version``.
|
||||
|
||||
Raises :class:`PublishGateError` with every problem found, so one run reports
|
||||
all of them instead of stopping at the first.
|
||||
"""
|
||||
repo_root = Path(repo_root)
|
||||
report_path = repo_root / "build" / "release" / "reports" / version / "preflight.json"
|
||||
if not report_path.is_file():
|
||||
raise PublishGateError(
|
||||
f"no preflight report at {report_path}. Run:\n"
|
||||
f" python build/run.py release preflight\n"
|
||||
f"(the publish commands only ship what preflight certified)"
|
||||
)
|
||||
|
||||
try:
|
||||
payload = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise PublishGateError(f"cannot read {report_path}: {exc}") from exc
|
||||
|
||||
wheel_dir = repo_root / "SDK" / version
|
||||
wheels = {path.name: path for path in sorted(wheel_dir.glob("*.whl"))}
|
||||
if not wheels:
|
||||
raise PublishGateError(
|
||||
f"no wheels in {wheel_dir}; build first (`python build/run.py wheel-win` / "
|
||||
"`manylinux`) and re-run preflight"
|
||||
)
|
||||
|
||||
problems, warnings = evaluate_gate(payload, version, wheels)
|
||||
if problems:
|
||||
raise PublishGateError(
|
||||
f"the preflight report at {report_path} does not certify orisys {version}:\n "
|
||||
+ "\n ".join(problems)
|
||||
)
|
||||
|
||||
git_payload = payload.get("git") if isinstance(payload.get("git"), dict) else {}
|
||||
return GateResult(
|
||||
version=version,
|
||||
git_commit=git_payload.get("commit"),
|
||||
report_path=report_path,
|
||||
wheels=[
|
||||
WheelRecord(name=name, sha256=sha256_file(path), size=path.stat().st_size)
|
||||
for name, path in wheels.items()
|
||||
],
|
||||
warnings=warnings,
|
||||
)
|
||||
@@ -0,0 +1,472 @@
|
||||
"""Upload the certified wheels to TestPyPI, then (on confirmation) to PyPI.
|
||||
|
||||
``python build/run.py release pypi`` runs:
|
||||
|
||||
1. certified-wheels gate (same contract as ``release github``)
|
||||
2. ``twine check`` on every wheel - the wheel metadata is generated, so its
|
||||
long description and classifiers are worth validating before an upload that
|
||||
cannot be undone
|
||||
3. refuse if the version already exists on the target index (read-only probe)
|
||||
4. upload to TestPyPI, then verify by fetching it back from the index
|
||||
5. stop there unless ``--confirm-pypi`` is given; then repeat 3-4 against PyPI
|
||||
|
||||
PyPI uploads are permanent: a version number can never be reused, and a bad
|
||||
artifact cannot be replaced. Hence the TestPyPI stage, the read-only existence
|
||||
probe, and ``--apply`` being required even to reach TestPyPI.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import REPO_ROOT # noqa: E402
|
||||
|
||||
from release import auth # noqa: E402
|
||||
from release.publish import GateResult, PublishGateError, load_gate, sha256_file # noqa: E402
|
||||
|
||||
INDEX_JSON = {name: str(meta["json_url"]) for name, meta in auth.INDEXES.items()}
|
||||
INDEX_SIMPLE = {name: str(meta["simple_url"]) for name, meta in auth.INDEXES.items()}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Options:
|
||||
version: str
|
||||
apply: bool = False
|
||||
confirm_pypi: bool = False
|
||||
verify_install: bool = True
|
||||
skip_check: bool = False
|
||||
# Upload straight to PyPI, skipping the TestPyPI rehearsal. Needed when you
|
||||
# only hold a production token, since the two indexes reject each other's.
|
||||
pypi_only: bool = False
|
||||
# Offer to enter a missing token now instead of just failing.
|
||||
prompt_credentials: bool = False
|
||||
# Token supplied non-interactively (e.g. --token-stdin), used once.
|
||||
token: str | None = None
|
||||
|
||||
|
||||
def upload_phases(options: Options) -> list[str]:
|
||||
"""The indexes to upload to, in order."""
|
||||
if options.pypi_only:
|
||||
return ["pypi"]
|
||||
return ["testpypi", "pypi"] if options.confirm_pypi else ["testpypi"]
|
||||
|
||||
|
||||
@dataclass
|
||||
class IndexState:
|
||||
versions: list[str] = field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
@property
|
||||
def known(self) -> bool:
|
||||
return self.error is None
|
||||
|
||||
|
||||
def twine_check_argv(wheels: list[Path]) -> list[str]:
|
||||
return [sys.executable, "-m", "twine", "check", *[str(w) for w in wheels]]
|
||||
|
||||
|
||||
def twine_upload_argv(wheels: list[Path], repository: str | None) -> list[str]:
|
||||
argv = [sys.executable, "-m", "twine", "upload"]
|
||||
if repository:
|
||||
argv += ["--repository", repository]
|
||||
argv += [str(wheel) for wheel in wheels]
|
||||
return argv
|
||||
|
||||
|
||||
def credentials_hint(index: str) -> str | None:
|
||||
"""Guidance when ``index`` has no usable credentials, else ``None``.
|
||||
|
||||
More than "does the file exist": a placeholder or the wrong ``username`` is
|
||||
treated as missing, because twine would only find out at upload time.
|
||||
"""
|
||||
state = auth.credential_state(index)
|
||||
if state.configured:
|
||||
return None
|
||||
lines = [f"no usable credentials for [{index}]: {state.problem}"]
|
||||
lines.append(f" set it up with: python build/run.py release auth --set-token {index}")
|
||||
if index == "testpypi":
|
||||
lines.append(
|
||||
" (TestPyPI needs its own account and token; if you only hold a "
|
||||
"production token, use --pypi-only to skip the rehearsal)"
|
||||
)
|
||||
lines.append(
|
||||
" or set TWINE_USERNAME=__token__ and TWINE_PASSWORD for this session only"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _ensure_credentials(
|
||||
indexes: list[str], options: Options, *, interactive: bool | None = None
|
||||
) -> bool:
|
||||
"""Make sure every index that will be uploaded to has credentials.
|
||||
|
||||
Prompts for a missing token when there is a terminal to prompt on; a piped or
|
||||
CI run fails fast instead of hanging on a hidden-input prompt.
|
||||
"""
|
||||
missing = [index for index in indexes if not auth.credential_state(index).configured]
|
||||
if not missing:
|
||||
for index in indexes:
|
||||
state = auth.credential_state(index)
|
||||
print(f"[ok ] credentials for [{index}] ({state.source})")
|
||||
return True
|
||||
|
||||
if interactive is None:
|
||||
interactive = sys.stdin.isatty()
|
||||
allow_prompt = options.prompt_credentials or (interactive and options.apply)
|
||||
|
||||
if not allow_prompt:
|
||||
for index in missing:
|
||||
print(f"[{'FAIL' if options.apply else 'warn'}] {credentials_hint(index)}")
|
||||
if not options.apply:
|
||||
print(
|
||||
"[info] dry run continues so you can review the plan; "
|
||||
"--apply would stop here until the credentials exist"
|
||||
)
|
||||
return True
|
||||
if not interactive:
|
||||
print(
|
||||
"[info] stdin is not a terminal, so nothing was prompted; run "
|
||||
"`release auth --set-token <index>` or pass --prompt-credentials"
|
||||
)
|
||||
return False
|
||||
|
||||
for index in missing:
|
||||
print(f"[warn] {credentials_hint(index)}")
|
||||
token = options.token or auth.prompt_for_token(index)
|
||||
if not token:
|
||||
print(f"[FAIL] no token entered for [{index}]")
|
||||
return False
|
||||
for problem in auth.validate_token(token):
|
||||
print(f"[FAIL] {problem.removeprefix('ERROR ')}")
|
||||
return False
|
||||
for warning in auth.token_warnings(token):
|
||||
print(f"[warn] {warning.removeprefix('WARN ')}")
|
||||
path = auth.write_token(index, token)
|
||||
print(f"[ok ] wrote [{index}] credentials to {path}")
|
||||
options.token = None # only the first index may consume a piped token
|
||||
return True
|
||||
|
||||
|
||||
def index_state(index: str, *, timeout: float = 20.0) -> IndexState:
|
||||
"""Read the published versions of orisys from an index (read-only)."""
|
||||
url = INDEX_JSON[index]
|
||||
request = urllib.request.Request(
|
||||
url, headers={"User-Agent": "orisys-release-publish", "Accept": "application/json"}
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
payload = json.load(response)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return IndexState(versions=[])
|
||||
return IndexState(error=f"{index} returned HTTP {exc.code}")
|
||||
except Exception as exc: # network, DNS, TLS, JSON
|
||||
return IndexState(error=f"{index} lookup failed: {type(exc).__name__}: {exc}")
|
||||
|
||||
releases = payload.get("releases")
|
||||
if not isinstance(releases, dict):
|
||||
return IndexState(error=f"unexpected {index} response (no 'releases' mapping)")
|
||||
return IndexState(
|
||||
versions=sorted(name for name, files in releases.items() if files)
|
||||
)
|
||||
|
||||
|
||||
def index_files(
|
||||
index: str, version: str, *, timeout: float = 20.0
|
||||
) -> tuple[list[str] | None, str | None]:
|
||||
"""Filenames published for ``version``, as ``(filenames, error)``.
|
||||
|
||||
``(None, None)`` - the index does not serve this version (yet)
|
||||
``(None, "why")`` - the lookup itself failed, which is NOT evidence of absence
|
||||
``([...], None)`` - the version is published with those files
|
||||
|
||||
Keeping those apart matters: reporting a network hiccup as "not published
|
||||
yet" is how a successful upload gets mistaken for a failure.
|
||||
"""
|
||||
url = INDEX_JSON[index]
|
||||
request = urllib.request.Request(
|
||||
url, headers={"User-Agent": "orisys-release-publish", "Accept": "application/json"}
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
payload = json.load(response)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return None, None
|
||||
return None, f"{index} returned HTTP {exc.code}"
|
||||
except Exception as exc:
|
||||
return None, f"{index} lookup failed: {type(exc).__name__}: {exc}"
|
||||
|
||||
files = (payload.get("releases") or {}).get(version)
|
||||
if not isinstance(files, list) or not files:
|
||||
return None, None
|
||||
return sorted(str(item.get("filename", "")) for item in files), None
|
||||
|
||||
|
||||
def wait_for_release(
|
||||
index: str,
|
||||
version: str,
|
||||
*,
|
||||
attempts: int = 6,
|
||||
delay: float = 5.0,
|
||||
sleep=None,
|
||||
timeout: float = 20.0,
|
||||
) -> tuple[list[str] | None, str | None]:
|
||||
"""Poll until the index serves ``version`` (uploads are not instantly visible).
|
||||
|
||||
TestPyPI in particular lags the upload response by seconds. Returns the last
|
||||
``(filenames, error)`` observed.
|
||||
"""
|
||||
sleep = sleep or time.sleep
|
||||
last_error: str | None = None
|
||||
for attempt in range(1, max(1, attempts) + 1):
|
||||
filenames, error = index_files(index, version, timeout=timeout)
|
||||
if filenames is not None:
|
||||
return filenames, None
|
||||
last_error = error
|
||||
if attempt < attempts:
|
||||
print(
|
||||
f"[info] {index} does not serve {version} yet "
|
||||
f"({'lookup failed' if error else 'indexing'}), retry {attempt}/{attempts - 1}",
|
||||
flush=True,
|
||||
)
|
||||
sleep(delay)
|
||||
return None, last_error
|
||||
|
||||
|
||||
def existing_release_state(index: str, version: str, expected: set[str]) -> str:
|
||||
"""``"satisfied"`` | ``"conflict"`` | ``"absent"`` for this index and version.
|
||||
|
||||
An index version can never be replaced, so "already published with exactly
|
||||
these files" is the only successful terminal state; treating it as an error
|
||||
would make a re-run after a partial failure look like a new failure. A
|
||||
different file set is a genuine conflict and must stop the run.
|
||||
"""
|
||||
published, error = index_files(index, version)
|
||||
if published is None:
|
||||
if error:
|
||||
print(f"[warn] could not check {index} for an existing release: {error}")
|
||||
return "absent"
|
||||
if set(published) == expected:
|
||||
print(
|
||||
f"[ok ] {index} already serves orisys {version} with all "
|
||||
f"{len(published)} file(s)"
|
||||
)
|
||||
return "satisfied"
|
||||
extra = sorted(set(published) - expected)
|
||||
missing = sorted(expected - set(published))
|
||||
print(
|
||||
f"[FAIL] {index} already serves orisys {version} with a different file set"
|
||||
+ (f"; unexpected: {extra}" if extra else "")
|
||||
+ (f"; missing: {missing}" if missing else "")
|
||||
)
|
||||
return "conflict"
|
||||
|
||||
|
||||
def _upload(wheels: list[Path], repository: str | None) -> int:
|
||||
argv = twine_upload_argv(wheels, repository)
|
||||
print(f"[run ] {' '.join(str(part) for part in argv)}")
|
||||
completed = subprocess.run(argv, check=False)
|
||||
return completed.returncode
|
||||
|
||||
|
||||
def _fetch_back(index: str, version: str, wheel: Path) -> int:
|
||||
"""Fetch the exact Python/platform wheel and verify its certified bytes."""
|
||||
with tempfile.TemporaryDirectory(prefix="orisys-fetch-") as tmp:
|
||||
target = Path(tmp) / "out"
|
||||
python_tag, abi_tag, platform_tags = wheel.stem.rsplit("-", 3)[1:]
|
||||
argv = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"download",
|
||||
"--no-deps",
|
||||
"--no-cache-dir",
|
||||
"--only-binary=:all:",
|
||||
"--implementation", "cp",
|
||||
"--python-version", python_tag[2:],
|
||||
"--abi", abi_tag,
|
||||
"--index-url",
|
||||
INDEX_SIMPLE[index],
|
||||
"--dest",
|
||||
str(target),
|
||||
f"orisys=={version}",
|
||||
]
|
||||
for platform_tag in platform_tags.split("."):
|
||||
argv += ["--platform", platform_tag]
|
||||
# Index verification must not inherit a fallback index or wheelhouse.
|
||||
env = {key: value for key, value in os.environ.items() if not key.startswith("PIP_")}
|
||||
env["PIP_CONFIG_FILE"] = os.devnull
|
||||
print(f"[run ] {' '.join(argv)}")
|
||||
completed = subprocess.run(argv, check=False, env=env)
|
||||
if completed.returncode != 0:
|
||||
return completed.returncode
|
||||
|
||||
fetched = sorted(target.glob(f"orisys-{version}*"))
|
||||
if not fetched:
|
||||
print(f"[FAIL] pip download produced nothing for orisys=={version}")
|
||||
return 1
|
||||
print(f"[ok ] fetched {[p.name for p in fetched]} from {index}")
|
||||
if {path.name for path in fetched} != {wheel.name}:
|
||||
raise PublishGateError(f"{index} returned a different wheel than {wheel.name}")
|
||||
if sha256_file(fetched[0]) != sha256_file(wheel):
|
||||
raise PublishGateError(f"{index} returned different bytes for {wheel.name}")
|
||||
return 0
|
||||
|
||||
|
||||
def publish(options: Options) -> int:
|
||||
version = options.version
|
||||
mode = "APPLY" if options.apply else "DRY RUN"
|
||||
print(f"=== release pypi: orisys {version} ({mode}) ===")
|
||||
|
||||
# --- 1. gate -------------------------------------------------------
|
||||
try:
|
||||
gate: GateResult = load_gate(REPO_ROOT, version)
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
print(f"[ok ] preflight report: {gate.report_path}")
|
||||
for wheel in gate.wheels:
|
||||
print(f" [ok] {wheel.name} sha256={wheel.sha256[:12]}")
|
||||
for warning in gate.warnings:
|
||||
print(f"[warn] {warning}")
|
||||
|
||||
wheels = [REPO_ROOT / "SDK" / version / record.name for record in gate.wheels]
|
||||
missing = [str(wheel) for wheel in wheels if not wheel.is_file()]
|
||||
if missing:
|
||||
print(f"[FAIL] wheels missing from disk: {missing}")
|
||||
return 1
|
||||
|
||||
# --- 2. credentials + twine check (before anything irreversible) ----
|
||||
phases = upload_phases(options)
|
||||
if options.pypi_only:
|
||||
print(
|
||||
"[warn] --pypi-only: uploading straight to PyPI with no TestPyPI "
|
||||
"rehearsal. An index version can never be reused or replaced."
|
||||
)
|
||||
if not _ensure_credentials(phases, options):
|
||||
return 1
|
||||
|
||||
if not options.skip_check:
|
||||
argv = twine_check_argv(wheels)
|
||||
print(f"[run ] {' '.join(str(part) for part in argv)}")
|
||||
completed = subprocess.run(argv, check=False)
|
||||
if completed.returncode != 0:
|
||||
print("[FAIL] twine check reported problems; fix the wheel metadata first")
|
||||
return 1
|
||||
print("[ok ] twine check passed")
|
||||
|
||||
# --- 3. phase plan -------------------------------------------------
|
||||
expected = {wheel.name for wheel in wheels}
|
||||
remaining: list[str] = []
|
||||
for index in phases:
|
||||
state = existing_release_state(index, version, expected)
|
||||
if state == "conflict":
|
||||
return 1
|
||||
if state == "absent":
|
||||
remaining.append(index)
|
||||
phases = remaining
|
||||
|
||||
print()
|
||||
for index in phases:
|
||||
state = index_state(index)
|
||||
if not state.known:
|
||||
print(f"[FAIL] {state.error}; cannot safely check for duplicate uploads")
|
||||
return 1
|
||||
if version in state.versions:
|
||||
print(
|
||||
f"[FAIL] orisys {version} already exists on {index} "
|
||||
f"(published: {', '.join(state.versions) or 'none'}) - "
|
||||
"an index version cannot be reused"
|
||||
)
|
||||
return 1
|
||||
print(f"[ok ] {index}: {version} not present (published: {', '.join(state.versions) or 'none'})")
|
||||
|
||||
if not phases:
|
||||
print()
|
||||
print("=== release pypi: every requested index already has this release ===")
|
||||
return 0
|
||||
|
||||
print()
|
||||
for index in phases:
|
||||
print(f"[plan] upload {len(wheels)} wheel(s) to {index}")
|
||||
if phases == ["testpypi"]:
|
||||
print(
|
||||
"[plan] stop after TestPyPI. Re-run with "
|
||||
"`--confirm-pypi --apply` to publish to the real PyPI."
|
||||
)
|
||||
print()
|
||||
|
||||
if not options.apply:
|
||||
print("dry run: nothing was uploaded. Re-run with --apply to execute.")
|
||||
return 0
|
||||
|
||||
# --- 4. upload ----------------------------------------------------
|
||||
for index in phases:
|
||||
if _upload(wheels, index) != 0:
|
||||
print(f"[FAIL] twine upload to {index} failed")
|
||||
return 1
|
||||
print(f"[ok ] uploaded to {index}")
|
||||
|
||||
published, error = wait_for_release(index, version)
|
||||
if published is None:
|
||||
# The upload itself succeeded, so this is a confirmation problem, not
|
||||
# a publishing failure. Say so plainly and do not imply data loss.
|
||||
reason = f"lookup kept failing ({error})" if error else "the index has not indexed it yet"
|
||||
print(
|
||||
f"[warn] uploaded, but could not confirm it on {index} yet: {reason}.\n"
|
||||
f" check https://{'test.' if index == 'testpypi' else ''}pypi.org/project/orisys/{version}/"
|
||||
"\n re-running this command is safe: an already-published version is skipped"
|
||||
)
|
||||
print(f"\n=== release pypi: {index} upload SUCCEEDED (verification inconclusive) ===")
|
||||
return 0
|
||||
if set(published) != expected:
|
||||
print(
|
||||
f"[FAIL] {index} file set mismatch:\n expected {sorted(expected)}\n"
|
||||
f" got {sorted(published)}"
|
||||
)
|
||||
return 1
|
||||
print(f"[ok ] {index} serves all {len(published)} file(s) for {version}")
|
||||
|
||||
if options.verify_install:
|
||||
fetch_pending = False
|
||||
for wheel in wheels:
|
||||
try:
|
||||
if _fetch_back(index, version, wheel) != 0:
|
||||
fetch_pending = True
|
||||
print(f"[warn] {wheel.name} is uploaded but download verification is inconclusive")
|
||||
else:
|
||||
print(f"[ok ] fetched and hash-verified {wheel.name} from {index}")
|
||||
except PublishGateError as exc:
|
||||
print(f"[FAIL] {exc}")
|
||||
return 1
|
||||
if fetch_pending:
|
||||
print(f"[warn] {index} upload succeeded but download verification is inconclusive; "
|
||||
"later indexes were not uploaded. Re-run after checking propagation.")
|
||||
return 0
|
||||
|
||||
if phases == ["testpypi"]:
|
||||
print()
|
||||
print("=== release pypi: TestPyPI stage PASSED; PyPI untouched ===")
|
||||
print("next: python build/run.py release pypi --confirm-pypi --apply")
|
||||
return 0
|
||||
|
||||
print()
|
||||
if options.pypi_only:
|
||||
print("=== release pypi PASSED (PyPI only, no TestPyPI rehearsal) ===")
|
||||
else:
|
||||
print("=== release pypi PASSED (TestPyPI + PyPI) ===")
|
||||
return 0
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Structured preflight reports.
|
||||
|
||||
A preflight run writes both a machine-readable ``preflight.json`` (the contract a
|
||||
future publish step will consume and re-verify) and a human ``preflight.txt``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
REPORTS_RELPATH = Path("build") / "release" / "reports"
|
||||
SCHEMA_VERSION = 1
|
||||
|
||||
|
||||
def reports_root(repo_root: Path, override: Path | None = None) -> Path:
|
||||
return Path(override) if override else Path(repo_root) / REPORTS_RELPATH
|
||||
|
||||
|
||||
def render_text(payload: dict) -> str:
|
||||
git = payload.get("git") or {}
|
||||
interpreter = payload.get("interpreter") or {}
|
||||
lines = [
|
||||
f"orisys release preflight :: version {payload.get('version')}",
|
||||
f" result: {'PASS' if payload.get('ok') else 'FAIL'}",
|
||||
f" git: {git.get('branch')} @ {git.get('commit')} (dirty={git.get('dirty')})",
|
||||
f" interpreter: {interpreter.get('version')} at {interpreter.get('executable')}",
|
||||
f" duration: {payload.get('duration_s')}s",
|
||||
"",
|
||||
]
|
||||
|
||||
for step in payload.get("steps", []):
|
||||
flag = "ok " if step.get("ok") else "FAIL"
|
||||
lines.append(f"[{flag}] {step.get('name')} ({step.get('duration_s')}s)")
|
||||
for detail_line in str(step.get("detail") or "").splitlines():
|
||||
lines.append(f" {detail_line}")
|
||||
lines.append("")
|
||||
|
||||
wheels = payload.get("wheels") or []
|
||||
if wheels:
|
||||
lines.append("wheels:")
|
||||
for wheel in wheels:
|
||||
lines.append(
|
||||
f" {wheel.get('name')} {wheel.get('size')} bytes sha256={wheel.get('sha256')}"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
options = payload.get("options") or {}
|
||||
waived = [
|
||||
key
|
||||
for key, value in options.items()
|
||||
if value and key in {"skip_tests", "allow_dirty", "force", "no_runtime", "no_assets"}
|
||||
]
|
||||
if waived:
|
||||
lines.append(f"waived guards: {', '.join(sorted(waived))}")
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
|
||||
|
||||
def write_report(payload: dict, out_dir: Path) -> tuple[Path, Path]:
|
||||
out_dir = Path(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
payload = {**payload, "schema": SCHEMA_VERSION}
|
||||
json_path = out_dir / "preflight.json"
|
||||
text_path = out_dir / "preflight.txt"
|
||||
|
||||
json_path.write_text(json.dumps(payload, indent=2, sort_keys=False), encoding="utf-8")
|
||||
text_path.write_text(render_text(payload), encoding="utf-8")
|
||||
return json_path, text_path
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Run Linux wheel release checks in the matching manylinux Docker image."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from packaging.utils import parse_wheel_filename
|
||||
|
||||
PLATFORM_LABELS = {
|
||||
"win_amd64": "Windows x64",
|
||||
"x86_64": "Linux x86_64",
|
||||
"aarch64": "Linux ARM64",
|
||||
}
|
||||
|
||||
|
||||
def wheel_target(wheel: Path) -> tuple[str, str]:
|
||||
"""Return the CPython tag and supported OS/architecture; reject ambiguity."""
|
||||
_, _, _, tags = parse_wheel_filename(wheel.name)
|
||||
identities = set()
|
||||
for tag in tags:
|
||||
if (
|
||||
tag.interpreter not in {"cp310", "cp311", "cp312"}
|
||||
or tag.abi != tag.interpreter
|
||||
):
|
||||
raise RuntimeError(f"Unsupported Python ABI in {wheel.name}")
|
||||
if tag.platform == "win_amd64":
|
||||
target = "win_amd64"
|
||||
elif tag.platform.startswith("manylinux") and tag.platform.endswith("_aarch64"):
|
||||
target = "aarch64"
|
||||
elif tag.platform.startswith("manylinux") and tag.platform.endswith("_x86_64"):
|
||||
target = "x86_64"
|
||||
else:
|
||||
raise RuntimeError(f"Unsupported release platform in {wheel.name}")
|
||||
identities.add((tag.interpreter, target))
|
||||
if len(identities) != 1:
|
||||
raise RuntimeError(f"Ambiguous release platform in {wheel.name}")
|
||||
return identities.pop()
|
||||
|
||||
|
||||
_CONTAINER_CHECK = """
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, '/src/build')
|
||||
from common.verify.wheel import verify_wheel
|
||||
record = verify_wheel(Path(sys.argv[1]), video=Path(sys.argv[2]),
|
||||
config_name=sys.argv[3], frames=int(sys.argv[4]), check_assets=sys.argv[5] == '1')
|
||||
print('ORISYS_RUNTIME_RESULT=' + json.dumps(record))
|
||||
"""
|
||||
|
||||
|
||||
def verify_manylinux_wheel(
|
||||
wheel: Path,
|
||||
*,
|
||||
repo_root: Path,
|
||||
video: Path,
|
||||
config_name: str,
|
||||
frames: int,
|
||||
check_assets: bool = True,
|
||||
) -> dict:
|
||||
tag, architecture = wheel_target(wheel)
|
||||
if architecture == "win_amd64":
|
||||
raise RuntimeError("Windows wheels must be verified with a Windows interpreter")
|
||||
platform = "linux/arm64" if architecture == "aarch64" else "linux/amd64"
|
||||
image = (
|
||||
"orisys-manylinux-aarch64" if architecture == "aarch64" else "orisys-manylinux"
|
||||
)
|
||||
abi = f"{tag}-{tag}"
|
||||
argv = [
|
||||
"docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--pull=never",
|
||||
"--platform",
|
||||
platform,
|
||||
"-v",
|
||||
f"{repo_root.resolve()}:/src:ro",
|
||||
"-v",
|
||||
f"{wheel.resolve().parent}:/wheels:ro",
|
||||
"-v",
|
||||
f"{video.resolve().parent}:/video:ro",
|
||||
"-e",
|
||||
"PIP_NO_INDEX=1",
|
||||
"-e",
|
||||
f"PIP_FIND_LINKS=/opt/orisys-wheelhouse/{abi}",
|
||||
"--entrypoint",
|
||||
f"/opt/python/{abi}/bin/python",
|
||||
image,
|
||||
"-c",
|
||||
_CONTAINER_CHECK,
|
||||
f"/wheels/{wheel.name}",
|
||||
f"/video/{video.name}",
|
||||
config_name,
|
||||
str(frames),
|
||||
"1" if check_assets else "0",
|
||||
]
|
||||
print(f" verifying {wheel.name} in {image} ({platform}) ...", flush=True)
|
||||
result = subprocess.run(argv, capture_output=True, text=True, check=False)
|
||||
if result.returncode:
|
||||
raise RuntimeError(
|
||||
f"{image} verification failed:\n{result.stdout}\n{result.stderr}"
|
||||
)
|
||||
marker = "ORISYS_RUNTIME_RESULT="
|
||||
records = [
|
||||
line[len(marker) :]
|
||||
for line in result.stdout.splitlines()
|
||||
if line.startswith(marker)
|
||||
]
|
||||
if len(records) != 1:
|
||||
raise RuntimeError(
|
||||
f"{image} returned no unique verification record:\n{result.stdout}"
|
||||
)
|
||||
record = json.loads(records[0])
|
||||
if record.get("wheel") != wheel.name or record.get("frames") != frames:
|
||||
raise RuntimeError(f"Unexpected verification record for {wheel.name}: {record}")
|
||||
return record
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Version consistency and duplicate-release guards for the release gate."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
PYPI_JSON_URL = "https://pypi.org/pypi/orisys/json"
|
||||
|
||||
# Sibling repository that carries the public GitHub releases and their tags.
|
||||
RELEASE_REPO_DIRNAME = "orisys-sdk-release"
|
||||
|
||||
# orisys-<version>-<pytag>-<abitag>-<platformtag>.whl
|
||||
_WHEEL_NAME_RE = re.compile(r"^orisys-(?P<version>[^-]+)-[^-]+-[^-]+-.+\.whl$")
|
||||
|
||||
|
||||
def wheel_version(wheel_path: Path) -> str | None:
|
||||
match = _WHEEL_NAME_RE.match(Path(wheel_path).name)
|
||||
return match.group("version") if match else None
|
||||
|
||||
|
||||
def sdk_versions(repo_root: Path) -> list[str]:
|
||||
"""Version directory names present under ``SDK/``, in ascending order.
|
||||
|
||||
Only digit-leading names count as versions, so stray directories (``tmp``,
|
||||
``latest``) never appear as releases.
|
||||
"""
|
||||
sdk_root = Path(repo_root) / "SDK"
|
||||
if not sdk_root.is_dir():
|
||||
return []
|
||||
|
||||
return sorted(
|
||||
(
|
||||
entry.name
|
||||
for entry in sdk_root.iterdir()
|
||||
if entry.is_dir() and entry.name[:1].isdigit()
|
||||
),
|
||||
key=_version_sort_key,
|
||||
)
|
||||
|
||||
|
||||
def sdk_wheels(repo_root: Path, version: str) -> list[Path]:
|
||||
"""Wheels already published into ``SDK/<version>/``."""
|
||||
version_dir = Path(repo_root) / "SDK" / version
|
||||
if not version_dir.is_dir():
|
||||
return []
|
||||
return sorted(version_dir.glob("*.whl"))
|
||||
|
||||
|
||||
def dynamic_version_attr(repo_root: Path) -> str | None:
|
||||
"""Return pyproject's configured dynamic version attr (e.g. orisys.__version__)."""
|
||||
from common.pyproject_utils import load_pyproject
|
||||
|
||||
config = (
|
||||
load_pyproject(Path(repo_root))
|
||||
.get("tool", {})
|
||||
.get("setuptools", {})
|
||||
.get("dynamic", {})
|
||||
.get("version")
|
||||
)
|
||||
if isinstance(config, dict):
|
||||
attr = config.get("attr")
|
||||
if isinstance(attr, str):
|
||||
return attr
|
||||
return None
|
||||
|
||||
|
||||
def pypi_released_versions(timeout: float = 20.0) -> tuple[list[str] | None, str | None]:
|
||||
"""Read the published version list from PyPI.
|
||||
|
||||
Returns ``(versions, error)``; ``versions`` is ``None`` when the lookup
|
||||
failed. This is the only network call in the release gate and it is
|
||||
read-only, so it is safe to run before an irreversible upload.
|
||||
"""
|
||||
request = urllib.request.Request(
|
||||
PYPI_JSON_URL,
|
||||
headers={"User-Agent": "orisys-release-preflight", "Accept": "application/json"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
payload = json.load(response)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return [], None
|
||||
return None, f"PyPI returned HTTP {exc.code}"
|
||||
except Exception as exc: # network, DNS, TLS, JSON
|
||||
return None, f"{type(exc).__name__}: {exc}"
|
||||
|
||||
releases = payload.get("releases")
|
||||
if not isinstance(releases, dict):
|
||||
return None, "unexpected PyPI response shape (no 'releases' mapping)"
|
||||
listed = [
|
||||
str(version)
|
||||
for version, files in releases.items()
|
||||
if isinstance(files, list) and files
|
||||
]
|
||||
return sorted(listed, key=_version_sort_key), None
|
||||
|
||||
|
||||
def _version_sort_key(name: str):
|
||||
parts = re.findall(r"\d+", name)
|
||||
return ([int(part) for part in parts] or [0], name)
|
||||
|
||||
|
||||
def git_tags(repo_root: Path) -> list[str]:
|
||||
"""Local git tags from this repo and the sibling release repo.
|
||||
|
||||
Tags are the only offline evidence that a version was already released
|
||||
(``v0.5.1`` in ``orisys-sdk-release`` marks a published GitHub release), so
|
||||
the duplicate guard works without network access.
|
||||
"""
|
||||
roots = [Path(repo_root)]
|
||||
sibling = Path(repo_root).parent / RELEASE_REPO_DIRNAME
|
||||
if sibling.is_dir():
|
||||
roots.append(sibling)
|
||||
|
||||
tags: set[str] = set()
|
||||
for root in roots:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
["git", "-C", str(root), "tag", "--list"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
except OSError:
|
||||
continue
|
||||
if completed.returncode != 0:
|
||||
continue
|
||||
tags.update(
|
||||
line.strip() for line in (completed.stdout or "").splitlines() if line.strip()
|
||||
)
|
||||
return sorted(tags)
|
||||
|
||||
|
||||
def released_tags_for(version: str, tags: list[str]) -> list[str]:
|
||||
"""Tag spellings that mark ``version`` as already released."""
|
||||
return sorted({version, f"v{version}"} & set(tags))
|
||||
|
||||
|
||||
# Version literals that are deliberately not the package version. Keyed by
|
||||
# repo-relative path; each entry documents *why* the literal may stay.
|
||||
VERSION_STRING_ALLOWLIST: dict[str, frozenset[str]] = {
|
||||
# Historical release records: rewriting these would destroy the audit trail.
|
||||
"docs/publishing.md": frozenset({"0.4.2", "0.4.3", "0.4.4", "0.5.0", "0.5.1", "0.5.2"}),
|
||||
# Unrelated third-party/CUDA versions quoted in prose.
|
||||
"docs/get_started.md": frozenset({"12.9.0"}),
|
||||
}
|
||||
|
||||
# Paths never scanned: tests deliberately use synthetic versions, reports are
|
||||
# generated output, and this module's own allowlist/sentinel literals would
|
||||
# otherwise flag themselves.
|
||||
VERSION_SCAN_EXCLUDED_PREFIXES = (
|
||||
"tests/",
|
||||
"build/release/reports/",
|
||||
"build/nuitka/",
|
||||
"build/lib/",
|
||||
"dist/",
|
||||
"SDK/",
|
||||
"data/",
|
||||
)
|
||||
VERSION_SCAN_EXCLUDED_FILES = ("build/release/version.py",)
|
||||
|
||||
_VERSION_LITERAL_RE = re.compile(
|
||||
r"(?<![<>=!~\w.])v?(\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.]+)?)(?![0-9A-Za-z.])"
|
||||
)
|
||||
_SCANNED_SUFFIXES = (".md", ".py", ".txt", ".toml", ".cfg", ".ini", ".rst")
|
||||
|
||||
|
||||
def tracked_text_files(repo_root: Path) -> list[Path]:
|
||||
"""Tracked files worth scanning for version literals."""
|
||||
repo_root = Path(repo_root)
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
["git", "-C", str(repo_root), "ls-files", "-z"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
names = [n for n in (completed.stdout or "").split("\0") if n]
|
||||
except OSError:
|
||||
names = []
|
||||
|
||||
if not names:
|
||||
names = [
|
||||
str(path.relative_to(repo_root)).replace("\\", "/")
|
||||
for path in repo_root.rglob("*")
|
||||
if path.is_file()
|
||||
]
|
||||
|
||||
selected = []
|
||||
for name in names:
|
||||
if name.startswith(VERSION_SCAN_EXCLUDED_PREFIXES):
|
||||
continue
|
||||
if name in VERSION_SCAN_EXCLUDED_FILES:
|
||||
continue
|
||||
if not name.endswith(_SCANNED_SUFFIXES):
|
||||
continue
|
||||
selected.append(Path(name))
|
||||
return selected
|
||||
|
||||
|
||||
def find_version_literals(
|
||||
repo_root: Path, version: str
|
||||
) -> list[tuple[str, int, str]]:
|
||||
"""``(path, lineno, literal)`` for every version-looking string != ``version``.
|
||||
|
||||
Dependency constraints are skipped by the regex lookbehind (``numpy>=1.24.4``
|
||||
is not a version reference), and historical/third-party literals require an
|
||||
explicit entry in :data:`VERSION_STRING_ALLOWLIST`.
|
||||
"""
|
||||
repo_root = Path(repo_root)
|
||||
findings: list[tuple[str, int, str]] = []
|
||||
for relative in tracked_text_files(repo_root):
|
||||
absolute = repo_root / relative
|
||||
try:
|
||||
text = absolute.read_text(encoding="utf-8", errors="ignore")
|
||||
except OSError:
|
||||
continue
|
||||
if "\0" in text:
|
||||
continue
|
||||
allowed = VERSION_STRING_ALLOWLIST.get(relative.as_posix(), frozenset())
|
||||
for lineno, line in enumerate(text.splitlines(), start=1):
|
||||
for match in _VERSION_LITERAL_RE.finditer(line):
|
||||
literal = match.group(1)
|
||||
if literal == version or literal in allowed:
|
||||
continue
|
||||
findings.append((relative.as_posix(), lineno, literal))
|
||||
return findings
|
||||
|
||||
|
||||
def conda_envs_root() -> Path | None:
|
||||
"""Locate the conda envs directory the same way build/wheel/win.bat does."""
|
||||
override = os.getenv("CONDA_ENVS_ROOT")
|
||||
if override:
|
||||
path = Path(override)
|
||||
return path if path.is_dir() else None
|
||||
|
||||
home = Path.home()
|
||||
candidates = (
|
||||
home / "anaconda3" / "envs",
|
||||
home / "miniconda3" / "envs",
|
||||
Path(os.getenv("LOCALAPPDATA", "")) / "anaconda3" / "envs",
|
||||
Path(os.getenv("LOCALAPPDATA", "")) / "miniconda3" / "envs",
|
||||
)
|
||||
for candidate in candidates:
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def python_for_tag(tag: str, envs_root: Path | None = None) -> Path | None:
|
||||
"""Resolve an interpreter for a wheel tag like ``cp310`` (env ``py310``)."""
|
||||
digits = tag[2:] if tag.startswith("cp") else tag
|
||||
env_name = f"py{digits}"
|
||||
root = envs_root or conda_envs_root()
|
||||
if root is None:
|
||||
return None
|
||||
|
||||
for relative in (
|
||||
Path(env_name) / "python.exe",
|
||||
Path(env_name) / "bin" / "python",
|
||||
):
|
||||
candidate = root / relative
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
return None
|
||||
+118
@@ -0,0 +1,118 @@
|
||||
"""Unified Orisys packaging entrypoint.
|
||||
|
||||
Usage (from repo root):
|
||||
|
||||
python build/run.py wheel [--target internal|pypi-linux] [--verify]
|
||||
python build/run.py wheel-win
|
||||
python build/run.py wheel-manylinux [PowerShell args...]
|
||||
python build/run.py gui finger|mini|box
|
||||
python build/run.py verify [wheel.whl]
|
||||
python build/run.py verify --gui-dist [--product ory_finger]
|
||||
python build/run.py release preflight [--version V]
|
||||
python build/run.py release audit <wheel.whl>
|
||||
|
||||
See build/README.md for details.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parent
|
||||
_REPO_ROOT = _BUILD_ROOT.parent
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
|
||||
def _run_python(script: Path, argv: list[str]) -> int:
|
||||
cmd = [sys.executable, str(script), *argv]
|
||||
return subprocess.call(cmd, cwd=_REPO_ROOT)
|
||||
|
||||
|
||||
def _cmd_wheel(argv: list[str]) -> int:
|
||||
return _run_python(_BUILD_ROOT / "wheel" / "build.py", argv)
|
||||
|
||||
|
||||
def _cmd_wheel_win(argv: list[str]) -> int:
|
||||
bat = _BUILD_ROOT / "wheel" / "win.bat"
|
||||
return subprocess.call(["cmd", "/c", str(bat), *argv], cwd=_REPO_ROOT)
|
||||
|
||||
|
||||
def _cmd_manylinux(argv: list[str]) -> int:
|
||||
ps1 = _BUILD_ROOT / "wheel" / "docker" / "run.ps1"
|
||||
cmd = [
|
||||
"powershell",
|
||||
"-ExecutionPolicy",
|
||||
"Bypass",
|
||||
"-File",
|
||||
str(ps1),
|
||||
*argv,
|
||||
]
|
||||
return subprocess.call(cmd, cwd=_REPO_ROOT)
|
||||
|
||||
|
||||
def _cmd_gui(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog="build/run.py gui")
|
||||
parser.add_argument(
|
||||
"product",
|
||||
choices=("finger", "mini", "box", "ory_finger", "ory_mini", "ory_box"),
|
||||
help="GUI product to package",
|
||||
)
|
||||
args, rest = parser.parse_known_args(argv)
|
||||
product = {
|
||||
"finger": "ory_finger",
|
||||
"mini": "ory_mini",
|
||||
"box": "ory_box",
|
||||
}.get(args.product, args.product)
|
||||
return _run_python(_BUILD_ROOT / "gui" / product / "build.py", rest)
|
||||
|
||||
|
||||
def _cmd_verify(argv: list[str]) -> int:
|
||||
from common.verify.cli import main as verify_main
|
||||
|
||||
return verify_main(argv)
|
||||
|
||||
|
||||
def _cmd_release(argv: list[str]) -> int:
|
||||
from release.cli import main as release_main
|
||||
|
||||
return release_main(argv)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
argv = list(sys.argv[1:] if argv is None else argv)
|
||||
if not argv or argv[0] in ("-h", "--help"):
|
||||
print(__doc__)
|
||||
print("Commands: wheel | wheel-win | wheel-manylinux | gui | verify | release")
|
||||
print("Examples:")
|
||||
print(" python build/run.py wheel --help")
|
||||
print(" python build/run.py gui finger")
|
||||
print(" python build/run.py wheel-manylinux -Architecture aarch64 -PythonVersions 3.10")
|
||||
print(" python build/run.py release preflight --version <version>")
|
||||
print(" python build/run.py release audit SDK\\<version>\\orisys-<version>-cp312-cp312-win_amd64.whl")
|
||||
return 0
|
||||
|
||||
command, rest = argv[0], argv[1:]
|
||||
dispatch = {
|
||||
"wheel": _cmd_wheel,
|
||||
"wheel-win": _cmd_wheel_win,
|
||||
"wheel-manylinux": _cmd_manylinux,
|
||||
"gui": _cmd_gui,
|
||||
"verify": _cmd_verify,
|
||||
"release": _cmd_release,
|
||||
}
|
||||
if command not in dispatch:
|
||||
print(f"Unknown command: {command!r}", file=sys.stderr)
|
||||
print(
|
||||
"Choose from: wheel, wheel-win, wheel-manylinux, gui, verify, release",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
return dispatch[command](rest)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,82 +0,0 @@
|
||||
"""Post-build smoke tests for packaged wheels and standalone GUI builds."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import venv
|
||||
from pathlib import Path
|
||||
|
||||
from paths import GUI_DIST_DIR
|
||||
|
||||
|
||||
def _venv_python(venv_dir: Path) -> Path:
|
||||
if sys.platform == "win32":
|
||||
return venv_dir / "Scripts" / "python.exe"
|
||||
return venv_dir / "bin" / "python"
|
||||
|
||||
|
||||
def verify_built_wheel(wheel_path: Path) -> None:
|
||||
"""Install the wheel in a temporary venv and smoke-test finger imports."""
|
||||
wheel_path = wheel_path.resolve()
|
||||
with tempfile.TemporaryDirectory(prefix="orisys-wheel-verify-") as tmp:
|
||||
venv_dir = Path(tmp) / "venv"
|
||||
venv.create(venv_dir, with_pip=True)
|
||||
python = _venv_python(venv_dir)
|
||||
|
||||
subprocess.check_call(
|
||||
[python, "-m", "pip", "install", "-q", str(wheel_path)],
|
||||
)
|
||||
subprocess.check_call(
|
||||
[
|
||||
python,
|
||||
"-c",
|
||||
(
|
||||
"from orisys.finger import UVMeshViewer, build_uv_surface_mesh, FingerRuntime; "
|
||||
"from orisys.finger.mesh import UVSurfaceMesh; "
|
||||
"from orisys.finger.viewer import init_uv_mesh_viewer; "
|
||||
"print('finger mesh/viewer imports OK')"
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def verify_gui_dist(dist_dir: Path | None = None) -> None:
|
||||
"""Smoke-test a standalone GUI folder produced by build_gui_win.py."""
|
||||
dist_dir = (dist_dir or GUI_DIST_DIR).resolve()
|
||||
exe = dist_dir / "OrisysDashboard.exe"
|
||||
if not exe.is_file():
|
||||
raise FileNotFoundError(f"GUI executable not found: {exe}")
|
||||
|
||||
required = [
|
||||
dist_dir / "config" / "dd02-ov.json",
|
||||
dist_dir / "assets" / "finger" / "visualization" / "world_to_cam.npy",
|
||||
dist_dir / "assets" / "finger" / "30fps" / "mesh.npz",
|
||||
]
|
||||
missing = [str(path.relative_to(dist_dir)) for path in required if not path.is_file()]
|
||||
if missing:
|
||||
raise FileNotFoundError(f"Missing bundled files in {dist_dir}: {', '.join(missing)}")
|
||||
|
||||
subprocess.check_call([str(exe), "--help"], cwd=dist_dir)
|
||||
print(f"GUI dist smoke test OK: {exe}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Verify packaged Orisys artifacts.")
|
||||
parser.add_argument("--gui-dist", action="store_true", help="Verify dist/OrisysDashboard/")
|
||||
parser.add_argument("--dist-dir", type=Path, default=None, help="Override GUI dist folder")
|
||||
parser.add_argument("wheel", nargs="?", type=Path, help="Wheel path for import smoke test")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.gui_dist:
|
||||
verify_gui_dist(args.dist_dir)
|
||||
return
|
||||
if args.wheel is None:
|
||||
parser.error("Provide a wheel path or pass --gui-dist")
|
||||
verify_built_wheel(args.wheel)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
"""Nuitka-compiled SDK wheel packaging."""
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Build a Nuitka-compiled orisys wheel and publish it to SDK/{version}/.
|
||||
|
||||
Pipeline:
|
||||
1. Compile src/orisys → extension module (.pyd / .so)
|
||||
2. Stage wheel tree (compiled module + bundled files from pyproject.toml)
|
||||
3. Run bdist_wheel; on Linux, auditwheel-repair to a manylinux tag
|
||||
4. Smoke-test imports in a temporary venv (required for --target pypi-linux)
|
||||
|
||||
Targets:
|
||||
internal – local/SDK delivery (default). Linux still repairs with auditwheel.
|
||||
pypi-linux – PyPI binary wheel; requires Linux + CPython 3.10/3.11/3.12 + compliant tag.
|
||||
|
||||
Run from repo root:
|
||||
python build/run.py wheel
|
||||
python build/run.py wheel --target pypi-linux
|
||||
python build/wheel/build.py --target pypi-linux
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import (
|
||||
NUITKA_COMPILE_DIR,
|
||||
NUITKA_WHEEL_DIR,
|
||||
REPO_ROOT,
|
||||
repaired_wheel_dir,
|
||||
repo_path,
|
||||
)
|
||||
from common.pyproject_utils import (
|
||||
bundle_items,
|
||||
extras_require,
|
||||
get_version,
|
||||
install_requires,
|
||||
project_metadata,
|
||||
)
|
||||
from common.verify.wheel import verify_built_wheel
|
||||
|
||||
from wheel.compile import compile_orisys, copy_compiled_orisys, install_nuitka
|
||||
from wheel.layout import (
|
||||
copy_bundle_items,
|
||||
copy_orisys_blend,
|
||||
package_data_from_bundle,
|
||||
)
|
||||
from wheel.package import (
|
||||
is_compliant_manylinux_wheel,
|
||||
publish_wheel_to_sdk,
|
||||
read_long_description,
|
||||
repair_linux_wheel,
|
||||
run_bdist_wheel,
|
||||
write_wheel_setup,
|
||||
)
|
||||
|
||||
VALID_TARGETS = ("internal", "pypi-linux")
|
||||
# ABI tags that manylinux images ship under /opt/python/cpXYY-cpXYY.
|
||||
PYPI_LINUX_PYTHON_VERSIONS = ((3, 10), (3, 11), (3, 12))
|
||||
|
||||
|
||||
def _reset_dir(path: Path) -> None:
|
||||
"""Remove and recreate a build output directory."""
|
||||
if path.exists():
|
||||
shutil.rmtree(path)
|
||||
path.mkdir(parents=True)
|
||||
|
||||
|
||||
def _validate_pypi_linux_environment() -> None:
|
||||
if sys.platform != "linux":
|
||||
raise SystemExit(
|
||||
"[ERROR] --target pypi-linux must run inside a Linux manylinux container "
|
||||
"(manylinux_2_28_x86_64 or manylinux_2_28_aarch64 via Docker Desktop)."
|
||||
)
|
||||
if sys.version_info[:2] not in PYPI_LINUX_PYTHON_VERSIONS:
|
||||
supported = ", ".join(f"{major}.{minor}" for major, minor in PYPI_LINUX_PYTHON_VERSIONS)
|
||||
raise SystemExit(
|
||||
f"[ERROR] --target pypi-linux requires CPython {supported}, got "
|
||||
f"{sys.version_info.major}.{sys.version_info.minor}. "
|
||||
"Use /opt/python/cp310-cp310 (or cp311/cp312) in the manylinux_2_28 image."
|
||||
)
|
||||
|
||||
|
||||
def build(*, target: str = "internal", verify: bool | None = None) -> Path:
|
||||
if target not in VALID_TARGETS:
|
||||
raise SystemExit(f"[ERROR] Unknown target {target!r}; choose from {VALID_TARGETS}")
|
||||
|
||||
if target == "pypi-linux":
|
||||
_validate_pypi_linux_environment()
|
||||
|
||||
# pypi-linux always verifies; internal opts in with --verify.
|
||||
if verify is None:
|
||||
verify = target == "pypi-linux"
|
||||
|
||||
# --- 1. Compile ---
|
||||
install_nuitka()
|
||||
|
||||
version = get_version(REPO_ROOT)
|
||||
_reset_dir(NUITKA_COMPILE_DIR)
|
||||
_reset_dir(NUITKA_WHEEL_DIR)
|
||||
|
||||
print(f"\n=== Compiling with Nuitka (target={target}, version={version}) ===")
|
||||
env = os.environ.copy()
|
||||
src_dir = repo_path("src")
|
||||
env["PYTHONPATH"] = src_dir.as_posix() + os.pathsep + env.get("PYTHONPATH", "")
|
||||
ext_file = compile_orisys(NUITKA_COMPILE_DIR, env)
|
||||
|
||||
# --- 2. Stage wheel contents ---
|
||||
print("\n=== Preparing Wheel Structure ===")
|
||||
pkg_dir = NUITKA_WHEEL_DIR / "wheel_root" / "orisys"
|
||||
pkg_dir.mkdir(parents=True)
|
||||
|
||||
ext_suffix = copy_compiled_orisys(pkg_dir, ext_file)
|
||||
|
||||
wheel_root = NUITKA_WHEEL_DIR / "wheel_root"
|
||||
blend_count = copy_orisys_blend(REPO_ROOT, wheel_root)
|
||||
print(f"Staged orisys_blend Python modules: {blend_count}")
|
||||
|
||||
# Bundle rules live in pyproject.toml → [[tool.orisys.bundle.items]]
|
||||
bundle = bundle_items(REPO_ROOT)
|
||||
copied_counts = copy_bundle_items(REPO_ROOT, pkg_dir, bundle)
|
||||
for item in bundle:
|
||||
destination = f"orisys/{item['target']}/"
|
||||
print(
|
||||
f"Copied {copied_counts.get(item['name'], 0)} file(s) "
|
||||
f"for bundle item {item['name']!r} to {destination}"
|
||||
)
|
||||
|
||||
write_wheel_setup(
|
||||
NUITKA_WHEEL_DIR,
|
||||
version=version,
|
||||
package_data=package_data_from_bundle(ext_suffix, bundle),
|
||||
install_requires=install_requires(REPO_ROOT),
|
||||
extras_require=extras_require(REPO_ROOT),
|
||||
project=project_metadata(REPO_ROOT),
|
||||
long_description=read_long_description(REPO_ROOT),
|
||||
)
|
||||
|
||||
# --- 3. Build wheel, repair on Linux, publish to SDK/ ---
|
||||
wheel_path = run_bdist_wheel(NUITKA_WHEEL_DIR)
|
||||
if wheel_path is None:
|
||||
raise SystemExit("\n[ERROR] Wheel not found after bdist_wheel.")
|
||||
|
||||
repair_dir = repaired_wheel_dir()
|
||||
final_candidate = repair_linux_wheel(wheel_path, repair_dir)
|
||||
|
||||
if target == "pypi-linux" and not is_compliant_manylinux_wheel(final_candidate):
|
||||
raise SystemExit(
|
||||
f"[ERROR] Wheel is not PyPI-compliant: {final_candidate.name}. "
|
||||
"Expected a supported manylinux x86_64 or aarch64 platform tag."
|
||||
)
|
||||
|
||||
# Keep a durable copy under dist/repaired for upload tooling.
|
||||
dist_repaired = REPO_ROOT / "dist" / "repaired"
|
||||
dist_repaired.mkdir(parents=True, exist_ok=True)
|
||||
dist_copy = dist_repaired / final_candidate.name
|
||||
if dist_copy.resolve() != final_candidate.resolve():
|
||||
shutil.copy2(final_candidate, dist_copy)
|
||||
|
||||
final_wheel = publish_wheel_to_sdk(final_candidate, version)
|
||||
print(f"\n[SUCCESS] Wheel created and copied to: {final_wheel}")
|
||||
print(f"[SUCCESS] Upload candidate also at: {dist_copy}")
|
||||
|
||||
# --- 4. Verify (isolated venv; does not touch the active environment) ---
|
||||
if verify:
|
||||
print("\n=== Verifying wheel imports in isolated venv ===")
|
||||
try:
|
||||
verify_built_wheel(final_wheel)
|
||||
print("[SUCCESS] Wheel smoke test passed.")
|
||||
except subprocess.CalledProcessError as exc:
|
||||
if target == "pypi-linux":
|
||||
raise SystemExit(
|
||||
f"[ERROR] Wheel verify failed (required for pypi-linux): {exc}"
|
||||
) from exc
|
||||
print(f"\n[WARNING] Wheel verify failed: {exc}")
|
||||
print("The wheel was built successfully; verification runs in an isolated temp venv.")
|
||||
else:
|
||||
print("\n[INFO] Skipping wheel verify (pass --verify to enable).")
|
||||
|
||||
return final_wheel
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> None:
|
||||
parser = argparse.ArgumentParser(description="Build a Nuitka-compiled orisys wheel.")
|
||||
parser.add_argument(
|
||||
"--target",
|
||||
choices=VALID_TARGETS,
|
||||
default="internal",
|
||||
help="internal: SDK delivery; pypi-linux: manylinux PyPI binary wheel",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verify",
|
||||
action="store_true",
|
||||
help="Run isolated venv smoke test (always on for --target pypi-linux)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--skip-verify",
|
||||
action="store_true",
|
||||
help="Skip smoke test when using --target internal",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
verify: bool | None
|
||||
if args.target == "pypi-linux" or args.verify:
|
||||
verify = True
|
||||
elif args.skip_verify:
|
||||
verify = False
|
||||
else:
|
||||
verify = None
|
||||
build(target=args.target, verify=verify)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -8,21 +8,44 @@ import sys
|
||||
from importlib.machinery import EXTENSION_SUFFIXES
|
||||
from pathlib import Path
|
||||
|
||||
from paths import repo_path
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import repo_path
|
||||
|
||||
NUITKA_INCLUDE_PACKAGES = (
|
||||
"orisys",
|
||||
"orisys.capture",
|
||||
"orisys.finger",
|
||||
"orisys.stitch",
|
||||
"orisys.stitch.backends",
|
||||
"orisys.unroll",
|
||||
"orisys.hhd",
|
||||
"orisys.hhd.solvers",
|
||||
)
|
||||
|
||||
NUITKA_NOFOLLOW_IMPORTS = (
|
||||
# Keep as runtime deps (shipped via wheel install_requires / extras).
|
||||
"numpy",
|
||||
"scipy",
|
||||
"cv2",
|
||||
"numba",
|
||||
"orisys_blend",
|
||||
# Optional accelerators / GUIs: imported behind try/except, resolved from the
|
||||
# user's environment at runtime. Listing them here is not optional - without
|
||||
# it, Nuitka follows whatever happens to be installed in the *build*
|
||||
# environment, which blows up compile time and makes wheel contents depend on
|
||||
# ambient packages. tests/test_nuitka_nofollow.py enforces full coverage.
|
||||
"cupy",
|
||||
"cupyx",
|
||||
"pyopencl",
|
||||
"pyvkfft",
|
||||
"open3d",
|
||||
"PyQt5",
|
||||
"pyqtgraph",
|
||||
"torch",
|
||||
"torchvision",
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# manylinux_2_28: modern glibc baseline with binary wheels for numpy/scipy/opencv.
|
||||
# Prefer this over manylinux2014 — recent NumPy requires GCC >= 10.3 and will try
|
||||
# (and fail) to build from source on the older image.
|
||||
ARG MANYLINUX_ARCH=x86_64
|
||||
FROM quay.io/pypa/manylinux_2_28_${MANYLINUX_ARCH}:latest
|
||||
|
||||
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
PIP_ROOT_USER_ACTION=ignore \
|
||||
ORISYS_WHEELHOUSE_ROOT=/opt/orisys-wheelhouse
|
||||
|
||||
# Per-ABI build tooling plus a per-ABI wheelhouse of every package the release
|
||||
# flow installs at build time:
|
||||
#
|
||||
# * build tooling - nuitka/auditwheel etc. baked in, so the entrypoint does
|
||||
# not have to re-resolve them from PyPI on every container start
|
||||
# (ORISYS_UPDATE_BUILD_TOOLS=1 opts back in to that).
|
||||
# * wheelhouse - `build/wheel/build.py --target pypi-linux` force-enables
|
||||
# verification, which creates a throwaway venv and pip-installs the built
|
||||
# wheel plus its declared runtime deps. Runtime deps are deliberately NOT
|
||||
# installed into the image interpreter (users get them from the wheel
|
||||
# metadata), so without a wheelhouse every build re-downloaded
|
||||
# numpy/scipy/opencv/numba — roughly 200 MB, three times over, once per ABI.
|
||||
# The entrypoint points pip at the wheelhouse with
|
||||
# PIP_NO_INDEX/PIP_FIND_LINKS, which keeps the venv isolated and still
|
||||
# exercises real dependency resolution, just without the network.
|
||||
#
|
||||
# --only-binary=:all: makes a missing wheel a loud image-build failure instead of
|
||||
# a slow source build inside the container.
|
||||
RUN set -eux; \
|
||||
for TAG in cp310-cp310 cp311-cp311 cp312-cp312; do \
|
||||
PYBIN="/opt/python/${TAG}/bin"; \
|
||||
echo "=== build tooling for ${TAG} ==="; \
|
||||
"${PYBIN}/python" -m pip install --upgrade pip setuptools wheel; \
|
||||
"${PYBIN}/python" -m pip install \
|
||||
"nuitka>=2.0" \
|
||||
ordered-set \
|
||||
zstandard \
|
||||
auditwheel; \
|
||||
echo "=== wheelhouse for ${TAG} ==="; \
|
||||
"${PYBIN}/python" -m pip download \
|
||||
--only-binary=:all: \
|
||||
--dest "${ORISYS_WHEELHOUSE_ROOT}/${TAG}" \
|
||||
pip \
|
||||
setuptools \
|
||||
wheel \
|
||||
auditwheel \
|
||||
numpy \
|
||||
scipy \
|
||||
opencv-python \
|
||||
numba; \
|
||||
done; \
|
||||
du -sh "${ORISYS_WHEELHOUSE_ROOT}"/*
|
||||
|
||||
WORKDIR /src
|
||||
ENTRYPOINT ["/bin/bash", "/src/build/wheel/docker/entrypoint.sh"]
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs inside the selected manylinux_2_28 image with the repo mounted at /src.
|
||||
# Builds one Nuitka wheel per CPython ABI listed in ORISYS_PYTHON_TAGS.
|
||||
set -euo pipefail
|
||||
|
||||
TARGET="${ORISYS_BUILD_TARGET:-pypi-linux}"
|
||||
# Comma-separated manylinux tags, e.g. "cp310-cp310,cp311-cp311,cp312-cp312"
|
||||
PYTHON_TAGS="${ORISYS_PYTHON_TAGS:-cp310-cp310,cp311-cp311,cp312-cp312}"
|
||||
WHEELHOUSE_ROOT="${ORISYS_WHEELHOUSE_ROOT:-/opt/orisys-wheelhouse}"
|
||||
# The image already bakes in pip/setuptools/wheel/nuitka/auditwheel. Re-resolving
|
||||
# them from PyPI on every container start costs a network round-trip per ABI and
|
||||
# lets the running container drift from the image it was built from.
|
||||
UPDATE_BUILD_TOOLS="${ORISYS_UPDATE_BUILD_TOOLS:-0}"
|
||||
|
||||
cd /src
|
||||
|
||||
echo "=== manylinux entrypoint ==="
|
||||
echo "Target: ${TARGET}"
|
||||
echo "Python tags: ${PYTHON_TAGS}"
|
||||
echo "Wheelhouse root: ${WHEELHOUSE_ROOT}"
|
||||
uname -a
|
||||
|
||||
IFS=',' read -r -a TAG_ARRAY <<< "${PYTHON_TAGS}"
|
||||
FAILED=0
|
||||
BUILT=()
|
||||
|
||||
for tag in "${TAG_ARRAY[@]}"; do
|
||||
tag="$(echo "${tag}" | tr -d '[:space:]')"
|
||||
if [[ -z "${tag}" ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
PYBIN="/opt/python/${tag}/bin"
|
||||
if [[ ! -x "${PYBIN}/python" ]]; then
|
||||
echo "[ERROR] Python not found for tag ${tag}: ${PYBIN}/python" >&2
|
||||
FAILED=1
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo "Building orisys wheel: ${tag}"
|
||||
echo "Python: $("${PYBIN}/python" -V)"
|
||||
echo "========================================"
|
||||
|
||||
export PATH="${PYBIN}:${PATH}"
|
||||
export PYTHONPATH="/src/src${PYTHONPATH:+:${PYTHONPATH}}"
|
||||
|
||||
# Point pip at the per-ABI wheelhouse baked into the image. The pypi-linux
|
||||
# target force-enables verification, which creates a throwaway venv and
|
||||
# pip-installs the built wheel plus its declared runtime deps; serving those
|
||||
# from the wheelhouse keeps the venv isolated (no --system-site-packages) while
|
||||
# removing a ~200 MB download per ABI. PIP_* is inherited by every pip
|
||||
# subprocess, including that venv.
|
||||
wheelhouse="${WHEELHOUSE_ROOT}/${tag}"
|
||||
if [[ -d "${wheelhouse}" ]]; then
|
||||
export PIP_NO_INDEX=1
|
||||
export PIP_FIND_LINKS="${wheelhouse}"
|
||||
echo "[info] pip is offline, installing from ${wheelhouse}"
|
||||
else
|
||||
echo "[warn] no wheelhouse at ${wheelhouse}; pip will use the network." >&2
|
||||
echo "[warn] rebuild the image without -SkipImageBuild to bake it in." >&2
|
||||
unset PIP_NO_INDEX PIP_FIND_LINKS || true
|
||||
fi
|
||||
|
||||
if [[ "${UPDATE_BUILD_TOOLS}" == "1" ]]; then
|
||||
echo "[info] refreshing build tooling (ORISYS_UPDATE_BUILD_TOOLS=1)"
|
||||
"${PYBIN}/python" -m pip install -q --upgrade pip setuptools wheel auditwheel
|
||||
fi
|
||||
|
||||
if "${PYBIN}/python" build/wheel/build.py --target "${TARGET}"; then
|
||||
BUILT+=("${tag}")
|
||||
else
|
||||
echo "[ERROR] Build failed for ${tag}" >&2
|
||||
FAILED=1
|
||||
fi
|
||||
|
||||
unset PIP_NO_INDEX PIP_FIND_LINKS || true
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=== Build artifacts ==="
|
||||
ls -la /src/dist/repaired/ || true
|
||||
ls -la /src/SDK/*/ || true
|
||||
|
||||
if [[ ${#BUILT[@]} -gt 0 ]]; then
|
||||
echo "Built tags: ${BUILT[*]}"
|
||||
fi
|
||||
|
||||
if [[ "${FAILED}" -ne 0 ]]; then
|
||||
echo "[ERROR] One or more Python ABI builds failed." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "[SUCCESS] All requested manylinux wheels built."
|
||||
@@ -0,0 +1,119 @@
|
||||
# Build x86_64 or aarch64 manylinux wheels using Docker Desktop.
|
||||
#
|
||||
# Usage (from repo root, PowerShell):
|
||||
# .\build\wheel\docker\run.ps1
|
||||
# .\build\wheel\docker\run.ps1 -Architecture aarch64 -PythonVersions 3.10
|
||||
#
|
||||
# From cmd.exe:
|
||||
# powershell -ExecutionPolicy Bypass -File .\build\wheel\docker\run.ps1
|
||||
|
||||
param(
|
||||
[ValidateSet("pypi-linux", "internal")]
|
||||
[string]$Target = "pypi-linux",
|
||||
[string]$PythonVersions = "3.10,3.11,3.12",
|
||||
[ValidateSet("x86_64", "aarch64")]
|
||||
[string]$Architecture = "x86_64",
|
||||
[switch]$SkipImageBuild,
|
||||
[string]$ImageName = ""
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$Architecture = $Architecture.ToLowerInvariant()
|
||||
|
||||
function Convert-ToManylinuxTag {
|
||||
param([string]$Version)
|
||||
$v = $Version.Trim()
|
||||
switch ($v) {
|
||||
"3.10" { return "cp310-cp310" }
|
||||
"3.11" { return "cp311-cp311" }
|
||||
"3.12" { return "cp312-cp312" }
|
||||
"cp310-cp310" { return "cp310-cp310" }
|
||||
"cp311-cp311" { return "cp311-cp311" }
|
||||
"cp312-cp312" { return "cp312-cp312" }
|
||||
default {
|
||||
throw "Unsupported Python version '$Version'. Use 3.10, 3.11, and/or 3.12."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
$RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..\..")).Path
|
||||
Set-Location $RepoRoot
|
||||
|
||||
$tags = @()
|
||||
foreach ($part in ($PythonVersions -split ",")) {
|
||||
if ([string]::IsNullOrWhiteSpace($part)) { continue }
|
||||
$tags += (Convert-ToManylinuxTag $part)
|
||||
}
|
||||
if ($tags.Count -eq 0) {
|
||||
throw "No Python versions specified."
|
||||
}
|
||||
$pythonTags = [string]::Join(",", $tags)
|
||||
$dockerArchitecture = if ($Architecture -eq "aarch64") { "arm64" } else { "amd64" }
|
||||
$dockerPlatform = "linux/$dockerArchitecture"
|
||||
if ([string]::IsNullOrWhiteSpace($ImageName)) {
|
||||
$ImageName = if ($Architecture -eq "aarch64") { "orisys-manylinux-aarch64" } else { "orisys-manylinux" }
|
||||
}
|
||||
|
||||
Write-Host "=== Orisys manylinux Docker build ==="
|
||||
Write-Host "Repo: $RepoRoot"
|
||||
Write-Host "Target: $Target"
|
||||
Write-Host "Image: $ImageName"
|
||||
Write-Host "Architecture: $Architecture ($dockerPlatform)"
|
||||
Write-Host "Python tags: $pythonTags"
|
||||
|
||||
# Docker Desktop must be in Linux container mode for manylinux.
|
||||
$dockerInfo = docker version --format "{{.Server.Os}}" 2>$null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "Docker is not available. Start Docker Desktop and retry."
|
||||
}
|
||||
if ($dockerInfo -ne "linux") {
|
||||
throw "Docker server OS is '$dockerInfo'. Switch Docker Desktop to Linux containers."
|
||||
}
|
||||
|
||||
$entrypoint = Join-Path $RepoRoot "build\wheel\docker\entrypoint.sh"
|
||||
# Normalize CRLF -> LF so bash in the container can execute the script.
|
||||
$raw = [System.IO.File]::ReadAllText($entrypoint)
|
||||
$normalized = $raw -replace "`r`n", "`n" -replace "`r", "`n"
|
||||
[System.IO.File]::WriteAllText($entrypoint, $normalized)
|
||||
|
||||
$dockerfile = "build/wheel/docker/Dockerfile.manylinux"
|
||||
|
||||
if (-not $SkipImageBuild) {
|
||||
Write-Host "`n=== Building Docker image ==="
|
||||
docker build --platform $dockerPlatform --build-arg "MANYLINUX_ARCH=$Architecture" -f $dockerfile -t $ImageName .
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "docker build failed"
|
||||
}
|
||||
}
|
||||
|
||||
# A custom or reused image must contain the requested architecture.
|
||||
$imageArchitecture = docker image inspect --format "{{.Architecture}}" $ImageName
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "Cannot inspect image '$ImageName'. Rebuild without -SkipImageBuild."
|
||||
}
|
||||
if ($imageArchitecture -ne $dockerArchitecture) {
|
||||
throw "Image '$ImageName' architecture is '$imageArchitecture'; expected '$dockerArchitecture'. Rebuild without -SkipImageBuild."
|
||||
}
|
||||
|
||||
Write-Host "`n=== Running manylinux container ==="
|
||||
# Mount the repo so artifacts land under dist/repaired and SDK/<version>/ on the host.
|
||||
docker run --rm `
|
||||
--platform $dockerPlatform `
|
||||
-e "ORISYS_BUILD_TARGET=$Target" `
|
||||
-e "ORISYS_PYTHON_TAGS=$pythonTags" `
|
||||
-v "${RepoRoot}:/src" `
|
||||
$ImageName
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "manylinux container build failed"
|
||||
}
|
||||
|
||||
Write-Host "`n=== Host artifacts ==="
|
||||
$repaired = Join-Path $RepoRoot "dist\repaired"
|
||||
if (Test-Path $repaired) {
|
||||
Get-ChildItem $repaired -Filter *.whl | ForEach-Object { Write-Host $_.FullName }
|
||||
} else {
|
||||
Write-Warning "No dist/repaired directory found."
|
||||
}
|
||||
|
||||
Write-Host "`nNext steps: see docs/publishing.md (TestPyPI then PyPI)."
|
||||
@@ -5,6 +5,21 @@ from __future__ import annotations
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
# Required files that must ship with the wheel for ORY-FINGER runtime/GUI.
|
||||
# Accept both the flat layout and a nested assets/finger/ checkout tree.
|
||||
REQUIRED_ASSET_RELPATHS = (
|
||||
"finger/assets/30fps-0703/mesh.npz",
|
||||
"finger/assets/30fps-0703/camera_intrinsics.npz",
|
||||
"finger/assets/30fps-0703/world_to_cam.npy",
|
||||
"finger/assets/visualization/world_to_cam.npy",
|
||||
)
|
||||
REQUIRED_ASSET_RELPATH_ALTERNATES = {
|
||||
"finger/assets/30fps-0703/mesh.npz": "finger/assets/finger/30fps-0703/mesh.npz",
|
||||
"finger/assets/30fps-0703/camera_intrinsics.npz": "finger/assets/finger/30fps-0703/camera_intrinsics.npz",
|
||||
"finger/assets/30fps-0703/world_to_cam.npy": "finger/assets/finger/30fps-0703/world_to_cam.npy",
|
||||
"finger/assets/visualization/world_to_cam.npy": "finger/assets/finger/visualization/world_to_cam.npy",
|
||||
}
|
||||
|
||||
|
||||
def resolve_bundle_source_files(repo_root: Path, item: dict) -> list[Path]:
|
||||
"""Resolve explicit files and/or glob patterns for one bundle item."""
|
||||
@@ -37,6 +52,36 @@ def resolve_bundle_source_files(repo_root: Path, item: dict) -> list[Path]:
|
||||
return resolved
|
||||
|
||||
|
||||
def verify_required_assets(repo_root: Path, items: list[dict]) -> None:
|
||||
"""Fail early when required finger assets are missing from the bundle."""
|
||||
asset_items = [item for item in items if item.get("name") == "sensors"]
|
||||
if not asset_items:
|
||||
return
|
||||
|
||||
resolved: set[str] = set()
|
||||
for item in asset_items:
|
||||
src_dir = (repo_root / item["source"]).resolve()
|
||||
for path in resolve_bundle_source_files(repo_root, item):
|
||||
try:
|
||||
rel = path.resolve().relative_to(src_dir).as_posix()
|
||||
except ValueError:
|
||||
continue
|
||||
resolved.add(rel)
|
||||
|
||||
missing = []
|
||||
for rel in REQUIRED_ASSET_RELPATHS:
|
||||
alt = REQUIRED_ASSET_RELPATH_ALTERNATES.get(rel)
|
||||
if rel not in resolved and (alt is None or alt not in resolved):
|
||||
missing.append(rel)
|
||||
if missing:
|
||||
joined = ", ".join(missing)
|
||||
raise FileNotFoundError(
|
||||
"Required finger assets are missing from the wheel bundle. "
|
||||
f"Missing: {joined}. Update [[tool.orisys.bundle.items]] patterns "
|
||||
"or restore assets under sensors/finger/assets/ before building."
|
||||
)
|
||||
|
||||
|
||||
def package_data_paths_for_item(item: dict) -> list[str]:
|
||||
"""Return setuptools package_data paths relative to the orisys package."""
|
||||
target = str(item["target"]).replace("\\", "/")
|
||||
@@ -54,6 +99,7 @@ def data_file_entries_for_item(item: dict) -> list[str]:
|
||||
|
||||
|
||||
def copy_bundle_items(repo_root: Path, pkg_dir: Path, items: list[dict]) -> dict[str, int]:
|
||||
verify_required_assets(repo_root, items)
|
||||
copied_counts: dict[str, int] = {}
|
||||
for item in items:
|
||||
name = item["name"]
|
||||
@@ -82,10 +128,23 @@ def copy_bundle_items(repo_root: Path, pkg_dir: Path, items: list[dict]) -> dict
|
||||
def package_data_from_bundle(ext_suffix: str, items: list[dict]) -> dict[str, list[str]]:
|
||||
package_data: dict[str, list[str]] = {
|
||||
"orisys": [f"*{ext_suffix}"],
|
||||
"orisys.examples": ["*.py", "README.md", "app_paths.py", "viz/**/*.py", "pipeline/**/*.py", "gui/**/*.py"],
|
||||
"orisys.examples": ["*.py", "README.md", "viz/**/*.py", "pipeline/**/*.py", "gui/**/*.py"],
|
||||
"orisys_blend": ["*.py"],
|
||||
}
|
||||
for item in items:
|
||||
if item["name"] == "examples":
|
||||
continue
|
||||
package_data["orisys"].extend(package_data_paths_for_item(item))
|
||||
return package_data
|
||||
|
||||
|
||||
def copy_orisys_blend(repo_root: Path, wheel_root: Path) -> int:
|
||||
"""Copy pure-Python orisys_blend (Numba kernels) next to the Nuitka orisys .pyd."""
|
||||
src = repo_root / "src" / "orisys_blend"
|
||||
dst = wheel_root / "orisys_blend"
|
||||
if dst.exists():
|
||||
shutil.rmtree(dst)
|
||||
shutil.copytree(src, dst)
|
||||
count = sum(1 for p in dst.rglob("*.py") if p.is_file())
|
||||
print(f"Copied orisys_blend ({count} .py files) to {dst}")
|
||||
return count
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Assemble and build the Nuitka wheel with setuptools."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BUILD_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_BUILD_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_BUILD_ROOT))
|
||||
|
||||
from common.paths import sdk_dir
|
||||
|
||||
# PyPI rejects bare tags like "manylinux_x86_64"; require a glibc ABI version.
|
||||
_MANYLINUX_TAG_RE = re.compile(
|
||||
r"(?:manylinux2014|manylinux_2_(?:17|24|27|28|31|34|35))_(?:x86_64|aarch64)(?=\.|$)"
|
||||
)
|
||||
|
||||
# Source of the PyPI project description (see read_long_description).
|
||||
PYPI_README_RELPATH = Path("docs") / "pypi_readme.md"
|
||||
|
||||
|
||||
def _license_text(project: dict) -> str | None:
|
||||
"""Extract free-text license from either pyproject license form."""
|
||||
declared = project.get("license")
|
||||
if isinstance(declared, str):
|
||||
return declared
|
||||
if isinstance(declared, dict):
|
||||
text = declared.get("text")
|
||||
if isinstance(text, str):
|
||||
return text
|
||||
return None
|
||||
|
||||
|
||||
def _person_strings(entries) -> list[str]:
|
||||
"""Render PEP 621 author/maintainer tables as ``Name <email>`` strings."""
|
||||
rendered: list[str] = []
|
||||
for entry in entries or []:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
name = (entry.get("name") or "").strip()
|
||||
email = (entry.get("email") or "").strip()
|
||||
if name and email:
|
||||
rendered.append(f"{name} <{email}>")
|
||||
elif name or email:
|
||||
rendered.append(name or email)
|
||||
return rendered
|
||||
|
||||
|
||||
def metadata_kwargs(project: dict) -> dict:
|
||||
"""Translate a pyproject ``[project]`` table into ``setup()`` keywords.
|
||||
|
||||
The Nuitka wheel is built by a generated ``setup.py`` outside the normal
|
||||
setuptools pipeline, so every PyPI-visible field has to be passed through
|
||||
explicitly. Omitting them is what left the early PyPI releases with no
|
||||
``Requires-Python``, classifiers, project URLs or license on the project page.
|
||||
"""
|
||||
authors = _person_strings(project.get("authors"))
|
||||
maintainers = _person_strings(project.get("maintainers"))
|
||||
candidates = {
|
||||
"description": project.get("description"),
|
||||
"python_requires": project.get("requires-python"),
|
||||
"classifiers": list(project.get("classifiers") or []),
|
||||
"project_urls": dict(project.get("urls") or {}),
|
||||
"license": _license_text(project),
|
||||
"author": authors[0] if authors else None,
|
||||
"author_email": None,
|
||||
"maintainer": maintainers[0] if maintainers else None,
|
||||
}
|
||||
return {key: value for key, value in candidates.items() if value}
|
||||
|
||||
|
||||
def read_long_description(repo_root: Path) -> str | None:
|
||||
"""Return the PyPI long description, or ``None`` when it is missing.
|
||||
|
||||
Deliberately not the repository ``README.md``: that file documents the
|
||||
closed-source build/publishing layout, which should not end up on the PyPI
|
||||
project page.
|
||||
"""
|
||||
path = Path(repo_root) / PYPI_README_RELPATH
|
||||
if not path.is_file():
|
||||
print(
|
||||
f"[WARN] {PYPI_README_RELPATH.as_posix()} not found; the wheel will be "
|
||||
"published without a long description."
|
||||
)
|
||||
return None
|
||||
return path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def write_wheel_setup(
|
||||
wheel_dir: Path,
|
||||
*,
|
||||
version: str,
|
||||
package_data: dict[str, list[str]],
|
||||
install_requires: list[str],
|
||||
extras_require: dict[str, list[str]],
|
||||
project: dict | None = None,
|
||||
long_description: str | None = None,
|
||||
) -> Path:
|
||||
metadata_lines = ""
|
||||
if project:
|
||||
for key, value in sorted(metadata_kwargs(project).items()):
|
||||
metadata_lines += f" {key}={value!r},\n"
|
||||
if long_description:
|
||||
metadata_lines += " long_description=_LONG_DESCRIPTION,\n"
|
||||
metadata_lines += ' long_description_content_type="text/markdown",\n'
|
||||
|
||||
setup_py_path = wheel_dir / "setup.py"
|
||||
setup_content = f"""
|
||||
from setuptools import setup, Distribution
|
||||
|
||||
class BinaryDistribution(Distribution):
|
||||
def has_ext_modules(self):
|
||||
return True
|
||||
|
||||
{'_LONG_DESCRIPTION = ' + repr(long_description) if long_description else ''}
|
||||
|
||||
setup(
|
||||
name="orisys",
|
||||
version="{version}",
|
||||
{metadata_lines} packages=['orisys', 'orisys.examples', 'orisys_blend'],
|
||||
package_dir={{'': 'wheel_root'}},
|
||||
package_data={package_data},
|
||||
include_package_data=True,
|
||||
install_requires={install_requires!r},
|
||||
extras_require={extras_require!r},
|
||||
distclass=BinaryDistribution,
|
||||
zip_safe=False,
|
||||
)
|
||||
"""
|
||||
setup_py_path.write_text(setup_content, encoding="utf-8")
|
||||
return setup_py_path
|
||||
|
||||
|
||||
def run_bdist_wheel(wheel_dir: Path) -> Path | None:
|
||||
print("\n=== Building Wheel ===")
|
||||
subprocess.check_call([sys.executable, "setup.py", "bdist_wheel"], cwd=wheel_dir)
|
||||
wheels = sorted((wheel_dir / "dist").glob("*.whl"))
|
||||
return wheels[0] if wheels else None
|
||||
|
||||
|
||||
def _ensure_auditwheel() -> None:
|
||||
try:
|
||||
subprocess.check_call(
|
||||
[sys.executable, "-m", "auditwheel", "--version"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
except (subprocess.CalledProcessError, FileNotFoundError):
|
||||
print("Installing auditwheel...")
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "auditwheel", "-q"])
|
||||
|
||||
|
||||
def is_compliant_manylinux_wheel(wheel_path: Path) -> bool:
|
||||
"""True if the wheel filename uses a supported manylinux x86_64 or aarch64 tag."""
|
||||
return bool(_MANYLINUX_TAG_RE.search(wheel_path.name))
|
||||
|
||||
|
||||
def repair_linux_wheel(wheel_path: Path, repair_dir: Path) -> Path:
|
||||
"""Run auditwheel show/repair and return a PyPI-compliant manylinux wheel.
|
||||
|
||||
Raises RuntimeError if repair fails or the result still lacks a valid tag.
|
||||
On non-Linux platforms the input wheel is returned unchanged.
|
||||
"""
|
||||
if sys.platform != "linux":
|
||||
print("Skipping auditwheel repair (not running on Linux).")
|
||||
return wheel_path
|
||||
|
||||
_ensure_auditwheel()
|
||||
repair_dir = repair_dir.resolve()
|
||||
if repair_dir.exists():
|
||||
shutil.rmtree(repair_dir)
|
||||
repair_dir.mkdir(parents=True)
|
||||
|
||||
print("\n=== auditwheel show ===")
|
||||
subprocess.check_call([sys.executable, "-m", "auditwheel", "show", str(wheel_path)])
|
||||
|
||||
print("\n=== auditwheel repair ===")
|
||||
subprocess.check_call(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"auditwheel",
|
||||
"repair",
|
||||
str(wheel_path),
|
||||
"--wheel-dir",
|
||||
str(repair_dir),
|
||||
]
|
||||
)
|
||||
|
||||
repaired = sorted(repair_dir.glob("*.whl"))
|
||||
if not repaired:
|
||||
raise RuntimeError(f"auditwheel repair produced no wheels in {repair_dir}")
|
||||
|
||||
compliant = [path for path in repaired if is_compliant_manylinux_wheel(path)]
|
||||
if not compliant:
|
||||
names = ", ".join(path.name for path in repaired)
|
||||
raise RuntimeError(
|
||||
"auditwheel repair did not produce a PyPI-compliant manylinux tag. "
|
||||
f"Got: {names}. Rebuild inside the manylinux_2_28 image for the "
|
||||
"target architecture instead of a generic Linux host."
|
||||
)
|
||||
|
||||
final = compliant[0]
|
||||
print(f"Repaired wheel: {final.name}")
|
||||
return final
|
||||
|
||||
|
||||
def publish_wheel_to_sdk(wheel_path: Path, version: str) -> Path:
|
||||
sdk_version_dir = sdk_dir(version)
|
||||
sdk_version_dir.mkdir(parents=True, exist_ok=True)
|
||||
dest_wheel = sdk_version_dir / wheel_path.name
|
||||
if dest_wheel.exists():
|
||||
dest_wheel.unlink()
|
||||
shutil.copy2(str(wheel_path), str(dest_wheel))
|
||||
return dest_wheel
|
||||
@@ -3,7 +3,7 @@ REM Build Nuitka wheels for Python 3.10, 3.11, and 3.12.
|
||||
REM Expects conda envs named py310, py311, py312 (or set CONDA_ENVS_ROOT).
|
||||
setlocal enabledelayedexpansion
|
||||
|
||||
cd /d "%~dp0\.."
|
||||
cd /d "%~dp0\..\.."
|
||||
if errorlevel 1 (
|
||||
echo ERROR: Could not change to repository root.
|
||||
exit /b 1
|
||||
@@ -32,7 +32,7 @@ for %%E in (py310 py311 py312) do (
|
||||
|
||||
set "OK=0"
|
||||
if !HAS_CONDA! equ 1 (
|
||||
call conda run --no-capture-output -n %%E python build\build_nuitka.py
|
||||
call conda run --no-capture-output -n %%E python build\wheel\build.py
|
||||
if not errorlevel 1 set "OK=1"
|
||||
)
|
||||
|
||||
@@ -44,7 +44,7 @@ for %%E in (py310 py311 py312) do (
|
||||
if not exist "!PYEXE!" (
|
||||
echo ERROR: Python not found: !PYEXE!
|
||||
) else (
|
||||
"!PYEXE!" build\build_nuitka.py
|
||||
"!PYEXE!" build\wheel\build.py
|
||||
if not errorlevel 1 set "OK=1"
|
||||
)
|
||||
)
|
||||
@@ -1,62 +0,0 @@
|
||||
"""Assemble and build the Nuitka wheel with setuptools."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from paths import sdk_dir
|
||||
|
||||
|
||||
def write_wheel_setup(
|
||||
wheel_dir: Path,
|
||||
*,
|
||||
version: str,
|
||||
package_data: dict[str, list[str]],
|
||||
install_requires: list[str],
|
||||
extras_require: dict[str, list[str]],
|
||||
) -> Path:
|
||||
setup_py_path = wheel_dir / "setup.py"
|
||||
setup_content = f"""
|
||||
from setuptools import setup, Distribution
|
||||
|
||||
class BinaryDistribution(Distribution):
|
||||
def has_ext_modules(self):
|
||||
return True
|
||||
|
||||
setup(
|
||||
name="orisys",
|
||||
version="{version}",
|
||||
description="Orisys - Optical Tactile Sensor SDK (Nuitka Compiled)",
|
||||
author="Orisys",
|
||||
packages=['orisys', 'orisys.examples'],
|
||||
package_dir={{'': 'wheel_root'}},
|
||||
package_data={package_data},
|
||||
include_package_data=True,
|
||||
install_requires={install_requires!r},
|
||||
extras_require={extras_require!r},
|
||||
distclass=BinaryDistribution,
|
||||
zip_safe=False,
|
||||
)
|
||||
"""
|
||||
setup_py_path.write_text(setup_content, encoding="utf-8")
|
||||
return setup_py_path
|
||||
|
||||
|
||||
def run_bdist_wheel(wheel_dir: Path) -> Path | None:
|
||||
print("\n=== Building Wheel ===")
|
||||
subprocess.check_call([sys.executable, "setup.py", "bdist_wheel"], cwd=wheel_dir)
|
||||
wheels = sorted((wheel_dir / "dist").glob("*.whl"))
|
||||
return wheels[0] if wheels else None
|
||||
|
||||
|
||||
def publish_wheel_to_sdk(wheel_path: Path, version: str) -> Path:
|
||||
sdk_version_dir = sdk_dir(version)
|
||||
sdk_version_dir.mkdir(parents=True, exist_ok=True)
|
||||
dest_wheel = sdk_version_dir / wheel_path.name
|
||||
if dest_wheel.exists():
|
||||
dest_wheel.unlink()
|
||||
shutil.move(str(wheel_path), str(dest_wheel))
|
||||
return dest_wheel
|
||||
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"name": "dd06-ov",
|
||||
"type": "single_camera",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"fps": 30,
|
||||
"resolution_stitch": [
|
||||
195,
|
||||
275
|
||||
],
|
||||
"source_scale": 0.4,
|
||||
"old": [
|
||||
672,
|
||||
728,
|
||||
1,
|
||||
0.5
|
||||
],
|
||||
"work": [
|
||||
504,
|
||||
546,
|
||||
1
|
||||
],
|
||||
"work2": [
|
||||
420,
|
||||
450,
|
||||
0.8
|
||||
],
|
||||
"work3": [
|
||||
350,
|
||||
375,
|
||||
4,
|
||||
0.7
|
||||
],
|
||||
"work4": [
|
||||
280,
|
||||
300,
|
||||
4,
|
||||
0.4,
|
||||
0.2
|
||||
]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": [
|
||||
"raft",
|
||||
"dis"
|
||||
],
|
||||
"hhd_downsample": 3
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold": 0.2,
|
||||
"div_threshold": 0.022,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"denoise_sigma": 12
|
||||
},
|
||||
"roi": {
|
||||
"type": "polygon",
|
||||
"image_size": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"created_at": "2026-07-26T11:56:15",
|
||||
"points": [
|
||||
[
|
||||
242,
|
||||
1
|
||||
],
|
||||
[
|
||||
402,
|
||||
597
|
||||
],
|
||||
[
|
||||
799,
|
||||
606
|
||||
],
|
||||
[
|
||||
976,
|
||||
2
|
||||
]
|
||||
],
|
||||
"mask_path": "dd06-ov_mask.png",
|
||||
"metadata": {
|
||||
"source": "./data/dd06/0726.mp4"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -11,14 +11,16 @@
|
||||
},
|
||||
"stitcher": {
|
||||
"min_area_frac": 0.01,
|
||||
"max_area_frac": 0.04,
|
||||
"max_aspect_ratio": 100,
|
||||
"max_area_frac": 0.05,
|
||||
"max_aspect_ratio": 50,
|
||||
"index": [1,2,3,6,7,8,11,12,13],
|
||||
"margin": 10,
|
||||
"nominal_ov_pos": 100,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [220, 220]
|
||||
"box_size": [250, 250],
|
||||
"blend":"numba",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
@@ -33,11 +35,11 @@
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold":1,
|
||||
"div_threshold":0.02,
|
||||
"flow_threshold":5,
|
||||
"div_threshold":0.05,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [5,5],
|
||||
"denoise_sigma": 15
|
||||
"denoise_sigma": 12
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"name": "dd06-ov",
|
||||
"type": "single_camera",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"fps": 30,
|
||||
"resolution_stitch": [
|
||||
195,
|
||||
275
|
||||
],
|
||||
"source_scale": 0.4,
|
||||
"old": [
|
||||
672,
|
||||
728,
|
||||
1,
|
||||
0.5
|
||||
],
|
||||
"work": [
|
||||
504,
|
||||
546,
|
||||
1
|
||||
],
|
||||
"work2": [
|
||||
420,
|
||||
450,
|
||||
0.8
|
||||
],
|
||||
"work3": [
|
||||
350,
|
||||
375,
|
||||
4,
|
||||
0.7
|
||||
],
|
||||
"work4": [
|
||||
280,
|
||||
300,
|
||||
4,
|
||||
0.4,
|
||||
0.2
|
||||
]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": [
|
||||
"raft",
|
||||
"dis"
|
||||
],
|
||||
"hhd_downsample": 3
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold": 0.2,
|
||||
"div_threshold": 0.022,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"denoise_sigma": 12
|
||||
},
|
||||
"roi": {
|
||||
"type": "polygon",
|
||||
"image_size": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"points": [
|
||||
[
|
||||
628,
|
||||
532
|
||||
],
|
||||
[
|
||||
1145,
|
||||
694
|
||||
],
|
||||
[
|
||||
1161,
|
||||
8
|
||||
],
|
||||
[
|
||||
630,
|
||||
126
|
||||
]
|
||||
],
|
||||
"created_at": "2026-08-10T14:26:31",
|
||||
"mask_path": "dd06-ov-0810_mask.png",
|
||||
"metadata": {
|
||||
"source": ".\\data\\dd06\\2026-08-10-142316.webm"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"name": "dd06-ov",
|
||||
"type": "single_camera",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"fps": 30,
|
||||
"resolution_stitch": [
|
||||
195,
|
||||
275
|
||||
],
|
||||
"source_scale": 0.4,
|
||||
"old": [
|
||||
672,
|
||||
728,
|
||||
1,
|
||||
0.5
|
||||
],
|
||||
"work": [
|
||||
504,
|
||||
546,
|
||||
1
|
||||
],
|
||||
"work2": [
|
||||
420,
|
||||
450,
|
||||
0.8
|
||||
],
|
||||
"work3": [
|
||||
350,
|
||||
375,
|
||||
4,
|
||||
0.7
|
||||
],
|
||||
"work4": [
|
||||
280,
|
||||
300,
|
||||
4,
|
||||
0.4,
|
||||
0.2
|
||||
]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": [
|
||||
"raft",
|
||||
"dis"
|
||||
],
|
||||
"hhd_downsample": 3
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold": 0.2,
|
||||
"div_threshold": 0.022,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"denoise_sigma": 12
|
||||
},
|
||||
"roi": {
|
||||
"type": "polygon",
|
||||
"image_size": [
|
||||
1280,
|
||||
720
|
||||
],
|
||||
"created_at": "2026-07-26T11:56:15",
|
||||
"points": [
|
||||
[
|
||||
242,
|
||||
1
|
||||
],
|
||||
[
|
||||
402,
|
||||
597
|
||||
],
|
||||
[
|
||||
799,
|
||||
606
|
||||
],
|
||||
[
|
||||
976,
|
||||
2
|
||||
]
|
||||
],
|
||||
"mask_path": "dd06-ov_mask.png",
|
||||
"metadata": {
|
||||
"source": "./data/dd06/0726.mp4"
|
||||
}
|
||||
}
|
||||
}
|
||||
+4
-2
@@ -18,7 +18,9 @@
|
||||
"nominal_ov_pos": 100,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [250, 250]
|
||||
"box_size": [250, 250],
|
||||
"blend":"numba",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
@@ -29,7 +31,7 @@
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
"hhd_downsample": 3
|
||||
"hhd_downsample": 1
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
|
||||
@@ -18,7 +18,9 @@
|
||||
"nominal_ov_pos": 44,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [98, 98]
|
||||
"box_size": [98, 98],
|
||||
"blend":"numpy",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
|
||||
@@ -18,7 +18,9 @@
|
||||
"nominal_ov_pos": 67,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [180, 180]
|
||||
"box_size": [180, 180],
|
||||
"blend":"numpy",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
@@ -29,7 +31,7 @@
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
"hhd_downsample": 3,
|
||||
"hhd_downsample": 1,
|
||||
"backend": "cpu"
|
||||
},
|
||||
"contact": {
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
{
|
||||
"name": "prototype",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [1280, 720],
|
||||
"fps": 15,
|
||||
"units": 15,
|
||||
"units_used": 9,
|
||||
"resolution_stitch": [263, 263],
|
||||
"stitch_res_list": [365,263,123,63]
|
||||
},
|
||||
"stitcher": {
|
||||
"min_area_frac": 0.01,
|
||||
"max_area_frac": 0.04,
|
||||
"max_aspect_ratio": 25,
|
||||
"index": [1,2,3,6,7,8,11,12,13],
|
||||
"margin": 10,
|
||||
"nominal_ov_pos": 67,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [180, 180]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
"hhd_downsample": 3,
|
||||
"backend": "cpu"
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold":1,
|
||||
"div_threshold":0.02,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [5,5],
|
||||
"denoise_sigma": 10
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "dd02-ov",
|
||||
"type": "single_camera",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [1280, 720],
|
||||
"fps": 30,
|
||||
"resolution_stitch": [280, 300],
|
||||
"source_scale": 0.4,
|
||||
"old":[672, 728, 1,0.5],
|
||||
"work": [504, 546,1],
|
||||
"work2": [420, 450,0.8],
|
||||
"work3": [350, 375,4,0.7],
|
||||
"work4": [280, 300,4,0.4,0.2]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 1,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
"hhd_downsample": 5
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold": 0.2,
|
||||
"div_threshold": 0.022,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [5, 5],
|
||||
"denoise_sigma": 12
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"name": "prototype",
|
||||
"version": "1.0",
|
||||
"camera": {
|
||||
"resolution_raw": [1280, 720],
|
||||
"fps": 30,
|
||||
"units": 15,
|
||||
"units_used": 9,
|
||||
"resolution_stitch": [263, 263],
|
||||
"stitch_res_list": [365,263,123,63]
|
||||
},
|
||||
"stitcher": {
|
||||
"min_area_frac": 0.01,
|
||||
"max_area_frac": 0.04,
|
||||
"max_aspect_ratio": 25,
|
||||
"index": [1,2,3,6,7,8,11,12,13],
|
||||
"margin": 10,
|
||||
"nominal_ov_pos": 67,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [180, 180],
|
||||
"blend":"numpy",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
"hhd_downsample": 3
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold":1,
|
||||
"div_threshold":0.02,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [5,5],
|
||||
"denoise_sigma": 10
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -6,25 +6,28 @@
|
||||
"fps": 30,
|
||||
"units": 15,
|
||||
"units_used": 9,
|
||||
"resolution_stitch": [384, 384]
|
||||
"resolution_stitch": [365, 365],
|
||||
"stitch_res_list": [365,263,123,63]
|
||||
},
|
||||
"stitcher": {
|
||||
"min_area_frac": 0.01,
|
||||
"max_area_frac": 0.04,
|
||||
"max_aspect_ratio": 100,
|
||||
"max_area_frac": 0.05,
|
||||
"max_aspect_ratio": 50,
|
||||
"index": [1,2,3,6,7,8,11,12,13],
|
||||
"margin": 10,
|
||||
"nominal_ov_pos": 100,
|
||||
"n": [3,3],
|
||||
"row_tol": 0.6,
|
||||
"box_size": [220, 220]
|
||||
"box_size": [250, 250],
|
||||
"blend":"numba",
|
||||
"blend_list": ["numpy", "numba","none"]
|
||||
},
|
||||
"tracker": {
|
||||
"adaptive": true,
|
||||
"lower_threshold": 12.0,
|
||||
"upper_threshold": 18.0,
|
||||
"loop_closure_interval": 1,
|
||||
"algorithm": "raft",
|
||||
"algorithm": "dis",
|
||||
"dis_preset": "medium",
|
||||
"dis_mode": 3,
|
||||
"algorithm_list": ["raft", "dis"],
|
||||
@@ -32,11 +35,11 @@
|
||||
},
|
||||
"contact": {
|
||||
"min_contour_area": 3000,
|
||||
"flow_threshold":1.5,
|
||||
"div_threshold":0.08,
|
||||
"flow_threshold":5,
|
||||
"div_threshold":0.05,
|
||||
"expansion_threshold": 0.01,
|
||||
"kernel_size": [5,5],
|
||||
"denoise_sigma": 15
|
||||
"denoise_sigma": 12
|
||||
}
|
||||
}
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,147 @@
|
||||
# Public API surface audit (2026-09-30)
|
||||
|
||||
A snapshot of what the SDK declares as public, what is documented, and where the gaps are.
|
||||
Produced by [`tools/api_audit.py`](../tools/api_audit.py), which is static and read-only — it
|
||||
never imports the package, so it costs no numpy/cv2 import and cannot be affected by the
|
||||
environment. Re-run it after any API change:
|
||||
|
||||
```powershell
|
||||
cd D:\orisys_ws\orisys-sdk
|
||||
python tools/api_audit.py # text report
|
||||
python tools/api_audit.py --json # machine-readable
|
||||
python tools/api_audit.py --quiet # headline numbers
|
||||
```
|
||||
|
||||
## 1. Headline
|
||||
|
||||
| Measure | Value |
|
||||
|---|---|
|
||||
| Public classes / methods / functions | **393** (53 / 214 / 126) |
|
||||
| With a docstring | **256 (65 %)** |
|
||||
| Fully annotated signature (params + return) | **125 (32 %)** |
|
||||
| Docstring documents its return value | **38** |
|
||||
|
||||
Docstring conventions actually in use:
|
||||
|
||||
| Convention | Count |
|
||||
|---|---|
|
||||
| plain prose (no structured sections) | 243 |
|
||||
| none | 137 |
|
||||
| Google (`Args:` / `Returns:`) | 9 |
|
||||
| NumPy (`Parameters` + `---`) | 4 |
|
||||
|
||||
**Reading:** the dominant convention is not a convention. Structured sections are absent from
|
||||
almost every docstring, and two thirds of signatures are missing type information. The hand-written
|
||||
reference compensates in the places it covers; a generated reference would not.
|
||||
|
||||
## 2. The declared public surface
|
||||
|
||||
The root `__all__` (`src/orisys/__init__.py`) declares 18 names. Sub-surfaces declare a further 88.
|
||||
|
||||
### 2.1 Root exports — 9 of 18 are absent from `docs/api.md`
|
||||
|
||||
| Documented | Not mentioned anywhere in `docs/api.md` |
|
||||
|---|---|
|
||||
| `Sensor`, `util`, `finger`, `signal_processing`, `capture`, `list_config`, `export_config`, `export_sensor_bundle`, `export_sensor_bundles` | **`StreamPipeline`**, **`FrameSnapshot`**, **`PipelineStats`**, **`StreamRecorder`**, **`mosaic_videos`**, **`FFCFrameProcessor`**, `CompoundEyeSensor`, `CameraSensor`, `calibration` |
|
||||
|
||||
`StreamPipeline` is the engine behind the published "5 × 30 FPS on one desktop, 4.2× headroom"
|
||||
claim and is used by `examples/multi_stream.py`. `StreamRecorder` and `mosaic_videos` are the
|
||||
offline-recording path. None of them has a Markdown reference anywhere in the repository.
|
||||
|
||||
### 2.2 Sub-surfaces with no coverage in `api.md` at all
|
||||
|
||||
| Module | Exports | Documented | Note |
|
||||
|---|---|---|---|
|
||||
| `orisys.unroll` | 18 | 0 | UV domain, remap, geometry — the finger pipeline's core maths |
|
||||
| `orisys.capture` | 17 | 0 | capture sources, workers, reconnect policy, camera discovery |
|
||||
| `orisys.hhd` + `hhd.solvers` | 7 | 0 | **no docstrings either** — see §3 |
|
||||
| `orisys.pipeline` | 3 | 0 | `StreamPipeline` etc. |
|
||||
| `orisys.export` | 2 | 0 | `StreamRecorder`, `mosaic_videos` |
|
||||
|
||||
### 2.3 Exports with no docstring — 20 of 88
|
||||
|
||||
Includes `CaptureWorker`, `pick_default_camera`, `available_blend_backends`,
|
||||
`compute_final_crop_rect`, `coverage_ratio`, `largest_fully_covered_rect`,
|
||||
`match_bgr_brightness`, `barycentric_grid`, `face_uvs_from_loop_data`, all four `orisys.hhd`
|
||||
exports, and named constants (`DEFAULT_LAPTOP_WEBCAM_PATTERNS`, `MAX_SIMULTANEOUS_CAPTURES`,
|
||||
`DEFAULT_FINGER_EXPORT_DIR`, `DEFAULT_FINGER_MAP_CACHE`, `BLEND_MODES`).
|
||||
|
||||
## 3. Coverage by module
|
||||
|
||||
The extreme cases, which are where effort pays off:
|
||||
|
||||
| Module | Symbols | Docstrings | Annotated | Comment |
|
||||
|---|---|---|---|---|
|
||||
| `orisys.finger.runtime` | 31 | **35 %** | **3 %** | `FingerRuntime` is the documented entry point for finger users, yet its implementation module is the thinnest in the package |
|
||||
| `orisys.hhd.timing` | 4 | **0 %** | 0 % | |
|
||||
| `orisys.stitch.feature` | 10 | 20 % | 0 % | |
|
||||
| `orisys.stitch.tiles` | 5 | 20 % | 0 % | |
|
||||
| `orisys.stitch.api` | 10 | 30 % | 10 % | |
|
||||
| `orisys.frame_processors` | 21 | 48 % | **0 %** | |
|
||||
| `orisys.capture.source` | 32 | 50 % | 91 % | annotations good, prose thin |
|
||||
| `orisys.sensor` | 21 | 90 % | **38 %** | the most-documented module, still missing annotations |
|
||||
| `orisys.configs` | 13 | 100 % | 69 % | |
|
||||
| `orisys.unroll.precompute`, `domain` | 6 / 6 | 100 % | 0 % | documented but untyped |
|
||||
|
||||
Annotated and documented do not move together: `capture.source` is 91 % annotated but 50 %
|
||||
documented; `hhd.domain` is 100 % documented but 0 % annotated.
|
||||
|
||||
## 4. Used by examples and tests, absent from the reference — 74 names
|
||||
|
||||
These are APIs users demonstrably reach for and the reference never explains. The highest-traffic
|
||||
group is the runtime/typing surface:
|
||||
|
||||
`StreamPipeline`, `FrameSnapshot`, `PipelineStats`, `StreamRecorder`, `mosaic_videos`,
|
||||
`FFCFrameProcessor`, `UVDomain`, `UVRemap`, `UVSurfaceMap`, `UVSurfaceMesh`, `UVMeshViewer`,
|
||||
`Tracker`, `Contact3D`, `CalSample`, `CalibrationError`, `StitchStatus`, `CameraInfo`,
|
||||
`CameraFilter`, `CaptureSource`, `CaptureConfig`, `CaptureHealth`, `ReconnectPolicy`,
|
||||
`PoissonSolver`, `nHHD`
|
||||
|
||||
Plus the finger/geometry helpers (`build_uv_surface_mesh`, `compute_uv_surface_frames`,
|
||||
`lift_uv_flow_to_3d`, `flow_magnitude_vertex_colors`, `scalar_map_vertex_colors`,
|
||||
`sample_field_at_mesh_vertices`, `mesh_uv_to_world_scale`, `resolve_export_paths`,
|
||||
`resolve_finger_geometry`, `load_finger_export_bundle`, `init_uv_mesh_viewer`,
|
||||
`update_uv_mesh_viewer`), the capture helpers (`open_capture`, `normalize_source`,
|
||||
`list_cameras`, `pick_default_camera`), the calibration helpers (`check`, `build`, `parse`,
|
||||
`load`, `calibrate_normal_force`, `calibrate_shear_force`, `calibrate_shear_components`),
|
||||
and the config resolvers (`load_config`, `list_sensor_bundles`, `resolve_sensor_config_path`,
|
||||
`resolve_sensor_resource`, `resolve_sensor_asset_dir`, `pipeline_from_sensor_json`).
|
||||
|
||||
## 5. Public classes exported nowhere — 21
|
||||
|
||||
Defined, carrying no name-mangling underscore, but in no `__all__` and not reachable as a declared
|
||||
API: `BlendBackend`, `CalSample`, `Calibration`, `CalibrationError`, `CausalMedianFilter`,
|
||||
`ChannelCurve`, `CheckResult`, `CompoundEyeFrameProcessor`, `Contact3D`,
|
||||
`ForceChannelCalibration`, `FrameUnwarper`, `LowpassFilter`, `MaskedDomain`,
|
||||
`MaskedPoissonConfig`, `NumbaBlendBackend`, `NumpyBlendBackend`, `SampleRateEstimator`,
|
||||
`SensorBundle`, `SingleCameraFrameProcessor`, `Tracker`, `VideoCaptureLike`.
|
||||
|
||||
Two of these are the clearest signals of a boundary that was never decided:
|
||||
|
||||
- **`LowpassFilter`** is documented in `docs/api.md` as `orisys.signal_processing.LowpassFilter`
|
||||
and is imported by examples — but it appears in neither the submodule's `__all__` nor the root's.
|
||||
It is public by documentation and private by declaration.
|
||||
- **`Calibration`** / `CalibrationError` / `CalSample` / `CheckResult` are the entire force
|
||||
calibration object model, and `calibration.check()` / `calibration.build()` are used directly by
|
||||
the test suite. The `orisys.calibration` package is exported from the root, but none of its
|
||||
public names are declared or documented.
|
||||
|
||||
Each of the 21 needs one decision: promote into the relevant `__all__`, or rename with a leading
|
||||
underscore to mark it internal.
|
||||
|
||||
## 6. What this implies
|
||||
|
||||
1. **Fix the declaration boundary first.** Nine root exports have no reference and 21 classes sit
|
||||
in the gap between documented and declared. Publishing a reference on top of an undecided
|
||||
public surface multiplies the confusion rather than resolving it.
|
||||
2. **Pick one docstring convention before generating anything.** With 243 prose docstrings, 137
|
||||
absent, and 13 structured, a generator's output would be thin and inconsistent — the tool
|
||||
renders what is there.
|
||||
3. **Annotations are the larger gap.** 32 % complete signatures means generated parameter tables
|
||||
would be mostly bare. This is mechanical work and it improves IDE help, not only docs.
|
||||
4. **The omissions are not random.** `StreamPipeline`, `capture` and `unroll` are the *newest*
|
||||
subsystems, and they are the ones with no coverage. The reference drifted toward the parts that
|
||||
existed when it was written.
|
||||
5. **`orisys.hhd` is documented nowhere and has no docstrings**, yet is exported from the root
|
||||
package. Either document it or stop exporting it — leaving it in `__all__` promises support that
|
||||
does not exist.
|
||||
+421
-138
@@ -1,86 +1,30 @@
|
||||
# orisys API 文档
|
||||
---
|
||||
title: API Reference
|
||||
description: Orisys SDK Python API 中文参考
|
||||
---
|
||||
|
||||
本文件提供了 `orisys` 套件的中文文档
|
||||
# API Reference
|
||||
|
||||
## 准备 Python 开发环境
|
||||
推荐下载 Anaconda,并使用Python 3.10
|
||||
```
|
||||
# 创建并激活虚拟环境
|
||||
conda create -n orisys python=3.10
|
||||
conda activate orisys
|
||||
```
|
||||
## 安装 CUDA Toolkit 和 cuDNN
|
||||
在官方链接下载和安装:https://developer.nvidia.com/cuda/toolkit
|
||||
或
|
||||
在Anaconda环境中安装(例子使用cuda12.9):
|
||||
https://anaconda.org/channels/nvidia/packages/cuda/overview
|
||||
```
|
||||
conda install nvidia/label/cuda-12.9.0::cuda-toolkit nvidia::cudnn
|
||||
```
|
||||
## SDK安装方法
|
||||
1. 根据系统(win / linux),平台(amd64, aarch64), python(如p38、p39、p310) 版本下载对应的`.whl`文件
|
||||
2. 运行pip命令安装:
|
||||
```bash
|
||||
pip install orisys-{version}-{python-version}-{python-version}-{os}_{machine}.whl
|
||||
```
|
||||
如
|
||||
```bash
|
||||
pip install orisys-0.4.0-cp310-cp310-win_amd64.whl
|
||||
```
|
||||
本文件提供 `orisys` 套件的 Python API 中文参考。安装、CUDA、CuPy 与运行示例的基础步骤请参考 [快速开始](./get_started.md)。
|
||||
|
||||
3. **安装 CuPy(根据您的 CUDA 版本选择):**
|
||||
|
||||
**方法 1:使用 extras(推荐)**
|
||||
```bash
|
||||
# 对于 CUDA 12.x
|
||||
pip install orisys[cuda12]
|
||||
|
||||
# 对于 CUDA 13.x
|
||||
pip install orisys[cuda13]
|
||||
```
|
||||
|
||||
**方法 2:手动安装**
|
||||
```bash
|
||||
# 对于 CUDA 12.x
|
||||
pip install cupy-cuda12x>=12.3.0
|
||||
|
||||
# 对于 CUDA 13.x
|
||||
pip install cupy-cuda13x>=13.0.0
|
||||
```
|
||||
## Sensor API
|
||||
|
||||
4. 测试导入:
|
||||
```python
|
||||
import orisys
|
||||
print(orisys.__version__)
|
||||
```
|
||||
|
||||
5 安装opencl(实验):
|
||||
|
||||
在不带nvidia显卡的设备,可选择通过opencl进行gpu 计算加速,需要确保opencl的库或驱动已经安装,然后在conda环境中安装依赖:
|
||||
|
||||
``` bash
|
||||
pip install pyopencl
|
||||
conda install conda-forge::pyvkfft
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## `Sensor` 类别
|
||||
### 描述
|
||||
### `Sensor` 类别
|
||||
#### 描述
|
||||
`orisys`套件的核心类,用于从视频文件或实时摄像头中读取传感器图像,创建一个具体的传感器对象(实例)。并完成形变、光流及力信息的计算与管理。
|
||||
|
||||
### 输入参数
|
||||
#### 输入参数
|
||||
|
||||
* **vid_src** (`int | str`, 必填): 传感器 ID、相机编号或视频路径。默认为 `0`。
|
||||
* **config_name** (`str`,可选): 传感器配置文件名称或 `json` 文件路径,用于加载对应的参数配置;例如 `ddjx01` 或 `./config/ddjx01.json`。
|
||||
* **cal_path** (`str`,可选,已弃用): 该参数仅为兼容旧代码保留,当前版本中不再使用,传入后也会被忽略。
|
||||
* **config_name** (`str`,可选): 传感器配置文件名称或 `json` 文件路径,用于加载对应的参数配置;例如 `box` 或 `./config/box.json`。
|
||||
* **cal_path** (`str`,可选): 显式的 `force_calibration.json` 路径;传入后优先于 `sensors/<type>/force_calibration.json`。不传时按 `config_name` 自动加载产品包中的标定文件。
|
||||
* **backend** (`str`,可选): 光流场解耦计算后端,默认为 `auto`,可选`cupy`,`opencl`,`cpu`。
|
||||
* **verbose** (`bool`,可选): 是否输出初始化及运行过程中的详细日志信息,默认为 `False`。
|
||||
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
* `Sensor` 实例,用于后续图像采集、形变计算和数据读取操作。
|
||||
### 示例
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
# Example 1: 用相机编号创建
|
||||
@@ -91,63 +35,62 @@ sensor = Sensor(0)
|
||||
sensor = Sensor("/data/data.mp4")
|
||||
```
|
||||
|
||||
***
|
||||
### `Sensor.get_img` 方法
|
||||
|
||||
## 2. `Sensor.get_img` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
|
||||
采集并预处理一帧图像并存到`Sensor` 类别中。
|
||||
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
* `bool`:是否成功读取图像。
|
||||
|
||||
### 状态更新
|
||||
#### 状态更新
|
||||
* 成功时更新`sensor`类里面的`sensor.img`。处理后的BGR格式图像(`uint8`,尺寸为json文件中`resolution_stitch`对应分辨率)。
|
||||
### 示例
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
sensor.get_img()
|
||||
```
|
||||
***
|
||||
### `Sensor.compute_deformation` 方法
|
||||
|
||||
## 3. `Sensor.compute_deformation` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
|
||||
基于当前帧图像计算光流/形变向量场,并通过 HHD 分解将形变场分解为法向/切向分量,同时估计法向力、切向力与扭矩等标量输出。该方法会更新
|
||||
`sensor`的形变场与力相关状态,并返回当前帧的光流场。
|
||||
调用前置条件:需要先通过`get_img()`成功更新图像。
|
||||
|
||||
### 输入参数
|
||||
* **check_motion**(`bool`,可选): 为 `True` 时先根据光流判断是否有运动,有运动才做 HHD 分解;为 `False` 时每帧都做 HHD。默认 `True`。
|
||||
* **threshold**(`float`,可选): **check_motion** 功能的阈值,默认为`6`。
|
||||
* **tail_frames** (`int`,可选): 光流变化低于阈值后继续解算多少帧,确保传感器回复,,默认为 `10`.
|
||||
#### 输入参数
|
||||
* **decompose** (`bool`,可选): 是否在计算光流后继续执行 HHD 分解,默认 `True`。
|
||||
* **check_motion** (`bool`,可选): 是否启用运动检测门控。当前不推荐依赖该功能;新代码建议显式传入 `False`,让每帧都执行 HHD 分解。为兼容旧代码,函数默认值仍为 `True`。
|
||||
|
||||
### 返回
|
||||
#### 兼容参数(暂不推荐)
|
||||
|
||||
* `bool`:是否接触到物体,如果**check_motion**为`False`时返回`None`。
|
||||
以下参数只在 `check_motion=True` 时生效。由于运动检测门控目前不稳定,建议在新代码中暂时不要使用:
|
||||
|
||||
### 示例
|
||||
* **threshold** (`float`,可选): 运动检测阈值,默认 `6`。
|
||||
* **tail_frames** (`int`,可选): 运动低于阈值后继续解算多少帧,默认 `10`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `None`: 当 `check_motion=False` 时返回。推荐用法下不要依赖该返回值判断接触状态。
|
||||
* `bool`: 仅当 `check_motion=True` 时返回运动检测结果;该模式目前暂不推荐。
|
||||
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
sensor.get_img()
|
||||
|
||||
is_touched = sensor.compute_deformation(check_motion=True, threshold=6)
|
||||
# 步骤 2.3: 计算触碰点
|
||||
if is_touched:
|
||||
sensor.compute_contact()
|
||||
sensor.compute_deformation(check_motion=False)
|
||||
sensor.compute_contact()
|
||||
```
|
||||
***
|
||||
## 4. `Sensor.compute_contact` 方法
|
||||
### `Sensor.compute_contact` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
基于已计算的法向形变场进行接触区域分析,输出接触质心与轮廓等信息,并生成接触深度图,默认配置在`json`文件中,可按需调节。
|
||||
|
||||
调用前置条件:需先执行`compute_deformation()`。
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
`dict`为接触分析结果字典,至少包含以下键:
|
||||
|
||||
* **expansion_mask**: 扩张区域掩码(`np.ndarray`)。
|
||||
@@ -157,12 +100,12 @@ if is_touched:
|
||||
* **centroid_found**: 是否检测到接触区域(`bool`)。
|
||||
* **contour**: 最大轮廓或轮廓列表(可选字段,存在时返回)。
|
||||
|
||||
### 状态更新
|
||||
#### 状态更新
|
||||
|
||||
* **sensor.centroid**: 接触质心(单点为 `(x, y)`,多点为列表)。
|
||||
* **sensor.centroid_found**: 是否找到质心(`bool`)。
|
||||
|
||||
### 示例
|
||||
#### 示例
|
||||
```python
|
||||
sensor.compute_contact()
|
||||
centroid, contour, depth = sensor.read_info(
|
||||
@@ -171,19 +114,18 @@ centroid, contour, depth = sensor.read_info(
|
||||
sensor.info.DEPTH,
|
||||
)
|
||||
```
|
||||
***
|
||||
## 4. `Sensor.read_info` 方法
|
||||
### `Sensor.read_info` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
|
||||
按指定的信息类型(`sensor.info`枚举值)批量读取传感器状态数据,并按传入顺序返回对应值的元组。
|
||||
该方法通常用于在完成形变计算后,读取光流场、分量场、力、帧率等信息。
|
||||
|
||||
### 输入参数
|
||||
#### 输入参数
|
||||
|
||||
* **args**: 任意数量的 `Sensor.info` 枚举,用于指定需要获取的数据类型。支持的枚举值及对应数据如下:
|
||||
|
||||
### 支持的枚 info keys:
|
||||
#### 支持的 info keys
|
||||
|
||||
向量/图像类
|
||||
|
||||
@@ -197,6 +139,9 @@ centroid, contour, depth = sensor.read_info(
|
||||
* FNORMAL: 法向力,`float`。
|
||||
* FSHEARX: X切向力,`float`。
|
||||
* FSHEARY: Y切向力,`float`。
|
||||
* FNORMAL_RAW: 未经过低通滤波的原始法向力,`float`。
|
||||
* FSHEARX_RAW: 未经过低通滤波的原始 X 切向力,`float`。
|
||||
* FSHEARY_RAW: 未经过低通滤波的原始 Y 切向力,`float`。
|
||||
* FPS: 输出帧率,`float`。
|
||||
* CENTROID: 触碰点的坐标,(x, y)。
|
||||
* CONTOUR: 触碰区域轮廓,`np.ndarray`, shape=(H, W), 未生成时返回`np.empty`
|
||||
@@ -207,7 +152,7 @@ centroid, contour, depth = sensor.read_info(
|
||||
|
||||
注:H/W 由当前`json`文件中的`resolution_stitch`配置分辨率决定。
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
|
||||
`Tuple`
|
||||
|
||||
@@ -219,7 +164,7 @@ centroid, contour, depth = sensor.read_info(
|
||||
- 标量类数据:`float`
|
||||
- 时间戳:`str`(`TIMESTAMP`,格式为 `HH:MM:SS`)
|
||||
|
||||
### 示例
|
||||
#### 示例
|
||||
```python
|
||||
# Example 1:读取标量数据
|
||||
fps, fn, fx, fy = sensor.read_info(
|
||||
@@ -232,23 +177,351 @@ fps, fn, fx, fy = sensor.read_info(
|
||||
# Example 2: 读取单个向量场(注意解包方式)
|
||||
flow = sensor.read_info(sensor.info.VRAW)
|
||||
```
|
||||
***
|
||||
### `Sensor.disconnect` 方法
|
||||
|
||||
## 5. `Sensor.disconnect` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
|
||||
释放传感器相关资源并停止后台异步线程。 调用后将关闭视频/摄像头输入源,并销毁所有 OpenCV 窗口。
|
||||
|
||||
建议在程序退出前调用以确保资源被正确释放。
|
||||
|
||||
***
|
||||
***
|
||||
# 可视化相关
|
||||
### `Sensor.configure` 低通滤波设置
|
||||
|
||||
## 1. `util.draw_arrows` 方法
|
||||
#### 描述
|
||||
|
||||
### 输入参数
|
||||
运行时更新传感器设置,无需重新连接相机。常用于开启力信号低通滤波,降低 `FNORMAL`、`FSHEARX`、`FSHEARY` 的高频噪声。
|
||||
|
||||
默认采样率按 30 fps 计算,推荐低通截止频率为 5 Hz:
|
||||
|
||||
```python
|
||||
sensor.configure(lowpass_cutoff_hz=5.0, lowpass_sample_rate_hz=30.0)
|
||||
```
|
||||
|
||||
注意:`lowpass_cutoff_hz` 必须小于 Nyquist 频率,即 `lowpass_sample_rate_hz / 2`。例如 30 fps 时,截止频率必须小于 15 Hz。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **lowpass_cutoff_hz** (`float`,可选): 力信号低通截止频率,单位 Hz。`0` 表示关闭滤波;正数默认开启滤波。
|
||||
* **lowpass_sample_rate_hz** (`float`,可选): 力信号采样率,默认 `30.0`。
|
||||
* **lowpass_order** (`int`,可选): Butterworth 滤波器阶数,默认 `2`。
|
||||
* **lowpass_enabled** (`bool`,可选): 显式开启或关闭低通滤波。
|
||||
|
||||
#### 读取结果
|
||||
|
||||
开启低通滤波后,`read_info()` 读取的力输出会被滤波:
|
||||
|
||||
```python
|
||||
fn, fx, fy = sensor.read_info(
|
||||
sensor.info.FNORMAL,
|
||||
sensor.info.FSHEARX,
|
||||
sensor.info.FSHEARY,
|
||||
)
|
||||
```
|
||||
|
||||
原始力输出仍可通过 `FNORMAL_RAW`、`FSHEARX_RAW`、`FSHEARY_RAW` 读取,便于调试或对比。
|
||||
|
||||
## Signal Processing API
|
||||
|
||||
### `orisys.signal_processing.lowpass_filter` 函数
|
||||
|
||||
#### 描述
|
||||
|
||||
对一维、等间隔采样数据执行零相位 Butterworth 低通滤波,适合示例程序或图表中对历史数据做平滑处理。
|
||||
|
||||
默认参数为 `cutoff_hz=5.0`、`sample_rate_hz=30.0`、`order=2`。这表示在 30 fps 数据上使用 5 Hz 低通滤波。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **data**: 一维数据序列。
|
||||
* **cutoff_hz** (`float`,可选): 截止频率,默认 `5.0`。传入 `0` 时不做滤波并返回数据副本。
|
||||
* **sample_rate_hz** (`float`,可选): 采样率,默认 `30.0`。
|
||||
* **order** (`int`,可选): Butterworth 滤波器阶数,默认 `2`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `np.ndarray`: 滤波后的 `float64` 数组。
|
||||
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
from orisys.signal_processing import lowpass_filter
|
||||
|
||||
filtered_force = lowpass_filter(force_values, cutoff_hz=5.0, sample_rate_hz=30.0)
|
||||
```
|
||||
|
||||
### `orisys.signal_processing.LowpassFilter` 类别
|
||||
|
||||
#### 描述
|
||||
|
||||
有状态的因果 Butterworth 低通滤波器,适合逐帧处理实时数据。它会保留上一帧的滤波状态,并通过 `process()` 一次处理一个多通道样本。
|
||||
|
||||
`Sensor.configure()` 内部使用该类来滤波力信号。构造时默认 `cutoff_hz=0.0`,表示关闭滤波;若要按 30 fps 使用推荐滤波,设置 `cutoff_hz=5.0`、`sample_rate_hz=30.0`。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **cutoff_hz** (`float`,可选): 截止频率,默认 `0.0`,表示关闭滤波。
|
||||
* **sample_rate_hz** (`float`,可选): 采样率,默认 `30.0`。
|
||||
* **order** (`int`,可选): Butterworth 滤波器阶数,默认 `2`。
|
||||
* **n_channels** (`int`,可选): 每个样本的通道数,默认 `3`。
|
||||
|
||||
#### 常用方法
|
||||
|
||||
* **configure(...)**: 运行时更新 `cutoff_hz`、`sample_rate_hz` 或 `order`。
|
||||
* **reset(x0=None)**: 清除滤波器状态;传入 `x0` 时会用该样本初始化稳态。
|
||||
* **process(values)**: 滤波一个多通道样本,返回 shape 为 `(n_channels,)` 的 `np.ndarray`。
|
||||
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
from orisys.signal_processing import LowpassFilter
|
||||
|
||||
filt = LowpassFilter(cutoff_hz=5.0, sample_rate_hz=30.0, n_channels=3)
|
||||
filtered = filt.process([fnormal, fshearx, fsheary])
|
||||
```
|
||||
|
||||
## Finger Pipeline API
|
||||
|
||||
### `orisys.finger` 手指 UV pipeline
|
||||
|
||||
`examples/finger_pipeline.py` 展示了一个更适合手指/单相机场景的高层 API:
|
||||
|
||||
1. 使用 `FingerRuntime.open()` 打开视频源、加载 Blender 导出的资产,并构建 UV 展开映射。
|
||||
2. 用 `grab_and_process()` 或 `read_source() + process_frame()` 获取一帧展开后的触觉图像。
|
||||
3. 通过 `runtime.tactile`(`TactileProcessor`)调用 `compute_deformation()`、`compute_contact()`、`read_info()`。
|
||||
4. 退出前调用 `runtime.close()` 释放视频源和内部 `Sensor` 资源。
|
||||
|
||||
相比直接使用 `Sensor`,`FingerRuntime` 已封装了视频读取、相机尺寸设置、UV 展开和有效区域掩码传递,更适合作为手指 2D/3D 示例与上层应用的入口。
|
||||
|
||||
### `FingerRuntime.open` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
创建一个可复用的手指 UV runtime。该方法会完成以下初始化:
|
||||
|
||||
- 解析 `export_dir` 下的 `mesh.npz`、`camera_intrinsics.npz`、`world_to_cam.npy`
|
||||
- 根据配置和标定信息构建/加载 `unroll_map_uv.npz`
|
||||
- 打开相机或视频文件
|
||||
- 创建内部 `Sensor("external", ...)`
|
||||
- 创建 `FrameUnwarper` 与 `TactileProcessor`
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **export_dir** (`str`,可选): 手指导出资产目录,默认值来自 `orisys.finger.DEFAULT_FINGER_EXPORT_DIR`。目录下通常需要包含 `mesh.npz`、`camera_intrinsics.npz`、`world_to_cam.npy`。
|
||||
* **config** (`str`,可选): Orisys 配置文件名称或路径,默认 `finger`。
|
||||
* **map_cache** (`str`,可选): UV 展开缓存文件路径,默认值来自 `orisys.finger.DEFAULT_FINGER_MAP_CACHE`。
|
||||
* **source_scale** (`float`,可选): 输入相机分辨率缩放比例;未提供时根据配置自动推断。
|
||||
* **calib** (`str | None`,可选): 外部相机标定文件路径;未提供时优先使用导出资产中的内参。
|
||||
* **video** (`int | str`,可选): 相机编号、数字字符串或视频路径,默认 `0`。
|
||||
* **camera_id** (`int | None`,可选): `video` 的别名;若提供则优先使用。
|
||||
* **flip_source_vertical** (`bool`,可选): 是否在 UV 展开前垂直翻转输入源,默认 `True`。
|
||||
* **verbose** (`bool`,可选): 是否输出初始化日志。
|
||||
* **mirror** (`bool`,可选): 是否对内部触觉图再做镜像,默认 `False`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `FingerRuntime` 实例。
|
||||
|
||||
#### 示例
|
||||
|
||||
```python
|
||||
from orisys.finger import FingerRuntime
|
||||
|
||||
runtime = FingerRuntime.open(
|
||||
config="finger",
|
||||
video=0,
|
||||
verbose=True,
|
||||
)
|
||||
```
|
||||
|
||||
### `FingerRuntime` 常用属性
|
||||
|
||||
* **runtime.geometry**: `FingerGeometry`,包含输入/输出分辨率、导出资产路径、相机内参、缓存路径等一次性初始化结果。
|
||||
* **runtime.domain**: UV 展开域对象,内部包含 remap、valid mask 以及可选 surface 数据。
|
||||
* **runtime.tactile**: `TactileProcessor`,用于执行形变、接触和数据读取。
|
||||
* **runtime.capture**: OpenCV `VideoCapture` 对象。
|
||||
* **runtime.valid_mask**: 当前 UV 域有效像素掩码。
|
||||
|
||||
### `FingerRuntime.read_source` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
仅读取一帧原始视频源,等价于对内部 `cv2.VideoCapture` 调用 `read()`。适合需要把采集和 UV 展开分别计时的场景。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `(ok, frame)`:
|
||||
* **ok** (`bool`): 是否成功读取。
|
||||
* **frame** (`np.ndarray | None`): 原始 BGR 帧。
|
||||
|
||||
### `FingerRuntime.process_frame` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
对一帧原始输入执行源图预处理、UV 展开,并把展开后的结果送入内部 `TactileProcessor`。这是 `examples/finger_pipeline.py` 中 profile 模式使用的方法。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **frame** (`np.ndarray`,必填): 原始输入帧。
|
||||
* **color** (`bool`,可选): 为 `False` 时走灰度展开流程;为 `True` 时保留 BGR 颜色展开。默认 `False`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `np.ndarray`: 展开后的触觉图像。`color=False` 时通常为单通道灰度图;`color=True` 时为 BGR 图。
|
||||
|
||||
### `FingerRuntime.grab_and_process` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
先读取一帧视频源,再调用 `process_frame()` 完成 UV 展开并推入内部触觉处理器。非 profile 模式下,`examples/finger_pipeline.py` 使用该方法作为主循环入口。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **color** (`bool`,可选): 是否保留颜色,默认 `False`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `(ok, unrolled)`:
|
||||
* **ok** (`bool`): 是否成功读取并处理。
|
||||
* **unrolled** (`np.ndarray | None`): 展开后的触觉图像。
|
||||
|
||||
### `FingerRuntime.get_img` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
`Sensor.get_img()` 风格的别名:内部执行一次采集和 UV 展开,并在成功时返回当前展开图,否则返回 `None`。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **color** (`bool`,可选): 是否保留颜色,默认 `False`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `np.ndarray | None`
|
||||
|
||||
### `FingerRuntime.warmup` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
先执行一次采集与 UV 展开,然后调用 `runtime.tactile.warmup()` 预热内部光流/HHD 相关缓存。`examples/finger_pipeline.py` 在进入主循环前会先执行一次 warmup。
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **color** (`bool`,可选): 是否保留颜色,默认 `False`。
|
||||
|
||||
#### 返回
|
||||
|
||||
* `(ok, unrolled)`:
|
||||
* **ok** (`bool`): 是否成功完成 warmup 前的采集。
|
||||
* **unrolled** (`np.ndarray | None`): warmup 时得到的展开图。
|
||||
|
||||
### `FingerRuntime.close` 方法
|
||||
|
||||
#### 描述
|
||||
|
||||
释放内部 `VideoCapture` 并调用 `runtime.tactile.disconnect()` 清理 `Sensor` 资源。程序退出前应调用。
|
||||
|
||||
### `TactileProcessor` 类别
|
||||
|
||||
#### 描述
|
||||
|
||||
`TactileProcessor` 是对内部 `Sensor("external", ...)` 的窄封装。它保留了 `compute_deformation()`、`compute_contact()`、`read_info()`、`reset()` 等常用接口,同时屏蔽了视频采集与 UV 展开细节。
|
||||
|
||||
#### 常用属性
|
||||
|
||||
* **raw_sensor**: 底层 `Sensor` 对象。
|
||||
* **info**: 与 `Sensor.info` 相同的枚举入口。
|
||||
* **img / flow / vnormal / vshear / depth_map**: 当前帧处理结果。
|
||||
* **mask_stats**: 有效 UV 区域统计信息;`examples/finger_pipeline.py` 会读取 `mask_stats["coverage"]` 输出当前掩码覆盖率。
|
||||
* **valid_mask**: 当前有效区域掩码。
|
||||
|
||||
#### 说明
|
||||
|
||||
除 `push_frame()` / `warmup()` 之外,`TactileProcessor` 的 `compute_deformation()`、`compute_contact()`、`read_info()`、`reset()` 调用方式与 `Sensor` 基本一致,因此可直接复用前文 `Sensor` 章节中的参数说明。
|
||||
|
||||
### 最小手指 pipeline 示例
|
||||
|
||||
下面的流程与 `examples/finger_pipeline.py` 一致:
|
||||
|
||||
```python
|
||||
import time
|
||||
import cv2
|
||||
import orisys
|
||||
from orisys.finger import FingerRuntime
|
||||
|
||||
runtime = FingerRuntime.open(
|
||||
config="finger",
|
||||
video=0,
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
runtime.warmup(color=False)
|
||||
tactile = runtime.tactile
|
||||
|
||||
try:
|
||||
while True:
|
||||
ok, unrolled = runtime.grab_and_process(color=False)
|
||||
if not ok:
|
||||
break
|
||||
|
||||
tactile.compute_deformation(check_motion=True, threshold=0)
|
||||
tactile.compute_contact()
|
||||
|
||||
fps, flow, vnormal, img, contour, centroid, depth_map, fnormal, fshearx, fsheary = tactile.read_info(
|
||||
tactile.info.FPS,
|
||||
tactile.info.VRAW,
|
||||
tactile.info.VNORMAL,
|
||||
tactile.info.IMG,
|
||||
tactile.info.CONTOUR,
|
||||
tactile.info.CENTROID,
|
||||
tactile.info.DEPTH,
|
||||
tactile.info.FNORMAL,
|
||||
tactile.info.FSHEARX,
|
||||
tactile.info.FSHEARY,
|
||||
)
|
||||
|
||||
arrows = orisys.util.draw_arrows(
|
||||
unrolled,
|
||||
flow,
|
||||
threshold=0.4,
|
||||
grid_spacing=10,
|
||||
arrow_scale=1.0,
|
||||
)
|
||||
cv2.imshow("UV Unrolled", arrows)
|
||||
|
||||
print(
|
||||
f"FPS={fps:.2f} normal={fnormal:.2f} "
|
||||
f"shear=({fshearx:.2f}, {fsheary:.2f}) "
|
||||
f"mask={tactile.mask_stats['coverage']:.2f}",
|
||||
end="\r",
|
||||
)
|
||||
|
||||
key = cv2.waitKey(1) & 0xFF
|
||||
if key == ord("q"):
|
||||
break
|
||||
if key == ord("r"):
|
||||
tactile.reset()
|
||||
finally:
|
||||
runtime.close()
|
||||
cv2.destroyAllWindows()
|
||||
```
|
||||
|
||||
### 关于 profile 模式
|
||||
|
||||
`examples/finger_pipeline.py` 中的 `--profile` / `--profile-every` 是示例脚本参数,不属于 `orisys` SDK 核心 API。本质上它只是把主循环拆成:
|
||||
|
||||
1. `runtime.read_source()`
|
||||
2. `runtime.process_frame(...)`
|
||||
3. `tactile.compute_deformation(...)`
|
||||
4. `tactile.compute_contact()`
|
||||
5. `tactile.read_info(...)`
|
||||
|
||||
并对各阶段做耗时统计。因此在业务代码中,也可以按同样方式自行插入计时逻辑。
|
||||
|
||||
## 可视化相关
|
||||
|
||||
### `util.draw_arrows` 方法
|
||||
|
||||
#### 输入参数
|
||||
|
||||
* **frame**:背景图像
|
||||
* **optical_flow**:光流场
|
||||
@@ -260,12 +533,12 @@ flow = sensor.read_info(sensor.info.VRAW)
|
||||
* **below_threshold_color**:RGB格式的`uint8`颜色,如(0, 0, 255)
|
||||
* **edge_margin**: 不显示箭头边距(pixel)
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
`np.ndarray` :箭头光流可视化图像
|
||||
|
||||
## 2. `util.draw_contact` 方法
|
||||
### `util.draw_contact` 方法
|
||||
|
||||
### 输入参数
|
||||
#### 输入参数
|
||||
|
||||
* **image**:背景图像
|
||||
* **contour**:轮廓
|
||||
@@ -273,41 +546,55 @@ flow = sensor.read_info(sensor.info.VRAW)
|
||||
* **point_color**:触碰点的颜色 (BGR)
|
||||
* **region_color**:轮廓的颜色 (BGR)
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
`np.ndarray` :箭头光流可视化图像
|
||||
|
||||
***
|
||||
## 配置标定相关
|
||||
|
||||
# 配置标定相关
|
||||
### `list_config` 方法
|
||||
|
||||
## 1. `list_config` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
列出内置传感器配置参数字典。
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
|
||||
* `list`: 内置传感器配置参数字典的名字列表。
|
||||
***
|
||||
## 2. `export_config` 方法
|
||||
### `export_config` 方法
|
||||
|
||||
### 描述
|
||||
#### 描述
|
||||
导出当前传感器实例使用的配置参数字典。
|
||||
|
||||
### 输入参数
|
||||
#### 输入参数
|
||||
* **config_name**: 需要导出的配置参数字典名字,`str`
|
||||
* **path**: 导出的路径,默认`"./config/exported_config.json"`
|
||||
|
||||
### 返回
|
||||
#### 返回
|
||||
|
||||
* `dict`: 导出的配置参数字典。
|
||||
|
||||
### 保存
|
||||
#### 保存
|
||||
|
||||
* 导出内置的`config_name`配置参数字典至`path`
|
||||
|
||||
***
|
||||
***
|
||||
### `export_sensor_bundle` / `export_sensor_bundles` 方法
|
||||
|
||||
#### 描述
|
||||
将已安装的 `sensors/<name>/` 传感器包复制到目标目录,供下游应用(如桌面 App)做本地可编辑配置,无需硬编码 wheel 路径。解析顺序与 `resolve_sensor_bundle` 相同(含 `importlib.resources` 打包布局)。
|
||||
|
||||
#### 输入参数
|
||||
* **name** / **names**: 单个名字,或名字列表;`export_sensor_bundles` 省略 `names` 时导出所有可解析的产品包(`box` / `mini` / `finger`)
|
||||
* **dest_dir**: 目标根目录;实际写入 `dest_dir/<name>/`
|
||||
* **include_assets**: 默认 `False`,仅复制 `sensor.json` 以及存在的 `force_calibration.json` / `manifest.json`;为 `True` 时复制完整目录树(含 `assets/`)
|
||||
* **overwrite**: 默认 `True`;为 `False` 且目标已存在时抛出 `FileExistsError`
|
||||
|
||||
#### 返回
|
||||
* `Path` 或 `list[Path]`:导出后的包目录路径
|
||||
|
||||
#### CLI
|
||||
```bash
|
||||
python -m orisys.export_sensors --dest ./out/sensors --names box,mini
|
||||
python -m orisys.export_sensors --dest ./out/sensors --include-assets
|
||||
```
|
||||
|
||||
## 常见问题解答 (FAQ)
|
||||
### Q1:为什么必须先调用`get_img()`,再调用`compute_deformation()`?
|
||||
@@ -328,9 +615,5 @@ A:当需要接触相关信息(如接触质心、轮廓或深度/散度图)
|
||||
A:`read_info()`按传入的`sensor.info`参数顺序返回一个元组。
|
||||
返回值顺序与参数顺序一一对应,与内部存储顺序无关。
|
||||
|
||||
### Q5:如果没有标定文件,或拼接结果不正确,会发生什么?
|
||||
A:若未指定或未找到标定文件,系统会在需要时创建新的拼接标定数据并保存到默认路径。
|
||||
拼接结果异常通常与标定数据、拼接配置或输入图像质量有关,建议重新生成拼接标定并检查配置参数。
|
||||
|
||||
### Q6: 运行报错:“AttributeError: 'Stitcher' object has no attribute 'pos_default'”
|
||||
### Q5: 运行报错:“AttributeError: 'Stitcher' object has no attribute 'pos_default'”
|
||||
A: 请检查视频源的编号是否正确,以及是否能正常打开视频源。在linux 使用`v4l2-ctl --list-devices` 或 Windows使用`Get-PnpDevice -PresentOnly | Where-Object { $_.Class -match '^Camera' }` 列出摄像头设备。
|
||||
@@ -0,0 +1,248 @@
|
||||
# 统一力标定模型契约
|
||||
|
||||
Purpose: reference. Scope: SDK 标定文件选择、SN 绑定和推断接口。
|
||||
状态:2026-10-18 已实现 schema 3 的空间/全局增益;ML 后端仅保留契约,尚未实现。
|
||||
保持 `cal_path` 与现有单点文件;新增空间模型和机器学习模型使用同一入口。
|
||||
|
||||
## 现状与边界
|
||||
|
||||
现有实现见 [calibration.py](../src/orisys/calibration.py)、
|
||||
[Sensor](../src/orisys/sensor.py)、[profile resolver](../src/orisys/sensors/calibration.py)。
|
||||
当前读 schema 1/2,按 SN 加载要求三个通道,转换只接收三个力标量。
|
||||
`ForceReadout.apply` 会分别转换滤波前后数据,不能直接挂接带状态的空间或 ML 模型。
|
||||
|
||||
| 所有者 | 责任 |
|
||||
|---|---|
|
||||
| 离线实验 / AutoCal | 采集、训练、验证、导出模型;将实验坐标转换为契约坐标 |
|
||||
| SDK | 选文件、校验身份与输入、准备特征、推断、输出逐轴有效性 |
|
||||
| 应用 | 提供 SN 与文件选择,展示状态;不自己套用第二次增益 |
|
||||
| 设计资产 | 几何与坐标定义;不提供每台设备的实测标定值 |
|
||||
|
||||
SDK 不训练模型、不导入应用代码、不读取训练 CSV,不运行文件内指定的 Python。
|
||||
实验 `finger_multical.experiment_model/1` 必须显式导出,不能直接当 SDK 文件加载。
|
||||
|
||||
## 文件选择:保留路径,文件决定模型
|
||||
|
||||
不增加 `model_type` 构造参数或全局活动模型。`Sensor`、`FingerRuntime.open` 和
|
||||
profile 创建处理器统一复用内部选择器;现有入口参数只映射为选择请求。
|
||||
|
||||
| 请求 | 文件与行为 |
|
||||
|---|---|
|
||||
| 非空 `cal_path` | 精确加载该文件;缺失、错误或不兼容时报错,不换文件 |
|
||||
| `calibration_dir` + `device_id` | 保持 `<dir>/<sensor_type>/<device_id>.json`;缺文件保持 raw 并报告 missing,不回退产品模板 |
|
||||
| 两者都未提供 | 保持产品包默认解析;有 descriptor 时用 `default_calibration`,旧包保留既有候选顺序 |
|
||||
| profile 的 `CalibrationSelection.NONE` | 明确不标定;不赋予 `cal_path="none"` 新语义 |
|
||||
|
||||
`cal_path` 与 `calibration_dir` 仍互斥。构造入口 `device_id` 对应文件 `identity.sn`,
|
||||
不新造另一个 SN 字段。未指定设备的录像可显式选文件,但实际 SN 一致性标记为 unknown。
|
||||
有 `device_id` 时严格匹配 `sensor_type`、`sn`;不根据相机编号猜 SN。
|
||||
保留 `device_id` 单独传入时的参数错误;目录本身不存在也报错,不能当成 SN 文件缺失。
|
||||
|
||||
```python
|
||||
# 沿用现有调用参数;cal_path 可以选择 schema 1/2/3 文件。
|
||||
runtime = FingerRuntime.open(
|
||||
config="finger", video="test.mp4",
|
||||
device_id="DD02-GL-001",
|
||||
cal_path="calibrations/finger/DD02-GL-001.json",
|
||||
)
|
||||
# 归零值来自本次无载窗口,不是模型内固定常量。
|
||||
runtime.set_calibration_reference({"fn": measured_zero_raw}, reference_id="unloaded-1")
|
||||
# 自动按 SN 选择仍使用 calibration_dir="calibrations",不同时传 cal_path。
|
||||
```
|
||||
|
||||
同一 SN 比较不同模型时,显式指定不同 JSON;默认 SN 文件只代表一个选定版本。
|
||||
不扫描目录选“最新模型”,不在每帧重新加载。
|
||||
|
||||
## 最小文件:schema 3
|
||||
|
||||
复用 `force_calibration.json` 文件角色,增加 schema 3;文件名不参与模型类型判断。
|
||||
以下是三点结构示例,数值和 SHA 占位值不是设备标定结果。
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 3,
|
||||
"unit": "N",
|
||||
"identity": {"sensor_type": "finger", "sn": "DD02-GL-001"},
|
||||
"pipeline": {
|
||||
"stitch_resolution": [700, 750],
|
||||
"dis_mode": 4,
|
||||
"hhd_downsample": 3,
|
||||
"source_scale": 0.42,
|
||||
"filters": {"measurement": {"flow_ema_alpha": 1.0}},
|
||||
"feature_signature_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"geometry_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"coordinates": {
|
||||
"frame": "autocal_flat",
|
||||
"unit": "px",
|
||||
"origin_xy": [511, 1021],
|
||||
"map_sha256": "0000000000000000000000000000000000000000000000000000000000000000"
|
||||
}
|
||||
},
|
||||
"model": {
|
||||
"type": "spatial_gain/1",
|
||||
"input_profile": "fn_delta_autocal_uv/1",
|
||||
"output_frame": "sdk_force/1",
|
||||
"outputs": ["fn"],
|
||||
"params": {
|
||||
"points_uv": [[-145, 604], [145, 604], [5, 430]],
|
||||
"gains_N_per_raw": [0.00015, 0.00020, 0.00025],
|
||||
"triangles": [[0, 1, 2]],
|
||||
"valid_force_range_N": [0.0, 3.0]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 规则 |
|
||||
|---|---|
|
||||
| `schema_version` | 整数 3;未知版本拒绝 |
|
||||
| `unit` | 固定 `N` |
|
||||
| `identity` | `sensor_type`、非空 `sn` 必填;文件位置不能替代身份 |
|
||||
| `pipeline` | 输入 profile 要求的实际处理参数、几何和坐标身份;缺少必需项或不匹配则拒绝 |
|
||||
| `model.type` | SDK 已注册的版本化标识;决定参数解析和推断,不是模块路径 |
|
||||
| `input_profile` | SDK 已实现的特征、单位、形状、预处理和时序契约 |
|
||||
| `output_frame` | `sdk_force/1` 延续对应处理 profile 的 FN/FX/FY 符号和基方向;不是世界坐标合力模长 |
|
||||
| `outputs` | 非空、无重复的 `fn/fx/fy` 子集;支持 FN-only,不冒充三轴标定 |
|
||||
| `params` | 模型类型私有参数;通用加载器不解释点数或网络层 |
|
||||
|
||||
`provenance` 可选,保存数据摘要、训练版本和验证指标;不作为运行时输入。
|
||||
其余未知语义字段拒绝,不能靠忽略字段误读新模型。新增模型类型不必升级外层 schema;
|
||||
修改输入或算法语义必须升级相应 `/1` 标识。
|
||||
|
||||
`geometry_sha256` 是所用计算 mesh 文件字节的 SHA-256;`map_sha256` 是 SDK 坐标映射定义的
|
||||
附件字节摘要,包含坐标转换结果,不能只比较图幅。pipeline 的必需键由输入 profile 固定,
|
||||
覆盖该 backend 所有影响测量的设置;只检查示例中的部分配置不算兼容校验。
|
||||
|
||||
新增 profile 必须定义闭合的签名字段表。首个 profile 的 `feature_signature_sha256` 摘要
|
||||
对象包含 `feature_version`、`processing_backend`、有效的图像预处理/光流/HHD 配置和计算资产哈希;
|
||||
不含文件绝对路径、SN、显示资源或力读数滤波。以 UTF-8、键排序、无多余空格的 JSON 计算 SHA-256。
|
||||
特征定义或单位改变须升级 `feature_version`,配置覆盖后重新计算,不能用配置文件名代替。
|
||||
实现由 `calibration_context.build_context` 统一生成,特征版本为 `orisys_hhd_raw/1`。
|
||||
|
||||
## 空间增益模型
|
||||
|
||||
`spatial_gain/1` 固定输出 `fn`,使用 `fn_delta_autocal_uv/1`:
|
||||
|
||||
- 输入为同一帧的未经过力读数滤波的 FN raw、有效会话零点及接触 U/V。
|
||||
上游光流滤波属于 pipeline 身份;不得混用不同特征链。
|
||||
`delta_raw`、U、V 为有限浮点标量,前者单位 raw,后两者为上述有符号像素。
|
||||
- SDK 会话准备 `delta_raw = fn_raw - zero_raw`,模型不再次扣零。
|
||||
零点必须来自明确的归零操作或已标注窗口;模型文件不保存某段录像的启动零点。
|
||||
- U 向右、V 向上,`x=origin_x+U`、`y=origin_y-V`。
|
||||
SDK 提供并校验坐标转换;不能让模型偷偷按比例换算,不能运行时依赖 AutoCal。
|
||||
- `points_uv` 为 N×2,`gains_N_per_raw` 长度 N;N≥3 且非共线、坐标唯一、增益有限且为正。
|
||||
`triangles` 为任意数量的三索引,必须构成支持点凸包内完整、无重叠的非退化三角网。
|
||||
保存连接关系以避免重新剖分引入变化;点数由数组决定,不固定为 9/15。
|
||||
- 对三角形内查询,`g=Σw_i*g_i`,`fn_N=g*delta_raw`;不插值倒数,不外推。
|
||||
未覆盖、位置失败、归零无效均返回无效,不能偷偷改用 M0。
|
||||
- `valid_force_range_N` 必填。估计超范围时保留数值并标记 `out_of_range`,不截断。
|
||||
|
||||
FingerRuntime 使用 `artifacts` 中 `role="coordinate_map"` 的只读 NPZ:
|
||||
`uv_by_output_px` 为 H×W×2 有符号 UV,`valid_mask` 为 H×W bool;可附 `origin_xy` 与
|
||||
`flat_mapping_sha256`。`pipeline.output_domain_sha256` 绑定实际展开的 remap、mask 和 world_xyz;
|
||||
形状、几何或特征身份不同则在打开视频前拒绝。离线直接提供 UV 的模型可以没有映射附件。
|
||||
|
||||
单点/两点数据不自动切换算法;可导出独立的 `global_gain/1`,参数为各输出轴的增益和有效范围,
|
||||
输入 profile 为 `force_delta/1`,由 SDK 按同一归零语义准备对应 raw 增量。
|
||||
这不同于旧曲线的绝对 raw 输入;兼容适配器不得替旧文件额外扣零。
|
||||
|
||||
## 机器学习扩展
|
||||
|
||||
同一 SN、路径和外层字段不变,只替换 `model`。下面只预留 `onnx/1` 示例,
|
||||
不表示 SDK 已支持 ONNX;后端实现和输入 profile 验证通过前必须拒绝加载。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "onnx/1",
|
||||
"input_profile": "fn_delta_autocal_uv/1",
|
||||
"output_frame": "sdk_force/1",
|
||||
"outputs": ["fn"],
|
||||
"params": {
|
||||
"input_tensor": "features",
|
||||
"output_tensor": "force_N",
|
||||
"feature_order": ["delta_raw", "u", "v"],
|
||||
"dtype": "float32",
|
||||
"shape": [1, 3],
|
||||
"valid_force_range_N": [0.0, 3.0]
|
||||
},
|
||||
"artifacts": [
|
||||
{
|
||||
"role": "weights",
|
||||
"path": "DD02-GL-001/model.onnx",
|
||||
"sha256": "0000000000000000000000000000000000000000000000000000000000000000"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
此标量 profile 的规范输入即上述顺序、类型、形状;输出 float32 `[1, len(outputs)]`,
|
||||
轴序严格按 `outputs`。归一化及单位转换封装在已验证的模型图内,不由应用另做。
|
||||
`fn_delta_autocal_uv/1` 只允许 `outputs=["fn"]`,不能凭 FN 特征宣称三轴已标定。
|
||||
需要图像、完整光流或历史序列时新增输入 profile,明确 H/W、通道、掩码、单位和时间窗口;
|
||||
没有对应特征的处理器拒绝加载,不能用零张量凑齐输入。
|
||||
|
||||
`artifacts` 只在有外部文件时必填;路径相对所选 JSON 的父目录,禁止绝对路径、URL、
|
||||
`..` 或链接逃逸。校验 SHA-256、角色和文件完整性后才初始化受支持的后端。
|
||||
不加载 pickle、任意 Python 或模型自带扩展库;缺推断依赖须明确报错,不自动安装。
|
||||
文件与所有附件共同构成内容指纹,不能只哈希 JSON。
|
||||
附件间依赖也须逐项声明和校验,未声明的外部权重不可加载。
|
||||
|
||||
## 每帧推断与有效性
|
||||
|
||||
内部统一生命周期如下,`calibration.load` 分流到旧曲线或新模型:
|
||||
|
||||
```text
|
||||
load(selected_file, device_identity, pipeline) -> CalibrationModel
|
||||
model.predict(frame_id, timestamp_s, profile_features, reference_id) -> CalibrationResult
|
||||
model.reset(reference_id)
|
||||
|
||||
CalibrationResult:
|
||||
values_N: {axis: finite_float | null}
|
||||
status: {axis: ok | no_contact | invalid_input | zero_unavailable |
|
||||
coordinate_unmapped | outside_coverage | out_of_range}
|
||||
```
|
||||
|
||||
FN 空间模型输入仅允许单一接触;当前定位器只返回一个中心,无法证明不存在多接触,
|
||||
不能宣称已标定总力。全局 `force_delta/1` 不要求定位,未提供接触状态不等同于无接触。
|
||||
无接触、失败帧的校准输出为空,raw 另行保留;缺失值不是 0 N。
|
||||
`out_of_range` 有数值但不在验证范围,其余失败状态无数值;未声明轴不可用、单位仍为 raw。
|
||||
不把某轴 raw 填进 `values_N`,也不只靠全局 `force_unit` 判断三轴单位。
|
||||
schema 3 的旧数值读数接口以 NaN 表示已声明轴的无效输出,并同步暴露上述状态;
|
||||
`force_calibrated` 表示已加载模型能力,不表示当前帧有效。
|
||||
|
||||
一个模型实例属于一个流;按帧顺序每帧推断一次,读数和录像复用结果。
|
||||
清零、重连、倒播或更换模型时重置状态;不同 SN 不共享可变状态。
|
||||
schema 3 先做上游测量与归零、模型推断,再做有效 N 读数的显示滤波;失败帧不进滤波历史。
|
||||
schema 1/2 仍保留旧滤波/转换顺序。状态模型不能塞入会调用两次的旧 `convert` 回调。
|
||||
|
||||
加载诊断至少包含路径、来源、SN 校验状态、模型/输入 profile、轴集合、内容指纹和失败原因。
|
||||
加载不匹配失败与逐帧暂时无效分开;替换时先完成加载校验,失败不得破坏原活动模型。
|
||||
Sensor 提供 `bind_calibration_context`、`set_calibration_reference`、`apply_calibration_frame`;
|
||||
FingerRuntime 自动绑定真实上下文,并转发归零。`FrameSnapshot.force_units` 和
|
||||
`calibration_status` 随帧保存,不能用稍后读取的 Sensor 状态解释旧帧。
|
||||
归零与换模型须在计算线程或暂停处理时调用,不承诺并发修改安全。
|
||||
|
||||
## 兼容、实现位置与验收
|
||||
|
||||
新读取器并存支持 schema 1/2/3;旧文件不重写,SN 校验的旧三通道要求保持。
|
||||
schema 3 按声明轴校验,允许 FN-only;旧 SDK 遇 schema 3 必须拒绝,不能降级猜读。
|
||||
回退通过恢复旧文件的 `cal_path`;SN 默认文件切换前保留旧副本,不覆盖唯一证据。
|
||||
|
||||
| 修改位置 | 最小工作 |
|
||||
|---|---|
|
||||
| `calibration.py` | schema 分流、身份和附件校验;旧曲线适配器 |
|
||||
| `sensors/calibration.py` | 统一选择器与指纹;支持 SN 请求 |
|
||||
| 新 `calibration_models.py` | 受控类型表、空间增益与模型生命周期;ML 后端另行实现 |
|
||||
| `sensor.py`、`finger/runtime.py`、处理器适配层 | 提供同帧位置/零点,缓存逐轴结果和状态 |
|
||||
| Sensor 与 `ForceReadout` | 复用读数滤波器;schema 3 在推断后逐轴滤波,旧路径不变 |
|
||||
| 离线导出器 | 实验模型到 schema 3;SDK 不依赖实验目录 |
|
||||
|
||||
当前导出入口为 workspace 的 `scripts/finger_multical/export_sdk.py`,输出已有文件时拒绝覆盖。
|
||||
`data/dd02-gl/force_calibration.v3.json` 与同名 `.coordinates.npz` 是本次实验交付,
|
||||
`identity.sn="dd02-gl"` 为实验标识,未从硬件读取;原三边缘点与非单调点警告保留于 provenance。
|
||||
它不是生产包默认文件,需显式 `cal_path` 选择。
|
||||
|
||||
实现验收必须覆盖:旧结果与单位不变;显式路径/SN/默认/关闭选择;错误 SN 不串用;
|
||||
3/14/更多点、退化网格与凸包外;节点和插值预测;零点不重复扣除;缺坐标/无接触;
|
||||
FN-only 的 FX/FY 不冒充 N;未知模型、附件损坏和依赖缺失;状态模型每帧只调用一次及 reset。
|
||||
另以独立录像验证物理准确度,契约或单元测试通过不代表标定合格。
|
||||
@@ -0,0 +1,74 @@
|
||||
# Code review: the automated part
|
||||
|
||||
Four gates existed before this document (the test suites, `apps/studio/tools/ui_check.py`, the
|
||||
real-GL smoke tests, the binding-purity test) and none of them is a linter. This adds the missing
|
||||
*static* layer, deliberately small.
|
||||
|
||||
## Why it exists, in one measured fact
|
||||
|
||||
Until now the three repos had **no Ruff configuration at all**. `ruff check .` therefore ran
|
||||
whatever the installed version enables by default — Ruff 0.16 enables a very broad set
|
||||
(`UP`/`RUF`/`PLR`/`S`/`TRY`/`PERF`/`FURB`/`YTT`/`ASYNC`…) — so the same tree produced **1038 /
|
||||
438 / 117** findings in `orisys-app` / `orisys-sdk` / `orisys-sensor-design`, and a different
|
||||
Ruff version produced different ones. A gate built on that is not a gate. `ruff.toml` now pins it.
|
||||
|
||||
## What is gated, and what is deliberately not
|
||||
|
||||
| rule | meaning | gated? | why |
|
||||
|---|---|---|---|
|
||||
| `BLE001` | broad `except Exception` | **no** | A broad catch is the *right* choice around device I/O, GL teardown and camera probing. This codebase already records that choice inline as `# noqa: BLE001 - <reason>`. Gating it would produce ~100 findings whose only correct fix is "add a reasoned suppression". |
|
||||
| `S110`, `S112` | handler is only `pass` / `continue` | **yes** | The complaint is not the breadth of the catch, it is that the failure becomes **invisible**. Silence is acceptable only with a reasoned suppression or a handler that reports/re-raises. |
|
||||
| `E722` | bare `except:` | **yes** | Strictly worse than `except Exception:` — it also swallows `KeyboardInterrupt`/`SystemExit`. |
|
||||
| `F` | unused import, redefinition, undefined name, unused variable | **yes** | Real defects: `F821`/`F822` are latent `NameError`s, `F811` can be a shadowed definition. |
|
||||
| `B023` | closure over a loop variable | **yes** | Late binding silently produces wrong geometry in the headless Blender scripts. |
|
||||
| `PLW1510` | `subprocess.run` without `check` | **yes** | A swallowed tool failure. |
|
||||
| `RUF100` | unused suppression | reported, **never auto-fixed** | The `- reason` in a `noqa` comment is documentation. A machine must not delete it. Triage by hand. |
|
||||
| `UP`, `I`, `PIE`, `RET501`, `PLR1711`, `SIM114`, `RUF022`, `RUF023` | mechanical modernisation | in the lint config, **not** in the gate | Applied once by `--fix`; gating them would only churn AI-authored diffs. |
|
||||
|
||||
`E402` (module import not at top of file) is **selected but not gated**. `import` after
|
||||
`os.environ[...]` is deliberate in this codebase — the environment must be set before Qt/cv2/the SDK
|
||||
load — so gating it would be pure noise. But it is the only rule that would catch an *accidental*
|
||||
import moved above that setup, so it stays visible. When you touch one of those files, record the
|
||||
intent as `# noqa: E402 - <reason>`; `E701`/`E702` are style, which `ruff format` owns. A gate that
|
||||
cries wolf gets ignored, which is worse than no gate.
|
||||
|
||||
## The three commands
|
||||
|
||||
```powershell
|
||||
# 1. The agent loop, after every edit — mechanical, no review needed.
|
||||
ruff check <changed files> --fix # never --unsafe-fixes
|
||||
ruff check <changed files> --output-format json # what remains, machine-readable
|
||||
|
||||
# 2. The gate before a change lands. Changed files only, machine-readable, non-zero on findings.
|
||||
python tools/lint_gate.py # vs HEAD
|
||||
python tools/lint_gate.py --base origin/main
|
||||
python tools/lint_gate.py --all # gate rules over the whole repo
|
||||
python tools/lint_gate.py --json
|
||||
|
||||
# 3. Formatting, changed files only — never a whole-tree rewrite mid-diff.
|
||||
ruff format <changed files>
|
||||
```
|
||||
|
||||
`$RUFF` may point at a specific ruff binary; otherwise `tools/lint_gate.py` uses `PATH` or the one
|
||||
beside the interpreter. **Pin the version**: the gate prints the version it ran, and findings differ
|
||||
between versions, so a runner must install the same one the developers use (`ruff==0.16.9`).
|
||||
|
||||
## What this found when it was introduced
|
||||
|
||||
Filtering to rules that can indicate a defect produced ~18 candidates rather than style noise —
|
||||
which is the argument for the narrow set:
|
||||
|
||||
- six latent `NameError`s (`np`, `sys`, `Callable`, `FrameIdentity`, `tessellate_polygon`, and an
|
||||
`__all__` entry naming a function that does not exist) — none caught by any test;
|
||||
- a **duplicated** function definition that silently shadowed the first;
|
||||
- `lower == lower` / `upper == upper` in a range check, and the same pattern inside a **test
|
||||
assertion that could never fail**;
|
||||
- 13 closures over loop variables in the headless Blender/patch scripts;
|
||||
- two `subprocess.run` calls whose failure was silently ignored.
|
||||
|
||||
## Follow-up deliberately not done yet
|
||||
|
||||
`PLR0124` ("name compared with itself") is not in `select`. Eleven of its sites are the deliberate
|
||||
`value != value` NaN idiom and must become `math.isnan(...)` (or carry a reasoned suppression)
|
||||
before the rule can be gated — otherwise it is noise. Once those are converted, add `PLR0124` to
|
||||
`select` and to `GATE_SELECT` in `tools/lint_gate.py`.
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
# 滤波:三级、预设与可追溯性
|
||||
|
||||
`json.load` 不支持注释(见 `src/orisys/configs.py`),所以"建议的滤波"**不写在 JSON 注释里**,
|
||||
而是写成**代码里的预设**(`src/orisys/sensor.py::FILTER_PRESETS`)+ 这份文档 +
|
||||
`examples/filters/` 里可直接复制的配置。预设是可执行、可回显、可审计的;注释不是。
|
||||
|
||||
---
|
||||
|
||||
## 1. 三级滤波,位置不同、性质不同
|
||||
|
||||
| 级 | 配置键 | 作用位置 | 性质 |
|
||||
|---|---|---|---|
|
||||
| 光流时间 EMA | `flow_ema_alpha` | `compute_flow()` 内,**HHD 之前** | **改变测量**:改 `vnormal`/接触统计与 `_force_raw` 的值。两个 `alpha` 不同的录制**不可比** |
|
||||
| 力中值 | `force_median_window` | `_apply_force_filter()`:raw → 中值 → 低通 → 标定 | **读数滤波**:删除单点尖峰(因果滑动中值) |
|
||||
| 力低通 | `force_lowpass_cutoff_hz` / `_order` / `_sample_rate_hz` | 同上(中值之后) | **读数滤波**:因果 Butterworth IIR(`sosfilt` + 持久状态) |
|
||||
|
||||
因此 `filters_state()` 把它们分成两组:`measurement`(测量身份)与 `readout`(读数滤波)。
|
||||
标定/可比性检查应比对 `measurement`,记录里两者都记。
|
||||
|
||||
## 2. 取值规则(自描述,无隐藏覆盖)
|
||||
|
||||
- **每个级由自己的值开关**:`flow_ema_alpha >= 1`、`force_median_window <= 1`、
|
||||
`force_lowpass_cutoff_hz <= 0` 都是"关"。
|
||||
- **配置没写的键 = 关**:缺省不是"我们觉得合理的那档",而是"不滤波"。因此**没有 `filters`
|
||||
段的配置文件什么都不滤**,出厂 bundle 也把三个值显式写成全关 —— 想让哪一级跑起来,
|
||||
必须在文件里写出来(数值或 `preset`)。这样"文件说的"和"设备跑的"才总是一致。
|
||||
- **非法值 → 关**,绝不回落到"默认开"。
|
||||
- `filters.enabled` 是**废弃别名**,显式 `false` 表示"三级全关";没有任何开关能覆盖已写的值。
|
||||
- `filters.preset` 可引用下面的预设;**显式键优先于预设值**(可从预设出发只改一个旋钮)。
|
||||
- 任何时刻的**真值**是 `Sensor.filters_state()`,不是配置文件 —— 运行时 `configure()` 可以改它。
|
||||
|
||||
## 3. 预设(`FILTER_PRESETS`)
|
||||
|
||||
| 预设 | `flow_ema_alpha` | `force_median_window` | `force_lowpass_cutoff_hz` | 延迟 @30fps | 用途 |
|
||||
|---|---|---|---|---|---|
|
||||
| `off` | 1.0 | 1 | 0.0 | 0 帧 | 原始读数;文件说什么就跑什么 |
|
||||
| `low_latency` | 1.0 | 3 | 0.0 | ≈1 帧(33 ms) | 延迟敏感,只去尖峰 |
|
||||
| `balanced` | 0.5 | 3 | 0.0 | ≈2 帧(67 ms) | 建议起步档:EMA 干掉力级滤波够不到的光流抖动 |
|
||||
| `low_noise` | 0.3 | 3 | 10.0 | ≈3 帧(100 ms) | 底噪优先,读数再低通一道 |
|
||||
|
||||
延迟换算 @60fps 减半;数字来自 `_perf/`(该一次性目录已从 `orisys-sdk` 移除:幸存的脚本与片源
|
||||
迁到 `D:\orisys_ws\docs\performance\perf-measurements\`,其余已永久丢失;对照表见
|
||||
`D:\orisys_ws\docs\performance\orisys-sdk-perf-plan.md`,其中仍保留全部实测数字)。
|
||||
≥1 帧的延迟与相位差是**因果滤波的代价**:旧显示端曾用零相位 `filtfilt`,那需要"未来",
|
||||
无法在采集里做,也让"显示 == 记录"不可能——所以现在只有一条因果链。
|
||||
|
||||
## 4. 怎么用
|
||||
|
||||
```jsonc
|
||||
// sensors/mini/sensor.json(片段;完整可复制版本见 examples/filters/)
|
||||
"filters": { "preset": "balanced" }
|
||||
```
|
||||
```python
|
||||
# 或运行时(UI 的齿轮就走这条路;不重连相机)
|
||||
sensor.configure(flow_ema_alpha=0.5, force_median_window=3, lowpass_cutoff_hz=0.0)
|
||||
print(sensor.filters_state()) # 实际生效值 + preset + applied 标志
|
||||
```
|
||||
UI 想列出可选档位:`from orisys.sensor import available_filter_presets`。
|
||||
|
||||
## 5. 两个 buffer:滤波前 / 滤波后
|
||||
|
||||
```python
|
||||
t = sensor.force_triplets()
|
||||
t["filtered"]["raw"], t["filtered"]["newton"] # 记录与显示用的值(同一套)
|
||||
t["unfiltered"]["raw"], t["unfiltered"]["newton"] # 中值/低通之前;同帧、同标定系数
|
||||
```
|
||||
- `filtered.raw` = 链的输出(标定前),`unfiltered.raw` = 设备计数;
|
||||
- 两个 `newton` 用同一份 `force_calibration.json`,差异**只**来自滤波链;
|
||||
- 实测:注入 `10,10,10,999,10,10`(中值窗口 3)→ `filtered.raw` 在尖峰帧为 `10.0`、
|
||||
`unfiltered.raw` 为 `999.0`。
|
||||
|
||||
## 6. 可追溯性清单
|
||||
|
||||
1. 录制元数据写 `sensor.filters_state()` 的**生效值**(不是配置文件内容),并含 `preset` 与
|
||||
`"source": "sensor.filters_state"` + `sdk_version`。
|
||||
2. 界面显示"实际生效"而不是"我填的值";两者不一致时以实际为准并提示。
|
||||
3. 记录里主列 = 滤波后(与界面一致),未滤波列另存;列结构与格式版本可区分。
|
||||
4. 标定/可比性指纹用 `measurement` 组;`readout` 组变化只记录、不拦截。
|
||||
5. 双份数据在同一帧、同一份标定系数下产生,差异可完全归因于滤波链。
|
||||
|
||||
## 7. 指纹:标定文件里怎么记(已实现)
|
||||
|
||||
`orisys.calibration` 负责把滤波设置写进标定文件的 `meta`,并在读回时还原:
|
||||
|
||||
```python
|
||||
from orisys.calibration import filters_fingerprint, filter_mismatches, pipeline_from_sensor_json
|
||||
|
||||
pipeline = pipeline_from_sensor_json(config_path, filters_state=sensor.filters_state())
|
||||
# {"stitch_resolution": …, "dis_mode": …, "hhd_downsample": …, "config_ref": …,
|
||||
# "filters": {"measurement": {"flow_ema_alpha": 0.5},
|
||||
# "readout": {"force_median_window": 3, "force_lowpass_cutoff_hz": 0.0},
|
||||
# "preset": "balanced", "source": "sensor.filters_state"}}
|
||||
```
|
||||
|
||||
写入的文件长这样(`meta` 里同时保留扁平键供旧读者按名取用):
|
||||
|
||||
```json
|
||||
"meta": {
|
||||
"filters": {
|
||||
"measurement": {"flow_ema_alpha": 0.5},
|
||||
"readout": {"force_median_window": 3, "force_lowpass_cutoff_hz": 0.0},
|
||||
"preset": "balanced",
|
||||
"source": "sensor.filters_state"
|
||||
},
|
||||
"flow_ema_alpha": 0.5,
|
||||
"force_median_window": 3
|
||||
}
|
||||
```
|
||||
|
||||
规则:
|
||||
|
||||
- **生效值优先**:传了 `filters_state=` 就用它;没传才读文件的 `filters` 段,并标成
|
||||
`"source": "sensor.json.filters"`(那是"意图",不一定等于设备正在做的事)。
|
||||
- **不猜**:来源没写的键**不出现**在指纹里。`{}` 表示"没有任何记录",绝不表示"大概是关的"。
|
||||
旧文档写的扁平键(`meta.flow_ema_alpha` …)在读回时会被折进对应的组,因此仍可比。
|
||||
- **`enabled: false`** 只作为"全部关"的废弃别名保留(`flow_ema_alpha=1`、中值 `1`、低通 `0`)。
|
||||
- **分组判读**:`calibration.check(doc, pipeline=…)` 把两组的差异分开报告 —— measurement 组的
|
||||
差异进 `warnings`(措辞含 `not comparable`,即"两份记录不可比"),readout 组的差异进
|
||||
`notes`(只记录,不拦截)。`CheckResult.notes` 是为此新增的字段。
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: Get started with Orisys SDK development
|
||||
description: Set up a source checkout, run a first test, and find the right example.
|
||||
---
|
||||
|
||||
# Get started with Orisys SDK development
|
||||
|
||||
This guide is for contributors working from the SDK source checkout. Obtain the
|
||||
repository URL from your team, clone it, and change into the `orisys-sdk` directory.
|
||||
If you only need to use a published wheel, follow the
|
||||
[Chinese installation guide](get_started.md). Run the commands below from the
|
||||
repository root.
|
||||
|
||||
## First working loop
|
||||
|
||||
Use Python 3.10, 3.11, or 3.12. The existing README uses conda; an isolated
|
||||
environment keeps another installed `orisys` package from shadowing this checkout.
|
||||
|
||||
```bash
|
||||
conda create -n orisys-dev python=3.10
|
||||
conda activate orisys-dev
|
||||
python -m pip install -e . pytest
|
||||
python -c "import orisys; print(orisys.__version__)"
|
||||
python -m pytest -m "not slow and not hardware" tests/test_calibration.py
|
||||
```
|
||||
|
||||
The editable install makes changes under `src/orisys/` visible without rebuilding a
|
||||
wheel. `pytest.ini` adds `src/` to the test import path. Before debugging a strange
|
||||
import failure, check that `python` and `python -m pip` point to the same environment.
|
||||
The full non-hardware suite is:
|
||||
|
||||
```bash
|
||||
python -m pytest -m "not slow and not hardware"
|
||||
```
|
||||
|
||||
Some tests need a recorded clip. `tests/conftest.py` looks for a repository sample or
|
||||
the path in `ORISYS_TEST_VIDEO`; a missing clip can skip those tests. Inspect the test
|
||||
summary rather than treating a run with unexpected skips as coverage.
|
||||
|
||||
## Run a pipeline
|
||||
|
||||
The examples accept a camera index or, where documented, a video path. Start with
|
||||
the example matching your sensor. A camera example needs the device and its bundle;
|
||||
the test command above does not need physical hardware.
|
||||
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c box
|
||||
python examples/basic_pipeline.py -v 0 -c mini
|
||||
python examples/finger_pipeline.py --camera_id 0 --config finger
|
||||
```
|
||||
|
||||
For a video-only first run, see the exact arguments in
|
||||
[examples/README.md](../examples/README.md). GUI examples need the optional GUI
|
||||
dependencies (`python -m pip install -e ".[gui]"`). GPU support is optional; the
|
||||
declared extras are `cuda12` and `cuda13` in [pyproject.toml](../pyproject.toml).
|
||||
Start on CPU unless your task requires a GPU backend.
|
||||
|
||||
## Find the owner of a change
|
||||
|
||||
| What you are changing | Start here |
|
||||
| --- | --- |
|
||||
| SDK API or algorithms | `src/orisys/`, then a focused test in `tests/` |
|
||||
| Product configuration and shipped assets | `sensors/<product>/`; see [sensor bundles](sensor-bundles.md) |
|
||||
| Example behavior | `examples/` and [examples/README.md](../examples/README.md) |
|
||||
| Build or release tooling | `build/` and [build/README.md](../build/README.md) |
|
||||
| Studio UI | The separate `orisys-app` repository |
|
||||
|
||||
`config/` holds legacy configurations. New product-bundle work belongs under
|
||||
`sensors/`. For a physical device's calibration, use a user-selected calibration
|
||||
directory and `device_id`; the [SDK README](../README.md#configuration) describes
|
||||
the path and validation contract. Do not commit individual sensor calibration data
|
||||
to the generic product bundle by accident.
|
||||
|
||||
After a change, run the focused tests first, then the relevant non-hardware suite.
|
||||
If a change requires a camera, record the product, configuration, input, and observed
|
||||
result in your handoff. See [publishing](publishing.en.md) when preparing a release;
|
||||
ordinary development does not require a wheel build or a PyPI account.
|
||||
+131
-67
@@ -1,80 +1,104 @@
|
||||
叠动科技软件使用手册
|
||||
版本: 0.4.0
|
||||
更新日期: 2026年4月16日
|
||||
---
|
||||
title: 快速开始
|
||||
description: 安装并运行 Orisys SDK 的基础步骤
|
||||
---
|
||||
|
||||
# 简介
|
||||
本文件介绍如何上手和运行叠动视触觉传感器配套的开发套件。本套件是配合叠动视触觉传感器进行图像处理,计算形变、力以及接触信息。它可以从摄像头或视频文件中读取图像,处理后打印和展示处理后信息。
|
||||
# 快速开始
|
||||
|
||||
## 简介
|
||||
本文件介绍如何上手和运行叠动视触觉传感器配套的SDK,进行图像处理,计算形变、力以及接触信息,接受从摄像头或视频文件中读取图像,处理后打印和展示处理后信息。
|
||||
支持操作系统:Windows、Ubuntu
|
||||
建议Python 版本: 3.10
|
||||
建议Python 版本: 3.10, 3.11, 3.12
|
||||
|
||||
# 基础环境设置
|
||||
## 安装 Python 与 Anaconda
|
||||
1. 访问 [Anaconda 官网](https://www.anaconda.com/download)下载适合您操作系统的安装包(Windows、Linux)。
|
||||
2. 按照提示完成安装。(Widnows:建议在安装过程中勾选“将 Anaconda 添加到系统环境变量”选项,方便后续使用)。
|
||||
3. 安装完成后,打开命令行终端(Windows 用户可在”Anaconda Navigator”打开“Anaconda Prompt”),输入 `conda info` ,若能显示 conda 的信息,则说明安装成功。
|
||||
项目链接:
|
||||
|
||||
## 创建conda虚拟环境
|
||||
- PyPI 安装页:[https://pypi.org/project/orisys/](https://pypi.org/project/orisys/)
|
||||
- 项目文档/代码:[https://github.com/Orisys-Tech/orisys-sdk](https://github.com/Orisys-Tech/orisys-sdk)
|
||||
|
||||
## 基础环境设置
|
||||
### 安装 Python 与 Anaconda
|
||||
1. 访问 [Anaconda 官网](https://www.anaconda.com/download) 下载适合您操作系统的安装包(Windows、Linux)。
|
||||
2. 按照提示完成安装。
|
||||
3. 安装完成后,打开命令行终端(Windows 用户可在 “Anaconda Navigator” 打开 “Anaconda Prompt”),输入 `conda info`,若能显示 conda 的信息,则说明安装成功。
|
||||
|
||||
### 创建 conda 虚拟环境
|
||||
虚拟环境相当于一个独立的 Python 工作空间,可以避免不同项目所需的库版本冲突。
|
||||
打开命令行终端(或Anaconda Prompt),依次输入以下命令(以 python 3.10 为例):
|
||||
``` bash
|
||||
打开命令行终端(或 Anaconda Prompt),依次输入以下命令(以 Python 3.10 为例):
|
||||
```bash
|
||||
conda create -n orisys python=3.10
|
||||
conda activate orisys
|
||||
```
|
||||
|
||||
激活后,命令行提示符前会出现 `(orisys)` 字样,表示当前已进入虚拟环境。
|
||||
|
||||
## 创建工作区文件夹
|
||||
“Anaconda Prompt”启动后的默认工作路径在用户根文件夹中,建议创建工作区文件夹方便管理,可以手动创建或者运行`mkdir <path>`命令:, 例如
|
||||
### 创建工作区文件夹
|
||||
“Anaconda Prompt” 启动后的默认工作路径在用户根文件夹中,建议创建工作区文件夹方便管理。可以手动创建文件夹,也可以运行 `mkdir <path>` 命令。例如:
|
||||
|
||||
``` bash
|
||||
```bash
|
||||
mkdir orisys_ws
|
||||
```
|
||||
|
||||
创建后如果要更换工作区,使用 `cd <path>`命令,例如:
|
||||
创建后如果要更换工作区,使用 `cd <path>` 命令,例如:
|
||||
|
||||
``` bash
|
||||
```bash
|
||||
cd "C:\Users\orisys\orisys_ws"
|
||||
```
|
||||
<div style="page-break-after: always;"></div>
|
||||
|
||||
## 安装 CUDA 和 cuDNN(可选)
|
||||
### 安装 CUDA(可选)
|
||||
如果您拥有 NVIDIA 显卡,并希望利用 GPU 加速计算,则需要安装 CUDA 工具包和 cuDNN。
|
||||
|
||||
1. 方法一(官网下载):
|
||||
访问 [NVIDIA CUDA Toolkit官网](https://developer.nvidia.com/cuda/toolkit)下载对应系统的 CUDA Toolkit 安装包,并按照提示安装, 注意12.X 和 13.X 的版本对应不同的 CuPy 库版本。
|
||||
访问 [NVIDIA CUDA Toolkit 官网](https://developer.nvidia.com/cuda/toolkit) 下载对应系统的 CUDA Toolkit 安装包,并按照提示安装。注意 12.x 和 13.x 的版本对应不同的 CuPy 库版本。
|
||||
|
||||
2. 方法二(使用 Anaconda 安装,推荐):
|
||||
在已激活的 orisys 环境中直接运行以下命令(以 CUDA 12.9 为例):
|
||||
`conda install nvidia/label/cuda-12.9.0::cuda-toolkit nvidia::cudnn`
|
||||
该方法会自动配置好 CUDA 和 cuDNN,无需手动下载。
|
||||
|
||||
```bash
|
||||
conda install nvidia/label/cuda-12.9.0::cuda-toolkit nvidia::cudnn
|
||||
```
|
||||
|
||||
该方法会自动配置好 CUDA,无需手动下载。
|
||||
|
||||
注意:如果您的计算机没有 NVIDIA 显卡,或者不想使用 GPU 加速,可以跳过此步骤。套件也支持纯 CPU 运行(但速度可能较慢)。
|
||||
|
||||
# 安装 orisys SDK
|
||||
## 安装 orisys SDK
|
||||
|
||||
orisys SDK 以 .whl 文件形式提供,您需要根据操作系统、Python 版本和硬件平台选择正确的文件。
|
||||
推荐优先从 PyPI 安装。PyPI 会自动选择适合当前 Python 版本和操作系统的安装包,适合大多数用户。
|
||||
|
||||
## 下载正确的 .whl 文件
|
||||
### 使用 PyPI 安装(推荐)
|
||||
确认已经激活 `orisys` 虚拟环境后,运行:
|
||||
|
||||
```bash
|
||||
pip install orisys
|
||||
```
|
||||
|
||||
如果需要升级到最新版本,运行:
|
||||
|
||||
```bash
|
||||
pip install --upgrade orisys
|
||||
```
|
||||
|
||||
如果当前网络无法访问 PyPI,或需要安装客服提供的指定版本,可以使用下方 `.whl` 文件安装方式。
|
||||
|
||||
### 使用 .whl 文件安装(离线或指定版本)
|
||||
从官方渠道获取 .whl 文件,文件名格式通常为:
|
||||
orisys-{版本}-{Python版本}-{Python版本}-{操作系统}_{硬件架构}.whl
|
||||
例如:orisys-0.4.0-cp310-cp310-win_amd64.whl 表示:
|
||||
例如 `orisys-<版本>-cp310-cp310-win_amd64.whl` 表示:
|
||||
|
||||
版本:0.4.0
|
||||
版本:`<版本>`(以实际下载的文件名为准,可用 `python -c "import orisys; print(orisys.__version__)"` 确认)
|
||||
Python 版本:3.10(cp310)
|
||||
操作系统:Windows(win)
|
||||
硬件架构:64位(amd64)
|
||||
|
||||
请根据您的系统选择对应的文件,或者咨询客服索取:
|
||||
请根据您的系统选择对应的文件,或者咨询客服索取。
|
||||
|
||||
## 安装 .whl 文件
|
||||
将下载好的 .whl 文件放在一个文件夹中(如上述`orisys_ws`文件夹),打开命令行(确保已激活 orisys 环境或使用"Anaconda Prompt"),使用 `cd` 命令切换到该文件夹,然后运行:
|
||||
将下载好的 `.whl` 文件放在一个文件夹中(如上述 `orisys_ws` 文件夹),打开命令行(确保已激活 `orisys` 环境或使用 “Anaconda Prompt”),使用 `cd` 命令切换到该文件夹,然后运行:
|
||||
|
||||
``` bash
|
||||
```bash
|
||||
# 请将文件名替换为您实际下载的文件名
|
||||
pip install orisys-0.4.0-cp310-cp310-win_amd64.whl
|
||||
pip install orisys-<版本>-cp310-cp310-win_amd64.whl
|
||||
```
|
||||
|
||||
## 安装 CuPy(可选)
|
||||
### 安装 CuPy(可选)
|
||||
注意:如果您没有安装 CUDA,或者使用 CPU 运行,可以跳过这一步。
|
||||
|
||||
CuPy 是一个与 NumPy 类似但支持 GPU 加速的库,您需要根据您安装的 CUDA 版本选择对应的 CuPy 版本。
|
||||
@@ -84,55 +108,95 @@ CuPy 是一个与 NumPy 类似但支持 GPU 加速的库,您需要根据您安
|
||||
|
||||
```bash
|
||||
# 如果您的 CUDA 版本是 12.x
|
||||
pip install orisys[cuda12]
|
||||
pip install "orisys[cuda12]"
|
||||
|
||||
# 如果您的 CUDA 版本是 13.x
|
||||
pip install orisys[cuda13]
|
||||
pip install "orisys[cuda13]"
|
||||
```
|
||||
如果之前已经安装了 orisys,可以单独安装 CuPy:
|
||||
|
||||
方法二(手动安装):
|
||||
|
||||
``` bash
|
||||
```bash
|
||||
# CUDA 12.x
|
||||
pip install cupy-cuda12x>=12.3.0
|
||||
pip install "cupy-cuda12x>=12.3.0"
|
||||
|
||||
# CUDA 13.x
|
||||
pip install cupy-cuda13x>=13.0.0
|
||||
pip install "cupy-cuda13x>=13.0.0"
|
||||
```
|
||||
|
||||
## 测试安装
|
||||
安装完成后,在命令行输入” python“命令进入 Python 环境, 并导入 orisys 库并查看版本号,验证是否成功:
|
||||
``` bash
|
||||
### 测试安装
|
||||
安装完成后,在命令行输入 `python` 命令进入 Python 环境,并导入 `orisys` 库查看版本号,验证是否成功:
|
||||
```bash
|
||||
python
|
||||
```
|
||||
|
||||
进入 Python 后继续输入:
|
||||
|
||||
```python
|
||||
import orisys
|
||||
print(orisys.__version__)
|
||||
```
|
||||
如果输出版本号(例如 0.4.0),则说明安装成功。
|
||||
如果打印出版本号,则说明安装成功。
|
||||
|
||||
# 使用方法
|
||||
## 运行示例程序
|
||||
套件里面包含了例子展示用法,运行步骤如下:
|
||||
## 使用方法
|
||||
## 使用设备专属的力标定参数
|
||||
|
||||
### Windows:
|
||||
1. 检查是否有其他正在连接的USB摄像头设备:打开powershell, 运行命令`Get-PnpDevice -PresentOnly | Where-Object { $_.Class -match '^Camera' }`,如果没有的话不会有输出。
|
||||
2. 把传感器的USB头与电脑连接,再次运行步骤一的命令,会新增一行输出
|
||||
3. 如果第一步没有输出,传感器的ID就是“0”,如果第一步有一个设备,传感器的ID就是“1”,如此类推。
|
||||
4. 在确定了传感器的ID(下方例子假设为“0”)后,打开Anaconda Prompt, 切换到工作路径,然后在命令行中输入:
|
||||
``` bash
|
||||
python examples/basic_pipeline.py -v 0
|
||||
```
|
||||
5. 程序会运行,并自动打开可视化窗口和打印资料,在窗口上按键盘”q”键可以停止运行,按”r”键可以重设数据
|
||||
摄像头编号与设备 ID 是两个独立参数。命令中的 `-v 0` 或 `--camera_id 0` 只表示选择第 0 个视频源;`device_id` 才是传感器的设备编号(SN),目前需要人工确认并填写。
|
||||
|
||||
### Ubuntu:
|
||||
1. 检查是否有其他正在连接的USB摄像头设备:打开终端, 运行命令`v4l2 –list-devices`,如果没有的话不会有输出。
|
||||
2. 把传感器的USB头与电脑连接,再次运行步骤一的命令,会新增输出
|
||||
3. 根据新增的输出,例如”/dev/video0”,确定传感器的ID为“0”, 如果是”/dev/video3”的话ID为“3” ,如此类推。
|
||||
4. 在确定了传感器的ID(下方例子假设为“0”)后,打开终端, 激活相应的python环境,切换到工作路径,然后在命令行中输入:
|
||||
``` bash
|
||||
python examples/basic_pipeline.py -v 0
|
||||
```
|
||||
5. 程序会运行,并自动打开可视化窗口和打印资料,在窗口上按键盘”q”键可以停止运行,按”r”键可以重设数据
|
||||
将**这台设备实际测得**的标定 JSON 放在自行选择的目录中,不要放进 SDK 的 `config/`,也不要覆盖 `sensors/<产品>/force_calibration.json`。目录结构如下,路径可由用户更改:
|
||||
|
||||
## 开发套件使用方法
|
||||
```te
|
||||
D:/orisys_data/calibrations/
|
||||
├── box/
|
||||
│ └── BOX-001.json
|
||||
├── mini/
|
||||
│ └── MINI-001.json
|
||||
└── finger/
|
||||
└── FINGER-001.json
|
||||
```
|
||||
|
||||
例如,使用 `BOX-001` 时,`calibration_dir` 指向上图的 `calibrations` 目录,`device_id` 填 `BOX-001`:
|
||||
|
||||
```python
|
||||
import orisys
|
||||
|
||||
sensor = orisys.Sensor(
|
||||
0, # 摄像头编号,不是设备 ID
|
||||
config_name="box",
|
||||
calibration_dir="D:/orisys_data/calibrations",
|
||||
device_id="BOX-001",
|
||||
verbose=True,
|
||||
)
|
||||
try:
|
||||
print(sensor.calibration.source_path if sensor.calibration else "未加载标定")
|
||||
print(sensor.force_units) # 依次为 FN、FX、FY 的单位
|
||||
finally:
|
||||
sensor.disconnect()
|
||||
```
|
||||
或者使用命令:
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c box --calibration-dir data/calibrations --device-id BOX-001
|
||||
```
|
||||
|
||||
若该设备的标定文件不存在,SDK 不应用标定,力值保持 raw,也不会回退到产品默认标定。若文件存在,但其中的设备 ID、产品类型或测量设置与当前设备不匹配,SDK 会报错。FingerRuntime.open() 支持这两个参数,但 finger_pipeline.py 目前还没有对应的命令行参数。
|
||||
|
||||
### 运行示例程序
|
||||
|
||||
示例的完整清单、每个产品对应的入口、全部命令与参数,见 [`examples/README.md`](../examples/README.md)。这里只给第一次运行的最短路径。
|
||||
|
||||
先在仓库根目录(即 `config/`、`examples/`、`assets/` 同级)确认传感器 ID:
|
||||
|
||||
- Windows:在 PowerShell 运行 `Get-PnpDevice -PresentOnly | Where-Object { $_.Class -match '^Camera' }`。若插上传感器后多出一行,ID 为 `1`,否则为 `0`。
|
||||
- Ubuntu:运行 `v4l2 --list-devices`,从新增的设备名取编号(`/dev/video0` 即 ID `0`)。
|
||||
|
||||
然后运行(假设 ID 为 `0`):
|
||||
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c box
|
||||
```
|
||||
|
||||
窗口打开后按 `q` 停止,按 `r` 重设数据。若本地没有示例文件(例如通过 PyPI 安装),从项目链接下载。
|
||||
|
||||
### 开发套件使用方法
|
||||
参考开发套件附带的开发文档。
|
||||
@@ -0,0 +1,139 @@
|
||||
---
|
||||
kind: reference
|
||||
---
|
||||
|
||||
# Plan: move the GUI out of the SDK into `orisys-studio`
|
||||
|
||||
Status: **planned, then postponed (2026-09-30).** Nothing here is in progress. This document
|
||||
records what would move, what would stay, and the one coupling that makes the move non-trivial, so
|
||||
that the work can be picked up without re-deriving it. It is a plan, not a description of current
|
||||
behaviour — everything described below is still in the SDK exactly as listed.
|
||||
|
||||
## Why
|
||||
|
||||
`examples/gui/` is not an example. It is a full Qt + open3d + pyqtgraph application — 30 modules,
|
||||
4,815 lines — that ships inside the SDK wheel. The GUI belongs to the `orisys-studio` application,
|
||||
which already exists for that purpose. Leaving it in the SDK means the SDK ships, versions and
|
||||
installs an application it is not responsible for.
|
||||
|
||||
## Current state
|
||||
|
||||
| Path | Modules | Lines | What it is |
|
||||
|---|---|---|---|
|
||||
| `examples/gui/` | 30 | 4,815 | The application: `viewer2d/`, `viewer3d/`, `widgets/`, `app.py`, `log_config.py` |
|
||||
| `examples/viz/` | 11 | 2,298 | **Shared** drawing/rendering helpers — see the coupling below |
|
||||
| `examples/pipeline/` | 6 | 577 | Sensor service, CSV recorder, profiling. **SDK support code, not GUI** |
|
||||
|
||||
### Good news: the SDK core does not depend on the GUI
|
||||
|
||||
Verified by AST, not by grep: **no module under `src/orisys/` imports `gui`, `viz`, `pipeline` or
|
||||
`examples`.** The only match is `src/orisys/__init__.py` importing `orisys.pipeline`, which is the
|
||||
SDK's own module. The dependency runs one way, so the GUI can leave without touching the library.
|
||||
|
||||
Two apparent hits are prose, not code: `src/orisys/configs.py` mentions `examples/` in a docstring
|
||||
about run-root layout, and `export_sensors.py` has an `Examples::` docstring section. The
|
||||
`configs.py` anchor is a real behavioural dependency to preserve — the run-root heuristic keys off
|
||||
a sibling `examples/` directory, so that directory must keep existing in the packaged layout even
|
||||
after `gui/` leaves it.
|
||||
|
||||
### The coupling: `viz/` is shared, so it cannot simply follow the GUI
|
||||
|
||||
```
|
||||
examples/gui/ ──uses──▶ examples/viz/ ──uses──▶ orisys (SDK)
|
||||
examples/pipeline/service.py ──uses──▶ examples/viz/
|
||||
tests/test_contact_arrow_scale.py ──uses──▶ examples/viz/
|
||||
```
|
||||
|
||||
`examples/pipeline/service.py` and a test depend on `viz`, and `viz` is a mix: `mesh_camera.py`,
|
||||
`contact_arrow.py`, `flow_arrow.py` and `offscreen_mesh.py` are open3d rendering helpers, while
|
||||
`render_2d.py`, `recording.py`, `signal_filters.py` and `force_charts.py` are toolkit-independent.
|
||||
**Moving `viz/` wholesale would break `examples/pipeline/`.** It has to be split, not moved.
|
||||
|
||||
### The real coupling is in the tests, and it is inverted
|
||||
|
||||
`tests/test_qt_viewer_backend.py` imports `examples/gui` directly, and `tests/conftest.py` inserts
|
||||
`EXAMPLES_ROOT` into `sys.path` with a guard whose stated reason is that **`examples/gui` must win
|
||||
over `build/gui`, because both are packages named `gui`**. So the SDK's test suite currently
|
||||
depends on the GUI application being present and on a name-resolution guard existing for it.
|
||||
|
||||
This is the inversion to fix: the SDK must not depend on the GUI.
|
||||
|
||||
Two related facts:
|
||||
|
||||
- `examples/gui/` contains committed `__pycache__` directories (`adapter.cpython-310.pyc`,
|
||||
`cpython-313.pyc`, …) — build artefacts tracked alongside source.
|
||||
- `tests/test_wheel_contents.py` and `tests/test_pyproject.py` assert on what the wheel bundles, so
|
||||
they will need updating when the `gui/**/*.py` bundle pattern goes.
|
||||
|
||||
## What moves, what stays
|
||||
|
||||
**Moves to `orisys-studio`:**
|
||||
|
||||
- `examples/gui/**` — the whole application, as the single home for GUI code.
|
||||
- `examples/finger_pipeline_3dview.py` — a 91-line launcher for the dashboard; its only purpose is
|
||||
to start the GUI, so it belongs with the GUI.
|
||||
- The open3d rendering half of `examples/viz/`: `contact_arrow.py`, `flow_arrow.py`,
|
||||
`mesh_camera.py`, `offscreen_mesh.py`, plus the open3d-dependent `__init__` re-exports.
|
||||
- `tests/test_qt_viewer_backend.py` — it tests the GUI, so it moves with the GUI.
|
||||
|
||||
**Stays in the SDK:**
|
||||
|
||||
- `examples/pipeline/` — SDK support code (sensor service, CSV recording, profiling).
|
||||
- `examples/viz/` toolkit-independent half: `render_2d.py`, `recording.py`, `signal_filters.py`,
|
||||
`force_charts.py` — needed by `examples/pipeline/service.py`.
|
||||
- `examples/plot_data.py` — uses pyqtgraph for force charts only; 96 lines. Decide separately
|
||||
whether it follows the GUI; it is the one remaining example with a Qt dependency after the move.
|
||||
- `src/orisys/finger/viewer.py` (`UVMeshViewer`) — **see the deprecation note below.**
|
||||
|
||||
## The public-API problem, handled separately
|
||||
|
||||
`UVMeshViewer`, `init_uv_mesh_viewer` and `update_uv_mesh_viewer` are exported in
|
||||
`src/orisys/finger/__init__.py`'s `__all__`. **Removing them is a breaking change to a released
|
||||
API**, so it must not be bundled into a code move:
|
||||
|
||||
1. Release with them present and documented `DeprecationWarning`s emitted on construction.
|
||||
2. Remove them at the next major version, with the release notes naming the replacement.
|
||||
|
||||
The audit (`docs/api-audit.md`) already lists `UVMeshViewer` among the surface that is used by
|
||||
examples and absent from the reference.
|
||||
|
||||
## Sequence, lowest risk first
|
||||
|
||||
| # | Step | Risk | Notes |
|
||||
|---|---|---|---|
|
||||
| 1 | Add `docs/plan-gui-extraction.md` (this file) | none | Done |
|
||||
| 2 | Delete the committed `__pycache__` under `examples/gui/` and add it to `.gitignore` | none | Pure hygiene; ship independently |
|
||||
| 3 | Tag the SDK at the commit whose `examples/gui` is known-good, so the studio move has a pinned source | none | A tag, not a submodule or subtree |
|
||||
| 4 | In `orisys-studio`: import `examples/gui` + the open3d half of `viz` as its own provenance-stated change, reconciling against its existing viewer code | **medium** | The studio repo already has viewer code; this is a merge decision, not a copy |
|
||||
| 5 | In the SDK: move `tests/test_qt_viewer_backend.py` out, and simplify the `conftest.py` guard | **medium** | The guard exists for `gui`; with `gui` gone, `EXAMPLES_ROOT` may still be needed for `viz`/`pipeline`. Re-derive, do not guess |
|
||||
| 6 | In the SDK: retire `examples/gui/`, `finger_pipeline_3dview.py`, and the moved `viz` modules | low once 4 and 5 land | |
|
||||
| 7 | Update the bundle patterns in `pyproject.toml` (drop `gui/**/*.py`, adjust `viz/**`) and the wheel tests | low | `test_wheel_contents.py`, `test_pyproject.py` |
|
||||
| 8 | Update `examples/README.md` and the SDK reference to stop pointing at the dashboard | low | The README already says the GUI is moving |
|
||||
| 9 | Deprecate `UVMeshViewer` in a release; remove at the next major | low | Separate release cycle |
|
||||
|
||||
Steps 2 and 3 are safe to do immediately and independently. Steps 4–5 are the actual move and need
|
||||
both repositories. Nothing here requires a library behaviour change, which is what makes the move
|
||||
tractable.
|
||||
|
||||
## Open questions for the studio side
|
||||
|
||||
- **Merge or adopt?** `orisys-studio` already has its own viewer. If its code is the descendant of
|
||||
this one, take the SDK copy only where it is ahead; otherwise choose one and delete the other.
|
||||
This must be decided before step 4, because the answer changes what gets imported.
|
||||
- **Do the SDK's 2D and 3D viewers need to keep working headless for CI?** `examples/viz` contains
|
||||
offscreen rendering used to produce evidence images. If any SDK or sensor-design workflow depends
|
||||
on that, the offscreen half may need to stay reachable from the SDK even after the app moves.
|
||||
- **Does `plot_data.py` follow the GUI?** It is an example, not an application, and it is the last
|
||||
Qt-dependent file in `examples/`.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] `src/orisys/` still imports no `gui`/`viz`/`examples` module (re-run the AST check).
|
||||
- [ ] The SDK test suite passes with `examples/gui/` absent.
|
||||
- [ ] No module named `gui` ships in the wheel; `test_wheel_contents.py` asserts it.
|
||||
- [ ] `examples/pipeline/service.py` and `tests/test_contact_arrow_scale.py` still import their
|
||||
`viz` dependencies.
|
||||
- [ ] `UVMeshViewer` is either deprecated-but-present or removed by an explicit major-version
|
||||
decision — never dropped silently in a move.
|
||||
- [ ] The `examples/` sibling directory still exists in the packaged layout, because
|
||||
`configs.py`'s run-root anchor depends on it.
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
title: Build and publish the Orisys SDK
|
||||
description: Contributor release checklist and the verified build and publish entry points.
|
||||
---
|
||||
|
||||
# Build and publish the Orisys SDK
|
||||
|
||||
This guide gives contributors the path from a source change to release evidence.
|
||||
The build launcher is [build/run.py](../build/run.py), documented in
|
||||
[build/README.md](../build/README.md). Publishing to GitHub or PyPI changes external
|
||||
systems and is a maintainer action. The existing
|
||||
[Chinese release runbook](publishing.md) retains historical details; verify its
|
||||
version-specific examples against current code before using them.
|
||||
|
||||
## Before building
|
||||
|
||||
Run the non-hardware tests from the repository root with the intended Python
|
||||
environment:
|
||||
|
||||
```bash
|
||||
python -m pytest -m "not slow and not hardware"
|
||||
```
|
||||
|
||||
Check the current SDK version at `src/orisys/__init__.py`. `pyproject.toml` reads it
|
||||
as a dynamic version. Do not copy a previous release number into new commands or
|
||||
documentation. Check the source diff and confirm the intended product bundles and
|
||||
assets are present; `pyproject.toml` declares which non-code files are packaged.
|
||||
|
||||
## Build the candidate
|
||||
|
||||
Use the launcher appropriate to the target. `wheel-win` invokes the Windows build;
|
||||
`wheel-manylinux` invokes the Docker-backed Linux build. A single-interpreter `wheel`
|
||||
build is useful for a local candidate, but does not cover every supported ABI.
|
||||
|
||||
```bash
|
||||
python build/run.py wheel-win
|
||||
python build/run.py wheel-manylinux
|
||||
python build/run.py wheel-manylinux -Architecture aarch64
|
||||
```
|
||||
|
||||
Windows builds use the configured `py310`, `py311`, and `py312` environments.
|
||||
The Linux build needs its documented Docker setup. Built wheels are placed under
|
||||
`SDK/<version>/`; Linux upload candidates may also appear in `dist/repaired/`.
|
||||
See [build/README.md](../build/README.md) for build prerequisites and narrower
|
||||
commands. A contributor without these environments can still run tests and hand
|
||||
off a source change; do not describe it as a verified release candidate.
|
||||
|
||||
Build Windows x64, Linux x86_64 and Linux ARM64 before preparing the release
|
||||
report. With Python 3.10, 3.11 and 3.12 this produces nine wheels. Both publishers
|
||||
use every wheel in `SDK/<version>/`; files added or rebuilt after preflight
|
||||
invalidate its certification. ARM64 CPU wheels do not require CUDA.
|
||||
|
||||
## Run the release gate
|
||||
|
||||
After building, run `release preflight` on a clean, reviewed commit. It checks
|
||||
repository state, version consistency, tests, bundled assets, wheel contents, and
|
||||
an isolated runtime on recorded frames. `--check-pypi` adds a read-only duplicate
|
||||
version check against PyPI. The report is written under
|
||||
`build/release/reports/<version>/`.
|
||||
|
||||
```bash
|
||||
python build/run.py release preflight --all-wheel-pythons --check-pypi
|
||||
```
|
||||
|
||||
`--all-wheel-pythons` checks every staged ABI and architecture. Run the complete
|
||||
mixed-platform gate from Windows: Windows wheels use the `py310`, `py311` and
|
||||
`py312` conda environments, while Linux wheels use `orisys-manylinux` or
|
||||
`orisys-manylinux-aarch64` with the matching container Python. Docker must already
|
||||
have these locally built images; verification never pulls or rebuilds them.
|
||||
Linux dependencies come from each image's ABI wheelhouse, in an isolated venv.
|
||||
ARM64 verification on Intel/AMD uses QEMU and can take longer. Every wheel must
|
||||
pass imports, bundled-asset loading and the CPU video pipeline; emulation does
|
||||
not certify Q6A or Jetson hardware, GPU or NPU behavior.
|
||||
|
||||
For work in progress, `--allow-dirty --keep-going` can collect diagnostics, but
|
||||
that report does **not** certify a release. Do not use `--skip-tests`, `--no-runtime`,
|
||||
or `--no-assets` to turn missing verification into approval. If the runtime clip is
|
||||
missing, provide `--video <path>` or `ORISYS_VERIFY_VIDEO`; the gate should report
|
||||
failure rather than silently claiming a pass.
|
||||
|
||||
For one existing wheel, `release audit <wheel>` checks it without preparing a
|
||||
complete release. Add `--assets --runtime` when a suitable verification clip and
|
||||
environment are available. This is useful for investigating a candidate; it does
|
||||
not replace the full preflight gate.
|
||||
|
||||
## Release handoff and publication
|
||||
|
||||
Hand the maintainer the version, source commit, wheel filenames, preflight report,
|
||||
and any hardware or platform checks that were not run. The publish commands read
|
||||
the passing preflight report and verify the wheel hashes again.
|
||||
The report must also contain successful runtime evidence matching every wheel's
|
||||
hash. Older reports that checked only Windows wheels must be regenerated.
|
||||
|
||||
```bash
|
||||
python build/run.py release github --version <version>
|
||||
python build/run.py release pypi --version <version>
|
||||
```
|
||||
|
||||
These commands omit `--apply`, so they preview the publish plan. The GitHub preview
|
||||
may still fetch remote tags; the PyPI preview may check credentials and the index.
|
||||
Maintainers run the commands with `--apply` only after checking the target repository,
|
||||
release notes, credentials, and candidate files. `release pypi --apply` targets
|
||||
TestPyPI; publishing to the production index also requires `--confirm-pypi`.
|
||||
GitHub publication can sync the release mirror, commit, tag, push, and create a
|
||||
Release. See the launcher help and [build/README.md](../build/README.md) for the
|
||||
exact flags. A published version cannot be overwritten, so do not treat a rerun as
|
||||
a way to replace bad wheels.
|
||||
|
||||
The release order is build all candidates, commit source changes, run the clean
|
||||
preflight, preview both publishers, then execute the authorized publications.
|
||||
GitHub attaches all certified wheels and generates notes naming their actual
|
||||
platforms, including ARM64. PyPI defaults to TestPyPI first; after confirming it,
|
||||
use `--confirm-pypi --apply` for production. Each upload checks the complete
|
||||
filename set and downloads every Python/platform wheel from that index to
|
||||
compare its hash. TestPyPI download verification does not fall back to PyPI.
|
||||
If downloads are not yet available, the command reports the successful upload
|
||||
as unconfirmed and stops before uploading to later indexes; check propagation
|
||||
before rerunning.
|
||||
If a published version has a different file set, the commands stop rather than
|
||||
silently extending or replacing that release.
|
||||
@@ -0,0 +1,399 @@
|
||||
# 发布流程(构建 → 校验 → 发布)
|
||||
|
||||
本文档说明如何构建 orisys 的 Nuitka 二进制 wheel、如何在发布前用一条命令做完整门禁,以及当前仍需人工执行的发布步骤。
|
||||
|
||||
> 历史说明:`README.md` 与 `build/wheel/docker/run.ps1` 早就引用了本文件,但它此前并不存在。本文件补齐该引用。
|
||||
|
||||
---
|
||||
|
||||
## 0. 环境准备
|
||||
|
||||
推荐用 `py310` conda 环境执行构建与校验(它同时具备 pytest / twine / numba / Nuitka / open3d / PyQt5):
|
||||
|
||||
```powershell
|
||||
conda activate py310
|
||||
python -m pytest -m "not slow and not hardware" # 开发期快速回归
|
||||
```
|
||||
|
||||
`py311`、`py312` 仅作为 Nuitka 各 ABI 的编译环境按需调用(`build/wheel/win.bat` 会依次使用 `py310`/`py311`/`py312`)。
|
||||
|
||||
**不要**在同一个环境里既装旧版 `orisys` 又从源码目录跑测试:site-packages 里的旧包会静默遮蔽 `src/orisys`。
|
||||
`tests/test_harness_guards.py` 会显式报错提示执行 `pip uninstall -y orisys`。
|
||||
|
||||
---
|
||||
|
||||
## 1. 构建 wheel
|
||||
|
||||
```powershell
|
||||
# Windows:py310 / py311 / py312 三个 ABI
|
||||
python build/run.py wheel-win
|
||||
|
||||
# 单个解释器(当前环境)
|
||||
python build/run.py wheel
|
||||
|
||||
# Linux manylinux(Docker Desktop,Linux 容器模式)
|
||||
python build/run.py manylinux
|
||||
python build/run.py wheel-manylinux -PythonVersions 3.10
|
||||
```
|
||||
|
||||
产物落点:
|
||||
|
||||
- `SDK/<version>/` — 交付用 wheel(git 忽略)
|
||||
- `dist/repaired/` — 上传候选(git 忽略)
|
||||
|
||||
---
|
||||
|
||||
## 2. 发布前门禁:`release preflight`
|
||||
|
||||
```powershell
|
||||
# --version 可省略:默认取 pyproject 的 dynamic version(即 src/orisys/__init__.py 里的 __version__)
|
||||
python build/run.py release preflight
|
||||
```
|
||||
|
||||
按顺序执行 8 个步骤,任一步失败即整体失败(`--keep-going` 可改为收集全部失败):
|
||||
|
||||
| 步骤 | 检查内容 |
|
||||
| --- | --- |
|
||||
| `repo-state` | 是 git 仓库、工作树干净、记录 HEAD(`--allow-dirty` 可豁免,用于提交前预检) |
|
||||
| `version-consistency` | `src/orisys/__init__.py::__version__` == pyproject 的 dynamic attr == `SDK/<version>/` 中 wheel 文件名里的版本 |
|
||||
| `version-strings` | 追踪的文本文件里没有与当前版本不符的版本字面量(见 §5.3) |
|
||||
| `duplicate-guard` | 该版本是否**已经发布过**:本地 git tag(`v<version>` / `<version>`,含 sibling `orisys-sdk-release` 仓库)命中即拒绝;`--check-pypi` 时再查 PyPI。`SDK/<version>/` 里已有 wheel 只作信息展示——那是 preflight 的正常输入,不是错误 |
|
||||
| `tests` | `pytest -m "not slow and not hardware"`(`--longrun` 时包含 slow) |
|
||||
| `bundle-precheck` | 每个 `[[tool.orisys.bundle.items]]` 至少解析出 1 个文件;finger 必需资产存在 |
|
||||
| `wheel-inspection` | 逐个 wheel 做静态断言:文件名/版本/METADATA 字段/RECORD 哈希/压缩包路径/体积上限/无 pyc 与缓存/无 `orisys/**/*.py` 源码泄漏/与源码树逐文件字节一致 |
|
||||
| `wheel-runtime` | 每个 wheel **一个** venv,在里面依次做:导入冒烟 → 内置 configs/sensors 资产可加载 → 对真实视频跑 40 帧公开流水线(无需相机、无 GUI) |
|
||||
|
||||
正常顺序是**先构建、再 preflight**,不需要额外开关:
|
||||
|
||||
```powershell
|
||||
python build/run.py wheel-win
|
||||
python build/run.py manylinux
|
||||
python build/run.py release preflight --all-wheel-pythons --check-pypi
|
||||
```
|
||||
|
||||
提交前想先看结果(此时工作树通常还是脏的):加 `--allow-dirty --keep-going`。
|
||||
|
||||
产物报告:
|
||||
|
||||
- `build/release/reports/<version>/preflight.json` — 机器可读,含每个 wheel 的 sha256,供后续发布步骤复用
|
||||
- `build/release/reports/<version>/preflight.txt` — 人工阅读
|
||||
- `build/release/reports/<version>/junit.xml` — pytest 结果
|
||||
|
||||
(`build/release/reports/` 已加入 `.gitignore`。)
|
||||
|
||||
### 常用参数
|
||||
|
||||
| 参数 | 作用 |
|
||||
| --- | --- |
|
||||
| `--wheel-dir DIR` | 校验指定目录下的 wheel(默认 `SDK/<version>`) |
|
||||
| `--all-wheel-pythons` | cp310/cp311/cp312 各自在对应 conda 环境里独立校验 |
|
||||
| `--video PATH` / `--frames N` / `--config NAME` | 覆盖运行时校验输入(默认 `data/sample/s1_480p.mp4` / 40 帧 / `box`) |
|
||||
| `--offline` | 复用父环境的 numpy/opencv/numba,不去 PyPI 装依赖 |
|
||||
| `--check-pypi` | 唯一一处网络调用:只读 PyPI,防止重复发布 |
|
||||
| `--skip-tests` / `--no-runtime` / `--no-assets` | 显式跳过并在报告中留痕;**跳过的运行不能作为发布依据** |
|
||||
| `--report-dir DIR` | 覆盖报告根目录 |
|
||||
|
||||
`wheel-runtime` **不会静默跳过**:找不到视频时直接失败,提示 `--video` / `ORISYS_VERIFY_VIDEO`。
|
||||
|
||||
### 单体体检:`release audit`
|
||||
|
||||
对单个 wheel 做静态断言(+ 可选运行时校验),不需要版本上下文。适合体检历史产物:
|
||||
|
||||
```powershell
|
||||
python build/run.py release audit SDK\<version>\orisys-<version>-cp312-cp312-win_amd64.whl
|
||||
python build/run.py release audit dist\repaired\orisys-<version>-cp310-cp310-win_amd64.whl --strict-bundle
|
||||
|
||||
# 追加运行时校验(--assets 与 --runtime 同时给时共用一个 venv,只装一次依赖)
|
||||
python build/run.py release audit <wheel> --assets --runtime --video data/sample/s1_480p.mp4
|
||||
```
|
||||
|
||||
默认对「内置文件与当前源码树不一致」只告警(历史 wheel 与当前分支不一致是正常的);加 `--strict-bundle` 改成失败(刚构建完的 wheel 应当一致)。
|
||||
|
||||
旧的导入冒烟测试仍然可用,行为不变:
|
||||
|
||||
```powershell
|
||||
python build/run.py verify <wheel>
|
||||
python build/run.py verify <wheel> --contents # 静态断言
|
||||
python build/run.py verify <wheel> --assets --runtime # 资产 + 真实帧
|
||||
python build/run.py verify --gui-dist --product ory_finger
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 发布(两条命令)
|
||||
|
||||
先决条件:`release preflight` PASS,**而且是在干净提交上跑的**。两个发布命令都只发布「preflight 认证过的东西」——它们会读取 `build/release/reports/<version>/preflight.json`,逐个核对 wheel 的 sha256,并拒绝使用带豁免项(`--skip-tests` / `--allow-dirty` / `--no-runtime` / `--force`)的报告。所以正确顺序是:
|
||||
|
||||
```powershell
|
||||
git add -A; git commit -m "..."
|
||||
python build/run.py wheel-win && python build/run.py manylinux
|
||||
python build/run.py release preflight --all-wheel-pythons --check-pypi # 干净树 + 无豁免
|
||||
```
|
||||
|
||||
两个命令**默认都是 dry run**,只打印将要执行的动作;必须显式加 `--apply` 才会真正动手。GitHub 的 `gh release create`、`git push`,以及 PyPI 上传(版本号永久占用、不可覆盖)都属于不可撤销操作。
|
||||
|
||||
### 3.1 GitHub:同步公开仓库 + 上传 Release
|
||||
|
||||
```powershell
|
||||
python build/run.py release github --version 0.5.2 # 预览
|
||||
python build/run.py release github --version 0.5.2 --apply # 执行
|
||||
```
|
||||
|
||||
执行顺序:
|
||||
|
||||
1. 认证书校验(上面那份报告 + 逐 wheel sha256)
|
||||
2. `git fetch --tags`,若 `v0.5.2` 或 `0.5.2` 已存在则拒绝;`gh release view` 已存在也拒绝
|
||||
3. 按 `tools/downstream_sync_manifest.json` 同步公开内容到 `../orisys-sdk-release`,并运行 manifest 里的 `checks`
|
||||
4. 把 `SDK/0.5.2/*.whl` 放到 `orisys-sdk-release/release/0.5.2/`(该目录在 release 仓库里是 gitignore 的,只用于上传附件)
|
||||
5. 提交 → 打 tag `v0.5.2` → push 分支与 tag
|
||||
6. `gh release create v0.5.2 --verify-tag --latest <6 个 wheel>`,然后回读 `gh release view --json assets` 核对附件齐全
|
||||
|
||||
需要:`gh` 已安装并登录(`winget install --id GitHub.cli` 然后 `gh auth login`)。没装 gh 时:
|
||||
|
||||
- dry run 仍会完整打印计划(只告警)
|
||||
- `--apply` 会失败,或改用 `--skip-release-upload`:同步、提交、打 tag、push 都自动完成,然后打印出你自己要跑的 `gh release create`
|
||||
|
||||
其他开关:`--notes-file`(自定义 release notes)、`--attach-preflight`(把 `preflight.json` 也作为附件上传,便于审计;注意报告里有本机绝对路径)、`--allow-dirty-release`、`--skip-fetch`、`--skip-checks`。
|
||||
|
||||
> tag 拼写沿用历史风格 `v<version>`(注意历史里那个 typo tag `0.50` 指向 `v0.5.1`)。命令会在动手前拒绝已存在的 tag,所以不会重复发布。**如果 tag 已经 push 但 Release 上传失败(例如 `gh` 没配好),直接重跑同一条 `--apply` 命令即可续跑**:它会跳过创建 tag,只补做剩下的步骤;只有「GitHub Release 已存在」才拒绝。
|
||||
|
||||
### 3.2 PyPI:TestPyPI 先行,确认后再上正式仓库
|
||||
|
||||
```powershell
|
||||
python build/run.py release pypi --version 0.5.2 # 预览
|
||||
python build/run.py release pypi --version 0.5.2 --apply # 只发 TestPyPI
|
||||
python build/run.py release pypi --version 0.5.2 --confirm-pypi --apply # TestPyPI + 正式 PyPI
|
||||
|
||||
# 只有正式 token、没有 TestPyPI 账号时:跳过彩排直接发正式库
|
||||
python build/run.py release pypi --version 0.5.2 --pypi-only --apply
|
||||
```
|
||||
|
||||
`--pypi-only` 与 `--confirm-pypi` 互斥。**注意它的代价**:没有 TestPyPI 彩排,而正式库的版本号一旦占用就无法复用、无法替换。它仍然会做完整门禁、`twine check`、上传前查重、上传后回读索引核对文件、以及把产物重新拉回来验证。
|
||||
|
||||
执行顺序:
|
||||
|
||||
1. 认证书校验(与 `release github` 同一份报告)
|
||||
2. `twine check` 全部 wheel(校验 METADATA/long description,上传前就能发现问题)
|
||||
3. 只读查询目标索引,确认该版本不存在(TestPyPI 与 PyPI 各查各的)
|
||||
4. 上传 → **轮询**索引确认 6 个文件都在 → `pip download` 把刚上传的 wheel 拉回来,证明产物可获取
|
||||
5. 未加 `--confirm-pypi` 时只发 TestPyPI,然后停在这里并打印后续命令
|
||||
|
||||
两个针对「索引不即时一致」的设计(TestPyPI 尤其明显,上传成功后 JSON API 可能滞后若干秒):
|
||||
|
||||
- **上传后的确认会重试**(默认 6 次 × 5 秒)。确认不了也**不算失败**:twine 上传成功就说明产物已发布,命令会打印警告、给出项目页链接并以 0 退出,不会把一个成功的上传报成失败。真正的失败只有两种:twine 上传本身失败,或索引上的文件集与本地不一致。
|
||||
- **重复运行是安全的**:若索引上已经存在该版本且文件集完全一致,该阶段会被视为「已完成」而跳过(`[ok] testpypi already serves orisys 0.5.2 with all 6 file(s)`),不会重传、也不报错。这让你在确认不明确时可以直接重跑。如果文件集**不一致**(索引上是另一次构建),才会失败并停下。
|
||||
|
||||
所以「上传后确认失败」不等于要重新上传——索引版本号不可覆盖,重跑只会做一次只读核对。
|
||||
|
||||
### 3.3 凭证:`release auth` 与 `~/.pypirc`
|
||||
|
||||
```powershell
|
||||
python build/run.py release auth # 查看两个索引的凭证状态
|
||||
python build/run.py release auth --set-token pypi # 隐藏输入,写入 ~/.pypirc
|
||||
python build/run.py release auth --set-token testpypi
|
||||
python build/run.py release auth --repair # 修「有段但没列进 index-servers」
|
||||
```
|
||||
|
||||
`--set-token` 的 token 只从**隐藏输入**或 `--token-stdin`(供脚本/CI)读取,**不接受命令行参数**——避免落进 shell 历史和进程列表。写入采用定点文本编辑,文件里的说明注释会保留;同时会把该索引补进 `[distutils] index-servers`(这一步很关键,见下)。
|
||||
|
||||
`release auth` 只有在你真的运行 `--set-token` 时才写文件;不带参数时只读并报告,状态判断比「文件是否存在」严格:
|
||||
|
||||
```
|
||||
config file: C:\Users\cheun\.pypirc (exists)
|
||||
[-- ] [pypi] not usable: [pypi] still contains a placeholder token
|
||||
[-- ] [testpypi] not usable: C:\Users\cheun\.pypirc has no [testpypi] section
|
||||
```
|
||||
|
||||
**占位符、空密码、`username` 不是 `__token__`,都会被判定为「不可用」**——否则 twine 只会在真正上传时才发现,而那时已经动过索引了。
|
||||
|
||||
`release pypi` 在 `--apply` 时若凭证缺失:有终端就地提示输入(隐藏),无终端则直接失败并打印 `release auth --set-token <index>`;也可显式加 `--prompt-credentials`。dry run 只告警并继续展示计划,不会卡在输入上。
|
||||
|
||||
#### twine 直接读取的格式
|
||||
|
||||
默认路径 `~/.pypirc`(Windows 上即 `C:\Users\<你>\.pypirc`,由 `USERPROFILE` 展开)。**没有**环境变量可以改这个路径;要用别的文件就传 `--config-file`。
|
||||
|
||||
```ini
|
||||
[distutils]
|
||||
index-servers =
|
||||
pypi
|
||||
testpypi
|
||||
|
||||
[pypi]
|
||||
repository = https://upload.pypi.org/legacy/
|
||||
username = __token__
|
||||
password = pypi-你的正式token
|
||||
|
||||
[testpypi]
|
||||
repository = https://test.pypi.org/legacy/
|
||||
username = __token__
|
||||
password = pypi-你的TestPyPI token
|
||||
```
|
||||
|
||||
容易搞错的点:
|
||||
|
||||
- **`[distutils] index-servers` 必须列出该索引,否则 twine 完全忽略那一节** —— 这是最常见的坑:文件里明明有 `[testpypi]` 段,twine 却报 `InvalidConfiguration: Missing 'testpypi' section`。用 `release auth --repair` 修(把已存在但未列出的段补进 `index-servers`)。
|
||||
- 更细的语义差别:`index-servers =` **值为空**表示「一个都不认」;只有**整行缺失**时 twine 才会回退到默认的 `pypi testpypi`。所以手工编辑时别把名字删空。
|
||||
- `username` 就是字面量 `__token__`,不是账号名或邮箱
|
||||
- 两类 token **都以 `pypi-` 开头**,但**不能互换**:拿正式 token 打 `test.pypi.org` 会被 403 拒绝
|
||||
- TestPyPI 是**独立服务、独立账号**,注册在 <https://test.pypi.org/account/register/>,你的 PyPI 账号在那里不存在
|
||||
- 正式 token 建议限定到 `orisys` 项目(项目已存在,可以建 project-scoped token)
|
||||
- 该文件必须在**任何 git 仓库之外**(放在用户目录里);token 一旦提交进仓库,即使后来删文件也还在历史里——那属于安全事故
|
||||
- 临时替代方案:会话内设 `TWINE_USERNAME=__token__` + `TWINE_PASSWORD=<token>`,命令会识别这种情形
|
||||
- PyPI token **没有只读的校验方式**,有效性只能在真正上传时才会被索引确认;所以所有本地检查、查重都排在写操作之前
|
||||
|
||||
`release auth` 的「可用」判定**直接问 twine 自己**(`twine.utils.get_config` 解析出的 username/password),而不是自己解析 ini,这样报告结果与 twine 的行为不会分叉。twine 未安装时才回退到本地复刻同一套规则。
|
||||
|
||||
只需要 TestPyPI 时才把 `testpypi` 写进 `index-servers` 并填对应 token;只发正式库时 `--pypi-only` 用不到 `[testpypi]` 段。
|
||||
|
||||
### 3.4 已知的历史发布空洞
|
||||
|
||||
| 版本 | 本地 `SDK/` | PyPI | GitHub Release |
|
||||
| --- | --- | --- | --- |
|
||||
| 0.4.2 | — | 有 | 有 |
|
||||
| 0.4.3 | — | 有 | 有 |
|
||||
| 0.4.4 | 有 | **无** | **无** |
|
||||
| 0.5.0 | 有 | **无** | **无** |
|
||||
| 0.5.1 | 有 | 有 | 有(tag `v0.5.1` + typo tag `0.50`) |
|
||||
| 0.5.2 | 有 | **待发布** | **待发布**(`release preflight --check-pypi` 确认 PyPI 尚无 0.5.2) |
|
||||
|
||||
> 本表按 2026 年 0.5.2 构建时的实际状态填写。`0.5.1` 这一行曾被一次全局替换误改成 `0.5.2` 并丢失,请勿再用「全局把 0.5.1 替换成新版本号」的方式改本文件——历史事实不应该被改。
|
||||
|
||||
---
|
||||
|
||||
## 4. 环境变量
|
||||
|
||||
| 变量 | 用途 |
|
||||
| --- | --- |
|
||||
| `ORISYS_TEST_VIDEO` | pytest 用的视频;默认自动取 `data/sample/s1_480p.mp4` |
|
||||
| `ORISYS_TEST_CONFIG` | pytest 用的配置名;默认自动挑一个含 `stitcher.box_size` 的配置 |
|
||||
| `ORISYS_VERIFY_VIDEO` | 运行时校验用的视频 |
|
||||
| `ORISYS_VERIFY_CONFIG` | 运行时校验用的配置名(默认 `box`) |
|
||||
| `ORISYS_LONGRUN_BACKEND` / `ORISYS_LONGRUN_FRAMES` | `slow` 长稳测试的参数 |
|
||||
| `CONDA_ENVS_ROOT` | conda envs 根目录(`--all-wheel-pythons` 用它解析 `py310`/`py311`/`py312`) |
|
||||
|
||||
---
|
||||
|
||||
## 5. 三个必须记住的坑
|
||||
|
||||
### 5.1 Nuitka 编译包的 `__file__` 是假的
|
||||
|
||||
Nuitka 把整个 `orisys` 编译成单个扩展模块,其 `__init__` 会安装 `nuitka_module_loader`。于是:
|
||||
|
||||
```python
|
||||
>>> import orisys
|
||||
>>> orisys.__file__ # 合成的、磁盘上并不存在的路径
|
||||
'...\\site-packages\\orisys\\orisys\\__init__.py'
|
||||
>>> orisys.__path__ # 第二项是 Nuitka 的哨兵值
|
||||
['...\\site-packages\\orisys\\orisys', '\\not_existing']
|
||||
>>> orisys.__loader__.__class__.__name__
|
||||
'nuitka_module_loader'
|
||||
```
|
||||
|
||||
后果:**在 Nuitka wheel 里不能靠 `orisys.__file__` 或 `importlib.resources.files("orisys")` 定位内置数据**——它们会指向不存在的目录,静默返回「文件不存在」。请改用发行包定位:
|
||||
|
||||
```python
|
||||
from importlib.metadata import distribution
|
||||
root = distribution("orisys").locate_file("orisys") # site-packages/orisys
|
||||
```
|
||||
|
||||
`build/common/verify/wheel.py` 的内置资产校验就是这么做的,并且会显式断言该目录位于校验用 venv 内,避免被父环境里的旧 `orisys` 顶替。
|
||||
|
||||
### 5.2 `--offline` 会被父环境的脏安装污染
|
||||
|
||||
`--offline` 通过 venv 的 `--system-site-packages` 复用父环境的 numpy/numba 等依赖,这也顺带暴露了父环境里的 `orisys`。若父环境残留了半卸载的 editable 安装(典型症状是 `pip` 提示 `Ignoring invalid distribution -risys`,且 `site-packages/orisys/__init__.py` 是一个指向死路径的 shim),校验就会读到它。
|
||||
|
||||
处置:在构建环境里执行 `pip uninstall -y orisys`,或改用不带 `--offline` 的完全隔离 venv(会联网安装依赖)。
|
||||
|
||||
### 5.3 版本号只有一个地方,其他都是推导出来的
|
||||
|
||||
版本号的唯一来源是 **`src/orisys/__init__.py::__version__`**。以下全部由它推导,不需要手改:
|
||||
|
||||
| 产物 | 推导方式 |
|
||||
| --- | --- |
|
||||
| pyproject 的 `[project] version` | `[tool.setuptools.dynamic] version = {attr = "orisys.__version__"}` |
|
||||
| wheel 文件名里的版本 | `build/wheel/build.py` 调 `get_version()` |
|
||||
| wheel `METADATA` 的 `Version:` | 生成的 `setup.py` 里写入同一个 `version` |
|
||||
| `SDK/<version>/` 目录名 | 同上 |
|
||||
| `importlib.metadata.version("orisys")` | 从 wheel METADATA 读出 |
|
||||
|
||||
因此**不要**再往文档、示例、测试里写具体版本号:
|
||||
|
||||
- 用法示例写 `SDK/<version>/...`、`--version <version>`,不要写具体版本
|
||||
- 文档里不要写「版本:x.y.z」;需要时让用户用 `python -c "import orisys; print(orisys.__version__)"` 自己看
|
||||
- 示例脚本的 docstring 不要再写 `SDK version: x.y.z`
|
||||
- 测试里需要「一个版本」时用明显是假的版本号(如 `1.x.y` 这种不会与真实版本混淆的值),不要用当前版本 —— 否则每次发版都要改测试,而且一次全局替换就会悄悄改坏断言
|
||||
|
||||
`release preflight` 的 `version-strings` 步骤强制这条规则:扫描所有被追踪的文本文件,任何与当前版本不符的版本字面量都会导致失败(`tests/` 与生成物除外;依赖约束如 `numpy>=1.24.4` 由正则排除)。
|
||||
|
||||
历史记录属于例外,需显式登记在 `build/release/version.py::VERSION_STRING_ALLOWLIST` 里——登记本身就是「该字面量故意不是包版本」的文档。**不要用全局替换修改含历史版本的文件**:本文件的发布记录表就曾被一次全局替换改错(`0.50` typo tag 被挂到错误的版本上,`0.5.1` 那一行被覆盖丢失)。
|
||||
|
||||
### 5.4 wheel 里有什么、没有什么
|
||||
|
||||
打包内容由 `pyproject.toml` 的 `[[tool.orisys.bundle.items]]` 单点定义:
|
||||
|
||||
| 内容 | 是否进 wheel | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `config/*.json` | 是 | `orisys/configs/`,运行时配置解析要用 |
|
||||
| `sensors/**` | 是 | `orisys/sensors/`,含 finger 的 npz/npy 资产,运行时与 GUI 都要用 |
|
||||
| `examples/**` | 是 | `orisys/examples/`,用户装完就能跑示例 |
|
||||
| `docs/**` | **否** | 只随 GitHub 仓库与 PyPI 项目页提供。`docs/publishing.md` 是内部流程文档;`docs/pypi_readme.md` 仅在构建时被读取作为 long description |
|
||||
|
||||
因为 docs 不进 wheel,**改文档不会让已构建的 wheel 失效**,不需要重新构建。(改 `examples/`、`config/`、`sensors/` 会——它们确实在 wheel 里,`wheel-inspection` 的 `bundle-fidelity` 会报出来。)
|
||||
|
||||
如果以后需要离线 API 参考,加一个只包含 `docs/api.md` 的窄条目即可,不要整个 `docs/` 都塞进去。
|
||||
|
||||
---
|
||||
|
||||
## 6. 缩短 manylinux 构建时间
|
||||
|
||||
`python build/run.py manylinux` 走 `build/wheel/docker/run.ps1` → 容器内 `entrypoint.sh`,对 `cp310/cp311/cp312` **串行**各做一次 Nuitka 编译 + 打 wheel + auditwheel repair + 隔离 venv 校验。
|
||||
|
||||
### 6.1 镜像里已经烘好的东西(所以每次构建不必再拉)
|
||||
|
||||
`Dockerfile.manylinux` 在每个 ABI 下预置了两类内容:
|
||||
|
||||
1. **构建工具**:`pip`/`setuptools`/`wheel`/`nuitka`/`ordered-set`/`zstandard`/`auditwheel`。
|
||||
因此 `entrypoint.sh` 默认**不再**在每个 ABI 上重跑 `pip install --upgrade pip setuptools wheel auditwheel` —— 那是每次构建 3 次多余的 PyPI 往返,还会让容器里的工具版本与镜像不一致。需要用最新工具时显式开启:
|
||||
|
||||
```powershell
|
||||
docker run ... -e ORISYS_UPDATE_BUILD_TOOLS=1 ...
|
||||
```
|
||||
|
||||
2. **每个 ABI 一个 wheelhouse**(`/opt/orisys-wheelhouse/<tag>`),内含 `numpy`/`scipy`/`opencv-python`/`numba` 及其依赖闭包,外加 `pip`/`setuptools`/`wheel`/`auditwheel` 的 wheel。
|
||||
|
||||
`--target pypi-linux` 会**强制**校验:为每个 ABI 建一个一次性 venv,再 `pip install` 刚构建的 wheel 和它声明的运行期依赖。镜像故意不把运行期依赖装进解释器(用户是凭 wheel 元数据装的),所以以前每次构建都要现下约 200 MB × 3。`entrypoint.sh` 现在用 `PIP_NO_INDEX=1` + `PIP_FIND_LINKS=<wheelhouse>` 把 pip 指向本地 wheelhouse:**venv 仍然是隔离的**(没有 `--system-site-packages`),依赖解析与安装依然真实发生,只是不走网络。
|
||||
|
||||
找不到 wheelhouse 时会打印警告并回退到联网,不会静默失败。镜像因此增大约 400–500 MB(一次性代价,换每次构建省三次下载)。
|
||||
|
||||
### 6.2 不要为了省事直接重建镜像
|
||||
|
||||
只改了源码(`src/`、`config/`、`sensors/`、`examples/`、`pyproject.toml`)时加 `-SkipImageBuild`:
|
||||
|
||||
```powershell
|
||||
.\build\wheel\docker\run.ps1 -SkipImageBuild
|
||||
```
|
||||
|
||||
Dockerfile 的 `RUN` 层要装 3 份工具链并下载 3 个 wheelhouse,缓存未命中就是数分钟。**改了 `Dockerfile.manylinux` 或需要更新 wheelhouse 时才重建镜像**。
|
||||
|
||||
### 6.3 Nuitka 的排除列表必须完整
|
||||
|
||||
`build/wheel/compile.py::NUITKA_NOFOLLOW_IMPORTS` 列出所有**不打进 wheel、运行期从用户环境解析**的第三方包。漏一个意味着 Nuitka 会尝试把它跟进来:编译时间暴涨,而且 wheel 内容开始取决于构建机上恰好装了什么 —— `torch`、`torchvision`(`src/orisys/tracker.py` 的 RAFT 追踪器)和 `pyvkfft`(`backend_availability.py`)就是这样漏掉的,之前只是靠「构建机上没装」才没出事。
|
||||
|
||||
`tests/test_nuitka_nofollow.py` 会扫描 `src/orisys` 的全部第三方导入,任何不在排除列表里的都会让测试失败。**新增可选依赖时同步更新列表。**
|
||||
|
||||
### 6.4 想要并行?先读这一段
|
||||
|
||||
`--jobs` 的默认值是**全部 CPU 数**,单次编译的 C 阶段已经吃满所有核心。真正能靠并行省下的是 Nuitka 的单线程前端(分析/优化/代码生成)、scons 启动、`auditwheel` 与 venv 校验 —— 做法是 3 路并行 + 每个 build 限 `--jobs=<核数/3>`。
|
||||
|
||||
但**现在不能直接并行**:`build/common/paths.py` 里的工作目录是写死的共享路径
|
||||
|
||||
```
|
||||
build/nuitka/compile
|
||||
build/nuitka/wheel
|
||||
build/nuitka/wheel/repaired
|
||||
```
|
||||
|
||||
而 `build/wheel/build.py` 每次开头都 `_reset_dir()`(`rmtree`)这两个目录,`auditwheel repair` 又会再 `rmtree` 一次 `repaired`。仓库是挂载进容器的(`-v ${RepoRoot}:/src`),所以无论「一个容器 3 路并行」还是「3 个容器并行」,只要共享同一个挂载点,三个 build 就会互相清空暂存目录 —— 表现为随机失败或 wheel 内容错乱。
|
||||
|
||||
要并行必须先让工作目录带上 ABI 标签(如 `build/nuitka/<tag>/compile`)。输出路径不用动:`SDK/<version>/`、`dist/repaired/` 的文件名本来就按 ABI 区分,下游 `release preflight` 不受影响。另外 3 个 Nuitka 并行很吃内存,先把 Docker Desktop 内存调到 16 GB 以上再试。
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# Orisys SDK
|
||||
|
||||
叠动科技视触觉传感器配套的 Python SDK,提供图像采集、拼接、稠密光流、Helmholtz-Hodge 分解、接触与深度估计,以及可运行的示例。
|
||||
|
||||
本文件是 PyPI 项目描述(`long_description`)的源文件:构建时由 `docs/publishing.md` 所述流程从**检出目录**读取(`build/wheel/package.py::PYPI_README_RELPATH`),**不会**被打进 wheel。因此不要在这里写内部仓库布局、构建流水线或发布步骤——那些内容属于仓库内的 `README.md` 与 `docs/publishing.md`。
|
||||
|
||||
```bash
|
||||
python -m pip install orisys
|
||||
```
|
||||
|
||||
支持 Python 3.10 / 3.11 / 3.12,Windows x64 与 Ubuntu x64。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```bash
|
||||
python -c "import orisys; print(orisys.__version__)"
|
||||
python -c "import orisys; print(orisys.list_config())"
|
||||
```
|
||||
|
||||
## 安装选项
|
||||
|
||||
- `orisys` — CPU / 基础流程
|
||||
- `orisys[gui]` — Qt / Open3D 可视化示例
|
||||
- `orisys[cuda12]` / `orisys[cuda13]` — NVIDIA GPU 加速
|
||||
- `orisys[gui,cuda12]` — GUI + CUDA 12
|
||||
|
||||
`orisys[cuda]` 与 `orisys[cuda12]` 等价。
|
||||
|
||||
## 配置
|
||||
|
||||
配置解析顺序:
|
||||
|
||||
1. 显式 JSON 文件路径
|
||||
2. 内置配置名(`box`、`mini`、`finger` 等)
|
||||
3. 产品别名(如 `dd02` -> `dd02-ov`)
|
||||
4. 向后兼容的 JSON `"name"` 匹配
|
||||
|
||||
```python
|
||||
import orisys
|
||||
|
||||
sensor = orisys.Sensor(0, config_name="box")
|
||||
orisys.export_config("box", "./exported_config.json")
|
||||
```
|
||||
|
||||
## 示例
|
||||
|
||||
示例随 wheel 一起分发,位于 `orisys/examples/`。也可以从项目仓库获取:
|
||||
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c box
|
||||
python examples/finger_pipeline.py --camera_id 0 --config finger
|
||||
```
|
||||
|
||||
无相机时可以改用视频文件:
|
||||
|
||||
```bash
|
||||
python examples/finger_pipeline.py --video ./sample.mp4 --config finger
|
||||
```
|
||||
|
||||
## 支持与反馈
|
||||
|
||||
- 官网:[https://www.orisys-tech.cn](https://www.orisys-tech.cn)
|
||||
- 问题反馈:GitHub Issues
|
||||
@@ -0,0 +1,161 @@
|
||||
# Sensor bundles and the processing contract
|
||||
|
||||
A sensor bundle is a folder that describes one processing profile. This document owns the bundle
|
||||
descriptor, the resolver, the backend registry and the fingerprints. `sensor.json` keeps its
|
||||
existing meaning: it is the processing configuration, and its `type` field is the image-processing
|
||||
arrangement (`compound_eye`, `single_camera`, `uv_unroll`) - never a product family.
|
||||
|
||||
## Why the descriptor exists
|
||||
|
||||
"Which pipeline processes these frames" used to be inferred from a product name. Those are four
|
||||
different things, and collapsing them is what made Mini work live but be refused in replay:
|
||||
|
||||
| Concept | Meaning | Example |
|
||||
|---|---|---|
|
||||
| Product family | What the operator calls it | Box, Mini, Finger |
|
||||
| Processing backend | Which SDK pipeline processes the images | `compound_eye`, `uv_unroll` |
|
||||
| Configuration profile | Camera, geometry and algorithm settings for one design revision | `orisys.mini.prototype-r1` |
|
||||
| Device instance | The physical unit | Serial number, its calibration |
|
||||
|
||||
Several families can share one backend, one family can have several profiles, and a recorded clip
|
||||
has no device instance at all - only a profile and, optionally, a compatible calibration.
|
||||
|
||||
## Descriptor
|
||||
|
||||
`manifest.json` in the bundle root. Every field except the five identifying keys is optional, so a
|
||||
bundle with no manifest still resolves through the compatibility adapter.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"profile_id": "orisys.mini.prototype-r1",
|
||||
"display_name": "Mini prototype R1",
|
||||
"product_family": "mini",
|
||||
"processing_backend": "compound_eye",
|
||||
"sensor_json": "sensor.json",
|
||||
"default_calibration": "force_calibration.json",
|
||||
"computational_assets": { "unroll_map": "assets/30fps-0703/unroll_map_uv.npz" },
|
||||
"assets": { "scene": "scene/finger-display-1.0.0.oryscene" }
|
||||
}
|
||||
```
|
||||
|
||||
- **`computational_assets`** are required to process at all. A missing one fails the profile.
|
||||
- **`assets`** are visualization only. A missing one is reported and never blocks processing - a
|
||||
missing 3D scene is not a missing deformation field.
|
||||
- Paths resolve relative to the bundle root, so moving or renaming the folder changes nothing.
|
||||
Absolute paths and `..` escapes are rejected at parse time.
|
||||
- An unsupported `processing_backend` or `schema_version` raises `DescriptorError`. The supported
|
||||
descriptor schema version is 1.
|
||||
|
||||
## Compatibility for existing bundles
|
||||
|
||||
`resolve_bundle` reads a new-shape descriptor when `schema_version` is present. Otherwise - no
|
||||
manifest, or the older `name`/`sensor_json`/`assets_dir`/`viz_pose` shape - the adapter describes
|
||||
it. That is the only place a folder name is consulted, and only for the built-in set:
|
||||
|
||||
| Folder | Product family | Backend (when the config does not decide) |
|
||||
|---|---|---|
|
||||
| `box` | box | `compound_eye` |
|
||||
| `mini` | mini | `compound_eye` |
|
||||
| `finger` | finger | `uv_unroll` |
|
||||
| `finger-mems` | finger-mems | `compound_eye` |
|
||||
|
||||
For a bundle without a descriptor the backend follows the configuration, using the rule
|
||||
`Sensor._load_config` already applies: `type` when present, else `compound_eye` when the config has
|
||||
a `stitcher` block, else `single_camera`. Only `finger` declares `type` today; `box`, `mini` and
|
||||
`finger-mems` are decided by the stitcher. A folder that is neither built-in nor carries a
|
||||
descriptor is rejected - never routed to `compound_eye` because box works that way.
|
||||
|
||||
## Resolver
|
||||
|
||||
```python
|
||||
from orisys import resolve_bundle, validate_profile, create_processor, CalibrationSelection
|
||||
|
||||
profile = resolve_bundle("sensors/mini") # or any bundle folder
|
||||
profile = resolve_bundle("sensors/mini", calibration_selection=CalibrationSelection.NONE)
|
||||
session = validate_profile(profile) # re-check against the filesystem now
|
||||
rebuilt = create_processor(profile, source=0) # live camera
|
||||
rebuilt = create_processor(profile, source="clip.mp4") # recorded frames, same semantics
|
||||
```
|
||||
|
||||
`resolve_bundle(bundle_path, *, calibration_selection=None) -> ResolvedSensorProfile`. The profile
|
||||
is plain data - no Qt, no application preferences, no device handle - so the same object drives a
|
||||
camera and a recorded clip. `_set_calibration` on a device affects only its own calibration
|
||||
selection.
|
||||
|
||||
`validate_profile(profile) -> ValidationReport` re-reads the filesystem. It reports
|
||||
`fingerprint.stale` when a computational asset changed since the profile was resolved.
|
||||
|
||||
`create_processor(profile, *, sensor=None, source=None, include=..., contact=...)` returns a
|
||||
`ProcessingSession` with `start`/`wait`/`latest`/`reset`/`pause`/`resume`/`stop`. It raises
|
||||
`ProcessorCreationError` naming what is missing rather than returning a session that does nothing.
|
||||
|
||||
### Backends
|
||||
|
||||
| Identifier | Requires | Produces |
|
||||
|---|---|---|
|
||||
| `compound_eye` | `camera`, `stitcher`, `tracker` config | stitched frame, flow, force |
|
||||
| `single_camera` | `camera` config | frames only - no deformation field, so no force |
|
||||
| `uv_unroll` | `camera` config, the mapped computational asset | unrolled frame, flow, force |
|
||||
|
||||
`known_backends()` and `spec_for(...)` are the registry. `uv_unroll` is not yet materialised from a
|
||||
profile alone, because it needs geometry, a UV domain, a tactile processor and a capture object
|
||||
assembled together - that is `orisys.finger.FingerRuntime.open`'s shape. `create_processor` says so
|
||||
instead of pretending.
|
||||
|
||||
## Capabilities
|
||||
|
||||
Each capability is decided by the facts it depends on, not by the presence of a file:
|
||||
|
||||
| Capability | Unavailable when |
|
||||
|---|---|
|
||||
| Video processing | the configuration cannot be loaded |
|
||||
| Optical flow | no `tracker` block |
|
||||
| Raw force | the backend produces no deformation field |
|
||||
| Calibrated force | per axis: the calibration has no curve for it |
|
||||
| Deformation | the backend computes no deformation field |
|
||||
| 3D visualization | no scene asset is declared, or the declared one is missing |
|
||||
|
||||
Deformation is relative unless a calibration says otherwise; it is reported with its meaning rather
|
||||
than as a bare "available". `ValidationReport.state` is `ok`, `degraded` (usable, something
|
||||
optional is missing) or `failed`.
|
||||
|
||||
## Calibration
|
||||
|
||||
`CalibrationSelection` is `BUNDLE_DEFAULT` (the default), `NONE`, or an explicit path.
|
||||
|
||||
- `NONE` is a choice, not a failure: raw force stays available and the calibration fingerprint
|
||||
records `none`.
|
||||
- An explicitly selected calibration that is missing or unparseable is an **error**. It is never
|
||||
replaced by another file.
|
||||
- Applicability is `declared` when the document carries `meta.sensor_type`, `unknown` when legacy
|
||||
metadata is absent, or `none`. `unknown` means it applies but its fit is unverified - which is
|
||||
what the shipped `box` calibration reports.
|
||||
|
||||
| Read | Meaning |
|
||||
|---|---|
|
||||
| `FrameSnapshot.normal_force_raw` / `shear_x_raw` / `shear_y_raw` | device counts, pre-calibration |
|
||||
| `FrameSnapshot.normal_force` / `shear_x` / `shear_y` | newtons when a curve applies, counts otherwise |
|
||||
| `FrameSnapshot.depth`, `centroid`, `contour`, `vnormal` | relative quantities, not newtons |
|
||||
|
||||
`Sensor.force_triplets()` and `_force_raw` remain the raw and unfiltered view.
|
||||
|
||||
## Fingerprints
|
||||
|
||||
`processing_fingerprint` hashes the effective backend, the paired data, and each computational asset's role name with
|
||||
its content hash. It excludes the bundle location, `display_name`, and every visualization asset: a
|
||||
relocated or cosmetically renamed bundle keeps its fingerprint, and a missing scene never changes it.
|
||||
It includes the config minus `name` and `version`.
|
||||
|
||||
`calibration_fingerprint` hashes the selected document's content, or the constant `none`.
|
||||
|
||||
`profile.inventory()` returns the reproducible snapshot: bundle-relative paths, asset hashes and
|
||||
both fingerprints. Compare an inventory, not a path, to decide whether two installations would
|
||||
compute the same numbers.
|
||||
|
||||
## Resetting the reference
|
||||
|
||||
Both backends process externally supplied frames through the same profile. After a gap in the
|
||||
input - a pause, a seek, a scene change - call `session.reset()`. It is queued to the stage that
|
||||
owns the tracker, so the reference frame is rebased on the frame about to be computed rather than
|
||||
on a buffer the pipeline is still reading.
|
||||
+68
-30
@@ -1,52 +1,90 @@
|
||||
# Examples
|
||||
|
||||
Top-level demo scripts under `examples/`. Run them from the repository root.
|
||||
Runnable demos live at the `examples/` root. Shared helpers stay in `pipeline/`, `viz/`, and
|
||||
`gui/`. Run commands from the repository root.
|
||||
|
||||
## Top-Level Files
|
||||
## Start here
|
||||
|
||||
- `README.md` — this overview of the root-level example entry points
|
||||
- `app_paths.py` — path helpers used by the packaged / frozen dashboard entry point
|
||||
- `basic_pipeline.py` — basic single-sensor OpenCV demo for the compound-eye pipeline
|
||||
- `plot_data.py` — single-sensor demo with OpenCV views plus PyQtGraph force charts
|
||||
- `two_sensor_pipeline.py` — two-sensor multiprocessing demo for parallel acquisition / display
|
||||
- `finger_pipeline.py` — minimal finger UV-unroll pipeline with optional profiling
|
||||
- `finger_pipeline_3dview.py` — main finger dashboard entry point with the unified Qt UI
|
||||
Both recommended examples create one object, then read a frame at a time. Everything else -
|
||||
camera reading, stitching, preprocessing, the optical-flow and force stages - happens inside
|
||||
`StreamPipeline`, and each frame arrives as one `FrameSnapshot`. There is no stage sequence to
|
||||
remember and no tuple to unpack, and no backend setting to tune.
|
||||
|
||||
## Typical Usage
|
||||
| Product | Entry | Default config |
|
||||
|---------|-------|----------------|
|
||||
| ORY-BOX | `basic_pipeline.py` | `box` |
|
||||
| ORY-MINI | `basic_pipeline.py -c mini` | `mini` |
|
||||
| ORY-FINGER | `finger_pipeline.py` | `finger` |
|
||||
|
||||
For the Qt dashboard:
|
||||
```python
|
||||
# ORY-BOX / ORY-MINI
|
||||
sensor = orisys.Sensor(0, config_name="box")
|
||||
with orisys.StreamPipeline(sensor, sensor.get_img, contact=True) as pipe:
|
||||
while (snap := pipe.wait()) is not None:
|
||||
print(snap.normal_force, snap.shear_x, snap.shear_y)
|
||||
|
||||
```bash
|
||||
pip install -e ".[gui]"
|
||||
python examples/finger_pipeline_3dview.py -v 0 -c ./config/dd02-ov.json
|
||||
# ORY-FINGER - the runtime owns camera, Blender export, unroll map and valid mask
|
||||
runtime = FingerRuntime.open(config="finger", video=0)
|
||||
with runtime.pipeline(contact=True) as pipe:
|
||||
while (snap := pipe.wait()) is not None:
|
||||
print(snap.normal_force, snap.shear_x, snap.shear_y)
|
||||
```
|
||||
|
||||
For the minimal finger pipeline:
|
||||
`contact=True` runs contact analysis; `include=(...)` chooses which arrays each snapshot carries
|
||||
(`image`, `flow`, `vnormal`, `vshear`, `depth`, `centroid`, `contour`). Arrays you do not include
|
||||
are `None`, so you pay only for what you use. `pipe.reset()` resets the tracker safely from your
|
||||
loop; `snap.image_size_raw` reports the capture resolution before UV unroll or resize.
|
||||
|
||||
### Typical usage
|
||||
|
||||
```bash
|
||||
python examples/finger_pipeline.py --camera_id 0 --config ./config/dd02-ov.json
|
||||
python examples/basic_pipeline.py -v 0 -c box # first camera, ORY-BOX
|
||||
python examples/basic_pipeline.py -v 0 -c mini # ORY-MINI
|
||||
python examples/basic_pipeline.py -v recording.mp4 -c box
|
||||
python examples/basic_pipeline.py --no-show # headless: print only
|
||||
python examples/finger_pipeline.py --camera_id 0 --config finger
|
||||
python examples/finger_pipeline.py --video data/dd02-0703/1.mp4
|
||||
```
|
||||
|
||||
For the basic compound-eye pipeline:
|
||||
Keys in both: `q` quits, `r` resets the tracker.
|
||||
|
||||
```bash
|
||||
python examples/basic_pipeline.py -v 0 -c ./config/ddjx01.json
|
||||
## The manual form, for when you need the stages
|
||||
|
||||
`examples/archive/` holds the previous versions of these two examples. They call the stages by
|
||||
hand:
|
||||
|
||||
```python
|
||||
sensor.get_img()
|
||||
sensor.compute_deformation()
|
||||
sensor.compute_contact()
|
||||
fps, flow, vnormal, img, ... = sensor.read_info( ... ) # positional, 13 elements
|
||||
```
|
||||
|
||||
That path is still supported and still correct - it is the right choice if you are implementing
|
||||
your own scheduling. It is no longer the recommended starting point, and the archived files carry
|
||||
a header explaining the difference. They are kept so that code copied from them keeps working.
|
||||
|
||||
## Other root examples
|
||||
|
||||
- `multi_stream.py` - several sensors at once, using `StreamPipeline` with `StreamRecorder`
|
||||
- `two_sensor_pipeline.py` - two-sensor multiprocessing demo, manual staging
|
||||
- `plot_data.py` - single-sensor OpenCV views plus PyQtGraph force charts
|
||||
- `finger_pipeline_3dview.py` - the Qt/open3d dashboard; see the note below
|
||||
|
||||
## Requirements
|
||||
|
||||
- SDK installed from the repository root, for example `pip install -e .`
|
||||
- camera index or test video path
|
||||
- matching config files under `./config/`
|
||||
- SDK installed from the repository root: `pip install -e .`
|
||||
- a camera index or a test video path
|
||||
- matching config names under `./config/` or the packaged `orisys/configs/`
|
||||
|
||||
Qt / PyQtGraph examples additionally require:
|
||||
Qt / PyQtGraph examples additionally need `pip install -e ".[gui]"`.
|
||||
|
||||
```bash
|
||||
pip install -e ".[gui]"
|
||||
```
|
||||
## The GUI lives here for now
|
||||
|
||||
## Notes
|
||||
`examples/gui/` - the 2D viewer, the 3D dashboard and their widgets - is a full Qt + open3d
|
||||
application rather than an example, and its proper home is the **orisys-studio** application.
|
||||
That move is **planned but not started**; `finger_pipeline_3dview.py` and `examples/gui/` work
|
||||
exactly as before. The plan, including the coupling that makes it non-trivial, is in
|
||||
`docs/plan-gui-extraction.md`.
|
||||
|
||||
- `finger_pipeline_3dview.py` is the main interactive demo. It uses `app_paths.py` so the same entry point works in editable and packaged runs.
|
||||
- Dashboard controls: click **启动** to start acquisition; `Q` quits, `R` resets, `S` starts recording, `E` stops recording, and `Esc` toggles fullscreen.
|
||||
- `finger_pipeline.py` is the leanest finger example if you want to validate UV unroll + force/contact processing without the full dashboard.
|
||||
The practical guidance is only that new SDK work should not build on `examples/gui/`.
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Display helpers shared by the pipeline examples.
|
||||
|
||||
These exist so an example can show a result without hand-rolling OpenCV arithmetic. Everything
|
||||
here is presentation only: the SDK never draws, and nothing in ``orisys`` imports this module.
|
||||
|
||||
The examples import it through a sibling import, which works when the examples are run from the
|
||||
examples directory or with ``examples/`` on ``sys.path``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
|
||||
def draw_flow_arrows(image, flow, *, threshold=0.4, grid_spacing=10, arrow_scale=1.0,
|
||||
valid_mask=None):
|
||||
"""Return the deformation field drawn over ``image``."""
|
||||
import orisys
|
||||
|
||||
return orisys.util.draw_arrows(
|
||||
image,
|
||||
flow,
|
||||
threshold=threshold,
|
||||
grid_spacing=grid_spacing,
|
||||
arrow_scale=arrow_scale,
|
||||
valid_mask=valid_mask,
|
||||
)
|
||||
|
||||
|
||||
def depth_to_color(depth_map):
|
||||
"""Normalise a depth/divergence map to 8-bit and apply a colormap for the ``depth`` window.
|
||||
|
||||
Returns ``None`` for an empty map, which is what ``read_info`` gives when the field was never
|
||||
generated, so a caller can skip the window instead of showing saturated noise.
|
||||
"""
|
||||
if depth_map is None or not isinstance(depth_map, np.ndarray) or depth_map.size == 0:
|
||||
return None
|
||||
lo = float(depth_map.min())
|
||||
hi = float(depth_map.max())
|
||||
if hi <= lo:
|
||||
scaled = np.zeros_like(depth_map, dtype=np.uint8)
|
||||
else:
|
||||
scaled = ((depth_map - lo) / (hi - lo) * 255.0).astype(np.uint8)
|
||||
return cv2.applyColorMap(scaled, cv2.COLORMAP_JET)
|
||||
|
||||
|
||||
def draw_contact(image, contour, centroid):
|
||||
"""Return ``image`` with the contact region and centroid drawn on it."""
|
||||
import orisys
|
||||
|
||||
return orisys.util.draw_contact(image, contour, centroid)
|
||||
@@ -0,0 +1,159 @@
|
||||
"""
|
||||
ORY-BOX:单传感器基础流程(**已归档的手写版本**)
|
||||
|
||||
本示例演示 Orisys SDK 的标准处理流程:
|
||||
1. 初始化传感器(摄像头或视频文件)
|
||||
2. 获取图像并计算形变
|
||||
3. 按需计算接触区域
|
||||
4. 读取并可视化结果
|
||||
|
||||
---
|
||||
|
||||
**已归档。请改用 `examples/basic_pipeline.py`。**
|
||||
|
||||
保留本文件是为了兼容已经复制过这段代码的用户。它逐级调用
|
||||
`get_img()` → `compute_deformation()` → `compute_contact()` → `read_info()`,
|
||||
需要自己记住调用顺序,并按下标解包一个 13 元组。
|
||||
|
||||
推荐写法是交给 `orisys.StreamPipeline`:读取、预处理、力/接触解算在内部并发完成,每一帧以
|
||||
单个 `FrameSnapshot` 交回:
|
||||
|
||||
with orisys.StreamPipeline(sensor, sensor.get_img, contact=True) as pipe:
|
||||
snap = pipe.wait() # 一个对象,一次调用
|
||||
print(snap.normal_force, snap.shear_x, snap.shear_y)
|
||||
|
||||
本文件里的 `sensor.reset()` 在 pipeline 中对应 `pipe.reset()`:它会排队到流程线程执行,
|
||||
不会与光流阶段争用同一帧缓冲。
|
||||
|
||||
**注意**:本文件用中文 `print`,在 cp1252 控制台(未设 UTF-8 的 Windows 终端)会抛
|
||||
`UnicodeEncodeError`。这是归档前就存在的行为,没有在这里修正——新示例的输出去掉了非 ASCII
|
||||
字符,因此在任何终端都能跑。
|
||||
|
||||
"""
|
||||
import os
|
||||
|
||||
os.environ["OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS"] = "0"
|
||||
import argparse
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
import orisys
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--video","-v", type=str, default="0", help="视频源:摄像头编号或视频文件路径")
|
||||
parser.add_argument("--config","-c", type=str, default="box", help="配置文件名称或路径")
|
||||
parser.add_argument("--verbose","-verbose", type=bool, default=True, help="是否输出详细日志")
|
||||
parser.add_argument("--cal_path","-cal", type=str, default=None, help="标定文件路径")
|
||||
args = parser.parse_args()
|
||||
|
||||
# 步骤 1:创建传感器对象(必需)
|
||||
# config_name 可以是内置配置名称,也可以是自定义配置文件路径
|
||||
# cal_path 为标定文件路径;每台设备建议使用独立的标定文件
|
||||
|
||||
# 输入既可以是摄像头编号,也可以是视频文件路径
|
||||
if args.video.isdigit():
|
||||
sensor = orisys.Sensor(int(args.video), config_name=args.config, verbose=args.verbose, cal_path=args.cal_path)
|
||||
else:
|
||||
sensor = orisys.Sensor(args.video, config_name=args.config, verbose=args.verbose, cal_path=args.cal_path)
|
||||
|
||||
print("\n按键说明:'q' 退出程序,'r' 重置追踪器。")
|
||||
|
||||
while True:
|
||||
# 步骤 2:数据处理
|
||||
# 步骤 2.1:获取并拼接图像(必需)
|
||||
sensor.get_img()
|
||||
# 步骤 2.2:计算形变场(必需);返回值表示当前帧是否检测到接触/运动
|
||||
is_contact_now = sensor.compute_deformation()
|
||||
# 步骤 2.3:仅在当前帧检测到接触/运动时计算接触区域
|
||||
# if is_contact_now:
|
||||
sensor.compute_contact()
|
||||
|
||||
|
||||
# 步骤 2.4:读取结果(FNORMAL/FSHEAR* 在有 sensors/box/force_calibration.json 时为标定后牛顿力;
|
||||
# *_RAW 始终为原始积分量)
|
||||
(
|
||||
fps,
|
||||
flow,
|
||||
vnormal,
|
||||
img,
|
||||
contour,
|
||||
centroid,
|
||||
depth_map,
|
||||
fnormal,
|
||||
fshearx,
|
||||
fsheary,
|
||||
fnormal_raw,
|
||||
fshearx_raw,
|
||||
fsheary_raw,
|
||||
) = sensor.read_info(
|
||||
sensor.info.FPS,
|
||||
sensor.info.VRAW,
|
||||
sensor.info.VNORMAL,
|
||||
sensor.info.IMG,
|
||||
sensor.info.CONTOUR,
|
||||
sensor.info.CENTROID,
|
||||
sensor.info.DEPTH,
|
||||
sensor.info.FNORMAL,
|
||||
sensor.info.FSHEARX,
|
||||
sensor.info.FSHEARY,
|
||||
sensor.info.FNORMAL_RAW,
|
||||
sensor.info.FSHEARX_RAW,
|
||||
sensor.info.FSHEARY_RAW,
|
||||
)
|
||||
|
||||
# 步骤 3:可视化显示
|
||||
# 步骤 3.1:显示形变矢量场
|
||||
arrows = orisys.util.draw_arrows(
|
||||
img,
|
||||
flow,
|
||||
threshold=2,
|
||||
grid_spacing=20,
|
||||
arrow_scale=1.0
|
||||
)
|
||||
cv2.imshow("arrow", arrows)
|
||||
|
||||
# 步骤 3.2:显示接触区域和质心
|
||||
image_with_foe = orisys.util.draw_contact(img, contour, centroid)
|
||||
cv2.imshow("contact", image_with_foe)
|
||||
|
||||
# 步骤 3.3:显示深度图
|
||||
# 将深度值归一化到 0-255 范围
|
||||
div_abs = depth_map
|
||||
if div_abs.max() > div_abs.min():
|
||||
div_normalized = ((div_abs - div_abs.min()) / (div_abs.max() - div_abs.min()) * 255).astype(np.uint8)
|
||||
else:
|
||||
div_normalized = np.zeros_like(div_abs, dtype=np.uint8)
|
||||
|
||||
# 应用伪彩色映射
|
||||
cv2.imshow("depth", cv2.applyColorMap(div_normalized, cv2.COLORMAP_JET))
|
||||
|
||||
# 步骤 3.4:打印关键结果(标定力 + 原始力)
|
||||
print(
|
||||
f"FPS={fps:.2f}, "
|
||||
f"法向力={fnormal:.4f} N (raw={fnormal_raw:.4f}), "
|
||||
f"切向力X={fshearx:.4f} N (raw={fshearx_raw:.4f}), "
|
||||
f"切向力Y={fsheary:.4f} N (raw={fsheary_raw:.4f}), "
|
||||
f"深度图尺寸={depth_map.shape}, 原始分辨率={sensor.img_size_raw}"
|
||||
)
|
||||
|
||||
# 步骤 3.5:键盘控制
|
||||
key = cv2.waitKey(1) & 0xFF
|
||||
if key == ord("q"):
|
||||
print("\n正在退出示例程序...")
|
||||
break
|
||||
if key == ord("r"):
|
||||
# 当光流追踪出现漂移时,可手动重置追踪器
|
||||
sensor.reset()
|
||||
print("追踪器已重置。")
|
||||
|
||||
# =========================================================================
|
||||
# 4. 释放资源
|
||||
# =========================================================================
|
||||
sensor.disconnect() # 释放视频源及相关资源
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
ORY-FINGER: Minimum UV-unroll finger tactile pipeline. **ARCHIVED — manual version.**
|
||||
|
||||
Offline: load Blender export -> build_or_load_domain -> unroll_map_uv.npz
|
||||
Runtime: camera frame -> FrameUnwarper -> TactileProcessor -> forces/contact
|
||||
|
||||
Recommended entry:
|
||||
python examples/finger_pipeline.py --camera_id 0 --config finger
|
||||
|
||||
Current product config: ``finger`` (``dd02`` still resolves to ``dd02-ov``).
|
||||
|
||||
---
|
||||
|
||||
**ARCHIVED. Use ``examples/finger_pipeline.py`` instead.**
|
||||
|
||||
Kept so code copied from this file keeps working. It hand-sequences ``grab_and_process()`` →
|
||||
``compute_deformation()`` → ``compute_contact()`` → ``read_info()`` and unpacks a ten-element
|
||||
tuple by position.
|
||||
|
||||
The recommended form lets :class:`orisys.StreamPipeline` run those stages concurrently and hand
|
||||
back one immutable snapshot per frame:
|
||||
|
||||
with runtime.pipeline(contact=True, include=("image", "depth")) as pipe:
|
||||
snap = pipe.wait()
|
||||
print(snap.normal_force, snap.shear_x, snap.shear_y)
|
||||
|
||||
``tactile.reset()`` becomes ``pipe.reset()``, which is queued onto the flow-stage thread instead of
|
||||
racing it.
|
||||
|
||||
This archived version also has a defect: ``--show`` is declared ``store_true`` with
|
||||
``default=True``, so it can never be turned off. The replacement uses ``--no-show``.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
os.environ.setdefault("OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS", "0")
|
||||
|
||||
import cv2
|
||||
|
||||
import orisys
|
||||
from orisys.finger import FingerRuntime
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Minimum UV-unroll finger tactile pipeline")
|
||||
parser.add_argument("--camera_id", type=int, default=0)
|
||||
parser.add_argument("--video","-v", help="Video file path (overrides camera)")
|
||||
parser.add_argument("--config", "-c", default="finger", help="Orisys SDK config name or path")
|
||||
parser.add_argument("--show", action="store_true", default=True, help="Show unrolled image window")
|
||||
parser.add_argument("--verbose", action="store_true", help="SDK verbose logs")
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
video = args.camera_id
|
||||
if args.video:
|
||||
video = args.video
|
||||
runtime = FingerRuntime.open(
|
||||
config=args.config,
|
||||
video=video,
|
||||
verbose=args.verbose,
|
||||
)
|
||||
except RuntimeError as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.verbose:
|
||||
geometry = runtime.geometry
|
||||
print(
|
||||
f"UV geometry: native={geometry.native_width}x{geometry.native_height}, "
|
||||
f"source={geometry.source_width}x{geometry.source_height} "
|
||||
f"output={geometry.out_width}x{geometry.out_height} "
|
||||
)
|
||||
|
||||
runtime.warmup(color=False)
|
||||
tactile = runtime.tactile
|
||||
|
||||
print("Press 'q' to quit, 'r' to reset tracker.")
|
||||
try:
|
||||
while True:
|
||||
ok, unrolled = runtime.grab_and_process(color=False)
|
||||
if not ok:
|
||||
break
|
||||
|
||||
tactile.compute_deformation()
|
||||
tactile.compute_contact()
|
||||
|
||||
fps, flow, vnormal, img, contour, centroid, depth_map, fnormal, fshearx, fsheary = tactile.read_info(
|
||||
tactile.info.FPS,
|
||||
tactile.info.VRAW,
|
||||
tactile.info.VNORMAL,
|
||||
tactile.info.IMG,
|
||||
tactile.info.CONTOUR,
|
||||
tactile.info.CENTROID,
|
||||
tactile.info.DEPTH,
|
||||
tactile.info.FNORMAL,
|
||||
tactile.info.FSHEARX,
|
||||
tactile.info.FSHEARY,
|
||||
)
|
||||
|
||||
status = (
|
||||
f"FPS={fps:.2f} normal={fnormal:.2f} shear=({fshearx:.2f}, {fsheary:.2f}) "
|
||||
f"mask={tactile.mask_stats['coverage']:.2f}"
|
||||
)
|
||||
|
||||
if args.show:
|
||||
arrows = orisys.util.draw_arrows(
|
||||
unrolled,
|
||||
flow,
|
||||
threshold=0.4,
|
||||
grid_spacing=10,
|
||||
arrow_scale=1.0,
|
||||
valid_mask=tactile.valid_mask,
|
||||
)
|
||||
cv2.imshow("UV Unrolled", arrows)
|
||||
|
||||
print(status, end="\r")
|
||||
|
||||
key = cv2.waitKey(1) & 0xFF
|
||||
if key == ord("q"):
|
||||
print("\nExiting.")
|
||||
break
|
||||
if key == ord("r"):
|
||||
tactile.reset()
|
||||
print("\nTracker reset.")
|
||||
finally:
|
||||
runtime.close()
|
||||
cv2.destroyAllWindows()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+131
-104
@@ -1,115 +1,142 @@
|
||||
"""
|
||||
示例 1:单传感器基础流程
|
||||
"""ORY-BOX / ORY-MINI: read a sensor and get tactile information, in one loop.
|
||||
|
||||
本示例演示 Orisys SDK 的标准处理流程:
|
||||
1. 初始化传感器(摄像头或视频文件)
|
||||
2. 获取图像并计算形变
|
||||
3. 按需计算接触区域
|
||||
4. 读取并可视化结果
|
||||
The whole recommended flow:
|
||||
|
||||
SDK version: 0.3.1
|
||||
sensor -> StreamPipeline -> FrameSnapshot per frame
|
||||
|
||||
One object reads the camera, pre-processes (stitch, grey, resize), computes the deformation field
|
||||
and the forces, and hands back one immutable snapshot per frame. There is no stage sequence to
|
||||
remember and no tuple to unpack by position.
|
||||
|
||||
What you do NOT need to touch: the compute backend, the capture queue, the stitch mode, the
|
||||
low-pass filter, or the order in which the stages run. Those are the SDK's concern and
|
||||
``StreamPipeline`` sequences them on three concurrent threads. If you later need that control, the
|
||||
manual form is preserved in ``examples/archive/basic_pipeline_manual.py``.
|
||||
|
||||
Run:
|
||||
|
||||
python examples/basic_pipeline.py # first camera, box config
|
||||
python examples/basic_pipeline.py -v 0 -c mini
|
||||
python examples/basic_pipeline.py -v recording.mp4 -c box
|
||||
python examples/basic_pipeline.py --no-show # headless: print only
|
||||
python examples/basic_pipeline.py -v 0 -c box --calibration-dir data/calibrations --device-id BOX-001
|
||||
|
||||
Keys: 'q' quit, 'r' reset the tracker.
|
||||
"""
|
||||
import os
|
||||
os.environ["OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS"] = "0"
|
||||
import cv2
|
||||
import orisys
|
||||
import numpy as np
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--video","-v", type=str, default="0", help="视频源:摄像头编号或视频文件路径")
|
||||
parser.add_argument("--config","-c", type=str, default="./config/ddjx01.json", help="配置文件名称或路径")
|
||||
parser.add_argument("--verbose","-verbose", type=bool, default=True, help="是否输出详细日志")
|
||||
# Set before OpenCV is imported: stops the capture backend applying its own transform on top of
|
||||
# the SDK's preprocessing. This is a capture detail, not a tuning knob you need to own - it is
|
||||
# here so the file can be copied as a whole.
|
||||
os.environ.setdefault("OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS", "0")
|
||||
|
||||
import cv2
|
||||
|
||||
import orisys
|
||||
from _display import depth_to_color, draw_contact, draw_flow_arrows
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Read a sensor and print its tactile information.")
|
||||
parser.add_argument("-v", "--video", default="0",
|
||||
help="camera index or video file path (default: 0)")
|
||||
parser.add_argument("-c", "--config", default="box",
|
||||
help="product config: box, mini, finger, or a config path")
|
||||
calibration = parser.add_mutually_exclusive_group()
|
||||
calibration.add_argument("--cal", default=None,
|
||||
help="force calibration file; omit to use the sensor bundle's own")
|
||||
calibration.add_argument("--calibration-dir", default=None,
|
||||
help="root containing <product>/<device-id>.json")
|
||||
parser.add_argument("--device-id", default=None,
|
||||
help="physical sensor ID; required with --calibration-dir")
|
||||
parser.add_argument("--no-show", dest="show", action="store_false",
|
||||
help="do not open the display windows")
|
||||
parser.add_argument("-verbose", action="store_true", help="SDK verbose logs")
|
||||
args = parser.parse_args()
|
||||
if args.calibration_dir is not None and args.device_id is None:
|
||||
parser.error("--calibration-dir requires --device-id")
|
||||
if args.device_id is not None and args.calibration_dir is None and args.cal is None:
|
||||
parser.error("--device-id requires --calibration-dir or --cal")
|
||||
|
||||
# 步骤 1:创建传感器对象(必需)
|
||||
# config_name 可以是内置配置名称,也可以是自定义配置文件路径
|
||||
# cal_path 为标定文件路径;每台设备建议使用独立的标定文件
|
||||
|
||||
# 输入既可以是摄像头编号,也可以是视频文件路径
|
||||
try:
|
||||
input = int(args.video)
|
||||
is_camera = True
|
||||
sensor = orisys.Sensor(input, config_name=args.config, verbose=args.verbose)
|
||||
except:
|
||||
is_camera = False
|
||||
sensor = orisys.Sensor(args.video, config_name=args.config, verbose=args.verbose)
|
||||
|
||||
print("\n按键说明:'q' 退出程序,'r' 重置追踪器。")
|
||||
|
||||
while True:
|
||||
# 步骤 2:数据处理
|
||||
# 步骤 2.1:获取并拼接图像(必需)
|
||||
sensor.get_img()
|
||||
# 步骤 2.2:计算形变场(必需);返回值表示当前帧是否检测到接触/运动
|
||||
is_contact_now = sensor.compute_deformation(check_motion=True, threshold=0)
|
||||
# 步骤 2.3:仅在当前帧检测到接触/运动时计算接触区域
|
||||
# if is_contact_now:
|
||||
sensor.compute_contact()
|
||||
|
||||
|
||||
# 步骤 2.4:读取结果
|
||||
fps, flow, vnormal, img, contour, centroid, depth_map, fnormal, fshearx, fsheary = sensor.read_info(
|
||||
sensor.info.FPS,
|
||||
sensor.info.VRAW,
|
||||
sensor.info.VNORMAL,
|
||||
sensor.info.IMG,
|
||||
sensor.info.CONTOUR,
|
||||
sensor.info.CENTROID,
|
||||
sensor.info.DEPTH,
|
||||
sensor.info.FNORMAL,
|
||||
sensor.info.FSHEARX,
|
||||
sensor.info.FSHEARY
|
||||
)
|
||||
|
||||
# 步骤 3:可视化显示
|
||||
# 步骤 3.1:显示形变矢量场
|
||||
arrows = orisys.util.draw_arrows(
|
||||
img,
|
||||
flow,
|
||||
threshold=2,
|
||||
grid_spacing=20,
|
||||
arrow_scale=1.0
|
||||
)
|
||||
cv2.imshow("形变矢量场", arrows)
|
||||
|
||||
# 步骤 3.2:显示接触区域和质心
|
||||
image_with_foe = orisys.util.draw_contact(img, contour, centroid)
|
||||
cv2.imshow("接触区域与质心", image_with_foe)
|
||||
|
||||
# 步骤 3.3:显示深度图
|
||||
# 将深度值归一化到 0-255 范围
|
||||
div_abs = depth_map
|
||||
if div_abs.max() > div_abs.min():
|
||||
div_normalized = ((div_abs - div_abs.min()) / (div_abs.max() - div_abs.min()) * 255).astype(np.uint8)
|
||||
else:
|
||||
div_normalized = np.zeros_like(div_abs, dtype=np.uint8)
|
||||
|
||||
# 应用伪彩色映射
|
||||
cv2.imshow("深度图", cv2.applyColorMap(div_normalized, cv2.COLORMAP_JET))
|
||||
|
||||
# 步骤 3.4:打印关键结果
|
||||
# ------------------------------------------------------------------ create the object
|
||||
# That is the whole setup. The source may be a camera index or a video path.
|
||||
source = int(args.video) if args.video.isdigit() else args.video
|
||||
sensor = orisys.Sensor(
|
||||
source, config_name=args.config, cal_path=args.cal,
|
||||
calibration_dir=args.calibration_dir, device_id=args.device_id,
|
||||
verbose=args.verbose,
|
||||
)
|
||||
fn_unit, fx_unit, fy_unit = sensor.force_units
|
||||
selected = sensor.calibration.source_path if sensor.calibration else None
|
||||
print(f"Force calibration: {selected or 'none'}")
|
||||
print(f"Force units: FN={fn_unit}, FX={fx_unit}, FY={fy_unit}")
|
||||
if sensor.device_calibration_missing:
|
||||
print(
|
||||
f"FPS={fps:.2f}, 法向力={fnormal:.4f}, 切向力X={fshearx:.4f}, "
|
||||
f"切向力Y={fsheary:.4f}, 深度图尺寸={depth_map.shape}, 原始分辨率={sensor.img_size_raw}"
|
||||
f"Warning: no calibration for {args.device_id!r} under "
|
||||
f"{args.calibration_dir}; force is raw."
|
||||
)
|
||||
|
||||
# 步骤 3.5:键盘控制
|
||||
key = cv2.waitKey(1) & 0xFF
|
||||
if key == ord("q"):
|
||||
print("\n正在退出示例程序...")
|
||||
break
|
||||
if key == ord("r"):
|
||||
# 当光流追踪出现漂移时,可手动重置追踪器
|
||||
sensor.reset()
|
||||
print("追踪器已重置。")
|
||||
|
||||
# =========================================================================
|
||||
# 4. 释放资源
|
||||
# =========================================================================
|
||||
sensor.disconnect() # 释放视频源及相关资源
|
||||
|
||||
# contact=True runs contact analysis. include=... selects which arrays each snapshot carries;
|
||||
# arrays that are not included are None, so you pay only for what you use.
|
||||
include = ("image", "flow", "depth", "centroid", "contour") if args.show else ("image",)
|
||||
|
||||
if args.show:
|
||||
print("Keys: 'q' quit, 'r' reset tracker.")
|
||||
|
||||
try:
|
||||
with orisys.StreamPipeline(sensor, sensor.get_img, contact=True,
|
||||
include=include) as pipe:
|
||||
while True:
|
||||
# ------------------------------------------------------ read one frame
|
||||
# Blocks until a frame newer than the last one is ready. Everything in the
|
||||
# snapshot belongs to the same frame.
|
||||
snap = pipe.wait()
|
||||
if snap is None:
|
||||
print("\nInput ended.")
|
||||
break
|
||||
|
||||
# ------------------------------------------------------ use the results
|
||||
# The *_raw pair is always the unfiltered integrator output.
|
||||
touching = "yes" if snap.centroid is not None else "no"
|
||||
fn_unit, fx_unit, fy_unit = sensor.force_units
|
||||
print(
|
||||
f"FPS={snap.fps:5.1f} "
|
||||
f"normal={snap.normal_force:8.4f} {fn_unit} (raw {snap.normal_force_raw:10.2f}) "
|
||||
f"shear=({snap.shear_x:7.4f} {fx_unit}, {snap.shear_y:7.4f} {fy_unit}) "
|
||||
f"contact={touching} age={snap.age_ms:4.1f} ms",
|
||||
end="\r",
|
||||
)
|
||||
|
||||
if args.show and snap.image is not None:
|
||||
if snap.flow is not None:
|
||||
cv2.imshow("deformation", draw_flow_arrows(
|
||||
snap.image, snap.flow, threshold=2, grid_spacing=20))
|
||||
if snap.contour is not None and snap.centroid is not None:
|
||||
cv2.imshow("contact", draw_contact(
|
||||
snap.image, snap.contour, snap.centroid))
|
||||
colour = depth_to_color(snap.depth)
|
||||
if colour is not None:
|
||||
cv2.imshow("depth", colour)
|
||||
|
||||
key = cv2.waitKey(1) & 0xFF if args.show else -1
|
||||
if key == ord("q"):
|
||||
print("\nExiting.")
|
||||
break
|
||||
if key == ord("r"):
|
||||
# Queued onto the flow stage, so this is safe from the consumer loop.
|
||||
pipe.reset()
|
||||
print("\nTracker reset.")
|
||||
except KeyboardInterrupt:
|
||||
print("\nInterrupted.")
|
||||
finally:
|
||||
sensor.disconnect()
|
||||
if args.show:
|
||||
cv2.destroyAllWindows()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
# 滤波预设:可直接复制的 `filters` 段
|
||||
|
||||
`json.load` 不支持注释,所以这个目录就是"在配置里写说明"的替代品:**把下面任意一段整块替换**
|
||||
到 `sensors/<name>/sensor.json` 的 `"filters"` 段即可。原有的 `filters` 段请整段替换,
|
||||
不要只改其中一行(避免出现 `preset` 与数值混写的歧义——显式键优先于预设值,混写容易看错)。
|
||||
|
||||
配置里**没写**的滤波键等于"关":整段删掉就是什么都不滤,和 `off` 那段等价。
|
||||
|
||||
背景、代价与可追溯性约定见 `docs/filters.md`;延迟/噪声实测见
|
||||
`D:\orisys_ws\docs\performance\orisys-sdk-perf-plan.md`(原 `_perf/` 已从本仓库移除:幸存脚本与
|
||||
片源在 `D:\orisys_ws\docs\performance\perf-measurements\`,该文档保留全部实测数字)。
|
||||
|
||||
下面每一段都有一个同目录的机器可读副本 `filters_<预设名>.json`(四个文件),
|
||||
`tests/test_filter_presets.py` 会核对它们与 `FILTER_PRESETS` 完全一致 —— 所以改预设时
|
||||
这些文件会一起红,不会悄悄过时。
|
||||
|
||||
---
|
||||
|
||||
## `off` —— 原始读数(默认)
|
||||
|
||||
```json
|
||||
"filters": {
|
||||
"flow_ema_alpha": 1.0,
|
||||
"force_median_window": 1,
|
||||
"force_lowpass_cutoff_hz": 0.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## `low_latency` —— 只去尖峰(≈1 帧 @30fps)
|
||||
|
||||
```json
|
||||
"filters": {
|
||||
"flow_ema_alpha": 1.0,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 0.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## `balanced` —— 建议默认(≈2 帧 @30fps)
|
||||
|
||||
```json
|
||||
"filters": {
|
||||
"flow_ema_alpha": 0.5,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 0.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## `low_noise` —— 底噪优先(≈3 帧 @30fps)
|
||||
|
||||
```json
|
||||
"filters": {
|
||||
"flow_ema_alpha": 0.3,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 10.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
## 用预设名(等价写法,数值由代码展开)
|
||||
|
||||
```json
|
||||
"filters": { "preset": "balanced" }
|
||||
```
|
||||
|
||||
`off / low_latency / balanced / low_noise` 四个名字来自
|
||||
`src/orisys/sensor.py::FILTER_PRESETS`(单一真值);想从预设出发只改一个旋钮,就在同一段里写
|
||||
那一个键,例如:
|
||||
|
||||
```json
|
||||
"filters": { "preset": "balanced", "force_lowpass_cutoff_hz": 10.0 }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 复制后请核对"实际生效值"
|
||||
|
||||
配置只是**意图**;`Sensor.configure()` 可以在运行时再改,所以审计与记录一律看生效值:
|
||||
|
||||
```python
|
||||
from orisys.sensor import Sensor
|
||||
s = Sensor("external", config_name="mini", verbose=False) # 无需硬件即可核对
|
||||
print(s.filters_state()["preset"], s.filters_state()["measurement"], s.filters_state()["readout"])
|
||||
```
|
||||
|
||||
`flow_ema_applied` / `force_median_applied` / `force_lowpass_applied` 三个布尔就是
|
||||
"这一级到底有没有在滤"的答案;`flow_ema_alpha` 属于 **measurement**(改变测量,
|
||||
不同值的两次录制不可比),中值与低通属于 **readout**(只影响读数呈现)。
|
||||
|
||||
导出标定/写记录时把同一个状态交给指纹函数,文件里就会留下可核对的设置:
|
||||
|
||||
```python
|
||||
from orisys.calibration import pipeline_from_sensor_json
|
||||
pipeline = pipeline_from_sensor_json("sensors/mini/sensor.json", filters_state=s.filters_state())
|
||||
print(pipeline["filters"]) # measurement / readout / preset / source
|
||||
```
|
||||
|
||||
详见 `docs/filters.md` 第 7 节。
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"flow_ema_alpha": 0.5,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 0.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"flow_ema_alpha": 1.0,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 0.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"flow_ema_alpha": 0.3,
|
||||
"force_median_window": 3,
|
||||
"force_lowpass_cutoff_hz": 10.0,
|
||||
"force_lowpass_order": 2,
|
||||
"force_lowpass_sample_rate_hz": "auto"
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user