构建:从打包数据中剔除 .gitignore 等版本控制残留并加校验

collect_data_files("orisys") 会把 SDK 包内受 git 跟踪的 src/orisys/.gitignore 一并拷进 _internal\orisys;
现在 spec 统一过滤版本控制与缓存文件名,build.ps1 第 6 步对我们自己发布的三个包硬失败、其它位置仅告警。
验证:清理重建 exit 0,dist 内 .git* / __pycache__ / 缓存文件 0 命中,Open3D 检查通过,冒烟测试
打印 "the application started" 与 "the packaged 3D scene loaded",输出 488.7 MB。
This commit is contained in:
ykcheungab
2026-10-10 16:17:44 +08:00
parent f94b51e2de
commit c4663ef769
3 changed files with 89 additions and 11 deletions
+8 -4
View File
@@ -97,6 +97,7 @@ script checks the things that only show up when the EXE runs:
| `collect_data_files('orisys_app_shared', includes=['theme/*.json'])` is non-empty | the theme colours are package data; without them the app dies in `build_qss()` before logging anything | | `collect_data_files('orisys_app_shared', includes=['theme/*.json'])` is non-empty | the theme colours are package data; without them the app dies in `build_qss()` before logging anything |
| every `config\sensors\**\*.oryscene` is present, and the finger manifest/`sensor.json`/unroll map survive the config strip | the 3D pane refuses the scene when the device description cannot be derived | | every `config\sensors\**\*.oryscene` is present, and the finger manifest/`sensor.json`/unroll map survive the config strip | the 3D pane refuses the scene when the device description cannot be derived |
| no `open3d*` under `dist` | the offline tool leaking into the shipped bundle | | no `open3d*` under `dist` | the offline tool leaking into the shipped bundle |
| no source-control or cache files in the bundle (`_without_residue` in the spec, step 6 of the script) | a `.gitignore`, `__pycache__` or `.scratch` from the source trees shipping inside `dist` — the SDK keeps a `.gitignore` *inside* the `orisys` package, so `collect_data_files('orisys')` copies it |
| **smoke test**: start the produced EXE, wait for its own log to say `Studio shell started` (and `3D scene loaded` when a finger scene shipped), then stop it | a bundle that cannot start at all — missing runtime DLLs, an excluded module, missing data. The log is kept under `build\_smoke\` | | **smoke test**: start the produced EXE, wait for its own log to say `Studio shell started` (and `3D scene loaded` when a finger scene shipped), then stop it | a bundle that cannot start at all — missing runtime DLLs, an excluded module, missing data. The log is kept under `build\_smoke\` |
The smoke test seeds a `preference.json` beside the EXE to select the finger sensor with the The smoke test seeds a `preference.json` beside the EXE to select the finger sensor with the
@@ -116,13 +117,15 @@ never sources.
```text ```text
_internal\orisys_app_shared\theme\colors_dark.json <- package data, bundled (qss.py reads it) _internal\orisys_app_shared\theme\colors_dark.json <- package data, bundled (qss.py reads it)
_internal\orisys_app_shared\theme\colors_light.json _internal\orisys_app_shared\theme\colors_light.json
_internal\orisys\.gitignore <- stray non-.py file of the SDK package
_internal\branding\... <- logos _internal\branding\... <- logos
``` ```
(`collect_data_files("orisys")` exists for the case where the SDK is installed from a wheel that (`collect_data_files("orisys")` exists for the case where the SDK is installed from a wheel that
bundles `sensors/` assets; with a local editable checkout the only non-`.py` file is that bundles `sensors/` assets. With a local editable checkout its only non-`.py` file *was* the SDK's
`.gitignore`. It is harmless — the app prefers `config\sensors` — but it does look odd.) own `src\orisys\.gitignore` — a repository file with no business inside an importable package — so
it landed here as `_internal\orisys\.gitignore`. The spec now filters source-control and cache
files out of every `datas` entry, and step 6 of the script fails the build if any survive in a
package we ship.)
To list what is really in the archive: To list what is really in the archive:
@@ -185,7 +188,8 @@ Expect ~450 MB total (Qt + numpy/scipy/numba + OpenCV + PyOpenGL).
appear next to the EXE. appear next to the EXE.
4. Camera preview + box/mini/finger Run (video) with image modes and force plots. 4. Camera preview + box/mini/finger Run (video) with image modes and force plots.
5. Open the 3D pane on a finger source: the surface draws, the legend shows the field. 5. Open the 3D pane on a finger source: the surface draws, the legend shows the field.
6. Confirm no `open3d*` files under `dist\OrisysStudio`. 6. Confirm no `open3d*` files under `dist\OrisysStudio`, and no `.gitignore`/`__pycache__`/
`.scratch` anywhere inside it (step 6 of the build fails on residue in our own packages).
## Editable run (unchanged) ## Editable run (unchanged)
+32 -3
View File
@@ -11,7 +11,7 @@
1. make sure that provisioning happened (and run `setup.ps1` if it plainly has not), 1. make sure that provisioning happened (and run `setup.ps1` if it plainly has not),
2. verify the environment can import what the frozen app imports, 2. verify the environment can import what the frozen app imports,
3. build, 3. build,
4. verify the artefact — config, excludes, and a real start-up of the produced EXE. 4. verify the artefact — config, excludes, residue, and a real start-up of the produced EXE.
Step 2 and step 4 exist because the interesting failures are silent ones: a stale venv Step 2 and step 4 exist because the interesting failures are silent ones: a stale venv
still builds (the spec's pathex finds the modules) yet loses the theme data; an excluded still builds (the spec's pathex finds the modules) yet loses the theme data; an excluded
@@ -177,7 +177,36 @@ if ($o3dHits) {
Write-Host "Open3D check: none found under dist." Write-Host "Open3D check: none found under dist."
} }
# ----------------------------------------------------------------- 6. smoke test # -------------------------------------------- 6. no source-control or cache residue
# PyInstaller copies whatever data a package declares, and `collect_data_files("orisys")`
# copies every non-Python file the SDK package holds — including the tracked
# `src/orisys/.gitignore`, which is how `_internal\orisys\.gitignore` got shipped. A file
# like that is not a crash, so nothing else in this script would notice it; this check is
# the only thing that keeps it from coming back. It is a hard failure inside the bundles we
# own, and a warning elsewhere, where the file may belong to a third-party wheel.
$residueNames = '^(\.git.*|\.gitkeep|__pycache__|\.pytest_cache|\.mypy_cache|\.ruff_cache|\.scratch|\.tox|\.idea|\.vscode)$'
$ourPackages = @("_internal\orisys", "_internal\orisys_app_shared", "_internal\orisys_studio")
$residue = @(Get-ChildItem $OutDir -Recurse -Force -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match $residueNames })
$ownedResidue = @()
foreach ($item in $residue) {
$relative = $item.FullName.Substring($OutDir.Length + 1)
foreach ($package in $ourPackages) {
if ($relative -like "$package\*") { $ownedResidue += $item; break }
}
}
if ($ownedResidue.Count -gt 0) {
Write-Host "Residue inside the bundles we own:"
$ownedResidue | ForEach-Object { Write-Host (" " + $_.FullName.Substring($OutDir.Length + 1)) }
throw "the bundle ships $($ownedResidue.Count) source-control/cache file(s); drop them in build\studio\orisys-studio.spec and rebuild"
} elseif ($residue.Count -gt 0) {
Write-Warning "Residue outside our packages ($($residue.Count) item(s)), left alone:"
$residue | ForEach-Object { Write-Warning (" " + $_.FullName.Substring($OutDir.Length + 1)) }
} else {
Write-Host "Residue check: no .git* / __pycache__ / cache files under dist."
}
# ----------------------------------------------------------------- 7. smoke test
# Start the EXE the way a user would and require the application's own log to say it came up. # Start the EXE the way a user would and require the application's own log to say it came up.
# This is the step that would have caught every silent failure found in this build. # This is the step that would have caught every silent failure found in this build.
$smokeFailed = $false $smokeFailed = $false
@@ -276,7 +305,7 @@ if (-not $SkipSmokeTest -and (Test-Path $Exe)) {
} }
} }
# ------------------------------------------------------------------- 7. summary # ------------------------------------------------------------------- 8. summary
if (Test-Path $OutDir) { if (Test-Path $OutDir) {
$bytes = (Get-ChildItem $OutDir -Recurse -File -ErrorAction SilentlyContinue | $bytes = (Get-ChildItem $OutDir -Recurse -File -ErrorAction SilentlyContinue |
Measure-Object -Property Length -Sum).Sum Measure-Object -Property Length -Sum).Sum
+49 -4
View File
@@ -71,6 +71,46 @@ def _branding_datas() -> list[tuple[str, str]]:
return datas return datas
#: Source-control and tool-cache names that must never ship. ``collect_data_files`` copies
#: *every* non-Python file a package contains, and the SDK keeps a tracked
#: ``src/orisys/.gitignore`` — which is how ``_internal\orisys\.gitignore`` reached the
#: bundle. The list is deliberately narrow: a real asset (``theme/*.json``, cv2's own
#: ``config.py``) must survive, so only VCS/cache names are dropped.
_RESIDUE_NAMES = frozenset(
{
".git",
".gitignore",
".gitattributes",
".gitmodules",
".gitkeep",
"__pycache__",
".pytest_cache",
".mypy_cache",
".ruff_cache",
".scratch",
".tox",
".idea",
".vscode",
}
)
_RESIDUE_SUFFIXES = (".pyc", ".pyo", ".orig", ".rej", "~")
def _without_residue(datas: list[tuple[str, str]]) -> list[tuple[str, str]]:
"""Drop VCS/cache residue from collected package data, naming what was dropped."""
kept: list[tuple[str, str]] = []
dropped: list[str] = []
for source, dest in datas:
parts = Path(source).parts
if any(part in _RESIDUE_NAMES for part in parts) or source.endswith(_RESIDUE_SUFFIXES):
dropped.append(source)
continue
kept.append((source, dest))
for item in dropped:
print(f"[spec] dropped residue from package data: {item}")
return kept
block_cipher = None block_cipher = None
# Config is copied beside the EXE by build.ps1 (writable / editable). # Config is copied beside the EXE by build.ps1 (writable / editable).
@@ -92,15 +132,20 @@ hiddenimports = [
datas += collect_data_files("orisys_app_shared", includes=["theme/*.json"]) datas += collect_data_files("orisys_app_shared", includes=["theme/*.json"])
try: try:
datas += collect_data_files("orisys") datas += collect_data_files("orisys")
except Exception: except Exception as exc:
pass # Tolerated so a spec run by hand still works, but never silently: a package whose data
# cannot be collected loses assets, and that is invisible in the built EXE.
print(f"[spec] WARNING: could not collect orisys package data: {exc}")
try: try:
cv2_datas, cv2_binaries, cv2_hidden = collect_all("cv2") cv2_datas, cv2_binaries, cv2_hidden = collect_all("cv2")
datas += cv2_datas datas += cv2_datas
binaries += cv2_binaries binaries += cv2_binaries
hiddenimports += list(cv2_hidden) hiddenimports += list(cv2_hidden)
except Exception: except Exception as exc:
pass print(f"[spec] WARNING: could not collect cv2: {exc}")
# Applied last, so it covers every contributor above (including a third-party wheel).
datas = _without_residue(datas)
icon_path = BRANDING_DIR / "logo.ico" icon_path = BRANDING_DIR / "logo.ico"
icon = str(icon_path) if icon_path.is_file() else None icon = str(icon_path) if icon_path.is_file() else None