finished docker manylinux compliation for pypi
This commit is contained in:
ykcheungab
@@ -0,0 +1,167 @@
|
||||
# Publishing Orisys to PyPI
|
||||
|
||||
This document covers the **Linux manylinux binary wheel** path (Nuitka-compiled
|
||||
`orisys` for CPython 3.10 / 3.11 / 3.12). Windows wheels are still built on
|
||||
Windows hosts; they cannot be produced inside the manylinux container.
|
||||
|
||||
## Why Docker?
|
||||
|
||||
PyPI rejects wheels tagged `manylinux_x86_64` (missing glibc ABI version). A
|
||||
compliant tag looks like:
|
||||
|
||||
```text
|
||||
orisys-0.4.2-cp310-cp310-manylinux_2_28_x86_64.whl
|
||||
orisys-0.4.2-cp311-cp311-manylinux_2_28_x86_64.whl
|
||||
orisys-0.4.2-cp312-cp312-manylinux_2_28_x86_64.whl
|
||||
```
|
||||
|
||||
Those tags are produced by building inside the official manylinux image and
|
||||
running `auditwheel repair`. On Windows, use **Docker Desktop in Linux container
|
||||
mode**.
|
||||
|
||||
> Note: we use `manylinux_2_28_x86_64` (not 2014). Recent NumPy/SciPy no longer
|
||||
> provide manylinux2014 wheels and fail to build from source on that older GCC.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Docker Desktop installed and running (**Linux containers**, not Windows containers).
|
||||
2. Repository checked out locally (this tree is bind-mounted into the container).
|
||||
3. Version bumped in `src/orisys/__init__.py` (`__version__`) before every upload.
|
||||
PyPI never allows re-uploading the same filename/version.
|
||||
4. PyPI / TestPyPI project-scoped API tokens (never commit tokens or `.pypirc` with secrets).
|
||||
|
||||
Quick Docker check:
|
||||
|
||||
```powershell
|
||||
docker version --format "{{.Server.Os}}"
|
||||
# expect: linux
|
||||
docker run --rm hello-world
|
||||
```
|
||||
|
||||
## Build manylinux wheels (3.10 / 3.11 / 3.12)
|
||||
|
||||
From the repository root. Prefer PowerShell, or from **cmd.exe**:
|
||||
|
||||
```bat
|
||||
powershell -ExecutionPolicy Bypass -File .\build\build_manylinux_docker.ps1
|
||||
```
|
||||
|
||||
PowerShell:
|
||||
|
||||
```powershell
|
||||
.\build\build_manylinux_docker.ps1
|
||||
```
|
||||
|
||||
Default builds **all three** ABIs. To build a subset:
|
||||
|
||||
```powershell
|
||||
.\build\build_manylinux_docker.ps1 -PythonVersions 3.10,3.12
|
||||
.\build\build_manylinux_docker.ps1 -PythonVersions 3.11
|
||||
```
|
||||
|
||||
What this does:
|
||||
|
||||
1. Builds `orisys-manylinux` from [`build/Dockerfile.manylinux`](../build/Dockerfile.manylinux)
|
||||
(`quay.io/pypa/manylinux_2_28_x86_64` with Nuitka/auditwheel in cp310/cp311/cp312).
|
||||
2. Runs [`build/manylinux_entrypoint.sh`](../build/manylinux_entrypoint.sh), which loops:
|
||||
|
||||
```bash
|
||||
/opt/python/cp310-cp310/bin/python build/build_nuitka.py --target pypi-linux
|
||||
/opt/python/cp311-cp311/bin/python build/build_nuitka.py --target pypi-linux
|
||||
/opt/python/cp312-cp312/bin/python build/build_nuitka.py --target pypi-linux
|
||||
```
|
||||
|
||||
3. For each ABI: Nuitka compile → wheel → `auditwheel repair` → isolated smoke test.
|
||||
4. Artifacts land in:
|
||||
|
||||
- `dist/repaired/orisys-*-manylinux_2_28_x86_64.whl` (upload candidates)
|
||||
- `SDK/<version>/` (same compliant wheels)
|
||||
|
||||
Expect roughly **3×** the wall time of a single-ABI build.
|
||||
|
||||
Rebuild without refreshing the image:
|
||||
|
||||
```powershell
|
||||
.\build\build_manylinux_docker.ps1 -SkipImageBuild
|
||||
```
|
||||
|
||||
Equivalent manual `docker` invocation:
|
||||
|
||||
```powershell
|
||||
docker build -f build/Dockerfile.manylinux -t orisys-manylinux .
|
||||
docker run --rm `
|
||||
-e ORISYS_BUILD_TARGET=pypi-linux `
|
||||
-e ORISYS_PYTHON_TAGS=cp310-cp310,cp311-cp311,cp312-cp312 `
|
||||
-v "${PWD}:/src" `
|
||||
orisys-manylinux
|
||||
```
|
||||
|
||||
### Acceptance checks before upload
|
||||
|
||||
- Each filename contains `cp310` / `cp311` / `cp312` and `manylinux_2_28_x86_64`
|
||||
(or another accepted manylinux tag such as `manylinux_2_17_x86_64`).
|
||||
- Isolated smoke test printed `public API imports OK` for every ABI.
|
||||
- Do **not** upload any wheel still tagged `linux_x86_64` or bare `manylinux_x86_64`.
|
||||
|
||||
Optional re-verify on a Linux machine:
|
||||
|
||||
```bash
|
||||
python build/verify.py dist/repaired/orisys-*-cp310-cp310-manylinux_*.whl
|
||||
```
|
||||
|
||||
## Promote: TestPyPI then PyPI
|
||||
|
||||
Install upload tooling in a clean environment:
|
||||
|
||||
```powershell
|
||||
python -m pip install --upgrade twine
|
||||
```
|
||||
|
||||
### 1. Upload to TestPyPI
|
||||
|
||||
```powershell
|
||||
$env:TWINE_USERNAME = "__token__"
|
||||
$env:TWINE_PASSWORD = "<TestPyPI API token>"
|
||||
|
||||
python -m twine check dist/repaired/*.whl
|
||||
python -m twine upload --repository testpypi dist/repaired/*.whl
|
||||
```
|
||||
|
||||
Install from TestPyPI in a fresh venv (pull runtime deps from real PyPI), testing
|
||||
each Python you care about:
|
||||
|
||||
```powershell
|
||||
py -3.10 -m venv .venv-testpypi310
|
||||
.\.venv-testpypi310\Scripts\Activate.ps1
|
||||
python -m pip install --index-url https://test.pypi.org/simple/ `
|
||||
--extra-index-url https://pypi.org/simple `
|
||||
"orisys==<version>"
|
||||
python -c "import orisys; print(orisys.__version__)"
|
||||
```
|
||||
|
||||
### 2. Upload the same artifacts to PyPI
|
||||
|
||||
Only after TestPyPI install/import succeeds:
|
||||
|
||||
```powershell
|
||||
$env:TWINE_USERNAME = "__token__"
|
||||
$env:TWINE_PASSWORD = "<PyPI API token>"
|
||||
|
||||
python -m twine upload dist/repaired/*.whl
|
||||
```
|
||||
|
||||
Prefer CI **Trusted Publishing** (OIDC) for production uploads so long-lived tokens
|
||||
are not stored on developer machines.
|
||||
|
||||
## Related local targets
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `python build/build_nuitka.py` | Internal SDK wheel on the current host (Windows `.pyd` or Linux `.so`) |
|
||||
| `python build/build_nuitka.py --target pypi-linux` | Linux-only; requires manylinux + CPython 3.10–3.12 |
|
||||
| `.\build\build_nuitka_win.bat` | Windows wheels for py310/py311/py312 conda envs |
|
||||
| `.\build\build_manylinux_docker.ps1` | Linux manylinux wheels for 3.10/3.11/3.12 via Docker |
|
||||
| `python -m build` | Pure-Python / sdist path from `pyproject.toml` (no Nuitka) |
|
||||
|
||||
Windows binary wheels and Linux manylinux wheels must be built on their respective
|
||||
platforms. Never rename a platform tag to bypass PyPI validation.
|
||||
Reference in New Issue
Block a user