统一多相机连接与 UART 数据协议
启动时完整连接所有配置相机，移除旧命令与单路数据格式。通过 22 项回归测试、真实 SDK 的 1–4 路录像验证、Ruff 和文档检查。
This commit is contained in:
ykcheungab
+102
-98
@@ -1,141 +1,145 @@
|
||||
# UART finger service
|
||||
|
||||
`serial/example_uart.py` processes 1–4 finger cameras or recorded videos and
|
||||
publishes the SDK's normal and shear values over a separate data UART. Python
|
||||
3.10+, the board's installed `orisys` SDK and `pyserial` are required. Run inside
|
||||
the board environment prepared by `setup/sdk_setup.sh`; install `pyserial` there
|
||||
with `python -m pip install pyserial` if needed.
|
||||
Start the service with an explicit list of 1-4 camera sources. It opens and
|
||||
warms up **every source first**, then opens the data UART and starts publishing.
|
||||
If any source fails to prepare, all opened sources are closed and the process
|
||||
exits with status 1. No partial camera set is published.
|
||||
|
||||
## Launch from the login console
|
||||
## Connect the cameras and start once
|
||||
|
||||
The TTL login console and data UART must use **different device ports**. Start
|
||||
the script in the login shell; connect the data receiver to the port selected
|
||||
by `--port`. A shell/getty or another serial program must not own that data port.
|
||||
The default is `/dev/ttyHS1`, 115200 baud, 8N1, with no flow control. This service
|
||||
does not configure pin multiplexing or enable the board's UART hardware.
|
||||
Plug the USB cameras into the Q6A. In its TTL login shell, identify their actual
|
||||
capture device paths before launching; one USB camera can expose more than one
|
||||
`/dev/video*` node, so do not assume every node represents a different camera.
|
||||
For example, `v4l2-ctl --list-devices` shows device groups. Select one usable
|
||||
capture node per camera. Use the board environment prepared by `setup/sdk_setup.sh`;
|
||||
it needs Python 3.10+, the `orisys` SDK and `pyserial`.
|
||||
|
||||
From the board's `dragon-q6a` checkout:
|
||||
Run from the board's `dragon-q6a` directory. These paths are examples, not detected
|
||||
hardware identities:
|
||||
|
||||
```bash
|
||||
# Host commands control connection and streaming (default: one camera).
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --config finger
|
||||
# One camera.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources /dev/video0
|
||||
|
||||
# Four configured sources; use 1, 2, 3 or 4 entries as required.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources 0 2 4 6
|
||||
# Two cameras.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources /dev/video0 /dev/video2
|
||||
|
||||
# A passive receiver needs only to read: connection and streaming start automatically.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources 0 2 --read-only
|
||||
# Three cameras.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources /dev/video0 /dev/video2 /dev/video4
|
||||
|
||||
# Start streaming immediately, while still accepting UART commands.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources 0 2 --autostart
|
||||
# Four cameras.
|
||||
python serial/example_uart.py --port /dev/ttyHS1 --sources /dev/video0 /dev/video2 /dev/video4 /dev/video6
|
||||
```
|
||||
|
||||
Use actual camera indices/device paths discovered on the board; `0 2 4 6` is
|
||||
only an example. Slot IDs are fixed by `--sources` order, not camera index.
|
||||
Repeat `--config` once per source for different sensor configurations, or supply
|
||||
one shared configuration. Existing `--camera-id` and `--video` remain available
|
||||
for a single source. `--sources` replaces those options. `--show` requires a local
|
||||
GUI and is off by default. Logs go to the login console, replies to the data UART.
|
||||
The first source is sensor 1, the second sensor 2, and so on. The number of listed
|
||||
sources determines the sensor count. Camera indices such as `--sources 0 2` are
|
||||
also accepted. Sources must be distinct. A shared `--config finger` is the
|
||||
default; repeat `--config` once per source to use different configurations.
|
||||
The login console prints each sensor's source/config mapping during preparation.
|
||||
There is no UART connection command: physical connection and source selection
|
||||
happen before startup, and software connection happens automatically.
|
||||
|
||||
In `--read-only` mode, all incoming UART commands are ignored. End the process
|
||||
with Ctrl-C in the login shell. Startup still emits `OK`/`ERR` records; the receiver
|
||||
should select `force,` lines. Without `--autostart` or `--read-only`, a passive
|
||||
reader receives only `OK ready` until a host sends `connect` and `run`.
|
||||
The TTL login console and data UART must use **different ports**. The data
|
||||
UART defaults to `/dev/ttyHS1`, 115200 baud, 8N1 without flow control. It must
|
||||
not be occupied by getty or another serial program. This script does not enable
|
||||
UART pins or configure the board's hardware. Logs stay in the login console;
|
||||
protocol records go to the data UART. `--show` is optional desktop debugging.
|
||||
|
||||
## Command protocol
|
||||
For a passive receiver, append `--read-only` to the same startup command. Startup
|
||||
and output are identical, but incoming UART commands are discarded. Exit using
|
||||
Ctrl-C in the login shell. The receiver should filter `force,` records because
|
||||
startup and failures also produce status records.
|
||||
|
||||
Commands are UTF-8, case-insensitive, terminated by LF, CR or CRLF. Incomplete
|
||||
lines remain buffered across serial timeouts. Lines exceeding 1024 bytes are
|
||||
discarded through their terminator and produce `ERR command ...` in control mode.
|
||||
Quote file paths containing spaces. Linux device paths are recommended on Q6A.
|
||||
## One protocol for every sensor count
|
||||
|
||||
| Command | Behavior |
|
||||
|---|---|
|
||||
| `search` | List cameras, then `OK search N`; use before connecting. Rejected while streaming. |
|
||||
| `connect` | Connect slot 1 using its configured source; in single-source mode, prefer the latest search result for a camera. |
|
||||
| `connect SOURCE` | Legacy form: connect slot 1 to this camera index, device path or file. |
|
||||
| `connect ID SOURCE` | Connect/replace one slot; stop streaming that slot. Other slots keep their connections. |
|
||||
| `connect all` | Connect all configured slots; report each success/failure independently. |
|
||||
| `run [ID\|all]` | Stream the selected slot, or all connected slots when omitted. Requires a connection. |
|
||||
| `read [ID\|all]` | Publish one newly processed measurement per selected slot; defaults to all configured slots. Does not start streaming. |
|
||||
| `stop [ID\|all]` | Stop selected streaming, or all when omitted; keep cameras connected. |
|
||||
| `quit` / `exit` | Exit and release all camera and UART resources. |
|
||||
|
||||
`connect 2` means **camera index 2 on slot 1**, preserving the old protocol.
|
||||
To connect slot 2, send `connect 2 /dev/video2` or use `connect all`.
|
||||
There are no motor-control commands; these commands control acquisition only.
|
||||
|
||||
Single-source measurements retain the original format:
|
||||
After all sources are ready, the data UART reports `OK ready N`, then immediately
|
||||
streams each complete processing round in source order. Every measurement has
|
||||
five CSV fields, including a sensor ID even when only one camera is configured:
|
||||
|
||||
```text
|
||||
force,1.250000,-2.500000,3.750000
|
||||
```
|
||||
|
||||
When configured with multiple sources, measurements include the slot ID even if
|
||||
only one source connected successfully:
|
||||
|
||||
```text
|
||||
OK connect 2 /dev/video2 (camera)
|
||||
OK run 1 2
|
||||
OK ready 2
|
||||
force,1,1.250000,-2.500000,3.750000
|
||||
force,2,0.500000,0.100000,-0.200000
|
||||
ERR run 2 processing failed
|
||||
OK eof 1
|
||||
```
|
||||
|
||||
Force fields are `FNORMAL`, `FSHEARX`, `FSHEARY` in SDK axes. They may be raw
|
||||
or calibrated; **do not assume Newtons**. The Python API exposes SDK units per
|
||||
reading, using `unknown` if an older SDK does not expose them. CSV preserves
|
||||
the original fields and carries no units, timestamps or cross-sensor synchronization.
|
||||
The fields are `force,sensor_id,FNORMAL,FSHEARX,FSHEARY`. Axes follow the SDK;
|
||||
units depend on calibration and must not be assumed to be Newtons. CSV carries
|
||||
no timestamp or units. Frames are processed sequentially, not synchronized
|
||||
across cameras. All inputs for a round must succeed before that round is
|
||||
published. A transport failure can still interrupt transmission between lines.
|
||||
|
||||
Each processing failure or EOF stops only the affected slot. Processing exceptions
|
||||
produce `ERR`, never fabricated zero force. `--no-loop-video` stops files at EOF;
|
||||
the default tries one rewind, resets the tracker and retries one frame. Failed
|
||||
preparation releases the newly opened runtime; replacing a connection releases
|
||||
the old one before opening the replacement. A failed replacement requires retrying
|
||||
`connect`. Startup continues with any successfully connected slots and fails if
|
||||
none connect. Reconnect and run explicitly after a camera failure.
|
||||
Control mode accepts only four commands, each acting on the entire camera set:
|
||||
|
||||
| Command | Result |
|
||||
|---|---|
|
||||
| `stop` | Pause continuous output and reply `OK stop`; keep cameras open. |
|
||||
| `start` | Resume continuous output and reply `OK start`. |
|
||||
| `read` | Process and publish one complete round; preserve streaming state on success. |
|
||||
| `quit` | Reply `OK quit`, close all cameras and UART, then exit. |
|
||||
|
||||
Commands take no arguments. They are case-insensitive UTF-8 lines terminated by
|
||||
LF, CR or CRLF. Partial lines survive read timeouts; commands exceeding 1024
|
||||
bytes are discarded through their terminator. Unknown commands or arguments
|
||||
produce `ERR`. The former protocol and source-selection options have been
|
||||
removed; receivers must use this single CSV format and explicit `--sources`.
|
||||
|
||||
A capture or processing failure produces `ERR read sensor ID: ...` and pauses
|
||||
output for the whole set. Measurements are never replaced with zeros. Sources
|
||||
remain open; `start` retries processing, or restart the service to reopen them.
|
||||
For recorded files, `--no-loop-video` reports `OK eof sensor ID: video ended`
|
||||
and pauses the whole set at EOF. Otherwise, one rewind and tracker reset is
|
||||
attempted per read. Ctrl-C always ends the process from the login console.
|
||||
|
||||
## Callable acquisition API
|
||||
|
||||
`serial/finger_api.py` owns acquisition and runtime cleanup; importing it opens
|
||||
no UART or camera. Place that directory on the caller's module search path:
|
||||
`serial/finger_api.py` has no UART dependency and opens no resources on import.
|
||||
Put its directory on the caller's module search path. Use the same connection-first
|
||||
lifecycle as the service:
|
||||
|
||||
```python
|
||||
from finger_api import FingerSensorBank
|
||||
|
||||
bank = FingerSensorBank([0, 2], configs=["finger"])
|
||||
try:
|
||||
bank.connect(1)
|
||||
bank.connect(2)
|
||||
reading = bank.read(1)
|
||||
if reading is not None:
|
||||
print(reading.normal, reading.shear_x, reading.shear_y, reading.units)
|
||||
bank.connect_all()
|
||||
for reading in bank.read_all():
|
||||
print(reading.sensor_id, reading.normal, reading.shear_x, reading.shear_y, reading.units)
|
||||
finally:
|
||||
bank.close()
|
||||
```
|
||||
|
||||
The caller owns one thread and must close the bank. `read()` returns `None` on
|
||||
capture failure/EOF and raises on processing failure. It returns an image and
|
||||
three scalar forces from the same processed frame. One file or camera is read
|
||||
at a time; the API introduces no workers beyond those owned by the SDK.
|
||||
One thread owns the bank. `connect_all()` rolls back every opened runtime on
|
||||
failure. `read_all()` returns a tuple only when every source succeeds; it raises
|
||||
`EOFError` for file EOF and `RuntimeError` for capture/processing failure. Each
|
||||
reading contains the unrolled image, three scalar forces and SDK units from its
|
||||
own runtime. Older SDKs without a units property report `unknown`. Reconnection
|
||||
requires closing the bank before calling `connect_all()` again.
|
||||
|
||||
## Verification and limits
|
||||
## Backend evidence and remaining limits
|
||||
|
||||
Hardware-free regression tests cover command fragmentation, CR/LF, legacy output,
|
||||
1–4 slot identities, selective control, failure isolation, read-only startup and
|
||||
cleanup. They use in-memory serial and runtime fakes, not physical devices:
|
||||
The SDK's `FingerRuntime.open()` constructs a separate capture object,
|
||||
`FrameUnwarper`, `TactileProcessor` and `Sensor` for each source. Live sources
|
||||
have separate SDK capture workers. The service holds all runtimes open together
|
||||
and reads them sequentially. SDK implementation is unchanged.
|
||||
|
||||
Verification on 2026-10-14 used local SDK revision
|
||||
`3953747f1726b197a3f327a38c65d9878e333e12`: real SDK instances processed 1, 2, 3
|
||||
and 4 distinct recorded inputs, three complete rounds for each count. Capture
|
||||
handles and sensor objects were distinct; optical-flow arrays did not share
|
||||
memory. All capture resources were released after each case. Run the same check
|
||||
with a recording containing at least 12 frames; put generated clips and caches
|
||||
in an isolated task scratch directory:
|
||||
|
||||
```bash
|
||||
python tests/check_backend.py --recording /path/to/recording.avi --scratch .scratch/uart-connect-first/backend-run
|
||||
python -m unittest discover -s tests -v
|
||||
```
|
||||
|
||||
Q6A UART wiring, installed ARM SDK compatibility, camera bandwidth, sustained
|
||||
frame rates and latency remain unverified. The shipped ARM wheel is compiled
|
||||
`orisys 0.5.0`; local tests do not execute it. SDK capture and processing remain
|
||||
synchronous from this service's perspective. Commands are checked before each
|
||||
round; a slow grab, processing call, connection or camera scan can delay them.
|
||||
Serial writes have a finite timeout (default 1 second), and transport failure
|
||||
terminates the service with cleanup. At 115200 baud/8N1, the data budget is
|
||||
11,520 bytes/s, shared by all measurements and replies. Measure actual frame
|
||||
rates and stop latency on the board before choosing a higher baud rate.
|
||||
Unit tests separately verify startup ordering, full rollback, a uniform protocol,
|
||||
removed commands/options, command framing and failure behavior. These tests do
|
||||
not prove Q6A USB bandwidth or the shipped compiled ARM `orisys 0.5.0` wheel.
|
||||
Run the backend check in the board's installed environment to verify that SDK.
|
||||
Four live cameras, serial wiring, sustained frame rates and command latency
|
||||
still require hardware validation. A slow capture/processing call delays command
|
||||
handling. Serial writes time out after a configurable interval (default 1 second).
|
||||
At 115200 baud/8N1, all data and replies share 11,520 bytes/s.
|
||||
|
||||
Reference in New Issue
Block a user