dragon-q6a/docs/uart.md

UART finger service

Connect 1-4 sources individually through UART, or load an explicit setup file with their configurations and calibration paths. There is no camera discovery or hot-plug monitoring; every source is selected by the user.

Start the service and add cameras

In the Q6A TTL login shell, activate the board environment prepared by setup/sdk_setup.sh. It needs Python 3.10+, the orisys SDK and pyserial. From the board's dragon-q6a directory:

python serial/example_uart.py --port /dev/ttyHS1 --config finger

The service opens only the data UART and replies OK ready 0. No camera is opened and no measurement is sent until the host adds a source and starts output. Send these lines through the data UART, waiting for each reply:

add /dev/video0
OK added 1
add /dev/video2
OK added 2
start
OK start
force,1,1.250000,-2.500000,3.750000
force,2,0.500000,0.100000,-0.200000

Use actual capture nodes; the paths above are examples. One USB camera can expose multiple /dev/video* nodes, so do not assume every node represents a different camera. v4l2-ctl --list-devices shows device groups. Select one usable capture node per camera. Camera indices such as add 0 are also accepted. Sources must be distinct; 0 and /dev/video0 are treated as the same camera.

The first successful addition is sensor 1, the next sensor 2, up to sensor 4. No sensor count is declared in advance. A failed addition replies ERR add ..., closes its newly opened runtime if one exists, and leaves existing cameras and IDs unchanged. It does not consume an ID. The service remains available for retry.

To select this sensor's force calibration, append --calibration:

add /dev/video0 --calibration "/home/radxa/calibrations/finger/DD02-14.json"

An optional config still works: add SOURCE CONFIG --calibration PATH. Explicit calibration paths are passed to SDK FingerRuntime.open(cal_path=...). A missing or invalid file fails the addition, without falling back to raw values. Without the option, normal SDK calibration selection applies. This does not replace calibration on a camera already added; restart to change that mapping.

The Windows example D:\orisys_ws\data\calibrations\finger\DD02-14.json refers to a file on that computer. Copy it to the Q6A before using a board-local path. Quote Windows paths in UART commands, or use forward slashes. The user must match each camera to its physical sensor's calibration; no serial-number discovery is performed.

The TTL login console and data UART must use different ports. Data defaults to /dev/ttyHS1, 115200 baud, 8N1 without flow control. It must not be occupied by getty or another serial program. The script does not configure UART pins. Logs stay in the login console; protocol records go to the data UART. --show is optional local desktop debugging.

Batch setup and one-command startup

Copy sensors.example.json, then replace its camera paths and calibration filenames with the real mappings. SENSOR-2 through SENSOR-4 are placeholders. The JSON format is:

{
  "sensors": [
    {"source": "/dev/video0", "config": "finger", "calibration": "calibrations/finger/DD02-14.json"},
    {"source": "/dev/video2", "calibration": "calibrations/finger/SENSOR-2.json"}
  ]
}

List 1-4 entries. source is required; config defaults to startup --config. An omitted/null calibration uses normal SDK selection. Relative file paths resolve from the setup file's directory, independently of the working directory. Named configs such as finger remain names. Unknown fields are rejected.

Once saved as sensors.json, one terminal command connects the whole set and starts continuous UART output:

python serial/example_uart.py --port /dev/ttyHS1 --setup sensors.json --start

Omit --start to connect and wait for UART commands. With neither option, empty manual startup is unchanged. --start requires --setup. Preparation failure rolls back the entire startup set and exits before opening the UART.

An already-running service can also load the file:

load "/home/radxa/sensors.json"
OK loaded 1 2
start

load only connects; it does not start output. It can extend manually added sensors, up to four total. Batch failure removes only that batch's new sensors and leaves previously added cameras unchanged. add and load require stopped output. Wait for each reply before proceeding.

Commands and output

Command	Behavior
add SOURCE [CONFIG] [--calibration PATH]	Prepare one source while output is stopped; reply OK added ID. Config/calibration apply only to this sensor.
load SETUP.json	Add the sources and calibration mappings in a setup file while stopped; reply OK loaded ID ....
start	Begin continuous output for all added sensors; requires at least one sensor. Reply OK start.
stop	Pause continuous output for the whole set; keep all cameras open. Reply OK stop.
read	Publish one complete processing round; requires a sensor and preserves streaming state on success.
quit	Reply OK quit, close all cameras and UART, then exit.

To extend the camera set after streaming has started, pause before adding:

stop
OK stop
add /dev/video4
OK added 3
start
OK start

add while streaming replies ERR add stop streaming first and opens no source. A fifth source or duplicate source is rejected. Removing/replacing cameras requires quitting and starting a fresh service. start, stop, read and quit take no arguments and always act on the full set. The older protocol commands remain removed.

Commands are case-insensitive UTF-8 lines terminated by LF, CR or CRLF. Partial lines survive read timeouts. Lines exceeding 1024 bytes are discarded through their terminator. Quote source/config paths containing spaces:

add "/path/test clip.mp4" "/path/my config.json"

Every measurement has five CSV fields, even with only one sensor: force,sensor_id,FNORMAL,FSHEARX,FSHEARY. Axes follow the SDK; units depend on calibration and must not be assumed to be Newtons. CSV carries no units or timestamp. The receiver can remain passive after sending setup commands and start; it needs no further requests to receive continuous data. Filter force, lines to distinguish measurements from replies and errors.

Frames are processed sequentially, not synchronized across cameras. All sources must succeed before a processing round is published. A capture/processing failure produces ERR read sensor ID: ... and stops continuous output for the full set, without fabricating zero values or transmitting a partial processing round. A transport failure can still interrupt transmission between lines. start retries processing; restart the service to reopen an unavailable camera.

For recorded files, --no-loop-video replies OK eof sensor ID: video ended and pauses the whole set at EOF. Otherwise one rewind and tracker reset is attempted per read. Ctrl-C in the login shell ends the service and releases resources.

Callable acquisition API

serial/finger_api.py has no UART dependency and opens no resources on import. Place its directory on the caller's module search path. Adding sources is explicit:

from finger_api import FingerSensorBank

bank = FingerSensorBank(config="finger")
try:
    first_id = bank.add(0, cal_path="/home/radxa/calibrations/finger/DD02-14.json")
    second_id = bank.add(2, config="finger")
    for reading in bank.read_all():
        print(reading.sensor_id, reading.normal, reading.shear_x, reading.shear_y, reading.units)
finally:
    bank.close()

One thread owns the bank. add() returns a 1-based ID only after successful preparation; it raises on failure and preserves previously added runtimes. read_all() requires at least one source and returns a tuple only when all sources succeed. File EOF raises EOFError; capture/processing failure raises RuntimeError. Each reading contains its own unrolled image, three scalar forces and SDK units; older SDKs without a units property report unknown. close() attempts every runtime and clears the camera set. The API caller must pause its own processing loop before adding; the UART service enforces this for commands. bank.add_setup("sensors.json") uses the same transactional batch loader as the CLI and UART. bank.calibration_paths records explicit calibration paths by ID.

Backend evidence and remaining limits

The SDK's FingerRuntime.open() creates separate capture, unwarping and sensor state for each source. SDK code is unchanged. Verification on 2026-10-14 used local SDK revision 3953747f1726b197a3f327a38c65d9878e333e12: 1, 2, 3 and 4 recorded sources were added incrementally, with complete readings after each addition and three final rounds for each count. Existing runtime identities were preserved; capture handles and sensor objects were distinct, and optical-flow arrays did not share memory. Resources were released after each case. The explicit DD02-14.json calibration was also tested on 1-4 recorded sources: each instance loaded its assigned file and reported three calibrated N axes. Four-source JSON loading passed, and invalid explicit calibration was rejected without altering an existing sensor. This validates software binding, not physical accuracy.

To rerun with a recording containing at least 12 frames, put generated clips and caches in an isolated task scratch directory:

python tests/check_backend.py --recording /path/to/recording.avi --calibration /path/to/DD02-14.json --scratch .scratch/uart-calibration-setup/backend-run
python -m unittest discover -s tests -v

Unit tests separately verify empty startup, explicit additions, failure isolation, IDs/limits, stop-before-add, command framing, complete-round output and cleanup. The shipped compiled ARM orisys 0.5.0 wheel and Q6A physical camera/UART setup remain unverified. Run the backend check inside the board's installed environment to verify that SDK. USB bandwidth, sustained frame rates and command latency require hardware validation. Camera opening/warmup and synchronous reads can delay command handling. Serial writes have a configurable timeout, default 1 second. At 115200 baud/8N1, all data and replies share 11,520 bytes/s.