orisys-tech/orisys-app
3
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

app 侧滤波接口改为驱动 SDK 并读回生效值 - What: shared/force/filters.py 从"自己滤一遍"改成 app 的滤波控制面，只做三件事：(1) FilterSettings 改用 SDK 的键与"按值开关"语义（flow_ema_alpha / median_window / lowpass_hz / lowpass_order / sample_rate_hz / show_unfiltered / preset；α≥1、窗口≤1、截止≤0 各自表示关），normalised() 把不可用值一律落成"关"而不是落成会滤波的默认值，并新增 problems(sample_rate_hz) 指出"截止 ≥ Nyquist、设备根本不会滤"这类请求；(2) apply_to(sensor) 就是一次 Sensor.configure(...)，用 CONFIGURE_PARAMETERS 一处维护"状态键 → configure 形参"的映射（低通那一对在配置/状态里叫 force_lowpass_*、在调用里叫 lowpass_*，此前正是这里写错导致支持性判断把所有键都判成不支持），并返回该 SDK 支持不了的键而不是默默丢掉——老 SDK（0.5.2 之前）没有 flow_ema_alpha/中值，其光流 EMA 也关不掉，必须让人看见；(3) from_filters_state(state) 读回设备**实际**在跑的值（界面与记录都以此为准），metadata(sensor_state) 产出 session.json/CSV 头的 filters 块，measurement 组与 readout 组严格分开、带 describe 与 source（有设备时为 sensor.filters_state，只有请求时为 app.FilterSettings，两者不可混认）。新增 describe_state()/settings_from_mapping()；ForceFilterChain 降级为"参考实现"，只为 tests/test_force_filters.py 的等价性断言存在（app 不再调用它），并删掉随之无用的 _odd()。force/__init__.py 导出 describe_state。tests/test_force_filters.py 重写为 19 项：按值开关、不可用值=关、Nyquist 预警、一次 configure 调用且键名正确、老 SDK 缺键被上报而不是忽略、显示开关绝不发给设备、读回生效值与 sample_rate 往返、metadata 两组分离与来源标注、空的 preferences 映射等于"什么都不滤"、参考链仍与 SDK 滤波器逐位一致。 - Why: 用户要求"统一采集滤波和显示滤波、显示的和记录的必须一致、保留滤波前数据两个 buffer"，并决定整条链留在 SDK、app 通过 Sensor.configure 驱动、filters.py 保留为 wrapper 与 CSV 元数据出口。上一版的 FilterSettings 用 median_enabled/lowpass_enabled 布尔 + app 自己实现滤波，既与 SDK 的"按值开关"两套词汇，也会让同一份数据在界面与 CSV 之间出现第二条滤波路径。 - Verification: pytest shared/tests/test_force_filters.py → 19 passed（含与 orisys.signal_processing.LowpassFilter 的逐位等价断言）；shared 全量 153 passed；SDK 已升级到 0.5.2（editable），studio 85 项测试与 ui_check 554 项检查在其上全绿。 - Not in this change: sdk_worker/finger_worker 仍在硬编码 lowpass_cutoff_hz=10.0（下一步改为 apply_to）、曲线图仍在显示端做 filtfilt、齿轮对话框与 plot_* preferences、CSV 的 17 列与未滤波 buffer、session.json 写入 filters 块都还没接。

Browse Source
This commit is contained in:
ykcheungab
2026-09-14 10:56:04 +08:00
parent 65dafdc84d
commit 0d8b79b759
3 changed files with 583 additions and 141 deletions
Download Patch File Download Diff File Expand all files Collapse all files
shared/src/orisys_app_shared/force/__init__.py
+3 -1
View File
@@ -14,8 +14,9 @@ from orisys_app_shared.force.calibration_recorder import (
CalibrationRecorder,
)
from orisys_app_shared.force.filters import (
ForceFilterChain,
FilterSettings,
ForceFilterChain,
describe_state,
settings_from_mapping,
)
from orisys_app_shared.force.history import MetricHistory, PlotState, lowpass_filter
@@ -47,6 +48,7 @@ __all__ = [
"SessionSlice",
"SessionStore",
"build_force_calibration_v2",
"describe_state",
"export_force_calibration_v2",
"lowpass_filter",
"meta_from_sensor_json",
shared/src/orisys_app_shared/force/filters.py
+326 -74
View File
@@ -1,119 +1,370 @@
"""One causal filter chain for the force readings, and the values it replaced.
"""The app's filter control surface: what was asked for, and what is running.
The chain runs **once, in the acquisition loop**, so the numbers on the screen
and the numbers in ``forces.csv`` are the same filtered values — there is no
second, display-only smoothing that would make the chart disagree with the
record. The unfiltered readings are kept beside the filtered ones, so a filter
can be judged instead of trusted.
The filter *chain* lives in the SDK — ``compute_flow`` applies the flow EMA
before the HHD, and ``_apply_force_filter`` applies the median and the low-pass
*before* the device calibration. Filtering anywhere else would produce numbers no
recording could reproduce, and would leave the unfiltered newtons unrecoverable
(the calibration has already been applied by then). So this module does three
things, and nothing else:
Both stages are causal and per-sample, because acquisition cannot look ahead:
* turn the operator's choice into one ``Sensor.configure(...)`` call
(:meth:`FilterSettings.apply_to`);
* read the **effective** values back from ``Sensor.filters_state()``
(:meth:`FilterSettings.from_filters_state`), so the dialog and the record show
what the device is doing rather than what was typed into it;
* write the settings into the recording metadata (:meth:`FilterSettings.metadata`),
keeping the **measurement** group (``flow_ema_alpha``: it changes the flow, so
two recordings with different values are not comparable) apart from the
**readout** group (median, low-pass: they only smooth the force on its way out).
* **median** — a window of ``median_window`` samples (odd, ≥3), applied per
channel; it removes single-sample spikes without smearing a step;
* **low-pass** — a Butterworth IIR with persistent state (``sosfilt`` with
``zi``), the same shape as the SDK's own ``LowpassFilter``.
Order is median then low-pass: the median protects the IIR from a spike that
would otherwise ring for several samples.
``ForceFilterChain`` at the bottom is a reference implementation kept for
``tests/test_force_filters.py``, which pins that this app understands the SDK's
chain exactly. The app itself never filters.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable, Optional
import inspect
import logging
from dataclasses import dataclass, replace
from typing import Any, Iterable, Mapping, Optional
import numpy as np
#: Channels of the chain: three raw counts followed by the three newton values.
logger = logging.getLogger(__name__)
#: Channels of the reference chain: three raw counts then the three newtons.
CHANNELS = 6
#: The SDK settings this app drives, in chain order (measurement first). These
#: are the sensor's own *state* keys: the same words appear in ``sensor.json``,
#: ``filters_state()``, ``session.json`` and the preferences, so a reader never
#: has to translate between two vocabularies.
MEASUREMENT_KEYS: tuple[str, ...] = ("flow_ema_alpha",)
READOUT_KEYS: tuple[str, ...] = (
"force_median_window",
"force_lowpass_cutoff_hz",
"force_lowpass_order",
"force_lowpass_sample_rate_hz",
)
def _odd(value: Any, *, minimum: int = 3, maximum: int = 31) -> int:
#: State key -> ``Sensor.configure`` parameter. Not the same on both sides: the
#: low-pass pair is ``force_lowpass_*`` in the config/state and ``lowpass_*`` in
#: the call. One table, so "can the SDK take this?" and "what do we pass?" can
#: never disagree.
CONFIGURE_PARAMETERS: dict[str, str] = {
"flow_ema_alpha": "flow_ema_alpha",
"force_median_window": "force_median_window",
"force_lowpass_cutoff_hz": "lowpass_cutoff_hz",
"force_lowpass_order": "lowpass_order",
"force_lowpass_sample_rate_hz": "lowpass_sample_rate_hz",
}
#: A sensible cutoff to offer when the operator switches the low-pass on.
SUGGESTED_LOWPASS_HZ = 10.0
def _as_float(value: Any, default: float) -> float:
try:
number = int(value)
return float(value)
except (TypeError, ValueError):
return 0
if number < minimum:
return 0
if number % 2 == 0:
number += 1
return min(number, maximum)
return default
def _as_int(value: Any, default: int) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def _sample_rate(value: Any) -> float | str:
"""``"auto"``, or a positive rate in Hz. Anything else is ``"auto"``."""
if isinstance(value, str):
text = value.strip().lower()
return "auto" if text in ("", "auto", "measured") else "auto"
rate = _as_float(value, 0.0)
return rate if rate > 0 else "auto"
@dataclass(frozen=True)
class FilterSettings:
"""What the operator chose. ``enabled`` flags decide what actually runs."""
"""The filter settings the operator chose, in the SDK's own vocabulary.
median_enabled: bool = False
median_window: int = 5
lowpass_enabled: bool = True
lowpass_hz: float = 10.0
Every stage is switched by its **own value** (``flow_ema_alpha >= 1``,
``median_window <= 1``, ``lowpass_hz <= 0`` all mean "off"), so there is no
second boolean to disagree with the number. Values that cannot be used are
resolved to "off" by :meth:`normalised`, never to a default that filters.
"""
#: Flow-EMA exponent. **Measurement**: it changes the flow, the contact
#: statistics and ``_force_raw``, so two recordings with different values are
#: not comparable.
flow_ema_alpha: float = 1.0
#: Sliding-median window in samples. **Readout**.
median_window: int = 1
#: Low-pass cutoff in Hz. **Readout**.
lowpass_hz: float = 0.0
lowpass_order: int = 2
sample_rate_hz: float = 30.0
#: Show the unfiltered readings next to the filtered ones on the cards.
#: ``"auto"`` follows the measured frame rate (30 fps until measured); a
#: number pins the rate the coefficients were designed for.
sample_rate_hz: float | str = "auto"
#: Display only — draw the unfiltered curves beside the filtered ones. Never
#: sent to the sensor, and never part of what makes two records comparable.
show_unfiltered: bool = False
#: The preset this came from, when it came from one (kept so a record can say
#: "balanced" instead of only the numbers it expanded to).
preset: Optional[str] = None
# ------------------------------------------------------------- resolving
def normalised(self) -> "FilterSettings":
window = _odd(self.median_window)
return FilterSettings(
median_enabled=bool(self.median_enabled) and window >= 3,
median_window=window or 5,
lowpass_enabled=bool(self.lowpass_enabled) and float(self.lowpass_hz) > 0,
lowpass_hz=max(0.0, float(self.lowpass_hz)),
lowpass_order=max(1, min(6, int(self.lowpass_order))),
sample_rate_hz=max(1.0, float(self.sample_rate_hz)),
"""Clamp to what the SDK can run, with unusable values meaning "off"."""
alpha = _as_float(self.flow_ema_alpha, 1.0)
if not 0.0 < alpha < 1.0:
alpha = 1.0
window = max(1, _as_int(self.median_window, 1))
cutoff = max(0.0, _as_float(self.lowpass_hz, 0.0))
order = min(6, max(1, _as_int(self.lowpass_order, 2)))
return replace(
self,
flow_ema_alpha=alpha,
median_window=window,
lowpass_hz=cutoff,
lowpass_order=order,
sample_rate_hz=_sample_rate(self.sample_rate_hz),
show_unfiltered=bool(self.show_unfiltered),
)
def active(self) -> bool:
"""Whether any stage actually filters (a disabled chain is a pass-through)."""
settings = self.normalised()
return bool(settings.median_enabled or settings.lowpass_enabled)
@property
def flow_ema_on(self) -> bool:
return 0.0 < _as_float(self.flow_ema_alpha, 1.0) < 1.0
def as_dict(self) -> dict:
@property
def median_on(self) -> bool:
return _as_int(self.median_window, 1) > 1
@property
def lowpass_on(self) -> bool:
return _as_float(self.lowpass_hz, 0.0) > 0.0
def active(self) -> bool:
"""Whether any stage filters at all (an off chain is a pass-through)."""
return bool(self.flow_ema_on or self.median_on or self.lowpass_on)
def as_state(self) -> dict[str, Any]:
"""The settings in the shape ``Sensor.filters_state()`` reports."""
settings = self.normalised()
rate = settings.sample_rate_hz
readout: dict[str, Any] = {
"force_median_window": settings.median_window,
"force_lowpass_cutoff_hz": settings.lowpass_hz,
"force_lowpass_order": settings.lowpass_order,
"force_lowpass_rate_auto": rate == "auto",
}
if rate != "auto":
readout["force_lowpass_sample_rate_hz"] = float(rate)
return {
"measurement": {"flow_ema_alpha": settings.flow_ema_alpha},
"readout": readout,
}
def _values_by_state_key(self) -> dict[str, Any]:
"""This request, keyed the way the SDK's state names things."""
settings = self.normalised()
return {
"median_enabled": settings.median_enabled,
"median_window": settings.median_window,
"lowpass_enabled": settings.lowpass_enabled,
"lowpass_hz": settings.lowpass_hz,
"lowpass_order": settings.lowpass_order,
"sample_rate_hz": settings.sample_rate_hz,
"flow_ema_alpha": settings.flow_ema_alpha,
"force_median_window": settings.median_window,
"force_lowpass_cutoff_hz": settings.lowpass_hz,
"force_lowpass_order": settings.lowpass_order,
"force_lowpass_sample_rate_hz": settings.sample_rate_hz,
}
def as_dict(self) -> dict[str, Any]:
"""A flat mapping for the preferences file (SDK keys, plus display)."""
settings = self.normalised()
return {
"flow_ema_alpha": settings.flow_ema_alpha,
"force_median_window": settings.median_window,
"force_lowpass_cutoff_hz": settings.lowpass_hz,
"force_lowpass_order": settings.lowpass_order,
"force_lowpass_sample_rate_hz": settings.sample_rate_hz,
"show_unfiltered": settings.show_unfiltered,
"preset": settings.preset,
}
def describe(self) -> str:
"""A short human line for the recording metadata."""
"""A short line for the record, upstream stage first."""
return describe_state(self.as_state())
def problems(self, sample_rate_hz: Optional[float] = None) -> tuple[str, ...]:
"""Why the device would not do what these settings ask.
The SDK drops a low-pass whose cutoff is at or above Nyquist instead of
silently re-tuning it, so the dialog has to say so *before* the session
starts — otherwise the file would claim a filter the device is not
running.
"""
settings = self.normalised()
parts = []
if settings.median_enabled:
parts.append(f"median({settings.median_window})")
if settings.lowpass_enabled:
parts.append(
f"lowpass({settings.lowpass_hz:g}Hz,order{settings.lowpass_order})"
issues: list[str] = []
if settings.lowpass_on and sample_rate_hz:
nyquist = 0.5 * _as_float(sample_rate_hz, 0.0)
if nyquist > 0 and settings.lowpass_hz >= nyquist:
issues.append(
f"lowpass {settings.lowpass_hz:g} Hz is at or above Nyquist "
f"({nyquist:g} Hz): the device would not filter"
)
return tuple(issues)
# ---------------------------------------------------------- driving the SDK
def unsupported_by(self, sensor: object) -> tuple[str, ...]:
"""The SDK keys this sensor's ``configure()`` does not accept.
Reported rather than dropped: a stage the operator switched on but the
installed SDK cannot run is exactly the disagreement that ends up in a
recorded file. Older SDKs (before 0.5.2) have no ``flow_ema_alpha`` and
no median, and their flow EMA is not switchable at all.
"""
try:
parameters = inspect.signature(sensor.configure).parameters
except (AttributeError, TypeError, ValueError): # pragma: no cover - exotic fake
return tuple(CONFIGURE_PARAMETERS)
return tuple(
key
for key, parameter in CONFIGURE_PARAMETERS.items()
if parameter not in parameters
)
def configure_kwargs(self, skip: Iterable[str] = ()) -> dict[str, Any]:
"""The ``Sensor.configure(...)`` call these settings amount to."""
skipped = set(skip)
return {
CONFIGURE_PARAMETERS[key]: value
for key, value in self._values_by_state_key().items()
if key not in skipped
}
def apply_to(self, sensor: object) -> tuple[str, ...]:
"""Push these settings into the device; return what it could not take.
Called from the acquisition thread right after the sensor opens, so a
session is filtered the way the operator asked for from its first frame.
"""
unsupported = self.unsupported_by(sensor)
if unsupported:
logger.warning(
"The installed orisys SDK cannot apply %s; the effective filter "
"state is in Sensor.filters_state()",
", ".join(unsupported),
)
return " + ".join(parts) if parts else "none"
sensor.configure(**self.configure_kwargs(unsupported))
return unsupported
# ------------------------------------------------------------ reading back
@classmethod
def from_filters_state(
cls, state: Mapping[str, Any], *, show_unfiltered: bool = False
) -> "FilterSettings":
"""The settings a device reports as in effect.
This is what the dialog displays and what a record stores: a config file
or a request can say one thing while the running device does another.
"""
measurement = dict(state.get("measurement") or {})
readout = dict(state.get("readout") or {})
rate: Any = "auto"
if not readout.get("force_lowpass_rate_auto", True):
rate = readout.get("force_lowpass_sample_rate_hz", "auto")
return cls(
flow_ema_alpha=_as_float(measurement.get("flow_ema_alpha"), 1.0),
median_window=_as_int(readout.get("force_median_window"), 1),
lowpass_hz=_as_float(readout.get("force_lowpass_cutoff_hz"), 0.0),
lowpass_order=_as_int(readout.get("force_lowpass_order"), 2),
sample_rate_hz=rate,
show_unfiltered=bool(show_unfiltered),
preset=state.get("preset"),
).normalised()
def metadata(self, sensor_state: Optional[Mapping[str, Any]] = None) -> dict[str, Any]:
"""The ``filters`` block for ``session.json`` / a recording header.
With a sensor state this records what the device was doing
(``source: sensor.filters_state``); without one it records what the app
asked for and says so, so the two are never mistaken for each other.
"""
if sensor_state:
measurement = dict(sensor_state.get("measurement") or {})
readout = dict(sensor_state.get("readout") or {})
preset = sensor_state.get("preset")
source = str(sensor_state.get("source") or "sensor.filters_state")
describe = describe_state(sensor_state)
else:
state = self.as_state()
measurement = state["measurement"]
readout = state["readout"]
preset = self.normalised().preset
source = "app.FilterSettings"
describe = describe_state(state)
block: dict[str, Any] = {
"measurement": measurement,
"readout": readout,
"describe": describe,
"source": source,
"show_unfiltered": bool(self.show_unfiltered),
}
if preset:
block["preset"] = str(preset)
return block
def settings_from_mapping(data: Optional[dict]) -> FilterSettings:
def describe_state(state: Mapping[str, Any]) -> str:
"""``none`` / ``flowEMA(0.5) + median(3) + lowpass(10Hz,order2)``.
Written upstream-first, which is also the order the stages run and the order
the two groups are listed in: the measurement stage first.
"""
measurement = dict(state.get("measurement") or {})
readout = dict(state.get("readout") or {})
parts: list[str] = []
alpha = _as_float(measurement.get("flow_ema_alpha"), 1.0)
if 0.0 < alpha < 1.0:
parts.append(f"flowEMA({alpha:g})")
window = _as_int(readout.get("force_median_window"), 1)
if window > 1:
parts.append(f"median({window})")
cutoff = _as_float(readout.get("force_lowpass_cutoff_hz"), 0.0)
if cutoff > 0:
order = _as_int(readout.get("force_lowpass_order"), 2)
parts.append(f"lowpass({cutoff:g}Hz,order{order})")
return " + ".join(parts) if parts else "none"
def settings_from_mapping(data: Optional[Mapping[str, Any]]) -> FilterSettings:
"""Build settings from a preferences/metadata mapping, with defaults."""
source = dict(data or {})
defaults = FilterSettings()
base = FilterSettings()
return FilterSettings(
median_enabled=source.get("median_enabled", defaults.median_enabled),
median_window=source.get("median_window", defaults.median_window),
lowpass_enabled=source.get("lowpass_enabled", defaults.lowpass_enabled),
lowpass_hz=source.get("lowpass_hz", defaults.lowpass_hz),
lowpass_order=source.get("lowpass_order", defaults.lowpass_order),
sample_rate_hz=source.get("sample_rate_hz", defaults.sample_rate_hz),
show_unfiltered=source.get("show_unfiltered", defaults.show_unfiltered),
flow_ema_alpha=source.get("flow_ema_alpha", base.flow_ema_alpha),
median_window=source.get("force_median_window", base.median_window),
lowpass_hz=source.get("force_lowpass_cutoff_hz", base.lowpass_hz),
lowpass_order=source.get("force_lowpass_order", base.lowpass_order),
sample_rate_hz=source.get("force_lowpass_sample_rate_hz", base.sample_rate_hz),
show_unfiltered=source.get("show_unfiltered", base.show_unfiltered),
preset=source.get("preset", base.preset),
).normalised()
# ---------------------------------------------------------------------------
# Reference implementation: kept so the tests can prove this app understands the
# SDK's chain exactly. Nothing in the app calls it.
# ---------------------------------------------------------------------------
class ForceFilterChain:
"""The median + low-pass chain, one instance per acquisition session."""
"""The median + low-pass chain the SDK runs, reimplemented for comparison.
Both stages are causal and per-sample, because acquisition cannot look
ahead: the median removes single-sample spikes without smearing a step, and
the low-pass is a Butterworth IIR with persistent state (``sosfilt`` with
``zi``). Order is median then low-pass, so a spike cannot ring the IIR.
"""
def __init__(self, settings: Optional[FilterSettings] = None) -> None:
self._settings = (settings or FilterSettings()).normalised()
@@ -128,11 +379,12 @@ class ForceFilterChain:
self._sos = None
self._zi = None
settings = self._settings
if not settings.lowpass_enabled:
if not settings.lowpass_on:
return
from scipy.signal import butter
nyquist = 0.5 * settings.sample_rate_hz
rate = _as_float(settings.sample_rate_hz, 30.0)
nyquist = 0.5 * rate
cutoff = min(float(settings.lowpass_hz), nyquist * 0.95)
if cutoff <= 0:
return
@@ -182,7 +434,7 @@ class ForceFilterChain:
return sample.copy()
settings = self._settings
out = sample.copy()
if settings.median_enabled:
if settings.median_on:
window = int(settings.median_window)
for index in range(CHANNELS):
series = self._median[index]