The tests that cover per-channel metering ask a fabric which radio will carry a
packet, which an openhop_core from before the selector change cannot answer. Run
against one, fourteen of them failed -- on a repeater whose whole point is that
the core change is optional.
Tests that genuinely need the newer core (they drive build_radio_stack through a
real FabricRadio) are skipped when it is absent. The engine tests instead use a
fabric double that implements core's resolve rule and nothing else: explicit
radio_id wins, then the selector, then default_radio. That keeps the engine
measured against the contract rather than against a convenience, and the real
RFFabric is still what runs wherever the installed core supports it.
One fan-out test asserted the engine names the egress radio. What must not
change is which radio transmits; whether the engine names it or leaves the
choice to the fabric depends on the core underneath, so it accepts either.
Against openhop_core c95a684: 2154 pass, 6 skipped. Against dev with the
selector change: 2157 pass, 3 skipped.
A duty cycle is a limit on a channel, not on a node, and one AirtimeManager
charging every radio at the top-level radio block's modulation is wrong twice
over on a bridge. The same packet occupies a 62.5 kHz channel about eight times
longer than a 500 kHz one, so whichever bandwidth that block names, the other
radio is mis-charged by that ratio; and the two share one allowance, so a busy
local radio spends the backhaul's airtime on a band it never transmits on.
There is now one budget per channel, built from each radio's own profile and
its own band's limit -- 868.0-868.6 MHz allows 1% where 869.4-869.65 allows
10%, and one number cannot describe both. Two radios tuned alike share a
budget, because they contend for the same spectrum. duty_cycle.shared_budget
restores a single node-wide allowance for anyone who wants the conservative
reading, and a single-radio node is metered exactly as it was.
Metering a channel means knowing which radio will carry a packet, and the
engine used to work that out by restating openhop_core's dispatch rules. It
asks now. openhop_core passes a packet's ingress radio to the TX policy, so the
choice is a function of the packet rather than of whatever arrived most
recently; the repeater asks before scheduling and gets the answer the send will
reach. That also closes a race that predates this: sticky and bridge read the
node's last RX at send time, which after a retransmit delay on a busy node is
another packet's arrival, and moves a reply onto the wrong band.
The core change is optional. The keyword in resolve_tx_radio_id's signature is
the version check, and against a core without it the tx_mode selectors are
registered in their original one-argument form and behave exactly as they did.
What changes there is the gate: with no way to know which radio will transmit,
every radio the fabric could pick is held to its own budget, so a node cannot
transmit past a legal limit by guessing the quiet channel. Which mode is in
force is logged at startup.
Also fixed, each reproduced by an adversarial review of the earlier attempt:
- A node with one radios[] entry had its modulation and limit overwritten by
the stale top-level block on every radio save from the web UI. The refresh
path now branches on whether profiles were read, not on how many radios
there are.
- build_radio_profiles is all-or-nothing by design -- it feeds the UI -- so one
radio_type: none collapsed metering onto the top-level block for the radio
still transmitting. Metering has its own builder that degrades per radio.
- Carrying the whole node's spend onto an unrecognised channel compounded over
repeated rebuilds (100 -> 400 -> 1600 ms) and wedged transmission for the
rest of the window. Carry-over is keyed by channel, which makes an unchanged
rebuild idempotent, and an unrecognised one starts from the busiest single
channel rather than the sum of them.
- A rebuild worked the previous default manager out from the config again,
which by then already named the new default -- preserving the wrong object
and orphaning the one NeighborScopeHelper holds for the life of the process.
It is remembered rather than recomputed.
- Every caller that reports one airtime figure was reporting the default
radio's channel: the MeshCore wire field total_air_time_secs, companion
stats, the RRD and SQLite history, and /stats. They read a node total again
-- totals summed across channels, utilisation the highest of them, because a
node at its limit on one channel is at a limit however quiet the other is.
tests/test_fabric_fanout.py's rig never bound a tx_mode selector, so all 49 of
those tests ran against a fabric with no TX policy at all. It binds the real
one now; that gap is why an on-air change went unnoticed through two reviews.
A bridge sends one logical packet through both radios, and the two do not fare
alike: 500 kHz and 62.5 kHz are different channels with different neighbours and
different contention. The packet row holds one set of LBT figures, from the
primary egress, so the other radio's attempts and backoffs were dropped on the
floor -- and an egress that failed was never stored at all, leaving the half of
the story an operator most wants to see only in the log.
A new packet_egress table keeps one row per physical send: packet id, timestamp,
radio, whether it went out, and its LBT attempts, total backoff and channel-busy
flag. Written only on a node with two or more radios; with one radio the packet
row already says all of this, and nothing about single-radio behaviour changes.
The rows are written on the storage writer thread beside the packet, addressed
by the id the insert returns, and are kept off packet_record itself so they do
not ride along on every websocket, Glass and MQTT publish. Retention ages them
out on the packet cutoff, and purging packets takes them along.
/lbt_diagnostics gains radios[]: the same summary and bucket shape per radio,
read from the new table over a covering index, so one radio's contention is
never read as the other's. The combined figures above it are unchanged and still
count each packet once. Per-radio entries carry no rf block -- the RRD series
behind it is node-wide, and hanging it off one radio would read as that radio's
own RSSI and loss -- and no packet-type breakdown, which would cost a heap
lookup per egress.
The fan-out now reads a send's LBT metadata whether or not it succeeded. The
single-egress path has always done so, and the figures on a failed send are
exactly the ones that explain it -- as far as they reach us: a radio that raises
instead of returning metadata (an SX1262 giving up on a busy channel) reports
nothing to read, so that row records the failure with no attempts rather than
inventing them.
Redundancy copies are recorded too. A multi-ack burst is real airtime on both
radios, and leaving it out would undercount exactly the traffic multi-ack adds
to a channel already busy enough to need it.
/db_stats lists packet_egress only once something has been written to it, so a
single-radio node reports the tables it always reported and never pays for
counting rows it does not have.
The bucket and summary maths moved out of get_lbt_diagnostics into three module
functions so the per-radio series is the same code as the combined one rather
than a second copy of 150 lines. The combined answer is unchanged.
A bridge has one receiver per band, and until now one of them stood for the
node: the background timer read the default radio and stored the answer with no
radio attached. On the 910.100/910.525 pair that reported the wide radio's noise
floor as the whole node's, and charged its CRC counter with both radios' errors.
Every physical radio is now sampled, one after another, and stored against its
own id. Migration 19 adds a nullable radio_id to noise_floor and crc_errors with
a covering index each, so a per-radio history reads only its own rows. Rows
written before this stay NULL: they were never measured per radio, and
labelling them would claim an attribution nobody made.
CRC baselines are per radio. A shared one charges every radio's errors to
whichever was sampled last, which reports swings that never happened.
Reads take an optional radio_id: /noise_floor_history, /noise_floor_stats,
/crc_error_count and /crc_error_history. Naming a radio also makes offset paging
stable, which the RF Health page depends on -- paging the merged series shifts a
page's contents every time the other radio samples. History rows name their
radio only on a node with two or more radios, so a single-radio response is
byte-for-byte what it was.
The default radio is sampled first and is the only one published. /stats, the
MQTT status message and companion stats read its cached value, and publishing
per radio doubles the observer-feed cadence -- its own change, not this one.
The StorageCollector facade carries the new arguments too. The web endpoints
reach the database through it, so the tests that drive them autospec it: a bare
mock accepts any keyword and would have let a facade that had not kept up
answer happily in tests and raise TypeError on a real node.
Adding the radio filter and per-radio split by concatenating SQL fragments tripped bandit B608, failing the pre-commit check. The fragments were constants and every value was bound, but the queries are now three fixed strings: a NULL radio parameter disables the rx_radio_id filter, and the per-radio split selects its own query. Results are unchanged.
Picks up the per-radio analytics fixes from openHop_RepeaterUI feat/per-radio-airtime: settled and generation-checked loads, whole-minute chart buckets, per-radio duplicate series, and the radio selection kept in ?radio=.
The self-echo guard registered a packet's hash and forwarded path when the
engine decided to forward it and kept it for 30 s. Two things could then
hide a real neighbour from the link tracker:
- A neighbour whose 1-byte path hash collides with ours, repeating the
same packet from the same upstream path, carries exactly the path we
would have sent.
- The registration happened before the duty-cycle gate, so a packet that
was never transmitted still suppressed matching receptions.
Registration now happens at the physical send, inside the TX lock after
the duty-cycle gate, and records which radio is sending. A send that
fails is forgotten. A reception counts as our echo only when it arrives on
a radio other than the one that sent it, and while that send is on the
air or within 2 s of it finishing, which covers a remote modem's delivery
delay. Single-radio nodes keep no records at all, since a lone radio
cannot hear itself.
A colliding neighbour is now mistaken for our echo only if it repeats the
same packet on the same path, on the other radio, inside that window.
The engine now passes each reception's radio to the neighbour link
tracker, from both the forwarding path and the duplicate-recording path,
so /neighbor_links can report how each radio hears a neighbour.
Two radios sharing one frequency hear each other transmit. The echo of a
relayed flood ends in this node's own path hash, so the tracker recorded
the node as its own neighbour, at near-zero-dB signal. The engine now
remembers the packet hash and exact path of each transmission for 30 s,
registered before the first send because an echo can arrive while a
later egress of the same fan-out is still on the air, and skips tracker
observation for a reception that matches. A neighbour repeating our copy
appends its own hash and still counts. The echo is still stored and
published as the reception it is; only the neighbour table ignores it.
Matching the exact path rather than the last hop's hash keeps a real
neighbour whose 1-byte hash collides with ours.
On a Fabric node the analytics endpoints merged every radio together, with
one transmission per logical packet and nothing to tell a reception on one
radio from the other. Every packet row already records its ingress radio
and each radio that sent it, so the split needs no new data.
The attribution rules get_airtime_buckets introduced move into shared
helpers: packet_carriers() says which stored radio ids a row occupied, and
RadioResolver maps them onto the configured radios (one radio owns every
row; with two, an unknown id is unattributed, never guessed).
get_airtime_buckets now uses them with no change in output.
With two or more radio profiles:
- /packet_stats adds radios[] (received, duplicates, dropped, physical
transmissions, average RSSI/SNR) and unattributed counts. The legacy
totals come from the same grouped pass, so the 24 h aggregate the
WebSocket broadcasts does not gain a second scan. Totals still count
one transmission per packet.
- /route_stats adds the route mix of each radio's receptions, plus
originated for packets no radio received.
- /radio_packet_rates (new) returns receptions and transmissions per
radio per bucket, grouped in SQL on columns idx_packets_airtime covers.
A week-long window is an index-only scan instead of a Python loop over
every row: 0.17 s against 1.11 s for the airtime loop on a synthetic
1.55M-row table.
- /neighbor_links gives each link radios[], its stats as each radio heard
it. The tracker keeps them beside the merged figures without changing
its keys or entry count.
- /neighbor_link_history rows carry rx_radio_id, and the endpoint takes a
radio_id filter and a by_radio bucket split.
A single-radio node gets exactly the answers it had: the legacy queries
still run for it and every new field is omitted.
get_airtime_buckets selects rx_radio_id, tx_radio_id and tx_radio_ids to
charge airtime per radio, but idx_packets_airtime only covers timestamp,
length, payload_length and transmitted. SQLite therefore fell back to
idx_packets_timestamp with a heap lookup for every row in the window:
the access pattern the covering index was added to remove, and the one
that pushes the chart past the client timeout on an SD card. The test
meant to guard this asserted the plain timestamp index.
Migration 18 drops idx_packets_airtime and rebuilds it with the three
radio id columns. CREATE INDEX IF NOT EXISTS never reshapes an existing
index, so the base schema cannot do this. It keeps creating the original
four-column shape, which is safe on a database that predates the radio
id columns, and the migration widens it once migrations 16 and 17 have
added them. The rebuild reads the packets table once, at startup.
The bucket query is now a module constant shared with the test, which
asserts COVERING INDEX idx_packets_airtime for both airtime queries, so
a column added to the query cannot slip out of the index unnoticed.
openhop-dev/openhop_core#123 settled on fabric.origin_tx: default | all
as the name for sending locally originated packets out on every radio,
and openhop-dev/openhop_core#124 with openhop-dev/openhop_repeater#412
implement that knob in core. Use the same key here so a Fabric config
means one thing whichever implementation lands.
local_tx_mode is still read, so a config written against the first
version of this branch keeps its behaviour. If both keys are set to
different modes, startup rejects the config before any hardware is
opened, since there is no way to tell which one the operator meant.
While running, the engine reads the option through the same helper and
lets origin_tx win. Validation errors name the key the operator actually
wrote.
The radio stack meta and the startup log report origin_tx.
A two-radio Fabric bridge runs each side at its own SF, so one combined
utilization line computed from a single caller-supplied profile describes
neither radio. Report airtime per radio instead.
build_radio_stack now records the effective air settings of every radio it
builds, in configured order, as radio_stack_meta["radio_profiles"]. The
same builder is exposed as build_radio_profiles() so /stats and the airtime
endpoint can read current settings from the live config rather than a boot
snapshot. Air settings are read the way get_radio_for_board reads them, and
a field that cannot be read is reported as null rather than defaulted, so
nothing is charted from settings that were never on the air.
get_airtime_buckets takes those profiles and charges each stored packet to
the radios that actually carried it: its ingress rx_radio_id and every
successful egress in tx_radio_ids, falling back to the scalar tx_radio_id
for pre-fan-out rows. A relayed packet therefore appears as RX on the radio
that heard it and TX on each radio that sent it, each at that radio's own
time-on-air. Duplicate receptions are included; they occupied the receiver.
With one profile every packet belongs to that radio, so single-radio and
pre-Fabric databases are attributed exactly as before. With two or more, a
packet whose radio cannot be identified is counted as unattributed rather
than guessed onto the default radio.
The response is additive: legacy buckets/rx_total/tx_total remain as the
combined series, and radios[] carries one series per radio alongside
unattributed_rx_count/unattributed_tx_count. Airtime comes from the shared
core estimator instead of a second inline Semtech formula, memoised per
(radio, length). The sf/bw_hz/cr/preamble query parameters are kept only as
a fallback; server profiles are authoritative and are never applied across
both radios of a bridge.
Add two Fabric options for an exactly-two-radio node:
- fabric.repeat_on_ingress (tx_mode: bridge): a relayed packet is sent
on the other radio first, then back out on the radio that heard it.
- fabric.local_tx_mode: all: locally originated packets (companions,
adverts, room/protocol replies) go out once on every radio, default
radio first.
MeshCore processing (policy, dedupe, path mutation) still runs once per
logical packet; only the physical TX is duplicated, each egress with its
own Packet, serialised through the TX lock and gated independently for
duty cycle and link health. One successful egress is a successful
forward. Multi-ack extras share the egress set.
inject_packet registers its single ACK waiter before the first TX so an
ACK heard mid-fan-out outlives the dispatcher's ACK cache. Fan-out fixes
flood scope and path-hash mode once before copying and metering, so all
radios carry identical bytes and the duty-cycle gates see them.
Packet records gain tx_radio_ids (JSON column + migration); tx_radio_id
remains the primary successful egress. Invalid option combinations are
rejected before hardware is opened. Defaults preserve existing behaviour.
The companion frame proxy only accepted an operator JWT in `?token=`,
while `/ws/packets` also accepts a long-lived API token (X-API-Key
header, or the same query parameter). Apply the identical rule to the
frame proxy so a client that already holds a repeater API key can open
the companion link on the HTTP port without keeping a JWT session alive.
No behaviour change for JWT clients. Same fallback order, close codes
and log lines as PacketWebSocket.
After mesh.default_region was wired to the core dispatcher's send-time
scope resolver, a relayed plain flood was indistinguishable from a
locally originated one, so Dispatcher._apply_flood_scope() converted it
to the repeater's default transport scope and sent it out-of-region.
Successful flood forwards now mark their existing scope decision as
authoritative before entering Dispatcher.send_packet(): relayed plain
floods stay plain and relayed transport floods retain the originator's
codes. Locally originated floods and REPLY_SCOPE_DEFAULT fallback
replies still pick up the configured default scope.
Marking only on the flood_forward() success path leaves packets rejected
by validation, policy, duplicate, or loop detection untouched, and
leaves locally injected transmissions (which bypass forwarding) alone.
The SSE endpoints registered their bridge callbacks once behind a
_callbacks_registered latch. Anything that cleared the bridge's push
callbacks therefore killed the stream permanently — the latch stayed True
while the subscriptions were gone, so no later stream open repaired them.
Cache the callback objects instead and re-register them each time a stream
opens. Bridge registration is idempotent, so the repeat costs nothing while
a subscription that went missing is repaired rather than leaving the stream
silently dead until the daemon restarts.
The clear that made this reachable is fixed in openhop_core, where the
frame server now retracts only its own subscriptions instead of clearing
every callback on each companion client connection. This half is the
belt-and-braces: clear_push_callbacks() remains a legitimate full reset,
and the stream should survive one.
agessaman