Adds a "spread up to Nkm" part to the ping-bot's reply, computed as the
farthest any hearing station's own GPS fix was from whoever heard it
first -- the same signal View Path's map shows, surfaced directly in
the chat bubble so it's visible without opening the map. Deliberately
skips View Path's neighbor-centroid position fallback (too expensive
to run per-ping across a page of messages); a station without its own
GPS fix just doesn't contribute, so the part is silently omitted when
fewer than two stations have one.
"heard by N observers" already reported total station breadth per
transmission, so no separate count was added.
Adds DistanceFromFirstKm to PacketPathBranch: the great-circle
distance (haversine) between a branch's own Observer and First's
Observer. Zero for First itself. Deliberately omitted when either
side is positioned via Approx (a neighbor-centroid estimate) -- a
distance computed against a guess isn't a real measurement worth
surfacing.
Rendered in the observer tooltip as "42.3 km away" alongside the
elapsed-time label, giving a concrete sense of how far the flood
physically reached, not just how many hops or how long it took.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds IsBridge to PacketPathPoint/PacketPathObserver and PublicKey to
PacketPathObserver (needed to look bridges up, and useful on its own
for future features). Set by a new markBridgeRepeaters handler-level
step in routes.go, not GetPacketPath itself, since the underlying data
(TransportedScopes per repeater) lives in the in-memory store, not
SQL -- same "relays for 2+ distinct regions" definition and data
source as the Foreign Traffic tab's existing Bridge badge
(ScopeStatsResponse.bridgeRepeaters), just applied to whichever
handful of pubkeys one packet's path touches.
The map gives a bridge repeater a bold purple outline (on top of
whatever primary/approx styling already applies) plus a tooltip note,
so the mesh's backbone nodes stand out regardless of which branch
they're in.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PacketPathPoint already carried Role; PacketPathObserver now does too
(from the observer's own nodes row, when it's known as a mesh node
itself and not just an MQTT/API listener). Rendered as a small icon
prefix in the tooltip (📡 repeater, 🏠 room, 📱 client, 🌡️ sensor) --
markers stay plain circleMarker dots throughout, since a role-specific
shape would clash with the color/dash coding already carrying primary,
approx, and observer meaning.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
nearestPositionedNeighbor now also returns how many positioned
neighbors fed the weighted centroid and the widest distance between
any two of them (0 with a single contributor). Exposed via new
ApproxNeighborCount/ApproxSpreadKm fields on PacketPathPoint and
PacketPathObserver.
The map scales the approximate marker's size/opacity accordingly: one
neighbor (or several that disagree widely) renders as a bigger,
fainter ring; several agreeing neighbors render tighter and more
solid. The tooltip also states the contributor count.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds SecondsAfterFirst to PacketPathBranch: the gap between the
earliest-arriving observation (First) and this branch's own deepest
observation. Zero for First itself. Rendered in the observer's tooltip
as "+4.7s" (or "Nm Ss" for longer gaps, "first to arrive" for zero) --
gives a sense of the propagation order across the flood, not just
depth/breadth.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
nearestPositionedNeighbor picked exactly one neighbor (the strongest by
edge count) and used its exact coordinates -- so the approximate marker
always landed precisely on top of that neighbor's own dot. Since each
neighbor's own position is a real, precise fix, and only the
unpositioned node's position relative to them is unknown, more
positioned neighbors should narrow the estimate rather than being
ignored in favor of just the top one.
Now sums every positioned neighbor's coordinates weighted by
neighbor_edges.count and returns the centroid. With exactly one
positioned neighbor this is unchanged (reduces to that neighbor's exact
position); with several, the marker settles somewhere among them
instead of overlapping a single specific node.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A hop or observer with no position of its own (no self-advertised GPS,
no name match, no IATA) previously stayed unplotted entirely. Adds a
last-resort fallback: look up its strongest neighbor_edges neighbor
(ranked by observation count) and, if that neighbor has a real
position, borrow it as an approximate stand-in -- flagged via a new
`approx` field so callers never mistake it for a real fix.
The View Path map renders approximate markers hollow and dashed
instead of solid, with a status-line callout ("N approximate (via
nearest neighbor)"), and the "N hops without a known position" count
drops accordingly for whichever ones now have a stand-in.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GetPacketPath's node-position lookups (relay-hop points and observer
positioning) read raw lat/lon straight off the nodes row, so a node
with (0,0) stored literally instead of NULL -- MeshCore's "never
actually reported a GPS position" case in practice -- rendered as a
real point off the coast of Ghana instead of no position at all.
Excludes (0,0) the same way GetNodesForScopeAdoption and
geofilter.PassesFilter already do; the node's name is kept, just not
the bogus position.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GetPacketPath's branches are sorted deepest-first, so the map had no
natural "where did this start" anchor. Adds a `first` field: the
single earliest-arriving observation across every station regardless
of observer or hop depth (the same "first observation wins" pick
already used for a ping message's own meta line), positioned the same
way as any other observer (own GPS, then name match, then IATA).
The map draws it as a green ring on top of everything else, since it
usually coincides with one of the already-plotted branch dots, plus a
status-line callout -- an approximate landmark for where the message
entered the visible mesh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Some bridge-type observers (seen in the wild on openHop-Repeater
firmware) publish their MQTT status keyed by device name instead of
their mesh pubkey, so observers.id never matches nodes.public_key for
them even though the same physical device has a real, positioned node
row under its actual pubkey. The pubkey-based GPS lookup added in
94be933 silently missed these.
Now falls back to matching the observer's display name against nodes
when the pubkey lookup finds nothing, skipping the match entirely if
more than one positioned node shares that name rather than guessing
which one is the real observer.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GetPacketPath only ever tried the hardcoded iataCoords table to place
an observer. Any station whose configured IATA code isn't a real
airport (a custom/regional code, or a typo) fell out of the map
entirely -- even when the station is itself a mesh node that has
self-advertised a real GPS position, the same source /api/observers
and the Wardriving tab already treat as authoritative.
Now checks the observer's own node-table position first (folded into
the existing bulk pubkey lookup, no extra query) and only falls back
to the IATA table when it has none.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GetPacketPath used to resolve only the single farthest-traveled
observation and discard the rest. It now returns one branch per
distinct station (kept at that station's own deepest observation),
each with its own hop count and, where resolvable, relay chain --
so the map visualizes the packet's full flood spread instead of one
route. Stations whose path never resolved, or who heard the packet
directly (0 hops), still contribute a branch via their own position,
rather than being silently dropped.
The "View path" link in the ping-bot reply no longer requires hops > 0
to show, since even an all-direct ping still has observer spread worth
plotting.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both are already shown on the triggering message's own meta line right
above it, so repeating them in the pong reply was redundant. Drops the
handler-level appendAreaToBotReply pass (routes.go) entirely along with
its now-unused scope plumbing through pendingPing (db.go) and the
client-side pingBotReply (channels.js).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Trigger check moved from a single string comparison to a small
pingTriggerWords set (mirrored by hand in db.go and channels.js), so
adding more trigger words later is a one-line change in each. Still an
exact match after the existing @mention-stripping -- "/pingx" etc. don't
match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New GET /api/packets/{hash}/path resolves a packet's DEEPEST observation
(same "farthest leg is more informative" reasoning as the ping-bot reply
itself) to a geographic point sequence: each relay's name/role/lat/lon in
path order, plus the hearing observer's position (from its configured
IATA code, like the Wardriving tab). Hops that have never advertised a
GPS position come back with null lat/lon rather than being dropped, so
the frontend can draw a gap instead of guessing.
Frontend: public/packet-path-map.js is a small on-demand Leaflet modal
(reuses node-reach-map.js's tile/marker conventions, but draws an ordered
chain instead of a star) opened via a new "View path" link on the
ping-bot reply -- shown only when there's an actual multi-hop route and
packet hash to look up. Kept general (keyed by packet hash, not
ping-specific) since any packet with a resolved path could use it later.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Matches the term used everywhere else in CoreScope (/api/observers,
observer names, etc.) instead of an inconsistent synonym.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GetChannelMessages previously used "first observation wins" for a ping's
botReply -- if the same flood was heard by multiple stations, only
whichever one happened to be scanned first shaped the hop count, SNR, and
relay path. That understates reach: different stations legitimately hear
the same flood at different hop depths depending on which relay leg
reached them.
Now tracks, across every observation of the ping: the DEEPEST (max-hop)
observation's hops/SNR/relay path, and every DISTINCT station that heard
it. The reply shows the deepest leg's path and reports "heard by N
stations" once more than one did (falling back to the single station's
name when there's exactly one, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
REST-loaded channel history (GetChannelMessages) now resolves the ping
message's relay path to node names ("via RepeaterA → RepeaterB"), bulk-
resolving every referenced pubkey across the page in one query rather
than per-message. Falls back to the raw pubkey when a hop's node isn't
known. Also includes the message's region scope and (via a handler-level
pass, since area resolution needs server config db.go doesn't have)
its resolved area.
The client-side pingBotReply (WebSocket live-push + PSK-channel decrypt
paths) gains scope/area too, since both are already present in that data.
Relay-path names are REST-only for now: the live WS broadcast doesn't
carry a resolved_path, only historical/REST-loaded messages do.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A channel message whose text is exactly "ping" (mention-prefix like
"@MeshviewBot ping" stripped first) now gets a synthesized "pong" reply
showing hop count, SNR, and hearing observer -- computed at read time
from that message's own already-stored data, no new table.
Deliberately NOT transmitted back onto the mesh: CoreScope has no publish
path to a MeshCore broker/radio (confirmed: it only ever subscribes to
MQTT, never publishes). The reply is visible only in CoreScope's own
Channels view, rendered as a visually distinct dashed-border bubble with
an explicit "Not sent to the mesh" caveat so it's never mistaken for a
real bot reply the sender's own radio received.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MeshCore firmware sends two capability bytes on ADVERT packets (wire bits
per AdvertDataHelpers.h) whenever the HasFeat1/HasFeat2 flags are set.
CoreScope already decoded them per-packet but discarded them rather than
storing a per-node value.
- New feat1/feat2 columns on nodes/inactive_nodes (internal/dbschema,
idempotent ALTER like the existing multibyte_sup/multibyte_evidence
columns).
- Ingestor's UpdateNodeTelemetry now writes feat1/feat2 alongside
battery_mv/temperature_c in the same COALESCE-based UPDATE, from the
same ADVERT payload.
- Server exposes them on /api/nodes and /api/nodes/{pubkey} (nullable,
same shape as battery_mv/temperature_c).
- Node detail page shows them as raw hex (0x....) in the Overview panel
when present -- undecoded, since CoreScope doesn't know the individual
bit meanings, but no longer silently discarded.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AreaScopeAdoption.Matching entries now carry MatchedScopes (which of
the area's linked regionScopes each node actually matched, via
default_scope or by relaying it) instead of just a bare name/key ref.
A node can match more than one when an area links several scopes and
the node uses/relays more than one of them.
Frontend: an area linking more than one scope (e.g. Europa's "eu" and
"europe") now splits its Supporting list into one sub-group per scope
instead of a single flat list, so it's visible which nodes support
which specific scope.
AreaEntry.RegionScope (single string) -> RegionScopes ([]string), so a
broad umbrella area (e.g. Europa) can link more than one scope name
(e.g. both "eu" and "europe") -- a node matching any one of them now
counts as supporting the area in computeScopeAdoptionByArea, the
"Scope Adoption by Area" section, and the regionScope->label lookup
used to annotate region codes elsewhere on the Scopes tab.
dborup: "Danmark (alle)" showed almost nothing (0 of 10 support) even
though most real nodes use the generic #dk scope. Root cause: the
per-area node count used AreaKeyForPoint, which picks only the single
most-specific area for each node -- a node in "Odense by" never also
counted toward the broader "Fyn" or "Danmark (alle)" it geographically
sits inside, so a country-level area only ever saw the leftovers no
smaller area had already claimed.
New AreaKeysForPoint returns every containing area (not just the
best match), used by computeScopeAdoptionByArea so a node now counts
toward all of its containing areas -- Danmark (alle) genuinely
aggregates every Danish sub-area's nodes now, not just stragglers.
AreaForPoint/AreaKeyForPoint (single-match, used by the GPS-share and
session area badges) are unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup wanted per-node visibility, not just aggregate counts: for
each area with a linked region, list the actual nodes that support
it (own default_scope or ever relayed it) vs. the ones that sit there
but don't. Replaces the summary table with expandable per-area groups
(same pattern as Repeaters by Region), each showing a Supporting /
Not Supporting node list with links.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup: a repeater that has relayed dk-horsens traffic supports the
Horsens area, even if its own default_scope is something else (e.g.
the generic #dk most nodes actually use) or unset entirely. The
previous version only checked default_scope, missing this -- same
runs-this-region vs carried-this-region's-traffic distinction the tab
already draws between OriginatingNodesByRegion and RepeatersByRegion.
Both NodesWithAnyScope and NodesMatchingArea now also check the
node's entry in the cached RepeaterRelayInfo map (TransportedScopes),
reusing the same cache RepeatersByRegion already populates.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New geographic view, independent of the raw hashRegion-code-based
Region Utilization: buckets every positioned node by its configured
area (AreaKeyForPoint) and tallies how many have any default_scope
at all, and how many specifically match the area's own linked
region. Surfaces gaps Region Utilization can't see, since that only
knows about region strings that already appeared in traffic — a real
area with real nodes that never produced a single scoped message is
invisible there but shows up here as 0% adoption.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes#1838
## Problem
`/api/scope-stats` reported 100% scoped whenever any region was
configured. Reporter noticed on a scopeless instance that "unscoped" was
always zero — the pie visual is misleading to operators deciding on
`denyf *`.
## Root cause
`cmd/server/db.go:22` restricted the entire scope-stats denominator to
`route_type IN (0, 3)`. Per firmware `docs/packet_format.md § Route
Types`:
- `0` = `TRANSPORT_FLOOD`
- `1` = `FLOOD`
- `2` = `DIRECT`
- `3` = `TRANSPORT_DIRECT`
Only routes 0 and 3 carry `transport_code_1` (transport-level scope).
Routes 1 and 2 are inherently unscoped by protocol. The existing SQL was
correct for the "how many transport-scopable routes are actually scoped"
question, but the denominator was silently promoted to "all traffic" in
the UI. Bonus: the comment on `routeTypeTransportSQL` labelled routes
0+3 as "FLOOD (0) and DIRECT (3)" — wrong on both counts.
## Fix
- `cmd/server/db.go` — corrected the `routeTypeTransportSQL` comment;
added `routeTypeNonTransportSQL = "route_type IN (1, 2)"` alongside it.
- `GetScopeStats` runs a second `COUNT(*)` over `route_type IN (1,2) AND
first_seen >= ?` and folds that count into `Summary.Unscoped`. Same
index path as the existing query — one extra scan per `/api/scope-stats`
call (cached 30s per triage's carmack finding).
- `public/analytics.js` — Scopes tab header explains the denominator
(all observed transmissions) and which route types carry scope. Card
notes now render `X% of all traffic` for Scoped/Unscoped and `X% of
scoped` for Unknown Scope so the pie's denominator is explicit.
## TDD
- Red: `5554ffe4` — extended `TestGetScopeStats` +
`TestHandleScopeStats` with `route_type=1` and `route_type=2` rows and
asserted `Unscoped = 3` (1 transport-NULL + 2 non-transport). Ran the
tests and confirmed assertion failure (`Unscoped = 1, want 3`).
- Green: `ebbb9253` — implementation + label copy. Full `go test
./cmd/server/...` passes (54s).
## Preflight overrides
- check-branch-clean: justified — cross-stack fix by design (backend
semantics change + matching frontend label copy). All 4 files are
exactly the surface the triage comment identified.
## Verification
- `go test ./cmd/server/...` — 54s, all pass.
- Firmware confirmation: `firmware/docs/packet_format.md:20-24` (route
type table).
## Files touched
- `cmd/server/db.go` — comment fix + second COUNT query.
- `cmd/server/db_test.go` — extended fixture.
- `cmd/server/routes_test.go` — extended fixture + isolate from seed
data.
- `public/analytics.js` — labels and header copy.
---------
Co-authored-by: corescope-bot <bot@corescope.dev>
## Summary
Issue #1478 — surface observers whose envelope timestamps are being
clamped because they're emitting zone-less local-time strings (UTC-N
observers showed up perpetually as "Stale" before #1466, and per-packet
rxTime is still clamped to ingest time for them, muddying
propagation-delay analytics).
Now the UI tells operators which observers are misconfigured + how to
fix it.
## What changed
### Ingestor (cmd/ingestor)
- New `observers_clock_naive_v1` migration adds three columns to
`observers`:
- `clock_skew_seconds INTEGER` (signed: negative = behind UTC, positive
= ahead)
- `clock_skew_count_24h INTEGER` (rolling 24h event count)
- `clock_last_naive_at TEXT` (RFC3339 timestamp of last clamp)
- `resolveRxTime` now returns `(rxTime, naiveSkewSec)`. The
packet-handler call site invokes `store.RecordNaiveSkew(observerID,
deltaSec)` whenever a naive envelope is clamped (the existing >15 min
naive-tolerance path). The counter resets to 1 if no event in the prior
24h, else increments. Single INSERT-or-UPDATE round trip per clamp.
### Server (cmd/server)
- `Observer` struct + `GetObservers` / `GetObserverByID` extended to
scan the three new columns.
- `ObserverResp` gains four JSON fields exposed by `/api/observers` and
`/api/observers/{id}`:
- `clock_naive` (bool, derived from `clock_last_naive_at` being within
24h)
- `clock_skew_seconds`, `clock_skew_count_24h`, `clock_last_naive_at`
- Decay is **read-side**: a stale event yields `clock_naive=false` with
zero counts. No background sweep, no writes from the read-only server,
no race with the ingestor.
### Frontend (public)
- `window.ObserversNaiveChip.render(o)` — total render helper, returns
⚠️ chip HTML when `o.clock_naive===true`, `""` otherwise. Used inline in
the observers-list `name` cell and in the row-detail slide-over. Tooltip
explains magnitude + direction + count + fix.
- `window.ObserverDetailNaiveBanner.render(obs)` — yellow alert banner
at the top of the observer-detail page with the skew magnitude,
last-event timestamp, and the actionable fix ("Set host clock to UTC, OR
emit Z-suffixed/offset-aware timestamps from the observer script").
## TDD trail
- `5ddd5b42` red: backend `cmd/server/observer_naive_clock_1478_test.go`
(3 tests asserting JSON fields + 24h decay) + frontend
`test-observer-naive-clock-1478.js` (8 jsdom-style tests asserting
helpers exist and render correctly). Both failed on master with
field-missing / export-missing assertions.
- `4ecc79c8` green backend: schema + Observer / GetObservers /
ObserverResp / handler decay.
- `2137ab81` green frontend: chip + banner helpers and call sites.
## Tests
- `cd cmd/server && go test ./...` → all green (full suite, 46s)
- `cd cmd/ingestor && go test ./...` → all green (full suite, 98s)
- `node test-observer-naive-clock-1478.js` → 8/8 pass
- `node test-frontend-helpers.js` → unchanged from master (pre-existing
failures only)
## Acceptance (issue #1478)
- ✅ Observer running with `python datetime.now().isoformat()` (naive,
off by N hours) → `clock_naive=true` after the next clamp → UI shows ⚠️
chip + banner.
- ✅ Observer with `datetime.now(timezone.utc).isoformat()` (Z-suffixed)
→ never clamped → never flagged.
- ✅ Observer that fixed its clock → `clock_naive` returns to `false` 24h
after the last clamp event (read-side decay).
Closes#1478.
---------
Co-authored-by: openclaw <bot@openclaw.local>
Red commit: 702d82eb5e (CI: see Actions
tab for fix/issue-1366)
## What
Channel view emits the max observation timestamp (`tx.LatestSeen`)
instead of the analyzer's first-observation time (`tx.FirstSeen`) as the
rendered `timestamp` field. A new `first_seen` field is exposed
alongside for debug surfaces. `sender_timestamp` continues to be
returned in the JSON response but is intentionally NOT used as the
rendered time (client clocks are unreliable).
## Root cause
Two parallel call sites both emitted the wrong field:
- `cmd/server/store.go` — `GetChannelMessages` (~line 4807): set
`entry.Data["timestamp"] = strOrNil(tx.FirstSeen)` for every new dedup
entry. `tx.FirstSeen` is the analyzer's first-ever observation time of a
`transmissions.hash` row; for heartbeat-style packets (e.g. `BlorkoBot
🤖` posting the same status line periodically), the hash is stable, so
FirstSeen stays pinned at the very first observation while the message
keeps retransmitting hours later. Operator sees "old" message timestamps
for live messages.
- `cmd/server/db.go` — `GetChannelMessages` (~line 1757): same problem
against the SQLite-backed query path. Used `nullStr(fs)` (where `fs` is
`t.first_seen`) for the `timestamp` field.
### Repro from staging
Same packet, same hash `aba4f0493249de57`, sender `BlorkoBot 🤖`:
- `/api/channels/%23test/messages` → `timestamp: "2026-05-25T15:53:20Z"`
(FirstSeen, 7h+ in the past)
- `/api/packets?hash=aba4f0493249de57` → `first_seen:
"2026-05-25T22:53:19Z"` (latest obs), `observation_count: 84`
The packets view used max-obs correctly; the channels view did not. 7h
gap matches operator screenshot.
## TDD red → green
Red: `cmd/server/channels_message_order_1366_test.go` — three tests:
- `TestChannelMessages_TimestampUsesLatestSeen`: seeds a CHAN tx with
observations 7h apart, asserts returned `timestamp` ≈ latest observation
epoch (±1s). Fails under FirstSeen with Δ=−25200s.
- `TestChannelMessages_TimestampNotSenderTimestamp`: seeds a CHAN tx
whose decoded `sender_timestamp` is year-2000 (bad RTC). Asserts the
rendered `timestamp` parses to current year — guards against the
tempting "just use sender_timestamp" alt-fix that would let bad client
clocks corrupt the view.
- `TestChannelMessages_TimestampIsUTCZ`: asserts the emitted string is
unambiguously UTC (suffix `Z` or `+00:00`) so browsers don't apply a
local-zone shift.
Green commit changes:
- `store.go`: emit `tx.LatestSeen` (with FirstSeen fallback if no obs);
add `first_seen` field.
- `db.go`: join `o.timestamp` per-observation, track max epoch per tx,
emit RFC3339 UTC at the end; add `first_seen` field.
`sender_timestamp` remains in the response — unchanged shape, frontend
never read it for the rendered time (verified: only `msg.timestamp` is
consumed in `public/channels.js:1902`).
## Manual verification (post-merge)
1. Deploy to staging.
2. Curl `/api/channels/%23test/messages?limit=5` and
`/api/packets?hash=<recent>`. The channel `timestamp` field MUST equal
the packets `first_seen` (max obs) for the same hash, NOT lag it.
3. Send a fresh GRP_TXT via a MeshCore client into a watched channel.
Within 15s, refresh the Channels view at `/channels`. The new message
MUST render at the bottom with the correct (current) time.
## Why not `sender_timestamp`?
It's a per-client field, decoded from the payload. Many MeshCore
firmware builds run without RTC/NTP/GPS and report bogus values.
Trusting it for display would propagate bad client clocks into the
analyzer UI — the analyzer is the source of truth for UTC, not the
client.
Fixes#1366
---------
Co-authored-by: CoreScope Bot <bot@corescope>
Co-authored-by: bot <bot@kpa-clawbot.dev>
Co-authored-by: openclaw-bot <bot@openclaw.local>
@
## What this PR does
Implements region-scoped transport-route packet tracking with two
sub-features:
### Feature 1 — Scope statistics (`scope_name`)
- At ingest, transport-route packets (route_type 0/3) with Code1 !=
`0000` are HMAC-matched against configured `hashRegions` keys (mirroring
the `hashChannels` pattern). Matched region name (or `""` for unknown)
stored in new `transmissions.scope_name` column via migration
`scope_name_v1`.
- New `GET /api/scope-stats?window=` endpoint (1h/24h/7d, 30s
server-side TTL) returning transport totals, scoped/unscoped counts,
per-region breakdown, and time-series.
- New **Scopes** tab in Analytics with summary cards, per-region table,
and two-line SVG chart. Auto-refreshes every 60s.
### Feature 2 — Node default scope (`default_scope`)
- Per-node `default_scope` column on `nodes`/`inactive_nodes` (migration
`nodes_default_scope_v1`) tracks the most recently matched region for
each node, derived from transport-scoped ADVERT packets.
- `GET /api/nodes` response includes `default_scope` field when column
is present.
- Node detail panel displays the default scope badge.
- Async startup backfill (`BackfillDefaultScopeAsync`) populates the
column for nodes with pre-existing ADVERT data.
### Config
Add `hashRegions` to `config.json` (see `config.example.json`). One
entry per region name (with or without leading `#`).
@
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
Co-authored-by: openclaw-bot <bot@openclaw.local>
## Problem
`indexByNode()` was called during `Load()` immediately when each
`StoreTx` was created — before observations were appended and before
`pickBestObservation()` set `tx.ResolvedPath`. The resolved_path
indexing branch added in #708 was effectively dead code on every server
restart.
**Symptom:** After any restart, `byNode[relay_pubkey]` was empty for
relay-only nodes even when `resolved_path` was correctly persisted in
the DB. Analytics showed `totalPackets = 0` for repeater nodes despite
active relay traffic.
## Fix
Call `s.indexByNode(tx)` again in the post-load loop after
`pickBestObservation()`, where `ResolvedPath` is populated. Same fix
applied to `backfillResolvedPathsAsync()`, which also called
`pickBestObservation()` without re-indexing afterward.
The dedup in `nodeHashes` prevents double-counting: pubkeys already
indexed from decoded JSON fields are skipped; only the relay hop pubkeys
from `resolved_path` are new additions.
## Test
`TestLoadIndexesRelayHopsFromResolvedPath` — inserts a packet with
`resolved_path` containing a relay pubkey that does not appear in
`decoded_json`, calls `Load()`, and verifies `byNode[relay_pubkey]` is
populated.
## Related
Closes#692 (together with #707, #708, #711 already merged)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes#1143.
## Summary
Replaces the structurally unsound `decoded_json LIKE '%pubkey%'` (and
`OR LIKE '%name%'`) attribution path with an exact-match lookup on a
dedicated, indexed `transmissions.from_pubkey` column.
This closes both holes documented in #1143:
- **Hole 1** — same-name false positives via `OR LIKE '%name%'`
- **Hole 2a** — adversarial spoofing: a malicious node names itself with
another node's pubkey and gets attributed to the victim
- **Hole 2b** — accidental false positive when any free-text field (path
elements, channel names, message bodies) contains a 64-char hex
substring matching a real pubkey
- **Perf** — query now uses an index instead of a full-table scan
against `LIKE '%substring%'`
## TDD
Two-commit history shows red-then-green:
| Commit | Status | Purpose |
|---|---|---|
| `7f0f08e` | RED — tests assertion-fail on master behaviour |
Adversarial fixtures + spec |
| `59327db` | GREEN — schema + ingestor + server + migration |
Implementation |
The red commit's test schema includes the new column so the file
compiles, but the production code still uses LIKE — the assertions fail
because the malicious / same-name / free-text rows are returned. The
green commit changes the query plus adds the migration/ingest path.
## Changes
### Schema
- new column `transmissions.from_pubkey TEXT`
- new index `idx_transmissions_from_pubkey`
### Ingestor (`cmd/ingestor/`)
- `PacketData.FromPubkey` populated from decoded ADVERT `pubKey` at
write time. Cheap — already parsing `decoded_json`. Non-ADVERTs stay
NULL.
- `stmtInsertTransmission` writes the column.
- Migration `from_pubkey_v1` ALTERs legacy DBs to add the column +
index.
- Bonus: rewrote the recipe in the gated one-shot
`advert_count_unique_v1` migration to use `from_pubkey` (already marked
done on existing DBs; kept correct for fresh installs).
### Server (`cmd/server/`)
- `ensureFromPubkeyColumn` mirrors the ingestor migration so the server
can boot against a DB the ingestor has never touched (e2e fixture, fresh
installs).
- `backfillFromPubkeyAsync` runs **after** HTTP starts. Scans `WHERE
from_pubkey IS NULL AND payload_type = 4` in 5000-row chunks with a
100ms yield between chunks. Cannot block boot even on prod-sized DBs
(100K+ transmissions). Queries handle NULL gracefully (return empty for
that pubkey, same as today's unknown-pubkey path).
- All in-scope LIKE call sites switched to exact match:
| Site | Before | After |
|---|---|---|
| `buildPacketWhere` (was db.go:582) | `decoded_json LIKE '%pubkey%'` |
`from_pubkey = ?` |
| `buildTransmissionWhere` (was db.go:626) | `t.decoded_json LIKE
'%pubkey%'` | `t.from_pubkey = ?` |
| `GetRecentTransmissionsForNode` (was db.go:910) | `LIKE '%pubkey%' OR
LIKE '%name%'` | `t.from_pubkey = ?` |
| `QueryMultiNodePackets` (was db.go:1785) | `decoded_json LIKE
'%pubkey%' OR ...` | `t.from_pubkey IN (?, ?, ...)` |
| `advert_count_unique_v1` (was ingestor/db.go:257) | `decoded_json LIKE
'%' \|\| nodes.public_key \|\| '%'` | `t.from_pubkey = nodes.public_key`
|
`GetRecentTransmissionsForNode` signature simplifies: the `name`
parameter is gone (it was only ever used for the legacy `OR LIKE
'%name%'` fallback). Sole caller in `routes.go:1243` updated.
### Tests
- `cmd/server/from_pubkey_attribution_test.go` — adversarial fixtures +
Hole 1/2a/2b/QueryMultiNodePackets exact-match assertions, EXPLAIN QUERY
PLAN index check, migration backfill correctness.
- `cmd/ingestor/from_pubkey_test.go` — write-time correctness
(BuildPacketData populates FromPubkey for ADVERT only;
InsertTransmission persists it; non-ADVERTs stay NULL).
- Existing test schemas (server v2, server v3, coverage) get the new
column **plus a SQLite trigger** that auto-populates `from_pubkey` from
`decoded_json` on ADVERT inserts. This means existing fixtures (which
only seed `decoded_json`) keep attributing correctly without per-test
edits.
- `seedTestData`'s ADVERTs explicitly set `from_pubkey`.
## Performance — index is used
```
$ EXPLAIN QUERY PLAN SELECT id FROM transmissions WHERE from_pubkey = ?
SEARCH transmissions USING INDEX idx_transmissions_from_pubkey (from_pubkey=?)
```
Asserted in `TestFromPubkeyIndexUsed`.
## Migration approach
- **Sync at boot**: `ALTER TABLE transmissions ADD COLUMN from_pubkey
TEXT` is a metadata-only operation in SQLite — microseconds regardless
of table size. `CREATE INDEX IF NOT EXISTS
idx_transmissions_from_pubkey` is **not** metadata-only: it scans the
table once. Empirically a few hundred ms on a 100K-row table; expect a
few seconds on a 10M-row table (one-time cost, blocking boot during that
window). Subsequent boots no-op via `IF NOT EXISTS`. If this boot delay
becomes an operational concern at prod scale we can defer the `CREATE
INDEX` to a goroutine — for now a few-second one-time delay is
acceptable.
- **Async**: row-level backfill of legacy NULL ADVERTs (chunked 5000 /
100ms yield). On a 100K-ADVERT prod DB, this completes in seconds in the
background; HTTP is fully available throughout.
- **Safety**: queries handle NULL gracefully — a node whose ADVERTs
haven't backfilled yet returns empty, identical to today's behaviour for
unknown pubkeys. No half-state regression.
## Out of scope (intentionally)
The free-text `LIKE` paths the issue explicitly leaves alone (e.g.
user-typed packet search) are untouched. Only the pubkey-attribution
sites get the column treatment.
## Cycle-3 review fixes
| Finding | Status | Commit |
|---|---|---|
| **M1c** — async-contract test was tautological (test's own `go`, not
production's) | Fixed | `23ace71` (red) → `a05b50c` (green) |
| **m1c** — package-global atomic resets unsafe under `t.Parallel()` |
Fixed (`// DO NOT t.Parallel` comment + `Reset()` helper) | rolled into
`23ace71` / `241ec69` |
| **m2c** — `/api/healthz` read 3 atomics non-atomically (torn snapshot)
| Fixed (single RWMutex-guarded snapshot + race test) | `241ec69` |
| **n3c.m1** — vestigial OR-scaffolding in `QueryMultiNodePackets` |
Fixed (cleanup) | `5a53ceb` |
| **n3c.m2** — verify PR body language about `ALTER` vs `CREATE INDEX` |
Verified accurate (already corrected in cycle 2) | (no change) |
| **n3c.m3** — `json.Unmarshal` per row in backfill → could use SQL
`json_extract` | **Deferred as known followup** — pure perf optimization
(current per-row Unmarshal is correct, just slower); SQL rewrite would
unwind the chunked-yield architecture and is non-trivial. Acceptable for
one-time backfill at boot on legacy DBs. |
### M1c implementation detail
`startFromPubkeyBackfill(dbPath, chunkSize, yieldDuration)` is now the
single production entry point used by `main.go`. It internally does `go
backfillFromPubkeyAsync(...)`. The test calls `startFromPubkeyBackfill`
(no `go` prefix) and asserts the dispatch returns within 50ms — so if
anyone removes the `go` keyword inside the wrapper, the test fails.
**Manually verified**: removing the `go` keyword causes
`TestBackfillFromPubkey_DoesNotBlockBoot` to fail with "backfill
dispatch took ~1s (>50ms): not async — would block boot."
### m2c implementation detail
`fromPubkeyBackfillTotal/Processed/Done` are now plain `int64`/`bool`
package globals guarded by a single `sync.RWMutex`.
`fromPubkeyBackfillSnapshot()` returns all three under one RLock.
`TestHealthzFromPubkeyBackfillConsistentSnapshot` races a writer
(lock-step total/processed updates with periodic done flips) against 8
readers hammering `/api/healthz`, asserting `processed<=total` and
`(done => processed==total)` on every response. Verified the test
catches torn reads (manually injected a 3-RLock implementation; test
failed within milliseconds with "processed>total" and "done=true but
processed!=total" errors).
---------
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: openclaw-bot <bot@openclaw.dev>
## Summary
**Partial fix for #730 (M1 only — M2 frontend and M3 alerting
deferred).**
Today the ingestor **silently drops** ADVERTs whose GPS lies outside the
configured `geo_filter` polygon. That's the wrong default for an
analytics tool — operators get zero visibility into bridged or leaked
meshes.
This PR makes the new default **flag, don't drop**: foreign adverts are
stored, the node row is tagged `foreign_advert=1`, and the API surfaces
`"foreign": true` so dashboards / map overlays can be built on top.
## Behavior
| Mode | What happens to an ADVERT outside `geo_filter` |
|---|---|
| (default) flag | Stored, marked `foreign_advert=1`, exposed via API |
| drop (legacy) | Silently dropped (preserves old behavior for ops who
want it) |
## What's done (M1 — Backend)
- ingestor stores foreign adverts instead of dropping
- `nodes.foreign_advert` column added (migration)
- `/api/nodes` and `/api/nodes/{pk}` expose `foreign: true` field
- Config: `geofilter.action: "flag"|"drop"` (default `flag`)
- Tests + config docs
## What's NOT done (deferred to M2 + M3)
- **M2 — Frontend:** Map overlay showing foreign adverts as distinct
markers, foreign-advert filter on packets/nodes pages, dedicated
foreign-advert dashboard
- **M3 — Alerting:** Time-series detection of bridging events, alert
when foreign advert rate spikes, identify bridge entry-point nodes
Issue #730 remains open for M2 and M3.
---------
Co-authored-by: corescope-bot <bot@corescope>
Rebased version of #968 (which was itself a rebase of #905) — resolves
merge conflict with #906 (clock-skew UI) that landed on master.
## Conflict resolution
**`public/observers.js`** — master (#906) added "Clock Offset" column to
observer table; #968 split "Last Seen" into "Last Status" + "Last
Packet" columns. Combined both: the table now has Status | Name | Region
| Last Status | Last Packet | Packets | Packets/Hour | Clock Offset |
Uptime.
## What this PR adds (unchanged from #968/#905)
- `last_packet_at` column in observers DB table
- Separate "Last Status Update" and "Last Packet Observation" display in
observers list and detail page
- Server-side migration to add the column automatically
- Backfill heuristic for existing data
- Tests for ingestor and server
## Verification
- All Go tests pass (`cmd/server`, `cmd/ingestor`)
- Frontend tests pass (`test-packets.js`, `test-hash-color.js`)
- Built server, hit `/api/observers` — `last_packet_at` field present in
JSON
- Observer table header has all 9 columns including both Last Packet and
Clock Offset
## Prior PRs
- #905 — original (conflicts with master)
- #968 — first rebase (conflicts after #906 landed)
- This PR — second rebase, resolves#906 conflict
Supersedes #968. Closes#905.
---------
Co-authored-by: you <you@example.com>
## Bug
`/api/observers` returned soft-deleted (inactive=1) observers. Operators
saw stale observers in the UI even after the auto-prune marked them
inactive on schedule. Reproduced on staging: 14 observers older than 14
days returned by the API; all of them had `inactive=1` in the DB.
## Root cause
`DB.GetObservers()` (`cmd/server/db.go:974`) ran `SELECT ... FROM
observers ORDER BY last_seen DESC` with no WHERE filter. The
`RemoveStaleObservers` path correctly soft-deletes by setting
`inactive=1`, but the read path didn't honor it.
`statsRow` (`cmd/server/db.go:234`) had the same bug — `totalObservers`
count included soft-deleted rows.
## Fix
Add `WHERE inactive IS NULL OR inactive = 0` to both:
```go
// GetObservers
"SELECT ... FROM observers WHERE inactive IS NULL OR inactive = 0 ORDER BY last_seen DESC"
// statsRow.TotalObservers
"SELECT COUNT(*) FROM observers WHERE inactive IS NULL OR inactive = 0"
```
`NULL` check preserves backward compatibility with rows from before the
`inactive` migration.
## Tests
Added regression `TestGetObservers_ExcludesInactive`:
- Seed two observers, mark one inactive, assert `GetObservers()` returns
only the other.
- **Anti-tautology gate verified**: reverting the WHERE clause causes
the test to fail with `expected 1 observer, got 2` and `inactive
observer obs2 should be excluded`.
`go test ./...` passes (19.6s).
## Out of scope
- `GetObserverByID` lookup at line 1009 still returns inactive observers
— this is intentional, so an old deep link to `/observers/<id>` shows
"inactive" rather than 404.
- Frontend may also have its own caching layer; this fix is server-side
only.
---------
Co-authored-by: Kpa-clawbot <bot@example.invalid>
Co-authored-by: you <you@example.com>
Co-authored-by: KpaBap <kpabap@gmail.com>
## Problem
Each MeshCore observer receives a physically distinct over-the-air byte
sequence for the same transmission (different path bytes, flags/hops
remaining). The `observations` table stored only `path_json` per
observer — all observations pointed at one `transmissions.raw_hex`. This
prevented the hex pane from updating when switching observations in the
packet detail view.
## Changes
| Layer | Change |
|-------|--------|
| **Schema** | `ALTER TABLE observations ADD COLUMN raw_hex TEXT`
(nullable). Migration: `observations_raw_hex_v1` |
| **Ingestor** | `stmtInsertObservation` now stores per-observer
`raw_hex` from MQTT payload |
| **View** | `packets_v` uses `COALESCE(o.raw_hex, t.raw_hex)` —
backward compatible with NULL historical rows |
| **Server** | `enrichObs` prefers `obs.RawHex` when non-empty, falls
back to `tx.RawHex` |
| **Frontend** | No changes — `effectivePkt.raw_hex` already flows
through `renderDetail` |
## Tests
- **Ingestor**: `TestPerObservationRawHex` — two MQTT packets for same
hash from different observers → both stored with distinct raw_hex
- **Server**: `TestPerObservationRawHexEnrich` — enrichObs returns
per-obs raw_hex when present, tx fallback when NULL
- **E2E**: Playwright assertion in `test-e2e-playwright.js` for hex pane
update on observation switch
E2E assertion added: `test-e2e-playwright.js:1794`
## Scope
- Historical observations: raw_hex stays NULL, UI falls back to
transmission raw_hex silently
- No backfill, no path_json reconstruction, no frontend changes
Closes#881
---------
Co-authored-by: you <you@example.com>
## Problem
Channel API endpoints scan entire DB — 2.4s for channel list, 30s for
messages.
## Fix
- Added `channel_hash` column to transmissions (populated on ingest,
backfilled on startup)
- `GetChannels()` rewrites to GROUP BY channel_hash (one row per channel
vs scanning every packet)
- `GetChannelMessages()` filters by channel_hash at SQL level with
proper LIMIT/OFFSET
- 60s cache for channel list
- Index: `idx_tx_channel_hash` for fast lookups
Expected: 2.4s → <100ms for list, 30s → <500ms for messages.
Fixes#762
---------
Co-authored-by: you <you@example.com>
## Summary
Switches channel API endpoints to query SQLite instead of the in-memory
packet store, giving users access to the full message history.
Implements #725 (M1 only — DB-backed channel messages). Does NOT close
#725 — M2-M5 (custom channels, PSK, persistence, retroactive decryption)
remain.
## Problem
Channel endpoints (`/api/channels`, `/api/channels/{hash}/messages`)
preferred the in-memory packet store when available. The store is
bounded by `packetStore.maxMemoryMB` — typically showing only recent
messages. The SQLite database has the complete history (weeks/months of
channel messages) but was only used as a fallback when the store was nil
(never in production).
## Fix
Reversed the preference order: DB first, in-memory store fallback.
Region filtering added to the DB path.
Co-authored-by: you <you@example.com>
## Problem
The "Paths Through This Node" API endpoint (`/api/nodes/{pubkey}/paths`)
returns unrelated packets when two nodes share a hex prefix. For
example, querying paths for "Kpa Roof Solar" (`c0dedad4...`) returns 316
packets that actually belong to "C0ffee SF" (`C0FFEEC7...`) because both
share the `c0` prefix in the `byPathHop` index.
Fixes#655
## Root Cause
`handleNodePaths()` in `routes.go` collects candidates from the
`byPathHop` index using 2-char and 4-char hex prefixes for speed, but
never verifies that the target node actually appears in each candidate's
resolved path. The broad index lookup is intentional, but the
**post-filter was missing**.
## Fix
Added `nodeInResolvedPath()` helper in `store.go` that checks whether a
transmission's `resolved_path` (from the neighbor affinity graph via
`resolveWithContext`) contains the target node's full pubkey. The
filter:
- **Includes** packets where `resolved_path` contains the target node's
full pubkey
- **Excludes** packets where `resolved_path` resolved to a different
node (prefix collision)
- **Excludes** packets where `resolved_path` is nil/empty (ambiguous —
avoids false positives)
The check examines both the best observation's resolved_path
(`tx.ResolvedPath`) and all individual observations, so packets are
included if *any* observation resolved the target.
## Tests
- `TestNodeInResolvedPath` — unit test for the helper with 5 cases
(match, different node, nil, all-nil elements, match in observation
only)
- `TestNodePathsPrefixCollisionFilter` — integration test: two nodes
sharing `aa` prefix, verifies the collision packet is excluded from one
and included for the other
- Updated test DB schema to include `resolved_path` column and seed data
with resolved pubkeys
- All existing tests pass (165 additions, 8 modifications)
## Performance
No impact on hot paths. The filter runs once per API call on the
already-collected candidate set (typically small). `nodeInResolvedPath`
is O(observations × hops) per candidate — negligible since observations
per transmission are typically 1–5.
---------
Co-authored-by: you <you@example.com>
## M2: Airtime + Channel Quality + Battery Charts
Implements M2 of #600 — server-side delta computation and three new
charts in the RF Health detail view.
### Backend Changes
**Delta computation** for cumulative counters (`tx_air_secs`,
`rx_air_secs`, `recv_errors`):
- Computes per-interval deltas between consecutive samples
- **Reboot handling:** detects counter reset (current < previous), skips
that delta, records reboot timestamp
- **Gap handling:** if time between samples > 2× interval, inserts null
(no interpolation)
- Returns `tx_airtime_pct` and `rx_airtime_pct` as percentages
(delta_secs / interval_secs × 100)
- Returns `recv_error_rate` as delta_errors / (delta_recv +
delta_errors) × 100
**`resolution` query param** on `/api/observers/{id}/metrics`:
- `5m` (default) — raw samples
- `1h` — hourly aggregates (GROUP BY hour with AVG/MAX)
- `1d` — daily aggregates
**Schema additions:**
- `packets_sent` and `packets_recv` columns added to `observer_metrics`
(migration)
- Ingestor parses these fields from MQTT stats messages
**API response** now includes:
- `tx_airtime_pct`, `rx_airtime_pct`, `recv_error_rate` (computed
deltas)
- `reboots` array with timestamps of detected reboots
- `is_reboot_sample` flag on affected samples
### Frontend Changes
Three new charts in the RF Health detail view, stacked vertically below
noise floor:
1. **Airtime chart** — TX (red) + RX (blue) as separate SVG lines,
Y-axis 0-100%, direct labels at endpoints
2. **Error Rate chart** — `recv_error_rate` line, shown only when data
exists
3. **Battery chart** — voltage line with 3.3V low reference, shown only
when battery_mv > 0
All charts:
- Share X-axis and time range (aligned vertically)
- Reboot markers as vertical hairlines spanning all charts
- Direct labels on data (no legends)
- Resolution auto-selected: `1h` for 7d/30d ranges
- Charts hidden when no data exists
### Tests
- `TestComputeDeltas`: normal deltas, reboot detection, gap detection
- `TestGetObserverMetricsResolution`: 5m/1h/1d downsampling verification
- Updated `TestGetObserverMetrics` for new API signature
---------
Co-authored-by: you <you@example.com>
## RF Health Dashboard — M1: Observer Metrics Storage, API & Small
Multiples Grid
Implements M1 of #600.
### What this does
Adds a complete RF health monitoring pipeline: MQTT stats ingestion →
SQLite storage → REST API → interactive dashboard with small multiples
grid.
### Backend Changes
**Ingestor (`cmd/ingestor/`)**
- New `observer_metrics` table via migration system (`_migrations`
pattern)
- Parse `tx_air_secs`, `rx_air_secs`, `recv_errors` from MQTT status
messages (same pattern as existing `noise_floor` and `battery_mv`)
- `INSERT OR REPLACE` with timestamps rounded to nearest 5-min interval
boundary (using ingestor wall clock, not observer timestamps)
- Missing fields stored as NULLs — partial data is always better than no
data
- Configurable retention pruning: `retention.metricsDays` (default 30),
runs on startup + every 24h
**Server (`cmd/server/`)**
- `GET /api/observers/{id}/metrics?since=...&until=...` — per-observer
time-series data
- `GET /api/observers/metrics/summary?window=24h` — fleet summary with
current NF, avg/max NF, sample count
- `parseWindowDuration()` supports `1h`, `24h`, `3d`, `7d`, `30d` etc.
- Server-side metrics retention pruning (same config, staggered 2min
after packet prune)
### Frontend Changes
**RF Health tab (`public/analytics.js`, `public/style.css`)**
- Small multiples grid showing all observers simultaneously — anomalies
pop out visually
- Per-observer cell: name, current NF value, battery voltage, sparkline,
avg/max stats
- NF status coloring: warning (amber) at ≥-100 dBm, critical (red) at
≥-85 dBm — text color only, no background fills
- Click any cell → expanded detail view with full noise floor line chart
- Reference lines with direct text labels (`-100 warning`, `-85
critical`) — not color bands
- Min/max points labeled directly on the chart
- Time range selector: preset buttons (1h/3h/6h/12h/24h/3d/7d/30d) +
custom from/to datetime picker
- Deep linking: `#/analytics?tab=rf-health&observer=...&range=...`
- All charts use SVG, matching existing analytics.js patterns
- Responsive: 3-4 columns on desktop, 1 on mobile
### Design Decisions (from spec)
- Labels directly on data, not in legends
- Reference lines with text labels, not color bands
- Small multiples grid, not card+accordion (Tufte: instant visual fleet
comparison)
- Ingestor wall clock for all timestamps (observer clocks may drift)
### Tests Added
**Ingestor tests:**
- `TestRoundToInterval` — 5 cases for rounding to 5-min boundaries
- `TestInsertMetrics` — basic insertion with all fields
- `TestInsertMetricsIdempotent` — INSERT OR REPLACE deduplication
- `TestInsertMetricsNullFields` — partial data with NULLs
- `TestPruneOldMetrics` — retention pruning
- `TestExtractObserverMetaNewFields` — parsing tx_air_secs, rx_air_secs,
recv_errors
**Server tests:**
- `TestGetObserverMetrics` — time-series query with since/until filters,
NULL handling
- `TestGetMetricsSummary` — fleet summary aggregation
- `TestObserverMetricsAPIEndpoints` — DB query verification
- `TestMetricsAPIEndpoints` — HTTP endpoint response shape
- `TestParseWindowDuration` — duration parsing for h/d formats
### Test Results
```
cd cmd/ingestor && go test ./... → PASS (26s)
cd cmd/server && go test ./... → PASS (5s)
```
### What's NOT in this PR (deferred to M2+)
- Server-side delta computation for cumulative counters
- Airtime charts (TX/RX percentage lines)
- Channel quality chart (recv_error_rate)
- Battery voltage chart
- Reboot detection and chart annotations
- Resolution downsampling (1h, 1d aggregates)
- Pattern detection / automated diagnosis
---------
Co-authored-by: you <you@example.com>
## Summary
- `db.GetNodes` accepted a `region` param from the HTTP handler but
never used it — every region-filter selection was silently ignored and
all nodes were always returned
- Added a subquery filtering `nodes.public_key` against ADVERT
transmissions (payload_type=4) observed by observers with matching IATA
codes
- Handles both v2 (`observer_id TEXT`) and v3 (`observer_idx INT`)
schemas
## Test plan
- [x] 4 new subtests added to `TestGetNodesFiltering`: SJC (1 node), SFO
(1 node), SJC,SFO multi (1 node deduped), AMS unknown (0 nodes)
- [x] All existing Go tests still pass
- [x] Deploy to staging, open `/nodes`, select a region in the filter
bar — only nodes observed by observers in that region should appear
Closes#496🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
Pass region through channel message routes, apply DB/store filtering, normalize IATA at read and write boundaries, and add regression coverage for routes/server/ingestor.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change healthThresholds config from milliseconds to hours for readability.
Config keys: infraDegradedHours, infraSilentHours, nodeDegradedHours, nodeSilentHours.
Defaults: infra degraded 24h, silent 72h; node degraded 1h, silent 24h.
- Config stored in hours, converted to ms at comparison time
- /api/config/client sends ms to frontend (backward compatible)
- Frontend tooltips use dynamic thresholds instead of hardcoded strings
- Added healthThresholds section to config.example.json
- Updated Go and Node.js servers, tests
SQLite :memory: databases create separate databases per connection.
When the connection pool opens multiple connections (e.g. poller goroutine
vs main test goroutine), tables created on one connection are invisible
to others. Setting MaxOpenConns(1) ensures all queries use the same
in-memory database, fixing TestPollerBroadcastsMultipleObservations.
* refactor: remove all packets_v SQL fallbacks — store handles all queries
Remove DB fallback paths from all route handlers. The in-memory
PacketStore now handles all packet/node/analytics queries. Handlers
return empty results or 404 when no store is available instead of
falling back to direct DB queries.
- Remove else-DB branches from handlePacketDetail, handleNodeHealth,
handleNodeAnalytics, handleBulkHealth, handlePacketTimestamps, etc.
- Remove unused DB methods (GetPacketByHash, GetTransmissionByID,
GetPacketByID, GetObservationsForHash, GetTimestamps, GetNodeHealth,
GetNodeAnalytics, GetBulkHealth, etc.)
- Remove packets_v VIEW creation from schema
- Update tests for new behavior (no-store returns 404/empty, not 500)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR #220 review comments
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: KpaBap <kpabap@gmail.com>
dborup