Extends the #1812 per-node relay hop-count work with a network-wide view:
GET /api/analytics/hop-depth answers (1) whether scoped traffic actually
travels fewer hops than unscoped before hitting a repeater's flood.max cap
(Scopes tab Overview: new "Flood Containment" comparison), and (2) which
repeaters relay unscoped traffic that already traveled far vs merely
locally (Foreign Traffic tab: min/median/max hop columns joined onto the
existing unscoped-relay table by public key).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New GET /api/nodes/{pubkey}/hop_analytics?days= returns, for each recent
transmission that passed through this node as a relay, the node's own
0-based index within the packet's resolved path -- the value MeshCore
firmware compares against flood_max/flood_max_advert/flood_max_unscoped
in allowPacketForward (Packet.cpp path_len = header & 63; MyMesh.cpp
getPathHashCount() >= flood_max). Deliberately distinct from the
existing hopDistribution field on /analytics, which measures path
length to whichever observer reported the packet -- a different,
unrelated number, confirmed via the upstream issue's multi-round
firmware-source discussion.
Candidate-finding reuses the same byPathHop-index + resolved_path
confirmation approach as handleNodePaths (routes.go), written fresh
rather than refactored out of that function to avoid any regression
risk in its older, correctness-critical prefix-collision handling
(#929, #1197, #1278, #1352). Only transmissions with a canonical
resolved_path contribute a data point -- no resolved_path means no
reliable hop index, so it's skipped rather than guessed.
Frontend: new "Relay Hop-Count" card on the node analytics page --
histogram (Chart.js) + a hand-drawn boxplot canvas above it sharing the
same x-axis (no Chart.js boxplot plugin is loaded), filterable by chip
matching the firmware knob names (flood default, flood_advert,
flood_unscoped, direct).
The old single "Mesh live area" checkbox actually showed the
homeArea-linked geo_filter boundary, which was confusing paired with
the already-existing "Area: X" dropdown (AreaFilter) that filters
which nodes show on the map -- picking an area there drew nothing.
Now two independent checkboxes on both Map and Live:
- "Geo filter boundary" -- unchanged behavior, the geo_filter/homeArea
polygon (with buffer zone if configured).
- "Selected area outline" -- new, draws whichever area is currently
picked via the Area dropdown (falls back to a box when the area has
no polygon), checked by default whenever an area is selected.
Same "Show all N / Show fewer" pattern as the Wardriving tab's
Sessions/Entry Points sections -- both lists on Foreign Traffic can
grow long on a busy network, dwarfing the rest of the tab.
?hasScope=true|false keeps only nodes that have/haven't ever
transported a region-scoped packet; ?hashRegion=eu,be (comma-separated,
leading # optional) keeps only nodes that have transported at least
one of the given regions. Both combine with each other (AND) and with
the existing ?role= filter, matching the suggested API shape from
https://github.com/Kpa-clawbot/CoreScope/issues/1862.
Implemented as two new predicates in the existing nodeListPostFilters
(same post-SQL-LIMIT compensation-loop machinery geo_filter/blacklist/
area already use), backed by the same bulk relay-info map
(GetRepeaterRelayInfoMap) the Scopes tab's "Repeaters Never Relaying
Any Scope" section reads -- fetched once per request, not per DB page.
AreaScopeAdoption.Matching entries now carry MatchedScopes (which of
the area's linked regionScopes each node actually matched, via
default_scope or by relaying it) instead of just a bare name/key ref.
A node can match more than one when an area links several scopes and
the node uses/relays more than one of them.
Frontend: an area linking more than one scope (e.g. Europa's "eu" and
"europe") now splits its Supporting list into one sub-group per scope
instead of a single flat list, so it's visible which nodes support
which specific scope.
AreaEntry.RegionScope (single string) -> RegionScopes ([]string), so a
broad umbrella area (e.g. Europa) can link more than one scope name
(e.g. both "eu" and "europe") -- a node matching any one of them now
counts as supporting the area in computeScopeAdoptionByArea, the
"Scope Adoption by Area" section, and the regionScope->label lookup
used to annotate region codes elsewhere on the Scopes tab.
"Nodes Without a Default Scope" and "Repeaters Never Relaying Any
Scope" now have the same All/Domestic/Foreign filter as the Nodes tab,
classified live from each node's lat/lon against window.MC_GEO_FILTER
(same nodePassesGeoFilter helper, not the node's stale one-way
`foreign` DB flag -- see renderForeignTrafficTab's isForeignNode).
Refactored the never-relay section's inline render block into its own
renderNeverRelaySection, mirroring renderNoScopeSection, so its filter
can re-render reactively without a re-fetch.
IngestNewFromDB/IngestNewObservations call resolveEntryPointArea while
already holding s.mu.Lock() (write lock). resolveEntryPointArea then
tried to s.mu.RLock() the same non-reentrant mutex, deadlocking the
goroutine permanently and blocking every other s.mu waiter -- this
stalled LoadChunked mid-startup and hung /api/stats on stg.
getCachedNodesAndPM() guards itself with its own cacheMu and never
touches s.mu, so the RLock/RUnlock wrapping was unnecessary.
resolveEntryPointArea moves onto PacketStore (which already owns the
config and prefix map) so IngestNewFromDB/IngestNewObservations can
resolve area at broadcast time, not just on the next REST reload.
Previously a freshly-sent message only showed "Area:" after a page
refresh since the WS live-append path never computed it.
Keeping the scope tag plain ("Scope: #dk") -- the path[0]-resolved
"From: <area>" is the more meaningful, accurate signal for where the
sender was, so labeling the scope string too was redundant.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup: sitting in Aarhus but sending with the broad #dk scope should
still show "Aarhus by" -- the scope-linked area alone doesn't tell you
where the sender physically was. Adds a second, independent area
resolved from the message's own path[0] entry-point repeater (same
unique_prefix-only discipline as resolveEntryPointArea/Wardriving),
shown as "From: <area>" alongside the existing "Scope: #dk (Danmark
alle)" tag.
GetChannelMessages (both DB and in-memory paths) now captures path[0]
as an internal "entryPrefix" field; handleChannelMessages resolves it
to an area server-side via the existing resolveEntryPointArea, then
strips entryPrefix before the response goes out -- raw hash prefixes
never reach the client for this feature.
Scoped to the REST message list only, not the WebSocket live-append
path (shared broadcast infra used by several other pages) -- a
brand-new live message shows the scope-linked area only, until the
next full load picks up the resolved path[0] area.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
"Scope: #dk-aarhus" now renders as "Scope: #dk-aarhus (Aarhus by)"
when that region is linked to a configured area -- same regionScope
lookup already used on the Scopes tab, just reused here via a small
local cache (loadRegionAreaLabels) since channels.js doesn't share
analytics.js's closure.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers AreaEntry fields (label/polygon-or-box/regionScope), why
hierarchy is purely geometric (no parent field -- draw a broad area
generously enough to contain its sub-areas and rollup just works),
the two different area lookups (single most-specific match for
badges vs. all-containing-areas for aggregate counts), the homeArea
link to geo_filter and why it exists, and a verify-before-deploy
checklist for anyone drawing a new polygon.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds Config.HomeArea: when set to an existing area's key, that area's
geometry becomes the effective geo_filter (getGeoFilter), instead of
maintaining a second, independently-drawn boundary that can drift out
of sync -- exactly what happened with Germany's box bleeding into
southern Denmark earlier this session. Falls back to the standalone
GeoFilter field when HomeArea is unset or unresolved, so existing
deployments see no behavior change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup asked why the two sections' counts for the same region (e.g.
#dk) disagree. They're not meant to match: Scope Adoption by Area only
counts positioned nodes geographically inside the area, while
Repeaters by Region has no geographic restriction and reflects a
live, independently-refreshing relay-activity window. Added an
explicit note to each section pointing at the other and explaining
why their numbers differ, so this isn't mistaken for a bug again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup: "Danmark (alle)" showed almost nothing (0 of 10 support) even
though most real nodes use the generic #dk scope. Root cause: the
per-area node count used AreaKeyForPoint, which picks only the single
most-specific area for each node -- a node in "Odense by" never also
counted toward the broader "Fyn" or "Danmark (alle)" it geographically
sits inside, so a country-level area only ever saw the leftovers no
smaller area had already claimed.
New AreaKeysForPoint returns every containing area (not just the
best match), used by computeScopeAdoptionByArea so a node now counts
toward all of its containing areas -- Danmark (alle) genuinely
aggregates every Danish sub-area's nodes now, not just stragglers.
AreaForPoint/AreaKeyForPoint (single-match, used by the GPS-share and
session area badges) are unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup wanted per-node visibility, not just aggregate counts: for
each area with a linked region, list the actual nodes that support
it (own default_scope or ever relayed it) vs. the ones that sit there
but don't. Replaces the summary table with expandable per-area groups
(same pattern as Repeaters by Region), each showing a Supporting /
Not Supporting node list with links.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dborup: a repeater that has relayed dk-horsens traffic supports the
Horsens area, even if its own default_scope is something else (e.g.
the generic #dk most nodes actually use) or unset entirely. The
previous version only checked default_scope, missing this -- same
runs-this-region vs carried-this-region's-traffic distinction the tab
already draws between OriginatingNodesByRegion and RepeatersByRegion.
Both NodesWithAnyScope and NodesMatchingArea now also check the
node's entry in the cached RepeaterRelayInfo map (TransportedScopes),
reusing the same cache RepeatersByRegion already populates.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
NodesMatchingArea had omitempty, so a genuine 0 count (the meaningful
case this feature exists to surface -- "linked region, zero adoption")
dropped out of the JSON entirely. The frontend read undefined off the
missing key and called .toLocaleString() on it, throwing and blanking
Region Utilization/Repeaters by Region/Bridge Repeaters along with it
since they render later in the same updateData pass. Drop omitempty,
add a defensive `|| 0` on the client, and a regression test that
decodes into a raw map (not the typed struct, which hides this by
zero-valuing the field regardless of whether the key was present).
Caught live on stg immediately after deploying the feature -- verified
via browser before/after, not just the unit test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New geographic view, independent of the raw hashRegion-code-based
Region Utilization: buckets every positioned node by its configured
area (AreaKeyForPoint) and tallies how many have any default_scope
at all, and how many specifically match the area's own linked
region. Surfaces gaps Region Utilization can't see, since that only
knows about region strings that already appeared in traffic — a real
area with real nodes that never produced a single scoped message is
invisible there but shows up here as 0% adoption.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Expose regionScope on /api/config/areas and use it to enrich Region
Utilization's unused-region list, Repeaters/Nodes by Region, and
Bridge Repeaters with the linked area's human name (e.g.
"dk-aarhus (Aarhus by)") instead of a bare hashRegion code.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
For senders who never share a literal GPS fix, resolve the session's
path[0] entry-point prefix to a known repeater position (only when it
resolves unambiguously, same unique_prefix discipline as
/api/resolve-hops) and label it with the most specific configured
area — shown as a badge in the Sessions table, clearly marked
approximate since it's the repeater's position, not the sender's own.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds AreaForPoint (config.go), picking the most specific configured
area when several overlap, and wires it into the GPS Sharing table as
a badge next to each sender's shared position.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add AreaEntry.RegionScope so config.json's "areas" can be tied to a
hashRegions channel scope, plus ops/meshguide-sync/sync_areas.py to
pull polygons and scope confirmations from meshguide.dk's
community-maintained dataset instead of guessing from naming
conventions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same "Show all N / Show fewer" pattern just added for Top Senders and
Coverage by Observer, extended to Sessions and Entry Points — a 7-day
window can have a lot of rows in all four tables.
Entry Points splits the resolve step (async /resolve-hops call, runs
once) from the render step (sync, runs once per toggle) so expanding
doesn't re-hit the API. The Ambiguous bucket row, when present, stays
always-visible and doesn't count against the top-10 limit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signal Quality Trends now renders right after the message-volume
chart, above Top Senders and Coverage by Observer (previously it was
lower, after Sessions/Entry Points).
Top Senders and Coverage by Observer collapse to the top 10 rows by
default, with a "Show all N" / "Show fewer" toggle — same pattern
already used for node neighbor lists (nodes.js). Coverage's "% of
Observations" stays relative to ALL observers, not just the shown
top-10 slice, so the percentages don't lie when collapsed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Wardriving analytics tab (all 6 originally-proposed ideas): activity
over time, entry-point repeaters, per-observer coverage, signal
quality trends, session detection, and coordinate-broadcast/GPS-share
detection — plus per-sender message drill-down, live-map path
visualization (real GPS trail, approximate entry-point trail, and a
compare view between the two), and session airtime (LoRa Time-on-Air
x distinct relaying repeaters).
Also reapplies c4b4453 (Foreign Traffic tab Bornholm fix), which had
only existed on foreign-traffic-wip and never reached master.
User report: "Hvorfor er der bornholm repeater under Foreign Traffic
igen!?" — this exact bug was already fixed once this session (commit
c4b4453), but that fix only ever existed on foreign-traffic-wip and
was never merged into master. wardriving-analytics branched from
master, so it never had the fix, and every stg deploy of this branch
today silently reintroduced it.
Reapplies c4b4453's fix: renderForeignTrafficTab now classifies
"foreign" live from each node's own lat/lon via nodePassesGeoFilter +
window.MC_GEO_FILTER (same mechanism the Nodes page already uses),
instead of trusting the node's `foreign` DB flag — a one-way flag
written once by the ingestor's MarkNodeForeign and never cleared, so
a node flagged foreign under an old, narrower geo_filter box stays
flagged forever even after the box widens to include it.
Note: while investigating, found that foreign-traffic-wip also has
entire Foreign Traffic tab features (Entry Points, Distance vs Hop
Count sections) that never reached master either — out of scope for
this specific fix, worth a separate look.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Caught live on stg: every wardriving session showed airtimeMs=0, even
for senders with confirmed multi-hop relay paths. The in-memory store
is memory-bounded (maxMemoryMB/maxPackets) and had already evicted
these transmissions despite them still being well within the SQL
24h/7d window — AirtimeForTransmissions treated "not found in memory"
the same as "found, zero relays", so ok=true with a silent 0 came back
indistinguishable from a genuinely never-relayed message.
Now ok=false when NONE of the requested IDs are held in memory
(nothing knowable → omit the field). A partial match still returns
ok=true with the known subset's total, matching how every other
airtime metric already tolerates eviction rather than going dark
entirely once retention exceeds the in-memory window.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds AirtimeMs to WardrivingSession: total network airtime consumed
relaying a session's messages, using the exact same formula as the
Overview tab's "Relay Airtime Share" (issue #1768) — ToA(payload_bytes)
x COUNT(DISTINCT resolved repeater in path), summed per message.
That formula needs the in-memory store's resolved-path index
(resolvedPubkeyReverse), which db.go's SQL-only wardriving code
doesn't have — so buildWardrivingSessions now also tracks each
session's transmission IDs (unexported, json:"-"), and the route
handler calls the new PacketStore.AirtimeForTransmissions(txIDs) to
fill in AirtimeMs after the fact. Omitted entirely (nil) in DB-only
mode rather than shown as a misleading zero.
Frontend adds an Airtime column to the Sessions table (ms/s
formatting, dash when unavailable).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
For messages where a sender shared their real position AND have a
resolvable entry-point repeater, pairs them up: haversine distance
shown inline in the message table ("2.3 km from DK-XXX"), plus a new
"Compare shared position vs. entry point" drill-down button.
On the map (drawGPSTrail's new 'compare' kind), draws both trails —
real position (blue/red/green) and entry-point repeater (orange) —
with a thin dashed line connecting each matched pair, so you can see
at a glance how good the entry-point proxy is when ground truth is
available.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
For senders who don't share a literal position, each message's
resolved path[0] entry-point repeater has a known, fixed location.
Plotting those in chronological order gives a rough "which coverage
area were they in" trail — not the sender's real position, just the
nearest repeater that first relayed them at that moment. Consecutive
messages hitting the same repeater collapse into one point.
Adds a second drill-down button, "View approximate path via entry
points", alongside the existing GPS-share path button. drawGPSTrail
(map.js) now accepts an optional per-point label (shows the repeater
name in the popup) and an opts.kind so the on-map caption makes clear
when a trail is the approximate entry-point kind vs a real shared
position.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds drawGPSTrail (map.js) — a self-contained polyline+markers renderer
for a plain sequence of lat/lon points, distinct from drawPacketRoute's
mesh-hop/node-resolution path. Handed off via the same sessionStorage
pattern already used for packet routes (map-route-hops), keyed
map-gps-trail: {points, sender}. Start/end points are colored blue/red,
intermediate points green, each with a timestamped popup.
In the Wardriving tab's per-sender drill-down, a "View path on map"
button appears whenever a session/sender has 2+ shared positions in
the fetched range, ordered chronologically (the API returns
most-recent-first).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found in production: at least one wardriving client appends plaintext
"<lat>,<lon>" after the standard token (e.g.
"MM:c3e_zJ1rUA:55.59743,13.00128"), confirmed against live traffic.
The generic anomaly framing (flagging non-standard payload length,
showing a raw hex dump) was based on the wrong assumption that this
was an undocumented binary format — it's actually deliberate,
human-readable coordinate sharing by that sender's client.
Replaces WardrivingAnomaly/detectWardrivingAnomalies/
StandardPayloadCount with WardrivingGPSShare/detectWardrivingGPSShares:
detects the specific "<token>:<lat>,<lon>" suffix (with range
validation) and reports the most recent position per sender. The
per-message drill-down gets Lat/Lon instead of a standard/anomaly
flag. Frontend replaces the "Payload Anomalies" section with "GPS
Sharing", linking each position to the live map
(#/map?lat=..&lon=..&zoom=15, an existing deep-link format).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
.btn-link is a large touch-target utility class (min-height/width
48px, background/border chrome) used for toolbar icon buttons — not a
lightweight inline-link style. Applying it to sender names inside a
table cell made them look like broken buttons. Switched to inline
styles matching the existing plain-link pattern used elsewhere
(nodes.js's "Show all N neighbors" button): no background/border,
color: var(--link-color).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New GET /api/analytics/wardriving/sender-messages returns one sender's
individual messages (most-recent-first, capped at 200): resolved
entry-point path (path[0] first), per-observer SNR/RSSI, and payload
standard/anomaly classification. Pass since+until to scope to one
session's exact range; otherwise it covers the sender's whole window.
Frontend makes sender names in Top Senders and Sessions clickable —
clicking toggles an inline expansion row with that sender's messages,
resolving every distinct path prefix in one /resolve-hops call (same
unique_prefix-only discipline as the aggregate Entry Points table).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
decodeGrpTxt (cmd/ingestor/decoder.go) builds decoded_json.text as
"<sender>: <message>", so a wardriving message's real text is
"<sender>: MM:<base64>", not a bare "MM:<base64>" at the start of the
string. detectWardrivingAnomalies's HasPrefix(text, "MM:") check never
matched anything live, silently returning standardPayloadCount=0 and
no anomalies — caught after deploying idea 6 to stg and seeing 0/0
against traffic that should have had ~160 standard-format messages.
Fixed to match the exact "<sender>: MM:" prefix. The Go test fixtures
shared the same wrong assumption and are fixed too.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extends GET /api/analytics/wardriving with a payload-anomaly detector:
every "MM:<base64>" message is checked against the standard 7-byte
anonymous session token (confirmed empirically against live traffic).
Non-standard lengths or undecodable payloads are grouped per sender
as a candidate signal that MeshMapper's optional "Broadcast My
Coordinates" mode is active — surfaced as a raw hex dump, never
interpreted as lat/lon, since that mode's byte format is undocumented.
Messages without the "MM:" prefix (plain channel chat) are ignored
entirely rather than polluting either bucket.
Frontend adds a Payload Anomalies table plus a stat card, completing
all 6 originally-proposed wardriving analytics ideas.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extends GET /api/analytics/wardriving with each sender's messages
grouped into distinct sessions/runs — a gap over 15 minutes starts a
new one. Each session reports duration, message count, and how many
distinct entry-point repeaters/observers it touched. Frontend adds a
Sessions table (most-recent-first) and a session-count stat card.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extends GET /api/analytics/wardriving with avgSnr/avgRssi over the
same time buckets as the activity series, plus overall averages.
Frontend adds two min/max-scaled line charts (SNR has no natural
zero floor, RSSI is negative dBm, so these can't reuse the 0-baseline
message-volume chart) and two stat cards.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Surfaced by the new Wardriving Coverage-by-Observer table: every
observer on this Danish mesh showed a "—" location because iataCoords
only had US/global codes. Same root cause as issue #1786, just never
extended to the codes this specific mesh's observers actually use.
Added the 11 codes I could verify as real airport IATA codes; QXV,
KRP, MRW look non-standard and are left unresolved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New GET /api/analytics/wardriving endpoint plus an Analytics tab
covering the three requested angles: activity over time + top
senders, entry-point repeaters (path[0] tally, unique_prefix-only
name resolution), and per-observer coverage using observers' known
IATA coordinates. MeshMapper's on-air ping is an anonymous session
token by default, not the sender's live GPS, so sender position
itself isn't tracked — documented in the tab and OpenAPI description.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Scoped/Unscoped/Unknown counts answered "how much" but not "which
regions" — a channel spanning multiple hashRegions areas looked
identical to one using just one. New "Regions Used" column lists the
distinct scope_name values seen on each channel's scoped messages,
most-used first.
Backend: GetChannelScopeAdoption now also queries per-(channel,
scope_name) counts and attaches them as ChannelScopeAdoption.Regions.
Non-fatal on query failure — adoption counts still render without it.
[dijkstra] doc-noted that `total` in handleNodes' filtered path reflects
only the returned page-slice, not an all-pages count — matches
pre-existing semantics, not a regression, but worth being explicit that
fetchAllNodes (public/app.js) never reads it for pagination (relies on
page length instead).
[munger] the pagination compensation loop's maxIterations=50 safety cap
now logs a WARN breadcrumb when it's actually hit, instead of silently
returning a possibly-short page. New test seeds 51 consecutive
hidden-prefix nodes at limit=1 to force the cap and asserts both the
empty result and the log line.
