Per bot review on PR #1852 (comment 5012304813): renderForeignTrafficTab
shipped without a test, repeating the same TDD-policy finding flagged
on the prior round, and the tab lacked the 60s auto-refresh + stop-hook
convention its Roles/Scopes siblings use (data would go stale until the
user re-navigated away and back).
- Added _stopForeignTrafficRefresh + a 60s setInterval loop, wired into
the tab-switch handler and destroy(), matching _stopRolesRefresh/
_stopScopesRefresh exactly.
- New test-analytics-foreign-traffic-tab.js: loads analytics.js in a vm
sandbox (same pattern as test-frontend-helpers.js's
makeAnalyticsSandbox), stubs fetchAllNodes with a fixed node mix, and
asserts row sort order, role/zero-count exclusion, the empty state,
both foreignCount note variants, and that the new stop-hook is
exported and idempotent.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Configuring geo_filter alone was silently changing what GET /api/nodes
returned — every node outside the polygon and not yet foreign_advert
-tagged (which only happens on that node's next ADVERT after
geo_filter is set) vanished from every view built on this endpoint,
including the live map. There was no way to see them again short of
waiting for each one to re-advertise.
geo_filter's own purposes — the ingestor tagging foreign adverts, and
the explicit prune-geo-filter admin flow — are untouched; this only
gates the passive node-list declutter view behind an explicit query
param, so turning geo_filter on to build analytics (e.g. the Foreign
Traffic tab) doesn't have the side effect of hiding nodes from the map.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New "Foreign Traffic" tab under Analytics, sorted by
unscoped_relay_count_24h descending — the metric already existed in
the /api/nodes response (cmd/server/repeater_liveness.go,
repeater_enrich_bulk.go) with an openapi doc string calling out
exactly this: "A well-configured repeater sets flood.max.unscoped 0,
so a non-trivial count flags a base-config problem" — but had zero
frontend surface until now.
Phase 1 of an investigation into unscoped traffic from PL/DE/SE
leaking into the DK mesh. Phase 2 (cross-referencing which of this
volume traces back to a foreign-origin sender via path_json) needs
foreign_advert data to accumulate first — it's only set going forward
from when geo_filter was configured, not backfilled for existing
history.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Docker build failed after adding the new internal/regions module
(10eed4b): the Dockerfile COPYs each internal/ package individually
into the build context rather than the whole directory, so a new
package needs an explicit COPY line in both the server and ingestor
build stages.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Bridge Repeaters renders as a flat, non-collapsible table — unlike
Repeaters by Region and Nodes Running This Region, which group into
collapsible <details> per region. On stg.meshview.dk it currently has
80 rows (~2700px tall), which buried Nodes Running This Region far
down the page, reading as if the section had vanished entirely.
Reordering doesn't touch the row-count problem itself, but it stops
the shortest, most specific section from being the one that pays for
it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Working through the outstanding non-blocker findings across all five
automated review passes on the PR:
- MAJOR: GetChannelMessages' rows.Scan() error was silently discarded
— a schema mismatch would produce zero-valued messages instead of a
visible error. Now returns the error, matching sibling query loops.
- Extracted the hashRegions name-normalization rule (trim, "#"-prefix,
dedupe) shared between cmd/ingestor's loadRegionKeys and cmd/server's
region-utilization diff into a new internal/regions package, so the
two can no longer drift apart on the rule independently.
- Batched GetRepeaterNamesByKeys' SQL IN (...) clause in chunks of 500
— an unbounded clause risks SQLITE_MAX_VARIABLE_NUMBER on very large
deployments' byPathHop candidate sets.
- handleScopeStats' remaining silently-swallowed err==nil branches
(GetMatchedRegionNames, GetNodesByDefaultScope,
GetChannelMessageScopeStats, GetChannelScopeAdoption) now log a WARN
breadcrumb on failure instead of failing invisibly.
- Scope Adoption table was rendering 3 of the 4 ChannelScopeAdoption
fields (Unscoped omitted) — the visible numbers didn't reconcile to
the message total without doing the subtraction by hand. Added the
column.
- Removed a duplicate `vertical-align: middle` declaration on
.badge-transport (dead, not a behavior change).
- Deleted ChannelMessageResp — flagged for interface{} vs *string/*int
typing, but turned out to be completely unused dead code; removing
it resolves the finding more directly than retyping something
nothing constructs.
Left two NIT/MINOR items as-is with reasoning:
- renderRegionNodeGroups' inline styles match the same pattern used in
15+ other places in analytics.js — "fixing" only this one function
would make it less consistent with the file, not more.
- TransmissionResp's interface{} fields (a different struct than the
one just removed) are an established, actively-used pattern for
nullable SQL-scanned values across that whole response type;
retyping it is a much larger, unrelated refactor.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Per bot review on PR #1852 (comment 5011643190): two changes in the
recent delta had zero test coverage, which the repo's TDD policy
(AGENTS.md § Test-First Development) requires for both net-new UI
helpers and bug fixes on existing UI.
- scopeCellHtml (public/app.js): unit tests for all three states
(non-transport dash, resolved region, unresolved "unknown") plus an
XSS-escaping check.
- Column-prefs backfill (public/packets.js): extracted the inline
reconciliation logic into a standalone reconcileVisibleCols(),
exposed via _packetsTestAPI like the rest of packets.js's pure
helpers, and unit tested directly instead of only through the DOM.
Writing the test surfaced a real bug in the original backfill: it
couldn't distinguish "column didn't exist yet" from "user explicitly
unchecked this column" — both look identical as a missing key in the
saved array, so any existing column a user had hidden (e.g. Observer)
would get silently resurrected on next page load. Fixed by persisting
a second "known columns" baseline (packets-visible-cols-known)
alongside the visibility array: a missing key only gets backfilled as
newly-visible if it's also absent from the known-columns baseline: a
pre-existing key missing from the saved array is a deliberate user
choice and stays hidden. Falls back to the pre-#1852 column list for
visitors whose saved prefs predate persisting the baseline at all.
Also registers scopeCellHtml in .eslintrc.json's cross-file globals
list (same treatment as transportBadge) — packets.js referencing it
was tripping no-undef since app.js and packets.js are separate
non-module scripts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cmd/server/prune-requests/ is internal/prunequeue's on-disk job queue,
written next to the DB path at runtime — not source, shouldn't be
tracked.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Packets column-visibility toggle persists an explicit array of
visible column keys to localStorage. A returning visitor with a
pre-existing saved array (from before the Scope column existed) never
got it shown, since the load path used the saved array verbatim
instead of reconciling it against the current COL_DEFS. Missing keys
now default to visible (unless narrow-viewport-hidden), same as a
fresh visitor gets.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Region-scope was only visible as a small inline pill inside the Type
column (transportBadge). Split it into its own sortable "Scope" column
between Type and Observer, matching the same three states (region
name / unknown / not applicable for non-transport packets). Wired
into the existing column visibility toggle and mobile narrow-width
hide list.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Git tracked docs/DEPLOYMENT.md and docs/deployment.md as two separate
blobs. On case-insensitive filesystems (macOS/Windows) only one can
exist on disk, so the uppercase entry silently went stale while the
lowercase one kept receiving edits — a case-sensitive checkout (e.g.
Linux CI) would have gotten both, one of them outdated. Keep only
docs/deployment.md, the one actually maintained.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Brings in a pending restructuring of the deployment doc, and adds a
section on reloading hashChannels/hashRegions via SIGHUP instead of a
full ingestor restart, which would otherwise reset in-memory analytics
state (Repeaters-by-Region, Bridge Repeaters, etc).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
channelKeys and regionKeys (derived from config.json's hashChannels/
hashRegions) were loaded exactly once in main() and held in plain
maps for the process lifetime — adding a new hashtag channel or
region required a full ingestor restart to take effect, which also
resets the in-memory relay/scope-attribution state built up since the
last restart (see the Repeaters-by-Region cold-start discussion).
Wraps both maps in atomic.Pointer-backed hotKeys (same pattern already
used for Config.NodeBlacklist, just actually wired to a trigger this
time) and adds a SIGHUP handler that re-reads config.json and
atomically swaps in freshly-derived keys:
kill -HUP <ingestor-pid>
On a malformed config during reload, the previous keys are kept and
the error is logged — a bad edit must not blank out a working
ingestor. The MQTT message-handler closure now pulls a fresh snapshot
from the atomic pointers per message instead of closing over a static
map, so no other call sites needed to change.
Adds channelScopeAdoption to /api/scope-stats: the existing
ChannelMessages aggregate (scoped/unscoped/unknown for channel chat)
broken down PER CHANNEL — which specific channels (#test,
#wardriving, ...) actually use region scoping vs which never do.
Ordered by message volume, capped at the top 30 channels.
Frontend renders a compact table under "Channel Messages": channel
name, total messages, scoped count+%, unknown count.
Adds hourlyActivityByRegion to /api/scope-stats: each region's message
counts bucketed by hour-of-day (0-23 UTC), aggregated across every day
in the window — answers "when during a typical day is this region
active" rather than "how did volume change over the window" (that's
the existing chronological TimeSeries chart).
Frontend renders a compact heatmap: one row per region, 24 hour
columns, color intensity normalized per-row (each region's own busiest
hour) so a quiet region's daily shape stays visible next to a loud
one instead of being crushed toward zero.
Repeaters by Region, Nodes Running This Region, and Bridge Repeaters
all cleared their entire section (heading included) when the data was
empty, which reads as "this feature doesn't exist" rather than "no
data yet" — especially confusing right after a restart, when the
neighbor graph and path resolution need a few minutes to catch back
up before repeater-level scope data repopulates. Now always renders
the heading + description with an explanatory empty-state message
instead of hiding.
Adds bridgeRepeaters to /api/scope-stats: RepeatersByRegion inverted
into pubkey -> regions, keeping only repeaters that have relayed
traffic for MORE than one region. These are the mesh's literal
backbone nodes connecting otherwise-separate regional communities —
losing one is a more consequential failure than losing a
single-region repeater.
Computed inline while building RepeatersByRegion (reuses the same
byRegion map and role-filtered names lookup, no extra queries).
Frontend renders a small table under "Repeaters by Region": repeater
name (linked to its node detail page), region count, and the region
list.
Adds channelMessages to /api/scope-stats: the same scoped/unscoped/
unknown question as the main Summary, but restricted to payload_type=5
(channel chat) instead of all observed traffic. Most channel chat is
plain FLOOD rather than transport-scoped, so this can read very
differently from the all-traffic numbers — answers "how many of our
actual channel messages carry a region scope" directly instead of
requiring the reader to infer it from the broader stats.
New GetChannelMessageScopeStats() mirrors GetScopeStats' query shape
but scopes TotalMessages to ALL route types for payload_type=5 (not
just route_type 0/3), since restricting to transport routes would
answer a different question than "how many channel messages, period".
Frontend renders a small "Channel Messages" stat-card row under the
main summary cards, window-scoped like the rest of the tab.
Fixes#1849.
## Problem
The packets table "HB" (Hop Bytes / col-hashsize) column always shows
`1` for TRACE packets. TRACE packets use header path bytes as per-hop
SNR readings, not truncated hop hashes — see
`internal/packetpath/route.go` `PathBytesAreHops(TRACE) = false`. The
high-2-bit "hash_size" derivation applied to a SNR byte is meaningless
(typically `1`).
## Fix
At the 3 render sites in `public/packets.js` (`buildGroupRowHtml` header
row, its child rows, `buildFlatRowHtml`), when `payload_type === 9`
(TRACE) render `—` in the `col-hashsize` cell with a `title` tooltip
explaining that TRACE path bytes are SNR readings and directing users to
the sidebar decoder for the actual hop count. Non-TRACE rows unchanged.
## TDD
- Red commit: `dd7a4e78` — `test-issue-1849-trace-hashbytes.js` asserts
col-hashsize cell equals `—` with a title tooltip for TRACE, numeric for
non-TRACE. CI must fail on this commit.
- Green commit: `115368d0` — 3-site fix in `public/packets.js`.
## Verification
```
$ node test-issue-1849-trace-hashbytes.js
✅ All 4 tests passed
```
Preexisting failures in `test-packets.js` (13) are unchanged by this PR
(verified via `git stash`) — unrelated to the surface touched here.
## Scope
- `public/packets.js`: 3 render sites, +21/-8.
- `test-issue-1849-trace-hashbytes.js`: new unit test.
- `test-all.sh`: wire new test.
No public API change.
---------
Co-authored-by: clawbot <bot@example.invalid>
Root-caused via a real report: "Repeaters by Region" showed a hyper-
local scope (#dk-fyn-middelfart) as transported by repeaters spread
across the whole country, which shouldn't be possible — MeshCore
firmware only relays a TRANSPORT_FLOOD/DIRECT packet when the
repeater's OWN configured region matches the packet's transport code
(examples/simple_repeater/MyMesh.cpp allowPacketForward, gated by
RegionMap::findMatch against the repeater's local region list).
Confirmed against the meshcore-dev/MeshCore source.
The bug: both TransportedScopes computation paths (bulk
computeRepeaterRelayInfoMap and per-node GetRepeaterRelayInfo) fold a
full pubkey's byPathHop entries together with its matching 1-byte
raw-prefix bucket — an intentional, existing fallback for RelayCount/
LastRelayed ("this node is probably active") that tolerates the
1-byte hash's inherent ambiguity (any node sharing that first byte
gets folded in). Applying the SAME fold to TransportedScopes asserted
something far more specific than the ambiguous signal can support,
and something the protocol itself wouldn't allow.
Scope accumulation now only happens on the exact-key (resolved,
unambiguous) pass; RelayCount/LastRelayed/RelayActive keep the
prefix-bucket fold unchanged, since those remain intentionally
approximate. Added relayEntry.fromPrefix to carry this distinction
through the per-node path, and rewrote the test that had pinned the
old (incorrect) folding behavior.
Adds originatingNodesByRegion to /api/scope-stats: nodes whose OWN
default_scope (#899) is a given region, complementing the existing
repeatersByRegion (transported_scopes) breakdown. The distinction
matters — a repeater can relay traffic for a region it isn't itself
configured with, so "who runs this region" and "who has carried this
region's traffic" are different, both useful questions.
Frontend: refactored the per-region collapsible-list rendering (used
by both breakdowns) into a shared renderRegionNodeGroups() helper
instead of duplicating the HTML-building logic, and added a "Nodes
Running This Region" section alongside "Repeaters by Region".
byPathHop indexes both full pubkeys and short hex-prefix "bucket" keys
used internally for ambiguous-hop resolution — GetRepeaterRelayInfoMap
returns TransportedScopes for every one of those keys indiscriminately.
The first deploy of repeatersByRegion iterated the raw map and fell
back to showing the bucket key itself when no name matched, so a
handful of short internal keys were counted as "repeaters" (stg showed
594 "repeaters" for #dk — every active repeater plus every 2-6 char
bucket key that ever touched a #dk packet).
GetNodeNamesByKeys is now GetRepeaterNamesByKeys and filters
`role IN ('repeater','room')` in the SQL itself, so a key only survives
if it's a real node — bucket keys never match a nodes.public_key row
and are dropped rather than falling back to the raw key.
Adds repeatersByRegion to /api/scope-stats: for every region that has
ever matched a transmission, which distinct repeaters/rooms have
relayed traffic carrying that scope. Sourced from the same 5-min
background-recomputed bulk relay-info cache the Nodes page already
uses (GetRepeaterRelayInfoMap / TransportedScopes, #1751) — no new
expensive computation, just an inversion + name lookup.
Frontend renders a collapsible per-region repeater list (name links
to the node detail page) under a new "Repeaters by Region" section,
explicitly framed as a coverage/redundancy signal: a region carried
by only one repeater is a single point of failure for that area.
Adds configuredRegions/unusedRegions to /api/scope-stats: an all-time
(not window-scoped) diff between the operator's configured hashRegions
list and the set of scope_name values that have actually matched a
transmission still in retention. Surfaces how much of the region list
is dead weight — directly actionable evidence for pruning, which is
also the real fix for the HMAC-collision noise (fewer configured
regions -> lower birthday-collision probability per packet).
Server config now parses hashRegions (previously ingestor-only, same
config.json key) purely to read the configured names — no HMAC key
derivation happens server-side.
Frontend: a "Region Utilization" section on the Scopes tab shows
used/unused counts and a collapsible list of the unused region names.
The transport badge previously only surfaced a known scope in the
title tooltip, requiring a hover. Now the label itself reads
"T·#region" so it's visible at a glance, matching the "T?" unknown
case which was already inline. Badge gets a max-width + ellipsis so
long region names (e.g. "#dk-trekantsomraadet") don't blow out the
Type column — full name stays in the title.
The packet detail pane already showed scope (with an "unknown scope"
fallback for empty scope_name), but the packets table itself gave no
at-a-glance signal — the existing transportBadge() "T" marker only
encoded route type.
transportBadge() now takes an optional scopeName argument: a resolved
region enriches the tooltip, an empty scope_name (transport-eligible
but unmatched/ambiguous) renders as "T?" with a distinct muted badge
style instead of the confident amber, so it's visually distinguishable
from a resolved scope without relying on color alone. Existing callers
that don't pass scopeName (live.js) are unaffected.
QueryGroupedPackets (SQLite + in-memory) didn't select scope_name at
all — added it, since the Packets tab defaults to the grouped view.
GetChannelMessages already returned an empty scope string for
transport-eligible packets whose region couldn't be determined (no
configured region matched, or matchScope now reports an HMAC
collision as unknown), but the UI treated empty scope the same as
"not applicable" and rendered nothing — indistinguishable from a
plain FLOOD/DIRECT message that never carries a scope at all.
Adds route_type to the channel-message payload (both SQLite and
in-memory paths, plus the decrypt-candidate and live WS paths) so the
frontend can tell "not transport-scoped" (routeType 1/2, no tag) apart
from "transport-scoped but unresolved" (routeType 0/3 with empty
scope, now shown as "Scope: unknown").
transport_code_1 is only 16 bits, so with enough configured hashRegions
a different, unrelated region's HMAC can coincidentally also match a
packet's real code1 (expected rate ~= len(regionKeys)/65534 per
packet). matchScope used to return the first match found while
iterating the region-key map, whose order Go randomizes per process —
so an ambiguous packet's assigned scope was effectively a coin flip
that could even change across ingestor restarts.
Confirmed on stg.meshview.dk (1098 configured regions): a live GRP_TXT
message's code1=C417 matched both "#dk" (the sender's true region) and
"#dk1906" (an unrelated collider), and was observed resolving to
either name across sends.
matchScope now scans every configured region and only returns a name
when exactly one matches; two or more matches are reported as
unknown-scoped ("") rather than guessed, with a log line for operator
visibility. Doesn't fix the root collision-probability cause (that
needs fewer/coarser configured regions) but stops confidently
displaying a wrong single answer.
The WS packet broadcast (IngestNewFromDB / IngestNewObservations in
cmd/server/store.go) builds its own map independent of the REST
response helpers, so scope_name was missing there even after it was
added to GetChannelMessages and txToMap. Adds it to both broadcast
paths and wires channels.js's live-append handler to read it, so
brand-new messages show their scope immediately instead of waiting
for the next periodic REST refresh.
Adds scope_name to GetChannelMessages (both SQLite and in-memory
store paths) and to the general packet response shape, then renders
it as "Scope: <name>" in the channel chat meta line so operators can
see which region scope a message was transported under.
Red commit: aabe8143d1
## Summary
Drop `max-width: 1200px` on `.observers-page` in `public/style.css`. The
observers table is column-dense and already runs `data-priority`
auto-hide on narrower viewports, so the 1200px cap crushed columns on
wide monitors while the mobile hide-column CSS had already dropped
columns for narrower widths — the exact "starts auto-hiding but never
expands back" behavior described in #1846.
Removes the cap so the browser layout uses available viewport width (as
`.analytics-page` does).
## Change
- `public/style.css:2113` — remove `max-width: 1200px` from
`.observers-page`.
- `test-issue-1846-observers-width.js` — new regression guard. Parses
`.observers-page` rule from `style.css`; fails if a `max-width` cap
`<1600px` is re-introduced (matches the `.analytics-page` convention
referenced in triage).
- `.github/workflows/deploy.yml` — wire the new test into the frontend
unit test job.
## Test discipline
- Red commit `aabe8143`: test only, no CSS change — CI fails on
assertion (`max-width 1200px must be >= 1600px`).
- Green commit `665e88a6`: CSS fix — test passes.
## Browser verification
Browser tool unavailable in this session (gateway restart required).
Change is a pure CSS width-cap removal on a single class; no JS, no
HTML, no theming implications. The regression guard makes silent
reintroduction impossible.
## E2E assertion
Regression guard is source-grep, not DOM-level:
`test-issue-1846-observers-width.js:41` (`.observers-page` block
max-width allowlist). Justification: bug is a single declarative CSS
attribute; a DOM E2E for it would just assert
`getComputedStyle(...).maxWidth === 'none'`, which mirrors the source
assertion at higher cost. If a follow-up wants a Playwright
viewport-resize check, easy to add.
## Preflight
`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
— clean on the green HEAD.
Fixes#1846
---------
Co-authored-by: corescope-bot <bot@corescope.dev>
## Summary
QR codes on node-details pages didn't scan. Two `public/nodes.js` render
sites called `qr.createSvgTag(3, 0)`; the vendor lib treats the second
arg as an **absolute pixel margin**, so `0` removed the QR quiet zone.
QR spec requires ≥4 modules of light border, so every scanner rejected
the code.
With `cellSize=3`, four modules = 12 pixels. Changed both call sites to
`createSvgTag(3, 12)`:
- `nodes.js:813` — node-details list detail
- `nodes.js:1705` — node-map overlay (also swaps background to
transparent; with margin restored, dark modules stay 12px from the SVG
edge so the transparent overlay still parses over the map tile)
Out of scope, filed as follow-up in triage: contrast tuning of the
transparent-overlay branch.
## TDD
Red commit: `3be8552b` — [test-only, CI
red](https://github.com/Kpa-clawbot/CoreScope/commit/3be8552baff64795504e8dcb888ed25cbd6a50be)
Green commit: `6f7e83a6` — two-line fix + test passes
## Test
`test-issue-1843-node-qr-quiet-zone.js` — Node/vm harness that loads
`public/vendor/qrcode.js`, grep-asserts both `nodes.js` call sites use
margin ≥ 12px, renders the SVG, and checks the `<path>` `M` coordinates
keep all dark modules ≥ 12px from every viewBox edge (and viewBox =
`modules*cellSize + 2*margin`).
Verified red on parent commit (test fails with margin=0 leaving dark
modules at 0,0), green after fix (dark modules at 12,12 with 12px free
border).
## Preflight
`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
— all gates clean.
Browser verified: pending — will validate on staging after CI. (Frontend
UX bug; DOM/grep test above exercises the same SVG code path that
scanners consume.)
E2E assertion added: `test-issue-1843-node-qr-quiet-zone.js:63` (viewBox
+ `<path>` coord grep on real-rendered SVG).
Fixes#1843.
---------
Co-authored-by: corescope-bot <bot@corescope.dev>
Fixes#1838
## Problem
`/api/scope-stats` reported 100% scoped whenever any region was
configured. Reporter noticed on a scopeless instance that "unscoped" was
always zero — the pie visual is misleading to operators deciding on
`denyf *`.
## Root cause
`cmd/server/db.go:22` restricted the entire scope-stats denominator to
`route_type IN (0, 3)`. Per firmware `docs/packet_format.md § Route
Types`:
- `0` = `TRANSPORT_FLOOD`
- `1` = `FLOOD`
- `2` = `DIRECT`
- `3` = `TRANSPORT_DIRECT`
Only routes 0 and 3 carry `transport_code_1` (transport-level scope).
Routes 1 and 2 are inherently unscoped by protocol. The existing SQL was
correct for the "how many transport-scopable routes are actually scoped"
question, but the denominator was silently promoted to "all traffic" in
the UI. Bonus: the comment on `routeTypeTransportSQL` labelled routes
0+3 as "FLOOD (0) and DIRECT (3)" — wrong on both counts.
## Fix
- `cmd/server/db.go` — corrected the `routeTypeTransportSQL` comment;
added `routeTypeNonTransportSQL = "route_type IN (1, 2)"` alongside it.
- `GetScopeStats` runs a second `COUNT(*)` over `route_type IN (1,2) AND
first_seen >= ?` and folds that count into `Summary.Unscoped`. Same
index path as the existing query — one extra scan per `/api/scope-stats`
call (cached 30s per triage's carmack finding).
- `public/analytics.js` — Scopes tab header explains the denominator
(all observed transmissions) and which route types carry scope. Card
notes now render `X% of all traffic` for Scoped/Unscoped and `X% of
scoped` for Unknown Scope so the pie's denominator is explicit.
## TDD
- Red: `5554ffe4` — extended `TestGetScopeStats` +
`TestHandleScopeStats` with `route_type=1` and `route_type=2` rows and
asserted `Unscoped = 3` (1 transport-NULL + 2 non-transport). Ran the
tests and confirmed assertion failure (`Unscoped = 1, want 3`).
- Green: `ebbb9253` — implementation + label copy. Full `go test
./cmd/server/...` passes (54s).
## Preflight overrides
- check-branch-clean: justified — cross-stack fix by design (backend
semantics change + matching frontend label copy). All 4 files are
exactly the surface the triage comment identified.
## Verification
- `go test ./cmd/server/...` — 54s, all pass.
- Firmware confirmation: `firmware/docs/packet_format.md:20-24` (route
type table).
## Files touched
- `cmd/server/db.go` — comment fix + second COUNT query.
- `cmd/server/db_test.go` — extended fixture.
- `cmd/server/routes_test.go` — extended fixture + isolate from seed
data.
- `public/analytics.js` — labels and header copy.
---------
Co-authored-by: corescope-bot <bot@corescope.dev>
## Summary
Phase A of #1828: extract the 5 aggregate builders in
`handleObserverAnalytics` into pure helpers in a new
`cmd/server/observer_analytics.go`. Handler becomes a snapshot + filter
+ 5 composed calls.
Also adopts the `byTxID` direct-read in `buildPacketTypes` (issue body's
core observation): the payload-type histogram no longer allocates a full
`enrichObs` map + interface-boxed fields just to read `tx.PayloadType`.
That's the ~90% perf win the triage called out.
Scope is exactly Phase A per the second triage comment. Phase B
(sub-endpoints, caching, SQL migration) is deferred to a follow-up.
## Byte-identical output
- Timeline / NodesTimeline: same key set, same sort, same labels.
- PacketTypes: same keys/counts. Both legacy
(`enriched["payload_type"].(int)`) and new (`tx.PayloadType == nil`
guard) skip obs whose tx is missing or `PayloadType` is `nil`.
- SnrDistribution: same 2-unit floor bucketing (negative-side rounding
preserved), same ascending sort.
- RecentPackets: still the first 20 enriched observations (`enrichObs`
kept only here, where the extra fields are actually needed).
## TDD
- Red commit: `9dc62f43` — 7 unit tests fail on assertions (not build
errors) against stubs.
- Green commit: `8d41011d` — implementations + handler rewire. All new
tests + existing `TestObserverAnalytics*` handler tests pass.
## Preflight
`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
→ clean (all 8 hard gates + 3 warnings pass).
## Non-goals
- No new endpoints.
- No SQL migration.
- No public API signature change.
- Snapshot count unchanged (still one under RLock, per #1481 P0-2).
Fixes#1828.
---------
Co-authored-by: fix-1828-bot <bot@corescope.local>
Co-authored-by: clawbot <bot@corescope>
Fixes#1836.
Observer↔node cross-nav links from #1826 land on 404 because pubkeys are
stored lowercase in `nodes` and uppercase in `observers`, and the
backend `WHERE` lookups are case-sensitive. The two link builders now
normalize case at the boundary.
## Fix
- `public/observer-detail.js`: observer → node href passes
`currentId.toLowerCase()`.
- `public/nodes.js`: node → observer href passes
`n.public_key.toUpperCase()`.
## TDD
- Red: `test-issue-1836-crossnav-case-normalization.js` asserts the two
hrefs contain `.toLowerCase()` / `.toUpperCase()`. Fails on master.
- Green: 2-line production change makes the test pass.
Scope: 2 production line edits + 1 new test file.
---------
Co-authored-by: openclaw-bot <bot@openclaw.local>
Adds, per node, how many distinct FLOOD adverts it originated in the
last 7 days. Zero-hop adverts (route_type DIRECT) are excluded, so a
nearby observer hearing a node's cheap local adverts does not inflate
the number - the existing advert_count mixes both kinds and cannot tell
a chatty flooder (mesh-wide airtime) from
the recommended 240-minute zero-hop cadence (local only).
Consumers (the ArcScope repeater advisor) rate advert hygiene against
the community practice of one flood advert every ~49h; with the mixed
total, a correctly configured repeater looked chatty whenever an
observer sat within zero-hop range.
Implemented like the relay-liveness fields: a pure, unit-tested counter
over (first_seen, route_type, hash) entries with the same timestamp
parsing and hash dedup, fed by a from_pubkey-indexed query capped at the
2000 most recent advert rows. The flood route-type constant is named
advertRouteTypeFlood so this merges independently
of the open unscoped-relay PR (#1823).
---------
Co-authored-by: Waydroid Builder <build@waydroid.local>
## Summary
Fixes#1833 — the `.legend-toggle-btn` (palette icon) on the Live view
was hardcoded to `bottom: 1rem`, so on typical desktop viewports it sat
underneath the VCR playback bar and was unreachable. The reporter had to
hide the legend via `localStorage` to work around it.
## Fix
`public/live.css:1313` — one-character-class change:
```diff
.legend-toggle-btn {
position: fixed;
- bottom: 1rem;
+ bottom: calc(var(--vcr-bar-height, 58px) + 10px);
right: 1rem;
```
Mirrors the existing pattern already used by every other bottom-pinned
Live overlay:
- `.live-feed` (live.css:1204)
- `.live-overlay[data-position="br"]` (live.css:1372)
- `.feed-show-btn` (live.css:903)
`--vcr-bar-height` is maintained by the ResizeObserver on `.vcr-bar`, so
the button now tracks bar growth (mobile two-row layout,
safe-area-inset) instead of overlapping.
Same class of regression as #685 / #1206 / #1107 — an overlay that was
missed in the prior sweep.
## TDD
- Red commit `f6a938b7`: `test-issue-1833-legend-toggle-vcr-offset.js`
asserts `.legend-toggle-btn`'s `bottom` declaration references
`var(--vcr-bar-height`. Fails on assertion (not build error) against the
hardcoded `1rem`.
- Green commit `e399092b`: the CSS one-liner above. All 3 assertions
pass.
Grep-based CSS assertion per AGENTS.md § "E2E-DOM-grep exemption" —
there is no existing Playwright test in this area that measures the
toggle-button rect against `--vcr-bar-height`, and standing one up would
be disproportionate for a single-property fix that mirrors three
existing, tested overlay patterns.
## Preflight
`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
— all hard gates pass, no warnings.
## Browser verified
Not required per fix-issue skill (CSS-only, mirrors three existing
tested patterns). Staging will pick up the change on merge; visual
regression will be caught if the mirrored pattern breaks (grep test in
this PR + existing E2E tests on the sister overlays).
---------
Co-authored-by: corescope-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Follow-up to #1823: TestRepeaterUnscopedRelayCount and its _Bulk twin
were ~30 verbatim lines apart (DB, node insert, store seeding,
assertions), differing only in the lookup under test - seeding changes
had to land twice. Both now use a shared seedUnscopedRelayFixture +
assertUnscopedCounts and contain only their
respective lookup call. No behaviour change; the relay-liveness suite
passes.
Co-authored-by: Waydroid Builder <build@waydroid.local>
## What
Adds a per-node API field `unscoped_relay_count_24h` on repeater/room
nodes: the
number of the node's 24h relay-hops that were unscoped floods
(route_type == ROUTE_TYPE_FLOOD). A strict subset of relay_count_24h.
## Why
A well-configured repeater runs `flood.max.unscoped 0` and should not
rebroadcast
unscoped floods — each one is re-sent by every repeater that hears it,
so one
packet turns into mesh-wide traffic. Exposing this lets clients (the
ArcScope
repeater advisor) detect and flag that base-config problem from observed
packets.
## How
Computed like relay_count_24h in both paths (bulk /api/nodes + per-node
detail)
with a route_type==FLOOD filter; reuses the byPathHop index, no
migration. Wired
into both handlers + OpenAPI schema + unit tests (per-node and bulk).
Co-authored-by: Waydroid Builder <build@waydroid.local>
Adds cross-navigation between the observer detail page and the node
detail page for the same pubkey (community feature request from
cwichura).
**Changes**
- `public/observer-detail.js`: new `<a
href="#/nodes/${encodeURIComponent(currentId)}">View node detail →</a>`
inside `.page-header`, next to the `<h2 id="obsTitle">`.
- `public/nodes.js`: new sibling `<a
href="#/observers/${encodeURIComponent(n.public_key)}"
class="btn-primary">Observer →</a>` in the same button row as the
`Analytics` / `Reach` anchors on the full node detail page. Uses the
existing `ph-eye` phosphor icon.
**Test — TDD red→green**
- Red commit: `8c2315e1` (`test(#1825): red — observer<->node cross-link
anchors missing`) — 4/4 assertions fail on master; CI RED.
- Green commit: `ff8f6ed7` — minimum production change; 4/4 assertions
pass locally.
Test file: `test-issue-1825-observer-node-cross-links.js` —
static-source DOM-grep style consistent with the neighbouring
`test-issue-1789-observer-firmware-cols.js` /
`test-observers-headings.js` pattern. It asserts:
1. observer-detail.js contains
`href="#/nodes/${encodeURIComponent(currentId)}"`.
2. That anchor sits inside the `.page-header` block.
3. nodes.js contains
`href="#/observers/${encodeURIComponent(n.public_key)}"`.
4. That anchor is a sibling of the analytics/reach anchors in the same
flex row.
**Notes**
- Pubkeys are `encodeURIComponent`-escaped on both sides
(defense-in-depth; MeshCore pubkeys are hex only).
- No API changes. No CSS changes. No new dependencies.
Fixes#1825
---------
Co-authored-by: meshcore-bot <meshcore-bot@users.noreply.github.com>
## Fixes#1741
`TestBoundedLoad_OldestLoadedSet` (and any test building a 5000-row
fixture) hung/timed out, blocking reliable `go test ./cmd/server` and
CI.
## Root cause
The four test-DB builders in `cmd/server/bounded_load_test.go`
(`createTestDBAt`, `createTestDBWithObs`, `createTestDBWithAgedPackets`)
inserted rows in a loop with no `BEGIN`/`COMMIT`. With the pure-Go
`modernc.org/sqlite` driver every `Exec` auto-commits → one fsync per
row → ~2N fsyncs for N transmissions (tx + obs). At
`numTx=5000` that's ~10k fsyncs and the fixture blows past the test
timeout. Sibling tests with `numTx<=3000` happened to stay under the
timeout, so only the 5000-row cases visibly hung.
## Fix
Wrap each insert loop in a single `BEGIN`/`COMMIT` so the whole fixture
build becomes one commit. Fixtures now finish in well under a second
regardless of `numTx`; the tests' actual assertions (`oldestLoaded` set,
newest-first ordering, bounded load) are exercised instead of the
timeout masking them. Also made the
prepared-statement `Exec` calls check their error (previously discarded)
so a failed insert surfaces instead of silently leaving the DB short.
No production code changed — test infrastructure only.
## Verified
- `TestBoundedLoad_OldestLoadedSet`: **0.18s** (was: 30s timeout /
FAIL).
- Full `TestBoundedLoad*` + retention group: passes in ~1.2s.
- `go test ./...` in `cmd/server`: exit 0 (no longer blocks on this
test).
Co-authored-by: Waydroid Builder <build@waydroid.local>
Co-authored-by: Claude <noreply@anthropic.com>
## Fixes#1729
The firmware-default **Public** channel (channel-hash byte `0x11` = 17)
was rendered as an opaque **"Encrypted (0x11)"** row at the bottom of
the analytics Channels tab, despite the key being well-known and
builtin.
## Root cause
`computeAnalyticsChannels` applied the #978 rainbow-table validation
(`SHA256(SHA256("#name")[:16])[0]`, the **hashtag** hash scheme) to
every decoded channel name. The Public channel is a **PSK** channel
whose hash byte is key-derived (`SHA256(key)[0]` = 17), not
hashtag-derived (`186` for `#Public`). So the ingestor-decoded name
`"Public"` failed the hashtag check and was discarded, the row forced to
`encrypted=true, name="ch17"`.
## Fix
Trust the ingestor's `decryptionStatus`. The ingestor already persists
`decryptionStatus:"decrypted"` when it decoded a packet with a real key
(PSK), and `"no_key"` / `"decryption_failed"` otherwise. When the packet
is `decrypted`, skip the hashtag hash check and keep the name — it came
from a key-based decryption, not a
rainbow-table lookup. The #978 mismatch rejection still applies to
non-decrypted packets, so rainbow-table collisions are still caught.
Frontend needs no change: `encrypted=false, name="Public"` lands in the
"Network" group (top), not "Encrypted".
## Tests
- `makeGrpTx` gains `makeGrpTxWithStatus` companion to set
`decryptionStatus`.
- `TestComputeAnalyticsChannels_PublicChannelPreserved`: hash 17 /
"Public" / `decrypted` → name stays `"Public"`, `encrypted=false`.
- `TestComputeAnalyticsChannels_UndecryptedNameStillValidated`: a
non-`decrypted` name failing the hashtag check is still downgraded to
`ch17` (#978 regression guard).
All channel-analytics tests pass; `go build ./...` clean.
Co-authored-by: Waydroid Builder <build@waydroid.local>
Co-authored-by: Claude <noreply@anthropic.com>
Red commit: `3898dbc5` (verified locally — CI run URL pending)
## Problem
A standalone `mqtt-broker` container (`eclipse-mosquitto:2`) was
provisioned out-of-band on the staging VM. It now owns MQTT, is attached
to external docker network `meshcore-net`, and binds host port `8883`.
The current `docker-compose.staging.yml` still:
- Publishes `1883:1883` on the host (dead weight; conflicts the moment
the broker moves to that port).
- Defaults `DISABLE_MOSQUITTO=false`, so the in-container mosquitto
burns RAM and briefly contests the `mqtt-broker` docker DNS name on cold
start.
- Doesn't join `meshcore-net`, so the ingestor can't resolve
`mqtt-broker:1883` via docker DNS without manual surgery.
## Fix (`docker-compose.staging.yml` only)
1. Remove the `1883:1883` host port publish from `staging-go`.
2. Flip `DISABLE_MOSQUITTO` default from `false` to `true`. Operators
can opt back in with the env var.
3. Attach `staging-go` to both `default` and `meshcore-net`; declare
`meshcore-net` as `external: true` so the file never tries to
create/destroy operator state.
Healthcheck and Caddy/443 plumbing untouched (out of scope).
## Test added (TDD framing: Option A — Go shape-asserts)
`cmd/server/staging_compose_broker_test.go:1` adds four regex-based
assertions on the compose file shape:
- staging-go does **not** bind port `1883` in ANY form (quoted/unquoted
short form, or long-form `target: 1883` / `published: 1883`).
- `DISABLE_MOSQUITTO` uses the interpolated default form
`${DISABLE_MOSQUITTO:-true}` (preserves operator override). Bare literal
`true`, or a later `=false` override in the same env block, is rejected.
- Top-level `networks:` declares `meshcore-net` as `external: true`.
- `staging-go` attaches to `meshcore-net` via a real
`services.staging-go.networks:` sub-key (comment-stripped so an
in-comment example can't masquerade).
Regex (not YAML byte-equality) so cosmetic edits don't break the guard.
No new go module deps. Red commit `3898dbc5` fails all 4 assertions on
master. Green commit `38297ff4` makes them pass. Round-1 hardening
commit `9f7155e2` tightens the regexes (per adversarial + kent-beck
must-fixes) and was verified against master's YAML shape — all 4 tests
fail on `origin/master`'s compose, pass on branch, proving the tightened
regexes still gate a real regression.
## Risk
Low, with one intentional semantic change.
- **Semantic change (v3.7+):** `DISABLE_MOSQUITTO` in
`docker-compose.staging.yml` now defaults to `true`. This is a
**deliberate flip** — the standalone `mqtt-broker` container is now
authoritative on the staging host, and running the in-container
mosquitto alongside it wastes RAM and races the docker DNS name
`mqtt-broker` on cold start. Operators who want the pre-v3.7 shape
(in-container mosquitto + host-published `1883`) must explicitly opt
back in via env override AND re-add the `1883:1883` port mapping
(concrete snippet is inline in the compose file and in `DEPLOY.md` under
"Standalone MQTT broker (staging)"). This intent is called out in a
`SEMANTIC CHANGE (v3.7+)` header comment at the top of
`docker-compose.staging.yml`.
- **Deploy prereq:** the external `meshcore-net` docker network MUST
already exist on the host before `docker compose up`. If it doesn't,
compose refuses to start `staging-go`. This is documented inline in the
compose file (with the `docker network create meshcore-net` one-liner)
and in `DEPLOY.md`.
- **Only takes effect where the standalone broker is deployed** — which
it already is on staging today. The legacy `DISABLE_MOSQUITTO=false`
path remains reachable via env override; the ingestor's upstream config
is untouched.
Partial fix — no tracking issue; follow-up to operator-side broker
provisioning.
---------
Co-authored-by: corescope-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Partial fix for #1809.
Red commit: c9c782b5 (CI runs on PR open; standalone red-branch CI not
configured — local repro proves the gate, see below).
## Problem
Issue #1809: at startup the background fill loader logged `background
load FAILED` within seconds and `backgroundLoadFailed=true` was set,
leaving the coverage gate tripped even though LoadChunked itself
completed normally.
## Root cause
`main.go:225-245` spawned `go store.loadBackgroundChunks()` as soon as
`FirstChunkReady` fired (chunk #1 = 10000 tx). But `s.oldestLoaded` is
only assigned at the end of `LoadChunked` (`chunked_load.go:329-333`),
~tens of seconds later. The bg loader read `oldestLoaded==""` at
`store.go:1462-1466`, broke out immediately, walked zero chunks, and the
coverage gate at `store.go:1543-1554` flipped
`backgroundLoadFailed=true`.
## Fix (initial)
Introduce `PacketStore.RunStartupLoad(chunkSize)` (`chunked_load.go`).
It runs `LoadChunked` first; only on success and only when
`hotStartupHours > 0` does it call `loadBackgroundChunks`. `main.go`
invokes `RunStartupLoad` in the same goroutine pattern as before, so
`FirstChunkReady` still unblocks the HTTP listener bind at chunk #1 —
only the bg loader is gated.
## Round-1 followups (this push)
Reviewer-driven hardening on top of the initial fix:
### Production behavior (commit db5592f6)
- **Steady-state semantics tightened.** `RunStartupLoad` now picks a
terminal state on every branch:
- LoadChunked error → `backgroundLoadFailed=true` with captured error
(was: `done=false, failed=false` indefinite).
- `hotStartupHours == 0` → `backgroundLoadDone=true` immediately,
`progress=100` (was: `done=false` forever → healthz stuck on
`backgroundLoadComplete=false`).
- Successful hot-window path → terminal state is whatever
`loadBackgroundChunks` sets (#1690 semantics, unchanged).
- **Runtime invariant assertion (A7).** `loadBackgroundChunks` panics
when `oldestLoaded==""` and packets exist — a future refactor that
re-introduces the parallel-spawn race fails loudly instead of silently
shipping the same coverage regression.
- **`RunStartupLoad` cleanup.** Inlined the superfluous goroutine +
channel that wrapped `LoadChunked` (direct call is equivalent).
- **Logging.** Added an INFO line between `LoadChunked` completion and
bg-loader start (the #1809 post-mortem needed exactly this signal).
Fixed the lying `"background load will start"` log that fired even on
the `hotStartupHours==0` branch.
- **Immutability documented.** `hotStartupHours` is now explicitly
documented as immutable post-construction, so the lock-free reads in
`LoadChunked` / `RunStartupLoad` / `loadBackgroundChunks` are sound.
### Test coverage (commits db5592f6 + e9e12acf)
- **Tautology fix (B1, commit e9e12acf).** The original
`Test1809_StartupLoad_BgLoaderSeesOldestLoaded` fixture seeded all 100
rows inside the 1h hot window, so `LoadChunked` alone produced
coverage=1.0 — the test passed even if `loadBackgroundChunks` was a
no-op. Rewrote the fixture to spread 100 rows over 14 days with
`hotStartupHours=24`, so only ~7 rows are hot and the remaining ~93 MUST
be loaded by the bg loader for the assertions to hold. Original
red-commit assertions kept intact; added `len(packets) > hot-only cap`
and `oldestLoaded < hot-cutoff - 12h` assertions on top.
- **New tests (commit db5592f6, `runstartup_load_test.go`)** codify the
new contracts:
- `TestRunStartupLoad_HotStartupHoursZero_SetsDoneImmediately`
- `TestRunStartupLoad_LoadChunkedError_SetsFailedTerminal`
- `TestRunStartupLoad_EmptyDB_SetsDoneTerminal`
- `TestRunStartupLoad_BgLoaderRunsAfterLoadChunkedSets_OldestLoaded`
- `TestLoadBackgroundChunks_PanicsOnOldestLoadedEmpty_Invariant`
### Docs (commit 70fa16f7)
- Package-level doc in `chunked_load.go` now documents `RunStartupLoad`
as the orchestrator entry point alongside `LoadChunked` /
`loadStatusMiddleware` / `OnChunkLoaded`.
### Preflight (commit eec1b48c)
- Test-fixture DDL annotated with `// PREFLIGHT: async=true
reason="unit-test fixture"` so the async-migration gate distinguishes
ephemeral test schema from prod migration paths.
## What's still NOT fixed (left intentionally open)
This PR addresses the startup race specifically. Issue #1809 will be
closed by the operator after observing healthy startup logs (`background
load complete: ... coverage=100.0%`) in prod for a full restart cycle.
Do not auto-close — leave open until the operator verifies.
## Test
Local repro (red branch state, before green commit): `go test
./cmd/server/ -run Test1809_StartupLoad_BgLoaderSeesOldestLoaded` → FAIL
on assertion `backgroundLoadFailed=true ...
oldest="2026-06-30T18:56:09Z"`. After green commit + round-1 followups:
PASS. Full `cmd/server/...` suite: ok in ~70s.
## Risk
Low — startup path only. New behavior gates surfaced by the new tests;
coverage-gate semantics unchanged. Runtime panic is a new failure mode
but only fires on a state (`oldestLoaded=="" && len(packets)>0`) that is
unreachable on the current code path — it exists solely as a refactor
tripwire.
---------
Co-authored-by: mc-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: meshcore-bot <bot@meshcore>
# Partial fix for #1749 — MQTT watchdog escalation + panic recovery +
tick exposure
Red commit: 9912bbb3e9 (CI run:
https://github.com/Kpa-clawbot/CoreScope/actions/runs?branch=fix%2Fissue-1749)
## Problem
Production CoreScope v3.9.1 (and a more recent prod recurrence on
2026-06-30 with the wcmesh source on `ssl://mqtt2.wcmesh.com:8883`)
showed two distinct watchdog failure modes:
1. **Per-source paho machinery dies silently.** `IsConnectedFn` returns
false; paho's `SetAutoReconnect(true)` never retries. The watchdog's
`processLivenessTransition` deliberately stays silent on
`LivenessDisconnected`, trusting paho to recover — so there is no
escalation path when that trust is misplaced.
2. **Watchdog goroutine death.** Three sources went silent within ~60s
of each other; no `WATCHDOG` log lines for 75 min. The most plausible
single point of failure is a panic inside `emit` (e.g. a blocked log
pipe) killing the loop with no defer/recover.
## Changes
**`cmd/ingestor/mqtt_watchdog.go`**
- Added `disconnectedReconnectMultiplier = 5` constant.
- Added `DisconnectedSinceUnix int64` (atomic) on `SourceLivenessState`.
Stamped on the first tick the source is observed disconnected; cleared
on any non-disconnected tick.
- `processLivenessTransition` now escalates: when `(now -
DisconnectedSinceUnix) > multiplier × threshold`, emits a `WATCHDOG
ESCALATION` WARN and calls `maybeForceReconnect` (subject to existing
`forceReconnectThrottle`). Distinct from the existing `LivenessStalled`
path so operators can grep escalation events independently.
- Added package-level `watchdogLastTickUnix atomic.Int64` +
`WatchdogLastTickUnix()` getter. The loop stamps it BEFORE per-source
processing — a wedged source-handler does not freeze the clock for an
external observer.
- `runLivenessWatchdogLoop` wraps each per-source
`processLivenessTransition` call in `func() { defer recover; ... }()` so
a panic in `emit` (or in any per-source code path) is logged and
skipped, not fatal. The loop continues to the next source and the next
tick.
**`cmd/ingestor/stats_file.go`**
- Added `WatchdogLastTickUnix int64` field on `IngestorStatsSnapshot`
(additive, `omitempty`); populated from `WatchdogLastTickUnix()` each
stats tick.
**`cmd/server/mqtt_status.go`**
- `MqttStatusResponse` gains `WatchdogLastTickUnix int64` (additive,
`omitempty`) sourced from the ingestor stats file; surfaced via `GET
/api/mqtt/status`.
**`config.example.json`**
- No new config field added — the multiplier is a code constant (5×) per
the issue's "N×threshold (e.g. 5×)" recommendation. The active
per-source `threshold` is the 5-minute scan threshold hard-coded at the
sole `runLivenessWatchdog` callsite (`cmd/ingestor/main.go:460`:
`runLivenessWatchdog(60*time.Second, 5*time.Minute)`), so escalation
fires at ~25 minutes (5 × 5min) of continuous disconnect — plus a
deterministic per-source jitter of 0..30s (#1810 round-1, see Taleb #4)
to avoid synchronized escalation across N sources sharing an upstream
broker outage.
## Acceptance (#1749)
- [x] Persistent `LivenessDisconnected` > N×threshold → force-reconnect
+ WARN
- [x] Watchdog goroutine liveness clock exposed (`WatchdogLastTickUnix`
in `/api/mqtt/status`)
- [x] Test: `IsConnectedFn` false for >5×threshold → assert
`ForceReconnectFn` invoked at least once
- [x] Test: panic in `emit` → assert loop recovers and continues ticking
## Test plan
- 4 new tests in `cmd/ingestor/mqtt_watchdog_1749_test.go` (all RED on
master, GREEN on this PR).
- Existing watchdog tests (`mqtt_watchdog_force_reconnect_test.go`,
`mqtt_reconnect_test.go`, r1/r2/m1 suites) continue to pass — the
escalation path is additive.
## Preflight
- TDD: red commit pushed and asserted to fail BEFORE green commit
landed.
- PII grep: clean on diff and PR body.
- Worktree: `_wt-fix-1749` on branch `fix/issue-1749`.
---------
Co-authored-by: corescope-bot <bot@corescope.dev>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: bot <bot@local>
## Summary
Replaces the three drifted per-surface payload-type label vocabularies
with a single canonical map keyed by firmware enum name.
Per the locked triage comment on #1799
([comment-4823975431](https://github.com/Kpa-clawbot/CoreScope/issues/1799#issuecomment-4823975431)):
> Create `public/payload-labels.js` exporting `{GRP_DATA: {short:'Group
Data', long:'Group data packet', enumId:6}, ...}`. Migrate `packets.js
typeMap`, `packet-filter.js FW_PAYLOAD_TYPES`, `live.js TYPE_COLORS
legend` to consume it. E2E that scrapes each surface and asserts label
equality.
## Changes
- **`public/payload-labels.js`** (new) — canonical map exposed as
`window.PayloadLabels` and `window.PayloadLabelsApi`. Keys are firmware
enum names; values carry `{short, long, enumId}` plus derived
`SHORT_BY_ID` / `FW_PAYLOAD_TYPES` / `TYPE_ALIASES` for legacy callers.
- **`public/packets.js`** — `TYPE_NAMES` + `typeMap` now read from
`PayloadLabelsApi.SHORT_BY_ID`. Literal kept only as a defensive
fallback for the case where the script tag fails to load.
- **`public/packet-filter.js`** — `FW_PAYLOAD_TYPES` + `TYPE_ALIASES`
now sourced from `PayloadLabelsApi`. Literal fallback retained so `node
test-packet-filter.js` still works headlessly.
- **`public/live.js`** — legend `<li>` rows are now generated from
`window.PayloadLabels` in stable order, killing the third-vocabulary
`Message — Group text` / `Direct — Direct message` drift the #1797
review surfaced.
- **`public/index.html`** — `<script src="payload-labels.js">` loaded
before `roles.js` / `packet-filter.js` / `packets.js`.
- **`test-issue-1799-label-vocab-e2e.js`** (new) — Playwright E2E.
Scrapes `#liveLegend` rows and the `/packets` type-filter checklist,
asserts each label matches `window.PayloadLabels[ENUM].short` for
`TXT_MSG`, `GRP_TXT`, `GRP_DATA`. Also verifies `window.PacketFilter`
still recognises the enum names.
- **`.github/workflows/deploy.yml`** — wired the new E2E into the
existing Playwright block.
## TDD trail
- Red commit `eb392d4` — adds the failing E2E only (asserts
`window.PayloadLabels` exists and labels match; both fail).
- Green commit `44e902a` — introduces the canonical map and migrates the
three surfaces.
## Verification
- `node test-packet-filter.js` — 92/92 pass with the new fallback
wiring.
- Preflight: `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh
origin/master` — clean.
Browser verified: E2E `test-issue-1799-label-vocab-e2e.js` exercises
`/live` legend + `/packets` type filter against a Playwright headless
Chromium; CI's Playwright block runs it on every push.
E2E assertion added: `test-issue-1799-label-vocab-e2e.js:139` —
`assert(fromLegend === canon, ...)` and `assert(fromPackets === canon,
...)` per enum.
Fixes#1799
---------
Co-authored-by: mc-bot <bot@corescope>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: clawbot <clawbot@kpa.com>
Co-authored-by: clawbot <bot@clawbot.local>
The Analytics → Neighbor Graph tab fetches the full (uncapped) graph
and,
when it exceeds NODE_LIMIT (1000), skips the force simulation with a
"use filters to reduce the node count" notice. But filtering never
actually
re-enabled rendering:
- the node-count guard tested _ngState.allNodes (the immutable full
fetched
set, assigned once in createGraphState and never reassigned) instead of
the
displayed/filtered _ngState.nodes, so its verdict was fixed at load
time;
- the entire draw loop lives in startGraphRenderer(), which ran exactly
once
at load and was never called from applyNGFilters(), so a filter change
updated the node/edge arrays and stat cards but never un-hid the canvas
or
scheduled an animation frame -> the graph stayed blank no matter how few
nodes remained.
This explains both reported symptoms (selects too many nodes initially
AND
stays broken once restricted to fewer).
Fix: make the render lifecycle filter-aware.
- startGraphRenderer() now guards on the displayed set (_ngState.nodes),
cancels any running rAF loop before re-deciding, toggles the canvas plus
a
stable-id "skipped" notice, and restarts cleanly (no double loops).
- applyNGFilters() calls startGraphRenderer() so every filter change
re-evaluates the guard and (re)starts or stops the loop.
- the initial render now goes through applyNGFilters() so the first
paint
already respects the default filters (observers unchecked, saved
min-score)
instead of dumping the full fetched graph.
Test: `node --check public/analytics.js` passes. Manually: open
Analytics → Neighbor Graph on a
mesh with >1000 nodes → the "skipped" notice shows; tighten filters
(min-score up / roles
off) below 1000 → the graph now renders (was blank before); loosen again
→ notice returns.
Frontend-only change (`public/analytics.js`); no backend/API change.
---
**TDD note (review round 1):** Single-commit community bug-fix on an
existing UI surface (no "net-new UI" exemption). The e2e
`test-issue-1758-ng-filter-rerenders-e2e.js` is the red→green gate — it
fails on `origin/master` (the renderer kept the node-count guard on the
full fetched graph and never un-hid the canvas) and passes with the fix.
Per AGENTS.md the separate red/green-commit *form* is a bot rule, not a
contributor gate.
---------
Co-authored-by: Waydroid Builder <build@waydroid.local>
Co-authored-by: Waydroid Builder <claude@michael.arcan.de>
Two server tests flaked intermittently and reddened CI on unrelated
(frontend)
PRs that merged master:
- TestDistanceConcurrentRequestsDuringBuildReturn202 asserted all 10
concurrent
requests get 202 'during the build window', but the lazy distance build
on the
tiny test DB finishes almost instantly, so on a fast machine some
requests
raced past it and got 200 (~50% flake). Add a nil-by-default
distanceBuildHook
seam on PacketStore (zero overhead in prod) that the test uses to hold
the
build open until all requests have been served — making the window
guarantee
deterministic.
- TestHandleNodePaths_AnchorBiasInconsistency_Issue1278 queried /paths
right
after store.Load(), racing the path-hop index that Load() builds in a
background goroutine (#1008); the membership/canonical result was thus
non-deterministic (rarer flake, worse under suite load). Wait for
PathHopIndexReady() before querying.
Both run 30x green and pass -race. No production behavior change (hook
is nil).
Co-authored-by: Waydroid Builder <claude@michael.arcan.de>
