mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-26 16:07:55 +00:00
1aed3ee5c82a8a71fcbf6180e42acee6ea8d0cd2
2838
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dborup
|
1aed3ee5c8 |
feat: bridge-repeaters breakdown on the Scopes analytics tab
Adds bridgeRepeaters to /api/scope-stats: RepeatersByRegion inverted into pubkey -> regions, keeping only repeaters that have relayed traffic for MORE than one region. These are the mesh's literal backbone nodes connecting otherwise-separate regional communities — losing one is a more consequential failure than losing a single-region repeater. Computed inline while building RepeatersByRegion (reuses the same byRegion map and role-filtered names lookup, no extra queries). Frontend renders a small table under "Repeaters by Region": repeater name (linked to its node detail page), region count, and the region list. |
||
|
dborup
|
c74a005247 |
feat: channel-messages-only scoped/unscoped breakdown on the Scopes tab
Adds channelMessages to /api/scope-stats: the same scoped/unscoped/ unknown question as the main Summary, but restricted to payload_type=5 (channel chat) instead of all observed traffic. Most channel chat is plain FLOOD rather than transport-scoped, so this can read very differently from the all-traffic numbers — answers "how many of our actual channel messages carry a region scope" directly instead of requiring the reader to infer it from the broader stats. New GetChannelMessageScopeStats() mirrors GetScopeStats' query shape but scopes TotalMessages to ALL route types for payload_type=5 (not just route_type 0/3), since restricting to transport routes would answer a different question than "how many channel messages, period". Frontend renders a small "Channel Messages" stat-card row under the main summary cards, window-scoped like the rest of the tab. |
||
|
dborup
|
40dc297e2b |
Merge remote-tracking branch 'upstream/master'
# Conflicts: # public/packets.js |
||
|
Kpa-clawbotandclawbot
|
a06ac8aceb |
fix(#1849): render — for TRACE Hop Bytes column (path bytes are SNR, not hop hashes) (#1850)
Fixes #1849. ## Problem The packets table "HB" (Hop Bytes / col-hashsize) column always shows `1` for TRACE packets. TRACE packets use header path bytes as per-hop SNR readings, not truncated hop hashes — see `internal/packetpath/route.go` `PathBytesAreHops(TRACE) = false`. The high-2-bit "hash_size" derivation applied to a SNR byte is meaningless (typically `1`). ## Fix At the 3 render sites in `public/packets.js` (`buildGroupRowHtml` header row, its child rows, `buildFlatRowHtml`), when `payload_type === 9` (TRACE) render `—` in the `col-hashsize` cell with a `title` tooltip explaining that TRACE path bytes are SNR readings and directing users to the sidebar decoder for the actual hop count. Non-TRACE rows unchanged. ## TDD - Red commit: `dd7a4e78` — `test-issue-1849-trace-hashbytes.js` asserts col-hashsize cell equals `—` with a title tooltip for TRACE, numeric for non-TRACE. CI must fail on this commit. - Green commit: `115368d0` — 3-site fix in `public/packets.js`. ## Verification ``` $ node test-issue-1849-trace-hashbytes.js ✅ All 4 tests passed ``` Preexisting failures in `test-packets.js` (13) are unchanged by this PR (verified via `git stash`) — unrelated to the surface touched here. ## Scope - `public/packets.js`: 3 render sites, +21/-8. - `test-issue-1849-trace-hashbytes.js`: new unit test. - `test-all.sh`: wire new test. No public API change. --------- Co-authored-by: clawbot <bot@example.invalid> |
||
|
dborup
|
ea340554e9 |
fix: don't credit TransportedScopes from the ambiguous prefix bucket
Root-caused via a real report: "Repeaters by Region" showed a hyper-
local scope (#dk-fyn-middelfart) as transported by repeaters spread
across the whole country, which shouldn't be possible — MeshCore
firmware only relays a TRANSPORT_FLOOD/DIRECT packet when the
repeater's OWN configured region matches the packet's transport code
(examples/simple_repeater/MyMesh.cpp allowPacketForward, gated by
RegionMap::findMatch against the repeater's local region list).
Confirmed against the meshcore-dev/MeshCore source.
The bug: both TransportedScopes computation paths (bulk
computeRepeaterRelayInfoMap and per-node GetRepeaterRelayInfo) fold a
full pubkey's byPathHop entries together with its matching 1-byte
raw-prefix bucket — an intentional, existing fallback for RelayCount/
LastRelayed ("this node is probably active") that tolerates the
1-byte hash's inherent ambiguity (any node sharing that first byte
gets folded in). Applying the SAME fold to TransportedScopes asserted
something far more specific than the ambiguous signal can support,
and something the protocol itself wouldn't allow.
Scope accumulation now only happens on the exact-key (resolved,
unambiguous) pass; RelayCount/LastRelayed/RelayActive keep the
prefix-bucket fold unchanged, since those remain intentionally
approximate. Added relayEntry.fromPrefix to carry this distinction
through the per-node path, and rewrote the test that had pinned the
old (incorrect) folding behavior.
|
||
|
dborup
|
a44f2a4ad5 |
feat: nodes-running-this-region breakdown on the Scopes analytics tab
Adds originatingNodesByRegion to /api/scope-stats: nodes whose OWN default_scope (#899) is a given region, complementing the existing repeatersByRegion (transported_scopes) breakdown. The distinction matters — a repeater can relay traffic for a region it isn't itself configured with, so "who runs this region" and "who has carried this region's traffic" are different, both useful questions. Frontend: refactored the per-region collapsible-list rendering (used by both breakdowns) into a shared renderRegionNodeGroups() helper instead of duplicating the HTML-building logic, and added a "Nodes Running This Region" section alongside "Repeaters by Region". |
||
|
dborup
|
93144ee654 |
fix: exclude byPathHop bucket keys from repeaters-by-region
byPathHop indexes both full pubkeys and short hex-prefix "bucket" keys
used internally for ambiguous-hop resolution — GetRepeaterRelayInfoMap
returns TransportedScopes for every one of those keys indiscriminately.
The first deploy of repeatersByRegion iterated the raw map and fell
back to showing the bucket key itself when no name matched, so a
handful of short internal keys were counted as "repeaters" (stg showed
594 "repeaters" for #dk — every active repeater plus every 2-6 char
bucket key that ever touched a #dk packet).
GetNodeNamesByKeys is now GetRepeaterNamesByKeys and filters
`role IN ('repeater','room')` in the SQL itself, so a key only survives
if it's a real node — bucket keys never match a nodes.public_key row
and are dropped rather than falling back to the raw key.
|
||
|
dborup
|
87d2f479a7 |
feat: repeaters-by-region breakdown on the Scopes analytics tab
Adds repeatersByRegion to /api/scope-stats: for every region that has ever matched a transmission, which distinct repeaters/rooms have relayed traffic carrying that scope. Sourced from the same 5-min background-recomputed bulk relay-info cache the Nodes page already uses (GetRepeaterRelayInfoMap / TransportedScopes, #1751) — no new expensive computation, just an inversion + name lookup. Frontend renders a collapsible per-region repeater list (name links to the node detail page) under a new "Repeaters by Region" section, explicitly framed as a coverage/redundancy signal: a region carried by only one repeater is a single point of failure for that area. |
||
|
dborup
|
afa76c4f52 |
feat: region-utilization report on the Scopes analytics tab
Adds configuredRegions/unusedRegions to /api/scope-stats: an all-time (not window-scoped) diff between the operator's configured hashRegions list and the set of scope_name values that have actually matched a transmission still in retention. Surfaces how much of the region list is dead weight — directly actionable evidence for pruning, which is also the real fix for the HMAC-collision noise (fewer configured regions -> lower birthday-collision probability per packet). Server config now parses hashRegions (previously ingestor-only, same config.json key) purely to read the configured names — no HMAC key derivation happens server-side. Frontend: a "Region Utilization" section on the Scopes tab shows used/unused counts and a collapsible list of the unused region names. |
||
|
dborup
|
31f3e77bf6 |
fix: show resolved scope name directly in the packets-tab badge label
The transport badge previously only surfaced a known scope in the title tooltip, requiring a hover. Now the label itself reads "T·#region" so it's visible at a glance, matching the "T?" unknown case which was already inline. Badge gets a max-width + ellipsis so long region names (e.g. "#dk-trekantsomraadet") don't blow out the Type column — full name stays in the title. |
||
|
dborup
|
aaef080f94 |
feat: surface scope on the Packets tab's transport badge
The packet detail pane already showed scope (with an "unknown scope" fallback for empty scope_name), but the packets table itself gave no at-a-glance signal — the existing transportBadge() "T" marker only encoded route type. transportBadge() now takes an optional scopeName argument: a resolved region enriches the tooltip, an empty scope_name (transport-eligible but unmatched/ambiguous) renders as "T?" with a distinct muted badge style instead of the confident amber, so it's visually distinguishable from a resolved scope without relying on color alone. Existing callers that don't pass scopeName (live.js) are unaffected. QueryGroupedPackets (SQLite + in-memory) didn't select scope_name at all — added it, since the Packets tab defaults to the grouped view. |
||
|
dborup
|
a94d57ed4a |
feat: show "Scope: unknown" for transport-scoped messages with no match
GetChannelMessages already returned an empty scope string for transport-eligible packets whose region couldn't be determined (no configured region matched, or matchScope now reports an HMAC collision as unknown), but the UI treated empty scope the same as "not applicable" and rendered nothing — indistinguishable from a plain FLOOD/DIRECT message that never carries a scope at all. Adds route_type to the channel-message payload (both SQLite and in-memory paths, plus the decrypt-candidate and live WS paths) so the frontend can tell "not transport-scoped" (routeType 1/2, no tag) apart from "transport-scoped but unresolved" (routeType 0/3 with empty scope, now shown as "Scope: unknown"). |
||
|
dborup
|
8765de71e0 |
fix: matchScope reports unknown instead of guessing on HMAC collision
transport_code_1 is only 16 bits, so with enough configured hashRegions
a different, unrelated region's HMAC can coincidentally also match a
packet's real code1 (expected rate ~= len(regionKeys)/65534 per
packet). matchScope used to return the first match found while
iterating the region-key map, whose order Go randomizes per process —
so an ambiguous packet's assigned scope was effectively a coin flip
that could even change across ingestor restarts.
Confirmed on stg.meshview.dk (1098 configured regions): a live GRP_TXT
message's code1=C417 matched both "#dk" (the sender's true region) and
"#dk1906" (an unrelated collider), and was observed resolving to
either name across sends.
matchScope now scans every configured region and only returns a name
when exactly one matches; two or more matches are reported as
unknown-scoped ("") rather than guessed, with a log line for operator
visibility. Doesn't fix the root collision-probability cause (that
needs fewer/coarser configured regions) but stops confidently
displaying a wrong single answer.
|
||
|
dborup
|
350bf7eeee |
fix: surface scope_name on live WS channel-message broadcast
The WS packet broadcast (IngestNewFromDB / IngestNewObservations in cmd/server/store.go) builds its own map independent of the REST response helpers, so scope_name was missing there even after it was added to GetChannelMessages and txToMap. Adds it to both broadcast paths and wires channels.js's live-append handler to read it, so brand-new messages show their scope immediately instead of waiting for the next periodic REST refresh. |
||
|
dborup
|
c686ae3f37 |
feat: surface transport scope on channel messages
Adds scope_name to GetChannelMessages (both SQLite and in-memory store paths) and to the general packet response shape, then renders it as "Scope: <name>" in the channel chat meta line so operators can see which region scope a message was transported under. |
||
|
Kpa-clawbotandcorescope-bot
|
8c3e397d39 |
fix(1846): drop 1200px cap on .observers-page (#1847)
Red commit:
|
||
|
Kpa-clawbotandcorescope-bot
|
c7f0c6931f |
fix(1843): restore QR quiet zone on node-details codes (#1844)
## Summary QR codes on node-details pages didn't scan. Two `public/nodes.js` render sites called `qr.createSvgTag(3, 0)`; the vendor lib treats the second arg as an **absolute pixel margin**, so `0` removed the QR quiet zone. QR spec requires ≥4 modules of light border, so every scanner rejected the code. With `cellSize=3`, four modules = 12 pixels. Changed both call sites to `createSvgTag(3, 12)`: - `nodes.js:813` — node-details list detail - `nodes.js:1705` — node-map overlay (also swaps background to transparent; with margin restored, dark modules stay 12px from the SVG edge so the transparent overlay still parses over the map tile) Out of scope, filed as follow-up in triage: contrast tuning of the transparent-overlay branch. ## TDD Red commit: `3be8552b` — [test-only, CI red](https://github.com/Kpa-clawbot/CoreScope/commit/3be8552baff64795504e8dcb888ed25cbd6a50be) Green commit: `6f7e83a6` — two-line fix + test passes ## Test `test-issue-1843-node-qr-quiet-zone.js` — Node/vm harness that loads `public/vendor/qrcode.js`, grep-asserts both `nodes.js` call sites use margin ≥ 12px, renders the SVG, and checks the `<path>` `M` coordinates keep all dark modules ≥ 12px from every viewBox edge (and viewBox = `modules*cellSize + 2*margin`). Verified red on parent commit (test fails with margin=0 leaving dark modules at 0,0), green after fix (dark modules at 12,12 with 12px free border). ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` — all gates clean. Browser verified: pending — will validate on staging after CI. (Frontend UX bug; DOM/grep test above exercises the same SVG code path that scanners consume.) E2E assertion added: `test-issue-1843-node-qr-quiet-zone.js:63` (viewBox + `<path>` coord grep on real-rendered SVG). Fixes #1843. --------- Co-authored-by: corescope-bot <bot@corescope.dev> |
||
|
Kpa-clawbotandcorescope-bot
|
59cf5130d1 |
fix(1838): fold non-transport routes into scope-stats Unscoped (#1842)
Fixes #1838 ## Problem `/api/scope-stats` reported 100% scoped whenever any region was configured. Reporter noticed on a scopeless instance that "unscoped" was always zero — the pie visual is misleading to operators deciding on `denyf *`. ## Root cause `cmd/server/db.go:22` restricted the entire scope-stats denominator to `route_type IN (0, 3)`. Per firmware `docs/packet_format.md § Route Types`: - `0` = `TRANSPORT_FLOOD` - `1` = `FLOOD` - `2` = `DIRECT` - `3` = `TRANSPORT_DIRECT` Only routes 0 and 3 carry `transport_code_1` (transport-level scope). Routes 1 and 2 are inherently unscoped by protocol. The existing SQL was correct for the "how many transport-scopable routes are actually scoped" question, but the denominator was silently promoted to "all traffic" in the UI. Bonus: the comment on `routeTypeTransportSQL` labelled routes 0+3 as "FLOOD (0) and DIRECT (3)" — wrong on both counts. ## Fix - `cmd/server/db.go` — corrected the `routeTypeTransportSQL` comment; added `routeTypeNonTransportSQL = "route_type IN (1, 2)"` alongside it. - `GetScopeStats` runs a second `COUNT(*)` over `route_type IN (1,2) AND first_seen >= ?` and folds that count into `Summary.Unscoped`. Same index path as the existing query — one extra scan per `/api/scope-stats` call (cached 30s per triage's carmack finding). - `public/analytics.js` — Scopes tab header explains the denominator (all observed transmissions) and which route types carry scope. Card notes now render `X% of all traffic` for Scoped/Unscoped and `X% of scoped` for Unknown Scope so the pie's denominator is explicit. ## TDD - Red: `5554ffe4` — extended `TestGetScopeStats` + `TestHandleScopeStats` with `route_type=1` and `route_type=2` rows and asserted `Unscoped = 3` (1 transport-NULL + 2 non-transport). Ran the tests and confirmed assertion failure (`Unscoped = 1, want 3`). - Green: `ebbb9253` — implementation + label copy. Full `go test ./cmd/server/...` passes (54s). ## Preflight overrides - check-branch-clean: justified — cross-stack fix by design (backend semantics change + matching frontend label copy). All 4 files are exactly the surface the triage comment identified. ## Verification - `go test ./cmd/server/...` — 54s, all pass. - Firmware confirmation: `firmware/docs/packet_format.md:20-24` (route type table). ## Files touched - `cmd/server/db.go` — comment fix + second COUNT query. - `cmd/server/db_test.go` — extended fixture. - `cmd/server/routes_test.go` — extended fixture + isolate from seed data. - `public/analytics.js` — labels and header copy. --------- Co-authored-by: corescope-bot <bot@corescope.dev> |
||
|
|
d60188e481 |
refactor(1828): split handleObserverAnalytics into 5 helpers + byTxID fast-path (#1839)
## Summary Phase A of #1828: extract the 5 aggregate builders in `handleObserverAnalytics` into pure helpers in a new `cmd/server/observer_analytics.go`. Handler becomes a snapshot + filter + 5 composed calls. Also adopts the `byTxID` direct-read in `buildPacketTypes` (issue body's core observation): the payload-type histogram no longer allocates a full `enrichObs` map + interface-boxed fields just to read `tx.PayloadType`. That's the ~90% perf win the triage called out. Scope is exactly Phase A per the second triage comment. Phase B (sub-endpoints, caching, SQL migration) is deferred to a follow-up. ## Byte-identical output - Timeline / NodesTimeline: same key set, same sort, same labels. - PacketTypes: same keys/counts. Both legacy (`enriched["payload_type"].(int)`) and new (`tx.PayloadType == nil` guard) skip obs whose tx is missing or `PayloadType` is `nil`. - SnrDistribution: same 2-unit floor bucketing (negative-side rounding preserved), same ascending sort. - RecentPackets: still the first 20 enriched observations (`enrichObs` kept only here, where the extra fields are actually needed). ## TDD - Red commit: `9dc62f43` — 7 unit tests fail on assertions (not build errors) against stubs. - Green commit: `8d41011d` — implementations + handler rewire. All new tests + existing `TestObserverAnalytics*` handler tests pass. ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → clean (all 8 hard gates + 3 warnings pass). ## Non-goals - No new endpoints. - No SQL migration. - No public API signature change. - Snapshot count unchanged (still one under RLock, per #1481 P0-2). Fixes #1828. --------- Co-authored-by: fix-1828-bot <bot@corescope.local> Co-authored-by: clawbot <bot@corescope> |
||
|
Kpa-clawbotandopenclaw-bot
|
9d47a4adea |
fix(1836): normalize pubkey case on observer↔node cross-nav links (#1837)
Fixes #1836. Observer↔node cross-nav links from #1826 land on 404 because pubkeys are stored lowercase in `nodes` and uppercase in `observers`, and the backend `WHERE` lookups are case-sensitive. The two link builders now normalize case at the boundary. ## Fix - `public/observer-detail.js`: observer → node href passes `currentId.toLowerCase()`. - `public/nodes.js`: node → observer href passes `n.public_key.toUpperCase()`. ## TDD - Red: `test-issue-1836-crossnav-case-normalization.js` asserts the two hrefs contain `.toLowerCase()` / `.toUpperCase()`. Fails on master. - Green: 2-line production change makes the test pass. Scope: 2 production line edits + 1 new test file. --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Michael J. ArcanandWaydroid Builder
|
d2ef624c2e |
feat(api): flood_advert_count_7d on the node detail endpoint (#1831)
Adds, per node, how many distinct FLOOD adverts it originated in the last 7 days. Zero-hop adverts (route_type DIRECT) are excluded, so a nearby observer hearing a node's cheap local adverts does not inflate the number - the existing advert_count mixes both kinds and cannot tell a chatty flooder (mesh-wide airtime) from the recommended 240-minute zero-hop cadence (local only). Consumers (the ArcScope repeater advisor) rate advert hygiene against the community practice of one flood advert every ~49h; with the mixed total, a correctly configured repeater looked chatty whenever an observer sat within zero-hop range. Implemented like the relay-liveness fields: a pure, unit-tested counter over (first_seen, route_type, hash) entries with the same timestamp parsing and hash dedup, fed by a from_pubkey-indexed query capped at the 2000 most recent advert rows. The flood route-type constant is named advertRouteTypeFlood so this merges independently of the open unscoped-relay PR (#1823). --------- Co-authored-by: Waydroid Builder <build@waydroid.local> |
||
|
|
4f7bb245d4 |
fix(#1833): pin legend toggle button above VCR bar on Live view (#1834)
## Summary Fixes #1833 — the `.legend-toggle-btn` (palette icon) on the Live view was hardcoded to `bottom: 1rem`, so on typical desktop viewports it sat underneath the VCR playback bar and was unreachable. The reporter had to hide the legend via `localStorage` to work around it. ## Fix `public/live.css:1313` — one-character-class change: ```diff .legend-toggle-btn { position: fixed; - bottom: 1rem; + bottom: calc(var(--vcr-bar-height, 58px) + 10px); right: 1rem; ``` Mirrors the existing pattern already used by every other bottom-pinned Live overlay: - `.live-feed` (live.css:1204) - `.live-overlay[data-position="br"]` (live.css:1372) - `.feed-show-btn` (live.css:903) `--vcr-bar-height` is maintained by the ResizeObserver on `.vcr-bar`, so the button now tracks bar growth (mobile two-row layout, safe-area-inset) instead of overlapping. Same class of regression as #685 / #1206 / #1107 — an overlay that was missed in the prior sweep. ## TDD - Red commit `f6a938b7`: `test-issue-1833-legend-toggle-vcr-offset.js` asserts `.legend-toggle-btn`'s `bottom` declaration references `var(--vcr-bar-height`. Fails on assertion (not build error) against the hardcoded `1rem`. - Green commit `e399092b`: the CSS one-liner above. All 3 assertions pass. Grep-based CSS assertion per AGENTS.md § "E2E-DOM-grep exemption" — there is no existing Playwright test in this area that measures the toggle-button rect against `--vcr-bar-height`, and standing one up would be disproportionate for a single-property fix that mirrors three existing, tested overlay patterns. ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` — all hard gates pass, no warnings. ## Browser verified Not required per fix-issue skill (CSS-only, mirrors three existing tested patterns). Staging will pick up the change on merge; visual regression will be caught if the mirrored pattern breaks (grep test in this PR + existing E2E tests on the sister overlays). --------- Co-authored-by: corescope-bot <bot@corescope.local> Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Michael J. ArcanandWaydroid Builder
|
56fe844871 |
test: dedupe the unscoped-relay tests via a shared fixture (#1832)
Follow-up to #1823: TestRepeaterUnscopedRelayCount and its _Bulk twin were ~30 verbatim lines apart (DB, node insert, store seeding, assertions), differing only in the lookup under test - seeding changes had to land twice. Both now use a shared seedUnscopedRelayFixture + assertUnscopedCounts and contain only their respective lookup call. No behaviour change; the relay-liveness suite passes. Co-authored-by: Waydroid Builder <build@waydroid.local> |
||
|
Michael J. ArcanandWaydroid Builder
|
bd0a58e14c |
feat(api): add unscoped_relay_count_24h per-node field (#1823)
## What Adds a per-node API field `unscoped_relay_count_24h` on repeater/room nodes: the number of the node's 24h relay-hops that were unscoped floods (route_type == ROUTE_TYPE_FLOOD). A strict subset of relay_count_24h. ## Why A well-configured repeater runs `flood.max.unscoped 0` and should not rebroadcast unscoped floods — each one is re-sent by every repeater that hears it, so one packet turns into mesh-wide traffic. Exposing this lets clients (the ArcScope repeater advisor) detect and flag that base-config problem from observed packets. ## How Computed like relay_count_24h in both paths (bulk /api/nodes + per-node detail) with a route_type==FLOOD filter; reuses the byPathHop index, no migration. Wired into both handlers + OpenAPI schema + unit tests (per-node and bulk). Co-authored-by: Waydroid Builder <build@waydroid.local> |
||
|
Kpa-clawbotandmeshcore-bot
|
ba68069c23 |
fix(#1825): add cross-nav links between observer and node detail pages (#1826)
Adds cross-navigation between the observer detail page and the node
detail page for the same pubkey (community feature request from
cwichura).
**Changes**
- `public/observer-detail.js`: new `<a
href="#/nodes/${encodeURIComponent(currentId)}">View node detail →</a>`
inside `.page-header`, next to the `<h2 id="obsTitle">`.
- `public/nodes.js`: new sibling `<a
href="#/observers/${encodeURIComponent(n.public_key)}"
class="btn-primary">Observer →</a>` in the same button row as the
`Analytics` / `Reach` anchors on the full node detail page. Uses the
existing `ph-eye` phosphor icon.
**Test — TDD red→green**
- Red commit: `8c2315e1` (`test(#1825): red — observer<->node cross-link
anchors missing`) — 4/4 assertions fail on master; CI RED.
- Green commit: `ff8f6ed7` — minimum production change; 4/4 assertions
pass locally.
Test file: `test-issue-1825-observer-node-cross-links.js` —
static-source DOM-grep style consistent with the neighbouring
`test-issue-1789-observer-firmware-cols.js` /
`test-observers-headings.js` pattern. It asserts:
1. observer-detail.js contains
`href="#/nodes/${encodeURIComponent(currentId)}"`.
2. That anchor sits inside the `.page-header` block.
3. nodes.js contains
`href="#/observers/${encodeURIComponent(n.public_key)}"`.
4. That anchor is a sibling of the analytics/reach anchors in the same
flex row.
**Notes**
- Pubkeys are `encodeURIComponent`-escaped on both sides
(defense-in-depth; MeshCore pubkeys are hex only).
- No API changes. No CSS changes. No new dependencies.
Fixes #1825
---------
Co-authored-by: meshcore-bot <meshcore-bot@users.noreply.github.com>
|
||
|
|
096e16409c |
fix(#1741): wrap test-DB insert loops in a single transaction (#1819)
## Fixes #1741 `TestBoundedLoad_OldestLoadedSet` (and any test building a 5000-row fixture) hung/timed out, blocking reliable `go test ./cmd/server` and CI. ## Root cause The four test-DB builders in `cmd/server/bounded_load_test.go` (`createTestDBAt`, `createTestDBWithObs`, `createTestDBWithAgedPackets`) inserted rows in a loop with no `BEGIN`/`COMMIT`. With the pure-Go `modernc.org/sqlite` driver every `Exec` auto-commits → one fsync per row → ~2N fsyncs for N transmissions (tx + obs). At `numTx=5000` that's ~10k fsyncs and the fixture blows past the test timeout. Sibling tests with `numTx<=3000` happened to stay under the timeout, so only the 5000-row cases visibly hung. ## Fix Wrap each insert loop in a single `BEGIN`/`COMMIT` so the whole fixture build becomes one commit. Fixtures now finish in well under a second regardless of `numTx`; the tests' actual assertions (`oldestLoaded` set, newest-first ordering, bounded load) are exercised instead of the timeout masking them. Also made the prepared-statement `Exec` calls check their error (previously discarded) so a failed insert surfaces instead of silently leaving the DB short. No production code changed — test infrastructure only. ## Verified - `TestBoundedLoad_OldestLoadedSet`: **0.18s** (was: 30s timeout / FAIL). - Full `TestBoundedLoad*` + retention group: passes in ~1.2s. - `go test ./...` in `cmd/server`: exit 0 (no longer blocks on this test). Co-authored-by: Waydroid Builder <build@waydroid.local> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
6a32ec2b2d |
fix(#1729): preserve firmware-default Public channel (0x11) in analytics (#1817)
## Fixes #1729 The firmware-default **Public** channel (channel-hash byte `0x11` = 17) was rendered as an opaque **"Encrypted (0x11)"** row at the bottom of the analytics Channels tab, despite the key being well-known and builtin. ## Root cause `computeAnalyticsChannels` applied the #978 rainbow-table validation (`SHA256(SHA256("#name")[:16])[0]`, the **hashtag** hash scheme) to every decoded channel name. The Public channel is a **PSK** channel whose hash byte is key-derived (`SHA256(key)[0]` = 17), not hashtag-derived (`186` for `#Public`). So the ingestor-decoded name `"Public"` failed the hashtag check and was discarded, the row forced to `encrypted=true, name="ch17"`. ## Fix Trust the ingestor's `decryptionStatus`. The ingestor already persists `decryptionStatus:"decrypted"` when it decoded a packet with a real key (PSK), and `"no_key"` / `"decryption_failed"` otherwise. When the packet is `decrypted`, skip the hashtag hash check and keep the name — it came from a key-based decryption, not a rainbow-table lookup. The #978 mismatch rejection still applies to non-decrypted packets, so rainbow-table collisions are still caught. Frontend needs no change: `encrypted=false, name="Public"` lands in the "Network" group (top), not "Encrypted". ## Tests - `makeGrpTx` gains `makeGrpTxWithStatus` companion to set `decryptionStatus`. - `TestComputeAnalyticsChannels_PublicChannelPreserved`: hash 17 / "Public" / `decrypted` → name stays `"Public"`, `encrypted=false`. - `TestComputeAnalyticsChannels_UndecryptedNameStillValidated`: a non-`decrypted` name failing the hashtag check is still downgraded to `ch17` (#978 regression guard). All channel-analytics tests pass; `go build ./...` clean. Co-authored-by: Waydroid Builder <build@waydroid.local> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
750b8742a7 |
fix(staging-compose): decouple in-container mosquitto from standalone broker (#1813)
Red commit: `3898dbc5` (verified locally — CI run URL pending)
## Problem
A standalone `mqtt-broker` container (`eclipse-mosquitto:2`) was
provisioned out-of-band on the staging VM. It now owns MQTT, is attached
to external docker network `meshcore-net`, and binds host port `8883`.
The current `docker-compose.staging.yml` still:
- Publishes `1883:1883` on the host (dead weight; conflicts the moment
the broker moves to that port).
- Defaults `DISABLE_MOSQUITTO=false`, so the in-container mosquitto
burns RAM and briefly contests the `mqtt-broker` docker DNS name on cold
start.
- Doesn't join `meshcore-net`, so the ingestor can't resolve
`mqtt-broker:1883` via docker DNS without manual surgery.
## Fix (`docker-compose.staging.yml` only)
1. Remove the `1883:1883` host port publish from `staging-go`.
2. Flip `DISABLE_MOSQUITTO` default from `false` to `true`. Operators
can opt back in with the env var.
3. Attach `staging-go` to both `default` and `meshcore-net`; declare
`meshcore-net` as `external: true` so the file never tries to
create/destroy operator state.
Healthcheck and Caddy/443 plumbing untouched (out of scope).
## Test added (TDD framing: Option A — Go shape-asserts)
`cmd/server/staging_compose_broker_test.go:1` adds four regex-based
assertions on the compose file shape:
- staging-go does **not** bind port `1883` in ANY form (quoted/unquoted
short form, or long-form `target: 1883` / `published: 1883`).
- `DISABLE_MOSQUITTO` uses the interpolated default form
`${DISABLE_MOSQUITTO:-true}` (preserves operator override). Bare literal
`true`, or a later `=false` override in the same env block, is rejected.
- Top-level `networks:` declares `meshcore-net` as `external: true`.
- `staging-go` attaches to `meshcore-net` via a real
`services.staging-go.networks:` sub-key (comment-stripped so an
in-comment example can't masquerade).
Regex (not YAML byte-equality) so cosmetic edits don't break the guard.
No new go module deps. Red commit `3898dbc5` fails all 4 assertions on
master. Green commit `38297ff4` makes them pass. Round-1 hardening
commit `9f7155e2` tightens the regexes (per adversarial + kent-beck
must-fixes) and was verified against master's YAML shape — all 4 tests
fail on `origin/master`'s compose, pass on branch, proving the tightened
regexes still gate a real regression.
## Risk
Low, with one intentional semantic change.
- **Semantic change (v3.7+):** `DISABLE_MOSQUITTO` in
`docker-compose.staging.yml` now defaults to `true`. This is a
**deliberate flip** — the standalone `mqtt-broker` container is now
authoritative on the staging host, and running the in-container
mosquitto alongside it wastes RAM and races the docker DNS name
`mqtt-broker` on cold start. Operators who want the pre-v3.7 shape
(in-container mosquitto + host-published `1883`) must explicitly opt
back in via env override AND re-add the `1883:1883` port mapping
(concrete snippet is inline in the compose file and in `DEPLOY.md` under
"Standalone MQTT broker (staging)"). This intent is called out in a
`SEMANTIC CHANGE (v3.7+)` header comment at the top of
`docker-compose.staging.yml`.
- **Deploy prereq:** the external `meshcore-net` docker network MUST
already exist on the host before `docker compose up`. If it doesn't,
compose refuses to start `staging-go`. This is documented inline in the
compose file (with the `docker network create meshcore-net` one-liner)
and in `DEPLOY.md`.
- **Only takes effect where the standalone broker is deployed** — which
it already is on staging today. The legacy `DISABLE_MOSQUITTO=false`
path remains reachable via env override; the ingestor's upstream config
is untouched.
Partial fix — no tracking issue; follow-up to operator-side broker
provisioning.
---------
Co-authored-by: corescope-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
|
||
|
|
fa15ab0a30 |
fix(#1809): gate background loader on LoadChunked completion (#1811)
Partial fix for #1809. Red commit: |
||
|
|
242c7c609b |
fix(mqtt): escalate persistent paho disconnect + recover from emit panic + expose watchdog tick (#1749) (#1810)
# Partial fix for #1749 — MQTT watchdog escalation + panic recovery +
tick exposure
Red commit:
|
||
|
|
b74a64ccfa |
fix(ui): canonical payload label map across packets/live/packet-filter (#1799) (#1804)
## Summary Replaces the three drifted per-surface payload-type label vocabularies with a single canonical map keyed by firmware enum name. Per the locked triage comment on #1799 ([comment-4823975431](https://github.com/Kpa-clawbot/CoreScope/issues/1799#issuecomment-4823975431)): > Create `public/payload-labels.js` exporting `{GRP_DATA: {short:'Group Data', long:'Group data packet', enumId:6}, ...}`. Migrate `packets.js typeMap`, `packet-filter.js FW_PAYLOAD_TYPES`, `live.js TYPE_COLORS legend` to consume it. E2E that scrapes each surface and asserts label equality. ## Changes - **`public/payload-labels.js`** (new) — canonical map exposed as `window.PayloadLabels` and `window.PayloadLabelsApi`. Keys are firmware enum names; values carry `{short, long, enumId}` plus derived `SHORT_BY_ID` / `FW_PAYLOAD_TYPES` / `TYPE_ALIASES` for legacy callers. - **`public/packets.js`** — `TYPE_NAMES` + `typeMap` now read from `PayloadLabelsApi.SHORT_BY_ID`. Literal kept only as a defensive fallback for the case where the script tag fails to load. - **`public/packet-filter.js`** — `FW_PAYLOAD_TYPES` + `TYPE_ALIASES` now sourced from `PayloadLabelsApi`. Literal fallback retained so `node test-packet-filter.js` still works headlessly. - **`public/live.js`** — legend `<li>` rows are now generated from `window.PayloadLabels` in stable order, killing the third-vocabulary `Message — Group text` / `Direct — Direct message` drift the #1797 review surfaced. - **`public/index.html`** — `<script src="payload-labels.js">` loaded before `roles.js` / `packet-filter.js` / `packets.js`. - **`test-issue-1799-label-vocab-e2e.js`** (new) — Playwright E2E. Scrapes `#liveLegend` rows and the `/packets` type-filter checklist, asserts each label matches `window.PayloadLabels[ENUM].short` for `TXT_MSG`, `GRP_TXT`, `GRP_DATA`. Also verifies `window.PacketFilter` still recognises the enum names. - **`.github/workflows/deploy.yml`** — wired the new E2E into the existing Playwright block. ## TDD trail - Red commit `eb392d4` — adds the failing E2E only (asserts `window.PayloadLabels` exists and labels match; both fail). - Green commit `44e902a` — introduces the canonical map and migrates the three surfaces. ## Verification - `node test-packet-filter.js` — 92/92 pass with the new fallback wiring. - Preflight: `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` — clean. Browser verified: E2E `test-issue-1799-label-vocab-e2e.js` exercises `/live` legend + `/packets` type filter against a Playwright headless Chromium; CI's Playwright block runs it on every push. E2E assertion added: `test-issue-1799-label-vocab-e2e.js:139` — `assert(fromLegend === canon, ...)` and `assert(fromPackets === canon, ...)` per enum. Fixes #1799 --------- Co-authored-by: mc-bot <bot@corescope> Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@kpa.com> Co-authored-by: clawbot <bot@clawbot.local> |
||
|
Michael J. ArcanandWaydroid Builder
|
4654ce3386 |
feat(analytics): "My Repeaters" favorites monitoring dashboard (#1761)
My Repeaters monitoring dashboard. Closes #1765. --------- Co-authored-by: Waydroid Builder <build@waydroid.local> |
||
|
|
30e4151f7a |
fix: neighbor-graph tab never renders after filtering down (#1758)
The Analytics → Neighbor Graph tab fetches the full (uncapped) graph and, when it exceeds NODE_LIMIT (1000), skips the force simulation with a "use filters to reduce the node count" notice. But filtering never actually re-enabled rendering: - the node-count guard tested _ngState.allNodes (the immutable full fetched set, assigned once in createGraphState and never reassigned) instead of the displayed/filtered _ngState.nodes, so its verdict was fixed at load time; - the entire draw loop lives in startGraphRenderer(), which ran exactly once at load and was never called from applyNGFilters(), so a filter change updated the node/edge arrays and stat cards but never un-hid the canvas or scheduled an animation frame -> the graph stayed blank no matter how few nodes remained. This explains both reported symptoms (selects too many nodes initially AND stays broken once restricted to fewer). Fix: make the render lifecycle filter-aware. - startGraphRenderer() now guards on the displayed set (_ngState.nodes), cancels any running rAF loop before re-deciding, toggles the canvas plus a stable-id "skipped" notice, and restarts cleanly (no double loops). - applyNGFilters() calls startGraphRenderer() so every filter change re-evaluates the guard and (re)starts or stops the loop. - the initial render now goes through applyNGFilters() so the first paint already respects the default filters (observers unchecked, saved min-score) instead of dumping the full fetched graph. Test: `node --check public/analytics.js` passes. Manually: open Analytics → Neighbor Graph on a mesh with >1000 nodes → the "skipped" notice shows; tighten filters (min-score up / roles off) below 1000 → the graph now renders (was blank before); loosen again → notice returns. Frontend-only change (`public/analytics.js`); no backend/API change. --- **TDD note (review round 1):** Single-commit community bug-fix on an existing UI surface (no "net-new UI" exemption). The e2e `test-issue-1758-ng-filter-rerenders-e2e.js` is the red→green gate — it fails on `origin/master` (the renderer kept the node-count guard on the full fetched graph and never un-hid the canvas) and passes with the fix. Per AGENTS.md the separate red/green-commit *form* is a bot rule, not a contributor gate. --------- Co-authored-by: Waydroid Builder <build@waydroid.local> Co-authored-by: Waydroid Builder <claude@michael.arcan.de> |
||
|
Michael J. ArcanandWaydroid Builder
|
9ae547ed7b |
test: de-flake distance-202 and anchor-bias tests (deterministic timing) (#1808)
Two server tests flaked intermittently and reddened CI on unrelated (frontend) PRs that merged master: - TestDistanceConcurrentRequestsDuringBuildReturn202 asserted all 10 concurrent requests get 202 'during the build window', but the lazy distance build on the tiny test DB finishes almost instantly, so on a fast machine some requests raced past it and got 200 (~50% flake). Add a nil-by-default distanceBuildHook seam on PacketStore (zero overhead in prod) that the test uses to hold the build open until all requests have been served — making the window guarantee deterministic. - TestHandleNodePaths_AnchorBiasInconsistency_Issue1278 queried /paths right after store.Load(), racing the path-hop index that Load() builds in a background goroutine (#1008); the membership/canonical result was thus non-deterministic (rarer flake, worse under suite load). Wait for PathHopIndexReady() before querying. Both run 30x green and pass -race. No production behavior change (hook is nil). Co-authored-by: Waydroid Builder <claude@michael.arcan.de> |
||
|
Kpa-clawbotandopenclaw-bot
|
ec0ebeda2f |
fix(#1793): WebSocket CheckOrigin allowlist (block cross-origin scrapers) (#1795)
## Summary Closes the wide-open `/ws` WebSocket upgrader (`CheckOrigin: return true`) that lets any browser origin scrape live packet data. Replaces it with an explicit allowlist consulted from `cfg.CORSAllowedOrigins`, plus an implicit same-origin allowance and an empty-Origin (non-browser client) allowance. Fixes #1793. ## Rules (`Hub.checkOrigin`) - Empty `Origin` header → **allow** (non-browser clients; per-IP rate/deny gating tracked separately in #1794). - `Origin` host == request `Host` (case-insensitive) → **allow** (same-origin). - `Origin` matches an entry in `cfg.CORSAllowedOrigins` by exact case-insensitive match → **allow**. - `"*"` in `cfg.CORSAllowedOrigins` is **deliberately ignored** for `/ws`. A startup `[ws] WARNING:` is logged once when present. - Anything else → **reject** (gorilla returns 403). ### Deliberate divergence from CORS XHR CORS XHR (`corsMiddleware`) still honors `"*"` for read-only cross-origin GETs. The `/ws` upgrade does NOT, per OWASP's WebSocket Security Cheat Sheet: > Use an allowlist, not a denylist. Avoid wildcards or substring matching. — https://cheatsheetseries.owasp.org/cheatsheets/WebSocket_Security_Cheat_Sheet.html `"*"` on the WS path would re-open the exact CSWSH/scraping vector this PR closes, so it is rejected with a startup warning rather than silently honored. This intentional asymmetry is documented in the updated `_comment_corsAllowedOrigins` in `config.example.json`. ## TDD red → green - `e5974c6a` **RED** — adds `cmd/server/websocket_checkorigin_test.go` with five cases; `SetAllowedOrigins` introduced as an enforcement stub so the test compiles and fails on the assertion (CI fails on this commit by design). - `a4791dc3` **GREEN** — implements `Hub.checkOrigin`, wires `SetAllowedOrigins` from `main.go`, updates the config example. All tests pass. ## Tests added (`cmd/server/websocket_checkorigin_test.go`) - `TestCheckOriginRejectsForeignOrigin` — foreign Origin → 403 - `TestCheckOriginAllowsEmptyOrigin` — non-browser client → 101 - `TestCheckOriginAllowsSameHost` — same-origin → 101 - `TestCheckOriginAllowsAllowlistedOrigin` — exact allowlist match → 101 - `TestCheckOriginWildcardDoesNotAllowForeignOrigin` — `"*"` in allowlist still rejects foreign origin → 403 ## Files changed - `cmd/server/websocket.go` — `Hub.allowedOrigins`, `SetAllowedOrigins`, `checkOrigin`, wired into `Upgrader.CheckOrigin`. - `cmd/server/main.go` — `hub.SetAllowedOrigins(cfg.CORSAllowedOrigins)` at the single call site. - `cmd/server/websocket_checkorigin_test.go` — new test file. - `config.example.json` — updated `_comment_corsAllowedOrigins` to document `/ws` gating and the `"*"` divergence. ## Out of scope (follow-up) - **#1794** — per-IP rate limit / deny list / connection cap for non-browser clients (which still bypass Origin because they don't send one). Layered defense; not in this PR. ## Verification - `go test ./cmd/server/...` — all server tests pass locally (574s). - Preflight clean (`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`). --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Michael J. Arcan
|
ae2e3933dd |
feat(server): store memory diagnostics + drop redundant obs.RawHex (#1773)
Drops the redundant per-observation RawHex (~98MB on a live store; reader already falls back to tx.RawHex #881) and adds an opt-in /api/perf?mem=1 memory breakdown (flood-forward share + per-component bytes). Profiled against a live instance. **Savings substantiation:** live-instance profiling shows ~1.66M observations in the store, each previously carrying its own per-observation `raw_hex` (avg ≈118 hex chars ≈59 bytes) that exactly duplicates the parent transmission's `raw_hex`. Dropping the duplicate on every load/ingest path eliminates ≈98 MB of redundant in-memory storage plus ~1.66M string allocations, with no data loss — the read path (`enrichObs`) already falls back to `tx.RawHex` when `obs.RawHex` is empty (verified by the new safety-gate test). The patched build cannot be run against the live instance here; instead the new opt-in `/api/perf?mem=1` diagnostic lets operators measure the real before/after (`trackedMB` and the per-component breakdown) directly after deploy. |
||
|
|
707d70c738 |
fix(packets): clamp .col-details to one line on mobile (#1770 S path) (#1805)
## Summary Partial fix for #1770 (S quick-fix path only; L refactor remains as follow-up). The packets-view virtual-scroller assumes a constant `VSCROLL_ROW_HEIGHT`, but the base rule at `public/style.css` L1097 lets `td.col-details` wrap on narrow viewports (`white-space: normal; word-break: break-word`). Wrapped rows produce variable row heights → visible jitter when scrolling past ~900px on iOS. **Quick-fix (S path):** under the existing `@media (max-width: 640px)` block in `public/style.css`, clamp `.col-details` to a single line: ```css .data-table td.col-details { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; } ``` Trade-off accepted in triage: Details column truncates on mobile in exchange for smooth scrolling. The base rule keeps wrapping on desktop (≥641px) so nothing changes there. **Out of scope:** the full L-path fix (per-row measurement, `_rowHeightsPx[]`, cumulative offsets, re-measure on hop-resolver finalize) — tracked separately on #1770. ## TDD - **Red commit** `7f58bedc` — adds `test-issue-1770-mobile-row-clamp.js`, a CSS-grep test (same pattern as `test-issue-1364-pill-no-clamp.js`) that walks every `@media (max-width: 640px)` block in `public/style.css` and asserts a `.col-details` rule declares `white-space: nowrap`, `overflow: hidden`, and `text-overflow: ellipsis`. Verified to FAIL on master (assertion failure, not a parse error) and PASS after the CSS change. - **Green commit** `d46271b8` — applies the 5-line CSS clamp inside the existing mobile breakpoint at L2362. ## Files touched - `public/style.css` (+13) - `test-issue-1770-mobile-row-clamp.js` (+101, new) ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → all gates pass (PII, branch scope, red commit, css-vars, css self-fallback, LIKE-on-JSON, sync migration, async-migration, XSS). No warnings. --------- Co-authored-by: clawbot <bot@clawbot.local> Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Kpa-clawbotandclawbot
|
b3189c613a |
fix(#1802): decode CONTROL DISCOVER_REQ/RESP subtype + body fields (#1806)
## Summary Extend CONTROL packet decoding to surface DISCOVER_REQ / DISCOVER_RESP subtype plus body fields in the packet detail view. Previously only the byte0 zero-hop flag was decoded; the body was rendered as opaque hex. ## What changed **Backend** — `cmd/ingestor/decoder.go` `decodeControl()` - New `Payload` fields (all omitempty): `CtrlSubtype`, `CtrlFilter`, `CtrlTag`, `CtrlSince`, `CtrlNodeType`, `CtrlSNR`, `CtrlPubKey`. - Subtype derived from `byte0 & 0xF0`: `0x80` → `DISCOVER_REQ`, `0x90` → `DISCOVER_RESP`, otherwise `UNKNOWN`. - REQ body parsed when `len(buf) >= 6`: `filter:u8 | tag:u32 LE`, plus optional `since:u32 LE` when 4 more bytes remain. - RESP body parsed when `len(buf) >= 6`: `node_type` (low nibble of byte0), `snr:i8`, `tag:u32 LE`, and `pubkey` hex — 32 bytes when full, 8 bytes when prefix-only. - Every field gated on length; short/truncated bodies emit subtype only and never panic. - `CtrlZeroHop` retained for backwards compatibility (rename flagged for follow-up per triage). **Frontend** — `public/packets.js` `getDetailPreview()` - New `decoded.type === 'CONTROL'` branch renders subtype + present body fields (filter / tag / since / node_type / snr / pubkey). Each field shown only when populated, so truncated CONTROL still gets a subtype label. ## Wire format reference - `firmware/src/Mesh.cpp:69` — `CTL_TYPE_NODE_DISCOVER_REQ=0x80`, `CTL_TYPE_NODE_DISCOVER_RESP=0x90`. - `firmware/examples/simple_repeater/MyMesh.cpp:773-820` — body parse / build. ## Tests (red → green, per AGENTS.md STRICT TDD) - `cmd/ingestor/issue1802_test.go` — 6 cases: REQ full body (with since), REQ no-since, RESP 32B pubkey, RESP 8B prefix pubkey, RESP truncated pubkey (no panic, no pubkey emitted), short body (subtype only), unknown subtype. Red commit `43713d3a` → green commit `d4b28180`. Pre-existing CONTROL tests (`TestDecodeControlZeroHop`, `TestDecodeControlMultiHop`) still pass. - `test-packets.js` — 3 cases on `getDetailPreview`: DISCOVER_REQ (filter+tag rendered), DISCOVER_RESP (snr+pubkey rendered), UNKNOWN subtype label. Red commit `be23e349` → green commit `845d6c48`. ## Preflight overrides - `check-branch-clean` (cross-stack): justified — issue #1802 explicitly spans backend decoder (`cmd/ingestor/decoder.go`) and frontend renderer (`public/packets.js`) per triage comment. Tests in both layers. Single-purpose PR. ## Scope discipline Files touched: `cmd/ingestor/decoder.go`, `cmd/ingestor/issue1802_test.go`, `public/packets.js`, `test-packets.js`. No other files. No firmware changes. No `cmd/server/decoder.go` changes. No `CtrlZeroHop` rename (deferred per triage). Fixes #1802 --------- Co-authored-by: clawbot <bot@meshcore.local> |
||
|
Kpa-clawbotandopenclaw-bot
|
120ac052d3 |
fix(packets): add Multipart/Control/Raw Custom to type filter checklist (#1798) (#1803)
## Summary Fixes #1798. Extends the Packets-page `typeMap` in `public/packets.js` to include three firmware payload types that were previously missing from the multi-select checklist: - `10` — Multipart - `11` — Control - `15` — Raw Custom Other surfaces (`public/packet-filter.js` `FW_PAYLOAD_TYPES`, `public/live.js` `TYPE_COLORS`, `public/map.js`) already knew about these types; only the Packets-page checklist UI omitted them, forcing operators to hand-type filter expressions to filter on them. ## Red → green - Red commit: `359e3645ac41506e563c19dfbd49983fb4ec9638` — adds E2E that opens `#typeMenu` and asserts each new `data-type-id="10|11|15"` checkbox renders with the exact label. Fails on assertion (DOM selectors return null) against the pre-fix `typeMap`. - Green commit: `f484e8cb88659b14fed7aa7fefcdfb3f0eb6c186` — single-line literal extension; test goes green. ## E2E assertion added `test-e2e-playwright.js:576` — `Packets type filter includes Multipart/Control/Raw Custom (#1798)` (asserts the three new `data-type-id` checkboxes render with their exact labels in the rendered Packets-page checklist DOM). ## Files touched - `public/packets.js` — extend `typeMap` literal - `test-e2e-playwright.js` — new E2E test asserting the three checkboxes render ## Browser verified E2E test scrapes the rendered Packets-page DOM via Playwright; CI runs it against the local Go server fixture in the `e2e-test` job. Fixes #1798 --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Michael J. ArcanandWaydroid Builder
|
3efa37c46c |
feat(server): complete the #672 4-axis repeater usefulness score (#1762)
Adds Coverage (harmonic reach) + Redundancy (Tarjan articulation) axes + composite & grade. Closes #672. **TDD note (BLOCKER-1):** Community PR delivered as a single squashed commit, so there is no separate pre-fix failing-test commit — please accept as a community-PR exemption. The tests are *gating*, not just thorough: each axis test pins a specific topology outcome (coverage on line/star/disconnected/weight-sensitive; redundancy online/triangle/star/bridged-cliques), and an end-to-end `/api/nodes` surface test drives the whole pipeline and asserts the composite diverges from the Traffic axis. Inverting the `1/weight` distance, dropping the NaN/Inf reject, removing the `redundancyMinWeight` floor, or aliasing `usefulness_score` back onto `traffic_share_score` each break a specific assertion. The axis functions are pure (no hidden state), so the suite fully characterises the behavior without the red anchor. Co-authored-by: Waydroid Builder <build@waydroid.local> |
||
|
Kpa-clawbotandmeshcore-bot
|
5e096147e5 |
fix(#1800): add routed_through filter, clarify path, fix hex lexer error (#1801)
Fixes #1800. ## Three changes 1. **`routed_through` field** — new `FIELDS` entry. Resolves against `packet.resolved_path` (handles both the JSON-string form from `/api/packets/by-id` and the already-parsed-array form from `/api/packets`). Returns a space-joined lower-case hex string so `contains` / `starts_with` / `==` work the same way they already do for `hash`. 2. **`path` desc clarified + `path_prefixes` alias** — `path` desc now reads `Hop path as 1-byte prefixes joined (e.g. a3→7f). For pubkey search use routed_through.` `path_prefixes` is added as a discoverability alias and resolves to the same value. 3. **Lexer hex-token error** — when the number/duration tokenizer hits an unknown unit AND the slice (extended forward through any remaining `[0-9a-fA-F]`) is pure hex of length ≥ 4, the lexer now returns: ``` Hex value must be quoted: try 'field == "<hex>"' or use the starts_with/contains operator ``` instead of `Invalid duration unit 'f' at position N (expected s/m/h/d/w)`. The duration-unit error is preserved for non-hex cases (`age < 5x` still errors with the original message). ## TDD Red commit `e44ac00a` adds 7 assertions that fail with the unmodified code (proven by stashing the impl and re-running — output: `7 failed`). Green commit `7b623721` makes them pass. Tests added in `test-packet-filter.js` (`#1800: …`): - `routed_through starts_with "2f0b00"` matches packet with JSON-string `resolved_path` - same, against array-form `resolved_path` (handles real `/api/packets` shape) - `routed_through contains "<full-pubkey>"` matches - `routed_through contains "2f0b"` matches - `routed_through starts_with "deadbe"` does NOT match - `path 2f0b001247a047ca` → error contains `Hex value must be quoted` - regression: `path contains "a3"` still matches `path_json=["a3","7f"]` - `routed_through` listed in `FIELDS` - `path_prefixes` alias resolves like `path` `node test-packet-filter.js` → `=== Results: 92 passed, 0 failed ===` Sibling JS tests (`test-packet-filter-ux.js`, `test-packet-filter-time.js`, in-file self-tests) all green. ## Browser verification Browser tool was unavailable this session, so I executed `public/packet-filter.js` in a Node VM context (identical execution) and exercised it against a live `/api/packets?limit=200` response from staging: - `routed_through starts_with "41b1"` returned 1 matching packet (whose `resolved_path[0]` is `41b1eabc3c6e88997242051ee53fa5840761dff02ac5f6d9904f23985395ec31`) - `routed_through starts_with "bccf91"` matched a packet with that hop - `routed_through starts_with "deadbe"` matched nothing (correct) - `PF.suggest('routed_t', 8)` returned `['routed_through']` - `PF.suggest('route', 5)` returned `['route', 'routed_through']` - `PF.compile('path 2f0b001247a047ca').error` is verbatim: `Hex value must be quoted: try 'field == "<hex>"' or use the starts_with/contains operator` - `PF.compile('age < 5x').error` is still `Invalid duration unit 'x' at position 7 (expected s/m/h/d/w)` — duration-unit message preserved for non-hex cases. ## Out of scope (per issue) - No server-side filter pushdown. - No operator-list changes. - No `resolved_path` changes — it already ships on `/api/packets`, `/api/packets/by-id`, `/api/live`. --------- Co-authored-by: meshcore-bot <meshcore-bot@users.noreply.github.com> |
||
|
|
d5ceb27334 |
fix(#1792): decode GRP_DATA channel hash + inner data_type/len/blob in details cell (#1796)
Red commit:
|
||
|
|
770749a8ce |
fix(#1791): add 'Group Data' (payload_type=6) to packets type filter (#1797)
Fixes #1791. ## What Adds `6:'Group Data'` to the `typeMap` in `public/packets.js` so the Packets-view "message type" multi-select shows a Group Data checkbox. The filter pipeline already keys by integer payload_type, so this just registers the missing option. Also aligns the Live-view legend label in `public/live.js` to "Group Data" for cross-view consistency. ## Why Triage (in #1791) confirmed payload_type=6 (GRP_DATA) was the only ordinary type omitted from the static `typeMap`. `packet-filter.js`, `live.js`, `app.js`, and `map.js` all already know about it — only the Packets-page checklist was missing it. ## Test (TDD red → green) Branch history (4 production commits before round-1 review): - `19ed5beb` — **test-only red commit**: adds Playwright E2E that opens the type-filter menu, asserts a `data-type-id="6"` checkbox labeled "Group Data" exists, selects it, and asserts every visible row's type badge reads "Group Data". Also seeds one GRP_DATA packet into the CI fixture (`.github/workflows/deploy.yml`) so the filter has a row to match. - `823a7d8d` — adds the one-line `typeMap` entry. First CI run on this commit failed on an unrelated test (not the #1791 assertion); the #1791 test ran and passed. - `eec2428` — fixture cleanup: `path_json=[]`/`resolved_path=[]` so the seeded GRP_DATA hop-row count matches the raw_hex `path_len=0`. CI green. - `8f85f5f` — labels the type-6 entry "Group Data" (was briefly "Grp Data"). CI green. E2E assertion: `test-e2e-playwright.js` block `Packets type filter includes Group Data (#1791)`. ## Round-1 review follow-ups - `e3651c99` — `public/live.js` legend: `'Grp Data'` → `'Group Data'`. - `4475c2f7` — test cleanup hardening: error string aligned to assertion, duplicated selector extracted, regex tightened to strict equality, `#typeMenu` explicitly closed, `meshcore-time-window` localStorage key cleared, page reloaded so the in-memory `selectedTypes` Set is reset. - `b90bc33f` — `.github/workflows/deploy.yml`: drop self-referential `#1797` citation from fixture comment, switch synthetic fixture id from `-1` to `-1000000` sentinel with explanatory comment. ## Scope Single-line typeMap registration plus its E2E test scaffolding, fixture seed, and the live.js label alignment. --------- Co-authored-by: clawbot <bot@openclaw.dev> Co-authored-by: meshcore-bot <bot@meshcore.local> Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
|
17654dd090 |
docs(api): document per-node usefulness metrics in OpenAPI (#1769)
Documented Node schema (the four #672 usefulness axes + composite + A-F grade + relay fields) and response schemas on the node endpoints. Documentation-only; no behaviour change. Pairs with #1762 (documents the metrics it adds). Co-authored-by: Waydroid Builder <build@waydroid.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Kevin Cai
|
1adb0116b2 |
fix: don't re-render node dots when scrubbing the Live timeline (#1754)
## What Scrubbing the Live page timeline no longer re-renders all node dots. ## Why `vcrReplayFromTs()` ran `clearNodeMarkers()` (wiping `nodesLayer` and `nodeMarkers`) and then `loadNodes()` rebuilt every marker from scratch. A single scrub click destroyed and recreated the entire node layer; visible flicker plus unnecessary DOM work (even though `addNodeMarker()` already no-ops nodes that still exist). ## How - `vcrReplayFromTs()` now clears only the transient animation/path layers. - The time-scoped branch of `loadNodes()` reconciles against the existing markers: removes only nodes absent at the target time, adds genuinely new ones, leaves shared dots untouched. ## Testing - `test-live.js`: 95/95 pass - `node --check public/live.js` clean |
||
|
Michael J. Arcan
|
fc26fb6b3a |
feat(#1751): show transported region scopes in repeater sidebar (#1752)
Closes #1751. |
||
|
|
c03f2ebbcc |
live: cap animation-canvas DPR at 1.5 and redraw at ~60fps (#1737)
The live-map animation overlay re-clears and re-draws its full backing store every animation frame. Two unbounded multipliers make that expensive: 1. devicePixelRatio is uncapped in updateAnimCanvas(). The canvas is already ~1.4x the screen area (20% pad per side), so at DPR 2-3 it allocates and fills 5-12x the screen's pixels per frame. Cap at 1.5 — lines stay crisp, per-frame fill cost drops up to ~4x on hi-DPI displays. 2. renderAnimations() reschedules via rAF with no rate limit, so on 120/144Hz displays it does 2-2.4x the work for no visible gain. Add a ~60fps guard. Progress is time-based (tickDt, itself capped at 32ms), so skipping frames preserves motion exactly. Paused frames fall through to the existing sleep. No behavior change on a standard 60Hz / 1x-DPI display. Existing animation tests (test-live-dt-cap-1524, test-live-anims) unaffected. Co-authored-by: Michael <claude@michael.arcan.de> Co-authored-by: efiten <erwin.fiten@gmail.com> |
||
|
Kpa-clawbotandclawbot
|
9757178aad |
fix(#1789): add Firmware + Client columns to observers table (#1790)
## Summary Adds **Firmware** and **Client** columns to the observers table (`#/observers`). Both values already come back from `/api/observers` (`firmware`, `client_version`) — they were just never rendered. Fleet operators have been asking to sort/scan firmware versions to coordinate upgrades. Closes #1789. ## Changes - `public/observers.js` - Two new `<th data-priority="4" data-sort-key="...">` headers (Firmware, Client). Priority 4 matches Clock Offset / Uptime so `TableResponsive` hides them first on narrow viewports. - Two new `<td class="mono">` cells with `data-value="${escapeHtml(raw)}"` for sort and the rendered text escape-wrapped. - `truncateBuildSuffix()` helper trims the long `" Build: ..."` tail from firmware in the displayed text; the full string is preserved in `title=` for hover. - `test-issue-1789-observer-firmware-cols.js` — TDD red→green static-source regression test (same pattern as `test-observers-headings.js`). - `test-observers-headings.js` — updated expected heading list with the two new columns (existing #1039 invariant test). - `test-all.sh` — wires the new test into CI. ## TDD evidence - Red: `c6c4e594c1084d664730666ba069871ac6d9755c` — test commit fails on assertions (not import errors): 5/6 cases fail because the headers/cells/title attr don't yet exist; the column-count invariant still passes because both thead and tbody are unmodified. - Green: `02a0246a185950c903828ac1225d6795f5a3b2f4` — implementation; all 6 cases pass. ## Browser verified To be verified post-deploy on staging (`http://analyzer-stg.00id.net/#/observers`). No backend changes — purely additive frontend render of fields that are already on the wire. ## Perf No new API calls, no extra fetches, two extra template-literal cells per observer row (~10s of observers in prod). O(n) render unchanged. --------- Co-authored-by: clawbot <bot@meshcore.local> |
||
|
|
f0763aecce |
fix(#1726): clear stale "varies" hash size once a node settles (#1788)
Fixes #1726. ## Problem A MeshCore v1.16.0 repeater configured for 2-byte path hashes (`path.hash.mode=1`) — e.g. `36f6c7c7…` (`DK_3400_RAK_TEST`) — kept showing as **"varies"** / mixed 1-byte + 2-byte for the full 7-day advert window. Per the live data in the issue triage: of the node's ~20 recent adverts, exactly **one** (2026-06-09, across 15 distinct observer paths) was a genuine 1-byte flood advert; every other advert was 2-byte. The flip-flop heuristic in `computeNodeHashSizeInfo` weighs that stale advert equally with recent ones, so an operator who flips `path.hash.mode` mid-flight (or a single old 1-byte advert) stays flagged for the full window with no way to signal "the config is settled now." ## Fix Two coupled changes in `cmd/server/store.go` `computeNodeHashSizeInfo`: 1. **Chronological ordering.** `byPayloadType[4]` iterates in insertion order, not timestamp order, so `HashSize = Seq[last]` could pick the wrong advert under out-of-order MQTT ingest or chunked cold-load (the "carmack" concern from triage). We now collect `(FirstSeen, size)` pairs and **stable-sort by `FirstSeen`**; ties keep insertion order, preserving prior behavior when timestamps are equal. 2. **Recency decay.** After `transitions >= 2` raises the flip-flop flag, clear it when the most recent `hashSizeRecentAgreeCount` (= **3**) non-zero-hop adverts all agree on a single size. A node still flapping (recent adverts disagree) stays flagged. `3` mirrors the existing ≥3-observation threshold used to raise the flag. ## Policy note Triage marked this **needs-operator-input** because the decay is a behavior/policy change. This PR implements the rule the triage proposed ("if the last 3 adverts agree, clear inconsistent"), which matches the reporter's stated expectation. Happy to adjust the threshold or gate it differently per your call. ## Tests `cmd/server/issue1726_hash_decay_test.go`: - `TestIssue1726_SettledNodeNotInconsistent` — reporter's case (`[2,1,2,2,2]` within window) → `Inconsistent=false`, `HashSize=2`. - `TestIssue1726_HashSizeUsesChronologicallyLatest` — out-of-order insertion still reports the chronologically-latest size. - `TestIssue1726_ActiveFlapperStaysInconsistent` — a node whose recent adverts disagree stays flagged. Existing flip-flop / hash-collision tests unchanged and green; full `cmd/server` package suite passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Erwin Fiten <e.fiten@opteco.be> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
d437958474 |
fix(map): pin APC (Napa) and STS (Sonoma) observers (#1786) (#1787)
Fixes the map-coordinate gap in #1786. ## Problem Observers tagged with IATA code **APC** (Napa County) or **STS** (Charles M. Schulz–Sonoma County) render with no location and never pin on the map. ## Root cause `iataCoords` in `cmd/server/routes.go` is a hardcoded `IATA -> lat/lon` lookup used purely for placing observer/region markers on the map. It had no entry for APC or STS, so those observers had no coordinates to render with. This is **display-only**. Ingestion is not gated on these codes: `IsObserverIATAAllowed` (`cmd/ingestor/config.go`) short-circuits to `true` when the observer IATA whitelist is empty — which is the staging configuration. The reporter''s "packets disappear entirely" symptom is therefore **not** explained by this code path (likely an upstream `meshcoretomqtt`/broker topic issue; needs operator `mosquitto_sub` confirmation per triage). ## Fix - Add `APC {38.2132, -122.2807}` and `STS {38.509, -122.8128}` to `iataCoords`, matching the airports'' published coordinates. - Add a regression test (`TestIataCoordsIncludesNapaAndSonoma`) asserting both are present with the expected coordinates. ## Verification - `go test ./cmd/server/` — full package passes (`ok`). - `go vet ./cmd/server/` — clean. ## Scope note Checked the repo for other statically-enumerable region codes (`config.example.json` regions: SJC/SFO/OAK/MRY) — all already covered. The broader "are other in-use codes missing" question can only be answered against the live `cfg.Regions` + `db.GetDistinctIATAs()` set, which is operational, not in-tree. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Erwin Fiten <e.fiten@opteco.be> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |