mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-11 12:37:33 +00:00
efiten-patch-1
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Marek WojtaszekandClaude Opus 5.5
|
3100d7a4c9 |
fix(ui): route remaining payload label surfaces through payload-labels.js (#2373) (#2374)
Fixes #2373 ## Summary Finishes the #1799 / #1804 migration: every frontend surface that names a payload type now reads it from `public/payload-labels.js`. Node Analytics, Observer detail and Observer comparison showed GRP_DATA, MULTIPART and RAW_CUSTOM as `Type 6` / `Type 10` / `Type 15`. The map route sidebar had RAW_CUSTOM on id 12; the firmware uses `0x0F`. ## Changes - **`public/payload-labels.js`**: new `PayloadLabels.api.displayName(id)` returns the canonical short label, or `Type N` for ids the firmware does not define (e.g. a corrupted header byte stays visible as `Type 13`). It lives on `.api` only; the legacy root aliases are not extended. - **`public/node-analytics.js`, `public/observer-detail.js`, `public/compare.js`**: drop the local `PAYLOAD_LABELS` maps and use `displayName()`. - **`public/nodes.js`**: drop the local `PAYLOAD_TYPES` map; Recent Adverts uses `shortById()` and keeps its `Packet` fallback. - **`public/app.js`**: drop `PAYLOAD_TYPES`; `payloadTypeName()` uses `shortById()` and keeps its `UNKNOWN` fallback. - **`public/map.js`**: drop `PAYLOAD_TYPE_MAP`; `inferredType` uses `enumNameById()`, which fixes RAW_CUSTOM (15 → `RAW_CUSTOM` instead of `OTHER`). - **`public/packets.js`**: `typeName()` is now `displayName` (same output, one implementation). Consumers resolve labels at render time rather than at file load, so the many unit sandboxes that load these files without `payload-labels.js` keep working. `index.html` already loads `payload-labels.js` before all of them. ## Tests - **New `tests/unit/test-payload-labels-surfaces.js`** (listed in `test-all.sh`): - drift guard: fails if any `public/*.js` other than `payload-labels.js` pairs a numeric id with a short label or firmware enum name (it caught the `map.js` copy, which was not on the original list); - `displayName()` for every enum, string ids, and the `Type N` fallback; - loads the real `node-analytics.js` in a vm sandbox and asserts the doughnut labels for ids 6, 10, 15 and 13. - Before the fix: 11 failures. After: 82 passed. - **`tests/e2e/test-issue-1799-label-vocab-e2e.js`**: two new steps intercept `/api/nodes/:pk/analytics` and `/api/observers/:id/analytics` (the fixture has no GRP_DATA in the store), inject ids 6, 10, 15 and 13, and read the labels from `Chart.getChart()`. Against upstream `master`: 11 passed, 2 failed (`["Type 6","Type 10","Type 15","Type 13"]`). With this branch: 13 passed. - **Sandbox updates**: `test-compare-overlap.js` and `test-observer-detail-ownership.js` now load `payload-labels.js` before the page script, as `index.html` does. `test-issue-1418-raw-hex-extraction.js` asserted the hand-rolled `PAYLOAD_TYPE_MAP` literal; it now asserts that `inferredType` uses the canonical map and that no local map exists. - Unit suites, each `test-all.sh` entry run on its own: 223 pass on this branch vs 222 on `master` (+1 new suite). `test-manage-workflow.js` and `test-preflight-xss-gate.js` fail identically on `master` in my Windows environment (CRLF in the shell fixture, no `python`); they are unrelated to this change. - E2E, run against a local server on the CI fixture (freshened, seeded and migrated as in `deploy.yml`): `test-issue-1799-label-vocab-e2e.js`, `test-charts-fluid-1058-e2e.js`, `test-issue-1147-section-order-e2e.js`, `test-issue-1640-compare-discovery-e2e.js`, `test-issue-1648-m2/m3-icons-e2e.js` and `test-observer-detail-ownership-e2e.js` all pass. `test-e2e-playwright.js` fails one step (`Version info lives on Perf dashboard`, `#navStats` wait timeout) with the same result on `master` locally. ## Performance No hot-path change. Every lookup is still one object-key read per packet: the packets-table sort accessor calls `displayName(t)` instead of `TYPE_NAMES[t] ?? ...`. No new loops, requests or allocations. ## Browser validation Checked against the local fixture server, with the analytics responses patched in the page to contain GRP_DATA: - Node Analytics → Packet Types legend: `Group Data, Advert, Channel Msg, Direct Msg, Multipart, Raw Custom, Type 13` - Observer detail → doughnut and the first Recent Packets row read `Group Data` - Observer comparison → packet table rows read `Group Data` - Map route deep link (`#/map?packet=…`) renders the route panel; no new console errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
n30nexandn30nex
|
6618820836 |
fix(nodes): show timestamp context for Last Contact (#2377)
Partial fix for #1154 U10. Peer Interactions Last Contact now shows timestamp context through the existing shared formatter: relative text with an absolute tooltip, or the reverse when absolute display is selected. Existing timezone, precision and custom-format preferences apply; missing/invalid values remain a dash. Other U10 surfaces and tracking items stay open. The production change only formats the existing cell. Work stays O(P) over the existing maximum 20 peers; no requests, settings, dependencies, caches or render triggers are added. Tooltip and text are HTML-escaped. RED `c47cce20` reached [11 unit assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42316165311/job/128229216777) and [six browser missing-tooltip assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42316165311/job/128229216764). GREEN `2c29eccb` keeps those tests unchanged and passes 82 unit assertions, 10 focused browser cases (parent rerun), and all 224 frontend suites. Existing browser smoke reports 149 passes and three fixture skips. One separate navigation timeout passed on an unchanged serial retry. Browser verified: disposable Go fixture at `http://127.0.0.1:14953`, desktop/phone; tests retain the real API response and supply deterministic peer contacts. Its unrelated missing battery-metrics table is a fixture limitation. E2E assertion added: `tests/e2e/test-issue-2218-node-analytics-stale-e2e.js:123`. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42317025356) passes. Independent adversarial, UI and test-quality reviews found no must-fixes; Kent Beck Gate: PASS. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
n30nex
|
b5caea1d5f |
fix(analytics): render safe, correctly sized battery status icons (#2282)
Red commit: `4cc2efb3` ([failing CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40189921192)). Fixes #2232. Battery badges now render their fixed status icon and append voltage as text, including zero. Chart sizing excludes inline icons, and status colors use the existing theme variables. Unknown status values stay muted. - The original browser regression failed on literal SVG; a second local run with only the JavaScript fix failed on oversized icons. - Validation on `eef0872e`: all standalone frontend suites; 47 focused unit checks; 41 Chromium checks, including both themes, hostile voltage, accessibility and 16×16 icon bounds. - Browser verified: http://127.0.0.1:18851 on an isolated Go fixture. - E2E assertion added: `tests/e2e/test-issue-1648-m2-icons-e2e.js:134`. - Constant work for one badge; no new requests, dependencies, settings or schema changes. Privacy, whitespace, CSS-variable and repository XSS checks passed. The external home-directory preflight bundle is unavailable; its applicable checks were performed locally. Independent review found no outstanding fixes. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40190620324) passed on `eef0872e`, including all three browser shards and both architecture builds. |
||
|
efitenandClaude Opus 5.5
|
2bb000f2aa |
feat(notify): adaptive offline threshold and certain node analytics (#2260) (#2262)
Part 2 of #2260. **Builds on #2261** (certain liveness): review/merge that first; until then this diff also shows its commit. ## Adaptive offline threshold `userManagement.notifications.adaptiveOffline` — **off by default**: ```json "notifications": { "enabled": true, "adaptiveOffline": { "enabled": true, "floorMinutes": 30, "minObservers": 3 } } ``` The fixed `infraSilentHours` (72 h) treats every repeater the same. With this on, a watched node's `node.offline` window is learned from its own **certain** traffic (the #2261 rule): 1. Certain packets per hour over the week **before its last certain packet**. Ending there, not at now, keeps an outage from dragging the rate down until the fixed window returns and the node reads "back online". 2. Empty runs are classified: ≤ 6 h and recurring on another day at the same hours = normal silence (kept; the window is never shorter than the longest + 1 h); any other run = outage, left out of the learning so it neither stretches the window nor blocks learning. 3. λ = 10th percentile of the remaining hourly counts, so quiet nights don't alert. 4. Window = 14/λ hours: silence at rate λ has a < 10⁻⁶ chance (Poisson, ln 10⁶ ≈ 13.8). 5. Clamped to [`floorMinutes`, the role's silent window]. Kept on the fixed window: λ < 1/hour, under 24 h of history in memory, or a 10th-percentile hour heard by fewer than `minObservers` observers (one observer dropping out must not read as the node going offline). Silence is measured up to the newest packet in the store, so a stalling feed stops every node's clock. A node stored offline comes back only on evidence from the moment it went offline on. The notifier caches each learned window for 10 minutes and drops unwatched nodes. For the #2260 repeater (~840 packets/day) the mail would have gone out about 30 minutes into the outage, instead of never (or 72 h with #2261 alone). ## Certain node analytics - Each `activityTimeline` bucket carries `certain`: the part that can only be this node (`certainFor`, the packet-level form of the `certainHeard` rule). `TimeBucket.Certain` is `omitempty`, so other endpoints are unchanged. - Availability and longest silence use certain buckets only; the longest silence now includes one **still running** (an outage in progress is the silence worth seeing). - The chart stacks certain (`--accent`) and uncertain (`--text-muted`) — existing theme variables, so no new customizer entry. Descriptions updated. ## Performance `CertainActivity` walks a week of packets per watched node, at most once per node per 10 minutes, with the read lock taken **per node**: ``` BenchmarkCertainActivity-16 12.1 ms/op (50 watched repeaters × 6,000 packets; ~0.25 ms per lock hold) BenchmarkNoteCertainRelays-16 121 ns/op 0 allocs ``` Node analytics adds one `certainFor` per packet of the requested node (a prefix-map lookup, or the cached decoded JSON for its own packets). ## Tests - `adaptiveOfflineAfter`: busy repeater hits the floor; 5/hour; quiet nights set the rate; > 10 % empty hours, short history, too few observers → fixed. - `CertainActivity` ends at the last certain packet and counts only certain packets — verified red by mutating the window to end at now. - Notifier end to end: learned window passed → bad; inside → good; too few observers / setting off → fixed 72 h; cache refresh and prune. - `certainFor`, `longestSilence` (incl. running silence), config defaults, `/api/nodes/{pk}/analytics` splits a shared-prefix bucket. - Frontend: `NodeAnalyticsTimeline.series` (new `tests/unit/test-node-analytics-certain-timeline.js`). - `cmd/server` `go test ./...` and `test-all.sh` pass locally, also on a trial merge into a downstream fork. ## Validation Server on the freshened e2e fixture: analytics buckets carry `certain` (e.g. 89 of 90 packets certain for one repeater). **No browser screenshot**: Playwright isn't installed on this machine. ## Later (rule 8) `floorMinutes` / `minObservers` are server config, not theme settings; an admin UI for notification settings would be the place, not the customizer. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
3960fadd96 |
fix(pages): drop late responses that belong to an earlier page visit (#2218, #2230, #2200) (#2248)
Fixes #2218, fixes #2230, fixes #2200. Node analytics, the account page and analytics await their data, then write into the shared `#app` container. A response that lands after the user has navigated away replaces the new page. In #2218's reproduction, the Nodes list, with 200 rows, became an Analytics heading. ## The shared part `public/app.js` keeps one view generation, `viewGeneration()`: - `navigate()` bumps it before the old page's `destroy()`, so every page init counts as a new visit, including a switch to another node of the same page type. - A page captures the value at init, and after each `await` drops the result when the value has moved on. `.eslintrc.json` declares it as a global. ## Per page **#2218, `node-analytics.js`** - `loadAnalytics` and `loadBatteryChart` check the generation on both success and error. A page-local `loadSeq` makes a time-range click a new load on the same visit, so an older range can't land over a newer one. - `NodeHopAnalytics.load` already checked `container.isConnected`, so it is unchanged. **#2230, `account.js`** - The render after `CSAuth.ready()` runs only while the generation is unchanged. - `destroy()` was empty. It now clears `shown.app` and `shown.userId`. **#2200, `analytics.js`** - A page-local `_loadSeq` sits next to the generation. Results and errors of an older window/region selection or an earlier visit are dropped, so a late failure of an old load no longer shows an error over newer results. - The bulk load renders only the six tabs drawn from its data. A tab that loads its own data (Prefix Tool, Nodes, Roles, ...) is no longer rebuilt or replaced by the bulk result: a deep-linked one renders at init, and a window, region or area change re-renders it directly. - `renderTab` guards the six tabs drawn from the bulk data (overview, rf, topology, channels, hashsizes, collisions). Until the data arrives they show "Loading analytics…", or the current load's error. Overview no longer dereferences an undefined `rf`. - Three related fixes in the same lines: - The error message is now escaped with `esc()`; it was inserted raw. - `destroy()` removes the RegionFilter and AreaFilter listeners. They piled up on every visit and called `loadAnalytics` from other pages. - `destroy()` resets the stored error. ## Tests | File | Assertions | Covers | |---|---|---| | `tests/unit/test-issue-2218-node-analytics-stale-render.js` | 12 | Loads the real router: the generation rises on every init, including another node. Late success and error after leaving, a stale visit of the same node, an older range landing last, late battery success and error. The current response, battery and error still render | | `tests/unit/test-issue-2230-account-stale-render.js` | 6 | Leaving before ready (login view and "not enabled"), re-entering before ready, the current render, an auth change after `destroy()` | | `tests/unit/test-issue-2200-analytics-stale-load.js` | 19 | An older selection landing last, a late rejection of an older selection, leave and re-enter (success and error), the current error rendered escaped and kept on a tab click, Overview and RF clicked before the data | | `tests/e2e/test-issue-2218-node-analytics-stale-e2e.js` | 4 steps | Holds `/api/nodes/*/analytics` with `page.route`, returns to `#/nodes`, then releases the response as a 200 and as a 500. The 200 rows stay and no Analytics heading appears; a normal visit still renders | How I checked them: - With the production files reverted and the tests kept, 7 of 12, 4 of 6 and 7 of 12 unit assertions fail. With the fix, all pass. - The E2E fails on the unfixed frontend with "Nodes rows 200 before release, 0 after". - Single mutations, each reverted: - removing the success check in `loadAnalytics` (analytics) fails 2 assertions; - removing the check after `CSAuth.ready()` fails 3. Other test changes: - `test-analytics-theme-refresh-before-data.js` and `test-user-management-ui.js` get a `viewGeneration: () => 0` stub. - The new E2E is registered in `scripts/non-unit-tests.json` and `deploy.yml` (shard 2, next to the #748 suite). The unit tests are in `test-all.sh`. - The broad suite's "Node side panel Details link navigates" test selected an anchor that now opens Analytics, as #2218 notes. It now clicks `#nodesRight .node-detail-btn`. Results: - `sh test-all.sh` passes, and `npx eslint public/*.js` reports 0 errors. - E2E on a fixture seeded as in CI: | Suite | Result | |---|---| | `test-e2e-playwright.js` | 148 passed, 3 skipped | | `test-user-management-e2e.js` | 28/28 | | the new #2218 suite | 4/4 | | `test-issue-748-node-analytics-coverage-e2e.js` | 2/2 | | `test-analytics-fluid-charts.js` | 8/8 | | `test-issue-1657-analytics-channels-group-sprites-e2e.js` | 5/5 | ## Not done - Two tabs that each fetch their own data can still overwrite each other after a fast tab switch between them. #2200 does not ask for that. - `nodes.js` `navigateToNode` calls its own `init` without the router, so it does not bump the generation. None of the three pages fixed here depends on that. - I placed the E2E in shard 2 without measuring suite times, which the `deploy.yml` comment asks for. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
927a8eec8a |
fix(nodes): node analytics rates cover the data in memory, and say so (#748) (#2211)
Refs #748 (M3a: correct numbers within the in-memory window; the SQL history in M3b is a separate plan, see the end). ## Problem The node analytics range picker offers 24h, 7d, 30d and All (365d), but the store only holds its in-memory window (`retentionHours`, then eviction). `GetNodeAnalytics` still divided availability and packets/day by the requested range. Measured on our live instance (14-day window), repeater BE-BRE-ON8AR: | Range | Availability | Packets/day | |---|---|---| | 7d | 88.7% | 91 | | 30d | 42.6% | 265 | | All | 3.5% | 22 | Nothing on the page said that the data stopped at the edge of the window. ## What changes - `coverageStart` returns the edge of the in-memory window: the latest of `oldestLoaded` (background-load progress), `now - retentionHours`, and a new `evictedThrough` that eviction sets to the newest `FirstSeen` it removed, so memory-pressure eviction inside the retention window moves the edge too. - `s.packets[0]` is not that edge, and the first version of this PR used it. Chunks are selected by `last_seen`, so a transmission first heard weeks ago and heard again recently sits at the head with its old `FirstSeen`. On staging (7-day window) it returned 21/07. - When the edge lies more than a minute after the requested start, node analytics counts only transmissions from the edge, computes availability and packets/day over that span, and returns `timeRange.dataFrom` and `timeRange.partial`. - The minute of slack keeps a range equal to the retention window from reading as partial. Without it every 7d request on staging (168h) was partial, because the two edges came from separate clock reads. - The analytics page shows "Data from 2026-10-07: the server keeps about 7 days of packets in memory. Availability and packets/day cover that period, not the full 365 days." when partial. - **UTC fix.** The window bounds of node analytics and node hop analytics are formatted in UTC. They are compared as strings with UTC `FirstSeen` values, so on a non-UTC host the 7d window lost its oldest hours (2 at +02:00). Docker images run in UTC, which is why this did not show in production. - `nodeTxsSince` (one caller) is inclusive, so a packet exactly on the edge is kept. ## Tests - Go, `node_analytics_coverage_test.go`: - full coverage within memory; - 30d over 3 days in memory; - empty store; - a re-heard 85-day-old transmission at the head of `s.packets` (edge stays at retention, packet not counted, no bucket before `dataFrom`); - memory-eviction edge; - eviction records `evictedThrough`; - retention 30 s shorter than the range is not partial, while 30d is; - window bounds under a +02:00 `time.Local`. - Two tests were checked to fail on the old code: the local-zone test (166 of 168 packets, a `+02:00` bound) and the retention test without the slack (`dataFrom 09:57:08` vs `from 09:56:38`). - Full `cmd/server` suite passes locally (skipping `TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean master). One run of five reported `FAIL` and four reruns were green; I did not capture which test failed. - Frontend unit: `tests/unit/test-issue-748-node-analytics-coverage.js` (5 assertions on the real `node-analytics.js` in `vm`), in `test-all.sh`; the full suite passes. - E2E: `tests/e2e/test-issue-748-node-analytics-coverage-e2e.js` (registered in `deploy.yml` and `scripts/non-unit-tests.json`), against the real fixture response with no interception: 7d is not partial and shows no notice; All is partial and the notice names the real start date. 2/2 locally. ## Validation Staging (retentionHours 168), same repeater, after the background load completed: | Range | Before | After | |---|---|---| | 7d | 82.7% | 83.3%, not partial | | 30d | 19.3% | 83.3%, partial from 2026-10-07 | | All | 1.6% | 83.3%, partial from 2026-10-07 | In the browser the notice appears under All and not under 7d. ## Not done - M3b, older history from SQLite. In the database only the sender is indexed (`from_pubkey`). Recipients sit in `decoded_json`, and relay hops only in `observations.resolved_path` (filled for about 27% of observations on our live instance, not indexed). A fallback today would show only a node's own packets for older days, a different meaning than the in-memory part. Doing it properly needs a per-node index written by the ingestor (pubkey, transmission, role); that is a separate plan. - The other analytics endpoints with a range (observer analytics, RF) were not checked for the same dilution. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
a059299588 |
feat(node-analytics): hop-count statistics per node (#1812) (#2021)
## Summary
Adds per-node hop-count statistics so repeater operators can choose
`flood.max`, `flood.max.unscoped` and `flood.max.advert` from what their
node actually sees.
- New endpoint `GET /api/nodes/{pubkey}/hop_analytics?days=N`
(`cmd/server/routes.go:299`, `cmd/server/node_hop_analytics.go:312`),
separate from `/analytics` as requested in the issue.
- New card "Hop Count at This Node" on the node analytics page
(`public/node-hop-analytics.js`, wired at
`public/node-analytics.js:130,174`): histogram of hop counts with a box
plot on the same x axis, filters `flood.max` (default),
`flood.max.advert`, `flood.max.unscoped`, driven by the existing range
picker.
- The existing "Hop Distribution" chart is unchanged: it shows path
length at the observer, a different quantity.
- No `direct` tag, although the issue lists one: for DIRECT packets the
path is the remaining route and no flood limit applies, so there is no
hop count to report.
## Hop count definition (firmware 0679dbef)
- `src/helpers/RoutingPolicy.h:15-21`: limits compare
`getPathHashCount()`; `.unscoped` applies to route type FLOOD, `.advert`
to adverts.
- `src/Mesh.cpp:344-350`: `routeRecvPacket` checks with n hashes in the
path, then writes its own hash at index n. So hops = the node's
zero-based index in the path, no +1.
- `src/Mesh.cpp:265-285`: a node forwards a flood once;
`src/Mesh.cpp:651,680`: an originator never forwards its own flood.
- DIRECT packets are excluded: their path is the remaining route
(`src/Mesh.cpp:78-103,334-341`).
Response: `{timeRange, packets: [{hash, timestamp, hops, tags}],
ambiguous}`. Tags: `flood`, `scoped` or `unscoped`, `advert`. Documented
in `docs/api-spec.md:679` and `cmd/server/openapi.go:90`.
## Attribution
`cmd/server/node_hop_analytics.go:198-309`. The result depends only on
the observed paths, the prefix map and the neighbor graph, so it is the
same after a restart as after live ingest.
- Every observation of every flood packet in the window is read.
`byNode` holds the server resolver's pick at ingest and other picks
after a cold load; `byPathHop` indexes only each packet's longest path,
which for a busy relay often runs through another branch of the flood.
- A packet counts when the node's prefix sits at exactly one index
across its observations, and either the node is the only relay candidate
for that prefix (`prefixMap.relayCandidates`,
`cmd/server/store.go:6795`), or the hop resolves to the node under the
ingestor's strict rule (`cmd/ingestor/path_resolver.go:143-214`) in at
least one observation and to another node in none. Strict rule: earlier
hops identified without a tiebreak, exactly one candidate adjacent in
`neighbor_edges` to the previous hop (the originator for hop 0 of an
advert), nodes already on the path excluded.
- The server resolver's tiebreaks (affinity, GPS distance, advert count,
pubkey order) are not used.
- Everything else with the node's prefix goes to `ambiguous`. In
practice that is most packets with a colliding 1-byte path hash.
On a read-only 7-day dump of a 1,669-node mesh DB, for one busy
repeater: 23,081 packets attributed, 11,437 ambiguous. Taking candidates
from `byPathHop` instead gave 9,995 attributed, with the histogram mode
moved from 2 to 3-5 hops.
## Performance
Scans `s.packets` under the read lock, no SQL per packet. Per
observation: one substring test for the node's first prefix byte; the
hop scan only for observations containing it; the strict walk only for
colliding prefixes, with per-request caches for candidates and
adjacency. `BenchmarkNodeHopPackets` models one 7-day request at that
scale (73,782 flood packets, 1,430,280 observations): 44-87 ms/op, 13.4
MB, 40 allocs on a throttling laptop.
Response size for that repeater over 7 days: about 23k entries, 2.3 MB
JSON, 375 KB gzipped. `hash` and `timestamp` are 61% of the raw and 91%
of the gzipped bytes; they stay because the issue asks for them so a
client can join entries to packets and bin by time.
## Tests
- Go: `cmd/server/node_hop_analytics_test.go`: 12 unit tests, a
live-ingest test through `IngestNewFromDB` (a colliding prefix without
independent attribution goes to `ambiguous`, not to the node the
resolver picked), live ingest versus cold load of the same DB, route
test, benchmark. 15 mutations of the attribution logic each fail a test.
- JS: `test-node-hop-analytics.js` (filters, histogram, quartiles and
whiskers with a fixture that separates 1.5 IQR from 3 IQR, render),
registered in `test-all.sh` and `.github/workflows/deploy.yml`.
- `gofmt`, `go vet ./...`, `go test ./...` in `cmd/server`,
`scripts/check-css-vars.js` pass.
## Staging validation
Build `c646310f` (this PR's review follow-up together with the other
open follow-ups), after a container restart and full load, on a busy
Belgian repeater:
- `hop_analytics?days=7`: 23,302 packets, 11,548 ambiguous, median 4,
adverts never above hop 7 (matching the firmware default
`flood_max_advert = 8`, `examples/simple_repeater/MyMesh.cpp:922`), 1.2
s. The first version reported 23,035 packets and 86 ambiguous in 534 ms,
because it trusted the resolver's pick for colliding prefixes.
- The card rendered on the first version with no console errors; the
rework does not touch the frontend beyond a test fixture.
## Not verified
- Response time and lock hold for 30 days on the busiest node on a
14-day store.
- Server relay candidates exclude companions and listeners while the
ingestor's prefix index does not, so a few strict attributions can
differ from the ingestor's persisted `resolved_path`.
- Identical numbers across a second container restart were shown in a Go
test, not repeated on staging.
- Dark theme, phone width, and switching the range picker in the
browser.
- Filter state is not reflected in the URL hash (the range picker is not
either).
Fixes #1812
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
d954ea7444 |
feat(#1668): axe-core CI gate for WCAG AA color-contrast (M5) (#1696)
Partial fix for #1668 (M5 of 6). After M1 (audit), M2 (color tokens, #1676), M3 (typography floor, #1679), and M4 (per-route polish, #1681) cleared ~95% of contrast/typography violations, M5 **locks in the wins** by adding an axe-core CI gate that fails the build on any new WCAG AA color-contrast regression. ## What's in the box - `test-a11y-axe-1668.js` — Playwright + `@axe-core/playwright`. Runs every major CoreScope route × `{dark, light}` at 1200×900 desktop, injects axe, runs only the `color-contrast` rule, asserts net violations === 0. - `test-a11y-axe-1668-selftest.js` — fast, deterministic, browser-free unit test that exercises the YAML allowlist parser, the `violationAllowed` matcher, and the route/theme metadata. Runs in the JS unit block (no browser needed). - `tests/a11y-allowlist.yaml` — operator-flagged false-positive allowlist. **0 entries at M5 baseline.** ## Allowlist format Each entry MUST cite a GH issue # and an `expires_at` date. Missing fields = refused. Expired `expires_at` = refused (warning logged). This **forces a periodic revisit** — no permanent suppressions. ```yaml - route: /analytics?tab=channels selector: ".some-known-stale-element" rule: color-contrast issue: 1234 expires_at: 2026-09-01 ``` ## Routes covered (19 × 2 themes = 38 cells) `/`, `/packets`, `/nodes`, `/channels`, `/live`, `/map`, `/observers`, `/compare`, `/analytics?tab={overview,rf,topology,channels,hashsizes,collisions,roles,airtime}`, `/audio-lab`, `/customize`, `/replay`. ## TDD red→green - **RED** (`08adafdb`) — adds the gate + deliberately regresses `--text-muted` from `palette-gray-700` (~10:1) to `#9ca3af` (~2.4:1). axe-core fails on every light-theme cell. - **GREEN** (`f62fb1e0`) — restores the M2 token. Net violations = 0 across all 38 cells. ## Scope discipline - Only `color-contrast` (matches M2/M3/M4 scope). M6 owns `image-alt`, `aria-required-attr`, `label`, mobile viewports, and letsmesh A/B. - No new design tokens. - M2-M4 tokens untouched. ## CI wiring - `.github/workflows/deploy.yml:155` — selftest in JS unit block. - `.github/workflows/deploy.yml:367` — real axe browser run in the Playwright E2E block after the fixture server is up. ## Deps `@axe-core/playwright@4.11.3` + `axe-core@4.12.1` added to `devDependencies`. Pinned versions. --------- Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> |
||
|
Kpa-clawbotandopenclaw-bot
|
3062745437 |
M2: emoji → Phosphor Icons — page headers & table chrome (#1648) (#1650)
Red commit:
|
||
|
Kpa-clawbotandbot
|
3ab404b545 |
feat(node-battery): voltage trend chart + /api/nodes/{pubkey}/battery (#663) (#1082)
## Summary Closes #663 (Phase 2 + 3 partial — time-series tracking + thresholds for nodes that are also observers). Adds a per-node battery voltage trend chart and `/api/nodes/{pubkey}/battery` endpoint, sourced from the existing `observer_metrics.battery_mv` samples populated by observer status messages. No new ingest or schema changes — purely surfaces data we were already collecting. ## Scope (TDD red→green) **RED commit:** test(node-battery) — DB query, endpoint shape (200/404/no-data), and config getters all asserted. **GREEN commit:** feat(node-battery) — implementation only. ## Changes ### Backend - `cmd/server/node_battery.go` (new): - `DB.GetNodeBatteryHistory(pubkey, since)` — pulls `(timestamp, battery_mv)` rows from `observer_metrics WHERE LOWER(observer_id) = LOWER(public_key) AND battery_mv IS NOT NULL`. Case-insensitive join tolerates historical pubkey casing variation (observers persist uppercase, nodes lowercase in this DB). - `Server.handleNodeBattery` — `GET /api/nodes/{pubkey}/battery?days=N` (default 7, max 365). Returns `{public_key, days, samples[], latest_mv, latest_ts, status, thresholds}`. - `Config.LowBatteryMv()` / `CriticalBatteryMv()` — defaults 3300 / 3000 mV. - `cmd/server/config.go` — `BatteryThresholds *BatteryThresholdsConfig` field. - `cmd/server/routes.go` — route registration alongside existing `/health`, `/analytics`. ### Frontend - `public/node-analytics.js` — new "Battery Voltage" chart card with status badge (🔋 OK / ⚠️ Low / 🪫 Critical / No data). Renders dashed threshold lines at `lowMv` and `criticalMv`. Empty-state message when no samples in window. ### Config - `config.example.json` — `batteryThresholds: { lowMv: 3300, criticalMv: 3000 }` with `_comment` per Config Documentation Rule. ## Status semantics | latest_mv | status | |-----------------------|------------| | no samples in window | `unknown` | | `>= lowMv` | `ok` | | `< lowMv`, `>= critMv`| `low` | | `< criticalMv` | `critical` | ## What this PR does NOT do (deferred) The issue's full Phase 1 (writing decoded sensor advert telemetry into `nodes.battery_mv` / `temperature_c` from server-side decoder) and Phase 4 (firmware/active polling for repeaters without observers) are out of scope here. This PR delivers the requested Phase 2/3 surfacing for the data path that already lands rows: `observer_metrics`. Repeaters that are also observers (i.e. publish status to MQTT) will get a voltage trend immediately; pure passive nodes won't until Phase 1 lands. ## Tests - `TestGetNodeBatteryHistory_FromObserverMetrics` — case-insensitive join, NULL skipping, ordering. - `TestNodeBatteryEndpoint` — full happy path with thresholds + status. - `TestNodeBatteryEndpoint_NoData` — 200 + status=unknown. - `TestNodeBatteryEndpoint_404` — unknown node. - `TestBatteryThresholds_ConfigOverride` — config getters + defaults. `cd cmd/server && go test ./...` — green. ## Performance Endpoint is per-pubkey (called once on analytics page open), indexed by `(observer_id, timestamp)` PK on `observer_metrics`. No hot-path impact. --------- Co-authored-by: bot <bot@corescope> |
||
|
Kpa-clawbotandyou
|
736b09697d |
fix(analytics): apply customizer timestamp format to chart axes (closes #756) (#981)
## Summary Fixes #756 — the customizer timestamp format setting (ISO/ISO+ms/locale) and timezone (UTC/local) were not applied to chart X-axis labels, tooltips, or certain inline timestamps in the analytics pages. ## Changes ### `public/app.js` - Added `formatChartAxisLabel(date, shortForm)` — a shared helper that reads the customizer's `timestampFormat` and `timestampTimezone` preferences and formats dates for chart axes accordingly. `shortForm=true` returns time-only (for intra-day charts), `shortForm=false` returns date+time (for multi-day ranges). ### `public/analytics.js` - `rfXAxisLabels()`: now calls `formatChartAxisLabel()` instead of hardcoded `toLocaleTimeString()` - `rfTooltipCircles()`: tooltip timestamps now use `formatAbsoluteTimestamp()` instead of raw ISO - Subpath detail first/last seen: now uses `formatAbsoluteTimestamp()` - Neighbor graph last_seen: now uses `formatAbsoluteTimestamp()` ### `public/node-analytics.js` - Packet timeline chart labels: now use `formatChartAxisLabel()` (respects short vs long form based on time range) - SNR over time chart labels: now use `formatChartAxisLabel()` ## Behavior by setting | Setting | Chart axis (short) | Chart axis (long) | |---------|-------------------|-------------------| | ISO | `14:30` | `05-03 14:30` | | ISO+ms | `14:30:05` | `05-03 14:30:05` | | Locale | `2:30 PM` | `May 3, 2:30 PM` | All respect the UTC/local timezone toggle. ## Testing - Server builds cleanly (`go build`) - Served `app.js` contains `formatChartAxisLabel` (verified via curl) - Graceful fallback: all callsites check `typeof formatChartAxisLabel === 'function'` before calling, preserving backward compat if script load order changes --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
10f712f9d7 |
fix: restructure scroll containers for iOS status bar tap-to-scroll (#330) (#554)
## Summary Fixes #330 — iOS status bar tap-to-scroll broken because `#app` had `overflow: hidden`, preventing `<body>` from being the scroll container. ## Approach: Option B from the issue Instead of a JS polyfill, this restructures scroll containers so `<body>` is the primary scroll container by default, which iOS Safari requires for native status-bar tap-to-scroll. ### How it works **`#app` default (body-scroll mode):** Uses `min-height` instead of fixed `height`, no `overflow: hidden`. Content pushes beyond the viewport and body scrolls naturally. **`#app.app-fixed` (fixed-layout mode):** Restores the original `height: calc(100dvh - 52px); overflow: hidden` for pages that need constrained containers. The router in `app.js` toggles this class based on the current page. ### Fixed-layout pages (`.app-fixed`) These pages need fixed-height containers and are unchanged in behavior: - **packets** — virtual scroll requires fixed-height `.panel-left` to calculate visible rows - **nodes** — split-panel layout with independently scrollable panels - **map** — Leaflet requires fixed-dimension container - **live** — Leaflet map (also has its own `#app:has(.live-page)` override in live.css) - **channels** — split-panel chat layout - **audio-lab** — split-panel layout ### Body-scroll pages (no `.app-fixed`) These pages now let the body scroll, enabling iOS tap-to-scroll: - **analytics** — removed `overflow-y: auto; height: 100%` - **observers** — removed `overflow-y: auto; height: calc(100vh - 56px)` - **traces** — removed `overflow-y: auto; height: 100%` - **home** — removed `#app:has(.home-hero)` override (no longer needed) - **compare** — removed inline `overflow-y:auto; height:calc(100vh - 56px)` - **perf** — removed inline `height:100%; overflow-y:auto` - **observer-detail** — removed inline `overflow-y:auto; height:calc(100vh - 56px)` - **node-analytics** — removed inline `height:100%; overflow-y:auto` ### Files changed | File | Change | |------|--------| | `public/style.css` | `#app` default → `min-height`; added `.app-fixed` class | | `public/app.js` | Router toggles `.app-fixed` based on page | | `public/home.css` | Removed `#app:has()` workaround | | `public/compare.js` | Removed inline overflow/height | | `public/perf.js` | Removed inline overflow/height | | `public/observer-detail.js` | Removed inline overflow/height | | `public/node-analytics.js` | Removed inline overflow/height | ### What's preserved - Sticky nav (`position: sticky; top: 0`) — works with body scroll - Split-panel resize handles — unchanged, still in fixed containers - Virtual scroll on packets page — unchanged, `.panel-left` still has fixed height - Leaflet maps — unchanged, containers still have fixed dimensions - Mobile responsive overrides — unchanged Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandCopilot
|
71ec5e6fca |
rename: MeshCore Analyzer → CoreScope (frontend + .squad)
Phase 1 of the CoreScope rename — frontend display strings and squad agent metadata only. index.html: - <title>, og:title, twitter:title → CoreScope - Brand text span → CoreScope - og:image/twitter:image URLs → corescope repo (placeholder) - Cache busters bumped public/*.js headers (19 files): - All file header comments updated public/*.css headers: - style.css, home.css updated JavaScript strings: - app.js: GitHub URL → corescope - home.js: 3 fallback siteName references - customize.js: default siteName + heroTitle Tests: - test-e2e-playwright.js: title assertion → corescope - test-frontend-helpers.js: GitHub URL constant - benchmark.js: header string - test-all.sh: header string .squad: - team.md, casting/history.json - All 7 agent charters + 5 history files NOT renamed (intentional): - localStorage keys (meshcore-*) - CSS classes (.meshcore-marker) - Window globals (_meshcore*) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Kpa-clawbotandCopilot
|
f04f1b8e77 |
fix: accessibility — chart labels, table scope, form labels (#210, #211, #212)
#210: Add role="img" aria-label to 9 Chart.js canvases in node-analytics.js and observer-detail.js with descriptive labels. #211: Add scope="col" to all <th> elements across analytics.js, audio-lab.js, compare.js, node-analytics.js, nodes.js, observer-detail.js, observers.js, and packets.js (40+ headers). #212: Add aria-label to packet filter input and time window select in packets.js. Add for/id associations to all customize.js inputs: branding, theme colors, node/type colors, heatmap sliders, onboarding fields, and export controls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
you
|
a882aae681 |
M5: Frontend updates for dedup — observation_count badges, totalTransmissions
- packets.js: Show observation_count badge (👁 N) on grouped rows - nodes.js: Use totalTransmissions (fallback totalPackets), show observation badges on recent packets - home.js: Use totalTransmissions for network stats - node-analytics.js: Use totalTransmissions for throughput display - analytics.js: Use totalTransmissions for overview stats and node rankings - live.js: Use totalTransmissions in node detail, show observation badges in feed and recent packets - style.css: Add .badge-obs style for observation count badges - index.html: Bump cache busters on all changed JS/CSS files All changes have backward compat fallbacks to totalPackets. |
||
|
you
|
de658bfb0d |
perf: configurable cache TTLs via config.json — server + client fetch from /api/config/cache
All cache TTLs now read from config.json cacheTTL section (seconds). Client fetches config on load via GET /api/config/cache. config.example.json updated with defaults. Edit config.json, restart server — no code changes needed to tweak TTLs. |
||
|
you
|
e98e04553a |
feat: add frontend API response caching with TTL, in-flight dedup, and WebSocket invalidation
- Replace api() with caching version supporting TTL and request deduplication - Add appropriate TTLs to all api() call sites across all frontend JS files: - /stats: 5s TTL (was called 962 times in 3 min) - /nodes/:pubkey: 15s, /health: 30s, /observers: 30s - /channels: 15s, messages: 10s - /analytics/*: 60s, /bulk-health: 60s, /network-status: 60s - /nodes?*: 10s - Skip caching for real-time endpoints (/packets, /resolve-hops, /perf) - Invalidate /stats, /nodes, /channels caches on WebSocket messages - Deduplicate in-flight requests (same path returns same promise) - Add cache hit rate to window.apiPerf() console debugging - Update all cache busters in index.html |
||
|
you
|
ad5e12fc45 | fix: node analytics — compact layout, add descriptions to every stat card and chart | ||
|
you
|
b62c8c7b43 | fix: analytics page scroll — add overflow-y:auto to wrapper | ||
|
you
|
21b1cbc332 |
Add per-node analytics page with charts, stats, and heatmap
- New route: #/nodes/:pubkey/analytics with Chart.js v4 visualizations - Activity timeline (bar), SNR trend (line), packet type breakdown (doughnut) - Observer coverage (horizontal bar), hop distribution (bar) - Uptime heatmap (7x24 CSS grid, GitHub-style) - Peer interactions table with links to node details - Stat cards: availability, signal grade, packets/day, relay %, silence - Time range selector: 24h / 7d / 30d / All - Server: GET /api/nodes/:pubkey/analytics with full aggregation in SQLite - Analytics button added to both sidebar and full-screen node views |