mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-27 14:28:07 +00:00
a2ea18f7787c1cdbc783ec42ffabdde729e18fda
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 efitenandClaude Opus 5
|
a059299588 |
feat(node-analytics): hop-count statistics per node (#1812) (#2021)
## Summary
Adds per-node hop-count statistics so repeater operators can choose
`flood.max`, `flood.max.unscoped` and `flood.max.advert` from what their
node actually sees.
- New endpoint `GET /api/nodes/{pubkey}/hop_analytics?days=N`
(`cmd/server/routes.go:299`, `cmd/server/node_hop_analytics.go:312`),
separate from `/analytics` as requested in the issue.
- New card "Hop Count at This Node" on the node analytics page
(`public/node-hop-analytics.js`, wired at
`public/node-analytics.js:130,174`): histogram of hop counts with a box
plot on the same x axis, filters `flood.max` (default),
`flood.max.advert`, `flood.max.unscoped`, driven by the existing range
picker.
- The existing "Hop Distribution" chart is unchanged: it shows path
length at the observer, a different quantity.
- No `direct` tag, although the issue lists one: for DIRECT packets the
path is the remaining route and no flood limit applies, so there is no
hop count to report.
## Hop count definition (firmware 0679dbef)
- `src/helpers/RoutingPolicy.h:15-21`: limits compare
`getPathHashCount()`; `.unscoped` applies to route type FLOOD, `.advert`
to adverts.
- `src/Mesh.cpp:344-350`: `routeRecvPacket` checks with n hashes in the
path, then writes its own hash at index n. So hops = the node's
zero-based index in the path, no +1.
- `src/Mesh.cpp:265-285`: a node forwards a flood once;
`src/Mesh.cpp:651,680`: an originator never forwards its own flood.
- DIRECT packets are excluded: their path is the remaining route
(`src/Mesh.cpp:78-103,334-341`).
Response: `{timeRange, packets: [{hash, timestamp, hops, tags}],
ambiguous}`. Tags: `flood`, `scoped` or `unscoped`, `advert`. Documented
in `docs/api-spec.md:679` and `cmd/server/openapi.go:90`.
## Attribution
`cmd/server/node_hop_analytics.go:198-309`. The result depends only on
the observed paths, the prefix map and the neighbor graph, so it is the
same after a restart as after live ingest.
- Every observation of every flood packet in the window is read.
`byNode` holds the server resolver's pick at ingest and other picks
after a cold load; `byPathHop` indexes only each packet's longest path,
which for a busy relay often runs through another branch of the flood.
- A packet counts when the node's prefix sits at exactly one index
across its observations, and either the node is the only relay candidate
for that prefix (`prefixMap.relayCandidates`,
`cmd/server/store.go:6795`), or the hop resolves to the node under the
ingestor's strict rule (`cmd/ingestor/path_resolver.go:143-214`) in at
least one observation and to another node in none. Strict rule: earlier
hops identified without a tiebreak, exactly one candidate adjacent in
`neighbor_edges` to the previous hop (the originator for hop 0 of an
advert), nodes already on the path excluded.
- The server resolver's tiebreaks (affinity, GPS distance, advert count,
pubkey order) are not used.
- Everything else with the node's prefix goes to `ambiguous`. In
practice that is most packets with a colliding 1-byte path hash.
On a read-only 7-day dump of a 1,669-node mesh DB, for one busy
repeater: 23,081 packets attributed, 11,437 ambiguous. Taking candidates
from `byPathHop` instead gave 9,995 attributed, with the histogram mode
moved from 2 to 3-5 hops.
## Performance
Scans `s.packets` under the read lock, no SQL per packet. Per
observation: one substring test for the node's first prefix byte; the
hop scan only for observations containing it; the strict walk only for
colliding prefixes, with per-request caches for candidates and
adjacency. `BenchmarkNodeHopPackets` models one 7-day request at that
scale (73,782 flood packets, 1,430,280 observations): 44-87 ms/op, 13.4
MB, 40 allocs on a throttling laptop.
Response size for that repeater over 7 days: about 23k entries, 2.3 MB
JSON, 375 KB gzipped. `hash` and `timestamp` are 61% of the raw and 91%
of the gzipped bytes; they stay because the issue asks for them so a
client can join entries to packets and bin by time.
## Tests
- Go: `cmd/server/node_hop_analytics_test.go`: 12 unit tests, a
live-ingest test through `IngestNewFromDB` (a colliding prefix without
independent attribution goes to `ambiguous`, not to the node the
resolver picked), live ingest versus cold load of the same DB, route
test, benchmark. 15 mutations of the attribution logic each fail a test.
- JS: `test-node-hop-analytics.js` (filters, histogram, quartiles and
whiskers with a fixture that separates 1.5 IQR from 3 IQR, render),
registered in `test-all.sh` and `.github/workflows/deploy.yml`.
- `gofmt`, `go vet ./...`, `go test ./...` in `cmd/server`,
`scripts/check-css-vars.js` pass.
## Staging validation
Build `c646310f` (this PR's review follow-up together with the other
open follow-ups), after a container restart and full load, on a busy
Belgian repeater:
- `hop_analytics?days=7`: 23,302 packets, 11,548 ambiguous, median 4,
adverts never above hop 7 (matching the firmware default
`flood_max_advert = 8`, `examples/simple_repeater/MyMesh.cpp:922`), 1.2
s. The first version reported 23,035 packets and 86 ambiguous in 534 ms,
because it trusted the resolver's pick for colliding prefixes.
- The card rendered on the first version with no console errors; the
rework does not touch the frontend beyond a test fixture.
## Not verified
- Response time and lock hold for 30 days on the busiest node on a
14-day store.
- Server relay candidates exclude companions and listeners while the
ingestor's prefix index does not, so a few strict attributions can
differ from the ingestor's persisted `resolved_path`.
- Identical numbers across a second container restart were shown in a Go
test, not repeated on staging.
- Dark theme, phone width, and switching the range picker in the
browser.
- Filter state is not reflected in the URL hash (the range picker is not
either).
Fixes #1812
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
d954ea7444 |
feat(#1668): axe-core CI gate for WCAG AA color-contrast (M5) (#1696)
Partial fix for #1668 (M5 of 6). After M1 (audit), M2 (color tokens, #1676), M3 (typography floor, #1679), and M4 (per-route polish, #1681) cleared ~95% of contrast/typography violations, M5 **locks in the wins** by adding an axe-core CI gate that fails the build on any new WCAG AA color-contrast regression. ## What's in the box - `test-a11y-axe-1668.js` — Playwright + `@axe-core/playwright`. Runs every major CoreScope route × `{dark, light}` at 1200×900 desktop, injects axe, runs only the `color-contrast` rule, asserts net violations === 0. - `test-a11y-axe-1668-selftest.js` — fast, deterministic, browser-free unit test that exercises the YAML allowlist parser, the `violationAllowed` matcher, and the route/theme metadata. Runs in the JS unit block (no browser needed). - `tests/a11y-allowlist.yaml` — operator-flagged false-positive allowlist. **0 entries at M5 baseline.** ## Allowlist format Each entry MUST cite a GH issue # and an `expires_at` date. Missing fields = refused. Expired `expires_at` = refused (warning logged). This **forces a periodic revisit** — no permanent suppressions. ```yaml - route: /analytics?tab=channels selector: ".some-known-stale-element" rule: color-contrast issue: 1234 expires_at: 2026-09-01 ``` ## Routes covered (19 × 2 themes = 38 cells) `/`, `/packets`, `/nodes`, `/channels`, `/live`, `/map`, `/observers`, `/compare`, `/analytics?tab={overview,rf,topology,channels,hashsizes,collisions,roles,airtime}`, `/audio-lab`, `/customize`, `/replay`. ## TDD red→green - **RED** (`08adafdb`) — adds the gate + deliberately regresses `--text-muted` from `palette-gray-700` (~10:1) to `#9ca3af` (~2.4:1). axe-core fails on every light-theme cell. - **GREEN** (`f62fb1e0`) — restores the M2 token. Net violations = 0 across all 38 cells. ## Scope discipline - Only `color-contrast` (matches M2/M3/M4 scope). M6 owns `image-alt`, `aria-required-attr`, `label`, mobile viewports, and letsmesh A/B. - No new design tokens. - M2-M4 tokens untouched. ## CI wiring - `.github/workflows/deploy.yml:155` — selftest in JS unit block. - `.github/workflows/deploy.yml:367` — real axe browser run in the Playwright E2E block after the fixture server is up. ## Deps `@axe-core/playwright@4.11.3` + `axe-core@4.12.1` added to `devDependencies`. Pinned versions. --------- Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> |
||
|
Kpa-clawbotandopenclaw-bot
|
3062745437 |
M2: emoji → Phosphor Icons — page headers & table chrome (#1648) (#1650)
Red commit:
|
||
|
Kpa-clawbotandbot
|
3ab404b545 |
feat(node-battery): voltage trend chart + /api/nodes/{pubkey}/battery (#663) (#1082)
## Summary Closes #663 (Phase 2 + 3 partial — time-series tracking + thresholds for nodes that are also observers). Adds a per-node battery voltage trend chart and `/api/nodes/{pubkey}/battery` endpoint, sourced from the existing `observer_metrics.battery_mv` samples populated by observer status messages. No new ingest or schema changes — purely surfaces data we were already collecting. ## Scope (TDD red→green) **RED commit:** test(node-battery) — DB query, endpoint shape (200/404/no-data), and config getters all asserted. **GREEN commit:** feat(node-battery) — implementation only. ## Changes ### Backend - `cmd/server/node_battery.go` (new): - `DB.GetNodeBatteryHistory(pubkey, since)` — pulls `(timestamp, battery_mv)` rows from `observer_metrics WHERE LOWER(observer_id) = LOWER(public_key) AND battery_mv IS NOT NULL`. Case-insensitive join tolerates historical pubkey casing variation (observers persist uppercase, nodes lowercase in this DB). - `Server.handleNodeBattery` — `GET /api/nodes/{pubkey}/battery?days=N` (default 7, max 365). Returns `{public_key, days, samples[], latest_mv, latest_ts, status, thresholds}`. - `Config.LowBatteryMv()` / `CriticalBatteryMv()` — defaults 3300 / 3000 mV. - `cmd/server/config.go` — `BatteryThresholds *BatteryThresholdsConfig` field. - `cmd/server/routes.go` — route registration alongside existing `/health`, `/analytics`. ### Frontend - `public/node-analytics.js` — new "Battery Voltage" chart card with status badge (🔋 OK / ⚠️ Low / 🪫 Critical / No data). Renders dashed threshold lines at `lowMv` and `criticalMv`. Empty-state message when no samples in window. ### Config - `config.example.json` — `batteryThresholds: { lowMv: 3300, criticalMv: 3000 }` with `_comment` per Config Documentation Rule. ## Status semantics | latest_mv | status | |-----------------------|------------| | no samples in window | `unknown` | | `>= lowMv` | `ok` | | `< lowMv`, `>= critMv`| `low` | | `< criticalMv` | `critical` | ## What this PR does NOT do (deferred) The issue's full Phase 1 (writing decoded sensor advert telemetry into `nodes.battery_mv` / `temperature_c` from server-side decoder) and Phase 4 (firmware/active polling for repeaters without observers) are out of scope here. This PR delivers the requested Phase 2/3 surfacing for the data path that already lands rows: `observer_metrics`. Repeaters that are also observers (i.e. publish status to MQTT) will get a voltage trend immediately; pure passive nodes won't until Phase 1 lands. ## Tests - `TestGetNodeBatteryHistory_FromObserverMetrics` — case-insensitive join, NULL skipping, ordering. - `TestNodeBatteryEndpoint` — full happy path with thresholds + status. - `TestNodeBatteryEndpoint_NoData` — 200 + status=unknown. - `TestNodeBatteryEndpoint_404` — unknown node. - `TestBatteryThresholds_ConfigOverride` — config getters + defaults. `cd cmd/server && go test ./...` — green. ## Performance Endpoint is per-pubkey (called once on analytics page open), indexed by `(observer_id, timestamp)` PK on `observer_metrics`. No hot-path impact. --------- Co-authored-by: bot <bot@corescope> |
||
|
Kpa-clawbotandyou
|
736b09697d |
fix(analytics): apply customizer timestamp format to chart axes (closes #756) (#981)
## Summary Fixes #756 — the customizer timestamp format setting (ISO/ISO+ms/locale) and timezone (UTC/local) were not applied to chart X-axis labels, tooltips, or certain inline timestamps in the analytics pages. ## Changes ### `public/app.js` - Added `formatChartAxisLabel(date, shortForm)` — a shared helper that reads the customizer's `timestampFormat` and `timestampTimezone` preferences and formats dates for chart axes accordingly. `shortForm=true` returns time-only (for intra-day charts), `shortForm=false` returns date+time (for multi-day ranges). ### `public/analytics.js` - `rfXAxisLabels()`: now calls `formatChartAxisLabel()` instead of hardcoded `toLocaleTimeString()` - `rfTooltipCircles()`: tooltip timestamps now use `formatAbsoluteTimestamp()` instead of raw ISO - Subpath detail first/last seen: now uses `formatAbsoluteTimestamp()` - Neighbor graph last_seen: now uses `formatAbsoluteTimestamp()` ### `public/node-analytics.js` - Packet timeline chart labels: now use `formatChartAxisLabel()` (respects short vs long form based on time range) - SNR over time chart labels: now use `formatChartAxisLabel()` ## Behavior by setting | Setting | Chart axis (short) | Chart axis (long) | |---------|-------------------|-------------------| | ISO | `14:30` | `05-03 14:30` | | ISO+ms | `14:30:05` | `05-03 14:30:05` | | Locale | `2:30 PM` | `May 3, 2:30 PM` | All respect the UTC/local timezone toggle. ## Testing - Server builds cleanly (`go build`) - Served `app.js` contains `formatChartAxisLabel` (verified via curl) - Graceful fallback: all callsites check `typeof formatChartAxisLabel === 'function'` before calling, preserving backward compat if script load order changes --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
10f712f9d7 |
fix: restructure scroll containers for iOS status bar tap-to-scroll (#330) (#554)
## Summary Fixes #330 — iOS status bar tap-to-scroll broken because `#app` had `overflow: hidden`, preventing `<body>` from being the scroll container. ## Approach: Option B from the issue Instead of a JS polyfill, this restructures scroll containers so `<body>` is the primary scroll container by default, which iOS Safari requires for native status-bar tap-to-scroll. ### How it works **`#app` default (body-scroll mode):** Uses `min-height` instead of fixed `height`, no `overflow: hidden`. Content pushes beyond the viewport and body scrolls naturally. **`#app.app-fixed` (fixed-layout mode):** Restores the original `height: calc(100dvh - 52px); overflow: hidden` for pages that need constrained containers. The router in `app.js` toggles this class based on the current page. ### Fixed-layout pages (`.app-fixed`) These pages need fixed-height containers and are unchanged in behavior: - **packets** — virtual scroll requires fixed-height `.panel-left` to calculate visible rows - **nodes** — split-panel layout with independently scrollable panels - **map** — Leaflet requires fixed-dimension container - **live** — Leaflet map (also has its own `#app:has(.live-page)` override in live.css) - **channels** — split-panel chat layout - **audio-lab** — split-panel layout ### Body-scroll pages (no `.app-fixed`) These pages now let the body scroll, enabling iOS tap-to-scroll: - **analytics** — removed `overflow-y: auto; height: 100%` - **observers** — removed `overflow-y: auto; height: calc(100vh - 56px)` - **traces** — removed `overflow-y: auto; height: 100%` - **home** — removed `#app:has(.home-hero)` override (no longer needed) - **compare** — removed inline `overflow-y:auto; height:calc(100vh - 56px)` - **perf** — removed inline `height:100%; overflow-y:auto` - **observer-detail** — removed inline `overflow-y:auto; height:calc(100vh - 56px)` - **node-analytics** — removed inline `height:100%; overflow-y:auto` ### Files changed | File | Change | |------|--------| | `public/style.css` | `#app` default → `min-height`; added `.app-fixed` class | | `public/app.js` | Router toggles `.app-fixed` based on page | | `public/home.css` | Removed `#app:has()` workaround | | `public/compare.js` | Removed inline overflow/height | | `public/perf.js` | Removed inline overflow/height | | `public/observer-detail.js` | Removed inline overflow/height | | `public/node-analytics.js` | Removed inline overflow/height | ### What's preserved - Sticky nav (`position: sticky; top: 0`) — works with body scroll - Split-panel resize handles — unchanged, still in fixed containers - Virtual scroll on packets page — unchanged, `.panel-left` still has fixed height - Leaflet maps — unchanged, containers still have fixed dimensions - Mobile responsive overrides — unchanged Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandCopilot
|
71ec5e6fca |
rename: MeshCore Analyzer → CoreScope (frontend + .squad)
Phase 1 of the CoreScope rename — frontend display strings and squad agent metadata only. index.html: - <title>, og:title, twitter:title → CoreScope - Brand text span → CoreScope - og:image/twitter:image URLs → corescope repo (placeholder) - Cache busters bumped public/*.js headers (19 files): - All file header comments updated public/*.css headers: - style.css, home.css updated JavaScript strings: - app.js: GitHub URL → corescope - home.js: 3 fallback siteName references - customize.js: default siteName + heroTitle Tests: - test-e2e-playwright.js: title assertion → corescope - test-frontend-helpers.js: GitHub URL constant - benchmark.js: header string - test-all.sh: header string .squad: - team.md, casting/history.json - All 7 agent charters + 5 history files NOT renamed (intentional): - localStorage keys (meshcore-*) - CSS classes (.meshcore-marker) - Window globals (_meshcore*) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Kpa-clawbotandCopilot
|
f04f1b8e77 |
fix: accessibility — chart labels, table scope, form labels (#210, #211, #212)
#210: Add role="img" aria-label to 9 Chart.js canvases in node-analytics.js and observer-detail.js with descriptive labels. #211: Add scope="col" to all <th> elements across analytics.js, audio-lab.js, compare.js, node-analytics.js, nodes.js, observer-detail.js, observers.js, and packets.js (40+ headers). #212: Add aria-label to packet filter input and time window select in packets.js. Add for/id associations to all customize.js inputs: branding, theme colors, node/type colors, heatmap sliders, onboarding fields, and export controls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
you
|
a882aae681 |
M5: Frontend updates for dedup — observation_count badges, totalTransmissions
- packets.js: Show observation_count badge (👁 N) on grouped rows - nodes.js: Use totalTransmissions (fallback totalPackets), show observation badges on recent packets - home.js: Use totalTransmissions for network stats - node-analytics.js: Use totalTransmissions for throughput display - analytics.js: Use totalTransmissions for overview stats and node rankings - live.js: Use totalTransmissions in node detail, show observation badges in feed and recent packets - style.css: Add .badge-obs style for observation count badges - index.html: Bump cache busters on all changed JS/CSS files All changes have backward compat fallbacks to totalPackets. |
||
|
you
|
de658bfb0d |
perf: configurable cache TTLs via config.json — server + client fetch from /api/config/cache
All cache TTLs now read from config.json cacheTTL section (seconds). Client fetches config on load via GET /api/config/cache. config.example.json updated with defaults. Edit config.json, restart server — no code changes needed to tweak TTLs. |
||
|
you
|
e98e04553a |
feat: add frontend API response caching with TTL, in-flight dedup, and WebSocket invalidation
- Replace api() with caching version supporting TTL and request deduplication - Add appropriate TTLs to all api() call sites across all frontend JS files: - /stats: 5s TTL (was called 962 times in 3 min) - /nodes/:pubkey: 15s, /health: 30s, /observers: 30s - /channels: 15s, messages: 10s - /analytics/*: 60s, /bulk-health: 60s, /network-status: 60s - /nodes?*: 10s - Skip caching for real-time endpoints (/packets, /resolve-hops, /perf) - Invalidate /stats, /nodes, /channels caches on WebSocket messages - Deduplicate in-flight requests (same path returns same promise) - Add cache hit rate to window.apiPerf() console debugging - Update all cache busters in index.html |
||
|
you
|
ad5e12fc45 | fix: node analytics — compact layout, add descriptions to every stat card and chart | ||
|
you
|
b62c8c7b43 | fix: analytics page scroll — add overflow-y:auto to wrapper | ||
|
you
|
21b1cbc332 |
Add per-node analytics page with charts, stats, and heatmap
- New route: #/nodes/:pubkey/analytics with Chart.js v4 visualizations - Activity timeline (bar), SNR trend (line), packet type breakdown (doughnut) - Observer coverage (horizontal bar), hop distribution (bar) - Uptime heatmap (7x24 CSS grid, GitHub-style) - Peer interactions table with links to node details - Stat cards: availability, signal grade, packets/day, relay %, silence - Time range selector: 24h / 7d / 30d / All - Server: GET /api/nodes/:pubkey/analytics with full aggregation in SQLite - Analytics button added to both sidebar and full-screen node views |