mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-01 22:08:24 +00:00
docs/release-notes-v3.4
1198
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 you
|
9099154514 | docs: add v3.4 release notes v3.4.0 | ||
|
Kpa-clawbotandyou
|
924caaa680 |
fix: render both steps AND FAQ on home page (#525) (#529)
Fixes #525 The `checklist()` function in `home.js` treated steps and FAQ/checklist as mutually exclusive — if `homeCfg.checklist` existed, steps were skipped entirely. Adding a single FAQ via the customizer made all intro steps disappear. Now renders steps first, then FAQ below with a '❓ FAQ' header. Falls back to Bay Area hardcoded defaults only when neither exists. --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
ca95fc46aa |
fix: neighbor UI — show neighbors crash, dark mode contrast (#523) (#527)
## Summary Part of #523 — fixes bugs 5 and 7 (bug 6 was a duplicate of bug 7). ### Bug 5: Show Neighbors button throws `window._mapSelectRefNode is not a function` **Root cause:** Map popup HTML used inline `onclick` calling `window._mapSelectRefNode`, which was deleted on SPA page destroy. If a popup persisted after navigation, clicks would throw. **Fix:** Replaced inline `onclick` with event delegation. A document-level click handler catches all `[data-show-neighbors]` clicks and calls `selectReferenceNode` directly. The global `window._mapSelectRefNode` is still exposed for existing Playwright tests but is no longer relied upon by the UI. ### Bug 7: Blue text on dark blue background (dark mode contrast) **Root cause:** Neighbor table cells inside `.node-detail-section` / `.node-full-card` inherited accent/link color instead of using `var(--text)`, making text unreadable in dark mode. **Fix:** Added explicit `color: var(--text)` on `.node-detail-section .data-table td` and `.node-full-card .data-table td`. Only `<a>` tags within those cells retain `color: var(--accent)`. ### Files changed - `public/map.js` — event delegation for Show Neighbors - `public/style.css` — contrast fix for neighbor table cells --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
54fab0551e |
fix: add home defaults to server theme config (#525) (#526)
## Summary Fixes #525 — Customizer v2 home section shows empty fields and adding FAQ kills steps. ## Root Cause Server returned `home: null` from `/api/config/theme` when no home config existed in config.json or theme.json. The customizer had no built-in defaults, so all home fields appeared empty. When a user added a single override (e.g. FAQ), `computeEffective` started from `home: null`, created `home: {}`, and only applied the user's override — wiping steps and everything else. ## Fix ### Server-side (primary) In `handleConfigTheme()`, replaced the conditional `home` assignment with `mergeMap` using built-in defaults matching what `home.js` hardcodes: - `heroTitle`: "CoreScope" - `heroSubtitle`: "Real-time MeshCore LoRa mesh network analyzer" - `steps`: 4 default getting-started steps - `footerLinks`: Packets + Network Map links Config/theme overrides merge on top, so customization still works. ### Client-side (defense-in-depth) Added `DEFAULT_HOME` constant in `customize-v2.js`. `computeEffective()` now falls back to these defaults when server returns `home: null`, ensuring the customizer works even without server defaults. ## Tests - **Go**: `TestConfigThemeHomeDefaults` — verifies `/api/config/theme` returns non-null home with heroTitle, steps, footerLinks when no config is set - **JS**: Two new tests in `test-frontend-helpers.js` — verifies `computeEffective` provides defaults when home is null, and that user overrides merge correctly with defaults Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
0e1beac52f |
fix: neighbor affinity graph empty results + performance + accessibility (#523) (#524)
## Summary Fixes the neighbor affinity graph returning empty results despite abundant ADVERT data in the store. **Root cause:** `extractFromNode()` in `neighbor_graph.go` only checked for `"from_node"` and `"from"` fields in the decoded JSON, but real ADVERT packets store the originator public key as `"pubKey"`. This meant `fromNode` was always empty, so: - Zero-hop edges (originator↔observer) were never created - Originator↔path[0] edges were never created - Only observer↔path[last] edges could be created (and only for non-empty paths) **Fix:** Check `"pubKey"` first in `extractFromNode()`, then fall through to `"from_node"` and `"from"` for other packet types. ## Bugs Fixed | Bug | Issue | Fix | |-----|-------|-----| | Empty graph results | #522 | `extractFromNode()` now reads `pubKey` field from ADVERTs | | 3-4s response time | #523 comment | Graph was rebuilding correctly with 60s TTL cache — the slow response was due to iterating all packets finding zero matches. With edges now being found, the cache works as designed. | | Incomplete visualization | #523 comment | Downstream of bug 1+2 — fixed by fixing the builder | | Accessibility | #523 comment | Added text-based neighbor list, dynamic aria-label, keyboard focus CSS, dashed lines for ambiguous edges, confidence symbols | ## Changes - **`cmd/server/neighbor_graph.go`** — Fixed `extractFromNode()` to check `pubKey` field (real ADVERT format) - **`cmd/server/neighbor_graph_test.go`** — Added 2 new tests: `TestBuildNeighborGraph_AdvertPubKeyField` (real ADVERT format) and `TestBuildNeighborGraph_OneByteHashPrefixes` (1-byte prefix collision scenario) - **`public/analytics.js`** — Added accessible text-based neighbor list, dynamic aria-label, dashed line pattern for ambiguous edges - **`public/style.css`** — Added `:focus-visible` keyboard focus indicator for canvas ## Testing All Go tests pass (`go test ./... -count=1`). New tests verify the fix prevents regression. Fixes #523, Fixes #522 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
34489e0446 |
fix: customizer v2 — phantom overrides, missing defaults, stale dark mode (#518) (#520)
Fixes #518, Fixes #514, Fixes #515, Fixes #516 ## Summary Fixes all customizer v2 bugs from the consolidated tracker (#518). Both server and client changes. ## Server Changes (`routes.go`) - **typeColors defaults** — added all 10 type color defaults matching `roles.js` `TYPE_COLORS`. Previously returned `{}`, causing all type colors to render as black. - **themeDark defaults** — added 22 dark mode color defaults matching the Default preset. Previously returned `{}`, causing dark mode to have no server-side defaults. ## Client Changes (`customize-v2.js`) - [x] **P0: Phantom override cleanup on init** — new `_cleanPhantomOverrides()` runs on startup, scanning `cs-theme-overrides` and removing any values that match server defaults (arrays via `JSON.stringify`, scalars via `===`). - [x] **P1: `setOverride` auto-prunes matching defaults** — after debounced write, iterates the delta and removes any key whose value matches the server default. Prevents phantom overrides from accumulating. - [x] **P1: `_countOverrides` counts only real diffs** — now iterates keys and calls `_isOverridden()` instead of blindly counting `Object.keys().length`. Badge count reflects actual overrides only. - [x] **P1: `_isOverridden` handles arrays/objects** — uses `JSON.stringify` comparison for non-scalar values (home.steps, home.checklist, etc.). - [x] **P1: Type color fallback** — `_renderNodes()` falls back to `window.TYPE_COLORS` when effective typeColors are empty, preventing black color swatches. - [x] **P1: Dark/light toggle re-renders panel** — MutationObserver on `data-theme` now calls `_refreshPanel()` when panel is open, so switching modes updates the Theme tab immediately. ## Tests 6 new unit tests added to `test-customizer-v2.js`: - Phantom scalar overrides cleaned on init - Phantom array overrides cleaned on init - Real overrides preserved after cleanup - `isOverridden` handles matching arrays (returns false) - `isOverridden` handles differing arrays (returns true) - `setOverride` prunes value matching server default All 48 tests pass. Go tests pass. --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
58f791266d |
feat: affinity debugging tools (#482) — milestone 6 (#521)
## Summary Milestone 6 of #482: Observability & Debugging tools for the neighbor affinity system. These tools exist because someone will need them at 3 AM when "Show Neighbors is showing the wrong node for C0DE" and they have 5 minutes to diagnose it. ## Changes ### 1. Debug API — `GET /api/debug/affinity` - Full graph state dump: all edges with weights, observation counts, last-seen timestamps - Per-prefix resolution log with disambiguation reasoning (Jaccard scores, ratios, thresholds) - Query params: `?prefix=C0DE` filter to specific prefix, `?node=<pubkey>` for specific node's edges - Protected by API key (same auth as `/api/admin/prune`) - Response includes: edge count, node count, cache age, last rebuild time ### 2. Debug Overlay on Map - Toggle-able checkbox "🔍 Affinity Debug" in map controls - Draws lines between nodes showing affinity edges with color coding: - Green = high confidence (score ≥ 0.6) - Yellow = medium (0.3–0.6) - Red = ambiguous (< 0.3) - Line thickness proportional to weight, dashed for ambiguous - Unresolved prefixes shown as ❓ markers - Click edge → popup with observation count, last seen, score, observers - Hidden behind `debugAffinity` config flag or `localStorage.setItem('meshcore-affinity-debug', 'true')` ### 3. Per-Node Debug Panel - Expandable "🔍 Affinity Debug" section in node detail page (collapsed by default) - Shows: neighbor edges table with scores, prefix resolutions with reasoning trace - Candidates table with Jaccard scores, highlighting the chosen candidate - Graph-level stats summary ### 4. Server-Side Structured Logging - Integrated into `disambiguate()` — logs every resolution decision during graph build - Format: `[affinity] resolve C0DE: c0dedad4 score=47 Jaccard=0.82 vs c0dedad9 score=3 Jaccard=0.11 → neighbor_affinity (ratio 15.7×)` - Logs ambiguous decisions: `scores too close (12 vs 9, ratio 1.3×) → ambiguous` - Gated by `debugAffinity` config flag ### 5. Dashboard Stats Widget - Added to analytics overview tab when debug mode is enabled - Metrics: total edges/nodes, resolved/ambiguous counts (%), avg confidence, cold-start coverage, cache age, last rebuild ## Files Changed - `cmd/server/neighbor_debug.go` — new: debug API handler, resolution builder, cold-start coverage - `cmd/server/neighbor_debug_test.go` — new: 7 tests for debug API - `cmd/server/neighbor_graph.go` — added structured logging to disambiguate(), `logFn` field, `BuildFromStoreWithLog` - `cmd/server/neighbor_api.go` — pass debug flag through `BuildFromStoreWithLog` - `cmd/server/config.go` — added `DebugAffinity` config field - `cmd/server/routes.go` — registered `/api/debug/affinity` route, exposed `debugAffinity` in client config - `cmd/server/types.go` — added `DebugAffinity` to `ClientConfigResponse` - `public/map.js` — affinity debug overlay layer with edge visualization - `public/nodes.js` — per-node affinity debug panel - `public/analytics.js` — dashboard stats widget - `test-e2e-playwright.js` — 3 Playwright tests for debug UI ## Tests - ✅ 7 Go unit tests (API shape, prefix/node filters, auth, structured logging, cold-start coverage) - ✅ 3 Playwright E2E tests (overlay checkbox, toggle without crash, panel expansion) - ✅ All existing tests pass (`go test ./cmd/server/... -count=1`) Part of #482 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
9b1b82f29b |
fix: remove merge conflict marker from test-e2e-playwright.js (#519)
Removes a stale `<<<<<<< HEAD` conflict marker that was accidentally left in during the PR #510 rebase. This breaks Playwright E2E tests in CI. One-line fix — line 1311 deletion. Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
943eb69937 |
feat: neighbors section in node detail page (#482) — milestone 5 (#510)
## Summary Add a "Neighbors" section to the node detail page, showing first-hop neighbor relationships derived from the neighbor affinity graph (M2 API). Part of #482 — Milestone 5 per [spec](https://github.com/Kpa-clawbot/CoreScope/blob/spec/482-neighbor-affinity/docs/specs/neighbor-affinity-graph.md). ## What's Added ### Full-screen detail view (`#/nodes/{pubkey}`) - New `node-full-card` section between "Heard By" and "Paths Through This Node" - Table with columns: **Neighbor** (linked), **Role** (badge), **Score**, **Obs**, **Last Seen**, **Conf** (confidence indicator) - Confidence indicators per spec: - 🟢 HIGH: auto-resolved, ≥3 observations, score ≥ 0.5 - 🟡 MEDIUM: 2+ observations - 🔴 LOW: single observation - ⚠️ AMBIGUOUS: multiple candidates - Click neighbor name → navigate to their detail page - 📍 Map button per resolved neighbor row ### Condensed panel view (right panel) - Shows top 5 neighbors only - "View all N neighbors →" link navigates to full detail page with `?section=node-neighbors` ### Deep linking - `?section=node-neighbors` auto-scrolls to the neighbors section (uses existing scroll mechanism) ### Data fetching - `GET /api/nodes/{pubkey}/neighbors` via existing `api()` helper - Cached per-node for 5 minutes (panel lifetime) - Loading spinner, empty state, error state ### States - **Loading**: spinner with "Loading neighbors…" - **Empty**: "No neighbor data available yet. Neighbor relationships are built from observed packet paths over time." - **Error**: "Could not load neighbor data" ## Tests - 2 new Playwright E2E tests: 1. Section exists with correct table columns (or empty state) 2. Loading spinner visible during fetch ## Files Changed - `public/nodes.js` — neighbor section rendering + data fetching helpers - `test-e2e-playwright.js` — 2 new E2E tests --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
15634362c9 |
feat: neighbor graph visualization in analytics (#482) — milestone 7 (#513)
## Summary Adds a **Neighbor Graph** tab to the Analytics page — an interactive force-directed graph visualization of the mesh network's neighbor affinity data. Part of #482 (Milestone 7 — Analytics Graph Visualization) ## What's New ### Neighbor Graph Tab - New "Neighbor Graph" tab in the analytics tab bar - Force-directed graph layout using HTML5 Canvas (vanilla JS, no external libs) - Nodes rendered as circles, colored by role using existing `ROLE_COLORS` - Edges as lines with thickness proportional to affinity score - Ambiguous edges highlighted in yellow ### Interactions - **Click node** → navigates to node detail page (`#/nodes/{pubkey}`) - **Hover node** → tooltip showing name, role, neighbor count - **Drag nodes** → rearrange layout interactively - **Mouse wheel** → zoom in/out (towards cursor position) - **Drag background** → pan the view ### Filters - **Role checkboxes** — toggle repeater, companion, room, sensor visibility - **Minimum score slider** — filter out weak edges (0.00–1.00) - **Confidence filter** — show all / high confidence only / hide ambiguous ### Stats Summary Displays above the graph: total nodes, total edges, average score, resolved %, ambiguous count ### Data Source Uses `GET /api/analytics/neighbor-graph` endpoint from M2, with region filtering via the shared RegionFilter component. ## Performance - Canvas-based rendering (not SVG) for performance with large graphs - Force simulation uses `requestAnimationFrame` with cooling/dampening — stops iterating when layout stabilizes - O(n²) repulsion is acceptable for typical mesh sizes (~500 nodes); for larger meshes, a Barnes-Hut approximation could be added later - Animation frame is properly cleaned up on page destroy ## Tests - Updated tab count assertion (≥10 tabs) - New Playwright test: tab loads, canvas renders, stats shown (≥3 stat cards) - New Playwright test: filter changes update stats ## Files Changed - `public/analytics.js` — new tab + full graph visualization implementation - `test-e2e-playwright.js` — 2 new tests + updated assertion --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
5151030697 |
feat: affinity-aware hop resolution (#482) — milestone 4 (#511)
## Summary Milestone 4 of #482: adds affinity-aware hop resolution to improve disambiguation accuracy across all hop resolution in the app. ### What changed **Backend — `prefixMap.resolveWithContext()` (store.go)** New method that applies a 4-tier disambiguation priority when multiple nodes match a hop prefix: | Priority | Strategy | When it wins | |----------|----------|-------------| | 1 | **Affinity graph score** | Neighbor graph has data, score ratio ≥ 3× runner-up | | 2 | **Geographic proximity** | Context nodes have GPS, pick closest candidate | | 3 | **GPS preference** | At least one candidate has coordinates | | 4 | **First match** | No signal — current naive fallback | The existing `resolve()` method is unchanged for backward compatibility. New callers that have context (originator, observer, adjacent hops) can use `resolveWithContext()` for better results. **API — `handleResolveHops` (routes.go)** Enhanced `/api/resolve-hops` endpoint: - New query params: `from_node`, `observer` — provide context for affinity scoring - New response fields on `HopCandidate`: `affinityScore` (float, 0.0–1.0) - New response fields on `HopResolution`: `bestCandidate` (pubkey when confident), `confidence` (one of `unique_prefix`, `neighbor_affinity`, `ambiguous`) - Backward compatible: without context params, behavior is identical to before (just adds `confidence` field) **Types (types.go)** - `HopCandidate.AffinityScore *float64` - `HopResolution.BestCandidate *string` - `HopResolution.Confidence string` ### Tests - 7 unit tests for `resolveWithContext` covering all 4 priority tiers + edge cases - 2 unit tests for `geoDistApprox` - 4 API tests for enhanced `/api/resolve-hops` response shape - All existing tests pass (no regressions) ### Impact This improves ALL hop resolution across the app — analytics, route display, subpath analysis, and any future feature that resolves hop prefixes. The affinity graph (from M1/M2) now feeds directly into disambiguation decisions. Part of #482 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
813b424ca1 |
fix: Show Neighbors uses affinity API for collision disambiguation (#484) — milestone 3 (#512)
## Summary
Replace broken client-side path walking in `selectReferenceNode()` with
server-side `/api/nodes/{pubkey}/neighbors` API call, fixing #484 where
Show Neighbors returned zero results due to hash collision
disambiguation failures.
**Fixes #484** | Part of #482
## What changed
### `public/map.js` — `selectReferenceNode()` function
**Before:** Client-side path walking — fetched
`/api/nodes/{pubkey}/paths`, walked each path to find hops adjacent to
the selected node by comparing full pubkeys. This fails on hash
collisions because path hops only contain short prefixes (1-2 bytes),
and the hop resolver can pick the wrong collision candidate.
**After:** Server-side affinity resolution — fetches
`/api/nodes/{pubkey}/neighbors?min_count=3` which uses the neighbor
affinity graph (built in M1/M2) to return disambiguated neighbors. For
ambiguous edges, all candidates are included in the neighbor set (better
to show extra markers than miss real neighbors).
**Fallback:** When the affinity API returns zero neighbors (cold start,
insufficient data), the function falls back to the original path-walking
approach. This ensures the feature works even before the affinity graph
has accumulated enough observations.
## Tests
4 new Playwright E2E tests (in both `test-show-neighbors.js` and
`test-e2e-playwright.js`):
1. **Happy path** — Verifies the `/neighbors` API is called and the
reference node UI activates
2. **Hash collision disambiguation** — Two nodes sharing prefix "C0" get
different neighbor sets via the affinity API (THE critical test for
#484)
3. **Fallback to path walking** — Empty affinity response triggers
fallback to `/paths` API
4. **Ambiguous candidates** — Ambiguous edge candidates are included in
the neighbor set
All tests use Playwright route interception to mock API responses,
testing the frontend logic independently of server state.
## Spec reference
See [neighbor-affinity-graph.md](docs/specs/neighbor-affinity-graph.md),
sections:
- "Replacing Show Neighbors on the map" (lines ~461-504)
- "Milestone 3: Show Neighbors Fix (#484)" (lines ~1136-1152)
- Test specs a & b (lines ~754-800)
---------
Co-authored-by: you <you@example.com>
|
||
|
Kpa-clawbotandyou
|
e66085092e |
feat: neighbor affinity API endpoints (#482) — milestone 2 (#508)
## Summary Milestone 2 of the neighbor affinity graph (#482). Adds two API endpoints that expose the neighbor graph built in M1 (PR #507). ### Endpoints #### `GET /api/nodes/{pubkey}/neighbors` Returns neighbors for a specific node with affinity scores. **Query params:** `min_count` (default 1), `min_score` (default 0.0), `include_ambiguous` (default true) **Response shape:** ```json { "node": "pubkey", "neighbors": [ { "pubkey": "...", "prefix": "BB", "name": "...", "role": "repeater", "count": 847, "score": 0.95, "first_seen": "...", "last_seen": "...", "avg_snr": -8.2, "observers": ["obs1"], "ambiguous": false } ], "total_observations": 847 } ``` Ambiguous entries have `candidates` array; unresolved prefixes have `unresolved: true`. #### `GET /api/analytics/neighbor-graph` Returns full graph summary for analytics/visualization. **Query params:** `min_count` (default 5), `min_score` (default 0.1), `region` (IATA code filter) **Response shape:** ```json { "nodes": [{ "pubkey": "...", "name": "...", "role": "...", "neighbor_count": 5 }], "edges": [{ "source": "...", "target": "...", "weight": 847, "score": 0.95, "ambiguous": false }], "stats": { "total_nodes": 42, "total_edges": 87, "ambiguous_edges": 3, "avg_cluster_size": 4.2 } } ``` ### Wiring - `NeighborGraph` + `neighborMu` added to `Server` struct - Lazy initialization: graph built on first API call, cached with 60s TTL - Node name/role lookups via existing `getCachedNodesAndPM()` - Region filtering via existing `resolveRegionObservers()` ### Tests (15 tests) - Empty graph, single neighbor, multiple neighbors (sorted by score) - Ambiguous candidates with candidate list - Unresolved prefix (orphan) with `unresolved: true` - `min_count` filter, `min_score` filter, `include_ambiguous=false` filter - Unknown node returns 200 with empty neighbors - Graph endpoint: empty, with edges, default min_count, ambiguous count - Region filter (graceful when no store) - Response shape validation (all required keys present) All existing tests continue to pass. Part of #482 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
4a56be0b48 |
feat: neighbor affinity graph builder (#482) — milestone 1 (#507)
## Summary Milestone 1 of 7 for the neighbor affinity graph feature (#482). Implements the core `NeighborGraph` data structure and `BuildFromStore()` algorithm. **Spec:** `docs/specs/neighbor-affinity-graph.md` on `spec/482-neighbor-affinity` branch. ## What's Built ### `cmd/server/neighbor_graph.go` - **`NeighborGraph` struct** — thread-safe (sync.RWMutex) in-memory graph with edge map and per-node index - **`BuildFromStore(*PacketStore)`** — iterates all packets/observations to extract first-hop edges: - `originator ↔ path[0]` for ADVERT packets only (originator identity known) - `observer ↔ path[last]` for ALL packet types - Zero-hop ADVERTs: `originator ↔ observer` direct edge - **Affinity scoring** — `score = min(1.0, count/100) × exp(-λ × hours)` with 7-day half-life - **Jaccard disambiguation** — resolves ambiguous hash prefixes using mutual-neighbor overlap - **Confidence threshold** — auto-resolve only when best ≥ 3× second-best AND ≥ 3 observations - **Transitivity poisoning guard** — only fully-resolved edges used as evidence - **Orphan prefix handling** — unknown prefixes stored as unresolved markers - **Cache management** — 60s TTL, `IsStale()` check for rebuild triggering ### `cmd/server/neighbor_graph_test.go` 22 unit tests covering all spec requirements: | Test | What it validates | |------|-------------------| | EmptyStore | Empty graph from empty store | | AdvertSingleHopPath | Both edge types from single-hop ADVERT | | AdvertMultiHopPath | originator↔path[0] + observer↔path[last] | | AdvertZeroHop | Direct originator↔observer edge | | NonAdvertEmptyPath | No edges from non-ADVERT empty path | | NonAdvertOnlyObserverEdge | Only observer↔last_hop for non-ADVERTs | | NonAdvertSingleHop | observer↔path[0] only | | HashCollision | Ambiguous edge with candidates | | JaccardScoring | Jaccard coefficient computation | | ConfidenceAutoResolve | Auto-resolve when ratio ≥ 3× | | EqualScoresAmbiguous | Remains ambiguous with equal scores | | ObserverSelfEdgeGuard | No self-edges | | OrphanPrefix | Unresolved prefix handling | | AffinityScore_Fresh | Score ≈ 1.0 for fresh high-count | | AffinityScore_Decayed | Score ≈ 0.5 at 7-day half-life | | AffinityScore_LowCount | Score ≈ 0.05 for count=5 | | AffinityScore_StaleAndLow | Score ≈ 0 for old low-count | | CountAccumulation | 5 observations → count=5 | | MultipleObservers | Observer set tracks all witnesses | | TimeDecayOldObservations | Month-old edge scores very low | | ADVERTOnlyConstraint | Non-ADVERTs don't create originator edges | | CacheTTL | Stale detection works correctly | ## Not in scope (future milestones) - API endpoints (M2) - Frontend integration (M3-M5) - Debug tools (M6) - Analytics visualization (M7) Part of #482 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
64745f89b1 |
feat: customizer v2 — event-driven state management (#502) (#503)
## Summary Implements the customizer v2 per the [approved spec](docs/specs/customizer-rework.md), replacing the v1 customizer's scattered state management with a clean event-driven architecture. Resolves #502. ## What Changed ### New: `public/customize-v2.js` Complete rewrite of the customizer as a self-contained IIFE with: - **Single localStorage key** (`cs-theme-overrides`) replacing 7 scattered keys - **Three state layers:** server defaults (immutable) → user overrides (delta) → effective config (computed) - **Full data flow pipeline:** `write → read-back → merge → atomic SITE_CONFIG assign → apply CSS → dispatch theme-changed` - **Color picker optimistic CSS** (Decision #12): `input` events update CSS directly for responsiveness; `change` events trigger the full pipeline - **Override indicator dots** (●) on each field — click to reset individual values - **Section-level override count badges** on tabs - **Browser-local banner** in panel header: "These settings are saved in your browser only" - **Auto-save status indicator** in footer: "All changes saved" / "Saving..." / "⚠️ Storage full" - **Export/Import** with full shape validation (`validateShape()`) - **Presets** flow through the standard pipeline (`writeOverrides(presetData) → pipeline`) - **One-time migration** from 7 legacy localStorage keys (exact field mapping per spec) - **Validation** on all writes: color format, opacity range, timestamp enum values - **QuotaExceededError handling** with visible user warning ### Modified: `public/app.js` Replaced ~80 lines of inline theme application code with a 15-line `_customizerV2.init(cfg)` call. The customizer v2 handles all merging, CSS application, and global state updates. ### Modified: `public/index.html` Swapped `customize.js` → `customize-v2.js` script tag. ### Added: `docs/specs/customizer-rework.md` The full approved spec, included in the repo for reference. ## Migration On first page load: 1. Checks if `cs-theme-overrides` already exists → skip if yes 2. Reads all 7 legacy keys (`meshcore-user-theme`, `meshcore-timestamp-*`, `meshcore-heatmap-opacity`, `meshcore-live-heatmap-opacity`) 3. Maps them to the new delta format per the spec's field-by-field mapping 4. Writes to `cs-theme-overrides`, removes all legacy keys 5. Continues with normal init Users with existing customizations will see them preserved automatically. ## Dark/Light Mode - `theme` section stores light mode overrides, `themeDark` stores dark mode overrides - `meshcore-theme` localStorage key remains **separate** (view preference, not customization) - Switching modes re-runs the full pipeline with the correct section ## Testing - All existing tests pass (`test-packet-filter.js`, `test-aging.js`, `test-frontend-helpers.js`) - Old `customize.js` is NOT modified — left in place for reference but no longer loaded ## Not in Scope (per spec) - Undo/redo stack - Cross-tab synchronization - Server-side admin import endpoint - Map config / geo-filter overrides --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
c9c473279e |
fix: add null-guards to rAF callbacks in live page animations (#506)
## Summary Fixes #483 — navigating away from the live page while matrix/hop animations are running throws `TypeError: Cannot read properties of null (reading 'addLayer')`. ## Root Cause `destroy()` sets `animLayer = null` and `pathsLayer = null`, but in-flight `requestAnimationFrame` callbacks continue executing and attempt to call `.addTo(animLayer)` or `.removeLayer()` on the now-null references. The entry guards at the top of `drawMatrixLine()` and `drawAnimatedLine()` only protect the initial call — not the rAF continuation loops inside `tick()`, `fadeOut()`, `animateLine()`, and `animateFade()`. ## Fix Added null-guards (`if (!animLayer || !pathsLayer) return`) at the top of all four rAF callback functions in `live.js`: 1. **`tick()`** (line ~2203) — matrix animation main loop 2. **`fadeOut()`** (line ~2253) — matrix animation fade-out 3. **`animateLine()`** (line ~2302) — standard line animation main loop 4. **`animateFade()`** (line ~2337) — standard line fade-out This pattern is already used elsewhere in the file (e.g., line 1873, 1886) for the same purpose. ## Testing - All unit tests pass (`npm test` — 0 failures) - Go server tests pass (`cmd/server` + `cmd/ingestor`) - Change is defensive only (early return on null) — no behavioral change when layers exist --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
ad97c0fdd1 |
fix: clear stale parsed cache on observation packets (#505)
## Summary Fixes #504 — Expanding a packet in the packets UI showed the same path on every observation instead of each observation's unique path. ## Root Cause PR #400 (fixing #387) added caching of `JSON.parse` results as `_parsedPath` and `_parsedDecoded` properties on packet objects. When observation packets are created via object spread (`{...parentPacket, ...obs}`), these cache properties are copied from the parent. Subsequent calls to `getParsedPath(obsPacket)` hit the stale cache and return the parent's path, ignoring the observation's own `path_json`. ## Fix After every object spread that creates an observation packet from a parent packet, delete the cache properties so they get re-parsed from the observation's own data: ```js delete obsPacket._parsedPath; delete obsPacket._parsedDecoded; ``` Applied to all 5 spread sites in `public/packets.js`: - Line 271: detail pane observation selection - Line 504: flat view observation expansion - Line 840: grouped view observation expansion - Line 1012: child observation selection in grouped view - Line 1982: WebSocket live update observation expansion ## Tests Added 2 new tests in `test-frontend-helpers.js`: 1. Verifies observation packets get their own path after cache invalidation (not the parent's) 2. Verifies observation path differs from parent path after cache invalidation All 431 frontend helper tests pass. All 62 packet filter tests pass. --------- Co-authored-by: you <you@example.com> |
||
|
|
c7f655e419 |
perf(frontend): cache JSON.parse results for packet data (#400)
## Problem As described in #387, `JSON.parse()` is called repeatedly on the same packet data across render cycles. With 30K packets, each render cycle parses 60K+ JSON strings unnecessarily. ## Analysis The server sends `decoded_json` and `path_json` as JSON strings. The frontend parses them on-demand in multiple locations: - `renderTableRows()` — for every row, every render - WebSocket handling — when processing filtered packets - `loadPackets()` — during packet loading - Detail view rendering — when showing packet details This creates O(n×m) parsing overhead where n = packet count and m = render cycles. ## Solution Add cached parse helpers that store parsed results on the packet object: ```javascript function getParsedPath(p) { if (p._parsedPath === undefined) { try { p._parsedPath = JSON.parse(p.path_json || '[]'); } catch { p._parsedPath = []; } } return p._parsedPath; } ``` Same pattern for `getParsedDecoded()`. ## Changes - `public/packets.js`: Add helpers + replace 15+ JSON.parse calls - `public/live.js`: Add helpers + replace 5 JSON.parse calls ## Benchmarks Before: 60K+ JSON.parse calls per render cycle (30K packets) After: ~30K parse calls (one per packet, cached thereafter) Memory impact: Negligible (stores parsed objects that were already created temporarily) ## Notes - Cache uses `undefined` check to distinguish "not cached" from "cached empty result" - Property names `_parsedPath` and `_parsedDecoded` prefixed to avoid collision with server fields - No breaking changes to existing code paths Fixes #387 --------- Co-authored-by: P. Clawmogorov <262173731+Alm0stSurely@users.noreply.github.com> Co-authored-by: you <you@example.com> |
||
|
|
b1d89d7d9f |
fix: apply region filter in GetNodes — was silently ignored (#496) (#497)
## Summary - `db.GetNodes` accepted a `region` param from the HTTP handler but never used it — every region-filter selection was silently ignored and all nodes were always returned - Added a subquery filtering `nodes.public_key` against ADVERT transmissions (payload_type=4) observed by observers with matching IATA codes - Handles both v2 (`observer_id TEXT`) and v3 (`observer_idx INT`) schemas ## Test plan - [x] 4 new subtests added to `TestGetNodesFiltering`: SJC (1 node), SFO (1 node), SJC,SFO multi (1 node deduped), AMS unknown (0 nodes) - [x] All existing Go tests still pass - [x] Deploy to staging, open `/nodes`, select a region in the filter bar — only nodes observed by observers in that region should appear Closes #496 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: you <you@example.com> |
||
|
|
c173ab7e80 |
perf: skip JSON parse in indexByNode when no pubkey fields present (#376) (#499)
## Summary - `indexByNode` was calling `json.Unmarshal` for every packet during `Load()` and `IngestNewFromDB()`, even channel messages and other payloads that can never contain node pubkey fields - All three target fields (`"pubKey"`, `"destPubKey"`, `"srcPubKey"`) share the common substring `"ubKey"` — added a `strings.Contains` pre-check that skips the JSON parse entirely for packets that don't match - At 30K+ packets on startup, this eliminates the majority of `json.Unmarshal` calls in `indexByNode` (channel messages, status packets, etc. all bypass it) ## Test plan - [x] 5 new subtests in `TestIndexByNodePreCheck`: ADVERT with pubKey indexed, destPubKey indexed, channel message skipped, empty JSON skipped, duplicate hash deduped - [x] All existing Go tests pass - [x] Deploy to staging and verify node-filtered packet queries still work correctly Closes #376 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: you <you@example.com> |
||
|
Jukka Väisänenandyou
|
4664c90db4 |
fix: skip zero-hop adverts when checking node hash size (#493)
Fixes issue router IDs flapping between 1byte and multi-byte as described in https://github.com/Kpa-clawbot/CoreScope/issues/303 with a minimal patch + test coverage. This fix is critical for regions using multi-byte IDs. Closes https://github.com/Kpa-clawbot/CoreScope/issues/303 --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
2755dc3875 |
test: push ingestor coverage from 70% to 84% (#344) (#492)
## Summary Push Go ingestor test coverage from **70.2% → 84.0%** (92.8% excluding the untestable `main()` and `init()` functions). Part of #344 — ingestor coverage ## What Changed Added `coverage_boost_test.go` with 60+ new test functions covering previously untested code paths: ### Coverage Before → After by Function | Function | Before | After | |----------|--------|-------| | `NodeDaysOrDefault` | 0% | 100% | | `MoveStaleNodes` | 0% | 76.5% | | `NodePassesGeoFilter` | 40% | 100% | | `handleMessage` | 41.4% | 92.1% | | `ResolvedSources` | 71.4% | 100% | | `extractObserverMeta` | 100% | 100% | | `decodeAdvert` | 88.2% | 94.1% | | `decryptChannelMessage` | 88.4% | 93.0% | | **Total** | **70.2%** | **84.0%** | ### Test Categories Added - **Config**: `NodeDaysOrDefault` all branches, broker scheme normalization (`mqtt://` → `tcp://`, `mqtts://` → `ssl://`) - **Database**: `MoveStaleNodes` (stale/fresh/replace), duplicate transmission handling, default timestamps, telemetry updates, schema migration verification - **Decoder**: Sensor telemetry parsing, location + features with truncated data, `countNonPrintable` with invalid UTF-8, `decryptChannelMessage` error paths (invalid key/MAC/ciphertext/alignment), short payload handling - **Geo Filter**: All branches (nil filter, nil coords, inside/outside) - **Message Handler**: Channel messages (with/without sender, empty text), direct messages, geo-filtered adverts, corrupted adverts (all-zero pubkey), non-advert packets, `Score`/`Direction` case-insensitive fallbacks, status messages with full hardware metadata ### Why Not 90%+ The remaining ~16% uncovered statements are: - `main()` function (68 blocks) — program entry point with MQTT client setup, signal handling, goroutines — not unit-testable without major refactoring - `init()` function — `--version` flag + `os.Exit(0)` — kills the test process - `prepareStatements()` error returns — only trigger on corrupted/incompatible SQLite databases - `applySchema()` migration error paths — only trigger on filesystem/SQLite failures Excluding `main()` and `init()`, effective coverage is **92.8%**. ## Test Results All 100+ tests pass (existing + new): ``` ok github.com/corescope/ingestor 25.945s coverage: 84.0% of statements ``` --------- Co-authored-by: you <you@example.com> |
||
|
|
5228e67604 |
fix: use packet timestamp in bufferPacket instead of arrival time (#475) (#491)
## Summary - `bufferPacket()` was overwriting `_ts` with `Date.now()` (receive time) for every live WS packet - Packets arriving in the same batch all got identical timestamps, making the message history show the same "Xs ago" for every entry (e.g., all show "5s ago") - Fix: use `pkt.timestamp || pkt.created_at` (mirroring `dbPacketToLive`) so each packet reflects its actual origination time, falling back to `Date.now()` only when the packet has no timestamp ## Root cause ```js // before pkt._ts = Date.now(); // after pkt._ts = new Date(pkt.timestamp || pkt.created_at || Date.now()).getTime(); ``` The WS broadcast includes `timestamp` (= `tx.FirstSeen`) in the packet map (store.go:1182), so the field is always present for real packets. ## Test plan - [x] Open Live page, observe packets arriving — each should show its own relative time, not all the same value - [x] `node test-frontend-helpers.js` passes (235 tests, 0 failures) Closes #475 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
698514e5e6 |
test: comprehensive live.js coverage (71 tests) (#489)
## Summary Add comprehensive test coverage for `live.js` — the largest and most complex frontend file (2500+ lines) covering animation modes, VCR playback, WebSocket handling, audio integration, and the live map. Part of #344 — live.js coverage. ## What's Tested (71 tests) ### Pure function tests via `vm.createContext` - **`dbPacketToLive`** — DB packet → live format conversion, null `decoded_json`, `payload_type_name` fallback, `created_at` timestamp fallback - **`expandToBufferEntries`** — observation expansion (1→N entries), empty observations, multi-packet batches - **`SEG_MAP`** — 7-segment LCD digit mapping completeness (all digits, colon, space, VCR mode letters) - **VCR state machine** — mode transitions (`LIVE`→`PAUSED`→`REPLAY`), `frozenNow` lifecycle, speed cycling (1→2→4→8→1), pause idempotency - **`getFavoritePubkeys`** — localStorage merging from `meshcore-favorites` + `meshcore-my-nodes`, corrupt data handling, falsy filtering - **`packetInvolvesFavorite`** — sender pubKey matching, hop prefix matching, missing decoded fields - **`isNodeFavorited`** — basic favorite lookup, empty state - **`formatLiveTimestampHtml`** — timestamp formatting with tooltip, null input, numeric input, future warning icon - **`resolveHopPositions`** — HopResolver integration, ghost hop interpolation between known nodes - **`bufferPacket`** — VCR buffer management, 2000-entry cap with playhead adjustment, missed count in PAUSED mode ### Source-level safety checks (20 tests) - Null guards: `renderPacketTree`, `animatePath`, `pulseNode`, `nextHop` (all verified via source-level checks) - Animation limit enforcement (`MAX_CONCURRENT_ANIMS`) - Tab visibility optimization (skip animations when hidden, clear propagation buffer on restore) - WebSocket auto-reconnect - `addNodeMarker` deduplication - All toggle state persistence to localStorage (matrix, rain, realistic, favorites, ghost hops) - `clearNodeMarkers` resets HopResolver - `startReplay` pre-aggregates by hash - Orientation change retry delays - `vcrRewind` deduplicates buffer entries by ID ## Changes - `public/live.js` — expose 14 additional functions via `window._live*` for testing (following existing pattern) - `test-live.js` — new test file, 841 lines, 71 tests ## Constraints - No new dependencies - Tests run via `vm.createContext` against real code (not copies) - No build step — vanilla JS --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
cf3a383bb2 |
test: comprehensive app.js coverage — 100+ new tests (#490)
## Summary Adds 100+ new tests for previously untested `app.js` functions, significantly improving frontend coverage toward the 90%+ target. ## What's Tested All pure/testable functions from `app.js` that lacked coverage: | Function Group | Tests Added | Description | |---|---|---| | `payloadTypeColor` | 13 | All PAYLOAD_COLORS mappings + unknown/null/undefined fallback | | `pad2` / `pad3` | 10 | Zero-padding for 1-3 digit values, no truncation | | `formatIsoLike` | 5 | UTC/local timezone, with/without milliseconds, zero-padding | | `formatTimestampCustom` | 5 | Token replacement (YYYY/MM/DD/HH/mm/ss/SSS/Z), partial formats, invalid format rejection | | `formatAbsoluteTimestamp` | 3 | Custom format integration, locale+UTC, null/invalid date handling | | `getTimestamp*` getters | 11 | localStorage priority, server config fallback, invalid value rejection for Mode/Timezone/FormatPreset/CustomFormat | | `invalidateApiCache` | 3 | Prefix-based selective invalidation, full clear, cache→invalidate→re-fetch lifecycle | | `formatHex` | 5 | Byte spacing, single byte, null/empty, odd-length hex | | `createColoredHexDump` | 6 | Range-based coloring, override precedence, null/empty hex+ranges | | `buildHexLegend` | 5 | Label deduplication, correct swatch colors per label class, null/empty | | Favorites (`getFavorites`/`isFavorite`/`toggleFavorite`/`favStar`) | 9 | CRUD operations, corrupt JSON resilience, star HTML rendering with custom classes | | `debounce` | 3 | Delay behavior, timer reset on rapid calls, argument forwarding | | `mergeUserHomeConfig` | 5 | Null/missing siteConfig/userTheme, non-object home, missing home creation | | Constants | 2 | Exhaustive ROUTE_TYPES (4) and PAYLOAD_TYPES (13) mapping verification | ## Approach - Tests use the existing `vm.createContext` sandbox pattern from `test-frontend-helpers.js` - Tests the **real code** loaded from `public/app.js` — no copies - No new dependencies - Each `invalidateApiCache` test uses an isolated sandbox to avoid async race conditions ## Test Results ``` Frontend helpers: 343 passed, 0 failed ``` Part of #344 — app.js coverage --------- Co-authored-by: you <you@example.com> |
||
|
efitenandClaude Sonnet 4.6
|
a45ac71508 |
fix: restore color-coded hex breakdown in packet detail (#329) (#500)
## Summary
- `BuildBreakdown` was never ported from the deleted Node.js
`decoder.js` to Go — the server has returned `breakdown: {}` since the
Go migration (commit `742ed865`), so `createColoredHexDump()` and
`buildHexLegend()` in the frontend always received an empty `ranges`
array and rendered everything as monochrome
- Implemented `BuildBreakdown()` in `decoder.go` — computes labeled byte
ranges matching the frontend's `LABEL_CLASS` map: `Header`, `Transport
Codes`, `Path Length`, `Path`, `Payload`; ADVERT packets get sub-ranges:
`PubKey`, `Timestamp`, `Signature`, `Flags`, `Latitude`, `Longitude`,
`Name`
- Wired into `handlePacketDetail` (was `struct{}{}`)
- Also adds per-section color classes to the field breakdown table
(`section-header`, `section-transport`, `section-path`,
`section-payload`) so the table rows get matching background tints
## Test plan
- [x] Open any packet detail pane — hex dump should show color-coded
sections (red header, orange path length, blue transport codes, green
path hops, yellow/colored payload)
- [x] Legend below action buttons should appear with color swatches
- [x] ADVERT packets: PubKey/Timestamp/Signature/Flags each get their
own distinct color
- [x] Field breakdown table section header rows should be tinted per
section
- [x] 8 new Go tests: all pass
Closes #329
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Kpa-clawbotandyou
|
016b87b33c |
test: add 64 unit tests for packets.js (Part of #344) (#488)
## Summary Adds 64 unit tests for `packets.js` — the largest untested frontend file (2000+ lines) covering filter engine integration, time window logic, groupByHash rendering, and packet detail display. Part of #344 — packets.js coverage. ## Approach Follows the existing `test-frontend-helpers.js` pattern: loads real source files into a `vm.createContext` sandbox and tests actual code (no copies). Added a `window._packetsTestAPI` export at the end of the packets.js IIFE to expose pure functions for testing without changing any runtime behavior. ## What's Tested | Function | Tests | What it covers | |----------|-------|----------------| | `typeName` | 2 | Type code → name mapping, unknown fallback | | `obsName` | 2 | Observer name lookup, falsy/missing handling | | `kv` | 1 | Key-value HTML helper | | `sectionRow` / `fieldRow` | 3 | Table section/field HTML builders | | `getDetailPreview` | 17 | All packet types: CHAN, ADVERT (repeater/room/sensor/companion), GRP_TXT (no_key/decryption_failed/channelHashHex), TXT_MSG, PATH, REQ, RESPONSE, ANON_REQ, text fallback, public_key fallback, empty | | `getPathHopCount` | 4 | Valid path, empty, null, invalid JSON | | `sortGroupChildren` | 3 | Default observer sort, header update, null safety | | `renderTimestampCell` | 2 | Timestamp HTML output, null handling | | `renderPath` | 3 | Empty/null, multi-hop with arrows, single hop | | `renderDecodedPacket` | 6 | Header/path/payload/nested objects/null skip/raw hex | | `buildFieldTable` | 11 | All payload types (ADVERT with flags/location/name, GRP_TXT, CHAN, ACK, destHash, raw fallback), transport codes, path hops, hash_size calculation, empty hex | | `_getRowCount` | 1 | Virtual scroll row counting | | `buildFlatRowHtml` | 3 | Row rendering, size calculation, missing hex | | `buildGroupRowHtml` | 3 | Single/multi group, observation badge | | Test API exposure | 1 | Verifies window._packetsTestAPI | ## Constraints Met - No new test dependencies - Tests real code via `vm.createContext`, not copies - No build step — vanilla JS - All existing tests still pass (254 frontend-helpers, 62 packet-filter, 29 aging) Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
889107a5e1 |
fix: address PR #487 review feedback (#501)
## Summary Addresses review feedback from PR #487 (nodes.js coverage). ### Changes 1. **Replace fragile `exportInternals` regex source patching with stable test hooks** — `getStatusInfo` and `getStatusTooltip` are now exposed via `window._nodesGetStatusInfo` and `window._nodesGetStatusTooltip`, matching the existing pattern used by all other test-accessible functions. The brittle regex `.replace()` approach that modified source code at runtime has been removed entirely. 2. **Strengthen weak null assertion** — The `renderNodeTimestampHtml handles null` test previously asserted `html.includes('—') || html.length > 0`, which is a near-tautology (any non-empty string passes). Now strictly asserts `html.includes('—')`. ### Files changed - `public/nodes.js` — 2 new test hook lines - `test-frontend-helpers.js` — removed 21-line `exportInternals` branch, updated tests to use hooks ### Testing - All 309 frontend helper tests pass - All 62 packet filter tests pass - All 29 aging tests pass Closes review items from #487. Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
50f94603c1 |
test: P0 coverage for nodes.js — sort, status, timestamps, sync (#487)
## Summary Add 67 new unit tests for `nodes.js`, raising frontend helper test count from 233 to 300. Part of #344 — nodes.js coverage. ## What's Tested ### Sort System (`toggleSort`, `sortNodes`, `sortArrow`) - Direction toggling on same column (asc↔desc) - Default sort directions per column type (name→asc, last_seen→desc, advert_count→desc) - localStorage persistence of sort state - All 5 sort columns: `name`, `public_key`, `role`, `last_seen`, `advert_count` - Both ascending and descending for each column - Case-insensitive name sorting - Unnamed nodes sort last - Timestamp fallback chain: `last_heard` → `last_seen` → 0 - Missing timestamp handling - Empty array edge case - Unknown column graceful handling - `sortArrow` rendering for active (▲/▼) and inactive columns ### Status Calculation (`getStatusInfo`, `getStatusTooltip`) - `_lastHeard` takes priority over `last_heard` - `last_seen` used as fallback when `last_heard` missing - No-timestamp nodes return stale with `lastHeardMs: 0` - Infrastructure threshold (72h) for rooms - Standard threshold (24h) for sensors and companions - Explanation text varies by role and status - Unknown role defaults to gray color `#6b7280` - All role/status tooltip combinations ### Timestamp Rendering (`renderNodeTimestampHtml`, `renderNodeTimestampText`) - HTML output includes tooltip and `timestamp-text` class - Future timestamps show ⚠️ warning icon - Null input produces dash - Text output is plain (no HTML tags) ### Favorites Sync (`syncClaimedToFavorites`) - Claimed pubkeys added to favorites - No-op when all already synced - Empty my-nodes handled - Missing localStorage keys don't crash ## Implementation - Added test hooks on `window` for closure-scoped functions (non-invasive, follows existing pattern) - Tests use `vm.createContext` to load real `nodes.js` code — no copies - No new dependencies ## Test Results ``` Frontend helpers: 300 passed, 0 failed ``` --------- Co-authored-by: you <you@example.com> |
||
|
efiten
|
b799f54700 |
perf: bound memory growth and reduce render CPU on packets page (#421)
## Problem On a long-running session the packets page consumed 8 GB of browser memory and 20%+ CPU on an 8-core machine. Root causes: 1. **Unbounded `packets` array growth via WebSocket** — `packets.unshift()` was called for every new unique hash, but nothing ever trimmed the array. After hours of live traffic the array grew well past the initial 50 k load limit. 2. **Unbounded `pauseBuffer`** — all WS messages queued while paused, no cap. 3. **Unbounded `_children` growth** — expanded groups received a `unshift(p)` on every matching WS message with no size limit. 4. **O(n) `observers.find()` inside the O(n) render loop** — with 50 k rows, each render triggered up to 50 k linear scans through the observers list. 5. **Full DOM rebuild on every WS message** — `renderTableRows()` was called synchronously on every WebSocket batch, reconstructing the entire table on each incoming packet. ## Changes - `packets[]` is now trimmed to `PACKET_LIMIT` after each WS batch; evicted entries are also removed from `hashIndex` to prevent stale references. - `pauseBuffer` capped at 2 000 entries (oldest dropped). - `_children` capped at 200 entries on WS prepend. - `renderTableRows()` on the WS path is debounced to 200 ms, batching rapid updates into a single redraw. - `observersById = new Map()` pre-built from the observers array; all `observers.find()` calls in the render loop and WS filter replaced with O(1) `Map.get()`. ## Test plan - [x] Load the packets page and leave it running for several minutes with live WebSocket traffic — memory in DevTools should remain stable rather than growing continuously - [x] Pause live updates, wait for several messages, then resume — buffer replays correctly and display updates - [x] Expand a packet group and leave it open during live traffic — children update but don't grow past 200 - [x] Region filter still works correctly (relies on the observer Map lookup) - [x] Observer name / IATA badge renders correctly in grouped and flat mode 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Kpa-clawbotandyou
|
d5b300a8ba |
fix: derive version from git tags instead of package.json (#486)
## Summary Fixes #485 — the app version was derived from `package.json` via Node.js, which is a meaningless artifact for this Go project. This caused version mismatches (e.g., v3.3.0 release showing "3.2.0") when someone forgot to bump `package.json`. ## Changes ### `manage.sh` - **Line 43**: Replace `node -p "require('./package.json').version"` with `git describe --tags --match "v*"` — version is now derived automatically from git tags - **Line 515**: Add `--force` to `git fetch origin --tags` in setup command - **Line 1320**: Add `--force` to `git fetch origin --tags` in update command — prevents "would clobber existing tag" errors when tags are moved ### `package.json` - Version field set to `0.0.0-use-git-tags` to make it clear this is not the source of truth. File kept because npm scripts and devDependencies are still used for testing. ## How it works `git describe --tags --match "v*"` produces: - `v3.3.0` — when on an exact tag - `v3.3.0-3-gabcdef1` — when 3 commits after a tag (useful for debugging) - Falls back to `unknown` if no tags exist ## Testing - All Go tests pass (`cmd/server`, `cmd/ingestor`) - All frontend unit tests pass (254/254) - No changes to application logic — only build-time version derivation Co-authored-by: you <you@example.com> |
||
|
you
|
2af4259eca | chore: bump version to 3.3.0 v3.3.0 | ||
|
|
bf2e721dd7 |
feat: auto-inject cache busters at server startup — eliminates merge conflicts (#481)
## Problem Every PR that touches `public/` files requires manually bumping cache buster timestamps in `index.html` (e.g. `?v=1775111407`). Since all PRs change the same lines in the same file, this causes **constant merge conflicts** — it's been the #1 source of unnecessary PR friction. ## Solution Replace all hardcoded `?v=TIMESTAMP` values in `index.html` with a `?v=__BUST__` placeholder. The Go server replaces `__BUST__` with the current Unix timestamp **once at startup** when it reads `index.html`, then serves the pre-processed HTML from memory. Every server restart automatically picks up fresh cache busters — no manual intervention needed. ## What changed | File | Change | |------|--------| | `public/index.html` | All `v=1775111407` → `v=__BUST__` (28 occurrences) | | `cmd/server/main.go` | `spaHandler` reads index.html at init, replaces `__BUST__` with Unix timestamp, serves from memory for `/`, `/index.html`, and SPA fallback | | `cmd/server/helpers_test.go` | New `TestSpaHandlerCacheBust` — verifies placeholder replacement works for root, SPA fallback, and direct `/index.html` requests. Also added tests for root `/` and `/index.html` routes | | `AGENTS.md` | Rule 3 updated: cache busters are now automatic, agents should not manually edit them | ## Testing - `go build ./...` — compiles cleanly - `go test ./...` — all tests pass (including new cache-bust tests) - `node test-frontend-helpers.js && node test-packet-filter.js && node test-aging.js` — all frontend tests pass - No hardcoded timestamps remain in `index.html` --------- Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
f20431d816 |
fix: implement 'Show direct neighbors' map filter (#480)
## Summary Fixes #457 — The "Show direct neighbors" checkbox on the map was a UI stub that did nothing. This PR implements the full feature. ## What Changed ### `public/map.js` - **New state**: `selectedReferenceNode` (pubkey) and `neighborPubkeys` (Set) track which node is the reference and who its direct neighbors are - **`selectReferenceNode(pubkey, name)`**: Fetches `/api/nodes/{pubkey}/paths`, parses path hops to find all nodes directly adjacent to the reference node in any observed path, then auto-enables the neighbor filter - **Neighbor filter in `_renderMarkersInner()`**: When `filters.neighbors` is on and a reference node is selected, only the reference node and its direct (1-hop) neighbors are shown on the map - **Popup "Show Neighbors" link**: Each node popup now has a "Show Neighbors" action that sets it as the reference node - **Sidebar UI hints**: Shows the reference node name when selected, or a hint to click a node when the filter is enabled without a reference - **Cleanup on `destroy()`**: Clears reference state and global handler ### `test-frontend-helpers.js` - 6 new unit tests covering: - Filter off shows all nodes - Filter on without reference shows all nodes (graceful no-op) - Filter on with reference + neighbors filters correctly - Filter on with empty neighbor set shows only reference - Neighbor filter respects role filters - Neighbor extraction from path data ### `public/index.html` - Cache buster bump ## How It Works 1. User clicks a node marker on the map → popup shows "Show Neighbors" link 2. Clicking "Show Neighbors" fetches that node's paths from `/api/nodes/{pubkey}/paths` 3. Adjacent hops in each path are identified as direct neighbors 4. The map filters to show only the reference node + its neighbors 5. The sidebar shows which node is the reference 6. Unchecking the checkbox restores the full node view ## Test Results ``` Frontend helpers: 250 passed, 0 failed Packet filter: 62 passed, 0 failed ``` --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
f9cfad9cd4 |
fix: update observer last_seen on packet ingestion (#479)
## Summary Related to #463 (partial fix — addresses packet path, status message path still needs investigation) — Observers incorrectly showing as offline despite actively forwarding packets. ## Root Cause Observer `last_seen` was only updated when status topic messages (`meshcore/<region>/<observer_id>/status`) were received via `UpsertObserver`. When packets were ingested from an observer, the observer's `last_seen` was **not** updated — only the `observer_idx` was resolved for the observation record. This meant observers with low traffic that published status messages less frequently than the 10-minute online threshold would appear offline on the observers page, even though they were clearly alive and forwarding packets. ## Changes **`cmd/ingestor/db.go`:** - Added `stmtUpdateObserverLastSeen` prepared statement: `UPDATE observers SET last_seen = ? WHERE rowid = ?` - In `InsertTransmission`, after resolving `observer_idx`, update the observer's `last_seen` to the packet timestamp - This ensures any observer actively forwarding traffic stays marked as online **`cmd/ingestor/db_test.go`:** - Added `TestInsertTransmissionUpdatesObserverLastSeen` — verifies that inserting a packet from an observer updates its `last_seen` from a backdated value to the packet timestamp ## Performance The added `UPDATE` is a single-row update by `rowid` (primary key) — O(1) with no index overhead. It runs once per packet insertion when an observer is resolved, which was already doing a `SELECT` by `rowid` anyway. No measurable impact on ingestion throughput. ## Test Results All existing tests pass: - `cmd/ingestor`: 26.6s ✅ - `cmd/server`: 3.7s ✅ --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
96d0bbe487 |
fix: replace Euclidean distance with haversine in analytics hop distances (#478)
## Summary Fixes #433 — Replace the inaccurate Euclidean distance approximation in `analytics.js` hop distances with proper haversine calculation, matching the server-side computation introduced in PR #415. ## Problem PR #415 moved collision analysis server-side and switched from the frontend's Euclidean approximation (`dLat×111, dLon×85`) to proper haversine. However, the **hop distance** calculation in `analytics.js` (subpath detail panel) still used the old Euclidean formula. This caused: - **Inconsistent distances** between hop distances and collision distances - **Significant errors at high latitudes** — e.g., Oslo→Stockholm: Euclidean gives ~627km, haversine gives ~415km (51% error) - The `dLon×85` constant assumes ~40° latitude; at 60° latitude the real scale factor is ~55.5km/degree, not 85 ## Changes | File | Change | |------|--------| | `public/analytics.js` | Replace `dLat*111, dLon*85` Euclidean with `HopResolver.haversineKm()` (with inline fallback) | | `public/hop-resolver.js` | Export `haversineKm` in the public API for reuse | | `test-frontend-helpers.js` | Add 4 tests: export check, zero distance, SF→LA accuracy, Euclidean vs haversine divergence | | `cmd/server/helpers_test.go` | Add `TestHaversineKm`: zero, SF→LA, symmetry, Oslo→Stockholm accuracy | | `public/index.html` | Cache buster bump | ## Performance No performance impact — `haversineKm` replaces an inline arithmetic expression with another inline arithmetic expression of identical O(1) complexity. Only called per hop pair in the subpath detail panel (typically <10 hops). ## Testing - `node test-frontend-helpers.js` — 248 passed, 0 failed - `go test -run TestHaversineKm` — PASS Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
6712da7d7c |
fix: add region filtering to hash-collisions endpoint (#477)
## Summary The `/api/analytics/hash-collisions` endpoint always returned global results, ignoring the active region filter. Every other analytics endpoint (RF, topology, hash-sizes, channels, distance, subpaths) respected the `?region=` query parameter — this was the only one that didn't. Fixes #438 ## Changes ### Backend (`cmd/server/`) - **routes.go**: Extract `region` query param and pass to `GetAnalyticsHashCollisions(region)` - **store.go**: - `collisionCache` changed from `*cachedResult` → `map[string]*cachedResult` (keyed by region, `""` = global) — consistent with `rfCache`, `topoCache`, etc. - `GetAnalyticsHashCollisions(region)` and `computeHashCollisions(region)` now accept a region parameter - When region is specified, resolves regional observers, scans packets for nodes seen by those observers, and filters the node list before computing collisions - Cache invalidation updated to clear the map (not set to nil) ### Frontend (`public/`) - **analytics.js**: The hash-collisions fetch was missing `+ sep` (the region query string). All other fetches in the same `Promise.all` block had it — this was simply overlooked in PR #415. - **index.html**: Cache busters bumped ### Tests (`cmd/server/routes_test.go`) - `TestHashCollisionsRegionParamIgnored` → renamed to `TestHashCollisionsRegionParam` with updated comments reflecting that region is now accepted (with no configured regional observers, results match global — which the test verifies) ## Performance No new hot-path work. Region filtering adds one scan of `s.packets` (same as every other region-filtered analytics endpoint) only when `?region=` is provided. Results are cached per-region with the existing 60s TTL. Without `?region=`, behavior is unchanged. Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
6aef83c82a |
fix: remove duplicate return statement in _cumulativeRowOffsets() (#476)
## Summary Removes an unreachable duplicate `return offsets;` statement in the `_cumulativeRowOffsets()` function in `packets.js`. The second return was dead code found during review of PR #402. ## Changes - **`public/packets.js`**: Removed the duplicate `return offsets;` on what was line 1137 (the line immediately after the first, reachable `return offsets;`) - **`public/index.html`**: Cache buster bump ## Testing This is a dead code removal — the duplicate return was unreachable. No behavior change. No new tests needed as existing tests already cover `_cumulativeRowOffsets()` behavior. Fixes #447 Co-authored-by: you <you@example.com> |
||
|
|
9f14c74b3e |
ci: add Docker cleanup before build to prevent disk space exhaustion (#473)
## Summary Fixes #472 The Docker build job on the self-hosted runner fails with `no space left on device` because Docker build cache and Go module downloads accumulate between runs. The existing cleanup (line ~330) runs in the **deploy** step *after* the build — too late to help. ## Changes - Added a "Free disk space" step at the start of the build job, **before** "Build Go Docker image": - `docker system prune -af` — removes all unused images, containers, networks - `docker builder prune -af` — clears the build cache - `df -h /` — logs available disk space for visibility - Kept the existing post-deploy cleanup as belt-and-suspenders --------- Co-authored-by: you <you@example.com> Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com> |
||
|
|
0b8b1e91a6 |
perf(live): replace animation setIntervals with requestAnimationFrame and cap concurrency (#470)
## Summary Replace all `setInterval`-based animations in `live.js` with `requestAnimationFrame` loops and add a concurrency cap to prevent unbounded animation accumulation under high packet throughput. Fixes #384 ## Problem Under high throughput (≥5 packets/sec), the live map accumulated unbounded `setInterval` timers: - `pulseNode()`: 26ms interval per pulse ring - `drawAnimatedLine()`: 33ms interval per hop line + 52ms nested interval for fade-out - Ghost hop pulse: 600ms interval per ghost marker At 5 pkts/sec × 3 hops = **15+ concurrent intervals**, climbing without limit. This caused UI jank, rising CPU usage, and potential memory leaks from leaked Leaflet markers. ## Changes ### `public/live.js` | Function | Before | After | |----------|--------|-------| | `pulseNode()` | `setInterval` (26ms) + `setTimeout` safety | `requestAnimationFrame` loop, self-terminates at 2s or opacity ≤ 0 | | `drawAnimatedLine()` | `setInterval` (33ms) for line + nested `setInterval` (52ms) for fade | Two `requestAnimationFrame` loops (line advance + fade-out) | | Ghost hop pulse | `setInterval` (600ms) + `setTimeout` (3s) | `requestAnimationFrame` loop with 3s expiry | | `animatePath()` | No concurrency limit | Returns early when `activeAnims >= MAX_CONCURRENT_ANIMS` (20) | ### `public/index.html` - Cache buster version bump ### `test-live-anims.js` (new) - 7 tests verifying: - No `setInterval` in `pulseNode`, `drawAnimatedLine`, or `animatePath` - `MAX_CONCURRENT_ANIMS` defined and set to 20 - Concurrency check present in `animatePath` - No stale `setInterval` in animation hot paths ## Complexity & Scale - **Time complexity**: O(1) per animation frame (no change in per-frame work) - **Concurrency**: Hard-capped at 20 simultaneous animations (previously unbounded) - **At 5 pkts/sec, 3 hops**: Excess animations silently dropped instead of accumulating timers - **rAF benefit**: Browser coalesces all animations into single paint cycle; paused tabs stop animating automatically ## Test Results ``` === Animation interval elimination === ✅ pulseNode does not use setInterval ✅ drawAnimatedLine does not use setInterval ✅ ghost hop pulse does not use setInterval === Concurrency cap === ✅ MAX_CONCURRENT_ANIMS is defined ✅ MAX_CONCURRENT_ANIMS is set to 20 ✅ animatePath checks MAX_CONCURRENT_ANIMS before proceeding === Safety: no stale setInterval in animation functions === ✅ no setInterval remains in animation hot path 7 passed, 0 failed ``` All existing tests pass (packet-filter: 62, aging: 29, frontend-helpers: 241). ## Performance Proof (Rule 0 compliance) Benchmark: `node test-anim-perf.js` — simulates timer/animation accumulation under realistic throughput. ### Timer count: old (setInterval) vs new (rAF + cap) | Scenario | Old model (peak concurrent timers) | New model (peak concurrent animations) | |----------|-----------------------------------:|---------------------------------------:| | 5 pkt/s × 3 hops, 30s sustained | **123** | **20** | | 5 pkt/s × 3 hops, 5min sustained | **123** | **20** | | 20 pkt/s × 3 hops, 10s burst | **246** | **20** | **Before:** Each hop spawns 3 `setInterval` timers (pulse 26ms, line 33ms, fade 52ms) that live 0.6–2s each. At 5 pkt/s × 3 hops = 15 timers/sec, peak concurrent timers reach **123** (limited only by timer lifetime, not by any cap). Under burst traffic (20 pkt/s), this climbs to **246+**. **After:** `MAX_CONCURRENT_ANIMS = 20` hard-caps active animations. Excess packets are silently dropped. rAF loops replace all `setInterval` calls, coalescing into single paint cycles. Peak concurrent animations: **always ≤ 20**, regardless of throughput or duration. --------- Co-authored-by: you <you@example.com> Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com> |
||
|
Kpa-clawbotandyou
|
c678555e75 |
fix: display channel hash as hex instead of decimal (#471)
## Summary Fixes #465 — Channel hash was displaying in decimal instead of hexadecimal in `channels.js`. ## Changes - Added `formatHashHex()` helper to `channels.js` that formats numeric hashes as `0x` hex (e.g. `0x0A`) and passes string hashes through unchanged - Applied to both display sites: `renderChannelList` fallback name and `selectChannel` header text - Consistent with `packets.js` and `analytics.js` which already use `.toString(16).padStart(2, '0').toUpperCase()` ## Tests - 3 new tests in `test-frontend-helpers.js` verifying the helper exists, is used at display sites, and produces correct output for numeric and string inputs - All 244 frontend tests pass, plus packet-filter (62) and aging (29) tests Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
623ebc879b |
fix: add mutex synchronization to PerfStats to eliminate data races (#469)
## Summary Fixes #361 — `perfMiddleware()` wrote to shared `PerfStats` fields (`Requests`, `TotalMs`, `Endpoints` map, `SlowQueries` slice) without any synchronization, causing data races under concurrent HTTP requests. ## Changes ### `cmd/server/routes.go` - **Added `sync.Mutex` to `PerfStats` struct** — single mutex protects all fields - **`perfMiddleware`** — all shared state mutations (counter increments, endpoint map access, slice appends) now happen under lock. Key normalization (regex, mux route lookup) moved outside the lock since it uses no shared state - **`handleHealth`** — snapshots `Requests`, `TotalMs`, `SlowQueries` under lock before building response - **`handlePerf`** — copies all endpoint data and slow queries under lock into local snapshots, then does expensive work (sorting, percentile calculation) outside the lock - **`handlePerfReset`** — resets fields in-place instead of replacing the pointer (avoids unlocking a different mutex) ### `cmd/server/perfstats_race_test.go` (new) - Regression test: 50 concurrent writer goroutines + 10 concurrent reader goroutines hammering `PerfStats` simultaneously - Verifies no race conditions (via `-race` flag) and counter consistency ## Design Decisions - **Single mutex over atomics**: The issue suggested `atomic.Int64` for counters, but since slices/maps need a mutex anyway, a single mutex is simpler and the critical section is small (microseconds). No measurable contention at CoreScope's scale. - **Copy-under-lock pattern**: Expensive operations (sorting, percentile computation) happen outside the lock to minimize hold time. - **In-place reset**: `handlePerfReset` clears fields rather than replacing the `PerfStats` pointer, ensuring the mutex remains valid for concurrent goroutines. ## Testing - `go test -race -count=1 ./cmd/server/...` — **PASS** (all existing tests + new race test) - New `TestPerfStatsConcurrentAccess` specifically validates concurrent access patterns Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
0b1924d401 |
perf(packets): replace observers.find() linear scans with Map lookups (#468)
## Summary Replace all `observers.find()` linear scans in `packets.js` with O(1) `Map.get()` lookups, eliminating ~300K comparisons per render cycle at 30K+ rows. ## Changes - Added `observerMap` (`Map<id, observer>`) built once when observers load - Replaced all 6 `observers.find()` call sites with `observerMap.get()`: - `obsName()` — called per row for observer name display - Region filter check in packet filtering - Observer dropdown label in filter UI - Group header region lookup - Child row region lookup - Flat row region lookup - Map is cleared on reset and rebuilt on each `loadObservers()` call ## Complexity - **Before:** O(k) per row × 30K rows = O(30K × k) where k = observer count (~10) - **After:** O(1) per row × 30K rows = O(30K) - Map construction: O(k) once, negligible ## Testing - All Go tests pass (`cmd/server`, `cmd/ingestor`) - All frontend tests pass (`test-packet-filter.js`: 62 passed, `test-aging.js`: 29 passed, `test-frontend-helpers.js`: 241 passed) Fixes #383 Co-authored-by: you <you@example.com> |
||
|
efiten
|
0f502370c5 |
fix: VCR timeline and clock respect UTC/local timezone setting (#459)
## Problem Fixes #324. The VCR LCD clock and timeline hover/touch tooltip always showed local time, ignoring the UTC/local timezone setting in the customizer Display tab. ## Root cause Three sites in `live.js` bypassed the shared `getTimestampTimezone()` utility: - `updateVCRClock()` — used `d.getHours()` / `d.getMinutes()` / `d.getSeconds()` (always local) - Timeline mousemove tooltip — used `d.toLocaleTimeString()` (always local) - Timeline touchmove tooltip — same ## Fix Added `vcrFormatTime(tsMs)` helper that checks `getTimestampTimezone()` and uses `getUTC*` methods when set to `'utc'`, otherwise local `get*`. Applied to all three sites. Exposed as `window._vcrFormatTime` for testing. ## Tests 4 new unit tests in `test-frontend-helpers.js` covering UTC mode, local mode, and zero-padding. ## Checklist - [x] Branches from `upstream/master` - [x] No Matomo or local-only commits - [x] Cache busters bumped (`v=1775073838`) - [x] 233 tests pass, 0 fail 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
efitenandClaude Sonnet 4.6
|
e47c39ffda |
fix: null-guard animLayer and liveAnimCount in nextHop after destroy (#462)
## Summary - `nextHop()` schedules `setInterval`/`setTimeout` callbacks that can fire after `destroy()` has set `animLayer = null` and removed DOM elements - This caused three console errors on the Live page when navigating away mid-animation: `Cannot read properties of null (reading 'hasLayer')` and `Cannot set properties of null (setting 'textContent')` - Added null guards at each async callback site; no behavioral change when the page is active ## Changes - `public/live.js`: early return if `animLayer` is null at start of `nextHop()`; null-safe `animLayer.hasLayer` checks in `setInterval`/`setTimeout`; null-safe `liveAnimCount` element access - `public/index.html`: cache buster bumped - `test-frontend-helpers.js`: 4 source-inspection tests verifying the null guards are present ## Test plan - [ ] Open Live page, trigger some packet animations, navigate away quickly — no console errors - [ ] `node test-frontend-helpers.js` passes (233 tests, 0 failures) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
efiten
|
1499a55ba7 |
perf: upsert known nodes in-place on ADVERT, skip full reload (#461)
## Problem Fixes #399. On every ADVERT WebSocket batch the nodes page invalidated the entire `_allNodes` cache and triggered a full `/nodes?limit=5000` fetch — even when every advertising node was already cached. The 90s API TTL was actively bypassed. ## Root cause ```js wsHandler = debouncedOnWS(function (msgs) { if (msgs.some(isAdvertMessage)) { _allNodes = null; // wipe cache unconditionally invalidateApiCache('/nodes'); // bust API TTL loadNodes(true); // full 5k fetch } }, 5000); ``` ## Fix ADVERT decoded payloads include `pubKey`, `name`, `lat`, `lon` — enough to update known nodes in place: - **Known node** (pubKey found in `_allNodes`): upsert `name`, `lat`, `lon`, `last_seen` directly — no fetch, no cache bust, just re-render. - **New node** (pubKey not in cache) or **no pubKey** in payload: fall back to full reload as before. This covers the common case on an active mesh: all advertising nodes are already cached. The full reload path is preserved for node discovery. ## Tests 2 new unit tests: known-node upsert (asserts 0 API calls, fields updated) and unknown-node fallback (asserts full reload triggered). All 231 tests pass. ## Checklist - [x] Branches from `upstream/master` - [x] No Matomo or local-only commits - [x] Cache busters bumped - [x] 231 tests pass, 0 fail 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
efiten
|
f71e117cdd |
fix: reset restores home steps after SITE_CONFIG contamination (#460)
## Problem Fixes #325. Removing all home steps and clicking "Reset my theme" did not restore them. ## Root cause Two-part bug: **1. `SITE_CONFIG.home` permanently mutated at page load** `app.js` calls `mergeUserHomeConfig(SITE_CONFIG, userTheme)` which does `SITE_CONFIG.home = Object.assign({}, serverHome, userTheme.home)`. If the user had `steps: []` saved in localStorage, this sets `SITE_CONFIG.home.steps = []` globally — permanently for the lifetime of the page. **2. `initState()` reads the contaminated config** When the customizer opens (or Reset is clicked), `initState()` reads `cfg = window.SITE_CONFIG`. Since `SITE_CONFIG.home.steps` is already `[]`, `state.home.steps` stays `[]` even after `localStorage.removeItem`. `autoSave()` then re-saves `steps: []` straight back. **Secondary issue:** `data-rm-step` / add / move handlers didn't call `autoSave()`, making step persistence non-deterministic (only saved if a text field edit happened to be pending). ## Fix - **`app.js`**: snapshot `SITE_CONFIG.home` before `mergeUserHomeConfig` → `window._SITE_CONFIG_ORIGINAL_HOME` - **`customize.js`**: `initState()` uses `_SITE_CONFIG_ORIGINAL_HOME` instead of the contaminated `cfg.home` - **`customize.js`**: add `autoSave()` to rm/move/add handlers for steps, checklist, and footer links ## Tests 2 new unit tests covering the snapshot bypass and DEFAULTS fallback. 231 tests pass. ## Checklist - [x] Branches from `upstream/master` - [x] No Matomo or local-only commits - [x] Cache busters bumped - [x] 231 tests pass, 0 fail 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Kpa-clawbotandyou
|
75f1295a06 |
fix: always refresh staging config from prod (#467)
## Summary Fixes #466 — staging config was not refreshed from prod due to a stale `-nt` timestamp guard. ## Root Cause `prepare_staging_config()` only copied prod config when staging was missing or prod was newer by mtime. However, the `sed -i` that applies the STAGING siteName updated staging's mtime, making it appear newer than prod. Subsequent runs skipped the copy entirely. ## Changes - **`manage.sh`**: Removed the `-nt` timestamp conditional in `prepare_staging_config()`. Staging config is now always copied fresh from prod with the STAGING siteName applied. Note: `prepare_staging_db()` already copies unconditionally — no change needed there. Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbotandyou
|
b1b76acb77 |
feat: manage.sh update supports pinning to release tags (#456)
## Summary `manage.sh update` now supports pinning to specific release tags instead of always pulling tip of master. Fixes #455 ## Changes ### `cmd_update` — accepts optional version argument - **No argument**: fetches tags, checks out latest release tag (`git tag -l 'v*' --sort=-v:refname | head -1`) - **`latest`**: explicit opt-in to tip of master (bleeding edge) - **Specific tag** (e.g. `v3.1.0`): checks out that exact tag, with error message + available tags if not found ### `cmd_setup` — defaults to latest tag - After Docker check, fetches tags and pins to latest release tag - Skips if already on the latest tag - Uses state tracking (`version_pin`) so re-runs don't repeat ### `cmd_status` — shows version - Displays current version (exact tag name or short commit hash) at the top of status output ### Help text - Updated to reflect new `update [version]` syntax ## Usage ```bash ./manage.sh update # checkout latest release tag (e.g. v3.2.0) ./manage.sh update v3.1.0 # pin to specific version ./manage.sh update latest # explicit tip of master (bleeding edge) ./manage.sh status # now shows "Version: v3.2.0" ``` ## Testing - `bash -n manage.sh` passes (syntax valid) - Logic follows existing patterns (git fetch, checkout, rebuild, restart) --------- Co-authored-by: you <you@example.com> |
||
|
|
f87eb3601c |
fix: graceful container shutdown for reliable deployments (#453)
## Summary Fixes #450 — staging deployment flaky due to container not shutting down cleanly. ## Root Causes 1. **Server never closed DB on shutdown** — SQLite WAL lock held indefinitely, blocking new container startup 2. **`httpServer.Close()` instead of `Shutdown()`** — abruptly kills connections instead of draining them 3. **No `stop_grace_period` in compose configs** — Docker sends SIGTERM then immediately SIGKILL (default 10s is often not enough for WAL checkpoint) 4. **Supervisor didn't forward SIGTERM** — missing `stopsignal`/`stopwaitsecs` meant Go processes got SIGKILL instead of graceful shutdown 5. **Deploy scripts used default `docker stop` timeout** — only 10s grace period ## Changes ### Go Server (`cmd/server/`) - **Graceful HTTP shutdown**: `httpServer.Shutdown(ctx)` with 15s context timeout — drains in-flight requests before closing - **WebSocket cleanup**: New `Hub.Close()` method sends `CloseGoingAway` frames to all connected clients - **DB close on shutdown**: Explicitly closes DB after HTTP server stops (was never closed before) - **WAL checkpoint**: `PRAGMA wal_checkpoint(TRUNCATE)` before DB close — flushes WAL to main DB file and removes WAL/SHM lock files ### Go Ingestor (`cmd/ingestor/`) - **WAL checkpoint on shutdown**: New `Store.Checkpoint()` method, called before `Close()` - **Longer MQTT disconnect timeout**: 5s (was 1s) to allow in-flight messages to drain ### Docker Compose (all 4 variants) - Added `stop_grace_period: 30s` and `stop_signal: SIGTERM` ### Supervisor Configs (both variants) - Added `stopsignal=TERM` and `stopwaitsecs=20` to server and ingestor programs ### Deploy Scripts - `deploy-staging.sh`: `docker stop -t 30` with explicit grace period - `deploy-live.sh`: `docker stop -t 30` with explicit grace period ## Shutdown Sequence (after fix) 1. Docker sends SIGTERM to supervisord (PID 1) 2. Supervisord forwards SIGTERM to server + ingestor (waits up to 20s each) 3. Server: stops poller → drains HTTP (15s) → closes WS clients → checkpoints WAL → closes DB 4. Ingestor: stops tickers → disconnects MQTT (5s) → checkpoints WAL → closes DB 5. Docker waits up to 30s total before SIGKILL ## Tests All existing tests pass: - `cd cmd/server && go test ./...` ✅ - `cd cmd/ingestor && go test ./...` ✅ --------- Co-authored-by: you <you@example.com> Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com> |