## Summary
Implements #946 — deterministic HSL coloring of packet markers by hash
for visual propagation tracing.
### What's new
1. **`public/hash-color.js`** — Pure IIFE
(`window.HashColor.hashToHsl(hashHex, theme)`) deriving hue from first 2
bytes of packet hash. Theme-aware lightness with WCAG ≥3.0 contrast
against `--content-bg` (`#f4f5f7` light / `#0f0f23` dark,
`style.css:32,55`). Green/yellow zone (hue 45°-195°) uses L=30% in light
theme to maintain contrast.
2. **Live page dots + contrails** — `drawAnimatedLine` fills the flying
dot and tints the contrail polyline with the hash-derived HSL when
toggle is ON. Ghost-hop dots remain grey (`#94a3b8`). Matrix mode path
(`drawMatrixLine`) is untouched.
3. **Packets table stripe** — `border-left: 4px solid <hsl>` on `<tr>`
in both `buildGroupRowHtml` (group + child rows) and `buildFlatRowHtml`.
Absent when toggle OFF.
4. **Toggle UI** — "Color by hash" checkbox in `#liveControls` between
Realistic and Favorites. Default ON. Persisted to
`localStorage('meshcore-color-packets-by-hash')`. Dispatches `storage`
event for cross-tab sync. Packets page listens and re-renders.
### Performance
- `hashToHsl` is O(1) — two `parseInt` calls + arithmetic. No allocation
beyond the result string.
- Called once per `drawAnimatedLine` invocation (not per animation
frame).
- Packets table: called once per visible row during render (existing
virtualization applies).
### Tests
- `test-hash-color.js`: 16 unit tests — purity, theme split, yellow-zone
clamp, sentinel, variability (anti-tautology gate), WCAG sweep (step 15°
both themes).
- `test-packets.js`: 82 tests still passing (no regression).
- `test-e2e-playwright.js`: 4 new E2E tests — toggle presence/default,
persistence across reload, table stripe present when ON, absent when
OFF.
### Acceptance criteria addressed
All items from spec §6 implemented. TYPE_COLORS retained on
borders/lines. Ghost hops stay grey. Matrix mode suppressed. Cross-tab
storage event dispatched.
Closes#946
---------
Co-authored-by: you <you@example.com>
Co-authored-by: Kpa-clawbot <bot@example.invalid>
## Bug
Path Inspector "Show on Map" only rendered the first node of a candidate
path. Multi-hop candidates appeared as a single dot instead of a
polyline.
## Root cause
`public/path-inspector.js:186-188` and `public/map.js:574` (cross-page
handler) called:
```js
drawPacketRoute(candidate.path.slice(1), candidate.path[0]);
```
But `drawPacketRoute(hopKeys, origin)` (`public/map.js:390`) expects
`origin` to be an **object** with `pubkey`/`lat`/`lon`/`name` properties
— not a bare string. The code at lines 451-460 does `origin.lat` /
`origin.pubkey` lookups; with a string, both branches fail, `originPos`
stays null, and the originating node never gets prepended to
`positions`.
Combined with `slice(1)` stripping the head, the resulting polyline was
missing the first hop AND the origin marker — and short paths could
collapse to a single resolved node.
## Fix
Pass the full path as `hopKeys` and `null` as origin. `drawPacketRoute`
already iterates `hopKeys`, resolves each against `nodes[]`, and draws a
marker for every resolved hop. The "origin" arg was meant for cases
where the originator is a separate object (e.g., from packet detail with
sender metadata), not for paths where the origin IS the first hop.
```js
drawPacketRoute(candidate.path, null);
```
Two call sites fixed: in-page direct call (`path-inspector.js:188`) and
cross-page handler (`map.js:574`).
## Verification
**Code reading only.** I did NOT manually load the page or visually
verify the polyline renders. Reviewer should:
1. Open Path Inspector, query a multi-prefix path with ≥3 known hops
2. Click "Show on Map"
3. Confirm polyline draws through every resolved node, not just the
first
`drawPacketRoute` is hard to unit-test without a real Leaflet map, so no
automated test added.
---------
Co-authored-by: Kpa-clawbot <bot@example.invalid>
Co-authored-by: you <you@example.com>
## Summary
Fixes#947 — MQTT ingestor silently stalls after `pingresp not received`
disconnect due to paho's default 10-minute reconnect backoff and zero
observability of reconnect attempts.
## Changes
### `cmd/ingestor/main.go`
- **Extract `buildMQTTOpts()`** — encapsulates MQTT client option
construction for testability
- **`SetMaxReconnectInterval(30s)`** — bounds paho's default 10-minute
exponential backoff (source: `options.go:137` in
`paho.mqtt.golang@v1.5.0`)
- **`SetConnectTimeout(10s)`** — prevents stuck connect attempts from
blocking reconnect cycle
- **`SetWriteTimeout(10s)`** — prevents stuck publish writes
- **`SetReconnectingHandler`** — logs `MQTT [<tag>] reconnecting to
<broker>` on every reconnect attempt, giving operators visibility into
retry behavior
- **Enhanced `SetConnectionLostHandler`** — now includes broker address
in log line for multi-source disambiguation
### `cmd/ingestor/mqtt_opts_test.go` (new)
- Tests verify `MaxReconnectInterval`, `ConnectTimeout`, `WriteTimeout`
are set correctly
- Tests verify credential and TLS configuration
- Anti-tautology: tests fail if timing settings are removed from
`buildMQTTOpts()`
## Operator impact
After this change, a pingresp disconnect produces:
```
MQTT [staging] disconnected from tcp://broker:1883: pingresp not received, disconnecting
MQTT [staging] reconnecting to tcp://broker:1883
MQTT [staging] reconnecting to tcp://broker:1883
MQTT [staging] connected to tcp://broker:1883
MQTT [staging] subscribed to meshcore/#
```
Max gap between disconnect and first reconnect attempt: ~30s (was up to
10 minutes).
---------
Co-authored-by: you <you@example.com>
Adds a node filter input to the live controls bar. When active, only
packets whose hop chain passes through the selected node(s) are
animated. A counter shows "Showing X of Y" so operators know traffic is
filtered, not absent.
## Changes
- `packetInvolvesFilterNode(pkt, filterKeys)`: pure filter function
using same prefix-matching logic as the favorites filter
- `setNodeFilter(keys)`: sets filter state, resets counters, persists to
localStorage
- `updateNodeFilterUI()`: updates counter + clear button visibility +
datalist autocomplete
- Filter input in controls bar (after Favorites toggle): text input +
datalist autocomplete + × clear button + counter div
- Filter wired into `renderPacketTree`: increments total/shown counters,
returns early when packet doesn't match
- URL hash sync: `?node=ABCD1234` — read on init, written on filter
change
## Tests
8 new unit tests covering filter logic, localStorage persistence, and
edge cases; 78 pass, 0 regressions.
Closes#771 (M3 of 3)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- Adds `public/geofilter-docs.html` — a self-contained, app-served
documentation page for the geofilter feature, matching the builder's
dark theme
- Updates the GeoFilter Builder's help-bar "Documentation" link from
GitHub markdown URL to the local `/geofilter-docs.html`
## Docs coverage
Polygon syntax, coordinate ordering (`[lat, lon]` — not GeoJSON `[lon,
lat]`), multi-polygon clarification (single polygon only), examples
(Belgium rectangle + irregular shape), legacy bounding box format, prune
script usage.
## Test plan
- [x] Open `/geofilter-docs.html` — dark theme renders, all sections
visible
- [x] Open `/geofilter-builder.html` → click "Documentation" → navigates
to `/geofilter-docs.html` in same tab
- [x] Click "← GeoFilter Builder" on docs page → navigates back to
`/geofilter-builder.html`
Closes#820
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Problem
`Load()` loaded all transmissions from the DB regardless of
`retentionHours`, so `buildSubpathIndex()` processed the full DB history
on every startup. On a DB with ~280K paths this produces ~13.5M subpath
index entries, OOM-killing the process before it ever starts listening —
causing a supervisord crash loop with no useful error message.
## Fix
Apply the same `retentionHours` cutoff to `Load()`'s SQL that
`EvictStale()` already uses at runtime. Both conditions
(`retentionHours` window and `maxPackets` cap) are combined with AND so
neither safety limit is bypassed.
Startup now builds indexes only over the retention window, making
startup time and memory proportional to recent activity rather than
total DB history.
## Docs
- `config.example.json`: adds `retentionHours` to the `packetStore`
block with recommended value `168` (7 days) and a warning about `0` on
large DBs
- `docs/user-guide/configuration.md`: documents the field and adds an
explicit OOM warning
## Test plan
- [x] `cd cmd/server && go test ./... -run TestRetentionLoad` — covers
the retention-filtered load: verifies packets outside the window are
excluded, and that `retentionHours: 0` still loads everything
- [x] Deploy on an instance with a large DB (>100K paths) and
`retentionHours: 168` — server reaches "listening" in seconds instead of
OOM-crashing
- [x] Verify `config.example.json` has `retentionHours: 168` in the
`packetStore` block
- [x] Verify `docs/user-guide/configuration.md` documents the field and
warning
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
Closes#919
## Summary
Enables SQLite incremental auto-vacuum so the database file actually
shrinks after retention reaper deletes old data. Previously, `DELETE`
operations freed pages internally but never returned disk space to the
OS.
## Changes
### 1. Auto-vacuum on new databases
- `PRAGMA auto_vacuum = INCREMENTAL` set via DSN pragma before
`journal_mode(WAL)` in the ingestor's `OpenStoreWithInterval`
- Must be set before any tables are created; DSN ordering ensures this
### 2. Post-reaper incremental vacuum
- `PRAGMA incremental_vacuum(N)` runs after every retention reaper cycle
(packets, metrics, observers, neighbor edges)
- N defaults to 1024 pages, configurable via `db.incrementalVacuumPages`
- Noop on `auto_vacuum=NONE` databases (safe before migration)
- Added to both server and ingestor
### 3. Opt-in full VACUUM for existing databases
- Startup check logs a clear warning if `auto_vacuum != INCREMENTAL`
- `db.vacuumOnStartup: true` config triggers one-time `PRAGMA
auto_vacuum = INCREMENTAL; VACUUM`
- Logs start/end time for operator visibility
### 4. Documentation
- `docs/user-guide/configuration.md`: retention section notes that
lowering retention doesn't immediately shrink the DB
- `docs/user-guide/database.md`: new guide covering WAL, auto-vacuum,
migration, manual VACUUM
### 5. Tests
- `TestNewDBHasIncrementalAutoVacuum` — fresh DB gets `auto_vacuum=2`
- `TestExistingDBHasAutoVacuumNone` — old DB stays at `auto_vacuum=0`
- `TestVacuumOnStartupMigratesDB` — full VACUUM sets `auto_vacuum=2`
- `TestIncrementalVacuumReducesFreelist` — DELETE + vacuum shrinks
freelist
- `TestCheckAutoVacuumLogs` — handles both modes without panic
- `TestConfigIncrementalVacuumPages` — config defaults and overrides
## Migration path for existing databases
1. On startup, CoreScope logs: `[db] auto_vacuum=NONE — DB needs
one-time VACUUM...`
2. Set `db.vacuumOnStartup: true` in config.json
3. Restart — VACUUM runs (blocks startup, minutes on large DBs)
4. Remove `vacuumOnStartup` after migration
## Test results
```
ok github.com/corescope/server 19.448s
ok github.com/corescope/ingestor 30.682s
```
---------
Co-authored-by: you <you@example.com>
## Summary
- Fixes#912 — geofilter-builder generates out-of-range longitudes for
southern hemisphere locations
- Root cause: Leaflet's `latlng.lng` is unbounded; panning from Europe
to Australia produces values like `-210` instead of `150`
- Fix: call `latlng.wrap()` in `latLonPair()` to normalise longitude to
`[-180, 180]` before writing the config JSON
## Details
When the user opens the builder (default view: Europe, `[50.5, 4.4]`)
and pans east to Australia, Leaflet tracks the cumulative pan offset and
returns `lng = 150 - 360 = -210` to keep the path continuous. The
builder was passing that raw value straight into the output JSON,
producing coordinates that fall outside any valid bounding box.
`L.LatLng.wrap()` is Leaflet's built-in normalisation method — collapses
any longitude to `[-180, 180]` with no loss of precision.
## Test plan
- [x] Open the builder, navigate to NSW Australia, place a polygon —
confirm longitudes are `~141`–`154`, not `~-219`–`-206`
- [x] Repeat for a northern hemisphere location (e.g. Belgium) — confirm
output is unchanged
- [x] Paste the generated config into CoreScope — confirm nodes appear
on Maps and Live view
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
## Summary
Remove tautological/vacuous frontend tests identified in the test audit
(2026-04-30).
## Deleted Files
- **`test-anim-perf.js`** — Tests local simulation functions, not actual
`live.js` animation code. Behavioral equivalents exist in
`test-live-anims.js`.
- **`test-channel-add-ux.js`** — Pure HTML/CSS substring grep tests
(`src.includes(...)`) with zero behavioral value. Any refactor breaks
them without indicating a real bug.
## Edited Files
- **`test-aging.js`** — Removed `mockGetStatusInfo` (5 tests) and
`getStatusTooltip` (6 tests) blocks. These re-implement SUT logic inside
the test file and assert on the local copy, not the real code. The
GOLD-rated `getNodeStatus` boundary tests (18 assertions) and the BUG
CHECK section are preserved.
## Why
These tests are unfailable by construction — they pass regardless of
whether the code under test is correct. They inflate test counts without
providing regression protection, and they break on harmless refactors
(false negatives).
## Validation
- `npm run test:unit` passes (all 3 files: 62 + 18 + 601 assertions).
- 9 pre-existing failures in `test-frontend-helpers.js` (hop-resolver
affinity tests, unrelated to this change).
- No harness references (`test-all.sh`, `package.json`) needed updating
— deleted files were not listed there.
Co-authored-by: you <you@example.com>
## Summary
- `_debouncedWrite()` was calling `_refreshPanel()` 300ms after every
keystroke
- `_refreshPanel()` sets `container.innerHTML`, destroying the focused
input element
- On mobile, losing the focused input collapses the virtual keyboard
after each keypress
Guard the `_refreshPanel()` call so it is skipped when
`document.activeElement` is inside the panel. The pipeline
(`_runPipeline`) still runs immediately — CSS updates apply. Override
dots update on the next natural re-render (tab switch, dark-mode toggle,
panel reopen).
## Root cause
`customize-v2.js` → `_debouncedWrite()` → `_refreshPanel()` →
`_renderPanel()` → `container.innerHTML = ...`
## Test plan
- [ ] New Playwright E2E test: open Customize, focus a text field, type,
wait 500ms past debounce — asserts input element is still connected to
DOM and focus remains inside panel
- [ ] Manual: open Customize on mobile (or DevTools mobile emulation),
type in Site Name — keyboard must not collapse after each character
Fixes#896
## Problem
E2E test `Node detail loads` intermittently fails with:
> elementHandle.click: Element is not attached to the DOM
(e.g. PR #938 CI run job 73889426640.) Same flake class as #ngStats
hydration race fixed in #940.
## Root cause
```js
const firstRow = await page.$('table tbody tr');
await firstRow.click();
```
Between the `$()` and `.click()`, the nodes table re-renders from a
WebSocket push. The captured handle is detached from the new DOM, click
throws.
## Fix
Switch to a selector-based click with a small retry loop (3 attempts ×
200ms backoff), so a detach mid-attempt re-resolves a fresh element.
Test logic unchanged; just defensive against re-render between query and
click.
Co-authored-by: Kpa-clawbot <bot@example.invalid>
## Problem
E2E test "Analytics Neighbor Graph tab renders canvas and stats"
intermittently fails with `Neighbor Graph stats should have >=3 cards,
got 0` (e.g. run 25185836669).
The same suite passes on neighboring runs (master + PR #939) within
minutes. The failure correlates with timing/load, not code change.
## Root cause
`#ngStats` cards render asynchronously after `#ngCanvas` mounts. The
test waits for the canvas, then immediately reads `#ngStats .stat-card`
count. On slower runs the read happens before stats hydrate → 0 cards →
assert fail.
Other Analytics tabs in the same file already use
`page.waitForFunction(...)` to poll for content (e.g. Distance tab on
line 654). Neighbor Graph block was missing the equivalent wait.
## Fix
Add the same defensive wait before counting:
```js
await page.waitForFunction(
() => document.querySelectorAll('#ngStats .stat-card').length >= 3,
{ timeout: 8000 },
);
```
Test-only change. No frontend code touched. Bounded by 8s timeout
matching other Analytics waits.
Co-authored-by: Kpa-clawbot <bot@example.invalid>
## Summary
Direct packet links like `/#/packets?hash=<HASH>` silently returned zero
rows when the user's saved region filter excluded the packet's observer
region. The packet existed and rendered in the side panel (which fetches
without region filter), but the main packet table was empty — leaving
the user with no rows to click and no obvious diagnostic.
## Root cause
`loadPackets()` in `public/packets.js` always added the `region` query
param to `/api/packets`, even when `filters.hash` was set. The
time-window filter is already correctly suppressed when `filters.hash`
is present (see line 619: `if (windowMin > 0 && !filters.hash)`); the
region filter should follow the same rule. A specific hash is an exact
identifier — the user wants THAT packet regardless of where their saved
region selection points.
## Change
Extracted the param-building logic into a pure helper
`buildPacketsParams(...)` so it's testable, then suppressed the `region`
param when `filters.hash` is set.
## Tests
Added 7 unit tests in `test-packets.js` covering:
- hash filter suppresses region (the bug)
- hash filter suppresses region with default windowMin=0
- region applies normally when no hash filter
- empty regionParam doesn't produce spurious `region=` param
- node/observer/channel filters still pass through alongside a hash
- groupByHash=true / false flag handling
Anti-tautology gate verified: reverting the one-line fix (`!filters.hash
&&` → removed) causes 3 of the 7 new tests to fail. The fix is the
smallest change that makes them pass.
`node test-packets.js`: 80 passed, 0 failed.
## Reproduction
1. Set region filter to e.g. `SJC`
2. Open `/#/packets?hash=<HASH_FROM_ANOTHER_REGION>`
3. Before fix: empty table, no diagnostic
4. After fix: packet renders
---------
Co-authored-by: Kpa-clawbot <bot@example.invalid>
Reverts the `if: false` guard from #908.
## Why
- Azure subscription was blocked, staging VM `meshcore-runner-2`
deallocated.
- Subscription unblocked, VM started, runner online, smoke CI [run
#25117292530](https://github.com/Kpa-clawbot/CoreScope/actions/runs/25117292530)
passed.
- Time to resume automatic staging deploys on master pushes.
## Changes
- `deploy` job: `if: false` → `if: github.event_name == 'push'`
(original condition from before #908).
- `publish` job: `needs: [build-and-publish]` → `needs: [deploy]`
(original wiring restored).
## Verify after merge
- Next master push triggers the full chain: go-test → e2e-test →
build-and-publish → deploy → publish.
- `docker ps` on staging VM shows `corescope-staging-go` updated to the
new commit.
Co-authored-by: you <you@example.com>
Fixes#935
## Problem
`buildPrefixMap()` indexed ALL nodes regardless of role, causing
companions/sensors to appear as repeater hops when their pubkey prefix
collided with a path-hop hash byte.
## Fix
### Server (`cmd/server/store.go`)
- Added `canAppearInPath(role string) bool` — allowlist of roles that
can forward packets (repeater, room_server, room)
- `buildPrefixMap` now skips nodes that fail this check
### Client (`public/hop-resolver.js`)
- Added matching `canAppearInPath(role)` helper
- `init()` now only populates `prefixIdx` for path-eligible nodes
- `pubkeyIdx` remains complete — `resolveFromServer()` still resolves
any node type by full pubkey (for server-confirmed `resolved_path`
arrays)
## Tests
- `cmd/server/prefix_map_role_test.go`: 7 new tests covering role
filtering in prefix map and resolveWithContext
- `test-hop-resolver-affinity.js`: 4 new tests verifying client-side
role filter + pubkeyIdx completeness
- All existing tests updated to include `Role: "repeater"` where needed
- `go test ./cmd/server/...` — PASS
- `node test-hop-resolver-affinity.js` — 16/17 pass (1 pre-existing
centroid failure unrelated to this change)
## Commits
1. `fix: filter prefix map to only repeater/room roles (#935)` — server
implementation
2. `test: prefix map role filter coverage (#935)` — server tests
3. `ui: filter HopResolver prefix index to repeater/room roles (#935)` —
client implementation
4. `test: hop-resolver role filter coverage (#935)` — client tests
---------
Co-authored-by: you <you@example.com>
Kpa-clawbot