Compare commits

...

80 Commits

Author SHA1 Message Date
you 68c669f4a4 Extract validateAdvertSignature to shared package, accept []byte, add server tests
- Extract signature validation to internal/sigvalidate/ shared package
- Change function signature to accept []byte instead of hex strings,
  eliminating unnecessary hex encode/decode round-trip
- Add signature validation tests to cmd/server/decoder_test.go
- Both cmd/server and cmd/ingestor now import from the shared package

Addresses all review feedback on PR #686.
2026-04-10 00:44:48 +00:00
you a7ada12f6d Add tests for ed25519 signature validation
Test validateAdvertSignature with valid signatures, tampered data,
wrong timestamps, malformed inputs, and wrong-length keys/signatures.
Also test decodeAdvert with validation enabled vs disabled.
2026-04-10 00:36:23 +00:00
you c182e799bf Clean up bool pointer allocation in signature validation
Replace &[]bool{false}[0] idiom with explicit variable for clarity.
2026-04-10 00:36:17 +00:00
you e900502c68 Add missing badge-success and badge-danger CSS classes
The signature validation UI uses badge-success and badge-danger classes
that were not defined in style.css, resulting in unstyled badges.
Add them using the same color patterns as existing badge-hash variants.
2026-04-10 00:36:12 +00:00
Jeff Copeland 8e8ff85f1c for BYOP mode, perform signature validation on advert packets and display status 2026-04-08 12:03:22 -04:00
you 111b03cea1 docs: lead with pre-built Docker image as the headline 2026-04-08 07:22:07 +00:00
you 34c56d203e docs: promote API docs to own section with live analyzer.00id.net links, fix transition section 2026-04-08 07:21:11 +00:00
you cc9f25e5c8 docs: fix release notes — bind mount for caddy-data, no personal paths, add Caddyfile example 2026-04-08 07:20:02 +00:00
you 2e33eb7050 docs: add HTTPS/Caddyfile mount to release notes and upgrade steps 2026-04-08 07:14:15 +00:00
you 6dd0957507 docs: use v3.5.0 tag in release notes, :latest requires git tag 2026-04-08 07:05:58 +00:00
you e22ee3f0ad docs: docker run based upgrade, no compose 2026-04-08 07:03:05 +00:00
you f7f1bb08d0 docs: add cd to compose dir in upgrade steps 2026-04-08 07:01:43 +00:00
you 84da4d962d docs: release notes with juice 2026-04-08 07:00:59 +00:00
you ad0a10c009 docs: fix transition steps — compose-based, not docker run 2026-04-08 06:59:54 +00:00
you c1f268d3b9 docs: add concrete transition steps to release notes 2026-04-08 06:58:39 +00:00
you f5d25f75c6 docs: trim release notes — less book, more changelog 2026-04-08 06:56:50 +00:00
you cde62166cb docs: v3.5.0 release notes + API documentation across README, deployment guide, FAQ
- Release notes for 95 commits since v3.4.1
- OpenAPI/Swagger docs: /api/spec and /api/docs called out everywhere
- Deployment guide: new API Documentation section
- README: API docs link added
- FAQ: 'Where is the API documentation?' entry
- Test plans for v3.4.2 validation
2026-04-08 06:55:25 +00:00
Kpa-clawbot 5606bc639e fix: table sorting broken on all node tables — wrong data attribute (#679) (#680)
## Problem

All table sorting on the Nodes page was broken — clicking column headers
did nothing. Affected:
- Nodes list table
- Node detail → Neighbors table
- Node detail → Observers table

## Root Cause

**Not a race condition** — the actual bug was a **data attribute
mismatch**.

`TableSort.init()` (in `table-sort.js`) queries for `th[data-sort-key]`
to find sortable columns. But all table headers in `nodes.js` used
`data-sort="..."` instead of `data-sort-key="..."`. The selector never
matched any headers, so no click handlers were attached and sorting
silently failed.

Additionally, `data-type="number"` was used but TableSort's built-in
comparator is named `numeric`, causing numeric columns to fall back to
text comparison.

The packets table (`packets.js`) was unaffected because it already used
the correct `data-sort-key` and `data-type="numeric"` attributes.

## Fix

1. **`public/nodes.js`**: Changed all `data-sort="..."` to
`data-sort-key="..."` on `<th>` elements (nodes list, neighbors table,
observers table)
2. **`public/nodes.js`**: Changed `data-type="number"` to
`data-type="numeric"` to match TableSort's comparator names
3. **`public/packets.js`**: Added timestamp tiebreaker to packet sort
for stable ordering when primary column values are equal

## Testing

- All existing tests pass (`npm test`)
- No changes to test infrastructure needed — this was a pure HTML
attribute fix

Fixes #679

---------

Co-authored-by: you <you@example.com>
2026-04-07 23:30:31 -07:00
Kpa-clawbot 1373106b50 Fix panel corner toggle buttons invisible and scrolling away (#678)
## Summary

Panel corner toggle buttons (◫) were invisible due to small size, low
opacity, and `position: absolute` causing them to scroll away with panel
content.

## Changes

### Panel structure — non-scrolling header
All 3 live overlay panels (feed, node detail, legend) now use a flex
layout:
- **`.panel-header`** — non-scrolling row with corner toggle + close
button
- **`.panel-content`** — scrollable content area

### CSS updates
- `.live-overlay`: `display: flex; flex-direction: column`
- `.panel-header`: flex row, `flex-shrink: 0`
- `.panel-content`: `flex: 1; overflow-y: auto`
- `.panel-corner-btn`: removed `position: absolute`, increased to
28×28px, opacity 0.6, hover background

### JS updates
- Feed items now appended to `.panel-content` child instead of panel
root
- `rebuildFeedList` and `addFeedItem` updated to target `.panel-content`
- Resize handle still attaches to panel root (correct behavior)

## Testing
- All 490+ frontend helper tests pass
- All panel-corner tests pass (14/14)
- No test changes needed — tests exercise logic, not DOM structure

Fixes #677

---------

Co-authored-by: you <you@example.com>
2026-04-07 23:17:19 -07:00
Kpa-clawbot 68a4628edf fix: channel color picker — data shape mismatch + redesign for discoverability (#675)
## Fix: Channel Color Picker — Data Shape Mismatch + Redesign (#674)

### Problem

The channel color picker was completely non-functional — dead code.
Three locations in `live.js` attempted to read
`decoded.header.payloadTypeName` and `decoded.payload.channelName`, but:

1. The decoded payload structure is flat
(`decoded.payload.channelHash`), not nested with separate
`header`/`payload` objects within the payload
2. The field is `channelHash` (an integer), not `channelName`
3. `_ccChannel` was **never set** on any DOM element, so all picker
handlers exited early

Additionally, the picker had zero discoverability — hidden behind
right-click/long-press with no visual affordance.

### Changes

**M1 — Fix the data shape bug:**
- Fixed `_ccChannel` assignment in 3 locations in `live.js` to use
`decoded.payload.channelHash` (converted to string)
- Fixed `_getChannelStyle()` to use the same flat structure
- Channel colors now key on the hash string (e.g. `"5"`) matching the
channels API

**M2 — Redesign for discoverability:**
- Reduced palette from 10 to **8 maximally-distinct colors** (removed
teal/rose — too close to cyan/red)
- Removed `<input type="color">` custom picker, "Apply" button, title
bar, close button
- Popover is now just 8 circle swatches + "Clear color" — click outside
to dismiss
- Added **12px clickable color dots** next to channel names on the
channels page (primary configuration surface)
- Unassigned channels show a dashed-border empty circle; assigned show
filled
- Channel list items get `border-left: 3px solid` when colored
- **Removed long-press handler entirely** — dots handle mobile
interaction
- Mobile: bottom-sheet with 36px touch targets via `@media (pointer:
coarse)`

**M3 — Visual encoding:**
- Left border only (3px) — no background tint (per Tufte spec: minimum
effective dose)
- Consistent encoding across live feed items, channel list, packets
table

### Tests

17 new tests in `test-channel-color-picker.js`:
- `_ccChannel` correctly set for GRP_TXT with various `channelHash`
values (including 0)
- `_ccChannel` not set for non-GRP_TXT packets
- `getRowStyle` returns `border-left:3px` only (no background)
- Palette is exactly 8 colors, no teal/rose
- All existing tests pass (62 + 29 + 490)

Fixes #674

---------

Co-authored-by: you <you@example.com>
2026-04-07 23:03:57 -07:00
you 00953207fb ci: remove arm64 build + QEMU — amd64 only
Removes linux/arm64 from multi-platform build and drops QEMU setup.
All infra (prod + staging) is x86. QEMU emulation was adding ~12min
to every CI run for an unused architecture.
2026-04-08 05:23:41 +00:00
you 16a72b66a9 test: fix hash_size test for zero-hop behavior change (#653)
The buildFieldTable test expected hash_size=4 for path byte 0xC0 with
hash_count=0. After #653, zero hash_count shows 'hash_count=0 (direct
advert)' instead. Updated test and added new test verifying hash_size
IS shown when hash_count > 0.
2026-04-08 04:53:10 +00:00
Kpa-clawbot e0e9aaa324 feat: noise floor column chart with color-coded thresholds (#659)
## Noise Floor: Line Chart → Color-Coded Column Chart

Implements M3a from the [RF Health Dashboard
spec](https://github.com/Kpa-clawbot/CoreScope/issues/600#issuecomment-2784399622)
— replacing the noise floor line chart with discrete color-coded
columns.

### What changed

**`public/analytics.js`** — replaced `rfNFLineChart()` with
`rfNFColumnChart()`:

- **Color-coded bars by threshold**: green (`< -100 dBm`), yellow (`-100
to -85 dBm`), red (`≥ -85 dBm`)
- **Instant hover tooltips**: exact dBm value + UTC timestamp via native
SVG `<title>` — no delay
- **Column highlighting on hover**: CSS `:hover` with opacity change +
border stroke
- **Inline legend**: green/yellow/red threshold key in chart header
- **Removed reference lines**: the `-100 warning` and `-85 critical`
dashed lines are eliminated — threshold info is now encoded directly in
bar color (data-ink ratio improvement)
- **No gap detection**: column charts render discrete bars — each data
point is an independent observation, so line-chart-style gap detection
doesn't apply. Every sample gets a bar.
- **Reboot markers**: vertical dashed lines with "reboot" labels at
reboot timestamps (shared `rfRebootMarkers` helper, same as other RF
charts)
- **Division-by-zero guard**: constant values or single data points use
a ±5 dBm window so bars render with visible height
- **Sparklines unchanged**: fleet overview sparklines remain as
polylines (correct at 140×24px scale)

### Why columns instead of lines

A polyline connecting discrete 5-minute noise floor samples creates
false visual continuity — it implies interpolation between measurements
that doesn't exist. When readings jump between -115 and -95 irregularly,
the line becomes a jagged mess. Column bars encode each sample as a
discrete, independent observation: one bar = one measurement.

### Testing

- 12 unit tests in `test-frontend-helpers.js` covering: SVG output,
threshold color coding, tooltips, empty/single/constant data, legend
rendering, reboot markers, shared time axis
- All existing tests pass (packet-filter: 62, aging: 29,
frontend-helpers: 490)

### No backend changes

Pure frontend change — ~150 lines in `analytics.js`.

Fixes #600

---------

Co-authored-by: you <you@example.com>
2026-04-07 21:40:14 -07:00
Kpa-clawbot 22bf33700e Fix: filter path-hop candidates by resolved_path to prevent prefix collisions (#658)
## Problem

The "Paths Through This Node" API endpoint (`/api/nodes/{pubkey}/paths`)
returns unrelated packets when two nodes share a hex prefix. For
example, querying paths for "Kpa Roof Solar" (`c0dedad4...`) returns 316
packets that actually belong to "C0ffee SF" (`C0FFEEC7...`) because both
share the `c0` prefix in the `byPathHop` index.

Fixes #655

## Root Cause

`handleNodePaths()` in `routes.go` collects candidates from the
`byPathHop` index using 2-char and 4-char hex prefixes for speed, but
never verifies that the target node actually appears in each candidate's
resolved path. The broad index lookup is intentional, but the
**post-filter was missing**.

## Fix

Added `nodeInResolvedPath()` helper in `store.go` that checks whether a
transmission's `resolved_path` (from the neighbor affinity graph via
`resolveWithContext`) contains the target node's full pubkey. The
filter:

- **Includes** packets where `resolved_path` contains the target node's
full pubkey
- **Excludes** packets where `resolved_path` resolved to a different
node (prefix collision)
- **Excludes** packets where `resolved_path` is nil/empty (ambiguous —
avoids false positives)

The check examines both the best observation's resolved_path
(`tx.ResolvedPath`) and all individual observations, so packets are
included if *any* observation resolved the target.

## Tests

- `TestNodeInResolvedPath` — unit test for the helper with 5 cases
(match, different node, nil, all-nil elements, match in observation
only)
- `TestNodePathsPrefixCollisionFilter` — integration test: two nodes
sharing `aa` prefix, verifies the collision packet is excluded from one
and included for the other
- Updated test DB schema to include `resolved_path` column and seed data
with resolved pubkeys
- All existing tests pass (165 additions, 8 modifications)

## Performance

No impact on hot paths. The filter runs once per API call on the
already-collected candidate set (typically small). `nodeInResolvedPath`
is O(observations × hops) per candidate — negligible since observations
per transmission are typically 1–5.

---------

Co-authored-by: you <you@example.com>
2026-04-07 21:24:00 -07:00
Kpa-clawbot b8e9b04a97 feat: panel corner-position toggle (M0) (#657)
## Panel Corner-Position Toggle (M0)

Fixes #608

### What

Each overlay panel on the live map page (feed, legend, node detail) gets
a small corner-toggle button that cycles through **TL → TR → BR → BL**
placement. This solves the panel-blocking-map-data problem with minimal
complexity.

### Changes

**`public/live.css`** (~60 lines)
- CSS classes for 4 corner positions via `data-position` attribute
- Smooth transitions with `cubic-bezier` easing
- `prefers-reduced-motion` support
- Direction-aware hide animations for positioned panels
- `.panel-corner-btn` styling (subtle, hover-to-reveal)
- Mobile: corner buttons hidden (`<640px` — panels are hidden or
bottom-sheet)
- `.sr-only` class for screen reader announcements

**`public/live.js`** (~90 lines)
- `PANEL_DEFAULTS`, `CORNER_CYCLE`, `CORNER_ARROWS` constants
- `getPanelPositions()` — reads from localStorage with defaults
- `nextAvailableCorner()` — collision avoidance (skips occupied corners)
- `applyPanelPosition()` — sets `data-position` + updates button
- `onCornerClick()` — cycle logic + persistence + SR announcement
- `resetPanelPositions()` — clears saved positions
- Corner toggle buttons added to feed, legend, and node detail panel
HTML
- `initPanelPositions()` called during page init

**`test-panel-corner.js`** (14 tests)
- `nextAvailableCorner`: available, skip occupied, skip multiple,
self-exclusion
- `getPanelPositions`: defaults, saved values
- `applyPanelPosition`: attribute setting, button update, missing
element
- `onCornerClick`: cycling, collision avoidance
- `resetPanelPositions`: clear + restore defaults
- Cycle order and default position validation

### What this does NOT include

- Drag-and-drop (M1–M4)
- Snap-to-edge
- Z-index management
- Keyboard repositioning
- Any of the full drag system

### Design decisions

- **`data-position` + CSS classes** over inline transforms — avoids
conflict with existing show/hide `transform` animations
- **Cycle (TL→TR→BR→BL)** over toggle-to-opposite — predictable,
learnable
- **3 panels, 4 corners** — collision avoidance is trivial, always a
free corner
- **Header/stats panel excluded** — it's contextual chrome, not
repositionable

---------

Co-authored-by: you <you@example.com>
2026-04-07 21:20:29 -07:00
Kpa-clawbot 7d71dc857b feat: expose hopsCompleted for TRACE packets, show real path on live map (#656)
## Summary

TRACE packets on the live map previously animated the **full intended
route** regardless of how far the trace actually reached. This made it
impossible to distinguish a completed route from a failed one —
undermining the primary diagnostic purpose of trace packets.

## Changes

### Backend — `cmd/server/decoder.go`

- Added `HopsCompleted *int` field to the `Path` struct
- For TRACE packets, the header path contains SNR bytes (one per hop
that actually forwarded). Before overwriting `path.Hops` with the full
intended route from the payload, we now capture the header path's
`HashCount` as `hopsCompleted`
- This field is included in API responses and WebSocket broadcasts via
the existing JSON serialization

### Frontend — `public/live.js`

- For TRACE packets with `hopsCompleted < totalHops`:
  - Animate only the **completed** portion (solid line + pulse)
- Draw the **unreached** remainder as a dashed/ghosted line (25%
opacity, `6,8` dash pattern) with ghost markers
  - Dashed lines and ghost markers auto-remove after 10 seconds
- When `hopsCompleted` is absent or equals total hops, behavior is
unchanged

### Tests — `cmd/server/decoder_test.go`

- `TestDecodePacket_TraceHopsCompleted` — partial completion (2 of 4
hops)
- `TestDecodePacket_TraceNoSNR` — zero completion (trace not forwarded
yet)
- `TestDecodePacket_TraceFullyCompleted` — all hops completed

## How it works

The MeshCore firmware appends an SNR byte to `pkt->path[]` at each hop
that forwards a TRACE packet. The count of these SNR bytes (`path_len`)
indicates how far the trace actually got. CoreScope's decoder already
parsed the header path, but the TRACE-specific code overwrote it with
the payload hops (full intended route) without preserving the progress
information. Now we save that count first.

Fixes #651

---------

Co-authored-by: you <you@example.com>
2026-04-07 21:19:45 -07:00
Kpa-clawbot 088b4381c3 Fix: Hash Stats 'By Repeaters' includes non-repeater nodes (#654)
## Summary

The "By Repeaters" section on the Hash Stats analytics page was counting
**all** node types (companions, room servers, sensors, etc.) instead of
only repeaters. This made the "By Repeaters" distribution identical to
"Multi-Byte Hash Adopters", defeating the purpose of the breakdown.

Fixes #652

## Root Cause

`computeAnalyticsHashSizes()` in `cmd/server/store.go` built its
`byNode` map from advert packet data without cross-referencing node
roles from the node store. Both `distributionByRepeaters` and
`multiByteNodes` consumed this unfiltered map.

## Changes

### `cmd/server/store.go`
- Build a `nodeRoleByPK` lookup map from `getCachedNodesAndPM()` at the
start of the function
- Store `role` in each `byNode` entry when processing advert packets
- **`distributionByRepeaters`**: filter to only count nodes whose role
contains "repeater"
- **`multiByteNodes`**: include `role` field in output so the frontend
can filter/group by node type

### `cmd/server/coverage_test.go`
- Add `TestHashSizesDistributionByRepeatersFiltersRole`: verifies that
companion nodes are excluded from `distributionByRepeaters` but included
in `multiByteNodes` with correct role

### `cmd/server/routes_test.go`
- Fix `TestHashAnalyticsZeroHopAdvert`: invalidate node cache after DB
insert so role lookup works
- Fix `TestAnalyticsHashSizeSameNameDifferentPubkey`: insert node
records as repeaters + invalidate cache

## Testing

All `cmd/server` tests pass (68 insertions, 3 deletions across 3 files).

Co-authored-by: you <you@example.com>
2026-04-07 21:00:03 -07:00
you 1ff094b852 fix: staging compose — standard ports, remove 3GB memory limit
- HTTP: 82→80 (standard)
- MQTT: 1885→1883 (standard)
- Remove 3GB memory limit that was causing OOM on 1.5M observation DB
2026-04-08 03:50:07 +00:00
efiten 144e98bcdf fix: hide hash size for zero-hop direct adverts (#649) (#653)
## Fix: Zero-hop DIRECT packets report bogus hash_size

Closes #649

### Problem
When a DIRECT packet has zero hops (pathByte lower 6 bits = 0), the
generic `hash_size = (pathByte >> 6) + 1` formula produces a bogus value
(1-4) instead of 0/unknown. This causes incorrect hash size displays and
analytics for zero-hop direct adverts.

### Solution

**Frontend (JS):**
- `packets.js` and `nodes.js` now check `(pathByte & 0x3F) === 0` to
detect zero-hop packets and suppress bogus hash_size display.

**Backend (Go):**
- Both `cmd/server/decoder.go` and `cmd/ingestor/decoder.go` reset
`HashSize=0` for DIRECT packets where `pathByte & 0x3F == 0` (hash_count
is zero).
- TRACE packets are excluded since they use hashSize to parse hop data
from the payload.
- The condition uses `pathByte & 0x3F == 0` (not `pathByte == 0x00`) to
correctly handle the case where hash_size bits are non-zero but
hash_count is zero — matching the JS frontend approach.

### Testing

**Backend:**
- Added 4 tests each in `cmd/server/decoder_test.go` and
`cmd/ingestor/decoder_test.go`:
  - DIRECT + pathByte 0x00 → HashSize=0 
- DIRECT + pathByte 0x40 (hash_size bits set, hash_count=0) → HashSize=0

  - Non-DIRECT + pathByte 0x00 → HashSize=1 (unchanged) 
  - DIRECT + pathByte 0x01 (1 hop) → HashSize=1 (unchanged) 
- All existing tests pass (`go test ./...` in both cmd/server and
cmd/ingestor)

**Frontend:**
- Verified hash size display is suppressed for zero-hop direct adverts

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
2026-04-07 19:39:15 -07:00
efiten bd54707987 feat: distance unit preference — km, mi, or auto (#621) (#646)
## Summary

- **`app.js`**: `getDistanceUnit()`, `formatDistance(km)`,
`formatDistanceRound(km)` helpers. Auto mode uses `navigator.language` —
miles for `en-US`, `en-GB`, `my`, `lr`; km everywhere else.
- **`customize-v2.js`**: Distance Unit preference (km / mi / auto) in
Display Settings panel. Stored in
`localStorage['meshcore-distance-unit']` via the existing apply
pipeline. Override dot and reset work. Display tab badge counts it.
- **`nodes.js`**: Neighbor table distance cell uses `formatDistance()`.
- **`analytics.js`**: All rendered km values use `formatDistance()` or
`formatDistanceRound()`. Column headers (`km`/`mi`) respond to the
active unit. Collision classification thresholds (Local < 50 km /
Regional 50–200 km / Distant > 200 km) also adapt.

Default is `auto` — no change for existing users unless their locale
maps to miles.

## Test plan

- [x] `node test-frontend-helpers.js` — 456 passed, 0 failed (10 new
formatDistance tests)
- [ ] Set unit to **mi** in customize → Neighbors table shows `7.6 mi`
instead of `12.3 km`
- [ ] Analytics → Distance tab → stat cards, leaderboard, and column
headers all show miles
- [ ] Collision tool → Local/Regional/Distant thresholds show `31 mi` /
`124 mi`
- [ ] Route patterns popup shows miles per hop and total
- [ ] Reset override dot → unit returns to auto

Closes #621

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
2026-04-07 19:36:25 -07:00
efiten 1033555d00 fix: resolve originLat out-of-scope ReferenceError in resolveHopPositions (#647) (#648)
## Summary

- `originLat` was declared with `const` inside two block-scoped
`if`/`else` branches in `resolveHopPositions` (lines 1914 and 1921) but
referenced at line 1945 outside both blocks → `ReferenceError: originLat
is not defined` thrown on every packet render on the live page.
- Fix: introduce `senderLat` derived directly from
`payload.lat`/`payload.lon` at the point of use, using the same
null/zero guard as the existing declarations.

## Test plan

- [x] Live page no longer shows `ReferenceError: originLat is not
defined` in the console
- [x] Packet path animations still render correctly for packets with GPS
coords
- [x] Packets without GPS coords still handled (senderLat === null,
anchor not added)

Closes #647

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
2026-04-07 19:31:43 -07:00
Kpa-clawbot 37be3dcd1f fix: Prefix Tool text consistency — use 'repeaters' everywhere (#642) (#645)
## Summary

Fixes remaining text inconsistencies in the Prefix Tool after #643 added
the repeater filter.

The Torvalds review on #643 flagged:
1. **Must-fix (already addressed in #643):** "About these numbers" text
— fixed
2. **Out-of-scope:** Empty state says "No nodes" should say "No
repeaters"

This PR fixes ALL remaining "nodes" references in the Prefix Tool to say
"repeaters":

- Empty state: "No nodes in the network yet" → "No repeaters in the
network yet"
- Stat card label: "Total nodes" → "Total repeaters"
- Region note link: "Check all nodes →" → "Check all repeaters →"
- Recommendation text: "With N nodes" → "With N repeaters"

Verified: zero occurrences of stale "all nodes", "Total nodes", or "No
nodes" remain in the Prefix Tool section.

Closes #642

Co-authored-by: you <you@example.com>
2026-04-06 15:43:43 -07:00
efiten 2bff89a546 feat: deep link P1 UI states — nodes tab, packets filters, channels node panel (#536) (#618)
## Summary

- **nodes.js**: `#/nodes?tab=repeater` and `#/nodes?search=foo` — role
tab and search query are now URL-addressable; state resets to defaults
on re-navigation
- **packets.js**: `#/packets?timeWindow=60` and
`#/packets?region=US-SFO` — time window and region filter survive
refresh and are shareable
- **channels.js**: `#/channels/{hash}?node=Name` — node detail panel is
URL-addressable; auto-opens on load, URL updates on open/close
- **region-filter.js**: adds `RegionFilter.setSelected(codesArray)` to
public API (needed for URL-driven init)

All changes use `history.replaceState` (not `pushState`) to avoid
polluting browser history. URL params override localStorage on load;
localStorage remains fallback.

## Implementation notes

- Router strips query string before computing `routeParam`, so all pages
read URL params directly from `location.hash`
- `buildNodesQuery(tab, searchStr)` and `buildPacketsUrl(timeWindowMin,
regionParam)` are pure functions exposed on `window` for testability
- Region URL param is applied after `RegionFilter.init()` via a
`_pendingUrlRegion` module-level var to keep ordering explicit
- `showNodeDetail` captures `selectedHash` before the async `lookupNode`
call to avoid stale URL construction

## Test plan

- [x] `node test-frontend-helpers.js` — 459 passed, 0 failed (includes 6
`buildNodesQuery` + 5 `buildPacketsUrl` unit tests)
- [x] Navigate to `#/nodes?tab=repeater` — Repeaters tab active on load
- [x] Click a tab, verify URL updates to `#/nodes?tab=room`
- [x] Navigate to `#/packets?timeWindow=60` — time window dropdown shows
60 min
- [x] Change time window, verify URL updates
- [x] Navigate to `#/channels/{hash}` and click a sender name — URL
updates to `?node=Name`
- [x] Reload that URL — node panel re-opens

Closes #536

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 15:43:25 -07:00
Kpa-clawbot dc079064f5 fix: clarify Hash Issues vs Prefix Tool collision data discrepancy (#643)
## Summary

Hash Issues and Prefix Tool tabs showed different collision counts
because the Prefix Tool was including all node types (companions, rooms,
sensors) while Hash Issues correctly filtered to repeaters only.

**Only repeaters matter for prefix collisions** — they're the nodes that
relay packets using hash-based addressing. Non-repeater collisions are
harmless noise.

## Changes
1. **Filtered Prefix Tool to repeaters only** — matches Hash Issues'
scope
2. **Updated explanatory text** — both tabs now clearly state they cover
repeaters
3. **Added cross-reference links** between the two tabs
4. **Added hash_size badges** in Prefix Tool results

Both tabs should now agree on collision counts for each byte size.

## Review Status
-  Self-review
-  Torvalds review — caught stale 'regardless of role' text, fixed
-  All tests pass

Fixes #642

---------

Co-authored-by: you <you@example.com>
2026-04-05 19:52:19 -07:00
Kpa-clawbot 43098a0705 refactor: DRY hash matrix rendering in analytics.js (#419) (#640)
## Summary

Fixes #419 — DRY violation in `renderHashMatrixFromServer` in
analytics.js.

The 1-byte and 2-byte branches shared ~80% identical HTML structure
(stat cards, matrix grid, detail panel, legend, tooltip init, click
handlers). This refactor extracts four shared helpers:

### New helpers

| Helper | Purpose |
|--------|---------|
| `classifyHashCell(count, isConfirmed, isPossible)` | Unified cell
classification → `{cls, bg}` |
| `hashCellTd(hex, cellSize, cls, bg, count, tipHtml, fontWeight)` |
Shared `<td>` element generation |
| `hashTooltipHtml(hexLabel, statusText, nodesHtml)` | Tooltip HTML
assembly |
| `renderHashMatrixPanel(el, statCards, cellFn, detailWidth, legend,
clickFn)` | Full matrix assembly pipeline |

### What changed

- Both branches now call `renderHashMatrixPanel()` with branch-specific
callbacks for cell rendering and detail click handling
- Cell classification logic (empty → taken → possible → collision with
heat scaling) is unified in `classifyHashCell()`
- Tooltip and `<td>` generation consolidated — no more duplicated inline
template strings
- Zero behavioral changes — all existing rendering, tooltips, and click
interactions are preserved

### Tests

All existing tests pass (445 frontend helpers, 62 packet filter, 29
aging).

Co-authored-by: you <you@example.com>
2026-04-05 18:31:23 -07:00
Kpa-clawbot 2d260bbfed test: behavioral vscroll tests replacing source-grep (#405, #409) (#641)
## Summary

Replace source-grep virtual scroll tests with behavioral tests that
exercise actual logic. Fixes #405, Fixes #409.

## What changed

### packets.js
- **Extracted `_calcVisibleRange()`** — pure function containing the
binary-search range calculation logic previously inline in
`renderVisibleRows()`. Takes offsets, scroll position, viewport
dimensions, row height, thead offset, and buffer as parameters. Returns
`{ startIdx, endIdx, firstEntry, lastEntry }`.
- `renderVisibleRows()` now calls `_calcVisibleRange()` instead of
inline math — no behavioral change.
- Exported via `_packetsTestAPI` for direct testing.

### test-frontend-helpers.js
- **Removed 8 source-grep tests** that used
`packetsSource.includes(...)` to check strings exist in source code (not
behavior):
  - "renderVisibleRows uses cumulative offsets not flat entry count"
  - "renderVisibleRows skips DOM rebuild when range unchanged"
  - "lazy row generation — HTML built only for visible slice"
  - "observer filter Set is hoisted, not recreated per-packet"
  - "packets.js display filter checks _children for observer match"
  - "packets.js WS filter checks _children for observer match"
  - "buildFlatRowHtml has null-safe decoded_json"
  - "pathHops null guard in buildFlatRowHtml / detail pane"
  - "destroy cleans up virtual scroll state"

- **Added 11 behavioral tests for `_calcVisibleRange()`** loaded from
the actual packets.js via sandbox:
  - Top of list (scroll = 0)
  - Middle of list (scroll to row 50)
  - Bottom of list (scroll past end)
  - Empty array (0 entries)
  - Single item
  - Exact row boundary
  - Large dataset (30K items)
  - Various row heights (24px instead of 36px)
  - Thead offset shifting visible range
  - Expanded groups with variable row counts
  - Buffer clamped at boundaries

- **Kept all existing behavioral tests**: `cumulativeRowOffsets`,
`getRowCount`, observer filter logic (#537).

## Test count
- Removed: 8 source-grep tests
- Added: 11 behavioral tests
- Net: +3 tests (446 total, 0 failures)

## Why
Source-grep tests (`packetsSource.includes('...')`) are brittle — they
break on refactors even when behavior is preserved, and they pass even
when the tested code is buggy. Behavioral tests exercise real
inputs/outputs and catch actual regressions.

Co-authored-by: you <you@example.com>
2026-04-05 18:30:30 -07:00
Kpa-clawbot 1dd763bf44 feat: sortable nodes list + neighbor/observer tables (M2, #620) (#639)
## Summary

Implements M2 of the table sorting spec (#620): sortable nodes list +
neighbor/observer tables.

### Changes

**Shared utility (`public/table-sort.js`)**
- IIFE pattern, no dependencies, no build step
- DOM-reorder sorting (no innerHTML rebuild) — preserves event listeners
- `data-value` attributes for raw sortable values, `data-type` on `<th>`
for type detection
- Built-in comparators: text (`localeCompare`), number, date, dBm
- `aria-sort` attributes, keyboard support (Enter/Space), sort arrows
- localStorage persistence with `storageKey` option
- `onSort` callback for custom re-render triggers

**Nodes list table**
- Wired via `TableSort.init` with `onSort` callback that triggers
`renderRows()`
- Keeps JS-array-level sorting for claimed/favorites pinning (TableSort
can't handle pinned rows)
- Replaces old `sortState`, `toggleSort()`, `sortArrow()` with TableSort
controller
- Test hooks preserved for backward compatibility (fallback state for
non-DOM tests)

**Neighbor table**
- Added `data-sort` and `data-value` attributes to all columns (name,
role, score, count, last_seen, distance)
- Default sort: count descending
- `TableSort.init` called after neighbor data renders

**Observer table (full detail page)**
- Converted from plain `<table>` to sortable table with data attributes
- Sortable columns: observer, region, packets, avg SNR, avg RSSI
- Default sort: packets descending

### Testing
- 18 new unit tests for `table-sort.js` (custom DOM mock, no jsdom
dependency)
- All 445 existing frontend tests pass unchanged
- All packet-filter (62) and aging (29) tests pass

### Note
This branch includes `table-sort.js` since M1 hasn't merged yet. The
utility code is identical to the M1 spec.

---------

Co-authored-by: you <you@example.com>
2026-04-05 18:29:54 -07:00
you 6b9946d9c6 docs: timestamp-based packet filter spec (#289) 2026-04-06 01:22:15 +00:00
Kpa-clawbot 243de9fba1 fix: consolidate CI pipeline — build, publish to GHCR, then deploy staging (#636)
## Consolidate CI Pipeline — Build + Publish to GHCR + Deploy Staging

### What
Merges the separate `publish.yml` workflow into `deploy.yml`, creating a
single CI/CD pipeline:

**`go-test → e2e-test → build-and-publish → deploy → publish-badges`**

### Why
- Two workflows doing overlapping builds was wasteful and error-prone
- `publish.yml` had a bug: `BUILD_TIME=$(date ...)` in a `with:` block
never executed (literal string)
- The old build job had duplicate/conflicting `APP_VERSION` assignments

### Changes
- **`build-and-publish` job** replaces old `build` job — builds locally
for staging, then does multi-arch GHCR push (gated to push events only,
PRs skip)
- **Build metadata** computed in a dedicated step, passed via
`GITHUB_OUTPUT` — no more shell expansion bugs
- **`APP_VERSION`** is `v1.2.3` on tag push, `edge` on master push
- **Deploy** now pulls the `edge` image from GHCR and tags for compose
compatibility, with fallback to local build
- **`publish.yml` deleted** — no duplicate workflow
- **Top-level `permissions`** block with `packages:write` for GHCR auth
- **Triggers** now include `tags: ['v*']` for release publishing

### Status
-  Rebased onto master
-  Self-reviewed (all checklist items pass)
-  Ready for merge

Co-authored-by: you <you@example.com>
2026-04-05 18:09:20 -07:00
Kpa-clawbot 6f3e3535c9 feat: shared table sort utility + packets table sorting (M1, #620) (#638)
## Summary

Implements M1 of the table sorting spec (#620): a shared `TableSort`
utility module and integration with the packets table.

### What's included

**1. `public/table-sort.js` — Shared sort utility (IIFE, no
dependencies)**
- `TableSort.init(tableEl, options)` — attaches click-to-sort on `<th
data-sort-key="...">` elements
- Built-in comparators: text (localeCompare), numeric, date (ISO), dBm
(strips suffix)
- NaN/null values sort last consistently
- Visual: ▲/▼ `<span class="sort-arrow">` appended to active column
header
- Accessibility: `aria-sort="ascending|descending|none"`, keyboard
support (Enter/Space)
- DOM reorder via `appendChild` loop (no innerHTML rebuild)
- `domReorder: false` option for virtual scroll tables (packets)
- `storageKey` option for localStorage persistence
- Custom comparator override per column
- `onSort(column, direction)` callback
- `destroy()` for clean teardown

**2. Packets table integration**
- All columns sortable: region, time, hash, size, HB, type, observer,
path, rpt
- Default sort: time descending (matches existing behavior)
- Uses `domReorder: false` + `onSort` callback to sort the data array,
then re-render via virtual scroll
- Works with both grouped and ungrouped views
- WebSocket updates respect active sort column
- Sort preference persisted in localStorage (`meshcore-packets-sort`)

**3. Tests — 22 unit tests (`test-table-sort.js`)**
- All 4 built-in comparators (text, numeric, date, dBm)
- NaN/null edge cases
- Direction toggle on click
- aria-sort attribute correctness
- Visual indicator (▲/▼) presence and updates
- onSort callback
- domReorder: false behavior
- destroy() cleanup
- Custom comparator override

### Performance

Packets table sorting works at the data array level (single `Array.sort`
call), not DOM level. Virtual scroll then renders only visible rows. No
new DOM nodes are created during sort — it's purely a data reorder +
re-render of the existing visible window. Expected sort time for 30K
packets: ~50-100ms (array sort) + existing virtual scroll render time.

Closes #620 (M1)

Co-authored-by: you <you@example.com>
2026-04-05 15:29:14 -07:00
Kpa-clawbot cae14da05e fix: implement DISABLE_CADDY env var in Docker entrypoint (#629) (#637)
## Summary

Implements the `DISABLE_CADDY` environment variable in the Docker
entrypoint, fixing #629.

## Problem

The `DISABLE_CADDY` env var was documented but had no effect — the
entrypoint only handled `DISABLE_MOSQUITTO`.

## Changes

### New supervisord configs
- **`supervisord-go-no-caddy.conf`** — mosquitto + ingestor + server (no
Caddy)
- **`supervisord-go-no-mosquitto-no-caddy.conf`** — ingestor + server
only

### Updated entrypoint (`docker/entrypoint-go.sh`)
Handles all 4 combinations:
| DISABLE_MOSQUITTO | DISABLE_CADDY | Config used |
|---|---|---|
| false | false | `supervisord.conf` (default) |
| true | false | `supervisord-no-mosquitto.conf` |
| false | true | `supervisord-no-caddy.conf` |
| true | true | `supervisord-no-mosquitto-no-caddy.conf` |

### Dockerfiles
Added COPY lines for the new configs in both `Dockerfile` and
`Dockerfile.go`.

## Testing

```bash
# Verify correct config selection
docker run -e DISABLE_CADDY=true corescope
# Should log: [config] Caddy reverse proxy disabled (DISABLE_CADDY=true)

docker run -e DISABLE_CADDY=true -e DISABLE_MOSQUITTO=true corescope
# Should log both disabled messages
```

Fixes #629

Co-authored-by: you <you@example.com>
2026-04-05 15:26:40 -07:00
Kpa-clawbot e046a6f632 fix: mobile accessibility — touch targets, ARIA, small viewport support (#630) (#633)
## Summary

Fixes critical and major mobile accessibility items from #630, focused
on small phone viewports (320px–375px).

### Critical fixes
1. **Touch targets ≥ 44px** — All interactive elements (filter buttons,
tab buttons, search inputs, nav buttons, region pills, dropdowns) get
`min-height: 44px; min-width: 44px` via `@media (pointer: coarse)` —
desktop/mouse users are unaffected.
2. **ARIA live regions** — Added `aria-live="polite"` to: packet list
(`#pktLeft`), node list (`#nodesLeft`), analytics content
(`#analyticsContent`), live feed (`#liveFeed` with `role="log"`). Screen
readers now announce dynamic content updates.
3. **Color-only status indicators** — Status dots in live view marked
`aria-hidden="true"` (text labels like "Online"/"Degraded"/"Offline"
already present alongside).
4. **Detail panel on mobile** — Side panel (`panel-right`) renders as a
full-screen fixed overlay on ≤640px. Close button (✕) added to nodes
detail panel. Escape key closes both nodes and packets detail panels.

### Major fixes
5. **Analytics tabs overflow** — Tabs switch to `flex-wrap: nowrap;
overflow-x: auto` on ≤640px, preventing overflow on 320px screens.
6. **Table horizontal scroll** — Added `.table-scroll-wrap` class and
`min-width: 480px` on `.data-table` at ≤640px for horizontal scrolling
when columns don't fit.
7. **SPA focus management** — On every page navigation, focus moves to
first heading (`h1`/`h2`/`h3`) or falls back to `#app`. Uses
`requestAnimationFrame` for correct DOM timing.

### Bonus
- Analytics tabs get `role="tablist"` + `aria-label` for screen reader
semantics.

### Known follow-ups (not blocking)
- Individual tab buttons should get `role="tab"` + `aria-selected` +
`aria-controls` for complete ARIA tab pattern.
- `sr-status-label` and `table-scroll-wrap` CSS classes are defined but
not yet used in JS — ready for future use when status text labels and
table wrappers are wired up.

Closes #630

Co-authored-by: you <you@example.com>
2026-04-05 15:06:14 -07:00
Kpa-clawbot 0f5e2db5cf feat: auto-generated OpenAPI 3.0 spec endpoint + Swagger UI (#530) (#632)
## Summary

Auto-generated OpenAPI 3.0.3 spec endpoint (`/api/spec`) and Swagger UI
(`/api/docs`) for the CoreScope API.

## What

- **`cmd/server/openapi.go`** — Route metadata map
(`routeDescriptions()`) + spec builder that walks the mux router to
generate a complete OpenAPI 3.0.3 spec at runtime. Includes:
- All 47 API endpoints grouped by tag (admin, analytics, channels,
config, nodes, observers, packets)
- Query parameter documentation for key endpoints (packets, nodes,
search, resolve-hops)
  - Path parameter extraction from mux `{name}` patterns
  - `ApiKeyAuth` security scheme for API-key-protected endpoints
  - Swagger UI served as a self-contained HTML page using unpkg CDN

- **`cmd/server/openapi_test.go`** — Tests for spec endpoint (validates
JSON structure, required fields, path count, security schemes,
self-exclusion of `/api/spec` and `/api/docs`), Swagger UI endpoint, and
`extractPathParams` helper.

- **`cmd/server/routes.go`** — Stores router reference on `Server`
struct for spec generation; registers `/api/spec` and `/api/docs`
routes.

## Design Decisions

- **Runtime spec generation** vs static YAML: The spec walks the actual
router, so it can never drift from registered routes. Route metadata
(summaries, descriptions, tags, auth flags) is maintained in a parallel
map — the test enforces minimum path count to catch drift.
- **No external dependencies**: Uses only stdlib + existing gorilla/mux.
Swagger UI loaded from unpkg CDN (no vendored assets).
- **Security tagging**: Auth-protected endpoints (those behind
`requireAPIKey` middleware) are tagged with `security: [{ApiKeyAuth:
[]}]` in the spec, matching the actual middleware configuration.

## Testing

- `go test -run TestOpenAPI` — validates spec structure, field presence,
path count ≥ 20, security schemes
- `go test -run TestSwagger` — validates HTML response with swagger-ui
references
- `go test -run TestExtractPathParams` — unit tests for path parameter
extraction

---------

Co-authored-by: you <you@example.com>
2026-04-05 15:05:20 -07:00
Kpa-clawbot a068e3e086 feat: zero-config defaults + deployment docs (M3-M4, #610) (#631)
## Zero-Config Defaults + Deployment Docs

Make CoreScope start with zero configuration — no `config.json`
required. The ingestor falls back to sensible defaults (local MQTT
broker, standard topics, default DB path) when no config file exists.

### What changed

**`cmd/ingestor/config.go`** — `LoadConfig` no longer errors on missing
config file. Instead it logs a message and uses defaults. If no MQTT
sources are configured (from file or env), defaults to
`mqtt://localhost:1883` with `meshcore/#` topic.

**`cmd/ingestor/main.go`** — Removed redundant "no MQTT sources" fatal
(now handled in config layer). Improved the "no connections established"
fatal with actionable hints.

**`README.md`** — Replaced "Docker (Recommended)" section with a
one-command quickstart using the pre-built image. No build step, no
config file, just `docker run`.

**`docs/deployment.md`** — New comprehensive deployment guide covering
Docker, Compose, config reference, MQTT setup, TLS/HTTPS, monitoring,
backup, and troubleshooting.

### Zero-config flow

```
docker run -d -p 80:80 -v corescope-data:/app/data ghcr.io/kpa-clawbot/corescope:latest
```

1. No config.json found → defaults used, log message printed
2. No MQTT sources → defaults to `mqtt://localhost:1883`
3. Internal Mosquitto broker already running in container → connection
succeeds
4. Dashboard shows empty, ready for packets

### Review fixes (commit 13b89bb)

- Removed `DISABLE_CADDY` references from all docs — this env var was
never implemented in the entrypoint
- Fixed `/api/stats` example in deployment guide — showed nonexistent
fields (`mqttConnected`, `uptimeSeconds`, `activeNodes`)
- Improved MQTT connection failure message with actionable
troubleshooting hints

Closes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 15:04:49 -07:00
you 24335164d6 docs: table sorting consistency spec (#620) 2026-04-05 21:56:09 +00:00
Kpa-clawbot 7cef89e07b fix: mobile UX improvements for channel color picker (#619) (#626)
## Summary

Mobile UX fixes for the channel color picker (addresses #619).

## Changes

### Commit 1: Mobile UX improvements
- **Bottom-sheet pattern on mobile**: Color picker renders as a fixed
bottom sheet on touch devices (`@media (pointer: coarse)`) with
`env(safe-area-inset-bottom)` for notched phones
- **40px touch targets**: Swatches enlarged from default to 40×40px on
mobile
- **Native color picker hidden on touch**: `<input type="color">` is
hidden on mobile — preset swatches only
- **Scroll lock**: `document.body.style.overflow = 'hidden'` while
popover is open, restored on close
- **CSS context menu suppression**: `-webkit-touch-callout: none` and
`user-select: none` on `.live-feed-item`
- **Long-press with `passive: true`**: touchstart listener is passive to
avoid scroll jank

### Commit 2: Remove preventDefault on touchstart
- Removed `e.preventDefault()` from the touchstart handler — it was
blocking scroll initiation on feed items
- Context menu suppression handled entirely via CSS (see above)

## Desktop behavior
Unchanged. All mobile-specific styles scoped under `@media (pointer:
coarse)`. Desktop positioning logic unchanged.

## Review Status
-  Rebased onto master (no conflicts)
-  Self-review complete — all checklist items verified
-  Tufte analysis posted as comment

---------

Co-authored-by: you <you@example.com>
2026-04-05 14:51:13 -07:00
Kpa-clawbot dc5b5ce9a0 fix: reject weak/default API keys + startup warning (#532) (#628)
## Summary

Hardens API key security for write endpoints (fixes #532):

1. **Constant-time comparison** — uses
`crypto/subtle.ConstantTimeCompare` to prevent timing attacks on API key
validation
2. **Weak key blocklist** — rejects known default/example keys (`test`,
`password`, `change-me`, `your-secret-api-key-here`, etc.)
3. **Minimum length enforcement** — keys shorter than 16 characters are
rejected
4. **Startup warning** — logs a clear warning if the configured key is
weak or a known default
5. **Generic error messages** — HTTP 403 response uses opaque
"forbidden" message to prevent information leakage about why a key was
rejected

### Security Model
- **Empty key** → all write endpoints disabled (403)
- **Weak/default key** → all write endpoints disabled (403), startup
warning logged
- **Wrong key** → 401 unauthorized
- **Strong correct key** → request proceeds

### Files Changed
- `cmd/server/config.go` — `IsWeakAPIKey()` function + blocklist
- `cmd/server/routes.go` — constant-time comparison via
`constantTimeEqual()`, weak key rejection
- `cmd/server/main.go` — startup warning for weak keys
- `cmd/server/apikey_security_test.go` — comprehensive test coverage
- `cmd/server/routes_test.go` — existing tests updated to use strong
keys

### Reviews
-  Self-review: all security properties verified
-  djb Final Review: timing fix correct, blocklist pragmatic, error
messages opaque, tests comprehensive. **Verdict: Ship it.**

### Test Results
All existing + new tests pass. Coverage includes: weak key detection
(blocklist + length + case-insensitive), empty key handling, strong key
acceptance, wrong key rejection, and constant-time comparison.

---------

Co-authored-by: you <you@example.com>
2026-04-05 14:50:40 -07:00
Kpa-clawbot f59b4629b0 feat: publish Docker images to GHCR + simplified deploy (#610) (#627)
## Summary

Implements M1-M2 of the deployment simplification spec (#610). Adds
pre-built multi-arch Docker images published to GHCR, plus a simplified
deploy experience for operators.

**Spec:**
[docs/specs/deployment-simplification.md](https://github.com/Kpa-clawbot/CoreScope/blob/master/docs/specs/deployment-simplification.md)

## Files Added (no existing files modified)

### 1. `.github/workflows/publish.yml`
Multi-arch Docker publish workflow:
- Triggers on `v*` tags (releases) → produces `vX.Y.Z`, `vX.Y`, `vX`,
`latest`
- Triggers on master push → produces `edge` (unstable)
- `workflow_dispatch` for manual runs
- QEMU + buildx for `linux/amd64` + `linux/arm64`
- GHCR auth via `GITHUB_TOKEN`
- GHA layer caching for fast rebuilds

### 2. `docker-compose.example.yml`
20-line compose file that pulls from GHCR (no local build required):
- Env var overrides: `HTTP_PORT`, `DATA_DIR`, `DISABLE_CADDY`,
`DISABLE_MOSQUITTO`
- Health check included
- Volume mount for data persistence

### 3. `DEPLOY.md`
Operator documentation:
- One-line `docker run` deploy
- Tag reference (pinned vs latest vs edge)
- Environment variables table
- Update path (`docker compose pull && docker compose up -d`)
- TLS options (Caddy auto-TLS vs reverse proxy)
- **Migration guide for existing manage.sh users** — both paths
documented with command equivalency table

## Review Status

-  Self-review: Actions syntax, GHCR auth, multi-arch, tag strategy,
security — all verified
-  Torvalds: Deploy UX is clean, one-liner works, right level of
simplicity
-  BUILD_TIME fixed: uses `date` command instead of fragile
`head_commit.timestamp`
-  Migration guide added for existing manage.sh admins
- ⚠️ `DISABLE_CADDY` env var documented but not implemented in
entrypoint — pre-existing bug, filed as #629

Fixes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 14:33:57 -07:00
Kpa-clawbot f7000992ca fix(rf-health): auto-scale airtime Y-axis + hover tooltips (#600) (#623)
## Summary

Addresses user feedback on #600 — two improvements to RF Health detail
panel charts:

### 1. Auto-scale airtime Y-axis
Previously fixed 0-100% which made low-activity nodes unreadable (e.g.
0.1% TX barely visible). Now auto-scales to the actual data range with
20% headroom (minimum 1%), matching how the noise floor chart already
works.

### 2. Hover tooltips on all chart data points
Invisible SVG `<circle>` elements with native `<title>` tooltips on
every data point across all 4 charts:
- **Noise floor**: `NF: -112.3 dBm` + UTC timestamp
- **Airtime**: `TX: 2.1%` or `RX: 8.3%` + UTC timestamp  
- **Error rate**: `Err: 0.05%` + UTC timestamp
- **Battery**: `Batt: 3.85V` + UTC timestamp

Uses native browser SVG tooltips — zero dependencies, accessible, no JS
event handlers.

### Design rationale (Tufte)
- Auto-scaling increases data-ink ratio by eliminating wasted vertical
space
- Tooltips provide detail-on-demand without cluttering the chart with
labels on every point

### Spec update
Added M2 feedback improvements section to
`docs/specs/rf-health-dashboard.md`.

---------

Co-authored-by: you <you@example.com>
2026-04-05 13:08:05 -07:00
Kpa-clawbot 30e7e9ae3c docs: document lock ordering for cacheMu and channelsCacheMu (#624)
## Summary

Documents the lock ordering for all five mutexes in `PacketStore`
(`store.go`) to prevent future deadlocks.

## What changed

Added a comment block above the `PacketStore` struct documenting:

- All 5 mutexes (`mu`, `cacheMu`, `channelsCacheMu`, `groupedCacheMu`,
`regionObsMu`)
- What each mutex guards
- The required acquisition order (numbered 1–5)
- The nesting relationships that exist today (`cacheMu →
channelsCacheMu` in `invalidateCachesFor` and `rebuildAnalyticsCaches`)
- Confirmation that no reverse ordering exists (no deadlock risk)

## Verification

- Grepped all lock acquisition sites to confirm no reverse nesting
exists
- `go build ./...` passes — documentation-only change

Fixes #413

---------

Co-authored-by: you <you@example.com>
2026-04-05 13:00:35 -07:00
Kpa-clawbot 3415d3babb fix: measure VSCROLL_ROW_HEIGHT and theadHeight dynamically (#625)
## Summary

Replaces hardcoded `VSCROLL_ROW_HEIGHT = 36` and `theadHeight = 40` in
the virtual scroll logic with dynamic DOM measurement, so the values
stay correct if CSS changes.

## Changes

- `VSCROLL_ROW_HEIGHT`: measured once from the first rendered data row's
`offsetHeight` after the initial full rebuild. Falls back to 36px until
measurement occurs.
- `theadHeight`: measured from the actual `<thead>` element's
`offsetHeight` on every `renderVisibleRows` call. Falls back to 40px if
no thead is found.
- Both variables are now `let` instead of `const` to allow runtime
updates.

## Performance

No performance impact — both measurements are single `offsetHeight`
reads (no reflow triggered since the DOM was just written). Row height
measurement runs only once (guarded by `_vscrollRowHeightMeasured`
flag). Thead measurement is a single property read per scroll event.

Fixes #407

Co-authored-by: you <you@example.com>
2026-04-05 13:00:20 -07:00
Kpa-clawbot 05fbcb09dd fix: wire cacheTTL.analyticsHashSizes config to collision cache (#420) (#622)
## Summary

Fixes #420 — wires `cacheTTL` config values to server-side cache
durations that were previously hardcoded.

## Problem

`collisionCacheTTL` was hardcoded at 60s in `store.go`. The config has
`cacheTTL.analyticsHashSizes: 3600` (1 hour) but it was never read — the
`/api/config/cache` endpoint just passed the raw map to the client
without applying values server-side.

## Changes

- **`store.go`**: Add `cacheTTLSec()` helper to safely extract duration
values from the `cacheTTL` config map. `NewPacketStore` now accepts an
optional `cacheTTL` map (variadic, backward-compatible) and wires:
  - `cacheTTL.analyticsHashSizes` → `collisionCacheTTL`
  - `cacheTTL.analyticsRF` → `rfCacheTTL`
- **Default changed**: `collisionCacheTTL` default raised from 60s →
3600s (1 hour). Hash collision computation is expensive and data changes
rarely — 60s was causing unnecessary recomputation.
- **`main.go`**: Pass `cfg.CacheTTL` to `NewPacketStore`.
- **Tests**: Added `TestCacheTTLFromConfig` and `TestCacheTTLDefaults`
in eviction_test.go. Updated existing `TestHashCollisionsCacheTTL` for
the new default.

## Audit of other cacheTTL values

The remaining `cacheTTL` keys (`stats`, `nodeDetail`, `nodeHealth`,
`nodeList`, `bulkHealth`, `networkStatus`, `observers`, `channels`,
`channelMessages`, `analyticsTopology`, `analyticsChannels`,
`analyticsSubpaths`, `analyticsSubpathDetail`, `nodeAnalytics`,
`nodeSearch`, `invalidationDebounce`) are **client-side only** — served
via `/api/config/cache` and consumed by the frontend. They don't have
corresponding server-side caches to wire to. The only server-side caches
(`rfCache`, `topoCache`, `hashCache`, `chanCache`, `distCache`,
`subpathCache`, `collisionCache`) all use either `rfCacheTTL` or
`collisionCacheTTL`, both now configurable.

## Complexity

O(1) config lookup at store init time. No hot-path impact.

Co-authored-by: you <you@example.com>
2026-04-05 12:49:46 -07:00
efiten b587f20d1c feat: add distance column to neighbor table in node details (#617)
Closes #616

## What

Adds a **Distance** column to the neighbor table on the node detail
page.

When both the viewed node and a neighbor have GPS coordinates recorded,
the table shows the haversine distance between them (e.g. `3.2 km`).
When either node lacks GPS, the cell shows `—`.

## Changes

**Backend** (`cmd/server/neighbor_api.go`):
- Added `distance_km *float64` (omitempty) to `NeighborEntry`
- In `handleNodeNeighbors`: look up source node coords from `nodeMap`,
then for each resolved (non-ambiguous) neighbor with GPS, compute
`haversineKm` and set the field

**Frontend** (`public/nodes.js`):
- Added `Distance` column header between Last Seen and Conf
- Cell renders `X.X km` or `—` (muted) when unavailable

**Tests** (`cmd/server/neighbor_api_test.go`):
- `TestNeighborAPI_DistanceKm_WithGPS`: two nodes with real coords →
`distance_km` is positive
- `TestNeighborAPI_DistanceKm_NoGPS`: two nodes at 0,0 → `distance_km`
is nil

## Verification

Test at **https://staging.on8ar.eu** — navigate to any node detail page
and scroll to the Neighbors section. Nodes with GPS coordinates show a
distance; those without show `—`.
2026-04-05 12:33:23 -07:00
you af9754dbea ci: move staging build+deploy to meshcore-runner-2
Prod VM (meshcore-vm) is now prod-only. Staging builds and
deploys on the secondary runner.
2026-04-05 17:33:15 +00:00
Kpa-clawbot 767c8a5a3e perf: async chunked backfill — HTTP serves within 2 minutes (#612) (#614)
## Summary

Adds two config knobs for controlling backfill scope and neighbor graph
data retention, plus removes the dead synchronous backfill function.

## Changes

### Config knobs

#### `resolvedPath.backfillHours` (default: 24)
Controls how far back (in hours) the async backfill scans for
observations with NULL `resolved_path`. Transmissions with `first_seen`
older than this window are skipped, reducing startup time for instances
with large historical datasets.

#### `neighborGraph.maxAgeDays` (default: 30)
Controls the maximum age of `neighbor_edges` entries. Edges with
`last_seen` older than this are pruned from both SQLite and the
in-memory graph. Pruning runs on startup (after a 4-minute stagger) and
every 24 hours thereafter.

### Dead code removal
- Removed the synchronous `backfillResolvedPaths` function that was
replaced by the async version.

### Implementation details
- `backfillResolvedPathsAsync` now accepts a `backfillHours` parameter
and filters by `tx.FirstSeen`
- `NeighborGraph.PruneOlderThan(cutoff)` removes stale edges from the
in-memory graph
- `PruneNeighborEdges(conn, graph, maxAgeDays)` prunes both DB and
in-memory graph
- Periodic pruning ticker follows the same pattern as metrics pruning
(24h interval, staggered start)
- Graceful shutdown stops the edge prune ticker

### Config example
Both knobs added to `config.example.json` with `_comment` fields.

## Tests
- Config default/override tests for both knobs
- `TestGraphPruneOlderThan` — in-memory edge pruning
- `TestPruneNeighborEdgesDB` — SQLite + in-memory pruning together
- `TestBackfillRespectsHourWindow` — verifies old transmissions are
excluded by backfill window

---------

Co-authored-by: you <you@example.com>
2026-04-05 09:49:39 -07:00
Kpa-clawbot 382b3505dc feat: channel color quick-assign UI (M2, #271) (#611)
## Summary

Implements M2 of channel color highlighting (#271): a right-click
context menu popover for quick-assigning colors to hash channels.

Builds on M1 (PR #607) which provides `ChannelColors.set/get/remove`
storage primitives.

## What's new

### Color picker popover (`channel-color-picker.js`)
- **Right-click** any GRP_TXT/CHAN row in the **live feed** or **packets
table** → opens a color picker popover at the click point
- **Long-press** (500ms) on mobile triggers the same popover
- **10 preset swatches** — maximally distinct, ColorBrewer-inspired
palette
- **Custom hex** — native `<input type="color">` with Apply button
- **Clear button** — removes color assignment (hidden when no color
assigned)
- **Popover positioning** — auto-adjusts to avoid viewport overflow
- **Dismiss** — click outside or Escape key

### Immediate feedback
- Assigning a color instantly re-styles all visible live feed items with
that channel
- Packets table triggers `renderVisibleRows()` via exposed
`window._packetsRenderVisible`

### Wiring
- Feed items store `_ccPkt` packet reference for channel extraction
- Picker installed via `registerPage` init hooks in both `live.js` and
`packets.js`
- Single shared popover DOM element, repositioned on each open

### Styling
- Dark card with border, matching existing CoreScope dropdown patterns
- CSS in `style.css` under `.cc-picker-*` classes
- Uses CSS variables (`--surface-1`, `--border`, `--accent`, etc.) for
theme compatibility

## Files changed

| File | Change |
|------|--------|
| `public/channel-color-picker.js` | New — popover component (IIFE, no
dependencies except `ChannelColors`) |
| `public/index.html` | Script tag for picker |
| `public/live.js` | Store `_ccPkt` on feed items, install picker on
init |
| `public/packets.js` | Install picker on init, expose
`_packetsRenderVisible` |
| `public/style.css` | Popover CSS |
| `test-channel-colors.js` | 2 new tests for picker loading and graceful
degradation |

## Testing

- All 21 channel-colors tests pass (19 M1 + 2 M2)
- All 445 frontend-helpers tests pass
- All 62 packet-filter tests pass

## Performance

No hot-path impact. The popover is a single shared DOM element created
lazily on first use. Context menu handlers use event delegation on the
feed/table containers (one listener each, not per-row). The
`refreshVisibleRows` function only iterates currently-visible DOM
elements.

Closes milestone M2 of #271.

---------

Co-authored-by: you <you@example.com>
2026-04-05 06:45:13 -07:00
you dc635775b5 docs: TUI spec updated with expert feedback + MVP definition 2026-04-05 07:12:11 +00:00
you 8a94c43334 docs: startup performance spec — serve HTTP within 2 minutes on any DB size 2026-04-05 07:09:55 +00:00
you 6aaa5cdc20 docs: add user guide — getting started, pages, config, FAQ 2026-04-05 07:09:54 +00:00
you 788005bff7 docs: clarify Docker tag strategy — pin to vX.Y.Z for production, edge for testing 2026-04-05 07:09:44 +00:00
you af03f9aa57 docs: deployment simplification spec — pre-built Docker images + one-line deploy 2026-04-05 07:06:35 +00:00
Kpa-clawbot 3328ca4354 feat: channel color highlighting M1 — core model + feed row (#271) (#607)
## Summary

Implements M1 of the [channel color highlighting
spec](docs/specs/channel-color-highlighting.md) for issue #271.

Allows users to assign custom highlight colors to specific hash
channels. When a `GRP_TXT` packet arrives with an assigned channel
color, the feed row and packets table row get:
- **4px colored left border** in the assigned color
- **Subtle background tint** (color at 10% opacity)

## What's included

### `public/channel-colors.js` — Storage model
- `ChannelColors.get(channel)` → hex color or null
- `ChannelColors.set(channel, color)` — assign a color
- `ChannelColors.remove(channel)` — clear assignment
- `ChannelColors.getAll()` → all assignments
- `ChannelColors.getRowStyle(typeName, channel)` → inline CSS string for
row highlighting
- Uses `localStorage` key `live-channel-colors`
- Gracefully handles corrupt/missing localStorage data

### Feed row highlighting (`public/live.js`)
- Both `addFeedItem` (live WS) and `addFeedItemDOM` (replay/DB load)
apply channel color styles
- Reads `decoded.payload.channelName` from the packet

### Packets table highlighting (`public/packets.js`)
- `buildFlatRowHtml` and `buildGroupRowHtml` apply channel color styles
to `<tr>` elements
- Reads channel from `getParsedDecoded(p).channel`

### Tests (`test-channel-colors.js`)
- 16 unit tests covering storage CRUD, edge cases (null, empty, corrupt
data), and style generation
- Tests verify only GRP_TXT/CHAN types get coloring, other types are
unaffected

## Design decisions

- **Only GRP_TXT/CHAN packets** — other types retain default
`TYPE_COLORS` styling
- **Channel color takes priority** over default type colors for row
highlighting
- **No UI for assigning colors yet** — that's M2 (right-click context
menu + color picker)
- **Storage key abstracted** behind functions to ease future migration
if customizer rework (#288) lands
- **10% opacity tint** (`#hexcolor` + `1a` suffix) ensures readability
in both dark/light modes

## Performance

- `getRowStyle()` is O(1) — single localStorage read + JSON parse per
call
- No per-packet API calls; all data is client-side
- No impact on hot rendering paths beyond one localStorage read per row
render

Closes #271 (M1 only — further milestones in separate PRs)

---------

Co-authored-by: you <you@example.com>
2026-04-05 00:03:17 -07:00
you 14732135b7 docs: proposal for terminal/TUI interface into CoreScope 2026-04-05 06:56:33 +00:00
Kpa-clawbot e42477b810 feat: collapsible panels + medium breakpoint on live map (#606)
## Summary

Adds collapsible/minimizable UI panels on the live map page so overlay
panels don't block map content on medium-sized screens.

Fixes #279

## Changes

### Collapsible Legend Panel (all screen sizes)
- The legend toggle button (🎨/✕) is now visible at **all** screen sizes,
not just mobile
- Clicking it smoothly collapses/expands the legend with a CSS
transition
- Collapsed state persists in `localStorage` (`live-legend-hidden`)
- Feed panel already had hide/show with localStorage — no changes needed
there

### Medium Breakpoint (768px)
New `@media (max-width: 768px)` rules for tablet/small laptop screens:
- Feed panel: 360px → 280px wide, max-height 340px → 200px
- Node detail panel: 320px → 260px wide
- Legend: smaller font (10px) and tighter padding
- Header: reduced gap and padding
- Stats/toggles: smaller font sizes

### What's NOT changed
- Mobile (≤640px): existing behavior preserved (feed/legend hidden
entirely)
- Desktop (>768px): no changes — panels render at full size as before

## Testing
- `test-packet-filter.js`: 62 passed
- `test-aging.js`: 29 passed  
- `test-frontend-helpers.js`: 445 passed

---------

Co-authored-by: you <you@example.com>
2026-04-04 23:56:07 -07:00
you cbc3e3ce13 docs: movable UI panels spec — draggable panel positioning (#279) 2026-04-05 06:54:45 +00:00
you 1796493ec0 docs: channel color highlighting spec (#271)
Custom color assignment for hash channels in Live tab.
Reviewed by Tufte, Torvalds, and Doshi personas.
2026-04-05 06:45:53 +00:00
you 168866ecb6 fix: View Route on Map button works on packet detail page
The button click handler used document.getElementById() which fails on
/packet/[ID] pages because renderDetail() runs before the container is
appended to the DOM. Changed to panel.querySelector() which searches
within the detached element tree.

Fixes #601
2026-04-05 06:43:59 +00:00
you be9257cd26 chore: switch license to GPL v3
Copyleft ensures all derivative works remain open source.
2026-04-05 06:36:03 +00:00
you b5b6faf90a chore: switch license from MIT to Apache 2.0
Adds patent protection for contributors while maintaining the same
permissive usage rights.
2026-04-05 06:35:38 +00:00
you 592061ec7e chore: add MIT license 2026-04-05 06:32:28 +00:00
you 596ccf2322 fix(rf-health): offset TX/RX airtime labels when overlapping
When TX and RX values are within 12px, TX label shifts up and RX shifts
down to avoid rendering on top of each other.
2026-04-05 06:31:02 +00:00
Kpa-clawbot 232770a858 feat(rf-health): M2 — airtime, error rate, battery charts with delta computation (#605)
## M2: Airtime + Channel Quality + Battery Charts

Implements M2 of #600 — server-side delta computation and three new
charts in the RF Health detail view.

### Backend Changes

**Delta computation** for cumulative counters (`tx_air_secs`,
`rx_air_secs`, `recv_errors`):
- Computes per-interval deltas between consecutive samples
- **Reboot handling:** detects counter reset (current < previous), skips
that delta, records reboot timestamp
- **Gap handling:** if time between samples > 2× interval, inserts null
(no interpolation)
- Returns `tx_airtime_pct` and `rx_airtime_pct` as percentages
(delta_secs / interval_secs × 100)
- Returns `recv_error_rate` as delta_errors / (delta_recv +
delta_errors) × 100

**`resolution` query param** on `/api/observers/{id}/metrics`:
- `5m` (default) — raw samples
- `1h` — hourly aggregates (GROUP BY hour with AVG/MAX)
- `1d` — daily aggregates

**Schema additions:**
- `packets_sent` and `packets_recv` columns added to `observer_metrics`
(migration)
- Ingestor parses these fields from MQTT stats messages

**API response** now includes:
- `tx_airtime_pct`, `rx_airtime_pct`, `recv_error_rate` (computed
deltas)
- `reboots` array with timestamps of detected reboots
- `is_reboot_sample` flag on affected samples

### Frontend Changes

Three new charts in the RF Health detail view, stacked vertically below
noise floor:

1. **Airtime chart** — TX (red) + RX (blue) as separate SVG lines,
Y-axis 0-100%, direct labels at endpoints
2. **Error Rate chart** — `recv_error_rate` line, shown only when data
exists
3. **Battery chart** — voltage line with 3.3V low reference, shown only
when battery_mv > 0

All charts:
- Share X-axis and time range (aligned vertically)
- Reboot markers as vertical hairlines spanning all charts
- Direct labels on data (no legends)
- Resolution auto-selected: `1h` for 7d/30d ranges
- Charts hidden when no data exists

### Tests

- `TestComputeDeltas`: normal deltas, reboot detection, gap detection
- `TestGetObserverMetricsResolution`: 5m/1h/1d downsampling verification
- Updated `TestGetObserverMetrics` for new API signature

---------

Co-authored-by: you <you@example.com>
2026-04-04 23:17:17 -07:00
you 747aea37b7 fix(rf-health): add region filter support to metrics summary
Frontend passes RegionFilter query string to summary API.
Backend filters results by observer IATA region.
Added iata field to MetricsSummaryRow.
2026-04-05 06:00:42 +00:00
you 968c104e14 feat(rf-health): show observer detail in side panel instead of page bottom
- Change RF Health detail view from bottom-of-page to a right-sliding side panel
- Grid stays visible and stable when detail is open (no layout shift)
- Click another observer updates panel in place; close button (×) dismisses
- On mobile (<640px): panel stacks below grid at full width
- Filter out observers with insufficient data (<2 sparkline points) from grid entirely
- Follows the same split-layout pattern used by the nodes page
2026-04-05 05:53:42 +00:00
Kpa-clawbot 6f35d4d417 feat: RF Health Dashboard M1 — observer metrics + small multiples grid (#604)
## RF Health Dashboard — M1: Observer Metrics Storage, API & Small
Multiples Grid

Implements M1 of #600.

### What this does

Adds a complete RF health monitoring pipeline: MQTT stats ingestion →
SQLite storage → REST API → interactive dashboard with small multiples
grid.

### Backend Changes

**Ingestor (`cmd/ingestor/`)**
- New `observer_metrics` table via migration system (`_migrations`
pattern)
- Parse `tx_air_secs`, `rx_air_secs`, `recv_errors` from MQTT status
messages (same pattern as existing `noise_floor` and `battery_mv`)
- `INSERT OR REPLACE` with timestamps rounded to nearest 5-min interval
boundary (using ingestor wall clock, not observer timestamps)
- Missing fields stored as NULLs — partial data is always better than no
data
- Configurable retention pruning: `retention.metricsDays` (default 30),
runs on startup + every 24h

**Server (`cmd/server/`)**
- `GET /api/observers/{id}/metrics?since=...&until=...` — per-observer
time-series data
- `GET /api/observers/metrics/summary?window=24h` — fleet summary with
current NF, avg/max NF, sample count
- `parseWindowDuration()` supports `1h`, `24h`, `3d`, `7d`, `30d` etc.
- Server-side metrics retention pruning (same config, staggered 2min
after packet prune)

### Frontend Changes

**RF Health tab (`public/analytics.js`, `public/style.css`)**
- Small multiples grid showing all observers simultaneously — anomalies
pop out visually
- Per-observer cell: name, current NF value, battery voltage, sparkline,
avg/max stats
- NF status coloring: warning (amber) at ≥-100 dBm, critical (red) at
≥-85 dBm — text color only, no background fills
- Click any cell → expanded detail view with full noise floor line chart
- Reference lines with direct text labels (`-100 warning`, `-85
critical`) — not color bands
- Min/max points labeled directly on the chart
- Time range selector: preset buttons (1h/3h/6h/12h/24h/3d/7d/30d) +
custom from/to datetime picker
- Deep linking: `#/analytics?tab=rf-health&observer=...&range=...`
- All charts use SVG, matching existing analytics.js patterns
- Responsive: 3-4 columns on desktop, 1 on mobile

### Design Decisions (from spec)
- Labels directly on data, not in legends
- Reference lines with text labels, not color bands
- Small multiples grid, not card+accordion (Tufte: instant visual fleet
comparison)
- Ingestor wall clock for all timestamps (observer clocks may drift)

### Tests Added

**Ingestor tests:**
- `TestRoundToInterval` — 5 cases for rounding to 5-min boundaries
- `TestInsertMetrics` — basic insertion with all fields
- `TestInsertMetricsIdempotent` — INSERT OR REPLACE deduplication
- `TestInsertMetricsNullFields` — partial data with NULLs
- `TestPruneOldMetrics` — retention pruning
- `TestExtractObserverMetaNewFields` — parsing tx_air_secs, rx_air_secs,
recv_errors

**Server tests:**
- `TestGetObserverMetrics` — time-series query with since/until filters,
NULL handling
- `TestGetMetricsSummary` — fleet summary aggregation
- `TestObserverMetricsAPIEndpoints` — DB query verification
- `TestMetricsAPIEndpoints` — HTTP endpoint response shape
- `TestParseWindowDuration` — duration parsing for h/d formats

### Test Results
```
cd cmd/ingestor && go test ./... → PASS (26s)
cd cmd/server && go test ./... → PASS (5s)
```

### What's NOT in this PR (deferred to M2+)
- Server-side delta computation for cumulative counters
- Airtime charts (TX/RX percentage lines)
- Channel quality chart (recv_error_rate)
- Battery voltage chart
- Reboot detection and chart annotations
- Resolution downsampling (1h, 1d aggregates)
- Pattern detection / automated diagnosis

---------

Co-authored-by: you <you@example.com>
2026-04-04 22:21:35 -07:00
you aaf00d0616 docs: add M5 Prometheus/Grafana metrics export to RF Health spec 2026-04-05 05:02:36 +00:00
you 41c046c974 docs: RF Health Dashboard spec — observer radio metrics
Per-observer time-series charts for noise floor, TX/RX airtime, CRC errors,
and battery. Small multiples grid design. MVP-first milestones.

Reviewed by Carmack (perf), Munger (failure modes), radio expert (hardware),
Tufte (visualization), and Doshi (product strategy).
2026-04-05 04:42:32 +00:00
efiten 1fbdd1c3d3 feat: Prefix Tool tab on Analytics page (#347) (#599)
## Summary

- Adds a new **Prefix Tool** tab to the Analytics page (alongside Hash
Stats / Hash Issues)
- **Network Overview**: per-tier collision stats (1/2/3-byte) and a
network-size-based recommendation — collapsible, folded by default
- **Prefix Checker**: accepts a 1/2/3-byte hex prefix or full public
key; shows colliding nodes at each tier with severity badges ( / ⚠️ /
🔴); clicking a node navigates to its detail page
- **Prefix Generator**: picks a random collision-free prefix at the
chosen hash size; links to
[meshcore-web-keygen](https://agessaman.github.io/meshcore-web-keygen/)
with the prefix pre-filled
- **Hash Issues tab**: adds a "🔎 Check a prefix →" shortcut in the nav
- **Deep-link support**: `#/analytics?tab=prefix-tool&prefix=A3F1`
pre-fills and runs the checker; `?generate=2` pre-selects and runs the
generator
- **No new API endpoints** — 100% client-side using the existing
`/nodes` list

## Verification

Live on staging:
**https://staging.on8ar.eu/#/analytics?tab=prefix-tool**

## Test plan

- [x] Network Overview card is collapsed by default; expands on click;
stats are correct
- [x] Prefix Checker: 2-char input shows 1-byte results; 4-char shows
2-byte; 6-char shows 3-byte; 64-char pubkey shows all three tiers
- [x] Prefix Checker: invalid hex shows error; odd-length input shows
error
- [x] Prefix Generator: Generate picks an unused prefix; "Try another"
cycles; keygen link opens with prefix pre-filled
- [x] Deep link `?prefix=A3F1` pre-fills checker and scrolls to it
- [x] Deep link `?generate=2` pre-selects 2-byte and runs generator
- [x] Hash Issues tab shows "🔎 Check a prefix →" in the nav
- [x] FAQ link at bottom of generator opens correct MeshCore docs anchor

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 20:18:32 -07:00
efiten d34320fa6c fix: use _getColCount() in error-state row to match spacers (#406) (#597)
## Summary

The error-state `<tbody>` row (shown when packet loading fails)
hardcoded `colspan="10"`, while the virtual scroll spacers and the
empty-state row both use `_getColCount()` (which reads from the actual
`<thead>` and falls back to 11). One-line fix: replace the hardcoded
value with `_getColCount()`.

Fixes #406

## Test plan

- [x] Trigger the error state (e.g. kill the backend mid-load) — error
row should span all columns with no gap on the right
- [x] `node test-packets.js` — 72 passed, 0 failed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 19:41:55 -07:00
efiten 77b7c33d0f perf: incremental DOM diff in renderVisibleRows (#414) (#596)
## Summary

- Replace full \`tbody\` teardown+rebuild on every scroll frame with a
range-diff that only adds/removes the delta rows at the edges of the
visible window
- \`buildFlatRowHtml\` / \`buildGroupRowHtml\` now accept an
\`entryIdx\` parameter and emit \`data-entry-idx\` on every \`<tr>\` so
the diff can target rows precisely (including expanded group children)
- Full rebuild is retained for initial render and large scroll jumps
past the buffer (no range overlap)
- Also loads \`packet-helpers.js\` in the test sandbox, fixing 7
pre-existing test failures for the builder functions; adds 4 new tests
covering \`data-entry-idx\` output

Fixes #414

## Test plan

- [x] Open packets page with 500+ packets, scroll rapidly — DOM
inspector should show incremental \`<tr>\` adds/removes rather than full
\`tbody\` teardown
- [x] Expand a grouped packet, scroll away and back — expanded children
re-render correctly
- [x] Large scroll jump (jump to bottom via scrollbar) — full rebuild
fires, no visual glitch
- [x] \`node test-packets.js\` — 72 passed, 0 failed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
2026-04-04 19:41:33 -07:00
89 changed files with 13714 additions and 588 deletions
+82 -20
View File
@@ -3,10 +3,15 @@ name: CI/CD Pipeline
on:
push:
branches: [master]
tags: ['v*']
pull_request:
branches: [master]
workflow_dispatch:
permissions:
contents: read
packages: write
concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
@@ -18,8 +23,8 @@ env:
STAGING_CONTAINER: corescope-staging-go
# Pipeline (sequential, fail-fast):
# go-test → e2e-test → build → deploy → publish
# PRs stop after build. Master continues to deploy + publish.
# go-test → e2e-test → build-and-publish → deploy → publish-badges
# PRs stop after build-and-publish (no GHCR push). Master continues to deploy + badges.
jobs:
# ───────────────────────────────────────────────────────────────
@@ -231,51 +236,108 @@ jobs:
include-hidden-files: true
# ───────────────────────────────────────────────────────────────
# 3. Build Docker Image
# 3. Build & Publish Docker Image
# ───────────────────────────────────────────────────────────────
build:
name: "🏗️ Build Docker Image"
build-and-publish:
name: "🏗️ Build & Publish Docker Image"
needs: [e2e-test]
runs-on: [self-hosted, meshcore-vm]
runs-on: [self-hosted, meshcore-runner-2]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Set up Node.js 22
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Free disk space
run: |
docker system prune -af 2>/dev/null || true
docker builder prune -af 2>/dev/null || true
df -h /
- name: Build Go Docker image
- name: Compute build metadata
id: meta
run: |
echo "${GITHUB_SHA::7}" > .git-commit
APP_VERSION=$(node -p "require('./package.json').version") \
GIT_COMMIT="${GITHUB_SHA::7}" \
APP_VERSION=$(grep -oP 'APP_VERSION:-\K[^}]+' docker-compose.yml | head -1 || echo "3.0.0")
GIT_COMMIT=$(git rev-parse --short HEAD)
BUILD_TIME=$(date -u '+%Y-%m-%dT%H:%M:%SZ')
export APP_VERSION GIT_COMMIT BUILD_TIME
GIT_COMMIT="${GITHUB_SHA::7}"
if [[ "$GITHUB_REF" == refs/tags/v* ]]; then
APP_VERSION="${GITHUB_REF#refs/tags/}"
else
APP_VERSION="edge"
fi
echo "build_time=$BUILD_TIME" >> "$GITHUB_OUTPUT"
echo "git_commit=$GIT_COMMIT" >> "$GITHUB_OUTPUT"
echo "app_version=$APP_VERSION" >> "$GITHUB_OUTPUT"
echo "Build: version=$APP_VERSION commit=$GIT_COMMIT time=$BUILD_TIME"
- name: Build Go Docker image (local staging)
run: |
GIT_COMMIT="${{ steps.meta.outputs.git_commit }}" \
APP_VERSION="${{ steps.meta.outputs.app_version }}" \
BUILD_TIME="${{ steps.meta.outputs.build_time }}" \
docker compose -f "$STAGING_COMPOSE_FILE" -p corescope-staging build "$STAGING_SERVICE"
echo "Built Go staging image ✅"
- name: Set up Docker Buildx
if: github.event_name == 'push'
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract Docker metadata
if: github.event_name == 'push'
id: docker-meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/kpa-clawbot/corescope
tags: |
type=semver,pattern=v{{version}}
type=semver,pattern=v{{major}}.{{minor}}
type=semver,pattern=v{{major}}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=edge,branch=master
- name: Build and push to GHCR
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: .
push: true
platforms: linux/amd64
tags: ${{ steps.docker-meta.outputs.tags }}
labels: ${{ steps.docker-meta.outputs.labels }}
build-args: |
APP_VERSION=${{ steps.meta.outputs.app_version }}
GIT_COMMIT=${{ steps.meta.outputs.git_commit }}
BUILD_TIME=${{ steps.meta.outputs.build_time }}
cache-from: type=gha
cache-to: type=gha,mode=max
# ───────────────────────────────────────────────────────────────
# 4. Deploy Staging (master only)
# ───────────────────────────────────────────────────────────────
deploy:
name: "🚀 Deploy Staging"
if: github.event_name == 'push'
needs: [build]
runs-on: [self-hosted, meshcore-vm]
needs: [build-and-publish]
runs-on: [self-hosted, meshcore-runner-2]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Pull latest image from GHCR
run: |
# Try to pull the edge image from GHCR and tag for docker-compose compatibility
if docker pull ghcr.io/kpa-clawbot/corescope:edge; then
docker tag ghcr.io/kpa-clawbot/corescope:edge corescope-go:latest
echo "Pulled and tagged GHCR edge image ✅"
else
echo "⚠️ GHCR pull failed — falling back to locally built image"
fi
- name: Deploy staging
run: |
# Stop old container and release memory
+131
View File
@@ -0,0 +1,131 @@
# Deploy CoreScope
Pre-built images are published to GHCR for `linux/amd64` and `linux/arm64` (Raspberry Pi 4/5).
## Quick Start
### Docker run
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
-e DISABLE_CADDY=true \
ghcr.io/kpa-clawbot/corescope:latest
```
Open `http://localhost` — done.
### Docker Compose
```bash
curl -sL https://raw.githubusercontent.com/Kpa-clawbot/CoreScope/master/docker-compose.example.yml \
-o docker-compose.yml
docker compose up -d
```
## Image Tags
| Tag | Description |
|-----|-------------|
| `v3.4.1` | Pinned release (recommended for production) |
| `v3.4` | Latest patch in v3.4.x |
| `v3` | Latest minor+patch in v3.x |
| `latest` | Latest release tag |
| `edge` | Built from master — unstable, for testing |
## Configuration
Settings can be overridden via environment variables:
| Variable | Default | Description |
|----------|---------|-------------|
| `DISABLE_CADDY` | `false` | Skip internal Caddy (set `true` behind a reverse proxy) |
| `DISABLE_MOSQUITTO` | `false` | Skip internal MQTT broker (use external) |
| `HTTP_PORT` | `80` | Host port mapping |
| `DATA_DIR` | `./data` | Host path for persistent data |
For advanced configuration, mount a `config.json` into `/app/data/config.json`. See `config.example.json` in the repo.
## Updating
```bash
docker compose pull
docker compose up -d
```
## Data
All persistent data lives in `/app/data`:
- `meshcore.db` — SQLite database (packets, nodes)
- `config.json` — custom config (optional)
- `theme.json` — custom theme (optional)
**Backup:** `cp data/meshcore.db ~/backup/`
## TLS
Option A — **External reverse proxy** (recommended): Run with `DISABLE_CADDY=true`, put nginx/traefik/Cloudflare in front.
Option B — **Built-in Caddy**: Mount a custom Caddyfile at `/etc/caddy/Caddyfile` and expose ports 80+443.
---
## Migrating from manage.sh (existing admins)
If you're currently deploying with `manage.sh` (git clone + local build), you have two options going forward:
### Option A: Keep using manage.sh (no changes needed)
`manage.sh update` continues to work exactly as before — it fetches the latest tag, builds locally, and restarts. Nothing breaks.
```bash
./manage.sh update # latest release
./manage.sh update v3.5.0 # specific version
```
### Option B: Switch to pre-built images (recommended)
Pre-built images skip the build step entirely — faster updates, no Go toolchain needed.
**One-time migration:**
1. Stop the current deployment:
```bash
./manage.sh stop
```
2. Your data is in `~/meshcore-data/` (or whatever `PROD_DATA_DIR` is set to). It's untouched — the database, config, and theme files persist.
3. Copy `docker-compose.example.yml` to where you want to run from:
```bash
cp docker-compose.example.yml ~/docker-compose.yml
```
4. Start with the pre-built image:
```bash
cd ~ && docker compose up -d
```
5. Verify it picked up your existing data:
```bash
curl http://localhost/api/stats
```
**Updates after migration:**
```bash
docker compose pull && docker compose up -d
```
### What about manage.sh features?
| manage.sh command | Pre-built equivalent |
|---|---|
| `./manage.sh update` | `docker compose pull && docker compose up -d` |
| `./manage.sh stop` | `docker compose down` |
| `./manage.sh start` | `docker compose up -d` |
| `./manage.sh logs` | `docker compose logs -f` |
| `./manage.sh status` | `docker compose ps` |
| `./manage.sh setup` | Copy `docker-compose.example.yml`, edit env vars |
`manage.sh` remains available for advanced use cases (building from source, custom patches, development). Pre-built images are recommended for most production deployments.
+2
View File
@@ -42,6 +42,8 @@ RUN echo "unknown" > .git-commit
# Supervisor + Mosquitto + Caddy config
COPY docker/supervisord-go.conf /etc/supervisor/conf.d/supervisord.conf
COPY docker/supervisord-go-no-mosquitto.conf /etc/supervisor/conf.d/supervisord-no-mosquitto.conf
COPY docker/supervisord-go-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-caddy.conf
COPY docker/supervisord-go-no-mosquitto-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-mosquitto-no-caddy.conf
COPY docker/mosquitto.conf /etc/mosquitto/mosquitto.conf
COPY docker/Caddyfile /etc/caddy/Caddyfile
+3
View File
@@ -40,6 +40,9 @@ RUN if [ ! -f .git-commit ]; then echo "unknown" > .git-commit; fi
# Supervisor + Mosquitto + Caddy config
COPY docker/supervisord-go.conf /etc/supervisor/conf.d/supervisord.conf
COPY docker/supervisord-go-no-mosquitto.conf /etc/supervisor/conf.d/supervisord-no-mosquitto.conf
COPY docker/supervisord-go-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-caddy.conf
COPY docker/supervisord-go-no-mosquitto-no-caddy.conf /etc/supervisor/conf.d/supervisord-no-mosquitto-no-caddy.conf
COPY docker/mosquitto.conf /etc/mosquitto/mosquitto.conf
COPY docker/Caddyfile /etc/caddy/Caddyfile
+674
View File
@@ -0,0 +1,674 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
+18 -4
View File
@@ -74,9 +74,23 @@ Full experience on your phone — proper touch controls, iOS safe area support,
## Quick Start
### Docker (Recommended)
### Pre-built Image (Recommended)
No Go installation needed — everything builds inside the container.
No build step required — just run:
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
ghcr.io/kpa-clawbot/corescope:latest
```
Open `http://localhost` — done. No config file needed; CoreScope starts with sensible defaults.
See [DEPLOY.md](DEPLOY.md) for image tags, Docker Compose, and migration from `manage.sh`.
See [docs/deployment.md](docs/deployment.md) for the full deployment guide — MQTT setup, HTTPS options, backups, monitoring, and troubleshooting.
### Build from Source
```bash
git clone https://github.com/Kpa-clawbot/CoreScope.git
@@ -95,8 +109,6 @@ The setup wizard walks you through config, domain, HTTPS, build, and run.
./manage.sh help # All commands
```
See [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) for the full deployment guide — HTTPS options (auto cert, bring your own, Cloudflare Tunnel), MQTT security, backups, and troubleshooting.
### Configure
Copy `config.example.json` to `config.json` and edit:
@@ -242,6 +254,8 @@ Contributions welcome. Please read [AGENTS.md](AGENTS.md) for coding conventions
**Live instance:** [analyzer.00id.net](https://analyzer.00id.net) — all API endpoints are public, no auth required.
**API Documentation:** CoreScope auto-generates an OpenAPI 3.0 spec. Browse the interactive Swagger UI at [`/api/docs`](https://analyzer.00id.net/api/docs) or fetch the machine-readable spec at [`/api/spec`](https://analyzer.00id.net/api/spec).
## License
MIT
+48 -7
View File
@@ -2,7 +2,9 @@ package main
import (
"encoding/json"
"errors"
"fmt"
"log"
"os"
"strings"
@@ -36,6 +38,7 @@ type Config struct {
ChannelKeys map[string]string `json:"channelKeys,omitempty"`
HashChannels []string `json:"hashChannels,omitempty"`
Retention *RetentionConfig `json:"retention,omitempty"`
Metrics *MetricsConfig `json:"metrics,omitempty"`
GeoFilter *GeoFilterConfig `json:"geo_filter,omitempty"`
}
@@ -44,7 +47,29 @@ type GeoFilterConfig = geofilter.Config
// RetentionConfig controls how long stale nodes are kept before being moved to inactive_nodes.
type RetentionConfig struct {
NodeDays int `json:"nodeDays"`
NodeDays int `json:"nodeDays"`
MetricsDays int `json:"metricsDays"`
}
// MetricsConfig controls observer metrics collection.
type MetricsConfig struct {
SampleIntervalSec int `json:"sampleIntervalSec"`
}
// MetricsSampleInterval returns the configured sample interval or 300s default.
func (c *Config) MetricsSampleInterval() int {
if c.Metrics != nil && c.Metrics.SampleIntervalSec > 0 {
return c.Metrics.SampleIntervalSec
}
return 300
}
// MetricsRetentionDays returns configured metrics retention or 30 days default.
func (c *Config) MetricsRetentionDays() int {
if c.Retention != nil && c.Retention.MetricsDays > 0 {
return c.Retention.MetricsDays
}
return 30
}
// NodeDaysOrDefault returns the configured retention.nodeDays or 7 if not set.
@@ -56,15 +81,21 @@ func (c *Config) NodeDaysOrDefault() int {
}
// LoadConfig reads configuration from a JSON file, with env var overrides.
// If the config file does not exist, sensible defaults are used (zero-config startup).
func LoadConfig(path string) (*Config, error) {
var cfg Config
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("reading config %s: %w", path, err)
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parsing config %s: %w", path, err)
if !errors.Is(err, os.ErrNotExist) {
return nil, fmt.Errorf("reading config %s: %w", path, err)
}
// Config file doesn't exist — use defaults (zero-config mode)
log.Printf("config file %s not found, using sensible defaults", path)
} else {
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parsing config %s: %w", path, err)
}
}
// Env var overrides
@@ -98,6 +129,16 @@ func LoadConfig(path string) (*Config, error) {
}}
}
// Default MQTT source: connect to localhost broker when no sources configured
if len(cfg.MQTTSources) == 0 {
cfg.MQTTSources = []MQTTSource{{
Name: "local",
Broker: "mqtt://localhost:1883",
Topics: []string{"meshcore/#"},
}}
log.Printf("no MQTT sources configured, defaulting to mqtt://localhost:1883")
}
return &cfg, nil
}
+21 -5
View File
@@ -32,9 +32,25 @@ func TestLoadConfigValidJSON(t *testing.T) {
}
func TestLoadConfigMissingFile(t *testing.T) {
_, err := LoadConfig("/nonexistent/path/config.json")
if err == nil {
t.Error("expected error for missing file")
t.Setenv("DB_PATH", "")
t.Setenv("MQTT_BROKER", "")
cfg, err := LoadConfig("/nonexistent/path/config.json")
if err != nil {
t.Fatalf("missing config should not error (zero-config mode), got: %v", err)
}
if cfg.DBPath != "data/meshcore.db" {
t.Errorf("dbPath=%s, want data/meshcore.db", cfg.DBPath)
}
// Should default to localhost MQTT
if len(cfg.MQTTSources) != 1 {
t.Fatalf("mqttSources len=%d, want 1", len(cfg.MQTTSources))
}
if cfg.MQTTSources[0].Broker != "mqtt://localhost:1883" {
t.Errorf("default broker=%s, want mqtt://localhost:1883", cfg.MQTTSources[0].Broker)
}
if cfg.MQTTSources[0].Name != "local" {
t.Errorf("default source name=%s, want local", cfg.MQTTSources[0].Name)
}
}
@@ -196,8 +212,8 @@ func TestLoadConfigLegacyMQTTEmptyBroker(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if len(cfg.MQTTSources) != 0 {
t.Errorf("mqttSources should be empty when legacy broker is empty, got %d", len(cfg.MQTTSources))
if len(cfg.MQTTSources) != 1 || cfg.MQTTSources[0].Name != "local" {
t.Errorf("mqttSources should default to local broker when legacy broker is empty, got %v", cfg.MQTTSources)
}
}
+7 -7
View File
@@ -461,7 +461,7 @@ func TestDecodeAdvertLocationTruncated(t *testing.T) {
buf[100] = 0x11
// Only 4 bytes after flags — not enough for full location (needs 8)
p := decodeAdvert(buf[:105])
p := decodeAdvert(buf[:105], false)
if p.Error != "" {
t.Fatalf("error: %s", p.Error)
}
@@ -483,7 +483,7 @@ func TestDecodeAdvertFeat1Truncated(t *testing.T) {
buf[100] = 0x21
// Only 1 byte after flags — not enough for feat1 (needs 2)
p := decodeAdvert(buf[:102])
p := decodeAdvert(buf[:102], false)
if p.Feat1 != nil {
t.Error("feat1 should be nil with truncated data")
}
@@ -504,7 +504,7 @@ func TestDecodeAdvertFeat2Truncated(t *testing.T) {
buf[102] = 0x00
// Only 1 byte left — not enough for feat2
p := decodeAdvert(buf[:104])
p := decodeAdvert(buf[:104], false)
if p.Feat1 == nil {
t.Error("feat1 should be set")
}
@@ -544,7 +544,7 @@ func TestDecodeAdvertSensorBadTelemetry(t *testing.T) {
buf[105] = 0x20
buf[106] = 0x4E
p := decodeAdvert(buf[:107])
p := decodeAdvert(buf[:107], false)
if p.BatteryMv != nil {
t.Error("battery_mv=0 should be nil")
}
@@ -740,7 +740,7 @@ func TestDecodeAdvertSensorNoName(t *testing.T) {
buf[103] = 0xC4
buf[104] = 0x09
p := decodeAdvert(buf[:105])
p := decodeAdvert(buf[:105], false)
if p.Error != "" {
t.Fatalf("error: %s", p.Error)
}
@@ -835,7 +835,7 @@ func TestDecodePacketNoPathByteAfterHeader(t *testing.T) {
// Non-transport route, but only header byte (no path byte)
// Actually 0A alone = 1 byte, but we need >= 2
// Header + exactly at offset boundary
_, err := DecodePacket("0A", nil)
_, err := DecodePacket("0A", nil, false)
if err == nil {
t.Error("should error - too short")
}
@@ -856,7 +856,7 @@ func TestDecodeAdvertNameNoNull(t *testing.T) {
// Name without null terminator — goes to end of buffer
copy(buf[101:], []byte("LongNameNoNull"))
p := decodeAdvert(buf[:115])
p := decodeAdvert(buf[:115], false)
if p.Name != "LongNameNoNull" {
t.Errorf("name=%q, want LongNameNoNull", p.Name)
}
+140 -1
View File
@@ -39,11 +39,19 @@ type Store struct {
stmtGetObserverRowid *sql.Stmt
stmtUpdateObserverLastSeen *sql.Stmt
stmtUpdateNodeTelemetry *sql.Stmt
stmtUpsertMetrics *sql.Stmt
sampleIntervalSec int
}
// OpenStore opens or creates a SQLite DB at the given path, applying the
// v3 schema that is compatible with the Node.js server.
func OpenStore(dbPath string) (*Store, error) {
return OpenStoreWithInterval(dbPath, 300)
}
// OpenStoreWithInterval opens or creates a SQLite DB with a configurable sample interval.
func OpenStoreWithInterval(dbPath string, sampleIntervalSec int) (*Store, error) {
dir := filepath.Dir(dbPath)
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, fmt.Errorf("creating data dir: %w", err)
@@ -66,7 +74,7 @@ func OpenStore(dbPath string) (*Store, error) {
return nil, fmt.Errorf("applying schema: %w", err)
}
s := &Store{db: db}
s := &Store{db: db, sampleIntervalSec: sampleIntervalSec}
if err := s.prepareStatements(); err != nil {
return nil, fmt.Errorf("preparing statements: %w", err)
}
@@ -292,6 +300,51 @@ func applySchema(db *sql.DB) error {
log.Println("[migration] observations timestamp index created")
}
// observer_metrics table for RF health dashboard
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'observer_metrics_v1'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Creating observer_metrics table...")
_, err := db.Exec(`
CREATE TABLE IF NOT EXISTS observer_metrics (
observer_id TEXT NOT NULL,
timestamp TEXT NOT NULL,
noise_floor REAL,
tx_air_secs INTEGER,
rx_air_secs INTEGER,
recv_errors INTEGER,
battery_mv INTEGER,
PRIMARY KEY (observer_id, timestamp)
)
`)
if err != nil {
return fmt.Errorf("observer_metrics schema: %w", err)
}
db.Exec(`INSERT INTO _migrations (name) VALUES ('observer_metrics_v1')`)
log.Println("[migration] observer_metrics table created")
}
// Migration: add timestamp index for cross-observer time-range queries
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'observer_metrics_ts_idx'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Creating observer_metrics timestamp index...")
_, err := db.Exec(`CREATE INDEX IF NOT EXISTS idx_observer_metrics_timestamp ON observer_metrics(timestamp)`)
if err != nil {
return fmt.Errorf("observer_metrics timestamp index: %w", err)
}
db.Exec(`INSERT INTO _migrations (name) VALUES ('observer_metrics_ts_idx')`)
log.Println("[migration] observer_metrics timestamp index created")
}
// Migration: add packets_sent and packets_recv columns to observer_metrics
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'observer_metrics_packets_v1'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Adding packets_sent/packets_recv columns to observer_metrics...")
db.Exec(`ALTER TABLE observer_metrics ADD COLUMN packets_sent INTEGER`)
db.Exec(`ALTER TABLE observer_metrics ADD COLUMN packets_recv INTEGER`)
db.Exec(`INSERT INTO _migrations (name) VALUES ('observer_metrics_packets_v1')`)
log.Println("[migration] packets_sent/packets_recv columns added")
}
return nil
}
@@ -385,6 +438,14 @@ func (s *Store) prepareStatements() error {
return err
}
s.stmtUpsertMetrics, err = s.db.Prepare(`
INSERT OR REPLACE INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
`)
if err != nil {
return err
}
return nil
}
@@ -517,6 +578,11 @@ type ObserverMeta struct {
BatteryMv *int // millivolts, always integer
UptimeSecs *int64 // seconds, always integer
NoiseFloor *float64 // dBm, may have decimals
TxAirSecs *int // cumulative TX seconds since boot
RxAirSecs *int // cumulative RX seconds since boot
RecvErrors *int // cumulative CRC/decode failures since boot
PacketsSent *int // cumulative packets sent since boot
PacketsRecv *int // cumulative packets received since boot
}
// UpsertObserver inserts or updates an observer with optional hardware metadata.
@@ -568,6 +634,79 @@ func (s *Store) Close() error {
return s.db.Close()
}
// RoundToInterval rounds a time to the nearest sample interval boundary.
func RoundToInterval(t time.Time, intervalSec int) time.Time {
if intervalSec <= 0 {
intervalSec = 300
}
epoch := t.Unix()
half := int64(intervalSec) / 2
rounded := ((epoch + half) / int64(intervalSec)) * int64(intervalSec)
return time.Unix(rounded, 0).UTC()
}
// MetricsData holds the fields to insert into observer_metrics.
type MetricsData struct {
ObserverID string
NoiseFloor *float64
TxAirSecs *int
RxAirSecs *int
RecvErrors *int
BatteryMv *int
PacketsSent *int
PacketsRecv *int
}
// InsertMetrics inserts a metrics sample for an observer using ingestor wall clock.
func (s *Store) InsertMetrics(data *MetricsData) error {
ts := RoundToInterval(time.Now().UTC(), s.sampleIntervalSec)
tsStr := ts.Format(time.RFC3339)
var nf, txAir, rxAir, recvErr, batt, pktSent, pktRecv interface{}
if data.NoiseFloor != nil {
nf = *data.NoiseFloor
}
if data.TxAirSecs != nil {
txAir = *data.TxAirSecs
}
if data.RxAirSecs != nil {
rxAir = *data.RxAirSecs
}
if data.RecvErrors != nil {
recvErr = *data.RecvErrors
}
if data.BatteryMv != nil {
batt = *data.BatteryMv
}
if data.PacketsSent != nil {
pktSent = *data.PacketsSent
}
if data.PacketsRecv != nil {
pktRecv = *data.PacketsRecv
}
_, err := s.stmtUpsertMetrics.Exec(data.ObserverID, tsStr, nf, txAir, rxAir, recvErr, batt, pktSent, pktRecv)
if err != nil {
s.Stats.WriteErrors.Add(1)
return fmt.Errorf("insert metrics: %w", err)
}
return nil
}
// PruneOldMetrics deletes observer_metrics rows older than retentionDays.
func (s *Store) PruneOldMetrics(retentionDays int) (int64, error) {
cutoff := time.Now().UTC().AddDate(0, 0, -retentionDays).Format(time.RFC3339)
result, err := s.db.Exec(`DELETE FROM observer_metrics WHERE timestamp < ?`, cutoff)
if err != nil {
return 0, fmt.Errorf("prune metrics: %w", err)
}
n, _ := result.RowsAffected()
if n > 0 {
log.Printf("[metrics] Pruned %d rows older than %d days", n, retentionDays)
}
return n, nil
}
// Checkpoint forces a WAL checkpoint to release the WAL lock file,
// preventing lock contention with a new process starting up.
func (s *Store) Checkpoint() {
+185 -6
View File
@@ -576,7 +576,7 @@ func TestEndToEndIngest(t *testing.T) {
// Simulate full pipeline: decode + insert
rawHex := "120046D62DE27D4C5194D7821FC5A34A45565DCC2537B300B9AB6275255CEFB65D840CE5C169C94C9AED39E8BCB6CB6EB0335497A198B33A1A610CD3B03D8DCFC160900E5244280323EE0B44CACAB8F02B5B38B91CFA18BD067B0B5E63E94CFC85F758A8530B9240933402E0E6B8F84D5252322D52"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -764,7 +764,7 @@ func TestInsertTransmissionNilSNRRSSI(t *testing.T) {
func TestBuildPacketData(t *testing.T) {
rawHex := "0A00D69FD7A5A7475DB07337749AE61FA53A4788E976"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -818,7 +818,7 @@ func TestBuildPacketData(t *testing.T) {
func TestBuildPacketDataWithHops(t *testing.T) {
// A packet with actual hops in the path
raw := "0505AABBCCDDEE" + strings.Repeat("00", 10)
decoded, err := DecodePacket(raw, nil)
decoded, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -834,7 +834,7 @@ func TestBuildPacketDataWithHops(t *testing.T) {
}
func TestBuildPacketDataNilSNRRSSI(t *testing.T) {
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil)
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil, false)
msg := &MQTTPacketMessage{Raw: "0A00" + strings.Repeat("00", 10)}
pkt := BuildPacketData(msg, decoded, "", "")
@@ -1624,7 +1624,7 @@ func TestObsTimestampIndexMigration(t *testing.T) {
func TestBuildPacketDataScoreAndDirection(t *testing.T) {
rawHex := "0A00D69FD7A5A7475DB07337749AE61FA53A4788E976"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1647,7 +1647,7 @@ func TestBuildPacketDataScoreAndDirection(t *testing.T) {
}
func TestBuildPacketDataNilScoreDirection(t *testing.T) {
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil)
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil, false)
msg := &MQTTPacketMessage{Raw: "0A00" + strings.Repeat("00", 10)}
pkt := BuildPacketData(msg, decoded, "", "")
@@ -1703,3 +1703,182 @@ func TestInsertTransmissionWithScoreAndDirection(t *testing.T) {
}
func ptrFloat(f float64) *float64 { return &f }
func ptrInt(i int) *int { return &i }
func TestRoundToInterval(t *testing.T) {
tests := []struct {
input time.Time
interval int
want time.Time
}{
{time.Date(2026, 4, 5, 10, 2, 0, 0, time.UTC), 300, time.Date(2026, 4, 5, 10, 0, 0, 0, time.UTC)},
{time.Date(2026, 4, 5, 10, 3, 0, 0, time.UTC), 300, time.Date(2026, 4, 5, 10, 5, 0, 0, time.UTC)},
{time.Date(2026, 4, 5, 10, 2, 30, 0, time.UTC), 300, time.Date(2026, 4, 5, 10, 5, 0, 0, time.UTC)},
{time.Date(2026, 4, 5, 10, 5, 0, 0, time.UTC), 300, time.Date(2026, 4, 5, 10, 5, 0, 0, time.UTC)},
{time.Date(2026, 4, 5, 10, 7, 29, 0, time.UTC), 300, time.Date(2026, 4, 5, 10, 5, 0, 0, time.UTC)},
}
for _, tc := range tests {
got := RoundToInterval(tc.input, tc.interval)
if !got.Equal(tc.want) {
t.Errorf("RoundToInterval(%v, %d) = %v, want %v", tc.input, tc.interval, got, tc.want)
}
}
}
func TestInsertMetrics(t *testing.T) {
store, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer store.Close()
nf := -112.5
txAir := 100
rxAir := 500
recvErr := 3
batt := 3720
data := &MetricsData{
ObserverID: "obs1",
NoiseFloor: &nf,
TxAirSecs: &txAir,
RxAirSecs: &rxAir,
RecvErrors: &recvErr,
BatteryMv: &batt,
}
if err := store.InsertMetrics(data); err != nil {
t.Fatalf("InsertMetrics: %v", err)
}
// Verify insertion
var count int
store.db.QueryRow("SELECT COUNT(*) FROM observer_metrics WHERE observer_id = 'obs1'").Scan(&count)
if count != 1 {
t.Errorf("expected 1 row, got %d", count)
}
// Verify values
var gotNF float64
var gotTx, gotRx, gotErr, gotBatt int
store.db.QueryRow("SELECT noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv FROM observer_metrics WHERE observer_id = 'obs1'").Scan(&gotNF, &gotTx, &gotRx, &gotErr, &gotBatt)
if gotNF != -112.5 {
t.Errorf("noise_floor = %v, want -112.5", gotNF)
}
if gotTx != 100 {
t.Errorf("tx_air_secs = %d, want 100", gotTx)
}
}
func TestInsertMetricsIdempotent(t *testing.T) {
store, err := OpenStoreWithInterval(tempDBPath(t), 300)
if err != nil {
t.Fatal(err)
}
defer store.Close()
nf := -110.0
data := &MetricsData{ObserverID: "obs1", NoiseFloor: &nf}
// Insert twice — should result in 1 row (INSERT OR REPLACE)
store.InsertMetrics(data)
nf2 := -108.0
data.NoiseFloor = &nf2
store.InsertMetrics(data)
var count int
store.db.QueryRow("SELECT COUNT(*) FROM observer_metrics WHERE observer_id = 'obs1'").Scan(&count)
if count != 1 {
t.Errorf("expected 1 row (idempotent), got %d", count)
}
// Verify the value was replaced
var gotNF float64
store.db.QueryRow("SELECT noise_floor FROM observer_metrics WHERE observer_id = 'obs1'").Scan(&gotNF)
if gotNF != -108.0 {
t.Errorf("noise_floor = %v, want -108.0 (replaced)", gotNF)
}
}
func TestInsertMetricsNullFields(t *testing.T) {
store, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer store.Close()
nf := -115.0
data := &MetricsData{
ObserverID: "obs1",
NoiseFloor: &nf,
// All other fields nil
}
if err := store.InsertMetrics(data); err != nil {
t.Fatalf("InsertMetrics with nulls: %v", err)
}
var gotNF sql.NullFloat64
var gotTx sql.NullInt64
store.db.QueryRow("SELECT noise_floor, tx_air_secs FROM observer_metrics WHERE observer_id = 'obs1'").Scan(&gotNF, &gotTx)
if !gotNF.Valid || gotNF.Float64 != -115.0 {
t.Errorf("noise_floor = %v, want -115.0", gotNF)
}
if gotTx.Valid {
t.Errorf("tx_air_secs should be NULL, got %v", gotTx.Int64)
}
}
func TestPruneOldMetrics(t *testing.T) {
store, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer store.Close()
// Insert old and new metrics directly
oldTs := time.Now().UTC().AddDate(0, 0, -40).Format(time.RFC3339)
newTs := time.Now().UTC().Format(time.RFC3339)
store.db.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)", "obs1", oldTs, -110.0)
store.db.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)", "obs1", newTs, -112.0)
n, err := store.PruneOldMetrics(30)
if err != nil {
t.Fatalf("PruneOldMetrics: %v", err)
}
if n != 1 {
t.Errorf("pruned %d rows, want 1", n)
}
var count int
store.db.QueryRow("SELECT COUNT(*) FROM observer_metrics").Scan(&count)
if count != 1 {
t.Errorf("expected 1 row remaining, got %d", count)
}
}
func TestExtractObserverMetaNewFields(t *testing.T) {
msg := map[string]interface{}{
"model": "L1",
"stats": map[string]interface{}{
"noise_floor": -112.5,
"battery_mv": 3720.0,
"uptime_secs": 86400.0,
"tx_air_secs": 100.0,
"rx_air_secs": 500.0,
"recv_errors": 3.0,
},
}
meta := extractObserverMeta(msg)
if meta == nil {
t.Fatal("expected non-nil meta")
}
if meta.TxAirSecs == nil || *meta.TxAirSecs != 100 {
t.Errorf("TxAirSecs = %v, want 100", meta.TxAirSecs)
}
if meta.RxAirSecs == nil || *meta.RxAirSecs != 500 {
t.Errorf("RxAirSecs = %v, want 500", meta.RxAirSecs)
}
if meta.RecvErrors == nil || *meta.RecvErrors != 3 {
t.Errorf("RecvErrors = %v, want 3", meta.RecvErrors)
}
}
+23 -5
View File
@@ -11,6 +11,8 @@ import (
"math"
"strings"
"unicode/utf8"
"github.com/meshcore-analyzer/sigvalidate"
)
// Route type constants (header bits 1-0)
@@ -109,6 +111,7 @@ type Payload struct {
Timestamp uint32 `json:"timestamp,omitempty"`
TimestampISO string `json:"timestampISO,omitempty"`
Signature string `json:"signature,omitempty"`
SignatureValid *bool `json:"signatureValid,omitempty"`
Flags *AdvertFlags `json:"flags,omitempty"`
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
@@ -215,7 +218,7 @@ func decodeAck(buf []byte) Payload {
}
}
func decodeAdvert(buf []byte) Payload {
func decodeAdvert(buf []byte, validateSignatures bool) Payload {
if len(buf) < 100 {
return Payload{Type: "ADVERT", Error: "too short for advert", RawHex: hex.EncodeToString(buf)}
}
@@ -233,6 +236,11 @@ func decodeAdvert(buf []byte) Payload {
Signature: signature,
}
if validateSignatures {
valid := sigvalidate.ValidateAdvertSignature(buf[0:32], buf[36:100], timestamp, appdata)
p.SignatureValid = &valid
}
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
@@ -506,7 +514,7 @@ func decodeTrace(buf []byte) Payload {
return p
}
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload {
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string, validateSignatures bool) Payload {
switch payloadType {
case PayloadREQ:
return decodeEncryptedPayload("REQ", buf)
@@ -517,7 +525,7 @@ func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) P
case PayloadACK:
return decodeAck(buf)
case PayloadADVERT:
return decodeAdvert(buf)
return decodeAdvert(buf, validateSignatures)
case PayloadGRP_TXT:
return decodeGrpTxt(buf, channelKeys)
case PayloadANON_REQ:
@@ -532,7 +540,7 @@ func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) P
}
// DecodePacket decodes a hex-encoded MeshCore packet.
func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPacket, error) {
func DecodePacket(hexString string, channelKeys map[string]string, validateSignatures bool) (*DecodedPacket, error) {
hexString = strings.ReplaceAll(hexString, " ", "")
hexString = strings.ReplaceAll(hexString, "\n", "")
hexString = strings.ReplaceAll(hexString, "\r", "")
@@ -570,7 +578,7 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
offset += bytesConsumed
payloadBuf := buf[offset:]
payload := decodePayload(header.PayloadType, payloadBuf, channelKeys)
payload := decodePayload(header.PayloadType, payloadBuf, channelKeys, validateSignatures)
// TRACE packets store hop IDs in the payload (buf[9:]) rather than the header
// path field. The header path byte still encodes hashSize in bits 6-7, which
@@ -587,6 +595,16 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
}
}
// Zero-hop direct packets have hash_count=0 (lower 6 bits of pathByte),
// which makes the generic formula yield a bogus hashSize. Reset to 0
// (unknown) so API consumers get correct data. We mask with 0x3F to check
// only hash_count, matching the JS frontend approach — the upper hash_size
// bits are meaningless when there are no hops. Skip TRACE packets — they
// use hashSize to parse hops from the payload above.
if (header.RouteType == RouteDirect || header.RouteType == RouteTransportDirect) && pathByte&0x3F == 0 && header.PayloadType != PayloadTRACE {
path.HashSize = 0
}
return &DecodedPacket{
Header: header,
TransportCodes: tc,
+190 -37
View File
@@ -2,6 +2,7 @@ package main
import (
"crypto/aes"
"crypto/ed25519"
"crypto/hmac"
"crypto/sha256"
"encoding/binary"
@@ -9,6 +10,8 @@ import (
"math"
"strings"
"testing"
"github.com/meshcore-analyzer/sigvalidate"
)
func TestDecodeHeaderRoutTypes(t *testing.T) {
@@ -55,7 +58,7 @@ func TestDecodeHeaderPayloadTypes(t *testing.T) {
func TestDecodePathZeroHops(t *testing.T) {
// 0x00: 0 hops, 1-byte hashes
pkt, err := DecodePacket("0500"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0500"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -72,7 +75,7 @@ func TestDecodePathZeroHops(t *testing.T) {
func TestDecodePath1ByteHashes(t *testing.T) {
// 0x05: 5 hops, 1-byte hashes → 5 path bytes
pkt, err := DecodePacket("0505"+"AABBCCDDEE"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0505"+"AABBCCDDEE"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -95,7 +98,7 @@ func TestDecodePath1ByteHashes(t *testing.T) {
func TestDecodePath2ByteHashes(t *testing.T) {
// 0x45: 5 hops, 2-byte hashes
pkt, err := DecodePacket("0545"+"AA11BB22CC33DD44EE55"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0545"+"AA11BB22CC33DD44EE55"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -112,7 +115,7 @@ func TestDecodePath2ByteHashes(t *testing.T) {
func TestDecodePath3ByteHashes(t *testing.T) {
// 0x8A: 10 hops, 3-byte hashes
pkt, err := DecodePacket("058A"+strings.Repeat("AA11FF", 10)+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("058A"+strings.Repeat("AA11FF", 10)+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -131,7 +134,7 @@ func TestTransportCodes(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) should have transport codes
// Firmware order: header + transport_codes(4) + path_len + path + payload
hex := "14" + "AABB" + "CCDD" + "00" + strings.Repeat("00", 10)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -149,7 +152,7 @@ func TestTransportCodes(t *testing.T) {
}
// Route type 1 (FLOOD) should NOT have transport codes
pkt2, err := DecodePacket("0500"+strings.Repeat("00", 10), nil)
pkt2, err := DecodePacket("0500"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -169,7 +172,7 @@ func TestDecodeAdvertFull(t *testing.T) {
name := "546573744E6F6465" // "TestNode"
hex := "1200" + pubkey + timestamp + signature + flags + lat + lon + name
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -227,7 +230,7 @@ func TestDecodeAdvertTypeEnums(t *testing.T) {
makeAdvert := func(flagsByte byte) *DecodedPacket {
hex := "1200" + strings.Repeat("AA", 32) + "00000000" + strings.Repeat("BB", 64) +
strings.ToUpper(string([]byte{hexDigit(flagsByte>>4), hexDigit(flagsByte & 0x0f)}))
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -272,7 +275,7 @@ func hexDigit(v byte) byte {
func TestDecodeAdvertNoLocationNoName(t *testing.T) {
hex := "1200" + strings.Repeat("CC", 32) + "00000000" + strings.Repeat("DD", 64) + "02"
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -291,7 +294,7 @@ func TestDecodeAdvertNoLocationNoName(t *testing.T) {
}
func TestGoldenFixtureTxtMsg(t *testing.T) {
pkt, err := DecodePacket("0A00D69FD7A5A7475DB07337749AE61FA53A4788E976", nil)
pkt, err := DecodePacket("0A00D69FD7A5A7475DB07337749AE61FA53A4788E976", nil, false)
if err != nil {
t.Fatal(err)
}
@@ -314,7 +317,7 @@ func TestGoldenFixtureTxtMsg(t *testing.T) {
func TestGoldenFixtureAdvert(t *testing.T) {
rawHex := "120046D62DE27D4C5194D7821FC5A34A45565DCC2537B300B9AB6275255CEFB65D840CE5C169C94C9AED39E8BCB6CB6EB0335497A198B33A1A610CD3B03D8DCFC160900E5244280323EE0B44CACAB8F02B5B38B91CFA18BD067B0B5E63E94CFC85F758A8530B9240933402E0E6B8F84D5252322D52"
pkt, err := DecodePacket(rawHex, nil)
pkt, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -337,7 +340,7 @@ func TestGoldenFixtureAdvert(t *testing.T) {
func TestGoldenFixtureUnicodeAdvert(t *testing.T) {
rawHex := "120073CFF971E1CB5754A742C152B2D2E0EB108A19B246D663ED8898A72C4A5AD86EA6768E66694B025EDF6939D5C44CFF719C5D5520E5F06B20680A83AD9C2C61C3227BBB977A85EE462F3553445FECF8EDD05C234ECE217272E503F14D6DF2B1B9B133890C923CDF3002F8FDC1F85045414BF09F8CB3"
pkt, err := DecodePacket(rawHex, nil)
pkt, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -354,14 +357,14 @@ func TestGoldenFixtureUnicodeAdvert(t *testing.T) {
}
func TestDecodePacketTooShort(t *testing.T) {
_, err := DecodePacket("FF", nil)
_, err := DecodePacket("FF", nil, false)
if err == nil {
t.Error("expected error for 1-byte packet")
}
}
func TestDecodePacketInvalidHex(t *testing.T) {
_, err := DecodePacket("ZZZZ", nil)
_, err := DecodePacket("ZZZZ", nil, false)
if err == nil {
t.Error("expected error for invalid hex")
}
@@ -568,7 +571,7 @@ func TestDecodeTracePathParsing(t *testing.T) {
// Packet from issue #276: 260001807dca00000000007d547d
// Path byte 0x00 → hashSize=1, hops in payload at buf[9:] = 7d 54 7d
// Expected path: ["7D", "54", "7D"]
pkt, err := DecodePacket("260001807dca00000000007d547d", nil)
pkt, err := DecodePacket("260001807dca00000000007d547d", nil, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
@@ -590,7 +593,7 @@ func TestDecodeTracePathParsing(t *testing.T) {
}
func TestDecodeAdvertShort(t *testing.T) {
p := decodeAdvert(make([]byte, 50))
p := decodeAdvert(make([]byte, 50), false)
if p.Error != "too short for advert" {
t.Errorf("expected 'too short for advert' error, got %q", p.Error)
}
@@ -628,7 +631,7 @@ func TestDecodeEncryptedPayloadValid(t *testing.T) {
func TestDecodePayloadGRPData(t *testing.T) {
buf := []byte{0x01, 0x02, 0x03}
p := decodePayload(PayloadGRP_DATA, buf, nil)
p := decodePayload(PayloadGRP_DATA, buf, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("type=%s, want UNKNOWN", p.Type)
}
@@ -639,7 +642,7 @@ func TestDecodePayloadGRPData(t *testing.T) {
func TestDecodePayloadRAWCustom(t *testing.T) {
buf := []byte{0xFF, 0xFE}
p := decodePayload(PayloadRAW_CUSTOM, buf, nil)
p := decodePayload(PayloadRAW_CUSTOM, buf, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("type=%s, want UNKNOWN", p.Type)
}
@@ -647,49 +650,49 @@ func TestDecodePayloadRAWCustom(t *testing.T) {
func TestDecodePayloadAllTypes(t *testing.T) {
// REQ
p := decodePayload(PayloadREQ, make([]byte, 10), nil)
p := decodePayload(PayloadREQ, make([]byte, 10), nil, false)
if p.Type != "REQ" {
t.Errorf("REQ: type=%s", p.Type)
}
// RESPONSE
p = decodePayload(PayloadRESPONSE, make([]byte, 10), nil)
p = decodePayload(PayloadRESPONSE, make([]byte, 10), nil, false)
if p.Type != "RESPONSE" {
t.Errorf("RESPONSE: type=%s", p.Type)
}
// TXT_MSG
p = decodePayload(PayloadTXT_MSG, make([]byte, 10), nil)
p = decodePayload(PayloadTXT_MSG, make([]byte, 10), nil, false)
if p.Type != "TXT_MSG" {
t.Errorf("TXT_MSG: type=%s", p.Type)
}
// ACK
p = decodePayload(PayloadACK, make([]byte, 10), nil)
p = decodePayload(PayloadACK, make([]byte, 10), nil, false)
if p.Type != "ACK" {
t.Errorf("ACK: type=%s", p.Type)
}
// GRP_TXT
p = decodePayload(PayloadGRP_TXT, make([]byte, 10), nil)
p = decodePayload(PayloadGRP_TXT, make([]byte, 10), nil, false)
if p.Type != "GRP_TXT" {
t.Errorf("GRP_TXT: type=%s", p.Type)
}
// ANON_REQ
p = decodePayload(PayloadANON_REQ, make([]byte, 40), nil)
p = decodePayload(PayloadANON_REQ, make([]byte, 40), nil, false)
if p.Type != "ANON_REQ" {
t.Errorf("ANON_REQ: type=%s", p.Type)
}
// PATH
p = decodePayload(PayloadPATH, make([]byte, 10), nil)
p = decodePayload(PayloadPATH, make([]byte, 10), nil, false)
if p.Type != "PATH" {
t.Errorf("PATH: type=%s", p.Type)
}
// TRACE
p = decodePayload(PayloadTRACE, make([]byte, 20), nil)
p = decodePayload(PayloadTRACE, make([]byte, 20), nil, false)
if p.Type != "TRACE" {
t.Errorf("TRACE: type=%s", p.Type)
}
@@ -925,7 +928,7 @@ func TestComputeContentHashLongFallback(t *testing.T) {
func TestDecodePacketWithWhitespace(t *testing.T) {
raw := "0A 00 D6 9F D7 A5 A7 47 5D B0 73 37 74 9A E6 1F A5 3A 47 88 E9 76"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -936,7 +939,7 @@ func TestDecodePacketWithWhitespace(t *testing.T) {
func TestDecodePacketWithNewlines(t *testing.T) {
raw := "0A00\nD69F\r\nD7A5A7475DB07337749AE61FA53A4788E976"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -947,7 +950,7 @@ func TestDecodePacketWithNewlines(t *testing.T) {
func TestDecodePacketTransportRouteTooShort(t *testing.T) {
// TRANSPORT_FLOOD (route=0) but only 2 bytes total → too short for transport codes
_, err := DecodePacket("1400", nil)
_, err := DecodePacket("1400", nil, false)
if err == nil {
t.Error("expected error for transport route with too-short buffer")
}
@@ -1007,7 +1010,7 @@ func TestDecodeHeaderUnknownTypes(t *testing.T) {
func TestDecodePayloadMultipart(t *testing.T) {
// MULTIPART (0x0A) falls through to default → UNKNOWN
p := decodePayload(PayloadMULTIPART, []byte{0x01, 0x02}, nil)
p := decodePayload(PayloadMULTIPART, []byte{0x01, 0x02}, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("MULTIPART type=%s, want UNKNOWN", p.Type)
}
@@ -1015,7 +1018,7 @@ func TestDecodePayloadMultipart(t *testing.T) {
func TestDecodePayloadControl(t *testing.T) {
// CONTROL (0x0B) falls through to default → UNKNOWN
p := decodePayload(PayloadCONTROL, []byte{0x01, 0x02}, nil)
p := decodePayload(PayloadCONTROL, []byte{0x01, 0x02}, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("CONTROL type=%s, want UNKNOWN", p.Type)
}
@@ -1039,7 +1042,7 @@ func TestDecodePathTruncatedBuffer(t *testing.T) {
func TestDecodeFloodAdvert5Hops(t *testing.T) {
// From test-decoder.js Test 1
raw := "11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1410,7 +1413,7 @@ func TestDecodeAdvertWithTelemetry(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1449,7 +1452,7 @@ func TestDecodeAdvertWithTelemetryNegativeTemp(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1476,7 +1479,7 @@ func TestDecodeAdvertWithoutTelemetry(t *testing.T) {
name := hex.EncodeToString([]byte("Node1"))
hexStr := "1200" + pubkey + timestamp + signature + flags + name
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1503,7 +1506,7 @@ func TestDecodeAdvertNonSensorIgnoresTelemetryBytes(t *testing.T) {
extraBytes := "B40ED403" // battery-like and temp-like bytes
hexStr := "1200" + pubkey + timestamp + signature + flags + name + nullTerm + extraBytes
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1531,7 +1534,7 @@ func TestDecodeAdvertTelemetryZeroTemp(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1542,3 +1545,153 @@ func TestDecodeAdvertTelemetryZeroTemp(t *testing.T) {
t.Errorf("temperature_c=%f, want 0.0", *pkt.Payload.TemperatureC)
}
}
func repeatHex(byteHex string, n int) string {
s := ""
for i := 0; i < n; i++ {
s += byteHex
}
return s
}
func TestZeroHopDirectHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x00 → hash_count=0, hash_size bits=0 → should get HashSize=0
hex := "02" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("DIRECT zero-hop: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x40 → hash_count=0, hash_size bits=01 → should still get HashSize=0
hex := "02" + "40" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("DIRECT zero-hop with hash_size bits set: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestNonDirectZeroPathByteKeepsHashSize(t *testing.T) {
// FLOOD (RouteType=1) + REQ (PayloadType=0) → header byte = 0x01
// pathByte=0x00 → non-DIRECT should keep HashSize=1
hex := "01" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 1 {
t.Errorf("FLOOD zero pathByte: want HashSize=1, got %d", pkt.Path.HashSize)
}
}
func TestDirectNonZeroHopKeepsHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x01 → hash_count=1, hash_size=1 → should keep HashSize=1
hex := "02" + "01" + repeatHex("BB", 21)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 1 {
t.Errorf("DIRECT with 1 hop: want HashSize=1, got %d", pkt.Path.HashSize)
}
}
func TestValidateAdvertSignature(t *testing.T) {
// Generate a real ed25519 key pair
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1234567890
appdata := []byte{0x02, 0x11, 0x22} // flags + some data
// Build the message the same way ValidateAdvertSignature does
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Valid signature
if !sigvalidate.ValidateAdvertSignature(pub, sig, timestamp, appdata) {
t.Error("expected valid signature")
}
// Tampered appdata → invalid
badAppdata := []byte{0x03, 0x11, 0x22}
if sigvalidate.ValidateAdvertSignature(pub, sig, timestamp, badAppdata) {
t.Error("expected invalid signature with tampered appdata")
}
// Wrong timestamp → invalid
if sigvalidate.ValidateAdvertSignature(pub, sig, timestamp+1, appdata) {
t.Error("expected invalid signature with wrong timestamp")
}
// Wrong length pubkey → false
if sigvalidate.ValidateAdvertSignature([]byte{0xAA, 0xBB}, sig, timestamp, appdata) {
t.Error("expected false for short pubkey")
}
// Wrong length signature → false
if sigvalidate.ValidateAdvertSignature(pub, []byte{0xAA, 0xBB}, timestamp, appdata) {
t.Error("expected false for short signature")
}
}
func TestDecodeAdvertWithSignatureValidation(t *testing.T) {
// Generate key pair
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1000000
appdata := []byte{0x02} // repeater type, no location
// Build signed message
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Build advert buffer: pubkey(32) + timestamp(4) + signature(64) + appdata
buf := make([]byte, 0, 101)
buf = append(buf, pub...)
ts := make([]byte, 4)
binary.LittleEndian.PutUint32(ts, timestamp)
buf = append(buf, ts...)
buf = append(buf, sig...)
buf = append(buf, appdata...)
// With validation enabled
p := decodeAdvert(buf, true)
if p.Error != "" {
t.Fatalf("decode error: %s", p.Error)
}
if p.SignatureValid == nil {
t.Fatal("SignatureValid should be set when validation enabled")
}
if !*p.SignatureValid {
t.Error("expected valid signature")
}
// Without validation
p2 := decodeAdvert(buf, false)
if p2.SignatureValid != nil {
t.Error("SignatureValid should be nil when validation disabled")
}
}
+3
View File
@@ -5,11 +5,14 @@ go 1.22
require (
github.com/eclipse/paho.mqtt.golang v1.5.0
github.com/meshcore-analyzer/geofilter v0.0.0
github.com/meshcore-analyzer/sigvalidate v0.0.0-00010101000000-000000000000
modernc.org/sqlite v1.34.5
)
replace github.com/meshcore-analyzer/geofilter => ../../internal/geofilter
replace github.com/meshcore-analyzer/sigvalidate => ../../internal/sigvalidate
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
+67 -6
View File
@@ -49,11 +49,8 @@ func main() {
}
sources := cfg.ResolvedSources()
if len(sources) == 0 {
log.Fatal("no MQTT sources configured — set mqttSources in config or MQTT_BROKER env var")
}
store, err := OpenStore(cfg.DBPath)
store, err := OpenStoreWithInterval(cfg.DBPath, cfg.MetricsSampleInterval())
if err != nil {
log.Fatalf("db: %v", err)
}
@@ -64,6 +61,10 @@ func main() {
nodeDays := cfg.NodeDaysOrDefault()
store.MoveStaleNodes(nodeDays)
// Metrics retention: prune old metrics on startup
metricsDays := cfg.MetricsRetentionDays()
store.PruneOldMetrics(metricsDays)
// Daily ticker for node retention
retentionTicker := time.NewTicker(1 * time.Hour)
go func() {
@@ -72,6 +73,14 @@ func main() {
}
}()
// Daily ticker for metrics retention (every 24h)
metricsRetentionTicker := time.NewTicker(24 * time.Hour)
go func() {
for range metricsRetentionTicker.C {
store.PruneOldMetrics(metricsDays)
}
}()
// Periodic stats logging (every 5 minutes)
statsTicker := time.NewTicker(5 * time.Minute)
go func() {
@@ -151,7 +160,7 @@ func main() {
}
if len(clients) == 0 {
log.Fatal("no MQTT connections established")
log.Fatal("no MQTT connections established — check broker is running (default: mqtt://localhost:1883). Set MQTT_BROKER env var or configure mqttSources in config.json")
}
log.Printf("Running — %d MQTT source(s) connected", len(clients))
@@ -163,6 +172,7 @@ func main() {
log.Println("Shutting down...")
retentionTicker.Stop()
metricsRetentionTicker.Stop()
statsTicker.Stop()
store.LogStats() // final stats on shutdown
for _, c := range clients {
@@ -215,6 +225,22 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
if err := store.UpsertObserver(observerID, name, iata, meta); err != nil {
log.Printf("MQTT [%s] observer status error: %v", tag, err)
}
// Insert metrics sample from status message
if meta != nil {
metricsData := &MetricsData{
ObserverID: observerID,
NoiseFloor: meta.NoiseFloor,
TxAirSecs: meta.TxAirSecs,
RxAirSecs: meta.RxAirSecs,
RecvErrors: meta.RecvErrors,
BatteryMv: meta.BatteryMv,
PacketsSent: meta.PacketsSent,
PacketsRecv: meta.PacketsRecv,
}
if err := store.InsertMetrics(metricsData); err != nil {
log.Printf("MQTT [%s] metrics insert error: %v", tag, err)
}
}
log.Printf("MQTT [%s] status: %s (%s)", tag, firstNonEmpty(name, observerID), iata)
return
}
@@ -222,7 +248,7 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
// Format 1: Raw packet (meshcoretomqtt / Cisien format)
rawHex, _ := msg["raw"].(string)
if rawHex != "" {
decoded, err := DecodePacket(rawHex, channelKeys)
decoded, err := DecodePacket(rawHex, channelKeys, false)
if err != nil {
log.Printf("MQTT [%s] decode error: %v", tag, err)
return
@@ -616,6 +642,41 @@ func extractObserverMeta(msg map[string]interface{}) *ObserverMeta {
hasData = true
}
}
if v := nestedOrTopLevel(stats, msg, "tx_air_secs"); v != nil {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.TxAirSecs = &iv
hasData = true
}
}
if v := nestedOrTopLevel(stats, msg, "rx_air_secs"); v != nil {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.RxAirSecs = &iv
hasData = true
}
}
if v := nestedOrTopLevel(stats, msg, "recv_errors"); v != nil {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.RecvErrors = &iv
hasData = true
}
}
if v := nestedOrTopLevel(stats, msg, "packets_sent"); v != nil {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.PacketsSent = &iv
hasData = true
}
}
if v := nestedOrTopLevel(stats, msg, "packets_recv"); v != nil {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.PacketsRecv = &iv
hasData = true
}
}
if !hasData {
return nil
+111
View File
@@ -0,0 +1,111 @@
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestIsWeakAPIKey(t *testing.T) {
// Known defaults must be detected
for _, weak := range []string{
"your-secret-api-key-here", "change-me", "example", "test",
"password", "admin", "apikey", "api-key", "secret", "default",
} {
if !IsWeakAPIKey(weak) {
t.Errorf("expected %q to be weak", weak)
}
}
// Case-insensitive
if !IsWeakAPIKey("Password") {
t.Error("expected case-insensitive match for Password")
}
if !IsWeakAPIKey("YOUR-SECRET-API-KEY-HERE") {
t.Error("expected case-insensitive match")
}
// Short keys (<16 chars) are weak
if !IsWeakAPIKey("short") {
t.Error("expected short key to be weak")
}
if !IsWeakAPIKey("exactly15chars!") { // 15 chars
t.Error("expected 15-char key to be weak")
}
// Empty key is NOT weak (handled separately as "disabled")
if IsWeakAPIKey("") {
t.Error("empty key should not be flagged as weak")
}
// Strong keys pass
if IsWeakAPIKey("a-very-strong-key-1234") {
t.Error("expected strong key to pass")
}
if IsWeakAPIKey("xK9!mP2@nL5#qR8$") {
t.Error("expected 17-char random key to pass")
}
}
func TestRequireAPIKey_RejectsWeakKey(t *testing.T) {
s := &Server{cfg: &Config{APIKey: "test"}}
handler := s.requireAPIKey(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest("POST", "/api/packets", nil)
req.Header.Set("X-API-Key", "test")
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusForbidden {
t.Errorf("expected 403 for weak key, got %d", rr.Code)
}
}
func TestRequireAPIKey_AcceptsStrongKey(t *testing.T) {
strongKey := "a-very-strong-key-1234"
s := &Server{cfg: &Config{APIKey: strongKey}}
handler := s.requireAPIKey(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest("POST", "/api/packets", nil)
req.Header.Set("X-API-Key", strongKey)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Errorf("expected 200 for strong key, got %d", rr.Code)
}
}
func TestRequireAPIKey_EmptyKeyDisablesEndpoints(t *testing.T) {
s := &Server{cfg: &Config{APIKey: ""}}
handler := s.requireAPIKey(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest("POST", "/api/packets", nil)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusForbidden {
t.Errorf("expected 403 for empty key, got %d", rr.Code)
}
}
func TestRequireAPIKey_WrongKeyUnauthorized(t *testing.T) {
s := &Server{cfg: &Config{APIKey: "a-very-strong-key-1234"}}
handler := s.requireAPIKey(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest("POST", "/api/packets", nil)
req.Header.Set("X-API-Key", "wrong-key-entirely-here")
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Errorf("expected 401 for wrong key, got %d", rr.Code)
}
}
+132
View File
@@ -0,0 +1,132 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gorilla/mux"
)
// TestBackfillAsyncChunked verifies that backfillResolvedPathsAsync processes
// observations in chunks, yields between batches, and sets the completion flag.
func TestBackfillAsyncChunked(t *testing.T) {
store := &PacketStore{
packets: make([]*StoreTx, 0),
byHash: make(map[string]*StoreTx),
byTxID: make(map[int]*StoreTx),
byObsID: make(map[int]*StoreObs),
}
// No pending observations → should complete immediately.
backfillResolvedPathsAsync(store, "", 100, time.Millisecond, 24)
if !store.backfillComplete.Load() {
t.Fatal("expected backfillComplete to be true with empty store")
}
}
// TestBackfillStatusHeader verifies the X-CoreScope-Status header is set correctly.
func TestBackfillStatusHeader(t *testing.T) {
store := &PacketStore{
packets: make([]*StoreTx, 0),
byHash: make(map[string]*StoreTx),
byTxID: make(map[int]*StoreTx),
byObsID: make(map[int]*StoreObs),
}
srv := &Server{store: store}
handler := srv.backfillStatusMiddleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
}))
// Before backfill completes → backfilling
req := httptest.NewRequest("GET", "/api/stats", nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if got := rec.Header().Get("X-CoreScope-Status"); got != "backfilling" {
t.Fatalf("expected 'backfilling', got %q", got)
}
// After backfill completes → ready
store.backfillComplete.Store(true)
rec = httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if got := rec.Header().Get("X-CoreScope-Status"); got != "ready" {
t.Fatalf("expected 'ready', got %q", got)
}
}
// TestStatsBackfillFields verifies /api/stats includes backfill fields.
func TestStatsBackfillFields(t *testing.T) {
db := setupTestDBv2(t)
defer db.Close()
seedV2Data(t, db)
store := &PacketStore{
db: db,
packets: make([]*StoreTx, 0),
byHash: make(map[string]*StoreTx),
byTxID: make(map[int]*StoreTx),
byObsID: make(map[int]*StoreObs),
loaded: true,
}
cfg := &Config{Port: 0}
hub := NewHub()
srv := NewServer(db, cfg, hub)
srv.store = store
router := mux.NewRouter()
srv.RegisterRoutes(router)
// While backfilling
req := httptest.NewRequest("GET", "/api/stats", nil)
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
var resp map[string]interface{}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("failed to parse stats response: %v", err)
}
if backfilling, ok := resp["backfilling"]; !ok {
t.Fatal("missing 'backfilling' field in stats response")
} else if backfilling != true {
t.Fatalf("expected backfilling=true, got %v", backfilling)
}
if _, ok := resp["backfillProgress"]; !ok {
t.Fatal("missing 'backfillProgress' field in stats response")
}
// Check header
if got := rec.Header().Get("X-CoreScope-Status"); got != "backfilling" {
t.Fatalf("expected X-CoreScope-Status=backfilling, got %q", got)
}
// After backfill completes
store.backfillComplete.Store(true)
// Invalidate stats cache
srv.statsMu.Lock()
srv.statsCache = nil
srv.statsMu.Unlock()
rec = httptest.NewRecorder()
router.ServeHTTP(rec, req)
resp = nil
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("failed to parse stats response: %v", err)
}
if backfilling, ok := resp["backfilling"]; !ok || backfilling != false {
t.Fatalf("expected backfilling=false after completion, got %v", backfilling)
}
if got := rec.Header().Get("X-CoreScope-Status"); got != "ready" {
t.Fatalf("expected X-CoreScope-Status=ready, got %q", got)
}
}
+67 -2
View File
@@ -57,6 +57,47 @@ type Config struct {
Timestamps *TimestampConfig `json:"timestamps,omitempty"`
DebugAffinity bool `json:"debugAffinity,omitempty"`
ResolvedPath *ResolvedPathConfig `json:"resolvedPath,omitempty"`
NeighborGraph *NeighborGraphConfig `json:"neighborGraph,omitempty"`
}
// weakAPIKeys is the blocklist of known default/example API keys that must be rejected.
var weakAPIKeys = map[string]bool{
"your-secret-api-key-here": true,
"change-me": true,
"example": true,
"test": true,
"password": true,
"admin": true,
"apikey": true,
"api-key": true,
"secret": true,
"default": true,
}
// IsWeakAPIKey returns true if the key is in the blocklist or shorter than 16 characters.
func IsWeakAPIKey(key string) bool {
if key == "" {
return false // empty is handled separately (endpoints disabled)
}
if weakAPIKeys[strings.ToLower(key)] {
return true
}
if len(key) < 16 {
return true
}
return false
}
// ResolvedPathConfig controls async backfill behavior.
type ResolvedPathConfig struct {
BackfillHours int `json:"backfillHours"` // how far back (hours) to scan for NULL resolved_path (default 24)
}
// NeighborGraphConfig controls neighbor edge pruning.
type NeighborGraphConfig struct {
MaxAgeDays int `json:"maxAgeDays"` // edges older than this are pruned (default 5)
}
// PacketStoreConfig controls in-memory packet store limits.
@@ -69,10 +110,34 @@ type PacketStoreConfig struct {
type GeoFilterConfig = geofilter.Config
type RetentionConfig struct {
NodeDays int `json:"nodeDays"`
PacketDays int `json:"packetDays"`
NodeDays int `json:"nodeDays"`
PacketDays int `json:"packetDays"`
MetricsDays int `json:"metricsDays"`
}
// MetricsRetentionDays returns configured metrics retention or 30 days default.
func (c *Config) MetricsRetentionDays() int {
if c.Retention != nil && c.Retention.MetricsDays > 0 {
return c.Retention.MetricsDays
}
return 30
}
// BackfillHours returns configured backfill window or 24h default.
func (c *Config) BackfillHours() int {
if c.ResolvedPath != nil && c.ResolvedPath.BackfillHours > 0 {
return c.ResolvedPath.BackfillHours
}
return 24
}
// NeighborMaxAgeDays returns configured max edge age or 30 days default.
func (c *Config) NeighborMaxAgeDays() int {
if c.NeighborGraph != nil && c.NeighborGraph.MaxAgeDays > 0 {
return c.NeighborGraph.MaxAgeDays
}
return 5
}
type TimestampConfig struct {
DefaultMode string `json:"defaultMode"` // "ago" | "absolute"
+177
View File
@@ -0,0 +1,177 @@
package main
import (
"database/sql"
"path/filepath"
"testing"
"time"
_ "modernc.org/sqlite"
)
func TestBackfillHoursDefault(t *testing.T) {
cfg := &Config{}
if got := cfg.BackfillHours(); got != 24 {
t.Errorf("BackfillHours() = %d, want 24", got)
}
}
func TestBackfillHoursConfigured(t *testing.T) {
cfg := &Config{ResolvedPath: &ResolvedPathConfig{BackfillHours: 48}}
if got := cfg.BackfillHours(); got != 48 {
t.Errorf("BackfillHours() = %d, want 48", got)
}
}
func TestBackfillHoursZeroFallsBack(t *testing.T) {
cfg := &Config{ResolvedPath: &ResolvedPathConfig{BackfillHours: 0}}
if got := cfg.BackfillHours(); got != 24 {
t.Errorf("BackfillHours() = %d, want 24 (default for zero)", got)
}
}
func TestNeighborMaxAgeDaysDefault(t *testing.T) {
cfg := &Config{}
if got := cfg.NeighborMaxAgeDays(); got != 5 {
t.Errorf("NeighborMaxAgeDays() = %d, want 5", got)
}
}
func TestNeighborMaxAgeDaysConfigured(t *testing.T) {
cfg := &Config{NeighborGraph: &NeighborGraphConfig{MaxAgeDays: 7}}
if got := cfg.NeighborMaxAgeDays(); got != 7 {
t.Errorf("NeighborMaxAgeDays() = %d, want 7", got)
}
}
func TestGraphPruneOlderThan(t *testing.T) {
g := NewNeighborGraph()
now := time.Now().UTC()
// Add a recent edge
g.upsertEdge("aaa", "bbb", "bb", "obs1", nil, now)
// Add an old edge
g.upsertEdge("ccc", "ddd", "dd", "obs1", nil, now.Add(-60*24*time.Hour))
if len(g.AllEdges()) != 2 {
t.Fatalf("expected 2 edges, got %d", len(g.AllEdges()))
}
cutoff := now.Add(-30 * 24 * time.Hour)
pruned := g.PruneOlderThan(cutoff)
if pruned != 1 {
t.Errorf("PruneOlderThan pruned %d, want 1", pruned)
}
edges := g.AllEdges()
if len(edges) != 1 {
t.Fatalf("expected 1 edge after prune, got %d", len(edges))
}
if edges[0].NodeA != "aaa" && edges[0].NodeB != "aaa" {
t.Errorf("wrong edge survived prune: %+v", edges[0])
}
}
func TestPruneNeighborEdgesDB(t *testing.T) {
tmpDir := t.TempDir()
dbPath := filepath.Join(tmpDir, "test.db")
db, err := sql.Open("sqlite", "file:"+dbPath+"?_journal_mode=WAL")
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE neighbor_edges (
node_a TEXT NOT NULL,
node_b TEXT NOT NULL,
count INTEGER DEFAULT 1,
last_seen TEXT,
PRIMARY KEY (node_a, node_b)
)`)
if err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
old := now.Add(-60 * 24 * time.Hour)
db.Exec("INSERT INTO neighbor_edges (node_a, node_b, count, last_seen) VALUES (?, ?, 5, ?)",
"aaa", "bbb", now.Format(time.RFC3339))
db.Exec("INSERT INTO neighbor_edges (node_a, node_b, count, last_seen) VALUES (?, ?, 3, ?)",
"ccc", "ddd", old.Format(time.RFC3339))
g := NewNeighborGraph()
g.upsertEdge("aaa", "bbb", "bb", "obs1", nil, now)
g.upsertEdge("ccc", "ddd", "dd", "obs1", nil, old)
pruned, err := PruneNeighborEdges(dbPath, g, 30)
if err != nil {
t.Fatal(err)
}
if pruned != 1 {
t.Errorf("PruneNeighborEdges pruned %d DB rows, want 1", pruned)
}
var count int
db.QueryRow("SELECT COUNT(*) FROM neighbor_edges").Scan(&count)
if count != 1 {
t.Errorf("expected 1 row in DB after prune, got %d", count)
}
if len(g.AllEdges()) != 1 {
t.Errorf("expected 1 in-memory edge after prune, got %d", len(g.AllEdges()))
}
}
func TestBackfillRespectsHourWindow(t *testing.T) {
store := &PacketStore{}
now := time.Now().UTC()
oldTime := now.Add(-48 * time.Hour).Format(time.RFC3339Nano)
newTime := now.Add(-30 * time.Minute).Format(time.RFC3339Nano)
store.packets = []*StoreTx{
{
ID: 1,
Hash: "old-hash",
FirstSeen: oldTime,
Observations: []*StoreObs{
{ID: 1, PathJSON: `["abc"]`},
},
},
{
ID: 2,
Hash: "new-hash",
FirstSeen: newTime,
Observations: []*StoreObs{
{ID: 2, PathJSON: `["def"]`},
},
},
}
// With a 1-hour window, only the new tx should be processed.
// backfillResolvedPathsAsync will find no prefix map and finish quickly,
// but we can verify the pending count reflects the window.
go backfillResolvedPathsAsync(store, "", 100, time.Millisecond, 1)
// Wait for completion
for i := 0; i < 100; i++ {
if store.backfillComplete.Load() {
break
}
time.Sleep(10 * time.Millisecond)
}
if !store.backfillComplete.Load() {
t.Fatal("backfill did not complete")
}
// With no prefix map, total should be 0 (early exit) or just the new one
// The function exits early when pm == nil, so backfillTotal stays at 0
// if there were pending items but no pm. Let's verify it didn't process
// the old one by checking total <= 1.
total := store.backfillTotal.Load()
if total > 1 {
t.Errorf("backfill total = %d, want <= 1 (old tx should be excluded by hour window)", total)
}
}
+47
View File
@@ -2198,6 +2198,53 @@ func TestStoreGetAnalyticsHashSizes(t *testing.T) {
})
}
func TestHashSizesDistributionByRepeatersFiltersRole(t *testing.T) {
db := setupRichTestDB(t)
defer db.Close()
store := NewPacketStore(db, nil)
store.Load()
result := store.GetAnalyticsHashSizes("")
// distributionByRepeaters should only count repeater nodes.
// Rich test DB: aabbccdd11223344 = repeater (hash size 2), eeff00112233aabb = companion (hash size 3).
dbr, ok := result["distributionByRepeaters"].(map[string]int)
if !ok {
t.Fatal("expected distributionByRepeaters map")
}
// Only the repeater node should be counted.
if dbr["3"] != 0 {
t.Errorf("distributionByRepeaters[3] = %d, want 0 (companion should be excluded)", dbr["3"])
}
if dbr["2"] != 1 {
t.Errorf("distributionByRepeaters[2] = %d, want 1 (repeater)", dbr["2"])
}
// multiByteNodes should include role field for frontend filtering.
mbn, ok := result["multiByteNodes"].([]map[string]interface{})
if !ok {
t.Fatal("expected multiByteNodes slice")
}
for _, node := range mbn {
if _, hasRole := node["role"]; !hasRole {
t.Errorf("multiByteNodes entry missing 'role' field: %v", node)
}
}
// Verify companion is included in multiByteNodes (it's multi-byte) with correct role.
foundCompanion := false
for _, node := range mbn {
if node["pubkey"] == "eeff00112233aabb" {
foundCompanion = true
if node["role"] != "companion" {
t.Errorf("companion node role = %v, want 'companion'", node["role"])
}
}
}
if !foundCompanion {
t.Error("expected companion node in multiByteNodes (multi-byte adopters should include all roles)")
}
}
func TestStoreGetAnalyticsSubpaths(t *testing.T) {
db := setupRichTestDB(t)
defer db.Close()
+338
View File
@@ -1734,3 +1734,341 @@ func (db *DB) PruneOldPackets(days int) (int64, error) {
n, _ := res.RowsAffected()
return n, tx.Commit()
}
// MetricsSample represents a single row from observer_metrics with computed deltas.
type MetricsSample struct {
Timestamp string `json:"timestamp"`
NoiseFloor *float64 `json:"noise_floor"`
TxAirSecs *int `json:"tx_air_secs,omitempty"`
RxAirSecs *int `json:"rx_air_secs,omitempty"`
RecvErrors *int `json:"recv_errors,omitempty"`
BatteryMv *int `json:"battery_mv"`
PacketsSent *int `json:"packets_sent,omitempty"`
PacketsRecv *int `json:"packets_recv,omitempty"`
TxAirtimePct *float64 `json:"tx_airtime_pct"`
RxAirtimePct *float64 `json:"rx_airtime_pct"`
RecvErrorRate *float64 `json:"recv_error_rate"`
IsReboot bool `json:"is_reboot_sample,omitempty"`
}
// rawMetricsSample is the raw DB row before delta computation.
type rawMetricsSample struct {
Timestamp string
NoiseFloor *float64
TxAirSecs *int
RxAirSecs *int
RecvErrors *int
BatteryMv *int
PacketsSent *int
PacketsRecv *int
}
// GetObserverMetrics returns time-series metrics with server-side delta computation.
// resolution: "5m" (raw), "1h", "1d"
// sampleIntervalSec: expected interval between samples (default 300)
func (db *DB) GetObserverMetrics(observerID, since, until, resolution string, sampleIntervalSec int) ([]MetricsSample, []string, error) {
if sampleIntervalSec <= 0 {
sampleIntervalSec = 300
}
// Build query based on resolution
var query string
args := []interface{}{observerID}
// Determine the effective bucket size for gap threshold scaling.
// For raw data (5m), use sampleIntervalSec. For aggregated resolutions,
// use the bucket duration so consecutive buckets aren't treated as gaps.
bucketSizeSec := sampleIntervalSec
switch resolution {
case "1h":
bucketSizeSec = 3600
// Use LAST value per bucket (latest timestamp) instead of MAX to preserve
// reboot semantics: if a device reboots mid-bucket, the last sample is the
// post-reboot baseline, not the pre-reboot high-water mark.
query = `SELECT ts, noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv FROM (
SELECT
strftime('%Y-%m-%dT%H:00:00Z', timestamp) as ts,
noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv,
ROW_NUMBER() OVER (PARTITION BY observer_id, strftime('%Y-%m-%dT%H:00:00Z', timestamp) ORDER BY timestamp DESC) as rn
FROM observer_metrics WHERE observer_id = ?`
case "1d":
bucketSizeSec = 86400
query = `SELECT ts, noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv FROM (
SELECT
strftime('%Y-%m-%dT00:00:00Z', timestamp) as ts,
noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv,
ROW_NUMBER() OVER (PARTITION BY observer_id, strftime('%Y-%m-%dT00:00:00Z', timestamp) ORDER BY timestamp DESC) as rn
FROM observer_metrics WHERE observer_id = ?`
default: // "5m" or raw
query = `SELECT timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv, packets_sent, packets_recv
FROM observer_metrics WHERE observer_id = ?`
}
if since != "" {
query += " AND timestamp >= ?"
args = append(args, since)
}
if until != "" {
query += " AND timestamp <= ?"
args = append(args, until)
}
switch resolution {
case "1h", "1d":
query += ") WHERE rn = 1 ORDER BY ts ASC"
default:
query += " ORDER BY timestamp ASC"
}
rows, err := db.conn.Query(query, args...)
if err != nil {
return nil, nil, err
}
defer rows.Close()
var raw []rawMetricsSample
for rows.Next() {
var s rawMetricsSample
if err := rows.Scan(&s.Timestamp, &s.NoiseFloor, &s.TxAirSecs, &s.RxAirSecs, &s.RecvErrors, &s.BatteryMv, &s.PacketsSent, &s.PacketsRecv); err != nil {
return nil, nil, err
}
raw = append(raw, s)
}
if err := rows.Err(); err != nil {
return nil, nil, err
}
// Compute deltas between consecutive samples.
// bucketSizeSec determines gap threshold: for raw data it's sampleIntervalSec,
// for aggregated resolutions it's the bucket duration (3600 for 1h, 86400 for 1d).
return computeDeltas(raw, bucketSizeSec)
}
// computeDeltas computes per-interval rates from cumulative counters.
// Handles reboots (counter reset) and gaps (missing samples).
// bucketSizeSec is the expected interval between consecutive points
// (sampleInterval for raw data, bucket duration for aggregated resolutions).
func computeDeltas(raw []rawMetricsSample, bucketSizeSec int) ([]MetricsSample, []string, error) {
if len(raw) == 0 {
return nil, nil, nil
}
gapThreshold := float64(bucketSizeSec) * 2.0
result := make([]MetricsSample, 0, len(raw))
var reboots []string
for i, cur := range raw {
s := MetricsSample{
Timestamp: cur.Timestamp,
NoiseFloor: cur.NoiseFloor,
BatteryMv: cur.BatteryMv,
}
if i == 0 {
// First sample: no delta possible
result = append(result, s)
continue
}
prev := raw[i-1]
// Check for gap
curT, err1 := time.Parse(time.RFC3339, cur.Timestamp)
prevT, err2 := time.Parse(time.RFC3339, prev.Timestamp)
if err1 != nil || err2 != nil {
result = append(result, s)
continue
}
intervalSecs := curT.Sub(prevT).Seconds()
if intervalSecs > gapThreshold {
// Gap detected: insert null deltas (don't interpolate)
result = append(result, s)
continue
}
if intervalSecs <= 0 {
result = append(result, s)
continue
}
// Detect reboot: any cumulative counter decreased
isReboot := false
if cur.TxAirSecs != nil && prev.TxAirSecs != nil && *cur.TxAirSecs < *prev.TxAirSecs {
isReboot = true
}
if cur.RxAirSecs != nil && prev.RxAirSecs != nil && *cur.RxAirSecs < *prev.RxAirSecs {
isReboot = true
}
if cur.RecvErrors != nil && prev.RecvErrors != nil && *cur.RecvErrors < *prev.RecvErrors {
isReboot = true
}
if cur.PacketsSent != nil && prev.PacketsSent != nil && *cur.PacketsSent < *prev.PacketsSent {
isReboot = true
}
if cur.PacketsRecv != nil && prev.PacketsRecv != nil && *cur.PacketsRecv < *prev.PacketsRecv {
isReboot = true
}
if isReboot {
s.IsReboot = true
reboots = append(reboots, cur.Timestamp)
// Skip delta computation for reboot samples — use as new baseline
result = append(result, s)
continue
}
// Compute TX airtime percentage
if cur.TxAirSecs != nil && prev.TxAirSecs != nil {
delta := float64(*cur.TxAirSecs - *prev.TxAirSecs)
pct := (delta / intervalSecs) * 100.0
if pct < 0 {
pct = 0
}
if pct > 100 {
pct = 100
}
result_pct := math.Round(pct*100) / 100
s.TxAirtimePct = &result_pct
}
// Compute RX airtime percentage
if cur.RxAirSecs != nil && prev.RxAirSecs != nil {
delta := float64(*cur.RxAirSecs - *prev.RxAirSecs)
pct := (delta / intervalSecs) * 100.0
if pct < 0 {
pct = 0
}
if pct > 100 {
pct = 100
}
result_pct := math.Round(pct*100) / 100
s.RxAirtimePct = &result_pct
}
// Compute recv error rate
if cur.RecvErrors != nil && prev.RecvErrors != nil &&
cur.PacketsRecv != nil && prev.PacketsRecv != nil {
deltaErrors := float64(*cur.RecvErrors - *prev.RecvErrors)
deltaRecv := float64(*cur.PacketsRecv - *prev.PacketsRecv)
total := deltaRecv + deltaErrors
if total > 0 {
rate := (deltaErrors / total) * 100.0
rate = math.Round(rate*100) / 100
s.RecvErrorRate = &rate
}
}
result = append(result, s)
}
return result, reboots, nil
}
// MetricsSummaryRow holds summary data for one observer.
type MetricsSummaryRow struct {
ObserverID string `json:"observer_id"`
ObserverName *string `json:"observer_name"`
IATA string `json:"iata,omitempty"`
CurrentNF *float64 `json:"current_noise_floor"`
AvgNF *float64 `json:"avg_noise_floor_24h"`
MaxNF *float64 `json:"max_noise_floor_24h"`
CurrentBattMv *int `json:"battery_mv"`
SampleCount int `json:"sample_count"`
Sparkline []*float64 `json:"sparkline"`
}
// GetMetricsSummary returns a fleet summary of observer metrics within a time window.
// Uses a CTE with ROW_NUMBER to get latest values in a single pass (no correlated subqueries).
// Also returns sparkline data (noise_floor time series) per observer.
func (db *DB) GetMetricsSummary(since string) ([]MetricsSummaryRow, error) {
query := `
WITH ranked AS (
SELECT observer_id, noise_floor, battery_mv,
ROW_NUMBER() OVER (PARTITION BY observer_id ORDER BY timestamp DESC) as rn
FROM observer_metrics
WHERE timestamp >= ?
)
SELECT m.observer_id, o.name, COALESCE(o.iata, '') as iata,
r.noise_floor as current_nf,
AVG(m.noise_floor) as avg_nf,
MAX(m.noise_floor) as max_nf,
r.battery_mv as current_batt,
COUNT(*) as sample_count
FROM observer_metrics m
LEFT JOIN observers o ON o.id = m.observer_id
LEFT JOIN ranked r ON r.observer_id = m.observer_id AND r.rn = 1
WHERE m.timestamp >= ?
GROUP BY m.observer_id
ORDER BY max_nf DESC
`
rows, err := db.conn.Query(query, since, since)
if err != nil {
return nil, err
}
defer rows.Close()
var result []MetricsSummaryRow
for rows.Next() {
var s MetricsSummaryRow
if err := rows.Scan(&s.ObserverID, &s.ObserverName, &s.IATA, &s.CurrentNF, &s.AvgNF, &s.MaxNF, &s.CurrentBattMv, &s.SampleCount); err != nil {
return nil, err
}
result = append(result, s)
}
if err := rows.Err(); err != nil {
return nil, err
}
// Fetch sparkline data (noise_floor series) for all observers in one query
if len(result) > 0 {
sparkQuery := `SELECT observer_id, noise_floor FROM observer_metrics
WHERE timestamp >= ? ORDER BY observer_id, timestamp ASC`
sparkRows, err := db.conn.Query(sparkQuery, since)
if err != nil {
return nil, err
}
defer sparkRows.Close()
sparkMap := make(map[string][]*float64)
for sparkRows.Next() {
var oid string
var nf *float64
if err := sparkRows.Scan(&oid, &nf); err != nil {
return nil, err
}
sparkMap[oid] = append(sparkMap[oid], nf)
}
if err := sparkRows.Err(); err != nil {
return nil, err
}
for i := range result {
if s, ok := sparkMap[result[i].ObserverID]; ok {
result[i].Sparkline = s
}
}
}
return result, nil
}
// PruneOldMetrics deletes observer_metrics rows older than retentionDays.
func (db *DB) PruneOldMetrics(retentionDays int) (int64, error) {
dsn := fmt.Sprintf("file:%s?_journal_mode=WAL&_busy_timeout=10000", db.path)
rw, err := sql.Open("sqlite", dsn)
if err != nil {
return 0, err
}
rw.SetMaxOpenConns(1)
defer rw.Close()
cutoff := time.Now().UTC().AddDate(0, 0, -retentionDays).Format(time.RFC3339)
res, err := rw.Exec(`DELETE FROM observer_metrics WHERE timestamp < ?`, cutoff)
if err != nil {
return 0, err
}
n, _ := res.RowsAffected()
if n > 0 {
log.Printf("[metrics] Pruned %d observer_metrics rows older than %d days", n, retentionDays)
}
return n, nil
}
+389 -8
View File
@@ -72,15 +72,31 @@ func setupTestDB(t *testing.T) *DB {
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp INTEGER NOT NULL
timestamp INTEGER NOT NULL,
resolved_path TEXT
);
CREATE TABLE IF NOT EXISTS observer_metrics (
observer_id TEXT NOT NULL,
timestamp TEXT NOT NULL,
noise_floor REAL,
tx_air_secs INTEGER,
rx_air_secs INTEGER,
recv_errors INTEGER,
battery_mv INTEGER,
packets_sent INTEGER,
packets_recv INTEGER,
PRIMARY KEY (observer_id, timestamp)
);
CREATE INDEX IF NOT EXISTS idx_observer_metrics_timestamp ON observer_metrics(timestamp);
`
if _, err := conn.Exec(schema); err != nil {
t.Fatal(err)
}
return &DB{conn: conn, isV3: true}
return &DB{conn: conn, isV3: true, hasResolvedPath: true}
}
func seedTestData(t *testing.T, db *DB) {
@@ -117,14 +133,15 @@ func seedTestData(t *testing.T, db *DB) {
VALUES ('AA1F', 'def456abc1230099', ?, 1, 4, '{"pubKey":"aabbccdd11223344","name":"TestRepeater","type":"ADVERT","timestamp":1700000100,"timestampISO":"2023-11-14T22:14:40.000Z","signature":"fedcba","flags":{"isRepeater":true},"lat":37.5,"lon":-122.0}')`, yesterday)
// Seed observations (use unix timestamps)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 12.5, -90, '["aa","bb"]', ?)`, recentEpoch)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 2, 8.0, -95, '["aa"]', ?)`, recentEpoch-100)
// resolved_path contains full pubkeys parallel to path_json hops
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp, resolved_path)
VALUES (1, 1, 12.5, -90, '["aa","bb"]', ?, '["aabbccdd11223344","eeff00112233aabb"]')`, recentEpoch)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp, resolved_path)
VALUES (1, 2, 8.0, -95, '["aa"]', ?, '["aabbccdd11223344"]')`, recentEpoch-100)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (2, 1, 15.0, -85, '[]', ?)`, yesterdayEpoch)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (3, 1, 10.0, -92, '["cc"]', ?)`, yesterdayEpoch)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp, resolved_path)
VALUES (3, 1, 10.0, -92, '["cc"]', ?, '["1122334455667788"]')`, yesterdayEpoch)
}
func TestGetStats(t *testing.T) {
@@ -1537,3 +1554,367 @@ func TestNodeTelemetryFields(t *testing.T) {
func TestMain(m *testing.M) {
os.Exit(m.Run())
}
func TestGetObserverMetrics(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
now := time.Now().UTC()
t1 := now.Add(-2 * time.Hour).Format(time.RFC3339)
t2 := now.Add(-1 * time.Hour).Format(time.RFC3339)
t3 := now.Format(time.RFC3339)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, battery_mv) VALUES (?, ?, ?, ?, ?, ?, ?)",
"obs1", t1, -112.5, 100, 500, 3, 3720)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors) VALUES (?, ?, ?, ?, ?, ?)",
"obs1", t2, -110.0, 200, 800, 5)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors) VALUES (?, ?, ?, ?, ?, ?)",
"obs1", t3, -108.0, 300, 1100, 8)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)",
"obs2", t1, -115.0)
// Query all for obs1
since := now.Add(-3 * time.Hour).Format(time.RFC3339)
metrics, reboots, err := db.GetObserverMetrics("obs1", since, "", "5m", 3600)
if err != nil {
t.Fatal(err)
}
if len(metrics) != 3 {
t.Errorf("expected 3 metrics, got %d", len(metrics))
}
if len(reboots) != 0 {
t.Errorf("expected 0 reboots, got %d", len(reboots))
}
// Verify first row has noise_floor
if metrics[0].NoiseFloor == nil || *metrics[0].NoiseFloor != -112.5 {
t.Errorf("first noise_floor = %v, want -112.5", metrics[0].NoiseFloor)
}
// First row: no delta possible (first sample)
if metrics[0].TxAirtimePct != nil {
t.Errorf("first sample should have nil tx_airtime_pct, got %v", *metrics[0].TxAirtimePct)
}
// Second row should have computed deltas
// TX: (200-100) / 3600 * 100 ≈ 2.78%
if metrics[1].TxAirtimePct == nil {
t.Errorf("second sample tx_airtime_pct should not be nil")
} else if *metrics[1].TxAirtimePct < 2.0 || *metrics[1].TxAirtimePct > 3.5 {
t.Errorf("second sample tx_airtime_pct = %v, want ~2.78", *metrics[1].TxAirtimePct)
}
// Query with until filter
metrics2, _, err := db.GetObserverMetrics("obs1", since, t2, "5m", 3600)
if err != nil {
t.Fatal(err)
}
if len(metrics2) != 2 {
t.Errorf("expected 2 metrics with until filter, got %d", len(metrics2))
}
}
func TestGetMetricsSummary(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
now := time.Now().UTC()
t1 := now.Add(-2 * time.Hour).Format(time.RFC3339)
t2 := now.Add(-1 * time.Hour).Format(time.RFC3339)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, battery_mv) VALUES (?, ?, ?, ?)",
"obs1", t1, -112.0, 3720)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)",
"obs1", t2, -108.0)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)",
"obs2", t1, -115.0)
since := now.Add(-24 * time.Hour).Format(time.RFC3339)
summary, err := db.GetMetricsSummary(since)
if err != nil {
t.Fatal(err)
}
if len(summary) != 2 {
t.Fatalf("expected 2 observers in summary, got %d", len(summary))
}
// Results sorted by max_nf DESC
// obs1 has max -108, obs2 has max -115
if summary[0].ObserverID != "obs1" {
t.Errorf("first observer should be obs1 (highest max NF), got %s", summary[0].ObserverID)
}
if summary[0].CurrentNF == nil || *summary[0].CurrentNF != -108.0 {
t.Errorf("obs1 current NF = %v, want -108.0", summary[0].CurrentNF)
}
if summary[0].SampleCount != 2 {
t.Errorf("obs1 sample count = %d, want 2", summary[0].SampleCount)
}
// Verify sparkline data is included
if len(summary[0].Sparkline) != 2 {
t.Errorf("obs1 sparkline length = %d, want 2", len(summary[0].Sparkline))
}
if len(summary[1].Sparkline) != 1 {
t.Errorf("obs2 sparkline length = %d, want 1", len(summary[1].Sparkline))
}
// Sparkline should be ordered by timestamp ASC
if summary[0].Sparkline[0] != nil && *summary[0].Sparkline[0] != -112.0 {
t.Errorf("obs1 sparkline[0] = %v, want -112.0", *summary[0].Sparkline[0])
}
if summary[0].Sparkline[1] != nil && *summary[0].Sparkline[1] != -108.0 {
t.Errorf("obs1 sparkline[1] = %v, want -108.0", *summary[0].Sparkline[1])
}
}
func TestObserverMetricsAPIEndpoints(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
now := time.Now().UTC()
t1 := now.Add(-1 * time.Hour).Format(time.RFC3339)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)",
"obs1", t1, -112.0)
// Query directly to verify
metrics, _, err := db.GetObserverMetrics("obs1", "", "", "5m", 300)
if err != nil {
t.Fatal(err)
}
if len(metrics) != 1 {
t.Errorf("expected 1 metric, got %d", len(metrics))
}
}
func TestComputeDeltas(t *testing.T) {
intPtr := func(v int) *int { return &v }
floatPtr := func(v float64) *float64 { return &v }
t.Run("empty input", func(t *testing.T) {
result, reboots, err := computeDeltas(nil, 300)
if err != nil {
t.Fatal(err)
}
if result != nil {
t.Errorf("expected nil, got %v", result)
}
if reboots != nil {
t.Errorf("expected nil reboots, got %v", reboots)
}
})
t.Run("normal delta computation", func(t *testing.T) {
raw := []rawMetricsSample{
{Timestamp: "2026-04-05T00:00:00Z", NoiseFloor: floatPtr(-112), TxAirSecs: intPtr(100), RxAirSecs: intPtr(500), RecvErrors: intPtr(3), PacketsRecv: intPtr(1000)},
{Timestamp: "2026-04-05T00:05:00Z", NoiseFloor: floatPtr(-110), TxAirSecs: intPtr(115), RxAirSecs: intPtr(525), RecvErrors: intPtr(5), PacketsRecv: intPtr(1100)},
}
result, reboots, err := computeDeltas(raw, 300)
if err != nil {
t.Fatal(err)
}
if len(result) != 2 {
t.Fatalf("expected 2 results, got %d", len(result))
}
if len(reboots) != 0 {
t.Errorf("expected 0 reboots, got %d", len(reboots))
}
// First sample: no deltas
if result[0].TxAirtimePct != nil {
t.Errorf("first sample should have nil tx_airtime_pct")
}
// Second sample: TX delta = 15 secs / 300 secs * 100 = 5%
if result[1].TxAirtimePct == nil {
t.Fatal("second sample tx_airtime_pct should not be nil")
}
if *result[1].TxAirtimePct != 5.0 {
t.Errorf("tx_airtime_pct = %v, want 5.0", *result[1].TxAirtimePct)
}
// RX delta = 25 secs / 300 secs * 100 ≈ 8.33%
if result[1].RxAirtimePct == nil {
t.Fatal("second sample rx_airtime_pct should not be nil")
}
if *result[1].RxAirtimePct < 8.3 || *result[1].RxAirtimePct > 8.4 {
t.Errorf("rx_airtime_pct = %v, want ~8.33", *result[1].RxAirtimePct)
}
// Error rate: delta_errors=2, delta_recv=100, rate = 2/(100+2)*100 ≈ 1.96%
if result[1].RecvErrorRate == nil {
t.Fatal("second sample recv_error_rate should not be nil")
}
if *result[1].RecvErrorRate < 1.9 || *result[1].RecvErrorRate > 2.0 {
t.Errorf("recv_error_rate = %v, want ~1.96", *result[1].RecvErrorRate)
}
})
t.Run("reboot detection", func(t *testing.T) {
raw := []rawMetricsSample{
{Timestamp: "2026-04-05T00:00:00Z", TxAirSecs: intPtr(1000), RxAirSecs: intPtr(5000)},
{Timestamp: "2026-04-05T00:05:00Z", TxAirSecs: intPtr(10), RxAirSecs: intPtr(20)}, // reboot!
{Timestamp: "2026-04-05T00:10:00Z", TxAirSecs: intPtr(25), RxAirSecs: intPtr(45)},
}
result, reboots, err := computeDeltas(raw, 300)
if err != nil {
t.Fatal(err)
}
if len(reboots) != 1 {
t.Fatalf("expected 1 reboot, got %d", len(reboots))
}
if reboots[0] != "2026-04-05T00:05:00Z" {
t.Errorf("reboot timestamp = %s", reboots[0])
}
if !result[1].IsReboot {
t.Error("second sample should be marked as reboot")
}
// Reboot sample should have nil deltas
if result[1].TxAirtimePct != nil {
t.Error("reboot sample should have nil tx_airtime_pct")
}
// Third sample should have valid deltas from post-reboot baseline
if result[2].TxAirtimePct == nil {
t.Fatal("third sample tx_airtime_pct should not be nil")
}
if *result[2].TxAirtimePct != 5.0 { // 15/300*100
t.Errorf("third sample tx_airtime_pct = %v, want 5.0", *result[2].TxAirtimePct)
}
})
t.Run("gap detection", func(t *testing.T) {
raw := []rawMetricsSample{
{Timestamp: "2026-04-05T00:00:00Z", TxAirSecs: intPtr(100)},
{Timestamp: "2026-04-05T00:15:00Z", TxAirSecs: intPtr(200)}, // 15min gap > 2*300s
}
result, _, err := computeDeltas(raw, 300)
if err != nil {
t.Fatal(err)
}
// Gap sample should have nil deltas
if result[1].TxAirtimePct != nil {
t.Error("gap sample should have nil tx_airtime_pct")
}
})
}
func TestGetObserverMetricsResolution(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs) VALUES (?, ?, ?, ?)",
"obs1", "2026-04-05T00:00:00Z", -112.0, 100)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs) VALUES (?, ?, ?, ?)",
"obs1", "2026-04-05T00:05:00Z", -110.0, 200)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs) VALUES (?, ?, ?, ?)",
"obs1", "2026-04-05T01:00:00Z", -108.0, 500)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs) VALUES (?, ?, ?, ?)",
"obs1", "2026-04-05T01:05:00Z", -106.0, 600)
// 5m resolution: all 4 rows
m5, _, err := db.GetObserverMetrics("obs1", "2026-04-04T00:00:00Z", "", "5m", 300)
if err != nil {
t.Fatal(err)
}
if len(m5) != 4 {
t.Errorf("5m resolution: expected 4 rows, got %d", len(m5))
}
// 1h resolution: 2 buckets
m1h, _, err := db.GetObserverMetrics("obs1", "2026-04-04T00:00:00Z", "", "1h", 300)
if err != nil {
t.Fatal(err)
}
if len(m1h) != 2 {
t.Errorf("1h resolution: expected 2 rows, got %d", len(m1h))
}
// 1d resolution: 1 bucket
m1d, _, err := db.GetObserverMetrics("obs1", "2026-04-04T00:00:00Z", "", "1d", 300)
if err != nil {
t.Fatal(err)
}
if len(m1d) != 1 {
t.Errorf("1d resolution: expected 1 row, got %d", len(m1d))
}
}
func TestHourlyResolutionDeltasNotNull(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
// Two hourly buckets, each with one sample. With old MAX+hardcoded gap threshold,
// the 3600s gap would exceed sampleInterval*2 (600s) and deltas would be null.
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_hr", "2026-04-05T10:00:00Z", -110.0, 100, 200, 5, 50, 100)
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_hr", "2026-04-05T11:00:00Z", -108.0, 200, 400, 10, 80, 200)
m, _, err := db.GetObserverMetrics("obs_hr", "2026-04-04T00:00:00Z", "", "1h", 300)
if err != nil {
t.Fatal(err)
}
if len(m) != 2 {
t.Fatalf("expected 2 rows, got %d", len(m))
}
// Second row should have computed deltas (not null)
if m[1].TxAirtimePct == nil {
t.Error("1h resolution: tx_airtime_pct should not be nil — gap threshold must scale with resolution")
}
}
func TestLastValuePreservesReboot(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
// Hour bucket with two samples: pre-reboot (high) and post-reboot (low).
// With MAX(), the pre-reboot value wins and the reboot is hidden.
// With LAST (latest timestamp), the post-reboot value wins.
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_rb", "2026-04-05T10:00:00Z", -110.0, 1000, 2000, 500, 400, 800) // pre-reboot baseline
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_rb", "2026-04-05T10:20:00Z", -110.0, 5000, 6000, 900, 700, 1200) // pre-reboot peak
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_rb", "2026-04-05T10:40:00Z", -110.0, 10, 20, 1, 5, 10) // post-reboot (counter reset)
// Next hour bucket
db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor, tx_air_secs, rx_air_secs, recv_errors, packets_sent, packets_recv) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
"obs_rb", "2026-04-05T11:00:00Z", -108.0, 100, 120, 5, 20, 50)
m, reboots, err := db.GetObserverMetrics("obs_rb", "2026-04-04T00:00:00Z", "", "1h", 300)
if err != nil {
t.Fatal(err)
}
if len(m) != 2 {
t.Fatalf("expected 2 rows, got %d", len(m))
}
// First bucket should use the LAST value (post-reboot: tx_air_secs=10).
// Second bucket (tx_air_secs=100) is a normal increase from 10→100.
// With LAST-value semantics, the second bucket should have valid deltas (not a reboot).
// With MAX(), first bucket would have tx_air_secs=5000, and second=100 would
// trigger a false reboot detection.
if m[1].IsReboot {
t.Error("second bucket should NOT be flagged as reboot with LAST-value aggregation")
}
if m[1].TxAirtimePct == nil {
t.Error("second bucket should have non-nil tx_airtime_pct")
}
_ = reboots // reboots list is informational
}
func TestParseWindowDuration(t *testing.T) {
tests := []struct {
input string
want time.Duration
err bool
}{
{"1h", time.Hour, false},
{"24h", 24 * time.Hour, false},
{"3d", 3 * 24 * time.Hour, false},
{"30d", 30 * 24 * time.Hour, false},
{"invalid", 0, true},
}
for _, tc := range tests {
got, err := parseWindowDuration(tc.input)
if tc.err && err == nil {
t.Errorf("parseWindowDuration(%q) expected error", tc.input)
}
if !tc.err && got != tc.want {
t.Errorf("parseWindowDuration(%q) = %v, want %v", tc.input, got, tc.want)
}
}
}
+33 -8
View File
@@ -9,6 +9,8 @@ import (
"math"
"strings"
"time"
"github.com/meshcore-analyzer/sigvalidate"
)
// Route type constants (header bits 1-0)
@@ -60,9 +62,10 @@ type TransportCodes struct {
// Path holds decoded path/hop information.
type Path struct {
HashSize int `json:"hashSize"`
HashCount int `json:"hashCount"`
Hops []string `json:"hops"`
HashSize int `json:"hashSize"`
HashCount int `json:"hashCount"`
Hops []string `json:"hops"`
HopsCompleted *int `json:"hopsCompleted,omitempty"`
}
// AdvertFlags holds decoded advert flag bits.
@@ -91,6 +94,7 @@ type Payload struct {
Timestamp uint32 `json:"timestamp,omitempty"`
TimestampISO string `json:"timestampISO,omitempty"`
Signature string `json:"signature,omitempty"`
SignatureValid *bool `json:"signatureValid,omitempty"`
Flags *AdvertFlags `json:"flags,omitempty"`
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
@@ -187,7 +191,7 @@ func decodeAck(buf []byte) Payload {
}
}
func decodeAdvert(buf []byte) Payload {
func decodeAdvert(buf []byte, validateSignatures bool) Payload {
if len(buf) < 100 {
return Payload{Type: "ADVERT", Error: "too short for advert", RawHex: hex.EncodeToString(buf)}
}
@@ -205,6 +209,11 @@ func decodeAdvert(buf []byte) Payload {
Signature: signature,
}
if validateSignatures {
valid := sigvalidate.ValidateAdvertSignature(buf[0:32], buf[36:100], timestamp, appdata)
p.SignatureValid = &valid
}
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
@@ -307,7 +316,7 @@ func decodeTrace(buf []byte) Payload {
return p
}
func decodePayload(payloadType int, buf []byte) Payload {
func decodePayload(payloadType int, buf []byte, validateSignatures bool) Payload {
switch payloadType {
case PayloadREQ:
return decodeEncryptedPayload("REQ", buf)
@@ -318,7 +327,7 @@ func decodePayload(payloadType int, buf []byte) Payload {
case PayloadACK:
return decodeAck(buf)
case PayloadADVERT:
return decodeAdvert(buf)
return decodeAdvert(buf, validateSignatures)
case PayloadGRP_TXT:
return decodeGrpTxt(buf)
case PayloadANON_REQ:
@@ -333,7 +342,7 @@ func decodePayload(payloadType int, buf []byte) Payload {
}
// DecodePacket decodes a hex-encoded MeshCore packet.
func DecodePacket(hexString string) (*DecodedPacket, error) {
func DecodePacket(hexString string, validateSignatures bool) (*DecodedPacket, error) {
hexString = strings.ReplaceAll(hexString, " ", "")
hexString = strings.ReplaceAll(hexString, "\n", "")
hexString = strings.ReplaceAll(hexString, "\r", "")
@@ -371,12 +380,17 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
offset += bytesConsumed
payloadBuf := buf[offset:]
payload := decodePayload(header.PayloadType, payloadBuf)
payload := decodePayload(header.PayloadType, payloadBuf, validateSignatures)
// TRACE packets store hop IDs in the payload (buf[9:]) rather than the header
// path field. The header path byte still encodes hashSize in bits 6-7, which
// we use to split the payload path data into individual hop prefixes.
// The header path contains SNR bytes — one per hop that actually forwarded.
// We expose hopsCompleted (count of SNR bytes) so consumers can distinguish
// how far the trace got vs the full intended route.
if header.PayloadType == PayloadTRACE && payload.PathData != "" {
// The header path hops count represents SNR entries = completed hops
hopsCompleted := path.HashCount
pathBytes, err := hex.DecodeString(payload.PathData)
if err == nil && path.HashSize > 0 {
hops := make([]string, 0, len(pathBytes)/path.HashSize)
@@ -385,9 +399,20 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
}
path.Hops = hops
path.HashCount = len(hops)
path.HopsCompleted = &hopsCompleted
}
}
// Zero-hop direct packets have hash_count=0 (lower 6 bits of pathByte),
// which makes the generic formula yield a bogus hashSize. Reset to 0
// (unknown) so API consumers get correct data. We mask with 0x3F to check
// only hash_count, matching the JS frontend approach — the upper hash_size
// bits are meaningless when there are no hops. Skip TRACE packets — they
// use hashSize to parse hops from the payload above.
if (header.RouteType == RouteDirect || header.RouteType == RouteTransportDirect) && pathByte&0x3F == 0 && header.PayloadType != PayloadTRACE {
path.HashSize = 0
}
return &DecodedPacket{
Header: header,
TransportCodes: tc,
+252 -2
View File
@@ -1,7 +1,11 @@
package main
import (
"crypto/ed25519"
"encoding/binary"
"testing"
"github.com/meshcore-analyzer/sigvalidate"
)
func TestDecodeHeader_TransportFlood(t *testing.T) {
@@ -65,7 +69,7 @@ func TestDecodePacket_TransportFloodHasCodes(t *testing.T) {
// Path byte: 0x00 (hashSize=1, hashCount=0)
// Payload: at least some bytes for GRP_TXT
hex := "14AABBCCDD00112233445566778899"
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -85,7 +89,7 @@ func TestDecodePacket_FloodHasNoCodes(t *testing.T) {
// Path byte: 0x00 (no hops)
// Some payload bytes
hex := "110011223344556677889900AABBCCDD"
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -235,6 +239,87 @@ func assertRange(t *testing.T, ranges []HexRange, label string, wantStart, wantE
t.Errorf("range %q not found in %v", label, rangeLabels(ranges))
}
func TestZeroHopDirectHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x00 → hash_count=0, hash_size bits=0 → should get HashSize=0
// Need at least a few payload bytes after pathByte.
hex := "02" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("DIRECT zero-hop: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x40 → hash_count=0, hash_size bits=01 → should still get HashSize=0
// because hash_count is zero (lower 6 bits are 0).
hex := "02" + "40" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("DIRECT zero-hop with hash_size bits set: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopTransportDirectHashSize(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0x00 → hash_count=0 → should get HashSize=0
hex := "03" + "11223344" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("TRANSPORT_DIRECT zero-hop: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopTransportDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0xC0 → hash_count=0, hash_size bits=11 → should still get HashSize=0
hex := "03" + "11223344" + "C0" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("TRANSPORT_DIRECT zero-hop with hash_size bits set: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestNonDirectZeroPathByteKeepsHashSize(t *testing.T) {
// FLOOD (RouteType=1) + REQ (PayloadType=0) → header byte = 0x01
// pathByte=0x00 → even though hash_count=0, non-DIRECT should keep HashSize=1
hex := "01" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 1 {
t.Errorf("FLOOD zero pathByte: want HashSize=1 (unchanged), got %d", pkt.Path.HashSize)
}
}
func TestDirectNonZeroHopKeepsHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x01 → hash_count=1, hash_size=1 → should keep HashSize=1
// Need 1 hop hash byte after pathByte.
hex := "02" + "01" + repeatHex("BB", 21)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 1 {
t.Errorf("DIRECT with 1 hop: want HashSize=1, got %d", pkt.Path.HashSize)
}
}
func repeatHex(byteHex string, n int) string {
s := ""
for i := 0; i < n; i++ {
@@ -242,3 +327,168 @@ func repeatHex(byteHex string, n int) string {
}
return s
}
func TestDecodePacket_TraceHopsCompleted(t *testing.T) {
// Build a TRACE packet:
// header: route=FLOOD(1), payload=TRACE(9), version=0 → (0<<6)|(9<<2)|1 = 0x25
// path_length: hash_size bits=0b00 (1-byte), hash_count=2 (2 SNR bytes) → 0x02
// path: 2 SNR bytes: 0xAA, 0xBB
// payload: tag(4 LE) + authCode(4 LE) + flags(1) + 4 hop hashes (1 byte each)
hex := "2502AABB" + // header + path_length + 2 SNR bytes
"01000000" + // tag = 1
"02000000" + // authCode = 2
"00" + // flags = 0
"DEADBEEF" // 4 hops (1-byte hash each)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if pkt.Payload.Type != "TRACE" {
t.Fatalf("expected TRACE, got %s", pkt.Payload.Type)
}
// Full intended route = 4 hops from payload
if len(pkt.Path.Hops) != 4 {
t.Errorf("expected 4 hops, got %d: %v", len(pkt.Path.Hops), pkt.Path.Hops)
}
// HopsCompleted = 2 (from header path SNR count)
if pkt.Path.HopsCompleted == nil {
t.Fatal("expected HopsCompleted to be set")
}
if *pkt.Path.HopsCompleted != 2 {
t.Errorf("expected HopsCompleted=2, got %d", *pkt.Path.HopsCompleted)
}
}
func TestDecodePacket_TraceNoSNR(t *testing.T) {
// TRACE with 0 SNR bytes (trace hasn't been forwarded yet)
// path_length: hash_size=0b00 (1-byte), hash_count=0 → 0x00
hex := "2500" + // header + path_length (0 hops in header)
"01000000" + // tag
"02000000" + // authCode
"00" + // flags
"AABBCC" // 3 hops intended
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if pkt.Path.HopsCompleted == nil {
t.Fatal("expected HopsCompleted to be set")
}
if *pkt.Path.HopsCompleted != 0 {
t.Errorf("expected HopsCompleted=0, got %d", *pkt.Path.HopsCompleted)
}
if len(pkt.Path.Hops) != 3 {
t.Errorf("expected 3 hops, got %d", len(pkt.Path.Hops))
}
}
func TestDecodePacket_TraceFullyCompleted(t *testing.T) {
// TRACE where all hops completed (SNR count = hop count)
// path_length: hash_size=0b00 (1-byte), hash_count=3 → 0x03
hex := "2503AABBCC" + // header + path_length + 3 SNR bytes
"01000000" + // tag
"02000000" + // authCode
"00" + // flags
"DDEEFF" // 3 hops intended
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if pkt.Path.HopsCompleted == nil {
t.Fatal("expected HopsCompleted to be set")
}
if *pkt.Path.HopsCompleted != 3 {
t.Errorf("expected HopsCompleted=3, got %d", *pkt.Path.HopsCompleted)
}
if len(pkt.Path.Hops) != 3 {
t.Errorf("expected 3 hops, got %d", len(pkt.Path.Hops))
}
}
func TestValidateAdvertSignature(t *testing.T) {
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1234567890
appdata := []byte{0x02, 0x11, 0x22}
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Valid signature
if !sigvalidate.ValidateAdvertSignature(pub, sig, timestamp, appdata) {
t.Error("expected valid signature")
}
// Tampered appdata
if sigvalidate.ValidateAdvertSignature(pub, sig, timestamp, []byte{0x03, 0x11, 0x22}) {
t.Error("expected invalid with tampered appdata")
}
// Wrong timestamp
if sigvalidate.ValidateAdvertSignature(pub, sig, timestamp+1, appdata) {
t.Error("expected invalid with wrong timestamp")
}
// Short pubkey
if sigvalidate.ValidateAdvertSignature([]byte{0xAA}, sig, timestamp, appdata) {
t.Error("expected false for short pubkey")
}
// Short signature
if sigvalidate.ValidateAdvertSignature(pub, []byte{0xBB}, timestamp, appdata) {
t.Error("expected false for short signature")
}
}
func TestDecodeAdvertWithSignatureValidation(t *testing.T) {
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1000000
appdata := []byte{0x02} // repeater type
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Build advert buffer: pubkey(32) + timestamp(4) + signature(64) + appdata
buf := make([]byte, 0, 101)
buf = append(buf, pub...)
ts := make([]byte, 4)
binary.LittleEndian.PutUint32(ts, timestamp)
buf = append(buf, ts...)
buf = append(buf, sig...)
buf = append(buf, appdata...)
// With validation
p := decodeAdvert(buf, true)
if p.Error != "" {
t.Fatalf("decode error: %s", p.Error)
}
if p.SignatureValid == nil {
t.Fatal("SignatureValid should be set when validation enabled")
}
if !*p.SignatureValid {
t.Error("expected valid signature")
}
// Without validation
p2 := decodeAdvert(buf, false)
if p2.SignatureValid != nil {
t.Error("SignatureValid should be nil when validation disabled")
}
}
+26
View File
@@ -276,3 +276,29 @@ func TestNewPacketStoreNilConfig(t *testing.T) {
t.Fatalf("expected retentionHours=0, got %f", store.retentionHours)
}
}
func TestCacheTTLFromConfig(t *testing.T) {
// With config values: analyticsHashSizes and analyticsRF should override defaults.
cacheTTL := map[string]interface{}{
"analyticsHashSizes": float64(7200),
"analyticsRF": float64(300),
}
store := NewPacketStore(nil, nil, cacheTTL)
if store.collisionCacheTTL != 7200*time.Second {
t.Fatalf("expected collisionCacheTTL=7200s, got %v", store.collisionCacheTTL)
}
if store.rfCacheTTL != 300*time.Second {
t.Fatalf("expected rfCacheTTL=300s, got %v", store.rfCacheTTL)
}
}
func TestCacheTTLDefaults(t *testing.T) {
// Without config, defaults should apply.
store := NewPacketStore(nil, nil)
if store.collisionCacheTTL != 3600*time.Second {
t.Fatalf("expected default collisionCacheTTL=3600s, got %v", store.collisionCacheTTL)
}
if store.rfCacheTTL != 15*time.Second {
t.Fatalf("expected default rfCacheTTL=15s, got %v", store.rfCacheTTL)
}
}
+3
View File
@@ -6,11 +6,14 @@ require (
github.com/gorilla/mux v1.8.1
github.com/gorilla/websocket v1.5.3
github.com/meshcore-analyzer/geofilter v0.0.0
github.com/meshcore-analyzer/sigvalidate v0.0.0-00010101000000-000000000000
modernc.org/sqlite v1.34.5
)
replace github.com/meshcore-analyzer/geofilter => ../../internal/geofilter
replace github.com/meshcore-analyzer/sigvalidate => ../../internal/sigvalidate
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
+137 -22
View File
@@ -104,6 +104,8 @@ func main() {
}
if cfg.APIKey == "" {
log.Printf("[security] WARNING: no apiKey configured — write endpoints are BLOCKED (set apiKey in config.json to enable them)")
} else if IsWeakAPIKey(cfg.APIKey) {
log.Printf("[security] WARNING: API key is weak or a known default — write endpoints are vulnerable")
}
// Resolve DB path
@@ -139,7 +141,7 @@ func main() {
}
// In-memory packet store
store := NewPacketStore(database, cfg.PacketStore)
store := NewPacketStore(database, cfg.PacketStore, cfg.CacheTTL)
if err := store.Load(); err != nil {
log.Fatalf("[store] failed to load: %v", err)
}
@@ -153,7 +155,7 @@ func main() {
// NOTE on startup ordering (review item #10): ensureResolvedPathColumn runs AFTER
// OpenDB/detectSchema, so db.hasResolvedPath will be false on first run with a
// pre-existing DB. This means Load() won't SELECT resolved_path from SQLite.
// That's OK: backfillResolvedPaths (below) computes and persists them in-memory
// Async backfill runs after HTTP starts (see backfillResolvedPathsAsync below)
// AND to SQLite. On next restart, detectSchema finds the column and Load() reads it.
if err := ensureResolvedPathColumn(dbPath); err != nil {
log.Printf("[store] warning: could not add resolved_path column: %v", err)
@@ -166,27 +168,59 @@ func main() {
store.graph = loadNeighborEdgesFromDB(database.conn)
log.Printf("[neighbor] loaded persisted neighbor graph")
} else {
log.Printf("[neighbor] no persisted edges found, building from store...")
rw, rwErr := openRW(dbPath)
if rwErr == nil {
edgeCount := buildAndPersistEdges(store, rw)
rw.Close()
log.Printf("[neighbor] persisted %d edges", edgeCount)
log.Printf("[neighbor] no persisted edges found, will build in background...")
store.graph = NewNeighborGraph() // empty graph — gets populated by background goroutine
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("[neighbor] graph build panic recovered: %v", r)
}
}()
rw, rwErr := openRW(dbPath)
if rwErr == nil {
edgeCount := buildAndPersistEdges(store, rw)
rw.Close()
log.Printf("[neighbor] persisted %d edges", edgeCount)
}
built := BuildFromStore(store)
store.mu.Lock()
store.graph = built
store.mu.Unlock()
log.Printf("[neighbor] graph build complete")
}()
}
// Initial pickBestObservation runs in background — doesn't need to block HTTP.
// API serves best-effort data until this completes (~10s for 100K txs).
// Processes in chunks of 5000, releasing the lock between chunks so API
// handlers remain responsive.
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("[store] pickBestObservation panic recovered: %v", r)
}
}()
const chunkSize = 5000
store.mu.RLock()
totalPackets := len(store.packets)
store.mu.RUnlock()
for i := 0; i < totalPackets; i += chunkSize {
end := i + chunkSize
if end > totalPackets {
end = totalPackets
}
store.mu.Lock()
for j := i; j < end && j < len(store.packets); j++ {
pickBestObservation(store.packets[j])
}
store.mu.Unlock()
if end < totalPackets {
time.Sleep(10 * time.Millisecond) // yield to API handlers
}
}
store.graph = BuildFromStore(store)
}
// Backfill resolved_path for observations that don't have it yet
if backfilled := backfillResolvedPaths(store, dbPath); backfilled > 0 {
log.Printf("[store] backfilled resolved_path for %d observations", backfilled)
}
// Re-pick best observation now that resolved paths are populated
store.mu.Lock()
for _, tx := range store.packets {
pickBestObservation(tx)
}
store.mu.Unlock()
log.Printf("[store] initial pickBestObservation complete (%d transmissions)", totalPackets)
}()
// WebSocket hub
hub := NewHub()
@@ -234,6 +268,11 @@ func main() {
close(pruneDone)
}
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("[prune] panic recovered: %v", r)
}
}()
time.Sleep(1 * time.Minute)
if n, err := database.PruneOldPackets(days); err != nil {
log.Printf("[prune] error: %v", err)
@@ -256,6 +295,72 @@ func main() {
log.Printf("[prune] auto-prune enabled: packets older than %d days will be removed daily", days)
}
// Auto-prune old metrics
var stopMetricsPrune func()
{
metricsDays := cfg.MetricsRetentionDays()
metricsPruneTicker := time.NewTicker(24 * time.Hour)
metricsPruneDone := make(chan struct{})
stopMetricsPrune = func() {
metricsPruneTicker.Stop()
close(metricsPruneDone)
}
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("[metrics-prune] panic recovered: %v", r)
}
}()
time.Sleep(2 * time.Minute) // stagger after packet prune
database.PruneOldMetrics(metricsDays)
for {
select {
case <-metricsPruneTicker.C:
database.PruneOldMetrics(metricsDays)
case <-metricsPruneDone:
return
}
}
}()
log.Printf("[metrics-prune] auto-prune enabled: metrics older than %d days", metricsDays)
}
// Auto-prune old neighbor edges
var stopEdgePrune func()
{
maxAgeDays := cfg.NeighborMaxAgeDays()
edgePruneTicker := time.NewTicker(24 * time.Hour)
edgePruneDone := make(chan struct{})
stopEdgePrune = func() {
edgePruneTicker.Stop()
close(edgePruneDone)
}
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("[neighbor-prune] panic recovered: %v", r)
}
}()
time.Sleep(4 * time.Minute) // stagger after metrics prune
store.mu.RLock()
g := store.graph
store.mu.RUnlock()
PruneNeighborEdges(dbPath, g, maxAgeDays)
for {
select {
case <-edgePruneTicker.C:
store.mu.RLock()
g := store.graph
store.mu.RUnlock()
PruneNeighborEdges(dbPath, g, maxAgeDays)
case <-edgePruneDone:
return
}
}
}()
log.Printf("[neighbor-prune] auto-prune enabled: edges older than %d days", maxAgeDays)
}
// Graceful shutdown
httpServer := &http.Server{
Addr: fmt.Sprintf(":%d", cfg.Port),
@@ -278,6 +383,12 @@ func main() {
if stopPrune != nil {
stopPrune()
}
if stopMetricsPrune != nil {
stopMetricsPrune()
}
if stopEdgePrune != nil {
stopEdgePrune()
}
// 2. Gracefully drain HTTP connections (up to 15s)
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
@@ -297,6 +408,10 @@ func main() {
}()
log.Printf("[server] CoreScope (Go) listening on http://localhost:%d", cfg.Port)
// Start async backfill in background — HTTP is now available.
go backfillResolvedPathsAsync(store, dbPath, 5000, 100*time.Millisecond, cfg.BackfillHours())
if err := httpServer.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("[server] %v", err)
}
+25 -14
View File
@@ -20,19 +20,20 @@ type NeighborResponse struct {
}
type NeighborEntry struct {
Pubkey *string `json:"pubkey"`
Prefix string `json:"prefix"`
Name *string `json:"name"`
Role *string `json:"role"`
Count int `json:"count"`
Score float64 `json:"score"`
FirstSeen string `json:"first_seen"`
LastSeen string `json:"last_seen"`
AvgSNR *float64 `json:"avg_snr"`
Observers []string `json:"observers"`
Ambiguous bool `json:"ambiguous"`
Unresolved bool `json:"unresolved,omitempty"`
Candidates []CandidateEntry `json:"candidates,omitempty"`
Pubkey *string `json:"pubkey"`
Prefix string `json:"prefix"`
Name *string `json:"name"`
Role *string `json:"role"`
Count int `json:"count"`
Score float64 `json:"score"`
FirstSeen string `json:"first_seen"`
LastSeen string `json:"last_seen"`
AvgSNR *float64 `json:"avg_snr"`
DistanceKm *float64 `json:"distance_km,omitempty"`
Observers []string `json:"observers"`
Ambiguous bool `json:"ambiguous"`
Unresolved bool `json:"unresolved,omitempty"`
Candidates []CandidateEntry `json:"candidates,omitempty"`
}
type CandidateEntry struct {
@@ -115,9 +116,15 @@ func (s *Server) handleNodeNeighbors(w http.ResponseWriter, r *http.Request) {
edges := graph.Neighbors(pubkey)
now := time.Now()
// Build node info lookup for names/roles.
// Build node info lookup for names/roles/coordinates.
nodeMap := s.buildNodeInfoMap()
// Look up the queried node's GPS coordinates for distance computation.
var srcInfo nodeInfo
if nodeMap != nil {
srcInfo = nodeMap[pubkey]
}
var entries []NeighborEntry
totalObs := 0
@@ -170,6 +177,10 @@ func (s *Server) handleNodeNeighbors(w http.ResponseWriter, r *http.Request) {
if info, ok := nodeMap[strings.ToLower(neighborPK)]; ok {
entry.Name = &info.Name
entry.Role = &info.Role
if srcInfo.HasGPS && info.HasGPS {
d := haversineKm(srcInfo.Lat, srcInfo.Lon, info.Lat, info.Lon)
entry.DistanceKm = &d
}
}
}
+63
View File
@@ -347,6 +347,69 @@ func TestNeighborGraphAPI_AmbiguousEdgesCount(t *testing.T) {
}
}
func TestNeighborAPI_DistanceKm_WithGPS(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen)
VALUES ('aaaa', 'NodeA', 'repeater', 51.5074, -0.1278, '2026-01-01T00:00:00Z', '2025-01-01T00:00:00Z')`)
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen)
VALUES ('bbbb', 'NodeB', 'repeater', 51.5200, -0.1200, '2026-01-01T00:00:00Z', '2025-01-01T00:00:00Z')`)
cfg := &Config{Port: 3000}
hub := NewHub()
srv := NewServer(db, cfg, hub)
srv.store = NewPacketStore(db, nil)
now := time.Now()
srv.neighborGraph = makeTestGraph(newEdge("aaaa", "bbbb", "bb", 50, now))
rr := serveRequest(srv, "GET", "/api/nodes/aaaa/neighbors")
var resp NeighborResponse
json.Unmarshal(rr.Body.Bytes(), &resp)
if len(resp.Neighbors) != 1 {
t.Fatalf("expected 1 neighbor, got %d", len(resp.Neighbors))
}
n := resp.Neighbors[0]
if n.DistanceKm == nil {
t.Fatal("expected distance_km to be set for GPS-enabled nodes")
}
if *n.DistanceKm <= 0 {
t.Errorf("expected positive distance, got %f", *n.DistanceKm)
}
}
func TestNeighborAPI_DistanceKm_NoGPS(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Nodes with 0,0 coords → HasGPS=false
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen)
VALUES ('aaaa', 'NodeA', 'repeater', 0, 0, '2026-01-01T00:00:00Z', '2025-01-01T00:00:00Z')`)
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen)
VALUES ('bbbb', 'NodeB', 'repeater', 0, 0, '2026-01-01T00:00:00Z', '2025-01-01T00:00:00Z')`)
cfg := &Config{Port: 3000}
hub := NewHub()
srv := NewServer(db, cfg, hub)
srv.store = NewPacketStore(db, nil)
now := time.Now()
srv.neighborGraph = makeTestGraph(newEdge("aaaa", "bbbb", "bb", 50, now))
rr := serveRequest(srv, "GET", "/api/nodes/aaaa/neighbors")
var resp NeighborResponse
json.Unmarshal(rr.Body.Bytes(), &resp)
if len(resp.Neighbors) != 1 {
t.Fatalf("expected 1 neighbor, got %d", len(resp.Neighbors))
}
if resp.Neighbors[0].DistanceKm != nil {
t.Errorf("expected nil distance_km for nodes without GPS, got %f", *resp.Neighbors[0].DistanceKm)
}
}
func TestNeighborGraphAPI_RegionFilter(t *testing.T) {
now := time.Now()
// Edge with observer "obs-sjc" — would match region SJC if we had region resolution.
+21
View File
@@ -542,3 +542,24 @@ func minLen(s string, n int) int {
}
return n
}
// PruneOlderThan removes all edges with LastSeen before cutoff.
// Returns the number of edges removed.
func (g *NeighborGraph) PruneOlderThan(cutoff time.Time) int {
g.mu.Lock()
defer g.mu.Unlock()
pruned := 0
for key, edge := range g.edges {
if edge.LastSeen.Before(cutoff) {
// Remove from byNode index
g.removeFromByNode(edge.NodeA, edge)
if edge.NodeB != "" {
g.removeFromByNode(edge.NodeB, edge)
}
delete(g.edges, key)
pruned++
}
}
return pruned
}
+171 -77
View File
@@ -343,112 +343,175 @@ func unmarshalResolvedPath(s string) []*string {
return result
}
// backfillResolvedPaths resolves paths for all observations that have NULL resolved_path.
func backfillResolvedPaths(store *PacketStore, dbPath string) int {
// Collect pending observations and snapshot immutable fields under read lock.
// graph is set in main.go before backfill is called; nil-safe throughout (review item #6).
// backfillResolvedPathsAsync processes observations with NULL resolved_path in
// chunks, yielding between batches so HTTP handlers remain responsive. It sets
// store.backfillComplete when finished and re-picks best observations for any
// transmissions affected by newly resolved paths.
func backfillResolvedPathsAsync(store *PacketStore, dbPath string, chunkSize int, yieldDuration time.Duration, backfillHours int) {
defer func() {
if r := recover(); r != nil {
log.Printf("[store] backfillResolvedPathsAsync panic recovered: %v", r)
}
}()
// Collect ALL pending obs refs upfront in one pass under a single RLock (fix A).
type obsRef struct {
obsID int
pathJSON string
observerID string
txJSON string // snapshot of DecodedJSON for extractFromNode
obsID int
pathJSON string
observerID string
txJSON string
payloadType *int
txHash string // to re-pick best obs
}
cutoff := time.Now().UTC().Add(-time.Duration(backfillHours) * time.Hour)
store.mu.RLock()
pm := store.nodePM
graph := store.graph
var pending []obsRef
var allPending []obsRef
for _, tx := range store.packets {
// Skip transmissions older than the backfill window.
if tx.FirstSeen != "" {
if ts, err := time.Parse(time.RFC3339Nano, tx.FirstSeen); err == nil && ts.Before(cutoff) {
continue
}
// Also try the common SQLite format
if ts, err := time.Parse("2006-01-02 15:04:05", tx.FirstSeen); err == nil && ts.Before(cutoff) {
continue
}
}
for _, obs := range tx.Observations {
if obs.ResolvedPath == nil && obs.PathJSON != "" && obs.PathJSON != "[]" {
pending = append(pending, obsRef{
allPending = append(allPending, obsRef{
obsID: obs.ID,
pathJSON: obs.PathJSON,
observerID: obs.ObserverID,
txJSON: tx.DecodedJSON,
payloadType: tx.PayloadType,
txHash: tx.Hash,
})
}
}
}
store.mu.RUnlock()
if len(pending) == 0 || pm == nil {
return 0
totalPending := len(allPending)
if totalPending == 0 || pm == nil {
store.backfillComplete.Store(true)
log.Printf("[store] async resolved_path backfill: nothing to do")
return
}
// Resolve paths outside the lock — resolvePathForObs only reads pm and graph.
type resolved struct {
obsID int
rp []*string
rpJSON string
store.backfillTotal.Store(int64(totalPending))
store.backfillProcessed.Store(0)
log.Printf("[store] async resolved_path backfill starting: %d observations", totalPending)
// Open RW connection once before the chunk loop (fix B).
var rw *sql.DB
if dbPath != "" {
var err error
rw, err = openRW(dbPath)
if err != nil {
log.Printf("[store] async backfill: open rw error: %v", err)
}
}
var results []resolved
for _, ref := range pending {
// Build a minimal StoreTx for extractFromNode (only needs DecodedJSON + PayloadType).
fakeTx := &StoreTx{DecodedJSON: ref.txJSON, PayloadType: ref.payloadType}
rp := resolvePathForObs(ref.pathJSON, ref.observerID, fakeTx, pm, graph)
if len(rp) > 0 {
rpJSON := marshalResolvedPath(rp)
if rpJSON != "" {
results = append(results, resolved{ref.obsID, rp, rpJSON})
defer func() {
if rw != nil {
rw.Close()
}
}()
totalProcessed := 0
for totalProcessed < totalPending {
end := totalProcessed + chunkSize
if end > totalPending {
end = totalPending
}
chunk := allPending[totalProcessed:end]
// Re-read graph under RLock at the start of each chunk so we pick up
// a freshly-built graph once the background build goroutine completes,
// instead of using the potentially-empty graph captured at cold start.
store.mu.RLock()
graph := store.graph
store.mu.RUnlock()
// Resolve paths outside any lock.
type resolved struct {
obsID int
rp []*string
rpJSON string
txHash string
}
var results []resolved
for _, ref := range chunk {
fakeTx := &StoreTx{DecodedJSON: ref.txJSON, PayloadType: ref.payloadType}
rp := resolvePathForObs(ref.pathJSON, ref.observerID, fakeTx, pm, graph)
if len(rp) > 0 {
rpJSON := marshalResolvedPath(rp)
if rpJSON != "" {
results = append(results, resolved{ref.obsID, rp, rpJSON, ref.txHash})
}
}
}
}
if len(results) == 0 {
return 0
}
// Persist to SQLite using the shared connection.
if len(results) > 0 && rw != nil {
sqlTx, err := rw.Begin()
if err != nil {
log.Printf("[store] async backfill: begin tx error: %v", err)
} else {
stmt, err := sqlTx.Prepare("UPDATE observations SET resolved_path = ? WHERE id = ?")
if err != nil {
log.Printf("[store] async backfill: prepare error: %v", err)
sqlTx.Rollback()
} else {
var execErr error
for _, r := range results {
if _, e := stmt.Exec(r.rpJSON, r.obsID); e != nil && execErr == nil {
execErr = e
}
}
if execErr != nil {
log.Printf("[store] async backfill: exec error (first): %v", execErr)
}
stmt.Close()
if err := sqlTx.Commit(); err != nil {
log.Printf("[store] async backfill: commit error: %v", err)
}
}
}
// Persist to SQLite (no lock needed — separate RW connection).
rw, err := openRW(dbPath)
if err != nil {
log.Printf("[store] backfill: open rw error: %v", err)
return 0
}
defer rw.Close()
sqlTx, err := rw.Begin()
if err != nil {
log.Printf("[store] backfill: begin tx error: %v", err)
return 0
}
defer sqlTx.Rollback()
stmt, err := sqlTx.Prepare("UPDATE observations SET resolved_path = ? WHERE id = ?")
if err != nil {
log.Printf("[store] backfill: prepare error: %v", err)
return 0
}
defer stmt.Close()
var firstErr error
for _, r := range results {
if _, err := stmt.Exec(r.rpJSON, r.obsID); err != nil && firstErr == nil {
firstErr = err
// Update in-memory state and re-pick best observation under a single
// write lock. The per-tx pickBestObservation is O(observations) which is
// typically <10 per tx — negligible cost vs. the race risk of splitting
// the lock (pollAndMerge can append to tx.Observations concurrently).
store.mu.Lock()
affectedSet := make(map[string]bool)
for _, r := range results {
if obs, ok := store.byObsID[r.obsID]; ok {
obs.ResolvedPath = r.rp
}
if !affectedSet[r.txHash] {
affectedSet[r.txHash] = true
if tx, ok := store.byHash[r.txHash]; ok {
pickBestObservation(tx)
}
}
}
store.mu.Unlock()
}
}
if firstErr != nil {
log.Printf("[store] backfill resolved_path exec error (first): %v", firstErr)
totalProcessed += len(chunk)
store.backfillProcessed.Store(int64(totalProcessed))
pct := float64(totalProcessed) / float64(totalPending) * 100
log.Printf("[store] backfill progress: %d/%d observations (%.1f%%)", totalProcessed, totalPending, pct)
time.Sleep(yieldDuration)
}
if err := sqlTx.Commit(); err != nil {
log.Printf("[store] backfill: commit error: %v", err)
return 0
}
// Update in-memory state under write lock.
store.mu.Lock()
count := 0
for _, r := range results {
if obs, ok := store.byObsID[r.obsID]; ok {
obs.ResolvedPath = r.rp
count++
}
}
store.mu.Unlock()
return count
store.backfillComplete.Store(true)
log.Printf("[store] async resolved_path backfill complete: %d observations processed", totalProcessed)
}
// ─── Shared helpers ────────────────────────────────────────────────────────────
@@ -529,3 +592,34 @@ func openRW(dbPath string) (*sql.DB, error) {
rw.SetMaxOpenConns(1)
return rw, nil
}
// PruneNeighborEdges removes edges older than maxAgeDays from both SQLite and
// the in-memory graph. Uses openRW internally because the shared database.conn
// is opened with mode=ro and DELETEs would silently fail.
func PruneNeighborEdges(dbPath string, graph *NeighborGraph, maxAgeDays int) (int, error) {
cutoff := time.Now().UTC().Add(-time.Duration(maxAgeDays) * 24 * time.Hour)
// 1. Prune from SQLite using a read-write connection
var dbPruned int64
rw, err := openRW(dbPath)
if err != nil {
return 0, fmt.Errorf("prune neighbor_edges: open rw: %w", err)
}
defer rw.Close()
res, err := rw.Exec("DELETE FROM neighbor_edges WHERE last_seen < ?", cutoff.Format(time.RFC3339))
if err != nil {
return 0, fmt.Errorf("prune neighbor_edges: %w", err)
}
dbPruned, _ = res.RowsAffected()
// 2. Prune from in-memory graph
memPruned := 0
if graph != nil {
memPruned = graph.PruneOlderThan(cutoff)
}
if dbPruned > 0 || memPruned > 0 {
log.Printf("[neighbor-prune] removed %d DB rows, %d in-memory edges older than %d days", dbPruned, memPruned, maxAgeDays)
}
return int(dbPruned), nil
}
+359
View File
@@ -0,0 +1,359 @@
package main
import (
"encoding/json"
"fmt"
"net/http"
"sort"
"strings"
"github.com/gorilla/mux"
)
// routeMeta holds metadata for a single API route.
type routeMeta struct {
Summary string `json:"summary"`
Description string `json:"description,omitempty"`
Tag string `json:"tag"`
Auth bool `json:"auth,omitempty"`
QueryParams []paramMeta `json:"queryParams,omitempty"`
}
type paramMeta struct {
Name string `json:"name"`
Description string `json:"description"`
Required bool `json:"required,omitempty"`
Type string `json:"type"` // "string", "integer", "boolean"
}
// routeDescriptions returns metadata for all known API routes.
// Key format: "METHOD /path/pattern"
func routeDescriptions() map[string]routeMeta {
return map[string]routeMeta{
// Config
"GET /api/config/cache": {Summary: "Get cache configuration", Tag: "config"},
"GET /api/config/client": {Summary: "Get client configuration", Tag: "config"},
"GET /api/config/regions": {Summary: "Get configured regions", Tag: "config"},
"GET /api/config/theme": {Summary: "Get theme configuration", Description: "Returns color maps, CSS variables, and theme defaults.", Tag: "config"},
"GET /api/config/map": {Summary: "Get map configuration", Tag: "config"},
"GET /api/config/geo-filter": {Summary: "Get geo-filter configuration", Tag: "config"},
// Admin / system
"GET /api/health": {Summary: "Health check", Description: "Returns server health, uptime, and memory stats.", Tag: "admin"},
"GET /api/stats": {Summary: "Network statistics", Description: "Returns aggregate stats (node counts, packet counts, observer counts). Cached for 10s.", Tag: "admin"},
"GET /api/perf": {Summary: "Performance statistics", Description: "Returns per-endpoint request timing and slow query log.", Tag: "admin"},
"POST /api/perf/reset": {Summary: "Reset performance stats", Tag: "admin", Auth: true},
"POST /api/admin/prune": {Summary: "Prune old data", Description: "Deletes packets and nodes older than the configured retention period.", Tag: "admin", Auth: true},
"GET /api/debug/affinity": {Summary: "Debug neighbor affinity scores", Tag: "admin", Auth: true},
// Packets
"GET /api/packets": {Summary: "List packets", Description: "Returns decoded packets with filtering, sorting, and pagination.", Tag: "packets",
QueryParams: []paramMeta{
{Name: "limit", Description: "Max packets to return", Type: "integer"},
{Name: "offset", Description: "Pagination offset", Type: "integer"},
{Name: "sort", Description: "Sort field", Type: "string"},
{Name: "order", Description: "Sort order (asc/desc)", Type: "string"},
{Name: "type", Description: "Filter by packet type", Type: "string"},
{Name: "observer", Description: "Filter by observer ID", Type: "string"},
{Name: "timeRange", Description: "Time range filter (e.g. 1h, 24h, 7d)", Type: "string"},
{Name: "search", Description: "Full-text search", Type: "string"},
{Name: "groupByHash", Description: "Group duplicate packets by hash", Type: "boolean"},
}},
"POST /api/packets": {Summary: "Ingest a packet", Description: "Submit a raw packet for decoding and storage.", Tag: "packets", Auth: true},
"GET /api/packets/{id}": {Summary: "Get packet detail", Tag: "packets"},
"GET /api/packets/timestamps": {Summary: "Get packet timestamp ranges", Tag: "packets"},
"POST /api/packets/observations": {Summary: "Batch submit observations", Description: "Submit multiple observer sightings for existing packets.", Tag: "packets"},
// Decode
"POST /api/decode": {Summary: "Decode a raw packet", Description: "Decodes a hex-encoded packet without storing it.", Tag: "packets"},
// Nodes
"GET /api/nodes": {Summary: "List nodes", Description: "Returns all known mesh nodes with status and metadata.", Tag: "nodes",
QueryParams: []paramMeta{
{Name: "role", Description: "Filter by node role", Type: "string"},
{Name: "status", Description: "Filter by status (active/stale/offline)", Type: "string"},
}},
"GET /api/nodes/search": {Summary: "Search nodes", Description: "Search nodes by name or public key prefix.", Tag: "nodes", QueryParams: []paramMeta{{Name: "q", Description: "Search query", Type: "string", Required: true}}},
"GET /api/nodes/bulk-health": {Summary: "Bulk node health", Description: "Returns health status for all nodes in one call.", Tag: "nodes"},
"GET /api/nodes/network-status": {Summary: "Network status summary", Description: "Returns counts of active, stale, and offline nodes.", Tag: "nodes"},
"GET /api/nodes/{pubkey}": {Summary: "Get node detail", Description: "Returns full detail for a single node by public key.", Tag: "nodes"},
"GET /api/nodes/{pubkey}/health": {Summary: "Get node health", Tag: "nodes"},
"GET /api/nodes/{pubkey}/paths": {Summary: "Get node routing paths", Tag: "nodes"},
"GET /api/nodes/{pubkey}/analytics": {Summary: "Get node analytics", Description: "Per-node packet counts, timing, and RF stats.", Tag: "nodes"},
"GET /api/nodes/{pubkey}/neighbors": {Summary: "Get node neighbors", Description: "Returns neighbor nodes with affinity scores.", Tag: "nodes"},
// Analytics
"GET /api/analytics/rf": {Summary: "RF analytics", Description: "SNR/RSSI distributions and statistics.", Tag: "analytics"},
"GET /api/analytics/topology": {Summary: "Network topology", Description: "Hop-count distribution and route analysis.", Tag: "analytics"},
"GET /api/analytics/channels": {Summary: "Channel analytics", Description: "Message counts and activity per channel.", Tag: "analytics"},
"GET /api/analytics/distance": {Summary: "Distance analytics", Description: "Geographic distance calculations between nodes.", Tag: "analytics"},
"GET /api/analytics/hash-sizes": {Summary: "Hash size analysis", Description: "Distribution of hash prefix sizes across the network.", Tag: "analytics"},
"GET /api/analytics/hash-collisions": {Summary: "Hash collision detection", Description: "Identifies nodes sharing hash prefixes.", Tag: "analytics"},
"GET /api/analytics/subpaths": {Summary: "Subpath analysis", Description: "Common routing subpaths through the mesh.", Tag: "analytics"},
"GET /api/analytics/subpaths-bulk": {Summary: "Bulk subpath analysis", Tag: "analytics"},
"GET /api/analytics/subpath-detail": {Summary: "Subpath detail", Tag: "analytics"},
"GET /api/analytics/neighbor-graph": {Summary: "Neighbor graph", Description: "Full neighbor affinity graph for visualization.", Tag: "analytics"},
// Channels
"GET /api/channels": {Summary: "List channels", Description: "Returns known mesh channels with message counts.", Tag: "channels"},
"GET /api/channels/{hash}/messages": {Summary: "Get channel messages", Description: "Returns messages for a specific channel.", Tag: "channels"},
// Observers
"GET /api/observers": {Summary: "List observers", Description: "Returns all known packet observers/gateways.", Tag: "observers"},
"GET /api/observers/{id}": {Summary: "Get observer detail", Tag: "observers"},
"GET /api/observers/{id}/metrics": {Summary: "Get observer metrics", Description: "Packet rates, uptime, and performance metrics.", Tag: "observers"},
"GET /api/observers/{id}/analytics": {Summary: "Get observer analytics", Tag: "observers"},
"GET /api/observers/metrics/summary": {Summary: "Observer metrics summary", Description: "Aggregate metrics across all observers.", Tag: "observers"},
// Misc
"GET /api/resolve-hops": {Summary: "Resolve hop path", Description: "Resolves hash prefixes in a hop path to node names. Returns affinity scores and best candidates.", Tag: "nodes", QueryParams: []paramMeta{{Name: "hops", Description: "Comma-separated hop hash prefixes", Type: "string", Required: true}}},
"GET /api/traces/{hash}": {Summary: "Get packet traces", Description: "Returns all observer sightings for a packet hash.", Tag: "packets"},
"GET /api/iata-coords": {Summary: "Get IATA airport coordinates", Description: "Returns lat/lon for known airport codes (used for observer positioning).", Tag: "config"},
"GET /api/audio-lab/buckets": {Summary: "Audio lab frequency buckets", Description: "Returns frequency bucket data for audio analysis.", Tag: "analytics"},
}
}
// buildOpenAPISpec constructs an OpenAPI 3.0 spec by walking the mux router.
func buildOpenAPISpec(router *mux.Router, version string) map[string]interface{} {
descriptions := routeDescriptions()
// Collect routes from the router
type routeInfo struct {
path string
method string
authReq bool
}
var routes []routeInfo
router.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error {
path, err := route.GetPathTemplate()
if err != nil {
return nil
}
if !strings.HasPrefix(path, "/api/") {
return nil
}
// Skip the spec/docs endpoints themselves
if path == "/api/spec" || path == "/api/docs" {
return nil
}
methods, err := route.GetMethods()
if err != nil {
return nil
}
for _, m := range methods {
routes = append(routes, routeInfo{path: path, method: m})
}
return nil
})
// Sort routes for deterministic output
sort.Slice(routes, func(i, j int) bool {
if routes[i].path != routes[j].path {
return routes[i].path < routes[j].path
}
return routes[i].method < routes[j].method
})
// Build paths object
paths := make(map[string]interface{})
tagSet := make(map[string]bool)
for _, ri := range routes {
key := ri.method + " " + ri.path
meta, hasMeta := descriptions[key]
// Convert mux path params {name} to OpenAPI {name} (same format, convenient)
openAPIPath := ri.path
// Build operation
op := map[string]interface{}{
"summary": func() string {
if hasMeta {
return meta.Summary
}
return ri.path
}(),
"responses": map[string]interface{}{
"200": map[string]interface{}{
"description": "Success",
"content": map[string]interface{}{
"application/json": map[string]interface{}{
"schema": map[string]interface{}{"type": "object"},
},
},
},
},
}
if hasMeta {
if meta.Description != "" {
op["description"] = meta.Description
}
if meta.Tag != "" {
op["tags"] = []string{meta.Tag}
tagSet[meta.Tag] = true
}
if meta.Auth {
op["security"] = []map[string]interface{}{
{"ApiKeyAuth": []string{}},
}
}
// Add query parameters
if len(meta.QueryParams) > 0 {
params := make([]interface{}, 0, len(meta.QueryParams))
for _, qp := range meta.QueryParams {
p := map[string]interface{}{
"name": qp.Name,
"in": "query",
"required": qp.Required,
"schema": map[string]interface{}{"type": qp.Type},
}
if qp.Description != "" {
p["description"] = qp.Description
}
params = append(params, p)
}
op["parameters"] = params
}
}
// Extract path parameters from {name} patterns
pathParams := extractPathParams(openAPIPath)
if len(pathParams) > 0 {
existing, _ := op["parameters"].([]interface{})
for _, pp := range pathParams {
existing = append(existing, map[string]interface{}{
"name": pp,
"in": "path",
"required": true,
"schema": map[string]interface{}{"type": "string"},
})
}
op["parameters"] = existing
}
// Add to paths
methodLower := strings.ToLower(ri.method)
if _, ok := paths[openAPIPath]; !ok {
paths[openAPIPath] = make(map[string]interface{})
}
paths[openAPIPath].(map[string]interface{})[methodLower] = op
}
// Build tags array (sorted)
tagOrder := []string{"admin", "analytics", "channels", "config", "nodes", "observers", "packets"}
tagDescriptions := map[string]string{
"admin": "Server administration and diagnostics",
"analytics": "Network analytics and statistics",
"channels": "Mesh channel operations",
"config": "Server configuration",
"nodes": "Mesh node operations",
"observers": "Packet observer/gateway operations",
"packets": "Packet capture and decoding",
}
var tags []interface{}
for _, t := range tagOrder {
if tagSet[t] {
tags = append(tags, map[string]interface{}{
"name": t,
"description": tagDescriptions[t],
})
}
}
spec := map[string]interface{}{
"openapi": "3.0.3",
"info": map[string]interface{}{
"title": "CoreScope API",
"description": "MeshCore network analyzer — packet capture, node tracking, and mesh analytics.",
"version": version,
"license": map[string]interface{}{
"name": "MIT",
},
},
"paths": paths,
"tags": tags,
"components": map[string]interface{}{
"securitySchemes": map[string]interface{}{
"ApiKeyAuth": map[string]interface{}{
"type": "apiKey",
"in": "header",
"name": "X-API-Key",
},
},
},
}
return spec
}
// extractPathParams returns parameter names from a mux-style path like /api/nodes/{pubkey}.
func extractPathParams(path string) []string {
var params []string
for {
start := strings.Index(path, "{")
if start == -1 {
break
}
end := strings.Index(path[start:], "}")
if end == -1 {
break
}
params = append(params, path[start+1:start+end])
path = path[start+end+1:]
}
return params
}
// handleOpenAPISpec serves the OpenAPI 3.0 spec as JSON.
// The router is injected via RegisterRoutes storing it on the Server.
func (s *Server) handleOpenAPISpec(w http.ResponseWriter, r *http.Request) {
spec := buildOpenAPISpec(s.router, s.version)
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
if err := enc.Encode(spec); err != nil {
http.Error(w, fmt.Sprintf("failed to encode spec: %v", err), http.StatusInternalServerError)
}
}
// handleSwaggerUI serves a minimal Swagger UI page.
func (s *Server) handleSwaggerUI(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprint(w, swaggerUIHTML)
}
const swaggerUIHTML = `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>CoreScope API Swagger UI</title>
<link rel="stylesheet" href="https://unpkg.com/swagger-ui-dist@5/swagger-ui.css">
<style>
html { box-sizing: border-box; overflow-y: scroll; }
*, *:before, *:after { box-sizing: inherit; }
body { margin: 0; background: #fafafa; }
.topbar { display: none; }
</style>
</head>
<body>
<div id="swagger-ui"></div>
<script src="https://unpkg.com/swagger-ui-dist@5/swagger-ui-bundle.js"></script>
<script>
SwaggerUIBundle({
url: '/api/spec',
dom_id: '#swagger-ui',
deepLinking: true,
presets: [
SwaggerUIBundle.presets.apis,
SwaggerUIBundle.SwaggerUIStandalonePreset
],
layout: 'BaseLayout'
});
</script>
</body>
</html>`
+142
View File
@@ -0,0 +1,142 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestOpenAPISpecEndpoint(t *testing.T) {
_, r := setupTestServer(t)
req := httptest.NewRequest("GET", "/api/spec", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w.Code)
}
ct := w.Header().Get("Content-Type")
if ct != "application/json; charset=utf-8" {
t.Errorf("unexpected content-type: %s", ct)
}
var spec map[string]interface{}
if err := json.Unmarshal(w.Body.Bytes(), &spec); err != nil {
t.Fatalf("invalid JSON: %v", err)
}
// Check required OpenAPI fields
if spec["openapi"] != "3.0.3" {
t.Errorf("expected openapi 3.0.3, got %v", spec["openapi"])
}
info, ok := spec["info"].(map[string]interface{})
if !ok {
t.Fatal("missing info object")
}
if info["title"] != "CoreScope API" {
t.Errorf("unexpected title: %v", info["title"])
}
paths, ok := spec["paths"].(map[string]interface{})
if !ok {
t.Fatal("missing paths object")
}
// Should have at least 20 paths
if len(paths) < 20 {
t.Errorf("expected at least 20 paths, got %d", len(paths))
}
// Check a known path exists
if _, ok := paths["/api/nodes"]; !ok {
t.Error("missing /api/nodes path")
}
if _, ok := paths["/api/packets"]; !ok {
t.Error("missing /api/packets path")
}
// Check tags exist
tags, ok := spec["tags"].([]interface{})
if !ok || len(tags) == 0 {
t.Error("missing or empty tags")
}
// Check security schemes
components, ok := spec["components"].(map[string]interface{})
if !ok {
t.Fatal("missing components")
}
schemes, ok := components["securitySchemes"].(map[string]interface{})
if !ok {
t.Fatal("missing securitySchemes")
}
if _, ok := schemes["ApiKeyAuth"]; !ok {
t.Error("missing ApiKeyAuth security scheme")
}
// Spec should NOT contain /api/spec or /api/docs (self-referencing)
if _, ok := paths["/api/spec"]; ok {
t.Error("/api/spec should not appear in the spec")
}
if _, ok := paths["/api/docs"]; ok {
t.Error("/api/docs should not appear in the spec")
}
}
func TestSwaggerUIEndpoint(t *testing.T) {
_, r := setupTestServer(t)
req := httptest.NewRequest("GET", "/api/docs", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w.Code)
}
ct := w.Header().Get("Content-Type")
if ct != "text/html; charset=utf-8" {
t.Errorf("unexpected content-type: %s", ct)
}
body := w.Body.String()
if len(body) < 100 {
t.Error("response too short for Swagger UI HTML")
}
if !strings.Contains(body, "swagger-ui") {
t.Error("response doesn't contain swagger-ui reference")
}
if !strings.Contains(body, "/api/spec") {
t.Error("response doesn't point to /api/spec")
}
}
func TestExtractPathParams(t *testing.T) {
tests := []struct {
path string
expect []string
}{
{"/api/nodes", nil},
{"/api/nodes/{pubkey}", []string{"pubkey"}},
{"/api/channels/{hash}/messages", []string{"hash"}},
}
for _, tt := range tests {
got := extractPathParams(tt.path)
if len(got) != len(tt.expect) {
t.Errorf("extractPathParams(%q) = %v, want %v", tt.path, got, tt.expect)
continue
}
for i := range got {
if got[i] != tt.expect[i] {
t.Errorf("extractPathParams(%q)[%d] = %q, want %q", tt.path, i, got[i], tt.expect[i])
}
}
}
}
+170 -3
View File
@@ -1,6 +1,7 @@
package main
import (
"crypto/subtle"
"database/sql"
"encoding/json"
"fmt"
@@ -42,6 +43,9 @@ type Server struct {
// Neighbor affinity graph (lazy-built, cached with TTL)
neighborMu sync.Mutex
neighborGraph *NeighborGraph
// Router reference for OpenAPI spec generation
router *mux.Router
}
// PerfStats tracks request performance.
@@ -98,9 +102,13 @@ func (s *Server) getMemStats() runtime.MemStats {
// RegisterRoutes sets up all HTTP routes on the given router.
func (s *Server) RegisterRoutes(r *mux.Router) {
s.router = r
// Performance instrumentation middleware
r.Use(s.perfMiddleware)
// Backfill status header middleware
r.Use(s.backfillStatusMiddleware)
// Config endpoints
r.HandleFunc("/api/config/cache", s.handleConfigCache).Methods("GET")
r.HandleFunc("/api/config/client", s.handleConfigClient).Methods("GET")
@@ -154,12 +162,29 @@ func (s *Server) RegisterRoutes(r *mux.Router) {
r.HandleFunc("/api/resolve-hops", s.handleResolveHops).Methods("GET")
r.HandleFunc("/api/channels/{hash}/messages", s.handleChannelMessages).Methods("GET")
r.HandleFunc("/api/channels", s.handleChannels).Methods("GET")
r.HandleFunc("/api/observers/metrics/summary", s.handleMetricsSummary).Methods("GET")
r.HandleFunc("/api/observers/{id}/metrics", s.handleObserverMetrics).Methods("GET")
r.HandleFunc("/api/observers/{id}/analytics", s.handleObserverAnalytics).Methods("GET")
r.HandleFunc("/api/observers/{id}", s.handleObserverDetail).Methods("GET")
r.HandleFunc("/api/observers", s.handleObservers).Methods("GET")
r.HandleFunc("/api/traces/{hash}", s.handleTraces).Methods("GET")
r.HandleFunc("/api/iata-coords", s.handleIATACoords).Methods("GET")
r.HandleFunc("/api/audio-lab/buckets", s.handleAudioLabBuckets).Methods("GET")
// OpenAPI spec + Swagger UI
r.HandleFunc("/api/spec", s.handleOpenAPISpec).Methods("GET")
r.HandleFunc("/api/docs", s.handleSwaggerUI).Methods("GET")
}
func (s *Server) backfillStatusMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if s.store != nil && s.store.backfillComplete.Load() {
w.Header().Set("X-CoreScope-Status", "ready")
} else {
w.Header().Set("X-CoreScope-Status", "backfilling")
}
next.ServeHTTP(w, r)
})
}
func (s *Server) perfMiddleware(next http.Handler) http.Handler {
@@ -222,10 +247,15 @@ func (s *Server) requireAPIKey(next http.Handler) http.Handler {
writeError(w, http.StatusForbidden, "write endpoints disabled — set apiKey in config.json")
return
}
if r.Header.Get("X-API-Key") != s.cfg.APIKey {
key := r.Header.Get("X-API-Key")
if !constantTimeEqual(key, s.cfg.APIKey) {
writeError(w, http.StatusUnauthorized, "unauthorized")
return
}
if IsWeakAPIKey(key) {
writeError(w, http.StatusForbidden, "forbidden")
return
}
next.ServeHTTP(w, r)
})
}
@@ -519,6 +549,19 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
return
}
counts := s.db.GetRoleCounts()
// Compute backfill progress
backfilling := s.store != nil && !s.store.backfillComplete.Load()
var backfillProgress float64
if backfilling && s.store != nil && s.store.backfillTotal.Load() > 0 {
backfillProgress = float64(s.store.backfillProcessed.Load()) / float64(s.store.backfillTotal.Load())
if backfillProgress > 1 {
backfillProgress = 1
}
} else if !backfilling {
backfillProgress = 1
}
resp := &StatsResponse{
TotalPackets: stats.TotalPackets,
TotalTransmissions: &stats.TotalTransmissions,
@@ -538,6 +581,8 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
Companions: counts["companions"],
Sensors: counts["sensors"],
},
Backfilling: backfilling,
BackfillProgress: backfillProgress,
}
s.statsMu.Lock()
@@ -885,7 +930,7 @@ func (s *Server) handleDecode(w http.ResponseWriter, r *http.Request) {
writeError(w, 400, "hex is required")
return
}
decoded, err := DecodePacket(hexStr)
decoded, err := DecodePacket(hexStr, true)
if err != nil {
writeError(w, 400, err.Error())
return
@@ -917,7 +962,7 @@ func (s *Server) handlePostPacket(w http.ResponseWriter, r *http.Request) {
writeError(w, 400, "hex is required")
return
}
decoded, err := DecodePacket(hexStr)
decoded, err := DecodePacket(hexStr, false)
if err != nil {
writeError(w, 400, err.Error())
return
@@ -1131,6 +1176,17 @@ func (s *Server) handleNodePaths(w http.ResponseWriter, r *http.Request) {
}
}
// Post-filter: verify target node actually appears in each candidate's resolved_path.
// The byPathHop index uses short prefixes which can collide (e.g. "c0" matches multiple nodes).
// We lean on resolved_path (from neighbor affinity graph) to disambiguate.
filtered := candidates[:0] // reuse backing array
for _, tx := range candidates {
if nodeInResolvedPath(tx, lowerPK) {
filtered = append(filtered, tx)
}
}
candidates = filtered
type pathAgg struct {
Hops []PathHopResp
Count int
@@ -2153,6 +2209,112 @@ func nullFloatVal(n sql.NullFloat64) float64 {
return 0
}
func (s *Server) handleObserverMetrics(w http.ResponseWriter, r *http.Request) {
id := mux.Vars(r)["id"]
since := r.URL.Query().Get("since")
until := r.URL.Query().Get("until")
resolution := r.URL.Query().Get("resolution")
// Default to last 24h if no since provided
if since == "" {
since = time.Now().UTC().Add(-24 * time.Hour).Format(time.RFC3339)
}
// Validate resolution
if resolution == "" {
resolution = "5m"
}
switch resolution {
case "5m", "1h", "1d":
// valid
default:
writeError(w, 400, "invalid resolution: "+resolution+". Must be 5m, 1h, or 1d")
return
}
// Sample interval (default 300s = 5min)
sampleInterval := 300
metrics, reboots, err := s.db.GetObserverMetrics(id, since, until, resolution, sampleInterval)
if err != nil {
writeError(w, 500, err.Error())
return
}
if metrics == nil {
metrics = []MetricsSample{}
}
if reboots == nil {
reboots = []string{}
}
// Get observer name
obs, _ := s.db.GetObserverByID(id)
var name *string
if obs != nil {
name = obs.Name
}
writeJSON(w, map[string]interface{}{
"observer_id": id,
"observer_name": name,
"reboots": reboots,
"metrics": metrics,
})
}
func (s *Server) handleMetricsSummary(w http.ResponseWriter, r *http.Request) {
window := r.URL.Query().Get("window")
if window == "" {
window = "24h"
}
region := r.URL.Query().Get("region")
// Parse window duration
dur, err := parseWindowDuration(window)
if err != nil {
writeError(w, 400, "invalid window: "+window)
return
}
since := time.Now().UTC().Add(-dur).Format(time.RFC3339)
summary, err := s.db.GetMetricsSummary(since)
if err != nil {
writeError(w, 500, err.Error())
return
}
if summary == nil {
summary = []MetricsSummaryRow{}
}
// Filter by region if specified
if region != "" {
filtered := make([]MetricsSummaryRow, 0)
for _, row := range summary {
if strings.EqualFold(row.IATA, region) {
filtered = append(filtered, row)
}
}
summary = filtered
}
writeJSON(w, map[string]interface{}{
"observers": summary,
})
}
// parseWindowDuration parses strings like "24h", "3d", "7d", "30d".
func parseWindowDuration(window string) (time.Duration, error) {
if strings.HasSuffix(window, "d") {
daysStr := strings.TrimSuffix(window, "d")
days, err := strconv.Atoi(daysStr)
if err != nil || days <= 0 {
return 0, fmt.Errorf("invalid days: %s", daysStr)
}
return time.Duration(days) * 24 * time.Hour, nil
}
return time.ParseDuration(window)
}
func (s *Server) handleAdminPrune(w http.ResponseWriter, r *http.Request) {
days := 0
if d := r.URL.Query().Get("days"); d != "" {
@@ -2173,3 +2335,8 @@ func (s *Server) handleAdminPrune(w http.ResponseWriter, r *http.Request) {
log.Printf("[prune] deleted %d transmissions older than %d days", n, days)
writeJSON(w, map[string]interface{}{"deleted": n, "days": days})
}
// constantTimeEqual compares two strings in constant time to prevent timing attacks.
func constantTimeEqual(a, b string) bool {
return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
}
+167 -6
View File
@@ -6,6 +6,7 @@ import (
"net/http"
"net/http/httptest"
"strconv"
"strings"
"testing"
"time"
@@ -47,7 +48,7 @@ func setupTestServerWithAPIKey(t *testing.T, apiKey string) (*Server, *mux.Route
}
func TestWriteEndpointsRequireAPIKey(t *testing.T) {
_, router := setupTestServerWithAPIKey(t, "test-secret")
_, router := setupTestServerWithAPIKey(t, "test-secret-key-strong-enough")
t.Run("missing key returns 401", func(t *testing.T) {
req := httptest.NewRequest("POST", "/api/perf/reset", nil)
@@ -65,7 +66,7 @@ func TestWriteEndpointsRequireAPIKey(t *testing.T) {
t.Run("wrong key returns 401", func(t *testing.T) {
req := httptest.NewRequest("POST", "/api/perf/reset", nil)
req.Header.Set("X-API-Key", "wrong-secret")
req.Header.Set("X-API-Key", "wrong-secret-key-strong-enough")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusUnauthorized {
@@ -75,7 +76,7 @@ func TestWriteEndpointsRequireAPIKey(t *testing.T) {
t.Run("correct key passes", func(t *testing.T) {
req := httptest.NewRequest("POST", "/api/perf/reset", nil)
req.Header.Set("X-API-Key", "test-secret")
req.Header.Set("X-API-Key", "test-secret-key-strong-enough")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
@@ -2451,6 +2452,7 @@ func TestHashAnalyticsZeroHopAdvert(t *testing.T) {
pk := "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'ZeroHop', 'repeater')", pk)
store.InvalidateNodeCache()
decoded := `{"name":"ZeroHop","pubKey":"` + pk + `"}`
// header 0x05 → routeType=1 (FLOOD), pathByte=0x00 → hashSize=1
@@ -2504,6 +2506,11 @@ func TestAnalyticsHashSizeSameNameDifferentPubkey(t *testing.T) {
pk1 := "aaaa111122223333444455556666777788889999aaaabbbbccccddddeeee1111"
pk2 := "aaaa111122223333444455556666777788889999aaaabbbbccccddddeeee2222"
// Insert both nodes as repeaters so they appear in distributionByRepeaters.
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'SameName', 'repeater')", pk1)
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'SameName', 'repeater')", pk2)
store.InvalidateNodeCache()
decoded1 := `{"name":"SameName","pubKey":"` + pk1 + `"}`
decoded2 := `{"name":"SameName","pubKey":"` + pk2 + `"}`
@@ -3186,7 +3193,7 @@ func TestHashCollisionsClassification(t *testing.T) {
}
func TestHashCollisionsCacheTTL(t *testing.T) {
// Issue #420: collision cache should use dedicated TTL (60s), not rfCacheTTL (15s)
// Issue #420: collision cache should use dedicated TTL, default 3600s (1 hour)
db := setupTestDB(t)
seedTestData(t, db)
store := NewPacketStore(db, nil)
@@ -3194,8 +3201,8 @@ func TestHashCollisionsCacheTTL(t *testing.T) {
t.Fatalf("store.Load failed: %v", err)
}
if store.collisionCacheTTL != 60*time.Second {
t.Errorf("expected collisionCacheTTL=60s, got %v", store.collisionCacheTTL)
if store.collisionCacheTTL != 3600*time.Second {
t.Errorf("expected collisionCacheTTL=3600s, got %v", store.collisionCacheTTL)
}
if store.rfCacheTTL != 15*time.Second {
t.Errorf("expected rfCacheTTL=15s, got %v", store.rfCacheTTL)
@@ -3532,6 +3539,122 @@ func TestNodePathsEndpointUsesIndex(t *testing.T) {
}
}
func TestNodePathsPrefixCollisionFilter(t *testing.T) {
// Two nodes share the "aa" prefix: TestRepeater (aabbccdd11223344) and a
// second node (aacafe0000000000). Packets whose resolved_path points to
// the second node must NOT appear when querying TestRepeater's paths.
srv, router := setupTestServer(t)
// Manually inject a transmission whose raw path contains "aa" but whose
// resolved_path points to the other node (aacafe0000000000).
now := time.Now().UTC()
recent := now.Add(-30 * time.Minute).Format(time.RFC3339)
recentEpoch := now.Add(-30 * time.Minute).Unix()
// Insert a second node with the same 2-char prefix
srv.db.conn.Exec(`INSERT OR IGNORE INTO nodes (public_key, name, role, last_seen, first_seen, advert_count)
VALUES ('aacafe0000000000', 'CollisionNode', 'repeater', ?, '2026-01-01T00:00:00Z', 5)`, recent)
// Insert a transmission with path hop "aa" that resolves to the OTHER node
srv.db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('FF01', 'collision_test_hash', ?, 1, 4, '{}')`, recent)
// Get its ID
var collisionTxID int
srv.db.conn.QueryRow(`SELECT id FROM transmissions WHERE hash='collision_test_hash'`).Scan(&collisionTxID)
srv.db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp, resolved_path)
VALUES (?, 1, 10.0, -90, '["aa","bb"]', ?, '["aacafe0000000000","eeff00112233aabb"]')`,
collisionTxID, recentEpoch)
// Reload store to pick up new data
store := NewPacketStore(srv.db, nil)
if err := store.Load(); err != nil {
t.Fatalf("store.Load failed: %v", err)
}
srv.store = store
// Query paths for TestRepeater — should NOT include the collision packet
req := httptest.NewRequest("GET", "/api/nodes/aabbccdd11223344/paths", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var resp struct {
Paths []json.RawMessage `json:"paths"`
TotalTransmissions int `json:"totalTransmissions"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("bad JSON: %v", err)
}
// The collision packet should be filtered out. Only transmission 1 (and 3
// if prefix matches) should remain — but transmission 3 has path "cc" and
// resolved_path pointing to TestRoom, so only tx 1 should match.
// Check that collision_test_hash is not in any path group.
bodyStr := w.Body.String()
if strings.Contains(bodyStr, "collision_test_hash") {
t.Error("collision packet should have been filtered out but appeared in response")
}
// Query paths for CollisionNode — should include the collision packet
req2 := httptest.NewRequest("GET", "/api/nodes/aacafe0000000000/paths", nil)
w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req2)
if w2.Code != 200 {
t.Fatalf("expected 200 for CollisionNode, got %d: %s", w2.Code, w2.Body.String())
}
body2 := w2.Body.String()
if !strings.Contains(body2, "collision_test_hash") {
t.Error("collision packet should appear for CollisionNode but was missing")
}
}
func TestNodeInResolvedPath(t *testing.T) {
target := "aabbccdd11223344"
// Case 1: tx.ResolvedPath contains target
pk := "aabbccdd11223344"
tx1 := &StoreTx{ResolvedPath: []*string{&pk}}
if !nodeInResolvedPath(tx1, target) {
t.Error("should match when ResolvedPath contains target")
}
// Case 2: tx.ResolvedPath contains different node
other := "aacafe0000000000"
tx2 := &StoreTx{ResolvedPath: []*string{&other}}
if nodeInResolvedPath(tx2, target) {
t.Error("should not match when ResolvedPath contains different node")
}
// Case 3: nil ResolvedPath — should match (no data to disambiguate, keep it)
tx3 := &StoreTx{}
if !nodeInResolvedPath(tx3, target) {
t.Error("should match when ResolvedPath is nil (no data to disambiguate)")
}
// Case 4: ResolvedPath with nil elements only — has data but no match
tx4 := &StoreTx{ResolvedPath: []*string{nil, nil}}
if nodeInResolvedPath(tx4, target) {
t.Error("should not match when all ResolvedPath elements are nil")
}
// Case 5: target in observation but not in tx.ResolvedPath
tx5 := &StoreTx{
ResolvedPath: []*string{&other},
Observations: []*StoreObs{
{ResolvedPath: []*string{&pk}},
},
}
if !nodeInResolvedPath(tx5, target) {
t.Error("should match when observation's ResolvedPath contains target")
}
}
func TestPathHopIndexIncrementalUpdate(t *testing.T) {
// Test that addTxToPathHopIndex and removeTxFromPathHopIndex work correctly
idx := make(map[string][]*StoreTx)
@@ -3583,3 +3706,41 @@ func TestPathHopIndexIncrementalUpdate(t *testing.T) {
t.Error("expected resolved pubkey key to be deleted after removal")
}
}
func TestMetricsAPIEndpoints(t *testing.T) {
srv, router := setupTestServer(t)
now := time.Now().UTC()
t1 := now.Add(-1 * time.Hour).Format(time.RFC3339)
srv.db.conn.Exec("INSERT INTO observer_metrics (observer_id, timestamp, noise_floor) VALUES (?, ?, ?)",
"obs1", t1, -112.0)
// Test /api/observers/obs1/metrics
req := httptest.NewRequest("GET", "/api/observers/obs1/metrics", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("GET /api/observers/obs1/metrics = %d, want 200", w.Code)
}
var resp map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &resp)
metrics, ok := resp["metrics"].([]interface{})
if !ok || len(metrics) != 1 {
t.Errorf("expected 1 metric in response, got %v", resp["metrics"])
}
// Test /api/observers/metrics/summary
req2 := httptest.NewRequest("GET", "/api/observers/metrics/summary?window=24h", nil)
w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req2)
if w2.Code != 200 {
t.Fatalf("GET /api/observers/metrics/summary = %d, want 200", w2.Code)
}
var resp2 map[string]interface{}
json.Unmarshal(w2.Body.Bytes(), &resp2)
observers, ok := resp2["observers"].([]interface{})
if !ok || len(observers) != 1 {
t.Errorf("expected 1 observer in summary, got %v", resp2["observers"])
}
}
+131 -5
View File
@@ -80,6 +80,45 @@ func (tx *StoreTx) ParsedDecoded() map[string]interface{} {
}
// PacketStore holds all transmissions in memory with indexes for fast queries.
//
// Lock ordering
// =============
// PacketStore uses several mutexes. To prevent deadlocks, locks MUST be
// acquired in the order listed below. Never acquire a higher-numbered lock
// while holding a lower-numbered one.
//
// 1. mu (sync.RWMutex) — guards the core packet data: packets,
// indexes (byHash, byTxID, byObsID, byObserver, byNode,
// byPathHop, byPayloadType), counters, and loaded flag.
//
// 2. cacheMu (sync.Mutex) — guards analytics response caches:
// rfCache, topoCache, hashCache, collisionCache, chanCache,
// distCache, subpathCache, and their TTLs/hit counters.
// Also guards rate-limited invalidation state
// (lastInvalidated, pendingInv).
//
// 3. channelsCacheMu (sync.Mutex) — guards the short-lived GetChannels
// cache (channelsCacheKey/Exp/Res).
//
// 4. groupedCacheMu (sync.Mutex) — guards the short-lived
// QueryGroupedPackets cache.
//
// 5. regionObsMu (sync.Mutex) — guards the region→observer mapping
// cache (regionObsCache, regionObsCacheTime).
//
// 6. hashSizeInfoMu (sync.Mutex) — guards the cached hash-size-info
// result (hashSizeInfoCache). Acquired independently or
// under mu (in EvictStale).
//
// Nesting that occurs today:
// - IngestNew: mu → cacheMu → channelsCacheMu (1 → 2 → 3, OK)
// - IngestObservations: mu → cacheMu (1 → 2, OK)
// - RunEviction/EvictStale: mu → cacheMu → channelsCacheMu (1 → 2 → 3, OK)
// - RunEviction/EvictStale: mu → hashSizeInfoMu (1 → 6, OK)
// - invalidateCachesFor: cacheMu → channelsCacheMu (2 → 3, OK)
//
// All other locks are acquired independently (no nesting).
// When adding new lock acquisitions, respect this ordering.
type PacketStore struct {
mu sync.RWMutex
db *DB
@@ -157,6 +196,12 @@ type PacketStore struct {
// Persisted neighbor graph for hop resolution at ingest time.
graph *NeighborGraph
// Async backfill state: set after backfillResolvedPathsAsync completes.
backfillComplete atomic.Bool
// Progress tracking for async backfill (total pending and processed so far).
backfillTotal atomic.Int64 // set once at start of async backfill
backfillProcessed atomic.Int64
// Eviction config and stats
retentionHours float64 // 0 = unlimited
maxMemoryMB int // 0 = unlimited
@@ -201,8 +246,33 @@ type cachedResult struct {
expiresAt time.Time
}
// cacheTTLSec extracts a duration from the cacheTTL config map.
// Values may be float64 (from JSON) or int. Returns false if key is missing or non-positive.
func cacheTTLSec(m map[string]interface{}, key string) (time.Duration, bool) {
v, ok := m[key]
if !ok {
return 0, false
}
var sec float64
switch n := v.(type) {
case float64:
sec = n
case int:
sec = float64(n)
case int64:
sec = float64(n)
default:
return 0, false
}
if sec <= 0 {
return 0, false
}
return time.Duration(sec * float64(time.Second)), true
}
// NewPacketStore creates a new empty packet store backed by db.
func NewPacketStore(db *DB, cfg *PacketStoreConfig) *PacketStore {
// cacheTTLs is the optional cacheTTL map from config.json; keys are strings, values are seconds.
func NewPacketStore(db *DB, cfg *PacketStoreConfig, cacheTTLs ...map[string]interface{}) *PacketStore {
ps := &PacketStore{
db: db,
packets: make([]*StoreTx, 0, 65536),
@@ -223,7 +293,7 @@ func NewPacketStore(db *DB, cfg *PacketStoreConfig) *PacketStore {
distCache: make(map[string]*cachedResult),
subpathCache: make(map[string]*cachedResult),
rfCacheTTL: 15 * time.Second,
collisionCacheTTL: 60 * time.Second,
collisionCacheTTL: 3600 * time.Second,
invCooldown: 10 * time.Second,
spIndex: make(map[string]int, 4096),
spTxIndex: make(map[string][]*StoreTx, 4096),
@@ -233,6 +303,16 @@ func NewPacketStore(db *DB, cfg *PacketStoreConfig) *PacketStore {
ps.retentionHours = cfg.RetentionHours
ps.maxMemoryMB = cfg.MaxMemoryMB
}
// Wire cacheTTL config values to server-side cache durations.
if len(cacheTTLs) > 0 && cacheTTLs[0] != nil {
ct := cacheTTLs[0]
if v, ok := cacheTTLSec(ct, "analyticsHashSizes"); ok {
ps.collisionCacheTTL = v
}
if v, ok := cacheTTLSec(ct, "analyticsRF"); ok {
ps.rfCacheTTL = v
}
}
return ps
}
@@ -2077,6 +2157,40 @@ func resolvePayloadTypeName(pt *int) string {
return fmt.Sprintf("UNK(%d)", *pt)
}
// nodeInResolvedPath checks whether a transmission's resolved_path contains
// the target node's full pubkey. Returns true if at least one observation's
// resolved_path includes targetPK (lowercased). Excludes transmissions where
// resolved_path is nil/empty or the hop resolved to a different node.
func nodeInResolvedPath(tx *StoreTx, targetPK string) bool {
// If no resolved_path data exists anywhere on this tx, we can't
// disambiguate — return true to keep it (avoid dropping old data).
hasAny := false
// Check the best observation's resolved_path (stored on tx directly).
if tx.ResolvedPath != nil && len(tx.ResolvedPath) > 0 {
hasAny = true
for _, rp := range tx.ResolvedPath {
if rp != nil && strings.ToLower(*rp) == targetPK {
return true
}
}
}
// Also check all observations in case a non-best observation resolved it.
for _, obs := range tx.Observations {
if obs.ResolvedPath == nil || len(obs.ResolvedPath) == 0 {
continue
}
hasAny = true
for _, rp := range obs.ResolvedPath {
if rp != nil && strings.ToLower(*rp) == targetPK {
return true
}
}
}
// No resolved_path data at all — can't disambiguate, keep the candidate.
return !hasAny
}
// txGetParsedPath returns cached parsed path hops, parsing on first call.
func txGetParsedPath(tx *StoreTx) []string {
if tx.pathParsed {
@@ -4651,7 +4765,13 @@ func (s *PacketStore) computeAnalyticsHashSizes(region string) map[string]interf
regionObs = s.resolveRegionObservers(region)
}
_, pm := s.getCachedNodesAndPM()
allNodes, pm := s.getCachedNodesAndPM()
// Build pubkey→role map for filtering by node type.
nodeRoleByPK := make(map[string]string, len(allNodes))
for _, n := range allNodes {
nodeRoleByPK[n.PublicKey] = n.Role
}
distribution := map[string]int{"1": 0, "2": 0, "3": 0}
byHour := map[string]map[string]int{}
@@ -4728,9 +4848,11 @@ func (s *PacketStore) computeAnalyticsHashSizes(region string) map[string]interf
}
}
if byNode[pk] == nil {
role := nodeRoleByPK[pk] // empty if unknown
byNode[pk] = map[string]interface{}{
"hashSize": hashSize, "packets": 0,
"lastSeen": tx.FirstSeen, "name": name,
"role": role,
}
}
byNode[pk]["packets"] = byNode[pk]["packets"].(int) + 1
@@ -4822,7 +4944,7 @@ func (s *PacketStore) computeAnalyticsHashSizes(region string) map[string]interf
multiByteNodes = append(multiByteNodes, map[string]interface{}{
"name": data["name"], "hashSize": data["hashSize"],
"packets": data["packets"], "lastSeen": data["lastSeen"],
"pubkey": pk,
"pubkey": pk, "role": data["role"],
})
}
}
@@ -4830,9 +4952,13 @@ func (s *PacketStore) computeAnalyticsHashSizes(region string) map[string]interf
return multiByteNodes[i]["packets"].(int) > multiByteNodes[j]["packets"].(int)
})
// Distribution by repeaters: count unique nodes per hash size
// Distribution by repeaters: count unique REPEATER nodes per hash size
distributionByRepeaters := map[string]int{"1": 0, "2": 0, "3": 0}
for _, data := range byNode {
role, _ := data["role"].(string)
if !strings.Contains(strings.ToLower(role), "repeater") {
continue
}
hs := data["hashSize"].(int)
key := strconv.Itoa(hs)
distributionByRepeaters[key]++
+2
View File
@@ -68,6 +68,8 @@ type StatsResponse struct {
Commit string `json:"commit"`
BuildTime string `json:"buildTime"`
Counts RoleCounts `json:"counts"`
Backfilling bool `json:"backfilling"`
BackfillProgress float64 `json:"backfillProgress"`
}
// ─── Health ────────────────────────────────────────────────────────────────────
+73 -19
View File
@@ -8,13 +8,15 @@
},
"https": {
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
"key": "/path/to/key.pem",
"_comment": "TLS cert/key paths for direct HTTPS. Most deployments use Caddy (included in Docker) for auto-TLS instead."
},
"branding": {
"siteName": "CoreScope",
"tagline": "Real-time MeshCore LoRa mesh network analyzer",
"logoUrl": null,
"faviconUrl": null
"faviconUrl": null,
"_comment": "Customize site name, tagline, logo, and favicon. logoUrl/faviconUrl can be absolute URLs or relative paths."
},
"theme": {
"accent": "#4a9eff",
@@ -23,38 +25,75 @@
"navBg2": "#1a1a2e",
"statusGreen": "#45644c",
"statusYellow": "#b08b2d",
"statusRed": "#b54a4a"
"statusRed": "#b54a4a",
"_comment": "CSS color overrides. Use the in-app Theme Customizer for live preview, then export values here."
},
"nodeColors": {
"repeater": "#dc2626",
"companion": "#2563eb",
"room": "#16a34a",
"sensor": "#d97706",
"observer": "#8b5cf6"
"observer": "#8b5cf6",
"_comment": "Marker/badge colors per node role. Used on map, nodes list, and live feed."
},
"home": {
"heroTitle": "CoreScope",
"heroSubtitle": "Find your nodes to start monitoring them.",
"steps": [
{ "emoji": "📡", "title": "Connect", "description": "Link your node to the mesh" },
{ "emoji": "🔍", "title": "Monitor", "description": "Watch packets flow in real-time" },
{ "emoji": "📊", "title": "Analyze", "description": "Understand your network's health" }
{
"emoji": "\ud83d\udce1",
"title": "Connect",
"description": "Link your node to the mesh"
},
{
"emoji": "\ud83d\udd0d",
"title": "Monitor",
"description": "Watch packets flow in real-time"
},
{
"emoji": "\ud83d\udcca",
"title": "Analyze",
"description": "Understand your network's health"
}
],
"checklist": [
{ "question": "How do I add my node?", "answer": "Search for your node name or paste your public key." },
{ "question": "What regions are covered?", "answer": "Check the map page to see active observers and nodes." }
{
"question": "How do I add my node?",
"answer": "Search for your node name or paste your public key."
},
{
"question": "What regions are covered?",
"answer": "Check the map page to see active observers and nodes."
}
],
"footerLinks": [
{ "label": "📦 Packets", "url": "#/packets" },
{ "label": "🗺️ Network Map", "url": "#/map" },
{ "label": "🔴 Live", "url": "#/live" },
{ "label": "📡 All Nodes", "url": "#/nodes" },
{ "label": "💬 Channels", "url": "#/channels" }
]
{
"label": "\ud83d\udce6 Packets",
"url": "#/packets"
},
{
"label": "\ud83d\uddfa\ufe0f Network Map",
"url": "#/map"
},
{
"label": "\ud83d\udd34 Live",
"url": "#/live"
},
{
"label": "\ud83d\udce1 All Nodes",
"url": "#/nodes"
},
{
"label": "\ud83d\udcac Channels",
"url": "#/channels"
}
],
"_comment": "Customize the landing page hero, onboarding steps, FAQ, and footer links."
},
"mqtt": {
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
"topic": "meshcore/+/+/packets",
"_comment": "Legacy single-broker config. Prefer mqttSources[] for multiple brokers."
},
"mqttSources": [
{
@@ -150,11 +189,26 @@
"timezone": "local",
"formatPreset": "iso",
"customFormat": "",
"allowCustomFormat": false
"allowCustomFormat": false,
"_comment": "defaultMode: ago|local|iso. timezone: local|utc. formatPreset: iso|us|eu. customFormat: strftime-style (requires allowCustomFormat: true)."
},
"packetStore": {
"maxMemoryMB": 1024,
"estimatedPacketBytes": 450,
"_comment": "In-memory packet store. maxMemoryMB caps RAM usage. All packets loaded on startup, served from RAM."
}
}
},
"resolvedPath": {
"backfillHours": 24,
"_comment": "How far back (hours) the async backfill scans for observations with NULL resolved_path. Default: 24. Set higher to backfill older data, lower to speed up startup."
},
"neighborGraph": {
"maxAgeDays": 5,
"_comment": "Neighbor edges older than this many days are pruned on startup and daily. Default: 5."
},
"_comment_mqttSources": "Each source connects to an MQTT broker. topics: what to subscribe to. iataFilter: only ingest packets from these regions (optional).",
"_comment_channelKeys": "Hex keys for decrypting channel messages. Key name = channel display name. public channel key is well-known.",
"_comment_hashChannels": "Channel names whose keys are derived via SHA256. Key = SHA256(name)[:16]. Listed here so the ingestor can auto-derive keys.",
"_comment_defaultRegion": "IATA code shown by default in region filters.",
"_comment_mapDefaults": "Initial map center [lat, lon] and zoom level.",
"_comment_regions": "IATA code to display name mapping. Packets are tagged with region codes by MQTT topic structure."
}
+20
View File
@@ -0,0 +1,20 @@
# CoreScope — simple deployment using pre-built image from GHCR
# Usage: docker compose -f docker-compose.example.yml up -d
# Docs: https://github.com/Kpa-clawbot/CoreScope/blob/master/DEPLOY.md
services:
corescope:
image: ghcr.io/kpa-clawbot/corescope:latest
ports:
- "${HTTP_PORT:-80}:80"
volumes:
- ${DATA_DIR:-./data}:/app/data
environment:
- DISABLE_CADDY=${DISABLE_CADDY:-true}
- DISABLE_MOSQUITTO=${DISABLE_MOSQUITTO:-false}
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
interval: 30s
timeout: 5s
retries: 3
+2 -6
View File
@@ -15,15 +15,11 @@ services:
restart: unless-stopped
stop_grace_period: 30s
stop_signal: SIGTERM
deploy:
resources:
limits:
memory: 3g
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${STAGING_GO_HTTP_PORT:-82}:80"
- "${STAGING_GO_MQTT_PORT:-1885}:1883"
- "${STAGING_GO_HTTP_PORT:-80}:80"
- "${STAGING_GO_MQTT_PORT:-1883}:1883"
- "6060:6060" # pprof server
- "6061:6061" # pprof ingestor
volumes:
+8 -1
View File
@@ -15,9 +15,16 @@ if [ -f /app/data/theme.json ]; then
fi
SUPERVISORD_CONF="/etc/supervisor/conf.d/supervisord.conf"
if [ "${DISABLE_MOSQUITTO:-false}" = "true" ]; then
if [ "${DISABLE_MOSQUITTO:-false}" = "true" ] && [ "${DISABLE_CADDY:-false}" = "true" ]; then
echo "[config] internal MQTT broker disabled (DISABLE_MOSQUITTO=true)"
echo "[config] Caddy reverse proxy disabled (DISABLE_CADDY=true)"
SUPERVISORD_CONF="/etc/supervisor/conf.d/supervisord-no-mosquitto-no-caddy.conf"
elif [ "${DISABLE_MOSQUITTO:-false}" = "true" ]; then
echo "[config] internal MQTT broker disabled (DISABLE_MOSQUITTO=true)"
SUPERVISORD_CONF="/etc/supervisor/conf.d/supervisord-no-mosquitto.conf"
elif [ "${DISABLE_CADDY:-false}" = "true" ]; then
echo "[config] Caddy reverse proxy disabled (DISABLE_CADDY=true)"
SUPERVISORD_CONF="/etc/supervisor/conf.d/supervisord-no-caddy.conf"
fi
exec /usr/bin/supervisord -c "$SUPERVISORD_CONF"
+43
View File
@@ -0,0 +1,43 @@
[supervisord]
nodaemon=true
user=root
logfile=/dev/stdout
logfile_maxbytes=0
pidfile=/var/run/supervisord.pid
[program:mosquitto]
command=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:corescope-ingestor]
command=/app/corescope-ingestor -config /app/config.json
directory=/app
autostart=true
autorestart=true
startretries=10
startsecs=2
stopsignal=TERM
stopwaitsecs=20
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:corescope-server]
command=/app/corescope-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
directory=/app
autostart=true
autorestart=true
startretries=10
startsecs=2
stopsignal=TERM
stopwaitsecs=20
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
@@ -0,0 +1,34 @@
[supervisord]
nodaemon=true
user=root
logfile=/dev/stdout
logfile_maxbytes=0
pidfile=/var/run/supervisord.pid
[program:corescope-ingestor]
command=/app/corescope-ingestor -config /app/config.json
directory=/app
autostart=true
autorestart=true
startretries=10
startsecs=2
stopsignal=TERM
stopwaitsecs=20
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:corescope-server]
command=/app/corescope-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
directory=/app
autostart=true
autorestart=true
startretries=10
startsecs=2
stopsignal=TERM
stopwaitsecs=20
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
+447
View File
@@ -0,0 +1,447 @@
# CoreScope Deployment Guide
Comprehensive guide to deploying and operating CoreScope. For a quick start, see [DEPLOY.md](../DEPLOY.md).
## Table of Contents
- [System Requirements](#system-requirements)
- [Docker Deployment](#docker-deployment)
- [Configuration Reference](#configuration-reference)
- [MQTT Setup](#mqtt-setup)
- [TLS / HTTPS](#tls--https)
- [Monitoring & Health Checks](#monitoring--health-checks)
- [Backup & Restore](#backup--restore)
- [Troubleshooting](#troubleshooting)
---
## System Requirements
| Resource | Minimum | Recommended |
|----------|---------|-------------|
| RAM | 256 MB | 512 MB+ |
| Disk | 500 MB (image + DB) | 2 GB+ for long-term data |
| CPU | 1 core | 2+ cores |
| Architecture | `linux/amd64`, `linux/arm64` | — |
| Docker | 20.10+ | Latest stable |
CoreScope runs well on Raspberry Pi 4/5 (ARM64). The Go server uses ~300 MB RAM for 56K+ packets.
---
## Docker Deployment
### Quick Start (one command)
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
ghcr.io/kpa-clawbot/corescope:latest
```
Open `http://localhost` — you'll see an empty dashboard ready to receive packets.
No `config.json` is required. The server starts with sensible defaults:
- HTTP on port 3000 (Caddy proxies port 80 → 3000 internally)
- Internal Mosquitto MQTT broker on port 1883
- Ingestor connects to `mqtt://localhost:1883` automatically
- SQLite database at `/app/data/meshcore.db`
### Docker Compose (recommended for production)
Download the example compose file:
```bash
curl -sL https://raw.githubusercontent.com/Kpa-clawbot/CoreScope/master/docker-compose.example.yml \
-o docker-compose.yml
docker compose up -d
```
#### Compose environment variables
| Variable | Default | Description |
|----------|---------|-------------|
| `HTTP_PORT` | `80` | Host port for the web UI |
| `DATA_DIR` | `./data` | Host path for persistent data |
| `DISABLE_MOSQUITTO` | `false` | Set `true` to use an external MQTT broker |
### Image tags
| Tag | Use case |
|-----|----------|
| `v3.4.1` | Pinned release — recommended for production |
| `v3.4` | Latest patch in the v3.4.x series |
| `v3` | Latest minor+patch in v3.x |
| `latest` | Latest release tag |
| `edge` | Built from master on every push — unstable |
### Updating
```bash
docker compose pull
docker compose up -d
```
For `docker run` users:
```bash
docker pull ghcr.io/kpa-clawbot/corescope:latest
docker stop corescope && docker rm corescope
docker run -d --name corescope ... # same flags as before
```
Data is preserved in the volume — updates are non-destructive.
---
## Configuration Reference
CoreScope uses a layered configuration system (highest priority wins):
1. **Environment variables**`MQTT_BROKER`, `DB_PATH`, etc.
2. **`/app/data/config.json`** — full config file (volume-mounted)
3. **Built-in defaults** — work out of the box with no config
### Environment variable overrides
| Variable | Default | Description |
|----------|---------|-------------|
| `MQTT_BROKER` | `mqtt://localhost:1883` | MQTT broker URL (overrides config file) |
| `MQTT_TOPIC` | `meshcore/#` | MQTT topic subscription pattern |
| `DB_PATH` | `data/meshcore.db` | SQLite database path |
| `DISABLE_MOSQUITTO` | `false` | Skip the internal Mosquitto broker |
### config.json
For advanced configuration, create a `config.json` and mount it at `/app/data/config.json`:
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
-v ./config.json:/app/data/config.json:ro \
ghcr.io/kpa-clawbot/corescope:latest
```
See `config.example.json` in the repository for all available options including:
- MQTT sources (multiple brokers)
- Channel encryption keys
- Branding and theming
- Health thresholds
- Region filters
- Retention policies
- Geo-filtering
---
## MQTT Setup
CoreScope receives MeshCore packets via MQTT. The container ships with an internal Mosquitto broker — no setup needed for basic use.
### Internal broker (default)
The built-in Mosquitto broker listens on port 1883 inside the container. Point your MeshCore gateways at it:
```bash
# Expose MQTT port for external gateways
docker run -d --name corescope \
-p 80:80 -p 1883:1883 \
-v corescope-data:/app/data \
ghcr.io/kpa-clawbot/corescope:latest
```
### External broker
To use your own MQTT broker (Mosquitto, EMQX, HiveMQ, etc.):
1. Disable the internal broker:
```bash
-e DISABLE_MOSQUITTO=true
```
2. Point the ingestor at your broker:
```bash
-e MQTT_BROKER=mqtt://your-broker:1883
```
Or via `config.json`:
```json
{
"mqttSources": [
{
"name": "my-broker",
"broker": "mqtt://your-broker:1883",
"username": "user",
"password": "pass",
"topics": ["meshcore/#"]
}
]
}
```
### Multiple brokers
CoreScope can connect to multiple MQTT brokers simultaneously:
```json
{
"mqttSources": [
{
"name": "local",
"broker": "mqtt://localhost:1883",
"topics": ["meshcore/#"]
},
{
"name": "remote",
"broker": "mqtts://remote-broker:8883",
"username": "reader",
"password": "secret",
"topics": ["meshcore/+/+/packets"]
}
]
}
```
### MQTT topic format
MeshCore gateways typically publish to `meshcore/<gateway>/<region>/packets`. The default subscription `meshcore/#` catches all of them.
---
## TLS / HTTPS
### Option 1: External reverse proxy (recommended)
Run CoreScope behind nginx, Traefik, or Cloudflare Tunnel for TLS termination:
```nginx
# nginx example
server {
listen 443 ssl;
server_name corescope.example.com;
ssl_certificate /etc/ssl/certs/corescope.pem;
ssl_certificate_key /etc/ssl/private/corescope.key;
location / {
proxy_pass http://localhost:80;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
```
The `Upgrade` and `Connection` headers are required for WebSocket support.
### Option 2: Built-in Caddy (auto-TLS)
The container includes Caddy for automatic Let's Encrypt certificates:
1. Create a Caddyfile:
```
corescope.example.com {
reverse_proxy localhost:3000
}
```
2. Mount it and expose TLS ports:
```bash
docker run -d --name corescope \
-p 80:80 -p 443:443 \
-v corescope-data:/app/data \
-v caddy-certs:/data/caddy \
-v ./Caddyfile:/etc/caddy/Caddyfile:ro \
ghcr.io/kpa-clawbot/corescope:latest
```
Caddy handles certificate issuance and renewal automatically.
---
## API Documentation
CoreScope auto-generates an OpenAPI 3.0 specification from its route definitions. The spec is always in sync with the running server — no manual maintenance required.
### Endpoints
| URL | Description |
|-----|-------------|
| `/api/spec` | OpenAPI 3.0 JSON schema — machine-readable API definition |
| `/api/docs` | Interactive Swagger UI — browse and test all 40+ endpoints |
### Usage
**Browse the API interactively:**
```
http://your-instance/api/docs
```
**Fetch the spec programmatically:**
```bash
curl http://your-instance/api/spec | jq .
```
**For bot/integration developers:** The spec includes all request parameters, response schemas, and example values. Import it into Postman, Insomnia, or any OpenAPI-compatible tool.
### Public instance
The live instance at [analyzer.00id.net](https://analyzer.00id.net) has all API endpoints publicly accessible:
- Spec: [analyzer.00id.net/api/spec](https://analyzer.00id.net/api/spec)
- Docs: [analyzer.00id.net/api/docs](https://analyzer.00id.net/api/docs)
---
## Monitoring & Health Checks
### Docker health check
The container includes a built-in health check that hits `/api/stats`:
```bash
docker inspect --format='{{.State.Health.Status}}' corescope
```
Docker reports `healthy` or `unhealthy` automatically. The check runs every 30 seconds.
### Manual health check
```bash
curl -f http://localhost/api/stats
```
Returns JSON with packet counts, node counts, and version info:
```json
{
"totalPackets": 56234,
"totalNodes": 142,
"totalObservers": 12,
"packetsLastHour": 830,
"packetsLast24h": 19644,
"engine": "go",
"version": "v3.4.1"
}
```
### Log monitoring
```bash
# All logs
docker compose logs -f
# Server only
docker compose logs -f | grep '\[server\]'
# Ingestor only
docker compose logs -f | grep '\[ingestor\]'
```
### Resource monitoring
```bash
docker stats corescope
```
---
## Backup & Restore
### Backup
All persistent data lives in `/app/data`. The critical file is the SQLite database:
```bash
# Copy from the Docker volume
docker cp corescope:/app/data/meshcore.db ./backup-$(date +%Y%m%d).db
# Or if using a bind mount
cp ./data/meshcore.db ./backup-$(date +%Y%m%d).db
```
Optional files to back up:
- `config.json` — custom configuration
- `theme.json` — custom theme/branding
### Restore
```bash
# Stop the container
docker stop corescope
# Replace the database
docker cp ./backup.db corescope:/app/data/meshcore.db
# Restart
docker start corescope
```
### Automated backups
```bash
# cron: daily backup at 3 AM, keep 7 days
0 3 * * * docker cp corescope:/app/data/meshcore.db /backups/corescope-$(date +\%Y\%m\%d).db && find /backups -name "corescope-*.db" -mtime +7 -delete
```
---
## Troubleshooting
### Container starts but dashboard is empty
This is normal on first start with no MQTT sources configured. The dashboard shows data once packets arrive via MQTT. Either:
- Point a MeshCore gateway at the container's MQTT broker (port 1883)
- Configure an external MQTT source in `config.json`
### "no MQTT connections established" in logs
The ingestor couldn't connect to any MQTT broker. Check:
1. Is the internal Mosquitto running? (`DISABLE_MOSQUITTO` should be `false`)
2. Is the external broker reachable? Test with `mosquitto_sub -h broker -t meshcore/#`
3. Are credentials correct in `config.json`?
### WebSocket disconnects / real-time updates stop
If behind a reverse proxy, ensure WebSocket upgrade headers are forwarded:
```nginx
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
Also check proxy timeouts — set them to at least 300s for long-lived WebSocket connections.
### High memory usage
The in-memory packet store grows with retained packets. Configure retention limits in `config.json`:
```json
{
"packetStore": {
"retentionHours": 24,
"maxMemoryMB": 512
},
"retention": {
"nodeDays": 7,
"packetDays": 30
}
}
```
### Database locked errors
SQLite doesn't support concurrent writers well. Ensure only one CoreScope instance accesses the database file. If running multiple containers, each needs its own database.
### Container unhealthy
Check logs: `docker compose logs --tail 50`. Common causes:
- Port 3000 already in use inside the container
- Database file permissions (must be writable by the container user)
- Corrupted database — restore from backup
### ARM / Raspberry Pi issues
- Use `linux/arm64` images (Pi 4 and 5). Pi 3 (armv7) is not supported.
- First pull may be slow — the multi-arch manifest selects the right image automatically.
- If memory is tight, set `packetStore.maxMemoryMB` to limit RAM usage.
+132
View File
@@ -0,0 +1,132 @@
# Proposal: Terminal/TUI Interface for CoreScope
**Status:** Approved for MVP
**Issue:** TBD
## Problem
CoreScope's web UI requires a browser. Operators managing remote mesh deployments often work over SSH — headless servers, Raspberry Pis, field laptops with spotty connectivity. They need to check mesh health, view packet flow, and diagnose issues without opening a browser.
## Vision
A terminal-based user interface (TUI) that connects to a CoreScope instance's API and renders key views directly in the terminal. Think `htop` for mesh networks.
---
## Expert Review
### Carmack (Performance / Data Flow)
- **bubbletea is fine for this.** The TUI is a thin API consumer — it's not processing 7.3M observations locally. The server does the heavy lifting; the TUI just renders summary data from `/api/observers/metrics/summary` (dozens of rows, not millions). No performance concern here.
- **WebSocket in a TUI — one gotcha:** reconnection. SSH sessions drop, networks flake. The TUI MUST have automatic reconnect with exponential backoff. Don't let a dropped WS kill the whole UI — show a "reconnecting..." status and keep the last-known state visible.
- **Memory footprint:** Should be trivial. The TUI holds at most a few hundred packets in a ring buffer for the live feed + summary stats. Target <20MB RSS. bubbletea itself is lightweight. The danger is unbounded packet accumulation — use a fixed-size ring buffer (e.g., last 1000 packets) for the live feed, not an ever-growing slice.
- **Batch WS messages.** Don't re-render on every single packet. Coalesce WS messages and re-render at most 10fps (every 100ms). Terminal rendering is slow — flooding it with updates causes flicker and CPU burn.
### Torvalds (Simplicity / Scope)
- **The scope is too big for an MVP.** Node detail view, sparklines, SSH server mode, multi-instance, export — delete all of that from M1. You need TWO views to prove this works: fleet dashboard table and live packet feed. That's it.
- **bubbletea vs tview:** bubbletea. Not because Elm-architecture is "clean" — because it's what the Go community actually uses now, the examples are good, and lipgloss makes table rendering trivial. Don't overthink this.
- **Over-engineering risk is HIGH.** The proposal describes 4 views, stretch features, and SSH server mode before a single line of code exists. Build the two-view demo. Ship it. Then decide what's next based on whether anyone actually uses it.
- **Same repo, `cmd/tui/`.** Don't create a separate repo for what's going to be 500 lines of Go initially. It shares the same API types. Keep it together.
- **Kill the "Open Questions" section.** Answer them: Target user = anyone with SSH access. M1 = dashboard + live feed. Same repo. Name = `corescope-tui`. Done. Stop discussing, start building.
### Doshi (Strategy / Prioritization)
- **This is an N (Neutral) feature, not an L.** It doesn't change CoreScope's trajectory — the web UI already works. But it's a solid N: it unlocks a real use case (SSH-only operators) and proves CoreScope's API is a proper platform, not just a web app backend.
- **The MVP that proves the concept:** Can an operator SSH into a Pi, run `corescope-tui --url http://analyzer:3000`, and immediately see fleet health + live packets? If yes, the concept is proven. Everything else (node detail, sparklines, alerting) is M2+.
- **Defer list:** Node detail view, RF sparklines, SSH server mode, multi-instance, export, mouse support, true-color fallback, alerting. ALL of these are M2 or later.
- **Pre-mortem — why would this fail?**
1. Nobody uses it because the web UI is good enough (likely for most users — that's fine, this is for the SSH-only niche)
2. The API doesn't return what the TUI needs in the right shape (validate this FIRST — curl the endpoints before writing any TUI code)
3. Scope creep kills the demo — someone adds "just one more view" and it's never done
- **Opportunity cost:** Low. This is a day of work for the MVP. The API already exists. The risk is spending a week on polish nobody asked for.
---
## MVP Definition (Demo Target)
**Goal:** A working two-view TUI that connects to any CoreScope instance and displays real-time mesh data in a terminal. Buildable in one focused session.
### View 1: Fleet Dashboard (default)
```
┌─ CoreScope TUI ──────────────────────────────────────────┐
│ Connected: analyzer.00id.net | Observers: 35 | ● Live │
├──────────────────────────────────────────────────────────┤
│ Observer │ Nodes │ Pkts/hr │ NF │ Status │
│ GY889 Repeater │ 142 │ 312 │ -112 │ ● active │
│ C0ffee SF │ 89 │ 201 │ -108 │ ● active │
│ ELC-ONNIE-RPT-1 │ 67 │ 156 │ -95 │ ▲ warning │
│ Bar Repeater │ 12 │ 3 │ -76 │ ▼ stale │
└──────────────────────────────────────────────────────────┘
Tab: [Dashboard] [Live Feed] q: quit ?: help
```
- **Data source:** `GET /api/observers/metrics/summary`
- **Refresh:** Poll every 5s (simple, no WS needed for this view)
- **Sort:** By observer name initially. Stretch: column sort with arrow keys.
### View 2: Live Packet Feed
```
┌─ Live Feed ──────────────────────────────────────────────┐
│ 14:32:01 ADVERT GY889 Repeater → 3 hops -112dB │
│ 14:32:02 GRP_TXT #test "hello world" → 5 hops -98dB │
│ 14:32:03 TXT_MSG [encrypted] → 2 hops -105dB │
│ 14:32:04 CHAN #sf "anyone on?" → 8 hops -91dB │
│ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
└──────────────────────────────────────────────────────────┘
Tab: [Dashboard] [Live Feed] p: pause q: quit
```
- **Data source:** WebSocket (`/ws`)
- **Buffer:** Ring buffer, last 500 packets max
- **Render:** Coalesce updates, re-render at most 10fps
- **Reconnect:** Auto-reconnect with exponential backoff (1s, 2s, 4s, max 30s)
### What's NOT in MVP
- Node detail view
- RF sparklines
- SSH server mode (`--serve-ssh`)
- Multi-instance support
- Export to CSV/JSON
- Mouse support
- Alerting / terminal bell
- Color theme configuration
- Custom filters (/ to filter)
### Technical Decisions (Resolved)
| Question | Answer |
|---|---|
| Target user | SSH operators, power users, field techs |
| Library | bubbletea + lipgloss |
| Location | `cmd/tui/` in same repo |
| Binary name | `corescope-tui` |
| Min terminal | 256-color, 80x24 |
| State | Stateless — pure API consumer, no local DB |
### Implementation Plan
1. Scaffold `cmd/tui/main.go` — flag parsing (`--url`), bubbletea app init
2. Fleet dashboard model — fetch `/api/observers/metrics/summary`, render table
3. Live feed model — WebSocket connect, ring buffer, packet rendering
4. Tab switching between views
5. Status bar (connection state, help hints)
6. Test against `https://analyzer.00id.net`
---
## Future Milestones (post-MVP, not scheduled)
### M2: Navigation & Detail
- Node detail view (select observer → see its packets/neighbors)
- Keyboard navigation (j/k, Enter, Esc)
- `/` to filter packets
### M3: Visualization
- RF noise floor sparklines (`▁▂▃▅▇█`)
- Health history over time
- Color theme support
### M4: Advanced
- SSH server mode (`--serve-ssh :2222`)
- Multi-instance tabs
- Export current view to stdout (CSV/JSON)
- Desktop notifications on anomalies
+135
View File
@@ -0,0 +1,135 @@
# CoreScope v3.5.0 🚀
The "stop building from source and start analyzing your mesh" release. 95 commits.
---
## 🐳 Pre-built Docker Images
CoreScope now ships as a ready-to-run Docker image on GitHub Container Registry. No cloning, no building, no dependencies — just pull and run.
```bash
docker run -d --name corescope -p 80:80 -p 443:443 -p 1883:1883 \
-v corescope-data:/app/data \
ghcr.io/kpa-clawbot/corescope:v3.5.0
```
**Using HTTPS with a custom domain?** Mount your Caddyfile and certs directory:
```bash
docker run -d --name corescope -p 80:80 -p 443:443 -p 1883:1883 \
-v /your/data:/app/data \
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
-v /your/caddy-data:/data/caddy \
ghcr.io/kpa-clawbot/corescope:v3.5.0
```
Caddy auto-provisions Let's Encrypt certs. Your Caddyfile just needs:
```
yourdomain.example.com {
reverse_proxy localhost:3000
}
```
That's it. Zero config required — MQTT broker, Caddy HTTPS, and SQLite are built in.
**Already running CoreScope?**
```bash
# 1. Find your running container name
docker ps --format '{{.Names}}'
# 2. Stop and remove it
docker stop <container-name> && docker rm <container-name>
# 3. Pull the pre-built image
docker pull ghcr.io/kpa-clawbot/corescope:v3.5.0
# 4. Run with your existing data directory
docker run -d --name corescope -p 80:80 -p 443:443 -p 1883:1883 \
-v /your/data:/app/data \
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
-v /your/caddy-data:/data/caddy \
ghcr.io/kpa-clawbot/corescope:v3.5.0
```
Your data volume stays. Nothing to migrate.
Tags: `v3.5.0` (this release) · `latest` (latest tagged release) · `edge` (master tip, for testing). Env: `DISABLE_CADDY=true` / `DISABLE_MOSQUITTO=true` if you bring your own.
---
## ⚡ 83% Faster
35 performance commits. Packets endpoint p50 dropped from 16.7ms → 2.7ms. Server now serves HTTP within 2 minutes on *any* DB size — async background backfill means you're never staring at a loading screen. N+1 API calls killed everywhere. Prefix map memory cut 10x. WebSocket renders batched via rAF.
---
## 🔬 RF Health Dashboard
New Analytics tab. Per-observer noise floor as color-coded columns (green/yellow/red), airtime utilization, error rates, battery levels. Click any observer for the full breakdown. Region-filterable. This is the beginning of making CoreScope more than just a packet viewer.
---
## 🗺️ See Where Traces Actually Go
Send a trace → watch it on the live map. Solid animated line shows how far it got. Dashed ghost shows where it didn't reach. Finally know *where* your trace failed, not just *that* it failed.
---
## 📊 Things That Were Lying To You
- "By Repeaters" was counting companions. Fixed.
- Zero-hop adverts claimed "1 byte hash" when the hash size was unknowable. Fixed.
- "Packets through this node" showed packets through a *different* node with the same prefix. Fixed — now uses the neighbor affinity graph.
- Table sorting on nodes/neighbors/observers silently did nothing. Fixed.
---
## 🔗 Deep Links · 🎨 Channel Colors · 📱 Mobile · 🔑 Security
**Deep links** — every page state goes in the URL. Share a link to a specific node, filter, or analytics tab.
**Channel colors** — click the color dot next to any channel, pick from 8 colors, see it highlighted across the feed. Persists in localStorage.
**Distance units** — km, miles, or auto-detect from locale. Customizer → Display.
**Mobile** — 44px touch targets, ARIA labels, responsive breakpoints.
**Security** — weak API keys rejected at startup. License: GPL v3.
---
## 📡 Full API Documentation
Every endpoint is now documented with an auto-generated OpenAPI 3.0 spec — always in sync with the running server.
- **Interactive Swagger UI:** [analyzer.00id.net/api/docs](https://analyzer.00id.net/api/docs) — browse and test all 40+ endpoints
- **Machine-readable spec:** [analyzer.00id.net/api/spec](https://analyzer.00id.net/api/spec) — import into Postman, Insomnia, or use for bot/integration development
On your own instance: `/api/docs` and `/api/spec`.
---
## 🐛 14 Bugs Squashed
Live map crash, zero-hop hash lies, animation freezes, repeater miscounts, prefix collisions, dead channel picker, invisible buttons, broken sorting, memory leak, and more.
---
## Upgrade
```bash
docker stop <container-name> && docker rm <container-name>
docker pull ghcr.io/kpa-clawbot/corescope:v3.5.0
# HTTP only:
docker run -d --name corescope -p 80:80 -p 1883:1883 \
-v /your/data:/app/data \
ghcr.io/kpa-clawbot/corescope:v3.5.0
# With HTTPS (custom domain):
docker run -d --name corescope -p 80:80 -p 443:443 -p 1883:1883 \
-v /your/data:/app/data \
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
-v /your/caddy-data:/data/caddy \
ghcr.io/kpa-clawbot/corescope:v3.5.0
```
First start backfills `resolved_path` in the background. No downtime. No breaking changes.
+148
View File
@@ -0,0 +1,148 @@
# Channel Color Highlighting Spec
**Status:** Proposed
**Issue:** [#271](https://github.com/Kpa-clawbot/CoreScope/issues/271)
**Author:** Stinkmeaner (AI)
**Date:** 2026-04-05
## Problem
When monitoring multiple active hash channels simultaneously on the Live tab, all `GRP_TXT` traffic renders identically — same color, same styling. Users tracking specific channels (e.g. `#wardriving`) cannot visually distinguish their traffic from other channel activity without reading each row's channel field.
## Solution
Allow users to assign custom highlight colors to specific hash channels. Colors propagate across the Live feed, map animations, and timeline. Unassigned channels retain the default `GRP_TXT` styling.
### Data Model
**Storage:** Single `localStorage` key `live-channel-colors`
```json
{
"#wardriving": "#ef4444",
"#meshnet": "#3b82f6"
}
```
- Keyed by resolved channel name (e.g. `#wardriving`) or raw hash prefix if unresolved
- Included in customizer theme export/import for portability
- Maximum ~16 assignments (no hard limit, but UI should discourage excess — see Edge Cases)
### Channel Matching
- Match on the packet's `channel` or `group` field
- Handle both resolved channel names and raw hash prefixes
- Only applies to `GRP_TXT` packet types — other types retain their existing `TYPE_COLORS` styling
### Visual Treatment
**Feed rows (primary):**
- 4px colored left border
- Subtle background tint: channel color at 810% opacity
- Text color unchanged — contrast must remain accessible
**Map animations:**
- Packet arcs use the assigned channel color instead of default `TYPE_COLORS.GRP_TXT`
- Node markers retain role-based coloring (channel color does NOT override node markers)
**Timeline sparkline:**
- Dots/bars colored per channel assignment
- Unassigned channels use default color
**Auto-legend:**
- Generated from active assignments
- Displayed near the feed header
- Color swatch + channel name, compact horizontal layout
### Configuration UI
**Quick assign (primary workflow):**
- Right-click (long-press on mobile) a channel name in the Live feed
- Color picker popover with ~12 preset swatches + custom hex input
- "Clear" button to remove assignment
**Customizer panel (management):**
- New "Channel Colors" section under existing "Packet Type Colors"
- Lists all assigned channels with color swatches
- Add/edit/remove individual assignments
- "Clear All" button
- Synced with theme export/import
### Priority Rules
| Context | Color source |
|---------|-------------|
| Feed row background/border | Channel color (if assigned), else default |
| Feed row text | Always default (no override) |
| Map packet arcs | Channel color (if assigned), else `TYPE_COLORS.GRP_TXT` |
| Map node markers | Always role color (no override) |
| Timeline dots | Channel color (if assigned), else default |
## Edge Cases
- **10+ colors:** At ~10 simultaneous assignments, colors become hard to distinguish. The UI should show a soft warning ("Many colors assigned — consider clearing unused ones") but not block the user.
- **Color conflicts with role/type colors:** Channel color takes priority for feed row highlighting only. Role colors remain authoritative for node markers.
- **Removal:** Clearing a channel color reverts to default styling immediately — no page refresh needed.
- **Non-GRP_TXT packets:** Channel color never applied. These packets have no channel association.
- **Customizer rework (#288):** If the customizer rework lands first, the Channel Colors section should follow the new single-delta-object pattern (`cs-theme-overrides`). If it hasn't landed, use the standalone `live-channel-colors` key and migrate later.
- **Dark/light mode:** Channel colors are mode-independent (same color in both modes). The 810% opacity tint ensures readability in both themes.
## Milestones
### M1: Core model + feed row highlighting
- `localStorage` read/write for `live-channel-colors`
- Feed row rendering: left border + background tint
- Unit tests for storage CRUD and color application logic
### M2: Quick-assign UI
- Right-click / long-press context menu on channel names
- Color picker popover with presets + custom hex
- Clear button
- Playwright E2E test for assign/clear workflow
### M3: Map animation integration
- Packet arc color lookup from channel assignments
- Falls back to `TYPE_COLORS.GRP_TXT` when unassigned
- Visual verification via browser screenshot
### M4: Customizer section + export/import
- "Channel Colors" management panel in customizer
- Include channel colors in theme export JSON
- Import restores channel colors
- Unit tests for export/import round-trip
### M5: Timeline coloring + auto-legend
- Timeline sparkline uses channel colors
- Auto-legend renders near feed header
- Playwright E2E for legend visibility
## Testing
| Level | What | How |
|-------|------|-----|
| Unit | Storage CRUD, color lookup, merge with defaults | `test-frontend-helpers.js` via `vm.createContext` |
| Unit | Export/import round-trip with channel colors | Same |
| E2E | Quick-assign popover, color applied to feed rows | Playwright against localhost |
| E2E | Customizer channel colors section | Playwright |
| E2E | Legend appears when ≥1 channel colored | Playwright |
| Visual | Map arcs colored, dark/light mode readability | Browser screenshot |
## Expert Review Notes
### Tufte (Visualization)
- **Left border + tint is sound.** The 4px border is data-ink (encodes channel identity). The tint at 810% opacity provides grouping without overwhelming the data. This is information encoding, not decoration.
- **Risk at scale:** Beyond ~8 colors, perceptual distinguishability drops sharply. The spec correctly warns but doesn't enforce. Consider using a curated palette of maximally-distinct colors (like ColorBrewer qualitative sets) as the preset swatches rather than a free-form picker.
- **Auto-legend is correct:** Direct labeling on every row would be redundant (channel name already in the row). A compact legend near the feed is the right balance — it teaches the encoding once.
- **No chartjunk introduced.** The visual treatment adds information (channel identity) without decorative excess.
### Torvalds (Code Quality)
- **localStorage is fine** for user preferences with <1KB payloads. No need for IndexedDB or server-side storage.
- **5 milestones is appropriate.** Each is independently shippable and testable. No milestone depends on speculation about future milestones.
- **Watch the customizer coupling.** If #288 lands, the `live-channel-colors` key should merge into `cs-theme-overrides`. Design the read/write functions to abstract the storage key so migration is a one-line change, not a rewrite.
- **Keep the color picker simple.** Don't build a custom color picker — use `<input type="color">` with preset swatch buttons. The browser's native picker is fine.
### Doshi (Product Strategy)
- **This is N (Neutral).** It's a genuine usability improvement for multi-channel monitoring, but it doesn't change CoreScope's trajectory. It won't attract new users or unlock new use cases — it makes existing power users slightly more efficient.
- **Opportunity cost is low.** Each milestone is small (~1-2 hours of work). The total investment is modest.
- **5 milestones is fine** given each is small. Shipping M1+M2 alone delivers 80% of the value. M3M5 are polish. Consider M1+M2 as the MVP gate — if nobody uses channel colors after M2, stop there.
- **Pre-mortem:** This fails if users rarely monitor 2+ channels simultaneously, making the problem theoretical. Validate that multi-channel monitoring is a real workflow before M3.
+311
View File
@@ -0,0 +1,311 @@
# Deployment Simplification Spec
**Status:** Draft
**Author:** Kpa-clawbot
**Date:** 2026-04-05
## Current State
CoreScope deployment today requires:
1. **Clone the repo** and build from source (`docker compose build`)
2. **Create a config.json** — the example is 100+ lines with MQTT credentials, channel keys, theme colors, regions, cache TTLs, health thresholds, branding, and more. An operator must understand all of this before seeing a single packet.
3. **Set up a Caddyfile** for TLS (separate `caddy-config/` directory, bind-mounted)
4. **Understand the supervisord architecture** — the container runs 4 processes (mosquitto, ingestor, server, caddy) via supervisord. This is opaque to operators.
5. **No pre-built images** — there's no image on Docker Hub or GHCR. Every operator must `git clone` + `docker compose build`.
6. **Updates require rebuilding**`git pull && docker compose build && docker compose up -d`. No `docker compose pull`.
7. **manage.sh is 100+ lines** of bash wrapping `docker compose` with state files, confirmations, and color output. It's helpful for the maintainer but intimidating for new operators.
### What works well
- **Dockerfile is solid** — multi-stage Go build, Alpine runtime, small image
- **Health checks exist**`wget -qO- http://localhost:3000/api/stats`
- **Environment variable overrides** — ports and data dirs are configurable via `.env`
- **Data persistence** — bind mounts for DB (`~/meshcore-data`), named volume for Caddy certs
- **DISABLE_MOSQUITTO flag** — can use external MQTT broker
- **Graceful shutdown**`stop_grace_period: 30s`, SIGTERM handling
### What's painful
| Pain Point | Impact |
|---|---|
| Must build from source | Blocks anyone without Go/Docker buildx knowledge |
| 100-line config.json required | Operator doesn't know what's optional vs required |
| No sensible defaults for MQTT | Can't connect to public mesh without credentials |
| No pre-built multi-arch images | ARM users (Raspberry Pi) must cross-compile |
| No one-line deploy | Minimum 4 steps: clone, configure, build, start |
| Updates = rebuild | Slow, error-prone, requires git |
## Goal
An operator who has never seen the codebase should be able to run CoreScope with:
```bash
docker run -d -p 80:80 -v corescope-data:/app/data ghcr.io/kpa-clawbot/corescope:v3.4.1
```
And see live MeshCore packets from the public mesh within 60 seconds.
## Pre-built Images
Publish to **GHCR** (`ghcr.io/kpa-clawbot/corescope`) on every release tag.
- **Tags:**
- `vX.Y.Z` (e.g., `v3.4.1`) — specific release, pinned, recommended for production
- `vX.Y` (e.g., `v3.4`) — latest patch in a minor series, auto-updates patches only
- `vX` (e.g., `v3`) — latest minor+patch in a major series
- `latest` — latest release tag (NOT latest commit). Only moves on tagged releases, never on random master commits. Still, production deployments should pin to `vX.Y.Z`
- `edge` — built from master on every push. Unstable, for testing only. Clearly labeled as such
- **Architectures:** `linux/amd64`, `linux/arm64` (Raspberry Pi 4/5)
- **Build trigger:** GitHub Actions on `v*` tag push
- **CI workflow:** New job `publish` after existing `deploy`, uses `docker/build-push-action` with QEMU for multi-arch
```yaml
# .github/workflows/publish.yml (simplified)
on:
push:
tags: ['v*']
jobs:
publish:
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- uses: actions/checkout@v5
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action@v6
with:
push: true
platforms: linux/amd64,linux/arm64
tags: |
ghcr.io/kpa-clawbot/corescope:v3.4.1
ghcr.io/kpa-clawbot/corescope:${{ github.ref_name }}
build-args: |
APP_VERSION=${{ github.ref_name }}
GIT_COMMIT=${{ github.sha }}
BUILD_TIME=${{ github.event.head_commit.timestamp }}
```
## Configuration
### Hierarchy (highest priority wins)
1. **Environment variables**`CORESCOPE_MQTT_BROKER`, `CORESCOPE_PORT`, etc.
2. **`/app/data/config.json`** — full config file (volume-mounted)
3. **Built-in defaults** — work out of the box
### Environment variables for common settings
| Variable | Default | Description |
|---|---|---|
| `CORESCOPE_MQTT_BROKER` | `mqtt://localhost:1883` | Primary MQTT broker URL |
| `CORESCOPE_MQTT_TOPIC` | `meshcore/+/+/packets` | MQTT topic pattern |
| `CORESCOPE_PORT` | `3000` | HTTP server port (internal) |
| `CORESCOPE_DB_PATH` | `/app/data/meshcore.db` | SQLite database path |
| `CORESCOPE_SITE_NAME` | `CoreScope` | Branding site name |
| `CORESCOPE_DEFAULT_REGION` | (none) | Default region filter |
| `DISABLE_MOSQUITTO` | `false` | Skip internal MQTT broker |
| `DISABLE_CADDY` | `false` | Skip internal Caddy (when behind reverse proxy) |
### Built-in defaults that work out of the box
The Go server and ingestor already have reasonable defaults compiled in. The only missing piece is **a default public MQTT source** so a fresh instance can see packets immediately. Options:
- **Option A:** Ship with the internal Mosquitto broker only (no external sources). Operator sees an empty dashboard and must configure MQTT. Safe but unhelpful.
- **Option B:** Ship with a public read-only MQTT source pre-configured (e.g., `mqtt.meshtastic.org` or equivalent if one exists for MeshCore). Operator sees live data immediately. Better UX.
**Recommendation:** Option A as default (safe), with a documented one-liner to add a public source. The config.example.json already shows how to add `mqttSources`.
## Compose Profiles
A single `docker-compose.yml` with profiles:
```yaml
services:
corescope:
image: ghcr.io/kpa-clawbot/corescope:v3.4.1
profiles: ["", "standard", "full"] # runs in all profiles
ports:
- "${HTTP_PORT:-80}:80"
volumes:
- ${DATA_DIR:-./data}:/app/data
environment:
- DISABLE_MOSQUITTO=${DISABLE_MOSQUITTO:-false}
- DISABLE_CADDY=${DISABLE_CADDY:-false}
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
interval: 30s
timeout: 5s
retries: 3
restart: unless-stopped
```
**Note:** Since the container already bundles mosquitto + caddy + server + ingestor via supervisord, "profiles" are really just env var toggles:
| Profile | DISABLE_MOSQUITTO | DISABLE_CADDY | Use case |
|---|---|---|---|
| **minimal** | `true` | `true` | External MQTT + external reverse proxy |
| **standard** (default) | `false` | `true` | Internal MQTT, no TLS (behind nginx/traefik) |
| **full** | `false` | `false` | Everything including Caddy auto-TLS |
This avoids splitting into separate compose services. The monolithic container is actually fine for this use case — it's a single-purpose appliance.
## One-Line Deploy
### Simplest (Docker run, no TLS)
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
-e DISABLE_CADDY=true \
ghcr.io/kpa-clawbot/corescope:v3.4.1
```
### With Docker Compose
```bash
curl -sL https://raw.githubusercontent.com/Kpa-clawbot/CoreScope/master/docker-compose.simple.yml -o docker-compose.yml
docker compose up -d
```
Where `docker-compose.simple.yml` is a minimal 15-line file shipped in the repo.
## Update Path
```bash
docker compose pull
docker compose up -d
```
Or for `docker run` users:
```bash
docker pull ghcr.io/kpa-clawbot/corescope:v3.4.1
docker stop corescope && docker rm corescope
docker run -d --name corescope ... # same args as before
```
No rebuild. No git pull. No source code needed.
## Data Persistence
| Path | Content | Mount |
|---|---|---|
| `/app/data/meshcore.db` | SQLite database (all packets, nodes) | Required volume |
| `/app/data/config.json` | Custom configuration (optional) | Same volume |
| `/app/data/theme.json` | Custom theme (optional) | Same volume |
| `/data/caddy` | TLS certificates (Caddy-managed) | Named volume (automatic) |
**Backup:** `cp ~/corescope-data/meshcore.db ~/backup/` — it's just a SQLite file.
**Migration:** Existing `~/meshcore-data` directories work unchanged. Just point the volume at the same path.
## TLS/HTTPS
### Option 1: Caddy auto-TLS (built-in)
The container ships Caddy. To enable auto-TLS:
1. Mount a custom Caddyfile:
```bash
docker run -d \
-p 80:80 -p 443:443 \
-v corescope-data:/app/data \
-v caddy-certs:/data/caddy \
-v ./Caddyfile:/etc/caddy/Caddyfile:ro \
ghcr.io/kpa-clawbot/corescope:v3.4.1
```
2. Caddyfile:
```
your-domain.com {
reverse_proxy localhost:3000
}
```
### Option 2: External reverse proxy (recommended for production)
Run with `DISABLE_CADDY=true` and put nginx/traefik/cloudflare in front. This is the standard approach and what most operators already have.
## Health Checks
Already implemented. The container health check hits `/api/stats`:
```bash
# From outside the container
curl -f http://localhost/api/stats
# Response includes packet counts, node counts, uptime
```
Docker will mark the container as `healthy`/`unhealthy` automatically.
## Monitoring
**Future (M5 from RF health spec):** Expose a `/metrics` Prometheus endpoint with:
- `corescope_packets_total` — total packets ingested
- `corescope_nodes_active` — currently active nodes
- `corescope_mqtt_connected` — MQTT connection status
- `corescope_ingestor_lag_seconds` — time since last packet
This is not required for the deployment simplification work but should be designed alongside it.
## Migration from Current Setup
For existing operators using `manage.sh` + build-from-source:
1. **Keep your data directory** — the bind mount path is the same
2. **Keep your config.json** — it goes in the data directory as before
3. **Replace `docker compose build`** with `docker compose pull`
4. **Update docker-compose.yml** — change `build:` to `image: ghcr.io/kpa-clawbot/corescope:v3.4.1`
5. **manage.sh continues to work** — it wraps `docker compose` and will work with pre-built images
**Breaking changes:** None expected. The container interface (ports, volumes, env vars) stays the same.
## Milestones
### M1: Pre-built images (1-2 days)
- [ ] Create `.github/workflows/publish.yml` for multi-arch builds
- [ ] Push a test `v0.x.0` tag and verify image on GHCR
- [ ] Update README with `docker run` quickstart
- [ ] Create `docker-compose.simple.yml` (minimal compose file using pre-built image)
### M2: Environment variable configuration (1 day)
- [ ] Add env var parsing to Go server `config.go` (overlay on config.json)
- [ ] Add env var parsing to Go ingestor
- [ ] Add `DISABLE_CADDY` support to `entrypoint-go.sh`
- [ ] Document all env vars in README
### M3: Sensible defaults (0.5 day)
- [ ] Ensure server starts with zero config (no config.json required)
- [ ] Verify ingestor connects to localhost MQTT by default
- [ ] Test: `docker run` with no config produces a working (empty) dashboard
### M4: Documentation + migration guide (0.5 day)
- [ ] Write operator-facing deployment docs in `docs/deployment.md`
- [ ] Migration guide for existing users
- [ ] One-page quickstart
**Total estimate:** 3-4 days of work.
## Torvalds Review
> "Is this over-engineered?"
The spec is intentionally simple. Key decisions:
1. **No Kubernetes manifests, Helm charts, or Terraform.** Just Docker.
2. **No config management system.** Env vars + optional JSON file.
3. **Keep the monolithic container.** Splitting into 4 separate services (server, ingestor, mosquitto, caddy) would be "proper" microservices but is worse for operators who just want one thing to run. The supervisord approach is fine for an appliance.
4. **No custom CLI tool.** `docker compose` is the interface.
5. **Profiles are just env vars**, not separate compose files or services.
The simplest version is literally just M1: publish the existing image to GHCR. Everything else is polish. An operator can already `docker run` the image — they just can't `docker pull` it because it's not published anywhere.
+141
View File
@@ -0,0 +1,141 @@
# Movable UI Panels — Draggable Panel Positioning
**Status:** Proposed
**Related:** #279 (original request), PR #606 (collapsible panels — immediate fix)
**Date:** 2026-04-05
---
## Problem
The live map page overlays several UI panels on the map viewport: legend, live feed, node detail, and filters. On smaller screens or dense deployments, these panels obscure map content. Users have no control over where panels sit — they're CSS-fixed in corners, and when they collide with each other or with map data, the only option is to close them entirely. Closing a panel means losing access to the data it shows.
PR #606 addresses the immediate pain with collapsible panels and responsive breakpoints. This spec covers the next step: letting users reposition panels to wherever serves their workflow best.
## Solution
Panels become draggable within the map viewport. Users grab a handle, drag to a new position, release. Position persists in `localStorage` per panel ID. That's it.
### What each panel gets
| Affordance | Behavior |
|---|---|
| **Drag handle** | A subtle grip indicator (6-dot grid or `⋮⋮`) in the panel header. Cursor changes to `grab`/`grabbing`. The handle is the ONLY drag target — the panel body remains interactive (scrollable, clickable). |
| **Snap-to-edge** | When released within 20px of a viewport edge, the panel snaps flush to that edge. Prevents panels floating 3px from the side looking broken. |
| **Position persistence** | `localStorage` key per panel: `panel-pos-{id}``{ x, y }` as viewport percentages (not pixels — survives resize). |
| **Z-index on focus** | Clicking or dragging a panel brings it to front. Simple incrementing counter, reset on page load. |
| **Reset button** | Single button (in settings or as a map control) resets ALL panels to default positions. Clears all `panel-pos-*` keys. |
### What we do NOT build
- **Resizable panels.** Drag-to-resize adds complexity for marginal benefit. Panels have natural content-driven sizes.
- **Docking/tiling/splitting.** This is not a window manager. No snap-to-other-panel, no split view, no tiling grid.
- **Panel minimization to a taskbar.** Collapsible (PR #606) is sufficient.
- **Drag on mobile.** Touch-drag conflicts with map pan. Mobile keeps collapsible behavior from PR #606. Draggable is desktop-only (`pointer: fine` media query).
## Design Considerations
### Drag handle affordance
The handle must be visible enough that users discover it, but not so prominent that it becomes visual noise. A 6-dot grip icon (`⋮⋮`) in the panel title bar, styled at 60% opacity, rising to 100% on hover. The cursor change (`grab``grabbing`) provides the primary affordance.
### Snap-to-edge
Panels snap to the nearest edge when released within a 20px threshold. Snap positions: top-left, top-right, bottom-left, bottom-right, or any edge midpoint. This prevents the "floating at 47px from the left" awkwardness without constraining users to a rigid grid.
### Position persistence
Positions stored as viewport percentages: `{ xPct: 0.02, yPct: 0.15 }`. On window resize, panels stay proportionally positioned. If a resize would push a panel off-screen, clamp it to the nearest visible edge.
### Responsive breakpoints
Below the medium breakpoint (defined in PR #606), panels revert to their fixed/collapsible positions. The draggable behavior is a progressive enhancement for viewports wide enough to have meaningful repositioning space. Persisted positions are preserved in `localStorage` but not applied until the viewport is wide enough again.
### Z-index management
A module-level counter starting at 1000. Each panel interaction (click, drag start) sets that panel's z-index to `++counter`. On page load, counter resets to 1000. No panel can exceed z-index 9999 (modal/overlay territory) — if counter approaches that, compact all panel z-indices down.
### Accessibility
- Panels are focusable (`tabindex="0"` on the drag handle).
- Arrow keys reposition the focused panel by 10px per press (Shift+Arrow = 50px).
- `Escape` while dragging cancels and returns to the previous position.
- `Home` key resets the focused panel to its default position.
- Screen readers: `aria-label="Drag handle for {panel name}. Use arrow keys to reposition."` and `role="slider"` with `aria-valuenow` reflecting position.
## Implementation
### Milestones
**M1: Core drag mechanics** (~2 days)
- `DragManager` class: registers panels, handles pointer events, updates positions
- Snap-to-edge logic
- Z-index management
- No persistence yet — positions reset on reload
**M2: Persistence + reset** (~1 day)
- `localStorage` read/write for panel positions
- Reset-to-defaults button
- Viewport-percentage storage with resize clamping
**M3: Responsive + accessibility** (~1 day)
- Disable drag below medium breakpoint
- Keyboard repositioning (arrow keys)
- ARIA attributes
- Screen reader announcements on position change
**M4: Polish + testing** (~1 day)
- Playwright E2E tests: drag, snap, persist, reset, keyboard
- Performance validation: drag must not trigger layout thrash (use `transform: translate()`, not `top/left`)
- Edge case handling (see below)
### Technical approach
- **No library.** Pointer events (`pointerdown`, `pointermove`, `pointerup`) with `setPointerCapture`. ~150 lines of vanilla JS.
- **CSS transforms for positioning.** `transform: translate(Xpx, Ypx)` avoids layout reflow during drag. Only write to `style.transform`, never `top`/`left`.
- **Debounce persistence.** Write to `localStorage` on `pointerup`, not during drag.
- **Single file:** `public/drag-manager.js` — imported by `live.js`, no other dependencies.
## Edge Cases
| Case | Handling |
|---|---|
| Panel dragged partially off-screen | Clamp to viewport bounds on `pointerup` |
| Window resized while panel is near edge | Re-clamp on `resize` (debounced 200ms) |
| Two panels overlap after drag | Allowed — z-index determines which is on top. Users can move them. |
| `localStorage` full or unavailable | Graceful fallback to default positions. No error shown. |
| Panel content changes size after drag | Panel stays at dragged position; content reflows within. If panel grows past viewport edge, clamp. |
| User has old `localStorage` keys from a removed panel | Ignore unknown keys on load. Clean up stale keys on reset. |
| RTL layouts | Snap logic uses physical viewport edges, not logical start/end. Drag is inherently physical. |
## Expert Reviews
### Tufte (Information Design)
- **Draggability is justified** only if it serves data access — and here it does. Panels obscuring map data is a data-visibility problem, not a UI-decoration problem. Letting users clear their sightlines to the data is correct.
- **The drag handle must be minimal.** Six dots at 60% opacity is acceptable. Anything more prominent (colored bars, icons, labels) becomes chartjunk — UI chrome competing with data for attention.
- **Resist feature creep.** Resizable panels, docking zones, panel-to-panel snapping — all increase interface complexity without increasing data throughput. The spec correctly excludes these.
- **Snap-to-edge is good.** It prevents the visual noise of arbitrarily placed rectangles. Panels aligned to edges create clean negative space for the map data.
### Torvalds (Engineering Pragmatism)
- **This is borderline over-engineering.** The real question: do users actually need free-form drag, or would a simpler "pick a corner" toggle (TL/TR/BL/BR) cover 95% of use cases with 20% of the code?
- **The 4-corner toggle would be ~40 lines.** The full drag system is ~150+ lines plus persistence, snap logic, accessibility, resize handling, z-index management, and edge cases. That's a lot of surface area for "I want the legend on the right instead of the left."
- **Recommendation:** Ship the 4-corner toggle first (M0). If users actually request free-form drag after that, build it. YAGNI applies here.
- **If you do build drag:** the spec is sound. Pointer events + transforms + localStorage is the right stack. No library is correct. But test it on Firefox — pointer capture has quirks.
### Doshi (Product/Business)
- **This is an N (Nice-to-have), not an L (Leverage).** It improves UX for power users who spend hours on the live map, but it doesn't unlock new capabilities or new users.
- **Opportunity cost:** 5 developer-days on draggable panels is 5 days not spent on features that expand what CoreScope can do (new analytics, alerting, multi-site support).
- **The collapsible panels (PR #606) likely resolve the P1 pain.** Track whether users still complain about panel placement after #606 ships. If complaints drop to zero, this spec can stay on the shelf.
- **If built:** ship M1+M2 only (3 days). M3 accessibility can come later if adoption warrants it. M4 testing is non-negotiable.
### Feedback incorporated
Based on the reviews, the spec adds a **Milestone 0** recommendation:
**M0: Corner-position toggle** (~0.5 days)
Before building full drag, ship a simpler panel-position toggle: each panel's header gets a small button that cycles through TL → TR → BR → BL placement. Positions persist in `localStorage`. If this satisfies user needs, M1M4 become unnecessary.
**Decision gate:** Ship M0 with PR #606 or shortly after. Monitor feedback for 2 weeks. If users request free-form repositioning, proceed to M1. If corner toggle is sufficient, close this spec as "resolved by M0."
+493
View File
@@ -0,0 +1,493 @@
# Spec: RF Health Dashboard — Observer Radio Metrics
**Status:** Draft v3
**Purpose:** Enable operators to quickly identify RF jammers, deaf receivers, and radio health issues through per-observer time-series charts.
## Prerequisite Gate
**Before building anything, verify that stats messages arrive periodically from observers.**
The ingestor must receive radio stats messages at a predictable interval via MQTT. Confirmed: status messages arrive every ~5 minutes per observer.
**Verification steps (M0):**
1. Connect ≥3 observers to the MQTT bridge
2. Log all incoming stats messages with timestamps for 24h
3. Confirm messages arrive at a regular interval (expected: every few minutes)
4. If stats are NOT periodic, stop — a stats-request mechanism must be added to the MQTT bridge first (separate spec)
5. **Verify `triggerNoiseFloorCalibrate()` firing frequency.** If it fires on every stats cycle, noise floor readings may be artificially consistent (measuring calibration, not environment). If it fires only on boot, the first sample after reboot is unreliable — document which behavior the firmware uses.
Do not proceed to M1 until this gate passes.
## Problem
Operators currently have no visibility into RF environment quality over time. A jammer could be active for hours before anyone notices degraded mesh performance. A deaf receiver silently drops packets with no alert. There's no way to distinguish "the mesh is quiet" from "my observer can't hear anything."
## Solution
A new Analytics tab ("RF Health") showing per-observer time-series charts for noise floor, TX airtime, RX airtime, and receive errors over configurable time windows (1h to 30d, plus custom from/to range). Automated pattern detection (M3+) flags anomalies and suggests diagnoses after operators have used raw charts to provide feedback.
## Data Model
### New table: `observer_metrics`
```sql
CREATE TABLE IF NOT EXISTS observer_metrics (
observer_id TEXT NOT NULL,
timestamp TEXT NOT NULL, -- ISO 8601, rounded to nearest sample interval
noise_floor REAL, -- dBm, from radio stats (nullable — may arrive without airtime)
tx_air_secs INTEGER, -- cumulative TX seconds since boot (nullable)
rx_air_secs INTEGER, -- cumulative RX seconds since boot (nullable)
packets_sent INTEGER, -- cumulative packets sent since boot (nullable)
packets_recv INTEGER, -- cumulative packets received since boot (nullable)
recv_errors INTEGER, -- cumulative CRC/decode failures since boot (nullable)
battery_mv INTEGER, -- battery voltage in millivolts (nullable, for field/solar nodes)
PRIMARY KEY (observer_id, timestamp)
);
```
**Field notes:**
- **`recv_errors`** (CRC failure count) is the strongest single indicator of channel quality. A rising error rate with stable noise floor points to in-band digital interference rather than broadband jamming. This is more diagnostic than packet_count alone.
- **`packets_sent` / `packets_recv`** are tracked separately because the ratio reveals asymmetric link problems (e.g., observer can transmit but not receive, or vice versa). The old `packet_count` field conflated these.
- **`battery_mv`** is nullable and only relevant for field/solar deployments. Low battery causes erratic radio behavior (reduced TX power, missed RX windows) that looks like RF problems but isn't. Charting voltage alongside RF metrics prevents misdiagnosis.
- All cumulative counters (`tx_air_secs`, `rx_air_secs`, `packets_sent`, `packets_recv`, `recv_errors`) reset on reboot — see reboot handling below.
No additional indexes. The composite primary key covers all query patterns (per-observer time-range scans). At 70K rows, a full scan for any fleet-wide time query is fast enough.
### Clock source
**Always use the ingestor's wall clock for timestamps, not observer-reported timestamps.** Observer clocks may be wrong, drifted, or absent (no RTC). Round the ingestor wall clock to the nearest sample interval boundary (e.g., 5-minute marks) for consistent time alignment.
### Noise floor cold start caveat
**The first noise floor sample after a reboot may be unreliable.** The radio's noise floor reading requires settling time and may reflect calibration artifacts rather than the actual RF environment. Mark the first post-reboot sample with a `reboot` flag (see reboot handling) so the frontend can annotate it. Do not use first-post-reboot noise floor samples in baseline/median calculations.
### Sampling strategy
- **Interval:** Every 5 minutes (configurable via config.json `metrics.sampleIntervalSec`, default 300)
- **Source:** MQTT stats messages (`STATS_TYPE_RADIO`)
- **Insertion:** `INSERT OR REPLACE INTO observer_metrics (observer_id, timestamp, ...) VALUES (?, ?, ...)` with timestamp rounded to the nearest interval boundary. No need to track last-insert time per observer — rounding + `INSERT OR REPLACE` is idempotent and naturally deduplicates.
- **Storage:** ~10K rows/day for 35 observers. At configurable retention. Negligible.
- **Retention:** Configurable, configurable, default 30 days. Prune with a single `DELETE FROM observer_metrics WHERE timestamp < datetime('now', '-N days')` on startup and every 24h. Consider `PRAGMA auto_vacuum = INCREMENTAL` for embedded devices.
### Gap detection
If the time between two consecutive samples for an observer exceeds 2× the sample interval (e.g., >10 minutes for a 5-min interval), insert null values in the response to indicate a gap. This prevents charts from drawing misleading interpolation lines across outages.
### Reboot handling
Cumulative counters (`tx_air_secs`, `rx_air_secs`, `packets_sent`, `packets_recv`, `recv_errors`) reset on device reboot. Detect counter resets (current value < previous value) and:
1. Skip the delta computation for that interval (do not produce a negative value)
2. Log a reboot event for the observer with the timestamp
3. Use the current sample as the new baseline for subsequent deltas
4. **Include reboot timestamps in the API response** so the frontend can render them as annotations directly on the chart (see frontend design)
5. **Flag the first post-reboot noise floor sample** as potentially unreliable (cold start — see above)
### Delta computation (server-side)
Cumulative counters are converted to per-interval rates server-side. **Deltas are computed server-side, not in the frontend.** The API returns percentage/rate values directly. This keeps firmware implementation details (cumulative counters, reboot semantics) out of the UI layer, reduces payload size, and centralizes reboot-handling logic.
### Graceful degradation
Not all observers may report all metrics. If fields are absent:
- Store `NULL` for missing columns
- The API returns `null` for unavailable fields
- The frontend shows only the charts for which data exists — missing charts are hidden, not broken
- Status detection uses only available metrics
- `battery_mv` is expected to be absent on mains-powered observers — this is normal, not an error
Partial data is always better than no data. Never error or crash on missing optional fields.
### Required ingestor changes
1. Parse `tx_air_secs`, `rx_air_secs`, `packets_sent`, `packets_recv`, `recv_errors`, and `battery_mv` from MQTT stats messages (same pattern as existing `noise_floor`)
2. On each stats message, round ingestor wall clock to nearest interval, `INSERT OR REPLACE` into `observer_metrics`
3. Handle missing fields gracefully (insert NULLs for absent metrics)
4. Detect counter resets and record reboot events
5. Add new columns to `observers` table for current/latest values
### API endpoints
```
GET /api/observers/{id}/metrics?since=2026-04-04T00:00:00Z&until=2026-04-05T00:00:00Z&resolution=5m
```
**`resolution` query parameter** controls downsampling:
- `5m` (default) — raw samples
- `1h` — hourly aggregates (`GROUP BY strftime('%Y-%m-%dT%H:00:00', timestamp)` with MIN/MAX/AVG)
- `1d` — daily aggregates
Use `1h` resolution for 7d views to avoid shipping 2,016 points per observer. Essential for the fleet comparison view (35 observers × 2,016 = 70K points at raw resolution → 35 × 168 = 5,880 points at 1h resolution).
Returns:
```json
{
"observer_id": "1F445B...",
"observer_name": "GY889 Repeater",
"reboots": ["2026-04-04T03:15:00Z", "2026-04-04T18:22:00Z"],
"metrics": [
{
"timestamp": "2026-04-04T00:00:00Z",
"noise_floor": -112.5,
"tx_airtime_pct": 2.1,
"rx_airtime_pct": 8.3,
"packets_sent": 42,
"packets_recv": 342,
"recv_errors": 3,
"recv_error_rate": 0.87,
"battery_mv": 3720,
"is_reboot_sample": false
}
]
}
```
Notes:
- `tx_airtime_pct` and `rx_airtime_pct` are server-computed deltas as percentages. Null if airtime data unavailable.
- `recv_error_rate` = `recv_errors / (packets_recv + recv_errors)` as a percentage. Null if either field unavailable.
- `packets_sent` and `packets_recv` are per-interval deltas (not cumulative). Null if unavailable.
- `reboots` array contains timestamps of detected reboots within the queried window, for chart annotation.
- `is_reboot_sample` flags first-post-reboot samples where noise floor may be unreliable.
- `battery_mv` is null for mains-powered observers.
```
GET /api/observers/metrics/summary?window=24h
```
**Fleet summary is cached incrementally.** Maintain a rolling summary struct in memory, updated on each new sample insert (35 observers × 1 sample/5min = 7 inserts/min — trivially cheap). The endpoint reads from the cached struct, not from SQLite queries on every request.
Returns:
```json
{
"observers": [
{
"observer_id": "1F445B...",
"observer_name": "GY889 Repeater",
"current_noise_floor": -112.5,
"avg_noise_floor_24h": -114.2,
"max_noise_floor_24h": -95.0,
"tx_airtime_pct_24h": 2.1,
"rx_airtime_pct_24h": 8.3,
"recv_error_rate_24h": 0.87,
"battery_mv": 3720,
"status": "normal"
}
]
}
```
## Frontend Design
### Design Principles
The dashboard exists for one purpose: **let an operator glance at it at 3 AM and know immediately if something is wrong.** Every design decision follows from this. Decoration that doesn't serve comprehension is removed. Data that can be shown is shown — not hidden behind clicks or hovers.
Key rules (per Tufte):
- **Maximize data-ink ratio.** Every pixel must encode data or directly support reading it. Remove anything that doesn't.
- **No chartjunk.** No gradient fills, no 3D effects, no decorative borders, no ornamental chrome.
- **Labels on the data, not in legends.** Direct-label lines, annotate anomalies at the point they occur. The viewer should never look away from the data to understand it.
- **Show data variation, not design variation.** All observer charts use identical scales, formats, and typography. If two charts look different, it's because the data is different.
- **Respect the viewer's intelligence.** Dense, information-rich displays are fine. Oversimplified displays waste screen space and the operator's time.
### Page structure: small multiples grid
```
Analytics → RF Health tab
├── Time range: [1h] [3h] [6h] [12h] [24h] [3d] [7d] [30d] [Custom ▾]
│ ├── Presets: click to quick-set
│ └── Custom: two datetime inputs (from/to) with calendar picker
│ └── URL hash reflects selected range for deep linking
├── Small Multiples Grid (ALL observers, one cell per observer)
│ │
│ │ Each cell contains:
│ │ ┌─────────────────────────────────────────┐
│ │ │ GY889 Repeater -112.5 dBm 3.7V│ ← name, current NF, battery (if field node)
│ │ │ ┈┈┈╲┈┈┈┈┈┈╱┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈│ ← noise floor sparkline (24h)
│ │ │ err: 0.8% TX: 2.1% RX: 8.3% │ ← key rates, inline text
│ │ │ ▲reboot 03:15 │ ← reboot annotation (if any)
│ │ └─────────────────────────────────────────┘
│ │
│ │ Sorted by: worst status first, then highest noise floor
│ │ Grid: 34 columns on desktop, 2 on tablet, 1 on phone
│ │ Click any cell → expand to full detail below
│ │
│ └── Entire grid is visible at once — no pagination, no "show more"
│ (35 observers × ~60px per cell = ~700px — fits on one screen)
├── Expanded Detail (shown below grid when a cell is clicked)
│ │
│ │ Three time-aligned charts, stacked vertically, sharing X-axis:
│ │
│ │ 1. Noise Floor (dBm)
│ │ - SVG line chart, Y-axis inverted (higher dBm = worse = higher on chart)
│ │ - Thin reference lines at -100 dBm and -85 dBm, directly labeled
│ │ (e.g., "100 warning" / "85 critical") — no color bands
│ │ - Gaps (nulls) break the line — no interpolation across outages
│ │ - Reboot markers: vertical hairline at each reboot timestamp,
│ │ labeled "reboot" directly on the chart
│ │ - First-post-reboot sample marked with open circle (unreliable cold start)
│ │ - Direct labels on notable points (min, max, anomalies)
│ │
│ │ 2. Airtime (%) — hidden if no airtime data
│ │ - Two separate SVG lines (NOT stacked area — stacked areas
│ │ make it impossible to read the lower series accurately)
│ │ - TX line and RX line, directly labeled at their endpoints
│ │ ("TX 2.1%" / "RX 8.3%") — no legend box
│ │ - Same X-axis as noise floor chart above
│ │ - Gaps shown as breaks
│ │
│ │ 3. Channel Quality
│ │ - Receive error rate (%) as a line
│ │ - Packets recv as a light step-line for context
│ │ - Directly labeled — no legend
│ │ - High error rate + low packet count = dead channel
│ │ - High error rate + high packet count = interference
│ │
│ │ 4. Battery Voltage (shown only if battery_mv is non-null)
│ │ - Simple line chart, mV scale
│ │ - Directly labeled with current value
│ │ - Useful for correlating RF anomalies with low-battery behavior
│ │
│ │ All four charts share the same X-axis and time range.
│ │ Reboot markers appear as vertical hairlines across ALL charts
│ │ (same event, visible in all contexts — no hunting).
│ │
│ └── Current values shown as text below charts:
│ NF: 112.5 dBm | TX: 2.1% | RX: 8.3% | Err: 0.87% | Batt: 3.72V
│ 24h: avg 114.2 | max 95.0 | 3 reboots
└── Fleet Comparison (M4)
└── Small multiples of noise floor, one per observer, identical Y-scale
└── NOT an overlay chart — overlays become unreadable past 5 lines
└── Use 1h resolution for 7d views
```
### Why small multiples, not expandable accordion
An accordion (expand/collapse per observer) forces the operator to click through each observer sequentially. At 3 AM with 35 observers, that's unacceptable. The small multiples grid shows ALL observers simultaneously — the eye does the comparison, not the mouse. Anomalies pop out visually because they break the pattern of the grid. This is Tufte's core insight: **small multiples leverage the viewer's ability to detect pattern breaks across a consistent visual template.**
### Why no color bands on charts
Color bands (green/yellow/red zones) are decorative — they add ink that doesn't encode data. They also pre-judge what's "good" and "bad," which varies by deployment environment. Instead, use **thin reference lines with direct text labels** at the warning and critical thresholds. The reference lines take up negligible ink, the labels are informational, and the operator's eye naturally compares the data line against them.
### Why not stacked area for airtime
Stacked area charts are a common source of graphical dishonesty. The bottom series (TX) reads correctly against the X-axis, but the top series (RX) reads against the TX boundary — making it impossible to accurately judge RX values without mental subtraction. Two separate lines, directly labeled, are always more honest and more readable.
### Color usage
Color encodes data category, never decoration:
- **Noise floor line:** single muted color (the line IS the data — it doesn't need to be loud)
- **TX / RX lines:** two distinct colors, directly labeled at endpoints (no legend needed)
- **Error rate:** a third distinct color
- **Reboot markers:** gray hairlines (de-emphasized — context, not data)
- **Status text in grid cells:** text color only (not background fill) — red text for critical, amber for warning, default for normal
- No background color fills on cards. No colored borders. No badge backgrounds. Color on text only where it carries meaning.
### Labels and annotations
- **Reference lines** at threshold values, labeled directly ("100 dBm warning")
- **Reboot events** as vertical hairlines across all charts, labeled "reboot" at the top
- **Cold-start samples** marked with open circles and a subtle "?" annotation
- **Current values** as inline text on the sparkline cells and below detail charts
- **No separate legends.** Lines are labeled at their endpoints or directly on the chart.
- **Hover** shows exact timestamp + value — this is the only interactive element, and it reveals precision, not hidden data
### Data density
- The small multiples grid fits 35 observers in ~700px vertical space (one screen on desktop)
- Each cell is information-dense: name + current value + sparkline + rates + reboot count — all visible without clicking
- Detail charts are stacked vertically sharing the X-axis, eliminating redundant time labels
- No wasted whitespace between chart panels — they are a single visual unit
### Information hierarchy (3 AM glance test)
1. **Grid scan (2 seconds):** Are all sparklines flat and similar? Yes → everything's fine. One cell has a spike or red text → that's the problem.
2. **Cell read (3 seconds):** Which observer, what's the current NF, what's the error rate? All visible without clicking.
3. **Detail dive (10 seconds):** Click the cell, see time-series context, see if it correlates with reboots, check battery, check airtime.
An operator never needs to click anything to know if the fleet is healthy. Clicking only provides temporal detail for diagnosis.
### Mobile considerations
- Grid collapses to 1 column on phone (each cell is full-width, still showing sparkline + values)
- Detail charts fill the viewport width, Y-axis labels move above the chart to save horizontal space
- Touch targets: the entire grid cell is tappable (not a small icon)
- Time range selector uses segmented control (large touch targets) for presets, not a dropdown
- Custom range picker: two datetime inputs with calendar popup, positioned below the presets
- Selected range (preset or custom) persists in URL hash: `&range=24h` or `&from=2026-04-04T14:00:00Z&to=2026-04-04T16:00:00Z`
### Chart rendering
**Use SVG, not Canvas.** The existing analytics.js uses SVG for all charts (sparklines, bar charts, histograms). Canvas is only used for the force-directed neighbor graph. Follow the existing SVG patterns — reuse `sparkSvg()` for fleet overview sparklines.
2,016 SVG polyline points per chart is fine. For the fleet comparison view (M4), use hourly downsampling (168 points per observer) to avoid layout jank on mobile.
### Deep linking
```
#/analytics?tab=rf-health
#/analytics?tab=rf-health&observer=1F445B...&range=24h
```
## Pattern Detection (M3+)
**Pattern detection is deferred until after operators have used raw charts (M1M2) and provided feedback on what patterns actually matter.** Do not implement automated diagnosis until real-world usage informs the rules.
### Planned automated diagnosis
The server computes a `status` field per observer based on the last N samples:
| Pattern | Status | Indicator |
|---|---|---|
| NF stable, RX/TX normal, low error rate | `normal` | (no indicator — absence of alarm is the signal) |
| NF spike + RX drop (broadband interference) | `jammer_suspected` | Red text: "Jammer?" |
| NF normal, RX near zero, fleet active (≥5 observers) | `deaf` | Red text: "Deaf receiver" |
| High `recv_errors` rate + stable NF | `digital_interference` | Amber text: "CRC errors high" |
| TX approaching duty cycle warning | `tx_overload` | Amber text: "TX overload" |
| No samples in >15 min | `offline` | Gray text: "Offline" |
| NF gradually increasing over hours | `interference_trend` | Amber text: "Rising interference" |
| Battery voltage below threshold | `low_battery` | Amber text: "Low battery" |
**Jammer detection logic:** A jammer raises the noise floor AND causes RX to drop (the receiver can't hear legitimate signals over the interference). NF spike + RX spike would indicate a legitimate busy channel, not a jammer. The key signal is: NF goes up, RX goes down.
**Digital interference detection (new):** High `recv_errors` with a stable noise floor indicates in-band digital interference (another protocol sharing the frequency, or a malfunctioning node transmitting garbage). This is distinct from broadband jamming, which raises the noise floor. `recv_errors` is the strongest single signal for this.
**Deaf detection:** Requires a minimum fleet size of ≥5 active observers to establish a meaningful fleet median. With fewer observers, skip deaf detection — the sample size is too small for comparison.
### Status priority
When multiple status conditions apply simultaneously, use this priority order (highest first):
1. `offline` — no data trumps everything
2. `jammer_suspected` — active threat
3. `deaf` — hardware failure
4. `digital_interference` — channel quality issue
5. `tx_overload` — regulatory concern
6. `low_battery` — power issue causing RF symptoms
7. `interference_trend` — gradual degradation
8. `normal` — default
### Baseline computation
- **Baseline noise floor:** rolling median of last 24h, **excluding first-post-reboot samples** (cold start unreliable). Computed once on new sample arrival, cached — not recomputed per request.
- **Spike detection:** current sample exceeds an absolute threshold (configurable) AND exceeds baseline + spike delta. Both conditions must be met — a delta-only threshold could false-positive in environments where the absolute NF is already benign (e.g., -115 dBm + 15 dBm = -100 dBm, which is fine).
- **"Others active" check for deaf detection:** compare this observer's RX packet count against the fleet median. If this observer is <10% of fleet median AND fleet has ≥5 active observers, flag as potentially deaf.
- **Error rate baseline:** rolling average of `recv_error_rate` over 24h. Spike above 2× baseline triggers `digital_interference` status.
### Alert thresholds (configurable)
```json
{
"rfHealth": {
"noiseFloorWarning": -100,
"noiseFloorCritical": -85,
"spikeThresholdDb": 15,
"txDutyCycleWarning": 8,
"deafThresholdPct": 10,
"deafMinFleetSize": 5,
"offlineTimeoutSec": 900,
"sampleIntervalSec": 300,
"retentionDays": 30,
"errorRateWarning": 5,
"lowBatteryMv": 3300
}
}
```
Note: No hardcoded duty cycle limit line on charts. Duty cycle regulations vary by jurisdiction (e.g., 1% in EU 868MHz, 10% in some US ISM bands). The warning threshold is configurable but no "regulatory limit" line is drawn on charts.
## Implementation Milestones
### M0: Prerequisite — Verify stats message frequency ✅ PASSED
- **Confirmed 2026-04-05:** Live MQTT capture on staging shows status messages arriving every ~5 minutes per observer
- **Fields confirmed present:** `noise_floor`, `tx_air_secs`, `rx_air_secs`, `recv_errors`, `battery_mv`, `uptime_secs`
- **Fields NOT yet parsed by ingestor:** `tx_air_secs`, `rx_air_secs`, `recv_errors` (noise_floor and battery_mv already parsed)
- **Ingestor timestamps:** Use ingestor wall clock, not observer timestamps (confirmed in design)
- **Verified:** `triggerNoiseFloorCalibrate()` fires every 2 seconds (`NOISE_FLOOR_CALIB_INTERVAL = 2000ms` in `Dispatcher.cpp`). Continuous calibration with 64 RSSI samples per cycle. Noise floor data is always fresh.
- **Gate: PASSED.** Proceed to M1.
### M1: Store metrics + small multiples grid (MVP)
- Create `observer_metrics` table with all columns (migration)
- Ingestor: parse all available fields from stats, `INSERT OR REPLACE` with rounded timestamps
- Handle missing fields gracefully (store NULLs)
- Detect counter resets and record reboot events
- Add `/api/observers/{id}/metrics` endpoint (all available fields)
- Add `/api/observers/metrics/summary` endpoint (cached incrementally)
- Add "RF Health" tab to Analytics
- **Small multiples grid** with sparklines and inline values for all observers
- Per-observer detail view: noise floor line chart with reference lines (not color bands), reboot markers as vertical hairlines, cold-start sample annotation
- Time range selector (1h/3h/6h/12h/24h/3d/7d/30d + custom range picker)
- Deep linking
- Retention pruning
- Tests: sampling, insertion idempotency, retention, API responses, gap handling, reboot detection
### M2: Airtime + channel quality charts
- Server-side delta computation for all cumulative counters with reboot handling and gap detection
- Add `resolution` query param for downsampling (1h, 1d)
- Airtime charts: two separate lines (TX/RX), directly labeled — not stacked area
- Channel quality chart: recv_error_rate line + packets_recv step-line
- Battery voltage chart (shown only when data exists)
- All charts time-aligned, sharing X-axis, reboot markers spanning all charts
- Tests: delta computation, reboot handling, counter reset, gap insertion, downsampling, error rate calculation
#### M2 feedback improvements (post-M2)
- **Auto-scale airtime Y-axis**: clamp to min/max of actual data values (20% headroom, min 1%) instead of fixed 0-100%, matching noise floor chart behavior. Increases data-ink ratio for low-activity nodes.
- **Hover tooltips on all chart data points**: invisible SVG circles with `<title>` elements on every data point across all 4 charts (noise floor, airtime, error rate, battery). Shows exact value + UTC timestamp on hover. Detail-on-demand without cluttering the chart.
### M3: Pattern detection
- Implement after operators have used raw charts (M1M2) and provided feedback
- Jammer detection (NF spike + RX drop)
- Digital interference detection (high recv_errors + stable NF)
- Deaf receiver detection (with ≥5 fleet minimum)
- Low battery detection
- Interference trend detection
- Status text indicators with priority ordering (no emoji badges — text only)
- Baseline computation (rolling median excluding cold-start samples, cached)
- Configurable alert thresholds
- Tests: each pattern, edge cases, status priority
### M4: Fleet comparison + advanced views
- Fleet comparison as **small multiples** (one noise floor chart per observer, identical Y-scale) — not overlay
- Sort/filter fleet by status, noise floor, error rate
- Optional: per-observer historical baseline trend
- Use 1h resolution for 7d views
### M5: Metrics export — Prometheus / Grafana / external systems
- **Prometheus endpoint:** `GET /metrics` exposing observer radio metrics in Prometheus exposition format
- Gauges per observer: `corescope_observer_noise_floor_dbm{observer="...",name="..."}`, `corescope_observer_tx_air_secs_total`, `corescope_observer_rx_air_secs_total`, `corescope_observer_recv_errors_total`, `corescope_observer_battery_mv`, `corescope_observer_uptime_secs`
- Fleet-level: `corescope_observers_total`, `corescope_observers_online`
- Packet counters: `corescope_packets_total`, `corescope_observations_total`
- Standard `process_*` and `go_*` runtime metrics via `promhttp` handler
- **Configurable:** Enable/disable via `config.json` (`metrics.prometheusEnabled: true`, `metrics.prometheusPath: "/metrics"`)
- **Auth:** Optional bearer token or basic auth on the metrics endpoint (prevents public scraping)
- **Labels:** Each observer metric labeled with `observer` (pubkey), `name` (friendly name), `region`
- **Why Prometheus format:** Industry standard, compatible with Grafana, Datadog, Victoria Metrics, Mimir, and any OpenMetrics consumer. Operators who already run monitoring stacks can integrate CoreScope without any custom work.
- **Implementation:** Use Go `prometheus/client_golang` library. Register collectors that read from the in-memory `PacketStore` and `observer_metrics` table. No additional polling — just expose current state on each scrape.
- **Grafana dashboard template:** Ship a JSON dashboard template (`docs/grafana-dashboard.json`) that operators can import for instant RF health visualization in Grafana. Pre-configured panels matching the built-in RF Health tab.
- **OpenTelemetry (future):** If demand exists, add OTLP export alongside Prometheus. Not in M5 scope.
## Design Decisions
1. **Per-observer, not per-device.** Even if two observers share hardware, their RF environments may differ (different antennas, channels). observer_id is already the natural key.
2. **Poll-on-tab-switch, not WebSocket push.** Data changes every 5 minutes. Users check this tab when investigating issues, not for live monitoring. WebSocket push adds complexity for no UX benefit.
3. **SVG charts.** Matches existing analytics.js patterns. Canvas only if fleet comparison proves too slow with SVG.
4. **Server-side deltas.** Keeps firmware details out of the frontend. Single point for reboot/gap handling logic.
5. **Incremental fleet summary cache.** 7 inserts/min is trivially cheap to process. No need to query SQLite on every summary request.
6. **No standalone timestamp index.** The composite PK handles all query patterns. A standalone index wastes write amplification.
7. **Ingestor wall clock for timestamps.** Observer clocks are unreliable. Consistent time source prevents alignment issues.
8. **Small multiples over accordion/cards.** Enables instant visual fleet comparison without clicking. Anomalies break the visual pattern of the grid. (Tufte: "Small multiples are the best design solution for a wide range of problems in data presentation.")
9. **Reference lines, not color bands.** Color bands add non-data ink and pre-judge thresholds. Reference lines are minimal and informational.
10. **Two lines, not stacked area for airtime.** Stacked areas make the upper series unreadable. Two lines with direct labels are always more honest.
11. **Text status indicators, not emoji badges.** Emoji badges are decorative chrome. Plain text with semantic color (red/amber/default) is higher data-ink ratio and more accessible.
12. **Reboot markers as cross-chart annotations.** Reboots affect all metrics simultaneously. Showing them as vertical hairlines across all charts prevents the operator from having to correlate events across separate views.
13. **Separate packets_sent/packets_recv.** The ratio reveals asymmetric link problems invisible in a combined count.
14. **recv_errors as a first-class metric.** CRC failures are the strongest channel quality signal — more diagnostic than noise floor alone for in-band interference.
15. **Exclude cold-start samples from baseline.** First-post-reboot noise floor readings may reflect calibration artifacts, not the RF environment. Including them would bias the baseline.
## Open Questions
1. **Multiple observers on same channel:** If two observers share a channel, their noise floors should correlate. Could be useful for validation but doesn't change the data model.
2. **EMA vs median for baseline:** Exponential moving average is cheaper (no sort) and smoother than median. Consider for M3 implementation — but median is more robust against outliers. Decision deferred to M3.
3. **`triggerNoiseFloorCalibrate()` frequency:** Must be verified in M0. If it fires on every stats cycle, noise floor readings may be artificially smoothed. If only on boot, cold-start caveat applies. This affects how much weight to give noise floor vs. recv_errors for interference detection.
4. **Battery voltage thresholds:** 3.3V is a reasonable default for LiPo cells, but varies by chemistry and regulator. May need per-observer configuration.
+212
View File
@@ -0,0 +1,212 @@
# Startup Performance: Serve HTTP Within 2 Minutes on Any Database Size
## Problem
CoreScope takes 3045 minutes to start on large databases (325K transmissions, 7.3M observations, 1.4GB SQLite). The HTTP server is completely unavailable during this time. Operators cannot restart without 30+ minutes of downtime.
### Where time goes (7.3M observation benchmark)
| Phase | Time | Blocking? |
|---|---|---|
| `Load()` — read SQLite → memory | ~90s | Yes |
| Build subpath index | ~20s | Yes |
| Build distance index | ~15s | Yes |
| Build path-hop index | <1s | Yes |
| Load neighbor edges from SQLite | <1s | Yes |
| **Backfill `resolved_path` for NULL observations** | **2030+ min** | **Yes — the killer** |
| Re-pick best observations | ~10s | Yes |
The backfill calls `resolvePathForObs` for every observation with `resolved_path IS NULL`, then writes results back to SQLite and updates in-memory state. On first run (or after schema migration), this means resolving all 7.3M observations.
### Root cause
`backfillResolvedPaths()` in `neighbor_persist.go` runs synchronously in `main()` before `httpServer.ListenAndServe()`. It:
1. Collects all observations with `ResolvedPath == nil` under a read lock
2. Resolves paths (CPU-bound, ~millions of calls to `resolvePathForObs`)
3. Writes results to SQLite in a single transaction
4. Updates in-memory state under a write lock
Steps 24 block the main goroutine for 2030 minutes.
## Solution: Async Chunked Backfill
### Design
Move `backfillResolvedPaths` out of the startup critical path. Start the HTTP server immediately after loading data and building indexes. Run backfill in a background goroutine with chunked processing that yields between batches.
### Startup sequence (new)
```
1. OpenDB, verify tables (~1s)
2. store.Load() (~90s)
3. ensureNeighborEdgesTable (<1s)
4. ensureResolvedPathColumn (<1s)
5. Load/build neighbor graph (<1s)
6. Build subpath/distance/path-hop indexes (~35s)
7. pickBestObservation (with whatever (~10s)
resolved_path data exists)
8. *** START HTTP SERVER *** — serving at ~2min mark
9. Background: backfillResolvedPaths (20-30 min, non-blocking)
→ chunked, yields between batches
→ updates in-memory + SQLite incrementally
→ re-picks best obs for affected txs
```
Total time to first HTTP response: **~2 minutes** regardless of database size.
### Implementation details
#### 1. Background backfill goroutine
```go
// In main(), after starting HTTP server:
go func() {
backfillResolvedPathsAsync(store, dbPath, 5000, 100*time.Millisecond)
}()
```
The async backfill processes observations in chunks of N (e.g., 5,000):
```go
func backfillResolvedPathsAsync(store *PacketStore, dbPath string, chunkSize int, yieldDuration time.Duration) {
for {
n := backfillResolvedPathsChunk(store, dbPath, chunkSize)
if n == 0 {
break // done
}
log.Printf("[store] backfilled resolved_path for %d observations (async)", n)
time.Sleep(yieldDuration) // yield to HTTP handlers
}
log.Printf("[store] async resolved_path backfill complete")
}
```
Each chunk:
1. Takes a read lock, collects up to `chunkSize` pending observations, releases lock
2. Resolves paths (no lock held — `resolvePathForObs` only reads immutable data)
3. Opens a separate RW SQLite connection, writes results in a transaction
4. Takes a write lock, updates in-memory `obs.ResolvedPath` and re-picks best obs for affected transmissions, releases lock
5. Sleeps briefly to yield CPU/lock time to HTTP handlers
#### 2. Readiness flag and API degraded-mode header
Add a boolean to `PacketStore`:
```go
type PacketStore struct {
// ...
backfillComplete atomic.Bool
}
```
API responses include a header during backfill:
```
X-CoreScope-Status: backfilling
X-CoreScope-Backfill-Remaining: 4523000
```
After backfill completes:
```
X-CoreScope-Status: ready
```
The frontend can read this header and show a subtle banner: *"Resolving hop paths… some paths may show abbreviated pubkeys."*
#### 3. Index rebuilds
The subpath, distance, and path-hop indexes are built during startup from whatever data exists. During backfill, newly resolved paths need to update these indexes incrementally.
Options (in order of preference):
**Option A: Defer index updates to end of backfill.** Indexes work fine with unresolved paths — they just produce slightly less precise results. After backfill completes, rebuild indexes once. Simple, correct, low risk.
**Option B: Incremental index updates per chunk.** After each chunk, update affected index entries. More complex, better real-time accuracy. Only worth it if index accuracy during backfill matters for production use.
**Recommendation: Option A.** The indexes are usable with unresolved paths. A single rebuild at the end (~35s) is cheap compared to the backfill duration. The API works throughout — results just improve after backfill finishes.
#### 4. SQLite contention
The backfill opens a separate RW connection for writes. The main server uses a read-only connection for polling. SQLite WAL mode (already in use) allows concurrent readers and one writer. Contention risk is minimal:
- Write transactions are small (5,000 UPDATEs per chunk, batched in a single tx)
- Read queries from HTTP handlers are unaffected by WAL writes
- The 100ms yield between chunks prevents sustained write pressure
#### 5. Lock contention
The write lock is held only during the in-memory update phase of each chunk (~5,000 pointer assignments + re-picks). This takes microseconds. HTTP handlers acquire read locks for API responses — they will not be blocked for any perceptible duration.
#### 6. Frontend handling
The `hop-resolver.js` module already handles unresolved (prefix) hops gracefully — it shows abbreviated pubkeys. No frontend changes are required for correctness.
Optional enhancement: read the `X-CoreScope-Status` header and show a transient info banner during backfill. This is cosmetic and can be done in a follow-up.
### What about first-run specifically?
On first run with a pre-existing database (e.g., migrating from a version without `resolved_path`), ALL 7.3M observations need backfill. The async approach handles this identically — it just takes longer in the background while HTTP is already serving.
On subsequent restarts, `resolved_path` is already persisted in SQLite and loaded by `store.Load()`. The backfill loop finds zero pending observations and exits immediately.
### What about new observations during backfill?
The poller ingests new packets continuously. New observations written by the ingestor already have `resolved_path` set at ingest time (this is already implemented). The backfill only processes observations with `ResolvedPath == nil`, so there's no conflict with new data.
## Alternatives considered
### Lazy resolution (resolve on API access)
Resolve `resolved_path` only when an observation is accessed via API, cache the result.
**Rejected because:**
- Adds latency to every API call that touches unresolved observations
- Cache invalidation complexity (when does a cached resolution become stale?)
- Doesn't help with index accuracy — indexes still need full data
- The backfill is a one-time cost; lazy resolution makes it a recurring cost
### Progressive loading (recent data first)
Load only the last 24h into memory, start serving, load historical data in background.
**Rejected because:**
- Significantly more complex — all store operations need "is this data loaded yet?" checks
- Memory implications: need to track which time ranges are loaded
- Historical queries return wrong results during loading (not just degraded — wrong)
- The actual bottleneck is backfill, not `Load()`. Even loading all 7.3M observations takes only ~90s.
### Chunked blocking backfill (yield to HTTP between chunks, but keep in main startup)
Process N observations per tick with `runtime.Gosched()` between chunks, but still in `main()` before `ListenAndServe`.
**Rejected because:**
- HTTP still isn't available until all chunks complete
- Adds complexity without solving the core problem
## Carmack Review (Performance)
**The approach is sound.** Moving a 2030 minute blocking operation to a background goroutine is the right call. Some notes:
1. **Chunk size tuning.** 5,000 is a reasonable starting point. Monitor: if write lock contention shows up in pprof (unlikely with microsecond hold times), reduce chunk size. If backfill is too slow, increase it or reduce yield time.
2. **Memory is not a concern.** The observations are already fully loaded in memory by `Load()`. The backfill only mutates the `ResolvedPath` field on existing objects — no additional memory allocation beyond temporary slices for the chunk.
3. **No hidden costs in `resolvePathForObs`.** It reads `nodePM` (a `PrefixMatcher`, immutable after startup) and `graph` (neighbor graph, immutable after startup). No locks needed during resolution. This is embarrassingly parallelizable if needed, but single-goroutine processing with chunking is sufficient.
4. **The index rebuild at the end is O(n) and takes ~35s.** This is a one-time cost after the first backfill. Not worth optimizing further unless the profile shows otherwise.
5. **Risk: `pickBestObservation` during backfill.** API responses may flip their "best" observation as resolved paths become available. This is cosmetically noisy but functionally correct. Document this as expected behavior.
6. **Future optimization if needed:** The backfill loop could be parallelized across multiple goroutines (partition observations by transmission hash). The resolution step is CPU-bound and read-only. This would reduce backfill wall time from 30 min to ~5 min on 8 cores. Not needed for MVP — the goal is HTTP availability, not backfill speed.
## Implementation plan
1. **Refactor `backfillResolvedPaths` into chunked async version** — new function `backfillResolvedPathsAsync` that processes in chunks and yields
2. **Move backfill call in `main.go` to after `ListenAndServe`** — wrap in goroutine
3. **Add `backfillComplete` atomic flag to `PacketStore`** — set after backfill finishes
4. **Add `X-CoreScope-Status` response header** — middleware reads the flag
5. **Rebuild indexes after backfill completes** — single call to rebuild subpath/distance/path-hop
6. **Tests:** unit test for chunked backfill (mock store with N unresolved obs, verify chunks process correctly)
7. **Frontend (follow-up):** optional banner during backfill state
Estimated effort: 12 hours for steps 15, plus tests.
+144
View File
@@ -0,0 +1,144 @@
# Table Sorting Consistency Spec (#620)
## Problem
CoreScope has 20+ data tables. Only 2 are sortable (nodes list, channel activity). Those 2 use incompatible implementations — different property names (`column`/`direction` vs `col`/`dir`), different data attributes (`data-sort` vs `data-sort-col`), different function signatures. The remaining 18+ tables, including the packets table (30K+ rows), have zero sorting.
This violates AGENTS.md DRY rules and frustrates users who can see data but can't reorder it.
## Solution
One shared `TableSort` module. Every data table uses it. Same UX everywhere.
## Shared Utility Design
### Module: `public/table-sort.js`
IIFE pattern (like `channel-colors.js`). No dependencies. No build step.
```js
window.TableSort = (function() {
return { init, sort, destroy };
})();
```
### API
```js
TableSort.init(tableEl, {
defaultColumn: 'last_seen', // initial sort column
defaultDirection: 'desc', // 'asc' or 'desc'
storageKey: 'nodes-sort', // localStorage key (optional)
comparators: { // custom comparators for non-string columns
time: (a, b) => ...,
snr: (a, b) => ...,
},
onSort: (column, direction) => {} // callback after sort completes
});
```
### How It Works
1. Scans `<th>` elements for `data-sort="columnName"` attribute
2. Attaches click handlers — click toggles asc/desc
3. On sort: reads `<td data-value="...">` (raw sortable value) from each row
4. Sorts rows in-place via DOM reorder (no innerHTML rebuild — important for 30K rows)
5. Updates visual indicator and `aria-sort` on active `<th>`
### Visual Indicator
Active column header gets `▲` (ascending) or `▼` (descending) appended as a `<span class="sort-arrow">`. Inactive columns show no arrow. CSS class `.sort-active` on the active `<th>`.
### Built-in Comparators
| Type | Detected From | Behavior |
|------|--------------|----------|
| `numeric` | `data-type="number"` on `<th>` | `Number(a) - Number(b)`, NaN sorts last |
| `text` | default | `localeCompare` |
| `date` | `data-type="date"` | Parse as timestamp, numeric compare |
| `dbm` | `data-type="dbm"` | Strip " dBm" suffix, numeric compare |
Custom comparators in `options.comparators` override built-in types.
### Accessibility
- `aria-sort="ascending"`, `"descending"`, or `"none"` on every sortable `<th>`
- `role="columnheader"` (already implicit for `<th>`)
- `cursor: pointer` and `:hover` style on sortable headers
- Keyboard: sortable headers are focusable, Enter/Space triggers sort
### Performance (Critical for Packets Table)
- Sort via DOM node reorder (`appendChild` loop), not `innerHTML`. Browser batches reflows.
- `data-value` attributes hold raw values — no parsing during sort.
- For 30K rows: expected sort time ~100-200ms (single `Array.sort` + DOM reorder). If >500ms, add a virtual scroll layer in a follow-up — but don't pre-optimize.
- No re-render of row content. Sort only changes order.
## Milestones
### M1: Shared utility + packets table
- Create `public/table-sort.js`
- Unit tests: `test-table-sort.js` (Node.js, jsdom or vm.createContext)
- Integrate with packets table (highest impact — 30K rows, currently unsortable)
- Default sort: time descending
- Columns: all current packets columns (Region, Time, Hash, Size, HB, Type, Observer, Path, Rpt, Details)
- Browser validation: sort 30K rows, verify <500ms
### M2: Nodes list + node detail tables
- Migrate nodes list from custom sort to `TableSort.init()`
- Add sorting to neighbor table (side pane + detail page)
- Add sorting to observer stats table (detail page)
- Remove old `sortState`/`sortArrow` code from `nodes.js`
### M3: Analytics tables
- Hash collisions tables (node table, sizes table, collision prefixes)
- RF statistics table
- Route frequency, co-appearance, topology tables
- Node health tables (top by packets/SNR/observers, recently active)
- Distance tables (by link type, top 20 longest)
- Per-node analytics: peer contacts
### M4: Channels list + observers list + comparison table
- Channel activity table: migrate from custom sort to `TableSort.init()`
- Remove old `_channelSortState` code from `analytics.js`
- Observers list table
- Comparison table (`compare.js`)
### M5: Cleanup
- Remove all old sorting code (both implementations)
- Verify no dead CSS/JS from old sort code
- Final consistency audit: every data table uses `TableSort.init()`
### Out of Scope
- `packets.js` hex breakdown (structural decode, fixed order)
- `audio-lab.js` debug tables (not user-facing)
- Virtual scroll / pagination (separate issue if perf requires it)
## Testing
### Unit Tests (`test-table-sort.js`)
- Numeric sort ascending/descending
- Text sort with localeCompare
- Date sort
- dBm sort (strip suffix)
- Custom comparator override
- NaN/null/undefined sort to end
- Toggle direction on repeated click
- `aria-sort` attribute updates
- localStorage persistence (read + write)
- `data-value` attribute used over text content
### Integration (per milestone)
- Playwright test: click column header, verify row order changes
- Playwright test: click again, verify direction toggles
- Playwright test: visual indicator present on active column
### Performance
- Unit test: sort 30K mock rows in <500ms (assert timing)
- Required per AGENTS.md: perf claims need proof
## Migration Path
Existing sort code in `nodes.js` and `analytics.js` will be replaced, not wrapped. Both current implementations are <100 lines each — replacing is simpler than adapting. The shared utility subsumes all their functionality.
Old localStorage keys (`nodes-sort-*`, channel sort state) should be migrated or cleared on first use of the new utility.
+266
View File
@@ -0,0 +1,266 @@
# Timestamp-Based Packet Filters
**Issue:** #289
**Status:** Draft
**Depends on:** #286 (timestamp display config)
## Summary
Extend the existing filter engine (`packet-filter.js`) with a `time` field type supporting absolute ISO timestamps, relative durations, and range expressions. The filter compiles date expressions to epoch milliseconds at parse time so per-packet evaluation is a single numeric comparison — no date parsing in the hot path.
## Syntax
### Absolute (ISO 8601)
```
time > "2024-01-01T00:00:00Z"
time <= "2024-06-15"
time == "2024-03-01"
```
Quoted strings after `time` are parsed as dates. Partial dates (`"2024-01-01"`) are treated as midnight UTC. All absolute values are interpreted as UTC regardless of the user's display preference.
### Relative
```
time > 2h ago
time > 30m ago
time > 7d ago
```
The lexer recognizes `<number><unit> ago` as a relative time literal. Supported units: `s` (seconds), `m` (minutes), `h` (hours), `d` (days). At compile time, the relative offset is resolved to an absolute epoch ms value (`Date.now() - offset`). This means a compiled filter's relative thresholds are frozen at compile time — recompile to refresh.
### Shorthand
```
time.ago < 30m
time.ago < 2h
```
`time.ago` resolves to `Date.now() - packet.timestamp`. The comparison value is a duration literal (`30m`, `2h`, `7d`). This is syntactic sugar and semantically equivalent to the relative form but reads more naturally for "show me recent packets."
### Range
```
time between "2024-01-01" "2024-01-02"
time between 1h ago 30m ago
```
`between` is a ternary operator: `field between <low> <high>`. Compiles to `low <= field && field <= high`. Both bounds are inclusive.
### Combinable with existing filters
```
type == Advert && time > 1h ago
snr > 5 && time between "2024-01-01" "2024-01-02"
(type == GRP_TXT || type == TXT_MSG) && time.ago < 30m
```
## Grammar Extension
### New token types
| Token | Pattern | Example |
|-------|---------|---------|
| `DURATION` | `/^\d+[smhd]$/` | `30m`, `2h`, `7d` |
| `AGO` | keyword `ago` | `ago` |
| `BETWEEN` | keyword `between` | `between` |
### Lexer changes
1. After reading an identifier that matches `\d+[smhd]`, emit `DURATION` token instead of `FIELD`.
2. Recognize `ago` and `between` as keywords (like `and`/`or`).
### Parser changes
In `parseComparison()`:
1. **Relative time:** If field is `time` and value tokens are `DURATION AGO`, compute `Date.now() - durationToMs(duration)` and store as a numeric epoch ms value in the AST node.
2. **Absolute time:** If field is `time` and value is a `STRING`, attempt `new Date(value).getTime()`. If `NaN`, return parse error. Store epoch ms.
3. **`time.ago` shorthand:** If field is `time.ago`, the value is a `DURATION`. Store the duration in ms. At evaluation, compute `now - packet_ts` and compare against the duration.
4. **`between`:** If operator token is `BETWEEN`, consume two values (same type resolution as above). Emit `{ type: 'between', field, low, high }`.
### AST node shapes
```js
// Absolute/relative (pre-resolved to epoch ms)
{ type: 'comparison', field: 'time', op: '>', value: 1704067200000 }
// time.ago (duration in ms)
{ type: 'comparison', field: 'time.ago', op: '<', value: 1800000 }
// between (both bounds as epoch ms)
{ type: 'between', field: 'time', low: 1704067200000, high: 1704153600000 }
```
## Field Resolution
Add to `resolveField()`:
```js
if (field === 'time') return packet.timestamp; // epoch ms
if (field === 'time.ago') return Date.now() - packet.timestamp;
```
`packet.timestamp` is the packet's capture time in epoch milliseconds. This field already exists in the data model (populated from the DB `created_at` column).
## Time Semantics
- **Filter expressions:** Always UTC. `"2024-01-01"` means `2024-01-01T00:00:00Z`.
- **Display:** Follows the user's timestamp config from #286 (UTC/local/relative).
- **Relative times:** Computed against `Date.now()` at compile time. The compiled filter is a snapshot — if the filter stays active for hours, relative thresholds drift. This is acceptable; filters are typically short-lived or recompiled on interaction.
**No timezone specifiers in the filter syntax.** UTC only. This avoids ambiguity and parsing complexity. Users who think in local time can use the relative syntax (`time > 2h ago`) which is timezone-agnostic.
## Performance
### Compile-time work (once)
- Parse date strings → epoch ms via `new Date().getTime()` (~1μs per date)
- Parse duration strings → ms via multiplication (~0ns, trivial arithmetic)
- Relative `ago``Date.now() - offset` (~0ns)
### Per-packet evaluation (hot path)
- `time` comparison: one numeric read + one numeric compare. Same cost as `snr > 5`.
- `time.ago`: one subtraction + one compare. Two arithmetic ops. **Important:** cache `Date.now()` once per filter pass (e.g., in a closure variable set before iterating packets), not per-packet. 30K `Date.now()` calls are ~1ms but it's a pointless syscall tax.
- `between`: two numeric compares.
**No `Date` objects created per packet. No string parsing per packet. No regex per packet.**
At 30K packets, the time filter adds ~0.1ms total to filter evaluation — dominated by the existing field resolution and AST walk overhead. No measurable regression.
### Implementation note: `between` as sugar
`between` should compile to `{ type: 'and', left: { type: 'comparison', field, op: '>=', value: low }, right: { type: 'comparison', field, op: '<=', value: high } }` — reusing existing comparison evaluation. No new AST node type, no new evaluator branch. The parser desugars it; the evaluator never sees `between`.
### Implementation note: `time.ago` and `Date.now()` caching
The `compile()` function should return a filter that accepts an optional `now` parameter:
```js
var compiled = compile('time.ago < 30m');
var now = Date.now();
packets.filter(function(p) { return compiled.filter(p, now); });
```
If `now` is not passed, `Date.now()` is called once on the first invocation and reused for the entire filter pass. This avoids 30K syscalls and ensures consistent evaluation within a single pass.
## Carmack Review Notes
Reviewed with a performance-first lens (30K+ packets, real-time updates):
1. **✅ No allocations in hot path.** All date parsing happens at compile time. Per-packet evaluation is pure numeric comparison — same cost as existing `snr > 5` filters.
2. **⚠️ `Date.now()` per-packet for `time.ago`.** Fixed above — cache once per filter pass via optional `now` parameter or closure. Without this, 30K packets × `Date.now()` = ~1ms wasted on a monotonic clock syscall that returns the same value.
3. **`between` as sugar, not a new node type.** Desugar in the parser to reuse existing `and` + `comparison` nodes. Zero new code paths in the evaluator = zero new bugs in the evaluator.
4. **✅ Parser complexity is bounded.** Three new token types, one new keyword. The parser remains LL(1) — no backtracking, no ambiguity. `DURATION AGO` is a clear two-token lookahead only when field is `time`.
5. **✅ Memory impact negligible.** Compiled time filters add one or two floats to the AST. At 16 bytes per node, even complex expressions with multiple time clauses are <100 bytes.
6. **⚠️ Compiled filter staleness for relative times.** Spec acknowledges this. Acceptable for a web UI where filters are recompiled on user interaction. If filters persist across long WebSocket sessions, consider recompiling on a timer (every 60s). This is a future concern, not a blocker.
7. **✅ No regex in hot path.** Duration parsing uses a simple char check on the last character + `parseInt`. Cheaper than any regex.
A compiled time filter adds one or two 64-bit float values to the AST. Negligible — roughly 16 bytes per time comparison node.
## URL Integration
Time filters appear in the URL hash query string like any other filter:
```
#/packets?filter=time%20%3E%201h%20ago
#/packets?filter=type%20%3D%3D%20Advert%20%26%26%20time%20%3E%20%222024-01-01%22
```
The filter text is URL-encoded and round-trips through `encodeURIComponent`/`decodeURIComponent`. No special handling needed — the existing filter-in-URL mechanism (#286 or current) works unchanged.
For convenience, a future milestone could add dedicated `timeFrom`/`timeTo` query params that inject into the filter, but this is not required for the initial implementation.
## Wireshark Compatibility
| Wireshark syntax | CoreScope equivalent | Notes |
|------------------|---------------------|-------|
| `frame.time >= "2024-01-01"` | `time >= "2024-01-01"` | We use `time` instead of `frame.time` for brevity. Could alias `frame.time``time` later. |
| `frame.time_relative < 60` | `time.ago < 60s` | Wireshark uses seconds float; we use duration literals |
| `frame.time_delta` | Not supported | Inter-packet delta is a different feature |
We intentionally diverge from Wireshark where their syntax is verbose or requires pcap-specific concepts. CoreScope's filter language prioritizes brevity and readability for a web UI. A `frame.time` alias for `time` can be added trivially in the field resolver if users request it.
## Milestones
### M1: Core time filtering (parser + evaluator)
- Add `DURATION`, `AGO`, `BETWEEN` tokens to lexer
- Extend parser for `time` field special handling
- Add `time` and `time.ago` to `resolveField()`
- Implement `between` AST node evaluation
- Unit tests: absolute, relative, ago, between, combined with existing filters, edge cases (bad dates, invalid units)
- **Test:** filter 30K packets by time in <50ms (assert in test)
### M2: UI integration
- Filter bar autocomplete hints for time syntax
- Help tooltip / cheat sheet update with time examples
- Verify URL round-trip with time filters
- Playwright E2E test: enter time filter, verify packet list updates
### M3: Polish
- `frame.time` alias
- Error messages for common mistakes ("did you mean `time > 1h ago`?")
- Consider dedicated time range picker UI widget (out of scope for this spec)
## Testing
### Unit tests (add to `test-packet-filter.js`)
```js
// Absolute time
c = compile('time > "2024-01-01"');
assert(c.filter({ timestamp: new Date('2024-06-01').getTime() }), 'after 2024-01-01');
assert(!c.filter({ timestamp: new Date('2023-06-01').getTime() }), 'before 2024-01-01');
// Relative time
c = compile('time > 1h ago');
assert(c.filter({ timestamp: Date.now() - 30 * 60000 }), '30m ago passes 1h filter');
assert(!c.filter({ timestamp: Date.now() - 2 * 3600000 }), '2h ago fails 1h filter');
// time.ago shorthand
c = compile('time.ago < 30m');
assert(c.filter({ timestamp: Date.now() - 10 * 60000 }), '10m ago < 30m');
assert(!c.filter({ timestamp: Date.now() - 60 * 60000 }), '60m ago not < 30m');
// between
c = compile('time between "2024-01-01" "2024-01-02"');
assert(c.filter({ timestamp: new Date('2024-01-01T12:00:00Z').getTime() }), 'in range');
assert(!c.filter({ timestamp: new Date('2024-01-03').getTime() }), 'out of range');
// Combined
c = compile('type == Advert && time > 1h ago');
assert(c.filter({ payload_type: 4, timestamp: Date.now() - 1000 }), 'combined pass');
assert(!c.filter({ payload_type: 4, timestamp: Date.now() - 7200000 }), 'combined fail time');
assert(!c.filter({ payload_type: 1, timestamp: Date.now() - 1000 }), 'combined fail type');
// Error cases
c = compile('time > "not-a-date"');
assert(c.error, 'invalid date string');
c = compile('time > 5x ago');
assert(c.error, 'invalid duration unit');
// Performance
var start = Date.now();
c = compile('time > 1h ago && type == Advert');
var packets = [];
for (var i = 0; i < 30000; i++) {
packets.push({ payload_type: i % 5, timestamp: Date.now() - i * 1000 });
}
packets.forEach(function(p) { c.filter(p); });
assert(Date.now() - start < 50, 'filter 30K packets in <50ms');
```
### Playwright tests
- Enter `time > 1h ago` in filter bar → verify packet count decreases
- Enter invalid time filter → verify error message appears
- Reload page with time filter in URL → verify filter is applied
@@ -0,0 +1,674 @@
# Deep Linking P1 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make P1 UI states in nodes, packets, and channels URL-addressable so they survive refresh and can be shared.
**Architecture:** Each page reads URL params from `location.hash.split('?')[1]` on init (router strips query string before passing `routeParam`, so pages must read `location.hash` directly). State changes call `history.replaceState` to keep the URL in sync. localStorage remains the fallback default; URL params override when present.
**Tech Stack:** Vanilla JS (ES5/6), browser History API, URLSearchParams
---
## Files Changed
| File | Changes |
|---|---|
| `public/region-filter.js` | Add `setSelected(codesArray)`, track `_container` for re-render |
| `public/nodes.js` | Read `?tab=`/`?search=` on init; `updateNodesUrl()` on tab/search change; expose `buildNodesQuery` on `window` |
| `public/packets.js` | Read `?timeWindow=`/`?region=` on init; `updatePacketsUrl()` on timeWindow/region change; expose `buildPacketsUrl` on `window` |
| `public/channels.js` | Read `?node=` on init; update URL in `showNodeDetail`/`closeNodeDetail` |
| `test-frontend-helpers.js` | Add unit tests for `buildNodesQuery` and `buildPacketsUrl` |
| `test-e2e-playwright.js` | Add Playwright tests: tab URL persistence, timeWindow URL persistence |
---
## Task 1: Add `setSelected` to RegionFilter
**Files:**
- Modify: `public/region-filter.js`
- [ ] **Step 1: Write the failing unit test**
Add to `test-frontend-helpers.js` before the `// ===== SUMMARY =====` line:
```javascript
// ===== REGION-FILTER.JS: setSelected =====
console.log('\n=== region-filter.js: setSelected ===');
{
const ctx = makeSandbox();
ctx.fetch = () => Promise.resolve({ json: () => Promise.resolve({ 'US-SFO': 'San Jose', 'US-LAX': 'Los Angeles' }) });
loadInCtx(ctx, 'public/region-filter.js');
const RF = ctx.RegionFilter;
RF.init(document.createElement('div'));
test('setSelected sets region codes', async () => {
await RF.init(document.createElement('div'));
RF.setSelected(['US-SFO', 'US-LAX']);
assert.strictEqual(RF.getRegionParam(), 'US-SFO,US-LAX');
});
test('setSelected with null clears selection', async () => {
await RF.init(document.createElement('div'));
RF.setSelected(['US-SFO']);
RF.setSelected(null);
assert.strictEqual(RF.getRegionParam(), '');
});
test('setSelected with empty array clears selection', async () => {
await RF.init(document.createElement('div'));
RF.setSelected(['US-SFO']);
RF.setSelected([]);
assert.strictEqual(RF.getRegionParam(), '');
});
}
```
- [ ] **Step 2: Run test to verify it fails**
```bash
node test-frontend-helpers.js 2>&1 | grep -A2 "setSelected"
```
Expected: `❌ setSelected sets region codes: RF.setSelected is not a function`
- [ ] **Step 3: Add `_container` tracking and `setSelected` to region-filter.js**
In `region-filter.js`, add `var _container = null;` after the existing module-level vars (after line 9 `var _listeners = [];`):
```javascript
var _listeners = [];
var _container = null; // ← add this line
var _loaded = false;
```
In `initFilter`, save the container:
```javascript
async function initFilter(container, opts) {
_container = container; // ← add this line
if (opts && opts.dropdown) container._forceDropdown = true;
await fetchRegions();
render(container);
}
```
Add `setSelected` function before `// Expose globally`:
```javascript
/** Override selected regions (e.g. from URL param). Persists to localStorage and re-renders. */
function setSelected(codesArray) {
_selected = (codesArray && codesArray.length > 0) ? new Set(codesArray) : null;
saveToStorage();
if (_container) render(_container);
}
```
Add `setSelected` to the public API object:
```javascript
window.RegionFilter = {
init: initFilter,
render: render,
getSelected: getSelected,
getRegionParam: getRegionParam,
regionQueryString: regionQueryString,
onChange: onChange,
offChange: offChange,
fetchRegions: fetchRegions,
setSelected: setSelected, // ← add this line
};
```
- [ ] **Step 4: Run test to verify it passes**
```bash
node test-frontend-helpers.js 2>&1 | grep -E "(setSelected|FAIL|passed|failed)"
```
Expected: 3 passing `setSelected` tests, overall pass.
- [ ] **Step 5: Commit**
```bash
git add public/region-filter.js test-frontend-helpers.js
git commit -m "feat: add RegionFilter.setSelected for URL param initialization (#536)"
```
---
## Task 2: nodes.js — tab and search deep linking
**Files:**
- Modify: `public/nodes.js`
- Test: `test-frontend-helpers.js`
- Test: `test-e2e-playwright.js`
- [ ] **Step 1: Write the unit test (add to test-frontend-helpers.js)**
Add before the `// ===== SUMMARY =====` line:
```javascript
// ===== NODES.JS: buildNodesQuery =====
console.log('\n=== nodes.js: buildNodesQuery ===');
{
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
// Provide required globals for nodes.js IIFE to execute
ctx.registerPage = () => {};
ctx.RegionFilter = { init: () => Promise.resolve(), onChange: () => () => {}, offChange: () => {}, getSelected: () => null, getRegionParam: () => '' };
ctx.onWS = () => {};
ctx.offWS = () => {};
ctx.debouncedOnWS = () => () => {};
ctx.invalidateApiCache = () => {};
ctx.favStar = () => '';
ctx.bindFavStars = () => {};
ctx.getFavorites = () => [];
ctx.isFavorite = () => false;
ctx.connectWS = () => {};
ctx.HopResolver = { init: () => {}, resolve: () => ({}), ready: () => false };
ctx.initTabBar = () => {};
ctx.debounce = (fn) => fn;
ctx.copyToClipboard = () => {};
ctx.api = () => Promise.resolve({});
ctx.escapeHtml = (s) => s;
ctx.timeAgo = () => '';
ctx.formatTimestampWithTooltip = () => '';
ctx.getTimestampMode = () => 'ago';
ctx.CLIENT_TTL = {};
ctx.qrcode = null;
try {
const src = fs.readFileSync('public/nodes.js', 'utf8');
vm.runInContext(src, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
} catch (e) {
console.log(' ⚠️ nodes.js sandbox load failed:', e.message.slice(0, 120));
}
const buildNodesQuery = ctx.buildNodesQuery;
if (buildNodesQuery) {
test('buildNodesQuery: all tab + no search = empty', () => {
assert.strictEqual(buildNodesQuery('all', ''), '');
});
test('buildNodesQuery: repeater tab only', () => {
assert.strictEqual(buildNodesQuery('repeater', ''), '?tab=repeater');
});
test('buildNodesQuery: search only (all tab)', () => {
assert.strictEqual(buildNodesQuery('all', 'foo'), '?search=foo');
});
test('buildNodesQuery: tab + search combined', () => {
assert.strictEqual(buildNodesQuery('companion', 'bar'), '?tab=companion&search=bar');
});
test('buildNodesQuery: null search treated as empty', () => {
assert.strictEqual(buildNodesQuery('all', null), '');
});
test('buildNodesQuery: sensor tab', () => {
assert.strictEqual(buildNodesQuery('sensor', ''), '?tab=sensor');
});
} else {
console.log(' ⚠️ buildNodesQuery not exposed — skipping');
}
}
```
- [ ] **Step 2: Run test to verify it fails (or skips)**
```bash
node test-frontend-helpers.js 2>&1 | grep -A3 "buildNodesQuery"
```
Expected: `⚠️ buildNodesQuery not exposed — skipping`
- [ ] **Step 3: Add URL param reading and helpers to nodes.js**
**3a.** Add `buildNodesQuery` and `updateNodesUrl` functions inside the nodes.js IIFE, after the `TABS` definition (around line 86, before `function renderNodeTimestampHtml`):
```javascript
function buildNodesQuery(tab, searchStr) {
var parts = [];
if (tab && tab !== 'all') parts.push('tab=' + encodeURIComponent(tab));
if (searchStr) parts.push('search=' + encodeURIComponent(searchStr));
return parts.length ? '?' + parts.join('&') : '';
}
window.buildNodesQuery = buildNodesQuery;
function updateNodesUrl() {
history.replaceState(null, '', '#/nodes' + buildNodesQuery(activeTab, search));
}
```
**3b.** In the list-view branch of `init` (after the `return;` that ends the full-screen block at line 317), add URL param reading before `app.innerHTML`:
```javascript
// Read URL params for list view (router strips query string from routeParam)
const _listUrlParams = new URLSearchParams(location.hash.split('?')[1] || '');
const _urlTab = _listUrlParams.get('tab');
const _urlSearch = _listUrlParams.get('search');
if (_urlTab && TABS.some(function(t) { return t.key === _urlTab; })) activeTab = _urlTab;
if (_urlSearch) search = _urlSearch;
app.innerHTML = `<div class="nodes-page">
```
**3c.** After `app.innerHTML = ...` (after the closing backtick at line ~330), populate the search input:
```javascript
if (search) {
var _si = document.getElementById('nodeSearch');
if (_si) _si.value = search;
}
```
**3d.** In the search input event listener (around line 335), add `updateNodesUrl()`:
```javascript
document.getElementById('nodeSearch').addEventListener('input', debounce(e => {
search = e.target.value;
updateNodesUrl();
loadNodes();
}, 250));
```
**3e.** In the tab click handler inside `renderLeft` (around line 875), add `updateNodesUrl()`:
```javascript
btn.addEventListener('click', () => { activeTab = btn.dataset.tab; updateNodesUrl(); loadNodes(); });
```
- [ ] **Step 4: Run unit tests**
```bash
node test-frontend-helpers.js 2>&1 | grep -E "(buildNodesQuery|✅|❌)" | grep -v "helpers"
```
Expected: 6 passing `buildNodesQuery` tests.
- [ ] **Step 5: Write Playwright test (add to test-e2e-playwright.js)**
Add before the closing `await browser.close()` line:
```javascript
// --- Group: Deep linking (#536) ---
// Test: nodes tab deep link
await test('Nodes tab deep link restores active tab', async () => {
await page.goto(BASE + '#/nodes?tab=repeater', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.node-tab', { timeout: 8000 });
const activeTab = await page.$('.node-tab.active');
assert(activeTab, 'No active tab found');
const tabText = await activeTab.textContent();
assert(tabText.includes('Repeater'), `Expected Repeater tab active, got: ${tabText}`);
const url = page.url();
assert(url.includes('tab=repeater'), `URL should contain tab=repeater, got: ${url}`);
});
// Test: nodes tab click updates URL
await test('Nodes tab click updates URL', async () => {
await page.goto(BASE + '#/nodes', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.node-tab', { timeout: 8000 });
const roomTab = await page.$('.node-tab[data-tab="room"]');
if (roomTab) {
await roomTab.click();
await page.waitForTimeout(300);
const url = page.url();
assert(url.includes('tab=room'), `URL should contain tab=room after click, got: ${url}`);
}
});
```
- [ ] **Step 6: Run full test suite**
```bash
node test-frontend-helpers.js
```
Expected: all tests pass.
- [ ] **Step 7: Commit**
```bash
git add public/nodes.js test-frontend-helpers.js test-e2e-playwright.js
git commit -m "feat: deep link nodes tab and search query (#536)"
```
---
## Task 3: packets.js — timeWindow and region deep linking
**Files:**
- Modify: `public/packets.js`
- Test: `test-frontend-helpers.js`
- Test: `test-e2e-playwright.js`
> Depends on Task 1 (RegionFilter.setSelected).
- [ ] **Step 1: Write the unit test**
Add to `test-frontend-helpers.js` before `// ===== SUMMARY =====`:
```javascript
// ===== PACKETS.JS: buildPacketsUrl =====
console.log('\n=== packets.js: buildPacketsUrl ===');
{
// Test the pure helper function
// (loaded via packets.js after it exposes window.buildPacketsUrl)
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
ctx.registerPage = () => {};
ctx.RegionFilter = { init: () => Promise.resolve(), onChange: () => () => {}, offChange: () => {}, getSelected: () => null, getRegionParam: () => '', setSelected: () => {} };
ctx.onWS = () => {};
ctx.offWS = () => {};
ctx.debouncedOnWS = () => () => {};
ctx.invalidateApiCache = () => {};
ctx.api = () => Promise.resolve({});
ctx.observerMap = new Map();
ctx.getParsedPath = () => [];
ctx.getParsedDecoded = () => ({});
ctx.clearParsedCache = () => {};
ctx.escapeHtml = (s) => s;
ctx.timeAgo = () => '';
ctx.formatTimestampWithTooltip = () => '';
ctx.getTimestampMode = () => 'ago';
ctx.copyToClipboard = () => {};
ctx.CLIENT_TTL = {};
ctx.debounce = (fn) => fn;
ctx.initTabBar = () => {};
try {
const src = fs.readFileSync('public/packet-helpers.js', 'utf8');
vm.runInContext(src, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
const src2 = fs.readFileSync('public/packets.js', 'utf8');
vm.runInContext(src2, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
} catch (e) {
console.log(' ⚠️ packets.js sandbox load failed:', e.message.slice(0, 120));
}
const buildPacketsUrl = ctx.buildPacketsUrl;
if (buildPacketsUrl) {
test('buildPacketsUrl: default (15min, no region) = bare #/packets', () => {
assert.strictEqual(buildPacketsUrl(15, ''), '#/packets');
});
test('buildPacketsUrl: non-default timeWindow', () => {
assert.strictEqual(buildPacketsUrl(60, ''), '#/packets?timeWindow=60');
});
test('buildPacketsUrl: region only', () => {
assert.strictEqual(buildPacketsUrl(15, 'US-SFO'), '#/packets?region=US-SFO');
});
test('buildPacketsUrl: timeWindow + region', () => {
assert.strictEqual(buildPacketsUrl(30, 'US-SFO,US-LAX'), '#/packets?timeWindow=30&region=US-SFO%2CUS-LAX');
});
test('buildPacketsUrl: timeWindow=0 treated as default', () => {
assert.strictEqual(buildPacketsUrl(0, ''), '#/packets');
});
} else {
console.log(' ⚠️ buildPacketsUrl not exposed — skipping');
}
}
```
- [ ] **Step 2: Run to verify it skips**
```bash
node test-frontend-helpers.js 2>&1 | grep -A2 "buildPacketsUrl"
```
Expected: `⚠️ buildPacketsUrl not exposed — skipping`
- [ ] **Step 3: Add helpers and URL param reading to packets.js**
**3a.** Add `buildPacketsUrl` and `updatePacketsUrl` inside the packets.js IIFE, after the existing constants at the top (around line 36, after `let showHexHashes`):
```javascript
function buildPacketsUrl(timeWindowMin, regionParam) {
var parts = [];
if (timeWindowMin && timeWindowMin !== 15) parts.push('timeWindow=' + timeWindowMin);
if (regionParam) parts.push('region=' + encodeURIComponent(regionParam));
return '#/packets' + (parts.length ? '?' + parts.join('&') : '');
}
window.buildPacketsUrl = buildPacketsUrl;
function updatePacketsUrl() {
history.replaceState(null, '', buildPacketsUrl(savedTimeWindowMin, RegionFilter.getRegionParam()));
}
```
**3b.** In the `init` function (around line 263), add URL param reading after the existing `routeParam`/`directObsId` parsing and before `app.innerHTML`:
```javascript
// Read URL params for filter state (router strips query from routeParam; read from location.hash)
var _initUrlParams = new URLSearchParams(location.hash.split('?')[1] || '');
var _urlTimeWindow = Number(_initUrlParams.get('timeWindow'));
if (Number.isFinite(_urlTimeWindow) && _urlTimeWindow > 0) {
savedTimeWindowMin = _urlTimeWindow;
localStorage.setItem('meshcore-time-window', String(_urlTimeWindow));
}
var _urlRegion = _initUrlParams.get('region');
if (_urlRegion) {
RegionFilter.setSelected(_urlRegion.split(',').filter(Boolean));
}
app.innerHTML = `<div class="split-layout detail-collapsed">
```
**3c.** In the time window change handler (around line 865), add `updatePacketsUrl()`:
```javascript
fTimeWindow.addEventListener('change', () => {
savedTimeWindowMin = Number(fTimeWindow.value);
if (!Number.isFinite(savedTimeWindowMin) || savedTimeWindowMin <= 0) savedTimeWindowMin = 15;
localStorage.setItem('meshcore-time-window', fTimeWindow.value);
updatePacketsUrl();
loadPackets();
});
```
**3d.** In the RegionFilter.onChange callback (around line 719), add `updatePacketsUrl()`:
```javascript
RegionFilter.onChange(function() { updatePacketsUrl(); loadPackets(); });
```
- [ ] **Step 4: Run unit tests**
```bash
node test-frontend-helpers.js 2>&1 | grep -E "(buildPacketsUrl|✅|❌)" | grep -v "helpers"
```
Expected: 5 passing `buildPacketsUrl` tests.
- [ ] **Step 5: Write Playwright test (add to test-e2e-playwright.js, inside the deep-linking group)**
```javascript
// Test: packets timeWindow deep link
await test('Packets timeWindow deep link restores dropdown', async () => {
await page.goto(BASE + '#/packets?timeWindow=60', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fTimeWindow', { timeout: 8000 });
const val = await page.$eval('#fTimeWindow', el => el.value);
assert(val === '60', `Expected timeWindow dropdown = 60, got: ${val}`);
const url = page.url();
assert(url.includes('timeWindow=60'), `URL should still contain timeWindow=60, got: ${url}`);
});
// Test: timeWindow change updates URL
await test('Packets timeWindow change updates URL', async () => {
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fTimeWindow', { timeout: 8000 });
await page.selectOption('#fTimeWindow', '30');
await page.waitForTimeout(300);
const url = page.url();
assert(url.includes('timeWindow=30'), `URL should contain timeWindow=30 after change, got: ${url}`);
});
```
- [ ] **Step 6: Run full test suite**
```bash
node test-frontend-helpers.js
```
Expected: all tests pass.
- [ ] **Step 7: Commit**
```bash
git add public/packets.js test-frontend-helpers.js test-e2e-playwright.js
git commit -m "feat: deep link packets timeWindow and region filter (#536)"
```
---
## Task 4: channels.js — node panel deep linking
**Files:**
- Modify: `public/channels.js`
No unit tests needed for this task — the URL manipulation is side-effectful (DOM + History API). Playwright tests cover it.
- [ ] **Step 1: Write the Playwright test (add to test-e2e-playwright.js, inside the deep-linking group)**
```javascript
// Test: channels selected channel survives refresh (already implemented, verify it still works)
await test('Channels channel selection is URL-addressable', async () => {
await page.goto(BASE + '#/channels', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.ch-item', { timeout: 8000 }).catch(() => null);
const firstChannel = await page.$('.ch-item');
if (firstChannel) {
await firstChannel.click();
await page.waitForTimeout(500);
const url = page.url();
assert(url.includes('#/channels/') || url.includes('#/channels'), `URL should reflect channel selection, got: ${url}`);
}
});
```
- [ ] **Step 2: Update `showNodeDetail` to write `?node=` to the URL**
In `channels.js`, in `showNodeDetail` (around line 171), add the URL update right after `selectedNode = name;`:
```javascript
async function showNodeDetail(name) {
_nodePanelTrigger = document.activeElement;
if (_focusTrapCleanup) { _focusTrapCleanup(); _focusTrapCleanup = null; }
const node = await lookupNode(name);
selectedNode = name;
var _chBase = selectedHash ? '#/channels/' + encodeURIComponent(selectedHash) : '#/channels';
history.replaceState(null, '', _chBase + '?node=' + encodeURIComponent(name));
let panel = document.getElementById('chNodePanel');
```
- [ ] **Step 3: Update `closeNodeDetail` to strip `?node=` from the URL**
In `closeNodeDetail` (around line 232), add URL restore right after `selectedNode = null;`:
```javascript
function closeNodeDetail() {
if (_focusTrapCleanup) { _focusTrapCleanup(); _focusTrapCleanup = null; }
const panel = document.getElementById('chNodePanel');
if (panel) panel.classList.remove('open');
selectedNode = null;
var _chRestoreUrl = selectedHash ? '#/channels/' + encodeURIComponent(selectedHash) : '#/channels';
history.replaceState(null, '', _chRestoreUrl);
if (_nodePanelTrigger && typeof _nodePanelTrigger.focus === 'function') {
```
- [ ] **Step 4: Read `?node=` on init and auto-open panel**
In `channels.js` `init` (line 316), add URL param reading at the very top of the function (before `app.innerHTML`):
```javascript
function init(app, routeParam) {
var _initUrlParams = new URLSearchParams(location.hash.split('?')[1] || '');
var _pendingNode = _initUrlParams.get('node');
app.innerHTML = `<div class="ch-layout">
```
Then update the `loadChannels().then(...)` call (around line 350) to auto-open the node panel:
```javascript
loadChannels().then(async function () {
if (routeParam) await selectChannel(routeParam);
if (_pendingNode) showNodeDetail(_pendingNode);
});
```
- [ ] **Step 5: Run full test suite**
```bash
node test-frontend-helpers.js
```
Expected: all tests pass (no channels unit tests, but regression tests still pass).
- [ ] **Step 6: Commit**
```bash
git add public/channels.js
git commit -m "feat: deep link channels node panel via ?node= (#536)"
```
---
## Task 5: Run E2E Playwright tests
- [ ] **Step 1: Start the local server**
```bash
cd cmd/server && go run . &
```
Wait for it to be ready (check `http://localhost:3000`).
- [ ] **Step 2: Run Playwright tests**
```bash
node test-e2e-playwright.js
```
Expected: all tests pass including the new deep-linking group.
- [ ] **Step 3: If any deep-linking test fails, debug**
Common failures:
- Selector `.node-tab.active` not found: check that nodes.js correctly reads `?tab=` from URL before rendering
- `#fTimeWindow` value wrong: check that `savedTimeWindowMin` is overridden before the DOM is built
- URL doesn't update: check `history.replaceState` calls in the change handlers
- [ ] **Step 4: Final commit (if any fixes needed)**
```bash
git add public/nodes.js public/packets.js public/channels.js
git commit -m "fix: deep linking E2E adjustments (#536)"
```
---
## Self-Review
**Spec coverage check:**
- ✅ P1: Nodes role tab → Task 2
- ✅ P1: Packets time window → Task 3
- ✅ P1: Packets region filter → Task 3 (depends on Task 1)
- ✅ P1: Channels selected channel → Already implemented via `#/channels/{hash}` (verified in channels.js init line 351)
- ✅ P1: Channels node panel → Task 4
- ✅ P2+ items → explicitly out of scope per issue
**Architecture note:** The router in `app.js` strips the query string at line 422 (`const route = hash.split('?')[0]`) before computing `basePage` and `routeParam`. Therefore `#/nodes?tab=repeater` gives `routeParam=null` (not `?tab=repeater`). All pages must read URL params from `location.hash` directly, not from `routeParam`. This is the established pattern in `analytics.js` and `nodes.js` (section scroll).
**Placeholder scan:** No TBDs, no "implement later", all code blocks complete. ✅
**Type consistency:**
- `buildNodesQuery(tab, searchStr)` — used consistently in `updateNodesUrl()` and in tests ✅
- `buildPacketsUrl(timeWindowMin, regionParam)` — used consistently in `updatePacketsUrl()` and in tests ✅
- `RegionFilter.setSelected(codesArray)` — defined in Task 1, used in Task 3 ✅
+162
View File
@@ -0,0 +1,162 @@
# v3.4.2 Manual Validation Checklist
**Tester:** _______________
**Staging:** http://20.109.157.39
**Prod:** https://analyzer.00id.net (READ ONLY — do not deploy until staging passes)
**Browser:** Chrome + Firefox + Safari (mobile for responsive items)
**Time estimate:** ~45 minutes
---
## 🔴 HIGH RISK — Test First
### 1. Zero-hop hash size display (#649, #653)
- [ ] Go to Packets page, find a DIRECT advert (route_type=2, 0 hops)
- [ ] Open packet detail — hash size should say "Unknown (zero-hop)" or be hidden, NOT "1 byte"
- [ ] Check "Path Length" field shows `hash_count=0 (direct advert)`
- [ ] Find a FLOOD advert with 0 hops — it SHOULD show hash size (this is different from DIRECT)
### 2. TRACE packet real path (#651, #656)
- [ ] Send a trace from your companion
- [ ] Watch Live map — the animated dot should only travel along completed hops (solid line)
- [ ] Unreached hops should show as dashed/ghosted line at reduced opacity
- [ ] If trace completes fully, entire path should be solid
- [ ] Ghost line should auto-clean after ~10 seconds
### 3. "Paths through this node" accuracy (#655, #658)
- [ ] Go to: http://20.109.157.39/#/nodes/c0dedad4208acb6cbe44b848943fc6d3c5d43cf38a21e48b43826a70862980e4
- [ ] Check "Packets through this node" — packets should actually have this node in their path
- [ ] Compare with a node that shares a 2-char prefix (e.g. C0ffee SF) — they should show DIFFERENT packets
- [ ] Spot-check 3-4 packets: click through, verify path contains the node
### 4. Hash Stats "By Repeaters" (#652, #654)
- [ ] Go to Analytics → Hash Stats
- [ ] "By Repeaters" section should only show repeater-role nodes
- [ ] Compare count in "Multi-Byte Hash Adopters" vs "By Repeaters" — adopters may include companions, repeaters section should not
- [ ] Check that companions/rooms/sensors are excluded from the repeater distribution
### 5. Noise floor column chart (#600, #659)
- [ ] Go to Analytics → RF Health
- [ ] Noise floor chart should show vertical color-coded bars, NOT a line
- [ ] Green bars (< -100 dBm), yellow (-100 to -85), red (≥ -85)
- [ ] Hover over a bar — tooltip should show exact dBm + timestamp
- [ ] Check with only 1 observer selected — chart should still render (division by zero edge case)
- [ ] Reboot markers (if any) should show as vertical dashed lines
### 6. Async backfill on startup
- [ ] SSH to staging: `ssh -i ~/.ssh/id_ed25519 runner@20.109.157.39`
- [ ] `docker restart corescope-staging-go`
- [ ] Within 30 seconds, hit `curl http://localhost:82/api/stats` — should return data (not hang)
- [ ] Check `backfilling` and `backfillProgress` fields in stats response
- [ ] Server should be serving HTTP while backfill runs in background
---
## 🟡 MEDIUM RISK — Features
### 7. Distance unit preference (#621, #646)
- [ ] Go to Customizer → Display tab
- [ ] Change distance unit to "mi" — all distances should show in miles
- [ ] Change to "km" — all distances should show in km
- [ ] Change to "auto" — should use locale (US = miles, EU = km)
- [ ] Check Analytics page distances update after customizer change (no page reload needed)
- [ ] Check Node detail → Neighbors table distances
- [ ] Very small distances (<0.1 mi) should show in feet, not "0.0 mi"
### 8. Panel corner toggle (#608, #657)
- [ ] Go to Live map page
- [ ] Each panel (feed, legend, node detail) should have a small corner-toggle button
- [ ] Click the button — panel should snap to next corner (TL → TR → BR → BL)
- [ ] Refresh page — panel positions should persist (localStorage)
- [ ] Move two panels to same corner — collision avoidance should skip to next free corner
- [ ] On mobile viewport (<768px?) — toggle buttons should be hidden
### 9. Deep linking (#536, #618)
- [ ] Navigate to Nodes page, click a node → URL should update with pubkey hash
- [ ] Copy URL, open in new tab → should land on same node
- [ ] Apply packet filters → URL hash should include filter params
- [ ] Channels page: select a node → URL should reflect selection
- [ ] Analytics tabs: switch tabs → URL should include tab name
- [ ] Share a deep link with someone — they should see the same view
### 10. Sortable tables (#620, #638, #639)
- [ ] Nodes list: click column headers — should sort ascending/descending
- [ ] Sort indicator (arrow) should be visible on active column
- [ ] Node detail → Neighbors table: sortable
- [ ] Node detail → Observers table: sortable
- [ ] Packets table: sortable by column headers
### 11. Channel color highlighting (#271, #607, #611)
- [ ] Go to Channels page
- [ ] Assign a color to a channel using the color picker
- [ ] Feed rows should highlight with that color
- [ ] Change color — should update immediately
- [ ] Refresh — color assignment should persist
### 12. Collapsible panels (#606)
- [ ] Live map: panels should have collapse/expand toggle
- [ ] Collapsed panel should show just the header
- [ ] State should persist across page navigations
### 13. Mobile accessibility (#630, #633)
- [ ] Open staging on phone (or Chrome DevTools mobile emulation)
- [ ] Touch targets should be at least 44×44px
- [ ] Channel color picker should work on mobile
- [ ] No horizontal scroll on any page
- [ ] ARIA labels present on interactive elements (inspect with accessibility tools)
### 14. Map byte-size filter (#565, #568)
- [ ] Go to Map page
- [ ] Find the byte-size filter control
- [ ] Filter by packet size — map should update to show only matching packets
- [ ] Clear filter — all packets should return
### 15. API key security (#532, #628)
- [ ] Try accessing a write endpoint without API key — should be blocked
- [ ] Try with a weak key (e.g., "test", "admin") — should be rejected at startup
- [ ] Check staging logs for API key warning: `docker logs corescope-staging-go 2>&1 | grep -i "apiKey\|api_key\|security"`
### 16. OpenAPI/Swagger (#530, #632)
- [ ] Hit http://20.109.157.39/api/spec — should return valid OpenAPI 3.0 spec
- [ ] Hit http://20.109.157.39/api/docs — should show Swagger UI
- [ ] Try an endpoint from Swagger UI — should work
---
## 🟢 LOW RISK — Verify Quickly
### 17. View Route on Map button
- [ ] Go to any packet detail page
- [ ] Click "View Route on Map" — should navigate to map with route highlighted
### 18. og-image compression
- [ ] Check page source or network tab — og-image.png should be < 300KB (was 1.1MB)
### 19. Prefix Tool
- [ ] Analytics → Prefix Tool tab should load
- [ ] Should show collision data
### 20. License
- [ ] Check repo footer/LICENSE — should be GPL v3
### 21. Docker DISABLE_CADDY
- [ ] (If testable) Set DISABLE_CADDY=true — Caddy should not start
### 22. Region filter on RF Health
- [ ] RF Health tab: change region filter — charts should update
---
## 🏁 Sign-off
| Section | Status | Notes |
|---------|--------|-------|
| High risk (1-6) | ☐ | |
| Medium risk (7-16) | ☐ | |
| Low risk (17-22) | ☐ | |
| **Overall** | ☐ | |
**Tested by:** _______________
**Date:** _______________
**Staging version:** `curl -s http://20.109.157.39/api/stats | jq .version`
**Ready for release:** ☐ Yes / ☐ No — blockers: _______________
+309
View File
@@ -0,0 +1,309 @@
# v3.4.2 Release Test Plan
**Scope:** 90 commits since v3.4.1 (84 files, +14,931 / -1,005)
**Categories:** 19 perf, 19 feat, 18 fix, 15 docs, 3 chore, 1 test, 1 refactor, 1 ci
**Date:** 2026-04-08
---
## A. Automated Tests — Verify All Pass
### Go Backend
```bash
cd cmd/server && go test -race -count=1 ./...
cd cmd/ingestor && go test -race -count=1 ./...
```
**Test files (27 total):**
| File | Tests For |
|------|-----------|
| `cmd/server/decoder_test.go` | Hash size zero-hop, TRACE hopsCompleted, transport direct |
| `cmd/server/backfill_async_test.go` | **NEW** — Async chunked backfill |
| `cmd/server/eviction_test.go` | Memory eviction with runtime heap stats |
| `cmd/server/apikey_security_test.go` | **NEW** — Weak/default API key rejection |
| `cmd/server/openapi_test.go` | **NEW** — OpenAPI spec generation |
| `cmd/server/routes_test.go` | Batch observations endpoint, subpaths-bulk, expand=observations |
| `cmd/server/cache_invalidation_test.go` | cacheTTL config wiring |
| `cmd/server/config_knobs_test.go` | cacheTTLSec helper |
| `cmd/server/helpers_test.go` | constantTimeEqual, IsWeakAPIKey |
| `cmd/server/obs_dedup_test.go` | UniqueObserverCount tracking |
| `cmd/server/neighbor_*.go` (4 files) | Neighbor graph, affinity, persistence |
| `cmd/server/perfstats_race_test.go` | Perf stats concurrency |
| `cmd/server/resolve_context_test.go` | Resolved path filtering |
| `cmd/server/advert_pubkey_test.go` | Advert pubkey tracking |
| `cmd/server/db_test.go` | SQLite operations |
| `cmd/server/config_test.go` | Config loading |
| `cmd/server/coverage_test.go` | Coverage helpers |
| `cmd/server/parity_test.go` | Go/JS decoder parity |
| `cmd/server/websocket_test.go` | WebSocket broadcast |
| `cmd/ingestor/decoder_test.go` | Ingestor decoder (hash size zero-hop) |
| `cmd/ingestor/db_test.go` | Ingestor DB writes |
| `cmd/ingestor/config_test.go` | Ingestor config |
| `cmd/ingestor/main_test.go` | Ingestor entry |
| `cmd/ingestor/coverage_boost_test.go` | Coverage helpers |
### Frontend Unit Tests
```bash
node test-packet-filter.js
node test-aging.js
node test-frontend-helpers.js
node test-table-sort.js # NEW — shared table sort utility
node test-channel-colors.js # NEW — channel color model
node test-panel-corner.js # NEW — panel corner toggle
node test-packets.js # NEW — packets page logic
node test-hop-resolver-affinity.js
node test-customizer-v2.js
node test-live.js
node test-live-dedup.js
```
### E2E / Playwright
```bash
BASE_URL=http://localhost:13581 node test-e2e-playwright.js
```
**Expected:** All existing tests pass + new tests added for sortable tables, deep linking, collapsible panels.
---
## B. Manual Browser Verification
### B1. HIGH RISK — Data Correctness
| # | Feature | Page | What to Check |
|---|---------|------|---------------|
| 1 | Hash size zero-hop | Packets detail | Find a direct (route_type=0) packet → hash_size should show 0, not a bogus computed value |
| 2 | TRACE hopsCompleted | Packets detail / Live map | Find a TRACE packet → verify `hopsCompleted` shows in decoded JSON, live map shows real path length vs intended |
| 3 | Transport direct hash size | Packets detail | Find route_type=RouteTransportDirect packet → hash_size=0 |
| 4 | resolved_path filtering | Node detail → Paths tab | Verify path-hop candidates use resolved_path, no prefix collision false positives |
| 5 | Hash stats repeater filter | Analytics → Hash Issues | "By Repeaters" should only show nodes with repeater role, not companions/sensors |
| 6 | Async chunked backfill | Server startup | Start server with large DB → verify HTTP serves within 2 minutes, `X-CoreScope-Status: backfilling` header present, then transitions to `ready` |
| 7 | Memory eviction (heap stats) | Admin/stats | Verify `/api/stats` shows realistic memory numbers from runtime heap, not the old estimation |
| 8 | Distance/subpath/path-hop indexes | Analytics → Distances, Subpaths | Verify analytics data matches v3.4.1 output (no missing or extra entries) |
| 9 | cacheTTL config wiring | Config | Set `cacheTTL.analyticsHashSizes: 300` in config → verify collision cache respects it |
### B2. MEDIUM RISK — User-Facing Features
| # | Feature | Page | What to Check |
|---|---------|------|---------------|
| 10 | Distance unit preference | Nodes detail, Map | Toggle km/mi/auto in settings → distances update throughout UI |
| 11 | Panel corner toggle | Live page | Click corner toggle → panel moves to opposite corner, persists on reload |
| 12 | Noise floor column chart | Analytics → RF | Verify column chart renders with color-coded thresholds, hover shows values |
| 13 | Deep linking UI states | All pages | Navigate to `#/nodes?tab=neighbors`, `#/packets?observer=X`, `#/channels?node=Y` → correct state loads. Copy URL, open in new tab → same state |
| 14 | Sortable tables | Nodes list, Neighbors, Observers | Click column headers → sort asc/desc, indicator arrow shows, persists correctly |
| 15 | Channel color highlighting | Channels, Live feed | Assign color to channel → feed rows show that color, persists on reload |
| 16 | Mobile accessibility | All pages (phone viewport) | Touch targets ≥44px, ARIA labels present, small viewport doesn't overflow |
| 17 | Collapsible panels | Live map | Collapse/expand panels, medium breakpoint auto-collapses, state persists |
| 18 | Byte-size map filter | Map page | Filter by byte size → markers update correctly |
| 19 | OpenAPI/Swagger | `/api/spec`, `/api/docs` | Spec loads valid JSON, Swagger UI renders and all endpoints are documented |
| 20 | API key rejection | Protected endpoints | Send weak key (e.g. "changeme", "test123") → 403 forbidden |
| 21 | Channel color picker mobile | Channels (phone viewport) | Color picker usable on touch, doesn't overflow |
| 22 | RF Health dashboard | Analytics → RF Health | Observer metrics grid, airtime charts, battery charts, error rate, region filter |
| 23 | Prefix Tool tab | Analytics → Prefix Tool | Renders correctly, collision data consistent with Hash Issues |
| 24 | View Route on Map | Packet detail page | Button works and shows route on map |
### B3. LOWER RISK — Performance (Verify No Regressions)
| # | Feature | Page | What to Check |
|---|---------|------|---------------|
| 25 | Incremental DOM diff | Packets (30K+) | Virtual scroll renders smoothly, no visible flicker |
| 26 | Coalesced WS renders | Live page | Rapid packets don't cause frame drops (rAF coalescing) |
| 27 | Marker reposition on zoom | Map | Zoom/resize → markers move smoothly, no full rebuild flash |
| 28 | Parallel replay fetches | Live → VCR | Replay loads quickly (parallel observation fetches) |
| 29 | Batch observations API | Packets page (sort change) | Changing sort fetches observations in batch (network tab: 1 POST not N GETs) |
| 30 | Client-side network status | Analytics | No separate API call for network status |
| 31 | og-image compression | `/og-image.png` | Verify loads, ~235KB not ~1.1MB |
---
## C. API Regression Tests
Run against a local server with test-fixture DB:
```bash
BASE=http://localhost:13581
# Core endpoints — verify response shape
curl -s "$BASE/api/stats" | jq '.totalPackets, .backfilling, .backfillProgress'
curl -s "$BASE/api/packets?limit=5" | jq '.packets[0] | keys'
curl -s "$BASE/api/packets?limit=5&expand=observations" | jq '.packets[0].observations | length'
curl -s "$BASE/api/nodes?limit=5" | jq '.[0] | keys'
# New endpoints
curl -s -X POST "$BASE/api/packets/observations" \
-H 'Content-Type: application/json' \
-d '{"hashes":["test123"]}' | jq '.results | keys'
curl -s "$BASE/api/analytics/subpaths-bulk?hops=A,B&hops=B,C" | jq 'keys'
curl -s "$BASE/api/observers/metrics/summary" | jq 'type'
curl -s "$BASE/api/spec" | jq '.openapi'
curl -s "$BASE/api/docs" | head -5 # Should return HTML
# Backfill status header
curl -sI "$BASE/api/stats" | grep X-CoreScope-Status
# API key rejection
curl -s -H 'X-API-Key: changeme' "$BASE/api/debug/vars" | jq '.error'
curl -s -H 'X-API-Key: test' "$BASE/api/debug/vars" | jq '.error'
# Existing endpoints — verify not broken
curl -s "$BASE/api/analytics/rf?timeRange=24h" | jq 'keys'
curl -s "$BASE/api/analytics/hash-sizes" | jq 'type'
curl -s "$BASE/api/analytics/distances" | jq 'type'
curl -s "$BASE/api/analytics/subpaths" | jq 'type'
curl -s "$BASE/api/channels" | jq 'type'
curl -s "$BASE/api/config/client" | jq 'keys'
```
### Expected response shape changes from v3.4.1:
- `/api/stats` now includes `backfilling` (bool) and `backfillProgress` (float 0-1)
- `/api/packets` no longer strips observations by default (lazy via `ExpandObservations` flag) — verify `observations` key absent without `expand=observations`
- Decoded packets with route_type=direct now have `hashSize: 0`
- TRACE packets now have `path.hopsCompleted` field
---
## D. Performance Regression Tests
### D1. Server Startup Time
```bash
# Start server with production-size DB (~30K packets)
# Measure time from process start to first successful HTTP response
time curl -s http://localhost:13581/api/stats > /dev/null
# Target: < 2 minutes (async backfill requirement)
```
### D2. Go Benchmarks
```bash
cd cmd/server && go test -bench=. -benchmem -count=3
```
Key benchmarks to compare with v3.4.1 baseline:
- `BenchmarkQueryPackets` — should not regress with new indexes
- `BenchmarkEvictStale` — batch removal from secondary indexes
- `BenchmarkGetStoreStats` — 2 concurrent queries vs 5 sequential
- `BenchmarkIngestNew` — additional index maintenance overhead
### D3. Frontend Performance
- Open Packets page with 30K+ packets → measure initial render time (DevTools Performance tab)
- Scroll rapidly through virtual scroll → should maintain 60fps
- Switch sort column on packets → single batch POST, not N+1 GETs
- Open Analytics page → no redundant API calls in network tab
### D4. Memory Usage
- After loading 30K packets, check `/api/stats` memory figure
- Compare with v3.4.1 baseline (prefix map cap at 8 chars should reduce ~10x)
- Verify eviction triggers at correct memory threshold using runtime heap stats
---
## E. Infrastructure / Deployment Tests
### E1. Docker Build
```bash
docker build -t corescope:test .
docker run --rm -p 13581:13581 corescope:test
# Verify: container starts, HTTP responds, WebSocket connects
```
### E2. GHCR Publish (CI)
- Verify CI publishes to `ghcr.io/kpa-clawbot/corescope`
- Verify tags: `edge` (master), `vX.Y.Z` (release)
### E3. Staging Deploy
```bash
# Verify staging compose works with standard ports
docker compose -f docker-compose.staging.yml up -d
# Check: no 3GB memory limit, standard port binding
```
### E4. DISABLE_CADDY
```bash
docker run --rm -e DISABLE_CADDY=true corescope:test
# Verify: Caddy not started, Go server serves directly
```
### E5. CI Pipeline
- Verify consolidated pipeline: build → publish GHCR → deploy staging
- Verify runs on `meshcore-runner-2`
---
## F. Edge Cases & Integration Tests
### F1. Cross-Feature Interactions
| Scenario | Risk |
|----------|------|
| Deep link to sorted table → sort state matches URL params | Medium |
| Channel color + deep link → color persists in linked URL | Medium |
| Panel corner toggle + collapsible panels → both states persist independently | Low |
| Distance unit pref + neighbor table sort by distance → sort uses correct unit | Medium |
| Noise floor chart + region filter → chart respects filter | Medium |
| Byte-size map filter + channel color highlighting → both active simultaneously | Low |
### F2. Data Correctness Edge Cases
| Scenario | Risk |
|----------|------|
| Zero-hop TRACE packet (should NOT reset hashSize — TRACE exemption) | **High** |
| Packet with all hops having same 2-char prefix → resolved_path filtering prevents false match | **High** |
| Node that switches role (repeater → companion) → hash stats updates | Medium |
| Backfill interrupted mid-chunk (server restart) → resumes or completes on next start | Medium |
| Empty DB startup → no errors, backfill completes instantly | Low |
| DB with 100K+ packets → async backfill doesn't OOM, progress reported | **High** |
### F3. Concurrency / Race Conditions
| Scenario | Risk |
|----------|------|
| Concurrent API requests during backfill → no deadlock (lock ordering documented) | **High** |
| Eviction running while analytics query in progress → no stale pointer panic | **High** |
| Multiple WebSocket clients during high ingest rate → coalesced broadcasts don't drop | Medium |
| `time.NewTicker` cleanup on graceful shutdown (replaced `time.Tick`) | Low |
### F4. API Key Security
| Scenario | Expected |
|----------|----------|
| No API key configured → write endpoints disabled | 403 "write endpoints disabled" |
| Weak key "changeme" → rejected even if configured | 403 "forbidden" |
| Timing-safe comparison → no timing oracle | Constant-time via `crypto/subtle` |
| Empty string key → rejected | 401 "unauthorized" |
### F5. Browser Compatibility
- Test on Chrome, Firefox, Safari (latest)
- Test on iOS Safari, Android Chrome
- Verify touch targets on mobile (44px minimum)
- Verify ARIA labels with screen reader
---
## G. Test Coverage Gaps — Action Items
| Gap | Priority | Action |
|-----|----------|--------|
| No automated test for distance unit preference rendering | Medium | Add Playwright test |
| No automated test for noise floor column chart | Medium | Add Playwright test |
| No automated test for deep link state restoration | **High** | Add Playwright tests for each deep-linkable state |
| No automated test for channel color persistence | Medium | `test-channel-colors.js` covers model; need Playwright for UI |
| No automated test for mobile viewport behavior | Medium | Add Playwright test with mobile viewport |
| No automated test for backfill progress header | Low | Add to `routes_test.go` |
| No automated test for `time.NewTicker` cleanup | Low | Add to graceful shutdown test |
| Observer metrics endpoints not covered in route tests | Medium | Add to `routes_test.go` |
| Subpaths-bulk endpoint needs test | Medium | Add to `routes_test.go` |
| No load test for batch observations endpoint (200 hash limit) | Low | Add boundary test |
---
## H. Release Checklist
- [ ] All Go tests pass with `-race` flag
- [ ] All frontend unit tests pass
- [ ] Playwright E2E tests pass
- [ ] Manual browser verification (Section B) complete
- [ ] API regression tests (Section C) pass
- [ ] Docker build succeeds
- [ ] Staging deploy verified
- [ ] No console errors on any page
- [ ] Performance spot-checks (Section D) — no regressions
- [ ] Coverage badges updated (backend ≥85%, frontend ≥42%)
- [ ] CHANGELOG updated
- [ ] Tag `v3.4.2` created
+88
View File
@@ -0,0 +1,88 @@
# Analytics
The Analytics page provides deep-dive charts and tables about your mesh network. Select a tab to explore different aspects.
[Screenshot: analytics page with tab bar]
## Overview
Summary dashboard with key network metrics at a glance. Quick sparklines and counts across all data dimensions.
## RF / Signal
Radio frequency analysis:
- **SNR distribution** — histogram of signal-to-noise ratios across all packets
- **RSSI distribution** — histogram of received signal strength
- **SNR by observer** — which observers are getting the best signals
- **Signal trends** — how signal quality changes over time
Use this to identify weak links or noisy observers.
## Topology
Network structure analysis:
- **Hop count distribution** — how many relay hops packets typically take
- **Top relay nodes** — which repeaters handle the most traffic
- **Node connectivity** — how well-connected each node is
## Channels
Channel message statistics:
- **Messages per channel** — which channels are most active
- **Channel activity over time** — traffic trends by channel
- **Top senders** — most active nodes per channel
## Hash Stats
Mesh hash size analysis:
- **Hash size distribution** — how many bytes nodes use for addressing
- **Hash sizes by role** — do repeaters use different hash sizes than companions?
## Hash Issues
Potential hash collision detection:
- **Collision pairs** — nodes whose short hash prefixes overlap
- **Risk assessment** — how likely collisions are at current hash sizes
Hash collisions can cause packet misrouting. If you see collisions here, consider increasing hash sizes on affected nodes.
## Route Patterns (Subpaths)
Common routing paths through the mesh:
- **Frequent subpaths** — which relay chains appear most often
- **Path reliability** — how consistently each path is used
- **Path detail** — click a subpath to see every packet that used it
## Nodes
Per-node analytics with sortable metrics across the fleet.
## Distance
Estimated distances between nodes based on GPS coordinates, correlated with signal quality.
## Neighbor Graph
Interactive visualization of which nodes can directly hear each other. Shows the mesh topology as a network graph.
## RF Health
Per-observer signal health over time. Identifies observers with degrading reception.
## Prefix Tool
Test hash prefix lengths to see how many collisions different sizes would produce. Useful for deciding on hash_size settings.
## Region filter
All analytics tabs respect the **region filter** at the top. Select a region to scope the data to observers in that area.
## Deep linking
Each tab is deep-linkable. Share a URL like `#/analytics?tab=collisions` to point someone directly at hash issues.
+68
View File
@@ -0,0 +1,68 @@
# Channels
The Channels page shows decrypted MeshCore channel messages — like a group chat viewer for your mesh.
[Screenshot: channels page with message list]
## What are channels?
MeshCore nodes can send messages on named channels (like `#LongFast` or `#test`). These are group messages broadcast through the mesh. Any observer that hears the packet captures it.
CoreScope can decrypt and display these messages if you provide the channel encryption key.
## How it works
1. Observers capture encrypted channel packets from the mesh
2. CoreScope matches the packet's channel hash to a known channel name
3. If a decryption key is configured, the message content is decrypted and displayed
4. Without a key, you'll see the packet metadata but not the message text
## Viewing messages
Select a channel from the list on the left. Messages appear in chronological order on the right.
Each message shows:
- **Sender** — node name or hash
- **Text** — decrypted message content
- **Observer** — which observer captured it
- **Time** — when it was received
The message list auto-scrolls to show new messages as they arrive via WebSocket.
## Channel keys
To decrypt messages, add channel keys to your `config.json`:
```json
{
"channelKeys": {
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
}
}
```
The key name (e.g., `"public"`) is a label for your reference. The value is the 16-byte hex encryption key for that channel.
See [Configuration](configuration.md) for details on `channelKeys` and `hashChannels`.
## Hash channels
The `hashChannels` config lists channel names that CoreScope should try to match by hash:
```json
{
"hashChannels": ["#LongFast", "#test", "#sf"]
}
```
CoreScope computes the hash of each name and matches incoming packets to identify which channel they belong to.
## Region filter
Channels respect the region filter. Select a region to see only messages captured by observers in that area.
## Tips
- The default MeshCore "public" channel key is well-known — most community meshes use it
- If messages appear but show garbled text, your key may be wrong
- Not all packets are channel messages — only type "Channel Msg" (GRP_TXT) appears here
+181
View File
@@ -0,0 +1,181 @@
# Configuration
CoreScope is configured via `config.json` in the server's working directory. Copy `config.example.json` to get started.
## Core settings
| Field | Default | Description |
|-------|---------|-------------|
| `port` | `3000` | HTTP server port |
| `apiKey` | — | Secret key for admin API endpoints (POST/PUT routes) |
| `dbPath` | — | Path to SQLite database file (optional, defaults to `meshcore.db`) |
## MQTT
```json
"mqtt": {
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
}
```
The ingestor connects to this MQTT broker and subscribes to the topic pattern.
### Multiple MQTT sources
Use `mqttSources` for multiple brokers:
```json
"mqttSources": [
{
"name": "local",
"broker": "mqtt://localhost:1883",
"topics": ["meshcore/#"]
},
{
"name": "remote",
"broker": "mqtts://mqtt.example.com:8883",
"username": "user",
"password": "pass",
"topics": ["meshcore/SJC/#"]
}
]
```
## Branding
| Field | Description |
|-------|-------------|
| `branding.siteName` | Site title shown in the nav bar |
| `branding.tagline` | Subtitle on the home page |
| `branding.logoUrl` | URL to a custom logo image |
| `branding.faviconUrl` | URL to a custom favicon |
## Theme
Colors used throughout the UI. All values are hex color codes.
| Field | Description |
|-------|-------------|
| `theme.accent` | Primary accent color (links, buttons) |
| `theme.navBg` | Navigation bar background |
| `theme.navBg2` | Secondary nav background |
| `theme.statusGreen` | Healthy status color |
| `theme.statusYellow` | Degraded status color |
| `theme.statusRed` | Silent/error status color |
See [Customization](customization.md) for the full list — the theme customizer exposes every color.
## Node colors
Default marker colors by role:
```json
"nodeColors": {
"repeater": "#dc2626",
"companion": "#2563eb",
"room": "#16a34a",
"sensor": "#d97706",
"observer": "#8b5cf6"
}
```
## Health thresholds
How long (in hours) before a node is marked degraded or silent:
| Field | Default | Description |
|-------|---------|-------------|
| `healthThresholds.infraDegradedHours` | `24` | Repeaters/rooms → degraded after this many hours |
| `healthThresholds.infraSilentHours` | `72` | Repeaters/rooms → silent after this many hours |
| `healthThresholds.nodeDegradedHours` | `1` | Companions/others → degraded |
| `healthThresholds.nodeSilentHours` | `24` | Companions/others → silent |
## Retention
| Field | Default | Description |
|-------|---------|-------------|
| `retention.nodeDays` | `7` | Nodes not seen in N days move to inactive |
| `retention.packetDays` | `30` | Packets older than N days are deleted daily |
## Channel decryption
| Field | Description |
|-------|-------------|
| `channelKeys` | Object of `"label": "hex-key"` pairs for decrypting channel messages |
| `hashChannels` | Array of channel names (e.g., `"#LongFast"`) to match by hash |
See [Channels](channels.md) for details.
## Map defaults
```json
"mapDefaults": {
"center": [37.45, -122.0],
"zoom": 9
}
```
Initial map center and zoom level.
## Regions
```json
"regions": {
"SJC": "San Jose, US",
"SFO": "San Francisco, US"
}
```
Named regions for the region filter dropdown. The `defaultRegion` field sets which region is selected by default.
## Cache TTL
All values in seconds. Controls how long the server caches API responses:
```json
"cacheTTL": {
"stats": 10,
"nodeList": 90,
"nodeDetail": 300,
"analyticsRF": 1800
}
```
Lower values = fresher data but more server load.
## Packet store
| Field | Default | Description |
|-------|---------|-------------|
| `packetStore.maxMemoryMB` | `1024` | Maximum RAM for in-memory packet store |
| `packetStore.estimatedPacketBytes` | `450` | Estimated bytes per packet (for memory budgeting) |
## Timestamps
| Field | Default | Description |
|-------|---------|-------------|
| `timestamps.defaultMode` | `"ago"` | Display mode: `"ago"` (relative) or `"absolute"` |
| `timestamps.timezone` | `"local"` | `"local"` or `"utc"` |
| `timestamps.formatPreset` | `"iso"` | Date format preset |
## Live map
| Field | Default | Description |
|-------|---------|-------------|
| `liveMap.propagationBufferMs` | `5000` | How long to buffer observations before animating |
## HTTPS
```json
"https": {
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
}
```
Provide cert and key paths to enable HTTPS.
## Home page
The `home` section customizes the onboarding experience. See `config.example.json` for the full structure including `steps`, `checklist`, and `footerLinks`.
+78
View File
@@ -0,0 +1,78 @@
# Customization
CoreScope includes a built-in theme customizer. Access it from **Tools → Customization** in the navigation menu.
[Screenshot: theme customizer panel with color pickers]
## What you can customize
### Branding
- **Site name** — displayed in the nav bar and browser tab
- **Tagline** — shown on the home page
- **Logo URL** — replace the default logo
- **Favicon URL** — custom browser tab icon
### Theme colors (Light & Dark)
Every color in the UI is customizable:
- **Accent** — primary color for links, buttons, highlights
- **Navigation** — nav bar background, text, and muted text colors
- **Background** — page background and content area
- **Surfaces** — cards, panels, input fields, detail panes
- **Status** — green (healthy), yellow (degraded), red (silent)
- **Text** — primary text, muted text, borders
- **Tables** — row stripe, hover, and selected row colors
Both light and dark themes are independently configurable.
### Node colors
Set the color for each role: repeater, companion, room, sensor, observer. These colors appear on the map, in node badges, and throughout the UI.
### Packet type colors
Customize the color for each packet type: Advert, Channel Msg, Direct Msg, ACK, Request, Response, Trace, Path.
### Home page
Customize the onboarding experience:
- Hero title and subtitle
- Getting-started steps (emoji, title, description for each)
- FAQ items
- Footer links
### Timestamps
- **Display mode** — relative ("5 min ago") or absolute
- **Timezone** — local or UTC
- **Format preset** — ISO or other presets
## Live preview
Changes apply instantly as you edit. You see the result in real time without saving.
## Exporting a theme
Click **Export JSON** to download your customizations as a JSON file. This produces a config-compatible block you can paste into your `config.json`.
## Importing a theme
Click **Import JSON** and paste a previously exported theme. The customizer loads all values and applies them immediately.
## Resetting
Click **Reset to Defaults** to restore all settings to the built-in defaults.
## How it works
The customizer writes CSS custom properties (variables) to override the defaults. Exported JSON maps directly to the `theme`, `nodeColors`, `branding`, and `home` sections of [config.json](configuration.md).
## Tips
- Start with the accent color — it cascades through buttons, links, and highlights
- Dark mode has its own color set (`themeDark`), independent of light mode
- Node colors affect the [Map](map.md), [Live](live.md) page, and node badges everywhere
- Export your theme before upgrading CoreScope, then re-import it after
+65
View File
@@ -0,0 +1,65 @@
# FAQ
## 1. How do I add my node to CoreScope?
Go to the **Home** page, search for your node by name or public key, and click **+ Claim**. Your node appears on the dashboard with live status.
## 2. Why does my node show as "Silent"?
Your node hasn't been heard by any observer within the configured threshold. For companions, the default is 24 hours. For repeaters, it's 72 hours. Check that your node is advertising and within range of an observer. See [Configuration](configuration.md) for threshold settings.
## 3. What's the difference between "Last seen" and "Last heard"?
**Last seen** updates only when a node sends an advertisement. **Last heard** updates on *any* traffic from that node. CoreScope uses whichever is more recent for status calculations.
## 4. Why can't I read channel messages?
You need the channel encryption key in your `config.json`. See [Channels](channels.md) for how to configure `channelKeys`.
## 5. What do the packet types mean?
| Type | Meaning |
|------|---------|
| Advert | Node announcing itself to the mesh |
| Channel Msg | Group message on a named channel |
| Direct Msg | Private message between two nodes |
| ACK | Acknowledgment of a received packet |
| Request | Query sent to the mesh |
| Response | Reply to a request |
| Trace | Route tracing packet |
| Path | Path discovery/announcement |
## 6. How do I filter packets by a specific node?
On the [Packets](packets.md) page, use the filter bar and type `from:NodeName` or click a node's name anywhere in the UI to jump to its packets.
## 7. Why do some nodes appear faded on the map?
Faded markers indicate **stale** nodes — they haven't been heard recently. The threshold depends on the node's role.
## 8. Can I run CoreScope without MQTT?
Yes. You can POST packets directly to the `/api/packets` endpoint using the API key. However, MQTT is the standard way to ingest data from mesh observers.
## 9. How do I change the map's default location?
Set `mapDefaults.center` and `mapDefaults.zoom` in your `config.json`. See [Configuration](configuration.md).
## 10. How do I share a link to a specific packet or view?
CoreScope uses URL hashes for deep linking. Copy the URL from your browser — it includes the current page, filters, and selected items. Examples:
- `#/packets/abc123` — a specific packet
- `#/analytics?tab=collisions` — the hash issues tab
- `#/nodes/pubkey123` — a specific node's detail page
### Where is the API documentation?
CoreScope auto-generates an OpenAPI 3.0 specification from its route definitions:
- **Interactive docs (Swagger UI):** `/api/docs` — browse and test all 40+ endpoints from your browser
- **Machine-readable spec:** `/api/spec` — import into Postman, Insomnia, or any OpenAPI tool
The spec is always in sync with the running server. No manual maintenance needed.
On the public instance: [analyzer.00id.net/api/docs](https://analyzer.00id.net/api/docs)
+70
View File
@@ -0,0 +1,70 @@
# Getting Started
## What is CoreScope?
CoreScope is a web-based analyzer for **MeshCore LoRa mesh networks**. It shows you every node, packet, and signal path in your mesh — in real time.
Use it to monitor node health, debug connectivity, view decrypted channel messages, and understand how your mesh is performing.
## What you need
- A running CoreScope server (Go binary + SQLite database)
- An MQTT broker feeding mesh packets into the CoreScope ingestor
- A modern web browser
## Quick start
### 1. Configure
Copy `config.example.json` to `config.json` and edit it:
```json
{
"port": 3000,
"apiKey": "pick-a-secret-key",
"mqtt": {
"broker": "mqtt://your-broker:1883",
"topic": "meshcore/+/+/packets"
}
}
```
See [Configuration](configuration.md) for all options.
### 2. Run
Start both the ingestor (reads MQTT → writes to SQLite) and the server (serves the UI + API):
```bash
./corescope-ingestor &
./corescope-server
```
### 3. Open the UI
Go to `http://localhost:3000`. You'll see the **Home** page.
- **New to MeshCore?** Choose "I'm new" for setup guides and tips.
- **Already set up?** Choose "I know what I'm doing" to jump straight in.
Search for your node by name or public key, then click **+ Claim** to add it to your personal dashboard.
## What's on each page
| Page | What it does |
|------|-------------|
| [Home](getting-started.md) | Your personal mesh dashboard — claimed nodes, health, stats |
| [Nodes](nodes.md) | Browse all nodes with status, role, and filters |
| [Packets](packets.md) | Inspect every packet — grouped or raw, with hex breakdown |
| [Map](map.md) | See node locations on a live map |
| [Live](live.md) | Watch packets flow in real time with map animations |
| [Analytics](analytics.md) | Deep-dive charts: RF, topology, routes, hash stats |
| [Channels](channels.md) | Read decrypted channel messages |
## Home page features
- **Claim nodes** — search and add nodes to "My Mesh" for at-a-glance status cards
- **Node cards** — show status (🟢 Active / 🟡 Degraded / 🔴 Silent), SNR, hops, packet count, and 24h sparkline
- **Health detail** — click a card to see full health: observers, recent packets, mini map
- **Packet journey** — click a recent packet to see sender → observer flow
- **Network stats** — total transmissions, nodes, observers, and 24h activity
+76
View File
@@ -0,0 +1,76 @@
# Live
The Live page shows packets flowing through your mesh in real time, with animated map visualizations.
[Screenshot: live page with map animations and packet feed]
## Real-time feed
Packets appear as they arrive via WebSocket. Each entry shows:
- Packet type icon and color
- Sender name
- Observer that captured it
- SNR and hop count
- Timestamp
The feed scrolls automatically. New packets appear at the top.
## Map animations
When a packet arrives, the Live map animates the signal path:
- A pulse appears at the sender's location
- Lines animate from sender to each observer that heard the packet
- Observer markers flash briefly on reception
### Realistic propagation
Enable **Realistic Propagation** in the controls to buffer observations of the same packet and animate them simultaneously — showing how a single transmission ripples through the mesh.
### Ghost hops
When enabled, intermediate relay hops are shown as faded markers even if they don't have known locations. Disable to show only nodes with GPS coordinates.
## VCR mode
The Live page has a built-in VCR (video cassette recorder) for packet replay.
| Button | Action |
|--------|--------|
| ⏸ Pause | Freeze the feed. New packets are buffered but not displayed. |
| ▶ Play | Resume live feed or start replay. |
| ⏪ Rewind | Step backward through packet history. |
| ⏩ Fast-forward | Replay at 2×, 4×, or 8× speed. |
While paused, a badge shows how many packets arrived that you haven't seen yet.
## Timeline
The timeline bar at the bottom shows packet activity over the selected time scope (default: 1 hour). Click anywhere on the timeline to jump to that point in time.
## Packet type legend
Each packet type has a color and icon:
| Type | Icon | Color |
|------|------|-------|
| Advert | 📡 | Green |
| Channel Msg | 💬 | Blue |
| Direct Msg | ✉️ | Amber |
| ACK | ✓ | Gray |
| Request | ❓ | Purple |
| Response | 📨 | Cyan |
| Trace | 🔍 | Pink |
| Path | 🛤️ | Teal |
## Controls
- **Favorites only** — show only packets from your claimed nodes
- **Matrix mode** — visual effect overlay (just for fun)
## Tips
- Use VCR pause when you spot something interesting — then step through packet by packet
- Realistic propagation mode is best for understanding multi-path reception
- The timeline sparkline shows traffic patterns — useful for spotting quiet periods or bursts
+71
View File
@@ -0,0 +1,71 @@
# Map
The Map page shows all nodes on an interactive map, color-coded by role.
[Screenshot: map with colored markers and controls panel]
## Marker shapes and colors
Each node role has a distinct shape and color:
| Role | Shape | Default Color |
|------|-------|---------------|
| Repeater | Diamond | Red |
| Companion | Circle | Blue |
| Room | Square | Green |
| Sensor | Triangle | Orange |
| Observer | Star | Purple |
Stale nodes (not heard recently) appear faded.
## Hash labels
Repeaters can display their short mesh hash ID instead of a plain marker. Toggle **Hash Labels** in the map controls to switch between icon markers and hash-labeled markers.
## Map controls
Open the controls panel with the ⚙️ button (top-right corner).
### Node types
Check or uncheck roles to show/hide them on the map. All roles are visible by default.
### Byte size filter
Filter nodes by packet size category: All, Small, Medium, Large.
### Status filter
Show only active, degraded, or silent nodes.
### Last heard filter
Limit the map to nodes heard within a time window (e.g., 24h, 7d, 30d).
### Clustering
Enable clustering to group nearby nodes into cluster bubbles. Zoom in to expand clusters.
### Neighbor filter
Select a reference node to highlight only its direct neighbors.
## Show Route
Click a node marker, then click **Show Route** in the popup to see the paths packets take to reach that node. Routes are drawn as lines between nodes.
## Popups
Click any marker to see:
- Node name and role
- Public key
- Last seen timestamp
- Link to the full node detail page
## Tips
- Zoom in on dense areas to see individual nodes
- Use the role checkboxes to isolate repeaters and understand coverage
- The neighbor filter is great for seeing which nodes can directly hear each other
- Node colors are [customizable](customization.md) in the theme settings
+70
View File
@@ -0,0 +1,70 @@
# Nodes
The Nodes page lists every node your mesh has seen — repeaters, companions, rooms, and sensors.
[Screenshot: nodes list with status indicators]
## What you see
Each row shows:
- **Name** — the node's advertised name (or public key if unnamed)
- **Role** — Repeater, Companion, Room, or Sensor
- **Status** — color-coded health indicator
- **Last seen** — when the node was last heard
- **Advert count** — how many advertisements this node has sent
## Status indicators
| Indicator | Meaning |
|-----------|---------|
| 🟢 Active | Heard recently (within threshold for its role) |
| 🟡 Degraded | Not heard for a while but not yet silent |
| 🔴 Silent | Not heard for an extended period |
Thresholds differ by role. Infrastructure nodes (repeaters, rooms) have longer grace periods than companions. See [Configuration](configuration.md) for `healthThresholds`.
## Filtering
### Role tabs
Click **All**, **Repeaters**, **Rooms**, **Companions**, or **Sensors** to filter by role.
### Search
Type in the search box to filter by name or public key. The filter applies instantly.
### Status filter
Filter to show only active, degraded, or silent nodes.
### Last heard filter
Filter nodes by how recently they were heard (e.g., last hour, last 24h).
## Sorting
Click any column header to sort. Click again to reverse the order. Your sort preference is saved across sessions.
## Node detail
Click a node row to open the **detail pane** on the right. It shows:
- Full public key
- Role and status explanation
- Location (if known)
- Recent packets involving this node
- Neighbor nodes
- Signal statistics
Click the node name in the detail pane to open the **full node page** with complete history, analytics, and health data.
## Favorites
Nodes you've claimed on the Home page appear as favorites. You can also star nodes directly from the Nodes page.
## Tips
- Use the search box for quick lookups — it matches partial names and keys
- Sort by "Last seen" descending to find the most active nodes
- The status explanation tells you exactly why a node is marked degraded or silent
+78
View File
@@ -0,0 +1,78 @@
# Packets
The Packets page shows every transmission captured by your mesh observers.
[Screenshot: packets table with grouped view]
## Grouped vs ungrouped view
By default, packets are **grouped by hash**. Each row represents one unique transmission, with a count of how many observers heard it.
Click **Ungroup** to see every individual observation as its own row.
Click the **▶** arrow on a grouped row to expand it and see all observations of that packet.
## What each row shows
- **Time** — when the packet was received
- **From** — sender node name or hash prefix
- **Type** — packet type (Advert, Channel Msg, Direct Msg, ACK, Request, Response, Trace, Path)
- **Observer** — which observer captured the packet
- **SNR** — signal-to-noise ratio in dB
- **RSSI** — received signal strength
- **Hops** — how many relay hops the packet took
## Filters
### Observer filter
Select a specific observer to see only packets it captured. Saved across sessions.
### Type filter
Filter by packet type (e.g., show only Adverts or Channel Messages).
### Time window
Choose how far back to look: 15 minutes, 1 hour, 6 hours, 24 hours, etc. On mobile, the window is capped at 3 hours for performance.
### Wireshark-style filter bar
Type filter expressions for advanced filtering:
```
type:advert snr>5 hops<3
from:MyNode observer:SJC
```
See the filter bar's help tooltip for all supported fields and operators.
## Packet detail
Click any row to open the **detail pane** on the right showing:
- Full packet metadata (hash, type, size, timestamp)
- Decoded payload fields
- Hop path with resolved node names
- All observers that heard this packet, sorted by SNR
### Hex breakdown
The detail pane includes a hex dump of the raw packet bytes with field boundaries highlighted.
## Observation sorting
When viewing a grouped packet's observations, they're sorted by SNR (best signal first). This helps you see which observer had the clearest reception.
## Display options
- **Hex hashes** — toggle to show packet hashes in hex format
- **Panel resize** — drag the detail pane border to resize it
- **Keyboard shortcuts** — press `Esc` to close the detail pane
## Tips
- Grouped view is best for understanding what's happening on the mesh
- Ungrouped view is best for debugging signal paths and comparing observers
- The time window filter is your best friend for managing large datasets
- Packet hashes in the URL are deep-linkable — share a link to a specific packet
+3
View File
@@ -0,0 +1,3 @@
module github.com/meshcore-analyzer/sigvalidate
go 1.22
+23
View File
@@ -0,0 +1,23 @@
// Package sigvalidate provides Ed25519 signature validation for MeshCore adverts.
package sigvalidate
import (
"crypto/ed25519"
"encoding/binary"
)
// ValidateAdvertSignature verifies an Ed25519 signature over a MeshCore advert.
// The signed message is: pubKey (32 bytes) || timestamp (4 bytes LE) || appdata.
// Returns false if pubKey is not 32 bytes or signature is not 64 bytes.
func ValidateAdvertSignature(pubKey, signature []byte, timestamp uint32, appdata []byte) bool {
if len(pubKey) != 32 || len(signature) != 64 {
return false
}
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pubKey)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
return ed25519.Verify(ed25519.PublicKey(pubKey), message, signature)
}
+1076 -89
View File
File diff suppressed because it is too large Load Diff
+46
View File
@@ -104,6 +104,46 @@ function timeAgo(iso) {
return value + suffix + ' ago';
}
function getHashParams() {
return new URLSearchParams(location.hash.split('?')[1] || '');
}
function getDistanceUnit() {
var stored = localStorage.getItem('meshcore-distance-unit');
if (stored === 'km') return 'km';
if (stored === 'mi') return 'mi';
// 'auto' or no value — locale detection
var milesLocales = ['en-us', 'en-gb'];
var lang = (typeof navigator !== 'undefined' && navigator.language || '').toLowerCase();
for (var i = 0; i < milesLocales.length; i++) {
if (lang === milesLocales[i] || lang.startsWith(milesLocales[i] + '-')) return 'mi';
}
return 'km';
}
window.getDistanceUnit = getDistanceUnit;
function formatDistance(km) {
if (km == null || isNaN(+km)) return '—';
var d = +km;
var unit = getDistanceUnit();
if (unit === 'mi') {
var mi = d / 1.60934;
if (mi < 0.1) return Math.round(mi * 5280) + ' ft';
return mi.toFixed(1) + ' mi';
}
if (d < 1) return Math.round(d * 1000) + ' m';
return d.toFixed(1) + ' km';
}
window.formatDistance = formatDistance;
function formatDistanceRound(km) {
if (km == null || isNaN(+km)) return '—';
var unit = getDistanceUnit();
if (unit === 'mi') return Math.round(+km / 1.60934) + ' mi';
return Math.round(+km) + ' km';
}
window.formatDistanceRound = formatDistanceRound;
function getTimestampMode() {
const saved = localStorage.getItem('meshcore-timestamp-mode');
if (saved === 'ago' || saved === 'absolute') return saved;
@@ -472,6 +512,12 @@ function navigate() {
const ms = performance.now() - t0;
if (ms > 100) console.warn(`[SLOW PAGE] ${basePage} init took ${Math.round(ms)}ms`);
app.classList.remove('page-enter'); void app.offsetWidth; app.classList.add('page-enter');
// #630-7: SPA focus management — move focus to first heading or main content
requestAnimationFrame(function() {
var heading = app.querySelector('h1, h2, h3, [role="heading"]');
if (heading) { heading.setAttribute('tabindex', '-1'); heading.focus({ preventScroll: true }); }
else { app.setAttribute('tabindex', '-1'); app.focus({ preventScroll: true }); }
});
} else {
app.innerHTML = `<div style="padding:40px;text-align:center;color:#6b7280"><h2>${route}</h2><p>Page not yet implemented.</p></div>`;
}
+273
View File
@@ -0,0 +1,273 @@
/**
* Channel Color Picker Simplified popover with 8-color constrained palette (#674)
*
* Click a color dot next to channel names (channels page, live feed) to open picker.
* Right-click on live feed items retained as power-user shortcut (desktop only).
* No long-press. No custom color input. 8 preset colors.
*
* Uses ChannelColors.set/get/remove from channel-colors.js.
*/
(function() {
'use strict';
// 8 maximally-distinct colors on dark backgrounds (#674 Tufte spec)
var CHANNEL_PALETTE = [
'#ef4444', // red
'#f97316', // orange
'#eab308', // yellow
'#22c55e', // green
'#06b6d4', // cyan
'#3b82f6', // blue
'#8b5cf6', // violet
'#ec4899' // pink
];
var popoverEl = null;
var currentChannel = null;
function createPopover() {
if (popoverEl) return popoverEl;
var el = document.createElement('div');
el.className = 'cc-picker-popover';
el.setAttribute('role', 'dialog');
el.setAttribute('aria-label', 'Channel color picker');
el.style.display = 'none';
el.innerHTML =
'<div class="cc-picker-swatches" role="group" aria-label="Color swatches"></div>' +
'<button class="cc-picker-clear">Clear color</button>';
// Build swatches
var swatchContainer = el.querySelector('.cc-picker-swatches');
for (var i = 0; i < CHANNEL_PALETTE.length; i++) {
var sw = document.createElement('button');
sw.className = 'cc-swatch';
sw.style.background = CHANNEL_PALETTE[i];
sw.setAttribute('data-color', CHANNEL_PALETTE[i]);
sw.setAttribute('aria-label', CHANNEL_PALETTE[i]);
sw.title = CHANNEL_PALETTE[i];
sw.setAttribute('tabindex', '0');
swatchContainer.appendChild(sw);
}
// Event: swatch click
swatchContainer.addEventListener('click', function(e) {
var btn = e.target.closest('.cc-swatch');
if (!btn) return;
assignColor(btn.getAttribute('data-color'));
});
// Keyboard navigation for swatches
swatchContainer.addEventListener('keydown', function(e) {
var btn = e.target.closest('.cc-swatch');
if (!btn) return;
var swatches = swatchContainer.querySelectorAll('.cc-swatch');
var idx = Array.prototype.indexOf.call(swatches, btn);
if (idx < 0) return;
var next = -1;
if (e.key === 'ArrowRight' || e.key === 'ArrowDown') next = (idx + 1) % swatches.length;
else if (e.key === 'ArrowLeft' || e.key === 'ArrowUp') next = (idx - 1 + swatches.length) % swatches.length;
else if (e.key === 'Enter' || e.key === ' ') { assignColor(btn.getAttribute('data-color')); e.preventDefault(); return; }
if (next >= 0) { swatches[next].focus(); e.preventDefault(); }
});
// Event: clear
el.querySelector('.cc-picker-clear').addEventListener('click', function() {
if (currentChannel && window.ChannelColors) {
window.ChannelColors.remove(currentChannel);
refreshVisibleRows();
}
hidePopover();
});
// Prevent right-click on the popover itself
el.addEventListener('contextmenu', function(e) { e.preventDefault(); });
document.body.appendChild(el);
popoverEl = el;
return el;
}
function assignColor(color) {
if (currentChannel && window.ChannelColors) {
window.ChannelColors.set(currentChannel, color);
refreshVisibleRows();
}
hidePopover();
}
function showPopover(channel, x, y) {
var el = createPopover();
currentChannel = channel;
// Highlight current color
var current = window.ChannelColors ? window.ChannelColors.get(channel) : null;
var swatches = el.querySelectorAll('.cc-swatch');
for (var i = 0; i < swatches.length; i++) {
swatches[i].classList.toggle('cc-swatch-active', swatches[i].getAttribute('data-color') === current);
}
// Show/hide clear button
el.querySelector('.cc-picker-clear').style.display = current ? '' : 'none';
// Position
el.style.display = '';
var isTouch = window.matchMedia('(pointer: coarse)').matches;
if (!isTouch) {
el.style.left = '0';
el.style.top = '0';
var rect = el.getBoundingClientRect();
var pw = rect.width;
var ph = rect.height;
var vw = window.innerWidth;
var vh = window.innerHeight;
var finalX = x + pw > vw ? Math.max(0, vw - pw - 8) : x;
var finalY = y + ph > vh ? Math.max(0, vh - ph - 8) : y;
el.style.left = finalX + 'px';
el.style.top = finalY + 'px';
}
// Lock background scroll while popover is open
document.body.style.overflow = 'hidden';
// Focus first swatch for keyboard accessibility
var firstSwatch = el.querySelector('.cc-swatch');
if (firstSwatch) setTimeout(function() { firstSwatch.focus(); }, 0);
// Listen for outside click / Escape
setTimeout(function() {
document.addEventListener('click', onOutsideClick, true);
document.addEventListener('keydown', onEscape, true);
}, 0);
}
function hidePopover() {
if (popoverEl) popoverEl.style.display = 'none';
currentChannel = null;
document.body.style.overflow = '';
document.removeEventListener('click', onOutsideClick, true);
document.removeEventListener('keydown', onEscape, true);
}
function onOutsideClick(e) {
if (popoverEl && !popoverEl.contains(e.target)) {
hidePopover();
}
}
function onEscape(e) {
if (e.key === 'Escape') {
hidePopover();
e.stopPropagation();
}
// Trap Tab within the popover
if (e.key === 'Tab' && popoverEl && popoverEl.style.display !== 'none') {
var focusable = popoverEl.querySelectorAll('button, [tabindex]');
if (focusable.length === 0) return;
var first = focusable[0];
var last = focusable[focusable.length - 1];
if (e.shiftKey && document.activeElement === first) {
last.focus(); e.preventDefault();
} else if (!e.shiftKey && document.activeElement === last) {
first.focus(); e.preventDefault();
}
}
}
/** Refresh channel color styles on all visible feed items, channel list, and packet rows. */
function refreshVisibleRows() {
if (!window.ChannelColors) return;
// Live feed items
var feedItems = document.querySelectorAll('.live-feed-item');
for (var i = 0; i < feedItems.length; i++) {
var item = feedItems[i];
var ch = item._ccChannel;
if (!ch) continue;
var color = window.ChannelColors.get(ch);
item.style.borderLeft = color ? '3px solid ' + color : '';
}
// Update color dots everywhere
var dots = document.querySelectorAll('.ch-color-dot');
for (var j = 0; j < dots.length; j++) {
var dot = dots[j];
var dotCh = dot.getAttribute('data-channel');
if (!dotCh) continue;
var dotColor = window.ChannelColors.get(dotCh);
dot.style.background = dotColor || '';
}
// Channel list items — update border
var chItems = document.querySelectorAll('.ch-item[data-hash]');
for (var k = 0; k < chItems.length; k++) {
var chItem = chItems[k];
var hash = chItem.getAttribute('data-hash');
if (!hash) continue;
var chColor = window.ChannelColors.get(hash);
chItem.style.borderLeft = chColor ? '3px solid ' + chColor : '';
}
// Packets table — trigger re-render via custom event
document.dispatchEvent(new CustomEvent('channel-colors-changed'));
}
/**
* Install context-menu (right-click) handler on the live feed.
* No long-press color dots handle mobile interaction.
*/
function installLiveFeedHandlers() {
var feed = document.getElementById('liveFeed');
if (!feed) return;
// Click on color dot opens picker (#674)
feed.addEventListener('click', function(e) {
var dot = e.target.closest('.feed-color-dot');
if (!dot) return;
e.stopPropagation();
var ch = dot.getAttribute('data-channel');
if (ch) showPopover(ch, e.clientX, e.clientY);
});
feed.addEventListener('contextmenu', function(e) {
var item = e.target.closest('.live-feed-item');
if (!item || !item._ccChannel) return;
e.preventDefault();
showPopover(item._ccChannel, e.clientX, e.clientY);
});
}
/**
* Install context-menu handler on the packets table.
*/
function installPacketsTableHandlers() {
var table = document.getElementById('packetsTableBody');
if (!table) return;
table.addEventListener('contextmenu', function(e) {
var row = e.target.closest('tr');
if (!row) return;
var chanTag = row.querySelector('.chan-tag');
if (chanTag) {
var ch = chanTag.textContent.trim();
if (ch) {
e.preventDefault();
showPopover(ch, e.clientX, e.clientY);
return;
}
}
});
}
// Export
window.ChannelColorPicker = {
install: function() {
installLiveFeedHandlers();
installPacketsTableHandlers();
},
installLiveFeed: installLiveFeedHandlers,
installPacketsTable: installPacketsTableHandlers,
show: showPopover,
hide: hidePopover,
PALETTE: CHANNEL_PALETTE
};
})();
+109
View File
@@ -0,0 +1,109 @@
/**
* Channel Color Highlighting Storage Model (M1)
*
* localStorage key: 'live-channel-colors'
* Value: JSON object mapping channel names to hex colors
* e.g. { "#wardriving": "#ef4444", "#meshnet": "#3b82f6" }
*
* Only applies to GRP_TXT packets. Other types retain default styling.
*/
(function() {
'use strict';
var STORAGE_KEY = 'live-channel-colors';
function _load() {
try {
return JSON.parse(localStorage.getItem(STORAGE_KEY)) || {};
} catch (e) {
return {};
}
}
function _save(colors) {
localStorage.setItem(STORAGE_KEY, JSON.stringify(colors));
}
/** Validate hex color format: #RGB or #RRGGBB */
var HEX_RE = /^#(?:[0-9a-fA-F]{3}|[0-9a-fA-F]{6})$/;
function _isValidHex(color) {
return typeof color === 'string' && HEX_RE.test(color);
}
/** Normalize 3-digit hex to 6-digit: #abc → #aabbcc */
function _normalize(color) {
if (color.length === 4) {
return '#' + color[1] + color[1] + color[2] + color[2] + color[3] + color[3];
}
return color;
}
/**
* Get the assigned color for a channel, or null if unassigned.
* @param {string} channel - Channel name (e.g. "#test")
* @returns {string|null} Hex color or null
*/
function getChannelColor(channel) {
if (!channel) return null;
var colors = _load();
return colors[channel] || null;
}
/**
* Assign a color to a channel.
* @param {string} channel - Channel name
* @param {string} color - Hex color (e.g. "#ef4444")
*/
function setChannelColor(channel, color) {
if (!channel || !color) return;
if (!_isValidHex(color)) return;
var colors = _load();
colors[channel] = _normalize(color);
_save(colors);
}
/**
* Remove the color assignment for a channel.
* @param {string} channel - Channel name
*/
function removeChannelColor(channel) {
if (!channel) return;
var colors = _load();
delete colors[channel];
_save(colors);
}
/**
* Get all channel-color assignments.
* @returns {Object} Map of channel name hex color
*/
function getAllChannelColors() {
return _load();
}
/**
* Compute inline style string for a feed row / table row based on channel color.
* Returns empty string if no channel color is assigned.
* @param {string} typeName - Packet type name (e.g. "GRP_TXT", "CHAN")
* @param {string|null} channel - Channel name from decoded payload
* @returns {string} Inline style string or empty
*/
function getChannelRowStyle(typeName, channel) {
// Only GRP_TXT / CHAN packets get channel coloring
if (typeName !== 'GRP_TXT' && typeName !== 'CHAN') return '';
if (!channel) return '';
var color = getChannelColor(channel);
if (!color) return '';
// 3px left border only — minimal Tufte-style encoding (#674)
return 'border-left:3px solid ' + color + ';';
}
// Export to window for use by live.js and packets.js
window.ChannelColors = {
get: getChannelColor,
set: setChannelColor,
remove: removeChannelColor,
getAll: getAllChannelColors,
getRowStyle: getChannelRowStyle
};
})();
+26 -3
View File
@@ -171,8 +171,11 @@
async function showNodeDetail(name) {
_nodePanelTrigger = document.activeElement;
if (_focusTrapCleanup) { _focusTrapCleanup(); _focusTrapCleanup = null; }
var _capturedHash = selectedHash;
const node = await lookupNode(name);
selectedNode = name;
var _chBase = _capturedHash ? '#/channels/' + encodeURIComponent(_capturedHash) : '#/channels';
history.replaceState(null, '', _chBase + '?node=' + encodeURIComponent(name));
let panel = document.getElementById('chNodePanel');
if (!panel) {
@@ -234,6 +237,8 @@
const panel = document.getElementById('chNodePanel');
if (panel) panel.classList.remove('open');
selectedNode = null;
var _chRestoreUrl = selectedHash ? '#/channels/' + encodeURIComponent(selectedHash) : '#/channels';
history.replaceState(null, '', _chRestoreUrl);
if (_nodePanelTrigger && typeof _nodePanelTrigger.focus === 'function') {
_nodePanelTrigger.focus();
_nodePanelTrigger = null;
@@ -314,6 +319,9 @@
let regionChangeHandler = null;
function init(app, routeParam) {
var _initUrlParams = getHashParams();
var _pendingNode = _initUrlParams.get('node');
app.innerHTML = `<div class="ch-layout">
<div class="ch-sidebar" aria-label="Channel list">
<div class="ch-sidebar-header">
@@ -347,8 +355,9 @@
});
loadObserverRegions();
loadChannels().then(() => {
if (routeParam) selectChannel(routeParam);
loadChannels().then(async function () {
if (routeParam) await selectChannel(routeParam);
if (_pendingNode && _pendingNode.length < 200) await showNodeDetail(_pendingNode);
});
// #89: Sidebar resize handle
@@ -394,6 +403,14 @@
// Event delegation for channel selection (touch-friendly)
document.getElementById('chList').addEventListener('click', (e) => {
// Color dot click — open picker, don't select channel
const dot = e.target.closest('.ch-color-dot');
if (dot && window.ChannelColorPicker) {
e.stopPropagation();
var ch = dot.getAttribute('data-channel');
if (ch) ChannelColorPicker.show(ch, e.clientX, e.clientY);
return;
}
const item = e.target.closest('.ch-item[data-hash]');
if (item) selectChannel(item.dataset.hash);
});
@@ -670,12 +687,18 @@
: `${ch.messageCount} messages`;
const sel = selectedHash === ch.hash ? ' selected' : '';
const abbr = name.startsWith('#') ? name.slice(0, 3) : name.slice(0, 2).toUpperCase();
// Channel color dot for color picker (#674)
const chColor = window.ChannelColors ? window.ChannelColors.get(ch.hash) : null;
const dotStyle = chColor ? ` style="background:${chColor}"` : '';
// Left border for assigned color
const borderStyle = chColor ? ` style="border-left:3px solid ${chColor}"` : '';
return `<button class="ch-item${sel}" data-hash="${ch.hash}" type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}">
return `<button class="ch-item${sel}" data-hash="${ch.hash}"${borderStyle} type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}">
<div class="ch-badge" style="background:${color}" aria-hidden="true">${escapeHtml(abbr)}</div>
<div class="ch-item-body">
<div class="ch-item-top">
<span class="ch-item-name">${escapeHtml(name)}</span>
<span class="ch-color-dot" data-channel="${escapeHtml(ch.hash)}"${dotStyle} title="Change channel color" aria-label="Change color for ${escapeHtml(name)}"></span>
<span class="ch-item-time" data-channel-hash="${ch.hash}">${time}</span>
</div>
<div class="ch-item-preview">${escapeHtml(preview)}</div>
+30 -8
View File
@@ -33,9 +33,10 @@
'meshcore-live-heatmap-opacity'
];
var VALID_SECTIONS = ['branding', 'theme', 'themeDark', 'nodeColors', 'typeColors', 'home', 'timestamps', 'heatmapOpacity', 'liveHeatmapOpacity'];
var VALID_SECTIONS = ['branding', 'theme', 'themeDark', 'nodeColors', 'typeColors', 'home', 'timestamps', 'heatmapOpacity', 'liveHeatmapOpacity', 'distanceUnit'];
var OBJECT_SECTIONS = ['branding', 'theme', 'themeDark', 'nodeColors', 'typeColors', 'home', 'timestamps'];
var SCALAR_SECTIONS = ['heatmapOpacity', 'liveHeatmapOpacity'];
var DISTANCE_UNIT_VALUES = ['km', 'mi', 'auto'];
// CSS variable mapping (theme key → CSS custom property)
var THEME_CSS_MAP = {
@@ -503,6 +504,11 @@
localStorage.setItem('meshcore-live-heatmap-opacity', effectiveConfig.liveHeatmapOpacity);
}
// Distance unit → sync to localStorage for all pages
if (typeof effectiveConfig.distanceUnit === 'string' && DISTANCE_UNIT_VALUES.indexOf(effectiveConfig.distanceUnit) >= 0) {
localStorage.setItem('meshcore-distance-unit', effectiveConfig.distanceUnit);
}
// Nav gradient
if (themeSection.navBg) {
var nav = document.querySelector('.top-nav');
@@ -744,6 +750,10 @@
}
}
}
// Validate distanceUnit
if (key === 'distanceUnit' && DISTANCE_UNIT_VALUES.indexOf(obj[key]) === -1) {
errors.push('Invalid distanceUnit: "' + obj[key] + '" — must be km, mi, or auto');
}
}
return { valid: errors.length === 0, errors: errors };
}
@@ -895,7 +905,7 @@
{ id: 'theme', label: '🎨', title: 'Theme', badge: _tabBadge(isDarkMode() ? 'themeDark' : 'theme') },
{ id: 'nodes', label: '🎯', title: 'Colors', badge: (function () { var n = _countOverrides('nodeColors') + _countOverrides('typeColors'); return n ? ' <span class="cv2-tab-badge">' + n + '</span>' : ''; })() },
{ id: 'home', label: '🏠', title: 'Home', badge: _tabBadge('home') },
{ id: 'display', label: '🖥️', title: 'Display', badge: _tabBadge('timestamps') },
{ id: 'display', label: '🖥️', title: 'Display', badge: (function () { var n = _countOverrides('timestamps') + (_isOverridden(null, 'distanceUnit') ? 1 : 0); return n ? ' <span class="cv2-tab-badge">' + n + '</span>' : ''; })() },
{ id: 'export', label: '📤', title: 'Export' }
];
return '<div class="cust-tabs">' + tabs.map(function (t) {
@@ -1059,6 +1069,7 @@
function _renderDisplay() {
var eff = _getEffective();
var distUnit = typeof eff.distanceUnit === 'string' && DISTANCE_UNIT_VALUES.indexOf(eff.distanceUnit) >= 0 ? eff.distanceUnit : 'auto';
var ts = (eff.timestamps) || {};
var tsMode = ts.defaultMode === 'absolute' ? 'absolute' : 'ago';
var tsTz = ts.timezone === 'utc' ? 'utc' : 'local';
@@ -1086,6 +1097,13 @@
'<option value="locale"' + (tsFmt === 'locale' ? ' selected' : '') + '>Locale (browser)</option></select></div>' +
(canCustom ? '<div class="cust-field" data-ts-abs="custom"' + showAbs + '><label>Custom Format' + _overrideDot('timestamps', 'customFormat') + '</label>' +
'<input type="text" data-cv2-field="timestamps.customFormat" value="' + escAttr(customFmt) + '" placeholder="YYYY-MM-DD HH:mm:ss"></div>' : '') +
'<p class="cust-section-title" style="font-size:14px;margin:16px 0 8px">Distances</p>' +
'<div class="cust-field"><label>Distance Unit' + _overrideDot(null, 'distanceUnit') + '</label>' +
'<select data-cv2-select="distanceUnit" style="width:100%;padding:6px 8px;border:1px solid var(--border);border-radius:6px;background:var(--input-bg);color:var(--text)">' +
'<option value="auto"' + (distUnit === 'auto' ? ' selected' : '') + '>Auto (browser locale)</option>' +
'<option value="km"' + (distUnit === 'km' ? ' selected' : '') + '>Kilometers (km)</option>' +
'<option value="mi"' + (distUnit === 'mi' ? ' selected' : '') + '>Miles (mi)</option>' +
'</select></div>' +
'</div>';
}
@@ -1324,12 +1342,16 @@
container.querySelectorAll('[data-cv2-select]').forEach(function (sel) {
sel.addEventListener('change', function () {
var parts = sel.dataset.cv2Select.split('.');
setOverride(parts[0], parts[1], sel.value);
// Show/hide absolute-only fields
if (parts[1] === 'defaultMode') {
container.querySelectorAll('[data-ts-abs]').forEach(function (el) {
el.style.display = sel.value === 'absolute' ? '' : 'none';
});
if (parts.length === 1) {
setOverride(null, parts[0], sel.value);
} else {
setOverride(parts[0], parts[1], sel.value);
// Show/hide absolute-only fields
if (parts[1] === 'defaultMode') {
container.querySelectorAll('[data-ts-abs]').forEach(function (el) {
el.style.display = sel.value === 'absolute' ? '' : 'none';
});
}
}
window.dispatchEvent(new CustomEvent('timestamp-mode-changed'));
});
+4
View File
@@ -92,12 +92,16 @@
<script src="hop-display.js?v=__BUST__"></script>
<script src="app.js?v=__BUST__"></script>
<script src="home.js?v=__BUST__"></script>
<script src="table-sort.js?v=__BUST__"></script>
<script src="packet-filter.js?v=__BUST__"></script>
<script src="packet-helpers.js?v=__BUST__"></script>
<script src="channel-colors.js?v=__BUST__"></script>
<script src="channel-color-picker.js?v=__BUST__"></script>
<script src="packets.js?v=__BUST__"></script>
<script src="geo-filter-overlay.js?v=__BUST__"></script>
<script src="map.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="table-sort.js?v=__BUST__"></script>
<script src="nodes.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
+99 -10
View File
@@ -19,6 +19,36 @@
position: absolute;
z-index: 1000;
pointer-events: auto;
display: flex;
flex-direction: column;
}
/* ---- Panel header (non-scrolling) ---- */
.panel-header {
display: flex;
justify-content: space-between;
align-items: center;
flex-shrink: 0;
padding: 4px 6px;
}
/* ---- Panel content (scrollable) ---- */
.panel-content {
flex: 1;
overflow-y: auto;
min-height: 0;
}
.live-feed .panel-content {
display: flex;
flex-direction: column;
gap: 1px;
}
.live-legend .panel-content {
display: flex;
flex-direction: column;
gap: 3px;
}
/* ---- Header / Stats ---- */
@@ -106,7 +136,6 @@
right: 12px;
width: 320px;
max-height: calc(100vh - 140px);
overflow-y: auto;
background: color-mix(in srgb, var(--surface-1) 95%, transparent);
backdrop-filter: blur(12px);
border-radius: 10px;
@@ -126,16 +155,12 @@
left: 12px;
width: 360px;
max-height: 340px;
overflow-y: auto;
background: color-mix(in srgb, var(--surface-1) 92%, transparent);
backdrop-filter: blur(12px);
border-radius: 10px;
border: 1px solid var(--border);
box-shadow: 0 4px 24px rgba(0, 0, 0, 0.5);
padding: 6px;
display: flex;
flex-direction: column;
gap: 1px;
}
.live-feed-item {
@@ -198,9 +223,15 @@
box-shadow: 0 4px 24px rgba(0, 0, 0, 0.5);
color: var(--text-muted);
font-size: 11px;
display: flex;
flex-direction: column;
gap: 3px;
transition: opacity 0.3s, transform 0.3s;
}
/* Collapsible legend (#279) */
.live-legend.hidden {
opacity: 0;
transform: translateX(100%);
pointer-events: none;
visibility: hidden;
}
.legend-title {
@@ -272,6 +303,16 @@
background: rgba(59, 130, 246, 0.2) !important;
}
/* ---- Medium breakpoint (#279) ---- */
@media (max-width: 768px) {
.live-feed { width: 280px; max-height: 200px; }
.live-node-detail { width: 260px; }
.live-legend { font-size: 10px; padding: 8px 10px; }
.live-header { gap: 8px; padding: 6px 12px; }
.live-stat-pill { font-size: 11px; padding: 2px 8px; }
.live-toggles { font-size: 10px; gap: 6px; }
}
/* ---- Responsive ---- */
@media (max-width: 640px) {
.live-feed { display: none !important; }
@@ -702,9 +743,8 @@
border: 0;
}
/* Legend toggle button for mobile (#60) */
/* Legend toggle button — visible at all sizes (#60, #279) */
.legend-toggle-btn {
display: none;
position: absolute;
bottom: 82px;
right: 12px;
@@ -760,3 +800,52 @@
}
.nav-pin-btn:hover { opacity: 0.8; }
.nav-pin-btn.pinned { opacity: 1; filter: drop-shadow(0 0 4px rgba(59,130,246,0.5)); }
/* ========== Panel Corner Positioning (#608 M0) ========== */
/* Corner positions — applied via data-position attribute on .live-overlay panels */
.live-overlay[data-position="tl"] { top: 64px; left: 12px; bottom: auto; right: auto; }
.live-overlay[data-position="tr"] { top: 64px; right: 12px; bottom: auto; left: auto; }
.live-overlay[data-position="bl"] { bottom: 12px; left: 12px; top: auto; right: auto; }
.live-overlay[data-position="br"] { bottom: 12px; right: 12px; top: auto; left: auto; }
/* Override hide animations for positioned panels — slide toward nearest edge */
.live-overlay[data-position="tl"].hidden,
.live-overlay[data-position="bl"].hidden { transform: translateX(-100%); }
.live-overlay[data-position="tr"].hidden,
.live-overlay[data-position="br"].hidden { transform: translateX(100%); }
.live-overlay[data-position].hidden { opacity: 0; pointer-events: none; visibility: hidden; }
/* Corner toggle button */
.panel-corner-btn {
width: 28px;
height: 28px;
padding: 0;
border: none;
background: transparent;
color: var(--text-muted);
cursor: pointer;
opacity: 0.6;
transition: opacity 0.15s, background 0.15s;
font-size: 14px;
line-height: 28px;
text-align: center;
flex-shrink: 0;
border-radius: 4px;
}
.panel-corner-btn:hover { opacity: 1; background: color-mix(in srgb, var(--text) 12%, transparent); }
.panel-corner-btn:focus-visible {
opacity: 1;
outline: 2px solid var(--accent);
outline-offset: 2px;
border-radius: 3px;
}
/* On mobile, corner toggle is not useful (panels are hidden or bottom-sheet) */
@media (max-width: 640px) {
.panel-corner-btn { display: none !important; }
.live-overlay[data-position] {
top: unset !important; bottom: unset !important;
left: unset !important; right: unset !important;
}
}
+231 -24
View File
@@ -58,6 +58,92 @@
REQUEST: '❓', RESPONSE: '📨', TRACE: '🔍', PATH: '🛤️'
};
/* ---- Panel Corner Positioning (#608 M0) ---- */
var PANEL_DEFAULTS = { liveFeed: 'bl', liveLegend: 'br', liveNodeDetail: 'tr' };
var CORNER_CYCLE = ['tl', 'tr', 'br', 'bl'];
var CORNER_ARROWS = { tl: '↘', tr: '↙', bl: '↗', br: '↖' };
var CORNER_LABELS = { tl: 'top-left', tr: 'top-right', bl: 'bottom-left', br: 'bottom-right' };
var PANEL_NAMES = { liveFeed: 'Feed', liveLegend: 'Legend', liveNodeDetail: 'Node detail' };
function getPanelPositions() {
var pos = {};
for (var id in PANEL_DEFAULTS) {
try { pos[id] = localStorage.getItem('panel-corner-' + id) || PANEL_DEFAULTS[id]; }
catch (_) { pos[id] = PANEL_DEFAULTS[id]; }
}
return pos;
}
function nextAvailableCorner(panelId, desired, allPositions) {
var idx = CORNER_CYCLE.indexOf(desired);
for (var i = 0; i < 4; i++) {
var candidate = CORNER_CYCLE[(idx + i) % 4];
var occupied = false;
for (var otherId in allPositions) {
if (otherId !== panelId && allPositions[otherId] === candidate) { occupied = true; break; }
}
if (!occupied) return candidate;
}
return desired; // all occupied (impossible with 3 panels, 4 corners)
}
function applyPanelPosition(id, corner) {
var el = document.getElementById(id);
if (!el) return;
el.setAttribute('data-position', corner);
var btn = el.querySelector('.panel-corner-btn');
if (btn) {
btn.textContent = CORNER_ARROWS[corner];
btn.setAttribute('aria-label',
'Move ' + (PANEL_NAMES[id] || 'panel') + ' to next corner (currently ' + CORNER_LABELS[corner] + ')');
}
}
function initPanelPositions() {
var positions = getPanelPositions();
for (var id in positions) {
applyPanelPosition(id, positions[id]);
}
// Wire up click handlers on corner buttons
var btns = document.querySelectorAll('.panel-corner-btn[data-panel]');
for (var i = 0; i < btns.length; i++) {
btns[i].addEventListener('click', function(e) {
e.stopPropagation();
var panelId = this.getAttribute('data-panel');
onCornerClick(panelId);
});
}
}
function onCornerClick(panelId) {
var positions = getPanelPositions();
var current = positions[panelId];
var nextIdx = (CORNER_CYCLE.indexOf(current) + 1) % 4;
var next = nextAvailableCorner(panelId, CORNER_CYCLE[nextIdx], positions);
try { localStorage.setItem('panel-corner-' + panelId, next); } catch (_) { /* quota */ }
applyPanelPosition(panelId, next);
// Announce for screen readers
var announce = document.getElementById('panelPositionAnnounce');
if (announce) announce.textContent = (PANEL_NAMES[panelId] || 'Panel') + ' moved to ' + CORNER_LABELS[next];
}
function resetPanelPositions() {
for (var id in PANEL_DEFAULTS) {
try { localStorage.removeItem('panel-corner-' + id); } catch (_) { /* ignore */ }
applyPanelPosition(id, PANEL_DEFAULTS[id]);
}
}
// Export for testing
if (typeof window !== 'undefined') {
window._panelCorner = {
PANEL_DEFAULTS: PANEL_DEFAULTS, CORNER_CYCLE: CORNER_CYCLE,
getPanelPositions: getPanelPositions, nextAvailableCorner: nextAvailableCorner,
applyPanelPosition: applyPanelPosition, onCornerClick: onCornerClick,
resetPanelPositions: resetPanelPositions
};
}
function formatLiveTimestampHtml(isoLike) {
if (typeof formatTimestampWithTooltip !== 'function' || typeof getTimestampMode !== 'function') {
return escapeHtml(typeof timeAgo === 'function' ? timeAgo(isoLike) : '—');
@@ -755,15 +841,26 @@
</div>
</div>
<div class="live-overlay live-feed" id="liveFeed">
<button class="feed-hide-btn" id="feedHideBtn" title="Hide feed"></button>
<div class="panel-header">
<button class="panel-corner-btn" data-panel="liveFeed" title="Move panel to next corner" aria-label="Move panel to next corner"></button>
<button class="feed-hide-btn" id="feedHideBtn" title="Hide feed"></button>
</div>
<div class="panel-content" aria-live="polite" aria-relevant="additions" role="log"></div>
</div>
<button class="feed-show-btn hidden" id="feedShowBtn" title="Show feed">📋</button>
<div class="live-overlay live-node-detail hidden" id="liveNodeDetail">
<button class="feed-hide-btn" id="nodeDetailClose" title="Close"></button>
<div id="nodeDetailContent"></div>
<div class="panel-header">
<button class="panel-corner-btn" data-panel="liveNodeDetail" title="Move panel to next corner" aria-label="Move panel to next corner"></button>
<button class="feed-hide-btn" id="nodeDetailClose" title="Close"></button>
</div>
<div class="panel-content" id="nodeDetailContent"></div>
</div>
<button class="legend-toggle-btn hidden" id="legendToggleBtn" aria-label="Show legend" title="Show legend">🎨</button>
<button class="legend-toggle-btn" id="legendToggleBtn" aria-label="Show legend" title="Show legend">🎨</button>
<div class="live-overlay live-legend" id="liveLegend" role="region" aria-label="Map legend">
<div class="panel-header">
<button class="panel-corner-btn" data-panel="liveLegend" title="Move panel to next corner" aria-label="Move panel to next corner"></button>
</div>
<div class="panel-content">
<h3 class="legend-title">PACKET TYPES</h3>
<ul class="legend-list">
<li><span class="live-dot" style="background:${TYPE_COLORS.ADVERT}" aria-hidden="true"></span> Advert Node advertisement</li>
@@ -774,9 +871,11 @@
</ul>
<h3 class="legend-title" style="margin-top:8px">NODE ROLES</h3>
<ul class="legend-list" id="roleLegendList"></ul>
</div>
</div>
<!-- VCR Bar -->
<div class="sr-only" id="panelPositionAnnounce" aria-live="polite"></div>
<div class="vcr-bar" id="vcrBar">
<div class="vcr-controls">
<button id="vcrRewindBtn" class="vcr-btn" title="Rewind" aria-label="Rewind"></button>
@@ -1043,14 +1142,25 @@
const legendEl = document.getElementById('liveLegend');
const legendToggleBtn = document.getElementById('legendToggleBtn');
if (legendToggleBtn && legendEl) {
// Restore legend collapsed state from localStorage (#279)
try {
if (localStorage.getItem('live-legend-hidden') === 'true') {
legendEl.classList.add('hidden');
legendToggleBtn.setAttribute('aria-label', 'Show legend');
legendToggleBtn.textContent = '🎨';
}
} catch (_) { /* private browsing / storage disabled */ }
legendToggleBtn.addEventListener('click', () => {
const isVisible = legendEl.classList.toggle('legend-mobile-visible');
legendToggleBtn.setAttribute('aria-label', isVisible ? 'Hide legend' : 'Show legend');
legendToggleBtn.textContent = isVisible ? '' : '🎨';
const nowHidden = legendEl.classList.toggle('hidden');
legendToggleBtn.setAttribute('aria-label', nowHidden ? 'Show legend' : 'Hide legend');
legendToggleBtn.textContent = nowHidden ? '🎨' : '';
try { localStorage.setItem('live-legend-hidden', String(nowHidden)); } catch (_) { /* ignore */ }
});
}
// Populate role legend from shared roles.js
// Initialize panel corner positions (#608 M0)
initPanelPositions();
const roleLegendList = document.getElementById('roleLegendList');
if (roleLegendList) {
for (const role of (window.ROLE_SORT || ['repeater', 'companion', 'room', 'sensor', 'observer'])) {
@@ -1314,7 +1424,7 @@
let html = `
<div style="padding:16px;">
<div style="display:flex;align-items:center;gap:8px;margin-bottom:12px;">
<span class="${statusDot}" style="font-size:18px"></span>
<span class="${statusDot}" style="font-size:18px" aria-hidden="true"></span>
<h3 style="margin:0;font-size:16px;font-weight:700;">${escapeHtml(n.name || 'Unknown')}</h3>
</div>
<div style="margin-bottom:12px;">
@@ -1493,7 +1603,9 @@
function rebuildFeedList() {
const feed = document.getElementById('liveFeed');
if (!feed) return;
feed.querySelectorAll('.live-feed-item').forEach(el => el.remove());
const feedContent = feed.querySelector('.panel-content');
if (!feedContent) return;
feedContent.querySelectorAll('.live-feed-item').forEach(el => el.remove());
feedDedup.clear();
// Aggregate VCR buffer by hash, then create one feed item per unique hash
@@ -1541,6 +1653,10 @@
const hopStr = longestHops.length ? `<span class="feed-hops">${longestHops.length}⇢</span>` : '';
const obsBadge = group.count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${group.count}</span>` : '';
var _ccPayload = (pkt.decoded || {}).payload || {};
var _ccChan1 = (typeName === 'GRP_TXT' || typeName === 'CHAN') ? (_ccPayload.channel || null) : null;
var dotHtml1 = _ccChan1 ? _feedColorDot(_ccChan1) : '';
const item = document.createElement('div');
item.className = 'live-feed-item';
item.setAttribute('tabindex', '0');
@@ -1550,12 +1666,13 @@
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
${dotHtml1}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(group.latestTs || Date.now())}</span>
`;
if (_ccChan1) item._ccChannel = _ccChan1; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
feed.appendChild(item);
feedContent.appendChild(item);
// Register in dedup map so replay and live updates work
if (group.hash) {
@@ -1889,11 +2006,66 @@
}
}
firstPathDone = true;
animatePath(allPaths[ai].hopPositions, typeName, color, allPaths[ai].raw, onHop);
// For TRACE packets, split at hopsCompleted: solid for completed, dashed for remaining
var hopsCompleted = decoded.path && decoded.path.hopsCompleted;
if (typeName === 'TRACE' && hopsCompleted != null && hopsCompleted < allPaths[ai].hopPositions.length) {
var completedPositions = allPaths[ai].hopPositions.slice(0, hopsCompleted + 1);
var remainingPositions = allPaths[ai].hopPositions.slice(hopsCompleted);
if (completedPositions.length >= 2) {
animatePath(completedPositions, typeName, color, allPaths[ai].raw, onHop);
} else if (completedPositions.length === 1) {
pulseNode(completedPositions[0].key, completedPositions[0].pos, typeName);
}
if (remainingPositions.length >= 2) {
drawDashedPath(remainingPositions, color);
}
} else {
animatePath(allPaths[ai].hopPositions, typeName, color, allPaths[ai].raw, onHop);
}
}
}
// Draw a static dashed/ghosted line for unreached TRACE hops
function drawDashedPath(hopPositions, color) {
var GHOST_TIMEOUT_MS = 10000;
var ghostColor = getComputedStyle(document.documentElement).getPropertyValue('--trace-ghost-color').trim() || '#94a3b8';
if (!pathsLayer) return;
for (var i = 0; i < hopPositions.length - 1; i++) {
var from = hopPositions[i].pos;
var to = hopPositions[i + 1].pos;
var line = L.polyline([from, to], {
color: color, weight: 2, opacity: 0.25, dashArray: '6, 8'
}).addTo(pathsLayer);
// Pulse the unreached hop nodes as ghost markers
if (i > 0) {
var hp = hopPositions[i];
if (!nodeMarkers[hp.key]) {
var ghost = L.circleMarker(hp.pos, {
radius: 3, fillColor: ghostColor, fillOpacity: 0.2, color: color, weight: 1, opacity: 0.3
}).addTo(pathsLayer);
setTimeout((function(g) { return function() { if (pathsLayer.hasLayer(g)) pathsLayer.removeLayer(g); }; })(ghost), GHOST_TIMEOUT_MS);
}
}
// Remove dashed line after timeout
setTimeout((function(l) { return function() { if (pathsLayer.hasLayer(l)) pathsLayer.removeLayer(l); }; })(line), GHOST_TIMEOUT_MS);
}
// Ghost marker for the final unreached hop
var last = hopPositions[hopPositions.length - 1];
if (!nodeMarkers[last.key]) {
var ghostEnd = L.circleMarker(last.pos, {
radius: 4, fillColor: ghostColor, fillOpacity: 0.25, color: color, weight: 1, opacity: 0.35
}).addTo(pathsLayer);
setTimeout(function() { if (pathsLayer.hasLayer(ghostEnd)) pathsLayer.removeLayer(ghostEnd); }, GHOST_TIMEOUT_MS);
}
}
function resolveHopPositions(hops, payload, resolvedPath) {
// Hoist sender GPS guard once — reject (0,0) as "no GPS"
const hasValidGps = payload.lat != null && payload.lon != null
&& !(payload.lat === 0 && payload.lon === 0);
const senderLat = hasValidGps ? payload.lat : null;
const senderLon = hasValidGps ? payload.lon : null;
// Prefer server-side resolved_path when available
var resolvedMap;
if (resolvedPath && resolvedPath.length === hops.length && window.HopResolver && HopResolver.ready()) {
@@ -1901,19 +2073,14 @@
// Fill in any null entries from client-side fallback, preserving sender GPS context
var nullHops = hops.filter(function(h, i) { return !resolvedPath[i] && !resolvedMap[h]; });
if (nullHops.length) {
const originLat = payload.lat != null && !(payload.lat === 0 && payload.lon === 0) ? payload.lat : null;
const originLon = payload.lon != null && !(payload.lon === 0 && payload.lon === 0) ? payload.lon : null;
var fallback = HopResolver.resolve(nullHops, originLat, originLon, null, null, null);
var fallback = HopResolver.resolve(nullHops, senderLat, senderLon, null, null, null);
for (var k in fallback) resolvedMap[k] = fallback[k];
}
} else {
// Delegate to shared HopResolver (from hop-resolver.js) instead of reimplementing
const originLat = payload.lat != null && !(payload.lat === 0 && payload.lon === 0) ? payload.lat : null;
const originLon = payload.lon != null && !(payload.lon === 0 && payload.lon === 0) ? payload.lon : null;
// Use HopResolver if available and initialized, otherwise fall back to simple lookup
resolvedMap = (window.HopResolver && HopResolver.ready())
? HopResolver.resolve(hops, originLat, originLon, null, null, null)
? HopResolver.resolve(hops, senderLat, senderLon, null, null, null)
: {};
}
@@ -1932,7 +2099,7 @@
});
// Add sender position as anchor if available
if (payload.pubKey && originLat != null) {
if (payload.pubKey && senderLat != null) {
const existing = raw.find(p => p.key === payload.pubKey);
if (!existing) {
raw.unshift({ key: payload.pubKey, pos: [payload.lat, payload.lon], name: payload.name || payload.pubKey.slice(0, 8), known: true });
@@ -2486,23 +2653,52 @@
if (heatLayer) { map.removeLayer(heatLayer); heatLayer = null; }
}
/** Extract channel row style from a packet (shared by feed item builders). */
function _getChannelStyle(pkt) {
if (!window.ChannelColors) return '';
var d = pkt.decoded || {};
var p = d.payload || {};
var typeName = p.type || (d.header || {}).payloadTypeName || '';
var ch = p.channel || null;
return window.ChannelColors.getRowStyle(typeName, ch);
}
/** Build a clickable 12×12 color dot for a channel feed item (#674). */
function _feedColorDot(channel) {
if (!channel || !window.ChannelColors) return '';
var c = window.ChannelColors.get(channel);
var bg = c || 'transparent';
var border = c ? c : 'var(--border-color, #555)';
var style = c
? 'background:' + bg + ';border:1px solid ' + border
: 'background:transparent;border:1px dashed ' + border;
return '<span class="feed-color-dot" data-channel="' + escapeHtml(channel) + '" style="display:inline-block;width:12px;height:12px;border-radius:50%;' + style + ';cursor:pointer;vertical-align:middle;margin-left:4px;flex-shrink:0" title="Set color for ' + escapeHtml(channel) + '"></span>';
}
function addFeedItemDOM(icon, typeName, payload, hops, color, pkt, feed) {
const text = payload.text || payload.name || '';
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = pkt.observation_count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${pkt.observation_count}</span>` : '';
var _ccPayload2 = (pkt.decoded || {}).payload || {};
var _ccChan = (typeName === 'GRP_TXT' || typeName === 'CHAN') ? (_ccPayload2.channel || null) : null;
var dotHtml = _ccChan ? _feedColorDot(_ccChan) : '';
const item = document.createElement('div');
item.className = 'live-feed-item';
item.setAttribute('tabindex', '0');
item.setAttribute('role', 'button');
item.style.cursor = 'pointer';
// Channel color highlighting for GRP_TXT packets (#271)
var _cs = _getChannelStyle(pkt);
if (_cs) item.style.cssText += _cs;
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
${dotHtml}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
`;
if (_ccChan) item._ccChannel = _ccChan; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
feed.appendChild(item);
}
@@ -2515,7 +2711,8 @@
const DEDUP_WINDOW = 30000;
function addFeedItem(icon, typeName, payload, hops, color, pkt) {
const feed = document.getElementById('liveFeed');
const feedPanel = document.getElementById('liveFeed');
const feed = feedPanel ? feedPanel.querySelector('.panel-content') : null;
if (!feed) return;
if (showOnlyFavorites && !packetInvolvesFavorite(pkt)) return;
@@ -2557,6 +2754,9 @@
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = incomingObs > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${incomingObs}</span>` : '';
var _ccPayload3 = (pkt.decoded || {}).payload || {};
var _ccChan3 = (typeName === 'GRP_TXT' || typeName === 'CHAN') ? (_ccPayload3.channel || null) : null;
var dotHtml3 = _ccChan3 ? _feedColorDot(_ccChan3) : '';
const item = document.createElement('div');
item.className = 'live-feed-item live-feed-enter';
@@ -2564,13 +2764,17 @@
item.setAttribute('role', 'button');
if (hash) item.setAttribute('data-hash', hash);
item.style.cursor = 'pointer';
// Channel color highlighting for GRP_TXT packets (#271)
var _chanStyle = _getChannelStyle(pkt);
if (_chanStyle) item.style.cssText += _chanStyle;
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
${dotHtml3}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
`;
if (_ccChan3) item._ccChannel = _ccChan3; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
feed.prepend(item);
requestAnimationFrame(() => requestAnimationFrame(() => item.classList.remove('live-feed-enter')));
@@ -2690,7 +2894,10 @@
if (activeNodeDetailKey) showNodeDetail(activeNodeDetailKey);
};
window.addEventListener('theme-refresh', _themeRefreshHandler);
return init(app, routeParam);
var result = init(app, routeParam);
// Install channel color picker (M2, #271)
if (window.ChannelColorPicker) window.ChannelColorPicker.installLiveFeed();
return result;
},
destroy: function() {
if (_themeRefreshHandler) { window.removeEventListener('theme-refresh', _themeRefreshHandler); _themeRefreshHandler = null; }
+153 -55
View File
@@ -20,26 +20,24 @@
let activeTab = 'all';
let search = '';
// Sort state: column + direction, persisted to localStorage
let sortState = (function () {
try {
const saved = JSON.parse(localStorage.getItem('meshcore-nodes-sort'));
if (saved && saved.column && saved.direction) return saved;
} catch {}
return { column: 'last_seen', direction: 'desc' };
})();
// Managed by TableSort utility (public/table-sort.js) when DOM is available,
// falls back to simple object for unit testing
var _nodesTableSortCtrl = null;
// TODO(M5): remove fallback when tests use DOM sandbox
var _fallbackSortState = null; // used when TableSort controller not initialized (tests)
function toggleSort(column) {
if (sortState.column === column) {
sortState.direction = sortState.direction === 'asc' ? 'desc' : 'asc';
} else {
// Default direction per column type
const descDefault = ['last_seen', 'advert_count'];
sortState = { column, direction: descDefault.includes(column) ? 'desc' : 'asc' };
}
localStorage.setItem('meshcore-nodes-sort', JSON.stringify(sortState));
function _getSortState() {
if (_nodesTableSortCtrl) return _nodesTableSortCtrl.getState();
if (_fallbackSortState) return _fallbackSortState;
try {
var saved = JSON.parse(localStorage.getItem('meshcore-nodes-sort'));
if (saved && saved.column && saved.direction) return saved;
} catch (e) { /* ignore */ }
return { column: 'last_seen', direction: 'desc' };
}
function sortNodes(arr) {
var sortState = _getSortState();
const col = sortState.column;
const dir = sortState.direction === 'asc' ? 1 : -1;
return arr.sort(function (a, b) {
@@ -66,11 +64,6 @@
return 0;
});
}
function sortArrow(col) {
if (sortState.column !== col) return '';
return '<span class="sort-arrow">' + (sortState.direction === 'asc' ? '▲' : '▼') + '</span>';
}
let lastHeard = localStorage.getItem('meshcore-nodes-last-heard') || '';
let statusFilter = localStorage.getItem('meshcore-nodes-status-filter') || 'all';
let wsHandler = null;
@@ -85,6 +78,18 @@
{ key: 'sensor', label: 'Sensors' },
];
function buildNodesQuery(tab, searchStr) {
var parts = [];
if (tab && tab !== 'all') parts.push('tab=' + encodeURIComponent(tab));
if (searchStr) parts.push('search=' + encodeURIComponent(searchStr));
return parts.length ? '?' + parts.join('&') : '';
}
window.buildNodesQuery = buildNodesQuery;
function updateNodesUrl() {
history.replaceState(null, '', '#/nodes' + buildNodesQuery(activeTab, search));
}
function renderNodeTimestampHtml(isoString) {
if (typeof formatTimestampWithTooltip !== 'function' || typeof getTimestampMode !== 'function') {
return escapeHtml(typeof timeAgo === 'function' ? timeAgo(isoString) : '—');
@@ -189,7 +194,7 @@
function renderNeighborRows(neighbors, limit) {
var sorted = neighbors.slice().sort(function(a, b) {
return (b.score || b.affinity || 0) - (a.score || a.affinity || 0);
return (b.count || 0) - (a.count || 0);
});
var items = limit ? sorted.slice(0, limit) : sorted;
return items.map(function(nb) {
@@ -204,15 +209,21 @@
: '<span class="text-muted">—</span>';
var scoreTitle = 'Observations: ' + nb.count;
if (nb.avg_snr != null) scoreTitle += ' · Avg SNR: ' + Number(nb.avg_snr).toFixed(1) + ' dB';
var distanceCell = nb.distance_km != null
? formatDistance(Number(nb.distance_km))
: '<span class="text-muted">—</span>';
var showOnMap = nb.pubkey
? ' <button class="btn-link neighbor-show-map" data-pubkey="' + escapeHtml(nb.pubkey) + '" style="font-size:11px;padding:1px 6px;white-space:nowrap">📍 Map</button>'
: '';
var lastSeenVal = nb.last_seen ? new Date(nb.last_seen).getTime() : 0;
var distanceVal = nb.distance_km != null ? Number(nb.distance_km) : '';
return '<tr>' +
'<td style="font-weight:600">' + nameHtml + '</td>' +
'<td>' + roleBadge + '</td>' +
'<td title="' + escapeHtml(scoreTitle) + '">' + Number(nb.score).toFixed(2) + '</td>' +
'<td>' + nb.count + '</td>' +
'<td>' + renderNodeTimestampHtml(nb.last_seen) + '</td>' +
'<td data-value="' + escapeHtml(name.toLowerCase()) + '" style="font-weight:600">' + nameHtml + '</td>' +
'<td data-value="' + escapeHtml(role.toLowerCase()) + '">' + roleBadge + '</td>' +
'<td data-value="' + Number(nb.score || 0) + '" title="' + escapeHtml(scoreTitle) + '">' + Number(nb.score).toFixed(2) + '</td>' +
'<td data-value="' + (nb.count || 0) + '">' + nb.count + '</td>' +
'<td data-value="' + lastSeenVal + '">' + renderNodeTimestampHtml(nb.last_seen) + '</td>' +
'<td data-value="' + distanceVal + '">' + distanceCell + '</td>' +
'<td><span title="' + conf.label + '">' + conf.icon + '</span></td>' +
'<td style="text-align:right">' + showOnMap + '</td>' +
'</tr>';
@@ -220,8 +231,16 @@
}
function renderNeighborTable(neighbors, limit) {
return '<table class="data-table" style="font-size:12px">' +
'<thead><tr><th>Neighbor</th><th>Role</th><th>Score</th><th>Obs</th><th>Last Seen</th><th>Conf</th><th></th></tr></thead>' +
return '<table class="data-table neighbor-sort-table" style="font-size:12px">' +
'<thead><tr>' +
'<th scope="col" data-sort-key="name">Neighbor</th>' +
'<th scope="col" data-sort-key="role">Role</th>' +
'<th scope="col" data-sort-key="score" data-type="numeric" data-sort-default="desc">Score</th>' +
'<th scope="col" data-sort-key="count" data-type="numeric" data-sort-default="desc">Obs</th>' +
'<th scope="col" data-sort-key="last_seen" data-type="numeric" data-sort-default="desc">Last Seen</th>' +
'<th scope="col" data-sort-key="distance" data-type="numeric">Distance</th>' +
'<th scope="col">Conf</th><th scope="col"></th>' +
'</tr></thead>' +
'<tbody>' + renderNeighborRows(neighbors, limit) + '</tbody></table>';
}
@@ -272,6 +291,15 @@
}
el.innerHTML = html;
// Initialize TableSort on neighbor table
var neighborTable = el.querySelector('.neighbor-sort-table');
if (neighborTable && window.TableSort) {
TableSort.init(neighborTable, {
defaultColumn: 'count',
defaultDirection: 'desc'
});
}
// Wire up "Show on Map" buttons via event delegation
el.addEventListener('click', function(e) {
var btn = e.target.closest('.neighbor-show-map');
@@ -313,6 +341,15 @@
return;
}
// Reset list-view state to defaults, then override from URL params
activeTab = 'all';
search = '';
const _listUrlParams = getHashParams();
const _urlTab = _listUrlParams.get('tab');
const _urlSearch = _listUrlParams.get('search');
if (_urlTab && TABS.some(function(t) { return t.key === _urlTab; })) activeTab = _urlTab;
if (_urlSearch) search = _urlSearch;
app.innerHTML = `<div class="nodes-page">
<div class="nodes-topbar">
<input type="text" class="nodes-search" id="nodeSearch" placeholder="Search nodes by name…" aria-label="Search nodes by name">
@@ -320,7 +357,7 @@
</div>
<div id="nodesRegionFilter" class="region-filter-container"></div>
<div class="split-layout">
<div class="panel-left" id="nodesLeft"></div>
<div class="panel-left" id="nodesLeft" aria-live="polite" aria-relevant="additions removals"></div>
<div class="panel-right empty" id="nodesRight"><span>Select a node to view details</span></div>
</div>
</div>`;
@@ -328,8 +365,14 @@
RegionFilter.init(document.getElementById('nodesRegionFilter'));
regionChangeHandler = RegionFilter.onChange(function () { _allNodes = null; loadNodes(); });
if (search) {
var _si = document.getElementById('nodeSearch');
if (_si) _si.value = search;
}
document.getElementById('nodeSearch').addEventListener('input', debounce(e => {
search = e.target.value;
updateNodesUrl();
loadNodes();
}, 250));
@@ -453,15 +496,21 @@
${observers.length ? `<div class="node-full-card" id="node-observers">
${(() => { const regions = [...new Set(observers.map(o => o.iata).filter(Boolean))]; return regions.length ? `<div style="margin-bottom:8px"><strong>Regions:</strong> ${regions.map(r => '<span class="badge" style="margin:0 2px">' + escapeHtml(r) + '</span>').join(' ')}</div>` : ''; })()}
<h4>Heard By (${observers.length} observer${observers.length > 1 ? 's' : ''})</h4>
<table class="data-table" style="font-size:12px">
<thead><tr><th scope="col">Observer</th><th scope="col">Region</th><th scope="col">Packets</th><th scope="col">Avg SNR</th><th scope="col">Avg RSSI</th></tr></thead>
<table class="data-table observer-sort-table" style="font-size:12px">
<thead><tr>
<th scope="col" data-sort-key="observer">Observer</th>
<th scope="col" data-sort-key="region">Region</th>
<th scope="col" data-sort-key="packets" data-type="numeric" data-sort-default="desc">Packets</th>
<th scope="col" data-sort-key="snr" data-type="numeric" data-sort-default="desc">Avg SNR</th>
<th scope="col" data-sort-key="rssi" data-type="numeric" data-sort-default="desc">Avg RSSI</th>
</tr></thead>
<tbody>
${observers.map(o => `<tr>
<td style="font-weight:600">${escapeHtml(o.observer_name || o.observer_id)}</td>
<td>${o.iata ? escapeHtml(o.iata) : '—'}</td>
<td>${o.packetCount}</td>
<td>${o.avgSnr != null ? Number(o.avgSnr).toFixed(1) + ' dB' : '—'}</td>
<td>${o.avgRssi != null ? Number(o.avgRssi).toFixed(0) + ' dBm' : '—'}</td>
<td data-value="${escapeHtml((o.observer_name || o.observer_id || '').toLowerCase())}" style="font-weight:600">${escapeHtml(o.observer_name || o.observer_id)}</td>
<td data-value="${escapeHtml((o.iata || '').toLowerCase())}">${o.iata ? escapeHtml(o.iata) : '—'}</td>
<td data-value="${o.packetCount || 0}">${o.packetCount}</td>
<td data-value="${o.avgSnr != null ? Number(o.avgSnr) : ''}">${o.avgSnr != null ? Number(o.avgSnr).toFixed(1) + ' dB' : '—'}</td>
<td data-value="${o.avgRssi != null ? Number(o.avgRssi) : ''}">${o.avgRssi != null ? Number(o.avgRssi).toFixed(0) + ' dBm' : '—'}</td>
</tr>`).join('')}
</tbody>
</table>
@@ -499,10 +548,12 @@
let hashSizeBadge = '';
if (n.hash_size_inconsistent && p.payload_type === 4 && p.raw_hex) {
const pb = parseInt(p.raw_hex.slice(2, 4), 16);
const hs = ((pb >> 6) & 0x3) + 1;
const hsColor = hs >= 3 ? '#16a34a' : hs === 2 ? '#86efac' : '#f97316';
const hsFg = hs === 2 ? '#064e3b' : '#fff';
hashSizeBadge = ` <span class="badge" style="background:${hsColor};color:${hsFg};font-size:9px;font-family:var(--mono)">${hs}B</span>`;
if ((pb & 0x3F) !== 0) {
const hs = ((pb >> 6) & 0x3) + 1;
const hsColor = hs >= 3 ? '#16a34a' : hs === 2 ? '#86efac' : '#f97316';
const hsFg = hs === 2 ? '#064e3b' : '#fff';
hashSizeBadge = ` <span class="badge" style="background:${hsColor};color:${hsFg};font-size:9px;font-family:var(--mono)">${hs}B</span>`;
}
}
return `<div class="node-activity-item">
<span class="node-activity-time">${renderNodeTimestampHtml(p.timestamp)}</span>
@@ -559,6 +610,15 @@
} catch {}
}
// Initialize TableSort on observer table (full detail page)
var observerTable = document.querySelector('#node-observers .observer-sort-table');
if (observerTable && window.TableSort) {
TableSort.init(observerTable, {
defaultColumn: 'packets',
defaultDirection: 'desc'
});
}
// Fetch neighbors for this node (full-screen view)
fetchAndRenderNeighbors(n.public_key, 'fullNeighborsContent', {
headerSelector: '#fullNeighborsHeader'
@@ -855,11 +915,11 @@
</div>
<table class="data-table" id="nodesTable">
<thead><tr>
<th scope="col" class="sortable${sortState.column==='name'?' sort-active':''}" data-sort="name">Name${sortArrow('name')}</th>
<th scope="col" class="col-pubkey sortable${sortState.column==='public_key'?' sort-active':''}" data-sort="public_key">Public Key${sortArrow('public_key')}</th>
<th scope="col" class="sortable${sortState.column==='role'?' sort-active':''}" data-sort="role">Role${sortArrow('role')}</th>
<th scope="col" class="sortable${sortState.column==='last_seen'?' sort-active':''}" data-sort="last_seen">Last Seen${sortArrow('last_seen')}</th>
<th scope="col" class="sortable${sortState.column==='advert_count'?' sort-active':''}" data-sort="advert_count">Adverts${sortArrow('advert_count')}</th>
<th scope="col" data-sort-key="name">Name</th>
<th scope="col" class="col-pubkey" data-sort-key="public_key">Public Key</th>
<th scope="col" data-sort-key="role">Role</th>
<th scope="col" data-sort-key="last_seen" data-sort-default="desc">Last Seen</th>
<th scope="col" data-sort-key="advert_count" data-sort-default="desc">Adverts</th>
</tr></thead>
<tbody id="nodesBody"></tbody>
</table>`;
@@ -868,7 +928,7 @@
const nodeTabs = document.getElementById('nodeTabs');
initTabBar(nodeTabs);
el.querySelectorAll('.node-tab').forEach(btn => {
btn.addEventListener('click', () => { activeTab = btn.dataset.tab; loadNodes(); });
btn.addEventListener('click', () => { activeTab = btn.dataset.tab; updateNodesUrl(); loadNodes(); });
});
// Filter changes
@@ -884,10 +944,18 @@
});
});
// Sortable column headers
el.querySelectorAll('th.sortable').forEach(th => {
th.addEventListener('click', () => { toggleSort(th.dataset.sort); renderLeft(); });
});
// Initialize TableSort on nodes table (handles header clicks, indicators, persistence)
// We use onSort callback to re-render rows (sorting is done at JS-array level in renderRows
// because of claimed/favorites pinning logic that TableSort can't handle)
var nodesTableEl = document.getElementById('nodesTable');
if (nodesTableEl && window.TableSort) {
_nodesTableSortCtrl = TableSort.init(nodesTableEl, {
defaultColumn: 'last_seen',
defaultDirection: 'desc',
storageKey: 'meshcore-nodes-sort',
onSort: function () { renderRows(); }
});
}
// Delegated click/keyboard handler for table rows
const tbody = document.getElementById('nodesBody');
@@ -916,6 +984,17 @@
}
});
// #630: Close button for node detail panel (important for mobile full-screen overlay)
document.getElementById('nodesRight').addEventListener('click', function(e) {
if (e.target.closest('.panel-close-btn')) {
const panel = document.getElementById('nodesRight');
panel.classList.add('empty');
panel.innerHTML = '<span>Select a node to view details</span>';
selectedKey = null;
renderRows();
}
});
renderRows();
}
@@ -1003,6 +1082,7 @@
const dupBadge = dupNameBadge(n.name, n.public_key, dupMap);
panel.innerHTML = `
<button class="panel-close-btn" title="Close detail pane (Esc)"></button>
<div class="node-detail">
<div class="node-detail-name">${escapeHtml(n.name || '(unnamed)')}${dupBadge}</div>
<div class="node-detail-role">${renderNodeBadges(n, roleColor)}
@@ -1196,11 +1276,29 @@
window._nodesIsAdvertMessage = isAdvertMessage;
window._nodesGetAllNodes = function() { return _allNodes; };
window._nodesSetAllNodes = function(n) { _allNodes = n; };
window._nodesToggleSort = toggleSort;
window._nodesToggleSort = function(col) {
if (_nodesTableSortCtrl) { _nodesTableSortCtrl.sort(col); return; }
// Fallback for tests without DOM
var st = _getSortState();
var descDefault = ['last_seen', 'advert_count'];
if (st.column === col) {
_fallbackSortState = { column: col, direction: st.direction === 'asc' ? 'desc' : 'asc' };
} else {
_fallbackSortState = { column: col, direction: descDefault.indexOf(col) >= 0 ? 'desc' : 'asc' };
}
localStorage.setItem('meshcore-nodes-sort', JSON.stringify(_fallbackSortState));
};
window._nodesSortNodes = sortNodes;
window._nodesSortArrow = sortArrow;
window._nodesGetSortState = function() { return sortState; };
window._nodesSetSortState = function(s) { sortState = s; };
window._nodesSortArrow = function(col) {
var st = _getSortState();
if (st.column !== col) return '';
return '<span class="sort-arrow">' + (st.direction === 'asc' ? '▲' : '▼') + '</span>';
};
window._nodesGetSortState = _getSortState;
window._nodesSetSortState = function(s) {
_fallbackSortState = s;
if (_nodesTableSortCtrl) _nodesTableSortCtrl.sort(s.column, s.direction);
};
window._nodesSyncClaimedToFavorites = syncClaimedToFavorites;
window._nodesRenderNodeTimestampHtml = renderNodeTimestampHtml;
window._nodesRenderNodeTimestampText = renderNodeTimestampText;
+252 -55
View File
@@ -33,7 +33,26 @@
let totalCount = 0;
let expandedHashes = new Set();
let hopNameCache = {};
let _tableSortInstance = null;
let _packetSortColumn = null;
let _packetSortDirection = 'desc';
let showHexHashes = localStorage.getItem('meshcore-hex-hashes') === 'true';
var _pendingUrlRegion = null;
var DEFAULT_TIME_WINDOW = 15;
function buildPacketsQuery(timeWindowMin, regionParam) {
var parts = [];
if (timeWindowMin && timeWindowMin !== DEFAULT_TIME_WINDOW) parts.push('timeWindow=' + timeWindowMin);
if (regionParam) parts.push('region=' + encodeURIComponent(regionParam));
return parts.length ? '?' + parts.join('&') : '';
}
window.buildPacketsQuery = buildPacketsQuery;
function updatePacketsUrl() {
history.replaceState(null, '', '#/packets' + buildPacketsQuery(savedTimeWindowMin, RegionFilter.getRegionParam()));
}
let filtersBuilt = false;
let _renderTimer = null;
function scheduleRender() {
@@ -63,7 +82,9 @@
const getParsedDecoded = window.getParsedDecoded;
// --- Virtual scroll state ---
const VSCROLL_ROW_HEIGHT = 36; // estimated row height in px
let VSCROLL_ROW_HEIGHT = 36; // measured dynamically on first render; fallback 36px
let _vscrollRowHeightMeasured = false;
let _vscrollTheadHeight = 40; // measured dynamically on first render; fallback 40px
const VSCROLL_BUFFER = 30; // extra rows above/below viewport
let _displayPackets = []; // filtered packets for current view
let _displayGrouped = false; // whether _displayPackets is in grouped mode
@@ -78,6 +99,37 @@
let _wsRenderDirty = false; // dirty flag for rAF render coalescing (#396)
let _observerFilterSet = null; // cached Set from filters.observer, hoisted above loops (#427)
// Pure function: calculate visible entry range from scroll state.
// Extracted for testability (#405, #409).
function _calcVisibleRange(offsets, entryCount, scrollTop, viewportHeight, rowHeight, theadHeight, buffer) {
const adjustedScrollTop = Math.max(0, scrollTop - theadHeight);
const firstDomRow = Math.floor(adjustedScrollTop / rowHeight);
const visibleDomCount = Math.ceil(viewportHeight / rowHeight);
// Binary search for first entry whose cumulative offset covers firstDomRow
let lo = 0, hi = entryCount;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (offsets[mid + 1] <= firstDomRow) lo = mid + 1;
else hi = mid;
}
const firstEntry = lo;
// Binary search for last visible entry
const lastDomRow = firstDomRow + visibleDomCount;
lo = firstEntry; hi = entryCount;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (offsets[mid + 1] <= lastDomRow) lo = mid + 1;
else hi = mid;
}
const lastEntry = Math.min(lo + 1, entryCount);
const startIdx = Math.max(0, firstEntry - buffer);
const endIdx = Math.min(entryCount, lastEntry + buffer);
return { startIdx, endIdx, firstEntry, lastEntry };
}
function closeDetailPanel() {
var panel = document.getElementById('pktRight');
if (panel) {
@@ -280,8 +332,19 @@
filters.node = routeParam;
}
}
// Read URL params (router strips query from routeParam; read from location.hash)
var _initUrlParams = getHashParams();
var _urlTimeWindow = Number(_initUrlParams.get('timeWindow'));
if (Number.isFinite(_urlTimeWindow) && _urlTimeWindow > 0) {
savedTimeWindowMin = _urlTimeWindow;
localStorage.setItem('meshcore-time-window', String(_urlTimeWindow));
}
var _urlRegion = _initUrlParams.get('region');
if (_urlRegion) _pendingUrlRegion = _urlRegion;
app.innerHTML = `<div class="split-layout detail-collapsed">
<div class="panel-left" id="pktLeft"></div>
<div class="panel-left" id="pktLeft" aria-live="polite" aria-relevant="additions removals"></div>
<div class="panel-right empty" id="pktRight" aria-live="polite">
<div class="panel-resize-handle" id="pktResizeHandle"></div>
${PANEL_CLOSE_HTML}
@@ -466,8 +529,12 @@
if (h) hashIndex.set(h, newGroup);
}
}
// Re-sort by latest DESC, then evict oldest beyond the limit
packets.sort((a, b) => (b.latest || '').localeCompare(a.latest || ''));
// Re-sort by active sort column (or latest DESC as default), then evict oldest beyond the limit
if (_packetSortColumn) {
sortPacketsArray();
} else {
packets.sort((a, b) => (b.latest || '').localeCompare(a.latest || ''));
}
if (packets.length > PACKET_LIMIT) {
const evicted = packets.splice(PACKET_LIMIT);
for (const p of evicted) { if (p.hash) hashIndex.delete(p.hash); }
@@ -488,6 +555,7 @@
clearTimeout(_renderTimer);
if (wsHandler) offWS(wsHandler);
wsHandler = null;
if (_tableSortInstance) { _tableSortInstance.destroy(); _tableSortInstance = null; }
detachVScrollListener();
clearTimeout(_wsRenderTimer);
if (_wsRafId) { cancelAnimationFrame(_wsRafId); _wsRafId = null; }
@@ -616,11 +684,12 @@
}
}
sortPacketsArray();
renderLeft();
} catch (e) {
console.error('Failed to load packets:', e);
const tbody = document.getElementById('pktBody');
if (tbody) tbody.innerHTML = '<tr><td colspan="10" class="text-center" style="padding:24px;color:var(--error,#ef4444)"><div role="alert" aria-live="polite">Failed to load packets. Please try again.</div></td></tr>';
if (tbody) tbody.innerHTML = '<tr><td colspan="' + _getColCount() + '" class="text-center" style="padding:24px;color:var(--error,#ef4444)"><div role="alert" aria-live="polite">Failed to load packets. Please try again.</div></td></tr>';
}
}
@@ -706,9 +775,9 @@
</div>
<table class="data-table" id="pktTable">
<thead><tr>
<th scope="col"></th><th scope="col" class="col-region">Region</th><th scope="col" class="col-time">Time</th><th scope="col" class="col-hash">Hash</th><th scope="col" class="col-size">Size</th>
<th scope="col" class="col-hashsize">HB</th>
<th scope="col" class="col-type">Type</th><th scope="col" class="col-observer">Observer</th><th scope="col" class="col-path">Path</th><th scope="col" class="col-rpt">Rpt</th><th scope="col" class="col-details">Details</th>
<th scope="col"></th><th scope="col" class="col-region" data-sort-key="region">Region</th><th scope="col" class="col-time" data-sort-key="time" data-type="date">Time</th><th scope="col" class="col-hash" data-sort-key="hash">Hash</th><th scope="col" class="col-size" data-sort-key="size" data-type="numeric">Size</th>
<th scope="col" class="col-hashsize" data-sort-key="hb" data-type="numeric">HB</th>
<th scope="col" class="col-type" data-sort-key="type">Type</th><th scope="col" class="col-observer" data-sort-key="observer">Observer</th><th scope="col" class="col-path" data-sort-key="path">Path</th><th scope="col" class="col-rpt" data-sort-key="rpt" data-type="numeric">Rpt</th><th scope="col" class="col-details">Details</th>
</tr></thead>
<tbody id="pktBody"></tbody>
</table>
@@ -716,7 +785,11 @@
// Init shared RegionFilter component
RegionFilter.init(document.getElementById('packetsRegionFilter'), { dropdown: true });
RegionFilter.onChange(function() { loadPackets(); });
if (_pendingUrlRegion) {
RegionFilter.setSelected(_pendingUrlRegion.split(',').filter(Boolean));
_pendingUrlRegion = null;
}
RegionFilter.onChange(function() { updatePacketsUrl(); loadPackets(); });
// --- Packet Filter Language ---
(function() {
@@ -866,6 +939,7 @@
savedTimeWindowMin = Number(fTimeWindow.value);
if (!Number.isFinite(savedTimeWindowMin) || savedTimeWindowMin <= 0) savedTimeWindowMin = 15;
localStorage.setItem('meshcore-time-window', fTimeWindow.value);
updatePacketsUrl();
loadPackets();
});
@@ -1101,10 +1175,37 @@
renderTableRows();
makeColumnsResizable('#pktTable', 'meshcore-pkt-col-widths');
// Initialize table sorting (virtual scroll — sort data array, not DOM)
if (window.TableSort) {
var pktTableEl = document.getElementById('pktTable');
if (pktTableEl) {
if (_tableSortInstance) _tableSortInstance.destroy();
_tableSortInstance = TableSort.init(pktTableEl, {
defaultColumn: 'time',
defaultDirection: 'desc',
storageKey: 'meshcore-packets-sort',
domReorder: false,
onSort: function(column, direction) {
_packetSortColumn = column;
_packetSortDirection = direction;
sortPacketsArray();
renderTableRows();
}
});
// Apply initial sort state from TableSort
if (_tableSortInstance) {
var st = _tableSortInstance.getState();
_packetSortColumn = st.column;
_packetSortDirection = st.direction;
sortPacketsArray();
}
}
}
}
// Build HTML for a single grouped packet row
function buildGroupRowHtml(p) {
function buildGroupRowHtml(p, entryIdx = -1) {
const isExpanded = expandedHashes.has(p.hash);
let headerObserverId = p.observer_id;
let headerPathJson = p.path_json;
@@ -1124,7 +1225,10 @@
const groupSize = p.raw_hex ? Math.floor(p.raw_hex.length / 2) : 0;
const groupHashBytes = ((parseInt(p.raw_hex?.slice(2, 4), 16) || 0) >> 6) + 1;
const isSingle = p.count <= 1;
let html = `<tr class="${isSingle ? '' : 'group-header'} ${isExpanded ? 'expanded' : ''}" data-hash="${p.hash}" data-action="${isSingle ? 'select-hash' : 'toggle-select'}" data-value="${p.hash}" tabindex="0" role="row">
// Channel color highlighting (#271)
const _grpDecoded = getParsedDecoded(p) || {};
const _grpChanStyle = window.ChannelColors ? window.ChannelColors.getRowStyle(_grpDecoded.type || groupTypeName, _grpDecoded.channel) : '';
let html = `<tr class="${isSingle ? '' : 'group-header'} ${isExpanded ? 'expanded' : ''}" data-hash="${p.hash}" data-action="${isSingle ? 'select-hash' : 'toggle-select'}" data-value="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row"${_grpChanStyle ? ' style="' + _grpChanStyle + '"' : ''}>
<td style="width:28px;text-align:center;cursor:pointer">${isSingle ? '' : (isExpanded ? '▼' : '▶')}</td>
<td class="col-region">${groupRegion ? `<span class="badge-region">${groupRegion}</span>` : '—'}</td>
<td class="col-time">${renderTimestampCell(p.latest)}</td>
@@ -1150,7 +1254,7 @@
const childRegion = c.observer_id ? (observerMap.get(c.observer_id)?.iata || '') : '';
const childPath = getParsedPath(c);
const childPathStr = renderPath(childPath, c.observer_id);
html += `<tr class="group-child" data-id="${c.id}" data-hash="${c.hash || ''}" data-action="select-observation" data-value="${c.id}" data-parent-hash="${p.hash}" tabindex="0" role="row">
html += `<tr class="group-child" data-id="${c.id}" data-hash="${c.hash || ''}" data-action="select-observation" data-value="${c.id}" data-parent-hash="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row">
<td></td><td class="col-region">${childRegion ? `<span class="badge-region">${childRegion}</span>` : ''}</td>
<td class="col-time">${renderTimestampCell(c.timestamp)}</td>
<td class="mono col-hash">${truncate(c.hash || '', 8)}</td>
@@ -1168,17 +1272,19 @@
}
// Build HTML for a single flat (ungrouped) packet row
function buildFlatRowHtml(p) {
function buildFlatRowHtml(p, entryIdx = -1) {
const decoded = getParsedDecoded(p) || {};
const pathHops = getParsedPath(p) || [];
const region = p.observer_id ? (observerMap.get(p.observer_id)?.iata || '') : '';
const typeName = payloadTypeName(p.payload_type);
const typeClass = payloadTypeColor(p.payload_type);
// Channel color highlighting (#271)
const _chanStyle = window.ChannelColors ? window.ChannelColors.getRowStyle(decoded.type || typeName, decoded.channel) : '';
const size = p.raw_hex ? Math.floor(p.raw_hex.length / 2) : 0;
const hashBytes = ((parseInt(p.raw_hex?.slice(2, 4), 16) || 0) >> 6) + 1;
const pathStr = renderPath(pathHops, p.observer_id);
const detail = getDetailPreview(decoded);
return `<tr data-id="${p.id}" data-hash="${p.hash || ''}" data-action="select-hash" data-value="${p.hash || p.id}" tabindex="0" role="row" class="${selectedId === p.id ? 'selected' : ''}">
return `<tr data-id="${p.id}" data-hash="${p.hash || ''}" data-action="select-hash" data-value="${p.hash || p.id}" data-entry-idx="${entryIdx}" tabindex="0" role="row" class="${selectedId === p.id ? 'selected' : ''}"${_chanStyle ? ' style="' + _chanStyle + '"' : ''}>
<td></td><td class="col-region">${region ? `<span class="badge-region">${region}</span>` : ''}</td>
<td class="col-time">${renderTimestampCell(p.timestamp)}</td>
<td class="mono col-hash">${truncate(p.hash || String(p.id), 8)}</td>
@@ -1239,6 +1345,7 @@
}
function renderVisibleRows() {
const _rvr_t0 = performance.now();
const tbody = document.getElementById('pktBody');
if (!tbody || !_displayPackets.length) return;
@@ -1271,38 +1378,23 @@
// Calculate visible range based on scroll position
const scrollTop = scrollContainer.scrollTop;
const viewportHeight = scrollContainer.clientHeight;
// Account for thead height (~40px)
const theadHeight = 40;
const adjustedScrollTop = Math.max(0, scrollTop - theadHeight);
// Account for thead height (measured dynamically)
const theadEl = scrollContainer.querySelector('thead');
if (theadEl) _vscrollTheadHeight = theadEl.offsetHeight || _vscrollTheadHeight;
// Find the first entry whose cumulative row offset covers the scroll position
const firstDomRow = Math.floor(adjustedScrollTop / VSCROLL_ROW_HEIGHT);
const visibleDomCount = Math.ceil(viewportHeight / VSCROLL_ROW_HEIGHT);
// Binary search for entry index containing firstDomRow
let lo = 0, hi = _displayPackets.length;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (offsets[mid + 1] <= firstDomRow) lo = mid + 1;
else hi = mid;
}
const firstEntry = lo;
// Find entry index covering last visible DOM row
const lastDomRow = firstDomRow + visibleDomCount;
lo = firstEntry; hi = _displayPackets.length;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (offsets[mid + 1] <= lastDomRow) lo = mid + 1;
else hi = mid;
}
const lastEntry = Math.min(lo + 1, _displayPackets.length);
const startIdx = Math.max(0, firstEntry - VSCROLL_BUFFER);
const endIdx = Math.min(_displayPackets.length, lastEntry + VSCROLL_BUFFER);
const { startIdx, endIdx } = _calcVisibleRange(
offsets, _displayPackets.length, scrollTop, viewportHeight,
VSCROLL_ROW_HEIGHT, _vscrollTheadHeight, VSCROLL_BUFFER
);
// Skip DOM rebuild if visible range hasn't changed
if (startIdx === _lastVisibleStart && endIdx === _lastVisibleEnd) return;
if (startIdx === _lastVisibleStart && endIdx === _lastVisibleEnd) {
if (window.__PERF_LOG_RENDER) console.log('[perf] renderVisibleRows: skip (no change) %.2fms', performance.now() - _rvr_t0);
return;
}
const prevStart = _lastVisibleStart;
const prevEnd = _lastVisibleEnd;
_lastVisibleStart = startIdx;
_lastVisibleEnd = endIdx;
@@ -1313,14 +1405,59 @@
topSpacer.firstChild.style.height = topPad + 'px';
bottomSpacer.firstChild.style.height = bottomPad + 'px';
// LAZY ROW GENERATION: only build HTML for the visible slice (#422)
const builder = _displayGrouped ? buildGroupRowHtml : buildFlatRowHtml;
const visibleSlice = _displayPackets.slice(startIdx, endIdx);
const visibleHtml = visibleSlice.map(p => builder(p)).join('');
tbody.innerHTML = '';
tbody.appendChild(topSpacer);
tbody.insertAdjacentHTML('beforeend', visibleHtml);
tbody.appendChild(bottomSpacer);
const hasOverlap = prevStart !== -1 && startIdx < prevEnd && endIdx > prevStart;
if (!hasOverlap) {
// Full rebuild: initial render or large scroll jump past buffer
const visibleHtml = _displayPackets.slice(startIdx, endIdx)
.map((p, i) => builder(p, startIdx + i)).join('');
tbody.innerHTML = '';
tbody.appendChild(topSpacer);
tbody.insertAdjacentHTML('beforeend', visibleHtml);
tbody.appendChild(bottomSpacer);
// Measure actual row height from first rendered data row (#407)
if (!_vscrollRowHeightMeasured) {
const firstRow = topSpacer.nextElementSibling;
if (firstRow && firstRow !== bottomSpacer) {
const h = firstRow.offsetHeight;
if (h > 0) { VSCROLL_ROW_HEIGHT = h; _vscrollRowHeightMeasured = true; }
}
}
if (window.__PERF_LOG_RENDER) console.log('[perf] renderVisibleRows: full rebuild %d entries, %.2fms', endIdx - startIdx, performance.now() - _rvr_t0);
return;
}
// Incremental update: remove rows that scrolled out at the top (positional)
const headRowCount = offsets[Math.min(startIdx, prevEnd)] - offsets[prevStart];
for (let r = 0; r < headRowCount; r++) {
const row = topSpacer.nextElementSibling;
if (row && row !== bottomSpacer) row.remove();
}
// Remove rows that scrolled out at the bottom (positional)
const tailFrom = Math.max(endIdx, prevStart);
const tailRowCount = offsets[prevEnd] - offsets[tailFrom];
for (let r = 0; r < tailRowCount; r++) {
const row = bottomSpacer.previousElementSibling;
if (row && row !== topSpacer) row.remove();
}
// Prepend rows that scrolled into view at the top
if (startIdx < prevStart) {
let html = '';
for (let i = startIdx; i < Math.min(prevStart, endIdx); i++) {
html += builder(_displayPackets[i], i);
}
topSpacer.insertAdjacentHTML('afterend', html);
}
// Append rows that scrolled into view at the bottom
if (endIdx > prevEnd) {
let html = '';
for (let i = Math.max(prevEnd, startIdx); i < endIdx; i++) {
html += builder(_displayPackets[i], i);
}
bottomSpacer.insertAdjacentHTML('beforebegin', html);
}
if (window.__PERF_LOG_RENDER) console.log('[perf] renderVisibleRows: incremental head=%d tail=%d, %.2fms', headRowCount, tailRowCount, performance.now() - _rvr_t0);
}
// Attach/detach scroll listener for virtual scrolling
@@ -1346,6 +1483,55 @@
_vsScrollHandler = null;
}
/** Sort the packets array by the current sort column. Called before renderTableRows. */
function sortPacketsArray() {
if (!_packetSortColumn || !packets.length) return;
var col = _packetSortColumn;
var dir = _packetSortDirection === 'asc' ? 1 : -1;
var accessor;
switch (col) {
case 'time': accessor = function(p) { return p.latest || p.timestamp || ''; }; break;
case 'type': accessor = function(p) { return typeName(p.payload_type); }; break;
case 'hash': accessor = function(p) { return p.hash || ''; }; break;
case 'observer': accessor = function(p) { return obsName(p.observer_id); }; break;
case 'size': accessor = function(p) { return p.packet_size || 0; }; break;
case 'hb': accessor = function(p) { return p.hash_byte_count != null ? p.hash_byte_count : (p.hash_size || 0); }; break;
case 'rpt': accessor = function(p) {
try { var pj = typeof p.path_json === 'string' ? JSON.parse(p.path_json) : p.path_json; return Array.isArray(pj) ? pj.length : 0; } catch(e) { return 0; }
}; break;
case 'region': accessor = function(p) { return (regionMap && regionMap[p.observer_id]) || ''; }; break;
case 'path': accessor = function(p) {
try { var pj = typeof p.path_json === 'string' ? JSON.parse(p.path_json) : p.path_json; return Array.isArray(pj) ? pj.join(',') : ''; } catch(e) { return ''; }
}; break;
default: return; // unsortable column
}
// Choose comparator based on column type
var isNumeric = (col === 'size' || col === 'hb' || col === 'rpt');
var isDate = (col === 'time');
packets.sort(function(a, b) {
var va = accessor(a), vb = accessor(b);
var result;
if (isDate) {
result = TableSort.comparators.date(va, vb);
} else if (isNumeric) {
result = TableSort.comparators.numeric(va, vb);
} else {
result = TableSort.comparators.text(va, vb);
}
// Stable tiebreaker: sort by timestamp (desc) when primary values are equal
if (result === 0 && !isDate) {
result = TableSort.comparators.date(
a.timestamp || a.first_seen || '',
b.timestamp || b.first_seen || ''
) * -1; // desc (newest first)
}
return dir * result;
});
}
async function renderTableRows() {
const tbody = document.getElementById('pktBody');
if (!tbody) return;
@@ -1576,7 +1762,7 @@
// Parse hash size from path byte
const rawPathByte = pkt.raw_hex ? parseInt(pkt.raw_hex.slice(2, 4), 16) : NaN;
const hashSize = isNaN(rawPathByte) ? null : ((rawPathByte >> 6) + 1);
const hashSize = (isNaN(rawPathByte) || (rawPathByte & 0x3F) === 0) ? null : ((rawPathByte >> 6) + 1);
const size = pkt.raw_hex ? Math.floor(pkt.raw_hex.length / 2) : 0;
const typeName = payloadTypeName(pkt.payload_type);
@@ -1737,7 +1923,7 @@
}
// Wire up view route on map button
const routeBtn = document.getElementById('viewRouteBtn');
const routeBtn = panel.querySelector('#viewRouteBtn');
if (routeBtn && pathHops.length) {
routeBtn.addEventListener('click', async () => {
try {
@@ -1798,7 +1984,7 @@
const pathByte0 = parseInt(buf.slice(2, 4), 16);
const hashSizeVal = isNaN(pathByte0) ? '?' : ((pathByte0 >> 6) + 1);
const hashCountVal = isNaN(pathByte0) ? '?' : (pathByte0 & 0x3F);
rows += fieldRow(1, 'Path Length', '0x' + (buf.slice(2, 4) || '??'), `hash_size=${hashSizeVal} byte${hashSizeVal !== 1 ? 's' : ''}, hash_count=${hashCountVal}`);
rows += fieldRow(1, 'Path Length', '0x' + (buf.slice(2, 4) || '??'), hashCountVal === 0 ? `hash_count=0 (direct advert)` : `hash_size=${hashSizeVal} byte${hashSizeVal !== 1 ? 's' : ''}, hash_count=${hashCountVal}`);
// Transport codes
let off = 2;
@@ -1826,7 +2012,7 @@
rows += sectionRow('Payload — ' + payloadTypeName(pkt.payload_type), 'section-payload');
if (decoded.type === 'ADVERT') {
rows += fieldRow(1, 'Advertised Hash Size', hashSizeVal + ' byte' + (hashSizeVal !== 1 ? 's' : ''), 'From path byte 0x' + (buf.slice(2, 4) || '??') + ' — bits 7-6 = ' + (hashSizeVal - 1));
if (hashCountVal !== 0) rows += fieldRow(1, 'Advertised Hash Size', hashSizeVal + ' byte' + (hashSizeVal !== 1 ? 's' : ''), 'From path byte 0x' + (buf.slice(2, 4) || '??') + ' — bits 7-6 = ' + (hashSizeVal - 1));
rows += fieldRow(off, 'Public Key (32B)', truncate(decoded.pubKey || '', 24), '');
rows += fieldRow(off + 32, 'Timestamp (4B)', decoded.timestampISO || '', 'Unix: ' + (decoded.timestamp || ''));
rows += fieldRow(off + 36, 'Signature (64B)', truncate(decoded.signature || '', 24), '');
@@ -1988,6 +2174,12 @@
html += kv(k, String(v));
}
}
// Special handling for advert signature validation
if (h.payloadType === 4 && p.signatureValid !== undefined) {
const status = p.signatureValid ? 'Valid' : 'Invalid';
const badgeClass = p.signatureValid ? 'badge-success' : 'badge-danger';
html += kv('Signature', `<span class="badge ${badgeClass}">${status}</span>`);
}
html += '</div></div>';
// Raw hex
@@ -2118,7 +2310,10 @@
init: function(app, routeParam) {
_themeRefreshHandler = () => { if (typeof renderTableRows === 'function') renderTableRows(); };
window.addEventListener('theme-refresh', _themeRefreshHandler);
return init(app, routeParam);
var result = init(app, routeParam);
// Install channel color picker on packets table (M2, #271)
if (window.ChannelColorPicker) window.ChannelColorPicker.installPacketsTable();
return result;
},
destroy: function() {
if (_themeRefreshHandler) { window.removeEventListener('theme-refresh', _themeRefreshHandler); _themeRefreshHandler = null; }
@@ -2129,6 +2324,7 @@
// Standalone packet detail page: #/packet/123 or #/packet/HASH
// Expose pure functions for unit testing (vm.createContext pattern)
if (typeof window !== 'undefined') {
document.addEventListener('channel-colors-changed', function() { renderVisibleRows(); });
window._packetsTestAPI = {
typeName,
obsName,
@@ -2148,6 +2344,7 @@
_refreshRowCountsIfDirty,
buildGroupRowHtml,
buildFlatRowHtml,
_calcVisibleRange,
};
}
+11 -1
View File
@@ -6,6 +6,7 @@
var _regions = {}; // { code: label }
var _selected = null; // Set of selected region codes, null = all
var _listeners = [];
var _container = null;
var _loaded = false;
function loadFromStorage() {
@@ -199,11 +200,19 @@
/** Initialize filter in a container, fetch regions, render, return promise.
* Options: { dropdown: true } to force dropdown mode regardless of region count */
async function initFilter(container, opts) {
_container = container;
if (opts && opts.dropdown) container._forceDropdown = true;
await fetchRegions();
render(container);
}
/** Override selected regions (e.g. from URL param). Persists to localStorage and re-renders. */
function setSelected(codesArray) {
_selected = (codesArray && codesArray.length > 0) ? new Set(codesArray) : null;
saveToStorage();
if (_container) render(_container);
}
// Expose globally
window.RegionFilter = {
init: initFilter,
@@ -213,6 +222,7 @@
regionQueryString: regionQueryString,
onChange: onChange,
offChange: offChange,
fetchRegions: fetchRegions
fetchRegions: fetchRegions,
setSelected: setSelected
};
})();
+253
View File
@@ -30,6 +30,7 @@
--content-bg: var(--surface-0);
--card-bg: var(--surface-1);
--hover-bg: rgba(0,0,0, 0.04);
--trace-ghost-color: #94a3b8;
}
/* DARK THEME VARIABLES KEEP BOTH BLOCKS IN SYNC
@@ -55,6 +56,7 @@
--input-bg: #1e1e34;
--selected-bg: #1e3a5f;
--hover-bg: rgba(255,255,255, 0.06);
--trace-ghost-color: #94a3b8;
--section-bg: #1e1e34;
}
}
@@ -78,6 +80,7 @@
--input-bg: #1e1e34;
--selected-bg: #1e3a5f;
--hover-bg: rgba(255,255,255, 0.06);
--trace-ghost-color: #94a3b8;
--section-bg: #1e1e34;
}
@@ -1184,6 +1187,8 @@ button.ch-item.ch-item-encrypted .ch-badge { filter: grayscale(0.6); }
.hash-bar-value { min-width: 120px; text-align: right; font-size: 13px; font-weight: 600; }
.badge-hash-1 { background: #ef444420; color: var(--status-red); }
.badge-hash-2 { background: #22c55e20; color: var(--status-green); }
.badge-success { background: #22c55e20; color: var(--status-green); }
.badge-danger { background: #ef444420; color: var(--status-red); }
.badge-hash-3 { background: #3b82f620; color: var(--accent); }
.timeline-legend { display: flex; gap: 16px; justify-content: center; margin-top: 8px; font-size: 12px; }
.legend-dot { display: inline-block; width: 10px; height: 10px; border-radius: 50%; margin-right: 4px; vertical-align: middle; }
@@ -1958,3 +1963,251 @@ tr[data-hops]:hover { background: rgba(59,130,246,0.1); }
#ngCanvas:focus:not(:focus-visible) {
outline: none;
}
/* ===================== RF Health Dashboard ===================== */
.rf-health-container { padding: 0; }
.rf-time-selector {
display: flex; flex-wrap: wrap; gap: 4px; align-items: center;
margin-bottom: 8px; padding: 8px 0;
}
.rf-range-btn {
padding: 4px 10px; border: 1px solid var(--border); border-radius: 4px;
background: var(--bg-secondary, var(--card-bg, #1e1e1e)); color: var(--text-primary, #e0e0e0);
cursor: pointer; font-size: 12px; transition: background 0.15s;
}
.rf-range-btn:hover { background: var(--bg-hover, #333); }
.rf-range-btn.active { background: var(--accent); color: #fff; border-color: var(--accent); }
.rf-custom-inputs { display: inline-flex; gap: 4px; align-items: center; margin-left: 8px; }
.rf-datetime {
padding: 3px 6px; border: 1px solid var(--border); border-radius: 4px;
background: var(--bg-secondary, var(--card-bg)); color: var(--text-primary); font-size: 12px;
}
.rf-health-split {
display: flex; height: calc(100vh - 180px); min-height: 300px; overflow: hidden;
}
.rf-health-grid {
flex: 1; min-width: 0; overflow-y: auto; padding: 0 8px 8px 0;
display: grid; grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
gap: 8px; align-content: start;
}
.rf-cell {
border: 1px solid var(--border); border-radius: 6px; padding: 8px 10px;
cursor: pointer; transition: border-color 0.15s, background 0.15s;
background: var(--bg-secondary, var(--card-bg, #1e1e1e));
}
.rf-cell:hover { border-color: var(--accent); }
.rf-cell:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; }
.rf-cell-selected { border-color: var(--accent); background: var(--bg-hover, rgba(96,165,250,0.08)); }
.rf-cell-header { display: flex; justify-content: space-between; align-items: baseline; gap: 6px; margin-bottom: 4px; }
.rf-cell-name { font-weight: 600; font-size: 13px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: 120px; }
.rf-cell-nf { font-size: 13px; font-variant-numeric: tabular-nums; white-space: nowrap; }
.rf-cell-batt { font-size: 11px; color: var(--text-muted); white-space: nowrap; }
.rf-nf-warning { color: var(--status-yellow, #f59e0b); }
.rf-nf-critical { color: var(--status-red, #ef4444); }
.rf-cell-sparkline { height: 24px; margin: 2px 0; overflow: hidden; }
.rf-cell-stats { display: flex; gap: 8px; font-size: 10px; color: var(--text-muted); }
/* Side panel for observer detail */
.rf-health-detail {
width: 420px; min-width: 280px; max-width: 50vw;
border-left: 1px solid var(--border); background: var(--bg-secondary, var(--card-bg));
overflow-y: auto; padding: 16px; position: relative;
animation: slideInRight 200ms ease-out;
}
.rf-health-detail.rf-panel-empty {
display: flex; align-items: center; justify-content: center;
color: var(--text-muted); font-size: 14px; animation: none;
}
.rf-detail-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px; }
.rf-detail-header h3 { margin: 0; font-size: 16px; }
.rf-detail-close {
background: none; border: none; color: var(--text-muted); cursor: pointer;
font-size: 18px; padding: 2px 6px; border-radius: 4px;
}
.rf-detail-close:hover { background: var(--bg-hover); }
.rf-detail-charts { display: flex; flex-direction: column; gap: 4px; }
.rf-detail-chart { margin: 0; overflow-x: auto; }
.rf-detail-summary { font-size: 12px; color: var(--text-muted); font-variant-numeric: tabular-nums; }
@media (max-width: 640px) {
.rf-health-split { flex-direction: column; height: auto; }
.rf-health-grid { grid-template-columns: 1fr; max-height: 50vh; }
.rf-health-detail {
width: 100% !important; max-width: 100%; min-width: 0;
border-left: none; border-top: 1px solid var(--border);
}
.rf-time-selector { gap: 3px; }
.rf-custom-inputs { margin-left: 0; margin-top: 4px; flex-wrap: wrap; }
}
/* Channel Color Picker Popover (M2, #271) */
/* === Channel Color Picker (#674) === */
.cc-picker-popover {
position: fixed;
z-index: 9999;
background: var(--bg-secondary, #1e1e1e);
border: 1px solid var(--border-color, #333);
border-radius: 8px;
padding: 8px;
box-shadow: 0 4px 12px rgba(0,0,0,0.3);
}
.cc-picker-swatches {
display: flex;
gap: 6px;
}
.cc-swatch {
width: 24px;
height: 24px;
border-radius: 50%;
border: 2px solid transparent;
cursor: pointer;
padding: 0;
transition: border-color 0.15s;
}
.cc-swatch:hover { border-color: rgba(255,255,255,0.6); }
.cc-swatch:focus-visible { border-color: #fff; outline: 2px solid var(--accent, #3b82f6); outline-offset: 1px; }
.cc-swatch-active { border-color: #fff; }
.cc-picker-clear {
display: block;
width: 100%;
margin-top: 6px;
padding: 4px 0;
font-size: 11px;
color: var(--text-muted, #888);
background: none;
border: none;
cursor: pointer;
text-align: center;
}
.cc-picker-clear:hover { color: var(--text-primary, #e0e0e0); }
/* Color dot affordance (#674) */
.ch-color-dot {
display: inline-block;
width: 12px;
height: 12px;
border-radius: 50%;
border: 1.5px solid rgba(255,255,255,0.3);
cursor: pointer;
vertical-align: middle;
margin-left: 6px;
flex-shrink: 0;
}
.ch-color-dot:not([style*="background"]) {
background: transparent;
border-style: dashed;
border-color: var(--text-muted, #888);
}
/* Mobile bottom-sheet + larger touch targets (#674) */
@media (pointer: coarse) {
.ch-color-dot {
width: 20px;
height: 20px;
margin-left: 8px;
}
.cc-swatch {
width: 36px;
height: 36px;
}
.cc-picker-swatches {
justify-content: center;
gap: 10px;
}
.cc-picker-popover {
position: fixed !important;
bottom: 0 !important;
left: 0 !important;
right: 0 !important;
top: auto !important;
width: 100% !important;
max-width: 100% !important;
border-radius: 12px 12px 0 0;
padding: 16px;
padding-bottom: calc(16px + env(safe-area-inset-bottom));
box-sizing: border-box;
}
}
/* === #630 — Mobile Accessibility Fixes === */
/* #630-1: Touch targets — minimum 44px on touch devices */
@media (pointer: coarse) {
.filter-bar .btn,
.filter-group .btn,
.tab-btn,
.filter-bar input,
.filter-bar select,
.nav-btn,
.region-pill,
.region-dropdown-trigger,
.multi-select-trigger,
.node-count-pill,
.analytics-time-range button,
.detail-back-btn,
.filter-toggle-btn {
min-height: 44px;
min-width: 44px;
}
.filter-bar input,
.filter-bar select {
height: 44px;
}
.region-dropdown-trigger,
.multi-select-trigger {
height: 44px;
}
}
/* #630-3: Status text labels — visually hidden text for screen readers */
.sr-status-label { font-size: 11px; margin-left: 4px; }
/* #630-4: Detail panel as full-width overlay on mobile */
@media (max-width: 640px) {
.split-layout .panel-right:not(.empty) {
display: block;
position: fixed;
top: 52px;
left: 0;
right: 0;
bottom: 0;
width: 100%;
min-width: 0;
z-index: 150;
overflow-y: auto;
-webkit-overflow-scrolling: touch;
}
}
/* #630-5: Analytics tabs — horizontal scroll on small screens */
@media (max-width: 640px) {
.analytics-tabs {
flex-wrap: nowrap;
overflow-x: auto;
-webkit-overflow-scrolling: touch;
scrollbar-width: thin;
padding-bottom: 4px;
}
.analytics-tabs .tab-btn {
flex-shrink: 0;
white-space: nowrap;
}
}
/* #630-6: Tables — horizontal scroll wrapper */
.table-scroll-wrap {
overflow-x: auto;
-webkit-overflow-scrolling: touch;
}
@media (max-width: 640px) {
.data-table { min-width: 480px; }
}
/* Table sorting indicators */
th[data-sort-key] { cursor: pointer; user-select: none; }
th[data-sort-key]:hover { background: var(--hover-bg, rgba(255,255,255,0.05)); }
th.sort-active { color: var(--accent, #60a5fa); }
.sort-arrow { font-size: 0.75em; opacity: 0.8; }
+224
View File
@@ -0,0 +1,224 @@
/* === CoreScope — table-sort.js === */
/* Shared table sorting utility. IIFE, no dependencies. */
'use strict';
window.TableSort = (function() {
/**
* Built-in comparators. Each takes two raw string values (from data-value or textContent)
* and returns a number for Array.sort.
*/
var comparators = {
text: function(a, b) {
if (a == null) a = '';
if (b == null) b = '';
return String(a).localeCompare(String(b));
},
numeric: function(a, b) {
var na = Number(a), nb = Number(b);
var aIsNaN = isNaN(na), bIsNaN = isNaN(nb);
if (aIsNaN && bIsNaN) return 0;
if (aIsNaN) return 1; // NaN sorts last
if (bIsNaN) return -1;
return na - nb;
},
date: function(a, b) {
var ta = a ? new Date(a).getTime() : NaN;
var tb = b ? new Date(b).getTime() : NaN;
var aIsNaN = isNaN(ta), bIsNaN = isNaN(tb);
if (aIsNaN && bIsNaN) return 0;
if (aIsNaN) return 1;
if (bIsNaN) return -1;
return ta - tb;
},
dbm: function(a, b) {
var na = parseFloat(String(a).replace(/\s*dBm\s*/i, ''));
var nb = parseFloat(String(b).replace(/\s*dBm\s*/i, ''));
var aIsNaN = isNaN(na), bIsNaN = isNaN(nb);
if (aIsNaN && bIsNaN) return 0;
if (aIsNaN) return 1;
if (bIsNaN) return -1;
return na - nb;
}
};
/**
* Resolve the comparator for a <th> element.
* Priority: custom comparator from options > data-type attribute > text default.
*/
function resolveComparator(key, thEl, customComparators) {
if (customComparators && customComparators[key]) return customComparators[key];
var type = thEl.getAttribute('data-type');
if (type && comparators[type]) return comparators[type];
return comparators.text;
}
/**
* Get the sort value for a <td>. Prefers data-value attribute, falls back to textContent.
*/
function getCellValue(td) {
if (!td) return '';
var dv = td.getAttribute('data-value');
return dv != null ? dv : td.textContent.trim();
}
/**
* Initialize sorting on a table element.
*
* @param {HTMLTableElement} tableEl - The table to make sortable
* @param {Object} [options]
* @param {string} [options.defaultColumn] - data-sort-key of initial sort column
* @param {string} [options.defaultDirection='asc'] - 'asc' or 'desc'
* @param {string} [options.storageKey] - localStorage key for persistence
* @param {Object} [options.comparators] - custom comparator functions keyed by column key
* @param {Function} [options.onSort] - callback(column, direction) after sort
* @param {boolean} [options.domReorder=true] - if false, skip DOM reorder (for virtual scroll tables)
* @returns {Object} instance with sort(), destroy(), getState() methods
*/
function init(tableEl, options) {
if (!tableEl) return null;
options = options || {};
var thead = tableEl.querySelector('thead');
if (!thead) return null;
var state = { column: options.defaultColumn || null, direction: options.defaultDirection || 'asc' };
var domReorder = options.domReorder !== false;
// Restore from localStorage
if (options.storageKey) {
try {
var saved = JSON.parse(localStorage.getItem(options.storageKey));
if (saved && saved.column) {
state.column = saved.column;
state.direction = saved.direction || 'asc';
}
} catch(e) { /* ignore */ }
}
var ths = thead.querySelectorAll('th[data-sort-key]');
var thMap = {}; // key → th element
var handlers = []; // for cleanup
for (var i = 0; i < ths.length; i++) {
(function(th) {
var key = th.getAttribute('data-sort-key');
thMap[key] = th;
th.style.cursor = 'pointer';
th.setAttribute('tabindex', '0');
th.setAttribute('aria-sort', 'none');
var handler = function(e) {
if (e.type === 'keydown' && e.key !== 'Enter' && e.key !== ' ') return;
if (e.type === 'keydown') e.preventDefault();
if (state.column === key) {
state.direction = state.direction === 'asc' ? 'desc' : 'asc';
} else {
state.column = key;
state.direction = options.defaultDirection || 'asc';
}
doSort();
};
th.addEventListener('click', handler);
th.addEventListener('keydown', handler);
handlers.push({ el: th, click: handler, keydown: handler });
})(ths[i]);
}
// Apply initial sort if defaultColumn is set
if (state.column && thMap[state.column]) {
updateArrows();
if (domReorder) sortDOM();
}
function doSort() {
updateArrows();
if (options.storageKey) {
try { localStorage.setItem(options.storageKey, JSON.stringify(state)); } catch(e) { /* ignore */ }
}
if (domReorder) sortDOM();
if (options.onSort) options.onSort(state.column, state.direction);
}
function updateArrows() {
for (var k in thMap) {
var th = thMap[k];
// Remove existing arrow
var arrow = th.querySelector('.sort-arrow');
if (arrow) arrow.remove();
if (k === state.column) {
th.classList.add('sort-active');
th.setAttribute('aria-sort', state.direction === 'asc' ? 'ascending' : 'descending');
var span = document.createElement('span');
span.className = 'sort-arrow';
span.textContent = state.direction === 'asc' ? ' ▲' : ' ▼';
th.appendChild(span);
} else {
th.classList.remove('sort-active');
th.setAttribute('aria-sort', 'none');
}
}
}
function sortDOM() {
var tbody = tableEl.querySelector('tbody');
if (!tbody) return;
var th = thMap[state.column];
if (!th) return;
var cmp = resolveComparator(state.column, th, options.comparators);
var colIndex = -1;
var allThs = thead.querySelectorAll('th');
for (var j = 0; j < allThs.length; j++) {
if (allThs[j] === th) { colIndex = j; break; }
}
if (colIndex < 0) return;
var rows = Array.prototype.slice.call(tbody.querySelectorAll('tr'));
var dir = state.direction === 'asc' ? 1 : -1;
rows.sort(function(rowA, rowB) {
var a = getCellValue(rowA.cells[colIndex]);
var b = getCellValue(rowB.cells[colIndex]);
return dir * cmp(a, b);
});
// DOM reorder via appendChild (no innerHTML rebuild)
for (var r = 0; r < rows.length; r++) {
tbody.appendChild(rows[r]);
}
}
function destroy() {
for (var h = 0; h < handlers.length; h++) {
handlers[h].el.removeEventListener('click', handlers[h].click);
handlers[h].el.removeEventListener('keydown', handlers[h].keydown);
// Clean up aria/classes
handlers[h].el.removeAttribute('aria-sort');
handlers[h].el.classList.remove('sort-active');
var arrow = handlers[h].el.querySelector('.sort-arrow');
if (arrow) arrow.remove();
}
handlers = [];
}
function sort(column, direction) {
if (column) state.column = column;
if (direction) state.direction = direction;
doSort();
}
function getState() {
return { column: state.column, direction: state.direction };
}
return { sort: sort, destroy: destroy, getState: getState };
}
return {
init: init,
comparators: comparators
};
})();
+177
View File
@@ -0,0 +1,177 @@
/**
* Tests for channel color picker fix (#674)
*
* Verifies:
* 1. _ccChannel is set correctly for GRP_TXT packets (flat decoded structure)
* 2. _ccChannel is NOT set for non-GRP_TXT packets
* 3. Channel color picker palette is 8 colors
* 4. getRowStyle uses border-left only (no background tint)
*/
'use strict';
const vm = require('vm');
const fs = require('fs');
const path = require('path');
let passed = 0;
let failed = 0;
function assert(condition, msg) {
if (condition) {
passed++;
console.log(`${msg}`);
} else {
failed++;
console.error(`${msg}`);
}
}
// --- Test 1: _ccChannel extraction logic (simulates live.js behavior) ---
console.log('\n=== _ccChannel assignment from flat decoded structure ===');
// Simulate the fixed logic from live.js — uses payload.channel (name string),
// NOT payload.channelHash (numeric byte). Channel colors are keyed by channel
// name (e.g. "public", "#test") matching the channels API hash field.
function extractCcChannel(typeName, pkt) {
var _ccPayload = (pkt.decoded || {}).payload || {};
if (typeName === 'GRP_TXT' || typeName === 'CHAN') {
return _ccPayload.channel || null;
}
return undefined; // not set
}
// CHAN with channel name (normal case — ingestor-decrypted WS broadcast)
var chanPkt = {
decoded: {
header: { payloadTypeName: 'CHAN' },
payload: { type: 'CHAN', channel: '#test', channelHash: 217, text: 'hello' }
}
};
assert(extractCcChannel('CHAN', chanPkt) === '#test', 'CHAN with channel="#test" → _ccChannel="#test"');
// CHAN with "public" channel
var publicPkt = {
decoded: {
header: { payloadTypeName: 'CHAN' },
payload: { type: 'CHAN', channel: 'public', text: 'hi' }
}
};
assert(extractCcChannel('CHAN', publicPkt) === 'public', 'CHAN with channel="public" → _ccChannel="public"');
// GRP_TXT without channel (encrypted, no decryption)
var encryptedPkt = {
decoded: {
header: { payloadTypeName: 'GRP_TXT' },
payload: { type: 'GRP_TXT', channelHash: 5, mac: 'ab12', encryptedData: 'ff' }
}
};
assert(extractCcChannel('GRP_TXT', encryptedPkt) === null, 'GRP_TXT without channel field → null');
// Non-GRP_TXT packet — should not set _ccChannel
var advertPkt = {
decoded: {
header: { payloadTypeName: 'ADVERT' },
payload: { type: 'ADVERT', name: 'Node1' }
}
};
assert(extractCcChannel('ADVERT', advertPkt) === undefined, 'ADVERT → _ccChannel not set');
// Empty decoded
var emptyPkt = { decoded: {} };
assert(extractCcChannel('GRP_TXT', emptyPkt) === null, 'GRP_TXT with empty payload → null');
// --- Test 2: _getChannelStyle fix (simulates fixed logic) ---
console.log('\n=== _getChannelStyle with flat structure ===');
function simulateGetChannelStyle(pkt, channelColors) {
var d = pkt.decoded || {};
var h = d.header || {};
var p = d.payload || {};
var ch = p.channel || null;
var typeName = h.payloadTypeName || '';
if (typeName !== 'GRP_TXT' && typeName !== 'CHAN') return '';
if (!ch) return '';
var color = channelColors[ch] || null;
if (!color) return '';
return 'border-left:3px solid ' + color + ';';
}
var colors = { '#test': '#ef4444' };
assert(
simulateGetChannelStyle(chanPkt, colors) === 'border-left:3px solid #ef4444;',
'getChannelStyle returns border-left for assigned color'
);
assert(
simulateGetChannelStyle(chanPkt, {}) === '',
'getChannelStyle returns empty for unassigned channel'
);
assert(
simulateGetChannelStyle(advertPkt, colors) === '',
'getChannelStyle returns empty for non-GRP_TXT'
);
// --- Test 3: channel-colors.js getRowStyle uses border-left only ---
console.log('\n=== channel-colors.js getRowStyle ===');
const ccSource = fs.readFileSync(path.join(__dirname, 'public', 'channel-colors.js'), 'utf8');
const ccCtx = {
window: {},
localStorage: {
_data: {},
getItem(k) { return this._data[k] || null; },
setItem(k, v) { this._data[k] = v; }
}
};
vm.createContext(ccCtx);
vm.runInContext(ccSource, ccCtx);
// Set a color
ccCtx.window.ChannelColors.set('5', '#3b82f6');
var style = ccCtx.window.ChannelColors.getRowStyle('GRP_TXT', '5');
assert(style === 'border-left:3px solid #3b82f6;', 'getRowStyle returns border-left:3px (no background tint)');
assert(!style.includes('background'), 'getRowStyle has no background property');
var noStyle = ccCtx.window.ChannelColors.getRowStyle('GRP_TXT', '99');
assert(noStyle === '', 'getRowStyle returns empty for unassigned channel');
var advertStyle = ccCtx.window.ChannelColors.getRowStyle('ADVERT', '5');
assert(advertStyle === '', 'getRowStyle returns empty for non-GRP_TXT type');
// --- Test 4: channel-color-picker.js palette ---
console.log('\n=== channel-color-picker.js palette ===');
const pickerSource = fs.readFileSync(path.join(__dirname, 'public', 'channel-color-picker.js'), 'utf8');
const pickerCtx = {
window: { ChannelColors: ccCtx.window.ChannelColors, matchMedia: () => ({ matches: false }) },
document: {
createElement: () => ({
className: '', style: {}, innerHTML: '',
setAttribute: () => {},
querySelector: () => ({ textContent: '', style: {}, addEventListener: () => {} }),
querySelectorAll: () => [],
appendChild: () => {},
addEventListener: () => {}
}),
body: { appendChild: () => {}, style: {} },
addEventListener: () => {},
removeEventListener: () => {},
activeElement: null
},
setTimeout: (fn) => fn(),
Array: Array
};
vm.createContext(pickerCtx);
vm.runInContext(pickerSource, pickerCtx);
assert(pickerCtx.window.ChannelColorPicker != null, 'ChannelColorPicker exported');
assert(Array.isArray(pickerCtx.window.ChannelColorPicker.PALETTE), 'PALETTE is exported');
assert(pickerCtx.window.ChannelColorPicker.PALETTE.length === 8, 'PALETTE has exactly 8 colors');
// Verify no teal/rose in palette
var palette = pickerCtx.window.ChannelColorPicker.PALETTE;
assert(!palette.includes('#14b8a6'), 'No teal in palette');
assert(!palette.includes('#f43f5e'), 'No rose in palette');
// --- Summary ---
console.log(`\n${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exit(failed > 0 ? 1 : 0);
+258
View File
@@ -0,0 +1,258 @@
/* Unit tests for channel color highlighting (M1) — #271 */
'use strict';
const vm = require('vm');
const fs = require('fs');
const assert = require('assert');
let passed = 0, failed = 0;
function test(name, fn) {
try {
fn();
passed++;
console.log(`${name}`);
} catch (e) {
failed++;
console.log(`${name}: ${e.message}`);
}
}
// Build minimal sandbox with localStorage mock
function makeSandbox() {
const store = {};
const localStorage = {
getItem: function(k) { return store[k] !== undefined ? store[k] : null; },
setItem: function(k, v) { store[k] = String(v); },
removeItem: function(k) { delete store[k]; },
clear: function() { for (var k in store) delete store[k]; }
};
const ctx = {
window: {},
localStorage: localStorage,
console: console,
JSON: JSON,
};
ctx.window.ChannelColors = undefined;
vm.createContext(ctx);
const src = fs.readFileSync(__dirname + '/public/channel-colors.js', 'utf8');
vm.runInContext(src, ctx);
return ctx;
}
console.log('\n🎨 Channel Colors — Storage CRUD');
test('getChannelColor returns null for unassigned channel', function() {
const ctx = makeSandbox();
assert.strictEqual(ctx.window.ChannelColors.get('#test'), null);
});
test('setChannelColor + getChannelColor round-trip', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#sf', '#ef4444');
assert.strictEqual(ctx.window.ChannelColors.get('#sf'), '#ef4444');
});
test('setChannelColor overwrites existing color', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#sf', '#ef4444');
ctx.window.ChannelColors.set('#sf', '#3b82f6');
assert.strictEqual(ctx.window.ChannelColors.get('#sf'), '#3b82f6');
});
test('removeChannelColor removes assignment', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#test', '#ff0000');
ctx.window.ChannelColors.remove('#test');
assert.strictEqual(ctx.window.ChannelColors.get('#test'), null);
});
test('removeChannelColor on non-existent channel is no-op', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.remove('#nonexistent');
assert.deepStrictEqual(ctx.window.ChannelColors.getAll(), {});
});
test('getAllChannelColors returns all assignments', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#a', '#111111');
ctx.window.ChannelColors.set('#b', '#222222');
const all = ctx.window.ChannelColors.getAll();
assert.strictEqual(JSON.stringify(all), JSON.stringify({ '#a': '#111111', '#b': '#222222' }));
});
test('getAllChannelColors returns empty object when none set', function() {
const ctx = makeSandbox();
assert.strictEqual(JSON.stringify(ctx.window.ChannelColors.getAll()), '{}');
});
test('handles corrupt localStorage gracefully', function() {
const ctx = makeSandbox();
ctx.localStorage.setItem('live-channel-colors', 'not-json{{{');
assert.strictEqual(ctx.window.ChannelColors.get('#test'), null);
assert.strictEqual(JSON.stringify(ctx.window.ChannelColors.getAll()), '{}');
});
test('set with null/empty channel is no-op', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('', '#ff0000');
ctx.window.ChannelColors.set(null, '#ff0000');
assert.strictEqual(JSON.stringify(ctx.window.ChannelColors.getAll()), '{}');
});
test('set rejects invalid hex colors', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#ch', 'red');
ctx.window.ChannelColors.set('#ch', '#xyz');
ctx.window.ChannelColors.set('#ch', '#12345');
ctx.window.ChannelColors.set('#ch', '#1234567');
ctx.window.ChannelColors.set('#ch', 'ff0000');
assert.strictEqual(ctx.window.ChannelColors.get('#ch'), null);
});
test('set normalizes 3-digit hex to 6-digit', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#ch', '#abc');
assert.strictEqual(ctx.window.ChannelColors.get('#ch'), '#aabbcc');
});
test('set accepts valid 6-digit hex', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#ch', '#ef4444');
assert.strictEqual(ctx.window.ChannelColors.get('#ch'), '#ef4444');
});
test('get with null/empty channel returns null', function() {
const ctx = makeSandbox();
assert.strictEqual(ctx.window.ChannelColors.get(''), null);
assert.strictEqual(ctx.window.ChannelColors.get(null), null);
});
console.log('\n🎨 Channel Colors — Row Style Generation');
test('getRowStyle returns empty string for non-GRP_TXT types', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#test', '#ff0000');
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('ADVERT', '#test'), '');
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('TXT_MSG', '#test'), '');
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('ACK', '#test'), '');
});
test('getRowStyle returns empty string for unassigned channel', function() {
const ctx = makeSandbox();
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('GRP_TXT', '#unassigned'), '');
});
test('getRowStyle returns empty string for null channel', function() {
const ctx = makeSandbox();
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('GRP_TXT', null), '');
});
test('getRowStyle returns border + background for assigned GRP_TXT channel', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#sf', '#ef4444');
const style = ctx.window.ChannelColors.getRowStyle('GRP_TXT', '#sf');
assert.ok(style.includes('border-left:4px solid #ef4444'), 'should have left border');
assert.ok(style.includes('background:#ef44441a'), 'should have 10% opacity background');
});
test('getRowStyle works with CHAN type (alias for GRP_TXT)', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#mesh', '#3b82f6');
const style = ctx.window.ChannelColors.getRowStyle('CHAN', '#mesh');
assert.ok(style.includes('border-left:4px solid #3b82f6'), 'should have left border');
assert.ok(style.includes('background:#3b82f61a'), 'should have background tint');
});
test('getRowStyle returns empty when channel has no assigned color', function() {
const ctx = makeSandbox();
ctx.window.ChannelColors.set('#other', '#ff0000');
assert.strictEqual(ctx.window.ChannelColors.getRowStyle('GRP_TXT', '#nope'), '');
});
// ── M2: Channel Color Picker tests ──
test('channel-color-picker.js loads without error in sandbox', function() {
const ctx = makeSandbox();
// Provide minimal DOM stubs for the picker
const elements = {};
const createdEls = [];
ctx.document = {
createElement: function(tag) {
var el = {
tagName: tag.toUpperCase(),
className: '', style: { cssText: '', display: '' },
innerHTML: '', textContent: '', title: '',
children: [],
_attrs: {},
_listeners: {},
setAttribute: function(k, v) { this._attrs[k] = v; },
getAttribute: function(k) { return this._attrs[k] || null; },
addEventListener: function(ev, fn, opts) { this._listeners[ev] = fn; },
removeEventListener: function() {},
appendChild: function(c) { this.children.push(c); return c; },
querySelector: function(sel) {
// Very basic selector matching for test
if (sel === '.cc-picker-swatches') return { addEventListener: function(){}, appendChild: function(c){} };
if (sel === '.cc-picker-apply') return { addEventListener: function(){} };
if (sel === '.cc-picker-clear') return { addEventListener: function(){}, style: {} };
if (sel === '.cc-picker-close') return { addEventListener: function(){} };
if (sel === '.cc-picker-title') return { textContent: '' };
if (sel === '.cc-picker-input') return { value: '#000000' };
return null;
},
querySelectorAll: function() { return []; },
classList: { toggle: function(){}, remove: function(){}, add: function(){} },
contains: function() { return false; },
closest: function() { return null; },
getBoundingClientRect: function() { return { width: 200, height: 200 }; }
};
createdEls.push(el);
return el;
},
getElementById: function() { return null; },
addEventListener: function() {},
removeEventListener: function() {},
body: { appendChild: function(c) {} },
querySelectorAll: function() { return []; }
};
ctx.setTimeout = function(fn) { fn(); };
ctx.window.innerWidth = 1024;
ctx.window.innerHeight = 768;
const pickerSrc = fs.readFileSync(__dirname + '/public/channel-color-picker.js', 'utf8');
vm.runInContext(pickerSrc, ctx);
assert.ok(ctx.window.ChannelColorPicker, 'ChannelColorPicker should be exported');
assert.strictEqual(typeof ctx.window.ChannelColorPicker.install, 'function');
assert.strictEqual(typeof ctx.window.ChannelColorPicker.show, 'function');
assert.strictEqual(typeof ctx.window.ChannelColorPicker.hide, 'function');
});
test('ChannelColorPicker.install does not throw when elements missing', function() {
const ctx = makeSandbox();
ctx.document = {
createElement: function() {
return { className: '', style: {}, innerHTML: '', _attrs: {}, children: [],
setAttribute: function(){}, getAttribute: function(){ return null; },
addEventListener: function(){}, appendChild: function(c){ this.children.push(c); return c; },
querySelector: function(){ return { addEventListener: function(){}, style: {}, textContent: '' }; },
querySelectorAll: function(){ return []; },
getBoundingClientRect: function(){ return {width:0,height:0}; },
contains: function(){ return false; }
};
},
getElementById: function() { return null; },
addEventListener: function() {},
removeEventListener: function() {},
body: { appendChild: function(){} },
querySelectorAll: function() { return []; }
};
ctx.setTimeout = function(fn) { fn(); };
ctx.window.innerWidth = 1024;
ctx.window.innerHeight = 768;
const pickerSrc = fs.readFileSync(__dirname + '/public/channel-color-picker.js', 'utf8');
vm.runInContext(pickerSrc, ctx);
// Should not throw when feed/table elements don't exist
ctx.window.ChannelColorPicker.install();
});
// Summary
console.log(`\n${passed} passed, ${failed} failed\n`);
process.exit(failed ? 1 : 0);
+60 -1
View File
@@ -1488,7 +1488,7 @@ async function run() {
const hasTable = await page.$('#fullNeighborsContent .data-table');
if (hasTable) {
// Check columns
const headers = await page.$$eval('#fullNeighborsContent thead th', ths => ths.map(t => t.textContent));
const headers = await page.$$eval('#fullNeighborsContent thead th', ths => ths.map(t => t.textContent.trim().replace(/\s*[▲▼]\s*$/, '')));
assert(headers.includes('Neighbor'), 'Should have Neighbor column');
assert(headers.includes('Role'), 'Should have Role column');
assert(headers.includes('Score'), 'Should have Score column');
@@ -1627,6 +1627,65 @@ async function run() {
}
} catch {}
// --- Group: Deep linking (#536) ---
// Test: nodes tab deep link
await test('Nodes tab deep link restores active tab', async () => {
await page.goto(BASE + '#/nodes?tab=repeater', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.node-tab', { timeout: 8000 });
const activeTab = await page.$('.node-tab.active');
assert(activeTab, 'No active tab found');
const tabText = await activeTab.textContent();
assert(tabText.includes('Repeater'), `Expected Repeater tab active, got: ${tabText}`);
const url = page.url();
assert(url.includes('tab=repeater'), `URL should contain tab=repeater, got: ${url}`);
});
// Test: nodes tab click updates URL
await test('Nodes tab click updates URL', async () => {
await page.goto(BASE + '#/nodes', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.node-tab', { timeout: 8000 });
const roomTab = await page.$('.node-tab[data-tab="room"]');
assert(roomTab, 'Room tab (data-tab="room") not found — nodes page may not have rendered or tab selector changed');
await roomTab.click();
await page.waitForTimeout(300);
const url = page.url();
assert(url.includes('tab=room'), `URL should contain tab=room after click, got: ${url}`);
});
// Test: packets timeWindow deep link
await test('Packets timeWindow deep link restores dropdown', async () => {
await page.goto(BASE + '#/packets?timeWindow=60', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fTimeWindow', { timeout: 8000 });
const val = await page.$eval('#fTimeWindow', el => el.value);
assert(val === '60', `Expected timeWindow dropdown = 60, got: ${val}`);
const url = page.url();
assert(url.includes('timeWindow=60'), `URL should still contain timeWindow=60, got: ${url}`);
});
// Test: timeWindow change updates URL
await test('Packets timeWindow change updates URL', async () => {
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fTimeWindow', { timeout: 8000 });
await page.selectOption('#fTimeWindow', '30');
await page.waitForTimeout(300);
const url = page.url();
assert(url.includes('timeWindow=30'), `URL should contain timeWindow=30 after change, got: ${url}`);
});
// Test: channels selected channel survives refresh (already implemented, verify it still works)
await test('Channels channel selection is URL-addressable', async () => {
await page.goto(BASE + '#/channels', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.ch-item', { timeout: 8000 }).catch(() => null);
const firstChannel = await page.$('.ch-item');
if (firstChannel) {
await firstChannel.click();
await page.waitForTimeout(500);
const url = page.url();
assert(url.includes('#/channels/') || url.includes('#/channels'), `URL should reflect channel selection, got: ${url}`);
}
});
await browser.close();
// Summary
+499 -70
View File
@@ -75,6 +75,7 @@ function makeSandbox() {
};
})(),
location: { hash: '' },
getHashParams: function() { return new URLSearchParams((ctx.location.hash.split('?')[1] || '')); },
CustomEvent: class CustomEvent {},
Map,
Promise,
@@ -2078,6 +2079,151 @@ console.log('\n=== analytics.js: sortChannels ===');
});
}
// ===== analytics.js: rfNFColumnChart =====
console.log('\n=== analytics.js: rfNFColumnChart ===');
{
function makeAnalyticsSandbox2() {
const ctx = makeSandbox();
ctx.getComputedStyle = () => ({ getPropertyValue: () => '' });
ctx.registerPage = () => {};
ctx.api = () => Promise.resolve({});
ctx.timeAgo = (iso) => iso ? 'x ago' : '—';
ctx.RegionFilter = { init: () => {}, onChange: () => {}, regionQueryString: () => '' };
ctx.onWS = () => {};
ctx.offWS = () => {};
ctx.connectWS = () => {};
ctx.invalidateApiCache = () => {};
ctx.makeColumnsResizable = () => {};
ctx.initTabBar = () => {};
ctx.IATA_COORDS_GEO = {};
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
try { loadInCtx(ctx, 'public/analytics.js'); } catch (e) {
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
}
return ctx;
}
const ctx2 = makeAnalyticsSandbox2();
const rfNFColumnChart = ctx2.window._analyticsRfNFColumnChart;
test('rfNFColumnChart is exposed', () => assert.ok(rfNFColumnChart, '_analyticsRfNFColumnChart must be exposed'));
test('returns SVG string with column bars', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -110 },
{ t: '2024-01-01T00:05:00Z', v: -95 },
{ t: '2024-01-01T00:10:00Z', v: -80 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(svg.includes('<svg'), 'should produce SVG');
assert.ok(svg.includes('class="nf-bar"'), 'should have column bars');
assert.ok(svg.includes('Noise floor column chart'), 'should have aria label');
});
test('color-codes bars by threshold', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -110 }, // green (< -100)
{ t: '2024-01-01T00:05:00Z', v: -95 }, // yellow (-100 to -85)
{ t: '2024-01-01T00:10:00Z', v: -80 }, // red (>= -85)
];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(svg.includes('var(--success'), 'green bar for < -100');
assert.ok(svg.includes('var(--warning'), 'yellow bar for -100 to -85');
assert.ok(svg.includes('var(--danger'), 'red bar for >= -85');
});
test('includes hover tooltips in bars', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -105 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(svg.includes('<title>NF: -105.0 dBm'), 'tooltip with dBm value');
});
test('handles empty data gracefully', () => {
const svg = rfNFColumnChart([], 700, 180, []);
assert.ok(svg.includes('<svg'), 'should return empty SVG');
});
test('handles single data point with visible bar', () => {
const data = [{ t: '2024-01-01T00:00:00Z', v: -100 }];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(svg.includes('class="nf-bar"'), 'should render single bar');
// Bar must have non-zero height (division-by-zero guard)
const m = svg.match(/height="([\d.]+)"/);
assert.ok(m && parseFloat(m[1]) > 0, 'single data point bar must have non-zero height');
assert.ok(!svg.includes('NaN'), 'must not contain NaN');
});
test('handles constant values with visible bars', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -95 },
{ t: '2024-01-01T00:05:00Z', v: -95 },
{ t: '2024-01-01T00:10:00Z', v: -95 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
const heights = [...svg.matchAll(/class="nf-bar"[^>]*height="([\d.]+)"/g)].map(m => parseFloat(m[1]));
assert.strictEqual(heights.length, 3, 'should render 3 bars');
assert.ok(heights.every(h => h > 0), 'all bars must have non-zero height');
assert.ok(!svg.includes('NaN'), 'must not contain NaN');
});
test('includes legend', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -110 },
{ t: '2024-01-01T00:05:00Z', v: -90 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(svg.includes('&lt; -100'), 'legend has green label');
assert.ok(svg.includes('-100…-85'), 'legend has yellow label');
assert.ok(svg.includes('≥ -85'), 'legend has red label');
});
test('no reference lines (removed per spec)', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -110 },
{ t: '2024-01-01T00:05:00Z', v: -80 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
assert.ok(!svg.includes('-100 warning'), 'no -100 warning reference line');
assert.ok(!svg.includes('-85 critical'), 'no -85 critical reference line');
assert.ok(!svg.includes('stroke-dasharray="4,2"'), 'no dashed reference lines');
});
test('renders all bars even with time gaps', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -110 },
{ t: '2024-01-01T06:00:00Z', v: -95 }, // 6h gap
{ t: '2024-01-01T06:05:00Z', v: -80 },
];
const svg = rfNFColumnChart(data, 700, 180, []);
const barCount = (svg.match(/class="nf-bar"/g) || []).length;
assert.strictEqual(barCount, 3, 'all 3 bars rendered despite time gap');
});
test('respects shared time axis', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -100 },
{ t: '2024-01-01T00:05:00Z', v: -95 },
];
const minT = new Date('2023-12-31T00:00:00Z').getTime();
const maxT = new Date('2024-01-02T00:00:00Z').getTime();
const svg = rfNFColumnChart(data, 700, 180, [], minT, maxT);
assert.ok(svg.includes('class="nf-bar"'), 'renders with shared time axis');
});
test('renders reboot markers when reboots provided', () => {
const data = [
{ t: '2024-01-01T00:00:00Z', v: -105 },
{ t: '2024-01-01T01:00:00Z', v: -95 },
];
const reboots = [new Date('2024-01-01T00:30:00Z').getTime()];
const svg = rfNFColumnChart(data, 700, 180, reboots);
assert.ok(svg.includes('reboot'), 'should render reboot marker');
});
}
// ===== CUSTOMIZE-V2.JS: core behavior =====
console.log('\n=== customize-v2.js: core behavior ===');
@@ -2718,7 +2864,6 @@ console.log('\n=== packets.js: savedTimeWindowMin defaults ===');
// ===== Packets page: virtual scroll infrastructure =====
{
console.log('\nPackets page — virtual scroll:');
const packetsSource = fs.readFileSync('public/packets.js', 'utf8');
// --- Behavioral tests using extracted logic ---
@@ -2743,6 +2888,17 @@ console.log('\n=== packets.js: savedTimeWindowMin defaults ===');
return 1 + childCount;
}
// Load _calcVisibleRange from the actual packets.js via sandbox
const pktCtx = makeSandbox();
pktCtx.registerPage = (name, handlers) => {};
pktCtx.onWS = () => {};
pktCtx.offWS = () => {};
pktCtx.api = () => Promise.resolve({});
pktCtx.window.getParsedPath = () => [];
pktCtx.window.getParsedDecoded = () => ({});
loadInCtx(pktCtx, 'public/packets.js');
const _calcVisibleRange = pktCtx.window._packetsTestAPI._calcVisibleRange;
test('cumulativeRowOffsets computes correct offsets for flat rows', () => {
const counts = [1, 1, 1, 1, 1];
const offsets = cumulativeRowOffsets(counts);
@@ -2804,37 +2960,101 @@ console.log('\n=== packets.js: savedTimeWindowMin defaults ===');
assert.strictEqual(getRowCount(p, true, expanded, null), 1);
});
test('renderVisibleRows uses cumulative offsets not flat entry count', () => {
assert.ok(packetsSource.includes('_cumulativeRowOffsets'),
'renderVisibleRows should use cumulative row offsets');
assert.ok(!packetsSource.includes('const totalRows = _displayPackets.length'),
'should NOT use flat array length for total row count');
// --- Behavioral tests for _calcVisibleRange (#405, #409) ---
test('_calcVisibleRange: top of list (scrollTop = 0)', () => {
const offsets = cumulativeRowOffsets([1,1,1,1,1,1,1,1,1,1]); // 10 flat items
const r = _calcVisibleRange(offsets, 10, 0, 360, 36, 0, 2);
assert.strictEqual(r.startIdx, 0, 'start should be 0');
assert.ok(r.endIdx <= 10, 'end should not exceed entry count');
assert.ok(r.endIdx >= 10, 'with buffer=2, should cover visible + buffer');
});
test('renderVisibleRows skips DOM rebuild when range unchanged', () => {
assert.ok(packetsSource.includes('startIdx === _lastVisibleStart && endIdx === _lastVisibleEnd'),
'should skip rebuild when range is unchanged');
test('_calcVisibleRange: middle of list', () => {
// 100 flat items, viewport shows ~10 rows, scroll to row 50
const offsets = cumulativeRowOffsets(new Array(100).fill(1));
const r = _calcVisibleRange(offsets, 100, 50 * 36, 360, 36, 0, 5);
assert.strictEqual(r.firstEntry, 50, 'firstEntry should be 50');
assert.strictEqual(r.startIdx, 45, 'startIdx = firstEntry - buffer');
assert.ok(r.endIdx <= 100);
assert.ok(r.endIdx >= 60, 'endIdx should cover visible + buffer');
});
test('lazy row generation — HTML built only for visible slice', () => {
assert.ok(!packetsSource.includes('_lastRenderedRows'),
'should NOT have pre-built row HTML cache');
assert.ok(packetsSource.includes('_displayPackets.slice(startIdx, endIdx)'),
'should slice display packets for visible range');
assert.ok(packetsSource.includes('visibleSlice.map(p => builder(p))'),
'should build HTML lazily per visible packet');
test('_calcVisibleRange: bottom of list', () => {
const offsets = cumulativeRowOffsets(new Array(100).fill(1));
// Scroll past the end
const r = _calcVisibleRange(offsets, 100, 99 * 36, 360, 36, 0, 5);
assert.strictEqual(r.endIdx, 100, 'endIdx clamped to entry count');
assert.ok(r.startIdx >= 84, 'startIdx should be near end minus buffer');
});
test('observer filter Set is hoisted, not recreated per-packet', () => {
assert.ok(packetsSource.includes('_observerFilterSet = filters.observer ? new Set(filters.observer.split'),
'observer filter Set should be created once in renderTableRows');
assert.ok(packetsSource.includes('_observerFilterSet.has(String(c.observer_id))'),
'buildGroupRowHtml should use hoisted _observerFilterSet');
test('_calcVisibleRange: empty array', () => {
const offsets = cumulativeRowOffsets([]);
const r = _calcVisibleRange(offsets, 0, 0, 360, 36, 0, 5);
assert.strictEqual(r.startIdx, 0);
assert.strictEqual(r.endIdx, 0);
});
test('_calcVisibleRange: single item', () => {
const offsets = cumulativeRowOffsets([1]);
const r = _calcVisibleRange(offsets, 1, 0, 360, 36, 0, 5);
assert.strictEqual(r.startIdx, 0);
assert.strictEqual(r.endIdx, 1);
});
test('_calcVisibleRange: exact row boundary', () => {
const offsets = cumulativeRowOffsets(new Array(20).fill(1));
// scrollTop exactly at row 5 boundary
const r = _calcVisibleRange(offsets, 20, 5 * 36, 360, 36, 0, 2);
assert.strictEqual(r.firstEntry, 5, 'firstEntry at exact boundary');
assert.strictEqual(r.startIdx, 3, 'startIdx = firstEntry - buffer');
});
test('_calcVisibleRange: large dataset (30K items)', () => {
const offsets = cumulativeRowOffsets(new Array(30000).fill(1));
const r = _calcVisibleRange(offsets, 30000, 15000 * 36, 360, 36, 30, 30);
// theadHeight=30 means adjustedScrollTop = 15000*36 - 30, so firstDomRow = floor((540000-30)/36) = 14999
assert.strictEqual(r.firstEntry, 14999);
assert.strictEqual(r.startIdx, 14969);
assert.ok(r.endIdx <= 30000);
assert.ok(r.endIdx >= 15040);
});
test('_calcVisibleRange: various row heights', () => {
const offsets = cumulativeRowOffsets(new Array(50).fill(1));
// rowHeight = 24 instead of 36
const r = _calcVisibleRange(offsets, 50, 10 * 24, 240, 24, 0, 3);
assert.strictEqual(r.firstEntry, 10);
assert.strictEqual(r.startIdx, 7);
});
test('_calcVisibleRange: thead offset shifts visible range', () => {
const offsets = cumulativeRowOffsets(new Array(20).fill(1));
// scrollTop = 40 but theadHeight = 40, so adjustedScrollTop = 0
const r = _calcVisibleRange(offsets, 20, 40, 360, 36, 40, 2);
assert.strictEqual(r.firstEntry, 0, 'thead offset should be subtracted');
});
test('_calcVisibleRange: expanded groups with variable row counts', () => {
// Simulate: item0=1row, item1=5rows(expanded group), item2=1row, item3=3rows, item4=1row
const offsets = cumulativeRowOffsets([1, 5, 1, 3, 1]);
// Scroll to DOM row 6 (in item2), viewport shows 3 DOM rows
const r = _calcVisibleRange(offsets, 5, 6 * 36, 108, 36, 0, 0);
assert.strictEqual(r.firstEntry, 2, 'should land in item2 (offsets[2]=6)');
assert.strictEqual(r.startIdx, 2);
});
test('_calcVisibleRange: buffer clamped at boundaries', () => {
const offsets = cumulativeRowOffsets(new Array(10).fill(1));
// At top with buffer=20 (larger than dataset)
const r = _calcVisibleRange(offsets, 10, 0, 360, 36, 0, 20);
assert.strictEqual(r.startIdx, 0, 'start clamped to 0');
assert.strictEqual(r.endIdx, 10, 'end clamped to entry count');
});
// --- Behavioral tests for observer filter logic (#537) ---
test('observer filter in grouped mode includes packet when child matches (#537)', () => {
// The display filter should keep a grouped packet whose primary observer_id
// does NOT match, but one of its _children does.
const obsIds = new Set(['OBS_B']);
const packets = [
{ observer_id: 'OBS_A', _children: [{ observer_id: 'OBS_A' }, { observer_id: 'OBS_B' }] },
@@ -2873,53 +3093,6 @@ console.log('\n=== packets.js: savedTimeWindowMin defaults ===');
const passes2 = obsSet.has(p2.observer_id) || (p2._children && p2._children.some(c => obsSet.has(String(c.observer_id))));
assert.ok(!passes2, 'WS filter should reject grouped packet with no matching observers');
});
test('packets.js display filter checks _children for observer match (#537)', () => {
// Verify the actual source code has the children check
assert.ok(
packetsSource.includes('p._children) return p._children.some(c => obsIds.has(String(c.observer_id))'),
'display filter should check _children for observer match'
);
});
test('packets.js WS filter checks _children for observer match (#537)', () => {
assert.ok(
packetsSource.includes('p._children && p._children.some(c => obsSet.has(String(c.observer_id)))'),
'WS filter should check _children for observer match'
);
});
test('buildFlatRowHtml has null-safe decoded_json', () => {
const flatBuilderMatch = packetsSource.match(/function buildFlatRowHtml[\s\S]*?(?=\n function )/);
assert.ok(flatBuilderMatch, 'buildFlatRowHtml should exist');
assert.ok(flatBuilderMatch[0].includes('getParsedDecoded(p)'),
'buildFlatRowHtml should use getParsedDecoded for null-safe decoded_json fallback');
});
test('pathHops null guard in buildFlatRowHtml (issue #451)', () => {
const flatBuilderMatch = packetsSource.match(/function buildFlatRowHtml[\s\S]*?(?=\n function )/);
assert.ok(flatBuilderMatch, 'buildFlatRowHtml should exist');
assert.ok(flatBuilderMatch[0].includes('getParsedPath(p)'),
'buildFlatRowHtml should use getParsedPath which guards against null');
});
test('pathHops null guard in detail pane (issue #451)', () => {
assert.ok(packetsSource.includes('getParsedPath(pkt)'),
'detail pane should use getParsedPath for null-safe path parsing');
assert.ok(packetsSource.includes('getParsedDecoded(pkt)'),
'detail pane should use getParsedDecoded for null-safe decoded parsing');
});
test('destroy cleans up virtual scroll state', () => {
assert.ok(packetsSource.includes('detachVScrollListener'),
'destroy should detach virtual scroll listener');
assert.ok(packetsSource.includes("_displayPackets = []"),
'destroy should reset display packets');
assert.ok(packetsSource.includes("_rowCounts = []"),
'destroy should reset row counts');
assert.ok(packetsSource.includes("_lastVisibleStart = -1"),
'destroy should reset visible start');
});
}
// ===== live.js: packetTimestamp =====
@@ -4489,6 +4662,262 @@ console.log('\n=== app.js: routeTypeName/payloadTypeName edge cases ===');
});
}
// ===== REGION-FILTER.JS: setSelected =====
console.log('\n=== region-filter.js: setSelected ===');
{
const ctx = makeSandbox();
ctx.fetch = () => Promise.resolve({ json: () => Promise.resolve({ 'US-SFO': 'San Jose', 'US-LAX': 'Los Angeles' }) });
// Patch createElement to return an object with style property
const origCreate = ctx.document.createElement;
ctx.document.createElement = () => ({
id: '', textContent: '', innerHTML: '',
style: {},
querySelector: () => null,
querySelectorAll: () => [],
onclick: null,
onchange: null,
addEventListener: () => {},
removeEventListener: () => {},
});
loadInCtx(ctx, 'public/region-filter.js');
const RF = ctx.RegionFilter;
test('setSelected sets region codes', async () => {
await RF.init(ctx.document.createElement('div'));
RF.setSelected(['US-SFO', 'US-LAX']);
assert.strictEqual(RF.getRegionParam(), 'US-SFO,US-LAX');
});
test('setSelected with null clears selection', async () => {
await RF.init(ctx.document.createElement('div'));
RF.setSelected(['US-SFO']);
RF.setSelected(null);
assert.strictEqual(RF.getRegionParam(), '');
});
test('setSelected with empty array clears selection', async () => {
await RF.init(ctx.document.createElement('div'));
RF.setSelected(['US-SFO']);
RF.setSelected([]);
assert.strictEqual(RF.getRegionParam(), '');
});
}
// ===== NODES.JS: buildNodesQuery =====
console.log('\n=== nodes.js: buildNodesQuery ===');
{
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
// Provide required globals for nodes.js IIFE to execute
ctx.registerPage = () => {};
ctx.RegionFilter = { init: () => Promise.resolve(), onChange: () => () => {}, offChange: () => {}, getSelected: () => null, getRegionParam: () => '' };
ctx.onWS = () => {};
ctx.offWS = () => {};
ctx.debouncedOnWS = () => () => {};
ctx.invalidateApiCache = () => {};
ctx.favStar = () => '';
ctx.bindFavStars = () => {};
ctx.getFavorites = () => [];
ctx.isFavorite = () => false;
ctx.connectWS = () => {};
ctx.HopResolver = { init: () => {}, resolve: () => ({}), ready: () => false };
ctx.initTabBar = () => {};
ctx.debounce = (fn) => fn;
ctx.copyToClipboard = () => {};
ctx.api = () => Promise.resolve({});
ctx.escapeHtml = (s) => s;
ctx.timeAgo = () => '';
ctx.formatTimestampWithTooltip = () => '';
ctx.getTimestampMode = () => 'ago';
ctx.CLIENT_TTL = {};
ctx.qrcode = null;
try {
const src = fs.readFileSync('public/nodes.js', 'utf8');
vm.runInContext(src, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
} catch (e) {
console.log(' ⚠️ nodes.js sandbox load failed:', e.message.slice(0, 120));
}
const buildNodesQuery = ctx.buildNodesQuery;
if (buildNodesQuery) {
test('buildNodesQuery: all tab + no search = empty', () => {
assert.strictEqual(buildNodesQuery('all', ''), '');
});
test('buildNodesQuery: repeater tab only', () => {
assert.strictEqual(buildNodesQuery('repeater', ''), '?tab=repeater');
});
test('buildNodesQuery: search only (all tab)', () => {
assert.strictEqual(buildNodesQuery('all', 'foo'), '?search=foo');
});
test('buildNodesQuery: tab + search combined', () => {
assert.strictEqual(buildNodesQuery('companion', 'bar'), '?tab=companion&search=bar');
});
test('buildNodesQuery: null search treated as empty', () => {
assert.strictEqual(buildNodesQuery('all', null), '');
});
test('buildNodesQuery: sensor tab', () => {
assert.strictEqual(buildNodesQuery('sensor', ''), '?tab=sensor');
});
} else {
console.log(' ⚠️ buildNodesQuery not exposed — skipping');
}
}
// ===== PACKETS.JS: buildPacketsQuery =====
console.log('\n=== packets.js: buildPacketsQuery ===');
{
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
ctx.registerPage = () => {};
ctx.RegionFilter = { init: () => Promise.resolve(), onChange: () => () => {}, offChange: () => {}, getSelected: () => null, getRegionParam: () => '', setSelected: () => {} };
ctx.onWS = () => {};
ctx.offWS = () => {};
ctx.debouncedOnWS = () => () => {};
ctx.invalidateApiCache = () => {};
ctx.api = () => Promise.resolve({});
ctx.observerMap = new Map();
ctx.getParsedPath = () => [];
ctx.getParsedDecoded = () => ({});
ctx.clearParsedCache = () => {};
ctx.escapeHtml = (s) => s;
ctx.timeAgo = () => '';
ctx.formatTimestampWithTooltip = () => '';
ctx.getTimestampMode = () => 'ago';
ctx.copyToClipboard = () => {};
ctx.CLIENT_TTL = {};
ctx.debounce = (fn) => fn;
ctx.initTabBar = () => {};
try {
const src = fs.readFileSync('public/packet-helpers.js', 'utf8');
vm.runInContext(src, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
const src2 = fs.readFileSync('public/packets.js', 'utf8');
vm.runInContext(src2, ctx);
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
} catch (e) {
console.log(' ⚠️ packets.js sandbox load failed:', e.message.slice(0, 120));
}
const buildPacketsQuery = ctx.buildPacketsQuery;
if (buildPacketsQuery) {
test('buildPacketsQuery: default (15min, no region) = empty string', () => {
assert.strictEqual(buildPacketsQuery(15, ''), '');
});
test('buildPacketsQuery: non-default timeWindow', () => {
assert.strictEqual(buildPacketsQuery(60, ''), '?timeWindow=60');
});
test('buildPacketsQuery: region only', () => {
assert.strictEqual(buildPacketsQuery(15, 'US-SFO'), '?region=US-SFO');
});
test('buildPacketsQuery: timeWindow + region', () => {
assert.strictEqual(buildPacketsQuery(30, 'US-SFO,US-LAX'), '?timeWindow=30&region=US-SFO%2CUS-LAX');
});
test('buildPacketsQuery: timeWindow=0 treated as default', () => {
assert.strictEqual(buildPacketsQuery(0, ''), '');
});
} else {
console.log(' ⚠️ buildPacketsQuery not exposed — skipping');
}
}
// ===== APP.JS: formatDistance / getDistanceUnit =====
console.log('\n=== app.js: formatDistance ===');
{
function makeDistCtx(localeLang, storageUnit) {
const ctx = makeSandbox();
if (storageUnit !== undefined) ctx.localStorage.setItem('meshcore-distance-unit', storageUnit);
ctx.navigator = { language: localeLang || 'en-BE' };
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
return ctx;
}
test('formatDistance: km mode, 12.3 km', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance(12.3), '12.3 km');
});
test('formatDistance: km mode, sub-1km shows meters', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance(0.45), '450 m');
});
test('formatDistance: mi mode, 12.3 km → 7.6 mi', () => {
const ctx = makeDistCtx('en-BE', 'mi');
assert.strictEqual(ctx.formatDistance(12.3), '7.6 mi');
});
test('formatDistance: auto + en-US locale → mi', () => {
const ctx = makeDistCtx('en-US', 'auto');
assert.strictEqual(ctx.getDistanceUnit(), 'mi');
});
test('formatDistance: auto + en-GB locale → mi', () => {
const ctx = makeDistCtx('en-GB', 'auto');
assert.strictEqual(ctx.getDistanceUnit(), 'mi');
});
test('formatDistance: auto + fr-BE locale → km', () => {
const ctx = makeDistCtx('fr-BE', 'auto');
assert.strictEqual(ctx.getDistanceUnit(), 'km');
});
test('formatDistance: null input returns —', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance(null), '—');
});
test('formatDistanceRound: 50 km → "50 km"', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistanceRound(50), '50 km');
});
test('formatDistanceRound: 50 km in mi mode → "31 mi"', () => {
const ctx = makeDistCtx('en-BE', 'mi');
assert.strictEqual(ctx.formatDistanceRound(50), '31 mi');
});
test('formatDistanceRound: 200 km in mi mode → "124 mi"', () => {
const ctx = makeDistCtx('en-BE', 'mi');
assert.strictEqual(ctx.formatDistanceRound(200), '124 mi');
});
test('formatDistance: 0 in km mode → "0 m"', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance(0), '0 m');
});
test('formatDistance: 0 in mi mode → "0 ft"', () => {
const ctx = makeDistCtx('en-BE', 'mi');
assert.strictEqual(ctx.formatDistance(0), '0 ft');
});
test('formatDistance: NaN input returns —', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance(NaN), '—');
});
test('formatDistance: "abc" input returns —', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistance('abc'), '—');
});
test('formatDistanceRound: null input returns —', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistanceRound(null), '—');
});
test('formatDistanceRound: NaN input returns —', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistanceRound(NaN), '—');
});
test('formatDistanceRound: 0 in km mode → "0 km"', () => {
const ctx = makeDistCtx('en-BE', 'km');
assert.strictEqual(ctx.formatDistanceRound(0), '0 km');
});
test('formatDistance: mi mode sub-0.1mi shows feet', () => {
const ctx = makeDistCtx('en-BE', 'mi');
assert.strictEqual(ctx.formatDistance(0.01), '33 ft');
});
}
// ===== SUMMARY =====
Promise.allSettled(pendingTests).then(() => {
console.log(`\n${'═'.repeat(40)}`);
+61 -1
View File
@@ -107,6 +107,7 @@ function loadPacketsSandbox() {
// Load dependencies first
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
loadInCtx(ctx, 'public/packet-helpers.js');
// HopDisplay stub (simpler than loading real file which may have DOM deps)
vm.runInContext(`
window.HopDisplay = {
@@ -628,10 +629,19 @@ console.log('\n=== packets.js: buildFieldTable ===');
});
test('buildFieldTable hash_size calculation', () => {
// Path byte 0xC0 → bits 7-6 = 3 → hash_size = 4
// Path byte 0xC0 → bits 7-6 = 3 → hash_size = 4, but hash_count = 0
// Since #653: when hashCount == 0, shows "hash_count=0 (direct advert)" instead of hash_size
const pkt = { raw_hex: '00C0', route_type: 1, payload_type: 0 };
const decoded = {};
const result = api.buildFieldTable(pkt, decoded, [], []);
assert(result.includes('hash_count=0 (direct advert)'));
});
test('buildFieldTable hash_size shown when hash_count > 0', () => {
// Path byte 0xC1 → bits 7-6 = 3 → hash_size = 4, hash_count = 1
const pkt = { raw_hex: '00C1aabbccdd', route_type: 1, payload_type: 0 };
const decoded = {};
const result = api.buildFieldTable(pkt, decoded, [], []);
assert(result.includes('hash_size=4'));
});
@@ -695,6 +705,26 @@ console.log('\n=== packets.js: buildFlatRowHtml ===');
const result = api.buildFlatRowHtml(p);
assert(result.includes('0B'));
});
test('buildFlatRowHtml emits data-entry-idx when provided', () => {
const p = {
id: 4, hash: 'z', timestamp: '', observer_id: null,
raw_hex: 'aabb', payload_type: 0, route_type: 0,
decoded_json: '{}', path_json: '[]'
};
const result = api.buildFlatRowHtml(p, 42);
assert(result.includes('data-entry-idx="42"'));
});
test('buildFlatRowHtml emits data-entry-idx=-1 by default', () => {
const p = {
id: 5, hash: 'w', timestamp: '', observer_id: null,
raw_hex: 'aabb', payload_type: 0, route_type: 0,
decoded_json: '{}', path_json: '[]'
};
const result = api.buildFlatRowHtml(p);
assert(result.includes('data-entry-idx="-1"'));
});
}
console.log('\n=== packets.js: buildGroupRowHtml ===');
@@ -740,6 +770,36 @@ console.log('\n=== packets.js: buildGroupRowHtml ===');
assert(result.includes('👁'));
assert(result.includes('5'));
});
test('buildGroupRowHtml emits data-entry-idx on header row', () => {
const p = {
hash: 'ei1', count: 1, latest: '2024-01-01T00:00:00Z',
observer_id: null, raw_hex: 'aa', payload_type: 0,
route_type: 0, decoded_json: '{}', path_json: '[]',
observation_count: 1, observer_count: 1
};
const result = api.buildGroupRowHtml(p, 7);
assert(result.includes('data-entry-idx="7"'));
});
test('buildGroupRowHtml emits data-entry-idx on child rows', () => {
const ctx2 = loadPacketsSandbox();
const api2 = ctx2._packetsTestAPI;
// Simulate expandedHashes having this hash
// We can't easily toggle expandedHashes from outside, so test via the
// fact that children only render when isExpanded is true.
// For this test, just verify the header row has the attribute (child rows
// are conditional on expandedHashes which we can't set from tests).
const p = {
hash: 'ei2', count: 3, latest: '2024-01-01T00:00:00Z',
observer_id: null, raw_hex: 'aabb', payload_type: 0,
route_type: 0, decoded_json: '{}', path_json: '[]',
observation_count: 3, observer_count: 2,
_children: []
};
const result = api2.buildGroupRowHtml(p, 15);
assert(result.includes('data-entry-idx="15"'));
});
}
console.log('\n=== packets.js: page registration ===');
+334
View File
@@ -0,0 +1,334 @@
/**
* Tests for panel corner positioning (#608 M0)
* Tests the pure logic functions extracted from live.js
*/
'use strict';
const assert = require('assert');
const vm = require('vm');
const fs = require('fs');
const path = require('path');
// Minimal DOM/browser stubs
function createContext() {
const storage = {};
const elements = {};
const listeners = {};
const mockEl = () => ({
style: {}, textContent: '', innerHTML: '',
classList: { add(){}, remove(){}, toggle(){}, contains(){ return false; } },
appendChild(c){ return c; }, removeChild(){ }, insertBefore(c){ return c; },
setAttribute(){}, getAttribute(){ return null; }, removeAttribute(){},
addEventListener(){}, removeEventListener(){},
querySelector(){ return null; }, querySelectorAll(){ return []; },
getBoundingClientRect(){ return {top:0,left:0,right:0,bottom:0,width:0,height:0}; },
closest(){ return null; }, matches(){ return false; },
children: [], childNodes: [], parentNode: null, parentElement: null,
focus(){}, blur(){}, click(){}, scrollTo(){},
dataset: {}, offsetWidth: 0, offsetHeight: 0,
getContext(){ return { clearRect(){}, fillRect(){}, beginPath(){}, moveTo(){}, lineTo(){}, stroke(){}, fill(){}, arc(){}, save(){}, restore(){}, translate(){}, rotate(){}, scale(){}, drawImage(){}, measureText(){ return {width:0}; }, createLinearGradient(){ return {addColorStop(){}}; }, canvas: {width:0,height:0} }; },
width: 0, height: 0,
});
const ctx = {
window: {},
document: {
getElementById: (id) => elements[id] || null,
querySelectorAll: (sel) => {
const results = [];
for (const id in elements) {
const el = elements[id];
if (el._btns) results.push(...el._btns);
}
return results;
},
querySelector: () => null,
documentElement: { getAttribute: () => null, style: {} },
addEventListener: () => {},
createElement: () => mockEl(),
createElementNS: () => mockEl(),
createTextNode: (t) => ({ textContent: t }),
createDocumentFragment: () => ({ appendChild(){}, children: [] }),
body: { appendChild(){}, removeChild(){}, style: {}, classList: { add(){}, remove(){} } },
head: { appendChild(){} },
},
localStorage: {
getItem: (k) => storage[k] !== undefined ? storage[k] : null,
setItem: (k, v) => { storage[k] = String(v); },
removeItem: (k) => { delete storage[k]; }
},
_storage: storage,
_elements: elements,
_addElement: function(id) {
const attrs = {};
const btns = [];
elements[id] = {
setAttribute: (k, v) => { attrs[k] = v; },
getAttribute: (k) => attrs[k] || null,
querySelector: (sel) => {
if (sel === '.panel-corner-btn') return btns[0] || null;
return null;
},
_attrs: attrs,
_btns: btns,
_addBtn: function(panelId) {
const btnAttrs = { 'data-panel': panelId };
const btn = {
textContent: '',
setAttribute: (k, v) => { btnAttrs[k] = v; },
getAttribute: (k) => btnAttrs[k] || null,
addEventListener: () => {},
_attrs: btnAttrs
};
btns.push(btn);
return btn;
}
};
return elements[id];
}
};
// Self-references
ctx.window = ctx;
ctx.self = ctx;
return ctx;
}
function loadLiveModule(ctx) {
// Load the REAL live.js in a VM context and return window._panelCorner.
// This tests the actual code, not a copy (per AGENTS.md "test the real code, not copies").
const src = fs.readFileSync(path.join(__dirname, 'public', 'live.js'), 'utf8');
// Minimal stubs for live.js dependencies (only what's needed to avoid errors)
ctx.registerPage = () => {};
ctx.escapeHtml = (s) => String(s || '');
ctx.timeAgo = () => '—';
ctx.getParsedPath = () => [];
ctx.getParsedDecoded = () => ({});
ctx.TYPE_COLORS = { ADVERT: '#22c55e', GRP_TXT: '#3b82f6', TXT_MSG: '#f59e0b', ACK: '#6b7280', REQUEST: '#a855f7', RESPONSE: '#06b6d4', TRACE: '#ec4899', PATH: '#14b8a6' };
ctx.ROLE_COLORS = {};
ctx.ROLE_LABELS = {};
ctx.ROLE_STYLE = {};
ctx.ROLE_SORT = [];
ctx.formatTimestampWithTooltip = () => '';
ctx.getTimestampMode = () => 'relative';
ctx.console = console;
ctx.setTimeout = setTimeout;
ctx.clearTimeout = clearTimeout;
ctx.setInterval = setInterval;
ctx.clearInterval = clearInterval;
ctx.requestAnimationFrame = (cb) => setTimeout(cb, 0);
ctx.cancelAnimationFrame = clearTimeout;
ctx.matchMedia = () => ({ matches: false, addEventListener: () => {} });
ctx.navigator = { userAgent: '' };
ctx.performance = { now: () => Date.now() };
ctx.L = undefined;
ctx.MutationObserver = class { observe() {} disconnect() {} };
ctx.ResizeObserver = class { observe() {} disconnect() {} };
ctx.IntersectionObserver = class { observe() {} disconnect() {} };
ctx.Image = class {};
ctx.AudioContext = undefined;
ctx.HTMLElement = class {};
ctx.Event = class {};
ctx.fetch = () => Promise.resolve({ ok: true, json: () => Promise.resolve([]) });
ctx.Number = Number; ctx.String = String; ctx.Array = Array; ctx.Object = Object;
ctx.JSON = JSON; ctx.Math = Math; ctx.Date = Date; ctx.RegExp = RegExp;
ctx.Error = Error; ctx.Map = Map; ctx.Set = Set; ctx.WeakMap = WeakMap;
ctx.parseInt = parseInt; ctx.parseFloat = parseFloat;
ctx.isNaN = isNaN; ctx.isFinite = isFinite;
ctx.encodeURIComponent = encodeURIComponent;
ctx.decodeURIComponent = decodeURIComponent;
ctx.Promise = Promise; ctx.Symbol = Symbol;
ctx.queueMicrotask = queueMicrotask;
// Self-references needed for the IIFE
ctx.self = ctx;
ctx.globalThis = ctx;
vm.createContext(ctx);
vm.runInContext(src, ctx, { timeout: 3000 });
return ctx.window._panelCorner;
}
// ---- Tests ----
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
passed++;
console.log(' ✓ ' + name);
} catch (e) {
failed++;
console.log(' ✗ ' + name);
console.log(' ' + e.message);
}
}
console.log('\nPanel Corner Positioning Tests (#608 M0)\n');
// --- nextAvailableCorner ---
console.log('nextAvailableCorner:');
test('returns desired corner when available', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const positions = { liveFeed: 'bl', liveLegend: 'br', liveNodeDetail: 'tr' };
assert.strictEqual(pc.nextAvailableCorner('liveFeed', 'tl', positions), 'tl');
});
test('skips occupied corner', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const positions = { liveFeed: 'bl', liveLegend: 'br', liveNodeDetail: 'tr' };
// liveFeed wants 'tr' but liveNodeDetail is there → should get 'br'? No, liveLegend is at br → skip to bl? No liveFeed is at bl → skip to tl
assert.strictEqual(pc.nextAvailableCorner('liveFeed', 'tr', positions), 'bl');
// Wait — liveFeed IS liveFeed, so bl is not occupied by "another" panel
// Actually liveFeed wants tr → tr occupied by nodeDetail → try br → occupied by legend → try bl → that's liveFeed itself (excluded from "occupied") → bl is free
});
test('skips multiple occupied corners', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const positions = { liveFeed: 'tl', liveLegend: 'tr', liveNodeDetail: 'br' };
// liveFeed wants 'tr' → occupied by legend → try 'br' → occupied by nodeDetail → try 'bl' → free
assert.strictEqual(pc.nextAvailableCorner('liveFeed', 'tr', positions), 'bl');
});
test('returns desired when only self occupies it', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const positions = { liveFeed: 'bl', liveLegend: 'br', liveNodeDetail: 'tr' };
// liveFeed wants bl — it's "occupied" by liveFeed itself, which is excluded
assert.strictEqual(pc.nextAvailableCorner('liveFeed', 'bl', positions), 'bl');
});
// --- getPanelPositions ---
console.log('\ngetPanelPositions:');
test('returns defaults when nothing in localStorage', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const pos = pc.getPanelPositions();
assert.strictEqual(pos.liveFeed, 'bl');
assert.strictEqual(pos.liveLegend, 'br');
assert.strictEqual(pos.liveNodeDetail, 'tr');
});
test('returns saved positions from localStorage', () => {
const ctx = createContext();
ctx.localStorage.setItem('panel-corner-liveFeed', 'tl');
ctx.localStorage.setItem('panel-corner-liveLegend', 'bl');
const pc = loadLiveModule(ctx);
const pos = pc.getPanelPositions();
assert.strictEqual(pos.liveFeed, 'tl');
assert.strictEqual(pos.liveLegend, 'bl');
assert.strictEqual(pos.liveNodeDetail, 'tr'); // still default
});
// --- applyPanelPosition ---
console.log('\napplyPanelPosition:');
test('sets data-position attribute on element', () => {
const ctx = createContext();
const el = ctx._addElement('liveFeed');
el._addBtn('liveFeed');
const pc = loadLiveModule(ctx);
pc.applyPanelPosition('liveFeed', 'tr');
assert.strictEqual(el._attrs['data-position'], 'tr');
});
test('updates button text and aria-label', () => {
const ctx = createContext();
const el = ctx._addElement('liveFeed');
const btn = el._addBtn('liveFeed');
const pc = loadLiveModule(ctx);
pc.applyPanelPosition('liveFeed', 'tr');
assert.strictEqual(btn.textContent, '↙');
assert.ok(btn._attrs['aria-label'].includes('top-right'));
});
test('handles missing element gracefully', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
// Should not throw
pc.applyPanelPosition('nonexistent', 'tl');
});
// --- onCornerClick ---
console.log('\nonCornerClick:');
test('cycles from default bl to tl for feed', () => {
const ctx = createContext();
const el = ctx._addElement('liveFeed');
el._addBtn('liveFeed');
ctx._addElement('liveLegend');
ctx._addElement('liveNodeDetail');
ctx._addElement('panelPositionAnnounce');
ctx._elements.panelPositionAnnounce.textContent = '';
const pc = loadLiveModule(ctx);
// Feed defaults to bl, cycle: bl → tl (next in cycle after bl is tl)
pc.onCornerClick('liveFeed');
assert.strictEqual(ctx._storage['panel-corner-liveFeed'], 'tl');
assert.strictEqual(el._attrs['data-position'], 'tl');
});
test('collision avoidance: skips occupied corner', () => {
const ctx = createContext();
ctx._addElement('liveFeed');
const legendEl = ctx._addElement('liveLegend');
legendEl._addBtn('liveLegend');
ctx._addElement('liveNodeDetail');
ctx._addElement('panelPositionAnnounce');
ctx._elements.panelPositionAnnounce.textContent = '';
const pc = loadLiveModule(ctx);
// Legend defaults to br. Click → next is bl. But bl is occupied by feed → skip to tl
pc.onCornerClick('liveLegend');
assert.strictEqual(ctx._storage['panel-corner-liveLegend'], 'tl');
});
// --- resetPanelPositions ---
console.log('\nresetPanelPositions:');
test('clears localStorage and restores defaults', () => {
const ctx = createContext();
ctx.localStorage.setItem('panel-corner-liveFeed', 'tr');
ctx.localStorage.setItem('panel-corner-liveLegend', 'tl');
const feedEl = ctx._addElement('liveFeed');
feedEl._addBtn('liveFeed');
const legendEl = ctx._addElement('liveLegend');
legendEl._addBtn('liveLegend');
const detailEl = ctx._addElement('liveNodeDetail');
detailEl._addBtn('liveNodeDetail');
const pc = loadLiveModule(ctx);
pc.resetPanelPositions();
assert.strictEqual(ctx._storage['panel-corner-liveFeed'], undefined);
assert.strictEqual(feedEl._attrs['data-position'], 'bl');
assert.strictEqual(legendEl._attrs['data-position'], 'br');
assert.strictEqual(detailEl._attrs['data-position'], 'tr');
});
// --- Corner cycle order ---
console.log('\nCorner cycle order:');
test('full cycle: tl → tr → br → bl → tl', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
const cycle = pc.CORNER_CYCLE;
assert.strictEqual(cycle.join(','), 'tl,tr,br,bl');
});
test('defaults match expected panel positions', () => {
const ctx = createContext();
const pc = loadLiveModule(ctx);
assert.strictEqual(pc.PANEL_DEFAULTS.liveFeed, 'bl');
assert.strictEqual(pc.PANEL_DEFAULTS.liveLegend, 'br');
assert.strictEqual(pc.PANEL_DEFAULTS.liveNodeDetail, 'tr');
});
// Summary
console.log('\n' + (passed + failed) + ' tests, ' + passed + ' passed, ' + failed + ' failed\n');
process.exit(failed > 0 ? 1 : 0);
+356
View File
@@ -0,0 +1,356 @@
/* test-table-sort.js — Unit tests for TableSort utility */
'use strict';
const { JSDOM } = require('jsdom');
const fs = require('fs');
const path = require('path');
const assert = require('assert');
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
passed++;
console.log(`${name}`);
} catch (e) {
failed++;
console.log(`${name}`);
console.log(` ${e.message}`);
}
}
function createDOM(html) {
const dom = new JSDOM(`<!DOCTYPE html><html><body>${html}</body></html>`, {
url: 'http://localhost',
runScripts: 'dangerously'
});
// Load TableSort into this DOM
const script = fs.readFileSync(path.join(__dirname, 'public', 'table-sort.js'), 'utf8');
const el = dom.window.document.createElement('script');
el.textContent = script;
dom.window.document.head.appendChild(el);
return dom;
}
function makeTable(headers, rows) {
// headers: [{key, type?, label}], rows: [[value, ...]]
let html = '<table id="t"><thead><tr>';
for (const h of headers) {
html += `<th data-sort-key="${h.key}"${h.type ? ` data-type="${h.type}"` : ''}>${h.label || h.key}</th>`;
}
html += '</tr></thead><tbody>';
for (const row of rows) {
html += '<tr>';
for (let i = 0; i < row.length; i++) {
const val = row[i];
if (typeof val === 'object' && val !== null) {
html += `<td data-value="${val.dataValue}">${val.text || ''}</td>`;
} else {
html += `<td data-value="${val}">${val}</td>`;
}
}
html += '</tr>';
}
html += '</tbody></table>';
return html;
}
function getColumnValues(dom, colIndex) {
const rows = dom.window.document.querySelectorAll('tbody tr');
return Array.from(rows).map(r => r.cells[colIndex].getAttribute('data-value'));
}
console.log('\nTableSort — comparators');
test('text comparator: basic alphabetical', () => {
const cmp = (() => {
const dom = createDOM('<div></div>');
return dom.window.TableSort.comparators.text;
})();
assert.ok(cmp('apple', 'banana') < 0);
assert.ok(cmp('banana', 'apple') > 0);
assert.strictEqual(cmp('same', 'same'), 0);
});
test('text comparator: null/undefined handling', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.text;
assert.strictEqual(cmp(null, null), 0);
assert.strictEqual(cmp(undefined, undefined), 0);
});
test('numeric comparator: basic numbers', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.numeric;
assert.ok(cmp('1', '2') < 0);
assert.ok(cmp('10', '2') > 0);
assert.strictEqual(cmp('5', '5'), 0);
});
test('numeric comparator: NaN sorts last', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.numeric;
assert.ok(cmp('abc', '5') > 0); // NaN > number (sorts last)
assert.ok(cmp('5', 'abc') < 0);
assert.strictEqual(cmp('abc', 'xyz'), 0); // both NaN
});
test('numeric comparator: negative numbers', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.numeric;
assert.ok(cmp('-10', '-5') < 0);
assert.ok(cmp('-5', '-10') > 0);
});
test('date comparator: ISO dates', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.date;
assert.ok(cmp('2024-01-01T00:00:00Z', '2024-06-01T00:00:00Z') < 0);
assert.ok(cmp('2024-06-01T00:00:00Z', '2024-01-01T00:00:00Z') > 0);
assert.strictEqual(cmp('2024-01-01', '2024-01-01'), 0);
});
test('date comparator: invalid dates sort last', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.date;
assert.ok(cmp('invalid', '2024-01-01') > 0);
assert.ok(cmp('2024-01-01', 'invalid') < 0);
});
test('dBm comparator: strips suffix', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.dbm;
assert.ok(cmp('-120 dBm', '-80 dBm') < 0);
assert.ok(cmp('-80 dBm', '-120 dBm') > 0);
assert.strictEqual(cmp('-95 dBm', '-95 dBm'), 0);
});
test('dBm comparator: works without suffix', () => {
const dom = createDOM('<div></div>');
const cmp = dom.window.TableSort.comparators.dbm;
assert.ok(cmp('-120', '-80') < 0);
});
console.log('\nTableSort — DOM sorting');
test('sort ascending by text column', () => {
const html = makeTable(
[{key: 'name'}],
[['Charlie'], ['Alice'], ['Bob']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
const inst = dom.window.TableSort.init(table, { defaultColumn: 'name', defaultDirection: 'asc' });
const vals = getColumnValues(dom, 0);
assert.deepStrictEqual(vals, ['Alice', 'Bob', 'Charlie']);
});
test('sort descending by numeric column', () => {
const html = makeTable(
[{key: 'val', type: 'numeric'}],
[['3'], ['1'], ['2']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'val', defaultDirection: 'desc' });
const vals = getColumnValues(dom, 0);
assert.deepStrictEqual(vals, ['3', '2', '1']);
});
test('click toggles direction', () => {
const html = makeTable(
[{key: 'name'}],
[['B'], ['A'], ['C']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
const inst = dom.window.TableSort.init(table, { defaultColumn: 'name', defaultDirection: 'asc' });
// Initially ascending
assert.deepStrictEqual(getColumnValues(dom, 0), ['A', 'B', 'C']);
// Click same header → descending
const th = dom.window.document.querySelector('th[data-sort-key="name"]');
th.click();
assert.deepStrictEqual(getColumnValues(dom, 0), ['C', 'B', 'A']);
// Click again → ascending
th.click();
assert.deepStrictEqual(getColumnValues(dom, 0), ['A', 'B', 'C']);
});
console.log('\nTableSort — aria-sort attributes');
test('aria-sort set correctly on active column', () => {
const html = makeTable(
[{key: 'a'}, {key: 'b'}],
[['1', 'x'], ['2', 'y']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
const thA = dom.window.document.querySelector('th[data-sort-key="a"]');
const thB = dom.window.document.querySelector('th[data-sort-key="b"]');
assert.strictEqual(thA.getAttribute('aria-sort'), 'ascending');
assert.strictEqual(thB.getAttribute('aria-sort'), 'none');
});
test('aria-sort updates on direction change', () => {
const html = makeTable(
[{key: 'a'}],
[['1'], ['2']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
const th = dom.window.document.querySelector('th[data-sort-key="a"]');
assert.strictEqual(th.getAttribute('aria-sort'), 'ascending');
th.click(); // toggle to desc
assert.strictEqual(th.getAttribute('aria-sort'), 'descending');
});
test('aria-sort updates when switching columns', () => {
const html = makeTable(
[{key: 'a'}, {key: 'b'}],
[['1', 'x'], ['2', 'y']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
const thB = dom.window.document.querySelector('th[data-sort-key="b"]');
thB.click(); // switch to column b
const thA = dom.window.document.querySelector('th[data-sort-key="a"]');
assert.strictEqual(thA.getAttribute('aria-sort'), 'none');
assert.strictEqual(thB.getAttribute('aria-sort'), 'ascending');
});
console.log('\nTableSort — visual indicator');
test('sort arrow shows on active column', () => {
const html = makeTable(
[{key: 'a'}],
[['1'], ['2']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
const arrow = dom.window.document.querySelector('.sort-arrow');
assert.ok(arrow, 'sort arrow should exist');
assert.ok(arrow.textContent.includes('▲'), 'ascending should show ▲');
});
test('sort arrow changes on direction toggle', () => {
const html = makeTable(
[{key: 'a'}],
[['1'], ['2']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
const th = dom.window.document.querySelector('th[data-sort-key="a"]');
th.click(); // desc
const arrow = dom.window.document.querySelector('.sort-arrow');
assert.ok(arrow.textContent.includes('▼'), 'descending should show ▼');
});
console.log('\nTableSort — onSort callback');
test('onSort fires with column and direction', () => {
const html = makeTable(
[{key: 'a'}, {key: 'b'}],
[['1', 'x'], ['2', 'y']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
let called = null;
dom.window.TableSort.init(table, {
domReorder: false,
onSort: function(col, dir) { called = { col, dir }; }
});
const th = dom.window.document.querySelector('th[data-sort-key="a"]');
th.click();
assert.ok(called, 'onSort should fire');
assert.strictEqual(called.col, 'a');
assert.strictEqual(called.dir, 'asc');
});
console.log('\nTableSort — domReorder: false');
test('domReorder: false skips DOM sorting', () => {
const html = makeTable(
[{key: 'name'}],
[['C'], ['A'], ['B']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'name', defaultDirection: 'asc', domReorder: false });
// DOM order should NOT change
const vals = getColumnValues(dom, 0);
assert.deepStrictEqual(vals, ['C', 'A', 'B']);
});
console.log('\nTableSort — destroy');
test('destroy removes event handlers and cleans up', () => {
const html = makeTable(
[{key: 'a'}],
[['2'], ['1']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
const inst = dom.window.TableSort.init(table, { defaultColumn: 'a', defaultDirection: 'asc' });
inst.destroy();
const th = dom.window.document.querySelector('th[data-sort-key="a"]');
assert.strictEqual(th.getAttribute('aria-sort'), null, 'aria-sort should be removed');
assert.ok(!th.classList.contains('sort-active'), 'sort-active should be removed');
assert.strictEqual(th.querySelector('.sort-arrow'), null, 'arrow should be removed');
});
console.log('\nTableSort — custom comparators');
test('custom comparator overrides built-in', () => {
const html = makeTable(
[{key: 'val', type: 'numeric'}],
[['3'], ['1'], ['2']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
// Custom: reverse numeric
dom.window.TableSort.init(table, {
defaultColumn: 'val', defaultDirection: 'asc',
comparators: { val: function(a, b) { return Number(b) - Number(a); } }
});
const vals = getColumnValues(dom, 0);
assert.deepStrictEqual(vals, ['3', '2', '1']); // reversed
});
console.log('\nTableSort — date sort with data-type="date"');
test('date column sorts correctly', () => {
const html = makeTable(
[{key: 'ts', type: 'date'}],
[['2024-06-15T10:00:00Z'], ['2024-01-01T00:00:00Z'], ['2024-12-25T23:59:59Z']]
);
const dom = createDOM(html);
const table = dom.window.document.getElementById('t');
dom.window.TableSort.init(table, { defaultColumn: 'ts', defaultDirection: 'asc' });
const vals = getColumnValues(dom, 0);
assert.deepStrictEqual(vals, ['2024-01-01T00:00:00Z', '2024-06-15T10:00:00Z', '2024-12-25T23:59:59Z']);
});
// Summary
console.log(`\n${passed + failed} tests, ${passed} passed, ${failed} failed\n`);
process.exit(failed > 0 ? 1 : 0);