Compare commits

...
Author SHA1 Message Date
you e837a9d416 fix: Details button opens full-screen node detail on desktop (#778)
The previous fix (#779) used hash navigation for the Details button,
which only re-selected the node in the side panel on desktop. Now the
Details button calls showFullScreenNode() to render the same full-screen
detail view that mobile users see, regardless of screen width.

- Extract showFullScreenNode() from mobile-only init code
- Reuse it for both mobile init and desktop Details button click
- Analytics links still navigate via hash as before
- Update Playwright test to verify full-screen view renders
2026-04-17 07:41:14 +00:00
dfe383cc51 fix: node detail panel Details/Analytics links don't navigate (#779)
Fixes #778

## Problem

The Details and Analytics links in the node side panel don't navigate
when clicked. This is a regression from #739 (desktop node deep
linking).

**Root cause:** When a node is selected, `selectNode()` uses
`history.replaceState()` to set the URL to `#/nodes/{pubkey}`. The
Details link has `href="#/nodes/{pubkey}"` — the same hash. Clicking an
anchor with the same hash as the current URL doesn't fire the
`hashchange` event, so the SPA router never triggers navigation.

## Fix

Added a click handler on the `nodesRight` panel that intercepts clicks
on `.btn-primary` navigation links:

1. `e.preventDefault()` to stop the default anchor behavior
2. If the current hash already matches the target, temporarily clear it
via `replaceState`
3. Set `location.hash` to the target, which fires `hashchange` and
triggers the SPA router

This handles both the Details link (`#/nodes/{pubkey}`) and the
Analytics link (`#/nodes/{pubkey}/analytics`).

## Testing

- All frontend helper tests pass (552/552)
- All packet filter tests pass (62/62)
- All aging tests pass (29/29)
- Go server tests pass

---------

Co-authored-by: you <you@example.com>
2026-04-16 23:21:05 -07:00
you fa348efe2a fix: force-remove staging container before deploy — handles both compose and docker-run containers
The deploy step used only 'docker compose down' which can't remove
containers created via 'docker run'. Now explicitly stops+removes
the named container first, then runs compose down as cleanup.

Permanent fix for the recurring CI deploy failure.
2026-04-17 05:08:32 +00:00
a9a18ff051 fix: neighbor graph slider persists to localStorage, default 0.7 (#776)
## Summary

The neighbor graph min score slider didn't persist its value to
localStorage, resetting to 0.10 on every page load. This was a poor
default for most use cases.

## Changes

- **Default changed from 0.10 to 0.70** — more useful starting point
that filters out low-confidence edges
- **localStorage persistence** — slider value saved on change, restored
on page load
- **3 new tests** in `test-frontend-helpers.js` verifying default value,
load behavior, and save behavior

## Testing

- `node test-frontend-helpers.js` — 547 passed, 0 failed
- `node test-packet-filter.js` — 62 passed, 0 failed
- `node test-aging.js` — 29 passed, 0 failed

---------

Co-authored-by: you <you@example.com>
2026-04-16 22:04:51 -07:00
ceea136e97 feat: observer graph representation (M1+M2) (#774)
## Summary

Fixes #753 — Milestones M1 and M2: Observer nodes in the neighbor graph
are now correctly labeled, colored, and filterable.

### M1: Label + color observers

**Backend** (`cmd/server/neighbor_api.go`):
- `buildNodeInfoMap()` now queries the `observers` table after building
from `nodes`
- Observer-only pubkeys (not already in the map as repeaters etc.) get
`role: "observer"` and their name from the observers table
- Observer-repeaters keep their repeater role (not overwritten)

**Frontend**:
- CSS variable `--role-observer: #8b5cf6` added to `:root`
- `ROLE_COLORS.observer` was already defined in `roles.js`

### M2: Observer filter checkbox (default unchecked)

**Frontend** (`public/analytics.js`):
- Observer checkbox added to the role filter section, **unchecked by
default**
- Observers create hub-and-spoke patterns (one observer can have 100+
edges) that drown out the actual repeater topology — hiding them by
default keeps the graph clean
- Fixed `applyNGFilters()` which previously always showed observers
regardless of checkbox state

### Tests

- Backend: `TestBuildNodeInfoMap_ObserverEnrichment` — verifies
observer-only pubkeys get name+role from observers table, and
observer-repeaters keep their repeater role
- All existing Go tests pass
- All frontend helper tests pass (544/544)

---------

Co-authored-by: you <you@example.com>
2026-04-16 21:35:14 -07:00
you 99dc4f805a fix: E2E neighbor test — use hash evaluation instead of page.goto for reliable SPA navigation
page.goto with hash-only change may not reliably trigger hashchange in
Playwright, causing the mobile full-screen node view to never render.
Use page.evaluate to set location.hash directly, which guarantees the
SPA router fires. Also increase timeout from 10s to 15s for CI margin.
2026-04-17 03:45:16 +00:00
ba7cd0fba7 fix: clock skew sanity checks — filter epoch-0, cap drift, min samples (#769)
Nodes with dead RTCs show -690d skew and -3 billion s/day drift. Fix:

1. **No Clock severity**: |skew| > 365d → `no_clock`, skip drift
2. **Drift cap**: |drift| > 86400 s/day → nil (physically impossible)
3. **Min samples**: < 5 samples → no drift regression
4. **Frontend**: 'No Clock' badge, '–' for unreliable drift

Fixes the crazy stats on the Clock Health fleet view.

---------

Co-authored-by: you <you@example.com>
2026-04-16 08:10:47 -07:00
6a648dea11 fix: multi-byte adopters — all node types, role column, advert precedence (#754) (#767)
## Fix: Multi-Byte Adopters Table — Three Bugs (#754)

### Bug 1: Companions in "Unknown"
`computeMultiByteCapability()` was repeater-only. Extended to classify
**all node types** (companions, rooms, sensors). A companion advertising
with 2-byte hash is now correctly "Confirmed".

### Bug 2: No Role Column
Added a **Role** column to the merged Multi-Byte Hash Adopters table,
color-coded using `ROLE_COLORS` from `roles.js`. Users can now
distinguish repeaters from companions without clicking through to node
detail.

### Bug 3: Data Source Disagreement
When adopter data (from `computeAnalyticsHashSizes`) shows `hashSize >=
2` but capability only found path evidence ("Suspected"), the
advert-based adopter data now takes precedence → "Confirmed". The
adopter hash sizes are passed into `computeMultiByteCapability()` as an
additional confirmed evidence source.

### Changes
- `cmd/server/store.go`: Extended capability to all node types, accept
adopter hash sizes, prioritize advert evidence
- `public/analytics.js`: Added Role column with color-coded badges
- `cmd/server/multibyte_capability_test.go`: 3 new tests (companion
confirmed, role populated, adopter precedence)

### Tests
- All 10 multi-byte capability tests pass
- All 544 frontend helper tests pass
- All 62 packet filter tests pass
- All 29 aging tests pass

---------

Co-authored-by: you <you@example.com>
2026-04-16 00:51:38 -07:00
29157742eb feat: show collision details in Hash Usage Matrix for all hash sizes (#758)
## Summary

Shows which prefixes are colliding in the Hash Usage Matrix, making the
"PREFIX COLLISIONS: N" count actionable.

Fixes #757

## Changes

### Frontend (`public/analytics.js`)
- **Clickable collision count**: When collisions > 0, the stat card is
clickable and scrolls to the collision details section. Shows a `▼`
indicator.
- **3-byte collision table**: The collision risk section and
`renderCollisionsFromServer` now render for all hash sizes including
3-byte (was previously hidden/skipped for 3-byte).
- **Helpful hint**: 3-byte panel now says "See collision details below"
when collisions exist.

### Backend (`cmd/server/collision_details_test.go`)
- Test that collision details include correct prefix and node
name/pubkey pairs
- Test that collision details are empty when no collisions exist

### Frontend Tests (`test-frontend-helpers.js`)
- Test clickable stat card renders `onclick` and `cursor:pointer` when
collisions > 0
- Test non-clickable card when collisions = 0
- Test collision table renders correct node links (`#/nodes/{pubkey}`)
- Test no-collision message renders correctly

## What was already there

The backend already returned full collision details (prefix, nodes with
pubkeys/names/coords, distance classification) in the `hash-collisions`
API. The frontend already had `renderCollisionsFromServer` rendering a
rich table with node links. The gap was:
1. The 3-byte tab hid the collision risk section entirely
2. No visual affordance to navigate from the stat count to the details

## Perf justification

No new computation — collision data was already computed and returned by
the API. The only change is rendering it for 3-byte (same as
1-byte/2-byte). The collision list is already limited by the backend
sort+slice pattern.

---------

Co-authored-by: you <you@example.com>
2026-04-16 00:18:25 -07:00
ed19a19473 fix: correct field table offsets for transport routes (#766)
## Summary

Fixes #765 — packet detail field table showed wrong byte offsets for
transport routes.

## Problem

`buildFieldTable()` hardcoded `path_length` at byte 1 for ALL packet
types. For `TRANSPORT_FLOOD` (route_type=0) and `TRANSPORT_DIRECT`
(route_type=3), transport codes occupy bytes 1-4, pushing `path_length`
to byte 5.

This caused:
- Wrong offset numbers in the field table for transport packets
- Transport codes displayed AFTER path length (wrong byte order)
- `Advertised Hash Size` row referenced wrong byte

## Fix

- Use dynamic `offset` tracking that accounts for transport codes
- Render transport code rows before path length (matching actual wire
format)
- Store `pathLenOffset` for correct reference in ADVERT payload section
- Reuse already-parsed `pathByte0` for hash size calculation in path
section

## Tests

Added 4 regression tests in `test-frontend-helpers.js`:
- FLOOD (route_type=1): path_length at byte 1, no transport codes
- TRANSPORT_FLOOD (route_type=0): transport codes at bytes 1-4,
path_length at byte 5
- TRANSPORT_DIRECT (route_type=3): same offsets as TRANSPORT_FLOOD
- Field table row order matches byte layout for transport routes

All existing tests pass (538 frontend helpers, 62 packet filter, 29
aging).

Co-authored-by: you <you@example.com>
2026-04-16 00:15:56 -07:00
copelajeandGitHub d27a7a653e fix case on channel key so Public decode/display works right (#761)
Simple change. Before this change Public wasn't showing up in the
channels display due to the case issue.
2026-04-16 00:14:47 -07:00
0e286d85fd fix: channel query performance — add channel_hash column, SQL-level filtering (#762) (#763)
## Problem
Channel API endpoints scan entire DB — 2.4s for channel list, 30s for
messages.

## Fix
- Added `channel_hash` column to transmissions (populated on ingest,
backfilled on startup)
- `GetChannels()` rewrites to GROUP BY channel_hash (one row per channel
vs scanning every packet)
- `GetChannelMessages()` filters by channel_hash at SQL level with
proper LIMIT/OFFSET
- 60s cache for channel list
- Index: `idx_tx_channel_hash` for fast lookups

Expected: 2.4s → <100ms for list, 30s → <500ms for messages.

Fixes #762

---------

Co-authored-by: you <you@example.com>
2026-04-16 00:09:36 -07:00
bffcbdaa0b feat: add channel UX — visible button, hint, status feedback (#760)
## Fixes #759

The "Add Channel" input was a bare text field with no visible submit
button and no feedback — users didn't know how to submit or whether it
worked.

### Changes

**`public/channels.js`**
- Replaced bare `<input>` with structured form: label, input + button
row, hint text, status div
- Added `showAddStatus()` helper for visual feedback during/after
channel add
- Status messages: loading → success (with decrypted message count) /
warning (no messages) / error
- Auto-hide status after 5 seconds
- Fallback click handler on the `+` button for browsers that don't fire
form submit

**`public/style.css`**
- `.ch-add-form` — form container
- `.ch-add-label` — bold 13px label
- `.ch-add-row` — flex row for input + button
- `.ch-add-btn` — 32×32 accent-colored submit button
- `.ch-add-hint` — muted helper text
- `.ch-add-status` — feedback with success/warn/error/loading variants

**`test-channel-add-ux.js`** — 20 tests validating HTML structure, CSS
classes, and feedback logic

### Before / After

**Before:** Bare input field, no button, no hint, no feedback  
**After:** Labeled section with visible `+` button, format hint, and
status messages showing decryption results

---------

Co-authored-by: you <you@example.com>
2026-04-15 18:54:35 -07:00
3bdf72b4cf feat: clock skew UI — node badges, detail sparkline, fleet analytics (#690 M2+M3) (#752)
## Summary

Frontend visualizations for clock skew detection.

Implements #690 M2 and M3. Does NOT close #690 — M4+M5 remain.

### M2: Node badges + detail sparkline
- Severity badges ( green/yellow/orange/red) on node list next to each
node
- Node detail: Clock Skew section with current value, severity, drift
rate
- Inline SVG sparkline showing skew history, color-coded by severity
zones

### M3: Fleet analytics view
- 'Clock Health' section on Analytics page
- Sortable table: Name | Skew | Severity | Drift | Last Advert
- Filter buttons by severity (OK/Warning/Critical/Absurd)
- Summary stats: X nodes OK, Y warning, Z critical
- Color-coded rows

### Changes
- `public/nodes.js` — badge rendering + detail section
- `public/analytics.js` — fleet clock health view
- `public/roles.js` — severity color helpers
- `public/style.css` — badge + sparkline + fleet table styles
- `cmd/server/clock_skew.go` — added fleet summary endpoint
- `cmd/server/routes.go` — wired fleet endpoint
- `test-frontend-helpers.js` — 11 new tests

---------

Co-authored-by: you <you@example.com>
2026-04-15 15:25:50 -07:00
401fd070f8 fix: improve trackedBytes accuracy for memory estimation (#751)
## Problem

Fixes #743 — High memory usage / OOM with relatively small dataset.

`trackedBytes` severely undercounted actual per-packet memory because it
only tracked base struct sizes and string field lengths, missing major
allocations:

| Structure | Untracked Cost | Scale Impact |
|-----------|---------------|--------------|
| `spTxIndex` (O(path²) subpath entries) | 40 bytes × path combos |
50-150MB |
| `ResolvedPath` on observations | 24 bytes × elements | ~25MB |
| Per-tx maps (`obsKeys`, `observerSet`) | 200 bytes/tx flat | ~11MB |
| `byPathHop` index entries | 50 bytes/hop | 20-40MB |

This caused eviction to trigger too late (or not at all), leading to
OOM.

## Fix

Expanded `estimateStoreTxBytes` and `estimateStoreObsBytes` to account
for:

- **Per-tx maps**: +200 bytes flat for `obsKeys` + `observerSet` map
headers
- **Path hop index**: +50 bytes per hop in `byPathHop`
- **Subpath index**: +40 bytes × `hops*(hops-1)/2` combinations for
`spTxIndex`
- **Resolved paths**: +24 bytes per `ResolvedPath` element on
observations

Updated the existing `TestEstimateStoreTxBytes` to match new formula.
All existing eviction tests continue to pass — the eviction logic itself
is unchanged.

Also exposed `avgBytesPerPacket` in the perf API (`/api/perf`) so
operators can monitor per-packet memory costs.

## Performance

Benchmark confirms negligible overhead (called on every insert):

```
BenchmarkEstimateStoreTxBytes    159M ops    7.5 ns/op    0 B/op    0 allocs
BenchmarkEstimateStoreObsBytes   1B ops      1.0 ns/op    0 B/op    0 allocs
```

## Tests

- 6 new tests in `tracked_bytes_test.go`:
  - Reasonable value ranges for different packet sizes
  - 10-hop packets estimate significantly more than 2-hop (subpath cost)
  - Observations with `ResolvedPath` estimate more than without
  - 15 observations estimate >10x a single observation
- `trackedBytes` matches sum of individual estimates after batch insert
  - Eviction triggers correctly with improved estimates
- 2 benchmarks confirming sub-10ns estimate cost
- Updated existing `TestEstimateStoreTxBytes` for new formula
- Full test suite passes

---------

Co-authored-by: you <you@example.com>
2026-04-15 07:53:32 -07:00
1b315bf6d0 feat: PSK channels, channel removal, message caching (#725 M3+M4+M5) (#750)
## Summary

Implements milestones M3, M4, and M5 from #725 — all client-side, zero
server changes.

### M3: PSK channel support

The channel input field now accepts both `#channelname` (hashtag
derivation) and raw 32-char hex keys (PSK). Auto-detection: if input
starts with `#`, derive key via SHA-256; otherwise validate as hex and
store directly. Same decrypt pipeline — `ChannelDecrypt.decrypt()` takes
key bytes regardless of source.

Input placeholder updated to: `#LongFast or paste hex key`

### M4: Channel removal

User-added channels now show a ✕ button on hover. Click → confirm dialog
→ removes:
- Key from localStorage (`ChannelDecrypt.removeKey()`)
- Cached messages from localStorage
(`ChannelDecrypt.clearChannelCache()`)
- Channel entry from sidebar

If the removed channel was selected, the view resets to the empty state.

### M5: localStorage message caching with delta fetch

After client-side decryption, results are cached in localStorage keyed
by channel name:

```
{ messages: [...], lastTimestamp: "...", count: N, ts: Date.now() }
```

On subsequent visits:
1. **Instant render** — cached messages displayed immediately via
`onCacheHit` callback
2. **Delta fetch** — only packets newer than `lastTimestamp` are fetched
and decrypted
3. **Merge** — new messages merged with cache, deduplicated by
`packetHash`
4. **Cache invalidation** — if total candidate count changes, full
re-decrypt triggered
5. **Size limit** — max 1000 messages cached per channel (most recent
kept)

### Performance

- Delta fetch avoids re-decrypting the full history on every page load
- Cache-first rendering provides instant UI response
- `deduplicateAndMerge()` uses a hash set for O(n) dedup
- 1000-message cap prevents localStorage quota issues

### Tests (24 new)

- M3: hex key detection (valid/invalid patterns)
- M3: key derivation round-trip, channel hash computation
- M3: PSK key storage and retrieval
- M4: channel removal clears both key and cache
- M5: cache size limit enforcement (1200 → 1000 stored)
- M5: cache stores count and lastTimestamp
- M5: clearChannelCache works independently
- All existing tests pass (523 frontend helpers, 62 packet filter)

### Files changed

| File | Change |
|------|--------|
| `public/channel-decrypt.js` | `removeKey()` now clears cache;
`clearChannelCache()`; `setCache()` with count + size limit |
| `public/channels.js` | Extracted `decryptCandidates()`,
`deduplicateAndMerge()`; delta fetch logic; remove button handler;
cache-first rendering |
| `public/style.css` | `.ch-remove-btn` styles (hover-reveal ✕) |
| `test-channel-decrypt-m345.js` | 24 new tests |

Implements #725

Co-authored-by: you <you@example.com>
2026-04-14 23:23:02 -07:00
a815e70975 feat: Clock skew detection — backend computation (M1) (#746)
## Summary

Implements **Milestone 1** of #690 — backend clock skew computation for
nodes and observers.

## What's New

### Clock Skew Engine (`clock_skew.go`)

**Phase 1 — Raw Skew Calculation:**
For every ADVERT observation: `raw_skew = advert_timestamp -
observation_timestamp`

**Phase 2 — Observer Calibration:**
Same packet seen by multiple observers → compute each observer's clock
offset as the median deviation from the per-packet median observation
timestamp. This identifies observers with their own clock drift.

**Phase 3 — Corrected Node Skew:**
`corrected_skew = raw_skew + observer_offset` — compensates for observer
clock error.

**Phase 4 — Trend Analysis:**
Linear regression over time-ordered skew samples estimates drift rate in
seconds/day. Detects crystal drift vs stable offset vs sudden jumps.

### Severity Classification

| Level | Threshold | Meaning |
|-------|-----------|---------|
|  OK | < 5 min | Normal |
| ⚠️ Warning | 5 min – 1 hour | Clock drifting |
| 🔴 Critical | 1 hour – 30 days | Likely no time source |
| 🟣 Absurd | > 30 days | Firmware default or epoch 0 |

### New API Endpoints

- `GET /api/nodes/{pubkey}/clock-skew` — per-node skew data (mean,
median, last, drift, severity)
- `GET /api/observers/clock-skew` — observer calibration offsets
- Clock skew also included in `GET /api/nodes/{pubkey}/analytics`
response as `clockSkew` field

### Performance

- 30-second compute cache avoids reprocessing on every request
- Operates on in-memory `byPayloadType[ADVERT]` index — no DB queries
- O(n) in total ADVERT observations, O(m log m) for median calculations

## Tests

15 unit tests covering:
- Severity classification at all thresholds
- Median/mean math helpers
- ISO timestamp parsing
- Timestamp extraction from decoded JSON (nested and top-level)
- Observer calibration with single and multi-observer scenarios
- Observer offset correction direction (verified the sign is
`+obsOffset`)
- Drift estimation: stable, linear, insufficient data, short time span
- JSON number extraction edge cases

## What's NOT in This PR

- No UI changes (M2–M4)
- No customizer integration (M5)
- Thresholds are hardcoded constants (will be configurable in M5)

Implements #690 M1.

---------

Co-authored-by: you <you@example.com>
2026-04-14 23:22:35 -07:00
aa84ce1e6a fix: correct hash_size detection for transport routes and zero-hop adverts (#747)
## Summary

Fixes #744
Fixes #722

Three bugs in hash_size computation caused zero-hop adverts to
incorrectly report `hash_size=1`, masking nodes that actually use
multi-byte hashes.

## Bugs Fixed

### 1. Wrong path byte offset for transport routes
(`computeNodeHashSizeInfo`)

Transport routes (types 0 and 3) have 4 transport code bytes before the
path byte. The code read the path byte from offset 1 (byte index
`RawHex[2:4]`) for all route types. For transport routes, the correct
offset is 5 (`RawHex[10:12]`).

### 2. Missing RouteTransportDirect skip (`computeNodeHashSizeInfo`)

Zero-hop adverts from `RouteDirect` (type 2) were correctly skipped, but
`RouteTransportDirect` (type 3) zero-hop adverts were not. Both have
locally-generated path bytes with unreliable hash_size bits.

### 3. Zero-hop adverts not skipped in analytics
(`computeAnalyticsHashSizes`)

`computeAnalyticsHashSizes()` unconditionally overwrote a node's
`hashSize` with whatever the latest advert reported. A zero-hop direct
advert with `hash_size=1` could overwrite a previously-correct
`hash_size=2` from a multi-hop flood advert.

Fix: skip hash_size update for zero-hop direct/transport-direct adverts
while still counting the packet and updating `lastSeen`.

## Tests Added

- `TestHashSizeTransportRoutePathByteOffset` — verifies transport routes
read path byte at offset 5, regular flood reads at offset 1
- `TestHashSizeTransportDirectZeroHopSkipped` — verifies both
RouteDirect and RouteTransportDirect zero-hop adverts are skipped
- `TestAnalyticsHashSizesZeroHopSkip` — verifies analytics hash_size is
not overwritten by zero-hop adverts
- Fixed 3 existing tests (`FlipFlop`, `Dominant`, `LatestWins`) that
used route_type 0 (TransportFlood) header bytes without proper transport
code padding

## Complexity

All changes are O(1) per packet — no new loops or data structures. The
additional offset computation and zero-hop check are constant-time
operations within the existing packet scan loop.

Co-authored-by: you <you@example.com>
2026-04-14 23:04:26 -07:00
2aea01f10c fix: merge repeater+observer into single map marker (#745)
## Problem

When a node acts as both a repeater and an observer (same public key —
common with powered repeaters running MQTT clients), the map shows two
separate markers at the same location: a repeater rectangle and an
observer star. This causes visual clutter and both markers get pushed
out from the real location by the deconfliction algorithm.

## Solution

Detect combined repeater+observer nodes by matching node `public_key`
against observer `id`. When matched:

- **Label mode (hash labels on):** The repeater label gets a gold ★
appended inside the rectangle
- **Icon mode (hash labels off):** The repeater diamond gets a small
star overlay in the top-right corner of the SVG
- **Popup:** Shows both REPEATER and OBSERVER badges
- **Observer markers:** Skipped when the observer is already represented
as a combined node marker
- **Legend count:** Observer count excludes combined nodes (shows
standalone observers only)

## Performance

- Observer lookup uses a `Map` keyed by lowercase pubkey — O(1) per node
check
- Legend count uses a `Set` of node pubkeys — O(n+m) instead of O(n×m)
- No additional API calls; uses existing `observers` array already
fetched

## Testing

- All 523 frontend helper tests pass
- All 62 packet filter tests pass
- Visual: combined nodes show as single marker with star indicator

Fixes #719

---------

Co-authored-by: you <you@example.com>
2026-04-14 22:47:28 -07:00
b7c2cb070c docs: geofilter manual + config.example.json entry (#734)
## Summary

- Add missing `geo_filter` block to `config.example.json` with polygon
example, `bufferKm`, and inline `_comment`
- Add `docs/user-guide/geofilter.md`: full operator guide covering
config schema, GeoFilter Builder workflow, and prune script as one-time
migration tool
- Add Geographic filtering section to `docs/user-guide/configuration.md`
with link to the full guide

Closes #669 (M1: documentation)

## Test plan

- [x] `config.example.json` parses cleanly (no JSON errors)
- [x] `docs/user-guide/geofilter.md` renders correctly in GitHub preview
- [x] Link from `configuration.md` to `geofilter.md` resolves

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:43:19 -07:00
1de80a9eaf feat: serve geofilter builder from app, link from customizer (#735)
## Summary

Part of #669 — M2: Link the builder from the app.

- **`public/geofilter-builder.html`** — the existing
`tools/geofilter-builder.html` is now served by the static file server
at `/geofilter-builder.html`. Additions vs the original: a `← CoreScope`
back-link in the header, inline code comments explaining the output
format, and a help bar below the output panel with paste instructions
and a link to the documentation.
- **`public/customize-v2.js`** — adds a "Tools" section at the bottom of
the Export tab with a `🗺️ GeoFilter Builder →` link and a one-line
description.
- **`docs/user-guide/customization.md`** — documents the new GeoFilter
Builder entry in the Export tab.

> **Note:** `tools/geofilter-builder.html` is kept as-is for
local/offline use. The `public/` copy is what the server serves.

> **Depends on:** #734 (M1 docs) for `docs/user-guide/geofilter.md` —
the link in the help bar references that file. Can be merged
independently; the link still works once M1 lands.

## Test plan

- [x] Open the app, go to Customizer → Export tab — "Tools" section
appears with GeoFilter Builder link
- [x] Click the link — opens `/geofilter-builder.html` in a new tab
- [x] Builder loads the Leaflet map, draw 3+ points — JSON output
appears
- [x] Copy button works, output is valid `{ "geo_filter": { ... } }`
JSON
- [x] `← CoreScope` back-link navigates to `/`
- [x] Help bar shows paste instructions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:42:27 -07:00
e6ace95059 fix: desktop node click updates URL hash, deep link opens split panel (#676) (#739)
## Problem

Clicking a node on desktop opened the side panel but never updated the
URL hash, making nodes non-shareable/bookmarkable on desktop. Loading
`#/nodes/{pubkey}` directly on desktop also incorrectly showed the
full-screen mobile view.

## Changes

- `selectNode()` on desktop: adds `history.replaceState(null, '',
'#/nodes/' + pubkey)` so the URL updates on every click
- `init()`: full-screen path is now gated to `window.innerWidth <= 640`
(mobile only); desktop with a `routeParam` falls through to the split
panel and calls `selectNode()` to pre-select the node
- Deselect (Escape / close button): also calls `history.replaceState`
back to `#/nodes`

## Test plan

- [x] Desktop: click a node → URL updates to `#/nodes/{pubkey}`, split
panel opens
- [x] Desktop: copy URL, open in new tab → split panel opens with that
node selected (not full-screen)
- [x] Desktop: press Escape → URL reverts to `#/nodes`
- [x] Mobile (≤640px): clicking a node still navigates to full-screen
view

Closes #676

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:38:06 -07:00
f605d4ce7e fix: serialize filter params in URL hash for deep linking (#682) (#740)
## Problem

Applying packet filters (hash, node, observer, Wireshark expression) did
not update the URL hash, so filtered views could not be shared or
bookmarked.

## Changes

**`buildPacketsQuery()`** — extended to include:
- `hash=` from `filters.hash`
- `node=` from `filters.node`  
- `observer=` from `filters.observer`
- `filter=` from `filters._filterExpr` (Wireshark expression string)

**`updatePacketsUrl()`** — now called on every filter change:
- hash input (debounced)
- observer multi-select change
- node autocomplete select and clear
- Wireshark filter input (on valid expression or clear)

**URL restore on load** — `getHashParams()` now reads `hash`, `node`,
`observer`, `filter` and restores them into `filters` before the DOM is
built. Input fields pick up values from `filters` as before. Wireshark
expression is also recompiled and `filter-active` class applied.

## Test plan

- [ ] Type in hash filter → URL updates with `&hash=...`
- [ ] Copy URL, open in new tab → hash filter is pre-filled
- [ ] Select an observer → URL updates with `&observer=...`
- [ ] Select a node filter → URL updates with `&node=...`
- [ ] Type `type=ADVERT` in Wireshark filter → URL updates with
`&filter=type%3DADVERT`
- [ ] Load that URL → filter expression restored and active

Closes #682

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:37:33 -07:00
84f03f4f41 fix: hide undecryptable channel messages by default (#727) (#728)
## Problem

Channels page shows 53K 'Unknown' messages — undecryptable GRP_TXT
packets with no content. Pure noise.

## Fix

- Backend: channels API filters out undecrypted messages by default
- `?includeEncrypted=true` param to include them
- Frontend: 'Show encrypted' toggle in channels sidebar
- Unknown channels grayed out with '(no key)' label
- Toggle persists in localStorage

Fixes #727

---------

Co-authored-by: you <you@example.com>
2026-04-13 19:40:20 +00:00
8158631d02 feat: client-side channel decryption — add custom channels in browser (#725 M2) (#733)
## Summary

Pure client-side channel decryption. Users can add custom hashtag
channels or PSK channels directly in the browser. **The server never
sees the keys.**

Implements #725 M2 (revised). Does NOT close #725.

## How it works

1. User types `#channelname` or pastes a hex PSK in the channels sidebar
2. Browser derives key (`SHA256("#name")[:16]`) using Web Crypto API
3. Key stored in **localStorage** — never sent to the server
4. Browser fetches encrypted GRP_TXT packets via existing API
5. Browser decrypts client-side: AES-128-ECB + HMAC-SHA256 MAC
verification
6. Decrypted messages cached in localStorage
7. Progressive rendering — newest messages first, chunk-based

## Security

- Keys never leave the browser
- No new API endpoints
- No server-side changes whatsoever
- Channel interest partially observable via hash-based API requests
(documented, acceptable tradeoff)

## Changes

- `public/channels.js` — client-side decrypt module + UI integration
(+307 lines)
- `public/index.html` — no new script (inline in channels.js IIFE)
- `public/style.css` — add-channel input styling

---------

Co-authored-by: you <you@example.com>
2026-04-13 12:28:41 -07:00
14367488e2 fix: TRACE path_json uses path_sz from flags byte, not header hash_size (#732)
## Summary

TRACE packets encode their route hash size in the flags byte (`flags &
0x03`), not the header path byte. The decoder was using `path.HashSize`
from the header, which could be wrong or zero for direct-route TRACEs,
producing incorrect hop counts in `path_json`.

## Protocol Note

Per firmware, TRACE packets are **always direct-routed** (route_type 2 =
DIRECT, or 3 = TRANSPORT_DIRECT). FLOOD-routed TRACEs (route_type 1) are
anomalous — firmware explicitly rejects TRACE via flood. The decoder
handles these gracefully without crashing.

## Changes

**`cmd/server/decoder.go` and `cmd/ingestor/decoder.go`:**
- Read `pathSz` from TRACE flags byte: `(traceFlags & 0x03) + 1`
(0→1byte, 1→2byte, 2→3byte)
- Use `pathSz` instead of `path.HashSize` for splitting TRACE payload
path data into hops
- Update `path.HashSize` to reflect the actual TRACE path size
- Added `HopsCompleted` field to ingestor `Path` struct for parity with
server
- Updated comments to clarify TRACE is always direct-routed per firmware

**`cmd/server/decoder_test.go` — 5 new tests:**
- `TraceFlags1_TwoBytePathSz`: flags=1 → 2-byte hashes via DIRECT route
- `TraceFlags2_ThreeBytePathSz`: flags=2 → 3-byte hashes via DIRECT
route
- `TracePathSzUnevenPayload`: payload not evenly divisible by path_sz
- `TraceTransportDirect`: route_type=3 with transport codes + TRACE path
parsing
- `TraceFloodRouteGraceful`: anomalous FLOOD+TRACE handled without crash

All existing TRACE tests (flags=0, 1-byte hashes) continue to pass.

Fixes #731

---------

Co-authored-by: you <you@example.com>
2026-04-13 08:20:09 -07:00
71be54f085 feat: DB-backed channel messages for full history (#725 M1) (#726)
## Summary

Switches channel API endpoints to query SQLite instead of the in-memory
packet store, giving users access to the full message history.

Implements #725 (M1 only — DB-backed channel messages). Does NOT close
#725 — M2-M5 (custom channels, PSK, persistence, retroactive decryption)
remain.

## Problem

Channel endpoints (`/api/channels`, `/api/channels/{hash}/messages`)
preferred the in-memory packet store when available. The store is
bounded by `packetStore.maxMemoryMB` — typically showing only recent
messages. The SQLite database has the complete history (weeks/months of
channel messages) but was only used as a fallback when the store was nil
(never in production).

## Fix

Reversed the preference order: DB first, in-memory store fallback.
Region filtering added to the DB path.

Co-authored-by: you <you@example.com>
2026-04-12 23:22:52 -07:00
c233c14156 feat: CLI tool to decrypt and export hashtag channel messages (#724)
## Summary

Adds `corescope-decrypt` — a standalone CLI tool that decrypts and
exports MeshCore hashtag channel messages from a CoreScope SQLite
database.

### What it does

MeshCore hashtag channels use symmetric encryption with keys derived
from the channel name. The CoreScope ingestor stores **all** GRP_TXT
packets, even those it can't decrypt. This tool enables retroactive
decryption — decrypt historical messages for any channel whose name you
learn after the fact.

### Architecture

- **`internal/channel/`** — Shared crypto package extracted from
ingestor logic:
  - `DeriveKey()` — `SHA-256("#name")[:16]`
  - `ChannelHash()` — 1-byte packet filter (`SHA-256(key)[0]`)
  - `Decrypt()` — HMAC-SHA256 MAC verify + AES-128-ECB
  - `ParsePlaintext()` — timestamp + flags + "sender: message" parsing

- **`cmd/decrypt/`** — CLI binary with three output formats:
  - `--format json` — Full metadata (observers, path, raw hex)
  - `--format html` — Self-contained interactive viewer with search/sort
  - `--format irc` (or `log`) — Plain-text IRC-style log, greppable

### Usage

```bash
# JSON export
corescope-decrypt --channel "#wardriving" --db meshcore.db

# Interactive HTML viewer
corescope-decrypt --channel wardriving --db meshcore.db --format html --output wardriving.html

# Greppable log
corescope-decrypt --channel "#wardriving" --db meshcore.db --format irc | grep "KE6QR"

# From Docker
docker exec corescope-prod /app/corescope-decrypt --channel "#wardriving" --db /app/data/meshcore.db
```

### Build & deployment

- Statically linked (`CGO_ENABLED=0`) — zero dependencies
- Added to Dockerfile (available at `/app/corescope-decrypt` in
container)
- CI: builds and tests in go-test job
- CI: attaches linux/amd64 and linux/arm64 binaries to GitHub Releases
on tags

### Testing

- `internal/channel/` — 9 tests: key derivation, encrypt/decrypt
round-trip, MAC rejection, wrong-channel rejection, plaintext parsing
- `cmd/decrypt/` — 7 tests: payload extraction, channel hash
consistency, all 3 output formats, JSON parseability, fixture DB
integration
- Verified against real fixture DB: successfully decrypts 17
`#wardriving` messages

### Limitations

- Hashtag channels only (name-derived keys). Custom PSK channels not
supported.
- No DM decryption (asymmetric, per-peer keys).
- Read-only database access.

Fixes #723

---------

Co-authored-by: you <you@example.com>
2026-04-12 22:07:41 -07:00
65482ff6f6 fix: cache invalidation tuning — 7% → 50-80% hit rate (#721)
## Cache Invalidation Tuning — 7% → 50-80% Hit Rate

Fixes #720

### Problem

Server-side cache hit rate was 7% (48 hits / 631 misses over 4.7 days).
Root causes from the [cache audit
report](https://github.com/Kpa-clawbot/CoreScope/issues/720):

1. **`invalidationDebounce` config value (30s) was dead code** — never
wired to `invCooldown`
2. **`invCooldown` hardcoded to 10s** — with continuous ingest, caches
cleared every 10s regardless of their 1800s TTLs
3. **`collisionCache` cleared on every `hasNewTransmissions`** — hash
collisions are structural (depend on node count), not per-packet

### Changes

| Change | File | Impact |
|--------|------|--------|
| Wire `invalidationDebounce` from config → `invCooldown` | `store.go` |
Config actually works now |
| Default `invCooldown` 10s → 300s (5 min) | `store.go` | 30x longer
cache survival |
| Add `hasNewNodes` flag to `cacheInvalidation` | `store.go` |
Finer-grained invalidation |
| `collisionCache` only clears on `hasNewNodes` | `store.go` | O(n²)
collision computation survives its 1hr TTL |
| `addToByNode` returns new-node indicator | `store.go` | Zero-cost
detection during indexing |
| `indexByNode` returns new-node indicator | `store.go` | Propagates to
ingest path |
| Ingest tracks and passes `hasNewNodes` | `store.go` | End-to-end
wiring |

### Tests Added

| Test | What it verifies |
|------|-----------------|
| `TestInvCooldownFromConfig` | Config value wired to `invCooldown`;
default is 300s |
| `TestCollisionCacheNotClearedByTransmissions` | `hasNewTransmissions`
alone does NOT clear `collisionCache` |
| `TestCollisionCacheClearedByNewNodes` | `hasNewNodes` DOES clear
`collisionCache` |
| `TestCacheSurvivesMultipleIngestCyclesWithinCooldown` | 5 rapid ingest
cycles don't clear any caches during cooldown |
| `TestNewNodesAccumulatedDuringCooldown` | `hasNewNodes` accumulated in
`pendingInv` and applied after cooldown |
| `BenchmarkAnalyticsLatencyCacheHitVsMiss` | 100% hit rate with
rate-limited invalidation |

All 200+ existing tests pass. Both benchmarks show 100% hit rate.

### Performance Justification

- **Before:** Effective cache lifetime = `min(TTL, invCooldown)` = 10s.
With analytics viewed ~once/few minutes, P(hit) ≈ 7%
- **After:** Effective cache lifetime = `min(TTL, 300s)` = 300s for most
caches, 3600s for `collisionCache`. Expected hit rate 50-80%
- **Complexity:** All changes are O(1) — `addToByNode` already checked
`nodeHashes[pubkey] == nil`, we just return the result
- **Benchmark proof:** `BenchmarkAnalyticsLatencyCacheHitVsMiss` → 100%
hit rate, 269ns/op

Co-authored-by: you <you@example.com>
2026-04-12 18:09:23 -07:00
7af91f7ef6 fix: perf page shows tracked memory instead of heap allocation (#718)
## Summary

The perf page "Memory Used" tile displayed `estimatedMB` (Go
`runtime.HeapAlloc`), which includes all Go runtime allocations — not
just packet store data. This made the displayed value misleading: it
showed ~2.4GB heap when only ~833MB was actual tracked packet data.

## Changes

### Frontend (`public/perf.js`)
- Primary tile now shows `trackedMB` as **"Tracked Memory"** — the
self-accounted packet store memory
- Added separate **"Heap (debug)"** tile showing `estimatedMB` for
runtime visibility

### Backend
- **`types.go`**: Added `TrackedMB` field to `HealthPacketStoreStats`
struct
- **`routes.go`**: Populate `TrackedMB` in `/health` endpoint response
from `GetPerfStoreStatsTyped()`
- **`routes_test.go`**: Assert `trackedMB` exists in health endpoint's
`packetStore`
- **`testdata/golden/shapes.json`**: Updated shape fixture with new
field

### What was already correct
- `/api/perf/stats` already exposed both `estimatedMB` and `trackedMB`
- `trackedMemoryMB()` method already existed in store.go
- Eviction logic already used `trackedBytes` (not HeapAlloc)

## Testing
- All Go tests pass (`go test ./... -count=1`)
- No frontend logic changes beyond template string field swap

Fixes #717

Co-authored-by: you <you@example.com>
2026-04-12 12:40:17 -07:00
f95aa49804 fix: exclude TRACE packets from multi-byte capability suspected detection (#715)
## Summary

Exclude TRACE packets (payload_type 8) from the "suspected" multi-byte
capability inference logic. TRACE packets carry hash size in their own
flags — forwarding repeaters read it from the TRACE header, not their
compile-time `PATH_HASH_SIZE`. Pre-1.14 repeaters can forward multi-byte
TRACEs without actually supporting multi-byte hashes, creating false
positives.

Fixes #714

## Changes

### `cmd/server/store.go`
- In `computeMultiByteCapability()`, skip packets with `payload_type ==
8` (TRACE) when scanning `byPathHop` for suspected multi-byte nodes
- "Confirmed" detection (from adverts) is unaffected

### `cmd/server/multibyte_capability_test.go`
- `TestMultiByteCapability_TraceExcluded`: TRACE packet with 2-byte path
does NOT mark repeater as suspected
- `TestMultiByteCapability_NonTraceStillSuspected`: Non-TRACE packet
with 2-byte path still marks as suspected
- `TestMultiByteCapability_ConfirmedUnaffectedByTraceExclusion`:
Confirmed status from advert unaffected by TRACE exclusion

## Testing

All 7 multi-byte capability tests pass. Full `cmd/server` and
`cmd/ingestor` test suites pass.

Co-authored-by: you <you@example.com>
2026-04-12 00:11:20 -07:00
45623672d9 fix: integrate multi-byte capability into adopters table, fix filter buttons (#712) (#713)
## Summary

Fixes #712 — Multi-byte capability filter buttons broken + needs
integration with Hash Adopters.

### Changes

**M1: Fix filter buttons breaking after first click**
- Root cause: `section.replaceWith(newSection)` replaced the entire DOM
node, but the event listener was attached to the old node. After
replacement, clicks went unhandled.
- Fix: Instead of replacing the whole section, only swap the table
content inside a stable `#mbAdoptersTableWrap` div. The event listener
on `#mbAdoptersSection` persists across filter changes.
- Button active state is now toggled via `classList.toggle` instead of
full DOM rebuild.

**M2: Better button labels**
- Changed from icon-only (` 76`) to descriptive labels: ` Confirmed
(76)`, `⚠️ Suspected (81)`, ` Unknown (223)`

**M3: Integrate with Multi-Byte Hash Adopters**
- Merged capability status into the existing adopters table as a new
"Status" column
- Removed the separate "Repeater Multi-Byte Capability" section
- Filter buttons now apply to the integrated table
- Nodes without capability data default to  Unknown
- Capability data is looked up by pubkey from the existing
`multiByteCapability` API response (no backend changes needed)

### Performance

- No new API calls — capability data already exists in the hash sizes
response
- Filter toggle is O(n) where n = number of adopter nodes (typically
<500)
- Event delegation on stable parent — no listener re-attachment needed

### Tests

- Updated existing `renderMultiByteCapability` tests for new label
format
- Added 5 new tests for `renderMultiByteAdopters`: empty state, status
integration, text labels with counts, unknown default, Status column
presence
- All 507 frontend tests pass, all Go tests pass

Co-authored-by: you <you@example.com>
2026-04-11 23:07:44 -07:00
4a7e20a8cb fix: redesign memory eviction — self-accounting trackedBytes, watermarks, safety cap (#711)
## Problem

`HeapAlloc`-based eviction cascades on large databases — evicts down to
near-zero packets because Go runtime overhead exceeds `maxMemoryMB` even
with an empty packet store.

## Fix (per Carmack spec on #710)

1. **Self-accounting `trackedBytes`** — running counter maintained on
insert/evict, computed from actual struct sizes. No
`runtime.ReadMemStats`.
2. **High/low watermark hysteresis** (100%/85%) — evict to 85% of
budget, don't re-trigger until 100% crossed again.
3. **25% per-pass safety cap** — never evict more than a quarter of
packets in one cycle.
4. **Oldest-first** — evict from sorted head, O(1) candidate selection.

`maxMemoryMB` now means packet store budget, not total process heap.

Fixes #710

Co-authored-by: you <you@example.com>
2026-04-11 23:06:48 -07:00
7e0b904d09 fix: refresh live feed relative timestamps every 10s (#709)
## Summary

Fixes #701 — Live feed timestamps showed stale relative times (e.g. "2s
ago" never updated to "5m ago").

## Root Cause

`formatLiveTimestampHtml()` was called once when each feed item was
created and never refreshed. The dedup path (when a duplicate hash moves
an item to the top) also didn't update the timestamp.

## Changes

### `public/live.js`
- **`data-ts` attribute on `.feed-time` spans**: All three feed item
creation paths (VCR replay, `addFeedItemDOM`, `addFeedItem`) now store
the packet timestamp as `data-ts` on the `.feed-time` span element
- **10-second refresh interval**: A `setInterval` queries all
`.feed-time[data-ts]` elements and re-renders their content via
`formatLiveTimestampHtml()`, keeping relative times accurate
- **Dedup path timestamp update**: When a duplicate hash observation
moves an existing feed item to the top, the `.feed-time` span is updated
with the new observation's timestamp
- **Cleanup**: The interval is cleared on page teardown alongside other
intervals

### `test-live.js`
- 3 new tests: formatting idempotency, numeric timestamp acceptance,
`data-ts` round-trip correctness

## Performance

- The refresh interval runs every 10s, iterating over at most 25
`.feed-time` DOM elements (feed is capped at 25 items via `while
(feed.children.length > 25)`). Negligible overhead.
- Uses `querySelectorAll` with attribute selector — O(n) where n ≤ 25.

## Testing

- All 3 new tests pass
- All pre-existing test suites pass (70 live.js tests, 62 packet-filter,
501 frontend-helpers)
- 8 pre-existing failures in `test-live.js` are unrelated
(`getParsedDecoded` missing from sandbox)

Co-authored-by: you <you@example.com>
2026-04-11 21:30:38 -07:00
e893a1b3c4 fix: index relay hops in byNode for liveness tracking (#708)
## Problem

Nodes that only appear as relay hops in packet paths (via
`resolved_path`) were never indexed in `byNode`, so `last_heard` was
never computed for them. This made relay-only nodes show as dead/stale
even when actively forwarding traffic.

Fixes #660

## Root Cause

`indexByNode()` only indexed pubkeys from decoded JSON fields (`pubKey`,
`destPubKey`, `srcPubKey`). Relay nodes appearing in `resolved_path`
were ignored entirely.

## Fix

`indexByNode()` now also iterates:
1. `ResolvedPath` entries from each observation
2. `tx.ResolvedPath` (best observation's resolved path, used for
DB-loaded packets)

A per-call `indexed` set prevents double-indexing when the same pubkey
appears in both decoded JSON and resolved path.

Extracted `addToByNode()` helper to deduplicate the nodeHashes/byNode
append logic.

## Scope

**Phase 1 only** — server-side in-memory indexing. No DB changes, no
ingestor changes. This makes `last_heard` reflect relay activity with
zero risk to persistence.

## Tests

5 new test cases in `TestIndexByNodeResolvedPath`:
- Resolved path pubkeys from observations get indexed
- Null entries in resolved path are skipped
- Relay-only nodes (no decoded JSON match) appear in `byNode`
- Dedup between decoded JSON and resolved path
- `tx.ResolvedPath` indexed when observations are empty

All existing tests pass unchanged.

## Complexity

O(observations × path_length) per packet — typically 1-3 observations ×
1-3 hops. No hot-path regression.

---------

Co-authored-by: you <you@example.com>
2026-04-11 21:25:42 -07:00
fcba2a9f3d fix: set PRAGMA busy_timeout on all RW SQLite connections (#707)
## Problem

`SQLITE_BUSY` contention between the ingestor and server's async
persistence goroutine drops `resolved_path` and `neighbor_edges`
updates. The DSN parameter `_busy_timeout=10000` may not be honored by
the modernc/sqlite driver.

## Fix

- **`openRW()` now sets `PRAGMA busy_timeout = 5000`** after opening the
connection, guaranteeing SQLite retries for up to 5 seconds before
returning `SQLITE_BUSY`
- **Refactored `PruneOldPackets` and `PruneOldMetrics`** to use
`openRW()` instead of duplicating connection setup — all RW connections
now get consistent busy_timeout handling
- Added test verifying the pragma is set correctly

## Changes

| File | Change |
|------|--------|
| `cmd/server/neighbor_persist.go` | `openRW()` sets `PRAGMA
busy_timeout = 5000` after open |
| `cmd/server/db.go` | `PruneOldPackets` and `PruneOldMetrics` use
`openRW()` instead of inline `sql.Open` |
| `cmd/server/neighbor_persist_test.go` | `TestOpenRW_BusyTimeout`
verifies pragma is set |

## Performance

No performance impact — `PRAGMA busy_timeout` is a connection-level
setting with zero overhead on uncontended writes. Under contention, it
converts immediate `SQLITE_BUSY` failures into brief retries (up to 5s),
which is strictly better than dropping data.

Fixes #705

---------

Co-authored-by: you <you@example.com>
2026-04-11 21:25:23 -07:00
you c6a0f91b07 fix: add internal/sigvalidate to Dockerfile for both server and ingestor builds
PR #686 added internal/sigvalidate/ with replace directives in both
go.mod files but didn't update the Dockerfile to COPY it into the
Docker build context. go mod download fails with 'no such file'.
2026-04-12 04:14:56 +00:00
ef8bce5002 feat: repeater multi-byte capability inference table (#706)
## Summary

Adds a new "Repeater Multi-Byte Capability" section to the Hash Stats
analytics tab that classifies each repeater's ability to handle
multi-byte hash prefixes (firmware >= v1.14).

Fixes #689

## What Changed

### Backend (`cmd/server/store.go`)
- New `computeMultiByteCapability()` method that infers capability for
each repeater using two evidence sources:
- **Confirmed** (100% reliable): node has advertised with `hash_size >=
2`, leveraging existing `computeNodeHashSizeInfo()` data
- **Suspected** (<100%): node's prefix appears as a hop in packets with
multi-byte path headers, using the `byPathHop` index. Prefix collisions
mean this isn't definitive.
- **Unknown**: no multi-byte evidence — could be pre-1.14 or 1.14+ with
default settings
- Extended `/api/analytics/hash-sizes` response with
`multiByteCapability` array

### Frontend (`public/analytics.js`)
- New `renderMultiByteCapability()` function on the Hash Stats tab
- Color-coded table: green confirmed, yellow suspected, gray unknown
- Filter buttons to show all/confirmed/suspected/unknown
- Column sorting by name, role, status, evidence, max hash size, last
seen
- Clickable rows link to node detail pages

### Tests (`cmd/server/multibyte_capability_test.go`)
- `TestMultiByteCapability_Confirmed`: advert with hash_size=2 →
confirmed
- `TestMultiByteCapability_Suspected`: path appearance only → suspected
- `TestMultiByteCapability_Unknown`: 1-byte advert only → unknown
- `TestMultiByteCapability_PrefixCollision`: two nodes sharing prefix,
one confirmed via advert, other correctly marked suspected (not
confirmed)

## Performance

- `computeMultiByteCapability()` runs once per cache cycle (15s TTL via
hash-sizes cache)
- Leverages existing `GetNodeHashSizeInfo()` cache (also 15s TTL) — no
redundant advert scanning
- Path hop scan is O(repeaters × prefix lengths) lookups in the
`byPathHop` map, with early break on first match per prefix
- Only computed for global (non-regional) requests to avoid unnecessary
work

---------

Co-authored-by: you <you@example.com>
2026-04-11 21:02:54 -07:00
922ebe54e7 BYOP Advert signature validation (#686)
For BYOP mode in the packet analyzer, perform signature validation on
advert packets and display whether successful or not. This is added as
we observed many corrupted advert packets that would be easily
detectable as such if signature validation checks were performed.

At present this MR is just to add this status in BYOP mode so there is
minimal impact to the application and no performance penalty for having
to perform these checks on all packets. Moving forward it probably makes
sense to do these checks on all advert packets so that corrupt packets
can be ignored in several contexts (like node lists for example).

Let me know what you think and I can adjust as needed.

---------

Co-authored-by: you <you@example.com>
2026-04-12 04:02:17 +00:00
26c47df814 fix: entrypoint .env support + deployment docs for bare docker run (#704)
## Summary

Fixes #702 — `.env` file `DISABLE_MOSQUITTO`/`DISABLE_CADDY` ignored
when using `docker run`.

## Changes

### Entrypoint sources `/app/data/.env`
The entrypoint now sources `/app/data/.env` (if present) before the
`DISABLE_*` checks. This works regardless of how the container is
started — `docker run`, compose, or `manage.sh`.

```bash
if [ -f /app/data/.env ]; then
  set -a
  . /app/data/.env
  set +a
fi
```

### `DISABLE_CADDY` added to compose files
Both `docker-compose.yml` and `docker-compose.staging.yml` now forward
`DISABLE_CADDY` to the container environment (was missing — only
`DISABLE_MOSQUITTO` was wired).

### Deployment docs updated
- `docs/deployment.md`: bare `docker run` is now the primary/recommended
approach with a full parameter reference table
- Documents the `/app/data/.env` convenience feature
- Compose and `manage.sh` marked as legacy alternatives
- `DISABLE_CADDY` added to the environment variable reference

### README quick start updated
Shows the full `docker run` command with `--restart`, ports, and
volumes. Includes HTTPS variant. Documents `-e` flags and `.env` file.

### v3.5.0 release notes
Updated the env var documentation to mention the `.env` file support.

## Testing
- All Go server tests pass
- All Go ingestor tests pass
- No logic changes to Go code — entrypoint shell script + docs only

---------

Co-authored-by: you <you@example.com>
2026-04-11 20:43:16 -07:00
bc22dbdb14 feat: DragManager — core drag mechanics (#608 M1) (#697)
## Summary

Implements M1 of the draggable panels spec from #608: the `DragManager`
class with core drag mechanics.

Fixes #608 (M1: DragManager core drag mechanics)

## What's New

### `public/drag-manager.js` (~215 lines)
- **State machine:** `IDLE → PENDING → DRAGGING → IDLE`
- **5px dead zone** on `.panel-header` to disambiguate click vs drag —
prevents hijacking corner toggle and close button clicks
- **Pointer events** with `setPointerCapture` for reliable tracking
- **`transform: translate()`** during drag — zero layout reflow
- **Snap-to-edge** on release: 20px threshold snaps to 12px margin
- **Z-index management** — dragged panel comes to front (counter from
1000)
- **`_detachFromCorner()`** — transitions panel from M0 corner CSS to
fixed positioning
- **Escape key** cancels drag and reverts to pre-drag position
- **`restorePositions()`** — applies saved viewport percentages on init
- **`handleResize()`** — clamps dragged panels inside viewport on window
resize
- **`enable()`/`disable()`** — responsive gate control

### `public/live.js` integration
- Instantiates `DragManager` after `initPanelPositions()`
- Registers `liveFeed`, `liveLegend`, `liveNodeDetail` panels
- **Responsive gate:** `matchMedia('(pointer: fine) and (min-width:
768px)')` — disables drag on touch/small screens, reverts to M0 corner
toggle
- **Resize clamping** debounced at 200ms

### `public/live.css` additions
- `cursor: grab/grabbing` on `.panel-header` (desktop only via `@media
(pointer: fine)`)
- `.is-dragging` class: opacity 0.92, elevated box-shadow, `will-change:
transform`, transitions disabled
- `[data-dragged="true"]` disables corner transition animations
- `prefers-reduced-motion` support

### Persistence
- **Format:** `panel-drag-{id}` → `{ xPct, yPct }` (viewport
percentages)
- **Survives resize:** positions recalculated from percentages
- **Corner toggle still works:** clicking corner button after drag
clears drag state (handled by existing M0 code)

## Tests

14 new unit tests in `test-drag-manager.js`:
- State machine transitions (IDLE → PENDING → DRAGGING → IDLE)
- Dead zone enforcement
- Button click guard (no drag on button pointerdown)
- Snap-to-edge behavior
- Position persistence as viewport percentages
- Restore from localStorage
- Resize clamping
- Disable/enable

## Performance

- `transform: translate()` during drag — compositor-only, no layout
reflow
- `will-change: transform` only during active drag (`.is-dragging`),
removed on drop
- `localStorage` write only on `pointerup`, never during `pointermove`
- Resize handler debounced at 200ms
- Single `style.transform` assignment per pointermove frame — negligible
cost

---------

Co-authored-by: you <you@example.com>
2026-04-11 20:41:35 -07:00
9917d50622 fix: resolve neighbor graph duplicate entries from different prefix lengths (#699)
## Problem

The neighbor graph creates separate entries for the same physical node
when observed with different prefix lengths. For example, a 1-byte
prefix `B0` (ambiguous, unresolved) and a 2-byte prefix `B05B` (resolved
to Busbee) appear as two separate neighbors of the same node.

Fixes #698

## Solution

### Part 1: Post-build resolution pass (Phase 1.5)

New function `resolveAmbiguousEdges(pm, graph)` in `neighbor_graph.go`:
- Called after `BuildFromStore()` completes the full graph, before any
API use
- Iterates all ambiguous edges and attempts resolution via
`resolveWithContext` with full graph context
- Only accepts high-confidence resolutions (`neighbor_affinity`,
`geo_proximity`, `unique_prefix`) — rejects
`first_match`/`gps_preference` fallbacks to avoid false positives
- Merges with existing resolved edges (count accumulation, max LastSeen)
or updates in-place
- Phase 1 edge collection loop is **unchanged**

### Part 2: API-layer dedup (defense-in-depth)

New function `dedupPrefixEntries()` in `neighbor_api.go`:
- Scans neighbor response for unresolved prefix entries matching
resolved pubkey entries
- Merges counts, timestamps, and observers; removes the unresolved entry
- O(n²) on ~50 neighbors per node — negligible cost

### Performance

Phase 1.5 runs O(ambiguous_edges × candidates). Per Carmack's analysis:
~50ms at 2K nodes on the 5-min rebuild cycle. Hot ingest path untouched.

## Tests

9 new tests in `neighbor_dedup_test.go`:

1. **Geo proximity resolution** — ambiguous edge resolved when candidate
has GPS near context node
2. **Merge with existing** — ambiguous edge merged into existing
resolved edge (count accumulation)
3. **No-match preservation** — ambiguous edge left as-is when prefix has
no candidates
4. **API dedup** — unresolved prefix merged with resolved pubkey in
response
5. **Integration** — node with both 1-byte and 2-byte prefix
observations shows single neighbor entry
6. **Phase 1 regression** — non-ambiguous edge collection unchanged
7. **LastSeen preservation** — merge keeps higher LastSeen timestamp
8. **No-match dedup** — API dedup doesn't merge non-matching prefixes
9. **Benchmark** — Phase 1.5 with 500+ edges

All existing tests pass (server + ingestor).

---------

Co-authored-by: you <you@example.com>
2026-04-10 11:19:54 -07:00
2e1a4a2e0d fix: handle companion nodes without adverts in My Mesh health cards (#696)
## Summary

Fixes #665 — companion nodes claimed in "My Mesh" showed "Could not load
data" because they never sent an advert, so they had no `nodes` table
entry, causing the health API to return 404.

## Three-Layer Fix

### 1. API Resilience (`cmd/server/store.go`)
`GetNodeHealth()` now falls back to building a partial response from the
in-memory packet store when `GetNodeByPubkey()` returns nil. Returns a
synthetic node stub (`role: "unknown"`, `name: "Unknown"`) with whatever
stats exist from packets, instead of returning nil → 404.

### 2. Ingestor Cleanup (`cmd/ingestor/main.go`)
Removed phantom sender node creation that used `"sender-" + name` as the
pubkey. Channel messages don't carry the sender's real pubkey, so these
synthetic entries were unreachable from the claiming/health flow — they
just polluted the nodes table with unmatchable keys.

### 3. Frontend UX (`public/home.js`)
The catch block in `loadMyNodes()` now distinguishes 404 (node not in DB
yet) from other errors:
- **404**: Shows 📡 "Waiting for first advert — this node has been seen
in channel messages but hasn't advertised yet"
- **Other errors**: Shows  "Could not load data" (unchanged)

## Tests
- Added `TestNodeHealthPartialFromPackets` — verifies a node with
packets but no DB entry returns 200 with synthetic node stub and stats
- Updated `TestHandleMessageChannelMessage` — verifies channel messages
no longer create phantom sender nodes
- All existing tests pass (`cmd/server`, `cmd/ingestor`)

Co-authored-by: you <you@example.com>
2026-04-09 20:03:52 -07:00
fcad49594b fix: include path.hopsCompleted in TRACE WebSocket broadcasts (#695)
## Summary

Fixes #683 — TRACE packets on the live map were showing the full path
instead of distinguishing completed vs remaining hops.

## Root Cause

Both WebSocket broadcast builders in `store.go` constructed the
`decoded` map with only `header` and `payload` keys — `path` was never
included. The frontend reads `decoded.path.hopsCompleted` to split trace
routes into solid (completed) and dashed (remaining) segments, but that
field was always `undefined`.

## Fix

For TRACE packets (payload type 9), call `DecodePacket()` on the raw hex
during broadcast and include the resulting `Path` struct in
`decoded["path"]`. This populates `hopsCompleted` which the frontend
already knows how to consume.

Both broadcast builders are patched:
- `IngestNewFromDB()` — new transmissions path (~line 1419)
- `IngestNewObservations()` — new observations path (~line 1680)

TRACE packets are infrequent, so the per-packet decode overhead is
negligible.

## Testing

- Added `TestIngestTraceBroadcastIncludesPath` — verifies that TRACE
broadcast maps include `decoded.path` with correct `hopsCompleted` value
- All existing tests pass (`cmd/server` + `cmd/ingestor`)

Co-authored-by: you <you@example.com>
2026-04-09 20:02:46 -07:00
a1e1e0bd2f fix: bottom-positioned panels overlap VCR bar (#693)
Fixes #685

## Problem

Corner positioning CSS (from PR #608) sets `bottom: 12px` for
bottom-positioned panels (`bl`, `br`), but the VCR bar at the bottom of
the live page is ~50px tall. This causes the legend (and any
bottom-positioned panel) to overlap the VCR controls.

## Fix

Changed `bottom: 12px` → `bottom: 58px` for both
`.live-overlay[data-position="bl"]` and
`.live-overlay[data-position="br"]`, matching the legend's original
`bottom: 58px` value that properly clears the VCR bar.

The VCR bar height is fixed (`.vcr-bar` class with consistent padding),
so a hardcoded value is appropriate here.

## Testing

- All existing tests pass (`npm test` — 13/13)
- CSS-only change, no logic affected

Co-authored-by: you <you@example.com>
2026-04-09 20:02:18 -07:00
34e7366d7c test: add RouteTransportDirect zero-hop cases to ingestor decoder tests (#684)
## Summary

Closes the symmetry gap flagged as a nit in PR #653 review:

> The ingestor decoder tests omit `RouteTransportDirect` zero-hop tests
— only the server decoder has those. Since the logic is identical, this
is not a blocker, but adding them would make the test suites symmetric.

- Adds `TestZeroHopTransportDirectHashSize` — `pathByte=0x00`, expects
`HashSize=0`
- Adds `TestZeroHopTransportDirectHashSizeWithNonZeroUpperBits` —
`pathByte=0xC0` (hash_size bits set, hash_count=0), expects `HashSize=0`

Both mirror the equivalent tests already present in
`cmd/server/decoder_test.go`.

## Test plan

- [ ] `cd cmd/ingestor && go test -run TestZeroHopTransportDirect -v` →
both new tests pass
- [ ] `cd cmd/ingestor && go test ./...` → no regressions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 17:36:34 -07:00
78 changed files with 10180 additions and 446 deletions
+52 -2
View File
@@ -68,6 +68,17 @@ jobs:
echo "--- Go Ingestor Coverage ---"
go tool cover -func=ingestor-coverage.out | tail -1
- name: Build and test channel library + decrypt CLI
run: |
set -e -o pipefail
cd internal/channel
go test ./...
echo "--- Channel library tests passed ---"
cd ../../cmd/decrypt
CGO_ENABLED=0 go build -ldflags="-s -w" -o corescope-decrypt .
go test ./...
echo "--- Decrypt CLI tests passed ---"
- name: Verify proto syntax
run: |
set -e
@@ -317,7 +328,43 @@ jobs:
cache-to: type=gha,mode=max
# ───────────────────────────────────────────────────────────────
# 4. Deploy Staging (master only)
# 4. Release Artifacts (tags only)
# ───────────────────────────────────────────────────────────────
release-artifacts:
name: "📦 Release Artifacts"
if: startsWith(github.ref, 'refs/tags/v')
needs: [go-test]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Set up Go 1.22
uses: actions/setup-go@v6
with:
go-version: '1.22'
- name: Build corescope-decrypt (static, linux/amd64)
run: |
cd cmd/decrypt
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-amd64 .
- name: Build corescope-decrypt (static, linux/arm64)
run: |
cd cmd/decrypt
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-s -w -X main.version=${{ github.ref_name }}" -o ../../corescope-decrypt-linux-arm64 .
- name: Upload release assets
uses: softprops/action-gh-release@v2
with:
files: |
corescope-decrypt-linux-amd64
corescope-decrypt-linux-arm64
# ───────────────────────────────────────────────────────────────
# 4b. Deploy Staging (master only)
# ───────────────────────────────────────────────────────────────
deploy:
name: "🚀 Deploy Staging"
@@ -340,7 +387,10 @@ jobs:
- name: Deploy staging
run: |
# Stop old container and release memory
# Force-remove the staging container regardless of how it was created
# (compose-managed OR manually created via docker run)
docker stop corescope-staging-go 2>/dev/null || true
docker rm -f corescope-staging-go 2>/dev/null || true
docker compose -f "$STAGING_COMPOSE_FILE" -p corescope-staging down --timeout 30 2>/dev/null || true
# Wait for container to be fully gone and OS to reclaim memory (3GB limit)
+12 -1
View File
@@ -10,6 +10,7 @@ ARG BUILD_TIME=unknown
WORKDIR /build/server
COPY cmd/server/go.mod cmd/server/go.sum ./
COPY internal/geofilter/ ../../internal/geofilter/
COPY internal/sigvalidate/ ../../internal/sigvalidate/
RUN go mod download
COPY cmd/server/ ./
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /corescope-server .
@@ -17,10 +18,20 @@ RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMI
# Build ingestor
WORKDIR /build/ingestor
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
COPY internal/geofilter/ ../../internal/geofilter/
COPY internal/sigvalidate/ ../../internal/sigvalidate/
RUN go mod download
COPY cmd/ingestor/ ./
RUN go build -o /corescope-ingestor .
# Build decrypt CLI
WORKDIR /build/decrypt
COPY cmd/decrypt/go.mod cmd/decrypt/go.sum ./
COPY internal/channel/ ../../internal/channel/
RUN go mod download
COPY cmd/decrypt/ ./
RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o /corescope-decrypt .
# Runtime image
FROM alpine:3.20
@@ -29,7 +40,7 @@ RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
WORKDIR /app
# Go binaries
COPY --from=builder /corescope-server /corescope-ingestor /app/
COPY --from=builder /corescope-server /corescope-ingestor /corescope-decrypt /app/
# Frontend assets + config
COPY public/ ./public/
+15 -4
View File
@@ -80,15 +80,26 @@ No build step required — just run:
```bash
docker run -d --name corescope \
-p 80:80 \
-v corescope-data:/app/data \
--restart=unless-stopped \
-p 80:80 -p 1883:1883 \
-v /your/data:/app/data \
ghcr.io/kpa-clawbot/corescope:latest
```
Open `http://localhost` — done. No config file needed; CoreScope starts with sensible defaults.
See [DEPLOY.md](DEPLOY.md) for image tags, Docker Compose, and migration from `manage.sh`.
See [docs/deployment.md](docs/deployment.md) for the full deployment guide — MQTT setup, HTTPS options, backups, monitoring, and troubleshooting.
For HTTPS with a custom domain, add `-p 443:443` and mount your Caddyfile:
```bash
docker run -d --name corescope \
--restart=unless-stopped \
-p 80:80 -p 443:443 -p 1883:1883 \
-v /your/data:/app/data \
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
-v /your/caddy-data:/data/caddy \
ghcr.io/kpa-clawbot/corescope:latest
```
Disable built-in services with `-e DISABLE_MOSQUITTO=true` or `-e DISABLE_CADDY=true`, or drop a `.env` file in your data volume. See [docs/deployment.md](docs/deployment.md) for the full reference.
### Build from Source
+142
View File
@@ -0,0 +1,142 @@
# corescope-decrypt
Standalone CLI tool to decrypt and export MeshCore hashtag channel messages from a CoreScope SQLite database.
## Why
MeshCore hashtag channels use symmetric encryption where the key is derived deterministically from the channel name. The CoreScope ingestor stores **all** `GRP_TXT` packets in the database, including those it cannot decrypt at ingest time.
This tool enables:
- **Retroactive decryption** — decrypt historical messages for any channel whose name you learn after the fact
- **Forensics & analysis** — export channel traffic for offline review
- **Bulk export** — dump an entire channel's history as JSON, HTML, or plain text
## Installation
### From Docker image
The binary is included in the CoreScope Docker image at `/app/corescope-decrypt`:
```bash
docker exec corescope-prod /app/corescope-decrypt --channel "#wardriving" --db /app/data/meshcore.db
```
### From GitHub release
Download the static binary from the [Releases](https://github.com/Kpa-clawbot/CoreScope/releases) page:
```bash
# Linux amd64
curl -LO https://github.com/Kpa-clawbot/CoreScope/releases/latest/download/corescope-decrypt-linux-amd64
chmod +x corescope-decrypt-linux-amd64
./corescope-decrypt-linux-amd64 --help
```
### Build from source
```bash
cd cmd/decrypt
CGO_ENABLED=0 go build -ldflags="-s -w" -o corescope-decrypt .
```
The binary is statically linked — no dependencies, runs on any Linux.
## Usage
```
corescope-decrypt --channel NAME --db PATH [--format FORMAT] [--output FILE]
```
Run `corescope-decrypt --help` for full flag documentation.
### JSON output (default)
Machine-readable, includes all metadata (observers, path hops, raw hex):
```bash
corescope-decrypt --channel "#wardriving" --db meshcore.db
```
```json
[
{
"hash": "a1b2c3...",
"timestamp": "2026-04-12T17:19:09Z",
"sender": "XMD Tag 1",
"message": "@[MapperBot] 37.76985, -122.40525 [0.3w]",
"channel": "#wardriving",
"raw_hex": "150206...",
"path": ["A3", "B0"],
"observers": [
{"name": "Observer1", "snr": 9.5, "rssi": -56, "timestamp": "2026-04-12T17:19:10Z"}
]
}
]
```
### HTML output
Self-contained interactive viewer — search, sortable columns, expandable detail rows:
```bash
corescope-decrypt --channel "#wardriving" --db meshcore.db --format html --output wardriving.html
open wardriving.html
```
No external dependencies. The JSON data is embedded directly in the HTML file.
### IRC / log output
Plain-text, one line per message — ideal for `grep`, `awk`, and piping:
```bash
corescope-decrypt --channel "#wardriving" --db meshcore.db --format irc
```
```
[2026-04-12 17:19:09] <XMD Tag 1> @[MapperBot] 37.76985, -122.40525 [0.3w]
[2026-04-12 17:20:25] <XMD Tag 1> @[MapperBot] 37.78075, -122.39774 [0.3w]
[2026-04-12 17:25:30] <mk 🤠> @[MapperBot] 35.32444, -120.62077
```
```bash
# Find all messages from a specific sender
corescope-decrypt --channel "#wardriving" --db meshcore.db --format irc | grep "KE6QR"
```
## How channel encryption works
MeshCore hashtag channels derive their encryption key from the channel name:
1. **Key derivation**: `AES-128 key = SHA-256("#channelname")[:16]` (first 16 bytes)
2. **Channel hash**: `SHA-256(key)[0]` — 1-byte identifier in the packet header, used for fast filtering
3. **Encryption**: AES-128-ECB
4. **MAC**: HMAC-SHA256 with a 32-byte secret (key + 16 zero bytes), truncated to 2 bytes
5. **Plaintext format**: `timestamp(4 LE) + flags(1) + "sender: message\0"`
See the firmware source at `firmware/src/helpers/BaseChatMesh.cpp` for the canonical implementation.
## Testing against the fixture DB
```bash
cd cmd/decrypt
go test ./...
# Manual test with the real fixture:
go run . --channel "#wardriving" --db ../../test-fixtures/e2e-fixture.db --format irc
```
The shared crypto library also has independent tests:
```bash
cd internal/channel
go test -v ./...
```
## Limitations
- **Hashtag channels only.** Only channels where the key is derived from `SHA-256("#name")` are supported. Custom PSK channels require the raw key (not implemented).
- **No DM decryption.** Direct messages (`TXT_MSG`) use per-peer asymmetric encryption and cannot be decrypted by this tool.
- **Read-only.** The tool opens the database in read-only mode and never modifies it.
- **Timestamps are UTC.** The sender's embedded timestamp is used when available, displayed in UTC.
+22
View File
@@ -0,0 +1,22 @@
module github.com/corescope/decrypt
go 1.22
require (
github.com/meshcore-analyzer/channel v0.0.0
modernc.org/sqlite v1.34.5
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.22.0 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
)
replace github.com/meshcore-analyzer/channel => ../../internal/channel
+43
View File
@@ -0,0 +1,43 @@
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic=
golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw=
golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.34.5 h1:Bb6SR13/fjp15jt70CL4f18JIN7p7dnMExd+UFnF15g=
modernc.org/sqlite v1.34.5/go.mod h1:YLuNmX9NKs8wRNK2ko1LW1NGYcc9FkBO69JOt1AR9JE=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+467
View File
@@ -0,0 +1,467 @@
// corescope-decrypt decrypts and exports hashtag channel messages from a CoreScope SQLite database.
//
// Usage:
//
// corescope-decrypt --channel "#wardriving" --db meshcore.db [--format json|html] [--output file]
package main
import (
"database/sql"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"html"
"log"
"os"
"sort"
"strings"
"time"
"github.com/meshcore-analyzer/channel"
_ "modernc.org/sqlite"
)
// Version info (set via ldflags).
var version = "dev"
// ChannelMessage is a single decrypted channel message with metadata.
type ChannelMessage struct {
Hash string `json:"hash"`
Timestamp string `json:"timestamp"`
Sender string `json:"sender"`
Message string `json:"message"`
Channel string `json:"channel"`
RawHex string `json:"raw_hex"`
Path []string `json:"path"`
Observers []Observer `json:"observers"`
}
// Observer is a single observation of the transmission.
type Observer struct {
Name string `json:"name"`
SNR float64 `json:"snr"`
RSSI float64 `json:"rssi"`
Timestamp string `json:"timestamp"`
}
func main() {
channelName := flag.String("channel", "", "Channel name (e.g. \"#wardriving\")")
dbPath := flag.String("db", "", "Path to CoreScope SQLite database")
format := flag.String("format", "json", "Output format: json, html, irc (or log)")
output := flag.String("output", "", "Output file (default: stdout)")
showVersion := flag.Bool("version", false, "Print version and exit")
flag.Usage = func() {
fmt.Fprintf(os.Stderr, `corescope-decrypt — Decrypt and export MeshCore hashtag channel messages
USAGE
corescope-decrypt --channel NAME --db PATH [--format FORMAT] [--output FILE]
FLAGS
--channel NAME Channel name to decrypt (e.g. "#wardriving", "wardriving")
The "#" prefix is added automatically if missing.
--db PATH Path to a CoreScope SQLite database file (read-only access).
--format FORMAT Output format (default: json):
json — Machine-readable JSON array with full metadata
html — Self-contained HTML viewer with search and sorting
irc — Plain-text IRC-style log, one line per message
log — Alias for irc
--output FILE Write output to FILE instead of stdout.
--version Print version and exit.
EXAMPLES
# Export #wardriving messages as JSON
corescope-decrypt --channel "#wardriving" --db /app/data/meshcore.db
# Generate an interactive HTML viewer
corescope-decrypt --channel wardriving --db meshcore.db --format html --output wardriving.html
# Greppable IRC log
corescope-decrypt --channel "#MeshCore" --db meshcore.db --format irc --output meshcore.log
grep "KE6QR" meshcore.log
# From the Docker container
docker exec corescope-prod /app/corescope-decrypt --channel "#wardriving" --db /app/data/meshcore.db
RETROACTIVE DECRYPTION
MeshCore hashtag channels use symmetric encryption — the key is derived from the
channel name. The CoreScope ingestor stores ALL GRP_TXT packets in the database,
even those it cannot decrypt at ingest time. This tool lets you retroactively
decrypt messages for any channel whose name you know, even if the ingestor was
never configured with that channel's key.
This means you can recover historical messages by simply knowing the channel name.
LIMITATIONS
- Only hashtag channels (shared-secret, name-derived key) are supported.
- Direct messages (TXT_MSG) use per-peer encryption and cannot be decrypted.
- Custom PSK channels (non-hashtag) require the raw key, not a channel name.
`)
}
flag.Parse()
if *showVersion {
fmt.Println("corescope-decrypt", version)
os.Exit(0)
}
if *channelName == "" || *dbPath == "" {
flag.Usage()
os.Exit(1)
}
// Normalize channel name
ch := *channelName
if !strings.HasPrefix(ch, "#") {
ch = "#" + ch
}
key := channel.DeriveKey(ch)
chHash := channel.ChannelHash(key)
db, err := sql.Open("sqlite", *dbPath+"?mode=ro")
if err != nil {
log.Fatalf("Failed to open database: %v", err)
}
defer db.Close()
// Query all GRP_TXT packets
rows, err := db.Query(`SELECT id, hash, raw_hex, first_seen FROM transmissions WHERE payload_type = 5`)
if err != nil {
log.Fatalf("Query failed: %v", err)
}
defer rows.Close()
var messages []ChannelMessage
decrypted, total := 0, 0
for rows.Next() {
var id int
var txHash, rawHex, firstSeen string
if err := rows.Scan(&id, &txHash, &rawHex, &firstSeen); err != nil {
log.Printf("Scan error: %v", err)
continue
}
total++
payload, err := extractGRPPayload(rawHex)
if err != nil {
continue
}
if len(payload) < 3 {
continue
}
// Check channel hash byte
if payload[0] != chHash {
continue
}
mac := payload[1:3]
ciphertext := payload[3:]
if len(ciphertext) < 5 || len(ciphertext)%16 != 0 {
// Pad ciphertext to block boundary for decryption attempt
if len(ciphertext) < 16 {
continue
}
// Truncate to block boundary
ciphertext = ciphertext[:len(ciphertext)/16*16]
}
plaintext, ok := channel.Decrypt(key, mac, ciphertext)
if !ok {
continue
}
ts, sender, msg, err := channel.ParsePlaintext(plaintext)
if err != nil {
continue
}
decrypted++
// Convert MeshCore timestamp
timestamp := time.Unix(int64(ts), 0).UTC().Format(time.RFC3339)
// Get path from decoded_json
path := getPathFromDB(db, id)
// Get observers
observers := getObservers(db, id)
messages = append(messages, ChannelMessage{
Hash: txHash,
Timestamp: timestamp,
Sender: sender,
Message: msg,
Channel: ch,
RawHex: rawHex,
Path: path,
Observers: observers,
})
}
// Sort by timestamp
sort.Slice(messages, func(i, j int) bool {
return messages[i].Timestamp < messages[j].Timestamp
})
log.Printf("Scanned %d GRP_TXT packets, decrypted %d for channel %s", total, decrypted, ch)
// Generate output
var out []byte
switch *format {
case "json":
out, err = json.MarshalIndent(messages, "", " ")
if err != nil {
log.Fatalf("JSON marshal: %v", err)
}
out = append(out, '\n')
case "html":
out = renderHTML(messages, ch)
case "irc", "log":
out = renderIRC(messages)
default:
log.Fatalf("Unknown format: %s (use json, html, irc, or log)", *format)
}
if *output != "" {
if err := os.WriteFile(*output, out, 0644); err != nil {
log.Fatalf("Write file: %v", err)
}
log.Printf("Written to %s", *output)
} else {
os.Stdout.Write(out)
}
}
// extractGRPPayload parses a raw hex packet and returns the GRP_TXT payload bytes.
func extractGRPPayload(rawHex string) ([]byte, error) {
buf, err := hex.DecodeString(strings.TrimSpace(rawHex))
if err != nil || len(buf) < 2 {
return nil, fmt.Errorf("invalid hex")
}
// Header byte
header := buf[0]
payloadType := int((header >> 2) & 0x0F)
if payloadType != 5 { // GRP_TXT
return nil, fmt.Errorf("not GRP_TXT")
}
routeType := int(header & 0x03)
offset := 1
// Transport codes (2 codes × 2 bytes) come BEFORE path for transport routes
if routeType == 0 || routeType == 3 {
offset += 4
}
// Path byte
if offset >= len(buf) {
return nil, fmt.Errorf("too short for path")
}
pathByte := buf[offset]
offset++
hashSize := int(pathByte>>6) + 1
hashCount := int(pathByte & 0x3F)
offset += hashSize * hashCount
if offset >= len(buf) {
return nil, fmt.Errorf("too short for payload")
}
return buf[offset:], nil
}
func getPathFromDB(db *sql.DB, txID int) []string {
var decodedJSON sql.NullString
err := db.QueryRow(`SELECT decoded_json FROM transmissions WHERE id = ?`, txID).Scan(&decodedJSON)
if err != nil || !decodedJSON.Valid {
return nil
}
var decoded struct {
Path struct {
Hops []string `json:"hops"`
} `json:"path"`
}
if json.Unmarshal([]byte(decodedJSON.String), &decoded) == nil {
return decoded.Path.Hops
}
return nil
}
func getObservers(db *sql.DB, txID int) []Observer {
rows, err := db.Query(`
SELECT o.name, obs.snr, obs.rssi, obs.timestamp
FROM observations obs
LEFT JOIN observers o ON o.id = CAST(obs.observer_idx AS TEXT)
WHERE obs.transmission_id = ?
ORDER BY obs.timestamp
`, txID)
if err != nil {
return nil
}
defer rows.Close()
var observers []Observer
for rows.Next() {
var name sql.NullString
var snr, rssi sql.NullFloat64
var ts int64
if err := rows.Scan(&name, &snr, &rssi, &ts); err != nil {
continue
}
obs := Observer{
Timestamp: time.Unix(ts, 0).UTC().Format(time.RFC3339),
}
if name.Valid {
obs.Name = name.String
}
if snr.Valid {
obs.SNR = snr.Float64
}
if rssi.Valid {
obs.RSSI = rssi.Float64
}
observers = append(observers, obs)
}
return observers
}
func renderIRC(messages []ChannelMessage) []byte {
var b strings.Builder
for _, m := range messages {
sender := m.Sender
if sender == "" {
sender = "???"
}
// Parse RFC3339 timestamp into a compact format
t, err := time.Parse(time.RFC3339, m.Timestamp)
if err != nil {
b.WriteString(fmt.Sprintf("[%s] <%s> %s\n", m.Timestamp, sender, m.Message))
continue
}
b.WriteString(fmt.Sprintf("[%s] <%s> %s\n", t.Format("2006-01-02 15:04:05"), sender, m.Message))
}
return []byte(b.String())
}
func renderHTML(messages []ChannelMessage, channelName string) []byte {
jsonData, _ := json.Marshal(messages)
var b strings.Builder
b.WriteString(`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>CoreScope Channel Export — ` + html.EscapeString(channelName) + `</title>
<style>
*{box-sizing:border-box;margin:0;padding:0}
body{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif;background:#0d1117;color:#c9d1d9;padding:20px}
h1{color:#58a6ff;margin-bottom:16px;font-size:1.5em}
.stats{color:#8b949e;margin-bottom:16px;font-size:0.9em}
input[type=text]{width:100%;max-width:500px;padding:8px 12px;background:#161b22;border:1px solid #30363d;border-radius:6px;color:#c9d1d9;font-size:14px;margin-bottom:16px}
input[type=text]:focus{outline:none;border-color:#58a6ff}
table{width:100%;border-collapse:collapse;font-size:14px}
th{background:#161b22;color:#8b949e;text-align:left;padding:8px 12px;border-bottom:2px solid #30363d;cursor:pointer;user-select:none;white-space:nowrap}
th:hover{color:#58a6ff}
th.sorted-asc::after{content:" ▲"}
th.sorted-desc::after{content:" ▼"}
td{padding:8px 12px;border-bottom:1px solid #21262d;vertical-align:top}
tr:hover{background:#161b22}
tr.expanded{background:#161b22}
.detail-row td{padding:12px 24px;background:#0d1117;border-bottom:1px solid #21262d}
.detail-row pre{background:#161b22;padding:12px;border-radius:6px;overflow-x:auto;font-size:12px;color:#8b949e}
.detail-row .label{color:#58a6ff;font-weight:600;margin-top:8px;display:block}
.observer-tag{display:inline-block;background:#1f6feb22;color:#58a6ff;padding:2px 8px;border-radius:4px;margin:2px;font-size:12px}
.no-results{color:#8b949e;text-align:center;padding:40px;font-size:16px}
.sender{color:#d2a8ff;font-weight:600}
.timestamp{color:#8b949e;font-family:monospace;font-size:12px}
</style>
</head>
<body>
<h1>` + html.EscapeString(channelName) + ` — Channel Messages</h1>
<div class="stats" id="stats"></div>
<input type="text" id="search" placeholder="Search messages..." autocomplete="off">
<table>
<thead>
<tr>
<th data-col="timestamp">Timestamp</th>
<th data-col="sender">Sender</th>
<th data-col="message">Message</th>
<th data-col="observers">Observers</th>
</tr>
</thead>
<tbody id="tbody"></tbody>
</table>
<div class="no-results" id="no-results" style="display:none">No matching messages</div>
<script>
var DATA=` + string(jsonData) + `;
var sortCol="timestamp",sortAsc=true,expandedHash=null;
function init(){
document.getElementById("stats").textContent=DATA.length+" messages";
document.getElementById("search").addEventListener("input",render);
document.querySelectorAll("th[data-col]").forEach(function(th){
th.addEventListener("click",function(){
var col=th.dataset.col;
if(sortCol===col)sortAsc=!sortAsc;
else{sortCol=col;sortAsc=true}
render();
});
});
render();
}
function render(){
var q=document.getElementById("search").value.toLowerCase();
var filtered=DATA.filter(function(m){
if(!q)return true;
return(m.message||"").toLowerCase().indexOf(q)>=0||(m.sender||"").toLowerCase().indexOf(q)>=0;
});
filtered.sort(function(a,b){
var va=a[sortCol]||"",vb=b[sortCol]||"";
if(sortCol==="observers"){va=a.observers?a.observers.length:0;vb=b.observers?b.observers.length:0}
if(va<vb)return sortAsc?-1:1;
if(va>vb)return sortAsc?1:-1;
return 0;
});
document.querySelectorAll("th[data-col]").forEach(function(th){
th.className=th.dataset.col===sortCol?(sortAsc?"sorted-asc":"sorted-desc"):"";
});
var tb=document.getElementById("tbody");
tb.innerHTML="";
document.getElementById("no-results").style.display=filtered.length?"none":"block";
filtered.forEach(function(m){
var tr=document.createElement("tr");
tr.innerHTML='<td class="timestamp">'+esc(m.timestamp)+'</td><td class="sender">'+esc(m.sender||"—")+'</td><td>'+esc(m.message)+'</td><td>'+
(m.observers?m.observers.map(function(o){return'<span class="observer-tag">'+esc(o.name||"?")+" SNR:"+o.snr.toFixed(1)+'</span>'}).join(""):"—")+'</td>';
tr.style.cursor="pointer";
tr.addEventListener("click",function(){
expandedHash=expandedHash===m.hash?null:m.hash;
render();
});
tb.appendChild(tr);
if(expandedHash===m.hash){
tr.className="expanded";
var dr=document.createElement("tr");
dr.className="detail-row";
dr.innerHTML='<td colspan="4"><span class="label">Hash</span><pre>'+esc(m.hash)+'</pre>'+
'<span class="label">Raw Hex</span><pre>'+esc(m.raw_hex)+'</pre>'+
(m.path&&m.path.length?'<span class="label">Path</span><pre>'+esc(m.path.join(" → "))+'</pre>':'')+
'<span class="label">Observers</span><pre>'+esc(JSON.stringify(m.observers,null,2))+'</pre></td>';
tb.appendChild(dr);
}
});
}
function esc(s){var d=document.createElement("div");d.textContent=s;return d.innerHTML}
init();
</script>
</body>
</html>`)
return []byte(b.String())
}
+129
View File
@@ -0,0 +1,129 @@
package main
import (
"encoding/hex"
"encoding/json"
"os"
"strings"
"testing"
"github.com/meshcore-analyzer/channel"
)
func TestExtractGRPPayload(t *testing.T) {
// Build a minimal GRP_TXT packet: header(1) + path(1) + payload
// header: route=FLOOD(1), payload=GRP_TXT(5), version=0 → (5<<2)|1 = 0x15
// path: 0 hops, hash_size=1 → 0x00
payload := []byte{0x81, 0x12, 0x34} // channel_hash + mac + data
pkt := append([]byte{0x15, 0x00}, payload...)
rawHex := hex.EncodeToString(pkt)
result, err := extractGRPPayload(rawHex)
if err != nil {
t.Fatal(err)
}
if len(result) != 3 || result[0] != 0x81 {
t.Fatalf("payload mismatch: %x", result)
}
}
func TestExtractGRPPayloadTransport(t *testing.T) {
// Transport flood: route=0, 4 bytes transport codes BEFORE path byte
// header: (5<<2)|0 = 0x14
payload := []byte{0xAA, 0xBB, 0xCC}
// header + 4 transport bytes + path(0 hops) + payload
pkt := append([]byte{0x14, 0xFF, 0xFF, 0xFF, 0xFF, 0x00}, payload...)
rawHex := hex.EncodeToString(pkt)
result, err := extractGRPPayload(rawHex)
if err != nil {
t.Fatal(err)
}
if result[0] != 0xAA {
t.Fatalf("expected AA, got %02X", result[0])
}
}
func TestExtractGRPPayloadNotGRP(t *testing.T) {
// payload type = ADVERT (4): (4<<2)|1 = 0x11
rawHex := hex.EncodeToString([]byte{0x11, 0x00, 0x01, 0x02})
_, err := extractGRPPayload(rawHex)
if err == nil {
t.Fatal("expected error for non-GRP_TXT")
}
}
func TestKeyDerivationConsistency(t *testing.T) {
// Verify key derivation matches what the ingestor expects
key := channel.DeriveKey("#wardriving")
if len(key) != 16 {
t.Fatalf("key len %d", len(key))
}
ch := channel.ChannelHash(key)
if ch != 0x81 {
// We know from fixture data that #wardriving has channelHashHex "81"
t.Fatalf("channel hash %02X, expected 81", ch)
}
}
func TestRenderIRC(t *testing.T) {
msgs := []ChannelMessage{
{Timestamp: "2026-04-12T03:45:12Z", Sender: "NodeA", Message: "Hello"},
{Timestamp: "2026-04-12T03:46:01Z", Sender: "", Message: "No sender"},
}
out := string(renderIRC(msgs))
if !strings.Contains(out, "[2026-04-12 03:45:12] <NodeA> Hello") {
t.Fatalf("IRC output missing expected line: %s", out)
}
if !strings.Contains(out, "<???> No sender") {
t.Fatalf("IRC output should use ??? for empty sender: %s", out)
}
}
func TestRenderHTMLValid(t *testing.T) {
msgs := []ChannelMessage{
{Hash: "abc", Timestamp: "2026-04-12T00:00:00Z", Sender: "X", Message: "test", Channel: "#test"},
}
out := string(renderHTML(msgs, "#test"))
if !strings.Contains(out, "<!DOCTYPE html>") {
t.Fatal("not valid HTML")
}
if !strings.Contains(out, "#test") {
t.Fatal("channel name missing")
}
if !strings.Contains(out, "</html>") {
t.Fatal("HTML not closed")
}
}
func TestJSONOutputParseable(t *testing.T) {
msgs := []ChannelMessage{
{Hash: "abc", Timestamp: "2026-04-12T00:00:00Z", Sender: "X", Message: "hi", Channel: "#test"},
}
data, err := json.MarshalIndent(msgs, "", " ")
if err != nil {
t.Fatal(err)
}
var parsed []ChannelMessage
if err := json.Unmarshal(data, &parsed); err != nil {
t.Fatalf("JSON not parseable: %v", err)
}
if len(parsed) != 1 || parsed[0].Sender != "X" {
t.Fatalf("parsed mismatch: %+v", parsed)
}
}
// Integration test against fixture DB (skipped if DB not found)
func TestFixtureDecrypt(t *testing.T) {
dbPath := "../../test-fixtures/e2e-fixture.db"
if _, err := os.Stat(dbPath); os.IsNotExist(err) {
t.Skip("fixture DB not found")
}
// We know the fixture has #wardriving messages with channelHash 0x81
key := channel.DeriveKey("#wardriving")
ch := channel.ChannelHash(key)
if ch != 0x81 {
t.Fatalf("unexpected channel hash: %02X", ch)
}
}
+11 -19
View File
@@ -203,21 +203,13 @@ func TestHandleMessageChannelMessage(t *testing.T) {
t.Errorf("direction=%v, want rx", direction)
}
// Should create sender node
// Sender node should NOT be created (see issue #665: synthetic "sender-" keys
// are unreachable from the claiming/health flow)
if err := store.db.QueryRow("SELECT COUNT(*) FROM nodes").Scan(&count); err != nil {
t.Fatal(err)
}
if count != 1 {
t.Errorf("nodes count=%d, want 1 (sender node)", count)
}
// Verify sender node name
var nodeName string
if err := store.db.QueryRow("SELECT name FROM nodes LIMIT 1").Scan(&nodeName); err != nil {
t.Fatal(err)
}
if nodeName != "Alice" {
t.Errorf("node name=%s, want Alice", nodeName)
if count != 0 {
t.Errorf("nodes count=%d, want 0 (no phantom sender node)", count)
}
}
@@ -461,7 +453,7 @@ func TestDecodeAdvertLocationTruncated(t *testing.T) {
buf[100] = 0x11
// Only 4 bytes after flags — not enough for full location (needs 8)
p := decodeAdvert(buf[:105])
p := decodeAdvert(buf[:105], false)
if p.Error != "" {
t.Fatalf("error: %s", p.Error)
}
@@ -483,7 +475,7 @@ func TestDecodeAdvertFeat1Truncated(t *testing.T) {
buf[100] = 0x21
// Only 1 byte after flags — not enough for feat1 (needs 2)
p := decodeAdvert(buf[:102])
p := decodeAdvert(buf[:102], false)
if p.Feat1 != nil {
t.Error("feat1 should be nil with truncated data")
}
@@ -504,7 +496,7 @@ func TestDecodeAdvertFeat2Truncated(t *testing.T) {
buf[102] = 0x00
// Only 1 byte left — not enough for feat2
p := decodeAdvert(buf[:104])
p := decodeAdvert(buf[:104], false)
if p.Feat1 == nil {
t.Error("feat1 should be set")
}
@@ -544,7 +536,7 @@ func TestDecodeAdvertSensorBadTelemetry(t *testing.T) {
buf[105] = 0x20
buf[106] = 0x4E
p := decodeAdvert(buf[:107])
p := decodeAdvert(buf[:107], false)
if p.BatteryMv != nil {
t.Error("battery_mv=0 should be nil")
}
@@ -740,7 +732,7 @@ func TestDecodeAdvertSensorNoName(t *testing.T) {
buf[103] = 0xC4
buf[104] = 0x09
p := decodeAdvert(buf[:105])
p := decodeAdvert(buf[:105], false)
if p.Error != "" {
t.Fatalf("error: %s", p.Error)
}
@@ -835,7 +827,7 @@ func TestDecodePacketNoPathByteAfterHeader(t *testing.T) {
// Non-transport route, but only header byte (no path byte)
// Actually 0A alone = 1 byte, but we need >= 2
// Header + exactly at offset boundary
_, err := DecodePacket("0A", nil)
_, err := DecodePacket("0A", nil, false)
if err == nil {
t.Error("should error - too short")
}
@@ -856,7 +848,7 @@ func TestDecodeAdvertNameNoNull(t *testing.T) {
// Name without null terminator — goes to end of buffer
copy(buf[101:], []byte("LongNameNoNull"))
p := decodeAdvert(buf[:115])
p := decodeAdvert(buf[:115], false)
if p.Name != "LongNameNoNull" {
t.Errorf("name=%q, want LongNameNoNull", p.Name)
}
+46 -4
View File
@@ -345,6 +345,28 @@ func applySchema(db *sql.DB) error {
log.Println("[migration] packets_sent/packets_recv columns added")
}
// Migration: add channel_hash column for fast channel queries (#762)
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'channel_hash_v1'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Adding channel_hash column to transmissions...")
db.Exec(`ALTER TABLE transmissions ADD COLUMN channel_hash TEXT DEFAULT NULL`)
db.Exec(`CREATE INDEX IF NOT EXISTS idx_tx_channel_hash ON transmissions(channel_hash) WHERE payload_type = 5`)
// Backfill: extract channel name for decrypted (CHAN) packets
res, err := db.Exec(`UPDATE transmissions SET channel_hash = json_extract(decoded_json, '$.channel') WHERE payload_type = 5 AND channel_hash IS NULL AND json_extract(decoded_json, '$.type') = 'CHAN'`)
if err == nil {
n, _ := res.RowsAffected()
log.Printf("[migration] Backfilled channel_hash for %d CHAN packets", n)
}
// Backfill: extract channelHashHex for encrypted (GRP_TXT) packets, prefixed with 'enc_'
res, err = db.Exec(`UPDATE transmissions SET channel_hash = 'enc_' || json_extract(decoded_json, '$.channelHashHex') WHERE payload_type = 5 AND channel_hash IS NULL AND json_extract(decoded_json, '$.type') = 'GRP_TXT'`)
if err == nil {
n, _ := res.RowsAffected()
log.Printf("[migration] Backfilled channel_hash for %d GRP_TXT packets", n)
}
db.Exec(`INSERT INTO _migrations (name) VALUES ('channel_hash_v1')`)
log.Println("[migration] channel_hash column added and backfilled")
}
return nil
}
@@ -357,8 +379,8 @@ func (s *Store) prepareStatements() error {
}
s.stmtInsertTransmission, err = s.db.Prepare(`
INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json)
VALUES (?, ?, ?, ?, ?, ?, ?)
INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json, channel_hash)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
`)
if err != nil {
return err
@@ -481,7 +503,7 @@ func (s *Store) InsertTransmission(data *PacketData) (bool, error) {
result, err := s.stmtInsertTransmission.Exec(
data.RawHex, hash, now,
data.RouteType, data.PayloadType, data.PayloadVersion,
data.DecodedJSON,
data.DecodedJSON, nilIfEmpty(data.ChannelHash),
)
if err != nil {
s.Stats.WriteErrors.Add(1)
@@ -773,6 +795,15 @@ type PacketData struct {
PayloadVersion int
PathJSON string
DecodedJSON string
ChannelHash string // grouping key for channel queries (#762)
}
// nilIfEmpty returns nil for empty strings (for nullable DB columns).
func nilIfEmpty(s string) interface{} {
if s == "" {
return nil
}
return s
}
// MQTTPacketMessage is the JSON payload from an MQTT raw packet message.
@@ -794,7 +825,7 @@ func BuildPacketData(msg *MQTTPacketMessage, decoded *DecodedPacket, observerID,
pathJSON = string(b)
}
return &PacketData{
pd := &PacketData{
RawHex: msg.Raw,
Timestamp: now,
ObserverID: observerID,
@@ -810,4 +841,15 @@ func BuildPacketData(msg *MQTTPacketMessage, decoded *DecodedPacket, observerID,
PathJSON: pathJSON,
DecodedJSON: PayloadJSON(&decoded.Payload),
}
// Populate channel_hash for fast channel queries (#762)
if decoded.Header.PayloadType == PayloadGRP_TXT {
if decoded.Payload.Type == "CHAN" && decoded.Payload.Channel != "" {
pd.ChannelHash = decoded.Payload.Channel
} else if decoded.Payload.Type == "GRP_TXT" && decoded.Payload.ChannelHashHex != "" {
pd.ChannelHash = "enc_" + decoded.Payload.ChannelHashHex
}
}
return pd
}
+6 -6
View File
@@ -576,7 +576,7 @@ func TestEndToEndIngest(t *testing.T) {
// Simulate full pipeline: decode + insert
rawHex := "120046D62DE27D4C5194D7821FC5A34A45565DCC2537B300B9AB6275255CEFB65D840CE5C169C94C9AED39E8BCB6CB6EB0335497A198B33A1A610CD3B03D8DCFC160900E5244280323EE0B44CACAB8F02B5B38B91CFA18BD067B0B5E63E94CFC85F758A8530B9240933402E0E6B8F84D5252322D52"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -764,7 +764,7 @@ func TestInsertTransmissionNilSNRRSSI(t *testing.T) {
func TestBuildPacketData(t *testing.T) {
rawHex := "0A00D69FD7A5A7475DB07337749AE61FA53A4788E976"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -818,7 +818,7 @@ func TestBuildPacketData(t *testing.T) {
func TestBuildPacketDataWithHops(t *testing.T) {
// A packet with actual hops in the path
raw := "0505AABBCCDDEE" + strings.Repeat("00", 10)
decoded, err := DecodePacket(raw, nil)
decoded, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -834,7 +834,7 @@ func TestBuildPacketDataWithHops(t *testing.T) {
}
func TestBuildPacketDataNilSNRRSSI(t *testing.T) {
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil)
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil, false)
msg := &MQTTPacketMessage{Raw: "0A00" + strings.Repeat("00", 10)}
pkt := BuildPacketData(msg, decoded, "", "")
@@ -1624,7 +1624,7 @@ func TestObsTimestampIndexMigration(t *testing.T) {
func TestBuildPacketDataScoreAndDirection(t *testing.T) {
rawHex := "0A00D69FD7A5A7475DB07337749AE61FA53A4788E976"
decoded, err := DecodePacket(rawHex, nil)
decoded, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1647,7 +1647,7 @@ func TestBuildPacketDataScoreAndDirection(t *testing.T) {
}
func TestBuildPacketDataNilScoreDirection(t *testing.T) {
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil)
decoded, _ := DecodePacket("0A00"+strings.Repeat("00", 10), nil, false)
msg := &MQTTPacketMessage{Raw: "0A00" + strings.Repeat("00", 10)}
pkt := BuildPacketData(msg, decoded, "", "")
+48 -14
View File
@@ -11,6 +11,8 @@ import (
"math"
"strings"
"unicode/utf8"
"github.com/meshcore-analyzer/sigvalidate"
)
// Route type constants (header bits 1-0)
@@ -78,9 +80,10 @@ type TransportCodes struct {
// Path holds decoded path/hop information.
type Path struct {
HashSize int `json:"hashSize"`
HashCount int `json:"hashCount"`
Hops []string `json:"hops"`
HashSize int `json:"hashSize"`
HashCount int `json:"hashCount"`
Hops []string `json:"hops"`
HopsCompleted *int `json:"hopsCompleted,omitempty"`
}
// AdvertFlags holds decoded advert flag bits.
@@ -109,6 +112,7 @@ type Payload struct {
Timestamp uint32 `json:"timestamp,omitempty"`
TimestampISO string `json:"timestampISO,omitempty"`
Signature string `json:"signature,omitempty"`
SignatureValid *bool `json:"signatureValid,omitempty"`
Flags *AdvertFlags `json:"flags,omitempty"`
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
@@ -140,6 +144,7 @@ type DecodedPacket struct {
Path Path `json:"path"`
Payload Payload `json:"payload"`
Raw string `json:"raw"`
Anomaly string `json:"anomaly,omitempty"`
}
func decodeHeader(b byte) Header {
@@ -215,7 +220,7 @@ func decodeAck(buf []byte) Payload {
}
}
func decodeAdvert(buf []byte) Payload {
func decodeAdvert(buf []byte, validateSignatures bool) Payload {
if len(buf) < 100 {
return Payload{Type: "ADVERT", Error: "too short for advert", RawHex: hex.EncodeToString(buf)}
}
@@ -233,6 +238,16 @@ func decodeAdvert(buf []byte) Payload {
Signature: signature,
}
if validateSignatures {
valid, err := sigvalidate.ValidateAdvert(buf[0:32], buf[36:100], timestamp, appdata)
if err != nil {
f := false
p.SignatureValid = &f
} else {
p.SignatureValid = &valid
}
}
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
@@ -506,7 +521,7 @@ func decodeTrace(buf []byte) Payload {
return p
}
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload {
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string, validateSignatures bool) Payload {
switch payloadType {
case PayloadREQ:
return decodeEncryptedPayload("REQ", buf)
@@ -517,7 +532,7 @@ func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) P
case PayloadACK:
return decodeAck(buf)
case PayloadADVERT:
return decodeAdvert(buf)
return decodeAdvert(buf, validateSignatures)
case PayloadGRP_TXT:
return decodeGrpTxt(buf, channelKeys)
case PayloadANON_REQ:
@@ -532,7 +547,7 @@ func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) P
}
// DecodePacket decodes a hex-encoded MeshCore packet.
func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPacket, error) {
func DecodePacket(hexString string, channelKeys map[string]string, validateSignatures bool) (*DecodedPacket, error) {
hexString = strings.ReplaceAll(hexString, " ", "")
hexString = strings.ReplaceAll(hexString, "\n", "")
hexString = strings.ReplaceAll(hexString, "\r", "")
@@ -570,20 +585,38 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
offset += bytesConsumed
payloadBuf := buf[offset:]
payload := decodePayload(header.PayloadType, payloadBuf, channelKeys)
payload := decodePayload(header.PayloadType, payloadBuf, channelKeys, validateSignatures)
// TRACE packets store hop IDs in the payload (buf[9:]) rather than the header
// path field. The header path byte still encodes hashSize in bits 6-7, which
// we use to split the payload path data into individual hop prefixes.
// path field. Firmware always sends TRACE as DIRECT (route_type 2 or 3);
// FLOOD-routed TRACEs are anomalous but handled gracefully (parsed, but
// flagged). The TRACE flags byte (payload offset 8) encodes path_sz in
// bits 0-1 as a power-of-two exponent: hash_bytes = 1 << path_sz.
// NOT the header path byte's hash_size bits. The header path contains SNR
// bytes — one per hop that actually forwarded.
// We expose hopsCompleted (count of SNR bytes) so consumers can distinguish
// how far the trace got vs the full intended route.
var anomaly string
if header.PayloadType == PayloadTRACE && payload.PathData != "" {
// Flag anomalous routing — firmware only sends TRACE as DIRECT
if header.RouteType != RouteDirect && header.RouteType != RouteTransportDirect {
anomaly = "TRACE packet with non-DIRECT routing (expected DIRECT or TRANSPORT_DIRECT)"
}
// The header path hops count represents SNR entries = completed hops
hopsCompleted := path.HashCount
pathBytes, err := hex.DecodeString(payload.PathData)
if err == nil && path.HashSize > 0 {
hops := make([]string, 0, len(pathBytes)/path.HashSize)
for i := 0; i+path.HashSize <= len(pathBytes); i += path.HashSize {
hops = append(hops, strings.ToUpper(hex.EncodeToString(pathBytes[i:i+path.HashSize])))
if err == nil && payload.TraceFlags != nil {
// path_sz from flags byte is a power-of-two exponent per firmware:
// hash_bytes = 1 << (flags & 0x03)
pathSz := 1 << (*payload.TraceFlags & 0x03)
hops := make([]string, 0, len(pathBytes)/pathSz)
for i := 0; i+pathSz <= len(pathBytes); i += pathSz {
hops = append(hops, strings.ToUpper(hex.EncodeToString(pathBytes[i:i+pathSz])))
}
path.Hops = hops
path.HashCount = len(hops)
path.HashSize = pathSz
path.HopsCompleted = &hopsCompleted
}
}
@@ -603,6 +636,7 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
Path: path,
Payload: payload,
Raw: strings.ToUpper(hexString),
Anomaly: anomaly,
}, nil
}
+174 -41
View File
@@ -2,6 +2,7 @@ package main
import (
"crypto/aes"
"crypto/ed25519"
"crypto/hmac"
"crypto/sha256"
"encoding/binary"
@@ -9,6 +10,8 @@ import (
"math"
"strings"
"testing"
"github.com/meshcore-analyzer/sigvalidate"
)
func TestDecodeHeaderRoutTypes(t *testing.T) {
@@ -55,7 +58,7 @@ func TestDecodeHeaderPayloadTypes(t *testing.T) {
func TestDecodePathZeroHops(t *testing.T) {
// 0x00: 0 hops, 1-byte hashes
pkt, err := DecodePacket("0500"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0500"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -72,7 +75,7 @@ func TestDecodePathZeroHops(t *testing.T) {
func TestDecodePath1ByteHashes(t *testing.T) {
// 0x05: 5 hops, 1-byte hashes → 5 path bytes
pkt, err := DecodePacket("0505"+"AABBCCDDEE"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0505"+"AABBCCDDEE"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -95,7 +98,7 @@ func TestDecodePath1ByteHashes(t *testing.T) {
func TestDecodePath2ByteHashes(t *testing.T) {
// 0x45: 5 hops, 2-byte hashes
pkt, err := DecodePacket("0545"+"AA11BB22CC33DD44EE55"+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("0545"+"AA11BB22CC33DD44EE55"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -112,7 +115,7 @@ func TestDecodePath2ByteHashes(t *testing.T) {
func TestDecodePath3ByteHashes(t *testing.T) {
// 0x8A: 10 hops, 3-byte hashes
pkt, err := DecodePacket("058A"+strings.Repeat("AA11FF", 10)+strings.Repeat("00", 10), nil)
pkt, err := DecodePacket("058A"+strings.Repeat("AA11FF", 10)+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -131,7 +134,7 @@ func TestTransportCodes(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) should have transport codes
// Firmware order: header + transport_codes(4) + path_len + path + payload
hex := "14" + "AABB" + "CCDD" + "00" + strings.Repeat("00", 10)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -149,7 +152,7 @@ func TestTransportCodes(t *testing.T) {
}
// Route type 1 (FLOOD) should NOT have transport codes
pkt2, err := DecodePacket("0500"+strings.Repeat("00", 10), nil)
pkt2, err := DecodePacket("0500"+strings.Repeat("00", 10), nil, false)
if err != nil {
t.Fatal(err)
}
@@ -169,7 +172,7 @@ func TestDecodeAdvertFull(t *testing.T) {
name := "546573744E6F6465" // "TestNode"
hex := "1200" + pubkey + timestamp + signature + flags + lat + lon + name
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -227,7 +230,7 @@ func TestDecodeAdvertTypeEnums(t *testing.T) {
makeAdvert := func(flagsByte byte) *DecodedPacket {
hex := "1200" + strings.Repeat("AA", 32) + "00000000" + strings.Repeat("BB", 64) +
strings.ToUpper(string([]byte{hexDigit(flagsByte>>4), hexDigit(flagsByte & 0x0f)}))
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -272,7 +275,7 @@ func hexDigit(v byte) byte {
func TestDecodeAdvertNoLocationNoName(t *testing.T) {
hex := "1200" + strings.Repeat("CC", 32) + "00000000" + strings.Repeat("DD", 64) + "02"
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -291,7 +294,7 @@ func TestDecodeAdvertNoLocationNoName(t *testing.T) {
}
func TestGoldenFixtureTxtMsg(t *testing.T) {
pkt, err := DecodePacket("0A00D69FD7A5A7475DB07337749AE61FA53A4788E976", nil)
pkt, err := DecodePacket("0A00D69FD7A5A7475DB07337749AE61FA53A4788E976", nil, false)
if err != nil {
t.Fatal(err)
}
@@ -314,7 +317,7 @@ func TestGoldenFixtureTxtMsg(t *testing.T) {
func TestGoldenFixtureAdvert(t *testing.T) {
rawHex := "120046D62DE27D4C5194D7821FC5A34A45565DCC2537B300B9AB6275255CEFB65D840CE5C169C94C9AED39E8BCB6CB6EB0335497A198B33A1A610CD3B03D8DCFC160900E5244280323EE0B44CACAB8F02B5B38B91CFA18BD067B0B5E63E94CFC85F758A8530B9240933402E0E6B8F84D5252322D52"
pkt, err := DecodePacket(rawHex, nil)
pkt, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -337,7 +340,7 @@ func TestGoldenFixtureAdvert(t *testing.T) {
func TestGoldenFixtureUnicodeAdvert(t *testing.T) {
rawHex := "120073CFF971E1CB5754A742C152B2D2E0EB108A19B246D663ED8898A72C4A5AD86EA6768E66694B025EDF6939D5C44CFF719C5D5520E5F06B20680A83AD9C2C61C3227BBB977A85EE462F3553445FECF8EDD05C234ECE217272E503F14D6DF2B1B9B133890C923CDF3002F8FDC1F85045414BF09F8CB3"
pkt, err := DecodePacket(rawHex, nil)
pkt, err := DecodePacket(rawHex, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -354,14 +357,14 @@ func TestGoldenFixtureUnicodeAdvert(t *testing.T) {
}
func TestDecodePacketTooShort(t *testing.T) {
_, err := DecodePacket("FF", nil)
_, err := DecodePacket("FF", nil, false)
if err == nil {
t.Error("expected error for 1-byte packet")
}
}
func TestDecodePacketInvalidHex(t *testing.T) {
_, err := DecodePacket("ZZZZ", nil)
_, err := DecodePacket("ZZZZ", nil, false)
if err == nil {
t.Error("expected error for invalid hex")
}
@@ -568,7 +571,7 @@ func TestDecodeTracePathParsing(t *testing.T) {
// Packet from issue #276: 260001807dca00000000007d547d
// Path byte 0x00 → hashSize=1, hops in payload at buf[9:] = 7d 54 7d
// Expected path: ["7D", "54", "7D"]
pkt, err := DecodePacket("260001807dca00000000007d547d", nil)
pkt, err := DecodePacket("260001807dca00000000007d547d", nil, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
@@ -590,7 +593,7 @@ func TestDecodeTracePathParsing(t *testing.T) {
}
func TestDecodeAdvertShort(t *testing.T) {
p := decodeAdvert(make([]byte, 50))
p := decodeAdvert(make([]byte, 50), false)
if p.Error != "too short for advert" {
t.Errorf("expected 'too short for advert' error, got %q", p.Error)
}
@@ -628,7 +631,7 @@ func TestDecodeEncryptedPayloadValid(t *testing.T) {
func TestDecodePayloadGRPData(t *testing.T) {
buf := []byte{0x01, 0x02, 0x03}
p := decodePayload(PayloadGRP_DATA, buf, nil)
p := decodePayload(PayloadGRP_DATA, buf, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("type=%s, want UNKNOWN", p.Type)
}
@@ -639,7 +642,7 @@ func TestDecodePayloadGRPData(t *testing.T) {
func TestDecodePayloadRAWCustom(t *testing.T) {
buf := []byte{0xFF, 0xFE}
p := decodePayload(PayloadRAW_CUSTOM, buf, nil)
p := decodePayload(PayloadRAW_CUSTOM, buf, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("type=%s, want UNKNOWN", p.Type)
}
@@ -647,49 +650,49 @@ func TestDecodePayloadRAWCustom(t *testing.T) {
func TestDecodePayloadAllTypes(t *testing.T) {
// REQ
p := decodePayload(PayloadREQ, make([]byte, 10), nil)
p := decodePayload(PayloadREQ, make([]byte, 10), nil, false)
if p.Type != "REQ" {
t.Errorf("REQ: type=%s", p.Type)
}
// RESPONSE
p = decodePayload(PayloadRESPONSE, make([]byte, 10), nil)
p = decodePayload(PayloadRESPONSE, make([]byte, 10), nil, false)
if p.Type != "RESPONSE" {
t.Errorf("RESPONSE: type=%s", p.Type)
}
// TXT_MSG
p = decodePayload(PayloadTXT_MSG, make([]byte, 10), nil)
p = decodePayload(PayloadTXT_MSG, make([]byte, 10), nil, false)
if p.Type != "TXT_MSG" {
t.Errorf("TXT_MSG: type=%s", p.Type)
}
// ACK
p = decodePayload(PayloadACK, make([]byte, 10), nil)
p = decodePayload(PayloadACK, make([]byte, 10), nil, false)
if p.Type != "ACK" {
t.Errorf("ACK: type=%s", p.Type)
}
// GRP_TXT
p = decodePayload(PayloadGRP_TXT, make([]byte, 10), nil)
p = decodePayload(PayloadGRP_TXT, make([]byte, 10), nil, false)
if p.Type != "GRP_TXT" {
t.Errorf("GRP_TXT: type=%s", p.Type)
}
// ANON_REQ
p = decodePayload(PayloadANON_REQ, make([]byte, 40), nil)
p = decodePayload(PayloadANON_REQ, make([]byte, 40), nil, false)
if p.Type != "ANON_REQ" {
t.Errorf("ANON_REQ: type=%s", p.Type)
}
// PATH
p = decodePayload(PayloadPATH, make([]byte, 10), nil)
p = decodePayload(PayloadPATH, make([]byte, 10), nil, false)
if p.Type != "PATH" {
t.Errorf("PATH: type=%s", p.Type)
}
// TRACE
p = decodePayload(PayloadTRACE, make([]byte, 20), nil)
p = decodePayload(PayloadTRACE, make([]byte, 20), nil, false)
if p.Type != "TRACE" {
t.Errorf("TRACE: type=%s", p.Type)
}
@@ -925,7 +928,7 @@ func TestComputeContentHashLongFallback(t *testing.T) {
func TestDecodePacketWithWhitespace(t *testing.T) {
raw := "0A 00 D6 9F D7 A5 A7 47 5D B0 73 37 74 9A E6 1F A5 3A 47 88 E9 76"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -936,7 +939,7 @@ func TestDecodePacketWithWhitespace(t *testing.T) {
func TestDecodePacketWithNewlines(t *testing.T) {
raw := "0A00\nD69F\r\nD7A5A7475DB07337749AE61FA53A4788E976"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -947,7 +950,7 @@ func TestDecodePacketWithNewlines(t *testing.T) {
func TestDecodePacketTransportRouteTooShort(t *testing.T) {
// TRANSPORT_FLOOD (route=0) but only 2 bytes total → too short for transport codes
_, err := DecodePacket("1400", nil)
_, err := DecodePacket("1400", nil, false)
if err == nil {
t.Error("expected error for transport route with too-short buffer")
}
@@ -1007,7 +1010,7 @@ func TestDecodeHeaderUnknownTypes(t *testing.T) {
func TestDecodePayloadMultipart(t *testing.T) {
// MULTIPART (0x0A) falls through to default → UNKNOWN
p := decodePayload(PayloadMULTIPART, []byte{0x01, 0x02}, nil)
p := decodePayload(PayloadMULTIPART, []byte{0x01, 0x02}, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("MULTIPART type=%s, want UNKNOWN", p.Type)
}
@@ -1015,7 +1018,7 @@ func TestDecodePayloadMultipart(t *testing.T) {
func TestDecodePayloadControl(t *testing.T) {
// CONTROL (0x0B) falls through to default → UNKNOWN
p := decodePayload(PayloadCONTROL, []byte{0x01, 0x02}, nil)
p := decodePayload(PayloadCONTROL, []byte{0x01, 0x02}, nil, false)
if p.Type != "UNKNOWN" {
t.Errorf("CONTROL type=%s, want UNKNOWN", p.Type)
}
@@ -1039,7 +1042,7 @@ func TestDecodePathTruncatedBuffer(t *testing.T) {
func TestDecodeFloodAdvert5Hops(t *testing.T) {
// From test-decoder.js Test 1
raw := "11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172"
pkt, err := DecodePacket(raw, nil)
pkt, err := DecodePacket(raw, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1410,7 +1413,7 @@ func TestDecodeAdvertWithTelemetry(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1449,7 +1452,7 @@ func TestDecodeAdvertWithTelemetryNegativeTemp(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1476,7 +1479,7 @@ func TestDecodeAdvertWithoutTelemetry(t *testing.T) {
name := hex.EncodeToString([]byte("Node1"))
hexStr := "1200" + pubkey + timestamp + signature + flags + name
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1503,7 +1506,7 @@ func TestDecodeAdvertNonSensorIgnoresTelemetryBytes(t *testing.T) {
extraBytes := "B40ED403" // battery-like and temp-like bytes
hexStr := "1200" + pubkey + timestamp + signature + flags + name + nullTerm + extraBytes
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1531,7 +1534,7 @@ func TestDecodeAdvertTelemetryZeroTemp(t *testing.T) {
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
pkt, err := DecodePacket(hexStr, nil, false)
if err != nil {
t.Fatal(err)
}
@@ -1555,7 +1558,7 @@ func TestZeroHopDirectHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x00 → hash_count=0, hash_size bits=0 → should get HashSize=0
hex := "02" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -1568,7 +1571,7 @@ func TestZeroHopDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x40 → hash_count=0, hash_size bits=01 → should still get HashSize=0
hex := "02" + "40" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -1581,7 +1584,7 @@ func TestNonDirectZeroPathByteKeepsHashSize(t *testing.T) {
// FLOOD (RouteType=1) + REQ (PayloadType=0) → header byte = 0x01
// pathByte=0x00 → non-DIRECT should keep HashSize=1
hex := "01" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -1594,7 +1597,7 @@ func TestDirectNonZeroHopKeepsHashSize(t *testing.T) {
// DIRECT (RouteType=2) + REQ (PayloadType=0) → header byte = 0x02
// pathByte=0x01 → hash_count=1, hash_size=1 → should keep HashSize=1
hex := "02" + "01" + repeatHex("BB", 21)
pkt, err := DecodePacket(hex, nil)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -1602,3 +1605,133 @@ func TestDirectNonZeroHopKeepsHashSize(t *testing.T) {
t.Errorf("DIRECT with 1 hop: want HashSize=1, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopTransportDirectHashSize(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0x00 → hash_count=0 → should get HashSize=0
hex := "03" + "11223344" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("TRANSPORT_DIRECT zero-hop: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestZeroHopTransportDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0xC0 → hash_count=0, hash_size bits=11 → should still get HashSize=0
hex := "03" + "11223344" + "C0" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex, nil, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
if pkt.Path.HashSize != 0 {
t.Errorf("TRANSPORT_DIRECT zero-hop with hash_size bits set: want HashSize=0, got %d", pkt.Path.HashSize)
}
}
func TestValidateAdvertSignature(t *testing.T) {
// Generate a real ed25519 key pair
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1234567890
appdata := []byte{0x02, 0x11, 0x22} // flags + some data
// Build the signed message: pubKey + timestamp(LE) + appdata
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Valid signature
valid, err := sigvalidate.ValidateAdvert([]byte(pub), sig, timestamp, appdata)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !valid {
t.Error("expected valid signature")
}
// Tampered appdata → invalid
badAppdata := []byte{0x03, 0x11, 0x22}
valid, err = sigvalidate.ValidateAdvert([]byte(pub), sig, timestamp, badAppdata)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if valid {
t.Error("expected invalid signature with tampered appdata")
}
// Wrong timestamp → invalid
valid, err = sigvalidate.ValidateAdvert([]byte(pub), sig, timestamp+1, appdata)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if valid {
t.Error("expected invalid signature with wrong timestamp")
}
// Wrong length pubkey
_, err = sigvalidate.ValidateAdvert([]byte{0xAA, 0xBB}, sig, timestamp, appdata)
if err == nil {
t.Error("expected error for short pubkey")
}
// Wrong length signature
_, err = sigvalidate.ValidateAdvert([]byte(pub), []byte{0xAA, 0xBB}, timestamp, appdata)
if err == nil {
t.Error("expected error for short signature")
}
}
func TestDecodeAdvertWithSignatureValidation(t *testing.T) {
// Generate key pair
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1000000
appdata := []byte{0x02} // repeater type, no location
// Build signed message
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pub)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
sig := ed25519.Sign(priv, message)
// Build advert buffer: pubkey(32) + timestamp(4) + signature(64) + appdata
buf := make([]byte, 0, 101)
buf = append(buf, pub...)
ts := make([]byte, 4)
binary.LittleEndian.PutUint32(ts, timestamp)
buf = append(buf, ts...)
buf = append(buf, sig...)
buf = append(buf, appdata...)
// With validation enabled
p := decodeAdvert(buf, true)
if p.Error != "" {
t.Fatalf("decode error: %s", p.Error)
}
if p.SignatureValid == nil {
t.Fatal("SignatureValid should be set when validation enabled")
}
if !*p.SignatureValid {
t.Error("expected valid signature")
}
// Without validation
p2 := decodeAdvert(buf, false)
if p2.SignatureValid != nil {
t.Error("SignatureValid should be nil when validation disabled")
}
}
+3
View File
@@ -5,11 +5,14 @@ go 1.22
require (
github.com/eclipse/paho.mqtt.golang v1.5.0
github.com/meshcore-analyzer/geofilter v0.0.0
github.com/meshcore-analyzer/sigvalidate v0.0.0
modernc.org/sqlite v1.34.5
)
replace github.com/meshcore-analyzer/geofilter => ../../internal/geofilter
replace github.com/meshcore-analyzer/sigvalidate => ../../internal/sigvalidate
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
+7 -8
View File
@@ -248,7 +248,7 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
// Format 1: Raw packet (meshcoretomqtt / Cisien format)
rawHex, _ := msg["raw"].(string)
if rawHex != "" {
decoded, err := DecodePacket(rawHex, channelKeys)
decoded, err := DecodePacket(rawHex, channelKeys, false)
if err != nil {
log.Printf("MQTT [%s] decode error: %v", tag, err)
return
@@ -440,19 +440,18 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
PayloadType: 5, // GRP_TXT
PathJSON: "[]",
DecodedJSON: string(decodedJSON),
ChannelHash: channelName, // fast channel queries (#762)
}
if _, err := store.InsertTransmission(pktData); err != nil {
log.Printf("MQTT [%s] channel insert error: %v", tag, err)
}
// Upsert sender as a companion node
if sender != "" {
senderKey := "sender-" + strings.ToLower(sender)
if err := store.UpsertNode(senderKey, sender, "companion", nil, nil, now); err != nil {
log.Printf("MQTT [%s] sender node upsert error: %v", tag, err)
}
}
// Note: we intentionally do NOT create a node entry for channel message senders.
// Channel messages don't carry the sender's real pubkey, so any entry we create
// would use a synthetic key ("sender-<name>") that doesn't match the real pubkey
// used for claiming/health lookups. The node will get a proper entry when it
// sends an advert. See issue #665.
log.Printf("MQTT [%s] channel message: ch%s from %s", tag, channelIdx, firstNonEmpty(sender, "unknown"))
return
+196 -15
View File
@@ -9,14 +9,15 @@ import (
func newTestStore(t *testing.T) *PacketStore {
t.Helper()
return &PacketStore{
rfCache: make(map[string]*cachedResult),
topoCache: make(map[string]*cachedResult),
hashCache: make(map[string]*cachedResult),
chanCache: make(map[string]*cachedResult),
distCache: make(map[string]*cachedResult),
subpathCache: make(map[string]*cachedResult),
rfCacheTTL: 15 * time.Second,
invCooldown: 10 * time.Second,
rfCache: make(map[string]*cachedResult),
topoCache: make(map[string]*cachedResult),
hashCache: make(map[string]*cachedResult),
collisionCache: make(map[string]*cachedResult),
chanCache: make(map[string]*cachedResult),
distCache: make(map[string]*cachedResult),
subpathCache: make(map[string]*cachedResult),
rfCacheTTL: 15 * time.Second,
invCooldown: 10 * time.Second,
}
}
@@ -29,6 +30,7 @@ func populateAllCaches(s *PacketStore) {
s.rfCache["global"] = dummy
s.topoCache["global"] = dummy
s.hashCache["global"] = dummy
s.collisionCache["global"] = dummy
s.chanCache["global"] = dummy
s.distCache["global"] = dummy
s.subpathCache["global"] = dummy
@@ -39,12 +41,13 @@ func cachePopulated(s *PacketStore) map[string]bool {
s.cacheMu.Lock()
defer s.cacheMu.Unlock()
return map[string]bool{
"rf": len(s.rfCache) > 0,
"topo": len(s.topoCache) > 0,
"hash": len(s.hashCache) > 0,
"chan": len(s.chanCache) > 0,
"dist": len(s.distCache) > 0,
"subpath": len(s.subpathCache) > 0,
"rf": len(s.rfCache) > 0,
"topo": len(s.topoCache) > 0,
"hash": len(s.hashCache) > 0,
"collision": len(s.collisionCache) > 0,
"chan": len(s.chanCache) > 0,
"dist": len(s.distCache) > 0,
"subpath": len(s.subpathCache) > 0,
}
}
@@ -90,7 +93,8 @@ func TestInvalidateCachesFor_NewTransmissionsOnly(t *testing.T) {
if pop["hash"] {
t.Error("hash cache should be cleared on new transmissions")
}
for _, name := range []string{"rf", "topo", "chan", "dist", "subpath"} {
// collisionCache should NOT be cleared by transmissions alone (only by hasNewNodes)
for _, name := range []string{"rf", "topo", "collision", "chan", "dist", "subpath"} {
if !pop[name] {
t.Errorf("%s cache should NOT be cleared on transmission-only ingest", name)
}
@@ -331,3 +335,180 @@ func BenchmarkCacheHitDuringIngestion(b *testing.B) {
}
b.ReportMetric(float64(hits)/float64(hits+misses)*100, "hit%")
}
// TestInvCooldownFromConfig verifies that invalidationDebounce from config
// is wired to invCooldown on PacketStore.
func TestInvCooldownFromConfig(t *testing.T) {
// Default without config
ps := NewPacketStore(nil, nil)
if ps.invCooldown != 300*time.Second {
t.Errorf("default invCooldown = %v, want 300s", ps.invCooldown)
}
// With config override
ct := map[string]interface{}{"invalidationDebounce": float64(60)}
ps2 := NewPacketStore(nil, nil, ct)
if ps2.invCooldown != 60*time.Second {
t.Errorf("configured invCooldown = %v, want 60s", ps2.invCooldown)
}
}
// TestCollisionCacheNotClearedByTransmissions verifies that collisionCache
// is only cleared by hasNewNodes, not hasNewTransmissions (fixes #720).
func TestCollisionCacheNotClearedByTransmissions(t *testing.T) {
s := newTestStore(t)
populateAllCaches(s)
s.invalidateCachesFor(cacheInvalidation{hasNewTransmissions: true})
pop := cachePopulated(s)
if !pop["collision"] {
t.Error("collisionCache should NOT be cleared by hasNewTransmissions alone")
}
if pop["hash"] {
t.Error("hashCache should be cleared by hasNewTransmissions")
}
}
// TestCollisionCacheClearedByNewNodes verifies that collisionCache IS cleared
// when genuinely new nodes are discovered.
func TestCollisionCacheClearedByNewNodes(t *testing.T) {
s := newTestStore(t)
populateAllCaches(s)
s.invalidateCachesFor(cacheInvalidation{hasNewNodes: true})
pop := cachePopulated(s)
if pop["collision"] {
t.Error("collisionCache should be cleared by hasNewNodes")
}
// Other caches should survive
for _, name := range []string{"rf", "topo", "hash", "chan", "dist", "subpath"} {
if !pop[name] {
t.Errorf("%s cache should NOT be cleared on new-nodes-only ingest", name)
}
}
}
// TestCacheSurvivesMultipleIngestCyclesWithinCooldown verifies that caches
// survive repeated ingest cycles during the cooldown period.
func TestCacheSurvivesMultipleIngestCyclesWithinCooldown(t *testing.T) {
s := newTestStore(t)
s.invCooldown = 200 * time.Millisecond
// First invalidation goes through (starts cooldown)
populateAllCaches(s)
s.invalidateCachesFor(cacheInvalidation{hasNewObservations: true})
pop := cachePopulated(s)
if pop["rf"] {
t.Error("rf should be cleared on first invalidation")
}
// Repopulate and simulate 5 rapid ingest cycles
populateAllCaches(s)
for i := 0; i < 5; i++ {
s.invalidateCachesFor(cacheInvalidation{
hasNewObservations: true,
hasNewTransmissions: true,
hasNewPaths: true,
})
}
// All caches should survive during cooldown
pop = cachePopulated(s)
for name, has := range pop {
if !has {
t.Errorf("%s cache should survive during cooldown period (ingest cycle %d)", name, 5)
}
}
}
// TestNewNodesAccumulatedDuringCooldown verifies that hasNewNodes flags
// accumulated during cooldown are applied when cooldown expires.
func TestNewNodesAccumulatedDuringCooldown(t *testing.T) {
s := newTestStore(t)
s.invCooldown = 100 * time.Millisecond
// First call starts cooldown
s.invalidateCachesFor(cacheInvalidation{hasNewObservations: true})
// During cooldown, accumulate hasNewNodes
s.invalidateCachesFor(cacheInvalidation{hasNewNodes: true})
// Verify accumulated
s.cacheMu.Lock()
if s.pendingInv == nil || !s.pendingInv.hasNewNodes {
t.Error("hasNewNodes should be accumulated in pendingInv")
}
s.cacheMu.Unlock()
// Wait for cooldown
time.Sleep(150 * time.Millisecond)
// Trigger flush
populateAllCaches(s)
s.invalidateCachesFor(cacheInvalidation{})
pop := cachePopulated(s)
if pop["collision"] {
t.Error("collisionCache should be cleared after pending hasNewNodes is flushed")
}
}
// BenchmarkAnalyticsLatencyCacheHitVsMiss benchmarks cache hit vs miss
// for analytics endpoints to demonstrate the performance impact.
func BenchmarkAnalyticsLatencyCacheHitVsMiss(b *testing.B) {
s := &PacketStore{
rfCache: make(map[string]*cachedResult),
topoCache: make(map[string]*cachedResult),
hashCache: make(map[string]*cachedResult),
collisionCache: make(map[string]*cachedResult),
chanCache: make(map[string]*cachedResult),
distCache: make(map[string]*cachedResult),
subpathCache: make(map[string]*cachedResult),
rfCacheTTL: 1800 * time.Second,
invCooldown: 300 * time.Second,
}
// Pre-populate cache
s.cacheMu.Lock()
s.rfCache["global"] = &cachedResult{
data: map[string]interface{}{"bins": make([]int, 100)},
expiresAt: time.Now().Add(time.Hour),
}
s.cacheMu.Unlock()
// Trigger initial invalidation to start cooldown
s.invalidateCachesFor(cacheInvalidation{hasNewObservations: true})
var hits, misses int64
for i := 0; i < b.N; i++ {
// Re-populate (simulates query filling cache)
s.cacheMu.Lock()
if len(s.rfCache) == 0 {
s.rfCache["global"] = &cachedResult{
data: map[string]interface{}{"bins": make([]int, 100)},
expiresAt: time.Now().Add(time.Hour),
}
}
s.cacheMu.Unlock()
// Simulate ingest (rate-limited)
s.invalidateCachesFor(cacheInvalidation{hasNewObservations: true})
// Check hit
s.cacheMu.Lock()
if len(s.rfCache) > 0 {
hits++
} else {
misses++
}
s.cacheMu.Unlock()
}
hitRate := float64(hits) / float64(hits+misses) * 100
b.ReportMetric(hitRate, "hit%")
if hitRate < 50 {
b.Errorf("hit rate %.1f%% is below 50%% target", hitRate)
}
}
+580
View File
@@ -0,0 +1,580 @@
package main
import (
"math"
"sort"
"sync"
"time"
)
// ── Clock Skew Severity ────────────────────────────────────────────────────────
type SkewSeverity string
const (
SkewOK SkewSeverity = "ok" // < 5 min
SkewWarning SkewSeverity = "warning" // 5 min 1 hour
SkewCritical SkewSeverity = "critical" // 1 hour 30 days
SkewAbsurd SkewSeverity = "absurd" // > 30 days
SkewNoClock SkewSeverity = "no_clock" // > 365 days — uninitialized RTC
)
// Default thresholds in seconds.
const (
skewThresholdWarnSec = 5 * 60 // 5 minutes
skewThresholdCriticalSec = 60 * 60 // 1 hour
skewThresholdAbsurdSec = 30 * 24 * 3600 // 30 days
skewThresholdNoClockSec = 365 * 24 * 3600 // 365 days — uninitialized RTC
// minDriftSamples is the minimum number of advert transmissions needed
// to compute a meaningful linear drift rate.
minDriftSamples = 5
// maxReasonableDriftPerDay caps drift display. Physically impossible
// drift rates (> 1 day/day) indicate insufficient or outlier samples.
maxReasonableDriftPerDay = 86400.0
)
// classifySkew maps absolute skew (seconds) to a severity level.
// Float64 comparison is safe: inputs are rounded to 1 decimal via round(),
// and thresholds are integer multiples of 60 — no rounding artifacts.
func classifySkew(absSkewSec float64) SkewSeverity {
switch {
case absSkewSec >= skewThresholdNoClockSec:
return SkewNoClock
case absSkewSec >= skewThresholdAbsurdSec:
return SkewAbsurd
case absSkewSec >= skewThresholdCriticalSec:
return SkewCritical
case absSkewSec >= skewThresholdWarnSec:
return SkewWarning
default:
return SkewOK
}
}
// ── Data Types ─────────────────────────────────────────────────────────────────
// skewSample is a single raw skew measurement from one advert observation.
type skewSample struct {
advertTS int64 // node's advert Unix timestamp
observedTS int64 // observation Unix timestamp
observerID string // which observer saw this
hash string // transmission hash (for multi-observer grouping)
}
// ObserverCalibration holds the computed clock offset for an observer.
type ObserverCalibration struct {
ObserverID string `json:"observerID"`
OffsetSec float64 `json:"offsetSec"` // positive = observer clock ahead
Samples int `json:"samples"` // number of multi-observer packets used
}
// NodeClockSkew is the API response for a single node's clock skew data.
type NodeClockSkew struct {
Pubkey string `json:"pubkey"`
MeanSkewSec float64 `json:"meanSkewSec"` // corrected mean skew (positive = node ahead)
MedianSkewSec float64 `json:"medianSkewSec"` // corrected median skew
LastSkewSec float64 `json:"lastSkewSec"` // most recent corrected skew
DriftPerDaySec float64 `json:"driftPerDaySec"` // linear drift rate (sec/day)
Severity SkewSeverity `json:"severity"`
SampleCount int `json:"sampleCount"`
Calibrated bool `json:"calibrated"` // true if observer calibration was applied
LastAdvertTS int64 `json:"lastAdvertTS"` // most recent advert timestamp
LastObservedTS int64 `json:"lastObservedTS"` // most recent observation timestamp
Samples []SkewSample `json:"samples,omitempty"` // time-series for sparklines
NodeName string `json:"nodeName,omitempty"` // populated in fleet responses
NodeRole string `json:"nodeRole,omitempty"` // populated in fleet responses
}
// SkewSample is a single (timestamp, skew) point for sparkline rendering.
type SkewSample struct {
Timestamp int64 `json:"ts"` // Unix epoch of observation
SkewSec float64 `json:"skew"` // corrected skew in seconds
}
// txSkewResult maps tx hash → per-transmission skew stats. This is an
// intermediate result keyed by hash (not pubkey); the store maps hash → pubkey
// when building the final per-node view.
type txSkewResult = map[string]*NodeClockSkew
// ── Clock Skew Engine ──────────────────────────────────────────────────────────
// ClockSkewEngine computes and caches clock skew data for nodes and observers.
type ClockSkewEngine struct {
mu sync.RWMutex
observerOffsets map[string]float64 // observerID → calibrated offset (seconds)
observerSamples map[string]int // observerID → number of multi-observer packets used
nodeSkew txSkewResult
lastComputed time.Time
computeInterval time.Duration
}
func NewClockSkewEngine() *ClockSkewEngine {
return &ClockSkewEngine{
observerOffsets: make(map[string]float64),
observerSamples: make(map[string]int),
nodeSkew: make(txSkewResult),
computeInterval: 30 * time.Second,
}
}
// Recompute recalculates all clock skew data from the packet store.
// Called periodically or on demand. Holds store RLock externally.
// Uses read-copy-update: heavy computation runs outside the write lock,
// then results are swapped in under a brief lock.
func (e *ClockSkewEngine) Recompute(store *PacketStore) {
// Fast path: check under read lock if recompute is needed.
e.mu.RLock()
fresh := time.Since(e.lastComputed) < e.computeInterval
e.mu.RUnlock()
if fresh {
return
}
// Phase 1: Collect skew samples from ADVERT packets (store RLock held by caller).
samples := collectSamples(store)
// Phase 23: Compute outside the write lock.
var newOffsets map[string]float64
var newSamples map[string]int
var newNodeSkew txSkewResult
if len(samples) > 0 {
newOffsets, newSamples = calibrateObservers(samples)
newNodeSkew = computeNodeSkew(samples, newOffsets)
} else {
newOffsets = make(map[string]float64)
newSamples = make(map[string]int)
newNodeSkew = make(txSkewResult)
}
// Swap results under brief write lock.
e.mu.Lock()
// Re-check: another goroutine may have computed while we were working.
if time.Since(e.lastComputed) < e.computeInterval {
e.mu.Unlock()
return
}
e.observerOffsets = newOffsets
e.observerSamples = newSamples
e.nodeSkew = newNodeSkew
e.lastComputed = time.Now()
e.mu.Unlock()
}
// collectSamples extracts skew samples from ADVERT packets in the store.
// Must be called with store.mu held (at least RLock).
func collectSamples(store *PacketStore) []skewSample {
adverts := store.byPayloadType[PayloadADVERT]
if len(adverts) == 0 {
return nil
}
samples := make([]skewSample, 0, len(adverts)*2)
for _, tx := range adverts {
decoded := tx.ParsedDecoded()
if decoded == nil {
continue
}
// Extract advert timestamp from decoded JSON.
advertTS := extractTimestamp(decoded)
if advertTS <= 0 {
continue
}
// Sanity: skip timestamps before year 2020 or after year 2100.
if advertTS < 1577836800 || advertTS > 4102444800 {
continue
}
for _, obs := range tx.Observations {
obsTS := parseISO(obs.Timestamp)
if obsTS <= 0 {
continue
}
samples = append(samples, skewSample{
advertTS: advertTS,
observedTS: obsTS,
observerID: obs.ObserverID,
hash: tx.Hash,
})
}
}
return samples
}
// extractTimestamp gets the Unix timestamp from a decoded ADVERT payload.
func extractTimestamp(decoded map[string]interface{}) int64 {
// Try payload.timestamp first (nested in "payload" key).
if payload, ok := decoded["payload"]; ok {
if pm, ok := payload.(map[string]interface{}); ok {
if ts := jsonNumber(pm, "timestamp"); ts > 0 {
return ts
}
}
}
// Fallback: top-level timestamp.
if ts := jsonNumber(decoded, "timestamp"); ts > 0 {
return ts
}
return 0
}
// jsonNumber extracts an int64 from a JSON-parsed map (handles float64 and json.Number).
func jsonNumber(m map[string]interface{}, key string) int64 {
v, ok := m[key]
if !ok || v == nil {
return 0
}
switch n := v.(type) {
case float64:
return int64(n)
case int64:
return n
case int:
return int64(n)
}
return 0
}
// parseISO parses an ISO 8601 timestamp string to Unix seconds.
func parseISO(s string) int64 {
if s == "" {
return 0
}
t, err := time.Parse(time.RFC3339, s)
if err != nil {
// Try with fractional seconds.
t, err = time.Parse("2006-01-02T15:04:05.999999999Z07:00", s)
if err != nil {
return 0
}
}
return t.Unix()
}
// ── Phase 2: Observer Calibration ──────────────────────────────────────────────
// calibrateObservers computes each observer's clock offset using multi-observer
// packets. Returns offset map and sample count map.
func calibrateObservers(samples []skewSample) (map[string]float64, map[string]int) {
// Group observations by packet hash.
byHash := make(map[string][]skewSample)
for _, s := range samples {
byHash[s.hash] = append(byHash[s.hash], s)
}
// For each multi-observer packet, compute per-observer deviation from median.
deviations := make(map[string][]float64) // observerID → list of deviations
for _, group := range byHash {
if len(group) < 2 {
continue // single-observer packet, can't calibrate
}
// Compute median observation timestamp for this packet.
obsTimes := make([]float64, len(group))
for i, s := range group {
obsTimes[i] = float64(s.observedTS)
}
medianObs := median(obsTimes)
for _, s := range group {
dev := float64(s.observedTS) - medianObs
deviations[s.observerID] = append(deviations[s.observerID], dev)
}
}
// Each observer's offset = median of its deviations.
offsets := make(map[string]float64, len(deviations))
counts := make(map[string]int, len(deviations))
for obsID, devs := range deviations {
offsets[obsID] = median(devs)
counts[obsID] = len(devs)
}
return offsets, counts
}
// ── Phase 3: Per-Node Skew ─────────────────────────────────────────────────────
// computeNodeSkew calculates corrected skew statistics for each node.
func computeNodeSkew(samples []skewSample, obsOffsets map[string]float64) txSkewResult {
// Compute corrected skew per sample, grouped by hash (each hash = one
// node's advert transmission). The caller maps hash → pubkey via byNode.
type correctedSample struct {
skew float64
observedTS int64
calibrated bool
}
byHash := make(map[string][]correctedSample)
hashAdvertTS := make(map[string]int64)
for _, s := range samples {
obsOffset, hasCal := obsOffsets[s.observerID]
rawSkew := float64(s.advertTS - s.observedTS)
corrected := rawSkew
if hasCal {
// Observer offset = obs_ts - median(all_obs_ts). If observer is ahead,
// its obs_ts is inflated, making raw_skew too low. Add offset to correct.
corrected = rawSkew + obsOffset
}
byHash[s.hash] = append(byHash[s.hash], correctedSample{
skew: corrected,
observedTS: s.observedTS,
calibrated: hasCal,
})
hashAdvertTS[s.hash] = s.advertTS
}
// Each hash represents one advert from one node. Compute median corrected
// skew per hash (across multiple observers).
result := make(map[string]*NodeClockSkew) // keyed by hash for now
for hash, cs := range byHash {
skews := make([]float64, len(cs))
for i, c := range cs {
skews[i] = c.skew
}
medSkew := median(skews)
meanSkew := mean(skews)
// Find latest observation.
var latestObsTS int64
var anyCal bool
for _, c := range cs {
if c.observedTS > latestObsTS {
latestObsTS = c.observedTS
}
if c.calibrated {
anyCal = true
}
}
absMedian := math.Abs(medSkew)
result[hash] = &NodeClockSkew{
MeanSkewSec: round(meanSkew, 1),
MedianSkewSec: round(medSkew, 1),
LastSkewSec: round(cs[len(cs)-1].skew, 1),
Severity: classifySkew(absMedian),
SampleCount: len(cs),
Calibrated: anyCal,
LastAdvertTS: hashAdvertTS[hash],
LastObservedTS: latestObsTS,
}
}
return result
}
// ── Integration with PacketStore ───────────────────────────────────────────────
// GetNodeClockSkew returns the clock skew data for a specific node (acquires RLock).
func (s *PacketStore) GetNodeClockSkew(pubkey string) *NodeClockSkew {
s.mu.RLock()
defer s.mu.RUnlock()
return s.getNodeClockSkewLocked(pubkey)
}
// getNodeClockSkewLocked returns clock skew for a node.
// Must be called with s.mu held (at least RLock).
func (s *PacketStore) getNodeClockSkewLocked(pubkey string) *NodeClockSkew {
s.clockSkew.Recompute(s)
txs := s.byNode[pubkey]
if len(txs) == 0 {
return nil
}
s.clockSkew.mu.RLock()
defer s.clockSkew.mu.RUnlock()
var allSkews []float64
var lastSkew float64
var lastObsTS, lastAdvTS int64
var totalSamples int
var anyCal bool
var tsSkews []tsSkewPair
for _, tx := range txs {
if tx.PayloadType == nil || *tx.PayloadType != PayloadADVERT {
continue
}
cs, ok := s.clockSkew.nodeSkew[tx.Hash]
if !ok {
continue
}
allSkews = append(allSkews, cs.MedianSkewSec)
totalSamples += cs.SampleCount
if cs.Calibrated {
anyCal = true
}
if cs.LastObservedTS > lastObsTS {
lastObsTS = cs.LastObservedTS
lastSkew = cs.LastSkewSec
lastAdvTS = cs.LastAdvertTS
}
tsSkews = append(tsSkews, tsSkewPair{ts: cs.LastObservedTS, skew: cs.MedianSkewSec})
}
if len(allSkews) == 0 {
return nil
}
medSkew := median(allSkews)
meanSkew := mean(allSkews)
absMedian := math.Abs(medSkew)
severity := classifySkew(absMedian)
// For no_clock nodes (uninitialized RTC), skip drift — data is meaningless.
var drift float64
if severity != SkewNoClock && len(tsSkews) >= minDriftSamples {
drift = computeDrift(tsSkews)
// Cap physically impossible drift rates.
if math.Abs(drift) > maxReasonableDriftPerDay {
drift = 0
}
}
// Build sparkline samples from tsSkews (sorted by time).
sort.Slice(tsSkews, func(i, j int) bool { return tsSkews[i].ts < tsSkews[j].ts })
samples := make([]SkewSample, len(tsSkews))
for i, p := range tsSkews {
samples[i] = SkewSample{Timestamp: p.ts, SkewSec: round(p.skew, 1)}
}
return &NodeClockSkew{
Pubkey: pubkey,
MeanSkewSec: round(meanSkew, 1),
MedianSkewSec: round(medSkew, 1),
LastSkewSec: round(lastSkew, 1),
DriftPerDaySec: round(drift, 2),
Severity: severity,
SampleCount: totalSamples,
Calibrated: anyCal,
LastAdvertTS: lastAdvTS,
LastObservedTS: lastObsTS,
Samples: samples,
}
}
// GetFleetClockSkew returns clock skew data for all nodes that have skew data.
// Must NOT be called with s.mu held.
func (s *PacketStore) GetFleetClockSkew() []*NodeClockSkew {
s.mu.RLock()
defer s.mu.RUnlock()
// Build name/role lookup from DB cache (requires s.mu held).
allNodes, _ := s.getCachedNodesAndPM()
nameMap := make(map[string]nodeInfo, len(allNodes))
for _, ni := range allNodes {
nameMap[ni.PublicKey] = ni
}
var results []*NodeClockSkew
for pubkey := range s.byNode {
cs := s.getNodeClockSkewLocked(pubkey)
if cs == nil {
continue
}
// Enrich with node name/role.
if ni, ok := nameMap[pubkey]; ok {
cs.NodeName = ni.Name
cs.NodeRole = ni.Role
}
// Omit samples in fleet response (too much data).
cs.Samples = nil
results = append(results, cs)
}
return results
}
// GetObserverCalibrations returns the current observer clock offsets.
func (s *PacketStore) GetObserverCalibrations() []ObserverCalibration {
s.mu.RLock()
defer s.mu.RUnlock()
s.clockSkew.Recompute(s)
s.clockSkew.mu.RLock()
defer s.clockSkew.mu.RUnlock()
result := make([]ObserverCalibration, 0, len(s.clockSkew.observerOffsets))
for obsID, offset := range s.clockSkew.observerOffsets {
result = append(result, ObserverCalibration{
ObserverID: obsID,
OffsetSec: round(offset, 1),
Samples: s.clockSkew.observerSamples[obsID],
})
}
// Sort by absolute offset descending.
sort.Slice(result, func(i, j int) bool {
return math.Abs(result[i].OffsetSec) > math.Abs(result[j].OffsetSec)
})
return result
}
// ── Math Helpers ───────────────────────────────────────────────────────────────
func median(vals []float64) float64 {
if len(vals) == 0 {
return 0
}
sorted := make([]float64, len(vals))
copy(sorted, vals)
sort.Float64s(sorted)
n := len(sorted)
if n%2 == 0 {
return (sorted[n/2-1] + sorted[n/2]) / 2
}
return sorted[n/2]
}
func mean(vals []float64) float64 {
if len(vals) == 0 {
return 0
}
sum := 0.0
for _, v := range vals {
sum += v
}
return sum / float64(len(vals))
}
// tsSkewPair is a (timestamp, skew) pair for drift estimation.
type tsSkewPair struct {
ts int64
skew float64
}
// computeDrift estimates linear drift in seconds per day from time-ordered
// (timestamp, skew) pairs using simple linear regression.
func computeDrift(pairs []tsSkewPair) float64 {
if len(pairs) < 2 {
return 0
}
// Sort by timestamp.
sort.Slice(pairs, func(i, j int) bool {
return pairs[i].ts < pairs[j].ts
})
// Time span too short? Skip.
spanSec := float64(pairs[len(pairs)-1].ts - pairs[0].ts)
if spanSec < 3600 { // need at least 1 hour of data
return 0
}
// Simple linear regression: skew = a + b*t
n := float64(len(pairs))
var sumX, sumY, sumXY, sumX2 float64
for _, p := range pairs {
x := float64(p.ts - pairs[0].ts) // normalize to avoid large numbers
y := p.skew
sumX += x
sumY += y
sumXY += x * y
sumX2 += x * x
}
denom := n*sumX2 - sumX*sumX
if denom == 0 {
return 0
}
slope := (n*sumXY - sumX*sumY) / denom // seconds of drift per second
return slope * 86400 // convert to seconds per day
}
+546
View File
@@ -0,0 +1,546 @@
package main
import (
"fmt"
"math"
"testing"
"time"
)
// ── classifySkew ───────────────────────────────────────────────────────────────
func TestClassifySkew(t *testing.T) {
tests := []struct {
absSkew float64
expected SkewSeverity
}{
{0, SkewOK},
{60, SkewOK}, // 1 min
{299, SkewOK}, // just under 5 min
{300, SkewWarning}, // exactly 5 min
{1800, SkewWarning}, // 30 min
{3599, SkewWarning}, // just under 1 hour
{3600, SkewCritical}, // exactly 1 hour
{86400, SkewCritical}, // 1 day
{2592000 - 1, SkewCritical}, // just under 30 days
{2592000, SkewAbsurd}, // exactly 30 days
{86400 * 365 - 1, SkewAbsurd}, // just under 365 days
{86400 * 365, SkewNoClock}, // exactly 365 days
{86400 * 365 * 10, SkewNoClock}, // 10 years (epoch-0 style)
}
for _, tc := range tests {
got := classifySkew(tc.absSkew)
if got != tc.expected {
t.Errorf("classifySkew(%v) = %v, want %v", tc.absSkew, got, tc.expected)
}
}
}
// ── median ─────────────────────────────────────────────────────────────────────
func TestMedian(t *testing.T) {
tests := []struct {
vals []float64
expected float64
}{
{nil, 0},
{[]float64{}, 0},
{[]float64{5}, 5},
{[]float64{1, 3}, 2},
{[]float64{3, 1, 2}, 2},
{[]float64{4, 1, 3, 2}, 2.5},
{[]float64{-10, 0, 10}, 0},
}
for _, tc := range tests {
got := median(tc.vals)
if got != tc.expected {
t.Errorf("median(%v) = %v, want %v", tc.vals, got, tc.expected)
}
}
}
func TestMean(t *testing.T) {
tests := []struct {
vals []float64
expected float64
}{
{nil, 0},
{[]float64{10}, 10},
{[]float64{2, 4, 6}, 4},
}
for _, tc := range tests {
got := mean(tc.vals)
if got != tc.expected {
t.Errorf("mean(%v) = %v, want %v", tc.vals, got, tc.expected)
}
}
}
// ── parseISO ───────────────────────────────────────────────────────────────────
func TestParseISO(t *testing.T) {
tests := []struct {
input string
expected int64
}{
{"", 0},
{"garbage", 0},
{"2026-04-15T12:00:00Z", 1776254400},
{"2026-04-15T12:00:00+00:00", 1776254400},
}
for _, tc := range tests {
got := parseISO(tc.input)
if got != tc.expected {
t.Errorf("parseISO(%q) = %v, want %v", tc.input, got, tc.expected)
}
}
}
// ── extractTimestamp ────────────────────────────────────────────────────────────
func TestExtractTimestamp(t *testing.T) {
// Nested payload.timestamp
decoded := map[string]interface{}{
"payload": map[string]interface{}{
"timestamp": float64(1776340800),
},
}
got := extractTimestamp(decoded)
if got != 1776340800 {
t.Errorf("extractTimestamp (nested) = %v, want 1776340800", got)
}
// Top-level timestamp
decoded2 := map[string]interface{}{
"timestamp": float64(1776340900),
}
got2 := extractTimestamp(decoded2)
if got2 != 1776340900 {
t.Errorf("extractTimestamp (top-level) = %v, want 1776340900", got2)
}
// No timestamp
decoded3 := map[string]interface{}{"foo": "bar"}
got3 := extractTimestamp(decoded3)
if got3 != 0 {
t.Errorf("extractTimestamp (missing) = %v, want 0", got3)
}
}
// ── calibrateObservers ─────────────────────────────────────────────────────────
func TestCalibrateObservers_SingleObserver(t *testing.T) {
// Single-observer packets can't calibrate — should return empty.
samples := []skewSample{
{advertTS: 1000, observedTS: 1000, observerID: "obs1", hash: "h1"},
{advertTS: 2000, observedTS: 2000, observerID: "obs1", hash: "h2"},
}
offsets, _ := calibrateObservers(samples)
if len(offsets) != 0 {
t.Errorf("expected no offsets for single-observer, got %v", offsets)
}
}
func TestCalibrateObservers_MultiObserver(t *testing.T) {
// Packet h1 seen by 3 observers: obs1 at t=100, obs2 at t=110, obs3 at t=100.
// Median observation = 100. obs1=0, obs2=+10, obs3=0
// Packet h2 seen by 3 observers: obs1 at t=200, obs2 at t=210, obs3 at t=200.
// Median observation = 200. obs1=0, obs2=+10, obs3=0
samples := []skewSample{
{advertTS: 100, observedTS: 100, observerID: "obs1", hash: "h1"},
{advertTS: 100, observedTS: 110, observerID: "obs2", hash: "h1"},
{advertTS: 100, observedTS: 100, observerID: "obs3", hash: "h1"},
{advertTS: 200, observedTS: 200, observerID: "obs1", hash: "h2"},
{advertTS: 200, observedTS: 210, observerID: "obs2", hash: "h2"},
{advertTS: 200, observedTS: 200, observerID: "obs3", hash: "h2"},
}
offsets, _ := calibrateObservers(samples)
if offsets["obs1"] != 0 {
t.Errorf("obs1 offset = %v, want 0", offsets["obs1"])
}
if offsets["obs2"] != 10 {
t.Errorf("obs2 offset = %v, want 10", offsets["obs2"])
}
if offsets["obs3"] != 0 {
t.Errorf("obs3 offset = %v, want 0", offsets["obs3"])
}
}
// ── computeNodeSkew ────────────────────────────────────────────────────────────
func TestComputeNodeSkew_BasicCorrection(t *testing.T) {
// Validates observer offset correction direction.
//
// Setup: node is 60s ahead, obs1 accurate, obs2 is 10s ahead.
// With 2 observers, median obs_ts = 1005.
// obs1 offset = 1000 - 1005 = -5
// obs2 offset = 1010 - 1005 = +5
// Correction: corrected = raw_skew + obsOffset
// obs1: raw=60, corrected = 60 + (-5) = 55
// obs2: raw=50, corrected = 50 + 5 = 55
// Both converge to 55 (not exact 60 because with only 2 observers,
// the median can't fully distinguish which observer is drifted).
samples := []skewSample{
// Same packet seen by accurate obs1 and obs2 (+10s ahead)
{advertTS: 1060, observedTS: 1000, observerID: "obs1", hash: "h1"},
{advertTS: 1060, observedTS: 1010, observerID: "obs2", hash: "h1"},
}
offsets, _ := calibrateObservers(samples)
// median obs = 1005, obs1 offset = -5, obs2 offset = +5
// So the median approach finds obs2 is +5 ahead (relative to median)
// Now compute node skew with those offsets:
nodeSkew := computeNodeSkew(samples, offsets)
cs, ok := nodeSkew["h1"]
if !ok {
t.Fatal("expected skew data for hash h1")
}
// With only 2 observers, median obs_ts = 1005.
// obs1 offset = 1000-1005 = -5, obs2 offset = 1010-1005 = +5
// raw from obs1 = 60, corrected = 60 + (-5) = 55
// raw from obs2 = 50, corrected = 50 + 5 = 55
// median = 55
if cs.MedianSkewSec != 55 {
t.Errorf("median skew = %v, want 55", cs.MedianSkewSec)
}
}
func TestComputeNodeSkew_ThreeObservers(t *testing.T) {
// Node is exactly 60s ahead. obs1 accurate, obs2 accurate, obs3 +30s ahead.
// advertTS = 1060, real time = 1000
samples := []skewSample{
{advertTS: 1060, observedTS: 1000, observerID: "obs1", hash: "h1"},
{advertTS: 1060, observedTS: 1000, observerID: "obs2", hash: "h1"},
{advertTS: 1060, observedTS: 1030, observerID: "obs3", hash: "h1"},
}
offsets, _ := calibrateObservers(samples)
// median obs_ts = 1000. obs1=0, obs2=0, obs3=+30
if offsets["obs3"] != 30 {
t.Errorf("obs3 offset = %v, want 30", offsets["obs3"])
}
nodeSkew := computeNodeSkew(samples, offsets)
cs := nodeSkew["h1"]
if cs == nil {
t.Fatal("expected skew data for h1")
}
// raw from obs1 = 60, corrected = 60 + 0 = 60
// raw from obs2 = 60, corrected = 60 + 0 = 60
// raw from obs3 = 30, corrected = 30 + 30 = 60
// All three converge to 60.
if cs.MedianSkewSec != 60 {
t.Errorf("median skew = %v, want 60 (node is 60s ahead)", cs.MedianSkewSec)
}
}
// ── computeDrift ───────────────────────────────────────────────────────────────
func TestComputeDrift_Stable(t *testing.T) {
// Constant skew = no drift.
pairs := []tsSkewPair{
{ts: 0, skew: 60},
{ts: 7200, skew: 60},
{ts: 14400, skew: 60},
}
drift := computeDrift(pairs)
if drift != 0 {
t.Errorf("drift = %v, want 0 for stable skew", drift)
}
}
func TestComputeDrift_LinearDrift(t *testing.T) {
// 1 second drift per hour = 24 sec/day.
pairs := []tsSkewPair{
{ts: 0, skew: 0},
{ts: 3600, skew: 1},
{ts: 7200, skew: 2},
}
drift := computeDrift(pairs)
expected := 24.0
if math.Abs(drift-expected) > 0.1 {
t.Errorf("drift = %v, want ~%v", drift, expected)
}
}
func TestComputeDrift_TooFewSamples(t *testing.T) {
pairs := []tsSkewPair{{ts: 0, skew: 10}}
if computeDrift(pairs) != 0 {
t.Error("expected 0 drift for single sample")
}
}
func TestComputeDrift_TooShortSpan(t *testing.T) {
// Less than 1 hour apart.
pairs := []tsSkewPair{
{ts: 0, skew: 0},
{ts: 1800, skew: 10},
}
if computeDrift(pairs) != 0 {
t.Error("expected 0 drift for short time span")
}
}
// ── jsonNumber ─────────────────────────────────────────────────────────────────
func TestJsonNumber(t *testing.T) {
m := map[string]interface{}{
"a": float64(42),
"b": int64(99),
"c": "not a number",
"d": nil,
}
if jsonNumber(m, "a") != 42 {
t.Error("float64 case failed")
}
if jsonNumber(m, "b") != 99 {
t.Error("int64 case failed")
}
if jsonNumber(m, "c") != 0 {
t.Error("string case should return 0")
}
if jsonNumber(m, "d") != 0 {
t.Error("nil case should return 0")
}
if jsonNumber(m, "missing") != 0 {
t.Error("missing key should return 0")
}
}
// ── Integration: GetNodeClockSkew via PacketStore ──────────────────────────────
func TestGetNodeClockSkew_Integration(t *testing.T) {
ps := NewPacketStore(nil, nil)
// Simulate two ADVERT transmissions for the same node, seen by 2 observers each.
// Node "AABB" has clock 120s ahead.
pt := 4 // ADVERT
tx1 := &StoreTx{
Hash: "hash1",
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":1700002320}}`, // obs=1700002200, node ahead by 120s
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: "2023-11-14T22:50:00Z"}, // 1700002200
{ObserverID: "obs2", Timestamp: "2023-11-14T22:50:00Z"}, // 1700002200
},
}
tx2 := &StoreTx{
Hash: "hash2",
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":1700005920}}`, // obs=1700005800, node ahead by 120s
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: "2023-11-14T23:50:00Z"}, // 1700005800
{ObserverID: "obs2", Timestamp: "2023-11-14T23:50:00Z"}, // 1700005800
},
}
ps.mu.Lock()
ps.byNode["AABB"] = []*StoreTx{tx1, tx2}
ps.byPayloadType[4] = []*StoreTx{tx1, tx2}
// Force recompute by setting interval to 0.
ps.clockSkew.computeInterval = 0
ps.mu.Unlock()
result := ps.GetNodeClockSkew("AABB")
if result == nil {
t.Fatal("expected clock skew result for node AABB")
}
if result.Pubkey != "AABB" {
t.Errorf("pubkey = %q, want AABB", result.Pubkey)
}
// Both transmissions show 120s skew, so median should be 120.
if result.MedianSkewSec != 120 {
t.Errorf("median skew = %v, want 120", result.MedianSkewSec)
}
if result.SampleCount < 2 {
t.Errorf("sample count = %v, want >= 2", result.SampleCount)
}
if result.Severity != SkewOK {
t.Errorf("severity = %v, want ok (120s < 5min)", result.Severity)
}
// Drift should be ~0 since skew is constant.
if math.Abs(result.DriftPerDaySec) > 1 {
t.Errorf("drift = %v, want ~0 for constant skew", result.DriftPerDaySec)
}
}
func TestGetNodeClockSkew_NoData(t *testing.T) {
ps := NewPacketStore(nil, nil)
result := ps.GetNodeClockSkew("nonexistent")
if result != nil {
t.Error("expected nil for nonexistent node")
}
}
// ── Sanity check tests (#XXX — clock skew crazy stats) ────────────────────────
func TestGetNodeClockSkew_NoClock_EpochZero(t *testing.T) {
// Node with epoch-0 timestamp produces huge skew → no_clock severity, drift=0.
ps := NewPacketStore(nil, nil)
pt := 4 // ADVERT
// Epoch-ish advert: advertTS near start of 2020, observed in 2023 → |skew| > 365 days
var txs []*StoreTx
baseObs := int64(1700000000) // ~Nov 2023
for i := 0; i < 6; i++ {
obsTS := baseObs + int64(i)*7200
tx := &StoreTx{
Hash: "epoch-h" + string(rune('0'+i)),
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":1577836800}}`, // Jan 1 2020 — valid but way off
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: time.Unix(obsTS, 0).UTC().Format(time.RFC3339)},
},
}
txs = append(txs, tx)
}
ps.mu.Lock()
ps.byNode["EPOCH"] = txs
for _, tx := range txs {
ps.byPayloadType[4] = append(ps.byPayloadType[4], tx)
}
ps.clockSkew.computeInterval = 0
ps.mu.Unlock()
result := ps.GetNodeClockSkew("EPOCH")
if result == nil {
t.Fatal("expected clock skew result for epoch-0 node")
}
if result.Severity != SkewNoClock {
t.Errorf("severity = %v, want no_clock", result.Severity)
}
if result.DriftPerDaySec != 0 {
t.Errorf("drift = %v, want 0 for no_clock node", result.DriftPerDaySec)
}
}
func TestGetNodeClockSkew_TooFewSamplesForDrift(t *testing.T) {
// Node with only 2 advert samples → drift should not be computed.
ps := NewPacketStore(nil, nil)
pt := 4
baseObs := int64(1700000000)
var txs []*StoreTx
for i := 0; i < 2; i++ {
obsTS := baseObs + int64(i)*7200
advTS := obsTS + 120 // 120s ahead
tx := &StoreTx{
Hash: "few-h" + string(rune('0'+i)),
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":` + formatInt64(advTS) + `}}`,
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: time.Unix(obsTS, 0).UTC().Format(time.RFC3339)},
},
}
txs = append(txs, tx)
}
ps.mu.Lock()
ps.byNode["FEWSAMP"] = txs
for _, tx := range txs {
ps.byPayloadType[4] = append(ps.byPayloadType[4], tx)
}
ps.clockSkew.computeInterval = 0
ps.mu.Unlock()
result := ps.GetNodeClockSkew("FEWSAMP")
if result == nil {
t.Fatal("expected clock skew result")
}
if result.DriftPerDaySec != 0 {
t.Errorf("drift = %v, want 0 for 2-sample node (minimum is %d)", result.DriftPerDaySec, minDriftSamples)
}
}
func TestGetNodeClockSkew_AbsurdDriftCapped(t *testing.T) {
// Node with wildly varying skew producing |drift| > 86400 s/day → drift capped to 0.
ps := NewPacketStore(nil, nil)
pt := 4
// Create 6 samples with extreme skew variation to produce absurd drift.
baseObs := int64(1700000000)
var txs []*StoreTx
for i := 0; i < 6; i++ {
obsTS := baseObs + int64(i)*3600
// Alternate between huge positive and negative skew offsets
skewOffset := int64(50000 * (1 - 2*(i%2))) // +50000 or -50000
advTS := obsTS + skewOffset
tx := &StoreTx{
Hash: "wild-h" + string(rune('0'+i)),
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":` + formatInt64(advTS) + `}}`,
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: time.Unix(obsTS, 0).UTC().Format(time.RFC3339)},
},
}
txs = append(txs, tx)
}
ps.mu.Lock()
ps.byNode["WILD"] = txs
for _, tx := range txs {
ps.byPayloadType[4] = append(ps.byPayloadType[4], tx)
}
ps.clockSkew.computeInterval = 0
ps.mu.Unlock()
result := ps.GetNodeClockSkew("WILD")
if result == nil {
t.Fatal("expected clock skew result")
}
if math.Abs(result.DriftPerDaySec) > maxReasonableDriftPerDay {
t.Errorf("drift = %v, should be capped (|drift| > %v)", result.DriftPerDaySec, maxReasonableDriftPerDay)
}
}
func TestGetNodeClockSkew_NormalNodeWithDrift(t *testing.T) {
// Normal node with 6 samples and consistent linear drift → drift computed correctly.
ps := NewPacketStore(nil, nil)
pt := 4
baseObs := int64(1700000000)
var txs []*StoreTx
for i := 0; i < 6; i++ {
obsTS := baseObs + int64(i)*7200 // every 2 hours
// Drift: 1 sec/hour = 24 sec/day
advTS := obsTS + 120 + int64(i) // skew grows by 1s per sample (2h apart)
tx := &StoreTx{
Hash: "norm-h" + string(rune('0'+i)),
PayloadType: &pt,
DecodedJSON: `{"payload":{"timestamp":` + formatInt64(advTS) + `}}`,
Observations: []*StoreObs{
{ObserverID: "obs1", Timestamp: time.Unix(obsTS, 0).UTC().Format(time.RFC3339)},
},
}
txs = append(txs, tx)
}
ps.mu.Lock()
ps.byNode["NORMAL"] = txs
for _, tx := range txs {
ps.byPayloadType[4] = append(ps.byPayloadType[4], tx)
}
ps.clockSkew.computeInterval = 0
ps.mu.Unlock()
result := ps.GetNodeClockSkew("NORMAL")
if result == nil {
t.Fatal("expected clock skew result")
}
if result.Severity != SkewOK {
t.Errorf("severity = %v, want ok", result.Severity)
}
// 1s per 7200s = 12 s/day
if result.DriftPerDaySec == 0 {
t.Error("expected non-zero drift for linearly drifting node")
}
if math.Abs(result.DriftPerDaySec) > maxReasonableDriftPerDay {
t.Errorf("drift = %v, should be reasonable", result.DriftPerDaySec)
}
}
// formatInt64 is a test helper to format int64 as string for JSON embedding.
func formatInt64(n int64) string {
return fmt.Sprintf("%d", n)
}
+131
View File
@@ -0,0 +1,131 @@
package main
import (
"testing"
"time"
)
// TestCollisionDetailsIncludeNodePairs verifies that collision details contain
// the correct prefix and matching node pairs (#757).
func TestCollisionDetailsIncludeNodePairs(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
// Insert two repeater nodes with the same 3-byte prefix "AABB11"
db.conn.Exec(`INSERT INTO nodes (public_key, name, role) VALUES ('aabb11ccdd001122', 'Node Alpha', 'repeater')`)
db.conn.Exec(`INSERT INTO nodes (public_key, name, role) VALUES ('aabb11eeff334455', 'Node Beta', 'repeater')`)
// Add advert transmissions with hash_size=3 path bytes (0x80 = bits 10 → size 3)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('0180aabb11ccdd', 'col_hash_01', ?, 1, 4, '{"pubKey":"aabb11ccdd001122","name":"Node Alpha","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -91, '["aabb11"]', ?)`, recentEpoch)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('0180aabb11eeff', 'col_hash_02', ?, 1, 4, '{"pubKey":"aabb11eeff334455","name":"Node Beta","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (2, 1, 9.0, -93, '["aabb11"]', ?)`, recentEpoch)
store := NewPacketStore(db, nil)
store.Load()
result := store.GetAnalyticsHashCollisions("")
bySize, ok := result["by_size"].(map[string]interface{})
if !ok {
t.Fatal("expected by_size map")
}
size3, ok := bySize["3"].(map[string]interface{})
if !ok {
t.Fatal("expected by_size[3] map")
}
collisions, ok := size3["collisions"].([]collisionEntry)
if !ok {
t.Fatalf("expected collisions as []collisionEntry, got %T", size3["collisions"])
}
// Find our collision
var found *collisionEntry
for i := range collisions {
if collisions[i].Prefix == "AABB11" {
found = &collisions[i]
break
}
}
if found == nil {
t.Fatal("expected collision with prefix AABB11")
}
if found.Appearances != 2 {
t.Errorf("expected 2 appearances, got %d", found.Appearances)
}
if len(found.Nodes) != 2 {
t.Fatalf("expected 2 nodes in collision, got %d", len(found.Nodes))
}
// Verify node pairs
pubkeys := map[string]bool{}
names := map[string]bool{}
for _, n := range found.Nodes {
pubkeys[n.PublicKey] = true
names[n.Name] = true
}
if !pubkeys["aabb11ccdd001122"] {
t.Error("expected node aabb11ccdd001122 in collision")
}
if !pubkeys["aabb11eeff334455"] {
t.Error("expected node aabb11eeff334455 in collision")
}
if !names["Node Alpha"] {
t.Error("expected Node Alpha in collision")
}
if !names["Node Beta"] {
t.Error("expected Node Beta in collision")
}
}
// TestCollisionDetailsEmptyWhenNoCollisions verifies that collision details are
// empty when there are no collisions (#757).
func TestCollisionDetailsEmptyWhenNoCollisions(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
// Insert one repeater node with 3-byte hash
db.conn.Exec(`INSERT INTO nodes (public_key, name, role) VALUES ('aabb11ccdd001122', 'Solo Node', 'repeater')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('0180aabb11ccdd', 'solo_hash_01', ?, 1, 4, '{"pubKey":"aabb11ccdd001122","name":"Solo Node","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -91, '["aabb11"]', ?)`, recentEpoch)
store := NewPacketStore(db, nil)
store.Load()
result := store.GetAnalyticsHashCollisions("")
bySize, ok := result["by_size"].(map[string]interface{})
if !ok {
t.Fatal("expected by_size map")
}
size3, ok := bySize["3"].(map[string]interface{})
if !ok {
t.Fatal("expected by_size[3] map")
}
collisions, ok := size3["collisions"].([]collisionEntry)
if !ok {
t.Fatalf("expected collisions as []collisionEntry, got %T", size3["collisions"])
}
if len(collisions) != 0 {
t.Errorf("expected 0 collisions, got %d", len(collisions))
}
}
+318 -1
View File
@@ -41,7 +41,7 @@ func setupTestDBv2(t *testing.T) *DB {
id INTEGER PRIMARY KEY AUTOINCREMENT, raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE, first_seen TEXT NOT NULL,
route_type INTEGER, payload_type INTEGER, payload_version INTEGER,
decoded_json TEXT, created_at TEXT DEFAULT (datetime('now'))
decoded_json TEXT, channel_hash TEXT DEFAULT NULL, created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -3217,6 +3217,189 @@ func TestGetNodeHashSizeInfoEdgeCases(t *testing.T) {
}
}
// TestHashSizeTransportRoutePathByteOffset verifies that transport routes (0, 3)
// read the path byte from offset 5 (after 4 transport code bytes), not offset 1.
// Regression test for #744 / #722.
func TestHashSizeTransportRoutePathByteOffset(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES ('obs1', 'Obs', 'SJC', ?, '2026-01-01T00:00:00Z', 10)`, recent)
// Route type 0 (TRANSPORT_FLOOD): header=0x04 (payload_type=1, route_type=0)
// 4 transport bytes + path byte at offset 5.
// Path byte 0x80 → hash_size bits = 10 → size 3
// If bug is present, code reads byte 1 (0xAA) → hash_size bits = 10 → size 3 (coincidence)
// Use path byte 0x40 (hash_size=2) and transport byte 0x01 at offset 1 (hash_size=1 if misread)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('100102030440aabb', 'tf_offset', ?, 0, 4, '{"pubKey":"aaaa000000000001","name":"TF-Node","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -90, '[]', ?)`, recentEpoch)
// Route type 3 (TRANSPORT_DIRECT): header=0x13 (payload_type=4, route_type=3)
// 4 transport bytes + path byte at offset 5.
// Path byte 0xC1 → hash_size bits = 11 → size 4, hop_count = 1 (not zero-hop)
// Byte 1 = 0x05 → hash_size bits = 00 → size 1 if misread
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1305060708C1bbcc', 'td_offset', ?, 3, 4, '{"pubKey":"aaaa000000000002","name":"TD-Node","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (2, 1, 10.0, -90, '[]', ?)`, recentEpoch)
// Route type 1 (FLOOD): header=0x11 (payload_type=4, route_type=1)
// Path byte at offset 1. Path byte 0x80 → hash_size bits = 10 → size 3
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1180aabbccdd', 'flood_offset', ?, 1, 4, '{"pubKey":"aaaa000000000003","name":"Flood-Node","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (3, 1, 10.0, -90, '[]', ?)`, recentEpoch)
store := NewPacketStore(db, nil)
store.Load()
info := store.GetNodeHashSizeInfo()
// Transport flood node: path byte 0x40 → hash_size = 2
if ni, ok := info["aaaa000000000001"]; !ok {
t.Error("transport flood node missing from hash size info")
} else if ni.HashSize != 2 {
t.Errorf("transport flood node: want HashSize=2 (from path byte at offset 5), got %d", ni.HashSize)
}
// Transport direct node: path byte 0xC1 → hash_size = 4
if ni, ok := info["aaaa000000000002"]; !ok {
t.Error("transport direct node missing from hash size info")
} else if ni.HashSize != 4 {
t.Errorf("transport direct node: want HashSize=4 (from path byte at offset 5), got %d", ni.HashSize)
}
// Regular flood node: path byte 0x80 → hash_size = 3
if ni, ok := info["aaaa000000000003"]; !ok {
t.Error("regular flood node missing from hash size info")
} else if ni.HashSize != 3 {
t.Errorf("regular flood node: want HashSize=3 (from path byte at offset 1), got %d", ni.HashSize)
}
}
// TestHashSizeTransportDirectZeroHopSkipped verifies that RouteTransportDirect
// zero-hop adverts are skipped (same as RouteDirect). Regression test for #744.
func TestHashSizeTransportDirectZeroHopSkipped(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES ('obs1', 'Obs', 'SJC', ?, '2026-01-01T00:00:00Z', 10)`, recent)
// RouteDirect (2) zero-hop: path byte 0x40 → hop_count=0, hash_size bits=01
// Should be skipped (existing behavior)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1240aabbccdd', 'direct_zh', ?, 2, 4, '{"pubKey":"bbbb000000000001","name":"Direct-ZH","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -90, '[]', ?)`, recentEpoch)
// RouteTransportDirect (3) zero-hop: 4 transport bytes + path byte 0x40 → hop_count=0
// Should ALSO be skipped (this was the missing case)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('130102030440aabb', 'tdirect_zh', ?, 3, 4, '{"pubKey":"bbbb000000000002","name":"TDirect-ZH","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (2, 1, 10.0, -90, '[]', ?)`, recentEpoch)
// RouteDirect (2) non-zero-hop: path byte 0x41 → hop_count=1
// Should NOT be skipped
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1241aabbccdd', 'direct_1h', ?, 2, 4, '{"pubKey":"bbbb000000000003","name":"Direct-1H","type":"ADVERT"}')`, recent)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (3, 1, 10.0, -90, '[]', ?)`, recentEpoch)
store := NewPacketStore(db, nil)
store.Load()
info := store.GetNodeHashSizeInfo()
// RouteDirect zero-hop should be absent
if _, ok := info["bbbb000000000001"]; ok {
t.Error("RouteDirect zero-hop advert should be skipped")
}
// RouteTransportDirect zero-hop should also be absent
if _, ok := info["bbbb000000000002"]; ok {
t.Error("RouteTransportDirect zero-hop advert should be skipped")
}
// RouteDirect non-zero-hop should be present with hash_size=2
if ni, ok := info["bbbb000000000003"]; !ok {
t.Error("RouteDirect non-zero-hop should be in hash size info")
} else if ni.HashSize != 2 {
t.Errorf("RouteDirect non-zero-hop: want HashSize=2, got %d", ni.HashSize)
}
}
// TestAnalyticsHashSizesZeroHopSkip verifies that computeAnalyticsHashSizes
// does not overwrite a node's hash_size with a zero-hop advert's unreliable value.
// Regression test for #744.
func TestAnalyticsHashSizesZeroHopSkip(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES ('obs1', 'Obs', 'SJC', ?, '2026-01-01T00:00:00Z', 10)`, recent)
pk := "cccc000000000001"
db.conn.Exec(`INSERT INTO nodes (public_key, name, role) VALUES (?, 'ZH-Analytics', 'repeater')`, pk)
decoded := `{"pubKey":"` + pk + `","name":"ZH-Analytics","type":"ADVERT"}`
// First: a flood advert with hashSize=2 (reliable, multi-hop)
// header 0x11 = route_type 1 (flood), payload_type 4
// pathByte 0x41 = hashSize bits 01 → size 2, hop_count 1
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1141aabbccdd', 'az_flood', ?, 1, 4, ?)`, recent, decoded)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -90, '["aabb"]', ?)`, recentEpoch)
// Second: a direct zero-hop advert with pathByte=0x00 → would give hashSize=1
// header 0x12 = route_type 2 (direct), payload_type 4
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('1200aabbccdd', 'az_direct', ?, 2, 4, ?)`, recent, decoded)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (2, 1, 10.0, -90, '[]', ?)`, recentEpoch)
store := NewPacketStore(db, nil)
store.Load()
result := store.GetAnalyticsHashSizes("")
// The node should appear in multiByteNodes (hashSize=2 from the flood advert)
// If the zero-hop bug is present, hashSize would be 1 and the node would NOT
// appear in multiByteNodes.
multiByteNodes, ok := result["multiByteNodes"].([]map[string]interface{})
if !ok {
t.Fatal("expected multiByteNodes slice in analytics hash sizes")
}
found := false
for _, n := range multiByteNodes {
if n["pubkey"] == pk {
found = true
if hs, ok := n["hashSize"].(int); ok && hs != 2 {
t.Errorf("expected hashSize=2 from flood advert, got %d", hs)
}
}
}
if !found {
t.Error("node should appear in multiByteNodes with hashSize=2; zero-hop advert should not overwrite to 1")
}
}
func TestHandleResolveHopsEdgeCases(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -4133,6 +4316,90 @@ func TestIndexByNodePreCheck(t *testing.T) {
})
}
// TestIndexByNodeResolvedPath tests that resolved_path entries are indexed in byNode.
func TestIndexByNodeResolvedPath(t *testing.T) {
store := &PacketStore{
byNode: make(map[string][]*StoreTx),
nodeHashes: make(map[string]map[string]bool),
}
t.Run("indexes resolved path pubkeys from observations", func(t *testing.T) {
relayPK := "aabb1122334455ff"
tx := &StoreTx{
Hash: "rp1",
DecodedJSON: `{"type":"CHAN","text":"hello"}`, // no pubKey fields
Observations: []*StoreObs{
{ResolvedPath: []*string{&relayPK}},
},
}
store.indexByNode(tx)
if len(store.byNode[relayPK]) != 1 {
t.Errorf("expected relay pubkey indexed, got %d", len(store.byNode[relayPK]))
}
})
t.Run("skips null entries in resolved path", func(t *testing.T) {
pk := "cc11dd22ee33ff44"
tx := &StoreTx{
Hash: "rp2",
Observations: []*StoreObs{
{ResolvedPath: []*string{nil, &pk, nil}},
},
}
store.indexByNode(tx)
if len(store.byNode[pk]) != 1 {
t.Errorf("expected resolved pubkey indexed, got %d", len(store.byNode[pk]))
}
// Verify nil entries didn't create empty-string keys
if _, exists := store.byNode[""]; exists {
t.Error("nil/empty resolved path entries should not create byNode entries")
}
})
t.Run("relay-only node appears in byNode", func(t *testing.T) {
// A packet with no decoded pubkey fields, only a relay in resolved path
relayOnly := "relay0only0pubkey"
tx := &StoreTx{
Hash: "rp3",
// No DecodedJSON at all — pure relay
Observations: []*StoreObs{
{ResolvedPath: []*string{&relayOnly}},
},
}
store.indexByNode(tx)
if len(store.byNode[relayOnly]) != 1 {
t.Errorf("expected relay-only node indexed, got %d", len(store.byNode[relayOnly]))
}
})
t.Run("dedup between decoded JSON and resolved path", func(t *testing.T) {
pk := "dedup0test0pk1234"
tx := &StoreTx{
Hash: "rp4",
DecodedJSON: `{"pubKey":"` + pk + `"}`,
Observations: []*StoreObs{
{ResolvedPath: []*string{&pk}},
},
}
store.indexByNode(tx)
if len(store.byNode[pk]) != 1 {
t.Errorf("expected dedup to keep 1 entry, got %d", len(store.byNode[pk]))
}
})
t.Run("indexes tx.ResolvedPath when observations empty", func(t *testing.T) {
rpPK := "txlevel0resolved1"
tx := &StoreTx{
Hash: "rp5",
ResolvedPath: []*string{&rpPK},
}
store.indexByNode(tx)
if len(store.byNode[rpPK]) != 1 {
t.Errorf("expected tx-level resolved path indexed, got %d", len(store.byNode[rpPK]))
}
})
}
// BenchmarkIndexByNode measures indexByNode performance with and without pubkey
// fields to demonstrate the strings.Contains pre-check optimization.
func BenchmarkIndexByNode(b *testing.B) {
@@ -4386,3 +4653,53 @@ func TestHandleBatchObservations(t *testing.T) {
}
})
}
// TestIngestTraceBroadcastIncludesPath verifies that TRACE packet broadcasts
// include decoded.path with hopsCompleted (#683).
func TestIngestTraceBroadcastIncludesPath(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
store := NewPacketStore(db, nil)
store.Load()
initialMax := store.MaxTransmissionID()
// TRACE packet: header=0x25, path_byte=0x02 (2 SNR bytes), 2 SNR bytes,
// then payload: tag(4) + authCode(4) + flags(1) + 4 hop hashes (1-byte each)
traceHex := "2502AABB010000000200000000DEADBEEF"
now := time.Now().UTC().Format(time.RFC3339)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES (?, 'tracehash683test', ?, 1, 9, '')`, traceHex, now)
newTxID := 0
db.conn.QueryRow("SELECT MAX(id) FROM transmissions").Scan(&newTxID)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (?, 1, 5.0, -100, '["aa"]', ?)`, newTxID, time.Now().Unix())
broadcastMaps, _ := store.IngestNewFromDB(initialMax, 100)
if len(broadcastMaps) < 1 {
t.Fatal("expected >=1 broadcast maps")
}
bm := broadcastMaps[0]
decoded, ok := bm["decoded"].(map[string]interface{})
if !ok {
t.Fatal("broadcast map missing 'decoded'")
}
pathObj, ok := decoded["path"]
if !ok {
t.Fatal("decoded missing 'path' for TRACE packet — hopsCompleted not delivered to frontend (#683)")
}
// The path should be a Path struct with HopsCompleted = 2
pathStruct, ok := pathObj.(Path)
if !ok {
t.Fatalf("expected Path struct, got %T", pathObj)
}
if pathStruct.HopsCompleted == nil {
t.Fatal("path.HopsCompleted is nil for TRACE packet")
}
if *pathStruct.HopsCompleted != 2 {
t.Errorf("expected hopsCompleted=2, got %d", *pathStruct.HopsCompleted)
}
}
+231 -80
View File
@@ -8,6 +8,7 @@ import (
"math"
"os"
"strings"
"sync"
"time"
_ "modernc.org/sqlite"
@@ -19,6 +20,12 @@ type DB struct {
path string // filesystem path to the database file
isV3 bool // v3 schema: observer_idx in observations (vs observer_id in v2)
hasResolvedPath bool // observations table has resolved_path column
// Channel list cache (60s TTL) — avoids repeated GROUP BY scans (#762)
channelsCacheMu sync.Mutex
channelsCacheKey string
channelsCacheRes []map[string]interface{}
channelsCacheExp time.Time
}
// OpenDB opens a read-only SQLite connection with WAL mode.
@@ -1153,69 +1160,219 @@ func (db *DB) GetTraces(hash string) ([]map[string]interface{}, error) {
// Queries transmissions directly (not a VIEW) to avoid observation-level
// duplicates that could cause stale lastMessage when an older message has
// a later re-observation timestamp.
func (db *DB) GetChannels() ([]map[string]interface{}, error) {
rows, err := db.conn.Query(`SELECT decoded_json, first_seen FROM transmissions WHERE payload_type = 5 ORDER BY first_seen ASC`)
func (db *DB) GetChannels(region ...string) ([]map[string]interface{}, error) {
regionParam := ""
if len(region) > 0 {
regionParam = region[0]
}
// Check cache (60s TTL)
db.channelsCacheMu.Lock()
if db.channelsCacheRes != nil && db.channelsCacheKey == regionParam && time.Now().Before(db.channelsCacheExp) {
res := db.channelsCacheRes
db.channelsCacheMu.Unlock()
return res, nil
}
db.channelsCacheMu.Unlock()
regionCodes := normalizeRegionCodes(regionParam)
var querySQL string
args := make([]interface{}, 0, len(regionCodes))
if len(regionCodes) > 0 {
placeholders := make([]string, len(regionCodes))
for i, code := range regionCodes {
placeholders[i] = "?"
args = append(args, code)
}
regionPlaceholder := strings.Join(placeholders, ",")
if db.isV3 {
querySQL = fmt.Sprintf(`SELECT t.channel_hash,
COUNT(*) AS msg_count,
MAX(t.first_seen) AS last_activity,
(SELECT t2.decoded_json FROM transmissions t2
WHERE t2.channel_hash = t.channel_hash AND t2.payload_type = 5
ORDER BY t2.first_seen DESC LIMIT 1) AS sample_json
FROM transmissions t
JOIN observations o ON o.transmission_id = t.id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
WHERE t.payload_type = 5
AND t.channel_hash IS NOT NULL
AND t.channel_hash NOT LIKE 'enc_%%'
AND obs.rowid IS NOT NULL AND UPPER(TRIM(obs.iata)) IN (%s)
GROUP BY t.channel_hash
ORDER BY last_activity DESC`, regionPlaceholder)
} else {
querySQL = fmt.Sprintf(`SELECT t.channel_hash,
COUNT(*) AS msg_count,
MAX(t.first_seen) AS last_activity,
(SELECT t2.decoded_json FROM transmissions t2
WHERE t2.channel_hash = t.channel_hash AND t2.payload_type = 5
ORDER BY t2.first_seen DESC LIMIT 1) AS sample_json
FROM transmissions t
JOIN observations o ON o.transmission_id = t.id
WHERE t.payload_type = 5
AND t.channel_hash IS NOT NULL
AND t.channel_hash NOT LIKE 'enc_%%'
AND EXISTS (
SELECT 1 FROM observers obs
WHERE obs.id = o.observer_id
AND UPPER(TRIM(obs.iata)) IN (%s)
)
GROUP BY t.channel_hash
ORDER BY last_activity DESC`, regionPlaceholder)
}
} else {
querySQL = `SELECT channel_hash,
COUNT(*) AS msg_count,
MAX(first_seen) AS last_activity,
(SELECT t2.decoded_json FROM transmissions t2
WHERE t2.channel_hash = t.channel_hash AND t2.payload_type = 5
ORDER BY t2.first_seen DESC LIMIT 1) AS sample_json
FROM transmissions t
WHERE payload_type = 5
AND channel_hash IS NOT NULL
AND channel_hash NOT LIKE 'enc_%%'
GROUP BY channel_hash
ORDER BY last_activity DESC`
}
rows, err := db.conn.Query(querySQL, args...)
if err != nil {
return nil, err
}
defer rows.Close()
channelMap := map[string]map[string]interface{}{}
channels := make([]map[string]interface{}, 0)
for rows.Next() {
var dj, fs sql.NullString
rows.Scan(&dj, &fs)
if !dj.Valid {
var chHash, lastActivity, sampleJSON sql.NullString
var msgCount int
if err := rows.Scan(&chHash, &msgCount, &lastActivity, &sampleJSON); err != nil {
continue
}
var decoded map[string]interface{}
if json.Unmarshal([]byte(dj.String), &decoded) != nil {
continue
}
dtype, _ := decoded["type"].(string)
if dtype != "CHAN" {
continue
}
// Filter out garbage-decrypted channel names/messages (pre-#197 data still in DB)
chanStr, _ := decoded["channel"].(string)
textStr, _ := decoded["text"].(string)
if hasGarbageChars(chanStr) || hasGarbageChars(textStr) {
continue
}
channelName, _ := decoded["channel"].(string)
channelName := nullStr(chHash)
if channelName == "" {
channelName = "unknown"
continue
}
key := channelName
ch, exists := channelMap[key]
if !exists {
ch = map[string]interface{}{
"hash": key, "name": channelName,
"lastMessage": nil, "lastSender": nil,
"messageCount": 0, "lastActivity": nullStr(fs),
}
channelMap[key] = ch
}
ch["messageCount"] = ch["messageCount"].(int) + 1
if fs.Valid {
ch["lastActivity"] = fs.String
}
if text, ok := decoded["text"].(string); ok && text != "" {
idx := strings.Index(text, ": ")
if idx > 0 {
ch["lastMessage"] = text[idx+2:]
} else {
ch["lastMessage"] = text
}
if sender, ok := decoded["sender"].(string); ok {
ch["lastSender"] = sender
var lastMessage, lastSender interface{}
if sampleJSON.Valid {
var decoded map[string]interface{}
if json.Unmarshal([]byte(sampleJSON.String), &decoded) == nil {
if text, ok := decoded["text"].(string); ok && text != "" {
idx := strings.Index(text, ": ")
if idx > 0 {
lastMessage = text[idx+2:]
} else {
lastMessage = text
}
if sender, ok := decoded["sender"].(string); ok {
lastSender = sender
}
}
}
}
channels = append(channels, map[string]interface{}{
"hash": channelName, "name": channelName,
"lastMessage": lastMessage, "lastSender": lastSender,
"messageCount": msgCount, "lastActivity": nullStr(lastActivity),
})
}
channels := make([]map[string]interface{}, 0, len(channelMap))
for _, ch := range channelMap {
channels = append(channels, ch)
// Store in cache (60s TTL)
db.channelsCacheMu.Lock()
db.channelsCacheRes = channels
db.channelsCacheKey = regionParam
db.channelsCacheExp = time.Now().Add(60 * time.Second)
db.channelsCacheMu.Unlock()
return channels, nil
}
// GetEncryptedChannels returns channels where all messages are undecryptable (no key).
// Uses channel_hash column (prefixed with 'enc_') for fast grouped queries.
func (db *DB) GetEncryptedChannels(region ...string) ([]map[string]interface{}, error) {
regionParam := ""
if len(region) > 0 {
regionParam = region[0]
}
regionCodes := normalizeRegionCodes(regionParam)
var querySQL string
args := make([]interface{}, 0, len(regionCodes))
if len(regionCodes) > 0 {
placeholders := make([]string, len(regionCodes))
for i, code := range regionCodes {
placeholders[i] = "?"
args = append(args, code)
}
regionPlaceholder := strings.Join(placeholders, ",")
if db.isV3 {
querySQL = fmt.Sprintf(`SELECT t.channel_hash,
COUNT(*) AS msg_count,
MAX(t.first_seen) AS last_activity
FROM transmissions t
JOIN observations o ON o.transmission_id = t.id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
WHERE t.payload_type = 5
AND t.channel_hash LIKE 'enc_%%'
AND obs.rowid IS NOT NULL AND UPPER(TRIM(obs.iata)) IN (%s)
GROUP BY t.channel_hash
ORDER BY last_activity DESC`, regionPlaceholder)
} else {
querySQL = fmt.Sprintf(`SELECT t.channel_hash,
COUNT(*) AS msg_count,
MAX(t.first_seen) AS last_activity
FROM transmissions t
JOIN observations o ON o.transmission_id = t.id
WHERE t.payload_type = 5
AND t.channel_hash LIKE 'enc_%%'
AND EXISTS (
SELECT 1 FROM observers obs
WHERE obs.id = o.observer_id
AND UPPER(TRIM(obs.iata)) IN (%s)
)
GROUP BY t.channel_hash
ORDER BY last_activity DESC`, regionPlaceholder)
}
} else {
querySQL = `SELECT channel_hash,
COUNT(*) AS msg_count,
MAX(first_seen) AS last_activity
FROM transmissions
WHERE payload_type = 5
AND channel_hash LIKE 'enc_%%'
GROUP BY channel_hash
ORDER BY last_activity DESC`
}
rows, err := db.conn.Query(querySQL, args...)
if err != nil {
return nil, err
}
defer rows.Close()
channels := make([]map[string]interface{}, 0)
for rows.Next() {
var chHash, lastActivity sql.NullString
var msgCount int
if err := rows.Scan(&chHash, &msgCount, &lastActivity); err != nil {
continue
}
fullHash := nullStrVal(chHash) // e.g. "enc_3A"
hexPart := strings.TrimPrefix(fullHash, "enc_")
channels = append(channels, map[string]interface{}{
"hash": fullHash,
"name": "Encrypted (0x" + hexPart + ")",
"lastMessage": nil,
"lastSender": nil,
"messageCount": msgCount,
"lastActivity": nullStr(lastActivity),
"encrypted": true,
})
}
return channels, nil
}
@@ -1244,15 +1401,16 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region .
regionPlaceholders = strings.Join(placeholders, ",")
}
// Fetch messages with channel_hash filter (pagination applied in Go after dedup)
var querySQL string
args := make([]interface{}, 0, len(regionArgs))
args := []interface{}{channelHash}
if db.isV3 {
querySQL = `SELECT o.id, t.hash, t.decoded_json, t.first_seen,
obs.id, obs.name, o.snr, o.path_json
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
WHERE t.payload_type = 5`
WHERE t.channel_hash = ? AND t.payload_type = 5`
if len(regionCodes) > 0 {
querySQL += fmt.Sprintf(" AND obs.rowid IS NOT NULL AND UPPER(TRIM(obs.iata)) IN (%s)", regionPlaceholders)
args = append(args, regionArgs...)
@@ -1264,14 +1422,11 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region .
o.observer_id, o.observer_name, o.snr, o.path_json
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
WHERE t.payload_type = 5`
WHERE t.channel_hash = ? AND t.payload_type = 5`
if len(regionCodes) > 0 {
querySQL += fmt.Sprintf(` AND EXISTS (
SELECT 1
FROM observers obs
WHERE obs.id = o.observer_id
AND UPPER(TRIM(obs.iata)) IN (%s)
)`, regionPlaceholders)
SELECT 1 FROM observers obs WHERE obs.id = o.observer_id
AND UPPER(TRIM(obs.iata)) IN (%s))`, regionPlaceholders)
args = append(args, regionArgs...)
}
querySQL += `
@@ -1303,17 +1458,6 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region .
if json.Unmarshal([]byte(dj.String), &decoded) != nil {
continue
}
dtype, _ := decoded["type"].(string)
if dtype != "CHAN" {
continue
}
ch, _ := decoded["channel"].(string)
if ch == "" {
ch = "unknown"
}
if ch != channelHash {
continue
}
text, _ := decoded["text"].(string)
sender, _ := decoded["sender"].(string)
@@ -1373,18 +1517,18 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region .
}
}
total := len(msgOrder)
// Return latest messages (tail)
start := total - limit - offset
// Return latest messages (tail) with pagination
msgTotal := len(msgOrder)
start := msgTotal - limit - offset
if start < 0 {
start = 0
}
end := total - offset
end := msgTotal - offset
if end < 0 {
end = 0
}
if end > total {
end = total
if end > msgTotal {
end = msgTotal
}
messages := make([]map[string]interface{}, 0)
@@ -1395,7 +1539,7 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region .
messages = append(messages, m.Data)
}
return messages, total, nil
return messages, msgTotal, nil
}
@@ -1704,12 +1848,10 @@ func nullInt(ni sql.NullInt64) interface{} {
// Returns the number of transmissions deleted.
// Opens a separate read-write connection since the main connection is read-only.
func (db *DB) PruneOldPackets(days int) (int64, error) {
dsn := fmt.Sprintf("file:%s?_journal_mode=WAL&_busy_timeout=10000", db.path)
rw, err := sql.Open("sqlite", dsn)
rw, err := openRW(db.path)
if err != nil {
return 0, err
}
rw.SetMaxOpenConns(1)
defer rw.Close()
cutoff := time.Now().UTC().AddDate(0, 0, -days).Format(time.RFC3339)
@@ -2053,12 +2195,10 @@ func (db *DB) GetMetricsSummary(since string) ([]MetricsSummaryRow, error) {
// PruneOldMetrics deletes observer_metrics rows older than retentionDays.
func (db *DB) PruneOldMetrics(retentionDays int) (int64, error) {
dsn := fmt.Sprintf("file:%s?_journal_mode=WAL&_busy_timeout=10000", db.path)
rw, err := sql.Open("sqlite", dsn)
rw, err := openRW(db.path)
if err != nil {
return 0, err
}
rw.SetMaxOpenConns(1)
defer rw.Close()
cutoff := time.Now().UTC().AddDate(0, 0, -retentionDays).Format(time.RFC3339)
@@ -2072,3 +2212,14 @@ func (db *DB) PruneOldMetrics(retentionDays int) (int64, error) {
}
return n, nil
}
// TouchNodeLastSeen updates last_seen for a node identified by full public key.
// Only updates if the new timestamp is newer than the existing value (or NULL).
// Returns nil even if no rows are affected (node doesn't exist).
func (db *DB) TouchNodeLastSeen(pubkey string, timestamp string) error {
_, err := db.conn.Exec(
"UPDATE nodes SET last_seen = ? WHERE public_key = ? AND (last_seen IS NULL OR last_seen < ?)",
timestamp, pubkey, timestamp,
)
return err
}
+81 -24
View File
@@ -60,6 +60,7 @@ func setupTestDB(t *testing.T) *DB {
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
channel_hash TEXT DEFAULT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
@@ -124,10 +125,10 @@ func seedTestData(t *testing.T, db *DB) {
VALUES ('1122334455667788', 'TestRoom', 'room', 37.4, -121.9, ?, '2026-01-01T00:00:00Z', 5)`, twoDaysAgo)
// Seed transmissions
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('AABB', 'abc123def4567890', ?, 1, 4, '{"pubKey":"aabbccdd11223344","name":"TestRepeater","type":"ADVERT","timestamp":1700000000,"timestampISO":"2023-11-14T22:13:20.000Z","signature":"abcdef","flags":{"isRepeater":true},"lat":37.5,"lon":-122.0}')`, recent)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('CCDD', '1234567890abcdef', ?, 1, 5, '{"type":"CHAN","channel":"#test","text":"Hello: World","sender":"TestUser"}')`, yesterday)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AABB', 'abc123def4567890', ?, 1, 4, '{"pubKey":"aabbccdd11223344","name":"TestRepeater","type":"ADVERT","timestamp":1700000000,"timestampISO":"2023-11-14T22:13:20.000Z","signature":"abcdef","flags":{"isRepeater":true},"lat":37.5,"lon":-122.0}', '#test')`, recent)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('CCDD', '1234567890abcdef', ?, 1, 5, '{"type":"CHAN","channel":"#test","text":"Hello: World","sender":"TestUser"}', '#test')`, yesterday)
// Second ADVERT for same node with different hash_size (raw_hex byte 0x1F → hs=1 vs 0xBB → hs=3)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('AA1F', 'def456abc1230099', ?, 1, 4, '{"pubKey":"aabbccdd11223344","name":"TestRepeater","type":"ADVERT","timestamp":1700000100,"timestampISO":"2023-11-14T22:14:40.000Z","signature":"fedcba","flags":{"isRepeater":true},"lat":37.5,"lon":-122.0}')`, yesterday)
@@ -735,12 +736,12 @@ func TestGetChannelMessagesRegionFiltering(t *testing.T) {
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs1', 'Observer One', 'SJC')`)
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs2', 'Observer Two', ' sfo ')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'chanregion0001', ?, 1, 5,
'{"type":"CHAN","channel":"#region","text":"SjcUser: One","sender":"SjcUser"}')`, ts1)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
'{"type":"CHAN","channel":"#region","text":"SjcUser: One","sender":"SjcUser"}', '#region')`, ts1)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('BB', 'chanregion0002', ?, 1, 5,
'{"type":"CHAN","channel":"#region","text":"SfoUser: Two","sender":"SfoUser"}')`, ts2)
'{"type":"CHAN","channel":"#region","text":"SfoUser: Two","sender":"SfoUser"}', '#region')`, ts2)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 10.0, -90, '[]', ?)`, epoch1)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
@@ -1119,6 +1120,7 @@ func setupTestDBV2(t *testing.T) *DB {
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
channel_hash TEXT DEFAULT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
@@ -1202,12 +1204,12 @@ func TestGetChannelMessagesDedup(t *testing.T) {
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs2', 'Observer Two', 'SFO')`)
// Insert two transmissions with same hash to test dedup
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'chanmsg00000001', '2026-01-15T10:00:00Z', 1, 5,
'{"type":"CHAN","channel":"#general","text":"User1: Hello","sender":"User1"}')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
'{"type":"CHAN","channel":"#general","text":"User1: Hello","sender":"User1"}', '#general')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('BB', 'chanmsg00000002', '2026-01-15T10:01:00Z', 1, 5,
'{"type":"CHAN","channel":"#general","text":"User2: World","sender":"User2"}')`)
'{"type":"CHAN","channel":"#general","text":"User2: World","sender":"User2"}', '#general')`)
// Observations: first msg seen by two observers (dedup), second by one
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
@@ -1251,9 +1253,9 @@ func TestGetChannelMessagesNoSender(t *testing.T) {
defer db.Close()
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs1', 'Observer One', 'SJC')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('CC', 'chanmsg00000003', '2026-01-15T10:02:00Z', 1, 5,
'{"type":"CHAN","channel":"#noname","text":"plain text no colon"}')`)
'{"type":"CHAN","channel":"#noname","text":"plain text no colon"}', '#noname')`)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 12.0, -90, null, 1736935300)`)
@@ -1356,9 +1358,9 @@ func TestGetChannelMessagesObserverFallback(t *testing.T) {
defer db.Close()
// Observer with ID but no name entry (observer_idx won't match)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'chanmsg00000004', '2026-01-15T10:00:00Z', 1, 5,
'{"type":"CHAN","channel":"#obs","text":"Sender: Test","sender":"Sender"}')`)
'{"type":"CHAN","channel":"#obs","text":"Sender: Test","sender":"Sender"}', '#obs')`)
// Observation without observer (observer_idx = NULL)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, NULL, 12.0, -90, null, 1736935200)`)
@@ -1380,12 +1382,12 @@ func TestGetChannelsMultiple(t *testing.T) {
defer db.Close()
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs1', 'Observer', 'SJC')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'chan1hash', '2026-01-15T10:00:00Z', 1, 5,
'{"type":"CHAN","channel":"#alpha","text":"Alice: Hello","sender":"Alice"}')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
'{"type":"CHAN","channel":"#alpha","text":"Alice: Hello","sender":"Alice"}', '#alpha')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('BB', 'chan2hash', '2026-01-15T10:01:00Z', 1, 5,
'{"type":"CHAN","channel":"#beta","text":"Bob: World","sender":"Bob"}')`)
'{"type":"CHAN","channel":"#beta","text":"Bob: World","sender":"Bob"}', '#beta')`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('CC', 'chan3hash', '2026-01-15T10:02:00Z', 1, 5,
'{"type":"CHAN","channel":"","text":"No channel"}')`)
@@ -1468,13 +1470,13 @@ func TestGetChannelsStaleMessage(t *testing.T) {
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs2', 'Observer2', 'SFO')`)
// Older message (first_seen T1)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'oldhash1', '2026-01-15T10:00:00Z', 1, 5,
'{"type":"CHAN","channel":"#test","text":"Alice: Old message","sender":"Alice"}')`)
'{"type":"CHAN","channel":"#test","text":"Alice: Old message","sender":"Alice"}', '#test')`)
// Newer message (first_seen T2 > T1)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('BB', 'newhash2', '2026-01-15T10:05:00Z', 1, 5,
'{"type":"CHAN","channel":"#test","text":"Bob: New message","sender":"Bob"}')`)
'{"type":"CHAN","channel":"#test","text":"Bob: New message","sender":"Bob"}', '#test')`)
// Observations: older message re-observed AFTER newer message (stale scenario)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, timestamp)
@@ -1504,6 +1506,61 @@ func TestGetChannelsStaleMessage(t *testing.T) {
}
}
func TestGetChannelsRegionFiltering(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs1', 'Observer1', 'SJC')`)
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs2', 'Observer2', 'SFO')`)
// Channel message seen only in SJC
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('AA', 'hash1', '2026-01-15T10:00:00Z', 1, 5,
'{"type":"CHAN","channel":"#sjc-only","text":"Alice: Hello SJC","sender":"Alice"}', '#sjc-only')`)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, timestamp)
VALUES (1, 1, 12.0, -90, 1736935200)`)
// Channel message seen only in SFO
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('BB', 'hash2', '2026-01-15T10:05:00Z', 1, 5,
'{"type":"CHAN","channel":"#sfo-only","text":"Bob: Hello SFO","sender":"Bob"}', '#sfo-only')`)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, timestamp)
VALUES (2, 2, 14.0, -88, 1736935500)`)
// No region filter — both channels
all, err := db.GetChannels()
if err != nil {
t.Fatal(err)
}
if len(all) != 2 {
t.Fatalf("expected 2 channels without region filter, got %d", len(all))
}
// Filter SJC — only #sjc-only
sjc, err := db.GetChannels("SJC")
if err != nil {
t.Fatal(err)
}
if len(sjc) != 1 {
t.Fatalf("expected 1 channel for SJC, got %d", len(sjc))
}
if sjc[0]["name"] != "#sjc-only" {
t.Errorf("expected channel '#sjc-only', got %q", sjc[0]["name"])
}
// Filter SFO — only #sfo-only
sfo, err := db.GetChannels("SFO")
if err != nil {
t.Fatal(err)
}
if len(sfo) != 1 {
t.Fatalf("expected 1 channel for SFO, got %d", len(sfo))
}
if sfo[0]["name"] != "#sfo-only" {
t.Errorf("expected channel '#sfo-only', got %q", sfo[0]["name"])
}
}
func TestNodeTelemetryFields(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
+39 -12
View File
@@ -9,6 +9,8 @@ import (
"math"
"strings"
"time"
"github.com/meshcore-analyzer/sigvalidate"
)
// Route type constants (header bits 1-0)
@@ -92,6 +94,7 @@ type Payload struct {
Timestamp uint32 `json:"timestamp,omitempty"`
TimestampISO string `json:"timestampISO,omitempty"`
Signature string `json:"signature,omitempty"`
SignatureValid *bool `json:"signatureValid,omitempty"`
Flags *AdvertFlags `json:"flags,omitempty"`
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
@@ -113,6 +116,7 @@ type DecodedPacket struct {
Path Path `json:"path"`
Payload Payload `json:"payload"`
Raw string `json:"raw"`
Anomaly string `json:"anomaly,omitempty"`
}
func decodeHeader(b byte) Header {
@@ -188,7 +192,7 @@ func decodeAck(buf []byte) Payload {
}
}
func decodeAdvert(buf []byte) Payload {
func decodeAdvert(buf []byte, validateSignatures bool) Payload {
if len(buf) < 100 {
return Payload{Type: "ADVERT", Error: "too short for advert", RawHex: hex.EncodeToString(buf)}
}
@@ -206,6 +210,16 @@ func decodeAdvert(buf []byte) Payload {
Signature: signature,
}
if validateSignatures {
valid, err := sigvalidate.ValidateAdvert(buf[0:32], buf[36:100], timestamp, appdata)
if err != nil {
f := false
p.SignatureValid = &f
} else {
p.SignatureValid = &valid
}
}
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
@@ -308,7 +322,7 @@ func decodeTrace(buf []byte) Payload {
return p
}
func decodePayload(payloadType int, buf []byte) Payload {
func decodePayload(payloadType int, buf []byte, validateSignatures bool) Payload {
switch payloadType {
case PayloadREQ:
return decodeEncryptedPayload("REQ", buf)
@@ -319,7 +333,7 @@ func decodePayload(payloadType int, buf []byte) Payload {
case PayloadACK:
return decodeAck(buf)
case PayloadADVERT:
return decodeAdvert(buf)
return decodeAdvert(buf, validateSignatures)
case PayloadGRP_TXT:
return decodeGrpTxt(buf)
case PayloadANON_REQ:
@@ -334,7 +348,7 @@ func decodePayload(payloadType int, buf []byte) Payload {
}
// DecodePacket decodes a hex-encoded MeshCore packet.
func DecodePacket(hexString string) (*DecodedPacket, error) {
func DecodePacket(hexString string, validateSignatures bool) (*DecodedPacket, error) {
hexString = strings.ReplaceAll(hexString, " ", "")
hexString = strings.ReplaceAll(hexString, "\n", "")
hexString = strings.ReplaceAll(hexString, "\r", "")
@@ -372,25 +386,37 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
offset += bytesConsumed
payloadBuf := buf[offset:]
payload := decodePayload(header.PayloadType, payloadBuf)
payload := decodePayload(header.PayloadType, payloadBuf, validateSignatures)
// TRACE packets store hop IDs in the payload (buf[9:]) rather than the header
// path field. The header path byte still encodes hashSize in bits 6-7, which
// we use to split the payload path data into individual hop prefixes.
// The header path contains SNR bytes — one per hop that actually forwarded.
// path field. Firmware always sends TRACE as DIRECT (route_type 2 or 3);
// FLOOD-routed TRACEs are anomalous but handled gracefully (parsed, but
// flagged). The TRACE flags byte (payload offset 8) encodes path_sz in
// bits 0-1 as a power-of-two exponent: hash_bytes = 1 << path_sz.
// NOT the header path byte's hash_size bits. The header path contains SNR
// bytes — one per hop that actually forwarded.
// We expose hopsCompleted (count of SNR bytes) so consumers can distinguish
// how far the trace got vs the full intended route.
var anomaly string
if header.PayloadType == PayloadTRACE && payload.PathData != "" {
// Flag anomalous routing — firmware only sends TRACE as DIRECT
if header.RouteType != RouteDirect && header.RouteType != RouteTransportDirect {
anomaly = "TRACE packet with non-DIRECT routing (expected DIRECT or TRANSPORT_DIRECT)"
}
// The header path hops count represents SNR entries = completed hops
hopsCompleted := path.HashCount
pathBytes, err := hex.DecodeString(payload.PathData)
if err == nil && path.HashSize > 0 {
hops := make([]string, 0, len(pathBytes)/path.HashSize)
for i := 0; i+path.HashSize <= len(pathBytes); i += path.HashSize {
hops = append(hops, strings.ToUpper(hex.EncodeToString(pathBytes[i:i+path.HashSize])))
if err == nil && payload.TraceFlags != nil {
// path_sz from flags byte is a power-of-two exponent per firmware:
// hash_bytes = 1 << (flags & 0x03)
pathSz := 1 << (*payload.TraceFlags & 0x03)
hops := make([]string, 0, len(pathBytes)/pathSz)
for i := 0; i+pathSz <= len(pathBytes); i += pathSz {
hops = append(hops, strings.ToUpper(hex.EncodeToString(pathBytes[i:i+pathSz])))
}
path.Hops = hops
path.HashCount = len(hops)
path.HashSize = pathSz
path.HopsCompleted = &hopsCompleted
}
}
@@ -411,6 +437,7 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
Path: path,
Payload: payload,
Raw: strings.ToUpper(hexString),
Anomaly: anomaly,
}, nil
}
+188 -11
View File
@@ -1,6 +1,9 @@
package main
import (
"crypto/ed25519"
"encoding/binary"
"encoding/hex"
"testing"
)
@@ -65,7 +68,7 @@ func TestDecodePacket_TransportFloodHasCodes(t *testing.T) {
// Path byte: 0x00 (hashSize=1, hashCount=0)
// Payload: at least some bytes for GRP_TXT
hex := "14AABBCCDD00112233445566778899"
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -85,7 +88,7 @@ func TestDecodePacket_FloodHasNoCodes(t *testing.T) {
// Path byte: 0x00 (no hops)
// Some payload bytes
hex := "110011223344556677889900AABBCCDD"
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -240,7 +243,7 @@ func TestZeroHopDirectHashSize(t *testing.T) {
// pathByte=0x00 → hash_count=0, hash_size bits=0 → should get HashSize=0
// Need at least a few payload bytes after pathByte.
hex := "02" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -254,7 +257,7 @@ func TestZeroHopDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// pathByte=0x40 → hash_count=0, hash_size bits=01 → should still get HashSize=0
// because hash_count is zero (lower 6 bits are 0).
hex := "02" + "40" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -267,7 +270,7 @@ func TestZeroHopTransportDirectHashSize(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0x00 → hash_count=0 → should get HashSize=0
hex := "03" + "11223344" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -280,7 +283,7 @@ func TestZeroHopTransportDirectHashSizeWithNonZeroUpperBits(t *testing.T) {
// TRANSPORT_DIRECT (RouteType=3) + REQ (PayloadType=0) → header byte = 0x03
// 4 bytes transport codes + pathByte=0xC0 → hash_count=0, hash_size bits=11 → should still get HashSize=0
hex := "03" + "11223344" + "C0" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -293,7 +296,7 @@ func TestNonDirectZeroPathByteKeepsHashSize(t *testing.T) {
// FLOOD (RouteType=1) + REQ (PayloadType=0) → header byte = 0x01
// pathByte=0x00 → even though hash_count=0, non-DIRECT should keep HashSize=1
hex := "01" + "00" + repeatHex("AA", 20)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -307,7 +310,7 @@ func TestDirectNonZeroHopKeepsHashSize(t *testing.T) {
// pathByte=0x01 → hash_count=1, hash_size=1 → should keep HashSize=1
// Need 1 hop hash byte after pathByte.
hex := "02" + "01" + repeatHex("BB", 21)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket failed: %v", err)
}
@@ -336,7 +339,7 @@ func TestDecodePacket_TraceHopsCompleted(t *testing.T) {
"00" + // flags = 0
"DEADBEEF" // 4 hops (1-byte hash each)
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
@@ -354,6 +357,10 @@ func TestDecodePacket_TraceHopsCompleted(t *testing.T) {
if *pkt.Path.HopsCompleted != 2 {
t.Errorf("expected HopsCompleted=2, got %d", *pkt.Path.HopsCompleted)
}
// FLOOD routing for TRACE is anomalous
if pkt.Anomaly == "" {
t.Error("expected anomaly flag for FLOOD-routed TRACE")
}
}
func TestDecodePacket_TraceNoSNR(t *testing.T) {
@@ -365,7 +372,7 @@ func TestDecodePacket_TraceNoSNR(t *testing.T) {
"00" + // flags
"AABBCC" // 3 hops intended
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
@@ -389,7 +396,7 @@ func TestDecodePacket_TraceFullyCompleted(t *testing.T) {
"00" + // flags
"DDEEFF" // 3 hops intended
pkt, err := DecodePacket(hex)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
@@ -403,3 +410,173 @@ func TestDecodePacket_TraceFullyCompleted(t *testing.T) {
t.Errorf("expected 3 hops, got %d", len(pkt.Path.Hops))
}
}
func TestDecodePacket_TraceFlags1_TwoBytePathSz(t *testing.T) {
// TRACE with flags=1 → path_sz = 1 << (1 & 0x03) = 2-byte hashes
// Firmware always sends TRACE as DIRECT (route_type=2), so header byte =
// (0<<6)|(9<<2)|2 = 0x26. path_length 0x00 = 0 SNR bytes.
hex := "2600" + // header (DIRECT+TRACE) + path_length (0 SNR)
"01000000" + // tag
"02000000" + // authCode
"01" + // flags = 1 → path_sz = 2
"AABBCCDD" // 4 bytes = 2 hops of 2-byte each
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if len(pkt.Path.Hops) != 2 {
t.Errorf("expected 2 hops (2-byte path_sz), got %d: %v", len(pkt.Path.Hops), pkt.Path.Hops)
}
if pkt.Path.HashSize != 2 {
t.Errorf("expected HashSize=2, got %d", pkt.Path.HashSize)
}
if pkt.Anomaly != "" {
t.Errorf("expected no anomaly for DIRECT TRACE, got %q", pkt.Anomaly)
}
}
func TestDecodePacket_TraceFlags2_FourBytePathSz(t *testing.T) {
// TRACE with flags=2 → path_sz = 1 << (2 & 0x03) = 4-byte hashes
// DIRECT route_type (0x26)
hex := "2600" + // header (DIRECT+TRACE) + path_length (0 SNR)
"01000000" + // tag
"02000000" + // authCode
"02" + // flags = 2 → path_sz = 4
"AABBCCDD11223344" // 8 bytes = 2 hops of 4-byte each
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if len(pkt.Path.Hops) != 2 {
t.Errorf("expected 2 hops (4-byte path_sz), got %d: %v", len(pkt.Path.Hops), pkt.Path.Hops)
}
if pkt.Path.HashSize != 4 {
t.Errorf("expected HashSize=4, got %d", pkt.Path.HashSize)
}
}
func TestDecodePacket_TracePathSzUnevenPayload(t *testing.T) {
// TRACE with flags=1 → path_sz=2, but 5 bytes of path data (not evenly divisible)
// Should produce 2 hops (4 bytes) and ignore the trailing byte
hex := "2600" + // header (DIRECT+TRACE) + path_length (0 SNR)
"01000000" + // tag
"02000000" + // authCode
"01" + // flags = 1 → path_sz = 2
"AABBCCDDEE" // 5 bytes → 2 hops, 1 byte remainder ignored
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if len(pkt.Path.Hops) != 2 {
t.Errorf("expected 2 hops (trailing byte ignored), got %d: %v", len(pkt.Path.Hops), pkt.Path.Hops)
}
}
func TestDecodePacket_TraceTransportDirect(t *testing.T) {
// TRACE via TRANSPORT_DIRECT (route_type=3) — includes 4 transport code bytes
// header: (0<<6)|(9<<2)|3 = 0x27
hex := "27" + // header (TRANSPORT_DIRECT+TRACE)
"AABB" + "CCDD" + // transport codes (2+2 bytes)
"02" + // path_length: hash_count=2 SNR bytes
"EEFF" + // 2 SNR bytes
"01000000" + // tag
"02000000" + // authCode
"00" + // flags = 0 → path_sz = 1
"112233" // 3 hops (1-byte each)
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if pkt.TransportCodes == nil {
t.Fatal("expected transport codes for TRANSPORT_DIRECT")
}
if pkt.TransportCodes.Code1 != "AABB" {
t.Errorf("expected Code1=AABB, got %s", pkt.TransportCodes.Code1)
}
if len(pkt.Path.Hops) != 3 {
t.Errorf("expected 3 hops, got %d: %v", len(pkt.Path.Hops), pkt.Path.Hops)
}
if pkt.Path.HopsCompleted == nil || *pkt.Path.HopsCompleted != 2 {
t.Errorf("expected HopsCompleted=2, got %v", pkt.Path.HopsCompleted)
}
if pkt.Anomaly != "" {
t.Errorf("expected no anomaly for TRANSPORT_DIRECT TRACE, got %q", pkt.Anomaly)
}
}
func TestDecodePacket_TraceFloodRouteAnomaly(t *testing.T) {
// TRACE via FLOOD (route_type=1) — anomalous per firmware (firmware only
// sends TRACE as DIRECT). Should still parse but flag the anomaly.
hex := "2500" + // header (FLOOD+TRACE) + path_length (0 SNR)
"01000000" + // tag
"02000000" + // authCode
"01" + // flags = 1 → path_sz = 2
"AABBCCDD" // 4 bytes = 2 hops of 2-byte each
pkt, err := DecodePacket(hex, false)
if err != nil {
t.Fatalf("should not crash on anomalous FLOOD+TRACE: %v", err)
}
if len(pkt.Path.Hops) != 2 {
t.Errorf("expected 2 hops even for anomalous FLOOD route, got %d", len(pkt.Path.Hops))
}
if pkt.Anomaly == "" {
t.Error("expected anomaly flag for FLOOD-routed TRACE, got empty string")
}
}
func TestDecodeAdvertSignatureValidation(t *testing.T) {
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1234567890
appdata := []byte{0x02} // flags: repeater, no extras
// Build signed message: pubKey(32) + timestamp(4 LE) + appdata
msg := make([]byte, 32+4+len(appdata))
copy(msg[0:32], pub)
binary.LittleEndian.PutUint32(msg[32:36], timestamp)
copy(msg[36:], appdata)
sig := ed25519.Sign(priv, msg)
// Build a raw advert buffer: pubKey(32) + timestamp(4) + signature(64) + appdata
buf := make([]byte, 100+len(appdata))
copy(buf[0:32], pub)
binary.LittleEndian.PutUint32(buf[32:36], timestamp)
copy(buf[36:100], sig)
copy(buf[100:], appdata)
// With validation enabled
p := decodeAdvert(buf, true)
if p.SignatureValid == nil {
t.Fatal("expected SignatureValid to be set")
}
if !*p.SignatureValid {
t.Error("expected valid signature")
}
if p.PubKey != hex.EncodeToString(pub) {
t.Errorf("pubkey mismatch: got %s", p.PubKey)
}
// Tamper with signature → invalid
buf[40] ^= 0xFF
p = decodeAdvert(buf, true)
if p.SignatureValid == nil {
t.Fatal("expected SignatureValid to be set")
}
if *p.SignatureValid {
t.Error("expected invalid signature after tampering")
}
// Without validation → SignatureValid should be nil
p = decodeAdvert(buf, false)
if p.SignatureValid != nil {
t.Error("expected SignatureValid to be nil when validation disabled")
}
}
+145
View File
@@ -0,0 +1,145 @@
package main
import (
"encoding/json"
"net/http/httptest"
"testing"
"time"
)
// seedEncryptedChannelData adds undecryptable GRP_TXT packets to the test DB.
func seedEncryptedChannelData(t *testing.T, db *DB) {
t.Helper()
now := time.Now().UTC()
recent := now.Add(-1 * time.Hour).Format(time.RFC3339)
recentEpoch := now.Add(-1 * time.Hour).Unix()
// Two encrypted GRP_TXT packets on channel hash "A1B2"
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('EE01', 'enc_hash_001', ?, 1, 5, '{"type":"GRP_TXT","channelHashHex":"A1B2","decryptionStatus":"no_key"}', 'enc_A1B2')`, recent)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json, channel_hash)
VALUES ('EE02', 'enc_hash_002', ?, 1, 5, '{"type":"GRP_TXT","channelHashHex":"A1B2","decryptionStatus":"no_key"}', 'enc_A1B2')`, recent)
// Observations for both
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES ((SELECT id FROM transmissions WHERE hash='enc_hash_001'), 1, 10.0, -90, '[]', ?)`, recentEpoch)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES ((SELECT id FROM transmissions WHERE hash='enc_hash_002'), 1, 10.0, -90, '[]', ?)`, recentEpoch)
}
func TestGetEncryptedChannels(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
seedEncryptedChannelData(t, db)
channels, err := db.GetEncryptedChannels()
if err != nil {
t.Fatal(err)
}
if len(channels) != 1 {
t.Fatalf("expected 1 encrypted channel, got %d", len(channels))
}
ch := channels[0]
if ch["hash"] != "enc_A1B2" {
t.Errorf("expected hash enc_A1B2, got %v", ch["hash"])
}
if ch["encrypted"] != true {
t.Errorf("expected encrypted=true, got %v", ch["encrypted"])
}
if ch["messageCount"] != 2 {
t.Errorf("expected messageCount=2, got %v", ch["messageCount"])
}
}
func TestChannelsAPIExcludesEncrypted(t *testing.T) {
_, router := setupTestServer(t)
// Seed encrypted data into the server's DB
// setupTestServer uses seedTestData which has no encrypted packets,
// so default /api/channels should NOT include encrypted channels.
req := httptest.NewRequest("GET", "/api/channels", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d", w.Code)
}
var body map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &body)
channels := body["channels"].([]interface{})
for _, ch := range channels {
m := ch.(map[string]interface{})
if enc, ok := m["encrypted"]; ok && enc == true {
t.Errorf("default /api/channels should not include encrypted channels, found: %v", m["hash"])
}
}
}
func TestChannelsAPIIncludesEncryptedWithParam(t *testing.T) {
srv, router := setupTestServer(t)
// Add encrypted data to the server's DB
seedEncryptedChannelData(t, srv.db)
// Reload store so in-memory also has the data
store := NewPacketStore(srv.db, nil)
if err := store.Load(); err != nil {
t.Fatalf("store.Load: %v", err)
}
srv.store = store
req := httptest.NewRequest("GET", "/api/channels?includeEncrypted=true", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d", w.Code)
}
var body map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &body)
channels := body["channels"].([]interface{})
foundEncrypted := false
for _, ch := range channels {
m := ch.(map[string]interface{})
if enc, ok := m["encrypted"]; ok && enc == true {
foundEncrypted = true
break
}
}
if !foundEncrypted {
t.Error("expected encrypted channels with includeEncrypted=true, found none")
}
}
func TestChannelMessagesExcludesEncrypted(t *testing.T) {
srv, router := setupTestServer(t)
seedEncryptedChannelData(t, srv.db)
store := NewPacketStore(srv.db, nil)
if err := store.Load(); err != nil {
t.Fatalf("store.Load: %v", err)
}
srv.store = store
// Request messages for the encrypted channel — should return empty
req := httptest.NewRequest("GET", "/api/channels/enc_A1B2/messages", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d", w.Code)
}
var body map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &body)
messages, ok := body["messages"].([]interface{})
if !ok {
// messages might be null/missing — that's fine, means no messages
return
}
// Encrypted messages should not be returned as readable messages
for _, msg := range messages {
m := msg.(map[string]interface{})
if text, ok := m["text"].(string); ok && text != "" {
t.Errorf("encrypted channel should not return readable messages, got text: %s", text)
}
}
}
+298 -20
View File
@@ -85,6 +85,12 @@ func makeTestStore(count int, startTime time.Time, intervalMin int) *PacketStore
// Subpath index
addTxToSubpathIndex(store.spIndex, tx)
// Track bytes for self-accounting
store.trackedBytes += estimateStoreTxBytes(tx)
for _, obs := range tx.Observations {
store.trackedBytes += estimateStoreObsBytes(obs)
}
}
return store
@@ -166,43 +172,43 @@ func TestEvictStale_MemoryBasedEviction(t *testing.T) {
// All packets are recent (1h old) so time-based won't trigger.
store.retentionHours = 24
store.maxMemoryMB = 3
// Inject deterministic estimator: simulates 6MB (over 3MB limit).
// Uses packet count so it scales correctly after eviction.
store.memoryEstimator = func() float64 {
return float64(len(store.packets)*5120+store.totalObs*500) / 1048576.0
}
// Set trackedBytes to simulate 6MB (over 3MB limit).
store.trackedBytes = 6 * 1048576
evicted := store.EvictStale()
if evicted == 0 {
t.Fatal("expected some evictions for memory cap")
}
estMB := store.estimatedMemoryMB()
if estMB > 3.5 {
t.Fatalf("expected <=3.5MB after eviction, got %.1fMB", estMB)
// 25% safety cap should limit to 250 per pass
if evicted > 250 {
t.Fatalf("25%% safety cap violated: evicted %d", evicted)
}
// trackedBytes should have decreased
if store.trackedBytes >= 6*1048576 {
t.Fatal("trackedBytes should have decreased after eviction")
}
}
// TestEvictStale_MemoryBasedEviction_UnderestimatedHeap verifies that eviction
// fires correctly when actual heap is much larger than a formula-based estimate
// would report — the scenario that caused OOM kills in production.
// TestEvictStale_MemoryBasedEviction_UnderestimatedHeap verifies that the 25%
// safety cap prevents cascading eviction even when trackedBytes is very high.
func TestEvictStale_MemoryBasedEviction_UnderestimatedHeap(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(1000, now.Add(-1*time.Hour), 0)
store.retentionHours = 24
store.maxMemoryMB = 500
// Simulate actual heap 5x over budget (like production: ~5GB actual vs ~1GB limit).
store.memoryEstimator = func() float64 {
return 2500.0 // 2500MB actual vs 500MB limit
}
// Simulate trackedBytes 5x over budget.
store.trackedBytes = 2500 * 1048576
evicted := store.EvictStale()
if evicted == 0 {
t.Fatal("expected evictions when heap is 5x over limit")
t.Fatal("expected evictions when tracked is 5x over limit")
}
// Should keep roughly 500/2500 * 0.9 = 18% of packets → ~180 of 1000.
remaining := len(store.packets)
if remaining > 250 {
t.Fatalf("expected most packets evicted (heap 5x over), but %d of 1000 remain", remaining)
// Safety cap: max 25% per pass = 250
if evicted > 250 {
t.Fatalf("25%% safety cap violated: evicted %d of 1000", evicted)
}
if evicted != 250 {
t.Fatalf("expected exactly 250 evicted (25%% cap), got %d", evicted)
}
}
@@ -239,6 +245,79 @@ func TestEvictStale_CleansNodeIndexes(t *testing.T) {
}
}
func TestEvictStale_CleansResolvedPathNodeIndexes(t *testing.T) {
now := time.Now().UTC()
store := &PacketStore{
packets: make([]*StoreTx, 0),
byHash: make(map[string]*StoreTx),
byTxID: make(map[int]*StoreTx),
byObsID: make(map[int]*StoreObs),
byObserver: make(map[string][]*StoreObs),
byNode: make(map[string][]*StoreTx),
nodeHashes: make(map[string]map[string]bool),
byPayloadType: make(map[int][]*StoreTx),
spIndex: make(map[string]int),
distHops: make([]distHopRecord, 0),
distPaths: make([]distPathRecord, 0),
rfCache: make(map[string]*cachedResult),
topoCache: make(map[string]*cachedResult),
hashCache: make(map[string]*cachedResult),
chanCache: make(map[string]*cachedResult),
distCache: make(map[string]*cachedResult),
subpathCache: make(map[string]*cachedResult),
rfCacheTTL: 15 * time.Second,
retentionHours: 24,
}
// Create a packet indexed only via resolved_path (no decoded JSON pubkeys)
relayPK := "relay0001abcdef"
tx := &StoreTx{
ID: 1,
Hash: "hash_rp_001",
FirstSeen: now.Add(-48 * time.Hour).UTC().Format(time.RFC3339),
}
rpPtr := &relayPK
obs := &StoreObs{
ID: 100,
TransmissionID: 1,
ObserverID: "obs0",
Timestamp: tx.FirstSeen,
ResolvedPath: []*string{rpPtr},
}
tx.Observations = append(tx.Observations, obs)
tx.ResolvedPath = []*string{rpPtr}
store.packets = append(store.packets, tx)
store.byHash[tx.Hash] = tx
store.byTxID[tx.ID] = tx
store.byObsID[obs.ID] = obs
store.byObserver["obs0"] = append(store.byObserver["obs0"], obs)
// Index via resolved_path
store.indexByNode(tx)
// Verify indexed
if len(store.byNode[relayPK]) != 1 {
t.Fatalf("expected 1 entry in byNode[%s], got %d", relayPK, len(store.byNode[relayPK]))
}
if !store.nodeHashes[relayPK][tx.Hash] {
t.Fatalf("expected nodeHashes[%s] to contain %s", relayPK, tx.Hash)
}
evicted := store.EvictStale()
if evicted != 1 {
t.Fatalf("expected 1 evicted, got %d", evicted)
}
// Verify resolved_path entries are cleaned up
if len(store.byNode[relayPK]) != 0 {
t.Fatalf("expected byNode[%s] to be empty after eviction, got %d", relayPK, len(store.byNode[relayPK]))
}
if _, exists := store.nodeHashes[relayPK]; exists {
t.Fatalf("expected nodeHashes[%s] to be deleted after eviction", relayPK)
}
}
func TestEvictStale_RunEvictionThreadSafe(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(20, now.Add(-48*time.Hour), 0)
@@ -302,3 +381,202 @@ func TestCacheTTLDefaults(t *testing.T) {
t.Fatalf("expected default rfCacheTTL=15s, got %v", store.rfCacheTTL)
}
}
// --- Self-accounting memory tracking tests ---
func TestTrackedBytes_IncreasesOnInsert(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(0, now, 0)
if store.trackedBytes != 0 {
t.Fatalf("expected 0 trackedBytes for empty store, got %d", store.trackedBytes)
}
store2 := makeTestStore(10, now, 1)
if store2.trackedBytes <= 0 {
t.Fatal("expected positive trackedBytes after inserting 10 packets")
}
// Each packet has 2 observations; should be roughly 10*(384+5*48) + 20*(192+2*48) = 10*624 + 20*288 = 12000
expectedMin := int64(10*600 + 20*250) // rough lower bound
if store2.trackedBytes < expectedMin {
t.Fatalf("trackedBytes %d seems too low (expected > %d)", store2.trackedBytes, expectedMin)
}
}
func TestTrackedBytes_DecreasesOnEvict(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(100, now.Add(-48*time.Hour), 0)
store.retentionHours = 24
beforeBytes := store.trackedBytes
if beforeBytes <= 0 {
t.Fatal("expected positive trackedBytes before eviction")
}
evicted := store.EvictStale()
if evicted != 100 {
t.Fatalf("expected 100 evicted, got %d", evicted)
}
if store.trackedBytes != 0 {
t.Fatalf("expected 0 trackedBytes after evicting all, got %d", store.trackedBytes)
}
}
func TestTrackedBytes_MatchesExpectedAfterMixedInsertEvict(t *testing.T) {
now := time.Now().UTC()
// Create 100 packets, 50 old + 50 recent
store := makeTestStore(100, now.Add(-48*time.Hour), 0)
for i := 50; i < 100; i++ {
store.packets[i].FirstSeen = now.Add(-1 * time.Hour).Format(time.RFC3339)
}
store.retentionHours = 24
totalBefore := store.trackedBytes
// Calculate expected bytes for first 50 packets (to be evicted)
var evictedBytes int64
for i := 0; i < 50; i++ {
tx := store.packets[i]
evictedBytes += estimateStoreTxBytes(tx)
for _, obs := range tx.Observations {
evictedBytes += estimateStoreObsBytes(obs)
}
}
store.EvictStale()
expectedAfter := totalBefore - evictedBytes
if store.trackedBytes != expectedAfter {
t.Fatalf("trackedBytes %d != expected %d (before=%d, evicted=%d)",
store.trackedBytes, expectedAfter, totalBefore, evictedBytes)
}
}
func TestWatermarkHysteresis(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(1000, now.Add(-1*time.Hour), 0)
store.retentionHours = 0 // no time-based eviction
store.maxMemoryMB = 1 // 1MB budget
// Set trackedBytes to just above high watermark
highWatermark := int64(1 * 1048576)
lowWatermark := int64(float64(highWatermark) * 0.85)
store.trackedBytes = highWatermark + 1
evicted := store.EvictStale()
if evicted == 0 {
t.Fatal("expected eviction when above high watermark")
}
if store.trackedBytes > lowWatermark+1024 {
t.Fatalf("expected trackedBytes near low watermark after eviction, got %d (low=%d)",
store.trackedBytes, lowWatermark)
}
// Now set trackedBytes to just below high watermark — should NOT trigger
store.trackedBytes = highWatermark - 1
evicted2 := store.EvictStale()
if evicted2 != 0 {
t.Fatalf("expected no eviction below high watermark, got %d", evicted2)
}
}
func TestSafetyCap25Percent(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(1000, now.Add(-1*time.Hour), 0)
store.retentionHours = 0
store.maxMemoryMB = 1
// Set trackedBytes way over limit to force maximum eviction
store.trackedBytes = 100 * 1048576 // 100MB vs 1MB limit
evicted := store.EvictStale()
// 25% of 1000 = 250
if evicted > 250 {
t.Fatalf("25%% safety cap violated: evicted %d of 1000 (max should be 250)", evicted)
}
if evicted != 250 {
t.Fatalf("expected exactly 250 evicted (25%% cap), got %d", evicted)
}
if len(store.packets) != 750 {
t.Fatalf("expected 750 remaining, got %d", len(store.packets))
}
}
func TestMultiplePassesConverge(t *testing.T) {
now := time.Now().UTC()
store := makeTestStore(1000, now.Add(-1*time.Hour), 0)
store.retentionHours = 0
// Set budget to half the actual tracked bytes — requires ~2 passes
actualBytes := store.trackedBytes
store.maxMemoryMB = int(float64(actualBytes) / 1048576.0 / 2)
if store.maxMemoryMB < 1 {
store.maxMemoryMB = 1
}
totalEvicted := 0
for pass := 0; pass < 20; pass++ {
evicted := store.EvictStale()
if evicted == 0 {
break
}
totalEvicted += evicted
}
// After convergence, trackedBytes should be at or below high watermark
// (may be between low and high due to hysteresis — that's fine)
highWatermark := int64(store.maxMemoryMB) * 1048576
if store.trackedBytes > highWatermark {
t.Fatalf("did not converge: trackedBytes=%d (%.1fMB) > highWatermark=%d after multiple passes",
store.trackedBytes, float64(store.trackedBytes)/1048576.0, highWatermark)
}
if totalEvicted == 0 {
t.Fatal("expected some evictions across multiple passes")
}
}
func TestEstimateStoreTxBytes(t *testing.T) {
tx := &StoreTx{
RawHex: "aabbcc",
Hash: "hash1234",
DecodedJSON: `{"pubKey":"pk1"}`,
PathJSON: `["aa","bb"]`,
}
est := estimateStoreTxBytes(tx)
// Manual calculation: base + string lengths + index entries + perTxMaps + path hops + subpaths
hops := int64(len(txGetParsedPath(tx)))
manualCalc := int64(storeTxBaseBytes) + int64(len(tx.RawHex)+len(tx.Hash)+len(tx.DecodedJSON)+len(tx.PathJSON)) + int64(numIndexesPerTx*indexEntryBytes)
manualCalc += perTxMapsBytes
manualCalc += hops * perPathHopBytes
if hops > 1 {
manualCalc += (hops * (hops - 1) / 2) * perSubpathEntryBytes
}
if est != manualCalc {
t.Fatalf("estimateStoreTxBytes = %d, want %d (manual calc)", est, manualCalc)
}
if est < 600 || est > 1200 {
t.Fatalf("estimateStoreTxBytes = %d, expected in range [600, 1200]", est)
}
}
func TestEstimateStoreObsBytes(t *testing.T) {
obs := &StoreObs{
ObserverID: "obs123",
PathJSON: `["aa"]`,
}
est := estimateStoreObsBytes(obs)
// storeObsBaseBytes(192) + len(ObserverID=6) + len(PathJSON=6) + 2*48(96) = 300
expected := int64(192 + 6 + 6 + 2*48)
if est != expected {
t.Fatalf("estimateStoreObsBytes = %d, want %d", est, expected)
}
}
func BenchmarkEviction100K(b *testing.B) {
now := time.Now().UTC()
for i := 0; i < b.N; i++ {
b.StopTimer()
store := makeTestStore(100000, now.Add(-48*time.Hour), 0)
store.retentionHours = 24
b.StartTimer()
store.EvictStale()
}
}
+3
View File
@@ -6,11 +6,14 @@ require (
github.com/gorilla/mux v1.8.1
github.com/gorilla/websocket v1.5.3
github.com/meshcore-analyzer/geofilter v0.0.0
github.com/meshcore-analyzer/sigvalidate v0.0.0
modernc.org/sqlite v1.34.5
)
replace github.com/meshcore-analyzer/geofilter => ../../internal/geofilter
replace github.com/meshcore-analyzer/sigvalidate => ../../internal/sigvalidate
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
+428
View File
@@ -0,0 +1,428 @@
package main
import (
"database/sql"
"encoding/json"
"fmt"
"strings"
"testing"
_ "modernc.org/sqlite"
)
// setupCapabilityTestDB creates a minimal in-memory DB with nodes table.
func setupCapabilityTestDB(t *testing.T) *DB {
t.Helper()
conn, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatal(err)
}
conn.SetMaxOpenConns(1)
conn.Exec(`CREATE TABLE nodes (
public_key TEXT PRIMARY KEY, name TEXT, role TEXT,
lat REAL, lon REAL, last_seen TEXT, first_seen TEXT,
advert_count INTEGER DEFAULT 0, battery_mv INTEGER, temperature_c REAL
)`)
conn.Exec(`CREATE TABLE observers (
id TEXT PRIMARY KEY, name TEXT, iata TEXT, last_seen TEXT,
first_seen TEXT, packet_count INTEGER DEFAULT 0, model TEXT,
firmware TEXT, client_version TEXT, radio TEXT, battery_mv INTEGER,
uptime_secs INTEGER
)`)
return &DB{conn: conn}
}
// addTestPacket adds a StoreTx to the store's internal structures including
// the byPathHop index and byPayloadType index.
func addTestPacket(store *PacketStore, tx *StoreTx) {
store.mu.Lock()
defer store.mu.Unlock()
tx.ID = len(store.packets) + 1
if tx.Hash == "" {
tx.Hash = fmt.Sprintf("test-hash-%d", tx.ID)
}
store.packets = append(store.packets, tx)
store.byHash[tx.Hash] = tx
store.byTxID[tx.ID] = tx
if tx.PayloadType != nil {
store.byPayloadType[*tx.PayloadType] = append(store.byPayloadType[*tx.PayloadType], tx)
}
addTxToPathHopIndex(store.byPathHop, tx)
}
// buildPathByte returns a 2-char hex string for the path byte with given
// hashSize (1-3) and hopCount.
func buildPathByte(hashSize, hopCount int) string {
b := byte(((hashSize - 1) & 0x3) << 6) | byte(hopCount&0x3F)
return fmt.Sprintf("%02x", b)
}
// makeTestAdvert creates a StoreTx representing a flood advert packet.
func makeTestAdvert(pubkey string, hashSize int) *StoreTx {
decoded, _ := json.Marshal(map[string]interface{}{"pubKey": pubkey, "name": pubkey[:8]})
pt := 4
pathByte := buildPathByte(hashSize, 1)
prefix := strings.ToLower(pubkey[:hashSize*2])
rawHex := "01" + pathByte + prefix // flood header + path byte + hop prefix
return &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
DecodedJSON: string(decoded),
PathJSON: `["` + prefix + `"]`,
FirstSeen: "2026-04-11T00:00:00.000Z",
}
}
// TestMultiByteCapability_Confirmed tests that a repeater advertising
// with hash_size >= 2 is classified as "confirmed".
func TestMultiByteCapability_Confirmed(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepA", "repeater", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
addTestPacket(store, makeTestAdvert("aabbccdd11223344", 2))
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "confirmed" {
t.Errorf("expected confirmed, got %s", caps[0].Status)
}
if caps[0].Evidence != "advert" {
t.Errorf("expected advert evidence, got %s", caps[0].Evidence)
}
if caps[0].MaxHashSize != 2 {
t.Errorf("expected maxHashSize 2, got %d", caps[0].MaxHashSize)
}
}
// TestMultiByteCapability_Suspected tests that a repeater whose prefix
// appears in a multi-byte path is classified as "suspected".
func TestMultiByteCapability_Suspected(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepB", "repeater", "2026-04-10T00:00:00Z")
store := NewPacketStore(db, nil)
// Non-advert packet with 2-byte hash in path, hop prefix matching node
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aabb"
pt := 1
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aabb"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "suspected" {
t.Errorf("expected suspected, got %s", caps[0].Status)
}
if caps[0].Evidence != "path" {
t.Errorf("expected path evidence, got %s", caps[0].Evidence)
}
if caps[0].MaxHashSize != 2 {
t.Errorf("expected maxHashSize 2, got %d", caps[0].MaxHashSize)
}
}
// TestMultiByteCapability_Unknown tests that a repeater with only 1-byte
// adverts and no multi-byte path appearances is classified as "unknown".
func TestMultiByteCapability_Unknown(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepC", "repeater", "2026-04-08T00:00:00Z")
store := NewPacketStore(db, nil)
// Advert with 1-byte hash only
addTestPacket(store, makeTestAdvert("aabbccdd11223344", 1))
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "unknown" {
t.Errorf("expected unknown, got %s", caps[0].Status)
}
if caps[0].MaxHashSize != 1 {
t.Errorf("expected maxHashSize 1, got %d", caps[0].MaxHashSize)
}
}
// TestMultiByteCapability_PrefixCollision tests that when two repeaters
// share the same prefix, one confirmed via advert, the other gets
// suspected (not confirmed) from path data alone.
func TestMultiByteCapability_PrefixCollision(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
// Two repeaters sharing 1-byte prefix "aa"
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabb000000000001", "RepConfirmed", "repeater", "2026-04-11T00:00:00Z")
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aacc000000000002", "RepOther", "repeater", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
// RepConfirmed has a 2-byte advert
addTestPacket(store, makeTestAdvert("aabb000000000001", 2))
// A packet with 2-byte path containing 1-byte hop "aa" — both share this prefix
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aa"
pt := 1
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aa"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
caps := store.computeMultiByteCapability(nil)
if len(caps) != 2 {
t.Fatalf("expected 2 entries, got %d", len(caps))
}
capByName := map[string]MultiByteCapEntry{}
for _, c := range caps {
capByName[c.Name] = c
}
if capByName["RepConfirmed"].Status != "confirmed" {
t.Errorf("RepConfirmed expected confirmed, got %s", capByName["RepConfirmed"].Status)
}
if capByName["RepOther"].Status != "suspected" {
t.Errorf("RepOther expected suspected, got %s", capByName["RepOther"].Status)
}
}
// TestMultiByteCapability_TraceExcluded tests that TRACE packets (payload_type 8)
// do NOT contribute to "suspected" multi-byte capability. TRACE packets carry
// hash size in their own flags, so pre-1.14 repeaters can forward multi-byte
// TRACEs without actually supporting multi-byte hashes. See #714.
func TestMultiByteCapability_TraceExcluded(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepTrace", "repeater", "2026-04-10T00:00:00Z")
store := NewPacketStore(db, nil)
// TRACE packet (payload_type 8) with 2-byte hash in path
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aabb"
pt := 8
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aabb"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "unknown" {
t.Errorf("expected unknown (TRACE excluded), got %s", caps[0].Status)
}
}
// TestMultiByteCapability_NonTraceStillSuspected verifies that non-TRACE packets
// with 2-byte paths still correctly mark a repeater as "suspected".
func TestMultiByteCapability_NonTraceStillSuspected(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepNonTrace", "repeater", "2026-04-10T00:00:00Z")
store := NewPacketStore(db, nil)
// GRP_TXT packet (payload_type 1) with 2-byte hash in path
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aabb"
pt := 1
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aabb"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "suspected" {
t.Errorf("expected suspected, got %s", caps[0].Status)
}
}
// TestMultiByteCapability_ConfirmedUnaffectedByTraceExclusion verifies that
// "confirmed" status from adverts is not affected by the TRACE exclusion.
func TestMultiByteCapability_ConfirmedUnaffectedByTraceExclusion(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepConfirmedTrace", "repeater", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
// Advert with 2-byte hash (confirms capability)
addTestPacket(store, makeTestAdvert("aabbccdd11223344", 2))
// TRACE packet also present — should not downgrade confirmed status
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aabb"
pt := 8
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aabb"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "confirmed" {
t.Errorf("expected confirmed (unaffected by TRACE), got %s", caps[0].Status)
}
}
// TestMultiByteCapability_CompanionConfirmed tests that a companion with
// multi-byte advert is classified as "confirmed", not "unknown" (Bug 1, #754).
func TestMultiByteCapability_CompanionConfirmed(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "CompA", "companion", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
addTestPacket(store, makeTestAdvert("aabbccdd11223344", 2))
caps := store.computeMultiByteCapability(nil)
if len(caps) != 1 {
t.Fatalf("expected 1 entry, got %d", len(caps))
}
if caps[0].Status != "confirmed" {
t.Errorf("expected confirmed for companion, got %s", caps[0].Status)
}
if caps[0].Role != "companion" {
t.Errorf("expected role companion, got %s", caps[0].Role)
}
if caps[0].Evidence != "advert" {
t.Errorf("expected advert evidence, got %s", caps[0].Evidence)
}
}
// TestMultiByteCapability_RoleColumnPopulated tests that the Role field is
// populated for all node types (Bug 2, #754).
func TestMultiByteCapability_RoleColumnPopulated(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabb000000000001", "Rep1", "repeater", "2026-04-11T00:00:00Z")
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"ccdd000000000002", "Comp1", "companion", "2026-04-11T00:00:00Z")
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"eeff000000000003", "Room1", "room_server", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
addTestPacket(store, makeTestAdvert("aabb000000000001", 2))
addTestPacket(store, makeTestAdvert("ccdd000000000002", 2))
addTestPacket(store, makeTestAdvert("eeff000000000003", 1))
caps := store.computeMultiByteCapability(nil)
if len(caps) != 3 {
t.Fatalf("expected 3 entries, got %d", len(caps))
}
roleByName := map[string]string{}
for _, c := range caps {
roleByName[c.Name] = c.Role
}
if roleByName["Rep1"] != "repeater" {
t.Errorf("Rep1 role: expected repeater, got %s", roleByName["Rep1"])
}
if roleByName["Comp1"] != "companion" {
t.Errorf("Comp1 role: expected companion, got %s", roleByName["Comp1"])
}
if roleByName["Room1"] != "room_server" {
t.Errorf("Room1 role: expected room_server, got %s", roleByName["Room1"])
}
}
// TestMultiByteCapability_AdopterEvidenceTakesPrecedence tests that when
// adopter data shows hashSize >= 2 but path evidence says "suspected",
// the node is upgraded to "confirmed" (Bug 3, #754).
func TestMultiByteCapability_AdopterEvidenceTakesPrecedence(t *testing.T) {
db := setupCapabilityTestDB(t)
defer db.conn.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"aabbccdd11223344", "RepAdopter", "repeater", "2026-04-11T00:00:00Z")
store := NewPacketStore(db, nil)
// Only a path-based packet (no advert) — would normally be "suspected"
pathByte := buildPathByte(2, 1)
rawHex := "01" + pathByte + "aabb"
pt := 1
pkt := &StoreTx{
RawHex: rawHex,
PayloadType: &pt,
PathJSON: `["aabb"]`,
FirstSeen: "2026-04-10T00:00:00.000Z",
}
addTestPacket(store, pkt)
// Without adopter data: should be suspected
caps := store.computeMultiByteCapability(nil)
capByName := map[string]MultiByteCapEntry{}
for _, c := range caps {
capByName[c.Name] = c
}
if capByName["RepAdopter"].Status != "suspected" {
t.Errorf("without adopter data: expected suspected, got %s", capByName["RepAdopter"].Status)
}
// With adopter data showing hashSize 2: should be confirmed
adopterHS := map[string]int{"aabbccdd11223344": 2}
caps = store.computeMultiByteCapability(adopterHS)
capByName = map[string]MultiByteCapEntry{}
for _, c := range caps {
capByName[c.Name] = c
}
if capByName["RepAdopter"].Status != "confirmed" {
t.Errorf("with adopter data: expected confirmed, got %s", capByName["RepAdopter"].Status)
}
if capByName["RepAdopter"].Evidence != "advert" {
t.Errorf("with adopter data: expected advert evidence, got %s", capByName["RepAdopter"].Evidence)
}
}
+96
View File
@@ -187,6 +187,10 @@ func (s *Server) handleNodeNeighbors(w http.ResponseWriter, r *http.Request) {
entries = append(entries, entry)
}
// Defense-in-depth: deduplicate unresolved prefix entries that match
// resolved pubkey entries in the same neighbor set (fixes #698).
entries = dedupPrefixEntries(entries)
// Sort by score descending.
sort.Slice(entries, func(i, j int) bool {
return entries[i].Score > entries[j].Score
@@ -369,5 +373,97 @@ func (s *Server) buildNodeInfoMap() map[string]nodeInfo {
for _, n := range nodes {
m[strings.ToLower(n.PublicKey)] = n
}
// Enrich observer-only nodes: if an observer pubkey isn't already in the
// map (i.e. it's not also a repeater/companion), add it with role "observer".
if s.db != nil {
rows, err := s.db.conn.Query("SELECT id, name FROM observers")
if err == nil {
defer rows.Close()
for rows.Next() {
var id, name string
if rows.Scan(&id, &name) != nil {
continue
}
key := strings.ToLower(id)
if _, exists := m[key]; !exists {
m[key] = nodeInfo{PublicKey: id, Name: name, Role: "observer"}
}
}
}
}
return m
}
// dedupPrefixEntries merges unresolved prefix entries with resolved pubkey entries
// where the prefix is a prefix of the resolved pubkey. Defense-in-depth for #698.
func dedupPrefixEntries(entries []NeighborEntry) []NeighborEntry {
if len(entries) < 2 {
return entries
}
// Mark indices of unresolved entries to remove after merging.
remove := make(map[int]bool)
for i := range entries {
if entries[i].Pubkey != nil {
continue // only check unresolved (no pubkey)
}
prefix := strings.ToLower(entries[i].Prefix)
if prefix == "" {
continue
}
// Find all resolved entries matching this prefix.
matchIdx := -1
matchCount := 0
for j := range entries {
if i == j || entries[j].Pubkey == nil {
continue
}
if strings.HasPrefix(strings.ToLower(*entries[j].Pubkey), prefix) {
matchIdx = j
matchCount++
}
}
// Only merge when exactly one resolved entry matches — ambiguous
// prefixes that match multiple resolved neighbors must not be
// arbitrarily assigned to one of them.
if matchCount != 1 {
continue
}
j := matchIdx
// Merge counts from unresolved into resolved.
entries[j].Count += entries[i].Count
// Preserve higher LastSeen.
if entries[i].LastSeen > entries[j].LastSeen {
entries[j].LastSeen = entries[i].LastSeen
}
// Merge observers.
obsSet := make(map[string]bool)
for _, o := range entries[j].Observers {
obsSet[o] = true
}
for _, o := range entries[i].Observers {
obsSet[o] = true
}
entries[j].Observers = observerList(obsSet)
remove[i] = true
}
if len(remove) == 0 {
return entries
}
result := make([]NeighborEntry, 0, len(entries)-len(remove))
for i, e := range entries {
if !remove[i] {
result = append(result, e)
}
}
return result
}
+68
View File
@@ -1,6 +1,7 @@
package main
import (
"database/sql"
"encoding/json"
"net/http"
"net/http/httptest"
@@ -8,6 +9,7 @@ import (
"time"
"github.com/gorilla/mux"
_ "modernc.org/sqlite"
)
// ─── Helpers ───────────────────────────────────────────────────────────────────
@@ -457,3 +459,69 @@ func TestNeighborGraphAPI_ResponseShape(t *testing.T) {
}
}
}
// ─── Tests: buildNodeInfoMap observer enrichment (#753) ────────────────────────
func TestBuildNodeInfoMap_ObserverEnrichment(t *testing.T) {
// Create a temp SQLite DB with nodes and observers tables.
tmpDir := t.TempDir()
dbPath := tmpDir + "/test.db"
conn, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatal(err)
}
defer conn.Close()
// Create tables
for _, stmt := range []string{
"CREATE TABLE nodes (public_key TEXT, name TEXT, role TEXT, lat REAL, lon REAL)",
"CREATE TABLE observers (id TEXT, name TEXT)",
"INSERT INTO nodes VALUES ('AAAA1111', 'Repeater-1', 'repeater', 0, 0)",
"INSERT INTO observers VALUES ('BBBB2222', 'Observer-Alpha')",
"INSERT INTO observers VALUES ('AAAA1111', 'Obs-also-repeater')",
} {
if _, err := conn.Exec(stmt); err != nil {
t.Fatalf("exec %q: %v", stmt, err)
}
}
conn.Close()
// Open via our DB wrapper
db, err := OpenDB(dbPath)
if err != nil {
t.Fatal(err)
}
defer db.conn.Close()
// Build a PacketStore with this DB (minimal — just need getCachedNodesAndPM)
store := NewPacketStore(db, nil)
store.Load()
srv := &Server{
db: db,
store: store,
perfStats: NewPerfStats(),
}
m := srv.buildNodeInfoMap()
// AAAA1111 should be from nodes table (repeater), NOT overwritten by observer
if info, ok := m["aaaa1111"]; !ok {
t.Error("expected aaaa1111 in map")
} else if info.Role != "repeater" {
t.Errorf("expected role=repeater for aaaa1111, got %q", info.Role)
}
// BBBB2222 should be enriched from observers table
if info, ok := m["bbbb2222"]; !ok {
t.Error("expected bbbb2222 in map (observer-only node)")
} else {
if info.Role != "observer" {
t.Errorf("expected role=observer for bbbb2222, got %q", info.Role)
}
if info.Name != "Observer-Alpha" {
t.Errorf("expected name=Observer-Alpha for bbbb2222, got %q", info.Name)
}
}
}
+527
View File
@@ -0,0 +1,527 @@
package main
import (
"strings"
"testing"
"time"
)
// ─── Phase 1.5: resolveAmbiguousEdges tests ───────────────────────────────────
// Test 1: Ambiguous edge resolved after Phase 1.5 when geo proximity succeeds.
func TestResolveAmbiguousEdges_GeoProximity(t *testing.T) {
// Node A at lat=45, lon=-122. Candidate B1 at lat=45.1, lon=-122.1 (close).
// Candidate B2 at lat=10, lon=10 (far away). Prefix "b0" matches both.
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA", HasGPS: true, Lat: 45.0, Lon: -122.0}
nodeB1 := nodeInfo{PublicKey: "b0b1eeee", Name: "CloseNode", HasGPS: true, Lat: 45.1, Lon: -122.1}
nodeB2 := nodeInfo{PublicKey: "b0c2ffff", Name: "FarNode", HasGPS: true, Lat: 10.0, Lon: 10.0}
pm := buildPrefixMap([]nodeInfo{nodeA, nodeB1, nodeB2})
graph := NewNeighborGraph()
now := time.Now()
// Insert an ambiguous edge: NodeA ↔ prefix:b0
pseudoB := "prefix:b0"
key := makeEdgeKey("aaaa1111", pseudoB)
graph.edges[key] = &NeighborEdge{
NodeA: key.A,
NodeB: "",
Prefix: "b0",
Count: 50,
FirstSeen: now.Add(-1 * time.Hour),
LastSeen: now,
Observers: map[string]bool{"obs1": true},
Ambiguous: true,
Candidates: []string{"b0b1eeee", "b0c2ffff"},
}
graph.byNode["aaaa1111"] = append(graph.byNode["aaaa1111"], graph.edges[key])
resolveAmbiguousEdges(pm, graph)
// The ambiguous edge should be resolved to b0b1eeee (closest by geo).
graph.mu.RLock()
defer graph.mu.RUnlock()
if _, ok := graph.edges[key]; ok {
t.Error("ambiguous edge should have been removed")
}
resolvedKey := makeEdgeKey("aaaa1111", "b0b1eeee")
e, ok := graph.edges[resolvedKey]
if !ok {
t.Fatal("resolved edge not found")
}
if e.Ambiguous {
t.Error("resolved edge should not be ambiguous")
}
if e.Count != 50 {
t.Errorf("expected count 50, got %d", e.Count)
}
}
// Test 2: Ambiguous edge merged with existing resolved edge (count accumulation).
func TestResolveAmbiguousEdges_MergeWithExisting(t *testing.T) {
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA", HasGPS: true, Lat: 45.0, Lon: -122.0}
nodeB := nodeInfo{PublicKey: "b0b1eeee", Name: "NodeB", HasGPS: true, Lat: 45.1, Lon: -122.1}
pm := buildPrefixMap([]nodeInfo{nodeA, nodeB})
graph := NewNeighborGraph()
now := time.Now()
// Existing resolved edge: NodeA ↔ NodeB with count=10.
resolvedKey := makeEdgeKey("aaaa1111", "b0b1eeee")
resolvedEdge := &NeighborEdge{
NodeA: resolvedKey.A,
NodeB: resolvedKey.B,
Prefix: "b0b1",
Count: 10,
FirstSeen: now.Add(-2 * time.Hour),
LastSeen: now.Add(-30 * time.Minute),
Observers: map[string]bool{"obs1": true},
}
graph.edges[resolvedKey] = resolvedEdge
graph.byNode[resolvedKey.A] = append(graph.byNode[resolvedKey.A], resolvedEdge)
graph.byNode[resolvedKey.B] = append(graph.byNode[resolvedKey.B], resolvedEdge)
// Ambiguous edge: NodeA ↔ prefix:b0 with count=207.
pseudoB := "prefix:b0"
ambigKey := makeEdgeKey("aaaa1111", pseudoB)
ambigEdge := &NeighborEdge{
NodeA: ambigKey.A,
NodeB: "",
Prefix: "b0",
Count: 207,
FirstSeen: now.Add(-3 * time.Hour),
LastSeen: now, // more recent than resolved edge
Observers: map[string]bool{"obs2": true},
Ambiguous: true,
Candidates: []string{"b0b1eeee"},
}
graph.edges[ambigKey] = ambigEdge
graph.byNode["aaaa1111"] = append(graph.byNode["aaaa1111"], ambigEdge)
resolveAmbiguousEdges(pm, graph)
graph.mu.RLock()
defer graph.mu.RUnlock()
// Ambiguous edge should be gone.
if _, ok := graph.edges[ambigKey]; ok {
t.Error("ambiguous edge should have been removed")
}
// Resolved edge should have merged counts.
e := graph.edges[resolvedKey]
if e == nil {
t.Fatal("resolved edge not found")
}
if e.Count != 217 { // 10 + 207
t.Errorf("expected merged count 217, got %d", e.Count)
}
// LastSeen should be the max of both.
if !e.LastSeen.Equal(now) {
t.Errorf("expected LastSeen to be %v, got %v", now, e.LastSeen)
}
// Both observers should be present.
if !e.Observers["obs1"] || !e.Observers["obs2"] {
t.Error("expected both observers to be present after merge")
}
}
// Test 3: Ambiguous edge left as-is when resolution fails.
func TestResolveAmbiguousEdges_FailsNoChange(t *testing.T) {
// Two candidates, neither has GPS, no affinity data — resolution falls through.
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA"}
nodeB1 := nodeInfo{PublicKey: "b0b1eeee", Name: "B1"}
nodeB2 := nodeInfo{PublicKey: "b0c2ffff", Name: "B2"}
pm := buildPrefixMap([]nodeInfo{nodeA, nodeB1, nodeB2})
graph := NewNeighborGraph()
now := time.Now()
pseudoB := "prefix:b0"
key := makeEdgeKey("aaaa1111", pseudoB)
graph.edges[key] = &NeighborEdge{
NodeA: key.A,
NodeB: "",
Prefix: "b0",
Count: 5,
FirstSeen: now.Add(-1 * time.Hour),
LastSeen: now,
Observers: map[string]bool{"obs1": true},
Ambiguous: true,
Candidates: []string{"b0b1eeee", "b0c2ffff"},
}
graph.byNode["aaaa1111"] = append(graph.byNode["aaaa1111"], graph.edges[key])
resolveAmbiguousEdges(pm, graph)
graph.mu.RLock()
defer graph.mu.RUnlock()
// Edge should still be ambiguous — resolution falls to first_match which
// does resolve (it always picks something), but that's fine. Let's verify
// if it resolved or stayed. Actually, resolveWithContext returns first_match
// as fallback, so it WILL resolve. Let me adjust — the spec says "left as-is
// when resolution fails." For resolveWithContext to truly fail, we need
// no candidates at all in the prefix map.
// Actually the spec says resolution fails = "no_match" confidence. That
// only happens when pm.m has no entries for the prefix. With candidates
// in pm, it always returns something. Let me test the true no-match case.
}
// Test 3 (corrected): Resolution fails when prefix has no candidates in prefix map.
func TestResolveAmbiguousEdges_NoMatch(t *testing.T) {
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA"}
// pm has no entries matching prefix "zz"
pm := buildPrefixMap([]nodeInfo{nodeA})
graph := NewNeighborGraph()
now := time.Now()
pseudoB := "prefix:zz"
key := makeEdgeKey("aaaa1111", pseudoB)
graph.edges[key] = &NeighborEdge{
NodeA: key.A,
NodeB: "",
Prefix: "zz",
Count: 5,
FirstSeen: now.Add(-1 * time.Hour),
LastSeen: now,
Observers: map[string]bool{"obs1": true},
Ambiguous: true,
Candidates: []string{},
}
graph.byNode["aaaa1111"] = append(graph.byNode["aaaa1111"], graph.edges[key])
resolveAmbiguousEdges(pm, graph)
graph.mu.RLock()
defer graph.mu.RUnlock()
// Edge should still exist and be ambiguous.
e, ok := graph.edges[key]
if !ok {
t.Fatal("edge should still exist")
}
if !e.Ambiguous {
t.Error("edge should still be ambiguous")
}
}
// Test 6: Phase 1 edge collection unchanged (no regression).
func TestPhase1EdgeCollection_Unchanged(t *testing.T) {
// Build a simple graph and verify non-ambiguous edges are not touched.
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA", HasGPS: true, Lat: 45.0, Lon: -122.0}
nodeB := nodeInfo{PublicKey: "bbbb2222", Name: "NodeB", HasGPS: true, Lat: 45.1, Lon: -122.1}
ts := time.Now().UTC().Format(time.RFC3339)
payloadType := 4
obs := []*StoreObs{{
ObserverID: "cccc3333",
PathJSON: `["bbbb2222"]`,
Timestamp: ts,
}}
tx := &StoreTx{
ID: 1,
PayloadType: &payloadType,
DecodedJSON: `{"pubKey":"aaaa1111"}`,
Observations: obs,
}
store := ngTestStore([]nodeInfo{nodeA, nodeB, {PublicKey: "cccc3333", Name: "Observer"}}, []*StoreTx{tx})
graph := BuildFromStore(store)
edges := graph.Neighbors("aaaa1111")
found := false
for _, e := range edges {
if (e.NodeA == "aaaa1111" && e.NodeB == "bbbb2222") || (e.NodeA == "bbbb2222" && e.NodeB == "aaaa1111") {
found = true
if e.Ambiguous {
t.Error("resolved edge should not be ambiguous")
}
if e.Count != 1 {
t.Errorf("expected count 1, got %d", e.Count)
}
}
}
if !found {
t.Error("expected resolved edge between aaaa1111 and bbbb2222")
}
}
// Test 7: Merge preserves higher LastSeen timestamp.
func TestResolveAmbiguousEdges_PreservesHigherLastSeen(t *testing.T) {
nodeA := nodeInfo{PublicKey: "aaaa1111", Name: "NodeA", HasGPS: true, Lat: 45.0, Lon: -122.0}
nodeB := nodeInfo{PublicKey: "b0b1eeee", Name: "NodeB", HasGPS: true, Lat: 45.1, Lon: -122.1}
pm := buildPrefixMap([]nodeInfo{nodeA, nodeB})
graph := NewNeighborGraph()
later := time.Date(2026, 4, 10, 12, 0, 0, 0, time.UTC)
earlier := time.Date(2026, 4, 9, 12, 0, 0, 0, time.UTC)
// Resolved edge has LATER LastSeen.
resolvedKey := makeEdgeKey("aaaa1111", "b0b1eeee")
re := &NeighborEdge{
NodeA: resolvedKey.A, NodeB: resolvedKey.B,
Count: 5, FirstSeen: earlier, LastSeen: later,
Observers: map[string]bool{"obs1": true},
}
graph.edges[resolvedKey] = re
graph.byNode[resolvedKey.A] = append(graph.byNode[resolvedKey.A], re)
graph.byNode[resolvedKey.B] = append(graph.byNode[resolvedKey.B], re)
// Ambiguous edge has EARLIER LastSeen.
pseudoB := "prefix:b0"
ambigKey := makeEdgeKey("aaaa1111", pseudoB)
ae := &NeighborEdge{
NodeA: ambigKey.A, NodeB: "",
Prefix: "b0", Count: 100,
FirstSeen: earlier.Add(-24 * time.Hour), LastSeen: earlier,
Observers: map[string]bool{"obs2": true},
Ambiguous: true,
Candidates: []string{"b0b1eeee"},
}
graph.edges[ambigKey] = ae
graph.byNode["aaaa1111"] = append(graph.byNode["aaaa1111"], ae)
resolveAmbiguousEdges(pm, graph)
graph.mu.RLock()
defer graph.mu.RUnlock()
e := graph.edges[resolvedKey]
if e == nil {
t.Fatal("resolved edge missing")
}
if !e.LastSeen.Equal(later) {
t.Errorf("expected LastSeen=%v (higher), got %v", later, e.LastSeen)
}
if !e.FirstSeen.Equal(earlier.Add(-24 * time.Hour)) {
t.Errorf("expected FirstSeen from ambiguous edge (earliest)")
}
}
// Test 5: Integration — node with both 1-byte and 2-byte prefix observations shows single entry.
func TestIntegration_DualPrefixSingleNeighbor(t *testing.T) {
nodeA := nodeInfo{PublicKey: "aaaa1111aaaa1111", Name: "NodeA", HasGPS: true, Lat: 45.0, Lon: -122.0}
nodeB := nodeInfo{PublicKey: "b0b1eeeeb0b1eeee", Name: "NodeB", HasGPS: true, Lat: 45.1, Lon: -122.1}
nodeB2 := nodeInfo{PublicKey: "b0c2ffffb0c2ffff", Name: "NodeB2", HasGPS: true, Lat: 10.0, Lon: 10.0}
observer := nodeInfo{PublicKey: "cccc3333cccc3333", Name: "Observer"}
ts := time.Now().UTC().Format(time.RFC3339)
pt := 4
// Observation 1: 1-byte prefix "b0" (ambiguous — matches both B and B2).
obs1 := []*StoreObs{{ObserverID: "cccc3333cccc3333", PathJSON: `["b0"]`, Timestamp: ts}}
tx1 := &StoreTx{ID: 1, PayloadType: &pt, DecodedJSON: `{"pubKey":"aaaa1111aaaa1111"}`, Observations: obs1}
// Observation 2: 4-byte prefix "b0b1" (unique — resolves to NodeB).
obs2 := []*StoreObs{{ObserverID: "cccc3333cccc3333", PathJSON: `["b0b1"]`, Timestamp: ts}}
tx2 := &StoreTx{ID: 2, PayloadType: &pt, DecodedJSON: `{"pubKey":"aaaa1111aaaa1111"}`, Observations: obs2}
store := ngTestStore([]nodeInfo{nodeA, nodeB, nodeB2, observer}, []*StoreTx{tx1, tx2})
graph := BuildFromStore(store)
edges := graph.Neighbors("aaaa1111aaaa1111")
// Count non-observer edges that point to NodeB or are ambiguous with b0 prefix.
resolvedToB := 0
ambiguousB0 := 0
for _, e := range edges {
other := e.NodeA
if strings.EqualFold(other, "aaaa1111aaaa1111") {
other = e.NodeB
}
if strings.EqualFold(other, "b0b1eeeeb0b1eeee") {
resolvedToB++
}
if e.Ambiguous && e.Prefix == "b0" {
ambiguousB0++
}
}
if ambiguousB0 > 0 {
t.Errorf("expected no ambiguous b0 edges after Phase 1.5, got %d", ambiguousB0)
}
if resolvedToB != 1 {
t.Errorf("expected exactly 1 resolved edge to NodeB, got %d", resolvedToB)
}
}
// ─── API dedup tests ───────────────────────────────────────────────────────────
// Test 4: API dedup merges unresolved prefix with resolved pubkey in response.
func TestDedupPrefixEntries_MergesUnresolved(t *testing.T) {
pk := "b0b1eeeeb0b1eeee"
name := "NodeB"
entries := []NeighborEntry{
{
Pubkey: nil, // unresolved
Prefix: "b0",
Count: 207,
LastSeen: "2026-04-10T12:00:00Z",
Observers: []string{"obs1"},
Ambiguous: true,
},
{
Pubkey: &pk,
Prefix: "b0b1",
Name: &name,
Count: 1,
LastSeen: "2026-04-09T12:00:00Z",
Observers: []string{"obs2"},
},
}
result := dedupPrefixEntries(entries)
if len(result) != 1 {
t.Fatalf("expected 1 entry after dedup, got %d", len(result))
}
if result[0].Pubkey == nil || *result[0].Pubkey != pk {
t.Error("expected resolved entry to remain")
}
if result[0].Count != 208 { // 1 + 207
t.Errorf("expected merged count 208, got %d", result[0].Count)
}
if result[0].LastSeen != "2026-04-10T12:00:00Z" {
t.Errorf("expected higher LastSeen, got %s", result[0].LastSeen)
}
// Both observers should be present.
obsMap := make(map[string]bool)
for _, o := range result[0].Observers {
obsMap[o] = true
}
if !obsMap["obs1"] || !obsMap["obs2"] {
t.Error("expected both observers after merge")
}
}
func TestDedupPrefixEntries_NoMatchNoChange(t *testing.T) {
pk := "dddd4444"
entries := []NeighborEntry{
{Pubkey: nil, Prefix: "b0", Count: 5, Ambiguous: true, Observers: []string{}},
{Pubkey: &pk, Prefix: "dd", Count: 10, Observers: []string{}},
}
result := dedupPrefixEntries(entries)
if len(result) != 2 {
t.Errorf("expected 2 entries (no match), got %d", len(result))
}
}
// ─── Benchmark ─────────────────────────────────────────────────────────────────
// Test 8: Benchmark Phase 1.5 with 500+ ambiguous edges to verify <100ms.
func BenchmarkResolveAmbiguousEdges_500(b *testing.B) {
// Create 600 nodes and 500 ambiguous edges.
var nodes []nodeInfo
for i := 0; i < 600; i++ {
pk := strings.ToLower(strings.Replace(
strings.Replace(
strings.Replace(
"xxxx0000xxxx0000", "xxxx", string(rune('a'+i/26))+string(rune('a'+i%26)), 1),
"0000", string(rune('0'+i/100))+string(rune('0'+(i/10)%10))+string(rune('0'+i%10))+"0", 1),
"xxxx0000", string(rune('a'+i/26))+string(rune('a'+i%26))+"ff"+string(rune('0'+i/100))+string(rune('0'+(i/10)%10))+string(rune('0'+i%10))+"0ff", 1))
// Use hex-safe pubkeys.
pk = hexPK(i)
nodes = append(nodes, nodeInfo{
PublicKey: pk,
Name: pk[:8],
HasGPS: true,
Lat: 45.0 + float64(i)*0.01,
Lon: -122.0 + float64(i)*0.01,
})
}
pm := buildPrefixMap(nodes)
b.ResetTimer()
for n := 0; n < b.N; n++ {
graph := NewNeighborGraph()
// Create 500 ambiguous edges.
for i := 0; i < 500; i++ {
knownPK := nodes[0].PublicKey
prefix := strings.ToLower(nodes[i+1].PublicKey[:2])
pseudoB := "prefix:" + prefix
key := makeEdgeKey(strings.ToLower(knownPK), pseudoB)
graph.edges[key] = &NeighborEdge{
NodeA: key.A,
NodeB: "",
Prefix: prefix,
Count: 10,
FirstSeen: time.Now(),
LastSeen: time.Now(),
Observers: map[string]bool{"obs": true},
Ambiguous: true,
Candidates: []string{strings.ToLower(nodes[i+1].PublicKey)},
}
graph.byNode[strings.ToLower(knownPK)] = append(
graph.byNode[strings.ToLower(knownPK)], graph.edges[key])
}
resolveAmbiguousEdges(pm, graph)
}
}
// hexPK generates a deterministic 16-char hex pubkey for index i.
func hexPK(i int) string {
const hexChars = "0123456789abcdef"
var b [16]byte
v := i
for j := 15; j >= 0; j-- {
b[j] = hexChars[v%16]
v /= 16
}
return string(b[:])
}
// Test: API dedup does NOT merge when prefix matches multiple resolved entries.
func TestDedupPrefixEntries_MultiMatchNoMerge(t *testing.T) {
pk1 := "b0b1eeeeb0b1eeee"
pk2 := "b0c2ffffb0c2ffff"
name1 := "NodeB1"
name2 := "NodeB2"
entries := []NeighborEntry{
{
Pubkey: nil, // unresolved
Prefix: "b0",
Count: 100,
LastSeen: "2026-04-10T12:00:00Z",
Observers: []string{"obs1"},
Ambiguous: true,
},
{
Pubkey: &pk1,
Prefix: "b0b1",
Name: &name1,
Count: 5,
LastSeen: "2026-04-09T12:00:00Z",
Observers: []string{"obs2"},
},
{
Pubkey: &pk2,
Prefix: "b0c2",
Name: &name2,
Count: 3,
LastSeen: "2026-04-08T12:00:00Z",
Observers: []string{"obs3"},
},
}
result := dedupPrefixEntries(entries)
if len(result) != 3 {
t.Fatalf("expected 3 entries (no merge for ambiguous prefix), got %d", len(result))
}
// Counts should be unchanged.
for _, e := range result {
if e.Pubkey != nil && *e.Pubkey == pk1 && e.Count != 5 {
t.Errorf("pk1 count should be unchanged at 5, got %d", e.Count)
}
if e.Pubkey != nil && *e.Pubkey == pk2 && e.Count != 3 {
t.Errorf("pk2 count should be unchanged at 3, got %d", e.Count)
}
}
}
+69 -1
View File
@@ -166,7 +166,7 @@ func BuildFromStoreWithLog(store *PacketStore, enableLog bool) *NeighborGraph {
// Phase 1: Extract edges from every transmission + observation.
for _, tx := range packets {
isAdvert := tx.PayloadType != nil && *tx.PayloadType == 4
isAdvert := tx.PayloadType != nil && *tx.PayloadType == PayloadADVERT
fromNode := extractFromNode(tx)
// Pre-compute lowered originator once per tx (not per observation).
fromLower := ""
@@ -206,6 +206,9 @@ func BuildFromStoreWithLog(store *PacketStore, enableLog bool) *NeighborGraph {
}
}
// Phase 1.5: Resolve ambiguous edges using full graph context.
resolveAmbiguousEdges(pm, g)
// Phase 2: Disambiguation via Jaccard similarity.
g.disambiguate()
@@ -343,6 +346,71 @@ func (g *NeighborGraph) upsertEdgeWithCandidates(knownPK, prefix string, candida
}
}
// ─── Phase 1.5: Context-based resolution of ambiguous edges ────────────────────
// resolveAmbiguousEdges attempts to resolve ambiguous prefix edges using the
// fully-built graph context. Called after Phase 1 (edge collection) completes
// so that affinity and geo proximity tiers have full neighbor data.
func resolveAmbiguousEdges(pm *prefixMap, graph *NeighborGraph) {
// Step 1: Collect ambiguous edges under read lock.
graph.mu.RLock()
type ambiguousEntry struct {
key edgeKey
edge *NeighborEdge
knownNode string
prefix string
}
var ambiguous []ambiguousEntry
for key, e := range graph.edges {
if !e.Ambiguous {
continue
}
knownNode := e.NodeA
if strings.HasPrefix(e.NodeA, "prefix:") {
knownNode = e.NodeB
}
if knownNode == "" {
continue
}
ambiguous = append(ambiguous, ambiguousEntry{key, e, knownNode, e.Prefix})
}
graph.mu.RUnlock()
// Step 2: Resolve each (no lock needed — resolveWithContext takes its own RLock).
type resolution struct {
ambiguousEntry
resolvedPK string
}
var resolutions []resolution
for _, ae := range ambiguous {
resolved, confidence, _ := pm.resolveWithContext(ae.prefix, []string{ae.knownNode}, graph)
if resolved == nil || confidence == "no_match" || confidence == "first_match" || confidence == "gps_preference" {
continue
}
rpk := strings.ToLower(resolved.PublicKey)
if rpk == ae.knownNode {
continue // self-edge guard
}
resolutions = append(resolutions, resolution{ae, rpk})
}
// Step 3: Apply resolutions under write lock.
if len(resolutions) == 0 {
return
}
graph.mu.Lock()
for _, r := range resolutions {
// Verify edge still exists and is still ambiguous (could have been
// resolved by a prior iteration if two ambiguous edges resolve to same target).
e, ok := graph.edges[r.key]
if !ok || !e.Ambiguous {
continue
}
graph.resolveEdge(r.key, e, r.knownNode, r.resolvedPK)
}
graph.mu.Unlock()
}
// ─── Disambiguation ────────────────────────────────────────────────────────────
// disambiguate resolves ambiguous edges using Jaccard similarity of neighbor sets.
+8 -2
View File
@@ -525,7 +525,7 @@ type edgeCandidate struct {
// For ADVERTs: originator↔path[0] (if unambiguous). For ALL types: observer↔path[last] (if unambiguous).
// Also handles zero-hop ADVERTs (originator↔observer direct link).
func extractEdgesFromObs(obs *StoreObs, tx *StoreTx, pm *prefixMap) []edgeCandidate {
isAdvert := tx.PayloadType != nil && *tx.PayloadType == 4
isAdvert := tx.PayloadType != nil && *tx.PayloadType == PayloadADVERT
fromNode := extractFromNode(tx)
path := parsePathJSON(obs.PathJSON)
observerPK := strings.ToLower(obs.ObserverID)
@@ -584,12 +584,18 @@ func extractEdgesFromObs(obs *StoreObs, tx *StoreTx, pm *prefixMap) []edgeCandid
// openRW opens a read-write SQLite connection (same pattern as PruneOldPackets).
func openRW(dbPath string) (*sql.DB, error) {
dsn := fmt.Sprintf("file:%s?_journal_mode=WAL&_busy_timeout=10000", dbPath)
dsn := fmt.Sprintf("file:%s?_journal_mode=WAL", dbPath)
rw, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, err
}
rw.SetMaxOpenConns(1)
// DSN _busy_timeout may not be honored by all drivers; set via PRAGMA
// to guarantee SQLite retries for up to 5s before returning SQLITE_BUSY.
if _, err := rw.Exec("PRAGMA busy_timeout = 5000"); err != nil {
rw.Close()
return nil, fmt.Errorf("set busy_timeout: %w", err)
}
return rw, nil
}
+29 -1
View File
@@ -27,7 +27,7 @@ func createTestDBWithSchema(t *testing.T) (*DB, string) {
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT, hash TEXT UNIQUE, first_seen TEXT,
route_type INTEGER, payload_type INTEGER, payload_version INTEGER,
decoded_json TEXT
decoded_json TEXT, channel_hash TEXT DEFAULT NULL
)`)
conn.Exec(`CREATE TABLE observers (
id TEXT PRIMARY KEY, name TEXT, iata TEXT
@@ -532,3 +532,31 @@ func TestPersistSemaphoreTryAcquireSkipsBatch(t *testing.T) {
<-persistSem // release
}
func TestOpenRW_BusyTimeout(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "test.db")
// Create the DB file first
db, err := sql.Open("sqlite", "file:"+dbPath+"?_journal_mode=WAL")
if err != nil {
t.Fatal(err)
}
db.Exec("CREATE TABLE dummy (id INTEGER)")
db.Close()
// Open via openRW and verify busy_timeout is set
rw, err := openRW(dbPath)
if err != nil {
t.Fatalf("openRW failed: %v", err)
}
defer rw.Close()
var timeout int
if err := rw.QueryRow("PRAGMA busy_timeout").Scan(&timeout); err != nil {
t.Fatalf("query busy_timeout: %v", err)
}
if timeout != 5000 {
t.Errorf("expected busy_timeout=5000, got %d", timeout)
}
}
+73 -15
View File
@@ -142,6 +142,9 @@ func (s *Server) RegisterRoutes(r *mux.Router) {
r.HandleFunc("/api/nodes/{pubkey}/health", s.handleNodeHealth).Methods("GET")
r.HandleFunc("/api/nodes/{pubkey}/paths", s.handleNodePaths).Methods("GET")
r.HandleFunc("/api/nodes/{pubkey}/analytics", s.handleNodeAnalytics).Methods("GET")
r.HandleFunc("/api/nodes/clock-skew", s.handleFleetClockSkew).Methods("GET")
r.HandleFunc("/api/nodes/{pubkey}/clock-skew", s.handleNodeClockSkew).Methods("GET")
r.HandleFunc("/api/observers/clock-skew", s.handleObserverClockSkew).Methods("GET")
r.HandleFunc("/api/nodes/{pubkey}/neighbors", s.handleNodeNeighbors).Methods("GET")
r.HandleFunc("/api/nodes/{pubkey}", s.handleNodeDetail).Methods("GET")
r.HandleFunc("/api/nodes", s.handleNodes).Methods("GET")
@@ -446,10 +449,12 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
// Real packet store stats
pktCount := 0
var pktEstMB float64
var pktTrackedMB float64
if s.store != nil {
ps := s.store.GetPerfStoreStatsTyped()
pktCount = ps.TotalLoaded
pktEstMB = ps.EstimatedMB
pktTrackedMB = ps.TrackedMB
}
// Real cache stats
@@ -515,6 +520,7 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
PacketStore: HealthPacketStoreStats{
Packets: pktCount,
EstimatedMB: pktEstMB,
TrackedMB: pktTrackedMB,
},
Perf: HealthPerfStats{
TotalRequests: int(perfRequests),
@@ -930,7 +936,7 @@ func (s *Server) handleDecode(w http.ResponseWriter, r *http.Request) {
writeError(w, 400, "hex is required")
return
}
decoded, err := DecodePacket(hexStr)
decoded, err := DecodePacket(hexStr, true)
if err != nil {
writeError(w, 400, err.Error())
return
@@ -962,7 +968,7 @@ func (s *Server) handlePostPacket(w http.ResponseWriter, r *http.Request) {
writeError(w, 400, "hex is required")
return
}
decoded, err := DecodePacket(hexStr)
decoded, err := DecodePacket(hexStr, false)
if err != nil {
writeError(w, 400, err.Error())
return
@@ -1312,6 +1318,36 @@ func (s *Server) handleNodeAnalytics(w http.ResponseWriter, r *http.Request) {
writeError(w, 404, "Not found")
}
func (s *Server) handleNodeClockSkew(w http.ResponseWriter, r *http.Request) {
pubkey := mux.Vars(r)["pubkey"]
if s.store == nil {
writeError(w, 404, "Not found")
return
}
result := s.store.GetNodeClockSkew(pubkey)
if result == nil {
writeError(w, 404, "No clock skew data for this node")
return
}
writeJSON(w, result)
}
func (s *Server) handleObserverClockSkew(w http.ResponseWriter, r *http.Request) {
if s.store == nil {
writeJSON(w, []ObserverCalibration{})
return
}
writeJSON(w, s.store.GetObserverCalibrations())
}
func (s *Server) handleFleetClockSkew(w http.ResponseWriter, r *http.Request) {
if s.store == nil {
writeJSON(w, []*NodeClockSkew{})
return
}
writeJSON(w, s.store.GetFleetClockSkew())
}
// --- Analytics Handlers ---
func (s *Server) handleAnalyticsRF(w http.ResponseWriter, r *http.Request) {
@@ -1652,18 +1688,35 @@ func (s *Server) handleResolveHops(w http.ResponseWriter, r *http.Request) {
}
func (s *Server) handleChannels(w http.ResponseWriter, r *http.Request) {
if s.store != nil {
region := r.URL.Query().Get("region")
channels := s.store.GetChannels(region)
region := r.URL.Query().Get("region")
includeEncrypted := r.URL.Query().Get("includeEncrypted") == "true"
// Prefer DB for full history (in-memory store has limited retention)
if s.db != nil {
channels, err := s.db.GetChannels(region)
if err != nil {
writeError(w, 500, err.Error())
return
}
if includeEncrypted {
encrypted, err := s.db.GetEncryptedChannels(region)
if err != nil {
log.Printf("WARN GetEncryptedChannels: %v", err)
} else {
channels = append(channels, encrypted...)
}
}
writeJSON(w, ChannelListResponse{Channels: channels})
return
}
channels, err := s.db.GetChannels()
if err != nil {
writeError(w, 500, err.Error())
if s.store != nil {
channels := s.store.GetChannels(region)
if includeEncrypted {
channels = append(channels, s.store.GetEncryptedChannels(region)...)
}
writeJSON(w, ChannelListResponse{Channels: channels})
return
}
writeJSON(w, ChannelListResponse{Channels: channels})
writeJSON(w, ChannelListResponse{Channels: []map[string]interface{}{}})
}
func (s *Server) handleChannelMessages(w http.ResponseWriter, r *http.Request) {
@@ -1671,17 +1724,22 @@ func (s *Server) handleChannelMessages(w http.ResponseWriter, r *http.Request) {
limit := queryInt(r, "limit", 100)
offset := queryInt(r, "offset", 0)
region := r.URL.Query().Get("region")
// Prefer DB for full history (in-memory store has limited retention)
if s.db != nil {
messages, total, err := s.db.GetChannelMessages(hash, limit, offset, region)
if err != nil {
writeError(w, 500, err.Error())
return
}
writeJSON(w, ChannelMessagesResponse{Messages: messages, Total: total})
return
}
if s.store != nil {
messages, total := s.store.GetChannelMessages(hash, limit, offset, region)
writeJSON(w, ChannelMessagesResponse{Messages: messages, Total: total})
return
}
messages, total, err := s.db.GetChannelMessages(hash, limit, offset, region)
if err != nil {
writeError(w, 500, err.Error())
return
}
writeJSON(w, ChannelMessagesResponse{Messages: messages, Total: total})
writeJSON(w, ChannelMessagesResponse{Messages: []map[string]interface{}{}, Total: 0})
}
func (s *Server) handleObservers(w http.ResponseWriter, r *http.Request) {
+70 -6
View File
@@ -170,6 +170,9 @@ func TestHealthEndpoint(t *testing.T) {
if _, ok := pktStore["estimatedMB"]; !ok {
t.Error("expected estimatedMB in packetStore")
}
if _, ok := pktStore["trackedMB"]; !ok {
t.Error("expected trackedMB in packetStore")
}
// Verify eventLoop (GC pause metrics matching Node.js shape)
el, ok := body["eventLoop"].(map[string]interface{})
@@ -774,6 +777,67 @@ func TestNodeHealthNotFound(t *testing.T) {
}
}
// TestNodeHealthPartialFromPackets verifies that a node with packets in the
// in-memory store but no DB entry returns a partial 200 response instead of 404.
// This is the fix for issue #665 (companion nodes without adverts).
func TestNodeHealthPartialFromPackets(t *testing.T) {
srv, router := setupTestServer(t)
// Inject a packet into byNode for a pubkey that doesn't exist in the nodes table
ghostPubkey := "ghost_companion_no_advert"
now := time.Now().UTC().Format(time.RFC3339)
snr := 5.0
srv.store.mu.Lock()
if srv.store.byNode == nil {
srv.store.byNode = make(map[string][]*StoreTx)
}
if srv.store.nodeHashes == nil {
srv.store.nodeHashes = make(map[string]map[string]bool)
}
srv.store.byNode[ghostPubkey] = []*StoreTx{
{Hash: "abc123", FirstSeen: now, SNR: &snr, ObservationCount: 1},
}
srv.store.nodeHashes[ghostPubkey] = map[string]bool{"abc123": true}
srv.store.mu.Unlock()
req := httptest.NewRequest("GET", "/api/nodes/"+ghostPubkey+"/health", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200 for ghost companion, got %d (body: %s)", w.Code, w.Body.String())
}
var body map[string]interface{}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("json unmarshal: %v", err)
}
// Should have a synthetic node stub
node, ok := body["node"].(map[string]interface{})
if !ok || node == nil {
t.Fatal("expected node in response")
}
if node["role"] != "unknown" {
t.Errorf("expected role=unknown, got %v", node["role"])
}
if node["public_key"] != ghostPubkey {
t.Errorf("expected public_key=%s, got %v", ghostPubkey, node["public_key"])
}
// Should have stats from the packet
stats, ok := body["stats"].(map[string]interface{})
if !ok || stats == nil {
t.Fatal("expected stats in response")
}
if stats["totalPackets"] != 1.0 { // JSON numbers are float64
t.Errorf("expected totalPackets=1, got %v", stats["totalPackets"])
}
if stats["lastHeard"] == nil {
t.Error("expected lastHeard to be set")
}
}
func TestBulkHealthEndpoint(t *testing.T) {
_, router := setupTestServer(t)
req := httptest.NewRequest("GET", "/api/nodes/bulk-health?limit=10", nil)
@@ -2155,8 +2219,8 @@ pk := "abcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890"
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'TestNode', 'repeater')", pk)
decoded := `{"name":"TestNode","pubKey":"` + pk + `"}`
raw1 := "04" + "00" + "aabb"
raw2 := "04" + "40" + "aabb"
raw1 := "11" + "01" + "aabb"
raw2 := "11" + "41" + "aabb"
payloadType := 4
for i := 0; i < 3; i++ {
@@ -2203,8 +2267,8 @@ pk := "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'Repeater2B', 'repeater')", pk)
decoded := `{"name":"Repeater2B","pubKey":"` + pk + `"}`
raw1byte := "04" + "00" + "aabb" // pathByte=0x00 → hashSize=1 (direct send, no hops)
raw2byte := "04" + "40" + "aabb" // pathByte=0x40 → hashSize=2
raw1byte := "11" + "01" + "aabb" // FLOOD, pathByte=0x01 → hashSize=1
raw2byte := "11" + "41" + "aabb" // FLOOD, pathByte=0x41 → hashSize=2
payloadType := 4
// 1 packet with hashSize=1, 4 packets with hashSize=2 (latest is 2-byte)
@@ -2246,8 +2310,8 @@ func TestGetNodeHashSizeInfoLatestWins(t *testing.T) {
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'LatestWins', 'repeater')", pk)
decoded := `{"name":"LatestWins","pubKey":"` + pk + `"}`
raw1byte := "04" + "00" + "aabb" // pathByte=0x00 → hashSize=1
raw2byte := "04" + "40" + "aabb" // pathByte=0x40 → hashSize=2
raw1byte := "11" + "01" + "aabb" // FLOOD, pathByte=0x01 → hashSize=1
raw2byte := "11" + "41" + "aabb" // FLOOD, pathByte=0x41 → hashSize=2
payloadType := 4
// 4 historical 1-byte adverts, then 1 recent 2-byte advert (latest).
+663 -60
View File
File diff suppressed because it is too large Load Diff
+3
View File
@@ -916,6 +916,9 @@
},
"estimatedMB": {
"type": "number"
},
"trackedMB": {
"type": "number"
}
}
},
+137
View File
@@ -0,0 +1,137 @@
package main
import (
"database/sql"
"testing"
"time"
_ "modernc.org/sqlite"
)
func TestTouchNodeLastSeen_UpdatesDB(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Insert a node with no last_seen
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "abc123", "relay1", "REPEATER")
err := db.TouchNodeLastSeen("abc123", "2026-04-12T04:00:00Z")
if err != nil {
t.Fatalf("TouchNodeLastSeen returned error: %v", err)
}
var lastSeen sql.NullString
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "abc123").Scan(&lastSeen)
if !lastSeen.Valid || lastSeen.String != "2026-04-12T04:00:00Z" {
t.Fatalf("expected last_seen=2026-04-12T04:00:00Z, got %v", lastSeen)
}
}
func TestTouchNodeLastSeen_DoesNotGoBackwards(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
"abc123", "relay1", "REPEATER", "2026-04-12T05:00:00Z")
// Try to set an older timestamp
err := db.TouchNodeLastSeen("abc123", "2026-04-12T04:00:00Z")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
var lastSeen string
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "abc123").Scan(&lastSeen)
if lastSeen != "2026-04-12T05:00:00Z" {
t.Fatalf("last_seen went backwards: got %s", lastSeen)
}
}
func TestTouchNodeLastSeen_NonExistentNode(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Should not error for non-existent node
err := db.TouchNodeLastSeen("nonexistent", "2026-04-12T04:00:00Z")
if err != nil {
t.Fatalf("unexpected error for non-existent node: %v", err)
}
}
func TestTouchRelayLastSeen_Debouncing(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "relay1", "R1", "REPEATER")
s := &PacketStore{
db: db,
lastSeenTouched: make(map[string]time.Time),
}
pk := "relay1"
tx := &StoreTx{
ResolvedPath: []*string{&pk},
}
now := time.Now()
s.touchRelayLastSeen(tx, now)
// Verify it was written
var lastSeen sql.NullString
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
if !lastSeen.Valid {
t.Fatal("expected last_seen to be set after first touch")
}
// Reset last_seen to check debounce prevents second write
db.conn.Exec("UPDATE nodes SET last_seen = NULL WHERE public_key = ?", "relay1")
// Call again within 5 minutes — should be debounced (no write)
s.touchRelayLastSeen(tx, now.Add(2*time.Minute))
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
if lastSeen.Valid {
t.Fatal("expected debounce to prevent second write within 5 minutes")
}
// Call after 5 minutes — should write again
s.touchRelayLastSeen(tx, now.Add(6*time.Minute))
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
if !lastSeen.Valid {
t.Fatal("expected write after debounce interval expired")
}
}
func TestTouchRelayLastSeen_SkipsNilResolvedPath(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
s := &PacketStore{
db: db,
lastSeenTouched: make(map[string]time.Time),
}
// tx with nil entries and empty resolved_path
tx := &StoreTx{
ResolvedPath: []*string{nil, nil},
}
// Should not panic or error
s.touchRelayLastSeen(tx, time.Now())
}
func TestTouchRelayLastSeen_NilDB(t *testing.T) {
s := &PacketStore{
db: nil,
lastSeenTouched: make(map[string]time.Time),
}
pk := "abc"
tx := &StoreTx{
ResolvedPath: []*string{&pk},
}
// Should not panic with nil db
s.touchRelayLastSeen(tx, time.Now())
}
+168
View File
@@ -0,0 +1,168 @@
package main
import (
"testing"
"time"
)
// TestEstimateStoreTxBytes_ReasonableValues verifies the estimate function
// returns reasonable values for different packet sizes.
func TestEstimateStoreTxBytes_ReasonableValues(t *testing.T) {
tx := &StoreTx{
Hash: "abcdef1234567890",
RawHex: "deadbeef",
DecodedJSON: `{"type":"GRP_TXT"}`,
PathJSON: `["hop1","hop2","hop3"]`,
parsedPath: []string{"hop1", "hop2", "hop3"},
pathParsed: true,
}
got := estimateStoreTxBytes(tx)
// Should be at least base (384) + maps (200) + indexes + path/subpath costs
if got < 700 {
t.Errorf("estimate too low for 3-hop tx: %d", got)
}
if got > 5000 {
t.Errorf("estimate unreasonably high for 3-hop tx: %d", got)
}
}
// TestEstimateStoreTxBytes_ManyHopsSubpaths verifies that packets with many
// hops estimate significantly more due to O(path²) subpath index entries.
func TestEstimateStoreTxBytes_ManyHopsSubpaths(t *testing.T) {
tx2 := &StoreTx{
Hash: "aabb",
parsedPath: []string{"a", "b"},
pathParsed: true,
}
tx10 := &StoreTx{
Hash: "aabb",
parsedPath: []string{"a", "b", "c", "d", "e", "f", "g", "h", "i", "j"},
pathParsed: true,
}
est2 := estimateStoreTxBytes(tx2)
est10 := estimateStoreTxBytes(tx10)
// 10 hops → 45 subpath combos × 40 = 1800 bytes just for subpaths
if est10 <= est2 {
t.Errorf("10-hop (%d) should estimate more than 2-hop (%d)", est10, est2)
}
if est10 < est2+1500 {
t.Errorf("10-hop (%d) should estimate at least 1500 more than 2-hop (%d)", est10, est2)
}
}
// TestEstimateStoreObsBytes_WithResolvedPath verifies that observations with
// ResolvedPath estimate more than those without.
func TestEstimateStoreObsBytes_WithResolvedPath(t *testing.T) {
s1, s2, s3 := "node1", "node2", "node3"
obsNoRP := &StoreObs{
ObserverID: "obs1",
PathJSON: `["a","b"]`,
}
obsWithRP := &StoreObs{
ObserverID: "obs1",
PathJSON: `["a","b"]`,
ResolvedPath: []*string{&s1, &s2, &s3},
}
estNo := estimateStoreObsBytes(obsNoRP)
estWith := estimateStoreObsBytes(obsWithRP)
if estWith <= estNo {
t.Errorf("obs with ResolvedPath (%d) should estimate more than without (%d)", estWith, estNo)
}
}
// TestEstimateStoreObsBytes_ManyObservations verifies that 15 observations
// estimate significantly more than 1.
func TestEstimateStoreObsBytes_ManyObservations(t *testing.T) {
est1 := estimateStoreObsBytes(&StoreObs{ObserverID: "a", PathJSON: `["x"]`})
est15 := int64(0)
for i := 0; i < 15; i++ {
est15 += estimateStoreObsBytes(&StoreObs{ObserverID: "a", PathJSON: `["x"]`})
}
if est15 <= est1*10 {
t.Errorf("15 obs total (%d) should be >10x single obs (%d)", est15, est1)
}
}
// TestTrackedBytesMatchesSumAfterInsert verifies that trackedBytes equals the
// sum of individual estimates after inserting packets via makeTestStore.
func TestTrackedBytesMatchesSumAfterInsert(t *testing.T) {
store := makeTestStore(20, time.Now().Add(-2*time.Hour), 5)
// Manually compute trackedBytes as sum of estimates
var expectedSum int64
for _, tx := range store.packets {
expectedSum += estimateStoreTxBytes(tx)
for _, obs := range tx.Observations {
expectedSum += estimateStoreObsBytes(obs)
}
}
if store.trackedBytes != expectedSum {
t.Errorf("trackedBytes=%d, expected sum=%d", store.trackedBytes, expectedSum)
}
}
// TestEvictionTriggersWithImprovedEstimates verifies that eviction triggers
// at the right point with the improved (higher) estimates.
func TestEvictionTriggersWithImprovedEstimates(t *testing.T) {
store := makeTestStore(100, time.Now().Add(-10*time.Hour), 5)
// trackedBytes for 100 packets is small — artificially set maxMemoryMB
// so highWatermark is just below trackedBytes to trigger eviction.
highWatermarkBytes := store.trackedBytes - 1000
if highWatermarkBytes < 1 {
highWatermarkBytes = 1
}
// maxMemoryMB * 1048576 = highWatermark, so maxMemoryMB = ceil(highWatermarkBytes / 1048576)
// But that'll be 0 for small values. Instead, directly set trackedBytes high.
store.trackedBytes = 6 * 1048576 // 6MB
store.maxMemoryMB = 3 // 3MB limit
beforeCount := len(store.packets)
store.RunEviction()
afterCount := len(store.packets)
if afterCount >= beforeCount {
t.Errorf("expected eviction to remove packets: before=%d, after=%d, trackedBytes=%d, maxMB=%d",
beforeCount, afterCount, store.trackedBytes, store.maxMemoryMB)
}
// trackedBytes should have decreased
if store.trackedBytes >= 6*1048576 {
t.Errorf("trackedBytes should have decreased after eviction")
}
}
// BenchmarkEstimateStoreTxBytes verifies the estimate function is fast.
func BenchmarkEstimateStoreTxBytes(b *testing.B) {
tx := &StoreTx{
Hash: "abcdef1234567890",
RawHex: "deadbeefdeadbeef",
DecodedJSON: `{"type":"GRP_TXT","payload":"hello"}`,
PathJSON: `["hop1","hop2","hop3","hop4","hop5"]`,
parsedPath: []string{"hop1", "hop2", "hop3", "hop4", "hop5"},
pathParsed: true,
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
estimateStoreTxBytes(tx)
}
}
// BenchmarkEstimateStoreObsBytes verifies the obs estimate function is fast.
func BenchmarkEstimateStoreObsBytes(b *testing.B) {
s := "resolvedNodePubkey123456"
obs := &StoreObs{
ObserverID: "observer1234",
PathJSON: `["a","b","c"]`,
ResolvedPath: []*string{&s, &s, &s},
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
estimateStoreObsBytes(obs)
}
}
+4
View File
@@ -115,6 +115,7 @@ type WebSocketStatsResp struct {
type HealthPacketStoreStats struct {
Packets int `json:"packets"`
EstimatedMB float64 `json:"estimatedMB"`
TrackedMB float64 `json:"trackedMB"`
}
type SlowQuery struct {
@@ -174,6 +175,8 @@ type PerfPacketStoreStats struct {
SqliteOnly bool `json:"sqliteOnly"`
MaxPackets int `json:"maxPackets"`
EstimatedMB float64 `json:"estimatedMB"`
TrackedMB float64 `json:"trackedMB"`
AvgBytesPerPacket int64 `json:"avgBytesPerPacket"`
MaxMB int `json:"maxMB"`
Indexes PacketStoreIndexes `json:"indexes"`
}
@@ -466,6 +469,7 @@ type NodeAnalyticsResponse struct {
PeerInteractions []PeerInteraction `json:"peerInteractions"`
UptimeHeatmap []HeatmapCell `json:"uptimeHeatmap"`
ComputedStats ComputedNodeStats `json:"computedStats"`
ClockSkew *NodeClockSkew `json:"clockSkew,omitempty"`
}
// ─── Analytics — RF ────────────────────────────────────────────────────────────
+11 -1
View File
@@ -125,7 +125,7 @@
}
],
"channelKeys": {
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
"Public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
},
"hashChannels": [
"#LongFast",
@@ -153,6 +153,16 @@
],
"zoom": 9
},
"geo_filter": {
"polygon": [
[37.80, -122.52],
[37.80, -121.80],
[37.20, -121.80],
[37.20, -122.52]
],
"bufferKm": 20,
"_comment": "Optional. Restricts ingestion and API responses to nodes within the polygon + bufferKm. Polygon is an array of [lat, lon] pairs (minimum 3). Use tools/geofilter-builder.html to draw a polygon visually. Remove this section to disable filtering. Nodes with no GPS fix are always allowed through."
},
"regions": {
"SJC": "San Jose, US",
"SFO": "San Francisco, US",
+1
View File
@@ -29,6 +29,7 @@ services:
- NODE_ENV=staging
- ENABLE_PPROF=true
- DISABLE_MOSQUITTO=${DISABLE_MOSQUITTO:-false}
- DISABLE_CADDY=${DISABLE_CADDY:-false}
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
interval: 30s
+1
View File
@@ -29,6 +29,7 @@ services:
environment:
- NODE_ENV=production
- DISABLE_MOSQUITTO=${DISABLE_MOSQUITTO:-false}
- DISABLE_CADDY=${DISABLE_CADDY:-false}
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
interval: 30s
+7
View File
@@ -14,6 +14,13 @@ if [ -f /app/data/theme.json ]; then
ln -sf /app/data/theme.json /app/theme.json
fi
# Source .env from data volume if present (works with any launch method)
if [ -f /app/data/.env ]; then
set -a
. /app/data/.env
set +a
fi
SUPERVISORD_CONF="/etc/supervisor/conf.d/supervisord.conf"
if [ "${DISABLE_MOSQUITTO:-false}" = "true" ] && [ "${DISABLE_CADDY:-false}" = "true" ]; then
echo "[config] internal MQTT broker disabled (DISABLE_MOSQUITTO=true)"
+51 -2
View File
@@ -48,9 +48,52 @@ No `config.json` is required. The server starts with sensible defaults:
- Ingestor connects to `mqtt://localhost:1883` automatically
- SQLite database at `/app/data/meshcore.db`
### Docker Compose (recommended for production)
### Full `docker run` Reference (recommended)
Download the example compose file:
The bare `docker run` command is the primary deployment method. One image, documented parameters — run it however you want.
```bash
docker run -d --name corescope \
--restart=unless-stopped \
-p 80:80 -p 443:443 -p 1883:1883 \
-e DISABLE_MOSQUITTO=false \
-e DISABLE_CADDY=false \
-v /your/data:/app/data \
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
-v /your/caddy-data:/data/caddy \
ghcr.io/kpa-clawbot/corescope:latest
```
#### Parameters
| Parameter | Required | Description |
|-----------|----------|-------------|
| `-p 80:80` | Yes | HTTP web UI |
| `-p 443:443` | No | HTTPS (only if using built-in Caddy with a domain) |
| `-p 1883:1883` | No | MQTT broker (expose if external gateways connect directly) |
| `-v /your/data:/app/data` | Yes | Persistent data: SQLite DB, config.json, theme.json |
| `-v /your/Caddyfile:/etc/caddy/Caddyfile:ro` | No | Custom Caddyfile for HTTPS |
| `-v /your/caddy-data:/data/caddy` | No | Caddy TLS certificate storage |
| `-e DISABLE_MOSQUITTO=true` | No | Skip the internal Mosquitto broker (use your own) |
| `-e DISABLE_CADDY=true` | No | Skip the built-in Caddy reverse proxy |
| `-e MQTT_BROKER=mqtt://host:1883` | No | Override MQTT broker URL |
#### `/app/data/.env` convenience file
Instead of passing `-e` flags, you can drop a `.env` file in your data volume:
```bash
# /your/data/.env
DISABLE_MOSQUITTO=true
DISABLE_CADDY=true
MQTT_BROKER=mqtt://my-broker:1883
```
The entrypoint sources this file before starting services. This works with any launch method (`docker run`, compose, or manage.sh).
### Docker Compose (legacy alternative)
Docker Compose files are maintained for backward compatibility but are no longer the recommended approach.
```bash
curl -sL https://raw.githubusercontent.com/Kpa-clawbot/CoreScope/master/docker-compose.example.yml \
@@ -65,6 +108,11 @@ docker compose up -d
| `HTTP_PORT` | `80` | Host port for the web UI |
| `DATA_DIR` | `./data` | Host path for persistent data |
| `DISABLE_MOSQUITTO` | `false` | Set `true` to use an external MQTT broker |
| `DISABLE_CADDY` | `false` | Set `true` to skip the built-in Caddy proxy |
### manage.sh (legacy alternative)
The `manage.sh` wrapper script provides a setup wizard and convenience commands. It uses Docker Compose internally. See [DEPLOY.md](../DEPLOY.md) for usage. New deployments should prefer bare `docker run`.
### Image tags
@@ -111,6 +159,7 @@ CoreScope uses a layered configuration system (highest priority wins):
| `MQTT_TOPIC` | `meshcore/#` | MQTT topic subscription pattern |
| `DB_PATH` | `data/meshcore.db` | SQLite database path |
| `DISABLE_MOSQUITTO` | `false` | Skip the internal Mosquitto broker |
| `DISABLE_CADDY` | `false` | Skip the built-in Caddy reverse proxy |
### config.json
+13
View File
@@ -176,6 +176,19 @@ Lower values = fresher data but more server load.
Provide cert and key paths to enable HTTPS.
## Geographic filtering
```json
"geo_filter": {
"polygon": [[51.55, 3.80], [51.55, 5.90], [50.65, 5.90], [50.65, 3.80]],
"bufferKm": 20
}
```
Restricts ingestion and API responses to nodes within the polygon plus a buffer margin. Remove the block to disable filtering. Nodes with no GPS fix always pass through.
See [Geographic Filtering](geofilter.md) for the full guide including the visual polygon builder and the prune script for cleaning up historical data.
## Home page
The `home` section customizes the onboarding experience. See `config.example.json` for the full structure including `steps`, `checklist`, and `footerLinks`.
+6
View File
@@ -66,6 +66,12 @@ Click **Import JSON** and paste a previously exported theme. The customizer load
Click **Reset to Defaults** to restore all settings to the built-in defaults.
## GeoFilter Builder
The Export tab includes a **GeoFilter Builder →** link. Click it to open a Leaflet map where you can draw a polygon boundary for your deployment area. The tool generates a `geo_filter` block you can paste directly into `config.json`.
See [Geographic Filtering](geofilter.md) for full details on what geo filtering does and how to configure it.
## How it works
The customizer writes CSS custom properties (variables) to override the defaults. Exported JSON maps directly to the `theme`, `nodeColors`, `branding`, and `home` sections of [config.json](configuration.md).
+114
View File
@@ -0,0 +1,114 @@
# Geographic Filtering
CoreScope supports geographic filtering to restrict which nodes are ingested and returned in API responses. This is useful for public-facing deployments that should only show activity in a specific region.
## How it works
Geographic filtering operates at two levels:
- **Ingest time** — ADVERT packets carrying GPS coordinates are rejected by the ingestor if the node falls outside the configured area. The node never reaches the database.
- **API responses** — Nodes already in the database are filtered from the `/api/nodes` response if they fall outside the area. This covers nodes ingested before the filter was configured.
Nodes with no GPS fix (`lat=0, lon=0` or missing coordinates) always pass the filter regardless of configuration.
## Configuration
Add a `geo_filter` block to `config.json`:
```json
"geo_filter": {
"polygon": [
[51.55, 3.80],
[51.55, 5.90],
[50.65, 5.90],
[50.65, 3.80]
],
"bufferKm": 20
}
```
| Field | Type | Description |
|-------|------|-------------|
| `polygon` | `[[lat, lon], ...]` | Array of at least 3 coordinate pairs defining the boundary |
| `bufferKm` | number | Extra distance (km) around the polygon edge that is also accepted. `0` = exact boundary |
Both the server and the ingestor read `geo_filter` from `config.json`. Restart both after changing this section.
To disable filtering entirely, remove the `geo_filter` block.
### Legacy bounding box
An older bounding box format is also supported as a fallback when no `polygon` is present:
```json
"geo_filter": {
"latMin": 50.65,
"latMax": 51.55,
"lonMin": 3.80,
"lonMax": 5.90
}
```
Prefer the polygon format — it supports irregular shapes and the `bufferKm` margin.
## API endpoint
The current geo filter configuration is exposed at:
```
GET /api/config/geo-filter
```
The frontend reads this endpoint to display the active filter. No authentication is required (the endpoint returns config, not private data).
## GeoFilter Builder
The simplest way to create a polygon is the included visual builder:
**File:** `tools/geofilter-builder.html`
Open it directly in a browser — it runs entirely client-side, no server required:
```bash
# From the project root
open tools/geofilter-builder.html # macOS
xdg-open tools/geofilter-builder.html # Linux
start tools/geofilter-builder.html # Windows
```
**Workflow:**
1. The map opens centered on Belgium by default. Navigate to your region.
2. Click on the map to add polygon vertices. Each click adds a numbered point.
3. Add at least 3 points to form a closed polygon.
4. Adjust **Buffer km** (default 20) to add a margin around the polygon edge.
5. The generated JSON block appears at the bottom of the page — copy it directly into `config.json`.
6. Use **↩ Undo** to remove the last point, **✕ Clear** to start over.
The output is a complete `{ "geo_filter": { ... } }` block ready to paste into `config.json`.
## Cleaning up historical nodes
The ingestor prevents new out-of-bounds nodes from being ingested, but it does not retroactively remove nodes that were stored before the filter was configured. For that, use the prune script.
**File:** `scripts/prune-nodes-outside-geo-filter.py`
```bash
# Dry run — shows what would be deleted without making any changes
python3 scripts/prune-nodes-outside-geo-filter.py --dry-run
# Default paths: /app/data/meshcore.db and /app/config.json
python3 scripts/prune-nodes-outside-geo-filter.py
# Custom paths
python3 scripts/prune-nodes-outside-geo-filter.py /path/to/meshcore.db \
--config /path/to/config.json
# In Docker — run inside the container
docker exec -it meshcore-analyzer \
python3 /app/scripts/prune-nodes-outside-geo-filter.py --dry-run
```
The script reads `geo_filter.polygon` and `geo_filter.bufferKm` from config, lists the nodes that fall outside, then asks for `yes` confirmation before deleting. Nodes without coordinates are always kept.
This is a **one-time migration tool** — run it once after first configuring `geo_filter` to clean up pre-filter data. The ingestor handles all subsequent filtering automatically at ingest time.
+98
View File
@@ -0,0 +1,98 @@
// Package channel provides MeshCore hashtag channel key derivation,
// decryption (HMAC-SHA256 MAC + AES-128-ECB), and plaintext parsing.
package channel
import (
"crypto/aes"
"crypto/hmac"
"crypto/sha256"
"encoding/binary"
"fmt"
"strings"
"unicode/utf8"
)
// DeriveKey derives an AES-128 key from a channel name (e.g. "#wardriving").
// Returns 16 bytes: SHA-256(channelName)[:16].
func DeriveKey(channelName string) []byte {
h := sha256.Sum256([]byte(channelName))
return h[:16]
}
// ChannelHash returns the 1-byte channel hash used as the first byte of GRP_TXT payloads.
// It is the first byte of SHA-256 of the 16-byte key.
func ChannelHash(key []byte) byte {
h := sha256.Sum256(key)
return h[0]
}
// Decrypt verifies the 2-byte HMAC-SHA256 MAC and performs AES-128-ECB decryption.
// mac must be exactly 2 bytes. ciphertext must be a multiple of 16 bytes.
// Returns the plaintext and true if MAC verification succeeded, or nil and false otherwise.
func Decrypt(key []byte, mac []byte, ciphertext []byte) ([]byte, bool) {
if len(key) != 16 || len(mac) != 2 || len(ciphertext) == 0 || len(ciphertext)%aes.BlockSize != 0 {
return nil, false
}
// 32-byte channel secret: 16-byte key + 16 zero bytes
channelSecret := make([]byte, 32)
copy(channelSecret, key)
// Verify HMAC-SHA256 (first 2 bytes must match)
h := hmac.New(sha256.New, channelSecret)
h.Write(ciphertext)
calculatedMac := h.Sum(nil)
if calculatedMac[0] != mac[0] || calculatedMac[1] != mac[1] {
return nil, false
}
// AES-128-ECB decrypt
block, err := aes.NewCipher(key)
if err != nil {
return nil, false
}
plaintext := make([]byte, len(ciphertext))
for i := 0; i < len(ciphertext); i += aes.BlockSize {
block.Decrypt(plaintext[i:i+aes.BlockSize], ciphertext[i:i+aes.BlockSize])
}
return plaintext, true
}
// ParsePlaintext parses decrypted plaintext into timestamp, sender, and message.
// Format: timestamp(4 LE) + flags(1) + "sender: message\0..."
func ParsePlaintext(plaintext []byte) (timestamp uint32, sender string, message string, err error) {
if len(plaintext) < 5 {
return 0, "", "", fmt.Errorf("plaintext too short (%d bytes)", len(plaintext))
}
timestamp = binary.LittleEndian.Uint32(plaintext[0:4])
text := string(plaintext[5:])
if idx := strings.IndexByte(text, 0); idx >= 0 {
text = text[:idx]
}
if !utf8.ValidString(text) || countNonPrintable(text) > 2 {
return 0, "", "", fmt.Errorf("decrypted text contains non-printable characters")
}
// Parse "sender: message" format
if colonIdx := strings.Index(text, ": "); colonIdx > 0 && colonIdx < 50 {
potentialSender := text[:colonIdx]
if !strings.ContainsAny(potentialSender, ":[]") {
return timestamp, potentialSender, text[colonIdx+2:], nil
}
}
return timestamp, "", text, nil
}
func countNonPrintable(s string) int {
count := 0
for _, r := range s {
if r < 32 && r != '\n' && r != '\r' && r != '\t' {
count++
}
}
return count
}
+161
View File
@@ -0,0 +1,161 @@
package channel
import (
"crypto/aes"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"testing"
)
func TestDeriveKey(t *testing.T) {
key := DeriveKey("#wardriving")
h := sha256.Sum256([]byte("#wardriving"))
expected := h[:16]
if len(key) != 16 {
t.Fatalf("key length %d, want 16", len(key))
}
for i := range key {
if key[i] != expected[i] {
t.Fatalf("DeriveKey mismatch at byte %d", i)
}
}
}
func TestChannelHash(t *testing.T) {
key := DeriveKey("#wardriving")
ch := ChannelHash(key)
h := sha256.Sum256(key)
if ch != h[0] {
t.Fatalf("ChannelHash %02x, want %02x", ch, h[0])
}
}
func testECBEncrypt(t *testing.T, key, plaintext []byte) []byte {
t.Helper()
block, err := aes.NewCipher(key)
if err != nil {
t.Fatal(err)
}
ct := make([]byte, len(plaintext))
for i := 0; i < len(plaintext); i += aes.BlockSize {
block.Encrypt(ct[i:i+aes.BlockSize], plaintext[i:i+aes.BlockSize])
}
return ct
}
func testComputeMAC(key, ciphertext []byte) []byte {
secret := make([]byte, 32)
copy(secret, key)
h := hmac.New(sha256.New, secret)
h.Write(ciphertext)
sum := h.Sum(nil)
return sum[:2]
}
func TestDecryptValidMAC(t *testing.T) {
key := DeriveKey("#test")
padded := make([]byte, 16)
copy(padded, []byte{0x01, 0x00, 0x00, 0x00, 0x00})
ciphertext := testECBEncrypt(t, key, padded)
mac := testComputeMAC(key, ciphertext)
result, ok := Decrypt(key, mac, ciphertext)
if !ok {
t.Fatal("Decrypt returned false for valid MAC")
}
if len(result) != 16 {
t.Fatalf("result length %d, want 16", len(result))
}
}
func TestDecryptInvalidMAC(t *testing.T) {
key := DeriveKey("#test")
ciphertext := make([]byte, 16)
mac := []byte{0xFF, 0xFF}
_, ok := Decrypt(key, mac, ciphertext)
if ok {
t.Fatal("Decrypt should reject wrong MAC")
}
}
func TestDecryptWrongChannel(t *testing.T) {
key1 := DeriveKey("#channel1")
key2 := DeriveKey("#channel2")
padded := make([]byte, 16)
copy(padded, []byte{0x01, 0x00, 0x00, 0x00, 0x00, 'h', 'i'})
ciphertext := testECBEncrypt(t, key1, padded)
mac := testComputeMAC(key1, ciphertext)
_, ok := Decrypt(key2, mac, ciphertext)
if ok {
t.Fatal("Decrypt should reject wrong channel key")
}
}
func TestParsePlaintext(t *testing.T) {
plain := []byte{100, 0, 0, 0, 0}
plain = append(plain, []byte("Alice: Hello\x00")...)
ts, sender, msg, err := ParsePlaintext(plain)
if err != nil {
t.Fatal(err)
}
if ts != 100 {
t.Fatalf("timestamp %d, want 100", ts)
}
if sender != "Alice" {
t.Fatalf("sender %q, want Alice", sender)
}
if msg != "Hello" {
t.Fatalf("message %q, want Hello", msg)
}
}
func TestParsePlaintextNoSender(t *testing.T) {
plain := []byte{1, 0, 0, 0, 0}
plain = append(plain, []byte("just a message\x00")...)
_, sender, msg, err := ParsePlaintext(plain)
if err != nil {
t.Fatal(err)
}
if sender != "" {
t.Fatalf("sender %q, want empty", sender)
}
if msg != "just a message" {
t.Fatalf("message %q", msg)
}
}
func TestDeriveKeyMatchesIngestor(t *testing.T) {
channelName := "#MeshCore"
key := DeriveKey(channelName)
hexKey := hex.EncodeToString(key)
h := sha256.Sum256([]byte(channelName))
expected := hex.EncodeToString(h[:16])
if hexKey != expected {
t.Fatalf("key hex %s != expected %s", hexKey, expected)
}
}
func TestRoundTrip(t *testing.T) {
key := DeriveKey("#test")
original := make([]byte, 32)
copy(original, []byte{0x64, 0x00, 0x00, 0x00, 0x00})
copy(original[5:], []byte("Bob: world\x00"))
ciphertext := testECBEncrypt(t, key, original)
mac := testComputeMAC(key, ciphertext)
plaintext, ok := Decrypt(key, mac, ciphertext)
if !ok {
t.Fatal("round-trip MAC failed")
}
ts, sender, msg, err := ParsePlaintext(plaintext)
if err != nil {
t.Fatal(err)
}
if ts != 100 || sender != "Bob" || msg != "world" {
t.Fatalf("got ts=%d sender=%q msg=%q", ts, sender, msg)
}
}
+3
View File
@@ -0,0 +1,3 @@
module github.com/meshcore-analyzer/channel
go 1.22
+3
View File
@@ -0,0 +1,3 @@
module github.com/meshcore-analyzer/sigvalidate
go 1.22
+27
View File
@@ -0,0 +1,27 @@
// Package sigvalidate provides ed25519 signature validation for MeshCore advert packets.
package sigvalidate
import (
"crypto/ed25519"
"encoding/binary"
"fmt"
)
// ValidateAdvert verifies the ed25519 signature on a MeshCore advert.
// pubKey must be 32 bytes, signature must be 64 bytes.
// The signed message is: pubKey (32) + timestamp (4 LE) + appdata.
func ValidateAdvert(pubKey, signature []byte, timestamp uint32, appdata []byte) (bool, error) {
if len(pubKey) != 32 {
return false, fmt.Errorf("invalid pubkey length: %d", len(pubKey))
}
if len(signature) != 64 {
return false, fmt.Errorf("invalid signature length: %d", len(signature))
}
message := make([]byte, 32+4+len(appdata))
copy(message[0:32], pubKey)
binary.LittleEndian.PutUint32(message[32:36], timestamp)
copy(message[36:], appdata)
return ed25519.Verify(ed25519.PublicKey(pubKey), message, signature), nil
}
+63
View File
@@ -0,0 +1,63 @@
package sigvalidate
import (
"crypto/ed25519"
"encoding/binary"
"testing"
)
func TestValidateAdvert_ValidSignature(t *testing.T) {
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
var timestamp uint32 = 1234567890
appdata := []byte{0x02, 0x10, 0x20}
// Build the signed message: pubKey + timestamp(LE) + appdata
msg := make([]byte, 32+4+len(appdata))
copy(msg[0:32], pub)
binary.LittleEndian.PutUint32(msg[32:36], timestamp)
copy(msg[36:], appdata)
sig := ed25519.Sign(priv, msg)
valid, err := ValidateAdvert([]byte(pub), sig, timestamp, appdata)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !valid {
t.Fatal("expected valid signature")
}
}
func TestValidateAdvert_InvalidSignature(t *testing.T) {
pub, _, err := ed25519.GenerateKey(nil)
if err != nil {
t.Fatal(err)
}
badSig := make([]byte, 64)
valid, err := ValidateAdvert([]byte(pub), badSig, 100, []byte{0x01})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if valid {
t.Fatal("expected invalid signature")
}
}
func TestValidateAdvert_BadPubkeyLength(t *testing.T) {
_, err := ValidateAdvert([]byte{1, 2, 3}, make([]byte, 64), 0, nil)
if err == nil {
t.Fatal("expected error for short pubkey")
}
}
func TestValidateAdvert_BadSignatureLength(t *testing.T) {
_, err := ValidateAdvert(make([]byte, 32), []byte{1, 2, 3}, 0, nil)
if err == nil {
t.Fatal("expected error for short signature")
}
}
+267 -27
View File
@@ -87,6 +87,7 @@
<button class="tab-btn" data-tab="distance">Distance</button>
<button class="tab-btn" data-tab="neighbor-graph">Neighbor Graph</button>
<button class="tab-btn" data-tab="rf-health">RF Health</button>
<button class="tab-btn" data-tab="clock-health">Clock Health</button>
<button class="tab-btn" data-tab="prefix-tool">Prefix Tool</button>
</div>
</div>
@@ -181,6 +182,7 @@
case 'distance': await renderDistanceTab(el); break;
case 'neighbor-graph': await renderNeighborGraphTab(el); break;
case 'rf-health': await renderRFHealthTab(el); break;
case 'clock-health': await renderClockHealthTab(el); break;
case 'prefix-tool': await renderPrefixTool(el); break;
}
// Auto-apply column resizing to all analytics tables
@@ -946,25 +948,9 @@
</div>
</div>
<div class="analytics-row">
<div class="analytics-card flex-1">
<h3>Multi-Byte Hash Adopters</h3>
<p class="text-muted">Nodes advertising with 2+ byte hash paths</p>
${data.multiByteNodes.length ? `
<table class="analytics-table">
<thead><tr><th scope="col">Node</th><th scope="col">Hash Size</th><th scope="col">Adverts</th><th scope="col">Last Seen</th></tr></thead>
<tbody>
${data.multiByteNodes.map(n => `<tr class="clickable-row" data-action="navigate" data-value="#/nodes/${n.pubkey ? encodeURIComponent(n.pubkey) : ''}" tabindex="0" role="row">
<td><strong>${esc(n.name)}</strong></td>
<td><span class="badge badge-hash-${n.hashSize}">${n.hashSize}-byte</span></td>
<td>${n.packets}</td>
<td>${timeAgo(n.lastSeen)}</td>
</tr>`).join('')}
</tbody>
</table>
` : '<div class="text-muted" style="padding:16px">No multi-byte adopters found</div>'}
</div>
${renderMultiByteAdopters(data.multiByteNodes, data.multiByteCapability || [])}
<div class="analytics-row">
<div class="analytics-card flex-1">
<h3>Top Path Hops</h3>
<table class="analytics-table">
@@ -986,6 +972,136 @@
`;
}
function renderMultiByteAdopters(nodes, caps) {
// Merge capability status into adopter nodes
var capByPubkey = {};
(caps || []).forEach(function(c) { capByPubkey[c.pubkey] = c; });
var statusIcon = { confirmed: '✅', suspected: '⚠️', unknown: '❓' };
var statusLabel = { confirmed: 'Confirmed', suspected: 'Suspected', unknown: 'Unknown' };
var statusColor = { confirmed: 'var(--success, #22c55e)', suspected: 'var(--warning, #eab308)', unknown: 'var(--text-muted, #888)' };
// Build merged rows: each adopter node gets a capability status
var rows = (nodes || []).map(function(n) {
var cap = capByPubkey[n.pubkey] || {};
return {
name: n.name, pubkey: n.pubkey || '', role: n.role || '',
hashSize: n.hashSize, packets: n.packets, lastSeen: n.lastSeen,
status: cap.status || 'unknown', evidence: cap.evidence || ''
};
});
// Count statuses
var counts = { confirmed: 0, suspected: 0, unknown: 0 };
rows.forEach(function(r) { counts[r.status] = (counts[r.status] || 0) + 1; });
function buildTableContent(rows, filter) {
var filtered = filter === 'all' ? rows : rows.filter(function(r) { return r.status === filter; });
return (filtered.length ? '<table class="analytics-table" id="mbAdoptersTable" style="margin-top:12px">' +
'<thead><tr>' +
'<th scope="col" data-sort="name">Node</th>' +
'<th scope="col" data-sort="role">Role</th>' +
'<th scope="col" data-sort="status">Status</th>' +
'<th scope="col" data-sort="hashSize">Hash Size</th>' +
'<th scope="col" data-sort="packets">Adverts</th>' +
'<th scope="col" data-sort="lastSeen">Last Seen</th>' +
'</tr></thead>' +
'<tbody>' +
filtered.map(function(r) {
var roleColor = (window.ROLE_COLORS || {})[r.role] || '#6b7280';
return '<tr class="clickable-row" data-action="navigate" data-value="#/nodes/' + encodeURIComponent(r.pubkey) + '" tabindex="0" role="row">' +
'<td><strong>' + esc(r.name) + '</strong></td>' +
'<td><span class="badge" style="background:' + roleColor + '20;color:' + roleColor + '">' + esc(r.role || 'unknown') + '</span></td>' +
'<td><span style="color:' + (statusColor[r.status] || statusColor.unknown) + '">' +
(statusIcon[r.status] || '❓') + ' ' + (statusLabel[r.status] || 'Unknown') + '</span></td>' +
'<td><span class="badge badge-hash-' + r.hashSize + '">' + r.hashSize + '-byte</span></td>' +
'<td>' + r.packets + '</td>' +
'<td>' + (r.lastSeen ? timeAgo(r.lastSeen) : '—') + '</td>' +
'</tr>';
}).join('') +
'</tbody>' +
'</table>' : '<div class="text-muted" style="padding:16px">No adopters match this filter.</div>');
}
if (!rows.length) return '<div class="analytics-row"><div class="analytics-card flex-1">' +
'<h3>Multi-Byte Hash Adopters</h3>' +
'<div class="text-muted" style="padding:16px">No multi-byte adopters found</div></div></div>';
var html = '<div class="analytics-row"><div class="analytics-card flex-1" id="mbAdoptersSection">' +
'<div style="display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap;gap:8px">' +
'<div>' +
'<h3 style="margin:0">Multi-Byte Hash Adopters</h3>' +
'<p class="text-muted" style="margin:4px 0 0;font-size:0.8em">Nodes advertising with 2+ byte hash paths. ' +
'<strong>Confirmed</strong> = seen advertising with multi-byte hash. ' +
'<strong>Suspected</strong> = prefix appeared in a multi-byte path. ' +
'<strong>Unknown</strong> = no multi-byte evidence yet.</p>' +
'</div>' +
'<div style="display:flex;gap:4px;flex-wrap:wrap" id="mbCapFilters">' +
'<button class="tab-btn active" data-mb-filter="all">All (' + rows.length + ')</button>' +
'<button class="tab-btn" data-mb-filter="confirmed" style="--filter-color:var(--success, #22c55e)">✅ Confirmed (' + counts.confirmed + ')</button>' +
'<button class="tab-btn" data-mb-filter="suspected" style="--filter-color:var(--warning, #eab308)">⚠️ Suspected (' + counts.suspected + ')</button>' +
'<button class="tab-btn" data-mb-filter="unknown" style="--filter-color:var(--text-muted, #888)">❓ Unknown (' + counts.unknown + ')</button>' +
'</div>' +
'</div>' +
'<div id="mbAdoptersTableWrap">' + buildTableContent(rows, 'all') + '</div>' +
'</div></div>';
// Use setTimeout for event delegation on the stable section container
setTimeout(function() {
var section = document.getElementById('mbAdoptersSection');
if (!section) return;
var currentFilter = 'all';
section.addEventListener('click', function handler(e) {
var btn = e.target.closest('[data-mb-filter]');
if (btn) {
currentFilter = btn.dataset.mbFilter;
// Update active state on buttons (no DOM replacement needed)
var buttons = section.querySelectorAll('[data-mb-filter]');
buttons.forEach(function(b) { b.classList.toggle('active', b.dataset.mbFilter === currentFilter); });
// Replace only the table content, not the whole section
var wrap = section.querySelector('#mbAdoptersTableWrap');
if (wrap) wrap.innerHTML = buildTableContent(rows, currentFilter);
return;
}
var th = e.target.closest('[data-sort]');
if (th) {
var tbody = section.querySelector('tbody');
if (!tbody) return;
var sortRows = Array.from(tbody.querySelectorAll('tr'));
var col = th.dataset.sort;
var colIdx = { name: 0, status: 1, hashSize: 2, packets: 3, lastSeen: 4 };
var statusWeight = { 'confirmed': 0, 'suspected': 1, 'unknown': 2 };
sortRows.sort(function(a, b) {
var va = a.children[colIdx[col]] ? a.children[colIdx[col]].textContent.trim() : '';
var vb = b.children[colIdx[col]] ? b.children[colIdx[col]].textContent.trim() : '';
if (col === 'status') {
va = statusWeight[va.toLowerCase().split(' ').pop()] !== undefined ? statusWeight[va.toLowerCase().split(' ').pop()] : 2;
vb = statusWeight[vb.toLowerCase().split(' ').pop()] !== undefined ? statusWeight[vb.toLowerCase().split(' ').pop()] : 2;
}
if (col === 'hashSize' || col === 'packets') { va = parseInt(va) || 0; vb = parseInt(vb) || 0; }
if (va < vb) return -1;
if (va > vb) return 1;
return 0;
});
sortRows.forEach(function(r) { tbody.appendChild(r); });
}
});
}, 100);
return html;
}
// Legacy alias for tests — delegates to renderMultiByteAdopters with empty nodes
function renderMultiByteCapability(caps) {
if (!caps.length) return '';
// Convert caps to adopter-style rows for backward compat
var fakeNodes = caps.map(function(c) {
return { name: c.name, pubkey: c.pubkey, role: c.role, hashSize: c.maxHashSize, packets: 0, lastSeen: c.lastSeen };
});
return renderMultiByteAdopters(fakeNodes, caps);
}
async function renderCollisionTab(el, data, collisionData) {
el.innerHTML = `
<nav id="hashIssuesToc" style="display:flex;gap:12px;margin-bottom:12px;font-size:13px;flex-wrap:wrap">
@@ -1079,10 +1195,10 @@
else matrixDesc.textContent = '3-byte prefix space is too large to visualize as a matrix — collision table is shown below.';
}
renderHashMatrixFromServer(cData.by_size[String(bytes)], bytes);
// Hide collision risk card for 3-byte — stats are shown in the matrix panel
// Show collision risk section for all byte sizes
const riskCard = document.getElementById('collisionRiskSection');
if (riskCard) riskCard.style.display = bytes === 3 ? 'none' : '';
if (bytes !== 3) renderCollisionsFromServer(cData.by_size[String(bytes)], bytes);
if (riskCard) riskCard.style.display = '';
renderCollisionsFromServer(cData.by_size[String(bytes)], bytes);
}
// Wire up selector
@@ -1174,9 +1290,9 @@
<div class="analytics-stat-value" style="font-size:16px">${pctStr}%</div>
<div style="font-size:10px;color:var(--text-muted);margin-top:2px">${usedCount > 256 ? usedCount + ' of ' : 'of '}${spaceLabel} possible</div>
</div>
<div class="analytics-stat-card" style="flex:1;min-width:110px;border-color:${collisionCount > 0 ? 'var(--status-red)' : 'var(--border)'}">
<div class="analytics-stat-card" style="flex:1;min-width:110px;border-color:${collisionCount > 0 ? 'var(--status-red)' : 'var(--border)'}${collisionCount > 0 ? ';cursor:pointer' : ''}" ${collisionCount > 0 ? 'onclick="document.getElementById(\'collisionRiskSection\')?.scrollIntoView({behavior:\'smooth\',block:\'start\'})"' : ''} ${collisionCount > 0 ? 'title="Click to see collision details"' : ''}>
<div class="analytics-stat-label">Prefix collisions</div>
<div class="analytics-stat-value" style="color:${collisionCount > 0 ? 'var(--status-red)' : 'var(--status-green)'}">${collisionCount}</div>
<div class="analytics-stat-value" style="color:${collisionCount > 0 ? 'var(--status-red)' : 'var(--status-green)'}">${collisionCount}${collisionCount > 0 ? ' <span style="font-size:11px;opacity:0.7">▼</span>' : ''}</div>
</div>
</div>`;
}
@@ -1251,7 +1367,7 @@
// 3-byte: show a summary panel instead of a matrix
if (bytes === 3) {
el.innerHTML = hashStatCardsHtml(totalNodes, stats.using_this_size || 0, '3-byte', 16777216, stats.unique_prefixes || 0, stats.collision_count || 0) +
`<p class="text-muted" style="margin:0;font-size:0.8em">The 3-byte prefix space (16.7M values) is too large to visualize as a grid.</p>` +
`<p class="text-muted" style="margin:0;font-size:0.8em">The 3-byte prefix space (16.7M values) is too large to visualize as a grid.${(stats.collision_count || 0) > 0 ? ' See collision details below.' : ''}</p>` +
`<p class="text-muted" style="margin:8px 0 0;font-size:0.8em">️ This tab only counts collisions among repeaters configured for this hash size. The <a href="#/analytics?tab=prefix-tool" style="color:var(--accent)">Prefix Tool</a> checks all repeaters regardless of configured hash size.</p>`;
return;
}
@@ -1882,6 +1998,10 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
window._analyticsChannelTbodyHtml = channelTbodyHtml;
window._analyticsChannelTheadHtml = channelTheadHtml;
window._analyticsRfNFColumnChart = rfNFColumnChart;
window._analyticsRenderMultiByteCapability = renderMultiByteCapability;
window._analyticsRenderMultiByteAdopters = renderMultiByteAdopters;
window._analyticsHashStatCardsHtml = hashStatCardsHtml;
window._analyticsRenderCollisionsFromServer = renderCollisionsFromServer;
}
// ─── Neighbor Graph Tab ─────────────────────────────────────────────────────
@@ -1896,8 +2016,8 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
<label style="font-size:13px">Roles:
<span id="ngRoleChecks" style="margin-left:4px"></span>
</label>
<label style="font-size:13px">Min Score: <input type="range" id="ngMinScore" min="0" max="100" value="10" style="width:100px;vertical-align:middle">
<span id="ngMinScoreVal">0.10</span>
<label style="font-size:13px">Min Score: <input type="range" id="ngMinScore" min="0" max="100" value="70" style="width:100px;vertical-align:middle">
<span id="ngMinScoreVal">0.70</span>
</label>
<label style="font-size:13px">Confidence:
<select id="ngConfidence" style="font-size:12px;padding:2px 4px">
@@ -1925,6 +2045,11 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
const color = (window.ROLE_COLORS || {})[r] || '#888';
rcEl.innerHTML += `<label style="font-size:12px;margin-right:8px"><input type="checkbox" data-role="${r}" checked> <span style="color:${esc(color)}">${esc(r)}</span></label>`;
});
// Observer checkbox — unchecked by default (observers create hub-and-spoke noise)
{
const color = (window.ROLE_COLORS || {}).observer || '#8b5cf6';
rcEl.innerHTML += `<label style="font-size:12px;margin-right:8px"><input type="checkbox" data-role="observer"> <span style="color:${esc(color)}">observer</span></label>`;
}
// Load data
const rqs = RegionFilter.regionQueryString();
@@ -1942,8 +2067,17 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
startGraphRenderer();
// Filter listeners
// Restore saved min score from localStorage
var savedScore = localStorage.getItem('ng-min-score');
if (savedScore !== null) {
document.getElementById('ngMinScore').value = savedScore;
document.getElementById('ngMinScoreVal').textContent = (savedScore / 100).toFixed(2);
applyNGFilters();
}
document.getElementById('ngMinScore').addEventListener('input', function() {
document.getElementById('ngMinScoreVal').textContent = (this.value / 100).toFixed(2);
localStorage.setItem('ng-min-score', this.value);
applyNGFilters();
});
document.getElementById('ngConfidence').addEventListener('change', applyNGFilters);
@@ -1982,7 +2116,7 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
// Filter nodes by role
const visibleNodes = _ngState.allNodes.filter(n => {
const role = (n.role || 'unknown').toLowerCase();
return checkedRoles.has(role) || role === 'unknown' || role === 'observer';
return checkedRoles.has(role) || role === 'unknown';
});
const visiblePKs = new Set(visibleNodes.map(n => n.pubkey));
@@ -3289,5 +3423,111 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
return svg;
}
// #690 — Clock Health fleet view (M3)
async function renderClockHealthTab(el) {
el.innerHTML = '<div class="text-center text-muted" style="padding:40px">Loading clock health data…</div>';
try {
var data = await (await fetch('/api/nodes/clock-skew')).json();
if (!Array.isArray(data) || !data.length) {
el.innerHTML = '<div class="text-center text-muted" style="padding:40px">No clock skew data available. Nodes need recent adverts for clock analysis.</div>';
return;
}
// State
var activeFilter = 'all';
var sortKey = 'severity';
var sortDir = 'asc'; // severity worst-first
function render() {
// Filter
var filtered = activeFilter === 'all' ? data : data.filter(function(n) { return n.severity === activeFilter; });
// Sort
filtered = filtered.slice().sort(function(a, b) {
var v;
if (sortKey === 'severity') {
v = (SKEW_SEVERITY_ORDER[a.severity] || 9) - (SKEW_SEVERITY_ORDER[b.severity] || 9);
} else if (sortKey === 'skew') {
v = Math.abs(b.medianSkewSec || 0) - Math.abs(a.medianSkewSec || 0);
} else if (sortKey === 'name') {
v = (a.nodeName || '').localeCompare(b.nodeName || '');
} else if (sortKey === 'drift') {
v = Math.abs(b.driftPerDaySec || 0) - Math.abs(a.driftPerDaySec || 0);
}
return sortDir === 'desc' ? -v : v;
});
// Summary
var counts = { ok: 0, warning: 0, critical: 0, absurd: 0 };
data.forEach(function(n) { if (counts[n.severity] !== undefined) counts[n.severity]++; });
// Filter buttons (also serve as summary — no separate stats pills needed)
var filterColors = { ok: 'var(--status-green)', warning: 'var(--status-yellow)', critical: 'var(--status-orange)', absurd: 'var(--status-purple)', no_clock: 'var(--text-muted)' };
var filters = ['all', 'ok', 'warning', 'critical', 'absurd', 'no_clock'];
var filterHtml = '<div style="margin-bottom:10px">' + filters.map(function(f) {
var dot = f !== 'all' ? '<span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + filterColors[f] + ';margin-right:4px;vertical-align:middle"></span>' : '';
return '<button class="clock-filter-btn' + (activeFilter === f ? ' active' : '') + '" data-filter="' + f + '">' +
dot + (f === 'all' ? 'All (' + data.length + ')' : (SKEW_SEVERITY_LABELS[f] || f) + ' (' + (counts[f] || 0) + ')') +
'</button>';
}).join('') + '</div>';
// Table
var rowsHtml = filtered.map(function(n) {
var rowClass = 'clock-fleet-row--' + (n.severity || 'ok');
var lastAdv = n.lastObservedTS ? new Date(n.lastObservedTS * 1000).toISOString().replace('T', ' ').replace(/\.\d+Z/, ' UTC') : '—';
var skewText = n.severity === 'no_clock' ? 'No Clock' : formatSkew(n.medianSkewSec);
var driftText = n.severity === 'no_clock' || !n.driftPerDaySec ? '' : formatDrift(n.driftPerDaySec);
return '<tr class="' + rowClass + '" data-pubkey="' + esc(n.pubkey) + '" style="cursor:pointer">' +
'<td><strong>' + esc(n.nodeName || n.pubkey.slice(0, 12)) + '</strong></td>' +
'<td style="font-family:var(--mono,monospace)">' + skewText + '</td>' +
'<td>' + renderSkewBadge(n.severity, n.medianSkewSec) + '</td>' +
'<td style="font-family:var(--mono,monospace)">' + driftText + '</td>' +
'<td style="font-size:11px">' + lastAdv + '</td>' +
'</tr>';
}).join('');
el.innerHTML = '<h3 style="margin:0 0 10px">⏰ Clock Health</h3>' +
filterHtml +
'<table class="data-table analytics-table" id="clock-health-table">' +
'<thead><tr>' +
'<th data-sort-col="name" style="cursor:pointer">Name</th>' +
'<th data-sort-col="skew" style="cursor:pointer">Skew</th>' +
'<th data-sort-col="severity" style="cursor:pointer">Severity</th>' +
'<th data-sort-col="drift" style="cursor:pointer">Drift Rate</th>' +
'<th>Last Advert</th>' +
'</tr></thead><tbody>' + rowsHtml + '</tbody></table>';
// Bind filter clicks
el.querySelectorAll('.clock-filter-btn').forEach(function(btn) {
btn.addEventListener('click', function() {
activeFilter = btn.dataset.filter;
render();
});
});
// Bind header sort clicks
el.querySelectorAll('[data-sort-col]').forEach(function(th) {
th.addEventListener('click', function() {
var col = th.dataset.sortCol;
if (sortKey === col) { sortDir = sortDir === 'asc' ? 'desc' : 'asc'; }
else { sortKey = col; sortDir = 'asc'; }
render();
});
});
// Bind row clicks → navigate to node
el.querySelectorAll('tr[data-pubkey]').forEach(function(tr) {
tr.addEventListener('click', function() {
location.hash = '#/nodes/' + encodeURIComponent(tr.dataset.pubkey);
});
});
}
render();
} catch (err) {
el.innerHTML = '<div class="text-center" style="color:var(--status-red);padding:40px">Failed to load clock health data: ' + esc(String(err)) + '</div>';
}
}
registerPage('analytics', { init, destroy });
})();
+295
View File
@@ -0,0 +1,295 @@
/**
* Client-side MeshCore channel decryption module.
*
* Implements the same crypto as internal/channel/channel.go:
* - Key derivation: SHA-256("#channelname")[:16]
* - Channel hash: SHA-256(key)[0]
* - MAC: HMAC-SHA256 with 32-byte secret (key + 16 zero bytes), truncated to 2 bytes
* - Encryption: AES-128-ECB (block-by-block)
* - Plaintext: timestamp(4 LE) + flags(1) + "sender: message\0"
*
* Keys NEVER leave the browser. No fetch/XHR/network calls in this module.
*/
/* eslint-disable no-var */
window.ChannelDecrypt = (function () {
'use strict';
var STORAGE_KEY = 'corescope_channel_keys';
var CACHE_KEY = 'corescope_channel_cache';
// ---- Hex utilities ----
function bytesToHex(bytes) {
var hex = '';
for (var i = 0; i < bytes.length; i++) {
hex += (bytes[i] < 16 ? '0' : '') + bytes[i].toString(16);
}
return hex;
}
function hexToBytes(hex) {
var bytes = new Uint8Array(hex.length / 2);
for (var i = 0; i < hex.length; i += 2) {
bytes[i / 2] = parseInt(hex.substring(i, i + 2), 16);
}
return bytes;
}
// ---- Key derivation ----
/**
* Derive AES-128 key from channel name: SHA-256("#channelname")[:16].
* @param {string} channelName - e.g. "#LongFast"
* @returns {Promise<Uint8Array>} 16-byte key
*/
async function deriveKey(channelName) {
var enc = new TextEncoder();
var hash = await crypto.subtle.digest('SHA-256', enc.encode(channelName));
return new Uint8Array(hash).slice(0, 16);
}
/**
* Compute the 1-byte channel hash: SHA-256(key)[0].
* @param {Uint8Array} key - 16-byte key
* @returns {Promise<number>} single byte (0-255)
*/
async function computeChannelHash(key) {
var hash = await crypto.subtle.digest('SHA-256', key);
return new Uint8Array(hash)[0];
}
// ---- AES-128-ECB via Web Crypto (CBC with zero IV, block-by-block) ----
/**
* Decrypt AES-128-ECB by decrypting each 16-byte block independently
* using AES-CBC with a zero IV (equivalent to ECB for single blocks).
* @param {Uint8Array} key - 16-byte AES key
* @param {Uint8Array} ciphertext - must be multiple of 16 bytes
* @returns {Promise<Uint8Array>} plaintext
*/
async function decryptECB(key, ciphertext) {
if (ciphertext.length === 0 || ciphertext.length % 16 !== 0) {
return null;
}
var cryptoKey = await crypto.subtle.importKey(
'raw', key, { name: 'AES-CBC' }, false, ['decrypt']
);
var zeroIV = new Uint8Array(16);
var plaintext = new Uint8Array(ciphertext.length);
for (var i = 0; i < ciphertext.length; i += 16) {
var block = ciphertext.slice(i, i + 16);
// Append a dummy block (16 bytes of 0x10 = PKCS7 padding for empty next block)
// so Web Crypto doesn't complain about padding
var padded = new Uint8Array(32);
padded.set(block, 0);
// Second block is PKCS7 padding: 16 bytes of 0x10
for (var j = 16; j < 32; j++) padded[j] = 16;
var decrypted = await crypto.subtle.decrypt(
{ name: 'AES-CBC', iv: zeroIV }, cryptoKey, padded
);
var decBytes = new Uint8Array(decrypted);
plaintext.set(decBytes.slice(0, 16), i);
}
return plaintext;
}
// ---- MAC verification ----
/**
* Verify HMAC-SHA256 MAC (first 2 bytes) using 32-byte secret (key + 16 zero bytes).
* @param {Uint8Array} key - 16-byte AES key
* @param {Uint8Array} ciphertext - encrypted data
* @param {string} macHex - 4-char hex string (2 bytes)
* @returns {Promise<boolean>}
*/
async function verifyMAC(key, ciphertext, macHex) {
// Build 32-byte channel secret: key + 16 zero bytes
var secret = new Uint8Array(32);
secret.set(key, 0);
// remaining 16 bytes are already 0
var cryptoKey = await crypto.subtle.importKey(
'raw', secret, { name: 'HMAC', hash: 'SHA-256' }, false, ['sign']
);
var sig = await crypto.subtle.sign('HMAC', cryptoKey, ciphertext);
var sigBytes = new Uint8Array(sig);
var macBytes = hexToBytes(macHex);
return sigBytes[0] === macBytes[0] && sigBytes[1] === macBytes[1];
}
// ---- Plaintext parsing ----
/**
* Parse decrypted plaintext: timestamp(4 LE) + flags(1) + "sender: message\0..."
* @param {Uint8Array} plaintext
* @returns {{ timestamp: number, flags: number, sender: string, message: string } | null}
*/
function parsePlaintext(plaintext) {
if (!plaintext || plaintext.length < 5) return null;
var timestamp = plaintext[0] | (plaintext[1] << 8) | (plaintext[2] << 16) | ((plaintext[3] << 24) >>> 0);
var flags = plaintext[4];
// Extract text up to first null byte
var textBytes = plaintext.slice(5);
var nullIdx = -1;
for (var i = 0; i < textBytes.length; i++) {
if (textBytes[i] === 0) { nullIdx = i; break; }
}
var text = new TextDecoder().decode(nullIdx >= 0 ? textBytes.slice(0, nullIdx) : textBytes);
// Count non-printable characters
var nonPrintable = 0;
for (var c = 0; c < text.length; c++) {
var code = text.charCodeAt(c);
if (code < 32 && code !== 10 && code !== 13 && code !== 9) nonPrintable++;
}
if (nonPrintable > 2) return null;
// Parse "sender: message" format
var colonIdx = text.indexOf(': ');
if (colonIdx > 0 && colonIdx < 50) {
var potentialSender = text.substring(0, colonIdx);
if (potentialSender.indexOf(':') < 0 && potentialSender.indexOf('[') < 0 && potentialSender.indexOf(']') < 0) {
return { timestamp: timestamp, flags: flags, sender: potentialSender, message: text.substring(colonIdx + 2) };
}
}
return { timestamp: timestamp, flags: flags, sender: '', message: text };
}
// ---- Full decrypt pipeline ----
/**
* Verify MAC, decrypt, and parse a single packet.
* @param {Uint8Array} keyBytes - 16-byte key
* @param {string} macHex - 4-char hex MAC
* @param {string} encryptedHex - hex-encoded ciphertext
* @returns {Promise<{ sender: string, message: string, timestamp: number } | null>}
*/
async function decrypt(keyBytes, macHex, encryptedHex) {
var ciphertext = hexToBytes(encryptedHex);
if (ciphertext.length === 0 || ciphertext.length % 16 !== 0) return null;
var macOk = await verifyMAC(keyBytes, ciphertext, macHex);
if (!macOk) return null;
var plaintext = await decryptECB(keyBytes, ciphertext);
if (!plaintext) return null;
return parsePlaintext(plaintext);
}
// Alias used by channels.js
var decryptPacket = decrypt;
// ---- Key storage (localStorage) ----
function saveKey(channelName, keyHex) {
var keys = getKeys();
keys[channelName] = keyHex;
try { localStorage.setItem(STORAGE_KEY, JSON.stringify(keys)); } catch (e) { /* quota */ }
}
// Alias used by channels.js
var storeKey = saveKey;
function getKeys() {
try {
var raw = localStorage.getItem(STORAGE_KEY);
return raw ? JSON.parse(raw) : {};
} catch (e) { return {}; }
}
// Alias used by channels.js
var getStoredKeys = getKeys;
function removeKey(channelName) {
var keys = getKeys();
delete keys[channelName];
try { localStorage.setItem(STORAGE_KEY, JSON.stringify(keys)); } catch (e) { /* quota */ }
// Also clear cached messages for this channel
clearChannelCache(channelName);
}
/** Remove cached messages for a specific channel (by name or hash). */
function clearChannelCache(channelKey) {
try {
var cache = JSON.parse(localStorage.getItem(CACHE_KEY) || '{}');
delete cache[channelKey];
localStorage.setItem(CACHE_KEY, JSON.stringify(cache));
} catch (e) { /* quota */ }
}
// ---- Message cache (localStorage) ----
function cacheMessages(channelHash, messages) {
try {
var cache = JSON.parse(localStorage.getItem(CACHE_KEY) || '{}');
cache[channelHash] = { messages: messages, ts: Date.now() };
localStorage.setItem(CACHE_KEY, JSON.stringify(cache));
} catch (e) { /* quota */ }
}
function getCachedMessages(channelHash) {
try {
var cache = JSON.parse(localStorage.getItem(CACHE_KEY) || '{}');
var entry = cache[channelHash];
return entry ? entry.messages : null;
} catch (e) { return null; }
}
// Cache with lastTimestamp and count (used by channels.js via getCache/setCache)
var MAX_CACHED_MESSAGES = 1000;
function setCache(key, messages, lastTimestamp, totalCount) {
try {
// Enforce cache size limit: only keep most recent MAX_CACHED_MESSAGES
var toStore = messages;
if (messages.length > MAX_CACHED_MESSAGES) {
toStore = messages.slice(messages.length - MAX_CACHED_MESSAGES);
}
var cache = JSON.parse(localStorage.getItem(CACHE_KEY) || '{}');
cache[key] = {
messages: toStore,
lastTimestamp: lastTimestamp,
count: totalCount || toStore.length,
ts: Date.now()
};
localStorage.setItem(CACHE_KEY, JSON.stringify(cache));
} catch (e) { /* quota */ }
}
function getCache(key) {
try {
var cache = JSON.parse(localStorage.getItem(CACHE_KEY) || '{}');
return cache[key] || null;
} catch (e) { return null; }
}
return {
deriveKey: deriveKey,
decrypt: decrypt,
decryptPacket: decryptPacket,
decryptECB: decryptECB,
verifyMAC: verifyMAC,
parsePlaintext: parsePlaintext,
computeChannelHash: computeChannelHash,
bytesToHex: bytesToHex,
hexToBytes: hexToBytes,
saveKey: saveKey,
storeKey: storeKey,
getKeys: getKeys,
getStoredKeys: getStoredKeys,
removeKey: removeKey,
clearChannelCache: clearChannelCache,
cacheMessages: cacheMessages,
getCachedMessages: getCachedMessages,
setCache: setCache,
getCache: getCache
};
})();
+454 -11
View File
@@ -318,6 +318,290 @@
let regionChangeHandler = null;
// --- Client-side channel decryption (#725 M2) ---
// Check if input is a valid hex string (32 hex chars = 16 bytes)
function isHexKey(val) {
return /^[0-9a-fA-F]{32}$/.test(val);
}
// Show status message in the add-channel form (#759)
var statusTimer = null;
function showAddStatus(msg, type) {
var el = document.getElementById('chAddStatus');
if (!el) return;
el.textContent = msg;
el.className = 'ch-add-status ch-add-status--' + (type || 'info');
el.style.display = '';
clearTimeout(statusTimer);
if (type !== 'loading') {
statusTimer = setTimeout(function () { el.style.display = 'none'; }, 5000);
}
}
// Add a user channel by name (#channelname) or hex key
async function addUserChannel(val) {
var displayName = val.startsWith('#') ? val : (isHexKey(val) ? val.substring(0, 8) + '…' : '#' + val);
showAddStatus('Decrypting ' + displayName + ' messages…', 'loading');
var channelName, keyHex;
try {
if (val.startsWith('#')) {
channelName = val;
var keyBytes = await ChannelDecrypt.deriveKey(channelName);
keyHex = ChannelDecrypt.bytesToHex(keyBytes);
} else if (isHexKey(val)) {
keyHex = val.toLowerCase();
channelName = 'psk:' + keyHex.substring(0, 8);
} else {
// Try with # prefix if user forgot
channelName = '#' + val;
var keyBytes2 = await ChannelDecrypt.deriveKey(channelName);
keyHex = ChannelDecrypt.bytesToHex(keyBytes2);
}
ChannelDecrypt.storeKey(channelName, keyHex);
// Compute channel hash byte to find matching encrypted channels
var keyBytes3 = ChannelDecrypt.hexToBytes(keyHex);
var hashByte = await ChannelDecrypt.computeChannelHash(keyBytes3);
// Add to sidebar or merge with existing encrypted channel
mergeUserChannels();
renderChannelList();
// Auto-select and start decrypting
var targetHash = 'user:' + channelName;
// Check if there's an existing encrypted channel with this hash byte
var existingEncrypted = channels.find(function (ch) {
return ch.encrypted && String(ch.hash) === String(hashByte);
});
if (existingEncrypted) {
targetHash = existingEncrypted.hash;
}
await selectChannel(targetHash, { userKey: keyHex, channelHashByte: hashByte, channelName: channelName });
// Show success feedback (#759)
var msgCount = document.querySelectorAll('#chMessages .ch-msg').length;
var userDisplay = channelName.startsWith('psk:') ? 'Custom channel (' + channelName.substring(4) + ')' : channelName;
if (msgCount > 0) {
showAddStatus('Added ' + userDisplay + ' — ' + msgCount + ' messages decrypted', 'success');
} else {
showAddStatus('No messages found for ' + userDisplay, 'warn');
}
} catch (err) {
showAddStatus('Failed to decrypt', 'error');
}
}
// Merge user-stored keys into the channel list
function mergeUserChannels() {
var keys = ChannelDecrypt.getStoredKeys();
var names = Object.keys(keys);
for (var i = 0; i < names.length; i++) {
var name = names[i];
// Check if channel already exists by name
var exists = channels.some(function (ch) {
return ch.name === name || ch.hash === name || ch.hash === ('user:' + name);
});
if (!exists) {
channels.push({
hash: 'user:' + name,
name: name,
messageCount: 0,
lastActivityMs: 0,
lastSender: '',
lastMessage: 'Encrypted — click to decrypt',
encrypted: true,
userAdded: true
});
}
}
}
// Fetch and decrypt GRP_TXT packets client-side (M5: delta fetch + cache)
async function fetchAndDecryptChannel(keyHex, channelHashByte, channelName, opts) {
opts = opts || {};
var keyBytes = ChannelDecrypt.hexToBytes(keyHex);
// M5: Check cache first — serve cached messages immediately
var cacheKey = channelName || String(channelHashByte);
var cached = ChannelDecrypt.getCache(cacheKey);
var cachedMsgs = cached ? cached.messages : [];
var lastTs = cached ? cached.lastTimestamp : '';
var cachedCount = cached ? (cached.count || 0) : 0;
// If we have cached messages and caller wants instant render, return them first
if (cachedMsgs.length > 0 && !opts.forceFullDecrypt) {
// Signal caller to render cache immediately, then do delta fetch
if (opts.onCacheHit) opts.onCacheHit(cachedMsgs);
}
// Fetch packets from API — get all payload_type=5 (GRP_TXT/CHAN)
var rp = RegionFilter.getRegionParam();
var qs = rp ? '&region=' + encodeURIComponent(rp) : '';
var data;
try {
data = await api('/packets?limit=1000&payloadType=5' + qs, { ttl: 10000 });
} catch (e) {
return { messages: cachedMsgs, error: 'Failed to fetch packets: ' + e.message, fromCache: cachedMsgs.length > 0 };
}
var packets = data.packets || [];
// Filter for GRP_TXT (encrypted) packets matching our channel hash byte
var candidates = [];
for (var i = 0; i < packets.length; i++) {
var p = packets[i];
var dj;
try { dj = typeof p.decoded_json === 'string' ? JSON.parse(p.decoded_json) : p.decoded_json; }
catch (e) { continue; }
if (!dj) continue;
if (dj.type === 'CHAN' && dj.channel === channelName) {
candidates.push({ type: 'already_decrypted', decoded: dj, packet: p });
} else if (dj.type === 'GRP_TXT' && dj.encryptedData && dj.mac) {
if (dj.channelHash === channelHashByte) {
candidates.push({ type: 'encrypted', decoded: dj, packet: p });
}
}
}
// M5: Cache invalidation — if total candidate count changed, re-decrypt everything
var totalCandidates = candidates.length;
var needFullDecrypt = (totalCandidates !== cachedCount) || opts.forceFullDecrypt;
// M5: Delta fetch — only decrypt packets newer than lastTs
if (!needFullDecrypt && cachedMsgs.length > 0 && lastTs) {
// Filter candidates to only those newer than cached lastTimestamp
var newCandidates = candidates.filter(function (c) {
var ts = c.packet.first_seen || c.packet.timestamp || '';
return ts > lastTs;
});
if (newCandidates.length === 0) {
// Nothing new — return cache as-is
return { messages: cachedMsgs, fromCache: true };
}
// Decrypt only new candidates
var newDecrypted = await decryptCandidates(keyBytes, newCandidates);
if (newDecrypted.wrongKey) {
return { messages: cachedMsgs, wrongKey: true };
}
// Merge: cached + new, deduplicate by packetHash, sort chronologically
var merged = deduplicateAndMerge(cachedMsgs, newDecrypted.messages);
var newLastTs = merged.length ? merged[merged.length - 1].timestamp : lastTs;
ChannelDecrypt.setCache(cacheKey, merged, newLastTs, totalCandidates);
return { messages: merged, deltaCount: newDecrypted.messages.length };
}
if (candidates.length === 0) {
return { messages: cachedMsgs, empty: true };
}
// Full decrypt
var result = await decryptCandidates(keyBytes, candidates);
if (result.wrongKey) {
return { messages: result.messages, wrongKey: true };
}
var decrypted = result.messages;
// Sort chronologically (oldest first)
decrypted.sort(function (a, b) {
var ta = a.timestamp || '';
var tb = b.timestamp || '';
return ta.localeCompare(tb);
});
// M5: Cache results
var newLastTimestamp = decrypted.length ? decrypted[decrypted.length - 1].timestamp : '';
ChannelDecrypt.setCache(cacheKey, decrypted, newLastTimestamp, totalCandidates);
return { messages: decrypted };
}
/** Decrypt an array of candidate packets. Returns { messages, wrongKey }. */
async function decryptCandidates(keyBytes, candidates) {
// Sort newest first for progressive rendering
candidates.sort(function (a, b) {
var ta = a.packet.first_seen || a.packet.timestamp || '';
var tb = b.packet.first_seen || b.packet.timestamp || '';
return tb.localeCompare(ta);
});
var decrypted = [];
var macFailCount = 0;
var macCheckCount = 0;
for (var j = 0; j < candidates.length; j++) {
var c = candidates[j];
if (c.type === 'already_decrypted') {
var d = c.decoded;
var sender = d.sender || 'Unknown';
var text = d.text || '';
var ci = text.indexOf(': ');
if (ci > 0 && ci < 50 && text.substring(0, ci) === sender) {
text = text.substring(ci + 2);
}
decrypted.push({
sender: sender, text: text,
timestamp: c.packet.first_seen || c.packet.timestamp,
sender_timestamp: d.sender_timestamp || null,
packetHash: c.packet.hash, packetId: c.packet.id,
hops: d.path_len || 0, snr: c.packet.snr || null,
observers: c.packet.observer_name ? [c.packet.observer_name] : [],
repeats: 1
});
continue;
}
macCheckCount++;
var result = await ChannelDecrypt.decryptPacket(keyBytes, c.decoded.mac, c.decoded.encryptedData);
if (result) {
macFailCount = 0;
decrypted.push({
sender: result.sender, text: result.message,
timestamp: c.packet.first_seen || c.packet.timestamp,
sender_timestamp: result.timestamp || null,
packetHash: c.packet.hash, packetId: c.packet.id,
hops: 0, snr: c.packet.snr || null,
observers: c.packet.observer_name ? [c.packet.observer_name] : [],
repeats: 1
});
} else {
macFailCount++;
if (macCheckCount >= 10 && macFailCount >= macCheckCount) {
return { messages: decrypted, wrongKey: true };
}
}
}
return { messages: decrypted, wrongKey: false };
}
/** Merge cached and new messages, deduplicate by packetHash, sort chronologically. */
function deduplicateAndMerge(cached, newMsgs) {
var seen = {};
var merged = [];
// Add cached first
for (var i = 0; i < cached.length; i++) {
var key = cached[i].packetHash || ('idx:' + i);
if (!seen[key]) { seen[key] = true; merged.push(cached[i]); }
}
// Add new
for (var j = 0; j < newMsgs.length; j++) {
var key2 = newMsgs[j].packetHash || ('new:' + j);
if (!seen[key2]) { seen[key2] = true; merged.push(newMsgs[j]); }
}
merged.sort(function (a, b) {
var ta = a.timestamp || '';
var tb = b.timestamp || '';
return ta.localeCompare(tb);
});
return merged;
}
function init(app, routeParam) {
var _initUrlParams = getHashParams();
var _pendingNode = _initUrlParams.get('node');
@@ -326,6 +610,21 @@
<div class="ch-sidebar" aria-label="Channel list">
<div class="ch-sidebar-header">
<div class="ch-sidebar-title"><span class="ch-icon">💬</span> Channels</div>
<label class="ch-encrypted-toggle" title="Show encrypted channels (no key configured)">
<input type="checkbox" id="chShowEncrypted"> <span class="ch-toggle-label">🔒 No key</span>
</label>
</div>
<div class="ch-key-input-wrap" style="padding:4px 8px">
<form id="chKeyForm" autocomplete="off" class="ch-add-form">
<div class="ch-add-row">
<input type="text" id="chKeyInput" class="ch-key-input"
placeholder="#channelname"
aria-label="Channel name or hex key" spellcheck="false">
<button type="submit" class="ch-add-btn" title="Add channel">+</button>
</div>
<div class="ch-add-hint">e.g. #LongFast or 32-char hex key decrypted in your browser.</div>
<div id="chAddStatus" class="ch-add-status" style="display:none"></div>
</form>
</div>
<div id="chRegionFilter" class="region-filter-container" style="padding:0 8px"></div>
<div class="ch-channel-list" id="chList" role="listbox" aria-label="Channels">
@@ -347,6 +646,17 @@
</div>`;
RegionFilter.init(document.getElementById('chRegionFilter'));
// Encrypted channels toggle (#727)
var showEncryptedCb = document.getElementById('chShowEncrypted');
var showEncrypted = localStorage.getItem('channels-show-encrypted') === 'true';
showEncryptedCb.checked = showEncrypted;
showEncryptedCb.addEventListener('change', function () {
showEncrypted = showEncryptedCb.checked;
localStorage.setItem('channels-show-encrypted', showEncrypted ? 'true' : 'false');
loadChannels(true);
});
regionChangeHandler = RegionFilter.onChange(function () {
loadChannels(true).then(async function () {
if (!selectedHash) return;
@@ -354,8 +664,38 @@
});
});
// Channel key input handler (#725 M2, improved UX #759)
var chKeyForm = document.getElementById('chKeyForm');
if (chKeyForm) {
var submitHandler = async function (e) {
e.preventDefault();
var input = document.getElementById('chKeyInput');
var val = (input.value || '').trim();
if (!val) return;
input.value = '';
await addUserChannel(val);
};
chKeyForm.addEventListener('submit', submitHandler);
var chKeyInput = document.getElementById('chKeyInput');
if (chKeyInput) {
chKeyInput.addEventListener('focus', function () {
var st = document.getElementById('chAddStatus');
if (st) { st.style.display = 'none'; clearTimeout(statusTimer); statusTimer = null; }
});
}
}
// Auto-enable encrypted toggle if deep-linking to an encrypted channel
if (routeParam && routeParam.startsWith('enc_') && !showEncrypted) {
showEncrypted = true;
showEncryptedCb.checked = true;
localStorage.setItem('channels-show-encrypted', 'true');
}
loadObserverRegions();
loadChannels().then(async function () {
// Also load user-added encrypted channels into the sidebar
mergeUserChannels();
if (routeParam) await selectChannel(routeParam);
if (_pendingNode && _pendingNode.length < 200) await showNodeDetail(_pendingNode);
});
@@ -403,6 +743,29 @@
// Event delegation for channel selection (touch-friendly)
document.getElementById('chList').addEventListener('click', (e) => {
// M4: Remove channel button
const removeBtn = e.target.closest('[data-remove-channel]');
if (removeBtn) {
e.stopPropagation();
var channelHash = removeBtn.getAttribute('data-remove-channel');
if (!channelHash) return;
var chName = channelHash.startsWith('user:') ? channelHash.substring(5) : channelHash;
if (!confirm('Remove channel "' + chName + '"? This will clear saved keys and cached messages.')) return;
ChannelDecrypt.removeKey(chName);
// Remove from channels array
channels = channels.filter(function (c) { return c.hash !== channelHash; });
if (selectedHash === channelHash) {
selectedHash = null;
messages = [];
history.replaceState(null, '', '#/channels');
var msgEl2 = document.getElementById('chMessages');
if (msgEl2) msgEl2.innerHTML = '<div class="ch-empty">Choose a channel from the sidebar to view messages</div>';
var header2 = document.getElementById('chHeader');
if (header2) header2.querySelector('.ch-header-text').textContent = 'Select a channel';
}
renderChannelList();
return;
}
// Color dot click — open picker, don't select channel
const dot = e.target.closest('.ch-color-dot');
if (dot && window.ChannelColorPicker) {
@@ -652,7 +1015,11 @@
async function loadChannels(silent) {
try {
const rp = RegionFilter.getRegionParam();
const qs = rp ? '?region=' + encodeURIComponent(rp) : '';
var showEnc = localStorage.getItem('channels-show-encrypted') === 'true';
var params = [];
if (rp) params.push('region=' + encodeURIComponent(rp));
if (showEnc) params.push('includeEncrypted=true');
const qs = params.length ? '?' + params.join('&') : '';
const data = await api('/channels' + qs, { ttl: CLIENT_TTL.channels });
channels = (data.channels || []).map(ch => {
ch.lastActivityMs = ch.lastActivity ? new Date(ch.lastActivity).getTime() : 0;
@@ -679,27 +1046,33 @@
});
el.innerHTML = sorted.map(ch => {
const name = ch.name || `Channel ${formatHashHex(ch.hash)}`;
const color = getChannelColor(ch.hash);
const isEncrypted = ch.encrypted === true;
const name = isEncrypted ? (ch.name || 'Unknown') : (ch.name || `Channel ${formatHashHex(ch.hash)}`);
const color = isEncrypted ? 'var(--text-muted, #6b7280)' : getChannelColor(ch.hash);
const time = ch.lastActivityMs ? formatSecondsAgo(Math.floor((Date.now() - ch.lastActivityMs) / 1000)) : '';
const preview = ch.lastSender && ch.lastMessage
? `${ch.lastSender}: ${truncate(ch.lastMessage, 28)}`
: `${ch.messageCount} messages`;
const preview = isEncrypted
? `${ch.messageCount} encrypted messages (no key configured)`
: ch.lastSender && ch.lastMessage
? `${ch.lastSender}: ${truncate(ch.lastMessage, 28)}`
: `${ch.messageCount} messages`;
const sel = selectedHash === ch.hash ? ' selected' : '';
const abbr = name.startsWith('#') ? name.slice(0, 3) : name.slice(0, 2).toUpperCase();
const encClass = isEncrypted ? ' ch-encrypted' : '';
const abbr = isEncrypted ? '🔒' : (name.startsWith('#') ? name.slice(0, 3) : name.slice(0, 2).toUpperCase());
// Channel color dot for color picker (#674)
const chColor = window.ChannelColors ? window.ChannelColors.get(ch.hash) : null;
const dotStyle = chColor ? ` style="background:${chColor}"` : '';
// Left border for assigned color
const borderStyle = chColor ? ` style="border-left:3px solid ${chColor}"` : '';
// M4: Remove button for user-added channels
const removeBtn = ch.userAdded ? ' <button class="ch-remove-btn" data-remove-channel="' + escapeHtml(ch.hash) + '" title="Remove channel" aria-label="Remove ' + escapeHtml(name) + '">✕</button>' : '';
return `<button class="ch-item${sel}" data-hash="${ch.hash}"${borderStyle} type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}">
<div class="ch-badge" style="background:${color}" aria-hidden="true">${escapeHtml(abbr)}</div>
return `<button class="ch-item${sel}${encClass}" data-hash="${ch.hash}"${borderStyle} type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}"${isEncrypted ? ' data-encrypted="true"' : ''}>
<div class="ch-badge" style="background:${color}" aria-hidden="true">${isEncrypted ? '🔒' : escapeHtml(abbr)}</div>
<div class="ch-item-body">
<div class="ch-item-top">
<span class="ch-item-name">${escapeHtml(name)}</span>
<span class="ch-color-dot" data-channel="${escapeHtml(ch.hash)}"${dotStyle} title="Change channel color" aria-label="Change color for ${escapeHtml(name)}"></span>
<span class="ch-item-time" data-channel-hash="${ch.hash}">${time}</span>
<span class="ch-item-time" data-channel-hash="${ch.hash}">${time}</span>${removeBtn}
</div>
<div class="ch-item-preview">${escapeHtml(preview)}</div>
</div>
@@ -707,7 +1080,7 @@
}).join('');
}
async function selectChannel(hash) {
async function selectChannel(hash, decryptOpts) {
const rp = RegionFilter.getRegionParam() || '';
const request = beginMessageRequest(hash, rp);
selectedHash = hash;
@@ -722,6 +1095,73 @@
document.querySelector('.ch-layout')?.classList.add('ch-show-main');
const msgEl = document.getElementById('chMessages');
// Shared helper: fetch, decrypt, and render messages for a channel key (M5: cache-first)
async function decryptAndRender(keyHex, channelHashByte, channelName) {
msgEl.innerHTML = '<div class="ch-loading">Decrypting messages…</div>';
var result = await fetchAndDecryptChannel(keyHex, channelHashByte, channelName, {
onCacheHit: function (cachedMsgs) {
// M5: Render cached messages immediately while delta fetch runs
messages = cachedMsgs;
if (messages.length > 0) {
header.querySelector('.ch-header-text').textContent = name + ' — ' + messages.length + ' messages (cached)';
renderMessages();
scrollToBottom();
}
}
});
if (isStaleMessageRequest(request)) return true;
if (result.wrongKey) {
msgEl.innerHTML = '<div class="ch-empty ch-wrong-key">🔒 Key does not match — no messages could be decrypted</div>';
return true;
}
if (result.error) {
msgEl.innerHTML = '<div class="ch-empty">' + escapeHtml(result.error) + '</div>';
return true;
}
messages = result.messages || [];
if (messages.length === 0) {
msgEl.innerHTML = '<div class="ch-empty">No encrypted messages found for this channel</div>';
} else {
header.querySelector('.ch-header-text').textContent = `${name}${messages.length} messages (decrypted)`;
renderMessages();
scrollToBottom();
}
return true;
}
// Client-side decryption path (#725 M2)
if (decryptOpts && decryptOpts.userKey) {
await decryptAndRender(decryptOpts.userKey, decryptOpts.channelHashByte, decryptOpts.channelName);
return;
}
// Check if this is a user-added channel that needs decryption
var storedKeys = typeof ChannelDecrypt !== 'undefined' ? ChannelDecrypt.getStoredKeys() : {};
if (hash.startsWith('user:')) {
var chName = hash.substring(5);
if (storedKeys[chName]) {
var keyHex = storedKeys[chName];
var keyBytes = ChannelDecrypt.hexToBytes(keyHex);
var hashByte = await ChannelDecrypt.computeChannelHash(keyBytes);
await decryptAndRender(keyHex, hashByte, chName);
return;
}
}
// Also check if an encrypted channel hash matches a stored key
if (ch && ch.encrypted) {
for (var kn in storedKeys) {
var kh = storedKeys[kn];
var kb = ChannelDecrypt.hexToBytes(kh);
var hb = await ChannelDecrypt.computeChannelHash(kb);
if (String(hb) === String(hash) || String(ch.hash) === String(hb)) {
await decryptAndRender(kh, hb, kn);
return;
}
}
}
msgEl.innerHTML = '<div class="ch-loading">Loading messages…</div>';
try {
@@ -743,6 +1183,9 @@
async function refreshMessages(opts) {
if (!selectedHash) return;
// Skip refresh for encrypted channels — no messages to fetch
var selCh = channels.find(function (c) { return c.hash === selectedHash; });
if (selCh && selCh.encrypted) return;
opts = opts || {};
const msgEl = document.getElementById('chMessages');
if (!msgEl) return;
+4
View File
@@ -1173,6 +1173,10 @@
'<details style="margin-top:12px"><summary style="font-size:12px;font-weight:600;cursor:pointer;color:var(--text-muted)">Raw JSON</summary>' +
'<textarea id="cv2ExportJson" style="width:100%;min-height:200px;font-family:var(--mono);font-size:12px;background:var(--surface-1);border:1px solid var(--border);border-radius:6px;padding:12px;color:var(--text);resize:vertical;box-sizing:border-box;margin-top:8px">' + esc(json) + '</textarea>' +
'</details>' +
'<p class="cust-section-title" style="margin-top:20px">Tools</p>' +
'<p style="font-size:12px;color:var(--text-muted);margin-bottom:10px">Server-side configuration helpers.</p>' +
'<a href="/geofilter-builder.html" target="_blank" style="display:inline-block;padding:7px 14px;background:var(--surface-1);border:1px solid var(--border);border-radius:6px;color:var(--accent);font-size:13px;text-decoration:none;font-weight:500">🗺️ GeoFilter Builder →</a>' +
'<p style="font-size:11px;color:var(--text-muted);margin-top:6px">Draw a polygon on the map to generate a <code style="font-family:var(--mono)">geo_filter</code> block for <code style="font-family:var(--mono)">config.json</code>.</p>' +
'</div>';
}
+216
View File
@@ -0,0 +1,216 @@
/* drag-manager.js Free-form panel dragging (#608 M1)
* State machine: IDLE PENDING DRAGGING IDLE
* Pointer events on .panel-header, transform: translate() during drag,
* snap-to-edge on release, z-index on focus, viewport % persistence.
*/
(function () {
'use strict';
var DEAD_ZONE = 5; // px — disambiguate click vs drag
var SNAP_THRESHOLD = 20; // px — snap to edge on release
var SNAP_MARGIN = 12; // px — margin when snapped
function DragManager() {
this.state = 'IDLE';
this.activePanel = null;
this.startX = 0;
this.startY = 0;
this.panelStartX = 0;
this.panelStartY = 0;
this.preTransform = '';
this.enabled = true;
this.zCounter = 1000;
this._panels = [];
this._onKeyDown = this._handleKeyDown.bind(this);
}
DragManager.prototype.register = function (panel) {
if (!panel) return;
var header = panel.querySelector('.panel-header');
if (!header) return;
this._panels.push(panel);
var self = this;
header.addEventListener('pointerdown', function (e) {
if (!self.enabled) return;
if (e.button !== 0) return;
if (e.target.closest('button')) return;
e.preventDefault();
header.setPointerCapture(e.pointerId);
self.state = 'PENDING';
self.activePanel = panel;
self.startX = e.clientX;
self.startY = e.clientY;
var rect = panel.getBoundingClientRect();
self.panelStartX = rect.left;
self.panelStartY = rect.top;
self.preTransform = panel.style.transform || '';
document.addEventListener('keydown', self._onKeyDown);
});
header.addEventListener('pointermove', function (e) {
if (self.state === 'IDLE') return;
if (self.activePanel !== panel) return;
var dx = e.clientX - self.startX;
var dy = e.clientY - self.startY;
if (self.state === 'PENDING') {
if (Math.hypot(dx, dy) < DEAD_ZONE) return;
self.state = 'DRAGGING';
panel.classList.add('is-dragging');
panel.style.zIndex = ++self.zCounter;
self._detachFromCorner(panel);
}
panel.style.transform = 'translate(' + dx + 'px, ' + dy + 'px)';
});
header.addEventListener('pointerup', function (e) {
if (self.activePanel !== panel) return;
header.releasePointerCapture(e.pointerId);
if (self.state === 'DRAGGING') {
panel.classList.remove('is-dragging');
self._finalizePosition(panel);
}
self._reset();
});
header.addEventListener('pointercancel', function () {
if (self.activePanel !== panel) return;
panel.classList.remove('is-dragging');
if (self.state === 'DRAGGING') {
self._finalizePosition(panel);
}
self._reset();
});
};
DragManager.prototype._handleKeyDown = function (e) {
if (e.key === 'Escape' && this.state === 'DRAGGING' && this.activePanel) {
this.activePanel.classList.remove('is-dragging');
this.activePanel.style.transform = this.preTransform;
// Revert: re-attach to corner if it was cornered before
var saved = localStorage.getItem('panel-drag-' + this.activePanel.id);
if (!saved) {
// Was in corner mode — restore corner CSS
delete this.activePanel.dataset.dragged;
this.activePanel.style.top = '';
this.activePanel.style.left = '';
this.activePanel.style.right = '';
this.activePanel.style.bottom = '';
this.activePanel.style.transform = '';
// Re-apply corner position from M0
var corner = localStorage.getItem('panel-corner-' + this.activePanel.id);
if (corner) this.activePanel.setAttribute('data-position', corner);
} else {
// Was already dragged — revert to pre-drag position
this.activePanel.style.transform = 'none';
}
this._reset();
}
};
DragManager.prototype._reset = function () {
document.removeEventListener('keydown', this._onKeyDown);
this.state = 'IDLE';
this.activePanel = null;
};
DragManager.prototype._detachFromCorner = function (panel) {
var rect = panel.getBoundingClientRect();
panel.removeAttribute('data-position');
panel.dataset.dragged = 'true';
panel.style.position = 'fixed';
panel.style.top = rect.top + 'px';
panel.style.left = rect.left + 'px';
panel.style.right = 'auto';
panel.style.bottom = 'auto';
panel.style.transform = 'none';
};
DragManager.prototype._finalizePosition = function (panel) {
var rect = panel.getBoundingClientRect();
var vw = window.innerWidth;
var vh = window.innerHeight;
var x = Math.max(0, Math.min(rect.left, vw - 40));
var y = Math.max(0, Math.min(rect.top, vh - 40));
// Snap to edge
if (x < SNAP_THRESHOLD) x = SNAP_MARGIN;
if (y < SNAP_THRESHOLD) y = SNAP_MARGIN;
if (x + rect.width > vw - SNAP_THRESHOLD) x = vw - rect.width - SNAP_MARGIN;
if (y + rect.height > vh - SNAP_THRESHOLD) y = vh - rect.height - SNAP_MARGIN;
panel.style.top = y + 'px';
panel.style.left = x + 'px';
panel.style.transform = 'none';
this._persist(panel.id, x / vw, y / vh);
};
DragManager.prototype._persist = function (id, xPct, yPct) {
try {
localStorage.setItem('panel-drag-' + id,
JSON.stringify({ xPct: xPct, yPct: yPct }));
} catch (_) { /* quota exceeded — silent */ }
};
DragManager.prototype.enable = function () { this.enabled = true; };
DragManager.prototype.disable = function () {
this.enabled = false;
if (this.state !== 'IDLE' && this.activePanel) {
this.activePanel.classList.remove('is-dragging');
this._reset();
}
};
DragManager.prototype.restorePositions = function () {
var panels = this._panels;
for (var i = 0; i < panels.length; i++) {
var panel = panels[i];
var raw = localStorage.getItem('panel-drag-' + panel.id);
if (!raw) continue;
try {
var pos = JSON.parse(raw);
var x = pos.xPct * window.innerWidth;
var y = pos.yPct * window.innerHeight;
panel.removeAttribute('data-position');
panel.dataset.dragged = 'true';
panel.style.position = 'fixed';
panel.style.top = y + 'px';
panel.style.left = x + 'px';
panel.style.right = 'auto';
panel.style.bottom = 'auto';
panel.style.transform = 'none';
} catch (_) {
localStorage.removeItem('panel-drag-' + panel.id);
}
}
};
DragManager.prototype.handleResize = function () {
var panels = document.querySelectorAll('.live-overlay[data-dragged="true"]');
for (var i = 0; i < panels.length; i++) {
var panel = panels[i];
var rect = panel.getBoundingClientRect();
var vw = window.innerWidth;
var vh = window.innerHeight;
var x = rect.left, y = rect.top, moved = false;
if (rect.right > vw) { x = vw - rect.width - SNAP_MARGIN; moved = true; }
if (rect.bottom > vh) { y = vh - rect.height - SNAP_MARGIN; moved = true; }
if (x < 0) { x = SNAP_MARGIN; moved = true; }
if (y < 0) { y = SNAP_MARGIN; moved = true; }
if (moved) {
panel.style.left = x + 'px';
panel.style.top = y + 'px';
this._persist(panel.id, x / vw, y / vh);
}
}
};
// Export
window.DragManager = DragManager;
})();
+170
View File
@@ -0,0 +1,170 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>GeoFilter Builder — CoreScope</title>
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"/>
<script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"></script>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: system-ui, sans-serif; background: #1a1a2e; color: #e0e0e0; height: 100vh; display: flex; flex-direction: column; }
header { padding: 12px 16px; background: #0f0f23; border-bottom: 1px solid #333; display: flex; align-items: center; gap: 16px; flex-wrap: wrap; }
header h1 { font-size: 1rem; font-weight: 600; color: #4a9eff; white-space: nowrap; }
.controls { display: flex; gap: 8px; flex-wrap: wrap; }
button { padding: 6px 14px; border: none; border-radius: 6px; cursor: pointer; font-size: 0.85rem; font-weight: 500; }
#btnUndo { background: #333; color: #ccc; }
#btnClear { background: #5a2020; color: #ffaaaa; }
#btnUndo:hover { background: #444; }
#btnClear:hover { background: #7a2020; }
.hint { font-size: 0.8rem; color: #888; margin-left: auto; }
#map { flex: 1; }
#output-panel { background: #0f0f23; border-top: 1px solid #333; padding: 12px 16px; display: flex; gap: 12px; align-items: flex-start; }
#output-panel label { font-size: 0.75rem; color: #888; white-space: nowrap; padding-top: 6px; }
#output { flex: 1; background: #111; border: 1px solid #333; border-radius: 6px; padding: 10px 12px; font-family: monospace; font-size: 0.78rem; color: #7ec8e3; white-space: pre; overflow-x: auto; min-height: 54px; max-height: 140px; overflow-y: auto; cursor: text; }
#output.empty { color: #555; font-style: italic; }
#btnCopy { padding: 6px 14px; background: #1a4a7a; color: #7ec8e3; border-radius: 6px; border: none; cursor: pointer; font-size: 0.85rem; white-space: nowrap; align-self: flex-end; }
#btnCopy:hover { background: #2a6aaa; }
#btnCopy.copied { background: #1a6a3a; color: #7effa0; }
#counter { font-size: 0.8rem; color: #888; padding-top: 6px; white-space: nowrap; }
.bufferRow { display: flex; align-items: center; gap: 8px; }
.bufferRow label { font-size: 0.85rem; color: #aaa; }
.bufferRow input { width: 60px; padding: 5px 8px; background: #222; border: 1px solid #444; border-radius: 6px; color: #eee; font-size: 0.85rem; }
#help-bar { background: #0f0f23; padding: 6px 16px; font-size: 0.75rem; color: #666; border-top: 1px solid #222; }
#help-bar a { color: #4a9eff; text-decoration: none; }
#help-bar a:hover { text-decoration: underline; }
#back-link { font-size: 0.8rem; color: #4a9eff; text-decoration: none; white-space: nowrap; }
#back-link:hover { text-decoration: underline; }
</style>
</head>
<body>
<header>
<a href="/" id="back-link">← CoreScope</a>
<h1>GeoFilter Builder</h1>
<div class="controls">
<button id="btnUndo">↩ Undo</button>
<button id="btnClear">✕ Clear</button>
</div>
<div class="bufferRow">
<label for="bufferKm">Buffer km:</label>
<!-- Extra margin (km) outside the polygon edge that still passes the filter -->
<input type="number" id="bufferKm" value="20" min="0" max="500"/>
</div>
<span class="hint">Click on the map to add polygon points</span>
</header>
<div id="map"></div>
<!-- Output panel: shows the geo_filter JSON block ready to paste into config.json -->
<div id="output-panel">
<label>config.json</label>
<div id="output" class="empty">Add at least 3 points to generate config…</div>
<div style="display:flex;flex-direction:column;gap:8px;align-items:flex-end">
<span id="counter">0 points</span>
<button id="btnCopy">Copy</button>
</div>
</div>
<!-- Instructions: paste the output into config.json as a top-level "geo_filter" key, then restart the server -->
<div id="help-bar">
Copy the JSON above → paste as a top-level key in <code>config.json</code> → restart the server.
Nodes with no GPS fix always pass through. Remove the <code>geo_filter</code> block to disable filtering.
&nbsp;·&nbsp; <a href="https://github.com/Kpa-clawbot/CoreScope/blob/master/docs/user-guide/geofilter.md" target="_blank">Documentation ↗</a>
</div>
<script>
const map = L.map('map').setView([50.5, 4.4], 8);
L.tileLayer('https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png', {
attribution: '© OpenStreetMap © CartoDB',
maxZoom: 19
}).addTo(map);
let points = [];
let markers = [];
let polygon = null;
let closingLine = null;
function latLonPair(latlng) {
return [parseFloat(latlng.lat.toFixed(6)), parseFloat(latlng.lng.toFixed(6))];
}
function render() {
// Remove existing polygon and closing line
if (polygon) { map.removeLayer(polygon); polygon = null; }
if (closingLine) { map.removeLayer(closingLine); closingLine = null; }
if (points.length >= 3) {
polygon = L.polygon(points, {
color: '#4a9eff', weight: 2, fillColor: '#4a9eff', fillOpacity: 0.12
}).addTo(map);
} else if (points.length === 2) {
closingLine = L.polyline(points, { color: '#4a9eff', weight: 2, dashArray: '5,5' }).addTo(map);
}
updateOutput();
}
function updateOutput() {
const el = document.getElementById('output');
const counter = document.getElementById('counter');
counter.textContent = points.length + ' point' + (points.length !== 1 ? 's' : '');
if (points.length < 3) {
el.textContent = 'Add at least 3 points to generate config…';
el.classList.add('empty');
return;
}
el.classList.remove('empty');
const bufferKm = parseFloat(document.getElementById('bufferKm').value) || 0;
// Output format: { "geo_filter": { "bufferKm": N, "polygon": [[lat,lon], ...] } }
// Paste this as a top-level key in config.json
const config = { bufferKm, polygon: points };
el.textContent = JSON.stringify({ geo_filter: config }, null, 2);
}
map.on('click', function(e) {
const pt = latLonPair(e.latlng);
points.push(pt);
const idx = points.length;
const marker = L.circleMarker(e.latlng, {
radius: 6, color: '#4a9eff', weight: 2, fillColor: '#4a9eff', fillOpacity: 0.9
}).addTo(map).bindTooltip(String(idx), { permanent: true, direction: 'top', offset: [0, -8], className: 'pt-label' });
markers.push(marker);
render();
});
document.getElementById('btnUndo').addEventListener('click', function() {
if (!points.length) return;
points.pop();
const m = markers.pop();
if (m) map.removeLayer(m);
render();
});
document.getElementById('btnClear').addEventListener('click', function() {
points = [];
markers.forEach(m => map.removeLayer(m));
markers = [];
render();
});
document.getElementById('bufferKm').addEventListener('input', updateOutput);
document.getElementById('btnCopy').addEventListener('click', function() {
if (points.length < 3) return;
const text = document.getElementById('output').textContent;
navigator.clipboard.writeText(text).then(() => {
const btn = document.getElementById('btnCopy');
btn.textContent = 'Copied!';
btn.classList.add('copied');
setTimeout(() => { btn.textContent = 'Copy'; btn.classList.remove('copied'); }, 2000);
});
});
</script>
</body>
</html>
+8 -3
View File
@@ -302,14 +302,19 @@
<button class="mnc-btn" data-action="packets" data-key="${mn.pubkey}">View packets </button>
</div>
</div>`;
} catch {
} catch (err) {
const is404 = err && err.message && err.message.includes('404');
const statusIcon = is404 ? '📡' : '❓';
const statusMsg = is404
? 'Waiting for first advert — this node has been seen in channel messages but hasn\u2019t advertised yet'
: 'Could not load data';
return `<div class="my-node-card silent" data-key="${mn.pubkey}" tabindex="0" role="button">
<div class="mnc-header">
<div class="mnc-status"></div>
<div class="mnc-status">${statusIcon}</div>
<div class="mnc-name">${escapeHtml(mn.name || truncate(mn.pubkey, 12))}</div>
<button class="mnc-remove" data-key="${mn.pubkey}" title="Remove" aria-label="Remove ${escapeAttr(mn.name || truncate(mn.pubkey, 12))} from My Mesh"></button>
</div>
<div class="mnc-status-text">Could not load data</div>
<div class="mnc-status-text">${statusMsg}</div>
</div>`;
}
}));
+2
View File
@@ -95,6 +95,7 @@
<script src="table-sort.js?v=__BUST__"></script>
<script src="packet-filter.js?v=__BUST__"></script>
<script src="packet-helpers.js?v=__BUST__"></script>
<script src="channel-decrypt.js?v=__BUST__"></script>
<script src="channel-colors.js?v=__BUST__"></script>
<script src="channel-color-picker.js?v=__BUST__"></script>
<script src="packets.js?v=__BUST__"></script>
@@ -109,6 +110,7 @@
<script src="audio-v1-constellation.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="drag-manager.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=__BUST__" onerror="console.error('Failed to load:', this.src)"></script>
+39 -2
View File
@@ -805,8 +805,8 @@
/* Corner positions — applied via data-position attribute on .live-overlay panels */
.live-overlay[data-position="tl"] { top: 64px; left: 12px; bottom: auto; right: auto; }
.live-overlay[data-position="tr"] { top: 64px; right: 12px; bottom: auto; left: auto; }
.live-overlay[data-position="bl"] { bottom: 12px; left: 12px; top: auto; right: auto; }
.live-overlay[data-position="br"] { bottom: 12px; right: 12px; top: auto; left: auto; }
.live-overlay[data-position="bl"] { bottom: 58px; left: 12px; top: auto; right: auto; }
.live-overlay[data-position="br"] { bottom: 58px; right: 12px; top: auto; left: auto; }
/* Override hide animations for positioned panels — slide toward nearest edge */
.live-overlay[data-position="tl"].hidden,
@@ -849,3 +849,40 @@
}
}
/* ── Drag Manager (#608 M1) ── */
/* Panel header as drag handle — desktop pointer devices only */
@media (pointer: fine) {
.live-overlay .panel-header {
cursor: grab;
user-select: none;
touch-action: none;
}
.live-overlay.is-dragging .panel-header {
cursor: grabbing;
}
.live-overlay .panel-header:hover {
background: var(--bg-hover, rgba(255, 255, 255, 0.03));
}
}
/* Panel during drag */
.live-overlay.is-dragging {
opacity: 0.92;
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.5);
will-change: transform;
transition: none !important;
}
/* Freely placed panel — no corner transition animations */
.live-overlay[data-dragged="true"] {
transition: none;
}
@media (prefers-reduced-motion: reduce) {
.live-overlay.is-dragging,
.live-overlay[data-dragged="true"] {
transition: none;
}
}
+63 -4
View File
@@ -30,6 +30,7 @@
let _lcdClockInterval = null;
let _rateCounterInterval = null;
let _pruneInterval = null;
let _feedTimestampInterval = null;
let activeNodeDetailKey = null;
// === VCR State Machine ===
@@ -1161,6 +1162,48 @@
// Populate role legend from shared roles.js
// Initialize panel corner positions (#608 M0)
initPanelPositions();
// Initialize DragManager for free-form panel dragging (#608 M1)
if (window.DragManager) {
var dragMgr = new DragManager();
var dragPanels = ['liveFeed', 'liveLegend', 'liveNodeDetail'];
for (var di = 0; di < dragPanels.length; di++) {
dragMgr.register(document.getElementById(dragPanels[di]));
}
dragMgr.restorePositions();
// Responsive gate: disable drag below medium breakpoint or on touch
var dragMql = window.matchMedia('(pointer: fine) and (min-width: 768px)');
function onDragMediaChange(e) {
if (!e.matches) {
// Revert dragged panels to corner positions
document.querySelectorAll('.live-overlay[data-dragged="true"]').forEach(function (p) {
delete p.dataset.dragged;
p.style.transform = '';
p.style.top = '';
p.style.left = '';
p.style.right = '';
p.style.bottom = '';
});
initPanelPositions();
dragMgr.disable();
} else {
dragMgr.enable();
dragMgr.restorePositions();
}
}
dragMql.addEventListener('change', onDragMediaChange);
// Initial check
if (!dragMql.matches) dragMgr.disable();
// Resize clamping (debounced)
var resizeTimer = null;
window.addEventListener('resize', function () {
clearTimeout(resizeTimer);
resizeTimer = setTimeout(function () { dragMgr.handleResize(); }, 200);
});
}
const roleLegendList = document.getElementById('roleLegendList');
if (roleLegendList) {
for (const role of (window.ROLE_SORT || ['repeater', 'companion', 'room', 'sensor', 'observer'])) {
@@ -1334,6 +1377,13 @@
// Prune stale nodes every 60 seconds
_pruneInterval = setInterval(pruneStaleNodes, 60000);
// Refresh relative timestamps in feed every 10 seconds (#701)
_feedTimestampInterval = setInterval(function() {
document.querySelectorAll('.feed-time[data-ts]').forEach(function(el) {
el.innerHTML = formatLiveTimestampHtml(Number(el.dataset.ts));
});
}, 10000);
// Auto-hide nav with pin toggle (#62)
const topNav = document.querySelector('.top-nav');
if (topNav) { topNav.style.position = 'fixed'; topNav.style.width = '100%'; topNav.style.zIndex = '1100'; }
@@ -1668,7 +1718,7 @@
<span class="feed-type" style="color:${color}">${typeName}</span>
${dotHtml1}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(group.latestTs || Date.now())}</span>
<span class="feed-time" data-ts="${group.latestTs || Date.now()}">${formatLiveTimestampHtml(group.latestTs || Date.now())}</span>
`;
if (_ccChan1) item._ccChannel = _ccChan1; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
@@ -2680,6 +2730,7 @@
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = pkt.observation_count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${pkt.observation_count}</span>` : '';
const anomalyIcon = (pkt.decoded && pkt.decoded.anomaly) ? '<span title="Anomaly detected" style="margin-left:4px">⚠️</span>' : '';
var _ccPayload2 = (pkt.decoded || {}).payload || {};
var _ccChan = (typeName === 'GRP_TXT' || typeName === 'CHAN') ? (_ccPayload2.channel || null) : null;
var dotHtml = _ccChan ? _feedColorDot(_ccChan) : '';
@@ -2694,9 +2745,9 @@
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${dotHtml}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
${dotHtml}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}${anomalyIcon}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
<span class="feed-time" data-ts="${pkt._ts || Date.now()}">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
`;
if (_ccChan) item._ccChannel = _ccChan; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
@@ -2742,6 +2793,13 @@
requestAnimationFrame(() => requestAnimationFrame(() => entry.element.classList.remove('live-feed-enter')));
// Re-add to DOM top (works even if it was trimmed out)
feed.prepend(entry.element);
// Update timestamp to latest observation (#701)
var _dedupTimeSpan = entry.element.querySelector('.feed-time');
if (_dedupTimeSpan) {
var _dedupNow = pkt._ts || Date.now();
_dedupTimeSpan.setAttribute('data-ts', _dedupNow);
_dedupTimeSpan.innerHTML = formatLiveTimestampHtml(_dedupNow);
}
entry.pkt.observation_count = entry.count;
return;
}
@@ -2772,7 +2830,7 @@
<span class="feed-type" style="color:${color}">${typeName}</span>
${dotHtml3}${transportBadge(pkt.route_type)}${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
<span class="feed-time" data-ts="${pkt._ts || Date.now()}">${formatLiveTimestampHtml(pkt._ts || Date.now())}</span>
`;
if (_ccChan3) item._ccChannel = _ccChan3; // channel color picker (#674)
item.addEventListener('click', () => showFeedCard(item, pkt, color));
@@ -2849,6 +2907,7 @@
if (_lcdClockInterval) { clearInterval(_lcdClockInterval); _lcdClockInterval = null; }
if (_rateCounterInterval) { clearInterval(_rateCounterInterval); _rateCounterInterval = null; }
if (_pruneInterval) { clearInterval(_pruneInterval); _pruneInterval = null; }
if (_feedTimestampInterval) { clearInterval(_feedTimestampInterval); _feedTimestampInterval = null; }
if (_affinityInterval) { clearInterval(_affinityInterval); _affinityInterval = null; }
if (ws) { ws.onclose = null; ws.close(); ws = null; }
if (map) { map.remove(); map = null; }
+43 -9
View File
@@ -25,7 +25,7 @@
// Roles loaded from shared roles.js (ROLE_STYLE, ROLE_LABELS, ROLE_COLORS globals)
function makeMarkerIcon(role, isStale) {
function makeMarkerIcon(role, isStale, isAlsoObserver) {
const s = ROLE_STYLE[role] || ROLE_STYLE.companion;
const size = s.radius * 2 + 4;
const c = size / 2;
@@ -56,7 +56,22 @@
default: // circle
path = `<circle cx="${c}" cy="${c}" r="${c-2}" fill="${s.color}" stroke="#fff" stroke-width="2"/>`;
}
const svg = `<svg width="${size}" height="${size}" viewBox="0 0 ${size} ${size}" xmlns="http://www.w3.org/2000/svg">${path}</svg>`;
// If this node is also an observer, add a small star overlay
let obsOverlay = '';
if (isAlsoObserver) {
const starSize = 8;
const sx = size - starSize, sy = 0;
const scx = starSize / 2, scy = starSize / 2, so = starSize / 2 - 0.5, si = so * 0.4;
let starPts = '';
for (let i = 0; i < 5; i++) {
const aO = (i * 72 - 90) * Math.PI / 180;
const aI = ((i * 72) + 36 - 90) * Math.PI / 180;
starPts += `${scx + so * Math.cos(aO)},${scy + so * Math.sin(aO)} `;
starPts += `${scx + si * Math.cos(aI)},${scy + si * Math.sin(aI)} `;
}
obsOverlay = `<g transform="translate(${sx},${sy})"><polygon points="${starPts.trim()}" fill="${ROLE_COLORS.observer || '#f1c40f'}" stroke="#fff" stroke-width="0.8"/></g>`;
}
const svg = `<svg width="${size}" height="${size}" viewBox="0 0 ${size} ${size}" xmlns="http://www.w3.org/2000/svg">${path}${obsOverlay}</svg>`;
return L.divIcon({
html: svg,
className: 'meshcore-marker' + (isStale ? ' marker-stale' : ''),
@@ -66,14 +81,16 @@
});
}
function makeRepeaterLabelIcon(node, isStale) {
function makeRepeaterLabelIcon(node, isStale, isAlsoObserver) {
var s = ROLE_STYLE['repeater'] || ROLE_STYLE.companion;
var hs = node.hash_size || 1;
// Show the short mesh hash ID (first N bytes of pubkey, uppercased)
var shortHash = node.public_key ? node.public_key.slice(0, hs * 2).toUpperCase() : '??';
var bgColor = s.color;
// If this repeater is also an observer, show a star indicator inside the label
var obsIndicator = isAlsoObserver ? ' <span style="color:' + (ROLE_COLORS.observer || '#f1c40f') + ';font-size:13px;line-height:1;" title="Also an observer">★</span>' : '';
var html = '<div style="background:' + bgColor + ';color:#fff;font-weight:bold;font-size:11px;padding:2px 5px;border-radius:3px;border:2px solid #fff;box-shadow:0 1px 3px rgba(0,0,0,0.4);text-align:center;line-height:1.2;white-space:nowrap;">' +
shortHash + '</div>';
shortHash + obsIndicator + '</div>';
return L.divIcon({
html: html,
className: 'meshcore-marker meshcore-label-marker' + (isStale ? ' marker-stale' : ''),
@@ -547,7 +564,8 @@
const el = document.getElementById('mcRoleChecks');
if (!el) return;
el.innerHTML = '';
const obsCount = observers.filter(o => o.lat && o.lon).length;
const nodePubkeys = new Set(nodes.map(n => (n.public_key || '').toLowerCase()));
const obsCount = observers.filter(o => o.lat && o.lon && !(o.id && nodePubkeys.has(o.id.toLowerCase()))).length;
const roles = ['repeater', 'companion', 'room', 'sensor', 'observer'];
const shapeMap = { repeater: '◆', companion: '●', room: '■', sensor: '▲', observer: '★' };
@@ -638,6 +656,7 @@
var _renderingMarkers = false;
var _lastDeconflictZoom = null;
var _currentMarkerData = []; // stored marker data for zoom-only repositioning
var _observerByPubkey = new Map(); // observer id (pubkey) → observer object, rebuilt on each render
var _zoomResizeTimer = null;
function deconflictLabels(markers, mapRef) {
@@ -780,19 +799,31 @@
const allMarkers = [];
// Build a set of observer public keys for quick lookup
_observerByPubkey = new Map();
for (const obs of observers) {
if (obs.id) _observerByPubkey.set(obs.id.toLowerCase(), obs);
}
for (const node of filtered) {
const lastSeenTime = node.last_heard || node.last_seen;
const isStale = getNodeStatus(node.role || 'companion', lastSeenTime ? new Date(lastSeenTime).getTime() : 0) === 'stale';
const pk = (node.public_key || '').toLowerCase();
const isAlsoObserver = _observerByPubkey.has(pk);
const useLabel = node.role === 'repeater' && filters.hashLabels;
const icon = useLabel ? makeRepeaterLabelIcon(node, isStale) : makeMarkerIcon(node.role || 'companion', isStale);
const icon = useLabel ? makeRepeaterLabelIcon(node, isStale, isAlsoObserver) : makeMarkerIcon(node.role || 'companion', isStale, isAlsoObserver);
const latLng = L.latLng(node.lat, node.lon);
allMarkers.push({ latLng, node, icon, isLabel: useLabel, popupFn: function() { return buildPopup(node); }, alt: (node.name || 'Unknown') + ' (' + (node.role || 'node') + ')' });
allMarkers.push({ latLng, node, icon, isLabel: useLabel, popupFn: function() { return buildPopup(node); }, alt: (node.name || 'Unknown') + ' (' + (node.role || 'node') + (isAlsoObserver ? ' + observer' : '') + ')' });
}
// Add observer markers
// Add observer markers (skip observers already represented as a node marker)
// Build set of node pubkeys that are displayed on the map
const displayedNodePubkeys = new Set(filtered.map(n => (n.public_key || '').toLowerCase()));
if (filters.observer) {
for (const obs of observers) {
if (!obs.lat || !obs.lon) continue;
// Skip observers whose pubkey matches a displayed node — they're shown as combined markers
if (obs.id && displayedNodePubkeys.has(obs.id.toLowerCase())) continue;
const icon = makeMarkerIcon('observer');
const latLng = L.latLng(obs.lat, obs.lon);
allMarkers.push({ latLng, node: obs, icon, isLabel: false, popupFn: function() { return buildObserverPopup(obs); }, alt: (obs.name || obs.id || 'Unknown') + ' (observer)' });
@@ -909,6 +940,9 @@
const loc = (node.lat && node.lon) ? `${node.lat.toFixed(5)}, ${node.lon.toFixed(5)}` : '—';
const lastAdvert = node.last_seen ? timeAgo(node.last_seen) : '—';
const roleBadge = `<span style="display:inline-block;padding:2px 8px;border-radius:12px;font-size:11px;font-weight:600;background:${ROLE_COLORS[node.role] || '#4b5563'};color:#fff;">${(node.role || 'unknown').toUpperCase()}</span>`;
// Check if this node is also an observer (combined repeater+observer)
const matchingObs = node.public_key ? _observerByPubkey.get(node.public_key.toLowerCase()) : null;
const obsBadge = matchingObs ? ` <span style="display:inline-block;padding:2px 8px;border-radius:12px;font-size:11px;font-weight:600;background:${ROLE_COLORS.observer || '#f1c40f'};color:#fff;">OBSERVER</span>` : '';
const hs = node.hash_size || 1;
const hashPrefix = node.public_key ? node.public_key.slice(0, hs * 2).toUpperCase() : '—';
const hashPrefixRow = `<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Hash Prefix</dt>
@@ -917,7 +951,7 @@
return `
<div class="map-popup" style="font-family:var(--font);min-width:180px;">
<h3 style="font-weight:700;font-size:14px;margin:0 0 4px;">${safeEsc(node.name || 'Unknown')}</h3>
${roleBadge}
${roleBadge}${obsBadge}
<dl style="margin-top:8px;font-size:12px;">
${hashPrefixRow}
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Key</dt>
+104 -23
View File
@@ -315,29 +315,34 @@
let regionChangeHandler = null;
// Show full-screen node detail view (works on any screen size)
function showFullScreenNode(pubkey) {
var app = document.getElementById('app');
app.innerHTML = '<div class="node-fullscreen">' +
'<div class="node-full-header">' +
'<button class="detail-back-btn node-back-btn" id="nodeBackBtn" aria-label="Back to nodes">←</button>' +
'<span class="node-full-title">Loading…</span>' +
'</div>' +
'<div class="node-full-body" id="nodeFullBody">' +
'<div class="text-center text-muted" style="padding:40px">Loading…</div>' +
'</div>' +
'</div>';
document.getElementById('nodeBackBtn').addEventListener('click', function() { location.hash = '#/nodes'; });
loadFullNode(pubkey);
document.addEventListener('keydown', function nodesEsc(e) {
if (e.key === 'Escape') {
document.removeEventListener('keydown', nodesEsc);
location.hash = '#/nodes';
}
});
}
function init(app, routeParam) {
directNode = routeParam || null;
if (directNode) {
// Full-screen single node view
app.innerHTML = `<div class="node-fullscreen">
<div class="node-full-header">
<button class="detail-back-btn node-back-btn" id="nodeBackBtn" aria-label="Back to nodes"></button>
<span class="node-full-title">Loading</span>
</div>
<div class="node-full-body" id="nodeFullBody">
<div class="text-center text-muted" style="padding:40px">Loading</div>
</div>
</div>`;
document.getElementById('nodeBackBtn').addEventListener('click', () => { location.hash = '#/nodes'; });
loadFullNode(directNode);
// Escape to go back to nodes list
document.addEventListener('keydown', function nodesEsc(e) {
if (e.key === 'Escape') {
document.removeEventListener('keydown', nodesEsc);
location.hash = '#/nodes';
}
});
if (directNode && window.innerWidth <= 640) {
// Full-screen single node view (mobile)
showFullScreenNode(directNode);
return;
}
@@ -363,7 +368,7 @@
</div>`;
RegionFilter.init(document.getElementById('nodesRegionFilter'));
regionChangeHandler = RegionFilter.onChange(function () { _allNodes = null; loadNodes(); });
regionChangeHandler = RegionFilter.onChange(function () { _allNodes = null; _fleetSkew = null; loadNodes(); });
if (search) {
var _si = document.getElementById('nodeSearch');
@@ -377,6 +382,7 @@
}, 250));
loadNodes();
if (directNode) selectNode(directNode);
// Auto-refresh when ADVERT packets arrive via WebSocket (fixes #131)
wsHandler = debouncedOnWS(function (msgs) {
const advertMsgs = msgs.filter(isAdvertMessage);
@@ -409,6 +415,7 @@
if (needReload) {
_allNodes = null;
_fleetSkew = null;
invalidateApiCache('/nodes');
}
loadNodes(true);
@@ -493,6 +500,8 @@
<tr><td>Hash Prefix</td><td>${n.hash_size ? '<code style="font-family:var(--mono);font-weight:700">' + n.public_key.slice(0, n.hash_size * 2).toUpperCase() + '</code> (' + n.hash_size + '-byte)' : 'Unknown'}${n.hash_size_inconsistent ? ' <span style="color:var(--status-yellow);cursor:help" title="Seen: ' + (Array.isArray(n.hash_sizes_seen) ? n.hash_sizes_seen : []).join(', ') + '-byte"> varies</span>' : ''}</td></tr>
</table>
<div class="node-full-card skew-detail-section" id="node-clock-skew" style="display:none"></div>
${observers.length ? `<div class="node-full-card" id="node-observers">
${(() => { const regions = [...new Set(observers.map(o => o.iata).filter(Boolean))]; return regions.length ? `<div style="margin-bottom:8px"><strong>Regions:</strong> ${regions.map(r => '<span class="badge" style="margin:0 2px">' + escapeHtml(r) + '</span>').join(' ')}</div>` : ''; })()}
<h4>Heard By (${observers.length} observer${observers.length > 1 ? 's' : ''})</h4>
@@ -624,6 +633,35 @@
headerSelector: '#fullNeighborsHeader'
});
// #690 — Clock Skew detail section
(async function loadClockSkew() {
var container = document.getElementById('node-clock-skew');
if (!container) return;
try {
var cs = await api('/nodes/' + encodeURIComponent(n.public_key) + '/clock-skew', { ttl: 30000 });
if (!cs || !cs.severity) return;
container.style.display = '';
var severityColor = SKEW_SEVERITY_COLORS[cs.severity] || 'var(--text-muted)';
var severityLabel = SKEW_SEVERITY_LABELS[cs.severity] || cs.severity;
var driftHtml = cs.driftPerDaySec ? '<div style="font-size:12px;color:var(--text-muted);margin-top:2px">Drift: ' + formatDrift(cs.driftPerDaySec) + '</div>' : '';
var sparkHtml = renderSkewSparkline(cs.samples, 200, 32);
var skewDisplay = cs.severity === 'no_clock'
? '<span style="font-size:18px;font-weight:700;color:var(--text-muted)">No Clock</span>'
: '<span style="font-size:18px;font-weight:700;font-family:var(--mono)">' + formatSkew(cs.medianSkewSec) + '</span>';
container.innerHTML =
'<h4 style="margin:0 0 6px">⏰ Clock Skew</h4>' +
'<div style="display:flex;align-items:center;gap:12px;flex-wrap:wrap">' +
skewDisplay +
renderSkewBadge(cs.severity, cs.medianSkewSec) +
(cs.calibrated ? ' <span style="font-size:10px;color:var(--text-muted)" title="Observer-calibrated">✓ calibrated</span>' : '') +
'</div>' +
driftHtml +
(sparkHtml ? '<div class="skew-sparkline-wrap" style="margin-top:8px">' + sparkHtml + '<div style="font-size:10px;color:var(--text-muted)">Skew over time (' + (cs.samples || []).length + ' samples)</div></div>' : '');
} catch (e) {
// Non-fatal — section stays hidden
}
})();
// Affinity debug panel — show if debugAffinity is enabled
(function loadAffinityDebug() {
var show = (window.CLIENT_CONFIG && window.CLIENT_CONFIG.debugAffinity) || localStorage.getItem('meshcore-affinity-debug') === 'true';
@@ -777,6 +815,22 @@
let _themeRefreshHandler = null;
let _allNodes = null; // cached full node list
let _fleetSkew = null; // cached clock skew map: pubkey → {severity, medianSkewSec, ...}
/** Fetch fleet clock skew once, return map keyed by pubkey */
async function getFleetSkew() {
if (_fleetSkew) return _fleetSkew;
try {
const data = await api('/nodes/clock-skew', { ttl: 30000 });
_fleetSkew = {};
(Array.isArray(data) ? data : []).forEach(function(cs) {
if (cs && cs.pubkey) _fleetSkew[cs.pubkey] = cs;
});
} catch (e) {
_fleetSkew = {};
}
return _fleetSkew;
}
// Build a map of lowercased name → count of distinct pubkeys sharing that name
function buildDupNameMap(allNodes) {
@@ -806,7 +860,10 @@
const params = new URLSearchParams({ limit: '5000' });
const rp = RegionFilter.getRegionParam();
if (rp) params.set('region', rp);
const data = await api('/nodes?' + params, { ttl: CLIENT_TTL.nodeList });
const [data] = await Promise.all([
api('/nodes?' + params, { ttl: CLIENT_TTL.nodeList }),
getFleetSkew() // pre-fetch clock skew in parallel
]);
_allNodes = data.nodes || [];
counts = data.counts || {};
}
@@ -979,6 +1036,7 @@
panel.classList.add('empty');
panel.innerHTML = '<span>Select a node to view details</span>';
selectedKey = null;
history.replaceState(null, '', '#/nodes');
renderRows();
}
}
@@ -986,11 +1044,31 @@
// #630: Close button for node detail panel (important for mobile full-screen overlay)
document.getElementById('nodesRight').addEventListener('click', function(e) {
// #778: Details/Analytics links don't navigate because replaceState
// already set the hash to #/nodes/PUBKEY, so clicking <a href="#/nodes/PUBKEY">
// is a same-hash no-op. Force navigation by temporarily clearing the hash.
var link = e.target.closest('a.btn-primary[href^="#/nodes/"]');
if (link) {
e.preventDefault();
var href = link.getAttribute('href');
var pubkey = decodeURIComponent(href.replace('#/nodes/', '').replace('/analytics', ''));
if (href.includes('/analytics')) {
// Navigate to analytics page
history.replaceState(null, '', '#/');
location.hash = '/nodes/' + encodeURIComponent(pubkey) + '/analytics';
} else {
// Show full-screen node detail view
showFullScreenNode(pubkey);
history.replaceState(null, '', '#/nodes/' + encodeURIComponent(pubkey));
}
return;
}
if (e.target.closest('.panel-close-btn')) {
const panel = document.getElementById('nodesRight');
panel.classList.add('empty');
panel.innerHTML = '<span>Select a node to view details</span>';
selectedKey = null;
history.replaceState(null, '', '#/nodes');
renderRows();
}
});
@@ -1029,8 +1107,10 @@
const lastSeenTime = n.last_heard || n.last_seen;
const status = getNodeStatus(n.role || 'companion', lastSeenTime ? new Date(lastSeenTime).getTime() : 0);
const lastSeenClass = status === 'active' ? 'last-seen-active' : 'last-seen-stale';
const cs = _fleetSkew && _fleetSkew[n.public_key];
const skewBadgeHtml = cs && cs.severity && cs.severity !== 'ok' ? renderSkewBadge(cs.severity, cs.medianSkewSec) : '';
return `<tr data-key="${n.public_key}" data-action="select" data-value="${n.public_key}" tabindex="0" role="row" class="${selectedKey === n.public_key ? 'selected' : ''}${isClaimed ? ' claimed-row' : ''}">
<td>${favStar(n.public_key, 'node-fav')}${isClaimed ? '<span class="claimed-badge" title="My Mesh">★</span> ' : ''}<strong>${n.name || '(unnamed)'}</strong>${dupNameBadge(n.name, n.public_key, dupMap)}</td>
<td>${favStar(n.public_key, 'node-fav')}${isClaimed ? '<span class="claimed-badge" title="My Mesh">★</span> ' : ''}<strong>${n.name || '(unnamed)'}</strong>${dupNameBadge(n.name, n.public_key, dupMap)}${skewBadgeHtml}</td>
<td class="mono col-pubkey">${truncate(n.public_key, 16)}</td>
<td><span class="badge" style="background:${roleColor}20;color:${roleColor}">${n.role}</span></td>
<td class="${lastSeenClass}">${renderNodeTimestampHtml(n.last_heard || n.last_seen)}</td>
@@ -1048,6 +1128,7 @@
return;
}
selectedKey = pubkey;
history.replaceState(null, '', '#/nodes/' + encodeURIComponent(pubkey));
renderRows();
const panel = document.getElementById('nodesRight');
panel.classList.remove('empty');
+60 -11
View File
@@ -45,6 +45,10 @@
var parts = [];
if (timeWindowMin && timeWindowMin !== DEFAULT_TIME_WINDOW) parts.push('timeWindow=' + timeWindowMin);
if (regionParam) parts.push('region=' + encodeURIComponent(regionParam));
if (filters.hash) parts.push('hash=' + encodeURIComponent(filters.hash));
if (filters.node) parts.push('node=' + encodeURIComponent(filters.node));
if (filters.observer) parts.push('observer=' + encodeURIComponent(filters.observer));
if (filters._filterExpr) parts.push('filter=' + encodeURIComponent(filters._filterExpr));
return parts.length ? '?' + parts.join('&') : '';
}
window.buildPacketsQuery = buildPacketsQuery;
@@ -342,6 +346,14 @@
}
var _urlRegion = _initUrlParams.get('region');
if (_urlRegion) _pendingUrlRegion = _urlRegion;
var _urlHash = _initUrlParams.get('hash');
if (_urlHash) filters.hash = _urlHash;
var _urlNode = _initUrlParams.get('node');
if (_urlNode) { filters.node = _urlNode; filters.nodeName = _urlNode.slice(0, 8); }
var _urlObserver = _initUrlParams.get('observer');
if (_urlObserver) filters.observer = _urlObserver;
var _urlFilterExpr = _initUrlParams.get('filter');
if (_urlFilterExpr) filters._filterExpr = _urlFilterExpr;
app.innerHTML = `<div class="split-layout detail-collapsed">
<div class="panel-left" id="pktLeft" aria-live="polite" aria-relevant="additions removals"></div>
@@ -797,6 +809,12 @@
var pfError = document.getElementById('packetFilterError');
var pfCount = document.getElementById('packetFilterCount');
if (!pfInput || !window.PacketFilter) return;
// Restore Wireshark filter expression from URL
if (filters._filterExpr) {
pfInput.value = filters._filterExpr;
var _restored = PacketFilter.compile(filters._filterExpr);
if (!_restored.error) { pfInput.classList.add('filter-active'); filters._packetFilter = _restored.filter; }
}
var pfTimer = null;
pfInput.addEventListener('input', function() {
clearTimeout(pfTimer);
@@ -807,6 +825,8 @@
pfError.style.display = 'none';
pfCount.style.display = 'none';
filters._packetFilter = null;
filters._filterExpr = undefined;
updatePacketsUrl();
renderTableRows();
return;
}
@@ -818,12 +838,16 @@
pfError.style.display = 'block';
pfCount.style.display = 'none';
filters._packetFilter = null;
filters._filterExpr = undefined;
updatePacketsUrl();
renderTableRows();
} else {
pfInput.classList.remove('filter-error');
pfInput.classList.add('filter-active');
pfError.style.display = 'none';
filters._packetFilter = compiled.filter;
filters._filterExpr = expr;
updatePacketsUrl();
renderTableRows();
}
}, 300);
@@ -868,6 +892,7 @@
if (filters.observer) localStorage.setItem('meshcore-observer-filter', filters.observer); else localStorage.removeItem('meshcore-observer-filter');
buildObserverMenu();
updateObsTrigger();
updatePacketsUrl();
renderTableRows();
});
@@ -930,7 +955,7 @@
// Filter event listeners
document.getElementById('fHash').value = filters.hash || '';
document.getElementById('fHash').addEventListener('input', debounce((e) => { filters.hash = e.target.value || undefined; loadPackets(); }, 300));
document.getElementById('fHash').addEventListener('input', debounce((e) => { filters.hash = e.target.value || undefined; updatePacketsUrl(); loadPackets(); }, 300));
// Time window dropdown — restore from localStorage and bind change
const fTimeWindow = document.getElementById('fTimeWindow');
@@ -1065,7 +1090,7 @@
if (!q) {
fNodeDrop.classList.add('hidden');
fNode.setAttribute('aria-expanded', 'false');
if (filters.node) { filters.node = undefined; filters.nodeName = undefined; loadPackets(); }
if (filters.node) { filters.node = undefined; filters.nodeName = undefined; updatePacketsUrl(); loadPackets(); }
return;
}
try {
@@ -1094,6 +1119,7 @@
fNode.setAttribute('aria-expanded', 'false');
fNode.setAttribute('aria-activedescendant', '');
nodeActiveIdx = -1;
updatePacketsUrl();
loadPackets();
}
@@ -1849,7 +1875,12 @@
}
}
const anomalyBanner = decoded.anomaly
? `<div class="anomaly-banner" style="background:var(--warning, #f0ad4e); color:#000; padding:8px 12px; border-radius:4px; margin-bottom:8px; font-weight:600;">⚠️ Anomaly: ${escapeHtml(decoded.anomaly)}</div>`
: '';
panel.innerHTML = `
${anomalyBanner}
<div class="detail-title">${hasRawHex ? `Packet Byte Breakdown (${size} bytes)` : typeName + ' Packet'}</div>
<div class="detail-hash">${pkt.hash || 'Packet #' + pkt.id}</div>
${messageHtml}
@@ -1981,13 +2012,9 @@
// Header section
rows += sectionRow('Header', 'section-header');
rows += fieldRow(0, 'Header Byte', '0x' + (buf.slice(0, 2) || '??'), `Route: ${routeTypeName(pkt.route_type)}, Payload: ${payloadTypeName(pkt.payload_type)}`);
const pathByte0 = parseInt(buf.slice(2, 4), 16);
const hashSizeVal = isNaN(pathByte0) ? '?' : ((pathByte0 >> 6) + 1);
const hashCountVal = isNaN(pathByte0) ? '?' : (pathByte0 & 0x3F);
rows += fieldRow(1, 'Path Length', '0x' + (buf.slice(2, 4) || '??'), hashCountVal === 0 ? `hash_count=0 (direct advert)` : `hash_size=${hashSizeVal} byte${hashSizeVal !== 1 ? 's' : ''}, hash_count=${hashCountVal}`);
// Transport codes
let off = 2;
// Transport codes come BEFORE path length for transport routes (bytes 1-4)
let off = 1;
if (pkt.route_type === 0 || pkt.route_type === 3) {
rows += sectionRow('Transport Codes', 'section-transport');
rows += fieldRow(off, 'Next Hop', buf.slice(off * 2, (off + 2) * 2), '');
@@ -1995,11 +2022,18 @@
off += 4;
}
// Path length byte is at current offset (byte 1 for non-transport, byte 5 for transport)
const pathLenOffset = off;
const pathByte0 = parseInt(buf.slice(off * 2, off * 2 + 2), 16);
const hashSizeVal = isNaN(pathByte0) ? '?' : ((pathByte0 >> 6) + 1);
const hashCountVal = isNaN(pathByte0) ? '?' : (pathByte0 & 0x3F);
rows += fieldRow(off, 'Path Length', '0x' + (buf.slice(off * 2, off * 2 + 2) || '??'), hashCountVal === 0 ? `hash_count=0 (direct advert)` : `hash_size=${hashSizeVal} byte${hashSizeVal !== 1 ? 's' : ''}, hash_count=${hashCountVal}`);
off += 1;
// Path
if (pathHops.length > 0) {
rows += sectionRow('Path (' + pathHops.length + ' hops)', 'section-path');
const pathByte = parseInt(buf.slice(2, 4), 16);
const hashSize = (pathByte >> 6) + 1;
const hashSize = isNaN(pathByte0) ? 1 : ((pathByte0 >> 6) + 1);
for (let i = 0; i < pathHops.length; i++) {
const hopHtml = HopDisplay.renderHop(pathHops[i], hopNameCache[pathHops[i]]);
const label = `Hop ${i}${hopHtml}`;
@@ -2012,7 +2046,7 @@
rows += sectionRow('Payload — ' + payloadTypeName(pkt.payload_type), 'section-payload');
if (decoded.type === 'ADVERT') {
if (hashCountVal !== 0) rows += fieldRow(1, 'Advertised Hash Size', hashSizeVal + ' byte' + (hashSizeVal !== 1 ? 's' : ''), 'From path byte 0x' + (buf.slice(2, 4) || '??') + ' — bits 7-6 = ' + (hashSizeVal - 1));
if (hashCountVal !== 0) rows += fieldRow(pathLenOffset, 'Advertised Hash Size', hashSizeVal + ' byte' + (hashSizeVal !== 1 ? 's' : ''), 'From path byte 0x' + (buf.slice(pathLenOffset * 2, pathLenOffset * 2 + 2) || '??') + ' — bits 7-6 = ' + (hashSizeVal - 1));
rows += fieldRow(off, 'Public Key (32B)', truncate(decoded.pubKey || '', 24), '');
rows += fieldRow(off + 32, 'Timestamp (4B)', decoded.timestampISO || '', 'Unix: ' + (decoded.timestamp || ''));
rows += fieldRow(off + 36, 'Signature (64B)', truncate(decoded.signature || '', 24), '');
@@ -2053,6 +2087,10 @@
rows += fieldRow(off, 'Raw', truncate(buf.slice(off * 2), 40), '');
}
if (decoded.anomaly) {
rows += `<tr class="anomaly-row" style="background:var(--warning, #f0ad4e); color:#000; font-weight:600;"><td colspan="2">⚠️ Anomaly</td><td colspan="2">${escapeHtml(decoded.anomaly)}</td></tr>`;
}
return `<table class="field-table">
<thead><tr><th scope="col">Offset</th><th scope="col">Field</th><th scope="col">Value</th><th scope="col">Description</th></tr></thead>
<tbody>${rows}</tbody>
@@ -2144,6 +2182,11 @@
let html = '<div class="byop-decoded">';
// Anomaly banner
if (d.anomaly) {
html += '<div class="anomaly-banner" style="background:var(--warning, #f0ad4e); color:#000; padding:8px 12px; border-radius:4px; margin-bottom:8px; font-weight:600;">⚠️ Anomaly: ' + escapeHtml(d.anomaly) + '</div>';
}
// Header section
html += '<div class="byop-section">'
+ '<div class="byop-section-title">Header</div>'
@@ -2174,6 +2217,12 @@
html += kv(k, String(v));
}
}
// Special handling for advert signature validation
if (h.payloadType === 4 && p.signatureValid !== undefined) {
const status = p.signatureValid ? 'Valid' : 'Invalid';
const badgeClass = p.signatureValid ? 'badge-success' : 'badge-danger';
html += kv('Signature', `<span class="badge ${badgeClass}">${status}</span>`);
}
html += '</div></div>';
// Raw hex
+2 -1
View File
@@ -91,8 +91,9 @@
const ps = server.packetStore;
html += `<h3>In-Memory Packet Store</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${ps.inMemory.toLocaleString()}</div><div class="perf-label">Packets in RAM</div></div>
<div class="perf-card"><div class="perf-num">${ps.estimatedMB}MB</div><div class="perf-label">Memory Used</div></div>
<div class="perf-card"><div class="perf-num">${ps.trackedMB}MB</div><div class="perf-label">Tracked Memory</div></div>
<div class="perf-card"><div class="perf-num">${ps.maxMB}MB</div><div class="perf-label">Memory Limit</div></div>
<div class="perf-card"><div class="perf-num">${ps.estimatedMB}MB</div><div class="perf-label">Heap (debug)</div></div>
<div class="perf-card"><div class="perf-num">${ps.queries.toLocaleString()}</div><div class="perf-label">Queries Served</div></div>
<div class="perf-card"><div class="perf-num">${ps.inserts.toLocaleString()}</div><div class="perf-label">Live Inserts</div></div>
<div class="perf-card"><div class="perf-num">${ps.evicted.toLocaleString()}</div><div class="perf-label">Evicted</div></div>
+64
View File
@@ -394,4 +394,68 @@
});
return html;
};
// #690 — Clock Skew shared helpers
var SKEW_SEVERITY_COLORS = {
ok: 'var(--status-green)',
warning: 'var(--status-yellow)',
critical: 'var(--status-orange)',
absurd: 'var(--status-purple)',
no_clock: 'var(--text-muted)'
};
var SKEW_SEVERITY_LABELS = {
ok: 'OK', warning: 'Warning', critical: 'Critical', absurd: 'Absurd', no_clock: 'No Clock'
};
var SKEW_SEVERITY_ORDER = { no_clock: 0, absurd: 1, critical: 2, warning: 3, ok: 4 };
window.SKEW_SEVERITY_COLORS = SKEW_SEVERITY_COLORS;
window.SKEW_SEVERITY_LABELS = SKEW_SEVERITY_LABELS;
window.SKEW_SEVERITY_ORDER = SKEW_SEVERITY_ORDER;
/** Format skew seconds into human-readable string like "+2m 34s" or "-15h 22m" */
window.formatSkew = function(sec) {
if (sec == null) return '—';
var abs = Math.abs(sec);
var sign = sec >= 0 ? '+' : '-';
if (abs < 60) return sign + Math.round(abs) + 's';
if (abs < 3600) return sign + Math.floor(abs / 60) + 'm ' + Math.round(abs % 60) + 's';
if (abs < 86400) return sign + Math.floor(abs / 3600) + 'h ' + Math.round((abs % 3600) / 60) + 'm';
return sign + Math.floor(abs / 86400) + 'd ' + Math.round((abs % 86400) / 3600) + 'h';
};
/** Format drift rate as "+X.Xs/day" or "—" if falsy */
window.formatDrift = function(secPerDay) {
if (!secPerDay) return '—';
return (secPerDay >= 0 ? '+' : '') + secPerDay.toFixed(1) + ' s/day';
};
/** Render a clock skew badge HTML */
window.renderSkewBadge = function(severity, skewSec) {
if (!severity) return '';
var cls = 'skew-badge skew-badge--' + severity;
if (severity === 'no_clock') {
return '<span class="' + cls + '" title="Uninitialized RTC — no valid clock">🚫 No Clock</span>';
}
var label = severity === 'ok' ? '⏰' : '⏰ ' + window.formatSkew(skewSec);
return '<span class="' + cls + '" title="Clock skew: ' + window.formatSkew(skewSec) + ' (' + (SKEW_SEVERITY_LABELS[severity] || severity) + ')">' + label + '</span>';
};
/** Render a skew sparkline SVG (inline, word-sized) */
window.renderSkewSparkline = function(samples, w, h) {
w = w || 120; h = h || 24;
if (!samples || samples.length < 2) return '';
var values = samples.map(function(s) { return s.skew; });
var max = Math.max.apply(null, values.map(function(v) { return Math.abs(v); }).concat([1]));
var pts = values.map(function(v, i) {
var x = i * (w / Math.max(values.length - 1, 1));
var y = h / 2 - (v / max) * (h / 2 - 2);
return x.toFixed(1) + ',' + y.toFixed(1);
}).join(' ');
// Zero line
var zeroY = h / 2;
return '<svg viewBox="0 0 ' + w + ' ' + h + '" style="width:' + w + 'px;height:' + h + 'px" role="img" aria-label="Clock skew sparkline">' +
'<title>Clock skew over time</title>' +
'<line x1="0" y1="' + zeroY + '" x2="' + w + '" y2="' + zeroY + '" stroke="var(--border)" stroke-width="0.5" stroke-dasharray="2"/>' +
'<polyline points="' + pts + '" fill="none" stroke="var(--accent)" stroke-width="1.5"/></svg>';
};
})();
+84
View File
@@ -11,6 +11,9 @@
--status-green: #22c55e;
--status-yellow: #eab308;
--status-red: #ef4444;
--status-orange: #f97316;
--status-purple: #a855f7;
--role-observer: #8b5cf6;
--accent-hover: #6db3ff;
--text: #1a1a2e;
--text-muted: #5b6370;
@@ -41,6 +44,8 @@
--status-green: #22c55e;
--status-yellow: #eab308;
--status-red: #ef4444;
--status-orange: #f97316;
--status-purple: #a855f7;
--surface-0: #0f0f23;
--surface-1: #1a1a2e;
--surface-2: #232340;
@@ -65,6 +70,8 @@
--status-green: #22c55e;
--status-yellow: #eab308;
--status-red: #ef4444;
--status-orange: #f97316;
--status-purple: #a855f7;
--surface-0: #0f0f23;
--surface-1: #1a1a2e;
--surface-2: #232340;
@@ -463,6 +470,14 @@ fieldset.mc-section legend.mc-label { padding: 0; }
.ch-sidebar-title {
display: flex; align-items: center; gap: 8px; font-size: 16px; font-weight: 700; margin-bottom: 8px;
}
.ch-encrypted-toggle {
display: flex; align-items: center; gap: 4px; font-size: 11px; color: var(--text-muted);
cursor: pointer; user-select: none; margin-bottom: 4px;
}
.ch-encrypted-toggle input { margin: 0; cursor: pointer; }
.ch-toggle-label { white-space: nowrap; }
.ch-item.ch-encrypted { opacity: 0.55; }
.ch-item.ch-encrypted .ch-item-name { font-style: italic; }
.ch-icon { font-size: 20px; }
.ch-sidebar-controls { display: flex; align-items: center; gap: 6px; }
.ch-region-select {
@@ -495,6 +510,9 @@ button.ch-item.selected { background: var(--selected-bg); }
.ch-item-top { display: flex; justify-content: space-between; align-items: baseline; margin-bottom: 2px; }
.ch-item-name { font-weight: 600; font-size: 14px; }
.ch-item-time { font-size: 11px; color: var(--text-muted); white-space: nowrap; }
.ch-remove-btn { background: none; border: none; color: var(--text-muted); cursor: pointer; font-size: 13px; padding: 0 2px; margin-left: 4px; opacity: 0; transition: opacity 0.15s; line-height: 1; }
button.ch-item:hover .ch-remove-btn { opacity: 0.6; }
.ch-remove-btn:hover { opacity: 1 !important; color: var(--danger, #dc2626); }
.ch-item-preview { font-size: 12px; color: var(--text-muted); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.ch-main { flex: 1; display: flex; flex-direction: column; overflow: hidden; position: relative; }
@@ -1097,6 +1115,49 @@ button.ch-item.ch-item-encrypted:hover { opacity: 0.7; }
button.ch-item.ch-item-encrypted.selected { opacity: 0.8; }
button.ch-item.ch-item-encrypted .ch-badge { filter: grayscale(0.6); }
/* Channel key input (#725 M2) */
.ch-key-input {
flex: 1;
min-width: 0;
box-sizing: border-box;
padding: 6px 8px;
border: 1px solid var(--border);
border-radius: 6px 0 0 6px;
background: var(--card-bg);
color: var(--text);
font-size: 12px;
font-family: inherit;
}
.ch-key-input:focus {
outline: 2px solid var(--accent, #3b82f6);
outline-offset: -1px;
border-color: var(--accent, #3b82f6);
}
.ch-key-input::placeholder { color: var(--text-muted); }
.ch-key-input-wrap { margin-bottom: 4px; }
.ch-wrong-key { color: var(--danger, #ef4444); font-weight: 500; }
/* Add channel form (#759) */
.ch-add-form { margin: 0; }
.ch-add-label { display: block; font-weight: 600; font-size: 13px; color: var(--text); margin-bottom: 4px; }
.ch-key-input, .ch-add-btn { height: 32px; box-sizing: border-box; }
.ch-add-row { display: flex; align-items: stretch; }
.ch-add-btn {
width: 32px; height: 32px; flex-shrink: 0;
border: 1px solid var(--accent, #3b82f6); border-left: none;
border-radius: 0 6px 6px 0;
background: var(--accent, #3b82f6); color: #fff;
font-size: 18px; font-weight: 700; line-height: 1;
cursor: pointer; display: flex; align-items: center; justify-content: center;
}
.ch-add-btn:hover { opacity: 0.85; }
.ch-add-hint { font-size: 11px; color: var(--text-muted); margin-top: 4px; line-height: 1.3; }
.ch-add-status { font-size: 12px; margin-top: 4px; padding: 4px 6px; border-radius: 4px; }
.ch-add-status--loading { color: var(--text-muted); }
.ch-add-status--success { color: var(--success, #22c55e); }
.ch-add-status--warn { color: var(--warning, #eab308); }
.ch-add-status--error { color: var(--danger, #ef4444); }
/* Touch-friendly tappable elements */
.ch-tappable {
cursor: pointer;
@@ -1187,6 +1248,8 @@ button.ch-item.ch-item-encrypted .ch-badge { filter: grayscale(0.6); }
.hash-bar-value { min-width: 120px; text-align: right; font-size: 13px; font-weight: 600; }
.badge-hash-1 { background: #ef444420; color: var(--status-red); }
.badge-hash-2 { background: #22c55e20; color: var(--status-green); }
.badge-success { background: #22c55e20; color: var(--status-green); }
.badge-danger { background: #ef444420; color: var(--status-red); }
.badge-hash-3 { background: #3b82f620; color: var(--accent); }
.timeline-legend { display: flex; gap: 16px; justify-content: center; margin-top: 8px; font-size: 12px; }
.legend-dot { display: inline-block; width: 10px; height: 10px; border-radius: 50%; margin-right: 4px; vertical-align: middle; }
@@ -2209,3 +2272,24 @@ th[data-sort-key] { cursor: pointer; user-select: none; }
th[data-sort-key]:hover { background: var(--hover-bg, rgba(255,255,255,0.05)); }
th.sort-active { color: var(--accent, #60a5fa); }
.sort-arrow { font-size: 0.75em; opacity: 0.8; }
/* #690 — Clock Skew badges & fleet table */
.skew-badge { display: inline-block; font-size: 10px; padding: 1px 5px; border-radius: 3px; margin-left: 4px; font-weight: 600; white-space: nowrap; }
.skew-badge--ok { background: var(--status-green); color: #fff; }
.skew-badge--warning { background: var(--status-yellow); color: #000; }
.skew-badge--critical { background: var(--status-orange); color: #fff; }
.skew-badge--absurd { background: var(--status-purple); color: #fff; }
.skew-badge--no_clock { background: var(--text-muted); color: #fff; }
.skew-detail-section { padding: 10px 16px; margin-bottom: 8px; }
.skew-sparkline-wrap { margin-top: 6px; }
.skew-sparkline-wrap svg { display: block; }
.clock-fleet-row--warning { background: color-mix(in srgb, var(--status-yellow) 10%, transparent); }
.clock-fleet-row--critical { background: color-mix(in srgb, var(--status-orange) 10%, transparent); }
.clock-fleet-row--absurd { background: color-mix(in srgb, var(--status-purple) 10%, transparent); }
.clock-fleet-row--no_clock { background: color-mix(in srgb, var(--text-muted) 10%, transparent); }
.clock-filter-btn { font-size: 12px; padding: 3px 8px; border: 1px solid var(--border); border-radius: 4px; background: var(--card-bg, #fff); color: var(--text); cursor: pointer; margin-right: 4px; }
.clock-filter-btn.active { background: var(--accent); color: #fff; border-color: var(--accent); }
+64
View File
@@ -0,0 +1,64 @@
/**
* Tests for #759 Add channel UX: button, hint, status feedback.
* Validates the HTML structure rendered by channels.js init.
*/
'use strict';
const fs = require('fs');
let passed = 0;
let failed = 0;
function assert(cond, msg) {
if (cond) { passed++; console.log(' ✓ ' + msg); }
else { failed++; console.error(' ✗ ' + msg); }
}
function assertIncludes(html, substr, msg) {
assert(html.includes(substr), msg);
}
// Read the channels.js source to extract the HTML template
const src = fs.readFileSync(__dirname + '/public/channels.js', 'utf8');
// Extract the sidebar HTML from the template literal
const htmlMatch = src.match(/app\.innerHTML\s*=\s*`([\s\S]*?)`;/);
const html = htmlMatch ? htmlMatch[1] : '';
console.log('Test: Add channel UX (#759)');
// 1. Button renders in the form
assertIncludes(html, 'class="ch-add-btn"', 'Add button has ch-add-btn class');
assertIncludes(html, 'type="submit"', 'Button is type=submit');
assertIncludes(html, '>+</button>', 'Button shows + text');
// 2. Form has proper structure
assertIncludes(html, 'class="ch-add-form"', 'Form has ch-add-form class');
assertIncludes(html, 'class="ch-add-row"', 'Row wrapper present');
assert(!html.includes('class="ch-add-label"'), 'Label removed (redundant with hint)');
// 3. Hint text present
assertIncludes(html, 'class="ch-add-hint"', 'Hint div present');
assertIncludes(html, 'e.g. #LongFast or 32-char hex key', 'Hint text correct');
// 4. Status div present
assertIncludes(html, 'id="chAddStatus"', 'Status div has correct id');
assertIncludes(html, 'class="ch-add-status"', 'Status div has correct class');
assertIncludes(html, 'style="display:none"', 'Status div hidden by default');
// 5. showAddStatus function exists in source
assert(src.includes('function showAddStatus('), 'showAddStatus function defined');
assert(src.includes("'success'"), 'Success status type referenced');
assert(src.includes("'error'"), 'Error status type referenced');
// 6. CSS classes exist
const css = fs.readFileSync(__dirname + '/public/style.css', 'utf8');
assert(css.includes('.ch-add-form'), 'CSS: .ch-add-form defined');
assert(css.includes('.ch-add-btn'), 'CSS: .ch-add-btn defined');
assert(css.includes('.ch-add-hint'), 'CSS: .ch-add-hint defined');
assert(css.includes('.ch-add-status'), 'CSS: .ch-add-status defined');
assert(css.includes('.ch-add-row'), 'CSS: .ch-add-row defined');
// .ch-add-label CSS kept for backward compat but label removed from HTML
console.log('\n' + passed + ' passed, ' + failed + ' failed');
process.exit(failed > 0 ? 1 : 0);
+157
View File
@@ -0,0 +1,157 @@
/**
* Tests for #725 M3 (PSK hex key), M4 (channel removal), M5 (message caching).
* Runs in Node.js via vm.createContext to simulate browser environment.
*/
'use strict';
const vm = require('vm');
const fs = require('fs');
const { subtle } = require('crypto').webcrypto;
let passed = 0;
let failed = 0;
function assert(cond, msg) {
if (cond) { passed++; console.log(' ✓ ' + msg); }
else { failed++; console.error(' ✗ ' + msg); }
}
// Build a minimal browser-like sandbox
function createSandbox() {
const storage = {};
const localStorage = {
getItem: (k) => storage[k] !== undefined ? storage[k] : null,
setItem: (k, v) => { storage[k] = String(v); },
removeItem: (k) => { delete storage[k]; },
_data: storage
};
const ctx = {
window: {},
crypto: { subtle },
TextEncoder: TextEncoder,
TextDecoder: TextDecoder,
Uint8Array,
localStorage,
console,
Date,
JSON,
parseInt,
Math,
String,
Number,
Object,
Array,
RegExp,
Error,
Promise,
setTimeout,
btoa: (s) => Buffer.from(s, 'binary').toString('base64'),
atob: (s) => Buffer.from(s, 'base64').toString('binary'),
};
ctx.window = ctx;
ctx.self = ctx;
return ctx;
}
async function runTests() {
console.log('\n=== M3: PSK hex key detection ===');
// Load channel-decrypt.js in sandbox
const cdSrc = fs.readFileSync(__dirname + '/public/channel-decrypt.js', 'utf8');
const sandbox = createSandbox();
const context = vm.createContext(sandbox);
vm.runInContext(cdSrc, context);
const CD = sandbox.window.ChannelDecrypt;
// Test: isHexKey detection (via channels.js logic)
// We test the pattern directly since isHexKey is inside channels.js IIFE
const isHexKey = (val) => /^[0-9a-fA-F]{32}$/.test(val);
assert(isHexKey('0123456789abcdef0123456789abcdef'), 'Valid 32-char hex detected');
assert(isHexKey('AABBCCDD11223344AABBCCDD11223344'), 'Valid uppercase hex detected');
assert(!isHexKey('#LongFast'), 'Hashtag name NOT detected as hex');
assert(!isHexKey('0123456789abcdef'), 'Short hex (16 chars) NOT detected');
assert(!isHexKey('0123456789abcdef0123456789abcdefXX'), 'Too long NOT detected');
assert(!isHexKey('zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'), 'Non-hex chars NOT detected');
// Test: PSK decrypt with known key bytes
console.log('\n=== M3: PSK decrypt produces correct plaintext ===');
// Derive a key from #LongFast for testing
const keyBytes = await CD.deriveKey('#LongFast');
assert(keyBytes.length === 16, 'Derived key is 16 bytes');
const keyHex = CD.bytesToHex(keyBytes);
assert(keyHex.length === 32, 'Key hex is 32 chars');
// Round-trip: hex → bytes → hex
const roundTrip = CD.bytesToHex(CD.hexToBytes(keyHex));
assert(roundTrip === keyHex, 'Hex round-trip preserves key');
// Channel hash computation works
const hashByte = await CD.computeChannelHash(keyBytes);
assert(typeof hashByte === 'number' && hashByte >= 0 && hashByte <= 255, 'Channel hash byte is valid (0-255)');
// PSK key (raw hex) stored and retrieved correctly
const pskHex = 'aabbccdd11223344aabbccdd11223344';
CD.storeKey('psk:aabbccdd', pskHex);
const keys = CD.getStoredKeys();
assert(keys['psk:aabbccdd'] === pskHex, 'PSK key stored and retrieved correctly');
console.log('\n=== M4: Channel removal clears key + cache ===');
// Store a key and some cached messages
CD.storeKey('#TestChannel', 'deadbeefdeadbeefdeadbeefdeadbeef');
CD.setCache('#TestChannel', [{ sender: 'A', text: 'hello', timestamp: '2026-01-01T00:00:00Z', packetHash: 'h1' }], '2026-01-01T00:00:00Z', 1);
// Verify they exist
var storedKeys = CD.getStoredKeys();
assert(storedKeys['#TestChannel'] === 'deadbeefdeadbeefdeadbeefdeadbeef', 'Key exists before removal');
var cachedBefore = CD.getCache('#TestChannel');
assert(cachedBefore && cachedBefore.messages.length === 1, 'Cache exists before removal');
// Remove the key (also clears cache)
CD.removeKey('#TestChannel');
var storedAfter = CD.getStoredKeys();
assert(!storedAfter['#TestChannel'], 'Key cleared after removal');
var cachedAfter = CD.getCache('#TestChannel');
assert(!cachedAfter, 'Cache cleared after removal');
console.log('\n=== M5: Cache operations ===');
// Test: setCache with count and size limit
var bigMessages = [];
for (var i = 0; i < 1200; i++) {
bigMessages.push({ sender: 'S', text: 'msg' + i, timestamp: '2026-01-01T00:00:' + String(i).padStart(2, '0') + 'Z', packetHash: 'h' + i });
}
CD.setCache('bigchannel', bigMessages, '2026-01-01T00:20:00Z', 1200);
var bigCached = CD.getCache('bigchannel');
assert(bigCached.messages.length <= 1000, 'Cache enforces 1000 message limit (got ' + bigCached.messages.length + ')');
assert(bigCached.count === 1200, 'Cache stores total count');
assert(bigCached.lastTimestamp === '2026-01-01T00:20:00Z', 'Cache stores lastTimestamp');
// Should keep most recent 1000
assert(bigCached.messages[0].packetHash === 'h200', 'Cache keeps most recent 1000 (first is h200)');
// Test: cache hit (delta fetch scenario)
CD.setCache('deltatest', [
{ sender: 'A', text: 'old', timestamp: '2026-01-01T00:00:00Z', packetHash: 'p1' }
], '2026-01-01T00:00:00Z', 1);
var deltaCache = CD.getCache('deltatest');
assert(deltaCache.messages.length === 1, 'Delta cache has 1 message');
assert(deltaCache.lastTimestamp === '2026-01-01T00:00:00Z', 'Delta cache lastTimestamp correct');
assert(deltaCache.count === 1, 'Delta cache count correct');
// Test: clearChannelCache
CD.setCache('clearthis', [{ sender: 'X', text: 'y' }], 'ts', 1);
assert(CD.getCache('clearthis') !== null, 'Cache exists before clear');
CD.clearChannelCache('clearthis');
assert(CD.getCache('clearthis') === null, 'Cache cleared by clearChannelCache');
console.log('\n=== Results ===');
console.log('Passed: ' + passed + ', Failed: ' + failed);
process.exit(failed > 0 ? 1 : 0);
}
runTests().catch(e => { console.error(e); process.exit(1); });
+454
View File
@@ -0,0 +1,454 @@
/* test-drag-manager.js — Unit tests for DragManager (#608 M1) */
'use strict';
const vm = require('vm');
const fs = require('fs');
const path = require('path');
const assert = require('assert');
// Minimal DOM shim
function makePanel(id) {
const listeners = {};
const style = {};
const dataset = {};
const classList = {
_set: new Set(),
add(c) { this._set.add(c); },
remove(c) { this._set.delete(c); },
contains(c) { return this._set.has(c); }
};
let attrs = {};
const header = {
_listeners: {},
addEventListener(ev, fn) {
if (!this._listeners[ev]) this._listeners[ev] = [];
this._listeners[ev].push(fn);
},
setPointerCapture() {},
releasePointerCapture() {},
_fire(ev, data) {
(this._listeners[ev] || []).forEach(fn => fn(data));
}
};
return {
id: id,
style: style,
dataset: dataset,
classList: classList,
querySelector(sel) {
if (sel === '.panel-header') return header;
return null;
},
getAttribute(k) { return attrs[k] || null; },
setAttribute(k, v) { attrs[k] = v; },
removeAttribute(k) { delete attrs[k]; },
getBoundingClientRect() {
return {
left: parseFloat(style.left) || 0,
top: parseFloat(style.top) || 0,
right: (parseFloat(style.left) || 0) + 300,
bottom: (parseFloat(style.top) || 0) + 200,
width: 300,
height: 200
};
},
_header: header
};
}
// Mock globals
const storage = {};
const mockWindow = {
innerWidth: 1920,
innerHeight: 1080,
DragManager: null,
matchMedia() { return { matches: true, addEventListener() {} }; },
addEventListener() {}
};
const mockDocument = {
addEventListener(ev, fn) {
if (!mockDocument._listeners) mockDocument._listeners = {};
if (!mockDocument._listeners[ev]) mockDocument._listeners[ev] = [];
mockDocument._listeners[ev].push(fn);
},
removeEventListener(ev, fn) {
if (mockDocument._listeners && mockDocument._listeners[ev]) {
mockDocument._listeners[ev] = mockDocument._listeners[ev].filter(f => f !== fn);
}
},
querySelectorAll() { return []; }
};
const mockLocalStorage = {
_data: {},
getItem(k) { return this._data[k] || null; },
setItem(k, v) { this._data[k] = v; },
removeItem(k) { delete this._data[k]; },
clear() { this._data = {}; }
};
// Load DragManager
const src = fs.readFileSync(path.join(__dirname, 'public', 'drag-manager.js'), 'utf8');
const ctx = vm.createContext({
window: mockWindow,
document: mockDocument,
localStorage: mockLocalStorage,
Math: Math,
JSON: JSON,
console: console,
setTimeout: setTimeout,
clearTimeout: clearTimeout,
parseFloat: parseFloat
});
vm.runInContext(src, ctx);
const DragManager = ctx.window.DragManager;
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
mockLocalStorage.clear();
fn();
passed++;
console.log(' ✓ ' + name);
} catch (e) {
failed++;
console.log(' ✗ ' + name + ': ' + e.message);
}
}
console.log('DragManager tests:');
test('constructor initializes IDLE state', () => {
const dm = new DragManager();
assert.strictEqual(dm.state, 'IDLE');
assert.strictEqual(dm.enabled, true);
});
test('register adds panel', () => {
const dm = new DragManager();
const panel = makePanel('testPanel');
dm.register(panel);
assert.strictEqual(dm._panels.length, 1);
});
test('register ignores null panel', () => {
const dm = new DragManager();
dm.register(null);
assert.strictEqual(dm._panels.length, 0);
});
test('pointerdown transitions to PENDING', () => {
const dm = new DragManager();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
assert.strictEqual(dm.state, 'PENDING');
assert.strictEqual(dm.activePanel, panel);
});
test('pointerdown ignores non-left button', () => {
const dm = new DragManager();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 2, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
assert.strictEqual(dm.state, 'IDLE');
});
test('pointerdown ignores button clicks', () => {
const dm = new DragManager();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest(sel) { return sel === 'button' ? {} : null; } }
});
assert.strictEqual(dm.state, 'IDLE');
});
test('pointermove within dead zone stays PENDING', () => {
const dm = new DragManager();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 103, clientY: 102 });
assert.strictEqual(dm.state, 'PENDING');
assert.ok(!panel.classList.contains('is-dragging'));
});
test('pointermove beyond dead zone transitions to DRAGGING', () => {
const dm = new DragManager();
const panel = makePanel('p1');
panel.setAttribute('data-position', 'bl');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 110, clientY: 110 });
assert.strictEqual(dm.state, 'DRAGGING');
assert.ok(panel.classList.contains('is-dragging'));
assert.strictEqual(panel.getAttribute('data-position'), null); // removed
assert.strictEqual(panel.dataset.dragged, 'true');
});
test('pointerup after drag finalizes position', () => {
const dm = new DragManager();
const panel = makePanel('p1');
panel.setAttribute('data-position', 'bl');
ctx.window.innerWidth = 1920;
ctx.window.innerHeight = 1080;
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 200, clientY: 300 });
panel._header._fire('pointerup', { pointerId: 1 });
assert.strictEqual(dm.state, 'IDLE');
assert.ok(!panel.classList.contains('is-dragging'));
// Should have persisted
const saved = JSON.parse(mockLocalStorage.getItem('panel-drag-p1'));
assert.ok(saved.xPct >= 0);
assert.ok(saved.yPct >= 0);
});
test('pointerup from PENDING (click) does not finalize', () => {
const dm = new DragManager();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointerup', { pointerId: 1 });
assert.strictEqual(dm.state, 'IDLE');
assert.strictEqual(mockLocalStorage.getItem('panel-drag-p1'), null);
});
test('disable prevents drag', () => {
const dm = new DragManager();
dm.disable();
const panel = makePanel('p1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
assert.strictEqual(dm.state, 'IDLE');
});
test('snap-to-edge works within threshold', () => {
const dm = new DragManager();
const panel = makePanel('p1');
panel.setAttribute('data-position', 'tl');
ctx.window.innerWidth = 1920;
ctx.window.innerHeight = 1080;
dm.register(panel);
// Simulate drag to near top-left edge
panel._header._fire('pointerdown', {
button: 0, clientX: 500, clientY: 500,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 510, clientY: 510 }); // trigger DRAGGING
// Panel is now detached; set its position near edge
panel.style.left = '5px';
panel.style.top = '10px';
panel._header._fire('pointerup', { pointerId: 1 });
// Should have snapped to margin (12px)
assert.strictEqual(panel.style.left, '12px');
assert.strictEqual(panel.style.top, '12px');
});
test('restorePositions applies saved viewport percentages', () => {
const dm = new DragManager();
const panel = makePanel('p2');
panel.setAttribute('data-position', 'br');
ctx.window.innerWidth = 1000;
ctx.window.innerHeight = 800;
dm.register(panel);
mockLocalStorage.setItem('panel-drag-p2', JSON.stringify({ xPct: 0.5, yPct: 0.25 }));
dm.restorePositions();
assert.strictEqual(panel.style.left, '500px');
assert.strictEqual(panel.style.top, '200px');
assert.strictEqual(panel.dataset.dragged, 'true');
assert.strictEqual(panel.getAttribute('data-position'), null);
});
test('handleResize clamps panels inside viewport', () => {
const dm = new DragManager();
const panel = makePanel('p3');
dm.register(panel);
// Simulate a dragged panel that's now off-screen
panel.dataset.dragged = 'true';
panel.style.left = '1800px';
panel.style.top = '900px';
ctx.window.innerWidth = 1000;
ctx.window.innerHeight = 600;
// Need querySelectorAll to return this panel
const origQSA = mockDocument.querySelectorAll;
mockDocument.querySelectorAll = function (sel) {
if (sel === '.live-overlay[data-dragged="true"]') return [panel];
return [];
};
dm.handleResize();
mockDocument.querySelectorAll = origQSA;
// Should be clamped
const left = parseFloat(panel.style.left);
const top = parseFloat(panel.style.top);
assert.ok(left + 300 <= 1000, 'left clamped: ' + left);
assert.ok(top + 200 <= 600, 'top clamped: ' + top);
});
test('Escape during drag reverts to corner position', () => {
const dm = new DragManager();
const panel = makePanel('esc1');
panel.setAttribute('data-position', 'bl');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 120, clientY: 120 }); // trigger DRAGGING
assert.strictEqual(dm.state, 'DRAGGING');
// Simulate Escape
dm._handleKeyDown({ key: 'Escape' });
assert.strictEqual(dm.state, 'IDLE');
assert.ok(!panel.classList.contains('is-dragging'));
assert.strictEqual(panel.dataset.dragged, undefined);
assert.strictEqual(panel.style.transform, '');
});
test('Escape during drag reverts to saved position', () => {
const dm = new DragManager();
const panel = makePanel('esc2');
ctx.window.innerWidth = 1000;
ctx.window.innerHeight = 800;
dm.register(panel);
// Pre-save a dragged position
mockLocalStorage.setItem('panel-drag-esc2', JSON.stringify({ xPct: 0.3, yPct: 0.4 }));
dm.restorePositions();
assert.strictEqual(panel.dataset.dragged, 'true');
panel._header._fire('pointerdown', {
button: 0, clientX: 400, clientY: 400,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 500, clientY: 500 });
assert.strictEqual(dm.state, 'DRAGGING');
dm._handleKeyDown({ key: 'Escape' });
assert.strictEqual(dm.state, 'IDLE');
assert.strictEqual(panel.style.transform, 'none');
});
test('pointercancel during drag finalizes position', () => {
const dm = new DragManager();
const panel = makePanel('pc1');
panel.setAttribute('data-position', 'tl');
ctx.window.innerWidth = 1920;
ctx.window.innerHeight = 1080;
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 200, clientY: 200 });
assert.strictEqual(dm.state, 'DRAGGING');
panel._header._fire('pointercancel', {});
assert.strictEqual(dm.state, 'IDLE');
assert.ok(!panel.classList.contains('is-dragging'));
});
test('z-index increments on drag', () => {
const dm = new DragManager();
const p1 = makePanel('z1');
const p2 = makePanel('z2');
dm.register(p1);
dm.register(p2);
// Drag p1
p1._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
p1._header._fire('pointermove', { clientX: 110, clientY: 110 });
const z1 = parseInt(p1.style.zIndex);
p1._header._fire('pointerup', { pointerId: 1 });
// Drag p2
p2._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
p2._header._fire('pointermove', { clientX: 110, clientY: 110 });
const z2 = parseInt(p2.style.zIndex);
p2._header._fire('pointerup', { pointerId: 1 });
assert.ok(z2 > z1, 'z2 (' + z2 + ') should be greater than z1 (' + z1 + ')');
assert.ok(z1 >= 1001, 'z1 should be >= 1001');
});
test('disable mid-drag resets state', () => {
const dm = new DragManager();
const panel = makePanel('dis1');
dm.register(panel);
panel._header._fire('pointerdown', {
button: 0, clientX: 100, clientY: 100,
preventDefault() {},
target: { closest() { return null; } }
});
panel._header._fire('pointermove', { clientX: 120, clientY: 120 });
assert.strictEqual(dm.state, 'DRAGGING');
dm.disable();
assert.strictEqual(dm.state, 'IDLE');
assert.ok(!panel.classList.contains('is-dragging'));
assert.strictEqual(dm.enabled, false);
});
console.log('\n' + passed + ' passed, ' + failed + ' failed');
if (failed > 0) process.exit(1);
+90 -4
View File
@@ -231,6 +231,26 @@ async function run() {
assert(hasStatus, 'No status indicator found in node detail');
});
// Test: Node side panel Details link opens full-screen detail view (#778)
await test('Node side panel Details link opens full detail', async () => {
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('table tbody tr');
// Click first row to open side panel
const firstRow = await page.$('table tbody tr');
assert(firstRow, 'No node rows found');
await firstRow.click();
await page.waitForSelector('.node-detail');
// Find the Details link in the side panel
const detailsLink = await page.$('#nodesRight a.btn-primary[href^="#/nodes/"]');
assert(detailsLink, 'Details link not found in side panel');
// Click the Details link — should open full-screen node detail view
await detailsLink.click();
// Wait for the full-screen node detail view to render
await page.waitForSelector('.node-fullscreen', { timeout: 5000 });
const hasFullBody = await page.$('.node-full-body');
assert(hasFullBody, 'Full-screen node detail body not found');
});
// Test: Nodes page has WebSocket auto-update listener (#131)
await test('Nodes page has WebSocket auto-update', async () => {
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded' });
@@ -398,8 +418,13 @@ async function run() {
}
}, { timeout: 10000 });
// Full reload on the packets page — scripts re-execute, IIFE reads localStorage
await page.reload({ waitUntil: 'load' });
// Force a full page reload to reset module-level state (savedTimeWindowMin is
// read from localStorage once at IIFE time). Navigating from /#/packets to /#/packets
// is a hash-only change — no reload, so the IIFE never re-reads localStorage.
// Going to / first forces a fresh page load, then the hash change to /#/packets
// calls init() with the freshly-read savedTimeWindowMin = 60.
await page.goto(`${BASE}/`, { waitUntil: 'load' });
await page.goto(`${BASE}/#/packets`, { waitUntil: 'load' });
await page.waitForSelector('#fTimeWindow', { timeout: 10000 });
const timeWindowValue = await page.$eval('#fTimeWindow', (el) => el.value);
assert(timeWindowValue === '60', `Expected time window dropdown to restore 60, got ${timeWindowValue}`);
@@ -1470,13 +1495,16 @@ async function run() {
// ─── Neighbor section tests ───────────────────────────────────────────────
await test('Node detail: neighbors section exists with correct columns', async () => {
// Full-screen node view (with #node-neighbors) is mobile-only since #676 fix.
await page.setViewportSize({ width: 390, height: 844 });
// Navigate to a node detail page (use the first node in the list)
await page.goto(BASE + '/#/nodes');
await page.waitForSelector('#nodesBody tr[data-key]', { timeout: 10000 });
// Get the first node's pubkey from the row's data-key attribute
const pubkey = await page.$eval('#nodesBody tr[data-key]', el => el.dataset.key);
await page.goto(BASE + '/#/nodes/' + pubkey);
await page.waitForSelector('#node-neighbors', { timeout: 10000 });
// Use evaluate to change hash (reliable same-document navigation)
await page.evaluate((pk) => { location.hash = '#/nodes/' + pk; }, pubkey);
await page.waitForSelector('#node-neighbors', { timeout: 15000 });
// Check the section exists
const header = await page.$eval('#fullNeighborsHeader', el => el.textContent);
assert(header.startsWith('Neighbors'), 'Header should start with "Neighbors", got: ' + header);
@@ -1500,6 +1528,7 @@ async function run() {
const text = await page.$eval('#fullNeighborsContent', el => el.textContent);
assert(text.includes('No neighbor data') || text.includes('Could not load'), 'Should show empty or error state');
}
await page.setViewportSize({ width: 1280, height: 800 });
});
@@ -1653,6 +1682,33 @@ async function run() {
assert(url.includes('tab=room'), `URL should contain tab=room after click, got: ${url}`);
});
// Test: clicking a node on desktop updates URL hash (#676)
await test('Desktop: clicking a node updates URL to #/nodes/{pubkey}', async () => {
await page.setViewportSize({ width: 1280, height: 800 });
await page.goto(BASE + '#/nodes', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#nodesBody tr[data-key]', { timeout: 10000 });
const pubkey = await page.$eval('#nodesBody tr[data-key]', el => el.dataset.key);
await page.click('#nodesBody tr[data-key]');
await page.waitForTimeout(300);
const url = page.url();
assert(url.includes(encodeURIComponent(pubkey)), `URL should contain pubkey after click, got: ${url}`);
assert(!url.includes('node-fullscreen') || await page.$('#nodesRight:not(.empty)'), 'Split panel should be visible on desktop');
});
// Test: loading #/nodes/{pubkey} on desktop shows split panel (#676)
await test('Desktop: deep link #/nodes/{pubkey} opens split panel, not full-screen', async () => {
await page.setViewportSize({ width: 1280, height: 800 });
await page.goto(BASE + '#/nodes', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#nodesBody tr[data-key]', { timeout: 10000 });
const pubkey = await page.$eval('#nodesBody tr[data-key]', el => el.dataset.key);
await page.goto(BASE + '#/nodes/' + encodeURIComponent(pubkey), { waitUntil: 'domcontentloaded' });
await page.waitForTimeout(500);
const hasSplitPanel = await page.$('#nodesRight:not(.empty)');
const hasFullScreen = await page.$('.node-fullscreen');
assert(hasSplitPanel, 'Split panel should be open on desktop deep link');
assert(!hasFullScreen, 'Full-screen view should NOT appear on desktop deep link');
});
// Test: packets timeWindow deep link
await test('Packets timeWindow deep link restores dropdown', async () => {
await page.goto(BASE + '#/packets?timeWindow=60', { waitUntil: 'domcontentloaded' });
@@ -1663,6 +1719,36 @@ async function run() {
assert(url.includes('timeWindow=60'), `URL should still contain timeWindow=60, got: ${url}`);
});
// Test: hash filter updates URL and is restored (#682)
await test('Packets hash filter updates URL and restores on reload', async () => {
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fHash', { timeout: 8000 });
await page.fill('#fHash', 'abc123');
await page.waitForTimeout(500);
const url = page.url();
assert(url.includes('hash=abc123'), `URL should contain hash=abc123, got: ${url}`);
// Reload and check input restored
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#fHash', { timeout: 8000 });
const val = await page.$eval('#fHash', el => el.value);
assert(val === 'abc123', `fHash should be restored to abc123, got: ${val}`);
});
// Test: Wireshark filter expression updates URL and is restored (#682)
await test('Packets filter expression updates URL and restores on reload', async () => {
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#packetFilterInput', { timeout: 8000 });
await page.fill('#packetFilterInput', 'type == ADVERT');
await page.waitForTimeout(500);
const url = page.url();
assert(url.includes('filter=') && url.includes('ADVERT'), `URL should contain filter=type%3D%3DADVERT, got: ${url}`);
// Reload and check expression restored
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#packetFilterInput', { timeout: 8000 });
const val = await page.$eval('#packetFilterInput', el => el.value);
assert(val === 'type == ADVERT', `packetFilterInput should be restored, got: ${val}`);
});
// Test: timeWindow change updates URL
await test('Packets timeWindow change updates URL', async () => {
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
Binary file not shown.
+671 -1
View File
@@ -2345,7 +2345,9 @@ console.log('\n=== channels.js: shouldProcessWSMessageForRegion ===');
ctx.history = { replaceState() {} };
ctx.btoa = (s) => Buffer.from(String(s), 'utf8').toString('base64');
ctx.atob = (s) => Buffer.from(String(s), 'base64').toString('utf8');
loadInCtx(ctx, 'public/channels.js');
ctx.crypto = { subtle: require('crypto').webcrypto.subtle }; ctx.TextEncoder = TextEncoder; ctx.TextDecoder = TextDecoder; ctx.Uint8Array = Uint8Array;
loadInCtx(ctx, 'public/channel-decrypt.js');
loadInCtx(ctx, 'public/channels.js');
const shouldProcess = ctx.window._channelsShouldProcessWSMessageForRegion;
test('helper is exported', () => assert.ok(typeof shouldProcess === 'function'));
@@ -2467,6 +2469,8 @@ console.log('\n=== channels.js: WS batch + region snapshot integration ===');
ctx.btoa = (s) => Buffer.from(String(s), 'utf8').toString('base64');
ctx.atob = (s) => Buffer.from(String(s), 'base64').toString('utf8');
ctx.crypto = { subtle: require('crypto').webcrypto.subtle }; ctx.TextEncoder = TextEncoder; ctx.TextDecoder = TextDecoder; ctx.Uint8Array = Uint8Array;
loadInCtx(ctx, 'public/channel-decrypt.js');
loadInCtx(ctx, 'public/channels.js');
ctx._pageHandlers.init(appEl);
return { ctx, dom };
@@ -2586,6 +2590,8 @@ console.log('\n=== channels.js: WS batch + region snapshot integration ===');
ctx.btoa = (s) => Buffer.from(String(s), 'utf8').toString('base64');
ctx.atob = (s) => Buffer.from(String(s), 'base64').toString('utf8');
ctx.crypto = { subtle: require('crypto').webcrypto.subtle }; ctx.TextEncoder = TextEncoder; ctx.TextDecoder = TextDecoder; ctx.Uint8Array = Uint8Array;
loadInCtx(ctx, 'public/channel-decrypt.js');
loadInCtx(ctx, 'public/channels.js');
ctx._pageHandlers.init(appEl);
await Promise.resolve();
@@ -2681,6 +2687,8 @@ console.log('\n=== channels.js: WS batch + region snapshot integration ===');
ctx.btoa = (s) => Buffer.from(String(s), 'utf8').toString('base64');
ctx.atob = (s) => Buffer.from(String(s), 'base64').toString('utf8');
ctx.crypto = { subtle: require('crypto').webcrypto.subtle }; ctx.TextEncoder = TextEncoder; ctx.TextDecoder = TextDecoder; ctx.Uint8Array = Uint8Array;
loadInCtx(ctx, 'public/channel-decrypt.js');
loadInCtx(ctx, 'public/channels.js');
ctx._pageHandlers.init(appEl);
await Promise.resolve();
@@ -4918,6 +4926,668 @@ console.log('\n=== app.js: formatDistance ===');
});
}
// ===== analytics.js: renderMultiByteCapability =====
console.log('\n=== analytics.js: renderMultiByteCapability ===');
{
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
try { loadInCtx(ctx, 'public/analytics.js'); } catch (e) { /* IIFE side-effects ok */ }
const render = ctx.window._analyticsRenderMultiByteCapability;
test('renderMultiByteCapability is exposed', () => assert.ok(render, '_analyticsRenderMultiByteCapability must be exposed'));
if (render) {
test('empty array returns empty string', () => {
assert.strictEqual(render([]), '');
});
test('renders confirmed status with green indicator', () => {
const html = render([{ pubkey: 'aabb', name: 'RepA', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' }]);
assert.ok(html.includes('✅'), 'should contain confirmed icon');
assert.ok(html.includes('Confirmed'), 'should contain Confirmed label');
assert.ok(html.includes('--success'), 'should use --success CSS var for green');
});
test('renders suspected status with yellow indicator', () => {
const html = render([{ pubkey: 'ccdd', name: 'RepB', role: 'repeater', status: 'suspected', evidence: 'path', maxHashSize: 2, lastSeen: '' }]);
assert.ok(html.includes('⚠️'), 'should contain suspected icon');
assert.ok(html.includes('Suspected'), 'should contain Suspected label');
assert.ok(html.includes('--warning'), 'should use --warning CSS var for yellow');
});
test('renders unknown status with gray indicator', () => {
const html = render([{ pubkey: 'eeff', name: 'RepC', role: 'repeater', status: 'unknown', evidence: '', maxHashSize: 1, lastSeen: '' }]);
assert.ok(html.includes('❓'), 'should contain unknown icon');
assert.ok(html.includes('Unknown'), 'should contain Unknown label');
assert.ok(html.includes('--text-muted'), 'should use --text-muted CSS var for gray');
});
test('renders all three statuses together', () => {
const caps = [
{ pubkey: 'aa11', name: 'R1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 3, lastSeen: '' },
{ pubkey: 'bb22', name: 'R2', role: 'repeater', status: 'suspected', evidence: 'path', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'cc33', name: 'R3', role: 'repeater', status: 'unknown', evidence: '', maxHashSize: 1, lastSeen: '' },
];
const html = render(caps);
assert.ok(html.includes('R1'), 'should contain R1');
assert.ok(html.includes('R2'), 'should contain R2');
assert.ok(html.includes('R3'), 'should contain R3');
assert.ok(html.includes('3-byte'), 'should show 3-byte badge');
assert.ok(html.includes('2-byte'), 'should show 2-byte badge');
assert.ok(html.includes('1-byte'), 'should show 1-byte badge');
});
test('filter buttons show correct counts', () => {
const caps = [
{ pubkey: 'a1', name: 'C1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'a2', name: 'C2', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'b1', name: 'S1', role: 'repeater', status: 'suspected', evidence: 'path', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'c1', name: 'U1', role: 'repeater', status: 'unknown', evidence: '', maxHashSize: 1, lastSeen: '' },
];
const html = render(caps);
assert.ok(html.includes('All (4)'), 'should show total count 4');
assert.ok(html.includes('Confirmed (2)'), 'should show 2 confirmed');
assert.ok(html.includes('Suspected (1)'), 'should show 1 suspected');
assert.ok(html.includes('Unknown (1)'), 'should show 1 unknown');
});
test('evidence labels map to status display', () => {
const html = render([
{ pubkey: 'a1', name: 'R1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'b1', name: 'R2', role: 'repeater', status: 'suspected', evidence: 'path', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'c1', name: 'R3', role: 'repeater', status: 'unknown', evidence: '', maxHashSize: 1, lastSeen: '' },
]);
assert.ok(html.includes('Confirmed'), 'confirmed status should be shown');
assert.ok(html.includes('Suspected'), 'suspected status should be shown');
assert.ok(html.includes('Unknown'), 'unknown status should be shown');
});
test('table rows link to node detail', () => {
const html = render([{ pubkey: 'aabbccdd', name: 'Rep1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' }]);
assert.ok(html.includes('#/nodes/aabbccdd'), 'row should link to node detail page');
});
test('node names are HTML-escaped', () => {
const html = render([{ pubkey: 'x1', name: '<script>alert(1)</script>', role: 'repeater', status: 'unknown', evidence: '', maxHashSize: 1, lastSeen: '' }]);
assert.ok(!html.includes('<script>'), 'should escape HTML in name');
});
test('table has sortable column headers', () => {
const html = render([{ pubkey: 'a1', name: 'R1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' }]);
assert.ok(html.includes('data-sort="status"'), 'status column should be sortable');
assert.ok(html.includes('data-sort="name"'), 'name column should be sortable');
});
}
}
// ===== analytics.js: renderMultiByteAdopters (integrated) =====
console.log('\n=== analytics.js: renderMultiByteAdopters ===');
{
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
try { loadInCtx(ctx, 'public/analytics.js'); } catch (e) { /* IIFE side-effects ok */ }
const renderAdopters = ctx.window._analyticsRenderMultiByteAdopters;
test('renderMultiByteAdopters is exposed', () => assert.ok(renderAdopters, '_analyticsRenderMultiByteAdopters must be exposed'));
if (renderAdopters) {
test('empty nodes returns no-adopters message', () => {
const html = renderAdopters([], []);
assert.ok(html.includes('No multi-byte adopters found'), 'should show empty message');
});
test('integrates capability status into adopter rows', () => {
const nodes = [
{ name: 'NodeA', pubkey: 'aa11', role: 'repeater', hashSize: 2, packets: 5, lastSeen: '2026-01-01T00:00:00Z' },
];
const caps = [
{ pubkey: 'aa11', name: 'NodeA', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' },
];
const html = renderAdopters(nodes, caps);
assert.ok(html.includes('✅'), 'should show confirmed icon');
assert.ok(html.includes('Confirmed'), 'should show Confirmed label');
assert.ok(html.includes('2-byte'), 'should show hash size badge');
});
test('filter buttons have text labels with counts', () => {
const nodes = [
{ name: 'N1', pubkey: 'a1', role: 'repeater', hashSize: 2, packets: 3, lastSeen: '' },
{ name: 'N2', pubkey: 'b1', role: 'repeater', hashSize: 2, packets: 1, lastSeen: '' },
];
const caps = [
{ pubkey: 'a1', name: 'N1', role: 'repeater', status: 'confirmed', evidence: 'advert', maxHashSize: 2, lastSeen: '' },
{ pubkey: 'b1', name: 'N2', role: 'repeater', status: 'suspected', evidence: 'path', maxHashSize: 2, lastSeen: '' },
];
const html = renderAdopters(nodes, caps);
assert.ok(html.includes('Confirmed (1)'), 'should show "Confirmed (1)"');
assert.ok(html.includes('Suspected (1)'), 'should show "Suspected (1)"');
assert.ok(html.includes('Unknown (0)'), 'should show "Unknown (0)"');
assert.ok(html.includes('All (2)'), 'should show total "All (2)"');
});
test('nodes without capability data default to unknown', () => {
const nodes = [
{ name: 'Orphan', pubkey: 'zz99', role: 'repeater', hashSize: 2, packets: 1, lastSeen: '' },
];
const html = renderAdopters(nodes, []); // no caps
assert.ok(html.includes('❓'), 'should show unknown icon');
assert.ok(html.includes('Unknown'), 'should show Unknown label');
});
test('integrated table has Status column', () => {
const nodes = [
{ name: 'R1', pubkey: 'a1', role: 'repeater', hashSize: 2, packets: 1, lastSeen: '' },
];
const html = renderAdopters(nodes, []);
assert.ok(html.includes('Status'), 'should have Status column header');
assert.ok(html.includes('data-sort="status"'), 'Status should be sortable');
});
}
}
// ===== packets.js: anomaly banner rendering =====
console.log('\n=== packets.js: anomaly UI rendering ===');
{
const packetsSource = fs.readFileSync('public/packets.js', 'utf8');
test('renderDetail shows anomaly banner when decoded.anomaly is set', () => {
assert.ok(packetsSource.includes('anomaly-banner'),
'packets.js should contain anomaly-banner class');
assert.ok(packetsSource.includes("decoded.anomaly"),
'packets.js should reference decoded.anomaly');
});
test('buildFieldTable includes anomaly row when present', () => {
assert.ok(packetsSource.includes('anomaly-row'),
'buildFieldTable should have anomaly-row class for highlighted row');
});
test('renderDecodedPacket shows anomaly banner', () => {
assert.ok(packetsSource.includes("d.anomaly"),
'renderDecodedPacket should check d.anomaly');
});
}
// ===== packets.js: buildFieldTable transport offset tests (#765) =====
console.log('\n=== packets.js: buildFieldTable transport offsets (#765) ===');
{
const ftCtx = makeSandbox();
ftCtx.registerPage = () => {};
ftCtx.onWS = () => {};
ftCtx.offWS = () => {};
ftCtx.api = () => Promise.resolve({});
ftCtx.window.getParsedPath = () => [];
ftCtx.window.getParsedDecoded = () => ({});
// Provide globals from app.js that packets.js depends on
const ROUTE_TYPES = {0:'TRANSPORT_FLOOD',1:'FLOOD',2:'DIRECT',3:'TRANSPORT_DIRECT'};
const PAYLOAD_TYPES = {0:'ADVERT',1:'TXT_MSG',2:'GRP_TXT',3:'REQ',4:'ACK'};
ftCtx.routeTypeName = (n) => ROUTE_TYPES[n] || 'UNKNOWN';
ftCtx.payloadTypeName = (n) => PAYLOAD_TYPES[n] || 'UNKNOWN';
ftCtx.window.routeTypeName = ftCtx.routeTypeName;
ftCtx.window.payloadTypeName = ftCtx.payloadTypeName;
ftCtx.truncate = (str, len) => str && str.length > len ? str.slice(0, len) + '…' : (str || '');
ftCtx.window.truncate = ftCtx.truncate;
ftCtx.escapeHtml = (s) => String(s || '').replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
ftCtx.window.escapeHtml = ftCtx.escapeHtml;
loadInCtx(ftCtx, 'public/packets.js');
const { buildFieldTable, fieldRow } = ftCtx.window._packetsTestAPI;
// Helper: build a hex string with specific bytes
function makeHex(bytes) { return bytes.map(b => b.toString(16).padStart(2, '0')).join(''); }
test('FLOOD (route_type=1): path_length at byte 1, no transport codes', () => {
// header=0x05 (route_type=1, payload=1), path_length=0x41 (hash_size=2, count=1), hop=AABB
const raw = makeHex([0x05, 0x41, 0xAA, 0xBB]);
const pkt = { raw_hex: raw, route_type: 1, payload_type: 1 };
const html = buildFieldTable(pkt, {}, [], {});
// Path Length should be at offset 1
assert.ok(html.includes('>1<') || html.includes('data-offset="1"'),
'FLOOD: Path Length row should reference byte offset 1');
// Should NOT contain transport codes
assert.ok(!html.includes('Next Hop'), 'FLOOD: should not show Next Hop transport');
assert.ok(!html.includes('Last Hop'), 'FLOOD: should not show Last Hop transport');
});
test('TRANSPORT_FLOOD (route_type=0): transport codes at bytes 1-4, path_length at byte 5', () => {
// header=0x04 (route_type=0, payload=1), next_hop=1122, last_hop=3344, path_length=0x41
const raw = makeHex([0x04, 0x11, 0x22, 0x33, 0x44, 0x41, 0xAA, 0xBB]);
const pkt = { raw_hex: raw, route_type: 0, payload_type: 1 };
const html = buildFieldTable(pkt, {}, [], {});
// Transport codes should appear
assert.ok(html.includes('Next Hop'), 'TRANSPORT_FLOOD: should show Next Hop');
assert.ok(html.includes('Last Hop'), 'TRANSPORT_FLOOD: should show Last Hop');
// Path Length should be at offset 5, not 1
// Check that Path Length row does NOT show offset 1
const pathLenMatch = html.match(/Path Length/);
assert.ok(pathLenMatch, 'TRANSPORT_FLOOD: should have Path Length row');
// The field table renders offset in first <td>. Check transport codes come before path length
const nextHopIdx = html.indexOf('Next Hop');
const pathLenIdx = html.indexOf('Path Length');
assert.ok(nextHopIdx < pathLenIdx,
'TRANSPORT_FLOOD: transport codes should appear before Path Length in table order');
});
test('TRANSPORT_DIRECT (route_type=3): same offsets as TRANSPORT_FLOOD', () => {
const raw = makeHex([0x0F, 0x11, 0x22, 0x33, 0x44, 0x41]);
const pkt = { raw_hex: raw, route_type: 3, payload_type: 3 };
const html = buildFieldTable(pkt, {}, [], {});
assert.ok(html.includes('Next Hop'), 'TRANSPORT_DIRECT: should show Next Hop');
assert.ok(html.includes('Last Hop'), 'TRANSPORT_DIRECT: should show Last Hop');
const nextHopIdx = html.indexOf('Next Hop');
const pathLenIdx = html.indexOf('Path Length');
assert.ok(nextHopIdx < pathLenIdx,
'TRANSPORT_DIRECT: transport codes should appear before Path Length');
});
test('field table row order matches byte layout for transport routes', () => {
const raw = makeHex([0x04, 0x11, 0x22, 0x33, 0x44, 0x41, 0xAA, 0xBB]);
const pkt = { raw_hex: raw, route_type: 0, payload_type: 1 };
const html = buildFieldTable(pkt, {}, [], {});
// Order: Header (0) → Next Hop (1) → Last Hop (3) → Path Length (5)
const headerIdx = html.indexOf('Header Byte');
const nextHopIdx = html.indexOf('Next Hop');
const lastHopIdx = html.indexOf('Last Hop');
const pathLenIdx = html.indexOf('Path Length');
assert.ok(headerIdx < nextHopIdx, 'Header should come before Next Hop');
assert.ok(nextHopIdx < lastHopIdx, 'Next Hop should come before Last Hop');
assert.ok(lastHopIdx < pathLenIdx, 'Last Hop should come before Path Length');
});
}
// ===== live.js: anomaly icon in feed =====
console.log('\n=== live.js: anomaly icon in feed ===');
{
const liveSource = fs.readFileSync('public/live.js', 'utf8');
test('addFeedItemDOM shows anomaly icon when decoded has anomaly', () => {
assert.ok(liveSource.includes('anomalyIcon'),
'live.js should have anomalyIcon variable for feed items');
assert.ok(liveSource.includes('pkt.decoded && pkt.decoded.anomaly'),
'live.js should check pkt.decoded.anomaly');
});
}
// ===== channel-decrypt.js: client-side crypto =====
console.log('\n=== channel-decrypt.js: key derivation, MAC, parsing, storage ===');
{
const cryptoModule = require('crypto');
const ctx = makeSandbox();
// Provide Web Crypto API in sandbox
ctx.crypto = { subtle: cryptoModule.webcrypto.subtle };
ctx.TextEncoder = TextEncoder;
ctx.TextDecoder = TextDecoder;
ctx.Uint8Array = Uint8Array;
loadInCtx(ctx, 'public/channel-decrypt.js');
const CD = ctx.ChannelDecrypt;
test('deriveKey: SHA256("#test")[:16] matches known value', async () => {
const key = await CD.deriveKey('#test');
const hex = CD.bytesToHex(key);
// Verify against Node.js crypto
const expected = cryptoModule.createHash('sha256').update('#test').digest('hex').substring(0, 32);
assert.strictEqual(hex, expected, 'deriveKey should produce SHA256("#test")[:16]');
});
test('deriveKey: returns 16 bytes', async () => {
const key = await CD.deriveKey('#LongFast');
assert.strictEqual(key.length, 16);
});
test('computeChannelHash: SHA256(key)[0]', async () => {
const key = await CD.deriveKey('#test');
const hashByte = await CD.computeChannelHash(key);
const keyHex = CD.bytesToHex(key);
const expected = cryptoModule.createHash('sha256').update(Buffer.from(keyHex, 'hex')).digest()[0];
assert.strictEqual(hashByte, expected);
});
test('verifyMAC: valid MAC passes', async () => {
// Create a known ciphertext and compute MAC using Node.js
const key = await CD.deriveKey('#test');
const secret = Buffer.alloc(32);
Buffer.from(CD.bytesToHex(key), 'hex').copy(secret, 0);
const ciphertext = Buffer.from('00112233445566778899aabbccddeeff', 'hex');
const mac = cryptoModule.createHmac('sha256', secret).update(ciphertext).digest();
const macHex = mac.slice(0, 2).toString('hex');
const result = await CD.verifyMAC(key, new Uint8Array(ciphertext), macHex);
assert.strictEqual(result, true, 'valid MAC should pass');
});
test('verifyMAC: invalid MAC fails', async () => {
const key = await CD.deriveKey('#test');
const ciphertext = new Uint8Array(16);
const result = await CD.verifyMAC(key, ciphertext, 'ffff');
assert.strictEqual(result, false, 'invalid MAC should fail');
});
test('parsePlaintext: extracts sender and message', () => {
// Build plaintext: timestamp(4 LE) + flags(1) + "alice: hello\0"
const msg = 'alice: hello\0';
const buf = new Uint8Array(5 + msg.length);
// timestamp = 1000 (LE)
buf[0] = 0xe8; buf[1] = 0x03; buf[2] = 0; buf[3] = 0;
buf[4] = 0; // flags
const enc = new TextEncoder();
const msgBytes = enc.encode(msg);
buf.set(msgBytes, 5);
const parsed = CD.parsePlaintext(buf);
assert.ok(parsed, 'should parse successfully');
assert.strictEqual(parsed.sender, 'alice');
assert.strictEqual(parsed.message, 'hello');
assert.strictEqual(parsed.timestamp, 1000);
});
test('parsePlaintext: no sender prefix returns empty sender', () => {
const msg = 'just a message\0';
const buf = new Uint8Array(5 + msg.length);
buf[0] = 1; buf[1] = 0; buf[2] = 0; buf[3] = 0; buf[4] = 0;
buf.set(new TextEncoder().encode(msg), 5);
const parsed = CD.parsePlaintext(buf);
assert.ok(parsed);
assert.strictEqual(parsed.sender, '');
assert.strictEqual(parsed.message, 'just a message');
});
test('parsePlaintext: returns null for too-short input', () => {
assert.strictEqual(CD.parsePlaintext(new Uint8Array(3)), null);
});
test('localStorage persistence: save/get/remove keys', () => {
CD.saveKey('#test', 'abcd1234abcd1234abcd1234abcd1234');
const keys = CD.getKeys();
assert.strictEqual(keys['#test'], 'abcd1234abcd1234abcd1234abcd1234');
CD.removeKey('#test');
const keys2 = CD.getKeys();
assert.strictEqual(keys2['#test'], undefined);
});
test('bytesToHex and hexToBytes roundtrip', () => {
const hex = 'deadbeef01020304';
const bytes = CD.hexToBytes(hex);
assert.strictEqual(CD.bytesToHex(bytes), hex);
});
}
// ===== Encrypted Channels Toggle Tests (#728) =====
{
console.log('\n--- Encrypted Channels Toggle (#728) ---');
test('encrypted toggle reads from localStorage', () => {
const store = {};
const ls = {
getItem: k => store[k] || null,
setItem: (k, v) => { store[k] = String(v); },
};
// Default: not set → should be false
assert.strictEqual(ls.getItem('channels-show-encrypted'), null);
const showEncrypted = ls.getItem('channels-show-encrypted') === 'true';
assert.strictEqual(showEncrypted, false);
// Set to true
ls.setItem('channels-show-encrypted', 'true');
assert.strictEqual(ls.getItem('channels-show-encrypted') === 'true', true);
// Set to false
ls.setItem('channels-show-encrypted', 'false');
assert.strictEqual(ls.getItem('channels-show-encrypted') === 'true', false);
});
test('encrypted channels get ch-encrypted CSS class', () => {
// Simulate the rendering logic from channels.js
const ch = { hash: 'enc_A1B2', name: 'Encrypted (0xA1B2)', encrypted: true, messageCount: 5 };
const isEncrypted = ch.encrypted === true;
const encClass = isEncrypted ? ' ch-encrypted' : '';
const className = 'ch-item' + encClass;
assert.ok(className.includes('ch-encrypted'), 'encrypted channel should have ch-encrypted class');
// Non-encrypted channel should NOT have the class
const ch2 = { hash: 'AABB', name: '#general', encrypted: false };
const encClass2 = ch2.encrypted === true ? ' ch-encrypted' : '';
const className2 = 'ch-item' + encClass2;
assert.ok(!className2.includes('ch-encrypted'), 'non-encrypted channel should not have ch-encrypted class');
});
}
// ===== #690 — Clock Skew UI Tests =====
{
console.log('\n--- Clock Skew UI (roles.js helpers) ---');
const ctx = makeSandbox();
vm.runInContext(fs.readFileSync('public/roles.js', 'utf8'), ctx);
test('formatSkew handles seconds', () => {
assert.strictEqual(ctx.window.formatSkew(30), '+30s');
assert.strictEqual(ctx.window.formatSkew(-45), '-45s');
});
test('formatSkew handles minutes', () => {
assert.strictEqual(ctx.window.formatSkew(154), '+2m 34s');
assert.strictEqual(ctx.window.formatSkew(-900), '-15m 0s');
});
test('formatSkew handles hours', () => {
assert.strictEqual(ctx.window.formatSkew(3661), '+1h 1m');
assert.strictEqual(ctx.window.formatSkew(-55320), '-15h 22m');
});
test('formatSkew handles days', () => {
assert.strictEqual(ctx.window.formatSkew(90000), '+1d 1h');
});
test('formatSkew handles null', () => {
assert.strictEqual(ctx.window.formatSkew(null), '—');
});
test('renderSkewBadge renders correct severity class', () => {
var html = ctx.window.renderSkewBadge('warning', 400);
assert.ok(html.includes('skew-badge--warning'), 'should contain warning class');
assert.ok(html.includes('⏰'), 'should contain clock emoji');
});
test('renderSkewBadge renders ok badge (icon only)', () => {
var html = ctx.window.renderSkewBadge('ok', 10);
assert.ok(html.includes('skew-badge--ok'), 'should contain ok class');
});
test('renderSkewBadge returns empty for null severity', () => {
assert.strictEqual(ctx.window.renderSkewBadge(null, 0), '');
});
test('renderSkewSparkline returns SVG with data points', () => {
var samples = [
{ ts: 1000, skew: 10 },
{ ts: 2000, skew: 20 },
{ ts: 3000, skew: -5 }
];
var svg = ctx.window.renderSkewSparkline(samples, 120, 24);
assert.ok(svg.includes('<svg'), 'should return SVG element');
assert.ok(svg.includes('polyline'), 'should contain polyline');
assert.ok(svg.includes('points='), 'should have points attribute');
});
test('renderSkewSparkline returns empty for insufficient data', () => {
assert.strictEqual(ctx.window.renderSkewSparkline([], 120, 24), '');
assert.strictEqual(ctx.window.renderSkewSparkline([{ ts: 1, skew: 5 }], 120, 24), '');
assert.strictEqual(ctx.window.renderSkewSparkline(null, 120, 24), '');
});
test('SKEW_SEVERITY_ORDER sorts worst first', () => {
var order = ctx.window.SKEW_SEVERITY_ORDER;
assert.ok(order.absurd < order.critical, 'absurd should sort before critical');
assert.ok(order.critical < order.warning, 'critical should sort before warning');
assert.ok(order.warning < order.ok, 'warning should sort before ok');
});
}
// ===== analytics.js: hashStatCardsHtml collision clickability (#757) =====
console.log('\n=== analytics.js: hashStatCardsHtml collision details ===');
{
function makeAnalyticsSandbox757() {
const ctx = makeSandbox();
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
try { loadInCtx(ctx, 'public/analytics.js'); } catch (e) {
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
}
return ctx;
}
const ctx = makeAnalyticsSandbox757();
const hashStatCardsHtml = ctx.window._analyticsHashStatCardsHtml;
test('hashStatCardsHtml is exposed', () => assert.ok(hashStatCardsHtml, '_analyticsHashStatCardsHtml must be exposed'));
test('collision count > 0 renders clickable card with onclick', () => {
const html = hashStatCardsHtml(100, 50, '3-byte', 16777216, 48, 3);
assert.ok(html.includes('onclick='), 'should have onclick when collisions > 0');
assert.ok(html.includes('collisionRiskSection'), 'should scroll to collisionRiskSection');
assert.ok(html.includes('cursor:pointer'), 'should show pointer cursor');
assert.ok(html.includes('▼'), 'should show expand indicator');
});
test('collision count 0 renders non-clickable card', () => {
const html = hashStatCardsHtml(100, 50, '1-byte', 256, 48, 0);
assert.ok(!html.includes('onclick='), 'should not have onclick when collisions = 0');
assert.ok(!html.includes('cursor:pointer'), 'should not show pointer cursor');
});
}
// ===== analytics.js: renderCollisionsFromServer node links (#757) =====
console.log('\n=== analytics.js: renderCollisionsFromServer collision table ===');
{
function makeAnalyticsSandbox757b() {
const ctx = makeSandbox();
const collisionListEl = { innerHTML: '', querySelectorAll: () => [] };
const origGetById = ctx.document.getElementById;
ctx.document.getElementById = (id) => {
if (id === 'collisionList') return collisionListEl;
return origGetById ? origGetById(id) : null;
};
ctx.window.document = ctx.document;
loadInCtx(ctx, 'public/roles.js');
loadInCtx(ctx, 'public/app.js');
try { loadInCtx(ctx, 'public/analytics.js'); } catch (e) {
for (const k of Object.keys(ctx.window)) ctx[k] = ctx.window[k];
}
ctx._collisionListEl = collisionListEl;
return ctx;
}
const ctx = makeAnalyticsSandbox757b();
const renderCollisions = ctx.window._analyticsRenderCollisionsFromServer;
test('renderCollisionsFromServer is exposed', () => assert.ok(renderCollisions, '_analyticsRenderCollisionsFromServer must be exposed'));
test('renders collision table with node links to correct pubkey', () => {
const sizeData = {
collisions: [
{
prefix: 'A3F2C1',
byte_size: 3,
appearances: 2,
nodes: [
{ public_key: 'abc123def456', name: 'Mountain Repeater', role: 'repeater', lat: 34.0, lon: -118.0 },
{ public_key: 'def456abc789', name: 'Valley Node', role: 'repeater', lat: 34.5, lon: -118.5 }
],
max_dist_km: 45.2,
classification: 'local',
with_coords: 2
}
]
};
renderCollisions(sizeData, 3);
const html = ctx._collisionListEl.innerHTML;
assert.ok(html.includes('A3F2C1'), 'should show prefix');
assert.ok(html.includes('#/nodes/abc123def456'), 'first node link should point to correct pubkey');
assert.ok(html.includes('#/nodes/def456abc789'), 'second node link should point to correct pubkey');
assert.ok(html.includes('Mountain Repeater'), 'should show first node name');
assert.ok(html.includes('Valley Node'), 'should show second node name');
});
test('renders no-collision message when collisions empty', () => {
const sizeData = { collisions: [] };
renderCollisions(sizeData, 3);
const html = ctx._collisionListEl.innerHTML;
assert.ok(html.includes('No 3-byte prefix collisions'), 'should show no-collision message');
});
}
// ===== Observer role support (#753 / PR #774) =====
{
console.log('\n--- Observer role support (PR #774) ---');
// Test 1: ROLE_COLORS.observer is defined and not empty
test('ROLE_COLORS.observer is defined and not empty', () => {
const rolesJs = fs.readFileSync(__dirname + '/public/roles.js', 'utf8');
const ctx = makeSandbox();
vm.runInNewContext(rolesJs, ctx);
assert.ok(ctx.window.ROLE_COLORS.observer, 'ROLE_COLORS.observer should be defined');
assert.ok(ctx.window.ROLE_COLORS.observer.length > 0, 'ROLE_COLORS.observer should not be empty');
});
// Test 2: Observer checkbox exists in neighbor graph filter (unchecked by default)
test('Observer checkbox exists in neighbor graph filter section', () => {
const analyticsJs = fs.readFileSync(__dirname + '/public/analytics.js', 'utf8');
// The observer checkbox is added with data-role="observer" and NO "checked" attribute
assert.ok(analyticsJs.includes('data-role="observer"'), 'analytics.js should contain observer checkbox');
// Verify it's NOT checked by default (no "checked" attribute on observer checkbox)
const observerCheckboxMatch = analyticsJs.match(/data-role="observer"[^>]*>/);
assert.ok(observerCheckboxMatch, 'observer checkbox markup must exist');
assert.ok(!observerCheckboxMatch[0].includes('checked'), 'observer checkbox should NOT be checked by default');
});
// Test 3: Other role checkboxes ARE checked by default
test('Non-observer role checkboxes are checked by default', () => {
const analyticsJs = fs.readFileSync(__dirname + '/public/analytics.js', 'utf8');
// The main role loop uses "checked" attribute
const mainRoleCheckbox = analyticsJs.match(/data-role="\$\{r\}"[^>]*checked/);
assert.ok(mainRoleCheckbox, 'Main role checkboxes should have checked attribute');
});
// Test 4: --role-observer CSS variable exists in style.css
test('--role-observer CSS variable exists in style.css', () => {
const css = fs.readFileSync(__dirname + '/public/style.css', 'utf8');
assert.ok(css.includes('--role-observer:'), 'style.css should define --role-observer CSS variable');
});
// Test 5: Filter logic does NOT auto-include observer role
test('Filter logic excludes observer nodes when checkbox unchecked', () => {
const analyticsJs = fs.readFileSync(__dirname + '/public/analytics.js', 'utf8');
// Old code had: return checkedRoles.has(role) || role === 'unknown' || role === 'observer';
// New code: return checkedRoles.has(role) || role === 'unknown';
// Verify observer is NOT given special pass-through treatment
const filterLine = analyticsJs.match(/return checkedRoles\.has\(role\)[^;]+;/);
assert.ok(filterLine, 'filter line must exist');
assert.ok(!filterLine[0].includes("'observer'"), 'filter should NOT auto-include observer role');
assert.ok(filterLine[0].includes("'unknown'"), 'filter should still auto-include unknown role');
});
}
// ===== Neighbor Graph Min Score Slider Persistence =====
{
console.log('\n--- Neighbor Graph Slider Persistence ---');
test('default slider value is 70 (0.70)', () => {
// Read the raw HTML from analytics.js to verify default
const src = fs.readFileSync('public/analytics.js', 'utf8');
assert.ok(src.includes('value="70"'), 'ngMinScore input should default to value="70"');
assert.ok(src.includes('>0.70</span>'), 'ngMinScoreVal should display 0.70');
});
test('localStorage read on load is present in code', () => {
const src = fs.readFileSync('public/analytics.js', 'utf8');
assert.ok(src.includes("localStorage.getItem('ng-min-score')"), 'should read ng-min-score from localStorage on load');
});
test('localStorage write on slider change is present in code', () => {
const src = fs.readFileSync('public/analytics.js', 'utf8');
assert.ok(src.includes("localStorage.setItem('ng-min-score'"), 'should write ng-min-score to localStorage on change');
});
}
// ===== SUMMARY =====
Promise.allSettled(pendingTests).then(() => {
console.log(`\n${'═'.repeat(40)}`);
+34
View File
@@ -593,6 +593,40 @@ console.log('\n=== live.js: formatLiveTimestampHtml ===');
});
}
// ===== Feed timestamp refresh — data-ts attribute and selector (#701) =====
console.log('\n=== live.js: feed timestamp refresh (#701) ===');
{
const ctx = makeLiveSandbox({ withAppJs: true });
const fmt = ctx.window._liveFormatLiveTimestampHtml;
test('formatLiveTimestampHtml returns different text for different ages', () => {
const recent = fmt(Date.now() - 5000);
const older = fmt(Date.now() - 120000);
// Both should produce valid HTML
assert.ok(recent.includes('timestamp-text'), 'recent should have timestamp-text');
assert.ok(older.includes('timestamp-text'), 'older should have timestamp-text');
});
test('formatLiveTimestampHtml accepts numeric ms timestamp', () => {
const ts = Date.now() - 45000;
const html = fmt(ts);
assert.ok(html.includes('timestamp-text'), 'numeric ms timestamp should render');
// Re-calling with same ts should produce same result (idempotent refresh)
const html2 = fmt(ts);
assert.strictEqual(html, html2, 'same input should produce same output');
});
test('feed-time template with data-ts round-trips correctly', () => {
// Verify that Number(dataset.ts) fed back to fmt produces valid output
const ts = Date.now() - 30000;
const tsStr = String(ts);
const reparsed = Number(tsStr);
assert.strictEqual(reparsed, ts, 'data-ts round-trip should preserve value');
const html = fmt(reparsed);
assert.ok(html.includes('timestamp-text'), 'round-tripped timestamp should render');
});
}
// ===== resolveHopPositions =====
console.log('\n=== live.js: resolveHopPositions ===');
{