Create public/hop-resolver.js that mirrors the server's disambiguateHops()
algorithm (prefix index, forward/backward pass, distance sanity check).
Replace all /api/resolve-hops fetch calls in packets.js with local
HopResolver.resolve() calls. The resolver lazily fetches and caches the
full nodes list via /api/nodes on first use.
The server endpoint is kept as fallback but no longer called by the UI,
eliminating 40+ HTTP requests per session.
- Create packets_v SQL view joining transmissions+observations to match old packets schema
- Replace all SELECT FROM packets with packets_v in db.js, packet-store.js, server.js
- Update countPackets/countRecentPackets to query observations directly
- Update seed() to use insertTransmission instead of insertPacket
- Remove insertPacket from exports (no longer called)
- Keep packets table schema intact (not dropped yet, pending testing)
SQLite WAL auto-checkpoint (every 1000 pages/4MB) was causing 200ms+
event loop spikes on a 381MB database. This is synchronous I/O that
blocks the Node.js event loop unpredictably.
Fix: disable auto-checkpoint, run PASSIVE (non-blocking) checkpoint
every 5 minutes. PASSIVE won't stall readers or writers — it only
checkpoints pages that aren't currently in use.
Previous pre-warm called computeAllSubpaths() synchronously (500ms+)
directly in the setTimeout callback, then sequentially warmed 8 more
endpoints. Any browser request arriving during this 1.5s+ window
waited the full duration (user saw 4816ms for a 3-hop resolve-hops).
Fix: ALL pre-warm now goes through self-HTTP-requests which yield the
event loop between each computation. Delayed to 5s so initial page
load requests complete first (they populate cache on-demand).
Removed the sync computeAllSubpaths() call and inline subpath cache
population — the /api/analytics/subpaths endpoint handles this itself.
Replaced inline resolveHopsInternal (allNodes.filter per hop) with
shared disambiguateHops (prefix-indexed). Also uses _parsedPath cache
and per-request disambig cache. /api/nodes/:pubkey/paths was 560ms cold,
should now be much faster.
200ms gaps meant clients hit cold caches themselves (worse).
Pre-warm should be fast and immediate — start at 100ms after listen,
setImmediate between endpoints to yield but not delay.
setImmediate wasn't enough — each analytics computation blocks for
200-400ms synchronously. Adding a 200ms setTimeout between pre-warm
requests gives pending client requests a window to complete between
the heavy computations.
- 8 separate 'SELECT * FROM nodes' queries replaced with getCachedNodes()
(refreshes every 30s, prefix index built once and reused)
- Region-filtered subpaths + master subpaths use _parsedPath cache
- Eliminates repeated SQLite queries + prefix index rebuilds across
back-to-back analytics endpoint calls
These endpoints were missing from the sequential pre-warm,
causing cold cache hits when clients connect before warm completes.
observers was 3s cold, distance was 600ms cold.
- Topology: replace O(N) allNodes.filter with prefix map + hop cache for resolveHop
- Topology: use _parsedPath cached JSON.parse for path_json (3 call sites)
- Topology: build observer map from already-filtered packets instead of second full scan
- Hash-sizes: prefix map for hop resolution instead of allNodes.find per hop
- Hash-sizes: use _parsedPath and _parsedDecoded cached parses
- Channels: use _parsedDecoded cached parse for decoded_json
- Build prefix map for O(1) hop resolution instead of O(N) linear scan per hop
- Cache resolved hops to avoid re-resolving same hex prefix across packets
- Pre-compute repeater set for O(1) role lookups
- Cache parsed path_json/decoded_json on packet objects (_parsedPath/_parsedDecoded)
The hash_size computation was scanning all 19K+ packets with JSON.parse
on every /api/nodes request, blocking the event loop for hundreds of ms.
Event loop p95 was 236ms, max 1732ms.
Now computed once at startup and updated incrementally on each new packet.
/api/nodes just does a Map.get per node instead of full scan.
Packets array is sorted newest-first. The previous 'last-wins'
approach (unconditional set) gave the OLDEST packet's hash_size.
Switched to first-wins (!has guard) which correctly uses the
newest ADVERT since we iterate newest-first.
Verified: Kpa Roof Solar has 1-byte ADVERTs (old firmware) and
2-byte ADVERTs (new firmware) interleaved. Newest are 2-byte.
Pass 1 correctly uses last-wins for ADVERT packets. But Pass 2
(fallback for nodes without ADVERTs) was also unconditionally
overwriting, so a stale 1-byte non-ADVERT packet would clobber
the correct 2-byte value from Pass 1.
Restored the !hashSizeMap.has() guard on Pass 2 only — it should
only fill gaps, never override ADVERT-derived hash_size.
The hashSizeMap was guarded by !hashSizeMap.has(pk), meaning the oldest
ADVERT determined a node's hash_size permanently. If a node upgraded
firmware from 1-byte to 2-byte hash prefix, the stale value persisted.
Remove the guard so newer packets overwrite older ones (last-seen-wins).
- New config.apiKey field — when set, POST endpoints require X-Api-Key header
- If apiKey not configured, endpoints remain open (dev/local mode)
- GET endpoints and /api/decode (read-only) remain public
- Closes the packet injection attack surface
- 🗺️ button on each top hop row → opens map with from/to markers + line
- 🗺️ button on each top path row → opens map with full multi-hop route
- Server now includes fromPk/toPk in topPaths hops for map resolution
- Uses existing drawPacketRoute() via sessionStorage handoff
- 1000km filter was too generous for LoRa (record ~250km)
- Uhuru kwa watu 📡 ↔ Bay Area hops at 880km were obviously wrong
- Node links in leaderboard now go to #/nodes/:pk (detail) not /analytics
New /api/analytics/distance endpoint that:
- Resolves path hops to nodes with valid GPS coordinates
- Calculates haversine distances between consecutive hops
- Separates stats by link type: R↔R, C↔R, C↔C
- Returns top longest hops, longest paths, category stats, histogram, time series
- Filters out invalid GPS (null, 0/0) and sanity-checks >1000km
- Supports region filtering and caching
New Distance tab in analytics UI with:
- Summary cards (total hops, paths, avg/max distance)
- Link type breakdown table
- Distance histogram
- Average distance over time sparkline
- Top 20 longest hops leaderboard
- Top 10 longest multi-hop paths table
The previous approach matched nodes via data packet hashes seen by
regional observers — but mesh packets propagate everywhere, so nearly
every node matched every region (550/558).
New approach: _advertByObserver index tracks which observers saw each
node's ADVERT packets. ADVERTs are local broadcasts that indicate
physical presence, so they're the correct signal for geographic filtering.
Also fixes role counts to reflect filtered results, not global totals.
Math.min(...arr) and Math.max(...arr) blow the call stack when arr
has tens of thousands of elements. Replaced with simple for-loop
arrMin/arrMax helpers.
Pills look out of place in the dense toolbar. Added { dropdown: true }
option to RegionFilter.init() to force dropdown mode regardless of
region count.
- Add aria-label and aria-haspopup='dialog' to BYOP trigger button
- Add aria-label to close button and textarea
- Add role='status' and aria-live='polite' to result container
- Add role='alert' to error messages for screen reader announcement
- Fix textarea focus style: visible outline instead of outline:none
- Update cache busters
- Separate region filtering from SNR filtering in /api/analytics/rf
- totalAllPackets now shows regional observation count (was global)
- Add totalTransmissions (unique hashes in regional set)
- Payload types and packet sizes use all regional data, not just SNR-filtered
- Signal stats (SNR, RSSI, scatter) use SNR-filtered subset
- Handle empty SNR/RSSI arrays gracefully (no Infinity/-Infinity)
- Fixes empty region dropdown (was populated before async regionMap loaded)
- Now uses multi-select RegionFilter component consistent with other pages
- Loads regions from /api/observers with proper async handling
- Supports multi-region filtering
The subagent used a non-existent column (sender_key) in the SQL join.
Reverted to the same byObserver + _nodeHashIndex approach used by
bulk-health and network-status endpoints.
The previous approach used pktStore._nodeHashIndex which only tracks
nodes appearing as sender/dest in decoded packet JSON. Most nodes only
send ADVERTs, so they had no entries in _nodeHashIndex and were filtered
out when a region was selected (showing 0 results).
Now uses a direct SQL join between observations and transmissions to find
all sender_keys observed by regional observers, which correctly includes
ADVERT-only nodes.
- Add RegionFilter.regionQueryString() to all API calls in renderSubpaths and renderNodesTab
- Add region filtering to /api/analytics/subpaths (filter packets by regional observer hashes)
- Add region filtering to /api/nodes/bulk-health (filter nodes by regional presence)
- Add region filtering to /api/nodes/network-status (filter node counts by region)
- Add region param to nodes lookup in hash collision tab
- Update cache keys to include region param for proper cache separation
Track current active tab in _currentTab variable so that
loadAnalytics() re-renders the current tab instead of always
resetting to 'overview' when region filter changes.
- Add 'Region:' label before filter controls
- ARIA roles: group with aria-label, checkbox roles on pills, aria-checked
- When >4 regions: render multi-select dropdown with checkboxes
- Trigger shows summary (All / selected names / N selected)
- All option at top, click-outside closes
- Pill bar mode unchanged for ≤4 regions (just added label + ARIA)
- channel-rainbow.json: 592 pre-computed SHA256-derived keys for common
channel names (cities, topics, ham radio, emergency, etc.)
- server.js: Load rainbow table at startup as lowest-priority key source
- config.example.json: Add #LongFast to hashChannels list
Key derivation verified against MeshCore source: SHA256('#name')[:16bytes].
Rainbow table boosted decryption from ~48% to ~88% in testing.
Channel keys for #test, #sf, #wardrive, #yo, #bot, #queer, #bookclub, #shtf
are all SHA256(channelName).slice(0,32) — no need to hardcode them. Move to
hashChannels array for auto-derivation. Only the MeshCore default public key
(8b3387e9c5cdea6ac9e5edbaa115cd72) needs explicit specification since it's
not derived from its channel name.
- Remove composite key scheme (ch_/unk_ prefixes) that broke URL routing
due to # in channel names. Use plain numeric channelHash as key instead.
- All packets with same hash byte go in one bucket; name is set from
first successful decryption.
- Add packet detail renderer for decoded CHAN type showing channel name,
sender, and sender timestamp.
- Update cache buster for packets.js.
The /api/channels endpoint was returning simple numeric hash (e.g. '45') while
/api/channels/:hash/messages was using composite keys (e.g. 'ch_#LongFast',
'unk_45') internally. This mismatch meant no channel ever matched, so all
messages appeared encrypted.
Fix: return the composite key as the hash field from /api/channels so the
frontend passes the correct identifier. Also add encodeURIComponent() to
channel API calls in the frontend since composite keys can contain '#'.
