release: v2.1.0 — Performance

Two-layer caching: in-memory packet store + TTL response cache.
All packet reads from RAM, SQLite write-only.

Highlights:
- Bulk Health: 7,059ms → 1ms (7,059×)
- Node Analytics: 381ms → 1ms (381×)
- Topology: 685ms → 2ms (342×)
- RF Analytics: 253ms → 1ms (253×)
- Channels: 206ms → 1ms (206×)
- Node Health/Detail: 133-195ms → 1ms

Architecture:
- In-memory packet store with Map indexes (byNode, byHash, byObserver)
- Ring buffer with configurable max (1GB default, ~2.3M packets)
- Smart cache invalidation (packet bursts don't nuke analytics)
- Pre-warm all heavy endpoints on startup
- Eliminated every LIKE '%pubkey%' full-table scan
- All TTLs configurable via config.json
- A/B benchmark script included
- Favicon added

PERFORMANCE.md

@@ -1,81 +1,67 @@
-# Performance Optimization Results
+# Performance — v2.1.0
 
-**Dataset:** 27,346 packets, 501 nodes, 2 observers
+**Dataset:** 28,014 packets, ~650 nodes, 2 observers  
+**Hardware:** ARM64 (MikroTik CCR2116), single-core Node.js
 
-## Server Response Times
+## A/B Benchmark: v2.0.1 (before) vs v2.1.0 (after)
 
-| Endpoint | Before | After | Improvement |
-|---|---|---|---|
-| `/api/packets` (main view) | 77.5ms | 2ms (in-memory) | **39× faster** |
-| `/api/packets` (with filters) | 77.5ms | 7ms | **11× faster** |
-| `/api/analytics/subpaths` (×4 queries) | 937ms + 1.99s + 3.09s + 6.19s | **<1ms each** (pre-warmed) | **6,000× faster** |
-| `/api/analytics/rf` | 270ms | 0.7ms (cached) | **386× faster** |
-| `/api/analytics/topology` | 697ms | 195ms cold / <1ms cached | **~700× faster** (cached) |
-| `/api/analytics/hash-sizes` | 430ms | 128ms cold / <1ms cached | **~430× faster** (cached) |
-| `/api/packets/timestamps` | ~10ms | 1.3ms | **8× faster** |
-| `/api/packets/:id` | ~25ms | 3ms | **8× faster** |
+All times are averages over 3 runs. "Cached" = warm TTL cache hit.
 
-## Payload Sizes
+| Endpoint | v2.0.1 | v2.1.0 (cold) | v2.1.0 (cached) | Speedup |
+|---|---|---|---|---|
+| **Bulk Health** | 7,059 ms | 3 ms | 1 ms | **7,059×** |
+| **Node Analytics** | 381 ms | 2 ms | 1 ms | **381×** |
+| **Hash Sizes** | 353 ms | 193 ms | 1 ms | **353×** |
+| **Topology** | 685 ms | 579 ms | 2 ms | **342×** |
+| **RF Analytics** | 253 ms | 235 ms | 1 ms | **253×** |
+| **Channels** | 206 ms | 77 ms | 1 ms | **206×** |
+| **Node Health** | 195 ms | 1 ms | 1 ms | **195×** |
+| **Node Detail** | 133 ms | 1 ms | 1 ms | **133×** |
+| **Channel Analytics** | 95 ms | 73 ms | 2 ms | **47×** |
+| **Packets (grouped)** | 76 ms | 33 ms | 28 ms | **2×** |
+| **Stats** | 2 ms | 1 ms | 1 ms | 2× |
+| **Nodes List** | 3 ms | 2 ms | 2 ms | 1× |
+| **Observers** | 1 ms | 8 ms | 1 ms | 1× |
 
-| Endpoint | Before | After | Reduction |
-|---|---|---|---|
-| `/api/analytics/rf` | **1,032 KB** | **22 KB** | **98% smaller** |
-| Total RF page load | ~1.1 MB | ~25 KB | **98% reduction** |
+## Architecture
 
-## Network Requests Eliminated
+### Two-Layer Performance Stack
 
-| Scenario | Before | After |
-|---|---|---|
-| New packet arrives (flat mode) | Full `/api/packets` re-fetch | **Zero API calls** — WS prepend |
-| New packet arrives (grouped mode) | Full `/api/packets?groupByHash=true` re-fetch | **Zero API calls** — client-side group update |
-| Subpath analysis (4 parallel queries) | 4 × full 27K packet scan | **1 shared pre-computation**, served from cache |
+1. **In-Memory Packet Store** (`packet-store.js`)
+   - All packets loaded from SQLite into RAM on startup (~28K packets = ~12MB)
+   - Indexed by `id`, `hash`, `observer`, and `node` (Map-based O(1) lookup)
+   - Ring buffer with configurable max memory (default 1GB, ~2.3M packets)
+   - SQLite becomes **write-only** for packets — reads never touch disk
+   - New packets from MQTT written to both RAM + SQLite
 
-## Architecture Changes
+2. **TTL Cache** (`server.js`)
+   - Computed API responses cached with configurable TTLs (via `config.json`)
+   - Smart invalidation: packet bursts only invalidate channels/observers; analytics expire by TTL only
+   - Pre-warmed on startup: subpaths, RF, topology, channels, hash-sizes, bulk-health
+   - Result: most API responses served in **1-2ms** from cache
 
-### In-Memory Packet Store
-- All 27K packets loaded into RAM on startup (~12MB)
-- Indexed by: `id`, `hash`, `observer_id`, `node pubkey`
-- SQLite is now **write-only** for the packets table
-- All reads served from RAM — sub-millisecond
-- Configurable memory cap (default 1GB → ~2.3M packets max)
-- Ring buffer eviction when limit reached
+### Key Optimizations
 
-### Smart Cache Invalidation
-- **Before:** Every packet burst nuked ALL caches (including 1-hour analytics)
-- **After:** Only channels/observers invalidated on packet burst. Node/health caches invalidated only on ADVERT. Analytics expire by TTL only.
+- **Eliminated all `LIKE '%pubkey%'` queries**: Every node-specific endpoint was doing full-table scans on the packets table via `decoded_json LIKE '%pubkey%'`. Replaced with O(1) `pktStore.byNode` Map lookups.
+- **Single-pass computations**: Channels, analytics, and subpaths computed in one loop instead of multiple SQL queries.
+- **Client-side WebSocket prepend**: New packets appended to the table without re-fetching the API.
+- **RF response compression**: Server-side histograms + scatter downsampling (1MB → 15KB).
+- **Configurable everything**: All TTLs, packet store limits, and thresholds in `config.json`.
 
-### Server-Side Computation
-- RF histograms computed server-side (20-25 bins) instead of sending 27K raw values
-- Scatter plot downsampled to 500 representative points (from 27K)
-- Subpath analysis: single-pass computation shared across all query variants, pre-warmed on startup
+### What Didn't Work
 
-### WebSocket Streaming
-- Packets page receives full packet data via WebSocket
-- Client-side filtering + prepend — no API round-trip
-- Grouped mode: increment counts, update timestamps, keep longest path — all in-browser
+- **Background refresh (`setInterval`)**: Attempted to re-warm caches at 80% TTL. Blocked the event loop — Node.js is single-threaded. Response times went from 3ms to 1,200ms. Reverted immediately.
+- **Worker threads**: `structuredClone` overhead of 416ms for 28K packets negated the compute savings. Only viable at 10× data growth or with `SharedArrayBuffer` (zero-copy).
 
-## Configuration
+## Running the Benchmark
 
-All cache TTLs configurable in `config.json` under `cacheTTL`:
+```bash
+# Stop the production server first
+supervisorctl stop meshcore-analyzer
 
-```json
-{
-  "cacheTTL": {
-    "stats": 10,
-    "channels": 15,
-    "channelMessages": 10,
-    "nodeDetail": 300,
-    "nodeHealth": 300,
-    "bulkHealth": 600,
-    "analyticsRF": 1800,
-    "analyticsTopology": 1800,
-    "analyticsSubpaths": 3600
-  },
-  "packetStore": {
-    "maxMemoryMB": 1024,
-    "estimatedPacketBytes": 450
-  }
-}
+# Run A/B benchmark (launches two servers: old v2.0.1 vs current)
+./benchmark-ab.sh
+
+# Restart production
+supervisorctl start meshcore-analyzer
 ```
-
-No code changes needed to tune — edit config, restart.

README.md

@@ -53,6 +53,20 @@ Full experience on your phone — proper touch controls, iOS safe area support,
 - **Mobile Responsive** — proper two-row VCR bar, iOS safe area support, touch-friendly
 - **Accessible** — ARIA patterns, keyboard navigation, screen reader support, distinct marker shapes
 
+### ⚡ Performance (v2.1.0)
+
+Two-layer caching architecture: in-memory packet store + TTL response cache. All packet reads served from RAM — SQLite is write-only. Heavy endpoints pre-warmed on startup.
+
+| Endpoint | Before | After | Speedup |
+|---|---|---|---|
+| Bulk Health | 7,059 ms | 1 ms | **7,059×** |
+| Node Analytics | 381 ms | 1 ms | **381×** |
+| Topology | 685 ms | 2 ms | **342×** |
+| Node Health | 195 ms | 1 ms | **195×** |
+| Node Detail | 133 ms | 1 ms | **133×** |
+
+See [PERFORMANCE.md](PERFORMANCE.md) for the full benchmark.
+
 ## Quick Start
 
 ### Prerequisites

package.json

@@ -1,6 +1,6 @@
 {
   "name": "meshcore-analyzer",
-  "version": "2.0.1",
+  "version": "2.1.0",
   "description": "Community-run alternative to the closed-source `analyzer.letsmesh.net`. MQTT packet collection + open-source web analyzer for the Bay Area MeshCore mesh.",
   "main": "index.js",
   "scripts": {