The planner inlined the view CTEs into consumers and re-ran the links
aggregation once per candidate row (loops=13070, 24s+ for getViableLinks).
Wrapping each view body in WITH v AS MATERIALIZED forces one evaluation
per query: viable-links query 24.3s -> 3.1s, WS initial state <1s.
Also drops the dead terrain_profile_json column from node_identity_links
(no readers; frontend terrain comes from DEM tiles).
- channelRegistry: 39 validated channel secrets committed as the service
default (Public + 38 community/hashtag channels recovered via the
derivation audit); MESHCORE_CHANNEL_SECRETS env appends on top
- shared buildSummary/identifyChannel (single implementation for ingest
and offline tools)
- tools/backfillDecrypt: keyset-paginated retroactive decryption of the
stored raw_hex corpus (idempotent, resumable)
- feed: channel scope sidebar now lists every decrypted channel
Backend: schedule ONE final multi-observer resolution per path-bearing
packet after the propagation window closes (default 60s, calibrated on
real first->last observer spread p50 11s / p95 34s / p99 50s; median 17
observers per packet). The final runs with the COMPLETE observer set and
pushes into the resolve cache after invalidation, so the next API read
serves the full-propagation answer. mode=slow on /path-beta/resolve-multi
returns 202 {status:pending, remainingMs} while inside the window; new
/path-beta/slow-mode status endpoint; observability table
path_slow_resolutions (migration 034). Env knobs:
PATH_SLOW_MODE_ENABLED / PATH_SLOW_MODE_WINDOW_MS / PATH_SLOW_MODE_MIN_HOPS
/ PATH_SLOW_MODE_PENDING_MAX. Best-effort: restart loss is harmless.
Frontend: pinned packets resolve in slow mode (bounded 202 retry), local
path renders meanwhile, upgraded when the final lands. Live feed unchanged.
Tests: 4 new slowMode unit tests (dedupe, disable, bound, window timing).
Backend build clean, 257/258 (1 pre-existing workerPool timing flake on
base too); frontend build clean, 74/74.
* Fix map node freshness consistency
* Harden output, ingest, caches, and WebSocket limits
* Enforce public visibility across derived data
* Harden proxy and operator deployment boundary
* Make owner grants authoritative and reconcile ACLs safely
* Bound path, spam, and statistics analysis
* Make link and coverage jobs crash-safe
* Implement strategic security remediation
* Fix production cutover configuration
* Fix disabled viewshed worker health signal
* Serve stale stats during background refresh
* Retain stale stats through refresh windows
* Bound analytics work to protect ingestion
* Prioritize summary warmup over chart scans
* Throttle path history rebuilds
* Bound path history result memory
* Stream path history aggregation
* Give bounded path rebuild one CPU
* Serve stale charts during bounded refresh
* Prioritize startup stats before chart scans
* Bound path history segment cardinality
* Pin path rebuild context to privacy generation
* Self-host original frontend fonts
* Allow bounded path rebuild to complete
* Improve live map UI and low-latency group feed
- Dock node details on the right with selection highlight and collapsible layers
- Add node legend, 24h activity sparkline, copy-link, and layout/overlap fixes
- Keep all repeaters visible during Live Path focus
- Send GroupText feed packets immediately over WebSocket (no batch delay)
- Cache expensive stats/observer activity more aggressively to protect ingest
- Remove stale local planning/audit markdown from the tree
* fix(ci): supply OPERATOR_SITE_TOKEN for compose validation
Workers/Compose CI failed because docker-compose requires
OPERATOR_SITE_TOKEN. Add CI placeholders for that and MQTT_PASSWORD.
Consolidates the stacked backend, privacy, network-intelligence, frontend, operations, mobile, and owner-cache changes after resolving main conflicts and passing the full CI suite.
Backend: derive observer IATA from packet topic instead of node table,
send messages separately in WS initial state, add tiered prefix/edge
priors to lazy path resolver for more accurate hash disambiguation.
Frontend: add SeoHead, JsonLd, and vite-seo plugin for per-route meta
tags and structured data. Add earth-curvature correction to custom LOS
tool. Owner portal last-hop chart supports hiding/showing individual
series with cookie persistence. Feed page and path map refactored.
Cache-busting build filenames. Track firstSeenTs on aggregated packets.
Infra: serve robots.txt and sitemap.xml via nginx, add manifest.json,
reduce Mosquitto log verbosity. Viewshed worker supports per-node
antenna heights, configurable profile step, and SNR-median calibration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Multi-node LOS: multiple repeaters can have LOS active simultaneously,
each auto-expiring after 15 seconds; LOS persists after popup closes
- Custom LOS tool: desktop-only map button to sample terrain between two
points (click map or repeater); renders per-segment green/clear or
red/obstructed based on terrain height; prohibited nodes are excluded
- Terrain exaggeration: deck.gl altitudes multiplied by exaggeration factor
(2×) to match MapLibre's visual terrain mesh
- LOS line rendering: bloom+core PathLayer tube effect with ScatterplotLayer
endpoint markers; popup z-index fix so it renders above deck.gl canvas
- Terrain sampler: client-side Terrarium PNG tile decoder for elevation sampling
- DB schema: terrain_profile_json JSONB column added to node_links
- Nginx: terrain tile serving block added
- Docker: terrain tiles volume mount added
Excluded: PacketDetailPanel, UKFeedPage, styles/globals.css (unrelated local
changes), backfill_profiles.py (server-side utility, not intended for repo)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Lazy path resolver: geographic hop reconstruction from path_hashes without
pre-training data; fixes observer self-hash trimming bug
- ACL watcher worker: auto-provisions Mosquitto ACL entries when PUBLISH is
denied for a known user; eliminates the recurring gnomemoe ACL removal issue
- connectionMonitor: increase historical scan to 50 MB; fix OOM by streaming
lines instead of buffering; fixes missed backfill on restart
- Feed page: replace path-beta with lazy paths; add clickable node markers
showing name and public key; observer column nowrap; summary text unclipped
- PacketFeed: GRP-only message store protected from ADV eviction; scroll to
bottom on new message; 24-hour message backfill on initial load
- getRecentMessages: separate DB query for last 50 GRP messages last 24h
- getPacketDetail: new endpoint for full packet detail including observations
- Grafana/Loki/Promtail observability stack with Docker log collection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New aclManager.ts rewrites a user's ACL block (all linked nodes) and
sends SIGHUP to the Mosquitto container whenever autoLinkOwnerNodeIds
runs at owner login, eliminating the need to manually edit the ACL
file when a user connects a second node.
- requireOwnerSession now resolves node IDs from owner_account_nodes
on every request instead of using the frozen session cookie list,
so manually- or auto-linked nodes are visible without re-login.
- docker-compose: mount mosquitto config dir (rw) and Docker socket
into backend so it can update the ACL file and signal Mosquitto.
- Add dockerode dependency for container SIGHUP signalling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- limit owner last-hop data to selected-node cached queries
- replace broad owner-node prewarm with rolling per-node cache updates
- incrementally refresh the latest cached hour and drop data outside the 7-day window
- narrow owner last-hop telemetry to the selected MQTT node and cache it per node with hourly background refresh
- tighten repeater-only last-hop classification and clean up the owner chart selector/filtering
- fix owner direct-sender map basemap, static behavior, and viewport fitting/centering
- allow bare ukmesh.com and teessidemesh.com host handling in backend/frontend config
- split backend API into route/service/repository boundaries across stats, owner, and pathing\n- extract shared API bootstrap and helper modules\n- reduce MapLibreMap into smaller frontend map modules with extracted builders, config, popup UI, and types\n- continue worker modularization by extracting RF terrain/tile helpers alongside RF config and loss helpers\n- add repo-local architecture, DB lifecycle, link model, pathing, frontend map, and contributing docs\n\nThis is the verified refactor checkpoint: backend and frontend builds passed, worker syntax checks passed, affected containers were rebuilt, and /healthz is OK.
Split the monolithic backend router into domain route modules for health, radio, node status, coverage, nodes, owner, telemetry, stats, pathing, and misc endpoints.
Introduce the first service layer extractions for stats and owner so route files are now thin wrappers around business logic instead of owning heavy query orchestration and cache policy directly.
Harden DB startup by separating schema/migration bootstrap from the normal request pool timeout path. Startup schema initialization and migrations now run through a dedicated no-timeout startup pool, while runtime query timeouts remain in place.
This refactor also replaces the old monolithic startup schema asset flow with base schema + migration support, adds DB/config asset resolution cleanup, and substantially reduces backend/src/api/routes.ts from its previous multi-thousand-line state to a smaller registration/helper module.
Current roadmap status:
- route decomposition is effectively complete enough to stop splitting route files further
- stats service extraction is complete
- owner service extraction is complete
- next planned slice is pathing service extraction, followed by repository extraction under stats, owner, and pathing
Previously the /owner/session endpoint used node IDs baked into the
encrypted cookie at login time. If a node's public key changed, the
stale IDs would persist until the user manually logged out and back in.
Now re-fetches fresh node IDs from owner_account_nodes using the stored
MQTT username, falling back to cookie IDs only if the DB returns nothing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Owner portal fixes:
- Fix session node ID case mismatch: decryptOwnerSession was lowercasing
node IDs but /owner/live uppercased the ?nodeId= param, so the find()
never matched — users always got "Node is not owned by this session"
- Revert sameSite strict → lax (broke sessions after March 18 deploy)
- Remove LOWER() wrappers from all owner live SQL (node IDs are uppercase)
- Add Fireball's second node (TS17 Yagi North) to owner_account_nodes
Tile worker:
- Server-side PNG tile rendering via @napi-rs/canvas
- Tile snapshot cache, Redis-backed tile cache, tile worker service
- /api/tiles/nodes/:z/:x/:y.png endpoint with rate limiter
- UK tile TTL logic
Cross-network connectivity:
- Add 7-day hourly history series to /cross-network-connectivity response
- Cache result for 60s to avoid repeated join-heavy query
- Fix query to use literal interval instead of interpolated variable
Other:
- Remove LOWER() from packet/node SQL throughout (IDs stored uppercase)
- Various performance and cleanup improvements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap MapView + DeckGLOverlay in .map-layer container so DeckGL canvas
anchors to the map area (not app-shell), eliminating the 24px topbar
offset that caused packet history lines to misalign with node dots
- Add EXPENSIVE_LIMITER (12 req/min) to /observer-activity and
/cross-network-connectivity endpoints
- Tighten owner login cookie to sameSite: strict
- Add Content-Security-Policy and Permissions-Policy response headers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pre-compute /api/stats/charts every 30 min with inflight dedup; cache served instantly to all visitors
- Fix CPU usage stat: systemStats() was called twice per health request, giving near-zero second reading
- De-duplicate observer region packet counts by packet_hash (was counting per observer row)
- Show repeater names on feed instead of rx counter / "heard by N"
- Observer names span full row width on /feed packet list
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- getViableLinks: replace correlated EXISTS subquery (2m 49s!) with CTE-based
join on pre-computed node set (39ms) — eliminates WS initial state timeouts
- WS initial state: switch from getLastNPackets (1,872ms, 24h correlated
subqueries) to getRecentPackets (16ms, 5-min CTE) — cold connect now 30ms
- Debounce emitNode/emitNodeUpsert with 500ms window to coalesce rapid advert
bursts before publishing to Redis (10-100x fewer publishes during active periods)
- Pre-normalise node IDs to lowercase at broadcast time; remove per-client
toLowerCase() allocations in shouldSendMessage (~200 allocs/sec eliminated)
- Cache /api/path-beta/history (60s TTL) and /api/coverage (30s TTL)
- Add 10-min TTL to resolveCache to prevent multi-day unbounded growth
- Fix LOWER() on node_status_samples queries suppressing index use
- Add Express compression() middleware; add gzip to nginx.website.conf
- Increase viableLinksCache TTL 30s→5min, INITIAL_STATE_TTL 30s→60s
- Pre-warm initial state cache at startup for teesside+ukmesh
- Add compression package dependency
Frontend:
- Lazy-load coverage (26MB GeoJSON) — only fetch when coverage toggle is on,
not on every page mount regardless of filter state
- Remove standalone 30s setInterval from useDashboardStats; feed stats from
App.tsx consolidated poll instead (-1 HTTP request per 30s)
- Pre-compute DeckGL packet-history segment colour+width inside useMemo;
getColor/getWidth become pure property lookups (~6000 log() calls/sec removed)
- DeckGLOverlay.tsx: add HistorySegmentWithColor type for pre-computed values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New www.teessidemesh.com single-page dashboard (TeessieDashboard.tsx)
with amber/charcoal theme, connectivity status, repeater cards with
7-day battery sparklines, MQTT observer grid, and decoded packet feed
- Backend: cross-network connectivity endpoint (hop-count packet matching),
radio-stats and radio-history proxy endpoints, observer-activity endpoint,
nodesDay stat field
- docker-compose: website-teesside service on port 3002
- Path-beta: worker pool, resolve cache, and pool refactor for performance
- Connection monitor, WS server and MQTT client improvements
- Mosquitto config, viewshed worker, and frontend UI polish
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New /repeater route with search and detailed repeater cards
- UK geographic bounds filter for viewshed calculations
- Advert packets API endpoint (GET /api/nodes/:id/adverts)
- Battery calculation fix (3.0V-4.2V range)
- NodeMarker optimizations (CircleMarker for repeaters)
- Map z-index adjustments for link lines
- Database performance indexes
- Nginx gzip compression
- Exclude mosquitto/acl and multipath.md from git
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gadgethd