Commit Graph
101 Commits
Author SHA1 Message Date
Ben 8db9a71e43 Rewrite map stack and tighten path topology 2026-03-19 22:42:35 +00:00
BenandClaude Sonnet 4.6 8b535b5eec fix(owner-portal): re-fetch node IDs from DB on session restore
Previously the /owner/session endpoint used node IDs baked into the
encrypted cookie at login time. If a node's public key changed, the
stale IDs would persist until the user manually logged out and back in.

Now re-fetches fresh node IDs from owner_account_nodes using the stored
MQTT username, falling back to cookie IDs only if the DB returns nothing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:36:38 +00:00
BenandClaude Sonnet 4.6 2d1c9cfa62 remove misleading 'tiles rendering' badge
The badge tracked Leaflet HTTP tile-load events, not actual server-side
rendering work (tiles are pre-rendered into Redis by the tile worker).
Counted normal browser tile fetches on pan/zoom, confusing users into
thinking the server was busy rendering. Removed badge, state, and CSS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:27:04 +00:00
BenandClaude Sonnet 4.6 83265e8df0 fix(comments): remove stale ScatterplotLayer references in MapView
DeckGLOverlay no longer has a ScatterplotLayer. Node dots are rendered
by server-side PNG tiles (/api/tiles/nodes). Update comments to reflect
the current architecture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:24:18 +00:00
BenandClaude Sonnet 4.6 acd71b0913 Fix owner portal login; add tile worker and cross-network improvements
Owner portal fixes:
- Fix session node ID case mismatch: decryptOwnerSession was lowercasing
  node IDs but /owner/live uppercased the ?nodeId= param, so the find()
  never matched — users always got "Node is not owned by this session"
- Revert sameSite strict → lax (broke sessions after March 18 deploy)
- Remove LOWER() wrappers from all owner live SQL (node IDs are uppercase)
- Add Fireball's second node (TS17 Yagi North) to owner_account_nodes

Tile worker:
- Server-side PNG tile rendering via @napi-rs/canvas
- Tile snapshot cache, Redis-backed tile cache, tile worker service
- /api/tiles/nodes/:z/:x/:y.png endpoint with rate limiter
- UK tile TTL logic

Cross-network connectivity:
- Add 7-day hourly history series to /cross-network-connectivity response
- Cache result for 60s to avoid repeated join-heavy query
- Fix query to use literal interval instead of interpolated variable

Other:
- Remove LOWER() from packet/node SQL throughout (IDs stored uppercase)
- Various performance and cleanup improvements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 15:55:47 +00:00
BenandClaude Sonnet 4.6 5d0281b922 Fix DeckGL/Leaflet 24px offset; add rate limits and security headers
- Wrap MapView + DeckGLOverlay in .map-layer container so DeckGL canvas
  anchors to the map area (not app-shell), eliminating the 24px topbar
  offset that caused packet history lines to misalign with node dots
- Add EXPENSIVE_LIMITER (12 req/min) to /observer-activity and
  /cross-network-connectivity endpoints
- Tighten owner login cookie to sameSite: strict
- Add Content-Security-Policy and Permissions-Policy response headers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 17:51:39 +00:00
BenandClaude Sonnet 4.6 b3562f774d Remove orphaned TeessieDashboard import from main.tsx
The teesside directory was gitignored in 3c148b6 but the static import
and routing branch in main.tsx were left behind, breaking tsc for fresh
clones. Remove both the import and the teesside routing branch; the
site now falls through to the default Layout, matching the intent of
the original commit.

Fixes gadgethd/ukmesh#1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 16:59:38 +00:00
BenandClaude Sonnet 4.6 9f00151ae1 Perf & UX improvements: stats cache, CPU fix, observer names on feed
- Pre-compute /api/stats/charts every 30 min with inflight dedup; cache served instantly to all visitors
- Fix CPU usage stat: systemStats() was called twice per health request, giving near-zero second reading
- De-duplicate observer region packet counts by packet_hash (was counting per observer row)
- Show repeater names on feed instead of rx counter / "heard by N"
- Observer names span full row width on /feed packet list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 22:56:24 +00:00
BenandClaude Sonnet 4.6 cc1118cb7c Performance improvements: fix WS initial state, debounce broadcasts, gzip, caching
Backend:
- getViableLinks: replace correlated EXISTS subquery (2m 49s!) with CTE-based
  join on pre-computed node set (39ms) — eliminates WS initial state timeouts
- WS initial state: switch from getLastNPackets (1,872ms, 24h correlated
  subqueries) to getRecentPackets (16ms, 5-min CTE) — cold connect now 30ms
- Debounce emitNode/emitNodeUpsert with 500ms window to coalesce rapid advert
  bursts before publishing to Redis (10-100x fewer publishes during active periods)
- Pre-normalise node IDs to lowercase at broadcast time; remove per-client
  toLowerCase() allocations in shouldSendMessage (~200 allocs/sec eliminated)
- Cache /api/path-beta/history (60s TTL) and /api/coverage (30s TTL)
- Add 10-min TTL to resolveCache to prevent multi-day unbounded growth
- Fix LOWER() on node_status_samples queries suppressing index use
- Add Express compression() middleware; add gzip to nginx.website.conf
- Increase viableLinksCache TTL 30s→5min, INITIAL_STATE_TTL 30s→60s
- Pre-warm initial state cache at startup for teesside+ukmesh
- Add compression package dependency

Frontend:
- Lazy-load coverage (26MB GeoJSON) — only fetch when coverage toggle is on,
  not on every page mount regardless of filter state
- Remove standalone 30s setInterval from useDashboardStats; feed stats from
  App.tsx consolidated poll instead (-1 HTTP request per 30s)
- Pre-compute DeckGL packet-history segment colour+width inside useMemo;
  getColor/getWidth become pure property lookups (~6000 log() calls/sec removed)
- DeckGLOverlay.tsx: add HistorySegmentWithColor type for pre-computed values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 22:36:42 +00:00
BenandClaude Sonnet 4.6 caaa45c5d6 Remove private radio bot URL from public config defaults
RADIO_BOT_URL is a private integration — strip the default service name
from docker-compose and remove it from the README env table.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:09:09 +00:00
BenandClaude Sonnet 4.6 441f80229c Update README and open source page
- README: correct GitHub URL, add path-history-worker, RADIO_BOT_URL env var,
  path resolver worker pool notes, updated services table and architecture diagram
- OpenSourcePage: fix GitHub link (teesside-mesh → ukmesh), add Recharts,
  polygon-clipping, cors, express-rate-limit; update geospatial section
  (GDAL, psycopg2, redis-py in place of rasterio; clarify world-atlas usage)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:07:22 +00:00
BenandClaude Sonnet 4.6 3c148b6df0 Remove Teesside Mesh site from repo
Teesside dashboard, CSS theme, and favicon are now gitignored —
they live on the VM but are not part of the ukmesh.com project.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:03:09 +00:00
BenandClaude Sonnet 4.6 0c5c97df39 Add Teesside Mesh dashboard and path-beta worker improvements
- New www.teessidemesh.com single-page dashboard (TeessieDashboard.tsx)
  with amber/charcoal theme, connectivity status, repeater cards with
  7-day battery sparklines, MQTT observer grid, and decoded packet feed
- Backend: cross-network connectivity endpoint (hop-count packet matching),
  radio-stats and radio-history proxy endpoints, observer-activity endpoint,
  nodesDay stat field
- docker-compose: website-teesside service on port 3002
- Path-beta: worker pool, resolve cache, and pool refactor for performance
- Connection monitor, WS server and MQTT client improvements
- Mosquitto config, viewshed worker, and frontend UI polish

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:01:54 +00:00
BenandClaude Opus 4.6 ad4ce0dbb1 Add repeater search page, UK bounds filter, and performance improvements
- New /repeater route with search and detailed repeater cards
- UK geographic bounds filter for viewshed calculations
- Advert packets API endpoint (GET /api/nodes/:id/adverts)
- Battery calculation fix (3.0V-4.2V range)
- NodeMarker optimizations (CircleMarker for repeaters)
- Map z-index adjustments for link lines
- Database performance indexes
- Nginx gzip compression
- Exclude mosquitto/acl and multipath.md from git

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 22:31:10 +00:00
Ben 6242fbaa98 Ship feed, observer, and pathing fixes 2026-03-14 13:45:30 +00:00
Ben 7300d53025 Stabilize redacted node uncertainty circles 2026-03-13 20:54:03 +00:00
Ben 007d538da9 Harden map privacy and telemetry displays 2026-03-13 19:21:48 +00:00
Ben ebb8fe55b4 Ship path history, site consolidation, and telemetry fixes 2026-03-12 23:12:33 +00:00
Ben 77dafab7db Fix Teesside node scope and add flasher nav 2026-03-09 21:54:28 +00:00
Ben c5c4053093 Prioritize repeater markers over client nodes 2026-03-09 19:03:28 +00:00
Ben 4793fb9a6e Label database clients by service 2026-03-09 02:48:59 +00:00
Ben ca94303fe9 Reduce website polling pressure 2026-03-09 02:41:48 +00:00
Ben 1f0e4addde Improve stats charts and path hash reporting 2026-03-09 02:20:00 +00:00
Ben 10136d5582 Add observer region stats cards 2026-03-08 21:25:28 +00:00
Ben f294b96592 Remove path simulation workers 2026-03-08 12:53:44 +00:00
Ben 9e25f876c9 Label control packets as CTL 2026-03-08 12:42:49 +00:00
BenandClaude Opus 4.6 b9a231ef36 Dedup relay copies per observer and increase DB connection limit
Drop duplicate MQTT packets when the same packet hash arrives at the
same observer with the same hop count within 120s. Increase TimescaleDB
max_connections to 300 to prevent connection exhaustion on startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:26:09 +00:00
BenandClaude Opus 4.6 2de39a871a Multi-observer path resolution: anchor-constrained solving with corridor relaxation
- Add server-side multi-observer resolver that solves a shared backbone
  once (anchor observer), then constrains other observers to reuse the
  same nodes for shared path hash prefixes, branching only at divergence
- New GET /api/path-beta/resolve-multi endpoint returns consistent
  per-observer results in a single request
- Relax corridor filtering for multi-observer packets: candidates
  accepted if within ANY observer's corridor, not just the primary
- Strengthen observer hop hint weights (0.09→0.18 confirmed,
  0.11→0.22 reachable, 0.10→0.20 fallback) for better cross-observer
  candidate steering
- Enumerate permutations from fork point for divergent suffixes
- Frontend prefers multi endpoint when 2+ observers, with cache
  keyed by sorted observer IDs for automatic invalidation on new
  observations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:19:27 +00:00
Ben 8eadea466e Handle exact and invalid multibyte paths 2026-03-08 11:59:35 +00:00
BenandClaude Opus 4.6 954b3ce853 Harden MeshCore packet decoding: extract routeType, validate paths, guard compat rewrite
- Extract and persist routeType from wire header byte (was always undefined)
- Log warnings on truncated path data and reserved hash-size modes
- Guard buildCompatRawHex against pathByteLength > 63 overflow
- Extract transport codes (4 bytes) when present in FLOOD/DIRECT+codes packets
- Validate path hash hex lengths against wire-format hash size in resolver
- Add routeType to LivePacket type for downstream consumers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 11:49:49 +00:00
Ben f0c2248c18 Add cookie banner and raw test feed view 2026-03-08 04:04:22 +00:00
Ben f5fcd68105 Refresh README for current public features 2026-03-08 04:00:53 +00:00
Ben b908fee98f Cull map rendering and correct health CPU metric 2026-03-08 03:53:29 +00:00
BenandClaude Sonnet 4.6 7b9ac180a9 Performance: push dedup to SQL, periodic cache eviction, owner/live cache
- getOwnerNodeIdsForUsername/getMappedOwnerNodeIds: move dedup and format
  validation into SQL (SELECT DISTINCT + regex filter) instead of JS post-processing
- viableLinksCache: add setInterval to evict stale entries every 30s so unused
  network/observer combos don't persist in memory indefinitely
- /owner/live: add 5s in-memory cache per nodeId to avoid 6 heavy parallel
  DB queries on every poll cycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 03:47:26 +00:00
BenandClaude Sonnet 4.6 0e8dba2b3d Fix low-severity security issues: headers, body limit, log redaction, telemetry validation
- Add X-Content-Type-Options, X-Frame-Options, and Referrer-Policy headers to all responses
- Cap express.json() body to 50kb to prevent memory exhaustion via large payloads
- Redact credentials from MQTT broker URL before logging (ws://user:pass@host pattern)
- Whitelist telemetry event kind field instead of accepting arbitrary strings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 03:44:09 +00:00
BenandClaude Sonnet 4.6 bbadebd761 Security hardening: input validation, bounded caches, and cookie secret enforcement
- Enforce OWNER_COOKIE_SECRET env var — throw on startup if unset rather than
  falling back to a weak derived secret
- Fix HTTPS detection to use exact string match instead of .includes()
- Cap and validate MQTT login credentials (32 char max, username format regex)
- Validate packetHash query param as hex before hitting the path resolver
- Validate node ID path params as 64-char hex in /nodes/:id/links and /nodes/:id/history
- Bound viableLinksCache to 50 entries with oldest-first eviction
- Bound countedAdvertHashes to 10,000 entries with oldest-first eviction

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 03:34:08 +00:00
Ben fca3de4bc1 Prefer rich packet rows for beta path resolution 2026-03-08 03:17:50 +00:00
Ben 7e4d9749ae Improve path resolution and backend connection handling 2026-03-08 03:03:58 +00:00
Ben 0d111e16c6 Tighten MQTT ingest and owner/path handling 2026-03-08 02:31:31 +00:00
Ben f84553bb3e Refactor path and packet helper modules 2026-03-08 02:06:00 +00:00
Ben 70d7b4c592 Improve beta path fallback and hide invalid map nodes 2026-03-08 01:56:45 +00:00
Ben bfafc8a4a5 Reduce live feed client retention 2026-03-08 01:28:08 +00:00
Ben a7503ef3f2 Refine coverage, owner links, and UK feed 2026-03-08 01:16:58 +00:00
Ben d2bac3317d Fix multi-observer packet path overlays 2026-03-08 00:09:40 +00:00
BenandClaude Sonnet 4.6 bf182b8ff9 Fix beta path hop ordering and suppress red segments overlapping purple paths
- Resolver now picks shortest-hop observation (ORDER BY hop_count ASC) instead of
  most recent arrival, matching what the live feed shows for flooding packets
- Red path segments that share an edge with any purple (high-confidence) path are
  filtered out so purple always takes visual precedence

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 23:32:55 +00:00
BenandClaude Sonnet 4.6 c9511d8eb9 Fix multi-observer path rendering, links flicker, stale node health, and login
- usePacketPathOverlay: don't cache null beta predictions; re-fetch pinned
  packet paths when new observers arrive after pin
- MapView: stable linkedNodesKey fingerprint so linkLines memo only recomputes
  when linked node positions/roles change, not on every heartbeat
- health/status + api/routes: exclude test-network-only nodes from stale count
  and mqttNodes stat so they don't inflate thresholds
- mosquitto/acl: add meshcore-test/# readwrite for backend user
- useNodes/PacketFeed/App: merge rx_count/tx_count from DB, ALL badge when all
  MQTT regions hear a packet, improved summary extraction
- db/index: getRecentPackets now deduplicates by packet_hash with observer
  aggregation; getLastNPackets adds rx_count/tx_count and _summary support

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 22:53:03 +00:00
Ben 41c01bb70b Add isolated test feed and fix live public stats 2026-03-07 21:27:59 +00:00
Ben cae0580609 Support multibyte path hashes across live pathing 2026-03-07 15:51:29 +00:00
Ben 814c94d01f Update README for owner portal and current coverage 2026-03-07 03:25:54 +00:00
Ben 05514fdc18 Add owner auth DB, website refresh, and coverage rollback 2026-03-07 03:23:52 +00:00