Commit Graph
83 Commits
Author SHA1 Message Date
BenandClaude Sonnet 4.6 015dfd2bab Fix terrain toggle, preference restore, and low-zoom crash
- Load terrain source and hillshade dynamically (only when 3D mode is on)
  to prevent tile loading at idle, fixing low-zoom WebGL crash
- Restore terrain correctly on page reload by applying inside map load handler
- Clean up source, hillshade, and sky layers fully when terrain is toggled off
- Raise minZoom to 6 to prevent crash at extreme zoom-out
- Reduce terrain exaggeration to 3x as a stable middle ground

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 01:12:41 +00:00
BenandClaude Sonnet 4.6 370348d9a1 Add UK terrain heightmap with hillshade to map
- Add SRTM raster-dem terrain source (AWS Terrarium tiles) to map style
- Add hillshade layer always visible for terrain relief in 2D and 3D
- Add sky layer for atmosphere effect in 3D mode
- Replace Links filter row with 3D Terrain toggle in the Layers panel
- Lock map pitch to 0 by default; terrain toggle unlocks and tilts to 45°
- Wire showTerrain prop through Filters → App → MapLibreMap via useEffect

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 00:38:47 +00:00
BenandClaude Sonnet 4.6 f8e8717454 Add lazy path resolver, ACL watcher, feed map, and observability stack
- Lazy path resolver: geographic hop reconstruction from path_hashes without
  pre-training data; fixes observer self-hash trimming bug
- ACL watcher worker: auto-provisions Mosquitto ACL entries when PUBLISH is
  denied for a known user; eliminates the recurring gnomemoe ACL removal issue
- connectionMonitor: increase historical scan to 50 MB; fix OOM by streaming
  lines instead of buffering; fixes missed backfill on restart
- Feed page: replace path-beta with lazy paths; add clickable node markers
  showing name and public key; observer column nowrap; summary text unclipped
- PacketFeed: GRP-only message store protected from ADV eviction; scroll to
  bottom on new message; 24-hour message backfill on initial load
- getRecentMessages: separate DB query for last 50 GRP messages last 24h
- getPacketDetail: new endpoint for full packet detail including observations
- Grafana/Loki/Promtail observability stack with Docker log collection

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 23:28:57 +00:00
Ben 89f06ca275 Refine path resolution and owner telemetry 2026-03-22 11:27:09 +00:00
Ben 7d57943b60 Improve owner dashboard telemetry and bare-domain support
- narrow owner last-hop telemetry to the selected MQTT node and cache it per node with hourly background refresh
- tighten repeater-only last-hop classification and clean up the owner chart selector/filtering
- fix owner direct-sender map basemap, static behavior, and viewport fitting/centering
- allow bare ukmesh.com and teessidemesh.com host handling in backend/frontend config
2026-03-21 00:16:21 +00:00
Ben 414945d875 Improve owner last-hop telemetry chart 2026-03-20 18:33:32 +00:00
Ben 21224a0525 Refactor backend, map, and worker boundaries
- split backend API into route/service/repository boundaries across stats, owner, and pathing\n- extract shared API bootstrap and helper modules\n- reduce MapLibreMap into smaller frontend map modules with extracted builders, config, popup UI, and types\n- continue worker modularization by extracting RF terrain/tile helpers alongside RF config and loss helpers\n- add repo-local architecture, DB lifecycle, link model, pathing, frontend map, and contributing docs\n\nThis is the verified refactor checkpoint: backend and frontend builds passed, worker syntax checks passed, affected containers were rebuilt, and /healthz is OK.
2026-03-20 17:45:47 +00:00
Ben 812aa95837 Tune RF link model and fix map stale link nodes 2026-03-20 00:41:02 +00:00
Ben 8db9a71e43 Rewrite map stack and tighten path topology 2026-03-19 22:42:35 +00:00
BenandClaude Sonnet 4.6 2d1c9cfa62 remove misleading 'tiles rendering' badge
The badge tracked Leaflet HTTP tile-load events, not actual server-side
rendering work (tiles are pre-rendered into Redis by the tile worker).
Counted normal browser tile fetches on pan/zoom, confusing users into
thinking the server was busy rendering. Removed badge, state, and CSS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:27:04 +00:00
BenandClaude Sonnet 4.6 83265e8df0 fix(comments): remove stale ScatterplotLayer references in MapView
DeckGLOverlay no longer has a ScatterplotLayer. Node dots are rendered
by server-side PNG tiles (/api/tiles/nodes). Update comments to reflect
the current architecture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 16:24:18 +00:00
BenandClaude Sonnet 4.6 acd71b0913 Fix owner portal login; add tile worker and cross-network improvements
Owner portal fixes:
- Fix session node ID case mismatch: decryptOwnerSession was lowercasing
  node IDs but /owner/live uppercased the ?nodeId= param, so the find()
  never matched — users always got "Node is not owned by this session"
- Revert sameSite strict → lax (broke sessions after March 18 deploy)
- Remove LOWER() wrappers from all owner live SQL (node IDs are uppercase)
- Add Fireball's second node (TS17 Yagi North) to owner_account_nodes

Tile worker:
- Server-side PNG tile rendering via @napi-rs/canvas
- Tile snapshot cache, Redis-backed tile cache, tile worker service
- /api/tiles/nodes/:z/:x/:y.png endpoint with rate limiter
- UK tile TTL logic

Cross-network connectivity:
- Add 7-day hourly history series to /cross-network-connectivity response
- Cache result for 60s to avoid repeated join-heavy query
- Fix query to use literal interval instead of interpolated variable

Other:
- Remove LOWER() from packet/node SQL throughout (IDs stored uppercase)
- Various performance and cleanup improvements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 15:55:47 +00:00
BenandClaude Sonnet 4.6 5d0281b922 Fix DeckGL/Leaflet 24px offset; add rate limits and security headers
- Wrap MapView + DeckGLOverlay in .map-layer container so DeckGL canvas
  anchors to the map area (not app-shell), eliminating the 24px topbar
  offset that caused packet history lines to misalign with node dots
- Add EXPENSIVE_LIMITER (12 req/min) to /observer-activity and
  /cross-network-connectivity endpoints
- Tighten owner login cookie to sameSite: strict
- Add Content-Security-Policy and Permissions-Policy response headers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 17:51:39 +00:00
BenandClaude Sonnet 4.6 b3562f774d Remove orphaned TeessieDashboard import from main.tsx
The teesside directory was gitignored in 3c148b6 but the static import
and routing branch in main.tsx were left behind, breaking tsc for fresh
clones. Remove both the import and the teesside routing branch; the
site now falls through to the default Layout, matching the intent of
the original commit.

Fixes gadgethd/ukmesh#1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 16:59:38 +00:00
BenandClaude Sonnet 4.6 9f00151ae1 Perf & UX improvements: stats cache, CPU fix, observer names on feed
- Pre-compute /api/stats/charts every 30 min with inflight dedup; cache served instantly to all visitors
- Fix CPU usage stat: systemStats() was called twice per health request, giving near-zero second reading
- De-duplicate observer region packet counts by packet_hash (was counting per observer row)
- Show repeater names on feed instead of rx counter / "heard by N"
- Observer names span full row width on /feed packet list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 22:56:24 +00:00
BenandClaude Sonnet 4.6 cc1118cb7c Performance improvements: fix WS initial state, debounce broadcasts, gzip, caching
Backend:
- getViableLinks: replace correlated EXISTS subquery (2m 49s!) with CTE-based
  join on pre-computed node set (39ms) — eliminates WS initial state timeouts
- WS initial state: switch from getLastNPackets (1,872ms, 24h correlated
  subqueries) to getRecentPackets (16ms, 5-min CTE) — cold connect now 30ms
- Debounce emitNode/emitNodeUpsert with 500ms window to coalesce rapid advert
  bursts before publishing to Redis (10-100x fewer publishes during active periods)
- Pre-normalise node IDs to lowercase at broadcast time; remove per-client
  toLowerCase() allocations in shouldSendMessage (~200 allocs/sec eliminated)
- Cache /api/path-beta/history (60s TTL) and /api/coverage (30s TTL)
- Add 10-min TTL to resolveCache to prevent multi-day unbounded growth
- Fix LOWER() on node_status_samples queries suppressing index use
- Add Express compression() middleware; add gzip to nginx.website.conf
- Increase viableLinksCache TTL 30s→5min, INITIAL_STATE_TTL 30s→60s
- Pre-warm initial state cache at startup for teesside+ukmesh
- Add compression package dependency

Frontend:
- Lazy-load coverage (26MB GeoJSON) — only fetch when coverage toggle is on,
  not on every page mount regardless of filter state
- Remove standalone 30s setInterval from useDashboardStats; feed stats from
  App.tsx consolidated poll instead (-1 HTTP request per 30s)
- Pre-compute DeckGL packet-history segment colour+width inside useMemo;
  getColor/getWidth become pure property lookups (~6000 log() calls/sec removed)
- DeckGLOverlay.tsx: add HistorySegmentWithColor type for pre-computed values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 22:36:42 +00:00
BenandClaude Sonnet 4.6 441f80229c Update README and open source page
- README: correct GitHub URL, add path-history-worker, RADIO_BOT_URL env var,
  path resolver worker pool notes, updated services table and architecture diagram
- OpenSourcePage: fix GitHub link (teesside-mesh → ukmesh), add Recharts,
  polygon-clipping, cors, express-rate-limit; update geospatial section
  (GDAL, psycopg2, redis-py in place of rasterio; clarify world-atlas usage)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:07:22 +00:00
BenandClaude Sonnet 4.6 3c148b6df0 Remove Teesside Mesh site from repo
Teesside dashboard, CSS theme, and favicon are now gitignored —
they live on the VM but are not part of the ukmesh.com project.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:03:09 +00:00
BenandClaude Sonnet 4.6 0c5c97df39 Add Teesside Mesh dashboard and path-beta worker improvements
- New www.teessidemesh.com single-page dashboard (TeessieDashboard.tsx)
  with amber/charcoal theme, connectivity status, repeater cards with
  7-day battery sparklines, MQTT observer grid, and decoded packet feed
- Backend: cross-network connectivity endpoint (hop-count packet matching),
  radio-stats and radio-history proxy endpoints, observer-activity endpoint,
  nodesDay stat field
- docker-compose: website-teesside service on port 3002
- Path-beta: worker pool, resolve cache, and pool refactor for performance
- Connection monitor, WS server and MQTT client improvements
- Mosquitto config, viewshed worker, and frontend UI polish

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 02:01:54 +00:00
BenandClaude Opus 4.6 ad4ce0dbb1 Add repeater search page, UK bounds filter, and performance improvements
- New /repeater route with search and detailed repeater cards
- UK geographic bounds filter for viewshed calculations
- Advert packets API endpoint (GET /api/nodes/:id/adverts)
- Battery calculation fix (3.0V-4.2V range)
- NodeMarker optimizations (CircleMarker for repeaters)
- Map z-index adjustments for link lines
- Database performance indexes
- Nginx gzip compression
- Exclude mosquitto/acl and multipath.md from git

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 22:31:10 +00:00
Ben 6242fbaa98 Ship feed, observer, and pathing fixes 2026-03-14 13:45:30 +00:00
Ben 7300d53025 Stabilize redacted node uncertainty circles 2026-03-13 20:54:03 +00:00
Ben 007d538da9 Harden map privacy and telemetry displays 2026-03-13 19:21:48 +00:00
Ben ebb8fe55b4 Ship path history, site consolidation, and telemetry fixes 2026-03-12 23:12:33 +00:00
Ben 77dafab7db Fix Teesside node scope and add flasher nav 2026-03-09 21:54:28 +00:00
Ben c5c4053093 Prioritize repeater markers over client nodes 2026-03-09 19:03:28 +00:00
Ben ca94303fe9 Reduce website polling pressure 2026-03-09 02:41:48 +00:00
Ben 1f0e4addde Improve stats charts and path hash reporting 2026-03-09 02:20:00 +00:00
Ben 10136d5582 Add observer region stats cards 2026-03-08 21:25:28 +00:00
Ben f294b96592 Remove path simulation workers 2026-03-08 12:53:44 +00:00
Ben 9e25f876c9 Label control packets as CTL 2026-03-08 12:42:49 +00:00
BenandClaude Opus 4.6 2de39a871a Multi-observer path resolution: anchor-constrained solving with corridor relaxation
- Add server-side multi-observer resolver that solves a shared backbone
  once (anchor observer), then constrains other observers to reuse the
  same nodes for shared path hash prefixes, branching only at divergence
- New GET /api/path-beta/resolve-multi endpoint returns consistent
  per-observer results in a single request
- Relax corridor filtering for multi-observer packets: candidates
  accepted if within ANY observer's corridor, not just the primary
- Strengthen observer hop hint weights (0.09→0.18 confirmed,
  0.11→0.22 reachable, 0.10→0.20 fallback) for better cross-observer
  candidate steering
- Enumerate permutations from fork point for divergent suffixes
- Frontend prefers multi endpoint when 2+ observers, with cache
  keyed by sorted observer IDs for automatic invalidation on new
  observations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 12:19:27 +00:00
Ben f0c2248c18 Add cookie banner and raw test feed view 2026-03-08 04:04:22 +00:00
Ben b908fee98f Cull map rendering and correct health CPU metric 2026-03-08 03:53:29 +00:00
Ben fca3de4bc1 Prefer rich packet rows for beta path resolution 2026-03-08 03:17:50 +00:00
Ben 7e4d9749ae Improve path resolution and backend connection handling 2026-03-08 03:03:58 +00:00
Ben 0d111e16c6 Tighten MQTT ingest and owner/path handling 2026-03-08 02:31:31 +00:00
Ben f84553bb3e Refactor path and packet helper modules 2026-03-08 02:06:00 +00:00
Ben 70d7b4c592 Improve beta path fallback and hide invalid map nodes 2026-03-08 01:56:45 +00:00
Ben bfafc8a4a5 Reduce live feed client retention 2026-03-08 01:28:08 +00:00
Ben a7503ef3f2 Refine coverage, owner links, and UK feed 2026-03-08 01:16:58 +00:00
Ben d2bac3317d Fix multi-observer packet path overlays 2026-03-08 00:09:40 +00:00
BenandClaude Sonnet 4.6 bf182b8ff9 Fix beta path hop ordering and suppress red segments overlapping purple paths
- Resolver now picks shortest-hop observation (ORDER BY hop_count ASC) instead of
  most recent arrival, matching what the live feed shows for flooding packets
- Red path segments that share an edge with any purple (high-confidence) path are
  filtered out so purple always takes visual precedence

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 23:32:55 +00:00
BenandClaude Sonnet 4.6 c9511d8eb9 Fix multi-observer path rendering, links flicker, stale node health, and login
- usePacketPathOverlay: don't cache null beta predictions; re-fetch pinned
  packet paths when new observers arrive after pin
- MapView: stable linkedNodesKey fingerprint so linkLines memo only recomputes
  when linked node positions/roles change, not on every heartbeat
- health/status + api/routes: exclude test-network-only nodes from stale count
  and mqttNodes stat so they don't inflate thresholds
- mosquitto/acl: add meshcore-test/# readwrite for backend user
- useNodes/PacketFeed/App: merge rx_count/tx_count from DB, ALL badge when all
  MQTT regions hear a packet, improved summary extraction
- db/index: getRecentPackets now deduplicates by packet_hash with observer
  aggregation; getLastNPackets adds rx_count/tx_count and _summary support

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 22:53:03 +00:00
Ben 41c01bb70b Add isolated test feed and fix live public stats 2026-03-07 21:27:59 +00:00
Ben cae0580609 Support multibyte path hashes across live pathing 2026-03-07 15:51:29 +00:00
Ben 05514fdc18 Add owner auth DB, website refresh, and coverage rollback 2026-03-07 03:23:52 +00:00
Ben 729a966e6a Add server-side beta path resolver and mobile layer controls
- move beta path resolution onto the backend with packet-level diagnostics and live API access
- add path simulation worker plumbing, health/schema updates, and compose wiring
- update app overlays to fetch server-computed beta paths with animated rendering and request caching
- refine mobile controls so map layers collapse behind a toggle and compact the links legend
2026-03-07 00:42:31 +00:00
Ben fe56f39007 Refine beta path rendering, confidence split, and adverts source anchoring 2026-03-06 00:44:07 +00:00
Ben 0d77a5027c feat: update owner auth and refresh stats visualizations 2026-03-05 23:05:29 +00:00