Kpa-clawbot 2435f2eaaf fix: observation timestamps, leaked fields, perf path normalization
- #178: Use strftime ISO 8601 format instead of datetime() for observation
  timestamps in all SQL queries (v3 + v2 views). Add normalizeTimestamp()
  helper for non-v3 paths that may store space-separated timestamps.

- #179: Strip internal fields (decoded_json, direction, payload_type,
  raw_hex, route_type, score, created_at) from ObservationResp. Only
  expose id, transmission_id, observer_id, observer_name, snr, rssi,
  path_json, timestamp — matching Node.js parity.

- #180: Remove _parsedDecoded and _parsedPath from node detail
  recentAdverts response. These internal/computed fields were leaking
  to the API. Updated golden shapes.json accordingly.

- #181: Use mux route template (GetPathTemplate) for perf stats path
  normalization, converting {param} to :param for Node.js parity.
  Fallback to hex regex for unmatched routes. Compile regexes once at
  package level instead of per-request.

fixes #178, fixes #179, fixes #180, fixes #181

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 18:09:36 -07:00
2026-03-27 20:22:33 +00:00
2026-03-27 18:02:13 -07:00
2026-03-24 21:06:04 +00:00
2026-03-20 05:38:23 +00:00

MeshCore Analyzer

Backend Tests Backend Coverage Frontend Tests Frontend Coverage Deploy

Self-hosted, open-source MeshCore packet analyzer — a community alternative to the closed-source analyzer.letsmesh.net.

Collects MeshCore packets via MQTT, decodes them, and presents a full web UI with live packet feed, node map, channel chat, packet tracing, per-node analytics, and more.

Features

📡 Live Trace Map

Real-time animated map with packet route visualization, VCR-style playback controls, and a retro LCD clock. Replay the last 24 hours of mesh activity, scrub through the timeline, or watch packets flow live at up to 4× speed.

Live VCR playback — watch packets flow across the Bay Area mesh

📦 Packet Feed

Filterable real-time packet stream with byte-level breakdown, Excel-like resizable columns, and a detail pane. Toggle "My Nodes" to focus on your mesh.

Packets view

🗺️ Network Overview

At-a-glance mesh stats — node counts, packet volume, observer coverage.

Network overview

🔀 Route Patterns

Visualize how packets traverse the mesh — see which repeaters carry the most traffic and identify routing patterns.

Route patterns

📊 Node Analytics

Per-node deep dive with 6 interactive charts: activity timeline, packet type breakdown, SNR distribution, hop count analysis, peer network graph, and hourly heatmap.

Node analytics

💬 Channel Chat

Decoded group messages with sender names, @mentions, timestamps — like reading a Discord channel for your mesh.

Channels

📱 Mobile Ready

Full experience on your phone — proper touch controls, iOS safe area support, and a compact VCR bar that doesn't fight your thumb.

Live view on iOS

And More

  • Node Directory — searchable list with role tabs, detail panel, QR codes, advert timeline, "Heard By" observer table
  • Packet Tracing — follow individual packets across observers with SNR/RSSI timeline
  • Observer Status — health monitoring, packet counts, uptime
  • Hash Collision Matrix — detect address collisions across the mesh
  • Claimed Nodes — star your nodes, always sorted to top, visual distinction
  • Dark / Light Mode — auto-detects system preference, instant toggle, map tiles swap too
  • Multi-Broker MQTT — connect to multiple MQTT brokers simultaneously with per-source IATA filtering
  • Observer Detail Pages — click any observer for analytics, charts, status, radio info, recent packets
  • Channel Key Auto-Derivation — hashtag channels (#channel) keys derived automatically via SHA256
  • Global Search — search packets, nodes, and channels (Ctrl+K)
  • Shareable URLs — deep links to individual packets, channels, and observer detail pages
  • Mobile Responsive — proper two-row VCR bar, iOS safe area support, touch-friendly
  • Accessible — ARIA patterns, keyboard navigation, screen reader support, distinct marker shapes

Performance (v2.1.1)

Two-layer caching architecture: in-memory packet store + TTL response cache. All packet reads served from RAM — SQLite is write-only. Heavy endpoints pre-warmed on startup.

Endpoint Before After Speedup
Bulk Health 7,059 ms 1 ms 7,059×
Node Analytics 381 ms 1 ms 381×
Topology 685 ms 2 ms 342×
Node Health 195 ms 1 ms 195×
Node Detail 133 ms 1 ms 133×

See PERFORMANCE.md for the full benchmark.

Quick Start

git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd meshcore-analyzer
./manage.sh setup

The setup wizard walks you through everything — config, domain, HTTPS, build, and run. Safe to cancel and re-run at any point.

After setup:

./manage.sh status       # Health check + packet/node counts
./manage.sh logs         # Follow logs
./manage.sh backup       # Backup database
./manage.sh update       # Pull latest + rebuild + restart
./manage.sh mqtt-test    # Check if observer data is flowing
./manage.sh help         # All commands

See docs/DEPLOYMENT.md for the full deployment guide, HTTPS options (auto cert, bring your own, Cloudflare Tunnel), MQTT security, backups, and troubleshooting.

Theme customization: Use the built-in customizer (Tools → Customize) to design your theme, download the theme.json file, and place it next to your config.json. Changes are picked up on page refresh.

Manual Install

Prerequisites

  • Node.js 18+ (tested with 22.x)
  • MQTT broker (Mosquitto recommended) — optional, can inject packets via API

Install

git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd meshcore-analyzer
npm install

Configure

Edit config.json:

{
  "port": 3000,
  "https": {
    "cert": "/path/to/cert.pem",
    "key": "/path/to/key.pem"
  },
  "mqtt": {
    "broker": "mqtt://localhost:1883",
    "topic": "meshcore/+/+/packets"
  },
  "mqttSources": [
    {
      "name": "remote-feed",
      "broker": "mqtts://remote-broker:8883",
      "topics": ["meshcore/+/+/packets", "meshcore/+/+/status"],
      "username": "user",
      "password": "pass",
      "rejectUnauthorized": false,
      "iataFilter": ["SJC", "SFO", "OAK"]
    }
  ],
  "channelKeys": {
    "public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
  },
  "defaultRegion": "SJC",
  "regions": {
    "SJC": "San Jose, US",
    "SFO": "San Francisco, US",
    "OAK": "Oakland, US"
  }
}
Field Description
port HTTP server port (default: 3000)
https.cert / https.key Optional PEM cert/key paths to enable native HTTPS (falls back to HTTP if omitted or unreadable)
mqtt.broker Local MQTT broker URL. Set to "" to disable
mqtt.topic MQTT topic pattern for packet ingestion
mqttSources Array of external MQTT broker connections (optional)
mqttSources[].name Friendly name for logging
mqttSources[].broker Broker URL (mqtt:// or mqtts:// for TLS)
mqttSources[].topics Array of MQTT topic patterns to subscribe to
mqttSources[].username / password Broker credentials
mqttSources[].rejectUnauthorized Set false for self-signed TLS certs
mqttSources[].iataFilter Only accept packets from these IATA regions
channelKeys Named channel decryption keys (hex). Hashtag channels auto-derived via SHA256
defaultRegion Default IATA region code for the UI
regions Map of IATA codes to human-readable region names

Run

node server.js

Open http://localhost:3000 in your browser.

Environment Variables

Variable Description
PORT Override config.json port
DB_PATH Override SQLite database path (default: data/meshcore.db)

Generate Test Data

# Generate and inject 200 packets via API
node tools/generate-packets.js --api --count 200

# Or output as JSON
node tools/generate-packets.js --json --count 50

Run Tests

# End-to-end test
DB_PATH=/tmp/test-e2e.db PORT=13590 node tools/e2e-test.js

# Frontend smoke test
DB_PATH=/tmp/test-fe.db PORT=13591 node tools/frontend-test.js

MQTT Setup

MeshCore packets flow into the analyzer via MQTT:

  1. Flash an observer node with MESH_PACKET_LOGGING=1 build flag
  2. Connect via USB to a host running meshcoretomqtt
  3. Configure meshcoretomqtt with your IATA region code and MQTT broker address
  4. Packets appear on topic meshcore/{IATA}/{PUBKEY}/packets

Alternatively, POST raw hex packets to POST /api/packets for manual injection.

Architecture

Observer Node → USB → meshcoretomqtt → MQTT Broker → Analyzer Server → WebSocket → Browser
                                                    → SQLite DB
                                                    → REST API

Project Structure

meshcore-analyzer/
├── Dockerfile           # Single-container build (Node + Mosquitto + Caddy)
├── .dockerignore
├── config.example.json  # Example config (copy to config.json)
├── config.json          # MQTT, channel keys, regions (gitignored)
├── server.js            # Express + WebSocket + MQTT + REST API
├── decoder.js           # Custom MeshCore packet decoder
├── db.js                # SQLite schema + queries
├── packet-store.js      # In-memory packet store (ring buffer, indexed)
├── docker/
│   ├── supervisord.conf # Process manager config
│   ├── mosquitto.conf   # MQTT broker config
│   ├── Caddyfile        # Default Caddy config (localhost)
│   └── entrypoint.sh    # Container entrypoint
├── data/
│   └── meshcore.db      # Packet database (auto-created)
├── public/
│   ├── index.html       # SPA shell
│   ├── style.css        # Theme (light/dark)
│   ├── app.js           # Router, WebSocket, utilities
│   ├── packets.js       # Packet feed + byte breakdown + detail page
│   ├── map.js           # Leaflet map with route visualization
│   ├── live.js          # Live trace page with VCR playback
│   ├── channels.js      # Channel chat
│   ├── nodes.js         # Node directory + detail views
│   ├── analytics.js     # Global analytics dashboard
│   ├── node-analytics.js # Per-node analytics with charts
│   ├── traces.js        # Packet tracing
│   ├── observers.js     # Observer status
│   ├── observer-detail.js # Observer detail with analytics
│   ├── home.js          # Dashboard home page
│   └── perf.js          # Performance monitoring dashboard
└── tools/
    ├── generate-packets.js  # Synthetic packet generator
    ├── e2e-test.js          # End-to-end API tests
    └── frontend-test.js     # Frontend smoke tests

License

MIT

Description
No description provided
Readme 105 MiB
Languages
JavaScript 49.5%
Go 38.4%
CSS 6.4%
Shell 3.7%
Python 1.4%
Other 0.5%