Compare commits

...

48 Commits

Author SHA1 Message Date
Kpa-clawbot 1191cfc017 fix(ci): restore Linux runner labels and action versions
The previous commit accidentally downgraded action versions (v5/v6 → v4)
and removed the Linux label from self-hosted runner selectors. This caused
Windows runners to pick up jobs with bash syntax, failing with PowerShell
parse errors (Missing '(' after 'if' in if statement).

Fixes:
- Restore runs-on: [self-hosted, Linux] for all self-hosted jobs
- Restore actions/checkout@v5, setup-go@v6, setup-node@v5
- Restore actions/upload-artifact@v5, download-artifact@v5
2026-03-29 15:39:46 +00:00
you 68bdaaca99 perf: speed up frontend coverage tests (~3x faster)
Three optimizations to the CI frontend test pipeline:

1. Run E2E tests and coverage collection concurrently
   - Previously sequential (E2E ~1.5min, then coverage ~5.75min)
   - Now both run in parallel against the same instrumented server
   - Expected savings: ~5 min (coverage runs alongside E2E instead of after)

2. Replace networkidle with domcontentloaded in coverage collector
   - SPA uses hash routing — networkidle waits 500ms for network silence
     on every navigation, adding ~10-15s of dead time across 23 navigations
   - domcontentloaded fires immediately once HTML is parsed; JS initializes
     the route handler synchronously
   - For in-page hash changes, use 200ms setTimeout instead of
     waitForLoadState (which would never re-fire for same-document nav)

3. Extract coverage from E2E tests too
   - E2E tests already exercise the app against the instrumented server
   - Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
   - nyc merges both coverage files for higher total coverage

Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
2026-03-29 14:35:05 +00:00
you 074f3d3760 ci: cancel workflow run immediately when any test job fails
When go-test or node-test fails, the workflow run is now cancelled
via the GitHub API so the sibling job doesn't sit queued/running.

Also fixed build job to need both go-test AND node-test (was only
waiting on go-test despite the pipeline comment saying both gate it).
2026-03-29 14:20:22 +00:00
you 206d9bd64a fix: use per-PR concurrency group to prevent cross-PR cancellation
The flat 'deploy' concurrency group caused ALL PRs to share one queue,
so pushing to any PR would cancel CI runs on other PRs.

Changed to deploy-${{ github.event.pull_request.number || github.ref }}
so each PR gets its own concurrency group while re-pushes to the same
PR still cancel the previous run.
2026-03-29 14:14:57 +00:00
efiten 3f54632b07 fix: cache /stats and GetNodeHashSizeInfo to eliminate slow API calls
- /api/stats: 10s server-side cache — was running 5 SQLite COUNT queries
  on every call, taking ~1500ms with 28 concurrent WS clients polling every 15s
- GetNodeHashSizeInfo: 15s cache — was doing a full O(n) scan + JSON unmarshal
  of all advert packets in memory on every /nodes request, taking ~1200ms

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 07:09:05 -07:00
Kpa-clawbot 609b12541e fix: add extra_hosts host.docker.internal to all services — fixes #238
Linux Docker doesn't resolve host.docker.internal by default.
Required when MQTT sources in config.json point to the host machine.
Harmless on Docker Desktop where it already works.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 18:58:31 -07:00
Kpa-clawbot 4369e58a3c Merge pull request #235 from Kpa-clawbot/fix/compose-build-directive
fix: docker-compose prod/staging need build: directive — fixes pull access denied
2026-03-28 18:36:21 -07:00
Kpa-clawbot 8ef321bf70 fix: add build context to prod and staging services in docker-compose.yml
Without build: directive, docker compose tries to pull corescope:latest
from Docker Hub instead of building locally.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 18:35:35 -07:00
Kpa-clawbot bee705d5d8 docs: add v3.1.0 release notes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 17:18:25 -07:00
Kpa-clawbot 9b2ad91512 Merge pull request #226 from Kpa-clawbot/rename/corescope-migration
docs: CoreScope rename migration guide
2026-03-28 16:44:56 -07:00
Kpa-clawbot 6740e53c18 Merge pull request #231 from Kpa-clawbot/refactor/manage-sh-compose-only
refactor: manage.sh uses docker compose only -- fixes #230
2026-03-28 16:26:01 -07:00
Kpa-clawbot b2e5b66f25 Merge remote-tracking branch 'origin/master' into refactor/manage-sh-compose-only 2026-03-28 16:25:47 -07:00
Kpa-clawbot 45b82ad390 Address PR #231 review: add docker compose check, document Caddy volumes
- Add preflight check for 'docker compose' in manage.sh (catches plugin missing)
- Document named Caddy volumes as cert storage, not user data

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 16:24:57 -07:00
KpaBap d538d2f3e7 Merge branch 'master' into rename/corescope-migration 2026-03-28 16:21:57 -07:00
Kpa-clawbot 746f7f2733 refactor: manage.sh uses docker compose only — fixes #230
Remove all legacy docker run code paths. manage.sh is now a pure
docker compose wrapper with no dual-mode branching.

Removed:
- COMPOSE_MODE flag and all if/else branches
- get_docker_run_args(), get_data_mount_args(), recreate_container()
- get_required_ports(), get_current_ports(), check_port_match()
- CONTAINER_NAME, DATA_VOLUME, CADDY_VOLUME variables
- All direct docker run/stop/start/rm invocations

All commands now delegate to docker compose:
- start → docker compose up -d prod
- stop → docker compose down / docker compose stop
- restart → docker compose up -d --force-recreate
- update → docker compose build prod + up -d --force-recreate
- reset → docker compose down --rmi local
- backup/restore use bind mount path from .env (PROD_DATA_DIR)
- verify_health, mqtt-test, status all use corescope-prod

Net result: -248 lines, zero dual-mode logic, identical behavior
to running docker compose directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 16:15:37 -07:00
Kpa-clawbot a1a67e89fb feat: manage.sh reads .env for data paths — consistent with docker compose
- Replace all hardcoded \C:\Users\KpaBap/meshcore-data with \ variable
- \ resolves from \ in .env or defaults to ~/meshcore-data
- Updated get_data_mount_args(), cmd_backup(), cmd_restore(), cmd_reset()
- Enhanced .env.example with detailed comments for each variable
- Both docker compose and manage.sh now read same .env file

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 16:06:13 -07:00
Kpa-clawbot 91fcbc5adc Fix: Use bind mounts instead of named volumes for data directory
PROBLEM:
manage.sh was using named Docker volumes (meshcore-data) as the default,
which hides the database and theme files inside Docker's internal storage.
Users couldn't find their DB on the filesystem for backups or inspection.

The function get_data_mount_args() had conditional logic that only used
bind mounts IF it detected an existing ~/meshcore-data with a DB file.
For new installs, it fell through to the named volume — silently hiding
all data in /var/lib/docker/volumes/.

FIXES:
1. get_data_mount_args() — Always use bind mount to ~/meshcore-data
   - Creates the directory if it doesn't exist
   - Removes all conditional logic and the named volume fallback

2. cmd_backup() — Use direct path C:\Users\KpaBap/meshcore-data/meshcore.db
   - No longer tries to inspect the named volume
   - Consistent with the bind mount approach

3. cmd_restore() — Use direct path for restore operations
   - Ensures directory exists before restoring files
   - No fallback to docker cp

4. cmd_reset() — Updated message to reflect bind mount location
   - Changed from 'docker volume rm' to '~/meshcore-data (not removed)'

5. docker-compose.yml — Added documentation comment
   - Clarifies that bind mounts are intentional, not named volumes
   - Ensures future changes maintain this pattern

VALIDATION:
- docker-compose.yml already used bind mounts correctly (\)
- Legacy 'docker run' mode now matches compose behavior
- All backup/restore operations reference the same bind mount path

DATABASE LOCATION:
- Always: ~/meshcore-data/meshcore.db
- Never: Hidden in Docker's volume storage

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Requested-by: Kpa-clawbot
2026-03-28 16:01:16 -07:00
Kpa-clawbot 5f5eae07b0 Merge pull request #222 from efiten/pr/perf-fix
perf: eliminate O(n) slice prepend on every packet ingest
2026-03-28 16:01:08 -07:00
efiten 380b1b1e28 fix: address review — observation ordering, stale comments, affected query functions
- Load() SQL: keep o.timestamp DESC (consistent with IngestNewFromDB) so
  pickBestObservation tie-breaking is identical on both load paths
- GetTimestamps: scan from tail instead of head (was breaking on first item
  assuming it was the newest, now correctly reads from newest end)
- QueryMultiNodePackets: apply same DESC/ASC tail-read pagination as
  QueryPackets (was sorting for ASC and assuming DESC as-is)
- GetNodeHealth recentPackets: read from tail to return 20 newest items
  (was reading from head = 20 oldest items)
- Remove stale "Prepend (newest first)" comments, replace with accurate
  "oldest-first; new items go to tail" wording

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 15:54:40 -07:00
efiten 03cfd114da perf: eliminate O(n) slice prepend on every packet ingest
s.packets and s.byPayloadType[t] were prepended on every new packet
to maintain newest-first order, copying the entire slice each time.
With 2-3M packets in memory this meant ~24MB of pointer copies per
ingest cycle, causing sustained high CPU and GC pressure.

Fix: store both slices oldest-first (append to tail). Load() SQL
changed to ASC ordering. QueryPackets DESC pagination now reads from
the tail in O(page_size) with no sort; GetChannelMessages switches
from reverse-iteration to forward-iteration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 15:54:40 -07:00
Kpa-clawbot df90de77a7 Merge pull request #219 from Kpa-clawbot/fix/hashchannels-derivation
fix: port hashChannels key derivation to Go ingestor (fixes #218)
2026-03-28 15:34:43 -07:00
copilot-swe-agent[bot] 7b97c532a1 test: fix env isolation and comment accuracy in channel key tests
Agent-Logs-Url: https://github.com/Kpa-clawbot/meshcore-analyzer/sessions/38b3e96f-861b-4929-8134-b1b9de39a7fc

Co-authored-by: KpaBap <746025+KpaBap@users.noreply.github.com>
2026-03-28 15:27:26 -07:00
Kpa-clawbot e0c2d37041 fix: port hashChannels key derivation to Go ingestor (fixes #218)
Add HashChannels config field and deriveHashtagChannelKey() to the Go
ingestor, matching the Node.js server-helpers.js algorithm:
SHA-256(channelName) -> first 32 hex chars (16 bytes AES-128 key).

Merge priority preserved: rainbow (lowest) -> derived -> explicit (highest).

Tests include cross-language vectors validated against Node.js output
and merge priority / normalization / skip-explicit coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 15:27:26 -07:00
Kpa-clawbot f5d0ce066b refactor: remove packets_v SQL fallbacks — store handles all queries (#220)
* refactor: remove all packets_v SQL fallbacks — store handles all queries

Remove DB fallback paths from all route handlers. The in-memory
PacketStore now handles all packet/node/analytics queries. Handlers
return empty results or 404 when no store is available instead of
falling back to direct DB queries.

- Remove else-DB branches from handlePacketDetail, handleNodeHealth,
  handleNodeAnalytics, handleBulkHealth, handlePacketTimestamps, etc.
- Remove unused DB methods (GetPacketByHash, GetTransmissionByID,
  GetPacketByID, GetObservationsForHash, GetTimestamps, GetNodeHealth,
  GetNodeAnalytics, GetBulkHealth, etc.)
- Remove packets_v VIEW creation from schema
- Update tests for new behavior (no-store returns 404/empty, not 500)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address PR #220 review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: KpaBap <kpabap@gmail.com>
2026-03-28 15:25:56 -07:00
Kpa-clawbot 202d0d87d7 ci: Add pull_request trigger to CI workflow
- Add pull_request trigger for PRs against master
- Add 'if: github.event_name == push' to build/deploy/publish jobs
- Test jobs (go-test, node-test) now run on both push and PRs
- Build/deploy/publish only run on push to master

This fixes the chicken-and-egg problem where branch protection requires
CI checks but CI doesn't run on PRs. Now PRs get test validation before
merge while keeping production deployments only on master pushes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 15:15:35 -07:00
Kpa-clawbot 99d2e67eb1 Rename Phase 1: MeshCore Analyzer -> CoreScope (backend + infra)
Reviewed by Kobayashi (gpt-5.3-codex). All comments addressed.
2026-03-28 14:45:24 -07:00
Kpa-clawbot a6413fb665 fix: address review — stale URLs, manage.sh branding, proto comment
- docs/go-migration.md: update clone URL meshcore-dev/meshcore-analyzer → Kpa-clawbot/meshcore-analyzer
- manage.sh: rename header comment and help footer from 'MeshCore Analyzer' to 'CoreScope'
- proto/config.proto: update default branding comment from 'MeshCore Analyzer' to 'CoreScope'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:44:53 -07:00
KpaBap 8a458c7c2a Merge pull request #227 from Kpa-clawbot/rename/corescope-frontend
rename: MeshCore Analyzer → CoreScope (frontend + .squad)
2026-03-28 14:39:06 -07:00
Kpa-clawbot 66b3c05da3 fix: remove stray backtick in template literal
Fixes malformed template literal in test assertion message that would cause a syntax error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:37:27 -07:00
Kpa-clawbot cdcaa476f2 rename: MeshCore Analyzer → CoreScope (Phase 1 — backend + infra)
Rename product branding, binary names, Docker images, container names,
Go modules, proto go_package, CI, manage.sh, and documentation.

Preserved (backward compat):
- meshcore.db database filename
- meshcore-data / meshcore-staging-data directory paths
- MQTT topics (meshcore/#, meshcore/+/+/packets, etc.)
- proto package namespace (meshcore.v1)
- localStorage keys

Changes by category:
- Go modules: github.com/corescope/{server,ingestor}
- Binaries: corescope-server, corescope-ingestor
- Docker images: corescope:latest, corescope-go:latest
- Containers: corescope-prod, corescope-staging, corescope-staging-go
- Supervisord programs: corescope, corescope-server, corescope-ingestor
- Branding: siteName, heroTitle, startup logs, fallback HTML
- Proto go_package: github.com/corescope/proto/v1
- CI: container refs, deploy path
- Docs: 8 markdown files updated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:08:15 -07:00
Kpa-clawbot 71ec5e6fca rename: MeshCore Analyzer → CoreScope (frontend + .squad)
Phase 1 of the CoreScope rename — frontend display strings and
squad agent metadata only.

index.html:
- <title>, og:title, twitter:title → CoreScope
- Brand text span → CoreScope
- og:image/twitter:image URLs → corescope repo (placeholder)
- Cache busters bumped

public/*.js headers (19 files):
- All file header comments updated

public/*.css headers:
- style.css, home.css updated

JavaScript strings:
- app.js: GitHub URL → corescope
- home.js: 3 fallback siteName references
- customize.js: default siteName + heroTitle

Tests:
- test-e2e-playwright.js: title assertion → corescope
- test-frontend-helpers.js: GitHub URL constant
- benchmark.js: header string
- test-all.sh: header string

.squad:
- team.md, casting/history.json
- All 7 agent charters + 5 history files

NOT renamed (intentional):
- localStorage keys (meshcore-*)
- CSS classes (.meshcore-marker)
- Window globals (_meshcore*)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:03:32 -07:00
Kpa-clawbot a94c24c550 fix: restore PR reviewer instructions with valid filename (was *.instructions.md)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:02:14 -07:00
Kpa-clawbot a1f95fee58 fix: Dockerfile .git-commit COPY fails on legacy builder — use RUN default
The glob trick COPY .git-commi[t] only works with BuildKit.
manage.sh uses legacy docker build. Just create a default via RUN.
Commit hash comes through --build-arg ldflags anyway.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 13:59:20 -07:00
Kpa-clawbot 24d76f8373 fix: remove file with * in name — breaks Windows/NTFS 2026-03-28 13:57:31 -07:00
Kpa-clawbot 1453fb6492 docs: add CoreScope rename migration guide
Documents what existing users need to update when the rename
from MeshCore Analyzer to CoreScope lands:
- Git remote URL update
- Docker image/container name changes
- Config branding.siteName (if customized)
- CI/CD references (if applicable)
- Confirms data dirs, MQTT, browser state unchanged

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 13:51:41 -07:00
KpaBap 8e18351c73 Merge pull request #221 from Kpa-clawbot/feat/telemetry-decode
feat: decode telemetry packets — battery voltage + temperature on nodes
2026-03-28 13:45:00 -07:00
Kpa-clawbot 5cc6064e11 fix: Dockerfile .git-commit COPY fails on legacy builder — use RUN default
The glob trick COPY .git-commi[t] only works with BuildKit.
manage.sh uses legacy docker build. Just create a default via RUN.
Commit hash comes through --build-arg ldflags anyway.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 13:36:37 -07:00
copilot-swe-agent[bot] a827fd3b43 fix: gate telemetry on sensor flag, fix 0°C emission, safe migration with PRAGMA check
Agent-Logs-Url: https://github.com/Kpa-clawbot/meshcore-analyzer/sessions/1c2af64b-0e8a-4dd0-ae80-e296f70437e9

Co-authored-by: KpaBap <746025+KpaBap@users.noreply.github.com>
2026-03-28 20:35:50 +00:00
KpaBap 467a307a8d Create MeshCore PR Reviewer instructions
Added instructions for the MeshCore PR Reviewer agent, detailing its role, core principles, review focus areas, and the review process.
2026-03-28 13:26:23 -07:00
KpaBap 077fca9038 Create MeshCore PR Reviewer agent
Added a new agent for reviewing pull requests in the meshcore-analyzer repository, focusing on best practices and code quality.
2026-03-28 13:16:03 -07:00
Kpa-clawbot b326e3f1a6 fix: pprof port conflict crashed Go server — non-fatal bind + separate ports
Server defaults to 6060, ingestor to 6061. Removed shared PPROF_PORT
env var. Bind failure logs warning instead of log.Fatal killing the process.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 13:01:41 -07:00
Kpa-clawbot 54cbc648e0 feat: decode telemetry from adverts — battery voltage + temperature on nodes
Sensor nodes embed telemetry (battery_mv, temperature_c) in their advert
appdata after the null-terminated name. This commit adds decoding and
storage for both the Go ingestor and Node.js backend.

Changes:
- decoder.go/decoder.js: Parse telemetry bytes from advert appdata
  (battery_mv as uint16 LE millivolts, temperature_c as int16 LE /100)
- db.go/db.js: Add battery_mv INTEGER and temperature_c REAL columns
  to nodes and inactive_nodes tables, with migration for existing DBs
- main.go/server.js: Update node telemetry on advert processing
- server db.go: Include battery_mv/temperature_c in node API responses
- Tests: Decoder telemetry tests (positive, negative temp, no telemetry),
  DB migration test, node telemetry update test, server API shape tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 12:07:42 -07:00
Kpa-clawbot aba4270ceb fix: undefined err in packets_v view creation (use vErr)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 12:00:04 -07:00
Kpa-clawbot 57b0188158 fix: create packets_v VIEW in Go ingestor schema (#217)
Fresh Go installs failed with 'no such table: packets_v' because the
ingestor created tables but never the VIEW that the Go server queries.

Add DROP VIEW IF EXISTS + CREATE VIEW packets_v to applySchema(), using
the v3 definition (observer_idx → observers.rowid JOIN). The view is
rebuilt on every startup to stay current with any definition changes.

Add tests: verify view exists after OpenStore, and verify it returns
correct observer_id/observer_name via the LEFT JOIN.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 11:28:38 -07:00
Kpa-clawbot f374a4a775 fix: enforce consistent types between Go ingestor writes and server reads
Schema:
- observers.noise_floor: INTEGER → REAL (dBm has decimals)
- battery_mv, uptime_secs remain INTEGER (always whole numbers)

Ingestor write side (cmd/ingestor/db.go):
- UpsertObserver now accepts ObserverMeta with battery_mv (int),
  uptime_secs (int64), noise_floor (float64)
- COALESCE preserves existing values when meta is nil
- Added migration: cast integer noise_floor values to REAL

Ingestor MQTT handler (cmd/ingestor/main.go — already updated):
- extractObserverMeta extracts hardware fields from status messages
- battery_mv/uptime_secs cast via math.Round to int on write

Server read side (cmd/server/db.go):
- Observer.BatteryMv: *float64 → *int (matches INTEGER storage)
- Observer.UptimeSecs: *float64 → *int64 (matches INTEGER storage)
- Observer.NoiseFloor: *float64 (unchanged, matches REAL storage)
- GetObservers/GetObserverByID: use sql.NullInt64 intermediaries
  for battery_mv/uptime_secs, sql.NullFloat64 for noise_floor

Proto (proto/observer.proto — already correct):
- battery_mv: int32, uptime_secs: int64, noise_floor: double

Tests:
- TestUpsertObserverWithMeta: verifies correct SQLite types via typeof()
- TestUpsertObserverMetaPreservesExisting: nil-meta preserves values
- TestExtractObserverMeta: float-to-int rounding, empty message
- TestSchemaNoiseFloorIsReal: PRAGMA table_info validation
- TestObserverTypeConsistency: server reads typed values correctly
- TestObserverTypesInGetObservers: list endpoint type consistency

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 11:22:14 -07:00
Kpa-clawbot 6d31cb2ad6 feat: add pprof profiling controlled by ENABLE_PPROF env var
Add net/http/pprof support to both Go server (default port 6060) and
ingestor (default port 6061). Profiling is off by default — only
starts the pprof HTTP listener when ENABLE_PPROF=true.

PPROF_PORT env var overrides the default port for each binary.

Enable on staging-go in docker-compose with exposed ports 6060/6061.
Not enabled on prod.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 11:18:33 -07:00
Kpa-clawbot 1619f4857e fix: noise_floor/battery_mv/uptime_secs scanned as float64 to handle REAL values
SQLite stores these as REAL on some instances. Go *int scan silently
fails, dropping the entire observer row (404 on detail, missing from list).
Reported for YC-Base-Repeater and YC-Work-Repeater.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 11:04:49 -07:00
Kpa-clawbot 58d19ec303 Merge pull request #214 from Kpa-clawbot/fix/sqlite-write-concurrency
Reviewed by Kobayashi — LGTM. Fixes SQLite BUSY contention with busy_timeout + single connection serialization.
2026-03-28 10:13:44 -07:00
93 changed files with 2529 additions and 1999 deletions
+33 -6
View File
@@ -1,17 +1,44 @@
# MeshCore Analyzer — Environment Configuration
# Copy to .env and customize. All values have sensible defaults in docker-compose.yml.
# Copy to .env and customize. All values have sensible defaults.
#
# This file is read by BOTH docker compose AND manage.sh — one source of truth.
# Each environment keeps config + data together in one directory:
# ~/meshcore-data/config.json, meshcore.db, Caddyfile, theme.json
# ~/meshcore-staging-data/config.json, meshcore.db, Caddyfile
# --- Production ---
PROD_HTTP_PORT=80
PROD_HTTPS_PORT=443
PROD_MQTT_PORT=1883
# Data directory (database, theme, etc.)
# Default: ~/meshcore-data
# Used by: docker compose, manage.sh
PROD_DATA_DIR=~/meshcore-data
# HTTP port for web UI
# Default: 80
# Used by: docker compose
PROD_HTTP_PORT=80
# HTTPS port for web UI (TLS via Caddy)
# Default: 443
# Used by: docker compose
PROD_HTTPS_PORT=443
# MQTT port for observer connections
# Default: 1883
# Used by: docker compose
PROD_MQTT_PORT=1883
# --- Staging (HTTP only, no HTTPS) ---
STAGING_HTTP_PORT=81
STAGING_MQTT_PORT=1884
# Data directory
# Default: ~/meshcore-staging-data
# Used by: docker compose
STAGING_DATA_DIR=~/meshcore-staging-data
# HTTP port
# Default: 81
# Used by: docker compose
STAGING_HTTP_PORT=81
# MQTT port
# Default: 1884
# Used by: docker compose
STAGING_MQTT_PORT=1884
+61
View File
@@ -0,0 +1,61 @@
---
name: "MeshCore PR Reviewer"
description: "A specialized agent for reviewing pull requests in the meshcore-analyzer repository. It focuses on SOLID, DRY, testing, Go best practices, frontend testability, observability, and performance to prevent regressions and maintain high code quality."
model: "gpt-5.3-codex"
tools: ["githubread", "add_issue_comment"]
---
# MeshCore PR Reviewer Agent
You are an expert software engineer specializing in Go and JavaScript-heavy network analysis tools. Your primary role is to act as a meticulous pull request reviewer for the `Kpa-clawbot/meshcore-analyzer` repository. You are deeply familiar with its architecture, as outlined in `AGENTS.md`, and you enforce its rules rigorously.
Your reviews are thorough, constructive, and aimed at maintaining the highest standards of code quality, performance, and stability on both the backend and frontend.
## Core Principles
1. **Context is King**: Before any review, consult the `AGENTS.md` file in the `Kpa-clawbot/meshcore-analyzer` repository to ground your feedback in the project's established architecture and rules.
2. **Enforce the Rules**: Your primary directive is to ensure every rule in `AGENTS.md` is followed. Call out any deviation.
3. **Go & JS Best Practices**: Apply your deep knowledge of Go and modern JavaScript idioms. Pay close attention to concurrency, error handling, performance, and state management, especially as they relate to a real-time data processing application.
4. **Constructive and Educational**: Your feedback should not only identify issues but also explain *why* they are issues and suggest idiomatic solutions. Your goal is to mentor and elevate the codebase and its contributors.
5. **Be a Guardian**: Protect the project from regressions, performance degradation, and architectural drift.
## Review Focus Areas
You will pay special attention to the following areas during your review:
### 1. Architectural Adherence & Design Principles
- **SOLID & DRY**: Does the change adhere to SOLID principles? Is there duplicated logic that could be refactored? Does it respect the existing separation of concerns?
- **Project Architecture**: Does the PR respect the single Node.js server + static frontend architecture? Are changes in the right place?
### 2. Testing and Validation
- **No commit without tests**: Is the backend logic change covered by unit tests? Is `test-packet-filter.js` or `test-aging.js` updated if necessary?
- **Browser Validation**: Has the contributor confirmed the change works in a browser? Is there a screenshot for visual changes?
- **Cache Busters**: If any `public/` assets (`.js`, `.css`) were modified, has the cache buster in `public/index.html` been bumped in the *same commit*? This is critical.
### 3. Go-Specific Concerns
- **Concurrency**: Are goroutines used safely? Are there potential race conditions? Is synchronization used correctly?
- **Error Handling**: Is error handling explicit and clear? Are errors wrapped with context where appropriate?
- **Performance**: Are there inefficient loops or memory allocation patterns? Scrutinize any new data processing logic.
- **Go Idioms**: Does the code follow standard Go idioms and formatting (`gofmt`)?
### 4. Frontend and UI Testability
- **Acknowledge Complexity**: Does the PR introduce complex client-side logic? Recognize that browser-based functionality is difficult to unit test.
- **Promote Testability**: Challenge the contributor to refactor UI code to improve testability. Are data manipulation, state management, and rendering logic separated? Logic should be in pure, testable functions, not tangled in DOM manipulation code.
- **UI Logic Purity**: Scrutinize client-side JavaScript. Are there large, monolithic functions? Could business logic be extracted from event handlers into standalone, easily testable functions?
- **State Management**: How is client-side state managed? Are there risks of race conditions or inconsistent states from asynchronous operations (e.g., API calls)?
### 5. Observability and Maintainability
- **Logging**: Are new logic paths and error cases instrumented with sufficient logging to be debuggable in production?
- **Configuration**: Are new configurable values (thresholds, timeouts) identified for future inclusion in the customizer, as per project rules?
- **Clarity**: Is the code clear, readable, and well-documented where complexity is unavoidable?
### 6. API and Data Integrity
- **API Response Shape**: If the PR adds a UI feature that consumes an API, is there evidence the author verified the actual API response?
- **Firmware as Source of Truth**: For any changes related to the MeshCore protocol, has the author referenced the `firmware/` source? Challenge any "magic numbers" or assumptions about packet structure.
## Review Process
1. **State Your Role**: Begin your review by announcing your function: "As the MeshCore PR Reviewer, I have analyzed this pull request based on the project's architectural guidelines and best practices."
2. **Provide a Summary**: Give a high-level summary of your findings (e.g., "This PR looks solid but needs additions to testing," or "I have several concerns regarding performance and frontend testability.").
3. **Detailed Feedback**: Use a bulleted list to present specific, actionable feedback, referencing file paths and line numbers. For each point, cite the relevant principle or project rule (e.g., "Missing Test Coverage (Rule #1)", "UI Logic Purity (Focus Area #4)").
4. **End with a Clear Approval Status**: Conclude with a clear statement of "Approved" (with minor optional suggestions), "Changes Requested," or "Rejected" (for significant violations).
@@ -0,0 +1,61 @@
---
name: "MeshCore PR Reviewer"
description: "A specialized agent for reviewing pull requests in the meshcore-analyzer repository. It focuses on SOLID, DRY, testing, Go best practices, frontend testability, observability, and performance to prevent regressions and maintain high code quality."
model: "gpt-5.3-codex"
tools: ["githubread", "add_issue_comment"]
---
# MeshCore PR Reviewer Agent
You are an expert software engineer specializing in Go and JavaScript-heavy network analysis tools. Your primary role is to act as a meticulous pull request reviewer for the `Kpa-clawbot/meshcore-analyzer` repository. You are deeply familiar with its architecture, as outlined in `AGENTS.md`, and you enforce its rules rigorously.
Your reviews are thorough, constructive, and aimed at maintaining the highest standards of code quality, performance, and stability on both the backend and frontend.
## Core Principles
1. **Context is King**: Before any review, consult the `AGENTS.md` file in the `Kpa-clawbot/meshcore-analyzer` repository to ground your feedback in the project's established architecture and rules.
2. **Enforce the Rules**: Your primary directive is to ensure every rule in `AGENTS.md` is followed. Call out any deviation.
3. **Go & JS Best Practices**: Apply your deep knowledge of Go and modern JavaScript idioms. Pay close attention to concurrency, error handling, performance, and state management, especially as they relate to a real-time data processing application.
4. **Constructive and Educational**: Your feedback should not only identify issues but also explain *why* they are issues and suggest idiomatic solutions. Your goal is to mentor and elevate the codebase and its contributors.
5. **Be a Guardian**: Protect the project from regressions, performance degradation, and architectural drift.
## Review Focus Areas
You will pay special attention to the following areas during your review:
### 1. Architectural Adherence & Design Principles
- **SOLID & DRY**: Does the change adhere to SOLID principles? Is there duplicated logic that could be refactored? Does it respect the existing separation of concerns?
- **Project Architecture**: Does the PR respect the single Node.js server + static frontend architecture? Are changes in the right place?
### 2. Testing and Validation
- **No commit without tests**: Is the backend logic change covered by unit tests? Is `test-packet-filter.js` or `test-aging.js` updated if necessary?
- **Browser Validation**: Has the contributor confirmed the change works in a browser? Is there a screenshot for visual changes?
- **Cache Busters**: If any `public/` assets (`.js`, `.css`) were modified, has the cache buster in `public/index.html` been bumped in the *same commit*? This is critical.
### 3. Go-Specific Concerns
- **Concurrency**: Are goroutines used safely? Are there potential race conditions? Is synchronization used correctly?
- **Error Handling**: Is error handling explicit and clear? Are errors wrapped with context where appropriate?
- **Performance**: Are there inefficient loops or memory allocation patterns? Scrutinize any new data processing logic.
- **Go Idioms**: Does the code follow standard Go idioms and formatting (`gofmt`)?
### 4. Frontend and UI Testability
- **Acknowledge Complexity**: Does the PR introduce complex client-side logic? Recognize that browser-based functionality is difficult to unit test.
- **Promote Testability**: Challenge the contributor to refactor UI code to improve testability. Are data manipulation, state management, and rendering logic separated? Logic should be in pure, testable functions, not tangled in DOM manipulation code.
- **UI Logic Purity**: Scrutinize client-side JavaScript. Are there large, monolithic functions? Could business logic be extracted from event handlers into standalone, easily testable functions?
- **State Management**: How is client-side state managed? Are there risks of race conditions or inconsistent states from asynchronous operations (e.g., API calls)?
### 5. Observability and Maintainability
- **Logging**: Are new logic paths and error cases instrumented with sufficient logging to be debuggable in production?
- **Configuration**: Are new configurable values (thresholds, timeouts) identified for future inclusion in the customizer, as per project rules?
- **Clarity**: Is the code clear, readable, and well-documented where complexity is unavoidable?
### 6. API and Data Integrity
- **API Response Shape**: If the PR adds a UI feature that consumes an API, is there evidence the author verified the actual API response?
- **Firmware as Source of Truth**: For any changes related to the MeshCore protocol, has the author referenced the `firmware/` source? Challenge any "magic numbers" or assumptions about packet structure.
## Review Process
1. **State Your Role**: Begin your review by announcing your function: "As the MeshCore PR Reviewer, I have analyzed this pull request based on the project's architectural guidelines and best practices."
2. **Provide a Summary**: Give a high-level summary of your findings (e.g., "This PR looks solid but needs additions to testing," or "I have several concerns regarding performance and frontend testability.").
3. **Detailed Feedback**: Use a bulleted list to present specific, actionable feedback, referencing file paths and line numbers. For each point, cite the relevant principle or project rule (e.g., "Missing Test Coverage (Rule #1)", "UI Logic Purity (Focus Area #4)").
4. **End with a Clear Approval Status**: Conclude with a clear statement of "Approved" (with minor optional suggestions), "Changes Requested," or "Rejected" (for significant violations).
+75 -30
View File
@@ -8,9 +8,16 @@ on:
- 'LICENSE'
- '.gitignore'
- 'docs/**'
pull_request:
branches: [master]
paths-ignore:
- '**.md'
- 'LICENSE'
- '.gitignore'
- 'docs/**'
concurrency:
group: deploy
group: deploy-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
@@ -34,10 +41,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up Go 1.22
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version: '1.22'
cache-dependency-path: |
@@ -115,9 +122,16 @@ jobs:
echo "| Server | ${SERVER_COV}% |" >> $GITHUB_STEP_SUMMARY
echo "| Ingestor | ${INGESTOR_COV}% |" >> $GITHUB_STEP_SUMMARY
- name: Cancel workflow on failure
if: failure()
run: |
curl -s -X POST \
-H "Authorization: Bearer ${{ github.token }}" \
"https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/cancel"
- name: Upload Go coverage badges
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v5
with:
name: go-badges
path: .badges/go-*.json
@@ -129,15 +143,15 @@ jobs:
# ───────────────────────────────────────────────────────────────
node-test:
name: "🧪 Node.js Tests"
runs-on: self-hosted
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
fetch-depth: 2
- name: Set up Node.js 22
uses: actions/setup-node@v4
uses: actions/setup-node@v5
with:
node-version: '22'
@@ -183,7 +197,11 @@ jobs:
- name: Install Playwright browser
if: steps.changes.outputs.frontend == 'true'
run: npx playwright install chromium --with-deps 2>/dev/null || true
run: |
# Install chromium (skips download if already cached on self-hosted runner)
npx playwright install chromium 2>/dev/null || true
# Install system deps only if missing (apt-get is slow)
npx playwright install-deps chromium 2>/dev/null || true
- name: Instrument frontend JS for coverage
if: steps.changes.outputs.frontend == 'true'
@@ -213,19 +231,32 @@ jobs:
sleep 1
done
- name: Run Playwright E2E tests
- name: Run Playwright E2E + coverage collection concurrently
if: steps.changes.outputs.frontend == 'true'
run: BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt
run: |
# Run E2E tests and coverage collection in parallel — both use the same server
BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt &
E2E_PID=$!
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt &
COV_PID=$!
- name: Collect frontend coverage report
# Wait for both — E2E must pass, coverage is best-effort
E2E_EXIT=0
wait $E2E_PID || E2E_EXIT=$?
wait $COV_PID || true
# Fail if E2E failed
[ $E2E_EXIT -ne 0 ] && exit $E2E_EXIT
true
- name: Generate frontend coverage badges
if: always() && steps.changes.outputs.frontend == 'true'
run: |
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1)
mkdir -p .badges
if [ -f .nyc_output/frontend-coverage.json ]; then
# Merge E2E + coverage collector data if both exist
if [ -f .nyc_output/frontend-coverage.json ] || [ -f .nyc_output/e2e-coverage.json ]; then
npx nyc report --reporter=text-summary --reporter=text 2>&1 | tee fe-report.txt
FE_COVERAGE=$(grep 'Statements' fe-report.txt | head -1 | grep -oP '[\d.]+(?=%)' || echo "0")
FE_COVERAGE=${FE_COVERAGE:-0}
@@ -252,13 +283,24 @@ jobs:
fuser -k 13581/tcp 2>/dev/null || true
PORT=13581 node server.js &
SERVER_PID=$!
sleep 5
# Wait for server to be ready (up to 15s)
for i in $(seq 1 15); do
curl -sf http://localhost:13581/api/stats > /dev/null 2>&1 && break
sleep 1
done
BASE_URL=http://localhost:13581 node test-e2e-playwright.js || true
kill $SERVER_PID 2>/dev/null || true
- name: Cancel workflow on failure
if: failure()
run: |
curl -s -X POST \
-H "Authorization: Bearer ${{ github.token }}" \
"https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/cancel"
- name: Upload Node.js test badges
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v5
with:
name: node-badges
path: .badges/
@@ -270,14 +312,15 @@ jobs:
# ───────────────────────────────────────────────────────────────
build:
name: "🏗️ Build Docker Image"
needs: [go-test]
runs-on: self-hosted
if: github.event_name == 'push'
needs: [go-test, node-test]
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up Node.js 22
uses: actions/setup-node@v4
uses: actions/setup-node@v5
with:
node-version: '22'
@@ -294,16 +337,17 @@ jobs:
# ───────────────────────────────────────────────────────────────
deploy:
name: "🚀 Deploy Staging"
if: github.event_name == 'push'
needs: [build]
runs-on: self-hosted
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Start staging on port 82
run: |
# Force remove stale containers
docker rm -f meshcore-staging-go 2>/dev/null || true
docker rm -f corescope-staging-go 2>/dev/null || true
# Clean up stale ports
fuser -k 82/tcp 2>/dev/null || true
docker compose --profile staging-go up -d staging-go
@@ -311,14 +355,14 @@ jobs:
- name: Healthcheck staging container
run: |
for i in $(seq 1 120); do
HEALTH=$(docker inspect meshcore-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
HEALTH=$(docker inspect corescope-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
if [ "$HEALTH" = "healthy" ]; then
echo "Staging healthy after ${i}s"
break
fi
if [ "$i" -eq 120 ]; then
echo "Staging failed health check after 120s"
docker logs meshcore-staging-go --tail 50
docker logs corescope-staging-go --tail 50
exit 1
fi
sleep 1
@@ -338,22 +382,23 @@ jobs:
# ───────────────────────────────────────────────────────────────
publish:
name: "📝 Publish Badges & Summary"
if: github.event_name == 'push'
needs: [deploy]
runs-on: self-hosted
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Download Go coverage badges
continue-on-error: true
uses: actions/download-artifact@v4
uses: actions/download-artifact@v5
with:
name: go-badges
path: .badges/
- name: Download Node.js test badges
continue-on-error: true
uses: actions/download-artifact@v4
uses: actions/download-artifact@v5
with:
name: node-badges
path: .badges/
@@ -378,6 +423,6 @@ jobs:
echo "To promote to production:" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
echo "ssh deploy@\$VM_HOST" >> $GITHUB_STEP_SUMMARY
echo "cd /opt/meshcore-deploy" >> $GITHUB_STEP_SUMMARY
echo "cd /opt/corescope-deploy" >> $GITHUB_STEP_SUMMARY
echo "./manage.sh promote" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
+2 -2
View File
@@ -1,10 +1,10 @@
# Bishop — Tester
Unit tests, Playwright E2E, coverage gates, and quality assurance for MeshCore Analyzer.
Unit tests, Playwright E2E, coverage gates, and quality assurance for CoreScope.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Node.js native test runner, Playwright, c8 + nyc (coverage), supertest
**User:** User
+1 -1
View File
@@ -2,7 +2,7 @@
## Project Context
MeshCore Analyzer has 14 test files, 4,290 lines of test code. Backend coverage 85%+, frontend 42%+. Tests use Node.js native runner, Playwright for E2E, c8/nyc for coverage, supertest for API routes. vm.createContext pattern used for testing frontend helpers in Node.js.
CoreScope has 14 test files, 4,290 lines of test code. Backend coverage 85%+, frontend 42%+. Tests use Node.js native runner, Playwright for E2E, c8/nyc for coverage, supertest for API routes. vm.createContext pattern used for testing frontend helpers in Node.js.
User: User
+2 -2
View File
@@ -1,10 +1,10 @@
# Hicks — Backend Dev
Server, decoder, packet-store, SQLite, API, MQTT, WebSocket, and performance for MeshCore Analyzer.
Server, decoder, packet-store, SQLite, API, MQTT, WebSocket, and performance for CoreScope.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Node.js 18+, Express 5, SQLite (better-sqlite3), MQTT (mqtt), WebSocket (ws)
**User:** User
+1 -1
View File
@@ -2,7 +2,7 @@
## Project Context
MeshCore Analyzer is a real-time LoRa mesh packet analyzer. Node.js + Express + SQLite backend, vanilla JS SPA frontend. Custom decoder.js fixes path_length bug from upstream library. In-memory packet store provides O(1) lookups for 30K+ packets. TTL response cache achieves 7,000× speedup on bulk health endpoint.
CoreScope is a real-time LoRa mesh packet analyzer. Node.js + Express + SQLite backend, vanilla JS SPA frontend. Custom decoder.js fixes path_length bug from upstream library. In-memory packet store provides O(1) lookups for 30K+ packets. TTL response cache achieves 7,000× speedup on bulk health endpoint.
User: User
+2 -2
View File
@@ -1,10 +1,10 @@
# Kobayashi — Lead
Architecture, code review, and decision-making for MeshCore Analyzer.
Architecture, code review, and decision-making for CoreScope.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Node.js 18+, Express 5, SQLite, vanilla JS frontend, Leaflet, WebSocket, MQTT
**User:** User
+1 -1
View File
@@ -2,7 +2,7 @@
## Project Context
MeshCore Analyzer is a real-time LoRa mesh packet analyzer. Node.js + Express + SQLite backend, vanilla JS SPA frontend with Leaflet maps, WebSocket live feed, MQTT ingestion. Production at v2.6.0, ~18K lines, 85%+ backend test coverage.
CoreScope is a real-time LoRa mesh packet analyzer. Node.js + Express + SQLite backend, vanilla JS SPA frontend with Leaflet maps, WebSocket live feed, MQTT ingestion. Production at v2.6.0, ~18K lines, 85%+ backend test coverage.
User: User
+2 -2
View File
@@ -1,10 +1,10 @@
# Newt — Frontend Dev
Vanilla JS UI, Leaflet maps, live visualization, theming, and all public/ modules for MeshCore Analyzer.
Vanilla JS UI, Leaflet maps, live visualization, theming, and all public/ modules for CoreScope.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Vanilla HTML/CSS/JavaScript (ES5/6), Leaflet maps, WebSocket, Canvas animations
**User:** User
+1 -1
View File
@@ -2,7 +2,7 @@
## Project Context
MeshCore Analyzer is a real-time LoRa mesh packet analyzer with a vanilla JS SPA frontend. 22 frontend modules, Leaflet maps, WebSocket live feed, VCR playback, Canvas animations, theme customizer with CSS variables. No build step, no framework. ES5/6 for broad browser support.
CoreScope is a real-time LoRa mesh packet analyzer with a vanilla JS SPA frontend. 22 frontend modules, Leaflet maps, WebSocket live feed, VCR playback, Canvas animations, theme customizer with CSS variables. No build step, no framework. ES5/6 for broad browser support.
User: User
+1 -1
View File
@@ -4,7 +4,7 @@ Tracks the work queue and keeps the team moving. Always on the roster.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**User:** User
## Responsibilities
+2 -2
View File
@@ -1,10 +1,10 @@
# Ripley — Support Engineer
Deep knowledge of every frontend behavior, API response, and user-facing feature in MeshCore Analyzer. Fields community questions, triages bug reports, and explains "why does X look like Y."
Deep knowledge of every frontend behavior, API response, and user-facing feature in CoreScope. Fields community questions, triages bug reports, and explains "why does X look like Y."
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Vanilla JS frontend (public/*.js), Node.js backend, SQLite, WebSocket, MQTT
**User:** Kpa-clawbot
+1 -1
View File
@@ -1,7 +1,7 @@
# Ripley — Support Engineer History
## Core Context
- Project: MeshCore Analyzer — real-time LoRa mesh packet analyzer
- Project: CoreScope — real-time LoRa mesh packet analyzer
- User: Kpa-clawbot
- Joined the team 2026-03-27 to handle community support and triage
+2 -2
View File
@@ -1,10 +1,10 @@
# Scribe — Session Logger
Silent agent that maintains decisions, logs, and cross-agent context for MeshCore Analyzer.
Silent agent that maintains decisions, logs, and cross-agent context for CoreScope.
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**User:** User
## Responsibilities
+1 -1
View File
@@ -5,7 +5,7 @@
"universe": "aliens",
"created_at": "2026-03-26T04:22:08Z",
"agents": ["Kobayashi", "Hicks", "Newt", "Bishop"],
"reason": "Initial team casting for MeshCore Analyzer project"
"reason": "Initial team casting for CoreScope project"
}
]
}
+2 -2
View File
@@ -1,8 +1,8 @@
# Squad — MeshCore Analyzer
# Squad — CoreScope
## Project Context
**Project:** MeshCore Analyzer — Real-time LoRa mesh packet analyzer
**Project:** CoreScope — Real-time LoRa mesh packet analyzer
**Stack:** Node.js 18+, Express 5, SQLite (better-sqlite3), vanilla JS frontend, Leaflet maps, WebSocket (ws), MQTT (mqtt)
**User:** User
**Description:** Self-hosted alternative to analyzer.letsmesh.net. Ingests MeshCore mesh network packets via MQTT, decodes with custom parser (decoder.js), stores in SQLite with in-memory indexing (packet-store.js), and serves a rich SPA with live visualization, packet analysis, node analytics, channel chat, observer health, and theme customizer. ~18K lines, 14 test files, 85%+ backend coverage. Production at v2.6.0.
+1 -1
View File
@@ -1,4 +1,4 @@
# AGENTS.md — MeshCore Analyzer
# AGENTS.md — CoreScope
Guide for AI agents working on this codebase. Read this before writing any code.
+6 -6
View File
@@ -11,14 +11,14 @@ WORKDIR /build/server
COPY cmd/server/go.mod cmd/server/go.sum ./
RUN go mod download
COPY cmd/server/ ./
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /meshcore-server .
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /corescope-server .
# Build ingestor
WORKDIR /build/ingestor
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
RUN go mod download
COPY cmd/ingestor/ ./
RUN go build -o /meshcore-ingestor .
RUN go build -o /corescope-ingestor .
# Runtime image
FROM alpine:3.20
@@ -28,15 +28,15 @@ RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
WORKDIR /app
# Go binaries
COPY --from=builder /meshcore-server /meshcore-ingestor /app/
COPY --from=builder /corescope-server /corescope-ingestor /app/
# Frontend assets + config
COPY public/ ./public/
COPY config.example.json channel-rainbow.json ./
# Bake git commit SHA (CI writes .git-commit before build; fallback for non-ldflags usage)
COPY .git-commi[t] ./
RUN if [ ! -f .git-commit ]; then echo "unknown" > .git-commit; fi
# Bake git commit SHA — manage.sh and CI write .git-commit before build
# Default to "unknown" if not provided
RUN echo "unknown" > .git-commit
# Supervisor + Mosquitto + Caddy config
COPY docker/supervisord-go.conf /etc/supervisor/conf.d/supervisord.conf
+3 -3
View File
@@ -11,14 +11,14 @@ WORKDIR /build/server
COPY cmd/server/go.mod cmd/server/go.sum ./
RUN go mod download
COPY cmd/server/ ./
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /meshcore-server .
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /corescope-server .
# Build ingestor
WORKDIR /build/ingestor
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
RUN go mod download
COPY cmd/ingestor/ ./
RUN go build -o /meshcore-ingestor .
RUN go build -o /corescope-ingestor .
# Runtime image
FROM alpine:3.20
@@ -28,7 +28,7 @@ RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
WORKDIR /app
# Go binaries
COPY --from=builder /meshcore-server /meshcore-ingestor /app/
COPY --from=builder /corescope-server /corescope-ingestor /app/
# Frontend assets + config
COPY public/ ./public/
+9 -9
View File
@@ -1,10 +1,10 @@
# MeshCore Analyzer
# CoreScope
[![Go Server Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/.badges/go-server-coverage.json)](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
[![Go Ingestor Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/.badges/go-ingestor-coverage.json)](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
[![Frontend Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/.badges/frontend-tests.json)](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
[![Frontend Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/.badges/frontend-coverage.json)](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
[![Deploy](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml/badge.svg)](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
[![Go Server Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/.badges/go-server-coverage.json)](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
[![Go Ingestor Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/.badges/go-ingestor-coverage.json)](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
[![Frontend Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/.badges/frontend-tests.json)](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
[![Frontend Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/.badges/frontend-coverage.json)](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
[![Deploy](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml/badge.svg)](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
> High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption.
@@ -79,8 +79,8 @@ Full experience on your phone — proper touch controls, iOS safe area support,
No Go installation needed — everything builds inside the container.
```bash
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd meshcore-analyzer
git clone https://github.com/Kpa-clawbot/corescope.git
cd corescope
./manage.sh setup
```
@@ -171,7 +171,7 @@ Or POST raw hex packets to `POST /api/packets` for manual injection.
## Project Structure
```
meshcore-analyzer/
corescope/
├── cmd/
│ ├── server/ # Go HTTP server + WebSocket + REST API
│ │ ├── main.go # Entry point
+4 -4
View File
@@ -73,8 +73,8 @@ Advert counts now reflect unique transmissions, not total observations. A packet
The Go backend is two binaries managed by supervisord inside Docker:
- **`meshcore-ingestor`** — connects to MQTT brokers, decodes packets, writes to SQLite, maintains the in-memory store
- **`meshcore-server`** — HTTP API, WebSocket broadcast, static file serving, analytics computation
- **`corescope-ingestor`** — connects to MQTT brokers, decodes packets, writes to SQLite, maintains the in-memory store
- **`corescope-server`** — HTTP API, WebSocket broadcast, static file serving, analytics computation
Both share the same SQLite database (WAL mode). The frontend is unchanged — same vanilla JS, same `public/` directory, served by the Go HTTP server through Caddy.
@@ -120,7 +120,7 @@ curl -s http://localhost/api/health | grep engine
The Node.js Dockerfile is preserved as `Dockerfile.node`:
```bash
docker build -f Dockerfile.node -t meshcore-analyzer:latest .
docker build -f Dockerfile.node -t corescope:latest .
docker compose up -d --force-recreate prod
```
@@ -152,7 +152,7 @@ This release wouldn't exist without the community:
- **LitBomb** — issue reports from production deployments
- **mibzzer15** — issue reports and edge case discovery
And to everyone running MeshCore Analyzer in the wild — your packet data, bug reports, and feature requests are what drive this project forward. The Go rewrite happened because the community outgrew what Node.js could handle. 56K packets, dozens of observers, sub-second queries. This is your tool. We just rewrote the engine.
And to everyone running CoreScope in the wild — your packet data, bug reports, and feature requests are what drive this project forward. The Go rewrite happened because the community outgrew what Node.js could handle. 56K packets, dozens of observers, sub-second queries. This is your tool. We just rewrote the engine.
---
+144
View File
@@ -0,0 +1,144 @@
# v3.1.0 — Now It's CoreScope
MeshCore Analyzer has a new name: **CoreScope**. Same mesh analysis you rely on, sharper identity, and a boatload of fixes and performance wins since v3.0.0.
48 commits, 30+ issues closed. Here's what changed.
---
## 🏷️ Renamed to CoreScope
The project is now **CoreScope** — frontend, backend, Docker images, manage.sh, docs, CI — everything has been updated. The URL, the API, the database, and your config all stay the same. Just a better name for the tool the community built.
---
## ⚡ Performance
| What | Before | After |
|------|--------|-------|
| Subpath analytics | 900 ms | **5 ms** (precomputed at ingest) |
| Distance analytics | 1.2 s | **15 ms** (precomputed at ingest) |
| Packet ingest (prepend) | O(n) slice copy | **O(1) append** |
| Go runtime stats | GC stop-the-world on every call | **cached ReadMemStats** |
| All analytics endpoints | computed per-request | **TTL-cached** |
The in-memory store now precomputes subpaths and distance data as packets arrive, eliminating expensive full-table scans on the analytics endpoints. The O(n) slice prepend on every ingest — the single hottest line in the server — is gone. `ReadMemStats` calls are cached to prevent GC pause spikes under load.
---
## 🆕 New Features
### Telemetry Decode
Sensor nodes now report **battery voltage** and **temperature** parsed from advert payloads. Telemetry is gated on the sensor flag — only real sensors emit data, and 0°C is no longer falsely reported. Safe migration with `PRAGMA` column checks.
### Channel Decryption for Custom Channels
The `hashChannels` config now works in the Go ingestor. Key derivation has been ported from Node.js with full AES-128-ECB support and garbage text detection — wrong keys silently fail instead of producing garbled output.
### Node Pruning
Stale nodes are automatically moved to an `inactive_nodes` table after the configurable retention window. Pruning runs hourly. Your active node list stays clean. (#202)
### Duplicate Node Name Badges
Nodes with the same display name but different public keys are flagged with a badge so you can spot collisions instantly.
### Sortable Channels Table
Channel columns are now sortable with click-to-sort headers. Sort preferences persist in `localStorage` across sessions. (#167)
### Go Runtime Metrics
The performance page exposes goroutine count, heap allocation, GC pause percentiles, and memory breakdown when connected to a Go backend.
---
## 🐛 Bug Fixes
- **Channel decryption regression** (#176) — full AES-128-ECB in Go, garbage text detection, hashChannels key derivation ported correctly (#218)
- **Packets page not live-updating** (#172) — WebSocket broadcast now includes the nested packet object and timestamp fields the frontend expects; multiple fixes across broadcast and render paths
- **Node detail page crashes** (#190) — `Number()` casts and `Array.isArray` guards prevent rendering errors on unexpected data shapes
- **Observation count staleness** (#174) — trace page and packet detail now show correct observation counts
- **Phantom node cleanup** (#133) — `autoLearnHopNodes` no longer creates fake nodes from 1-byte repeater IDs
- **Advert count inflation** (#200) — counts unique transmissions, not total observations (8 observers × 1 advert = 1, not 8)
- **SQLite BUSY contention** (#214) — `MaxOpenConns(1)` + `MaxIdleConns(1)` serializes writes; load-tested under concurrent ingest
- **Decoder bounds check** (#183) — corrupt/malformed packets no longer crash the decoder with buffer overruns
- **noise_floor / battery_mv type mismatches** — consistent `float64` scanning handles SQLite REAL values correctly
- **packetsLastHour always zero** (#182) — early `break` in observer loop prevented counting
- **Channels stale messages** (#171) — latest message sorted by observation timestamp, not first-seen
- **pprof port conflict** — non-fatal bind with separate ports prevents Go server crash on startup
---
## ♿ Accessibility & 📱 Mobile
### WCAG AA Compliance (10 fixes)
- Search results keyboard-accessible with `tabindex`, `role`, and arrow-key navigation (#208)
- 40+ table headers given `scope` attributes (#211)
- 9 Chart.js canvases given accessible names (#210)
- Form inputs in customizer/filters paired with labels (#212)
### Mobile Responsive
- **Live page**: bottom-sheet panel instead of full-screen overlay (#203)
- **Perf page**: responsive layout with stacked cards (#204)
- **Nodes table**: column hiding at narrow viewports (#205)
- **Analytics/Compare**: horizontal scroll wrappers (#206)
- **VCR bar**: 44px minimum touch targets (#207)
---
## 🏗️ Infrastructure
### manage.sh Refactored (#230)
`manage.sh` is now a thin wrapper around `docker compose` — no custom container management, no divergent logic. It reads `.env` for data paths, matching how `docker-compose.yml` works. One source of truth.
### .env Support
Data directory, ports, and image tags are configured via `.env`. Both `docker compose` and `manage.sh` read the same file.
### Branch Protection & CI on PRs
- Branch protection enabled on `master` — CI must pass, PRs required
- CI now triggers on `pull_request`, not just `push` — catch failures before merge (#199)
### Protobuf API Contract
10 `.proto` files, 33 golden fixtures, CI validation on every push. API shape drift is caught automatically.
### pprof Profiling
Controlled by `ENABLE_PPROF` env var. When enabled, exposes Go profiling endpoints on separate ports — zero overhead when off.
### Test Coverage
- Go backend: **92%+** coverage
- **49 Playwright E2E tests**
- Both tracks gate deploy in CI
---
## 📦 Upgrading
```bash
git pull
./manage.sh stop
./manage.sh setup
```
That's it. Your existing `config.json` and database work as-is. The rename is cosmetic — no schema changes, no API changes, no config changes.
### Verify
```bash
curl -s http://localhost/api/health | grep engine
# "engine": "go"
```
---
## ⚠️ Breaking Changes
**None.** All API endpoints, WebSocket messages, and config options are backwards-compatible. The rename affects branding only — Docker image names, page titles, and documentation.
---
## 🙏 Thank You
- **efiten** — PR #222 performance fix (O(n) slice prepend elimination)
- **jade-on-mesh**, **lincomatic**, **LitBomb**, **mibzzer15** — ongoing testing, feedback, and issue reports
And to everyone running CoreScope on their mesh networks — your real-world data drives every fix and feature in this release. 48 commits since v3.0.0, and every one of them came from something the community found, reported, or requested.
---
*Previous release: [v3.0.0](RELEASE-v3.0.0.md)*
+1 -1
View File
@@ -148,7 +148,7 @@ async function benchmarkEndpoints(port, endpoints, nocache = false) {
}
async function run() {
console.log(`\nMeshCore Analyzer Benchmark — ${RUNS} runs per endpoint`);
console.log(`\nCoreScope Benchmark — ${RUNS} runs per endpoint`);
console.log('Launching servers...\n');
// Launch both servers
+4 -4
View File
@@ -1,6 +1,6 @@
# MeshCore MQTT Ingestor (Go)
Standalone MQTT ingestion service for MeshCore Analyzer. Connects to MQTT brokers, decodes raw MeshCore packets, and writes to the same SQLite database used by the Node.js web server.
Standalone MQTT ingestion service for CoreScope. Connects to MQTT brokers, decodes raw MeshCore packets, and writes to the same SQLite database used by the Node.js web server.
This is the first step of a larger Go rewrite — separating MQTT ingestion from the web server.
@@ -23,19 +23,19 @@ Requires Go 1.22+.
```bash
cd cmd/ingestor
go build -o meshcore-ingestor .
go build -o corescope-ingestor .
```
Cross-compile for Linux (e.g., for the production VM):
```bash
GOOS=linux GOARCH=amd64 go build -o meshcore-ingestor .
GOOS=linux GOARCH=amd64 go build -o corescope-ingestor .
```
## Run
```bash
./meshcore-ingestor -config /path/to/config.json
./corescope-ingestor -config /path/to/config.json
```
The config file uses the same format as the Node.js `config.json`. The ingestor reads the `mqttSources` array (or legacy `mqtt` object) and `dbPath` fields.
+8 -7
View File
@@ -26,13 +26,14 @@ type MQTTLegacy struct {
// Config holds the ingestor configuration, compatible with the Node.js config.json format.
type Config struct {
DBPath string `json:"dbPath"`
MQTT *MQTTLegacy `json:"mqtt,omitempty"`
MQTTSources []MQTTSource `json:"mqttSources,omitempty"`
LogLevel string `json:"logLevel,omitempty"`
ChannelKeysPath string `json:"channelKeysPath,omitempty"`
ChannelKeys map[string]string `json:"channelKeys,omitempty"`
Retention *RetentionConfig `json:"retention,omitempty"`
DBPath string `json:"dbPath"`
MQTT *MQTTLegacy `json:"mqtt,omitempty"`
MQTTSources []MQTTSource `json:"mqttSources,omitempty"`
LogLevel string `json:"logLevel,omitempty"`
ChannelKeysPath string `json:"channelKeysPath,omitempty"`
ChannelKeys map[string]string `json:"channelKeys,omitempty"`
HashChannels []string `json:"hashChannels,omitempty"`
Retention *RetentionConfig `json:"retention,omitempty"`
}
// RetentionConfig controls how long stale nodes are kept before being moved to inactive_nodes.
+156 -11
View File
@@ -35,7 +35,8 @@ type Store struct {
stmtUpsertNode *sql.Stmt
stmtIncrementAdvertCount *sql.Stmt
stmtUpsertObserver *sql.Stmt
stmtGetObserverRowid *sql.Stmt
stmtGetObserverRowid *sql.Stmt
stmtUpdateNodeTelemetry *sql.Stmt
}
// OpenStore opens or creates a SQLite DB at the given path, applying the
@@ -81,7 +82,9 @@ func applySchema(db *sql.DB) error {
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE TABLE IF NOT EXISTS observers (
@@ -97,7 +100,7 @@ func applySchema(db *sql.DB) error {
radio TEXT,
battery_mv INTEGER,
uptime_secs INTEGER,
noise_floor INTEGER
noise_floor REAL
);
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
@@ -111,7 +114,9 @@ func applySchema(db *sql.DB) error {
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE INDEX IF NOT EXISTS idx_inactive_nodes_last_seen ON inactive_nodes(last_seen);
@@ -167,6 +172,25 @@ func applySchema(db *sql.DB) error {
}
}
// Create/rebuild packets_v view (v3 schema: observer_idx → observers.rowid)
// The Go server reads this view; without it fresh installs get "no such table: packets_v".
db.Exec(`DROP VIEW IF EXISTS packets_v`)
_, vErr := db.Exec(`
CREATE VIEW packets_v AS
SELECT o.id, t.raw_hex,
datetime(o.timestamp, 'unixepoch') AS timestamp,
obs.id AS observer_id, obs.name AS observer_name,
o.direction, o.snr, o.rssi, o.score, t.hash, t.route_type,
t.payload_type, t.payload_version, o.path_json, t.decoded_json,
t.created_at
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
`)
if vErr != nil {
return fmt.Errorf("packets_v view: %w", vErr)
}
// One-time migration: recalculate advert_count to count unique transmissions only
db.Exec(`CREATE TABLE IF NOT EXISTS _migrations (name TEXT PRIMARY KEY)`)
var migDone int
@@ -184,6 +208,77 @@ func applySchema(db *sql.DB) error {
log.Println("[migration] advert_count recalculated")
}
// One-time migration: change noise_floor from INTEGER to REAL affinity.
// SQLite doesn't support ALTER COLUMN, but existing float values are stored
// as REAL regardless of column affinity. New table definition already uses REAL.
// This migration casts any integer-stored noise_floor values to real.
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'noise_floor_real_v1'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Ensuring noise_floor values are stored as REAL...")
db.Exec(`UPDATE observers SET noise_floor = CAST(noise_floor AS REAL) WHERE noise_floor IS NOT NULL AND typeof(noise_floor) = 'integer'`)
db.Exec(`INSERT INTO _migrations (name) VALUES ('noise_floor_real_v1')`)
log.Println("[migration] noise_floor migration complete")
}
// One-time migration: add telemetry columns to nodes and inactive_nodes tables.
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'node_telemetry_v1'")
if row.Scan(&migDone) != nil {
log.Println("[migration] Adding telemetry columns to nodes/inactive_nodes...")
// checkAndAddColumn checks whether `column` already exists in `table`
// using PRAGMA table_info, and adds it if missing. All call sites pass
// hardcoded table/column/type literals so there is no SQL injection risk.
checkAndAddColumn := func(table, column, colType string) error {
rows, err := db.Query(fmt.Sprintf("PRAGMA table_info(%s)", table))
if err != nil {
return fmt.Errorf("querying table info for %s: %w", table, err)
}
defer rows.Close()
exists := false
for rows.Next() {
var cid int
var name, ctype string
var notnull, pk int
var dfltValue sql.NullString
if err := rows.Scan(&cid, &name, &ctype, &notnull, &dfltValue, &pk); err != nil {
return fmt.Errorf("scanning table info for %s: %w", table, err)
}
if name == column {
exists = true
break
}
}
if err := rows.Err(); err != nil {
return fmt.Errorf("iterating table info for %s: %w", table, err)
}
if exists {
return nil
}
if _, err := db.Exec(fmt.Sprintf("ALTER TABLE %s ADD COLUMN %s %s", table, column, colType)); err != nil {
return fmt.Errorf("adding column %s to %s: %w", column, table, err)
}
return nil
}
if err := checkAndAddColumn("nodes", "battery_mv", "INTEGER"); err != nil {
return err
}
if err := checkAndAddColumn("nodes", "temperature_c", "REAL"); err != nil {
return err
}
if err := checkAndAddColumn("inactive_nodes", "battery_mv", "INTEGER"); err != nil {
return err
}
if err := checkAndAddColumn("inactive_nodes", "temperature_c", "REAL"); err != nil {
return err
}
if _, err := db.Exec(`INSERT INTO _migrations (name) VALUES ('node_telemetry_v1')`); err != nil {
return fmt.Errorf("recording node_telemetry_v1 migration: %w", err)
}
log.Println("[migration] node telemetry columns added")
}
return nil
}
@@ -238,13 +333,16 @@ func (s *Store) prepareStatements() error {
}
s.stmtUpsertObserver, err = s.db.Prepare(`
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES (?, ?, ?, ?, ?, 1)
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
VALUES (?, ?, ?, ?, ?, 1, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
name = COALESCE(?, name),
iata = COALESCE(?, iata),
last_seen = ?,
packet_count = packet_count + 1
packet_count = packet_count + 1,
battery_mv = COALESCE(?, battery_mv),
uptime_secs = COALESCE(?, uptime_secs),
noise_floor = COALESCE(?, noise_floor)
`)
if err != nil {
return err
@@ -255,6 +353,16 @@ func (s *Store) prepareStatements() error {
return err
}
s.stmtUpdateNodeTelemetry, err = s.db.Prepare(`
UPDATE nodes SET
battery_mv = COALESCE(?, battery_mv),
temperature_c = COALESCE(?, temperature_c)
WHERE public_key = ?
`)
if err != nil {
return err
}
return nil
}
@@ -359,12 +467,49 @@ func (s *Store) IncrementAdvertCount(pubKey string) error {
return err
}
// UpsertObserver inserts or updates an observer.
func (s *Store) UpsertObserver(id, name, iata string) error {
// UpdateNodeTelemetry updates battery and temperature for a node.
func (s *Store) UpdateNodeTelemetry(pubKey string, batteryMv *int, temperatureC *float64) error {
var bv, tc interface{}
if batteryMv != nil {
bv = *batteryMv
}
if temperatureC != nil {
tc = *temperatureC
}
_, err := s.stmtUpdateNodeTelemetry.Exec(bv, tc, pubKey)
if err != nil {
s.Stats.WriteErrors.Add(1)
}
return err
}
// ObserverMeta holds optional observer hardware metadata.
type ObserverMeta struct {
BatteryMv *int // millivolts, always integer
UptimeSecs *int64 // seconds, always integer
NoiseFloor *float64 // dBm, may have decimals
}
// UpsertObserver inserts or updates an observer with optional hardware metadata.
func (s *Store) UpsertObserver(id, name, iata string, meta *ObserverMeta) error {
now := time.Now().UTC().Format(time.RFC3339)
var batteryMv, uptimeSecs, noiseFloor interface{}
if meta != nil {
if meta.BatteryMv != nil {
batteryMv = *meta.BatteryMv
}
if meta.UptimeSecs != nil {
uptimeSecs = *meta.UptimeSecs
}
if meta.NoiseFloor != nil {
noiseFloor = *meta.NoiseFloor
}
}
_, err := s.stmtUpsertObserver.Exec(
id, name, iata, now, now,
name, iata, now,
id, name, iata, now, now, batteryMv, uptimeSecs, noiseFloor,
name, iata, now, batteryMv, uptimeSecs, noiseFloor,
)
if err != nil {
s.Stats.WriteErrors.Add(1)
+297 -7
View File
@@ -1,6 +1,7 @@
package main
import (
"database/sql"
"fmt"
"os"
"path/filepath"
@@ -62,6 +63,16 @@ func TestOpenStore(t *testing.T) {
t.Errorf("missing table %s, got %v", e, tables)
}
}
// Verify packets_v view exists
var viewCount int
err = s.db.QueryRow("SELECT COUNT(*) FROM sqlite_master WHERE type='view' AND name='packets_v'").Scan(&viewCount)
if err != nil {
t.Fatal(err)
}
if viewCount != 1 {
t.Error("packets_v view not created")
}
}
func TestInsertTransmission(t *testing.T) {
@@ -114,6 +125,54 @@ func TestInsertTransmission(t *testing.T) {
}
}
func TestPacketsViewQueryable(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
// Insert observer so the LEFT JOIN resolves
if err := s.UpsertObserver("obs1", "TestObserver", "SJC", nil); err != nil {
t.Fatal(err)
}
snr := 3.5
rssi := -95.0
data := &PacketData{
RawHex: "AABB",
Timestamp: "2026-01-01T00:00:00Z",
ObserverID: "obs1",
Hash: "viewtesthash",
RouteType: 1,
PayloadType: 4,
PathJSON: "[]",
DecodedJSON: `{"type":"ADVERT"}`,
SNR: &snr,
RSSI: &rssi,
}
if _, err := s.InsertTransmission(data); err != nil {
t.Fatal(err)
}
// Query through packets_v — the view the Go server relies on
var obsID, obsName sql.NullString
var hash string
err = s.db.QueryRow("SELECT observer_id, observer_name, hash FROM packets_v LIMIT 1").Scan(&obsID, &obsName, &hash)
if err != nil {
t.Fatalf("packets_v query failed: %v", err)
}
if hash != "viewtesthash" {
t.Errorf("hash=%s, want viewtesthash", hash)
}
if !obsID.Valid || obsID.String != "obs1" {
t.Errorf("observer_id=%v, want obs1", obsID)
}
if !obsName.Valid || obsName.String != "TestObserver" {
t.Errorf("observer_name=%v, want TestObserver", obsName)
}
}
func TestUpsertNode(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
@@ -160,7 +219,7 @@ func TestUpsertObserver(t *testing.T) {
}
defer s.Close()
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
t.Fatal(err)
}
@@ -174,6 +233,165 @@ func TestUpsertObserver(t *testing.T) {
}
}
func TestUpsertObserverWithMeta(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
battery := 3500
uptime := int64(86400)
noise := -115.5
meta := &ObserverMeta{
BatteryMv: &battery,
UptimeSecs: &uptime,
NoiseFloor: &noise,
}
if err := s.UpsertObserver("obs1", "Observer1", "SJC", meta); err != nil {
t.Fatal(err)
}
// Verify correct types in DB
var batteryMv int
var uptimeSecs int64
var noiseFloor float64
err = s.db.QueryRow("SELECT battery_mv, uptime_secs, noise_floor FROM observers WHERE id = 'obs1'").
Scan(&batteryMv, &uptimeSecs, &noiseFloor)
if err != nil {
t.Fatal(err)
}
if batteryMv != 3500 {
t.Errorf("battery_mv=%d, want 3500", batteryMv)
}
if uptimeSecs != 86400 {
t.Errorf("uptime_secs=%d, want 86400", uptimeSecs)
}
if noiseFloor != -115.5 {
t.Errorf("noise_floor=%f, want -115.5", noiseFloor)
}
// Verify typeof returns correct SQLite types
var typBattery, typUptime, typNoise string
s.db.QueryRow("SELECT typeof(battery_mv), typeof(uptime_secs), typeof(noise_floor) FROM observers WHERE id = 'obs1'").
Scan(&typBattery, &typUptime, &typNoise)
if typBattery != "integer" {
t.Errorf("typeof(battery_mv)=%s, want integer", typBattery)
}
if typUptime != "integer" {
t.Errorf("typeof(uptime_secs)=%s, want integer", typUptime)
}
if typNoise != "real" {
t.Errorf("typeof(noise_floor)=%s, want real", typNoise)
}
}
func TestUpsertObserverMetaPreservesExisting(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
// First upsert with metadata
battery := 3500
noise := -115.5
meta := &ObserverMeta{
BatteryMv: &battery,
NoiseFloor: &noise,
}
if err := s.UpsertObserver("obs1", "Observer1", "SJC", meta); err != nil {
t.Fatal(err)
}
// Second upsert without metadata — should preserve existing values
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
t.Fatal(err)
}
var batteryMv int
var noiseFloor float64
s.db.QueryRow("SELECT battery_mv, noise_floor FROM observers WHERE id = 'obs1'").
Scan(&batteryMv, &noiseFloor)
if batteryMv != 3500 {
t.Errorf("battery_mv=%d after nil-meta upsert, want 3500 (preserved)", batteryMv)
}
if noiseFloor != -115.5 {
t.Errorf("noise_floor=%f after nil-meta upsert, want -115.5 (preserved)", noiseFloor)
}
}
func TestExtractObserverMeta(t *testing.T) {
// Float values from JSON (typical MQTT payload)
msg := map[string]interface{}{
"battery_mv": 3500.0,
"uptime_secs": 86400.0,
"noise_floor": -115.5,
}
meta := extractObserverMeta(msg)
if meta == nil {
t.Fatal("expected non-nil meta")
}
if meta.BatteryMv == nil || *meta.BatteryMv != 3500 {
t.Errorf("BatteryMv=%v, want 3500", meta.BatteryMv)
}
if meta.UptimeSecs == nil || *meta.UptimeSecs != 86400 {
t.Errorf("UptimeSecs=%v, want 86400", meta.UptimeSecs)
}
if meta.NoiseFloor == nil || *meta.NoiseFloor != -115.5 {
t.Errorf("NoiseFloor=%v, want -115.5", meta.NoiseFloor)
}
// Battery with fractional part should round
msg2 := map[string]interface{}{
"battery_mv": 3500.7,
}
meta2 := extractObserverMeta(msg2)
if meta2 == nil || meta2.BatteryMv == nil || *meta2.BatteryMv != 3501 {
t.Errorf("battery_mv rounding: got %v, want 3501", meta2)
}
// Empty message → nil
meta3 := extractObserverMeta(map[string]interface{}{})
if meta3 != nil {
t.Errorf("expected nil for empty message, got %v", meta3)
}
}
func TestSchemaNoiseFloorIsReal(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
// Check column type affinity via PRAGMA
rows, err := s.db.Query("PRAGMA table_info(observers)")
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var cid int
var colName, colType string
var notNull, pk int
var dflt interface{}
if rows.Scan(&cid, &colName, &colType, &notNull, &dflt, &pk) == nil {
if colName == "noise_floor" && colType != "REAL" {
t.Errorf("noise_floor column type=%s, want REAL", colType)
}
if colName == "battery_mv" && colType != "INTEGER" {
t.Errorf("battery_mv column type=%s, want INTEGER", colType)
}
if colName == "uptime_secs" && colType != "INTEGER" {
t.Errorf("uptime_secs column type=%s, want INTEGER", colType)
}
}
}
}
func TestInsertTransmissionWithObserver(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
@@ -182,7 +400,7 @@ func TestInsertTransmissionWithObserver(t *testing.T) {
defer s.Close()
// Insert observer first
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
t.Fatal(err)
}
@@ -639,7 +857,7 @@ func TestConcurrentWrites(t *testing.T) {
defer s.Close()
// Pre-create an observer for observer_idx resolution
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
t.Fatal(err)
}
@@ -681,7 +899,7 @@ func TestConcurrentWrites(t *testing.T) {
return
}
obsID := fmt.Sprintf("obs_%d_%d__________", gIdx, i)
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC"); err != nil {
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC", nil); err != nil {
errCh <- fmt.Errorf("goroutine %d observer upsert %d: %w", gIdx, i, err)
return
}
@@ -782,7 +1000,7 @@ func TestDBStats(t *testing.T) {
}
// Observer upsert
if err := s.UpsertObserver("obs1", "Obs1", "SJC"); err != nil {
if err := s.UpsertObserver("obs1", "Obs1", "SJC", nil); err != nil {
t.Fatal(err)
}
if s.Stats.ObserverUpserts.Load() != 1 {
@@ -801,7 +1019,7 @@ func TestLoadTestThroughput(t *testing.T) {
defer s.Close()
// Pre-create observer
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
t.Fatal(err)
}
@@ -867,7 +1085,7 @@ func TestLoadTestThroughput(t *testing.T) {
}
obsID := fmt.Sprintf("obs_%04d_%04d_____", gIdx, i)
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC"); err != nil {
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC", nil); err != nil {
totalErrors.Add(1)
if strings.Contains(err.Error(), "locked") || strings.Contains(err.Error(), "BUSY") {
busyErrors.Add(1)
@@ -933,3 +1151,75 @@ func TestLoadTestThroughput(t *testing.T) {
t.Errorf("transmissions=%d, want %d", txCount, totalMessages)
}
}
func TestUpdateNodeTelemetry(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
lat := 37.0
lon := -122.0
if err := s.UpsertNode("telem1", "TelemetryNode", "sensor", &lat, &lon, "2026-03-25T00:00:00Z"); err != nil {
t.Fatal(err)
}
battery := 3700
temp := 28.5
if err := s.UpdateNodeTelemetry("telem1", &battery, &temp); err != nil {
t.Fatal(err)
}
var bv int
var tc float64
err = s.db.QueryRow("SELECT battery_mv, temperature_c FROM nodes WHERE public_key = 'telem1'").Scan(&bv, &tc)
if err != nil {
t.Fatal(err)
}
if bv != 3700 {
t.Errorf("battery_mv=%d, want 3700", bv)
}
if tc != 28.5 {
t.Errorf("temperature_c=%f, want 28.5", tc)
}
newTemp := -5.0
if err := s.UpdateNodeTelemetry("telem1", nil, &newTemp); err != nil {
t.Fatal(err)
}
err = s.db.QueryRow("SELECT battery_mv, temperature_c FROM nodes WHERE public_key = 'telem1'").Scan(&bv, &tc)
if err != nil {
t.Fatal(err)
}
if bv != 3700 {
t.Errorf("battery_mv after nil update=%d, want 3700 (preserved)", bv)
}
if tc != -5.0 {
t.Errorf("temperature_c after update=%f, want -5.0", tc)
}
}
func TestTelemetryMigrationAddsColumns(t *testing.T) {
s, err := OpenStore(tempDBPath(t))
if err != nil {
t.Fatal(err)
}
defer s.Close()
_, err = s.db.Exec("SELECT battery_mv, temperature_c FROM nodes LIMIT 1")
if err != nil {
t.Errorf("nodes table should have battery_mv and temperature_c columns: %v", err)
}
_, err = s.db.Exec("SELECT battery_mv, temperature_c FROM inactive_nodes LIMIT 1")
if err != nil {
t.Errorf("inactive_nodes table should have battery_mv and temperature_c columns: %v", err)
}
var count int
s.db.QueryRow("SELECT COUNT(*) FROM _migrations WHERE name = 'node_telemetry_v1'").Scan(&count)
if count != 1 {
t.Errorf("migration node_telemetry_v1 should be recorded, count=%d", count)
}
}
+31 -2
View File
@@ -111,6 +111,8 @@ type Payload struct {
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
Name string `json:"name,omitempty"`
BatteryMv *int `json:"battery_mv,omitempty"`
TemperatureC *float64 `json:"temperature_c,omitempty"`
ChannelHash int `json:"channelHash,omitempty"`
ChannelHashHex string `json:"channelHashHex,omitempty"`
DecryptionStatus string `json:"decryptionStatus,omitempty"`
@@ -251,10 +253,37 @@ func decodeAdvert(buf []byte) Payload {
off += 8
}
if p.Flags.HasName {
name := string(appdata[off:])
name = strings.TrimRight(name, "\x00")
// Find null terminator to separate name from trailing telemetry bytes
nameEnd := len(appdata)
for i := off; i < len(appdata); i++ {
if appdata[i] == 0x00 {
nameEnd = i
break
}
}
name := string(appdata[off:nameEnd])
name = sanitizeName(name)
p.Name = name
off = nameEnd
// Skip null terminator(s)
for off < len(appdata) && appdata[off] == 0x00 {
off++
}
}
// Telemetry bytes after name: battery_mv(2 LE) + temperature_c(2 LE, signed, /100)
// Only sensor nodes (advType=4) carry telemetry bytes.
if p.Flags.Sensor && off+4 <= len(appdata) {
batteryMv := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
tempRaw := int16(binary.LittleEndian.Uint16(appdata[off+2 : off+4]))
tempC := float64(tempRaw) / 100.0
if batteryMv > 0 && batteryMv <= 10000 {
p.BatteryMv = &batteryMv
}
// Raw int16 / 100 → °C; accept -50°C to 100°C (raw: -5000 to 10000)
if tempRaw >= -5000 && tempRaw <= 10000 {
p.TemperatureC = &tempC
}
}
}
+151
View File
@@ -1355,3 +1355,154 @@ func TestDecodeGrpTxtGarbageMarkedFailed(t *testing.T) {
t.Errorf("type=%s, want GRP_TXT", p.Type)
}
}
func TestDecodeAdvertWithTelemetry(t *testing.T) {
pubkey := strings.Repeat("AA", 32)
timestamp := "78563412"
signature := strings.Repeat("BB", 64)
flags := "94" // sensor(4) | hasLocation(0x10) | hasName(0x80)
lat := "40933402"
lon := "E0E6B8F8"
name := hex.EncodeToString([]byte("Sensor1"))
nullTerm := "00"
batteryLE := make([]byte, 2)
binary.LittleEndian.PutUint16(batteryLE, 3700)
tempLE := make([]byte, 2)
binary.LittleEndian.PutUint16(tempLE, uint16(int16(2850)))
hexStr := "1200" + pubkey + timestamp + signature + flags + lat + lon +
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
if err != nil {
t.Fatal(err)
}
if pkt.Payload.Name != "Sensor1" {
t.Errorf("name=%s, want Sensor1", pkt.Payload.Name)
}
if pkt.Payload.BatteryMv == nil {
t.Fatal("battery_mv should not be nil")
}
if *pkt.Payload.BatteryMv != 3700 {
t.Errorf("battery_mv=%d, want 3700", *pkt.Payload.BatteryMv)
}
if pkt.Payload.TemperatureC == nil {
t.Fatal("temperature_c should not be nil")
}
if math.Abs(*pkt.Payload.TemperatureC-28.50) > 0.01 {
t.Errorf("temperature_c=%f, want 28.50", *pkt.Payload.TemperatureC)
}
}
func TestDecodeAdvertWithTelemetryNegativeTemp(t *testing.T) {
pubkey := strings.Repeat("CC", 32)
timestamp := "00000000"
signature := strings.Repeat("DD", 64)
flags := "84" // sensor(4) | hasName(0x80), no location
name := hex.EncodeToString([]byte("Cold"))
nullTerm := "00"
batteryLE := make([]byte, 2)
binary.LittleEndian.PutUint16(batteryLE, 4200)
tempLE := make([]byte, 2)
var negTemp int16 = -550
binary.LittleEndian.PutUint16(tempLE, uint16(negTemp))
hexStr := "1200" + pubkey + timestamp + signature + flags +
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
if err != nil {
t.Fatal(err)
}
if pkt.Payload.Name != "Cold" {
t.Errorf("name=%s, want Cold", pkt.Payload.Name)
}
if pkt.Payload.BatteryMv == nil || *pkt.Payload.BatteryMv != 4200 {
t.Errorf("battery_mv=%v, want 4200", pkt.Payload.BatteryMv)
}
if pkt.Payload.TemperatureC == nil {
t.Fatal("temperature_c should not be nil")
}
if math.Abs(*pkt.Payload.TemperatureC-(-5.50)) > 0.01 {
t.Errorf("temperature_c=%f, want -5.50", *pkt.Payload.TemperatureC)
}
}
func TestDecodeAdvertWithoutTelemetry(t *testing.T) {
pubkey := strings.Repeat("EE", 32)
timestamp := "00000000"
signature := strings.Repeat("FF", 64)
flags := "82" // repeater(2) | hasName(0x80)
name := hex.EncodeToString([]byte("Node1"))
hexStr := "1200" + pubkey + timestamp + signature + flags + name
pkt, err := DecodePacket(hexStr, nil)
if err != nil {
t.Fatal(err)
}
if pkt.Payload.Name != "Node1" {
t.Errorf("name=%s, want Node1", pkt.Payload.Name)
}
if pkt.Payload.BatteryMv != nil {
t.Errorf("battery_mv should be nil for advert without telemetry, got %d", *pkt.Payload.BatteryMv)
}
if pkt.Payload.TemperatureC != nil {
t.Errorf("temperature_c should be nil for advert without telemetry, got %f", *pkt.Payload.TemperatureC)
}
}
func TestDecodeAdvertNonSensorIgnoresTelemetryBytes(t *testing.T) {
// A repeater node with 4 trailing bytes after the name should NOT decode telemetry.
pubkey := strings.Repeat("AB", 32)
timestamp := "00000000"
signature := strings.Repeat("CD", 64)
flags := "82" // repeater(2) | hasName(0x80)
name := hex.EncodeToString([]byte("Rptr"))
nullTerm := "00"
extraBytes := "B40ED403" // battery-like and temp-like bytes
hexStr := "1200" + pubkey + timestamp + signature + flags + name + nullTerm + extraBytes
pkt, err := DecodePacket(hexStr, nil)
if err != nil {
t.Fatal(err)
}
if pkt.Payload.BatteryMv != nil {
t.Errorf("battery_mv should be nil for non-sensor node, got %d", *pkt.Payload.BatteryMv)
}
if pkt.Payload.TemperatureC != nil {
t.Errorf("temperature_c should be nil for non-sensor node, got %f", *pkt.Payload.TemperatureC)
}
}
func TestDecodeAdvertTelemetryZeroTemp(t *testing.T) {
// 0°C is a valid temperature and must be emitted.
pubkey := strings.Repeat("12", 32)
timestamp := "00000000"
signature := strings.Repeat("34", 64)
flags := "84" // sensor(4) | hasName(0x80)
name := hex.EncodeToString([]byte("FreezeSensor"))
nullTerm := "00"
batteryLE := make([]byte, 2)
binary.LittleEndian.PutUint16(batteryLE, 3600)
tempLE := make([]byte, 2) // tempRaw=0 → 0°C
hexStr := "1200" + pubkey + timestamp + signature + flags +
name + nullTerm +
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
pkt, err := DecodePacket(hexStr, nil)
if err != nil {
t.Fatal(err)
}
if pkt.Payload.TemperatureC == nil {
t.Fatal("temperature_c should not be nil for 0°C")
}
if *pkt.Payload.TemperatureC != 0.0 {
t.Errorf("temperature_c=%f, want 0.0", *pkt.Payload.TemperatureC)
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
module github.com/meshcore-analyzer/ingestor
module github.com/corescope/ingestor
go 1.22
+95 -8
View File
@@ -8,6 +8,9 @@ import (
"flag"
"fmt"
"log"
"math"
"net/http"
_ "net/http/pprof"
"os"
"os/signal"
"path/filepath"
@@ -19,6 +22,20 @@ import (
)
func main() {
// pprof profiling — off by default, enable with ENABLE_PPROF=true
if os.Getenv("ENABLE_PPROF") == "true" {
pprofPort := os.Getenv("PPROF_PORT")
if pprofPort == "" {
pprofPort = "6061"
}
go func() {
log.Printf("[pprof] ingestor profiling at http://localhost:%s/debug/pprof/", pprofPort)
if err := http.ListenAndServe(":"+pprofPort, nil); err != nil {
log.Printf("[pprof] failed to start: %v (non-fatal)", err)
}
}()
}
configPath := flag.String("config", "config.json", "path to config file")
flag.Parse()
@@ -193,7 +210,8 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
observerID := parts[2]
name, _ := msg["origin"].(string)
iata := parts[1]
if err := store.UpsertObserver(observerID, name, iata); err != nil {
meta := extractObserverMeta(msg)
if err := store.UpsertObserver(observerID, name, iata, meta); err != nil {
log.Printf("MQTT [%s] observer status error: %v", tag, err)
}
log.Printf("MQTT [%s] status: %s (%s)", tag, firstNonEmpty(name, observerID), iata)
@@ -252,6 +270,12 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
log.Printf("MQTT [%s] advert count error: %v", tag, err)
}
}
// Update telemetry if present in advert
if decoded.Payload.BatteryMv != nil || decoded.Payload.TemperatureC != nil {
if err := store.UpdateNodeTelemetry(decoded.Payload.PubKey, decoded.Payload.BatteryMv, decoded.Payload.TemperatureC); err != nil {
log.Printf("MQTT [%s] node telemetry update error: %v", tag, err)
}
}
} else {
log.Printf("MQTT [%s] skipping corrupted ADVERT: %s", tag, reason)
}
@@ -260,7 +284,7 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
// Upsert observer
if observerID != "" {
origin, _ := msg["origin"].(string)
if err := store.UpsertObserver(observerID, origin, region); err != nil {
if err := store.UpsertObserver(observerID, origin, region, nil); err != nil {
log.Printf("MQTT [%s] observer upsert error: %v", tag, err)
}
}
@@ -446,6 +470,39 @@ func toFloat64(v interface{}) (float64, bool) {
}
}
// extractObserverMeta extracts hardware metadata from an MQTT status message.
// Casts battery_mv and uptime_secs to integers (they're always whole numbers).
func extractObserverMeta(msg map[string]interface{}) *ObserverMeta {
meta := &ObserverMeta{}
hasData := false
if v, ok := msg["battery_mv"]; ok {
if f, ok := toFloat64(v); ok {
iv := int(math.Round(f))
meta.BatteryMv = &iv
hasData = true
}
}
if v, ok := msg["uptime_secs"]; ok {
if f, ok := toFloat64(v); ok {
iv := int64(math.Round(f))
meta.UptimeSecs = &iv
hasData = true
}
}
if v, ok := msg["noise_floor"]; ok {
if f, ok := toFloat64(v); ok {
meta.NoiseFloor = &f
hasData = true
}
}
if !hasData {
return nil
}
return meta
}
func firstNonEmpty(vals ...string) string {
for _, v := range vals {
if v != "" {
@@ -455,34 +512,64 @@ func firstNonEmpty(vals ...string) string {
return ""
}
// deriveHashtagChannelKey derives an AES-128 key from a channel name.
// Same algorithm as Node.js: SHA-256(channelName) → first 32 hex chars (16 bytes).
func deriveHashtagChannelKey(channelName string) string {
h := sha256.Sum256([]byte(channelName))
return hex.EncodeToString(h[:16])
}
// loadChannelKeys loads channel decryption keys from config and/or a JSON file.
// Priority: CHANNEL_KEYS_PATH env var > cfg.ChannelKeysPath > channel-rainbow.json next to config.
// Merge priority: rainbow (lowest) → derived from hashChannels → explicit config (highest).
func loadChannelKeys(cfg *Config, configPath string) map[string]string {
keys := make(map[string]string)
// Determine file path for rainbow keys
// 1. Rainbow table keys (lowest priority)
keysPath := os.Getenv("CHANNEL_KEYS_PATH")
if keysPath == "" {
keysPath = cfg.ChannelKeysPath
}
if keysPath == "" {
// Default: look for channel-rainbow.json next to config file
keysPath = filepath.Join(filepath.Dir(configPath), "channel-rainbow.json")
}
rainbowCount := 0
if data, err := os.ReadFile(keysPath); err == nil {
var fileKeys map[string]string
if err := json.Unmarshal(data, &fileKeys); err == nil {
for k, v := range fileKeys {
keys[k] = v
}
log.Printf("Loaded %d channel keys from %s", len(fileKeys), keysPath)
rainbowCount = len(fileKeys)
log.Printf("Loaded %d channel keys from %s", rainbowCount, keysPath)
} else {
log.Printf("Warning: failed to parse channel keys file %s: %v", keysPath, err)
}
}
// Merge inline config keys (override file keys)
// 2. Derived keys from hashChannels (middle priority)
derivedCount := 0
for _, raw := range cfg.HashChannels {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
continue
}
channelName := trimmed
if !strings.HasPrefix(channelName, "#") {
channelName = "#" + channelName
}
// Skip if explicit config already has this key
if _, exists := cfg.ChannelKeys[channelName]; exists {
continue
}
keys[channelName] = deriveHashtagChannelKey(channelName)
derivedCount++
}
if derivedCount > 0 {
log.Printf("[channels] %d derived from hashChannels", derivedCount)
}
// 3. Explicit config keys (highest priority — overrides rainbow + derived)
for k, v := range cfg.ChannelKeys {
keys[k] = v
}
@@ -495,7 +582,7 @@ var version = "dev"
func init() {
if len(os.Args) > 1 && os.Args[1] == "--version" {
fmt.Println("meshcore-ingestor", version)
fmt.Println("corescope-ingestor", version)
os.Exit(0)
}
}
+131
View File
@@ -3,6 +3,8 @@ package main
import (
"encoding/json"
"math"
"os"
"path/filepath"
"testing"
"time"
)
@@ -492,3 +494,132 @@ func TestAdvertRole(t *testing.T) {
})
}
}
func TestDeriveHashtagChannelKey(t *testing.T) {
// Test vectors validated against Node.js server-helpers.js
tests := []struct {
name string
want string
}{
{"#General", "649af2cab73ed5a890890a5485a0c004"},
{"#test", "9cd8fcf22a47333b591d96a2b848b73f"},
{"#MeshCore", "dcf73f393fa217f6b28fcec6ffc411ad"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := deriveHashtagChannelKey(tt.name)
if got != tt.want {
t.Errorf("deriveHashtagChannelKey(%q) = %q, want %q", tt.name, got, tt.want)
}
})
}
// Deterministic
k1 := deriveHashtagChannelKey("#foo")
k2 := deriveHashtagChannelKey("#foo")
if k1 != k2 {
t.Error("deriveHashtagChannelKey should be deterministic")
}
// Returns 32-char hex string (16 bytes)
if len(k1) != 32 {
t.Errorf("key length = %d, want 32", len(k1))
}
// Different inputs → different keys
k3 := deriveHashtagChannelKey("#bar")
if k1 == k3 {
t.Error("different inputs should produce different keys")
}
}
func TestLoadChannelKeysMergePriority(t *testing.T) {
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.json")
// Create a rainbow file with two keys: #rainbow (unique) and #override (to be overridden)
rainbowPath := filepath.Join(dir, "channel-rainbow.json")
t.Setenv("CHANNEL_KEYS_PATH", rainbowPath)
rainbow := map[string]string{
"#rainbow": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"#override": "rainbow_value_should_be_overridden",
}
rainbowJSON, err := json.Marshal(rainbow)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(rainbowPath, rainbowJSON, 0o644); err != nil {
t.Fatal(err)
}
cfg := &Config{
HashChannels: []string{"General", "#override"},
ChannelKeys: map[string]string{"#override": "explicit_wins"},
}
keys := loadChannelKeys(cfg, cfgPath)
// Rainbow key loaded
if keys["#rainbow"] != "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Errorf("rainbow key missing or wrong: %q", keys["#rainbow"])
}
// HashChannels derived #General
expected := deriveHashtagChannelKey("#General")
if keys["#General"] != expected {
t.Errorf("#General = %q, want %q (derived)", keys["#General"], expected)
}
// Explicit config wins over both rainbow and derived
if keys["#override"] != "explicit_wins" {
t.Errorf("#override = %q, want explicit_wins", keys["#override"])
}
}
func TestLoadChannelKeysHashChannelsNormalization(t *testing.T) {
t.Setenv("CHANNEL_KEYS_PATH", "")
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.json")
cfg := &Config{
HashChannels: []string{
"NoPound", // should become #NoPound
"#HasPound", // stays #HasPound
" Spaced ", // trimmed → #Spaced
"", // skipped
},
}
keys := loadChannelKeys(cfg, cfgPath)
if _, ok := keys["#NoPound"]; !ok {
t.Error("should derive key for #NoPound (auto-prefixed)")
}
if _, ok := keys["#HasPound"]; !ok {
t.Error("should derive key for #HasPound")
}
if _, ok := keys["#Spaced"]; !ok {
t.Error("should derive key for #Spaced (trimmed)")
}
if len(keys) != 3 {
t.Errorf("expected 3 keys, got %d", len(keys))
}
}
func TestLoadChannelKeysSkipExplicit(t *testing.T) {
t.Setenv("CHANNEL_KEYS_PATH", "")
dir := t.TempDir()
cfgPath := filepath.Join(dir, "config.json")
cfg := &Config{
HashChannels: []string{"General"},
ChannelKeys: map[string]string{"#General": "my_explicit_key"},
}
keys := loadChannelKeys(cfg, cfgPath)
// Explicit key should win — hashChannels derivation should be skipped
if keys["#General"] != "my_explicit_key" {
t.Errorf("#General = %q, want my_explicit_key", keys["#General"])
}
}
+9 -30
View File
@@ -26,12 +26,13 @@ func setupTestDBv2(t *testing.T) *DB {
schema := `
CREATE TABLE nodes (
public_key TEXT PRIMARY KEY, name TEXT, role TEXT,
lat REAL, lon REAL, last_seen TEXT, first_seen TEXT, advert_count INTEGER DEFAULT 0
lat REAL, lon REAL, last_seen TEXT, first_seen TEXT, advert_count INTEGER DEFAULT 0,
battery_mv INTEGER, temperature_c REAL
);
CREATE TABLE observers (
id TEXT PRIMARY KEY, name TEXT, iata TEXT, last_seen TEXT, first_seen TEXT,
packet_count INTEGER DEFAULT 0, model TEXT, firmware TEXT,
client_version TEXT, radio TEXT, battery_mv INTEGER, uptime_secs INTEGER, noise_floor INTEGER
client_version TEXT, radio TEXT, battery_mv INTEGER, uptime_secs INTEGER, noise_floor REAL
);
CREATE TABLE transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT, raw_hex TEXT NOT NULL,
@@ -45,14 +46,6 @@ func setupTestDBv2(t *testing.T) *DB {
observer_id TEXT, observer_name TEXT, direction TEXT,
snr REAL, rssi REAL, score INTEGER, path_json TEXT, timestamp INTEGER NOT NULL
);
CREATE VIEW packets_v AS
SELECT o.id, t.raw_hex,
strftime('%Y-%m-%dT%H:%M:%fZ', o.timestamp, 'unixepoch') AS timestamp,
o.observer_id, o.observer_name,
o.direction, o.snr, o.rssi, o.score, t.hash, t.route_type,
t.payload_type, t.payload_version, o.path_json, t.decoded_json, t.created_at
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id;
`
if _, err := conn.Exec(schema); err != nil {
t.Fatal(err)
@@ -550,8 +543,8 @@ func TestHandlePacketDetailNoStore(t *testing.T) {
req := httptest.NewRequest("GET", "/api/packets/abc123def4567890", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
if w.Code != 404 {
t.Fatalf("expected 404 (no store), got %d: %s", w.Code, w.Body.String())
}
})
@@ -559,8 +552,8 @@ func TestHandlePacketDetailNoStore(t *testing.T) {
req := httptest.NewRequest("GET", "/api/packets/1", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
if w.Code != 404 {
t.Fatalf("expected 404 (no store), got %d: %s", w.Code, w.Body.String())
}
})
@@ -1474,8 +1467,8 @@ func TestHandleObserverAnalyticsNoStore(t *testing.T) {
req := httptest.NewRequest("GET", "/api/observers/obs1/analytics", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
if w.Code != 503 {
t.Fatalf("expected 503, got %d: %s", w.Code, w.Body.String())
}
}
@@ -3271,20 +3264,6 @@ func TestHandlePacketDetailWithStoreAllPaths(t *testing.T) {
// --- Additional DB function coverage ---
func TestDBGetTimestamps(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
ts, err := db.GetTimestamps("2000-01-01")
if err != nil {
t.Fatal(err)
}
if len(ts) < 1 {
t.Error("expected >=1 timestamps")
}
}
func TestDBGetNewTransmissionsSince(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
+91 -209
View File
@@ -120,31 +120,33 @@ func (db *DB) scanTransmissionRow(rows *sql.Rows) map[string]interface{} {
// Node represents a row from the nodes table.
type Node struct {
PublicKey string `json:"public_key"`
Name *string `json:"name"`
Role *string `json:"role"`
Lat *float64 `json:"lat"`
Lon *float64 `json:"lon"`
LastSeen *string `json:"last_seen"`
FirstSeen *string `json:"first_seen"`
AdvertCount int `json:"advert_count"`
PublicKey string `json:"public_key"`
Name *string `json:"name"`
Role *string `json:"role"`
Lat *float64 `json:"lat"`
Lon *float64 `json:"lon"`
LastSeen *string `json:"last_seen"`
FirstSeen *string `json:"first_seen"`
AdvertCount int `json:"advert_count"`
BatteryMv *int `json:"battery_mv"`
TemperatureC *float64 `json:"temperature_c"`
}
// Observer represents a row from the observers table.
type Observer struct {
ID string `json:"id"`
Name *string `json:"name"`
IATA *string `json:"iata"`
LastSeen *string `json:"last_seen"`
FirstSeen *string `json:"first_seen"`
PacketCount int `json:"packet_count"`
Model *string `json:"model"`
Firmware *string `json:"firmware"`
ClientVersion *string `json:"client_version"`
Radio *string `json:"radio"`
BatteryMv *int `json:"battery_mv"`
UptimeSecs *int `json:"uptime_secs"`
NoiseFloor *int `json:"noise_floor"`
ID string `json:"id"`
Name *string `json:"name"`
IATA *string `json:"iata"`
LastSeen *string `json:"last_seen"`
FirstSeen *string `json:"first_seen"`
PacketCount int `json:"packet_count"`
Model *string `json:"model"`
Firmware *string `json:"firmware"`
ClientVersion *string `json:"client_version"`
Radio *string `json:"radio"`
BatteryMv *int `json:"battery_mv"`
UptimeSecs *int64 `json:"uptime_secs"`
NoiseFloor *float64 `json:"noise_floor"`
}
// Transmission represents a row from the transmissions table.
@@ -160,7 +162,7 @@ type Transmission struct {
CreatedAt *string `json:"created_at"`
}
// Observation (from packets_v view).
// Observation (observation-level data).
type Observation struct {
ID int `json:"id"`
RawHex *string `json:"raw_hex"`
@@ -433,7 +435,7 @@ func (db *DB) QueryGroupedPackets(q PacketQuery) (*PacketResult, error) {
w = "WHERE " + strings.Join(where, " AND ")
}
// Count total transmissions (fast — queries transmissions directly, not packets_v)
// Count total transmissions (fast — queries transmissions directly, not a VIEW)
var total int
if len(where) == 0 {
db.conn.QueryRow("SELECT COUNT(*) FROM transmissions").Scan(&total)
@@ -626,18 +628,6 @@ func (db *DB) resolveNodePubkey(nodeIDOrName string) string {
return pk
}
// GetPacketByID fetches a single packet/observation.
func (db *DB) GetPacketByID(id int) (map[string]interface{}, error) {
rows, err := db.conn.Query("SELECT id, raw_hex, timestamp, observer_id, observer_name, direction, snr, rssi, score, hash, route_type, payload_type, payload_version, path_json, decoded_json, created_at FROM packets_v WHERE id = ?", id)
if err != nil {
return nil, err
}
defer rows.Close()
if rows.Next() {
return scanPacketRow(rows), nil
}
return nil, nil
}
// GetTransmissionByID fetches from transmissions table with observer data.
func (db *DB) GetTransmissionByID(id int) (map[string]interface{}, error) {
@@ -671,24 +661,6 @@ func (db *DB) GetPacketByHash(hash string) (map[string]interface{}, error) {
return nil, nil
}
// GetObservationsForHash returns all observations for a given hash.
func (db *DB) GetObservationsForHash(hash string) ([]map[string]interface{}, error) {
rows, err := db.conn.Query(`SELECT id, raw_hex, timestamp, observer_id, observer_name, direction, snr, rssi, score, hash, route_type, payload_type, payload_version, path_json, decoded_json, created_at
FROM packets_v WHERE hash = ? ORDER BY timestamp DESC`, strings.ToLower(hash))
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]map[string]interface{}, 0)
for rows.Next() {
p := scanPacketRow(rows)
if p != nil {
result = append(result, p)
}
}
return result, nil
}
// GetNodes returns filtered, paginated node list.
func (db *DB) GetNodes(limit, offset int, role, search, before, lastHeard, sortBy, region string) ([]map[string]interface{}, int, map[string]int, error) {
@@ -739,7 +711,7 @@ func (db *DB) GetNodes(limit, offset int, role, search, before, lastHeard, sortB
var total int
db.conn.QueryRow(fmt.Sprintf("SELECT COUNT(*) FROM nodes %s", w), args...).Scan(&total)
querySQL := fmt.Sprintf("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count FROM nodes %s ORDER BY %s LIMIT ? OFFSET ?", w, order)
querySQL := fmt.Sprintf("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c FROM nodes %s ORDER BY %s LIMIT ? OFFSET ?", w, order)
qArgs := append(args, limit, offset)
rows, err := db.conn.Query(querySQL, qArgs...)
@@ -765,7 +737,7 @@ func (db *DB) SearchNodes(query string, limit int) ([]map[string]interface{}, er
if limit <= 0 {
limit = 10
}
rows, err := db.conn.Query(`SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count
rows, err := db.conn.Query(`SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c
FROM nodes WHERE name LIKE ? OR public_key LIKE ? ORDER BY last_seen DESC LIMIT ?`,
"%"+query+"%", query+"%", limit)
if err != nil {
@@ -785,7 +757,7 @@ func (db *DB) SearchNodes(query string, limit int) ([]map[string]interface{}, er
// GetNodeByPubkey returns a single node.
func (db *DB) GetNodeByPubkey(pubkey string) (map[string]interface{}, error) {
rows, err := db.conn.Query("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count FROM nodes WHERE public_key = ?", pubkey)
rows, err := db.conn.Query("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c FROM nodes WHERE public_key = ?", pubkey)
if err != nil {
return nil, err
}
@@ -796,30 +768,6 @@ func (db *DB) GetNodeByPubkey(pubkey string) (map[string]interface{}, error) {
return nil, nil
}
// GetRecentPacketsForNode returns recent packets referencing a node.
func (db *DB) GetRecentPacketsForNode(pubkey string, name string, limit int) ([]map[string]interface{}, error) {
if limit <= 0 {
limit = 20
}
pk := "%" + pubkey + "%"
np := "%" + name + "%"
rows, err := db.conn.Query(`SELECT id, raw_hex, timestamp, observer_id, observer_name, direction, snr, rssi, score, hash, route_type, payload_type, payload_version, path_json, decoded_json, created_at
FROM packets_v WHERE decoded_json LIKE ? OR decoded_json LIKE ?
ORDER BY timestamp DESC LIMIT ?`, pk, np, limit)
if err != nil {
return nil, err
}
defer rows.Close()
packets := make([]map[string]interface{}, 0)
for rows.Next() {
p := scanPacketRow(rows)
if p != nil {
packets = append(packets, p)
}
}
return packets, nil
}
// GetRecentTransmissionsForNode returns recent transmissions referencing a node (Node.js-compatible shape).
func (db *DB) GetRecentTransmissionsForNode(pubkey string, name string, limit int) ([]map[string]interface{}, error) {
@@ -958,9 +906,21 @@ func (db *DB) GetObservers() ([]Observer, error) {
var observers []Observer
for rows.Next() {
var o Observer
if err := rows.Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &o.BatteryMv, &o.UptimeSecs, &o.NoiseFloor); err != nil {
var batteryMv, uptimeSecs sql.NullInt64
var noiseFloor sql.NullFloat64
if err := rows.Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor); err != nil {
continue
}
if batteryMv.Valid {
v := int(batteryMv.Int64)
o.BatteryMv = &v
}
if uptimeSecs.Valid {
o.UptimeSecs = &uptimeSecs.Int64
}
if noiseFloor.Valid {
o.NoiseFloor = &noiseFloor.Float64
}
observers = append(observers, o)
}
return observers, nil
@@ -969,11 +929,23 @@ func (db *DB) GetObservers() ([]Observer, error) {
// GetObserverByID returns a single observer.
func (db *DB) GetObserverByID(id string) (*Observer, error) {
var o Observer
var batteryMv, uptimeSecs sql.NullInt64
var noiseFloor sql.NullFloat64
err := db.conn.QueryRow("SELECT id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor FROM observers WHERE id = ?", id).
Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &o.BatteryMv, &o.UptimeSecs, &o.NoiseFloor)
Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor)
if err != nil {
return nil, err
}
if batteryMv.Valid {
v := int(batteryMv.Int64)
o.BatteryMv = &v
}
if uptimeSecs.Valid {
o.UptimeSecs = &uptimeSecs.Int64
}
if noiseFloor.Valid {
o.NoiseFloor = &noiseFloor.Float64
}
return &o, nil
}
@@ -1019,103 +991,6 @@ func (db *DB) GetDistinctIATAs() ([]string, error) {
return codes, nil
}
// GetNodeHealth returns health info for a node (observers, stats, recent packets).
func (db *DB) GetNodeHealth(pubkey string) (map[string]interface{}, error) {
node, err := db.GetNodeByPubkey(pubkey)
if err != nil || node == nil {
return nil, err
}
name := ""
if n, ok := node["name"]; ok && n != nil {
name = fmt.Sprintf("%v", n)
}
pk := "%" + pubkey + "%"
np := "%" + name + "%"
whereClause := "decoded_json LIKE ? OR decoded_json LIKE ?"
if name == "" {
whereClause = "decoded_json LIKE ?"
np = pk
}
todayStart := time.Now().UTC().Truncate(24 * time.Hour).Format(time.RFC3339)
// Observers
observerSQL := fmt.Sprintf(`SELECT observer_id, observer_name, AVG(snr) as avgSnr, AVG(rssi) as avgRssi, COUNT(*) as packetCount
FROM packets_v WHERE (%s) AND observer_id IS NOT NULL GROUP BY observer_id ORDER BY packetCount DESC`, whereClause)
oRows, err := db.conn.Query(observerSQL, pk, np)
if err != nil {
return nil, err
}
defer oRows.Close()
observers := make([]map[string]interface{}, 0)
for oRows.Next() {
var obsID, obsName sql.NullString
var avgSnr, avgRssi sql.NullFloat64
var pktCount int
oRows.Scan(&obsID, &obsName, &avgSnr, &avgRssi, &pktCount)
observers = append(observers, map[string]interface{}{
"observer_id": nullStr(obsID),
"observer_name": nullStr(obsName),
"avgSnr": nullFloat(avgSnr),
"avgRssi": nullFloat(avgRssi),
"packetCount": pktCount,
})
}
// Stats
var packetsToday, totalPackets int
db.conn.QueryRow(fmt.Sprintf("SELECT COUNT(*) FROM packets_v WHERE (%s) AND timestamp > ?", whereClause), pk, np, todayStart).Scan(&packetsToday)
db.conn.QueryRow(fmt.Sprintf("SELECT COUNT(*) FROM packets_v WHERE (%s)", whereClause), pk, np).Scan(&totalPackets)
var avgSnr sql.NullFloat64
db.conn.QueryRow(fmt.Sprintf("SELECT AVG(snr) FROM packets_v WHERE (%s)", whereClause), pk, np).Scan(&avgSnr)
var lastHeard sql.NullString
db.conn.QueryRow(fmt.Sprintf("SELECT MAX(timestamp) FROM packets_v WHERE (%s)", whereClause), pk, np).Scan(&lastHeard)
// Avg hops
hRows, _ := db.conn.Query(fmt.Sprintf("SELECT path_json FROM packets_v WHERE (%s) AND path_json IS NOT NULL", whereClause), pk, np)
totalHops, hopCount := 0, 0
if hRows != nil {
defer hRows.Close()
for hRows.Next() {
var pj sql.NullString
hRows.Scan(&pj)
if pj.Valid {
var hops []interface{}
if json.Unmarshal([]byte(pj.String), &hops) == nil {
totalHops += len(hops)
hopCount++
}
}
}
}
avgHops := 0
if hopCount > 0 {
avgHops = int(math.Round(float64(totalHops) / float64(hopCount)))
}
// Recent packets
recentPackets, _ := db.GetRecentTransmissionsForNode(pubkey, name, 20)
return map[string]interface{}{
"node": node,
"observers": observers,
"stats": map[string]interface{}{
"totalTransmissions": totalPackets,
"totalObservations": totalPackets,
"totalPackets": totalPackets,
"packetsToday": packetsToday,
"avgSnr": nullFloat(avgSnr),
"avgHops": avgHops,
"lastHeard": nullStr(lastHeard),
},
"recentPackets": recentPackets,
}, nil
}
// GetNetworkStatus returns overall network health status.
func (db *DB) GetNetworkStatus(healthThresholds HealthThresholds) (map[string]interface{}, error) {
@@ -1164,10 +1039,28 @@ func (db *DB) GetNetworkStatus(healthThresholds HealthThresholds) (map[string]in
}, nil
}
// GetTraces returns observations for a hash.
// GetTraces returns observations for a hash using direct table queries.
func (db *DB) GetTraces(hash string) ([]map[string]interface{}, error) {
rows, err := db.conn.Query(`SELECT observer_id, observer_name, timestamp, snr, rssi, path_json
FROM packets_v WHERE hash = ? ORDER BY timestamp ASC`, strings.ToLower(hash))
var querySQL string
if db.isV3 {
querySQL = `SELECT obs.id AS observer_id, obs.name AS observer_name,
strftime('%Y-%m-%dT%H:%M:%fZ', o.timestamp, 'unixepoch') AS timestamp,
o.snr, o.rssi, o.path_json
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
WHERE t.hash = ?
ORDER BY o.timestamp ASC`
} else {
querySQL = `SELECT o.observer_id, o.observer_name,
strftime('%Y-%m-%dT%H:%M:%fZ', o.timestamp, 'unixepoch') AS timestamp,
o.snr, o.rssi, o.path_json
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
WHERE t.hash = ?
ORDER BY o.timestamp ASC`
}
rows, err := db.conn.Query(querySQL, strings.ToLower(hash))
if err != nil {
return nil, err
}
@@ -1193,7 +1086,7 @@ func (db *DB) GetTraces(hash string) ([]map[string]interface{}, error) {
}
// GetChannels returns channel list from GRP_TXT packets.
// Queries transmissions directly (not packets_v) to avoid observation-level
// Queries transmissions directly (not a VIEW) to avoid observation-level
// duplicates that could cause stale lastMessage when an older message has
// a later re-observation timestamp.
func (db *DB) GetChannels() ([]map[string]interface{}, error) {
@@ -1409,31 +1302,7 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int) ([]map[s
return messages, total, nil
}
// GetTimestamps returns packet timestamps since a given time.
func (db *DB) GetTimestamps(since string) ([]string, error) {
rows, err := db.conn.Query("SELECT timestamp FROM packets_v WHERE timestamp > ? ORDER BY timestamp ASC", since)
if err != nil {
return nil, err
}
defer rows.Close()
var timestamps []string
for rows.Next() {
var ts string
rows.Scan(&ts)
timestamps = append(timestamps, ts)
}
if timestamps == nil {
timestamps = []string{}
}
return timestamps, nil
}
// GetNodeCountsForPacket returns observation count for a hash.
func (db *DB) GetObservationCount(hash string) int {
var count int
db.conn.QueryRow("SELECT COUNT(*) FROM packets_v WHERE hash = ?", strings.ToLower(hash)).Scan(&count)
return count
}
// GetNewTransmissionsSince returns new transmissions after a given ID for WebSocket polling.
func (db *DB) GetNewTransmissionsSince(lastID int, limit int) ([]map[string]interface{}, error) {
@@ -1634,11 +1503,13 @@ func scanNodeRow(rows *sql.Rows) map[string]interface{} {
var name, role, lastSeen, firstSeen sql.NullString
var lat, lon sql.NullFloat64
var advertCount int
var batteryMv sql.NullInt64
var temperatureC sql.NullFloat64
if err := rows.Scan(&pk, &name, &role, &lat, &lon, &lastSeen, &firstSeen, &advertCount); err != nil {
if err := rows.Scan(&pk, &name, &role, &lat, &lon, &lastSeen, &firstSeen, &advertCount, &batteryMv, &temperatureC); err != nil {
return nil
}
return map[string]interface{}{
m := map[string]interface{}{
"public_key": pk,
"name": nullStr(name),
"role": nullStr(role),
@@ -1651,6 +1522,17 @@ func scanNodeRow(rows *sql.Rows) map[string]interface{} {
"hash_size": nil,
"hash_size_inconsistent": false,
}
if batteryMv.Valid {
m["battery_mv"] = int(batteryMv.Int64)
} else {
m["battery_mv"] = nil
}
if temperatureC.Valid {
m["temperature_c"] = temperatureC.Float64
} else {
m["temperature_c"] = nil
}
return m
}
func nullStr(ns sql.NullString) interface{} {
+133 -204
View File
@@ -28,7 +28,9 @@ func setupTestDB(t *testing.T) *DB {
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE TABLE observers (
@@ -44,7 +46,7 @@ func setupTestDB(t *testing.T) *DB {
radio TEXT,
battery_mv INTEGER,
uptime_secs INTEGER,
noise_floor INTEGER
noise_floor REAL
);
CREATE TABLE transmissions (
@@ -71,16 +73,6 @@ func setupTestDB(t *testing.T) *DB {
timestamp INTEGER NOT NULL
);
CREATE VIEW packets_v AS
SELECT o.id, t.raw_hex,
strftime('%Y-%m-%dT%H:%M:%fZ', o.timestamp, 'unixepoch') AS timestamp,
obs.id AS observer_id, obs.name AS observer_name,
o.direction, o.snr, o.rssi, o.score, t.hash, t.route_type,
t.payload_type, t.payload_version, o.path_json, t.decoded_json,
t.created_at
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx;
`
if _, err := conn.Exec(schema); err != nil {
t.Fatal(err)
@@ -369,6 +361,88 @@ func TestGetObserverByIDNotFound(t *testing.T) {
}
}
func TestObserverTypeConsistency(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Insert observer with typed metadata matching ingestor writes
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
VALUES ('obs_typed', 'TypedObs', 'SJC', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 10, 3500, 86400, -115.5)`)
obs, err := db.GetObserverByID("obs_typed")
if err != nil {
t.Fatal(err)
}
// battery_mv should be *int
if obs.BatteryMv == nil {
t.Fatal("BatteryMv should not be nil")
}
if *obs.BatteryMv != 3500 {
t.Errorf("BatteryMv=%d, want 3500", *obs.BatteryMv)
}
// uptime_secs should be *int64
if obs.UptimeSecs == nil {
t.Fatal("UptimeSecs should not be nil")
}
if *obs.UptimeSecs != 86400 {
t.Errorf("UptimeSecs=%d, want 86400", *obs.UptimeSecs)
}
// noise_floor should be *float64
if obs.NoiseFloor == nil {
t.Fatal("NoiseFloor should not be nil")
}
if *obs.NoiseFloor != -115.5 {
t.Errorf("NoiseFloor=%f, want -115.5", *obs.NoiseFloor)
}
// Verify NULL handling: observer without metadata
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES ('obs_null', 'NullObs', 'SFO', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 5)`)
obsNull, err := db.GetObserverByID("obs_null")
if err != nil {
t.Fatal(err)
}
if obsNull.BatteryMv != nil {
t.Errorf("BatteryMv should be nil for observer without metadata, got %d", *obsNull.BatteryMv)
}
if obsNull.UptimeSecs != nil {
t.Errorf("UptimeSecs should be nil for observer without metadata, got %d", *obsNull.UptimeSecs)
}
if obsNull.NoiseFloor != nil {
t.Errorf("NoiseFloor should be nil for observer without metadata, got %f", *obsNull.NoiseFloor)
}
}
func TestObserverTypesInGetObservers(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
VALUES ('obs1', 'Obs1', 'SJC', '2026-06-01T00:00:00Z', '2026-01-01T00:00:00Z', 10, 4200, 172800, -110.3)`)
observers, err := db.GetObservers()
if err != nil {
t.Fatal(err)
}
if len(observers) != 1 {
t.Fatalf("expected 1 observer, got %d", len(observers))
}
o := observers[0]
if o.BatteryMv == nil || *o.BatteryMv != 4200 {
t.Errorf("BatteryMv=%v, want 4200", o.BatteryMv)
}
if o.UptimeSecs == nil || *o.UptimeSecs != 172800 {
t.Errorf("UptimeSecs=%v, want 172800", o.UptimeSecs)
}
if o.NoiseFloor == nil || *o.NoiseFloor != -110.3 {
t.Errorf("NoiseFloor=%v, want -110.3", o.NoiseFloor)
}
}
func TestGetDistinctIATAs(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -485,51 +559,6 @@ func TestGetNewTransmissionsSince(t *testing.T) {
}
}
func TestGetObservationsForHash(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
obs, err := db.GetObservationsForHash("abc123def4567890")
if err != nil {
t.Fatal(err)
}
if len(obs) != 2 {
t.Errorf("expected 2 observations, got %d", len(obs))
}
}
func TestGetPacketByIDFound(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
pkt, err := db.GetPacketByID(1)
if err != nil {
t.Fatal(err)
}
if pkt == nil {
t.Fatal("expected packet, got nil")
}
if pkt["hash"] != "abc123def4567890" {
t.Errorf("expected hash abc123def4567890, got %v", pkt["hash"])
}
}
func TestGetPacketByIDNotFound(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
pkt, err := db.GetPacketByID(9999)
if err != nil {
t.Fatal(err)
}
if pkt != nil {
t.Error("expected nil for nonexistent packet ID")
}
}
func TestGetTransmissionByIDFound(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -572,34 +601,6 @@ func TestGetPacketByHashNotFound(t *testing.T) {
}
}
func TestGetRecentPacketsForNode(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
packets, err := db.GetRecentPacketsForNode("aabbccdd11223344", "TestRepeater", 20)
if err != nil {
t.Fatal(err)
}
if len(packets) == 0 {
t.Error("expected packets for TestRepeater")
}
}
func TestGetRecentPacketsForNodeDefaultLimit(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
packets, err := db.GetRecentPacketsForNode("aabbccdd11223344", "TestRepeater", 0)
if err != nil {
t.Fatal(err)
}
if packets == nil {
t.Error("expected non-nil result")
}
}
func TestGetObserverIdsForRegion(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -649,46 +650,6 @@ func TestGetObserverIdsForRegion(t *testing.T) {
})
}
func TestGetNodeHealth(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
t.Run("found", func(t *testing.T) {
result, err := db.GetNodeHealth("aabbccdd11223344")
if err != nil {
t.Fatal(err)
}
if result == nil {
t.Fatal("expected result, got nil")
}
node, ok := result["node"].(map[string]interface{})
if !ok {
t.Fatal("expected node object")
}
if node["name"] != "TestRepeater" {
t.Errorf("expected TestRepeater, got %v", node["name"])
}
stats, ok := result["stats"].(map[string]interface{})
if !ok {
t.Fatal("expected stats object")
}
if stats["totalPackets"] == nil {
t.Error("expected totalPackets in stats")
}
})
t.Run("not found", func(t *testing.T) {
result, err := db.GetNodeHealth("nonexistent")
if err != nil {
t.Fatal(err)
}
if result != nil {
t.Error("expected nil for nonexistent node")
}
})
}
func TestGetChannelMessages(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -731,48 +692,6 @@ func TestGetChannelMessages(t *testing.T) {
})
}
func TestGetTimestamps(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
t.Run("with results", func(t *testing.T) {
ts, err := db.GetTimestamps("2020-01-01")
if err != nil {
t.Fatal(err)
}
if len(ts) == 0 {
t.Error("expected timestamps")
}
})
t.Run("no results", func(t *testing.T) {
ts, err := db.GetTimestamps("2099-01-01")
if err != nil {
t.Fatal(err)
}
if len(ts) != 0 {
t.Errorf("expected 0 timestamps, got %d", len(ts))
}
})
}
func TestGetObservationCount(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
count := db.GetObservationCount("abc123def4567890")
if count != 2 {
t.Errorf("expected 2, got %d", count)
}
count = db.GetObservationCount("nonexistent")
if count != 0 {
t.Errorf("expected 0 for nonexistent, got %d", count)
}
}
func TestBuildPacketWhereFilters(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -1196,29 +1115,6 @@ func TestOpenDBInvalidPath(t *testing.T) {
}
}
func TestGetNodeHealthNoName(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Insert a node without a name
db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES ('obs1', 'Observer One', 'SJC')`)
db.conn.Exec(`INSERT INTO nodes (public_key, role, last_seen, first_seen, advert_count)
VALUES ('deadbeef12345678', 'repeater', '2026-01-15T10:00:00Z', '2026-01-01T00:00:00Z', 5)`)
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('DDEE', 'deadbeefhash1234', '2026-01-15T10:05:00Z', 1, 4,
'{"pubKey":"deadbeef12345678","type":"ADVERT"}')`)
db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 1, 11.0, -91, '["dd"]', 1736935500)`)
result, err := db.GetNodeHealth("deadbeef12345678")
if err != nil {
t.Fatal(err)
}
if result == nil {
t.Fatal("expected result, got nil")
}
}
func TestGetChannelMessagesObserverFallback(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
@@ -1299,20 +1195,6 @@ func TestQueryGroupedPacketsWithFilters(t *testing.T) {
}
}
func TestGetTracesEmpty(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
traces, err := db.GetTraces("nonexistenthash1")
if err != nil {
t.Fatal(err)
}
if len(traces) != 0 {
t.Errorf("expected 0 traces, got %d", len(traces))
}
}
func TestNullHelpers(t *testing.T) {
// nullStr
if nullStr(sql.NullString{Valid: false}) != nil {
@@ -1386,6 +1268,53 @@ func TestGetChannelsStaleMessage(t *testing.T) {
}
}
func TestNodeTelemetryFields(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
// Insert node with telemetry data
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c)
VALUES ('pk_telem1', 'SensorNode', 'sensor', 37.0, -122.0, '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 5, 3700, 28.5)`)
// Test via GetNodeByPubkey
node, err := db.GetNodeByPubkey("pk_telem1")
if err != nil {
t.Fatal(err)
}
if node == nil {
t.Fatal("expected node, got nil")
}
if node["battery_mv"] != 3700 {
t.Errorf("battery_mv=%v, want 3700", node["battery_mv"])
}
if node["temperature_c"] != 28.5 {
t.Errorf("temperature_c=%v, want 28.5", node["temperature_c"])
}
// Test via GetNodes
nodes, _, _, err := db.GetNodes(50, 0, "sensor", "", "", "", "", "")
if err != nil {
t.Fatal(err)
}
if len(nodes) != 1 {
t.Fatalf("expected 1 sensor node, got %d", len(nodes))
}
if nodes[0]["battery_mv"] != 3700 {
t.Errorf("GetNodes battery_mv=%v, want 3700", nodes[0]["battery_mv"])
}
// Test node without telemetry — fields should be nil
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, last_seen, first_seen, advert_count)
VALUES ('pk_notelem', 'PlainNode', 'repeater', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 3)`)
node2, _ := db.GetNodeByPubkey("pk_notelem")
if node2["battery_mv"] != nil {
t.Errorf("expected nil battery_mv for node without telemetry, got %v", node2["battery_mv"])
}
if node2["temperature_c"] != nil {
t.Errorf("expected nil temperature_c for node without telemetry, got %v", node2["temperature_c"])
}
}
func TestMain(m *testing.M) {
os.Exit(m.Run())
}
+1 -1
View File
@@ -1,4 +1,4 @@
module github.com/meshcore-analyzer/server
module github.com/corescope/server
go 1.22
+18 -3
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"log"
"net/http"
_ "net/http/pprof"
"os"
"os/exec"
"os/signal"
@@ -54,6 +55,20 @@ func resolveBuildTime() string {
}
func main() {
// pprof profiling — off by default, enable with ENABLE_PPROF=true
if os.Getenv("ENABLE_PPROF") == "true" {
pprofPort := os.Getenv("PPROF_PORT")
if pprofPort == "" {
pprofPort = "6060"
}
go func() {
log.Printf("[pprof] profiling UI at http://localhost:%s/debug/pprof/", pprofPort)
if err := http.ListenAndServe(":"+pprofPort, nil); err != nil {
log.Printf("[pprof] failed to start: %v (non-fatal)", err)
}
}()
}
var (
configDir string
port int
@@ -101,7 +116,7 @@ func main() {
var tableName string
err = database.conn.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name='transmissions'").Scan(&tableName)
if err == sql.ErrNoRows {
log.Fatalf("[db] table 'transmissions' not found — is this a MeshCore Analyzer database?")
log.Fatalf("[db] table 'transmissions' not found — is this a CoreScope database?")
}
stats, err := database.GetStats()
@@ -140,7 +155,7 @@ func main() {
log.Printf("[static] directory %s not found — API-only mode", absPublic)
router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(`<!DOCTYPE html><html><body><h1>MeshCore Analyzer</h1><p>Frontend not found. API available at /api/</p></body></html>`))
w.Write([]byte(`<!DOCTYPE html><html><body><h1>CoreScope</h1><p>Frontend not found. API available at /api/</p></body></html>`))
})
}
@@ -167,7 +182,7 @@ func main() {
httpServer.Close()
}()
log.Printf("[server] MeshCore Analyzer (Go) listening on http://localhost:%d", cfg.Port)
log.Printf("[server] CoreScope (Go) listening on http://localhost:%d", cfg.Port)
if err := httpServer.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("[server] %v", err)
}
+299 -799
View File
File diff suppressed because it is too large Load Diff
+38 -11
View File
@@ -18,7 +18,9 @@ func setupTestServer(t *testing.T) (*Server, *mux.Router) {
hub := NewHub()
srv := NewServer(db, cfg, hub)
store := NewPacketStore(db)
store.Load()
if err := store.Load(); err != nil {
t.Fatalf("store.Load failed: %v", err)
}
srv.store = store
router := mux.NewRouter()
srv.RegisterRoutes(router)
@@ -722,6 +724,9 @@ func TestNodePathsFound(t *testing.T) {
if body["paths"] == nil {
t.Error("expected paths in response")
}
if got, ok := body["totalTransmissions"].(float64); !ok || got < 1 {
t.Errorf("expected totalTransmissions >= 1, got %v", body["totalTransmissions"])
}
}
func TestNodePathsNotFound(t *testing.T) {
@@ -832,6 +837,9 @@ func TestObserverAnalytics(t *testing.T) {
if body["recentPackets"] == nil {
t.Error("expected recentPackets")
}
if recent, ok := body["recentPackets"].([]interface{}); !ok || len(recent) == 0 {
t.Errorf("expected non-empty recentPackets, got %v", body["recentPackets"])
}
})
t.Run("custom days", func(t *testing.T) {
@@ -1251,6 +1259,11 @@ func TestNodeAnalyticsNoNameNode(t *testing.T) {
cfg := &Config{Port: 3000}
hub := NewHub()
srv := NewServer(db, cfg, hub)
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store.Load failed: %v", err)
}
srv.store = store
router := mux.NewRouter()
srv.RegisterRoutes(router)
@@ -1282,6 +1295,11 @@ func TestNodeHealthForNoNameNode(t *testing.T) {
cfg := &Config{Port: 3000}
hub := NewHub()
srv := NewServer(db, cfg, hub)
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store.Load failed: %v", err)
}
srv.store = store
router := mux.NewRouter()
srv.RegisterRoutes(router)
@@ -1521,8 +1539,6 @@ func TestHandlerErrorPaths(t *testing.T) {
router := mux.NewRouter()
srv.RegisterRoutes(router)
// Drop the view to force query errors
db.conn.Exec("DROP VIEW IF EXISTS packets_v")
t.Run("stats error", func(t *testing.T) {
db.conn.Exec("DROP TABLE IF EXISTS transmissions")
@@ -1563,7 +1579,7 @@ func TestHandlerErrorTraces(t *testing.T) {
router := mux.NewRouter()
srv.RegisterRoutes(router)
db.conn.Exec("DROP VIEW IF EXISTS packets_v")
db.conn.Exec("DROP TABLE IF EXISTS observations")
req := httptest.NewRequest("GET", "/api/traces/abc123def4567890", nil)
w := httptest.NewRecorder()
@@ -1697,13 +1713,12 @@ func TestHandlerErrorTimestamps(t *testing.T) {
router := mux.NewRouter()
srv.RegisterRoutes(router)
db.conn.Exec("DROP VIEW IF EXISTS packets_v")
// Without a store, timestamps returns empty 200
req := httptest.NewRequest("GET", "/api/packets/timestamps?since=2020-01-01", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 500 {
t.Errorf("expected 500 for timestamps error, got %d", w.Code)
if w.Code != 200 {
t.Errorf("expected 200 for timestamps without store, got %d", w.Code)
}
}
@@ -1740,8 +1755,8 @@ func TestHandlerErrorBulkHealth(t *testing.T) {
req := httptest.NewRequest("GET", "/api/nodes/bulk-health", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 500 {
t.Errorf("expected 500, got %d", w.Code)
if w.Code != 200 {
t.Errorf("expected 200, got %d", w.Code)
}
}
@@ -1876,7 +1891,9 @@ func TestGetNodeHashSizeInfoFlipFlop(t *testing.T) {
db := setupTestDB(t)
seedTestData(t, db)
store := NewPacketStore(db)
store.Load()
if err := store.Load(); err != nil {
t.Fatalf("store.Load failed: %v", err)
}
pk := "abcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890"
db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, role) VALUES (?, 'TestNode', 'repeater')", pk)
@@ -1934,7 +1951,17 @@ for _, field := range arrayFields {
if body[field] == nil {
t.Errorf("field %q is null, expected []", field)
}
}
}
func TestObserverAnalyticsNoStore(t *testing.T) {
_, router := setupNoStoreServer(t)
req := httptest.NewRequest("GET", "/api/observers/obs1/analytics", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 503 {
t.Fatalf("expected 503, got %d", w.Code)
}
}
func min(a, b int) int {
if a < b {
+79 -49
View File
@@ -62,7 +62,7 @@ type StoreObs struct {
type PacketStore struct {
mu sync.RWMutex
db *DB
packets []*StoreTx // sorted by first_seen DESC
packets []*StoreTx // sorted by first_seen ASC (oldest first; newest at tail)
byHash map[string]*StoreTx // hash → *StoreTx
byTxID map[int]*StoreTx // transmission_id → *StoreTx
byObsID map[int]*StoreObs // observation_id → *StoreObs
@@ -98,6 +98,11 @@ type PacketStore struct {
// computed during Load() and incrementally updated on ingest.
distHops []distHopRecord
distPaths []distPathRecord
// Cached GetNodeHashSizeInfo result — recomputed at most once every 15s
hashSizeInfoMu sync.Mutex
hashSizeInfoCache map[string]*hashSizeNodeInfo
hashSizeInfoAt time.Time
}
// Precomputed distance records for fast analytics aggregation.
@@ -176,7 +181,7 @@ func (s *PacketStore) Load() error {
FROM transmissions t
LEFT JOIN observations o ON o.transmission_id = t.id
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
ORDER BY t.first_seen DESC, o.timestamp DESC`
ORDER BY t.first_seen ASC, o.timestamp DESC`
} else {
loadSQL = `SELECT t.id, t.raw_hex, t.hash, t.first_seen, t.route_type,
t.payload_type, t.payload_version, t.decoded_json,
@@ -184,7 +189,7 @@ func (s *PacketStore) Load() error {
o.snr, o.rssi, o.score, o.path_json, o.timestamp
FROM transmissions t
LEFT JOIN observations o ON o.transmission_id = t.id
ORDER BY t.first_seen DESC, o.timestamp DESC`
ORDER BY t.first_seen ASC, o.timestamp DESC`
}
rows, err := s.db.conn.Query(loadSQL)
@@ -368,28 +373,32 @@ func (s *PacketStore) QueryPackets(q PacketQuery) *PacketResult {
results := s.filterPackets(q)
total := len(results)
if q.Order == "ASC" {
sorted := make([]*StoreTx, len(results))
copy(sorted, results)
sort.Slice(sorted, func(i, j int) bool {
return sorted[i].FirstSeen < sorted[j].FirstSeen
})
results = sorted
}
// Paginate
// results is oldest-first (ASC). For DESC (default) read backwards from the tail;
// for ASC read forwards. Both are O(page_size) — no sort copy needed.
start := q.Offset
if start >= len(results) {
if start >= total {
return &PacketResult{Packets: []map[string]interface{}{}, Total: total}
}
end := start + q.Limit
if end > len(results) {
end = len(results)
pageSize := q.Limit
if start+pageSize > total {
pageSize = total - start
}
packets := make([]map[string]interface{}, 0, end-start)
for _, tx := range results[start:end] {
packets = append(packets, txToMap(tx))
packets := make([]map[string]interface{}, 0, pageSize)
if q.Order == "ASC" {
for _, tx := range results[start : start+pageSize] {
packets = append(packets, txToMap(tx))
}
} else {
// DESC: newest items are at the tail; page 0 = last pageSize items reversed
endIdx := total - start
startIdx := endIdx - pageSize
if startIdx < 0 {
startIdx = 0
}
for i := endIdx - 1; i >= startIdx; i-- {
packets = append(packets, txToMap(results[i]))
}
}
return &PacketResult{Packets: packets, Total: total}
}
@@ -719,15 +728,16 @@ func (s *PacketStore) GetTimestamps(since string) []string {
s.mu.RLock()
defer s.mu.RUnlock()
// packets sorted newest first — scan from start until older than since
// packets sorted oldest-first — scan from tail until we reach items older than since
var result []string
for _, tx := range s.packets {
for i := len(s.packets) - 1; i >= 0; i-- {
tx := s.packets[i]
if tx.FirstSeen <= since {
break
}
result = append(result, tx.FirstSeen)
}
// Reverse to get ASC order
// result is currently newest-first; reverse to return ASC order
for i, j := 0, len(result)-1; i < j; i, j = i+1, j-1 {
result[i], result[j] = result[j], result[i]
}
@@ -777,23 +787,30 @@ func (s *PacketStore) QueryMultiNodePackets(pubkeys []string, limit, offset int,
total := len(filtered)
if order == "ASC" {
sort.Slice(filtered, func(i, j int) bool {
return filtered[i].FirstSeen < filtered[j].FirstSeen
})
}
// filtered is oldest-first (built by iterating s.packets forward).
// Apply same DESC/ASC pagination logic as QueryPackets.
if offset >= total {
return &PacketResult{Packets: []map[string]interface{}{}, Total: total}
}
end := offset + limit
if end > total {
end = total
pageSize := limit
if offset+pageSize > total {
pageSize = total - offset
}
packets := make([]map[string]interface{}, 0, end-offset)
for _, tx := range filtered[offset:end] {
packets = append(packets, txToMap(tx))
packets := make([]map[string]interface{}, 0, pageSize)
if order == "ASC" {
for _, tx := range filtered[offset : offset+pageSize] {
packets = append(packets, txToMap(tx))
}
} else {
endIdx := total - offset
startIdx := endIdx - pageSize
if startIdx < 0 {
startIdx = 0
}
for i := endIdx - 1; i >= startIdx; i-- {
packets = append(packets, txToMap(filtered[i]))
}
}
return &PacketResult{Packets: packets, Total: total}
}
@@ -926,15 +943,14 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
DecodedJSON: r.decodedJSON,
}
s.byHash[r.hash] = tx
// Prepend (newest first)
s.packets = append([]*StoreTx{tx}, s.packets...)
s.packets = append(s.packets, tx) // oldest-first; new items go to tail
s.byTxID[r.txID] = tx
s.indexByNode(tx)
if tx.PayloadType != nil {
pt := *tx.PayloadType
// Prepend to maintain newest-first order (matches Load ordering)
// Append to maintain oldest-first order (matches Load ordering)
// so GetChannelMessages reverse iteration stays correct
s.byPayloadType[pt] = append([]*StoreTx{tx}, s.byPayloadType[pt]...)
s.byPayloadType[pt] = append(s.byPayloadType[pt], tx)
}
if _, exists := broadcastTxs[r.txID]; !exists {
@@ -1079,8 +1095,6 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
s.cacheMu.Unlock()
}
log.Printf("[poller] IngestNewFromDB: found %d new txs, maxID %d->%d", len(result), sinceID, newMaxID)
return result, newMaxID
}
@@ -1263,8 +1277,7 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
s.subpathCache = make(map[string]*cachedResult)
s.cacheMu.Unlock()
log.Printf("[poller] IngestNewObservations: updated %d existing txs, maxObsID %d->%d",
len(updatedTxs), sinceObsID, newMaxObsID)
// analytics caches cleared; no per-cycle log to avoid stdout overhead
}
return newMaxObsID
@@ -1888,7 +1901,7 @@ func (s *PacketStore) GetChannelMessages(channelHash string, limit, offset int)
msgMap := map[string]*msgEntry{}
var msgOrder []string
// Iterate type-5 packets oldest-first (byPayloadType is in load order = newest first)
// Iterate type-5 packets oldest-first (byPayloadType is ASC = oldest first)
type decodedMsg struct {
Type string `json:"type"`
Channel string `json:"channel"`
@@ -1899,8 +1912,7 @@ func (s *PacketStore) GetChannelMessages(channelHash string, limit, offset int)
}
grpTxts := s.byPayloadType[5]
for i := len(grpTxts) - 1; i >= 0; i-- {
tx := grpTxts[i]
for _, tx := range grpTxts {
if tx.DecodedJSON == "" {
continue
}
@@ -3715,8 +3727,26 @@ type hashSizeNodeInfo struct {
Inconsistent bool
}
// GetNodeHashSizeInfo scans advert packets to compute per-node hash size data.
// GetNodeHashSizeInfo returns cached per-node hash size data, recomputing at most every 15s.
func (s *PacketStore) GetNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
const ttl = 15 * time.Second
s.hashSizeInfoMu.Lock()
if s.hashSizeInfoCache != nil && time.Since(s.hashSizeInfoAt) < ttl {
cached := s.hashSizeInfoCache
s.hashSizeInfoMu.Unlock()
return cached
}
s.hashSizeInfoMu.Unlock()
result := s.computeNodeHashSizeInfo()
s.hashSizeInfoMu.Lock()
s.hashSizeInfoCache = result
s.hashSizeInfoAt = time.Now()
s.hashSizeInfoMu.Unlock()
return result
}
// computeNodeHashSizeInfo scans advert packets to compute per-node hash size data.
func (s *PacketStore) computeNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
s.mu.RLock()
defer s.mu.RUnlock()
@@ -4069,13 +4099,13 @@ func (s *PacketStore) GetNodeHealth(pubkey string) (map[string]interface{}, erro
lhVal = lastHeard
}
// Recent packets (up to 20, newest first — packets are already sorted DESC)
// Recent packets (up to 20, newest first — read from tail of oldest-first slice)
recentLimit := 20
if len(packets) < recentLimit {
recentLimit = len(packets)
}
recentPackets := make([]map[string]interface{}, 0, recentLimit)
for i := 0; i < recentLimit; i++ {
for i := len(packets) - 1; i >= len(packets)-recentLimit; i-- {
p := txToMap(packets[i])
delete(p, "observations")
recentPackets = append(recentPackets, p)
+12
View File
@@ -996,6 +996,12 @@
"elementShape": {
"type": "number"
}
},
"battery_mv": {
"type": "nullable_number"
},
"temperature_c": {
"type": "nullable_number"
}
}
},
@@ -1097,6 +1103,12 @@
},
"last_heard": {
"type": "string"
},
"battery_mv": {
"type": "nullable_number"
},
"temperature_c": {
"type": "nullable_number"
}
}
}
+2 -2
View File
@@ -10,7 +10,7 @@
"key": "/path/to/key.pem"
},
"branding": {
"siteName": "MeshCore Analyzer",
"siteName": "CoreScope",
"tagline": "Real-time MeshCore LoRa mesh network analyzer",
"logoUrl": null,
"faviconUrl": null
@@ -32,7 +32,7 @@
"observer": "#8b5cf6"
},
"home": {
"heroTitle": "MeshCore Analyzer",
"heroTitle": "CoreScope",
"heroSubtitle": "Find your nodes to start monitoring them.",
"steps": [
{ "emoji": "📡", "title": "Connect", "description": "Link your node to the mesh" },
+37 -3
View File
@@ -33,7 +33,9 @@ db.exec(`
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE TABLE IF NOT EXISTS observers (
@@ -60,7 +62,9 @@ db.exec(`
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
@@ -324,6 +328,22 @@ for (const col of ['model', 'firmware', 'client_version', 'radio', 'battery_mv',
}
}
// --- One-time migration: add telemetry columns to nodes and inactive_nodes ---
{
const done = db.prepare(`SELECT 1 FROM _migrations WHERE name = 'node_telemetry_v1'`).get();
if (!done) {
console.log('[migration] Adding telemetry columns to nodes/inactive_nodes...');
const nodeCols = db.pragma('table_info(nodes)').map(c => c.name);
if (!nodeCols.includes('battery_mv')) db.exec(`ALTER TABLE nodes ADD COLUMN battery_mv INTEGER`);
if (!nodeCols.includes('temperature_c')) db.exec(`ALTER TABLE nodes ADD COLUMN temperature_c REAL`);
const inactiveCols = db.pragma('table_info(inactive_nodes)').map(c => c.name);
if (!inactiveCols.includes('battery_mv')) db.exec(`ALTER TABLE inactive_nodes ADD COLUMN battery_mv INTEGER`);
if (!inactiveCols.includes('temperature_c')) db.exec(`ALTER TABLE inactive_nodes ADD COLUMN temperature_c REAL`);
db.prepare(`INSERT INTO _migrations (name) VALUES ('node_telemetry_v1')`).run();
console.log('[migration] node telemetry columns added');
}
}
// --- Prepared statements ---
const stmts = {
upsertNode: db.prepare(`
@@ -339,6 +359,12 @@ const stmts = {
incrementAdvertCount: db.prepare(`
UPDATE nodes SET advert_count = advert_count + 1 WHERE public_key = @public_key
`),
updateNodeTelemetry: db.prepare(`
UPDATE nodes SET
battery_mv = COALESCE(@battery_mv, battery_mv),
temperature_c = COALESCE(@temperature_c, temperature_c)
WHERE public_key = @public_key
`),
upsertObserver: db.prepare(`
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor)
VALUES (@id, @name, @iata, @last_seen, @first_seen, 1, @model, @firmware, @client_version, @radio, @battery_mv, @uptime_secs, @noise_floor)
@@ -511,6 +537,14 @@ function incrementAdvertCount(publicKey) {
stmts.incrementAdvertCount.run({ public_key: publicKey });
}
function updateNodeTelemetry(data) {
stmts.updateNodeTelemetry.run({
public_key: data.public_key,
battery_mv: data.battery_mv ?? null,
temperature_c: data.temperature_c ?? null,
});
}
function upsertNode(data) {
const now = new Date().toISOString();
stmts.upsertNode.run({
@@ -898,4 +932,4 @@ function moveStaleNodes(nodeDays) {
return moved;
}
module.exports = { db, schemaVersion, observerIdToRowid, resolveObserverIdx, insertTransmission, upsertNode, incrementAdvertCount, upsertObserver, updateObserverStatus, getPackets, getPacket, getTransmission, getNodes, getNode, getObservers, getStats, searchNodes, getNodeHealth, getNodeAnalytics, removePhantomNodes, moveStaleNodes };
module.exports = { db, schemaVersion, observerIdToRowid, resolveObserverIdx, insertTransmission, upsertNode, incrementAdvertCount, updateNodeTelemetry, upsertObserver, updateObserverStatus, getPackets, getPacket, getTransmission, getNodes, getNode, getObservers, getStats, searchNodes, getNodeHealth, getNodeAnalytics, removePhantomNodes, moveStaleNodes };
+24 -2
View File
@@ -135,10 +135,32 @@ function decodeAdvert(buf) {
off += 8;
}
if (result.flags.hasName) {
let name = appdata.subarray(off).toString('utf8');
// Strip non-printable characters (< 0x20 except tab/newline) and DEL
// Find null terminator to separate name from trailing telemetry bytes
let nameEnd = appdata.length;
for (let i = off; i < appdata.length; i++) {
if (appdata[i] === 0x00) { nameEnd = i; break; }
}
let name = appdata.subarray(off, nameEnd).toString('utf8');
name = name.replace(/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/g, '');
result.name = name;
off = nameEnd;
// Skip null terminator(s)
while (off < appdata.length && appdata[off] === 0x00) off++;
}
// Telemetry bytes after name: battery_mv(2 LE) + temperature_c(2 LE, signed, /100)
// Only sensor nodes (advType=4) carry telemetry bytes.
if (result.flags.sensor && off + 4 <= appdata.length) {
const batteryMv = appdata.readUInt16LE(off);
const tempRaw = appdata.readInt16LE(off + 2);
const tempC = tempRaw / 100.0;
if (batteryMv > 0 && batteryMv <= 10000) {
result.battery_mv = batteryMv;
}
// Raw int16 / 100 → °C; accept -50°C to 100°C (raw: -5000 to 10000)
if (tempRaw >= -5000 && tempRaw <= 10000) {
result.temperature_c = tempC;
}
}
}
+21 -7
View File
@@ -1,11 +1,16 @@
# Volume paths unified with manage.sh — see manage.sh lines 9-12, 56-68, 98-113
# All container config lives here. manage.sh is just a wrapper around docker compose.
# Override defaults via .env or environment variables.
# CRITICAL: All data mounts use bind mounts (~/path), NOT named volumes.
# This ensures the DB and theme are visible on the host filesystem for backup.
services:
prod:
image: meshcore-analyzer:latest
container_name: meshcore-prod
build: .
image: corescope:latest
container_name: corescope-prod
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${PROD_HTTP_PORT:-80}:${PROD_HTTP_PORT:-80}"
- "${PROD_HTTPS_PORT:-443}:${PROD_HTTPS_PORT:-443}"
@@ -24,9 +29,12 @@ services:
retries: 3
staging:
image: meshcore-analyzer:latest
container_name: meshcore-staging
build: .
image: corescope:latest
container_name: corescope-staging
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${STAGING_HTTP_PORT:-81}:${STAGING_HTTP_PORT:-81}"
- "${STAGING_MQTT_PORT:-1884}:1883"
@@ -52,18 +60,23 @@ services:
args:
APP_VERSION: ${APP_VERSION:-unknown}
GIT_COMMIT: ${GIT_COMMIT:-unknown}
image: meshcore-go:latest
container_name: meshcore-staging-go
image: corescope-go:latest
container_name: corescope-staging-go
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${STAGING_GO_HTTP_PORT:-82}:80"
- "${STAGING_GO_MQTT_PORT:-1885}:1883"
- "6060:6060" # pprof server
- "6061:6061" # pprof ingestor
volumes:
- ${STAGING_DATA_DIR:-~/meshcore-staging-data}/config.json:/app/config.json:ro
- ${STAGING_DATA_DIR:-~/meshcore-staging-data}:/app/data
- caddy-data-staging-go:/data/caddy
environment:
- NODE_ENV=staging
- ENABLE_PPROF=true
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
interval: 30s
@@ -73,6 +86,7 @@ services:
- staging-go
volumes:
# Named volumes for Caddy TLS certificates (not user data — managed by Caddy internally)
caddy-data:
caddy-data-staging:
caddy-data-staging-go:
+4 -4
View File
@@ -14,8 +14,8 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:meshcore-ingestor]
command=/app/meshcore-ingestor -config /app/config.json
[program:corescope-ingestor]
command=/app/corescope-ingestor -config /app/config.json
directory=/app
autostart=true
autorestart=true
@@ -24,8 +24,8 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:meshcore-server]
command=/app/meshcore-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
[program:corescope-server]
command=/app/corescope-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
directory=/app
autostart=true
autorestart=true
+1 -1
View File
@@ -14,7 +14,7 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:meshcore-analyzer]
[program:corescope]
command=node /app/server.js
directory=/app
autostart=true
+1 -1
View File
@@ -27,7 +27,7 @@ No restart needed. The server picks up changes to `theme.json` on every page loa
**Bare metal / PM2 / systemd:**
```bash
# Same directory as server.js and config.json
cp theme.json /path/to/meshcore-analyzer/
cp theme.json /path/to/corescope/
```
Check the server logs on startup — it tells you where it's looking:
+22 -22
View File
@@ -1,6 +1,6 @@
# Deploying MeshCore Analyzer
# Deploying CoreScope
Get MeshCore Analyzer running with automatic HTTPS on your own server.
Get CoreScope running with automatic HTTPS on your own server.
## Table of Contents
@@ -19,7 +19,7 @@ Get MeshCore Analyzer running with automatic HTTPS on your own server.
## What You'll End Up With
- MeshCore Analyzer running at `https://your-domain.com`
- CoreScope running at `https://your-domain.com`
- Automatic HTTPS certificates (via Let's Encrypt + Caddy)
- Built-in MQTT broker for receiving packets from observers
- SQLite database for packet storage (auto-created)
@@ -83,8 +83,8 @@ docker --version
The easiest way — use the management script:
```bash
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd meshcore-analyzer
git clone https://github.com/Kpa-clawbot/corescope.git
cd corescope
./manage.sh setup
```
@@ -111,8 +111,8 @@ flowchart LR
### 1. Download the code
```bash
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd meshcore-analyzer
git clone https://github.com/Kpa-clawbot/corescope.git
cd corescope
```
### 2. Create your config
@@ -153,10 +153,10 @@ Save and close. Caddy handles certificates, renewals, and HTTP→HTTPS redirects
### 4. Build and run
```bash
docker build -t meshcore-analyzer .
docker build -t corescope .
docker run -d \
--name meshcore-analyzer \
--name corescope \
--restart unless-stopped \
-p 80:80 \
-p 443:443 \
@@ -164,7 +164,7 @@ docker run -d \
-v $(pwd)/caddy-config/Caddyfile:/etc/caddy/Caddyfile:ro \
-v meshcore-data:/app/data \
-v caddy-data:/data/caddy \
meshcore-analyzer
corescope
```
What each flag does:
@@ -184,12 +184,12 @@ Open `https://your-domain.com`. You should see the analyzer home page.
Check the logs:
```bash
docker logs meshcore-analyzer
docker logs corescope
```
Expected output:
```
MeshCore Analyzer running on http://localhost:3000
CoreScope running on http://localhost:3000
MQTT [local] connected to mqtt://localhost:1883
[pre-warm] 12 endpoints in XXXms
```
@@ -215,7 +215,7 @@ Add a remote broker to `mqttSources` in your `config.json`:
}
```
Restart: `docker restart meshcore-analyzer`
Restart: `docker restart corescope`
### Option B: Run your own observer
@@ -271,12 +271,12 @@ If you already run a reverse proxy, skip Caddy entirely and proxy directly to th
```bash
docker run -d \
--name meshcore-analyzer \
--name corescope \
--restart unless-stopped \
-p 3000:3000 \
-v $(pwd)/config.json:/app/config.json:ro \
-v meshcore-data:/app/data \
meshcore-analyzer
corescope
```
Then configure your existing proxy to forward traffic to `localhost:3000`.
@@ -287,12 +287,12 @@ For local testing or a LAN-only setup, use the default Caddyfile that ships in t
```bash
docker run -d \
--name meshcore-analyzer \
--name corescope \
--restart unless-stopped \
-p 80:80 \
-v $(pwd)/config.json:/app/config.json:ro \
-v meshcore-data:/app/data \
meshcore-analyzer
corescope
```
## MQTT Security
@@ -315,7 +315,7 @@ password_file /etc/mosquitto/passwd
```
After starting the container, create users:
```bash
docker exec -it meshcore-analyzer mosquitto_passwd -c /etc/mosquitto/passwd myuser
docker exec -it corescope mosquitto_passwd -c /etc/mosquitto/passwd myuser
```
**Option 3: Use TLS** — For production, configure Mosquitto with TLS certificates. See the [Mosquitto docs](https://mosquitto.org/man/mosquitto-conf-5.html).
@@ -331,7 +331,7 @@ Packet data is stored in `meshcore.db` inside the data volume.
**Using manage.sh (easiest):**
```bash
./manage.sh backup # Saves to ./backups/meshcore-TIMESTAMP.db
./manage.sh backup # Saves to ./backups/corescope-TIMESTAMP/
./manage.sh backup ~/my-backup.db # Custom path
./manage.sh restore ./backups/some-file.db # Restore (backs up current DB first)
```
@@ -345,7 +345,7 @@ If you used `-v ./analyzer-data:/app/data` instead of a Docker volume, the datab
```bash
crontab -e
# Add:
0 3 * * * cd /path/to/meshcore-analyzer && ./manage.sh backup
0 3 * * * cd /path/to/corescope && ./manage.sh backup
```
## Updating
@@ -398,11 +398,11 @@ Center the map on your area in `config.json`:
| Problem | Likely cause | Fix |
|---------|-------------|-----|
| Site shows "connection refused" | Container not running | `docker ps` to check, `docker logs meshcore-analyzer` for errors |
| Site shows "connection refused" | Container not running | `docker ps` to check, `docker logs corescope` for errors |
| HTTPS not working | Port 80 blocked | Open port 80 — Caddy needs it for ACME challenges |
| "too many certificates" error | Let's Encrypt rate limit (5/domain/week) | Use a different subdomain, bring your own cert, or wait a week |
| Certificate won't provision | DNS not pointed at server | `dig your-domain` must show your server IP before starting |
| No packets appearing | No observer connected | `docker exec meshcore-analyzer mosquitto_sub -t 'meshcore/#' -C 1 -W 10` — if silent, no data is coming in |
| No packets appearing | No observer connected | `docker exec corescope mosquitto_sub -t 'meshcore/#' -C 1 -W 10` — if silent, no data is coming in |
| Container crashes on startup | Bad JSON in config | `python3 -c "import json; json.load(open('config.json'))"` to validate |
| "address already in use" | Another web server on 80/443 | Stop it: `sudo systemctl stop nginx apache2` |
| Slow on Raspberry Pi | First build is slow | Normal — subsequent builds use cache. Runtime performance is fine. |
+1 -1
View File
@@ -1,4 +1,4 @@
# Hash Prefix Disambiguation in MeshCore Analyzer
# Hash Prefix Disambiguation in CoreScope
## Section 1: Executive Summary
+2 -2
View File
@@ -1,4 +1,4 @@
# MeshCore Analyzer — API Contract Specification
# CoreScope — API Contract Specification
> **Authoritative contract.** Both the Node.js and Go backends MUST conform to this spec.
> The frontend relies on these exact shapes. Breaking changes require a spec update first.
@@ -1547,7 +1547,7 @@ Theme and branding configuration (merged from config.json + theme.json).
```jsonc
{
"branding": {
"siteName": string, // default: "MeshCore Analyzer"
"siteName": string, // default: "CoreScope"
"tagline": string // default: "Real-time MeshCore LoRa mesh network analyzer"
// ... additional branding keys from config/theme files
},
+17 -17
View File
@@ -1,6 +1,6 @@
# Migrating from Node.js to Go Engine
Guide for existing MeshCore Analyzer users switching from the Node.js Docker image to the Go version.
Guide for existing CoreScope users switching from the Node.js Docker image to the Go version.
> **Status (July 2025):** The Go engine is fully functional for production use.
> Go images are **not yet published to Docker Hub** — you build locally from source.
@@ -24,11 +24,11 @@ Guide for existing MeshCore Analyzer users switching from the Node.js Docker ima
## Prerequisites
- **Docker** 20.10+ and **Docker Compose** v2 (verify: `docker compose version`)
- An existing MeshCore Analyzer deployment running the Node.js image
- An existing CoreScope deployment running the Node.js image
- The repository cloned locally (needed to build the Go image):
```bash
git clone https://github.com/meshcore-dev/meshcore-analyzer.git
cd meshcore-analyzer
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
cd corescope
git pull # get latest
```
- Your `config.json` and `caddy-config/Caddyfile` in place (the same ones you use now)
@@ -122,7 +122,7 @@ docker compose --profile staging-go build staging-go
Or build directly:
```bash
docker build -f Dockerfile.go -t meshcore-go:latest \
docker build -f Dockerfile.go -t corescope-go:latest \
--build-arg APP_VERSION=$(git describe --tags 2>/dev/null || echo unknown) \
--build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo unknown) \
.
@@ -151,7 +151,7 @@ Once satisfied, update `docker-compose.yml` to use the Go image for prod:
```yaml
services:
prod:
image: meshcore-go:latest # was: meshcore-analyzer:latest
image: corescope-go:latest # was: corescope:latest
build:
context: .
dockerfile: Dockerfile.go # add this
@@ -174,9 +174,9 @@ docker compose up -d prod
./manage.sh stop
# Build the Go image
docker build -f Dockerfile.go -t meshcore-analyzer:latest .
docker build -f Dockerfile.go -t corescope:latest .
# Start (manage.sh uses the meshcore-analyzer:latest image)
# Start (manage.sh uses the corescope:latest image)
./manage.sh start
```
@@ -248,7 +248,7 @@ These should match (or be close to) your pre-migration numbers.
```bash
# Watch container logs for MQTT messages
docker logs -f meshcore-prod --tail 20
docker logs -f corescope-prod --tail 20
# Or use manage.sh
./manage.sh mqtt-test
@@ -279,13 +279,13 @@ If something goes wrong, switching back is straightforward:
```yaml
services:
prod:
image: meshcore-analyzer:latest # back to Node.js
image: corescope:latest # back to Node.js
# Remove the build.dockerfile line if you added it
```
```bash
# Rebuild Node.js image if needed
docker build -t meshcore-analyzer:latest .
docker build -t corescope:latest .
docker compose up -d --force-recreate prod
```
@@ -295,8 +295,8 @@ docker compose up -d --force-recreate prod
```bash
./manage.sh stop
# Rebuild Node.js image (overwrites the meshcore-analyzer:latest tag)
docker build -t meshcore-analyzer:latest .
# Rebuild Node.js image (overwrites the corescope:latest tag)
docker build -t corescope:latest .
./manage.sh start
```
@@ -310,9 +310,9 @@ docker build -t meshcore-analyzer:latest .
Or manually:
```bash
docker stop meshcore-prod
docker stop corescope-prod
cp backups/pre-go-migration/meshcore.db ~/meshcore-data/meshcore.db
docker start meshcore-prod
docker start corescope-prod
```
---
@@ -348,7 +348,7 @@ docker start meshcore-prod
|------|---------|-----|
| `engine` field in `/api/health` | Not present or `"node"` | Always `"go"` |
| MQTT URL scheme | Uses `mqtt://` / `mqtts://` natively | Auto-converts to `tcp://` / `ssl://` (transparent) |
| Process model | Single Node.js process (server + ingestor) | Two binaries: `meshcore-ingestor` + `meshcore-server` (managed by supervisord) |
| Process model | Single Node.js process (server + ingestor) | Two binaries: `corescope-ingestor` + `corescope-server` (managed by supervisord) |
| Memory management | Configurable via `packetStore.maxMemoryMB` | Loads all packets; no configurable limit |
| Startup time | Faster (no compilation) | Slightly slower (loads all packets from DB into memory) |
@@ -393,4 +393,4 @@ The following gaps have been identified. Check the GitHub issue tracker for curr
3. **Go ingestor missing `meshcore/self_info` handling** — The local node identity topic is not processed. Low impact but breaks parity.
4. **No Docker Hub publishing for Go images** — Users must build locally. CI/CD pipeline should publish `meshcore-go:latest` alongside the Node.js image.
4. **No Docker Hub publishing for Go images** — Users must build locally. CI/CD pipeline should publish `corescope-go:latest` alongside the Node.js image.
+101
View File
@@ -0,0 +1,101 @@
# CoreScope Migration Guide
MeshCore Analyzer has been renamed to **CoreScope**. This document covers what you need to update.
## What Changed
- **Repository name**: `meshcore-analyzer``corescope`
- **Docker image name**: `meshcore-analyzer:latest``corescope:latest`
- **Docker container prefixes**: `meshcore-*``corescope-*`
- **Default site name**: "MeshCore Analyzer" → "CoreScope"
## What Did NOT Change
- **Data directories**`~/meshcore-data/` stays as-is
- **Database filename**`meshcore.db` is unchanged
- **MQTT topics**`meshcore/#` topics are protocol-level and unchanged
- **Browser state** — Favorites, localStorage keys, and settings are preserved
- **Config file format**`config.json` structure is the same
---
## 1. Git Remote Update
Update your local clone to point to the new repository URL:
```bash
git remote set-url origin https://github.com/Kpa-clawbot/corescope.git
git pull
```
## 2. Docker (manage.sh) Users
Rebuild with the new image name:
```bash
./manage.sh stop
git pull
./manage.sh setup
```
The new image is `corescope:latest`. You can clean up the old image:
```bash
docker rmi meshcore-analyzer:latest
```
## 3. Docker Compose Users
Rebuild containers with the new names:
```bash
docker compose down
git pull
docker compose build
docker compose up -d
```
Container names change from `meshcore-*` to `corescope-*`. Old containers are removed by `docker compose down`.
## 4. Data Directories
**No action required.** The data directory `~/meshcore-data/` and database file `meshcore.db` are unchanged. Your existing data carries over automatically.
## 5. Config
If you customized `branding.siteName` in your `config.json`, update it to your preferred name. Otherwise the new default "CoreScope" applies automatically.
No other config keys changed.
## 6. MQTT
**No action required.** MQTT topics (`meshcore/#`) are protocol-level and are not affected by the rename.
## 7. Browser
**No action required.** Bookmarks/favorites will continue to work at the same host and port. localStorage keys are unchanged, so your settings and preferences are preserved.
## 8. CI/CD
If you have custom CI/CD pipelines that reference:
- The old repository URL (`meshcore-analyzer`)
- The old Docker image name (`meshcore-analyzer:latest`)
- Old container names (`meshcore-*`)
Update those references to use the new names.
---
## Summary Checklist
| Item | Action Required? | What to Do |
|------|-----------------|------------|
| Git remote | ✅ Yes | `git remote set-url origin …corescope.git` |
| Docker image | ✅ Yes | Rebuild; optionally `docker rmi` old image |
| Docker Compose | ✅ Yes | `docker compose down && build && up` |
| Data directories | ❌ No | Unchanged |
| Config | ⚠️ Maybe | Only if you customized `branding.siteName` |
| MQTT | ❌ No | Topics unchanged |
| Browser | ❌ No | Settings preserved |
| CI/CD | ⚠️ Maybe | Update if referencing old repo/image names |
+161 -411
View File
@@ -1,27 +1,21 @@
#!/bin/bash
# MeshCore Analyzer — Setup & Management Helper
# CoreScope — Setup & Management Helper
# Usage: ./manage.sh [command]
#
# All container management goes through docker compose.
# Container config lives in docker-compose.yml — this script is just a wrapper.
#
# Idempotent: safe to cancel and re-run at any point.
# Each step checks what's already done and skips it.
set -e
CONTAINER_NAME="meshcore-analyzer"
IMAGE_NAME="meshcore-analyzer"
DATA_VOLUME="meshcore-data"
CADDY_VOLUME="caddy-data"
IMAGE_NAME="corescope"
STATE_FILE=".setup-state"
# Source .env for port/path overrides (if present)
# Source .env for port/path overrides (same file docker compose reads)
[ -f .env ] && set -a && . ./.env && set +a
# Docker Compose mode detection
COMPOSE_MODE=false
if [ -f docker-compose.yml ]; then
COMPOSE_MODE=true
fi
# Resolved paths for prod/staging data
# Resolved paths for prod/staging data (must match docker-compose.yml)
PROD_DATA="${PROD_DATA_DIR:-$HOME/meshcore-data}"
STAGING_DATA="${STAGING_DATA_DIR:-$HOME/meshcore-staging-data}"
@@ -51,83 +45,6 @@ is_done() { [ -f "$STATE_FILE" ] && grep -qx "$1" "$STATE_FILE" 2>/dev/null;
# ─── Helpers ──────────────────────────────────────────────────────────────
# Determine the correct data volume/mount args for docker run.
# Detects existing host data directories and uses bind mounts if found.
get_data_mount_args() {
# Check for existing host data directories with a DB file
if [ -d "$HOME/meshcore-data" ] && [ -f "$HOME/meshcore-data/meshcore.db" ]; then
echo "-v $HOME/meshcore-data:/app/data"
return
fi
if [ -d "$(pwd)/data" ] && [ -f "$(pwd)/data/meshcore.db" ]; then
echo "-v $(pwd)/data:/app/data"
return
fi
# Default: Docker named volume
echo "-v ${DATA_VOLUME}:/app/data"
}
# Determine the required port mappings from Caddyfile
get_required_ports() {
local caddyfile_domain
caddyfile_domain=$(grep -v '^#' caddy-config/Caddyfile 2>/dev/null | head -1 | tr -d ' {')
if echo "$caddyfile_domain" | grep -qE '^:[0-9]+$'; then
# HTTP-only on a specific port (e.g., :80, :8080)
echo "${caddyfile_domain#:}"
else
# Domain name — needs 80 + 443 for Caddy auto-TLS
echo "80 443"
fi
}
# Get current container port mappings (just the host ports)
get_current_ports() {
docker inspect "$CONTAINER_NAME" 2>/dev/null | \
grep -oP '"HostPort":\s*"\K[0-9]+' | sort -u | tr '\n' ' ' | sed 's/ $//'
}
# Check if container port mappings match what's needed.
# Returns 0 if they match, 1 if mismatch.
check_port_match() {
local required current
required=$(get_required_ports | tr ' ' '\n' | sort | tr '\n' ' ' | sed 's/ $//')
current=$(get_current_ports | tr ' ' '\n' | sort | tr '\n' ' ' | sed 's/ $//')
[ "$required" = "$current" ]
}
# Build the docker run command args (ports + volumes)
get_docker_run_args() {
local ports_arg=""
for port in $(get_required_ports); do
ports_arg="$ports_arg -p ${port}:${port}"
done
local data_mount
data_mount=$(get_data_mount_args)
echo "$ports_arg \
-v $(pwd)/config.json:/app/config.json:ro \
-v $(pwd)/caddy-config/Caddyfile:/etc/caddy/Caddyfile:ro \
$data_mount \
-v ${CADDY_VOLUME}:/data/caddy"
}
# Recreate the container with current settings
recreate_container() {
info "Stopping and removing old container..."
docker stop "$CONTAINER_NAME" 2>/dev/null || true
docker rm "$CONTAINER_NAME" 2>/dev/null || true
local run_args
run_args=$(get_docker_run_args)
eval docker run -d \
--name "$CONTAINER_NAME" \
--restart unless-stopped \
$run_args \
"$IMAGE_NAME"
}
# Check config.json for placeholder values
check_config_placeholders() {
if [ -f config.json ]; then
@@ -140,7 +57,7 @@ check_config_placeholders() {
# Verify the running container is actually healthy
verify_health() {
local base_url="http://localhost:3000"
local container="corescope-prod"
local use_https=false
# Check if Caddyfile has a real domain (not :80)
@@ -156,7 +73,7 @@ verify_health() {
info "Waiting for server to respond..."
local healthy=false
for i in $(seq 1 45); do
if docker exec "$CONTAINER_NAME" wget -qO- http://localhost:3000/api/stats &>/dev/null; then
if docker exec "$container" wget -qO- http://localhost:3000/api/stats &>/dev/null; then
healthy=true
break
fi
@@ -172,7 +89,7 @@ verify_health() {
# Check for MQTT errors in recent logs
local mqtt_errors
mqtt_errors=$(docker logs "$CONTAINER_NAME" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
mqtt_errors=$(docker logs "$container" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
if [ -n "$mqtt_errors" ]; then
warn "MQTT errors detected in logs:"
echo "$mqtt_errors" | head -5 | sed 's/^/ /'
@@ -201,7 +118,7 @@ TOTAL_STEPS=6
cmd_setup() {
echo ""
echo "═══════════════════════════════════════"
echo " MeshCore Analyzer Setup"
echo " CoreScope Setup"
echo "═══════════════════════════════════════"
echo ""
@@ -234,6 +151,13 @@ cmd_setup() {
fi
log "Docker $(docker --version | grep -oP 'version \K[^ ,]+')"
# Check docker compose (separate check since it's a plugin/separate binary)
if ! docker compose version &>/dev/null; then
err "docker compose is required. Install Docker Desktop or docker-compose-plugin."
exit 1
fi
mark_done "docker"
# ── Step 2: Config ──
@@ -371,12 +295,12 @@ cmd_setup() {
if [ -n "$IMAGE_EXISTS" ] && is_done "build"; then
log "Image already built."
if confirm "Rebuild? (only needed if you updated the code)"; then
docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
docker compose build prod
log "Image rebuilt."
fi
else
info "This takes 1-2 minutes the first time..."
docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
docker compose build prod
log "Image built."
fi
mark_done "build"
@@ -385,45 +309,15 @@ cmd_setup() {
step 5 "Starting container"
# Detect existing data directories
if [ -d "$HOME/meshcore-data" ] && [ -f "$HOME/meshcore-data/meshcore.db" ]; then
info "Found existing data at \$HOME/meshcore-data/ — will use bind mount."
elif [ -d "$(pwd)/data" ] && [ -f "$(pwd)/data/meshcore.db" ]; then
info "Found existing data at ./data/ — will use bind mount."
if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
info "Found existing data at $PROD_DATA/ — will use bind mount."
fi
if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if docker ps --format '{{.Names}}' | grep -q "^corescope-prod$"; then
log "Container already running."
# Check port mappings match
if ! check_port_match; then
warn "Container port mappings don't match Caddyfile configuration."
warn "Current ports: $(get_current_ports)"
warn "Required ports: $(get_required_ports)"
if confirm "Recreate container with correct ports?"; then
recreate_container
log "Container recreated with correct ports."
fi
fi
elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
# Exists but stopped — check ports before starting
if ! check_port_match; then
warn "Stopped container has wrong port mappings."
warn "Current ports: $(get_current_ports)"
warn "Required ports: $(get_required_ports)"
if confirm "Recreate container with correct ports?"; then
recreate_container
log "Container recreated with correct ports."
else
info "Starting existing container (ports unchanged)..."
docker start "$CONTAINER_NAME"
log "Started (with old port mappings)."
fi
else
info "Container exists but is stopped. Starting..."
docker start "$CONTAINER_NAME"
log "Started."
fi
else
recreate_container
mkdir -p "$PROD_DATA"
docker compose up -d prod
log "Container started."
fi
mark_done "container"
@@ -431,7 +325,7 @@ cmd_setup() {
# ── Step 6: Verify ──
step 6 "Verifying"
if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if docker ps --format '{{.Names}}' | grep -q "^corescope-prod$"; then
verify_health
CADDYFILE_DOMAIN=$(grep -v '^#' caddy-config/Caddyfile 2>/dev/null | head -1 | tr -d ' {')
@@ -463,7 +357,7 @@ cmd_setup() {
err "Container failed to start."
echo ""
echo " Check what went wrong:"
echo " docker logs ${CONTAINER_NAME}"
echo " docker compose logs prod"
echo ""
echo " Common fixes:"
echo " • Invalid config.json — check JSON syntax"
@@ -501,7 +395,7 @@ prepare_staging_config() {
if [ ! -f "$staging_config" ] || [ "$prod_config" -nt "$staging_config" ]; then
info "Copying production config to staging..."
cp "$prod_config" "$staging_config"
sed -i 's/"siteName":\s*"[^"]*"/"siteName": "MeshCore Analyzer — STAGING"/' "$staging_config"
sed -i 's/"siteName":\s*"[^"]*"/"siteName": "CoreScope — STAGING"/' "$staging_config"
log "Staging config created at ${staging_config} with STAGING site name."
else
log "Staging config is up to date."
@@ -535,132 +429,72 @@ cmd_start() {
WITH_STAGING=true
fi
if $COMPOSE_MODE; then
if $WITH_STAGING; then
# Prepare staging data and config
prepare_staging_db
prepare_staging_config
if $WITH_STAGING; then
# Prepare staging data and config
prepare_staging_db
prepare_staging_config
info "Starting production container (meshcore-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
info "Starting staging container (meshcore-staging) on port ${STAGING_HTTP_PORT:-81}..."
docker compose --profile staging up -d
log "Production started on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}/${PROD_MQTT_PORT:-1883}"
log "Staging started on port ${STAGING_HTTP_PORT:-81} (MQTT: ${STAGING_MQTT_PORT:-1884})"
else
info "Starting production container (meshcore-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
docker compose up -d prod
log "Production started. Staging NOT running (use --with-staging to start both)."
fi
info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
info "Starting staging container (corescope-staging) on port ${STAGING_HTTP_PORT:-81}..."
docker compose --profile staging up -d
log "Production started on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}/${PROD_MQTT_PORT:-1883}"
log "Staging started on port ${STAGING_HTTP_PORT:-81} (MQTT: ${STAGING_MQTT_PORT:-1884})"
else
# Legacy single-container mode
if $WITH_STAGING; then
err "--with-staging requires docker-compose.yml. Run setup or add docker-compose.yml first."
exit 1
fi
warn "No docker-compose.yml found — using legacy single-container mode."
if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
warn "Already running."
elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if ! check_port_match; then
warn "Container port mappings don't match Caddyfile configuration."
warn "Current ports: $(get_current_ports)"
warn "Required ports: $(get_required_ports)"
if confirm "Recreate container with correct ports?"; then
recreate_container
log "Container recreated and started with correct ports."
return
fi
fi
docker start "$CONTAINER_NAME"
log "Started."
else
err "Container doesn't exist. Run './manage.sh setup' first."
exit 1
fi
info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
docker compose up -d prod
log "Production started. Staging NOT running (use --with-staging to start both)."
fi
}
cmd_stop() {
local TARGET="${1:-all}"
if $COMPOSE_MODE; then
case "$TARGET" in
prod)
info "Stopping production container (meshcore-prod)..."
docker compose stop prod
log "Production stopped."
;;
staging)
info "Stopping staging container (meshcore-staging)..."
docker compose stop staging
log "Staging stopped."
;;
all)
info "Stopping all containers..."
docker compose --profile staging --profile staging-go down 2>/dev/null
docker rm -f "$CONTAINER_NAME" 2>/dev/null
log "All containers stopped."
;;
*)
err "Usage: ./manage.sh stop [prod|staging|all]"
exit 1
;;
esac
else
# Legacy mode
docker stop "$CONTAINER_NAME" 2>/dev/null && log "Stopped." || warn "Not running."
fi
case "$TARGET" in
prod)
info "Stopping production container (corescope-prod)..."
docker compose stop prod
log "Production stopped."
;;
staging)
info "Stopping staging container (corescope-staging)..."
docker compose --profile staging stop staging
log "Staging stopped."
;;
all)
info "Stopping all containers..."
docker compose --profile staging --profile staging-go down
log "All containers stopped."
;;
*)
err "Usage: ./manage.sh stop [prod|staging|all]"
exit 1
;;
esac
}
cmd_restart() {
if $COMPOSE_MODE; then
local TARGET="${1:-prod}"
case "$TARGET" in
prod)
info "Restarting production container (meshcore-prod)..."
docker compose up -d --force-recreate prod
log "Production restarted."
;;
staging)
info "Restarting staging container (meshcore-staging)..."
docker compose --profile staging up -d --force-recreate staging
log "Staging restarted."
;;
all)
info "Restarting all containers..."
docker compose --profile staging up -d --force-recreate
log "All containers restarted."
;;
*)
err "Usage: ./manage.sh restart [prod|staging|all]"
exit 1
;;
esac
else
# Legacy mode
if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if ! check_port_match; then
warn "Port mappings have changed. Recreating container..."
recreate_container
log "Container recreated with correct ports."
else
docker restart "$CONTAINER_NAME"
log "Restarted."
fi
elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if ! check_port_match; then
warn "Port mappings have changed. Recreating container..."
recreate_container
log "Container recreated with correct ports."
else
docker start "$CONTAINER_NAME"
log "Started."
fi
else
err "Not running. Use './manage.sh setup'."
local TARGET="${1:-prod}"
case "$TARGET" in
prod)
info "Restarting production container (corescope-prod)..."
docker compose up -d --force-recreate prod
log "Production restarted."
;;
staging)
info "Restarting staging container (corescope-staging)..."
docker compose --profile staging up -d --force-recreate staging
log "Staging restarted."
;;
all)
info "Restarting all containers..."
docker compose --profile staging up -d --force-recreate
log "All containers restarted."
;;
*)
err "Usage: ./manage.sh restart [prod|staging|all]"
exit 1
fi
fi
;;
esac
}
# ─── Status ───────────────────────────────────────────────────────────────
@@ -695,143 +529,68 @@ show_container_status() {
cmd_status() {
echo ""
echo "═══════════════════════════════════════"
echo " CoreScope Status"
echo "═══════════════════════════════════════"
echo ""
if $COMPOSE_MODE; then
echo "═══════════════════════════════════════"
echo " MeshCore Analyzer Status (Compose)"
echo "═══════════════════════════════════════"
echo ""
# Production
show_container_status "meshcore-prod" "Production"
echo ""
# Staging
if container_running "meshcore-staging"; then
show_container_status "meshcore-staging" "Staging"
else
info "Staging (meshcore-staging): Not running (use --with-staging to start both)"
fi
echo ""
# Disk usage
if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
local db_size
db_size=$(du -h "$PROD_DATA/meshcore.db" 2>/dev/null | cut -f1)
info "Production DB: ${db_size}"
fi
if [ -d "$STAGING_DATA" ] && [ -f "$STAGING_DATA/meshcore.db" ]; then
local staging_db_size
staging_db_size=$(du -h "$STAGING_DATA/meshcore.db" 2>/dev/null | cut -f1)
info "Staging DB: ${staging_db_size}"
fi
# Production
show_container_status "corescope-prod" "Production"
echo ""
# Staging
if container_running "corescope-staging"; then
show_container_status "corescope-staging" "Staging"
else
# Legacy single-container status
if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
log "Container is running."
echo ""
docker ps --filter "name=${CONTAINER_NAME}" --format " Status: {{.Status}}"
docker ps --filter "name=${CONTAINER_NAME}" --format " Ports: {{.Ports}}"
echo ""
info "Service health:"
# Server
if docker exec "$CONTAINER_NAME" wget -qO /dev/null http://localhost:3000/api/stats 2>/dev/null; then
STATS=$(docker exec "$CONTAINER_NAME" wget -qO- http://localhost:3000/api/stats 2>/dev/null)
PACKETS=$(echo "$STATS" | grep -oP '"totalPackets":\K[0-9]+' 2>/dev/null || echo "?")
NODES=$(echo "$STATS" | grep -oP '"totalNodes":\K[0-9]+' 2>/dev/null || echo "?")
log " Server — ${PACKETS} packets, ${NODES} nodes"
else
err " Server — not responding"
fi
# Mosquitto
if docker exec "$CONTAINER_NAME" pgrep mosquitto &>/dev/null; then
log " Mosquitto — running"
else
err " Mosquitto — not running"
fi
# Caddy
if docker exec "$CONTAINER_NAME" pgrep caddy &>/dev/null; then
log " Caddy — running"
else
err " Caddy — not running"
fi
# Check for MQTT errors in recent logs
MQTT_ERRORS=$(docker logs "$CONTAINER_NAME" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
if [ -n "$MQTT_ERRORS" ]; then
echo ""
warn "MQTT errors in recent logs:"
echo "$MQTT_ERRORS" | head -3 | sed 's/^/ /'
fi
# Port mapping check
if ! check_port_match; then
echo ""
warn "Port mappings don't match Caddyfile. Run './manage.sh restart' to fix."
fi
# Disk usage
DB_SIZE=$(docker exec "$CONTAINER_NAME" du -h /app/data/meshcore.db 2>/dev/null | cut -f1)
if [ -n "$DB_SIZE" ]; then
echo ""
info "Database size: ${DB_SIZE}"
fi
else
err "Container is not running."
if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
echo " Start with: ./manage.sh start"
else
echo " Set up with: ./manage.sh setup"
fi
fi
info "Staging (corescope-staging): Not running (use --with-staging to start both)"
fi
echo ""
# Disk usage
if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
local db_size
db_size=$(du -h "$PROD_DATA/meshcore.db" 2>/dev/null | cut -f1)
info "Production DB: ${db_size}"
fi
if [ -d "$STAGING_DATA" ] && [ -f "$STAGING_DATA/meshcore.db" ]; then
local staging_db_size
staging_db_size=$(du -h "$STAGING_DATA/meshcore.db" 2>/dev/null | cut -f1)
info "Staging DB: ${staging_db_size}"
fi
echo ""
}
# ─── Logs ─────────────────────────────────────────────────────────────────
cmd_logs() {
if $COMPOSE_MODE; then
local TARGET="${1:-prod}"
local LINES="${2:-100}"
case "$TARGET" in
prod)
info "Tailing production logs..."
docker compose logs -f --tail="$LINES" prod
;;
staging)
if container_running "meshcore-staging"; then
info "Tailing staging logs..."
docker compose logs -f --tail="$LINES" staging
else
err "Staging container is not running."
info "Start with: ./manage.sh start --with-staging"
exit 1
fi
;;
*)
err "Usage: ./manage.sh logs [prod|staging] [lines]"
local TARGET="${1:-prod}"
local LINES="${2:-100}"
case "$TARGET" in
prod)
info "Tailing production logs..."
docker compose logs -f --tail="$LINES" prod
;;
staging)
if container_running "corescope-staging"; then
info "Tailing staging logs..."
docker compose logs -f --tail="$LINES" staging
else
err "Staging container is not running."
info "Start with: ./manage.sh start --with-staging"
exit 1
;;
esac
else
# Legacy mode
docker logs -f "$CONTAINER_NAME" --tail "${1:-100}"
fi
fi
;;
*)
err "Usage: ./manage.sh logs [prod|staging] [lines]"
exit 1
;;
esac
}
# ─── Promote ──────────────────────────────────────────────────────────────
cmd_promote() {
if ! $COMPOSE_MODE; then
err "Promotion requires Docker Compose setup (docker-compose.yml)."
exit 1
fi
echo ""
info "Promotion Flow: Staging → Production"
echo ""
@@ -843,10 +602,10 @@ cmd_promote() {
# Show what's currently running
local staging_image staging_created prod_image prod_created
staging_image=$(docker inspect meshcore-staging --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
staging_created=$(docker inspect meshcore-staging --format '{{.Created}}' 2>/dev/null || echo "N/A")
prod_image=$(docker inspect meshcore-prod --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
prod_created=$(docker inspect meshcore-prod --format '{{.Created}}' 2>/dev/null || echo "N/A")
staging_image=$(docker inspect corescope-staging --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
staging_created=$(docker inspect corescope-staging --format '{{.Created}}' 2>/dev/null || echo "N/A")
prod_image=$(docker inspect corescope-prod --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
prod_created=$(docker inspect corescope-prod --format '{{.Created}}' 2>/dev/null || echo "N/A")
echo " Staging: ${staging_image} (created ${staging_created})"
echo " Prod: ${prod_image} (created ${prod_created})"
@@ -863,8 +622,8 @@ cmd_promote() {
mkdir -p "$BACKUP_DIR"
if [ -f "$PROD_DATA/meshcore.db" ]; then
cp "$PROD_DATA/meshcore.db" "$BACKUP_DIR/"
elif container_running "meshcore-prod"; then
docker cp meshcore-prod:/app/data/meshcore.db "$BACKUP_DIR/"
elif container_running "corescope-prod"; then
docker cp corescope-prod:/app/data/meshcore.db "$BACKUP_DIR/"
else
warn "Could not backup production database."
fi
@@ -878,7 +637,7 @@ cmd_promote() {
info "Waiting for production health check..."
local i health
for i in $(seq 1 30); do
health=$(container_health "meshcore-prod")
health=$(container_health "corescope-prod")
if [ "$health" = "healthy" ]; then
log "Production healthy after ${i}s"
break
@@ -906,10 +665,10 @@ cmd_update() {
git pull
info "Rebuilding image..."
docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
docker compose build prod
info "Restarting with new image..."
recreate_container
docker compose up -d --force-recreate prod
log "Updated and restarted. Data preserved."
}
@@ -918,18 +677,19 @@ cmd_update() {
cmd_backup() {
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
BACKUP_DIR="${1:-./backups/meshcore-${TIMESTAMP}}"
BACKUP_DIR="${1:-./backups/corescope-${TIMESTAMP}}"
mkdir -p "$BACKUP_DIR"
info "Backing up to ${BACKUP_DIR}/"
# Database
DB_PATH=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/meshcore.db
# Always use bind mount path (from .env or default)
DB_PATH="$PROD_DATA/meshcore.db"
if [ -f "$DB_PATH" ]; then
cp "$DB_PATH" "$BACKUP_DIR/meshcore.db"
log "Database ($(du -h "$BACKUP_DIR/meshcore.db" | cut -f1))"
elif docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
docker cp "${CONTAINER_NAME}:/app/data/meshcore.db" "$BACKUP_DIR/meshcore.db" 2>/dev/null && \
elif container_running "corescope-prod"; then
docker cp corescope-prod:/app/data/meshcore.db "$BACKUP_DIR/meshcore.db" 2>/dev/null && \
log "Database (via docker cp)" || warn "Could not backup database"
else
warn "Database not found (container not running?)"
@@ -948,7 +708,8 @@ cmd_backup() {
fi
# Theme
THEME_PATH=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/theme.json
# Always use bind mount path (from .env or default)
THEME_PATH="$PROD_DATA/theme.json"
if [ -f "$THEME_PATH" ]; then
cp "$THEME_PATH" "$BACKUP_DIR/theme.json"
log "theme.json"
@@ -972,7 +733,7 @@ cmd_restore() {
if [ -d "./backups" ]; then
echo ""
echo " Available backups:"
ls -dt ./backups/meshcore-* 2>/dev/null | head -10 | while read d; do
ls -dt ./backups/meshcore-* ./backups/corescope-* 2>/dev/null | head -10 | while read d; do
if [ -d "$d" ]; then
echo " $d/ ($(ls "$d" | wc -l) files)"
elif [ -f "$d" ]; then
@@ -1019,17 +780,14 @@ cmd_restore() {
# Backup current state first
info "Backing up current state..."
cmd_backup "./backups/meshcore-pre-restore-$(date +%Y%m%d-%H%M%S)"
cmd_backup "./backups/corescope-pre-restore-$(date +%Y%m%d-%H%M%S)"
docker stop "$CONTAINER_NAME" 2>/dev/null || true
docker compose stop prod 2>/dev/null || true
# Restore database
DEST_DB=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/meshcore.db
if [ -d "$(dirname "$DEST_DB")" ]; then
cp "$DB_FILE" "$DEST_DB"
else
docker cp "$DB_FILE" "${CONTAINER_NAME}:/app/data/meshcore.db"
fi
mkdir -p "$PROD_DATA"
DEST_DB="$PROD_DATA/meshcore.db"
cp "$DB_FILE" "$DEST_DB"
log "Database restored"
# Restore config if present
@@ -1047,27 +805,25 @@ cmd_restore() {
# Restore theme if present
if [ -n "$THEME_FILE" ] && [ -f "$THEME_FILE" ]; then
DEST_THEME=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/theme.json
if [ -d "$(dirname "$DEST_THEME")" ]; then
cp "$THEME_FILE" "$DEST_THEME"
fi
DEST_THEME="$PROD_DATA/theme.json"
cp "$THEME_FILE" "$DEST_THEME"
log "theme.json restored"
fi
docker start "$CONTAINER_NAME"
docker compose up -d prod
log "Restored and restarted."
}
# ─── MQTT Test ────────────────────────────────────────────────────────────
cmd_mqtt_test() {
if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
if ! container_running "corescope-prod"; then
err "Container not running. Start with: ./manage.sh start"
exit 1
fi
info "Listening for MQTT messages (10 second timeout)..."
MSG=$(docker exec "$CONTAINER_NAME" mosquitto_sub -h localhost -t 'meshcore/#' -C 1 -W 10 2>/dev/null)
MSG=$(docker exec corescope-prod mosquitto_sub -h localhost -t 'meshcore/#' -C 1 -W 10 2>/dev/null)
if [ -n "$MSG" ]; then
log "Received MQTT message:"
echo " $MSG" | head -c 200
@@ -1084,28 +840,26 @@ cmd_mqtt_test() {
cmd_reset() {
echo ""
warn "This will remove the container, image, and setup state."
warn "Your config.json, Caddyfile, and data volume are NOT deleted."
warn "This will remove all containers, images, and setup state."
warn "Your config.json, Caddyfile, and data directory are NOT deleted."
echo ""
if ! confirm "Continue?"; then
echo " Aborted."
exit 0
fi
docker stop "$CONTAINER_NAME" 2>/dev/null || true
docker rm "$CONTAINER_NAME" 2>/dev/null || true
docker rmi "$IMAGE_NAME" 2>/dev/null || true
docker compose --profile staging --profile staging-go down --rmi local 2>/dev/null || true
rm -f "$STATE_FILE"
log "Reset complete. Run './manage.sh setup' to start over."
echo " Data volume preserved. To delete it: docker volume rm ${DATA_VOLUME}"
echo " Data directory: $PROD_DATA (not removed)"
}
# ─── Help ─────────────────────────────────────────────────────────────────
cmd_help() {
echo ""
echo "MeshCore Analyzer — Management Script"
echo "CoreScope — Management Script"
echo ""
echo "Usage: ./manage.sh <command>"
echo ""
@@ -1128,11 +882,7 @@ cmd_help() {
echo " restore <d> Restore from backup dir or .db file"
echo " mqtt-test Check if MQTT data is flowing"
echo ""
if $COMPOSE_MODE; then
info "Docker Compose mode detected (docker-compose.yml present)."
else
warn "Legacy mode (no docker-compose.yml). Some commands unavailable."
fi
echo "All commands use docker compose with docker-compose.yml."
echo ""
}
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "common.proto";
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
// Core Channel Type
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
// Pagination
+2 -2
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
//
// GET /api/config/theme Theme and branding configuration
@@ -10,7 +10,7 @@ option go_package = "github.com/meshcore-analyzer/proto/v1";
// Site branding configuration.
message Branding {
// Site name (default: "MeshCore Analyzer").
// Site name (default: "CoreScope").
string site_name = 1 [json_name = "siteName"];
// Site tagline.
string tagline = 2;
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
// Decoded Packet Structure
// Returned by POST /api/decode, POST /api/packets, and WS broadcast.
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "common.proto";
import "packet.proto";
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "common.proto";
import "packet.proto";
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "common.proto";
import "decoded.proto";
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "common.proto";
+1 -1
View File
@@ -2,7 +2,7 @@ syntax = "proto3";
package meshcore.v1;
option go_package = "github.com/meshcore-analyzer/proto/v1";
option go_package = "github.com/corescope/proto/v1";
import "decoded.proto";
import "packet.proto";
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — analytics.js (v2 — full nerd mode) === */
/* === CoreScope — analytics.js (v2 — full nerd mode) === */
'use strict';
(function () {
+2 -2
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — app.js === */
/* === CoreScope — app.js === */
'use strict';
// --- Route/Payload name maps ---
@@ -109,7 +109,7 @@ function formatVersionBadge(version, commit, engine) {
if (!version && !commit && !engine) return '';
var port = (typeof location !== 'undefined' && location.port) || '';
var isProd = !port || port === '80' || port === '443';
var GH = 'https://github.com/Kpa-clawbot/meshcore-analyzer';
var GH = 'https://github.com/Kpa-clawbot/corescope';
var parts = [];
if (version && isProd) {
var vTag = version.charAt(0) === 'v' ? version : 'v' + version;
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — audio-lab.js === */
/* === CoreScope — audio-lab.js === */
/* Audio Lab: Packet Jukebox for sound debugging & understanding */
'use strict';
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — channels.js === */
/* === CoreScope — channels.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — compare.js === */
/* === CoreScope — compare.js === */
/* Observer packet comparison — Fixes #129 */
'use strict';
+3 -3
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — customize.js === */
/* === CoreScope — customize.js === */
/* Tools → Customization: visual config builder with live preview & JSON export */
'use strict';
@@ -9,7 +9,7 @@
const DEFAULTS = {
branding: {
siteName: 'MeshCore Analyzer',
siteName: 'CoreScope',
tagline: 'Real-time MeshCore LoRa mesh network analyzer',
logoUrl: '',
faviconUrl: ''
@@ -45,7 +45,7 @@
ANON_REQ: '#f43f5e'
},
home: {
heroTitle: 'MeshCore Analyzer',
heroTitle: 'CoreScope',
heroSubtitle: 'Find your nodes to start monitoring them.',
steps: [
{ emoji: '💬', title: 'Join the Bay Area MeshCore Discord', description: 'The community Discord is the best place to get help and find local mesh enthusiasts.' },
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — home.css === */
/* === CoreScope — home.css === */
/* Override #app overflow:hidden for home page scrolling */
#app:has(.home-hero), #app:has(.home-chooser) { overflow-y: auto; }
+4 -4
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — home.js (My Mesh Dashboard) === */
/* === CoreScope — home.js (My Mesh Dashboard) === */
'use strict';
(function () {
@@ -39,7 +39,7 @@
function showChooser(container) {
container.innerHTML = `
<section class="home-chooser">
<h1>Welcome to ${escapeHtml(window.SITE_CONFIG?.branding?.siteName || 'MeshCore Analyzer')}</h1>
<h1>Welcome to ${escapeHtml(window.SITE_CONFIG?.branding?.siteName || 'CoreScope')}</h1>
<p>How familiar are you with MeshCore?</p>
<div class="chooser-options">
<button class="chooser-btn new" id="chooseNew">
@@ -63,7 +63,7 @@
const myNodes = getMyNodes();
const hasNodes = myNodes.length > 0;
const homeCfg = window.SITE_CONFIG?.home || null;
const siteName = window.SITE_CONFIG?.branding?.siteName || 'MeshCore Analyzer';
const siteName = window.SITE_CONFIG?.branding?.siteName || 'CoreScope';
container.innerHTML = `
<section class="home-hero">
@@ -324,7 +324,7 @@
loadMyNodes();
// Update title if no nodes left
const h1 = document.querySelector('.home-hero h1');
if (h1 && !getMyNodes().length) h1.textContent = 'MeshCore Analyzer';
if (h1 && !getMyNodes().length) h1.textContent = 'CoreScope';
});
});
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — hop-display.js === */
/* === CoreScope — hop-display.js === */
/* Shared hop rendering with conflict info for all pages */
'use strict';
+33 -33
View File
@@ -5,12 +5,12 @@
<link rel="icon" href="favicon.ico" type="image/x-icon">
<link rel="icon" href="favicon.svg" type="image/svg+xml">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<title>MeshCore Analyzer</title>
<title>CoreScope</title>
<!-- Open Graph / Discord embed -->
<meta property="og:title" content="MeshCore Analyzer">
<meta property="og:title" content="CoreScope">
<meta property="og:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, route analysis, and deep mesh analytics.">
<meta property="og:image" content="https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/public/og-image.png">
<meta property="og:image" content="https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/public/og-image.png">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta property="og:url" content="https://analyzer.00id.net">
@@ -19,12 +19,12 @@
<!-- Twitter Card -->
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="MeshCore Analyzer">
<meta name="twitter:title" content="CoreScope">
<meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis.">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1774690966">
<link rel="stylesheet" href="home.css?v=1774690966">
<link rel="stylesheet" href="live.css?v=1774690966">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1774731523">
<link rel="stylesheet" href="home.css?v=1774731523">
<link rel="stylesheet" href="live.css?v=1774731523">
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
crossorigin="anonymous">
@@ -40,7 +40,7 @@
<div class="nav-left">
<a href="#/" class="nav-brand">
<span class="brand-icon">🍄</span>
<span class="brand-text">MeshCore Analyzer</span>
<span class="brand-text">CoreScope</span>
<span class="live-dot" id="liveDot" title="WebSocket connected" aria-label="WebSocket connected"></span>
</a>
<div class="nav-links">
@@ -81,29 +81,29 @@
<main id="app" role="main"></main>
<script src="vendor/qrcode.js"></script>
<script src="roles.js?v=1774690966"></script>
<script src="customize.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="region-filter.js?v=1774690966"></script>
<script src="hop-resolver.js?v=1774690966"></script>
<script src="hop-display.js?v=1774690966"></script>
<script src="app.js?v=1774690966"></script>
<script src="home.js?v=1774690966"></script>
<script src="packet-filter.js?v=1774690966"></script>
<script src="packets.js?v=1774690966"></script>
<script src="map.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v1-constellation.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="compare.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1774690966" onerror="console.error('Failed to load:', this.src)"></script>
<script src="roles.js?v=1774731523"></script>
<script src="customize.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="region-filter.js?v=1774731523"></script>
<script src="hop-resolver.js?v=1774731523"></script>
<script src="hop-display.js?v=1774731523"></script>
<script src="app.js?v=1774731523"></script>
<script src="home.js?v=1774731523"></script>
<script src="packet-filter.js?v=1774731523"></script>
<script src="packets.js?v=1774731523"></script>
<script src="map.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v1-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="compare.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
</body>
</html>
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — map.js === */
/* === CoreScope — map.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — node-analytics.js === */
/* === CoreScope — node-analytics.js === */
'use strict';
(function () {
const PAYLOAD_LABELS = { 0: 'Request', 1: 'Response', 2: 'Direct Msg', 3: 'ACK', 4: 'Advert', 5: 'Channel Msg', 7: 'Anon Req', 8: 'Path', 9: 'Trace', 11: 'Control' };
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — nodes.js === */
/* === CoreScope — nodes.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — observer-detail.js === */
/* === CoreScope — observer-detail.js === */
'use strict';
(function () {
const PAYLOAD_LABELS = { 0: 'Request', 1: 'Response', 2: 'Direct Msg', 3: 'ACK', 4: 'Advert', 5: 'Channel Msg', 7: 'Anon Req', 8: 'Path', 9: 'Trace', 11: 'Control' };
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — observers.js === */
/* === CoreScope — observers.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — packets.js === */
/* === CoreScope — packets.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — perf.js === */
/* === CoreScope — perf.js === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — region-filter.js (shared region filter component) === */
/* === CoreScope — region-filter.js (shared region filter component) === */
'use strict';
(function () {
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — roles.js (shared config module) === */
/* === CoreScope — roles.js (shared config module) === */
'use strict';
/*
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — style.css === */
/* === CoreScope — style.css === */
:root {
--nav-bg: #0f0f23;
+1 -1
View File
@@ -1,4 +1,4 @@
/* === MeshCore Analyzer — traces.js === */
/* === CoreScope — traces.js === */
'use strict';
(function () {
+23 -23
View File
@@ -64,9 +64,9 @@ async function collectCoverage() {
// ══════════════════════════════════════════════
console.log(' [coverage] Home page — chooser...');
// Clear localStorage to get chooser
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(BASE, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click "I'm new"
await safeClick('#chooseNew');
@@ -105,7 +105,7 @@ async function collectCoverage() {
// Switch to experienced mode
await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#chooseExp');
// Interact with experienced home page
@@ -120,7 +120,7 @@ async function collectCoverage() {
// NODES PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Nodes page...');
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Sort by EVERY column
for (const col of ['name', 'public_key', 'role', 'last_seen', 'advert_count']) {
@@ -168,7 +168,7 @@ async function collectCoverage() {
try {
const firstNodeKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim());
if (firstNodeKey) {
await page.goto(`${BASE}/#/nodes/${firstNodeKey}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/nodes/${firstNodeKey}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click tabs on detail page
await clickAll('.tab-btn, [data-tab]', 10);
@@ -191,7 +191,7 @@ async function collectCoverage() {
try {
const firstKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim()).catch(() => null);
if (firstKey) {
await page.goto(`${BASE}/#/nodes/${firstKey}?scroll=paths`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/nodes/${firstKey}?scroll=paths`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
}
} catch {}
@@ -199,7 +199,7 @@ async function collectCoverage() {
// PACKETS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Packets page...');
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Open filter bar
await safeClick('#filterToggleBtn');
@@ -285,13 +285,13 @@ async function collectCoverage() {
} catch {}
// Navigate to specific packet by hash
await page.goto(`${BASE}/#/packets/deadbeef`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/packets/deadbeef`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// ══════════════════════════════════════════════
// MAP PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Map page...');
await page.goto(`${BASE}/#/map`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/map`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Toggle controls panel
await safeClick('#mapControlsToggle');
@@ -345,7 +345,7 @@ async function collectCoverage() {
// ANALYTICS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Analytics page...');
await page.goto(`${BASE}/#/analytics`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/analytics`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click EVERY analytics tab
const analyticsTabs = ['overview', 'rf', 'topology', 'channels', 'hashsizes', 'collisions', 'subpaths', 'nodes', 'distance'];
@@ -383,7 +383,7 @@ async function collectCoverage() {
// Deep-link to each analytics tab via URL
for (const tab of analyticsTabs) {
await page.goto(`${BASE}/#/analytics?tab=${tab}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/analytics?tab=${tab}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
}
// Region filter on analytics
@@ -396,7 +396,7 @@ async function collectCoverage() {
// CUSTOMIZE
// ══════════════════════════════════════════════
console.log(' [coverage] Customizer...');
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(BASE, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#customizeToggle');
// Click EVERY customizer tab
@@ -503,7 +503,7 @@ async function collectCoverage() {
// CHANNELS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Channels page...');
await page.goto(`${BASE}/#/channels`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/channels`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click channel rows/items
await clickAll('.channel-item, .channel-row, .channel-card', 3);
await clickAll('table tbody tr', 3);
@@ -512,7 +512,7 @@ async function collectCoverage() {
try {
const channelHash = await page.$eval('table tbody tr td:first-child', el => el.textContent.trim()).catch(() => null);
if (channelHash) {
await page.goto(`${BASE}/#/channels/${channelHash}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/channels/${channelHash}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
}
} catch {}
@@ -520,7 +520,7 @@ async function collectCoverage() {
// LIVE PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Live page...');
await page.goto(`${BASE}/#/live`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/live`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// VCR controls
await safeClick('#vcrPauseBtn');
@@ -603,14 +603,14 @@ async function collectCoverage() {
// TRACES PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Traces page...');
await page.goto(`${BASE}/#/traces`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/traces`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await clickAll('table tbody tr', 3);
// ══════════════════════════════════════════════
// OBSERVERS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Observers page...');
await page.goto(`${BASE}/#/observers`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/observers`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click observer rows
const obsRows = await page.$$('table tbody tr, .observer-card, .observer-row');
for (let i = 0; i < Math.min(obsRows.length, 3); i++) {
@@ -631,7 +631,7 @@ async function collectCoverage() {
// PERF PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Perf page...');
await page.goto(`${BASE}/#/perf`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/perf`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#perfRefresh');
await safeClick('#perfReset');
@@ -641,14 +641,14 @@ async function collectCoverage() {
console.log(' [coverage] App.js — router + global...');
// Navigate to bad route to trigger error/404
await page.goto(`${BASE}/#/nonexistent-route`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/nonexistent-route`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Navigate to every route via hash
const allRoutes = ['home', 'nodes', 'packets', 'map', 'live', 'channels', 'traces', 'observers', 'analytics', 'perf'];
for (const route of allRoutes) {
try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {});
await new Promise(r => setTimeout(r, 200));
} catch {}
}
@@ -788,14 +788,14 @@ async function collectCoverage() {
console.log(' [coverage] Region filter...');
try {
// Open region filter on nodes page
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#nodesRegionFilter');
await clickAll('#nodesRegionFilter input[type="checkbox"]', 3);
} catch {}
// Region filter on packets
try {
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#packetsRegionFilter');
await clickAll('#packetsRegionFilter input[type="checkbox"]', 3);
} catch {}
@@ -807,7 +807,7 @@ async function collectCoverage() {
for (const route of allRoutes) {
try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {});
await new Promise(r => setTimeout(r, 200));
} catch {}
}
+11 -3
View File
@@ -344,7 +344,7 @@ app.get('/api/config/theme', (req, res) => {
const theme = loadThemeFile();
res.json({
branding: {
siteName: 'MeshCore Analyzer',
siteName: 'CoreScope',
tagline: 'Real-time MeshCore LoRa mesh network analyzer',
...(cfg.branding || {}),
...(theme.branding || {})
@@ -716,6 +716,10 @@ for (const source of mqttSources) {
const role = p.flags ? (p.flags.repeater ? 'repeater' : p.flags.room ? 'room' : p.flags.sensor ? 'sensor' : 'companion') : 'companion';
db.upsertNode({ public_key: p.pubKey, name: p.name || null, role, lat: p.lat, lon: p.lon, last_seen: now });
if (txResult && txResult.isNew) db.incrementAdvertCount(p.pubKey);
// Update telemetry if present in advert
if (p.battery_mv != null || p.temperature_c != null) {
db.updateNodeTelemetry({ public_key: p.pubKey, battery_mv: p.battery_mv ?? null, temperature_c: p.temperature_c ?? null });
}
// Invalidate this node's caches on advert
cache.invalidate('node:' + p.pubKey);
cache.invalidate('health:' + p.pubKey);
@@ -1057,6 +1061,10 @@ app.post('/api/packets', requireApiKey, (req, res) => {
const role = p.flags ? (p.flags.repeater ? 'repeater' : p.flags.room ? 'room' : p.flags.sensor ? 'sensor' : 'companion') : 'companion';
db.upsertNode({ public_key: p.pubKey, name: p.name || null, role, lat: p.lat, lon: p.lon, last_seen: now });
if (txResult && txResult.isNew) db.incrementAdvertCount(p.pubKey);
// Update telemetry if present in advert
if (p.battery_mv != null || p.temperature_c != null) {
db.updateNodeTelemetry({ public_key: p.pubKey, battery_mv: p.battery_mv ?? null, temperature_c: p.temperature_c ?? null });
}
} else {
console.warn(`[advert] Skipping corrupted ADVERT (API): ${validation.reason}`);
}
@@ -2948,7 +2956,7 @@ app.get('/{*splat}', (req, res) => {
if (fs.existsSync(indexPath)) {
res.sendFile(indexPath);
} else {
res.status(200).send('<!DOCTYPE html><html><body><h1>MeshCore Analyzer</h1><p>Frontend not yet built.</p></body></html>');
res.status(200).send('<!DOCTYPE html><html><body><h1>CoreScope</h1><p>Frontend not yet built.</p></body></html>');
}
});
@@ -2959,7 +2967,7 @@ if (require.main === module) {
db.removePhantomNodes();
server.listen(listenPort, () => {
const protocol = isHttps ? 'https' : 'http';
console.log(`MeshCore Analyzer running on ${protocol}://localhost:${listenPort}`);
console.log(`CoreScope running on ${protocol}://localhost:${listenPort}`);
// Log theme file location
let themeFound = false;
for (const p of THEME_PATHS) {
+1 -1
View File
@@ -3,7 +3,7 @@
set -e
echo "═══════════════════════════════════════"
echo " MeshCore Analyzer — Test Suite"
echo " CoreScope — Test Suite"
echo "═══════════════════════════════════════"
echo ""
+42
View File
@@ -213,6 +213,48 @@ console.log('── Spec Tests: Advert Payload ──');
assertEq(p.name, undefined, 'advert no name: name undefined');
}
// Telemetry: sensor node with battery + positive temperature
{
const pubkey = 'AA'.repeat(32);
const sig = 'BB'.repeat(64);
const flags = '84'; // sensor(4) | hasName(0x80)
const name = Buffer.from('S1').toString('hex') + '00'; // null-terminated
const battBuf = Buffer.alloc(2); battBuf.writeUInt16LE(3700);
const tempBuf = Buffer.alloc(2); tempBuf.writeInt16LE(2850); // 28.50°C
const hex = '1200' + pubkey + '00000000' + sig + flags + name +
battBuf.toString('hex') + tempBuf.toString('hex');
const p = decodePacket(hex).payload;
assertEq(p.battery_mv, 3700, 'telemetry: battery_mv decoded');
assert(Math.abs(p.temperature_c - 28.50) < 0.01, 'telemetry: temperature_c positive');
}
// Telemetry: sensor node with 0°C must still emit temperature_c
{
const pubkey = 'CC'.repeat(32);
const sig = 'DD'.repeat(64);
const flags = '84'; // sensor(4) | hasName(0x80)
const name = Buffer.from('S2').toString('hex') + '00';
const battBuf = Buffer.alloc(2); battBuf.writeUInt16LE(3600);
const tempBuf = Buffer.alloc(2); // 0°C
const hex = '1200' + pubkey + '00000000' + sig + flags + name +
battBuf.toString('hex') + tempBuf.toString('hex');
const p = decodePacket(hex).payload;
assert(p.temperature_c === 0, 'telemetry: 0°C is valid and emitted');
}
// Telemetry: non-sensor node with trailing bytes must NOT decode telemetry
{
const pubkey = 'EE'.repeat(32);
const sig = 'FF'.repeat(64);
const flags = '82'; // repeater(2) | hasName(0x80)
const name = Buffer.from('R1').toString('hex') + '00';
const extraBytes = 'B40ED403'; // battery-like and temp-like bytes
const hex = '1200' + pubkey + '00000000' + sig + flags + name + extraBytes;
const p = decodePacket(hex).payload;
assertEq(p.battery_mv, undefined, 'telemetry: non-sensor node: battery_mv must be undefined');
assertEq(p.temperature_c, undefined, 'telemetry: non-sensor node: temperature_c must be undefined');
}
console.log('── Spec Tests: Encrypted Payload Format ──');
// NOTE: Spec says v1 encrypted payloads have dest(1) + src(1) + MAC(2) + ciphertext
+14 -1
View File
@@ -44,7 +44,7 @@ async function run() {
await page.goto(BASE, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('nav, .navbar, .nav, [class*="nav"]');
const title = await page.title();
assert(title.toLowerCase().includes('meshcore'), `Title "${title}" doesn't contain MeshCore`);
assert(title.toLowerCase().includes('corescope'), `Title "${title}" doesn't contain CoreScope`);
const nav = await page.$('nav, .navbar, .nav, [class*="nav"]');
assert(nav, 'Nav bar not found');
});
@@ -909,6 +909,19 @@ async function run() {
assert(hexDump, 'Hex dump should be visible after selecting a packet');
});
// Extract frontend coverage if instrumented server is running
try {
const coverage = await page.evaluate(() => window.__coverage__);
if (coverage) {
const fs = require('fs');
const path = require('path');
const outDir = path.join(__dirname, '.nyc_output');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
fs.writeFileSync(path.join(outDir, 'e2e-coverage.json'), JSON.stringify(coverage));
console.log(`Frontend coverage from E2E: ${Object.keys(coverage).length} files`);
}
} catch {}
await browser.close();
// Summary
+1 -1
View File
@@ -1304,7 +1304,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
loadInCtx(ctx, 'public/app.js');
return ctx;
}
const GH = 'https://github.com/Kpa-clawbot/meshcore-analyzer';
const GH = 'https://github.com/Kpa-clawbot/corescope';
test('returns empty string when all args missing', () => {
const { formatVersionBadge } = makeBadgeSandbox('');