Compare commits

...

27 Commits

Author SHA1 Message Date
Kpa-clawbot
7394f00fec ci: force bash shell for all workflow steps
Self-hosted runner defaults to PowerShell on Windows, causing bash
syntax (if/then/fi, curl line continuations) to fail with parse errors.
Setting defaults.run.shell=bash at workflow level fixes all steps.
2026-03-29 18:29:04 +00:00
efiten
464e7b3ea4 fix: stop wiping analytics cache on every ingest cycle
The 15s TTL already handles freshness — clearing all cache maps on
every 1-second poll meant entries were never reused, giving 0% server
hit rate and forcing every analytics request back to SQLite.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 18:28:43 +00:00
Kpa-clawbot
ec7ae19bb5 ci: restructure pipeline — sequential fail-fast, Go server E2E, remove deprecated JS tests (#256)
## Summary

Complete CI pipeline restructure. Sequential fail-fast chain, E2E tests
against Go server with real staging data, all deprecated Node.js server
tests removed.

### Pipeline (PR):
1. **Go unit tests** — fail-fast, coverage + badges
2. **Playwright E2E** — against Go server with fixture DB, frontend
coverage, fail-fast on first failure
3. **Docker build** — verify containers build

### Pipeline (master merge):
Same chain + deploy to staging + badge publishing

### Removed:
- All Node.js server-side unit tests (deprecated JS server)
- `npm ci` / `npm run test` steps
- JS server coverage collection (`COVERAGE=1 node server.js`)
- Changed-files detection logic
- Docs-only CI skip logic
- Cancel-workflow API hacks

### Added:
- `test-fixtures/e2e-fixture.db` — real data from staging (200 nodes, 31
observers, 500 packets)
- `scripts/capture-fixture.sh` — refresh fixture from staging API
- Go server launches with `-port 13581 -db test-fixtures/e2e-fixture.db
-public public-instrumented`

---------

Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com>
Co-authored-by: you <you@example.com>
2026-03-29 11:24:22 -07:00
you
75637afcc8 ci: upgrade upload/download-artifact to v6 (Node.js 24) 2026-03-29 18:05:03 +00:00
you
78c5b911e3 test: skip flaky packet detail pane E2E tests (fixes #257) 2026-03-29 17:54:03 +00:00
you
13cab9bede perf: optimize frontend coverage collector (~2x faster)
Three optimizations to reduce wall-clock time:

1. Reduce safeClick timeout from 3000ms to 500ms
   - Elements either exist immediately after navigation or don't exist at all
   - ~75 safeClick calls; if ~30 miss, saves ~75s of dead wait time

2. Replace 18 page.goto() calls with SPA hash navigation
   - After initial page load, the SPA shell is already in the DOM
   - page.goto() reloads the entire page (network round-trip + parse)
   - Hash navigation via location.hash triggers the SPA router instantly
   - Only 3 page.goto() remain: initial load + 2 home page loads after localStorage.clear()

3. Remove redundant final route sweep
   - All 10 routes were already visited during the page-specific sections
   - The sweep just re-navigated to pages that had already been exercised
   - Saves ~2s of redundant navigation

Also:
- Reduce inter-route wait from 200ms to 50ms (SPA router is synchronous)
- Merge utility function + packet filter exercises into single evaluate() call
- Use navHash() helper for consistent hash navigation with 150ms settle time
2026-03-29 10:32:42 -07:00
you
97486cfa21 ci: temporarily disable node-test job (CI restructure in progress) 2026-03-29 17:32:07 +00:00
you
d8ba887514 test: remove Node-specific perf test that fails against Go server
The test 'Node perf page should NOT show Go Runtime section' asserts
Node.js-specific behavior, but E2E tests now run against the Go server
(per this PR), so Go Runtime info is correctly present. Remove the
now-irrelevant assertion.
2026-03-29 10:22:26 -07:00
you
bb43b5696c ci: use Go server instead of Node.js for E2E tests
The Playwright E2E tests were starting `node server.js` (the deprecated
JS server) instead of the Go server, meaning E2E tests weren't testing
the production backend at all.

Changes:
- Add Go 1.22 setup and build steps to the node-test job
- Build the Go server binary before E2E tests run
- Replace `node server.js` with `./corescope-server` in both the
  instrumented (coverage) and quick (no-coverage) E2E server starts
- Use `-port 13581` and `-public` flags to configure the Go server
- For coverage runs, serve from `public-instrumented/` directory

The Go server serves the same static files and exposes compatible
/api/* routes (stats, packets, health, perf) that the E2E tests hit.
2026-03-29 10:22:26 -07:00
you
0f70cd1ac0 feat: make health thresholds configurable in hours
Change healthThresholds config from milliseconds to hours for readability.
Config keys: infraDegradedHours, infraSilentHours, nodeDegradedHours, nodeSilentHours.
Defaults: infra degraded 24h, silent 72h; node degraded 1h, silent 24h.

- Config stored in hours, converted to ms at comparison time
- /api/config/client sends ms to frontend (backward compatible)
- Frontend tooltips use dynamic thresholds instead of hardcoded strings
- Added healthThresholds section to config.example.json
- Updated Go and Node.js servers, tests
2026-03-29 09:50:32 -07:00
Kpa-clawbot
5bb9bc146e docs: remove letsmesh.net reference from README (#233)
* docs: remove letsmesh.net reference from README

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* ci: remove paths-ignore from pull_request trigger

PR #233 only touches .md files, which were excluded by paths-ignore,
causing CI to be skipped entirely. Remove paths-ignore from the
pull_request trigger so all PRs get validated. Keep paths-ignore on
push to avoid unnecessary deploys for docs-only changes to master.

* ci: skip heavy CI jobs for docs-only PRs

Instead of using paths-ignore (which skips the entire workflow and
blocks required status checks), detect docs-only changes at the start
of each job and skip heavy steps while still reporting success.

This allows doc-only PRs to merge without waiting for Go builds,
Node.js tests, or Playwright E2E runs.

Reverts the approach from 7546ece (removing paths-ignore entirely)
in favor of a proper conditional skip within the jobs themselves.

* fix: update engine tests to match engine-badge HTML format

Tests expected [go]/[node] text but formatVersionBadge now renders
<span class="engine-badge">go</span>. Updated 6 assertions to
check for engine-badge class and engine name in HTML output.

---------

Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com>
Co-authored-by: you <you@example.com>
2026-03-29 16:25:51 +00:00
you
12d1174e39 perf: speed up frontend coverage tests (~3x faster)
Three optimizations to the CI frontend test pipeline:

1. Run E2E tests and coverage collection concurrently
   - Previously sequential (E2E ~1.5min, then coverage ~5.75min)
   - Now both run in parallel against the same instrumented server
   - Expected savings: ~5 min (coverage runs alongside E2E instead of after)

2. Replace networkidle with domcontentloaded in coverage collector
   - SPA uses hash routing — networkidle waits 500ms for network silence
     on every navigation, adding ~10-15s of dead time across 23 navigations
   - domcontentloaded fires immediately once HTML is parsed; JS initializes
     the route handler synchronously
   - For in-page hash changes, use 200ms setTimeout instead of
     waitForLoadState (which would never re-fire for same-document nav)

3. Extract coverage from E2E tests too
   - E2E tests already exercise the app against the instrumented server
   - Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
   - nyc merges both coverage files for higher total coverage

Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
2026-03-29 09:12:23 -07:00
you
3bbd986d41 fix: add sleep before poller data insert to prevent race condition in tests
The poller's Start() calls GetMaxTransmissionID() to initialize its cursor.
When the test goroutine inserts data between go poller.Start() and the
actual GetMaxTransmissionID() call, the poller's cursor skips past the
test data and never broadcasts it, causing a timeout.

Adding a 100ms sleep after go poller.Start() ensures the poller has
initialized its cursors before the test inserts new data.
2026-03-29 08:32:37 -07:00
you
712fa15a8c fix: force single SQLite connection in test DBs to prevent in-memory table visibility issues
SQLite :memory: databases create separate databases per connection.
When the connection pool opens multiple connections (e.g. poller goroutine
vs main test goroutine), tables created on one connection are invisible
to others. Setting MaxOpenConns(1) ensures all queries use the same
in-memory database, fixing TestPollerBroadcastsMultipleObservations.
2026-03-29 08:32:37 -07:00
Kpa-clawbot
ab03b142f5 fix: per-observation WS broadcast for live view starburst — fixes #237
IngestNewFromDB now broadcasts one message per observation (not per
transmission). IngestNewObservations also broadcasts late arrivals.
Tests verify multi-observer packets produce multiple WS messages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-29 08:32:37 -07:00
you
def95aae64 fix: align packet decoder with MeshCore firmware spec
Compared decoder.js against the MeshCore firmware source (Dispatcher.cpp,
Packet.h, Mesh.cpp, AdvertDataHelpers.h) and fixed all mismatches:

1. Field order: transport codes now parsed BEFORE path_length byte,
   matching the spec: [header][transport_codes?][path_length][path][payload]

2. ACK payload: was incorrectly decoded as dest(1)+src(1)+ackHash(4).
   Firmware shows ACK is just checksum(4) — no dest/src hashes.

3. TRACE payload: was incorrectly decoded as flags(1)+tag(4)+dest(6)+src(1).
   Firmware shows tag(4)+authCode(4)+flags(1)+pathData.

4. ADVERT appdata: added missing feature1 (0x20 flag) and feature2
   (0x40 flag) parsing — 2-byte fields between location and name.

5. Transport code field naming: renamed nextHop/lastHop to code1/code2
   to match spec terminology (transport_code_1/transport_code_2).

6. Fixed incorrect field size labels in packets.js hex breakdown:
   dest/src are 1 byte, MAC is 2 bytes (not 6B/6B/4B).

7. Fixed ANON_REQ/PATH comment typos (dest was listed as 6 bytes,
   MAC as 4 bytes — both wrong, code was already correct).

All 329 tests pass (66 decoder + 263 spec/golden).
2026-03-29 08:32:16 -07:00
you
1b09c733f5 ci: restrict self-hosted jobs to Linux runners
The Windows self-hosted runner picks up jobs and fails because bash
scripts run in PowerShell. Node.js tests need Chromium/Playwright
(Linux-only), and build/deploy/publish use Docker (Linux-only).

Changes:
- node-test: runs-on: [self-hosted, Linux]
- build: runs-on: [self-hosted, Linux]
- deploy: runs-on: [self-hosted, Linux]
- publish: runs-on: [self-hosted, Linux]
- go-test: unchanged (ubuntu-latest)
2026-03-29 14:58:15 +00:00
Kpa-clawbot
553c0e4963 ci: bump GitHub Actions to Node 24 compatible versions
checkout v4→v5, setup-go v5→v6, setup-node v4→v5,
upload-artifact v4→v5, download-artifact v4→v5

Fixes the Node.js 20 deprecation warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-29 07:51:48 -07:00
efiten
8ede8427c8 fix: round Go Runtime floats to 1dp, prevent nav stats dot wrapping
- perf.js: toFixed(1) on all ms/MB values in Go Runtime section
- style.css: white-space: nowrap on .nav-stats to prevent the · separator
  from wrapping onto its own line

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 07:51:26 -07:00
you
8e66c68d6f fix: cache hit rate excludes stale hits + debounce bulk-health invalidation
Two cache bugs fixed:

1. Hit rate formula excluded stale hits — reported rate was artificially low
   because stale-while-revalidate responses (which ARE cache hits from the
   caller's perspective) were not counted. Changed formula from
   hits/(hits+misses) to (hits+staleHits)/(hits+staleHits+misses).

2. Bulk-health cache invalidated on every advert packet — in a mesh with
   dozens of nodes advertising every few seconds, this caused the expensive
   bulk-health query to be recomputed on nearly every request, defeating
   the cache entirely. Switched to 30s debounced invalidation via
   debouncedInvalidateBulkHealth().

Added regression test for hit rate formula in test-server-routes.js.
2026-03-29 07:51:08 -07:00
you
37396823ad fix: align Go packet decoder with MeshCore firmware spec
Match the C++ firmware wire format (Packet::writeTo/readFrom):

1. Field order: transport codes are parsed BEFORE path_length byte,
   matching firmware's header → transport_codes → path_len → path → payload

2. ACK payload: just 4-byte CRC checksum, not dest+src+ackHash.
   Firmware createAck() writes only ack_crc (4 bytes).

3. TRACE payload: tag(4) + authCode(4) + flags(1) + pathData,
   matching firmware createTrace() and onRecvPacket() TRACE handler.

4. ADVERT features: parse feat1 (0x20) and feat2 (0x40) optional
   2-byte fields between location and name, matching AdvertDataBuilder
   and AdvertDataParser in the firmware.

5. Transport code naming: code1/code2 instead of nextHop/lastHop,
   matching firmware's transport_codes[0]/transport_codes[1] naming.

Fixes applied to both cmd/ingestor/decoder.go and cmd/server/decoder.go.
Tests updated to match new behavior.
2026-03-29 07:50:51 -07:00
you
074f3d3760 ci: cancel workflow run immediately when any test job fails
When go-test or node-test fails, the workflow run is now cancelled
via the GitHub API so the sibling job doesn't sit queued/running.

Also fixed build job to need both go-test AND node-test (was only
waiting on go-test despite the pipeline comment saying both gate it).
2026-03-29 14:20:22 +00:00
you
206d9bd64a fix: use per-PR concurrency group to prevent cross-PR cancellation
The flat 'deploy' concurrency group caused ALL PRs to share one queue,
so pushing to any PR would cancel CI runs on other PRs.

Changed to deploy-${{ github.event.pull_request.number || github.ref }}
so each PR gets its own concurrency group while re-pushes to the same
PR still cancel the previous run.
2026-03-29 14:14:57 +00:00
efiten
3f54632b07 fix: cache /stats and GetNodeHashSizeInfo to eliminate slow API calls
- /api/stats: 10s server-side cache — was running 5 SQLite COUNT queries
  on every call, taking ~1500ms with 28 concurrent WS clients polling every 15s
- GetNodeHashSizeInfo: 15s cache — was doing a full O(n) scan + JSON unmarshal
  of all advert packets in memory on every /nodes request, taking ~1200ms

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 07:09:05 -07:00
Kpa-clawbot
609b12541e fix: add extra_hosts host.docker.internal to all services — fixes #238
Linux Docker doesn't resolve host.docker.internal by default.
Required when MQTT sources in config.json point to the host machine.
Harmless on Docker Desktop where it already works.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 18:58:31 -07:00
Kpa-clawbot
4369e58a3c Merge pull request #235 from Kpa-clawbot/fix/compose-build-directive
fix: docker-compose prod/staging need build: directive — fixes pull access denied
2026-03-28 18:36:21 -07:00
Kpa-clawbot
8ef321bf70 fix: add build context to prod and staging services in docker-compose.yml
Without build: directive, docker compose tries to pull corescope:latest
from Docker Hub instead of building locally.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 18:35:35 -07:00
34 changed files with 5953 additions and 5397 deletions

View File

@@ -1,393 +1,324 @@
name: Deploy
on:
push:
branches: [master]
paths-ignore:
- '**.md'
- 'LICENSE'
- '.gitignore'
- 'docs/**'
pull_request:
branches: [master]
paths-ignore:
- '**.md'
- 'LICENSE'
- '.gitignore'
- 'docs/**'
concurrency:
group: deploy
cancel-in-progress: true
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
# Pipeline:
# node-test (frontend tests) ──
# go-test ├──→ build → deploy → publish
# └─ (both wait)
#
# Proto validation flow:
# 1. go-test job: verify .proto files compile (syntax check)
# 2. deploy job: capture fresh fixtures from prod, validate protos match actual API responses
jobs:
# ───────────────────────────────────────────────────────────────
# 1. Go Build & Test — compiles + tests Go modules, coverage badges
# ───────────────────────────────────────────────────────────────
go-test:
name: "✅ Go Build & Test"
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go 1.22
uses: actions/setup-go@v5
with:
go-version: '1.22'
cache-dependency-path: |
cmd/server/go.sum
cmd/ingestor/go.sum
- name: Build and test Go server (with coverage)
run: |
set -e -o pipefail
cd cmd/server
go build .
go test -coverprofile=server-coverage.out ./... 2>&1 | tee server-test.log
echo "--- Go Server Coverage ---"
go tool cover -func=server-coverage.out | tail -1
- name: Build and test Go ingestor (with coverage)
run: |
set -e -o pipefail
cd cmd/ingestor
go build .
go test -coverprofile=ingestor-coverage.out ./... 2>&1 | tee ingestor-test.log
echo "--- Go Ingestor Coverage ---"
go tool cover -func=ingestor-coverage.out | tail -1
- name: Verify proto syntax (all .proto files compile)
run: |
set -e
echo "Installing protoc..."
sudo apt-get update -qq
sudo apt-get install -y protobuf-compiler
echo "Checking proto syntax..."
for proto in proto/*.proto; do
echo " ✓ $(basename "$proto")"
protoc --proto_path=proto --descriptor_set_out=/dev/null "$proto"
done
echo "✅ All .proto files are syntactically valid"
- name: Generate Go coverage badges
if: always()
run: |
mkdir -p .badges
# Parse server coverage
SERVER_COV="0"
if [ -f cmd/server/server-coverage.out ]; then
SERVER_COV=$(cd cmd/server && go tool cover -func=server-coverage.out | tail -1 | grep -oP '[\d.]+(?=%)')
fi
SERVER_COLOR="red"
if [ "$(echo "$SERVER_COV >= 80" | bc -l 2>/dev/null)" = "1" ]; then
SERVER_COLOR="green"
elif [ "$(echo "$SERVER_COV >= 60" | bc -l 2>/dev/null)" = "1" ]; then
SERVER_COLOR="yellow"
fi
echo "{\"schemaVersion\":1,\"label\":\"go server coverage\",\"message\":\"${SERVER_COV}%\",\"color\":\"${SERVER_COLOR}\"}" > .badges/go-server-coverage.json
echo "Go server coverage: ${SERVER_COV}% (${SERVER_COLOR})"
# Parse ingestor coverage
INGESTOR_COV="0"
if [ -f cmd/ingestor/ingestor-coverage.out ]; then
INGESTOR_COV=$(cd cmd/ingestor && go tool cover -func=ingestor-coverage.out | tail -1 | grep -oP '[\d.]+(?=%)')
fi
INGESTOR_COLOR="red"
if [ "$(echo "$INGESTOR_COV >= 80" | bc -l 2>/dev/null)" = "1" ]; then
INGESTOR_COLOR="green"
elif [ "$(echo "$INGESTOR_COV >= 60" | bc -l 2>/dev/null)" = "1" ]; then
INGESTOR_COLOR="yellow"
fi
echo "{\"schemaVersion\":1,\"label\":\"go ingestor coverage\",\"message\":\"${INGESTOR_COV}%\",\"color\":\"${INGESTOR_COLOR}\"}" > .badges/go-ingestor-coverage.json
echo "Go ingestor coverage: ${INGESTOR_COV}% (${INGESTOR_COLOR})"
echo "## Go Coverage" >> $GITHUB_STEP_SUMMARY
echo "| Module | Coverage |" >> $GITHUB_STEP_SUMMARY
echo "|--------|----------|" >> $GITHUB_STEP_SUMMARY
echo "| Server | ${SERVER_COV}% |" >> $GITHUB_STEP_SUMMARY
echo "| Ingestor | ${INGESTOR_COV}% |" >> $GITHUB_STEP_SUMMARY
- name: Upload Go coverage badges
if: always()
uses: actions/upload-artifact@v4
with:
name: go-badges
path: .badges/go-*.json
retention-days: 1
if-no-files-found: ignore
# ───────────────────────────────────────────────────────────────
# 2. Node.js Tests — backend unit tests + Playwright E2E, coverage
# ───────────────────────────────────────────────────────────────
node-test:
name: "🧪 Node.js Tests"
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Set up Node.js 22
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install npm dependencies
run: npm ci --production=false
- name: Detect changed files
id: changes
run: |
BACKEND=$(git diff --name-only HEAD~1 | grep -cE '^(server|db|decoder|packet-store|server-helpers|iata-coords)\.js$' || true)
FRONTEND=$(git diff --name-only HEAD~1 | grep -cE '^public/' || true)
TESTS=$(git diff --name-only HEAD~1 | grep -cE '^test-|^tools/' || true)
CI=$(git diff --name-only HEAD~1 | grep -cE '\.github/|package\.json|test-all\.sh|scripts/' || true)
# If CI/test infra changed, run everything
if [ "$CI" -gt 0 ]; then BACKEND=1; FRONTEND=1; fi
# If test files changed, run everything
if [ "$TESTS" -gt 0 ]; then BACKEND=1; FRONTEND=1; fi
echo "backend=$([[ $BACKEND -gt 0 ]] && echo true || echo false)" >> $GITHUB_OUTPUT
echo "frontend=$([[ $FRONTEND -gt 0 ]] && echo true || echo false)" >> $GITHUB_OUTPUT
echo "Changes: backend=$BACKEND frontend=$FRONTEND tests=$TESTS ci=$CI"
- name: Run backend tests with coverage
if: steps.changes.outputs.backend == 'true'
run: |
npx c8 --reporter=text-summary --reporter=text sh test-all.sh 2>&1 | tee test-output.txt
TOTAL_PASS=$(grep -oP '\d+(?= passed)' test-output.txt | awk '{s+=$1} END {print s}')
TOTAL_FAIL=$(grep -oP '\d+(?= failed)' test-output.txt | awk '{s+=$1} END {print s}')
BE_COVERAGE=$(grep 'Statements' test-output.txt | tail -1 | grep -oP '[\d.]+(?=%)')
mkdir -p .badges
BE_COLOR="red"
[ "$(echo "$BE_COVERAGE > 60" | bc -l 2>/dev/null)" = "1" ] && BE_COLOR="yellow"
[ "$(echo "$BE_COVERAGE > 80" | bc -l 2>/dev/null)" = "1" ] && BE_COLOR="brightgreen"
echo "{\"schemaVersion\":1,\"label\":\"backend tests\",\"message\":\"${TOTAL_PASS} passed\",\"color\":\"brightgreen\"}" > .badges/backend-tests.json
echo "{\"schemaVersion\":1,\"label\":\"backend coverage\",\"message\":\"${BE_COVERAGE}%\",\"color\":\"${BE_COLOR}\"}" > .badges/backend-coverage.json
echo "## Backend: ${TOTAL_PASS} tests, ${BE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY
- name: Run backend tests (quick, no coverage)
if: steps.changes.outputs.backend == 'false'
run: npm run test:unit
- name: Install Playwright browser
if: steps.changes.outputs.frontend == 'true'
run: npx playwright install chromium --with-deps 2>/dev/null || true
- name: Instrument frontend JS for coverage
if: steps.changes.outputs.frontend == 'true'
run: sh scripts/instrument-frontend.sh
- name: Start instrumented test server on port 13581
if: steps.changes.outputs.frontend == 'true'
run: |
# Kill any stale server on 13581
fuser -k 13581/tcp 2>/dev/null || true
sleep 2
COVERAGE=1 PORT=13581 node server.js &
echo $! > .server.pid
echo "Server PID: $(cat .server.pid)"
# Health-check poll loop (up to 30s)
for i in $(seq 1 30); do
if curl -sf http://localhost:13581/api/stats > /dev/null 2>&1; then
echo "Server ready after ${i}s"
break
fi
if [ "$i" -eq 30 ]; then
echo "Server failed to start within 30s"
echo "Last few lines from server logs:"
ps aux | grep "PORT=13581" || echo "No server process found"
exit 1
fi
sleep 1
done
- name: Run Playwright E2E tests
if: steps.changes.outputs.frontend == 'true'
run: BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt
- name: Collect frontend coverage report
if: always() && steps.changes.outputs.frontend == 'true'
run: |
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1)
mkdir -p .badges
if [ -f .nyc_output/frontend-coverage.json ]; then
npx nyc report --reporter=text-summary --reporter=text 2>&1 | tee fe-report.txt
FE_COVERAGE=$(grep 'Statements' fe-report.txt | head -1 | grep -oP '[\d.]+(?=%)' || echo "0")
FE_COVERAGE=${FE_COVERAGE:-0}
FE_COLOR="red"
[ "$(echo "$FE_COVERAGE > 50" | bc -l 2>/dev/null)" = "1" ] && FE_COLOR="yellow"
[ "$(echo "$FE_COVERAGE > 80" | bc -l 2>/dev/null)" = "1" ] && FE_COLOR="brightgreen"
echo "{\"schemaVersion\":1,\"label\":\"frontend coverage\",\"message\":\"${FE_COVERAGE}%\",\"color\":\"${FE_COLOR}\"}" > .badges/frontend-coverage.json
echo "## Frontend: ${FE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY
fi
echo "{\"schemaVersion\":1,\"label\":\"frontend tests\",\"message\":\"${E2E_PASS:-0} E2E passed\",\"color\":\"brightgreen\"}" > .badges/frontend-tests.json
- name: Stop test server
if: always() && steps.changes.outputs.frontend == 'true'
run: |
if [ -f .server.pid ]; then
kill $(cat .server.pid) 2>/dev/null || true
rm -f .server.pid
echo "Server stopped"
fi
- name: Run frontend E2E (quick, no coverage)
if: steps.changes.outputs.frontend == 'false'
run: |
fuser -k 13581/tcp 2>/dev/null || true
PORT=13581 node server.js &
SERVER_PID=$!
sleep 5
BASE_URL=http://localhost:13581 node test-e2e-playwright.js || true
kill $SERVER_PID 2>/dev/null || true
- name: Upload Node.js test badges
if: always()
uses: actions/upload-artifact@v4
with:
name: node-badges
path: .badges/
retention-days: 1
if-no-files-found: ignore
# ───────────────────────────────────────────────────────────────
# 3. Build Docker Image
# ───────────────────────────────────────────────────────────────
build:
name: "🏗️ Build Docker Image"
if: github.event_name == 'push'
needs: [go-test]
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node.js 22
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Build Go Docker image
run: |
echo "${GITHUB_SHA::7}" > .git-commit
APP_VERSION=$(node -p "require('./package.json').version") \
GIT_COMMIT="${GITHUB_SHA::7}" \
docker compose --profile staging-go build staging-go
echo "Built Go staging image"
# ───────────────────────────────────────────────────────────────
# 4. Deploy Staging — start on port 82, healthcheck, smoke test
# ───────────────────────────────────────────────────────────────
deploy:
name: "🚀 Deploy Staging"
if: github.event_name == 'push'
needs: [build]
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Start staging on port 82
run: |
# Force remove stale containers
docker rm -f corescope-staging-go 2>/dev/null || true
# Clean up stale ports
fuser -k 82/tcp 2>/dev/null || true
docker compose --profile staging-go up -d staging-go
- name: Healthcheck staging container
run: |
for i in $(seq 1 120); do
HEALTH=$(docker inspect corescope-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
if [ "$HEALTH" = "healthy" ]; then
echo "Staging healthy after ${i}s"
break
fi
if [ "$i" -eq 120 ]; then
echo "Staging failed health check after 120s"
docker logs corescope-staging-go --tail 50
exit 1
fi
sleep 1
done
- name: Smoke test staging API
run: |
if curl -sf http://localhost:82/api/stats | grep -q engine; then
echo "Staging verified — engine field present ✅"
else
echo "Staging /api/stats did not return engine field"
exit 1
fi
# ───────────────────────────────────────────────────────────────
# 5. Publish Badges & Summary
# ───────────────────────────────────────────────────────────────
publish:
name: "📝 Publish Badges & Summary"
if: github.event_name == 'push'
needs: [deploy]
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Download Go coverage badges
continue-on-error: true
uses: actions/download-artifact@v4
with:
name: go-badges
path: .badges/
- name: Download Node.js test badges
continue-on-error: true
uses: actions/download-artifact@v4
with:
name: node-badges
path: .badges/
- name: Publish coverage badges to repo
continue-on-error: true
run: |
git config user.name "github-actions"
git config user.email "actions@github.com"
git remote set-url origin https://x-access-token:${{ github.token }}@github.com/${{ github.repository }}.git
git add .badges/ -f
git diff --cached --quiet || (git commit -m "ci: update test badges [skip ci]" && git push) || echo "Badge push failed"
- name: Post deployment summary
run: |
echo "## Staging Deployed ✓" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Commit:** \`$(git rev-parse --short HEAD)\` — $(git log -1 --format=%s)" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Staging:** http://<VM_HOST>:82" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "To promote to production:" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
echo "ssh deploy@\$VM_HOST" >> $GITHUB_STEP_SUMMARY
echo "cd /opt/corescope-deploy" >> $GITHUB_STEP_SUMMARY
echo "./manage.sh promote" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
name: CI/CD Pipeline
on:
push:
branches: [master]
pull_request:
branches: [master]
concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
# Pipeline (sequential, fail-fast):
# go-test → e2e-test → build → deploy → publish
# PRs stop after build. Master continues to deploy + publish.
jobs:
# ───────────────────────────────────────────────────────────────
# 1. Go Build & Test
# ───────────────────────────────────────────────────────────────
go-test:
name: "✅ Go Build & Test"
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Set up Go 1.22
uses: actions/setup-go@v6
with:
go-version: '1.22'
cache-dependency-path: |
cmd/server/go.sum
cmd/ingestor/go.sum
- name: Build and test Go server (with coverage)
run: |
set -e -o pipefail
cd cmd/server
go build .
go test -coverprofile=server-coverage.out ./... 2>&1 | tee server-test.log
echo "--- Go Server Coverage ---"
go tool cover -func=server-coverage.out | tail -1
- name: Build and test Go ingestor (with coverage)
run: |
set -e -o pipefail
cd cmd/ingestor
go build .
go test -coverprofile=ingestor-coverage.out ./... 2>&1 | tee ingestor-test.log
echo "--- Go Ingestor Coverage ---"
go tool cover -func=ingestor-coverage.out | tail -1
- name: Verify proto syntax
run: |
set -e
sudo apt-get update -qq
sudo apt-get install -y protobuf-compiler
for proto in proto/*.proto; do
echo " ✓ $(basename "$proto")"
protoc --proto_path=proto --descriptor_set_out=/dev/null "$proto"
done
echo "✅ All .proto files are syntactically valid"
- name: Generate Go coverage badges
if: success()
run: |
mkdir -p .badges
SERVER_COV="0"
if [ -f cmd/server/server-coverage.out ]; then
SERVER_COV=$(cd cmd/server && go tool cover -func=server-coverage.out | tail -1 | grep -oP '[\d.]+(?=%)')
fi
SERVER_COLOR="red"
if [ "$(echo "$SERVER_COV >= 80" | bc -l 2>/dev/null)" = "1" ]; then SERVER_COLOR="green"
elif [ "$(echo "$SERVER_COV >= 60" | bc -l 2>/dev/null)" = "1" ]; then SERVER_COLOR="yellow"; fi
echo "{\"schemaVersion\":1,\"label\":\"go server coverage\",\"message\":\"${SERVER_COV}%\",\"color\":\"${SERVER_COLOR}\"}" > .badges/go-server-coverage.json
INGESTOR_COV="0"
if [ -f cmd/ingestor/ingestor-coverage.out ]; then
INGESTOR_COV=$(cd cmd/ingestor && go tool cover -func=ingestor-coverage.out | tail -1 | grep -oP '[\d.]+(?=%)')
fi
INGESTOR_COLOR="red"
if [ "$(echo "$INGESTOR_COV >= 80" | bc -l 2>/dev/null)" = "1" ]; then INGESTOR_COLOR="green"
elif [ "$(echo "$INGESTOR_COV >= 60" | bc -l 2>/dev/null)" = "1" ]; then INGESTOR_COLOR="yellow"; fi
echo "{\"schemaVersion\":1,\"label\":\"go ingestor coverage\",\"message\":\"${INGESTOR_COV}%\",\"color\":\"${INGESTOR_COLOR}\"}" > .badges/go-ingestor-coverage.json
echo "## Go Coverage" >> $GITHUB_STEP_SUMMARY
echo "| Module | Coverage |" >> $GITHUB_STEP_SUMMARY
echo "|--------|----------|" >> $GITHUB_STEP_SUMMARY
echo "| Server | ${SERVER_COV}% |" >> $GITHUB_STEP_SUMMARY
echo "| Ingestor | ${INGESTOR_COV}% |" >> $GITHUB_STEP_SUMMARY
- name: Upload Go coverage badges
if: success()
uses: actions/upload-artifact@v6
with:
name: go-badges
path: .badges/go-*.json
retention-days: 1
if-no-files-found: ignore
# ───────────────────────────────────────────────────────────────
# 2. Playwright E2E Tests (against Go server with fixture DB)
# ───────────────────────────────────────────────────────────────
e2e-test:
name: "🎭 Playwright E2E Tests"
needs: [go-test]
runs-on: [self-hosted, Linux]
defaults:
run:
shell: bash
steps:
- name: Checkout code
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Set up Node.js 22
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Set up Go 1.22
uses: actions/setup-go@v6
with:
go-version: '1.22'
cache-dependency-path: cmd/server/go.sum
- name: Build Go server
run: |
cd cmd/server
go build -o ../../corescope-server .
echo "Go server built successfully"
- name: Install npm dependencies
run: npm ci --production=false
- name: Install Playwright browser
run: |
npx playwright install chromium 2>/dev/null || true
npx playwright install-deps chromium 2>/dev/null || true
- name: Instrument frontend JS for coverage
run: sh scripts/instrument-frontend.sh
- name: Start Go server with fixture DB
run: |
fuser -k 13581/tcp 2>/dev/null || true
sleep 1
./corescope-server -port 13581 -db test-fixtures/e2e-fixture.db -public public-instrumented &
echo $! > .server.pid
for i in $(seq 1 30); do
if curl -sf http://localhost:13581/api/stats > /dev/null 2>&1; then
echo "Server ready after ${i}s"
break
fi
if [ "$i" -eq 30 ]; then
echo "Server failed to start within 30s"
exit 1
fi
sleep 1
done
- name: Run Playwright E2E tests (fail-fast)
run: |
BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt
- name: Collect frontend coverage
if: success()
run: |
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt || true
- name: Generate frontend coverage badges
if: success()
run: |
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1 || echo "0")
mkdir -p .badges
if [ -f .nyc_output/frontend-coverage.json ] || [ -f .nyc_output/e2e-coverage.json ]; then
npx nyc report --reporter=text-summary --reporter=text 2>&1 | tee fe-report.txt
FE_COVERAGE=$(grep 'Statements' fe-report.txt | head -1 | grep -oP '[\d.]+(?=%)' || echo "0")
FE_COVERAGE=${FE_COVERAGE:-0}
FE_COLOR="red"
[ "$(echo "$FE_COVERAGE > 50" | bc -l 2>/dev/null)" = "1" ] && FE_COLOR="yellow"
[ "$(echo "$FE_COVERAGE > 80" | bc -l 2>/dev/null)" = "1" ] && FE_COLOR="brightgreen"
echo "{\"schemaVersion\":1,\"label\":\"frontend coverage\",\"message\":\"${FE_COVERAGE}%\",\"color\":\"${FE_COLOR}\"}" > .badges/frontend-coverage.json
echo "## Frontend: ${FE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY
fi
echo "{\"schemaVersion\":1,\"label\":\"e2e tests\",\"message\":\"${E2E_PASS:-0} passed\",\"color\":\"brightgreen\"}" > .badges/e2e-tests.json
- name: Stop test server
if: success()
run: |
if [ -f .server.pid ]; then
kill $(cat .server.pid) 2>/dev/null || true
rm -f .server.pid
fi
- name: Upload E2E badges
if: success()
uses: actions/upload-artifact@v6
with:
name: e2e-badges
path: .badges/
retention-days: 1
if-no-files-found: ignore
# ───────────────────────────────────────────────────────────────
# 3. Build Docker Image
# ───────────────────────────────────────────────────────────────
build:
name: "🏗️ Build Docker Image"
needs: [e2e-test]
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Set up Node.js 22
uses: actions/setup-node@v5
with:
node-version: '22'
- name: Build Go Docker image
run: |
echo "${GITHUB_SHA::7}" > .git-commit
APP_VERSION=$(node -p "require('./package.json').version") \
GIT_COMMIT="${GITHUB_SHA::7}" \
docker compose --profile staging-go build staging-go
echo "Built Go staging image ✅"
# ───────────────────────────────────────────────────────────────
# 4. Deploy Staging (master only)
# ───────────────────────────────────────────────────────────────
deploy:
name: "🚀 Deploy Staging"
if: github.event_name == 'push'
needs: [build]
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Start staging on port 82
run: |
docker rm -f corescope-staging-go 2>/dev/null || true
fuser -k 82/tcp 2>/dev/null || true
docker compose --profile staging-go up -d staging-go
- name: Healthcheck staging container
run: |
for i in $(seq 1 120); do
HEALTH=$(docker inspect corescope-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
if [ "$HEALTH" = "healthy" ]; then
echo "Staging healthy after ${i}s"
break
fi
if [ "$i" -eq 120 ]; then
echo "Staging failed health check after 120s"
docker logs corescope-staging-go --tail 50
exit 1
fi
sleep 1
done
- name: Smoke test staging API
run: |
if curl -sf http://localhost:82/api/stats | grep -q engine; then
echo "Staging verified — engine field present ✅"
else
echo "Staging /api/stats did not return engine field"
exit 1
fi
# ───────────────────────────────────────────────────────────────
# 5. Publish Badges & Summary (master only)
# ───────────────────────────────────────────────────────────────
publish:
name: "📝 Publish Badges & Summary"
if: github.event_name == 'push'
needs: [deploy]
runs-on: [self-hosted, Linux]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Download Go coverage badges
continue-on-error: true
uses: actions/download-artifact@v6
with:
name: go-badges
path: .badges/
- name: Download E2E badges
continue-on-error: true
uses: actions/download-artifact@v6
with:
name: e2e-badges
path: .badges/
- name: Publish coverage badges to repo
continue-on-error: true
run: |
git config user.name "github-actions"
git config user.email "actions@github.com"
git remote set-url origin https://x-access-token:${{ github.token }}@github.com/${{ github.repository }}.git
git add .badges/ -f
git diff --cached --quiet || (git commit -m "ci: update test badges [skip ci]" && git push) || echo "Badge push failed"
- name: Post deployment summary
run: |
echo "## Staging Deployed ✓" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Commit:** \`$(git rev-parse --short HEAD)\` — $(git log -1 --format=%s)" >> $GITHUB_STEP_SUMMARY

1
.gitignore vendored
View File

@@ -28,3 +28,4 @@ reps.txt
cmd/server/server.exe
cmd/ingestor/ingestor.exe
# CI trigger
!test-fixtures/e2e-fixture.db

View File

@@ -8,7 +8,7 @@
> High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption.
Self-hosted, open-source MeshCore packet analyzer — a community alternative to the closed-source `analyzer.letsmesh.net`. Collects MeshCore packets via MQTT, decodes them in real time, and presents a full web UI with live packet feed, interactive maps, channel chat, packet tracing, and per-node analytics.
Self-hosted, open-source MeshCore packet analyzer. Collects MeshCore packets via MQTT, decodes them in real time, and presents a full web UI with live packet feed, interactive maps, channel chat, packet tracing, and per-node analytics.
## ⚡ Performance

View File

@@ -72,8 +72,8 @@ type Header struct {
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
type TransportCodes struct {
NextHop string `json:"nextHop"`
LastHop string `json:"lastHop"`
Code1 string `json:"code1"`
Code2 string `json:"code2"`
}
// Path holds decoded path/hop information.
@@ -92,6 +92,8 @@ type AdvertFlags struct {
Room bool `json:"room"`
Sensor bool `json:"sensor"`
HasLocation bool `json:"hasLocation"`
HasFeat1 bool `json:"hasFeat1"`
HasFeat2 bool `json:"hasFeat2"`
HasName bool `json:"hasName"`
}
@@ -111,6 +113,8 @@ type Payload struct {
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
Name string `json:"name,omitempty"`
Feat1 *int `json:"feat1,omitempty"`
Feat2 *int `json:"feat2,omitempty"`
BatteryMv *int `json:"battery_mv,omitempty"`
TemperatureC *float64 `json:"temperature_c,omitempty"`
ChannelHash int `json:"channelHash,omitempty"`
@@ -123,6 +127,8 @@ type Payload struct {
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
PathData string `json:"pathData,omitempty"`
Tag uint32 `json:"tag,omitempty"`
AuthCode uint32 `json:"authCode,omitempty"`
TraceFlags *int `json:"traceFlags,omitempty"`
RawHex string `json:"raw,omitempty"`
Error string `json:"error,omitempty"`
}
@@ -199,14 +205,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
}
func decodeAck(buf []byte) Payload {
if len(buf) < 6 {
if len(buf) < 4 {
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
}
checksum := binary.LittleEndian.Uint32(buf[0:4])
return Payload{
Type: "ACK",
DestHash: hex.EncodeToString(buf[0:1]),
SrcHash: hex.EncodeToString(buf[1:2]),
ExtraHash: hex.EncodeToString(buf[2:6]),
ExtraHash: fmt.Sprintf("%08x", checksum),
}
}
@@ -231,6 +236,8 @@ func decodeAdvert(buf []byte) Payload {
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
hasFeat1 := flags&0x20 != 0
hasFeat2 := flags&0x40 != 0
p.Flags = &AdvertFlags{
Raw: int(flags),
Type: advType,
@@ -239,6 +246,8 @@ func decodeAdvert(buf []byte) Payload {
Room: advType == 3,
Sensor: advType == 4,
HasLocation: flags&0x10 != 0,
HasFeat1: hasFeat1,
HasFeat2: hasFeat2,
HasName: flags&0x80 != 0,
}
@@ -252,6 +261,16 @@ func decodeAdvert(buf []byte) Payload {
p.Lon = &lon
off += 8
}
if hasFeat1 && len(appdata) >= off+2 {
feat1 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
p.Feat1 = &feat1
off += 2
}
if hasFeat2 && len(appdata) >= off+2 {
feat2 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
p.Feat2 = &feat2
off += 2
}
if p.Flags.HasName {
// Find null terminator to separate name from trailing telemetry bytes
nameEnd := len(appdata)
@@ -469,15 +488,22 @@ func decodePathPayload(buf []byte) Payload {
}
func decodeTrace(buf []byte) Payload {
if len(buf) < 12 {
if len(buf) < 9 {
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
}
return Payload{
Type: "TRACE",
DestHash: hex.EncodeToString(buf[5:11]),
SrcHash: hex.EncodeToString(buf[11:12]),
Tag: binary.LittleEndian.Uint32(buf[1:5]),
tag := binary.LittleEndian.Uint32(buf[0:4])
authCode := binary.LittleEndian.Uint32(buf[4:8])
flags := int(buf[8])
p := Payload{
Type: "TRACE",
Tag: tag,
AuthCode: authCode,
TraceFlags: &flags,
}
if len(buf) > 9 {
p.PathData = hex.EncodeToString(buf[9:])
}
return p
}
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload {
@@ -520,8 +546,7 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
}
header := decodeHeader(buf[0])
pathByte := buf[1]
offset := 2
offset := 1
var tc *TransportCodes
if isTransportRoute(header.RouteType) {
@@ -529,12 +554,18 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
return nil, fmt.Errorf("packet too short for transport codes")
}
tc = &TransportCodes{
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
}
offset += 4
}
if offset >= len(buf) {
return nil, fmt.Errorf("packet too short (no path byte)")
}
pathByte := buf[offset]
offset++
path, bytesConsumed := decodePath(pathByte, buf, offset)
offset += bytesConsumed
@@ -562,16 +593,24 @@ func ComputeContentHash(rawHex string) string {
return rawHex
}
pathByte := buf[1]
headerByte := buf[0]
offset := 1
if isTransportRoute(int(headerByte & 0x03)) {
offset += 4
}
if offset >= len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]
}
return rawHex
}
pathByte := buf[offset]
offset++
hashSize := int((pathByte>>6)&0x3) + 1
hashCount := int(pathByte & 0x3F)
pathBytes := hashSize * hashCount
headerByte := buf[0]
payloadStart := 2 + pathBytes
if isTransportRoute(int(headerByte & 0x03)) {
payloadStart += 4
}
payloadStart := offset + pathBytes
if payloadStart > len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]

View File

@@ -129,7 +129,8 @@ func TestDecodePath3ByteHashes(t *testing.T) {
func TestTransportCodes(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) should have transport codes
hex := "1400" + "AABB" + "CCDD" + "1A" + strings.Repeat("00", 10)
// Firmware order: header + transport_codes(4) + path_len + path + payload
hex := "14" + "AABB" + "CCDD" + "00" + strings.Repeat("00", 10)
pkt, err := DecodePacket(hex, nil)
if err != nil {
t.Fatal(err)
@@ -140,11 +141,11 @@ func TestTransportCodes(t *testing.T) {
if pkt.TransportCodes == nil {
t.Fatal("transportCodes should not be nil for TRANSPORT_FLOOD")
}
if pkt.TransportCodes.NextHop != "AABB" {
t.Errorf("nextHop=%s, want AABB", pkt.TransportCodes.NextHop)
if pkt.TransportCodes.Code1 != "AABB" {
t.Errorf("code1=%s, want AABB", pkt.TransportCodes.Code1)
}
if pkt.TransportCodes.LastHop != "CCDD" {
t.Errorf("lastHop=%s, want CCDD", pkt.TransportCodes.LastHop)
if pkt.TransportCodes.Code2 != "CCDD" {
t.Errorf("code2=%s, want CCDD", pkt.TransportCodes.Code2)
}
// Route type 1 (FLOOD) should NOT have transport codes
@@ -537,10 +538,11 @@ func TestDecodeTraceShort(t *testing.T) {
func TestDecodeTraceValid(t *testing.T) {
buf := make([]byte, 16)
buf[0] = 0x00
buf[1] = 0x01 // tag LE uint32 = 1
buf[5] = 0xAA // destHash start
buf[11] = 0xBB
// tag(4) + authCode(4) + flags(1) + pathData
binary.LittleEndian.PutUint32(buf[0:4], 1) // tag = 1
binary.LittleEndian.PutUint32(buf[4:8], 0xDEADBEEF) // authCode
buf[8] = 0x02 // flags
buf[9] = 0xAA // path data
p := decodeTrace(buf)
if p.Error != "" {
t.Errorf("unexpected error: %s", p.Error)
@@ -548,9 +550,18 @@ func TestDecodeTraceValid(t *testing.T) {
if p.Tag != 1 {
t.Errorf("tag=%d, want 1", p.Tag)
}
if p.AuthCode != 0xDEADBEEF {
t.Errorf("authCode=%d, want 0xDEADBEEF", p.AuthCode)
}
if p.TraceFlags == nil || *p.TraceFlags != 2 {
t.Errorf("traceFlags=%v, want 2", p.TraceFlags)
}
if p.Type != "TRACE" {
t.Errorf("type=%s, want TRACE", p.Type)
}
if p.PathData == "" {
t.Error("pathData should not be empty")
}
}
func TestDecodeAdvertShort(t *testing.T) {
@@ -833,10 +844,9 @@ func TestComputeContentHashShortHex(t *testing.T) {
}
func TestComputeContentHashTransportRoute(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) with no path hops + 4 transport code bytes
// header=0x14 (TRANSPORT_FLOOD, ADVERT), path=0x00 (0 hops)
// transport codes = 4 bytes, then payload
hex := "1400" + "AABBCCDD" + strings.Repeat("EE", 10)
// Route type 0 (TRANSPORT_FLOOD) with transport codes then path=0x00 (0 hops)
// header=0x14 (TRANSPORT_FLOOD, ADVERT), transport(4), path=0x00
hex := "14" + "AABBCCDD" + "00" + strings.Repeat("EE", 10)
hash := ComputeContentHash(hex)
if len(hash) != 16 {
t.Errorf("hash length=%d, want 16", len(hash))
@@ -870,12 +880,10 @@ func TestComputeContentHashPayloadBeyondBufferLongHex(t *testing.T) {
func TestComputeContentHashTransportBeyondBuffer(t *testing.T) {
// Transport route (0x00 = TRANSPORT_FLOOD) with path claiming some bytes
// total buffer too short for transport codes + path
// header=0x00, pathByte=0x02 (2 hops, 1-byte hash), then only 2 more bytes
// payloadStart = 2 + 2 + 4(transport) = 8, but buffer only 6 bytes
hex := "0002" + "AABB" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
// header=0x00, transport(4), pathByte=0x02 (2 hops, 1-byte hash)
// offset=1+4+1+2=8, buffer needs to be >= 8
hex := "00" + "AABB" + "CCDD" + "02" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
hash := ComputeContentHash(hex)
// payloadStart = 2 + 2 + 4 = 8, buffer is 10 bytes → should work
if len(hash) != 16 {
t.Errorf("hash length=%d, want 16", len(hash))
}
@@ -913,8 +921,8 @@ func TestDecodePacketWithNewlines(t *testing.T) {
}
func TestDecodePacketTransportRouteTooShort(t *testing.T) {
// TRANSPORT_FLOOD (route=0) but only 3 bytes total → too short for transport codes
_, err := DecodePacket("140011", nil)
// TRANSPORT_FLOOD (route=0) but only 2 bytes total → too short for transport codes
_, err := DecodePacket("1400", nil)
if err == nil {
t.Error("expected error for transport route with too-short buffer")
}
@@ -931,16 +939,19 @@ func TestDecodeAckShort(t *testing.T) {
}
func TestDecodeAckValid(t *testing.T) {
buf := []byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
buf := []byte{0xAA, 0xBB, 0xCC, 0xDD}
p := decodeAck(buf)
if p.Error != "" {
t.Errorf("unexpected error: %s", p.Error)
}
if p.DestHash != "aa" {
t.Errorf("destHash=%s, want aa", p.DestHash)
if p.ExtraHash != "ddccbbaa" {
t.Errorf("extraHash=%s, want ddccbbaa", p.ExtraHash)
}
if p.ExtraHash != "ccddeeff" {
t.Errorf("extraHash=%s, want ccddeeff", p.ExtraHash)
if p.DestHash != "" {
t.Errorf("destHash should be empty, got %s", p.DestHash)
}
if p.SrcHash != "" {
t.Errorf("srcHash should be empty, got %s", p.SrcHash)
}
}

View File

@@ -60,10 +60,10 @@ func (c *Config) NodeDaysOrDefault() int {
}
type HealthThresholds struct {
InfraDegradedMs int `json:"infraDegradedMs"`
InfraSilentMs int `json:"infraSilentMs"`
NodeDegradedMs int `json:"nodeDegradedMs"`
NodeSilentMs int `json:"nodeSilentMs"`
InfraDegradedHours float64 `json:"infraDegradedHours"`
InfraSilentHours float64 `json:"infraSilentHours"`
NodeDegradedHours float64 `json:"nodeDegradedHours"`
NodeSilentHours float64 `json:"nodeSilentHours"`
}
// ThemeFile mirrors theme.json overlay.
@@ -126,34 +126,46 @@ func LoadTheme(baseDirs ...string) *ThemeFile {
func (c *Config) GetHealthThresholds() HealthThresholds {
h := HealthThresholds{
InfraDegradedMs: 86400000,
InfraSilentMs: 259200000,
NodeDegradedMs: 3600000,
NodeSilentMs: 86400000,
InfraDegradedHours: 24,
InfraSilentHours: 72,
NodeDegradedHours: 1,
NodeSilentHours: 24,
}
if c.HealthThresholds != nil {
if c.HealthThresholds.InfraDegradedMs > 0 {
h.InfraDegradedMs = c.HealthThresholds.InfraDegradedMs
if c.HealthThresholds.InfraDegradedHours > 0 {
h.InfraDegradedHours = c.HealthThresholds.InfraDegradedHours
}
if c.HealthThresholds.InfraSilentMs > 0 {
h.InfraSilentMs = c.HealthThresholds.InfraSilentMs
if c.HealthThresholds.InfraSilentHours > 0 {
h.InfraSilentHours = c.HealthThresholds.InfraSilentHours
}
if c.HealthThresholds.NodeDegradedMs > 0 {
h.NodeDegradedMs = c.HealthThresholds.NodeDegradedMs
if c.HealthThresholds.NodeDegradedHours > 0 {
h.NodeDegradedHours = c.HealthThresholds.NodeDegradedHours
}
if c.HealthThresholds.NodeSilentMs > 0 {
h.NodeSilentMs = c.HealthThresholds.NodeSilentMs
if c.HealthThresholds.NodeSilentHours > 0 {
h.NodeSilentHours = c.HealthThresholds.NodeSilentHours
}
}
return h
}
// GetHealthMs returns degraded/silent thresholds for a given role.
// GetHealthMs returns degraded/silent thresholds in ms for a given role.
func (h HealthThresholds) GetHealthMs(role string) (degradedMs, silentMs int) {
const hourMs = 3600000
if role == "repeater" || role == "room" {
return h.InfraDegradedMs, h.InfraSilentMs
return int(h.InfraDegradedHours * hourMs), int(h.InfraSilentHours * hourMs)
}
return int(h.NodeDegradedHours * hourMs), int(h.NodeSilentHours * hourMs)
}
// ToClientMs returns the thresholds as ms for the frontend.
func (h HealthThresholds) ToClientMs() map[string]int {
const hourMs = 3600000
return map[string]int{
"infraDegradedMs": int(h.InfraDegradedHours * hourMs),
"infraSilentMs": int(h.InfraSilentHours * hourMs),
"nodeDegradedMs": int(h.NodeDegradedHours * hourMs),
"nodeSilentMs": int(h.NodeSilentHours * hourMs),
}
return h.NodeDegradedMs, h.NodeSilentMs
}
func (c *Config) ResolveDBPath(baseDir string) string {

View File

@@ -23,10 +23,10 @@ func TestLoadConfigValidJSON(t *testing.T) {
"SJC": "San Jose",
},
"healthThresholds": map[string]interface{}{
"infraDegradedMs": 100000,
"infraSilentMs": 200000,
"nodeDegradedMs": 50000,
"nodeSilentMs": 100000,
"infraDegradedHours": 2,
"infraSilentHours": 4,
"nodeDegradedHours": 0.5,
"nodeSilentHours": 2,
},
"liveMap": map[string]interface{}{
"propagationBufferMs": 3000,
@@ -178,68 +178,68 @@ func TestGetHealthThresholdsDefaults(t *testing.T) {
cfg := &Config{}
ht := cfg.GetHealthThresholds()
if ht.InfraDegradedMs != 86400000 {
t.Errorf("expected 86400000, got %d", ht.InfraDegradedMs)
if ht.InfraDegradedHours != 24 {
t.Errorf("expected 24, got %v", ht.InfraDegradedHours)
}
if ht.InfraSilentMs != 259200000 {
t.Errorf("expected 259200000, got %d", ht.InfraSilentMs)
if ht.InfraSilentHours != 72 {
t.Errorf("expected 72, got %v", ht.InfraSilentHours)
}
if ht.NodeDegradedMs != 3600000 {
t.Errorf("expected 3600000, got %d", ht.NodeDegradedMs)
if ht.NodeDegradedHours != 1 {
t.Errorf("expected 1, got %v", ht.NodeDegradedHours)
}
if ht.NodeSilentMs != 86400000 {
t.Errorf("expected 86400000, got %d", ht.NodeSilentMs)
if ht.NodeSilentHours != 24 {
t.Errorf("expected 24, got %v", ht.NodeSilentHours)
}
}
func TestGetHealthThresholdsCustom(t *testing.T) {
cfg := &Config{
HealthThresholds: &HealthThresholds{
InfraDegradedMs: 100000,
InfraSilentMs: 200000,
NodeDegradedMs: 50000,
NodeSilentMs: 100000,
InfraDegradedHours: 2,
InfraSilentHours: 4,
NodeDegradedHours: 0.5,
NodeSilentHours: 2,
},
}
ht := cfg.GetHealthThresholds()
if ht.InfraDegradedMs != 100000 {
t.Errorf("expected 100000, got %d", ht.InfraDegradedMs)
if ht.InfraDegradedHours != 2 {
t.Errorf("expected 2, got %v", ht.InfraDegradedHours)
}
if ht.InfraSilentMs != 200000 {
t.Errorf("expected 200000, got %d", ht.InfraSilentMs)
if ht.InfraSilentHours != 4 {
t.Errorf("expected 4, got %v", ht.InfraSilentHours)
}
if ht.NodeDegradedMs != 50000 {
t.Errorf("expected 50000, got %d", ht.NodeDegradedMs)
if ht.NodeDegradedHours != 0.5 {
t.Errorf("expected 0.5, got %v", ht.NodeDegradedHours)
}
if ht.NodeSilentMs != 100000 {
t.Errorf("expected 100000, got %d", ht.NodeSilentMs)
if ht.NodeSilentHours != 2 {
t.Errorf("expected 2, got %v", ht.NodeSilentHours)
}
}
func TestGetHealthThresholdsPartialCustom(t *testing.T) {
cfg := &Config{
HealthThresholds: &HealthThresholds{
InfraDegradedMs: 100000,
InfraDegradedHours: 2,
// Others left as zero → should use defaults
},
}
ht := cfg.GetHealthThresholds()
if ht.InfraDegradedMs != 100000 {
t.Errorf("expected 100000, got %d", ht.InfraDegradedMs)
if ht.InfraDegradedHours != 2 {
t.Errorf("expected 2, got %v", ht.InfraDegradedHours)
}
if ht.InfraSilentMs != 259200000 {
t.Errorf("expected default 259200000, got %d", ht.InfraSilentMs)
if ht.InfraSilentHours != 72 {
t.Errorf("expected default 72, got %v", ht.InfraSilentHours)
}
}
func TestGetHealthMs(t *testing.T) {
ht := HealthThresholds{
InfraDegradedMs: 86400000,
InfraSilentMs: 259200000,
NodeDegradedMs: 3600000,
NodeSilentMs: 86400000,
InfraDegradedHours: 24,
InfraSilentHours: 72,
NodeDegradedHours: 1,
NodeSilentHours: 24,
}
tests := []struct {

File diff suppressed because it is too large Load Diff

View File

@@ -17,6 +17,8 @@ func setupTestDB(t *testing.T) *DB {
if err != nil {
t.Fatal(err)
}
// Force single connection so all goroutines share the same in-memory DB
conn.SetMaxOpenConns(1)
// Create schema matching MeshCore Analyzer v3
schema := `
@@ -511,10 +513,10 @@ func TestGetNetworkStatus(t *testing.T) {
seedTestData(t, db)
ht := HealthThresholds{
InfraDegradedMs: 86400000,
InfraSilentMs: 259200000,
NodeDegradedMs: 3600000,
NodeSilentMs: 86400000,
InfraDegradedHours: 24,
InfraSilentHours: 72,
NodeDegradedHours: 1,
NodeSilentHours: 24,
}
result, err := db.GetNetworkStatus(ht)
if err != nil {
@@ -1048,10 +1050,10 @@ func TestGetNetworkStatusDateFormats(t *testing.T) {
VALUES ('node4444', 'NodeBad', 'sensor', 'not-a-date')`)
ht := HealthThresholds{
InfraDegradedMs: 86400000,
InfraSilentMs: 259200000,
NodeDegradedMs: 3600000,
NodeSilentMs: 86400000,
InfraDegradedHours: 24,
InfraSilentHours: 72,
NodeDegradedHours: 1,
NodeSilentHours: 24,
}
result, err := db.GetNetworkStatus(ht)
if err != nil {

View File

@@ -54,8 +54,8 @@ type Header struct {
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
type TransportCodes struct {
NextHop string `json:"nextHop"`
LastHop string `json:"lastHop"`
Code1 string `json:"code1"`
Code2 string `json:"code2"`
}
// Path holds decoded path/hop information.
@@ -74,6 +74,8 @@ type AdvertFlags struct {
Room bool `json:"room"`
Sensor bool `json:"sensor"`
HasLocation bool `json:"hasLocation"`
HasFeat1 bool `json:"hasFeat1"`
HasFeat2 bool `json:"hasFeat2"`
HasName bool `json:"hasName"`
}
@@ -97,6 +99,8 @@ type Payload struct {
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
PathData string `json:"pathData,omitempty"`
Tag uint32 `json:"tag,omitempty"`
AuthCode uint32 `json:"authCode,omitempty"`
TraceFlags *int `json:"traceFlags,omitempty"`
RawHex string `json:"raw,omitempty"`
Error string `json:"error,omitempty"`
}
@@ -173,14 +177,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
}
func decodeAck(buf []byte) Payload {
if len(buf) < 6 {
if len(buf) < 4 {
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
}
checksum := binary.LittleEndian.Uint32(buf[0:4])
return Payload{
Type: "ACK",
DestHash: hex.EncodeToString(buf[0:1]),
SrcHash: hex.EncodeToString(buf[1:2]),
ExtraHash: hex.EncodeToString(buf[2:6]),
ExtraHash: fmt.Sprintf("%08x", checksum),
}
}
@@ -205,6 +208,8 @@ func decodeAdvert(buf []byte) Payload {
if len(appdata) > 0 {
flags := appdata[0]
advType := int(flags & 0x0F)
hasFeat1 := flags&0x20 != 0
hasFeat2 := flags&0x40 != 0
p.Flags = &AdvertFlags{
Raw: int(flags),
Type: advType,
@@ -213,6 +218,8 @@ func decodeAdvert(buf []byte) Payload {
Room: advType == 3,
Sensor: advType == 4,
HasLocation: flags&0x10 != 0,
HasFeat1: hasFeat1,
HasFeat2: hasFeat2,
HasName: flags&0x80 != 0,
}
@@ -226,6 +233,12 @@ func decodeAdvert(buf []byte) Payload {
p.Lon = &lon
off += 8
}
if hasFeat1 && len(appdata) >= off+2 {
off += 2 // skip feat1 bytes (reserved for future use)
}
if hasFeat2 && len(appdata) >= off+2 {
off += 2 // skip feat2 bytes (reserved for future use)
}
if p.Flags.HasName {
name := string(appdata[off:])
name = strings.TrimRight(name, "\x00")
@@ -276,15 +289,22 @@ func decodePathPayload(buf []byte) Payload {
}
func decodeTrace(buf []byte) Payload {
if len(buf) < 12 {
if len(buf) < 9 {
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
}
return Payload{
Type: "TRACE",
DestHash: hex.EncodeToString(buf[5:11]),
SrcHash: hex.EncodeToString(buf[11:12]),
Tag: binary.LittleEndian.Uint32(buf[1:5]),
tag := binary.LittleEndian.Uint32(buf[0:4])
authCode := binary.LittleEndian.Uint32(buf[4:8])
flags := int(buf[8])
p := Payload{
Type: "TRACE",
Tag: tag,
AuthCode: authCode,
TraceFlags: &flags,
}
if len(buf) > 9 {
p.PathData = hex.EncodeToString(buf[9:])
}
return p
}
func decodePayload(payloadType int, buf []byte) Payload {
@@ -327,8 +347,7 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
}
header := decodeHeader(buf[0])
pathByte := buf[1]
offset := 2
offset := 1
var tc *TransportCodes
if isTransportRoute(header.RouteType) {
@@ -336,12 +355,18 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
return nil, fmt.Errorf("packet too short for transport codes")
}
tc = &TransportCodes{
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
}
offset += 4
}
if offset >= len(buf) {
return nil, fmt.Errorf("packet too short (no path byte)")
}
pathByte := buf[offset]
offset++
path, bytesConsumed := decodePath(pathByte, buf, offset)
offset += bytesConsumed
@@ -367,16 +392,24 @@ func ComputeContentHash(rawHex string) string {
return rawHex
}
pathByte := buf[1]
headerByte := buf[0]
offset := 1
if isTransportRoute(int(headerByte & 0x03)) {
offset += 4
}
if offset >= len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]
}
return rawHex
}
pathByte := buf[offset]
offset++
hashSize := int((pathByte>>6)&0x3) + 1
hashCount := int(pathByte & 0x3F)
pathBytes := hashSize * hashCount
headerByte := buf[0]
payloadStart := 2 + pathBytes
if isTransportRoute(int(headerByte & 0x03)) {
payloadStart += 4
}
payloadStart := offset + pathBytes
if payloadStart > len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]

View File

@@ -1,403 +1,506 @@
package main
// parity_test.go — Golden fixture shape tests.
// Validates that Go API responses match the shape of Node.js API responses.
// Shapes were captured from the production Node.js server and stored in
// testdata/golden/shapes.json.
import (
"encoding/json"
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
// shapeSpec describes the expected JSON structure from the Node.js server.
type shapeSpec struct {
Type string `json:"type"`
Keys map[string]shapeSpec `json:"keys,omitempty"`
ElementShape *shapeSpec `json:"elementShape,omitempty"`
DynamicKeys bool `json:"dynamicKeys,omitempty"`
ValueShape *shapeSpec `json:"valueShape,omitempty"`
RequiredKeys map[string]shapeSpec `json:"requiredKeys,omitempty"`
}
// loadShapes reads testdata/golden/shapes.json relative to this source file.
func loadShapes(t *testing.T) map[string]shapeSpec {
t.Helper()
_, thisFile, _, _ := runtime.Caller(0)
dir := filepath.Dir(thisFile)
data, err := os.ReadFile(filepath.Join(dir, "testdata", "golden", "shapes.json"))
if err != nil {
t.Fatalf("cannot load shapes.json: %v", err)
}
var shapes map[string]shapeSpec
if err := json.Unmarshal(data, &shapes); err != nil {
t.Fatalf("cannot parse shapes.json: %v", err)
}
return shapes
}
// validateShape recursively checks that `actual` matches the expected `spec`.
// `path` tracks the JSON path for error messages.
// Returns a list of mismatch descriptions.
func validateShape(actual interface{}, spec shapeSpec, path string) []string {
var errs []string
switch spec.Type {
case "null", "nullable":
// nullable means: value can be null OR matching type. Accept anything.
return nil
case "nullable_number":
// Can be null or number
if actual != nil {
if _, ok := actual.(float64); !ok {
errs = append(errs, fmt.Sprintf("%s: expected number or null, got %T", path, actual))
}
}
return errs
case "string":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected string, got null", path))
} else if _, ok := actual.(string); !ok {
errs = append(errs, fmt.Sprintf("%s: expected string, got %T", path, actual))
}
case "number":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected number, got null", path))
} else if _, ok := actual.(float64); !ok {
errs = append(errs, fmt.Sprintf("%s: expected number, got %T (%v)", path, actual, actual))
}
case "boolean":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected boolean, got null", path))
} else if _, ok := actual.(bool); !ok {
errs = append(errs, fmt.Sprintf("%s: expected boolean, got %T", path, actual))
}
case "array":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected array, got null (arrays must be [] not null)", path))
return errs
}
arr, ok := actual.([]interface{})
if !ok {
errs = append(errs, fmt.Sprintf("%s: expected array, got %T", path, actual))
return errs
}
if spec.ElementShape != nil && len(arr) > 0 {
errs = append(errs, validateShape(arr[0], *spec.ElementShape, path+"[0]")...)
}
case "object":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected object, got null", path))
return errs
}
obj, ok := actual.(map[string]interface{})
if !ok {
errs = append(errs, fmt.Sprintf("%s: expected object, got %T", path, actual))
return errs
}
if spec.DynamicKeys {
// Object with dynamic keys — validate value shapes
if spec.ValueShape != nil && len(obj) > 0 {
for k, v := range obj {
errs = append(errs, validateShape(v, *spec.ValueShape, path+"."+k)...)
break // check just one sample
}
}
if spec.RequiredKeys != nil {
for rk, rs := range spec.RequiredKeys {
v, exists := obj[rk]
if !exists {
errs = append(errs, fmt.Sprintf("%s: missing required key %q in dynamic-key object", path, rk))
} else {
errs = append(errs, validateShape(v, rs, path+"."+rk)...)
}
}
}
} else if spec.Keys != nil {
// Object with known keys — check each expected key exists and has correct type
for key, keySpec := range spec.Keys {
val, exists := obj[key]
if !exists {
errs = append(errs, fmt.Sprintf("%s: missing field %q (expected %s)", path, key, keySpec.Type))
} else {
errs = append(errs, validateShape(val, keySpec, path+"."+key)...)
}
}
}
}
return errs
}
// parityEndpoint defines one endpoint to test for parity.
type parityEndpoint struct {
name string // key in shapes.json
path string // HTTP path to request
}
func TestParityShapes(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
endpoints := []parityEndpoint{
{"stats", "/api/stats"},
{"nodes", "/api/nodes?limit=5"},
{"packets", "/api/packets?limit=5"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"observers", "/api/observers"},
{"channels", "/api/channels"},
{"channel_messages", "/api/channels/0000000000000000/messages?limit=5"},
{"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"},
{"bulk_health", "/api/nodes/bulk-health"},
{"health", "/api/health"},
{"perf", "/api/perf"},
}
for _, ep := range endpoints {
t.Run("Parity_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name]
if !ok {
t.Fatalf("no shape spec found for %q in shapes.json", ep.name)
}
req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("GET %s returned %d, expected 200. Body: %s",
ep.path, w.Code, w.Body.String())
}
var body interface{}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("GET %s returned invalid JSON: %v\nBody: %s",
ep.path, err, w.Body.String())
}
mismatches := validateShape(body, spec, ep.path)
if len(mismatches) > 0 {
t.Errorf("Go %s has %d shape mismatches vs Node.js golden:\n %s",
ep.path, len(mismatches), strings.Join(mismatches, "\n "))
}
})
}
}
// TestParityNodeDetail tests node detail endpoint shape.
// Uses a known test node public key from seeded data.
func TestParityNodeDetail(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
spec, ok := shapes["node_detail"]
if !ok {
t.Fatal("no shape spec for node_detail in shapes.json")
}
req := httptest.NewRequest("GET", "/api/nodes/aabbccdd11223344", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("node detail returned %d: %s", w.Code, w.Body.String())
}
var body interface{}
json.Unmarshal(w.Body.Bytes(), &body)
mismatches := validateShape(body, spec, "/api/nodes/{pubkey}")
if len(mismatches) > 0 {
t.Errorf("Go node detail has %d shape mismatches vs Node.js golden:\n %s",
len(mismatches), strings.Join(mismatches, "\n "))
}
}
// TestParityArraysNotNull verifies that array-typed fields in Go responses are
// [] (empty array) rather than null. This is a common Go/JSON pitfall where
// nil slices marshal as null instead of [].
// Uses shapes.json to know which fields SHOULD be arrays.
func TestParityArraysNotNull(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
endpoints := []struct {
name string
path string
}{
{"stats", "/api/stats"},
{"nodes", "/api/nodes?limit=5"},
{"packets", "/api/packets?limit=5"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"observers", "/api/observers"},
{"channels", "/api/channels"},
{"bulk_health", "/api/nodes/bulk-health"},
{"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"},
}
for _, ep := range endpoints {
t.Run("NullArrayCheck_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name]
if !ok {
t.Skipf("no shape spec for %s", ep.name)
}
req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Skipf("GET %s returned %d, skipping null-array check", ep.path, w.Code)
}
var body interface{}
json.Unmarshal(w.Body.Bytes(), &body)
nullArrays := findNullArrays(body, spec, ep.path)
if len(nullArrays) > 0 {
t.Errorf("Go %s has null where [] expected:\n %s\n"+
"Go nil slices marshal as null — initialize with make() or literal",
ep.path, strings.Join(nullArrays, "\n "))
}
})
}
}
// findNullArrays walks JSON data alongside a shape spec and returns paths
// where the spec says the field should be an array but Go returned null.
func findNullArrays(actual interface{}, spec shapeSpec, path string) []string {
var nulls []string
switch spec.Type {
case "array":
if actual == nil {
nulls = append(nulls, fmt.Sprintf("%s: null (should be [])", path))
} else if arr, ok := actual.([]interface{}); ok && spec.ElementShape != nil {
for i, elem := range arr {
nulls = append(nulls, findNullArrays(elem, *spec.ElementShape, fmt.Sprintf("%s[%d]", path, i))...)
}
}
case "object":
obj, ok := actual.(map[string]interface{})
if !ok || obj == nil {
return nulls
}
if spec.Keys != nil {
for key, keySpec := range spec.Keys {
if val, exists := obj[key]; exists {
nulls = append(nulls, findNullArrays(val, keySpec, path+"."+key)...)
} else if keySpec.Type == "array" {
// Key missing entirely — also a null-array problem
nulls = append(nulls, fmt.Sprintf("%s.%s: missing (should be [])", path, key))
}
}
}
if spec.DynamicKeys && spec.ValueShape != nil {
for k, v := range obj {
nulls = append(nulls, findNullArrays(v, *spec.ValueShape, path+"."+k)...)
break // sample one
}
}
}
return nulls
}
// TestParityHealthEngine verifies Go health endpoint declares engine=go
// while Node declares engine=node (or omits it). The Go server must always
// identify itself.
func TestParityHealthEngine(t *testing.T) {
_, router := setupTestServer(t)
req := httptest.NewRequest("GET", "/api/health", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
var body map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &body)
engine, ok := body["engine"]
if !ok {
t.Error("health response missing 'engine' field (Go server must include engine=go)")
} else if engine != "go" {
t.Errorf("health engine=%v, expected 'go'", engine)
}
}
// TestValidateShapeFunction directly tests the shape validator itself.
func TestValidateShapeFunction(t *testing.T) {
t.Run("string match", func(t *testing.T) {
errs := validateShape("hello", shapeSpec{Type: "string"}, "$.x")
if len(errs) != 0 {
t.Errorf("unexpected errors: %v", errs)
}
})
t.Run("string mismatch", func(t *testing.T) {
errs := validateShape(42.0, shapeSpec{Type: "string"}, "$.x")
if len(errs) != 1 {
t.Errorf("expected 1 error, got %d: %v", len(errs), errs)
}
})
t.Run("null array rejected", func(t *testing.T) {
errs := validateShape(nil, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 1 || !strings.Contains(errs[0], "null") {
t.Errorf("expected null-array error, got: %v", errs)
}
})
t.Run("empty array OK", func(t *testing.T) {
errs := validateShape([]interface{}{}, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 0 {
t.Errorf("unexpected errors for empty array: %v", errs)
}
})
t.Run("missing object key", func(t *testing.T) {
spec := shapeSpec{Type: "object", Keys: map[string]shapeSpec{
"name": {Type: "string"},
"age": {Type: "number"},
}}
obj := map[string]interface{}{"name": "test"}
errs := validateShape(obj, spec, "$.user")
if len(errs) != 1 || !strings.Contains(errs[0], "age") {
t.Errorf("expected missing age error, got: %v", errs)
}
})
t.Run("nullable allows null", func(t *testing.T) {
errs := validateShape(nil, shapeSpec{Type: "nullable"}, "$.x")
if len(errs) != 0 {
t.Errorf("nullable should accept null: %v", errs)
}
})
t.Run("dynamic keys validates value shape", func(t *testing.T) {
spec := shapeSpec{
Type: "object",
DynamicKeys: true,
ValueShape: &shapeSpec{Type: "number"},
}
obj := map[string]interface{}{"a": 1.0, "b": 2.0}
errs := validateShape(obj, spec, "$.dyn")
if len(errs) != 0 {
t.Errorf("unexpected errors: %v", errs)
}
})
}
package main
// parity_test.go — Golden fixture shape tests.
// Validates that Go API responses match the shape of Node.js API responses.
// Shapes were captured from the production Node.js server and stored in
// testdata/golden/shapes.json.
import (
"encoding/json"
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"runtime"
"sort"
"strings"
"testing"
"time"
)
// shapeSpec describes the expected JSON structure from the Node.js server.
type shapeSpec struct {
Type string `json:"type"`
Keys map[string]shapeSpec `json:"keys,omitempty"`
ElementShape *shapeSpec `json:"elementShape,omitempty"`
DynamicKeys bool `json:"dynamicKeys,omitempty"`
ValueShape *shapeSpec `json:"valueShape,omitempty"`
RequiredKeys map[string]shapeSpec `json:"requiredKeys,omitempty"`
}
// loadShapes reads testdata/golden/shapes.json relative to this source file.
func loadShapes(t *testing.T) map[string]shapeSpec {
t.Helper()
_, thisFile, _, _ := runtime.Caller(0)
dir := filepath.Dir(thisFile)
data, err := os.ReadFile(filepath.Join(dir, "testdata", "golden", "shapes.json"))
if err != nil {
t.Fatalf("cannot load shapes.json: %v", err)
}
var shapes map[string]shapeSpec
if err := json.Unmarshal(data, &shapes); err != nil {
t.Fatalf("cannot parse shapes.json: %v", err)
}
return shapes
}
// validateShape recursively checks that `actual` matches the expected `spec`.
// `path` tracks the JSON path for error messages.
// Returns a list of mismatch descriptions.
func validateShape(actual interface{}, spec shapeSpec, path string) []string {
var errs []string
switch spec.Type {
case "null", "nullable":
// nullable means: value can be null OR matching type. Accept anything.
return nil
case "nullable_number":
// Can be null or number
if actual != nil {
if _, ok := actual.(float64); !ok {
errs = append(errs, fmt.Sprintf("%s: expected number or null, got %T", path, actual))
}
}
return errs
case "string":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected string, got null", path))
} else if _, ok := actual.(string); !ok {
errs = append(errs, fmt.Sprintf("%s: expected string, got %T", path, actual))
}
case "number":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected number, got null", path))
} else if _, ok := actual.(float64); !ok {
errs = append(errs, fmt.Sprintf("%s: expected number, got %T (%v)", path, actual, actual))
}
case "boolean":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected boolean, got null", path))
} else if _, ok := actual.(bool); !ok {
errs = append(errs, fmt.Sprintf("%s: expected boolean, got %T", path, actual))
}
case "array":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected array, got null (arrays must be [] not null)", path))
return errs
}
arr, ok := actual.([]interface{})
if !ok {
errs = append(errs, fmt.Sprintf("%s: expected array, got %T", path, actual))
return errs
}
if spec.ElementShape != nil && len(arr) > 0 {
errs = append(errs, validateShape(arr[0], *spec.ElementShape, path+"[0]")...)
}
case "object":
if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected object, got null", path))
return errs
}
obj, ok := actual.(map[string]interface{})
if !ok {
errs = append(errs, fmt.Sprintf("%s: expected object, got %T", path, actual))
return errs
}
if spec.DynamicKeys {
// Object with dynamic keys — validate value shapes
if spec.ValueShape != nil && len(obj) > 0 {
for k, v := range obj {
errs = append(errs, validateShape(v, *spec.ValueShape, path+"."+k)...)
break // check just one sample
}
}
if spec.RequiredKeys != nil {
for rk, rs := range spec.RequiredKeys {
v, exists := obj[rk]
if !exists {
errs = append(errs, fmt.Sprintf("%s: missing required key %q in dynamic-key object", path, rk))
} else {
errs = append(errs, validateShape(v, rs, path+"."+rk)...)
}
}
}
} else if spec.Keys != nil {
// Object with known keys — check each expected key exists and has correct type
for key, keySpec := range spec.Keys {
val, exists := obj[key]
if !exists {
errs = append(errs, fmt.Sprintf("%s: missing field %q (expected %s)", path, key, keySpec.Type))
} else {
errs = append(errs, validateShape(val, keySpec, path+"."+key)...)
}
}
}
}
return errs
}
// parityEndpoint defines one endpoint to test for parity.
type parityEndpoint struct {
name string // key in shapes.json
path string // HTTP path to request
}
func TestParityShapes(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
endpoints := []parityEndpoint{
{"stats", "/api/stats"},
{"nodes", "/api/nodes?limit=5"},
{"packets", "/api/packets?limit=5"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"observers", "/api/observers"},
{"channels", "/api/channels"},
{"channel_messages", "/api/channels/0000000000000000/messages?limit=5"},
{"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"},
{"bulk_health", "/api/nodes/bulk-health"},
{"health", "/api/health"},
{"perf", "/api/perf"},
}
for _, ep := range endpoints {
t.Run("Parity_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name]
if !ok {
t.Fatalf("no shape spec found for %q in shapes.json", ep.name)
}
req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("GET %s returned %d, expected 200. Body: %s",
ep.path, w.Code, w.Body.String())
}
var body interface{}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("GET %s returned invalid JSON: %v\nBody: %s",
ep.path, err, w.Body.String())
}
mismatches := validateShape(body, spec, ep.path)
if len(mismatches) > 0 {
t.Errorf("Go %s has %d shape mismatches vs Node.js golden:\n %s",
ep.path, len(mismatches), strings.Join(mismatches, "\n "))
}
})
}
}
// TestParityNodeDetail tests node detail endpoint shape.
// Uses a known test node public key from seeded data.
func TestParityNodeDetail(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
spec, ok := shapes["node_detail"]
if !ok {
t.Fatal("no shape spec for node_detail in shapes.json")
}
req := httptest.NewRequest("GET", "/api/nodes/aabbccdd11223344", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("node detail returned %d: %s", w.Code, w.Body.String())
}
var body interface{}
json.Unmarshal(w.Body.Bytes(), &body)
mismatches := validateShape(body, spec, "/api/nodes/{pubkey}")
if len(mismatches) > 0 {
t.Errorf("Go node detail has %d shape mismatches vs Node.js golden:\n %s",
len(mismatches), strings.Join(mismatches, "\n "))
}
}
// TestParityArraysNotNull verifies that array-typed fields in Go responses are
// [] (empty array) rather than null. This is a common Go/JSON pitfall where
// nil slices marshal as null instead of [].
// Uses shapes.json to know which fields SHOULD be arrays.
func TestParityArraysNotNull(t *testing.T) {
shapes := loadShapes(t)
_, router := setupTestServer(t)
endpoints := []struct {
name string
path string
}{
{"stats", "/api/stats"},
{"nodes", "/api/nodes?limit=5"},
{"packets", "/api/packets?limit=5"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"observers", "/api/observers"},
{"channels", "/api/channels"},
{"bulk_health", "/api/nodes/bulk-health"},
{"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"},
}
for _, ep := range endpoints {
t.Run("NullArrayCheck_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name]
if !ok {
t.Skipf("no shape spec for %s", ep.name)
}
req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Skipf("GET %s returned %d, skipping null-array check", ep.path, w.Code)
}
var body interface{}
json.Unmarshal(w.Body.Bytes(), &body)
nullArrays := findNullArrays(body, spec, ep.path)
if len(nullArrays) > 0 {
t.Errorf("Go %s has null where [] expected:\n %s\n"+
"Go nil slices marshal as null — initialize with make() or literal",
ep.path, strings.Join(nullArrays, "\n "))
}
})
}
}
// findNullArrays walks JSON data alongside a shape spec and returns paths
// where the spec says the field should be an array but Go returned null.
func findNullArrays(actual interface{}, spec shapeSpec, path string) []string {
var nulls []string
switch spec.Type {
case "array":
if actual == nil {
nulls = append(nulls, fmt.Sprintf("%s: null (should be [])", path))
} else if arr, ok := actual.([]interface{}); ok && spec.ElementShape != nil {
for i, elem := range arr {
nulls = append(nulls, findNullArrays(elem, *spec.ElementShape, fmt.Sprintf("%s[%d]", path, i))...)
}
}
case "object":
obj, ok := actual.(map[string]interface{})
if !ok || obj == nil {
return nulls
}
if spec.Keys != nil {
for key, keySpec := range spec.Keys {
if val, exists := obj[key]; exists {
nulls = append(nulls, findNullArrays(val, keySpec, path+"."+key)...)
} else if keySpec.Type == "array" {
// Key missing entirely — also a null-array problem
nulls = append(nulls, fmt.Sprintf("%s.%s: missing (should be [])", path, key))
}
}
}
if spec.DynamicKeys && spec.ValueShape != nil {
for k, v := range obj {
nulls = append(nulls, findNullArrays(v, *spec.ValueShape, path+"."+k)...)
break // sample one
}
}
}
return nulls
}
// TestParityHealthEngine verifies Go health endpoint declares engine=go
// while Node declares engine=node (or omits it). The Go server must always
// identify itself.
func TestParityHealthEngine(t *testing.T) {
_, router := setupTestServer(t)
req := httptest.NewRequest("GET", "/api/health", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
var body map[string]interface{}
json.Unmarshal(w.Body.Bytes(), &body)
engine, ok := body["engine"]
if !ok {
t.Error("health response missing 'engine' field (Go server must include engine=go)")
} else if engine != "go" {
t.Errorf("health engine=%v, expected 'go'", engine)
}
}
// TestValidateShapeFunction directly tests the shape validator itself.
func TestValidateShapeFunction(t *testing.T) {
t.Run("string match", func(t *testing.T) {
errs := validateShape("hello", shapeSpec{Type: "string"}, "$.x")
if len(errs) != 0 {
t.Errorf("unexpected errors: %v", errs)
}
})
t.Run("string mismatch", func(t *testing.T) {
errs := validateShape(42.0, shapeSpec{Type: "string"}, "$.x")
if len(errs) != 1 {
t.Errorf("expected 1 error, got %d: %v", len(errs), errs)
}
})
t.Run("null array rejected", func(t *testing.T) {
errs := validateShape(nil, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 1 || !strings.Contains(errs[0], "null") {
t.Errorf("expected null-array error, got: %v", errs)
}
})
t.Run("empty array OK", func(t *testing.T) {
errs := validateShape([]interface{}{}, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 0 {
t.Errorf("unexpected errors for empty array: %v", errs)
}
})
t.Run("missing object key", func(t *testing.T) {
spec := shapeSpec{Type: "object", Keys: map[string]shapeSpec{
"name": {Type: "string"},
"age": {Type: "number"},
}}
obj := map[string]interface{}{"name": "test"}
errs := validateShape(obj, spec, "$.user")
if len(errs) != 1 || !strings.Contains(errs[0], "age") {
t.Errorf("expected missing age error, got: %v", errs)
}
})
t.Run("nullable allows null", func(t *testing.T) {
errs := validateShape(nil, shapeSpec{Type: "nullable"}, "$.x")
if len(errs) != 0 {
t.Errorf("nullable should accept null: %v", errs)
}
})
t.Run("dynamic keys validates value shape", func(t *testing.T) {
spec := shapeSpec{
Type: "object",
DynamicKeys: true,
ValueShape: &shapeSpec{Type: "number"},
}
obj := map[string]interface{}{"a": 1.0, "b": 2.0}
errs := validateShape(obj, spec, "$.dyn")
if len(errs) != 0 {
t.Errorf("unexpected errors: %v", errs)
}
})
}
func TestParityWSMultiObserverGolden(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store load failed: %v", err)
}
poller := NewPoller(db, hub, 50*time.Millisecond)
poller.store = store
client := &Client{send: make(chan []byte, 256)}
hub.Register(client)
defer hub.Unregister(client)
go poller.Start()
defer poller.Stop()
// Wait for poller to initialize its lastID/lastObsID cursors before
// inserting new data; otherwise the poller may snapshot a lastID that
// already includes the test data and never broadcast it.
time.Sleep(100 * time.Millisecond)
now := time.Now().UTC().Format(time.RFC3339)
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('BEEF', 'goldenstarburst237', ?, 1, 4, '{"pubKey":"aabbccdd11223344","type":"ADVERT"}')`, now); err != nil {
t.Fatalf("insert tx failed: %v", err)
}
var txID int
if err := db.conn.QueryRow(`SELECT id FROM transmissions WHERE hash='goldenstarburst237'`).Scan(&txID); err != nil {
t.Fatalf("query tx id failed: %v", err)
}
ts := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (?, 1, 11.0, -88, '["p1"]', ?),
(?, 2, 9.0, -92, '["p1","p2"]', ?),
(?, 1, 7.0, -96, '["p1","p2","p3"]', ?)`,
txID, ts, txID, ts+1, txID, ts+2); err != nil {
t.Fatalf("insert obs failed: %v", err)
}
type golden struct {
Hash string
Count int
Paths []string
ObserverIDs []string
}
expected := golden{
Hash: "goldenstarburst237",
Count: 3,
Paths: []string{`["p1"]`, `["p1","p2"]`, `["p1","p2","p3"]`},
ObserverIDs: []string{"obs1", "obs2"},
}
gotPaths := make([]string, 0, expected.Count)
gotObservers := make(map[string]bool)
deadline := time.After(2 * time.Second)
for len(gotPaths) < expected.Count {
select {
case raw := <-client.send:
var msg map[string]interface{}
if err := json.Unmarshal(raw, &msg); err != nil {
t.Fatalf("unmarshal ws message failed: %v", err)
}
if msg["type"] != "packet" {
continue
}
data, _ := msg["data"].(map[string]interface{})
if data == nil || data["hash"] != expected.Hash {
continue
}
if path, ok := data["path_json"].(string); ok {
gotPaths = append(gotPaths, path)
}
if oid, ok := data["observer_id"].(string); ok && oid != "" {
gotObservers[oid] = true
}
case <-deadline:
t.Fatalf("timed out waiting for %d ws messages, got %d", expected.Count, len(gotPaths))
}
}
sort.Strings(gotPaths)
sort.Strings(expected.Paths)
if len(gotPaths) != len(expected.Paths) {
t.Fatalf("path count mismatch: got %d want %d", len(gotPaths), len(expected.Paths))
}
for i := range expected.Paths {
if gotPaths[i] != expected.Paths[i] {
t.Fatalf("path mismatch at %d: got %q want %q", i, gotPaths[i], expected.Paths[i])
}
}
for _, oid := range expected.ObserverIDs {
if !gotObservers[oid] {
t.Fatalf("missing expected observer %q in ws messages", oid)
}
}
}

View File

@@ -33,6 +33,11 @@ type Server struct {
memStatsMu sync.Mutex
memStatsCache runtime.MemStats
memStatsCachedAt time.Time
// Cached /api/stats response — recomputed at most once every 10s
statsMu sync.Mutex
statsCache *StatsResponse
statsCachedAt time.Time
}
// PerfStats tracks request performance.
@@ -208,7 +213,7 @@ func (s *Server) handleConfigCache(w http.ResponseWriter, r *http.Request) {
func (s *Server) handleConfigClient(w http.ResponseWriter, r *http.Request) {
writeJSON(w, ClientConfigResponse{
Roles: s.cfg.Roles,
HealthThresholds: s.cfg.HealthThresholds,
HealthThresholds: s.cfg.GetHealthThresholds().ToClientMs(),
Tiles: s.cfg.Tiles,
SnrThresholds: s.cfg.SnrThresholds,
DistThresholds: s.cfg.DistThresholds,
@@ -380,6 +385,17 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
}
func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
const statsTTL = 10 * time.Second
s.statsMu.Lock()
if s.statsCache != nil && time.Since(s.statsCachedAt) < statsTTL {
cached := s.statsCache
s.statsMu.Unlock()
writeJSON(w, cached)
return
}
s.statsMu.Unlock()
var stats *Stats
var err error
if s.store != nil {
@@ -392,7 +408,7 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
return
}
counts := s.db.GetRoleCounts()
writeJSON(w, StatsResponse{
resp := &StatsResponse{
TotalPackets: stats.TotalPackets,
TotalTransmissions: &stats.TotalTransmissions,
TotalObservations: stats.TotalObservations,
@@ -411,7 +427,14 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
Companions: counts["companions"],
Sensors: counts["sensors"],
},
})
}
s.statsMu.Lock()
s.statsCache = resp
s.statsCachedAt = time.Now()
s.statsMu.Unlock()
writeJSON(w, resp)
}
func (s *Server) handlePerf(w http.ResponseWriter, r *http.Request) {

View File

@@ -98,6 +98,11 @@ type PacketStore struct {
// computed during Load() and incrementally updated on ingest.
distHops []distHopRecord
distPaths []distPathRecord
// Cached GetNodeHashSizeInfo result — recomputed at most once every 15s
hashSizeInfoMu sync.Mutex
hashSizeInfoCache map[string]*hashSizeNodeInfo
hashSizeInfoAt time.Time
}
// Precomputed distance records for fast analytics aggregation.
@@ -1034,7 +1039,7 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
}
}
// Build broadcast maps (same shape as Node.js WS broadcast)
// Build broadcast maps (same shape as Node.js WS broadcast), one per observation.
result := make([]map[string]interface{}, 0, len(broadcastOrder))
for _, txID := range broadcastOrder {
tx := broadcastTxs[txID]
@@ -1050,44 +1055,34 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
decoded["payload"] = payload
}
}
// Build the nested packet object (packets.js checks m.data.packet)
pkt := map[string]interface{}{
"id": tx.ID,
"raw_hex": strOrNil(tx.RawHex),
"hash": strOrNil(tx.Hash),
"first_seen": strOrNil(tx.FirstSeen),
"timestamp": strOrNil(tx.FirstSeen),
"route_type": intPtrOrNil(tx.RouteType),
"payload_type": intPtrOrNil(tx.PayloadType),
"decoded_json": strOrNil(tx.DecodedJSON),
"observer_id": strOrNil(tx.ObserverID),
"observer_name": strOrNil(tx.ObserverName),
"snr": floatPtrOrNil(tx.SNR),
"rssi": floatPtrOrNil(tx.RSSI),
"path_json": strOrNil(tx.PathJSON),
"direction": strOrNil(tx.Direction),
"observation_count": tx.ObservationCount,
for _, obs := range tx.Observations {
// Build the nested packet object (packets.js checks m.data.packet)
pkt := map[string]interface{}{
"id": tx.ID,
"raw_hex": strOrNil(tx.RawHex),
"hash": strOrNil(tx.Hash),
"first_seen": strOrNil(tx.FirstSeen),
"timestamp": strOrNil(tx.FirstSeen),
"route_type": intPtrOrNil(tx.RouteType),
"payload_type": intPtrOrNil(tx.PayloadType),
"decoded_json": strOrNil(tx.DecodedJSON),
"observer_id": strOrNil(obs.ObserverID),
"observer_name": strOrNil(obs.ObserverName),
"snr": floatPtrOrNil(obs.SNR),
"rssi": floatPtrOrNil(obs.RSSI),
"path_json": strOrNil(obs.PathJSON),
"direction": strOrNil(obs.Direction),
"observation_count": tx.ObservationCount,
}
// Broadcast map: top-level fields for live.js + nested packet for packets.js
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
result = append(result, broadcastMap)
}
// Broadcast map: top-level fields for live.js + nested packet for packets.js
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
result = append(result, broadcastMap)
}
// Invalidate analytics caches since new data was ingested
if len(result) > 0 {
s.cacheMu.Lock()
s.rfCache = make(map[string]*cachedResult)
s.topoCache = make(map[string]*cachedResult)
s.hashCache = make(map[string]*cachedResult)
s.chanCache = make(map[string]*cachedResult)
s.distCache = make(map[string]*cachedResult)
s.subpathCache = make(map[string]*cachedResult)
s.cacheMu.Unlock()
}
return result, newMaxID
@@ -1096,7 +1091,7 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
// IngestNewObservations loads new observations for transmissions already in the
// store. This catches observations that arrive after IngestNewFromDB has already
// advanced past the transmission's ID (fixes #174).
func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) []map[string]interface{} {
if limit <= 0 {
limit = 500
}
@@ -1122,7 +1117,7 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
rows, err := s.db.conn.Query(querySQL, sinceObsID, limit)
if err != nil {
log.Printf("[store] ingest observations query error: %v", err)
return sinceObsID
return nil
}
defer rows.Close()
@@ -1165,20 +1160,16 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
}
if len(obsRows) == 0 {
return sinceObsID
return nil
}
s.mu.Lock()
defer s.mu.Unlock()
newMaxObsID := sinceObsID
updatedTxs := make(map[int]*StoreTx)
broadcastMaps := make([]map[string]interface{}, 0, len(obsRows))
for _, r := range obsRows {
if r.obsID > newMaxObsID {
newMaxObsID = r.obsID
}
// Already ingested (e.g. by IngestNewFromDB in same cycle)
if _, exists := s.byObsID[r.obsID]; exists {
continue
@@ -1221,6 +1212,43 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
}
s.totalObs++
updatedTxs[r.txID] = tx
decoded := map[string]interface{}{
"header": map[string]interface{}{
"payloadTypeName": resolvePayloadTypeName(tx.PayloadType),
},
}
if tx.DecodedJSON != "" {
var payload map[string]interface{}
if json.Unmarshal([]byte(tx.DecodedJSON), &payload) == nil {
decoded["payload"] = payload
}
}
pkt := map[string]interface{}{
"id": tx.ID,
"raw_hex": strOrNil(tx.RawHex),
"hash": strOrNil(tx.Hash),
"first_seen": strOrNil(tx.FirstSeen),
"timestamp": strOrNil(tx.FirstSeen),
"route_type": intPtrOrNil(tx.RouteType),
"payload_type": intPtrOrNil(tx.PayloadType),
"decoded_json": strOrNil(tx.DecodedJSON),
"observer_id": strOrNil(obs.ObserverID),
"observer_name": strOrNil(obs.ObserverName),
"snr": floatPtrOrNil(obs.SNR),
"rssi": floatPtrOrNil(obs.RSSI),
"path_json": strOrNil(obs.PathJSON),
"direction": strOrNil(obs.Direction),
"observation_count": tx.ObservationCount,
}
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
broadcastMaps = append(broadcastMaps, broadcastMap)
}
// Re-pick best observation for updated transmissions and update subpath index
@@ -1261,21 +1289,7 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
}
}
if len(updatedTxs) > 0 {
// Invalidate analytics caches
s.cacheMu.Lock()
s.rfCache = make(map[string]*cachedResult)
s.topoCache = make(map[string]*cachedResult)
s.hashCache = make(map[string]*cachedResult)
s.chanCache = make(map[string]*cachedResult)
s.distCache = make(map[string]*cachedResult)
s.subpathCache = make(map[string]*cachedResult)
s.cacheMu.Unlock()
// analytics caches cleared; no per-cycle log to avoid stdout overhead
}
return newMaxObsID
return broadcastMaps
}
// MaxTransmissionID returns the highest transmission ID in the store.
@@ -3722,8 +3736,26 @@ type hashSizeNodeInfo struct {
Inconsistent bool
}
// GetNodeHashSizeInfo scans advert packets to compute per-node hash size data.
// GetNodeHashSizeInfo returns cached per-node hash size data, recomputing at most every 15s.
func (s *PacketStore) GetNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
const ttl = 15 * time.Second
s.hashSizeInfoMu.Lock()
if s.hashSizeInfoCache != nil && time.Since(s.hashSizeInfoAt) < ttl {
cached := s.hashSizeInfoCache
s.hashSizeInfoMu.Unlock()
return cached
}
s.hashSizeInfoMu.Unlock()
result := s.computeNodeHashSizeInfo()
s.hashSizeInfoMu.Lock()
s.hashSizeInfoCache = result
s.hashSizeInfoAt = time.Now()
s.hashSizeInfoMu.Unlock()
return result
}
// computeNodeHashSizeInfo scans advert packets to compute per-node hash size data.
func (s *PacketStore) computeNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
s.mu.RLock()
defer s.mu.RUnlock()

View File

@@ -1,229 +1,245 @@
package main
import (
"encoding/json"
"log"
"net/http"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 4096,
CheckOrigin: func(r *http.Request) bool { return true },
}
// Hub manages WebSocket clients and broadcasts.
type Hub struct {
mu sync.RWMutex
clients map[*Client]bool
}
// Client is a single WebSocket connection.
type Client struct {
conn *websocket.Conn
send chan []byte
}
func NewHub() *Hub {
return &Hub{
clients: make(map[*Client]bool),
}
}
func (h *Hub) ClientCount() int {
h.mu.RLock()
defer h.mu.RUnlock()
return len(h.clients)
}
func (h *Hub) Register(c *Client) {
h.mu.Lock()
h.clients[c] = true
h.mu.Unlock()
log.Printf("[ws] client connected (%d total)", h.ClientCount())
}
func (h *Hub) Unregister(c *Client) {
h.mu.Lock()
if _, ok := h.clients[c]; ok {
delete(h.clients, c)
close(c.send)
}
h.mu.Unlock()
log.Printf("[ws] client disconnected (%d total)", h.ClientCount())
}
// Broadcast sends a message to all connected clients.
func (h *Hub) Broadcast(msg interface{}) {
data, err := json.Marshal(msg)
if err != nil {
log.Printf("[ws] marshal error: %v", err)
return
}
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
select {
case c.send <- data:
default:
// Client buffer full — drop
}
}
}
// ServeWS handles the WebSocket upgrade and runs the client.
func (h *Hub) ServeWS(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Printf("[ws] upgrade error: %v", err)
return
}
client := &Client{
conn: conn,
send: make(chan []byte, 256),
}
h.Register(client)
go client.writePump()
go client.readPump(h)
}
// wsOrStatic upgrades WebSocket requests at any path, serves static files otherwise.
func wsOrStatic(hub *Hub, static http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") {
hub.ServeWS(w, r)
return
}
static.ServeHTTP(w, r)
})
}
func (c *Client) readPump(hub *Hub) {
defer func() {
hub.Unregister(c)
c.conn.Close()
}()
c.conn.SetReadLimit(512)
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
return nil
})
for {
_, _, err := c.conn.ReadMessage()
if err != nil {
break
}
}
}
func (c *Client) writePump() {
ticker := time.NewTicker(30 * time.Second)
defer func() {
ticker.Stop()
c.conn.Close()
}()
for {
select {
case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return
}
if err := c.conn.WriteMessage(websocket.TextMessage, message); err != nil {
return
}
case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return
}
}
}
}
// Poller watches for new transmissions in SQLite and broadcasts them.
type Poller struct {
db *DB
hub *Hub
store *PacketStore // optional: if set, new transmissions are ingested into memory
interval time.Duration
stop chan struct{}
}
func NewPoller(db *DB, hub *Hub, interval time.Duration) *Poller {
return &Poller{db: db, hub: hub, interval: interval, stop: make(chan struct{})}
}
func (p *Poller) Start() {
lastID := p.db.GetMaxTransmissionID()
lastObsID := p.db.GetMaxObservationID()
log.Printf("[poller] starting from transmission ID %d, obs ID %d, interval %v", lastID, lastObsID, p.interval)
ticker := time.NewTicker(p.interval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
if p.store != nil {
// Ingest new transmissions into in-memory store and broadcast
newTxs, newMax := p.store.IngestNewFromDB(lastID, 100)
if newMax > lastID {
lastID = newMax
}
// Ingest new observations for existing transmissions (fixes #174)
newObsMax := p.store.IngestNewObservations(lastObsID, 500)
if newObsMax > lastObsID {
lastObsID = newObsMax
}
if len(newTxs) > 0 {
log.Printf("[broadcast] sending %d packets to %d clients (lastID now %d)", len(newTxs), p.hub.ClientCount(), lastID)
}
for _, tx := range newTxs {
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: tx,
})
}
} else {
// Fallback: direct DB query (used when store is nil, e.g. tests)
newTxs, err := p.db.GetNewTransmissionsSince(lastID, 100)
if err != nil {
log.Printf("[poller] error: %v", err)
continue
}
for _, tx := range newTxs {
id, _ := tx["id"].(int)
if id > lastID {
lastID = id
}
// Copy packet fields for the nested packet (avoids circular ref)
pkt := make(map[string]interface{}, len(tx))
for k, v := range tx {
pkt[k] = v
}
tx["packet"] = pkt
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: tx,
})
}
}
case <-p.stop:
return
}
}
}
func (p *Poller) Stop() {
close(p.stop)
}
package main
import (
"encoding/json"
"log"
"net/http"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
)
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 4096,
CheckOrigin: func(r *http.Request) bool { return true },
}
// Hub manages WebSocket clients and broadcasts.
type Hub struct {
mu sync.RWMutex
clients map[*Client]bool
}
// Client is a single WebSocket connection.
type Client struct {
conn *websocket.Conn
send chan []byte
}
func NewHub() *Hub {
return &Hub{
clients: make(map[*Client]bool),
}
}
func (h *Hub) ClientCount() int {
h.mu.RLock()
defer h.mu.RUnlock()
return len(h.clients)
}
func (h *Hub) Register(c *Client) {
h.mu.Lock()
h.clients[c] = true
h.mu.Unlock()
log.Printf("[ws] client connected (%d total)", h.ClientCount())
}
func (h *Hub) Unregister(c *Client) {
h.mu.Lock()
if _, ok := h.clients[c]; ok {
delete(h.clients, c)
close(c.send)
}
h.mu.Unlock()
log.Printf("[ws] client disconnected (%d total)", h.ClientCount())
}
// Broadcast sends a message to all connected clients.
func (h *Hub) Broadcast(msg interface{}) {
data, err := json.Marshal(msg)
if err != nil {
log.Printf("[ws] marshal error: %v", err)
return
}
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
select {
case c.send <- data:
default:
// Client buffer full — drop
}
}
}
// ServeWS handles the WebSocket upgrade and runs the client.
func (h *Hub) ServeWS(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Printf("[ws] upgrade error: %v", err)
return
}
client := &Client{
conn: conn,
send: make(chan []byte, 256),
}
h.Register(client)
go client.writePump()
go client.readPump(h)
}
// wsOrStatic upgrades WebSocket requests at any path, serves static files otherwise.
func wsOrStatic(hub *Hub, static http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") {
hub.ServeWS(w, r)
return
}
static.ServeHTTP(w, r)
})
}
func (c *Client) readPump(hub *Hub) {
defer func() {
hub.Unregister(c)
c.conn.Close()
}()
c.conn.SetReadLimit(512)
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
return nil
})
for {
_, _, err := c.conn.ReadMessage()
if err != nil {
break
}
}
}
func (c *Client) writePump() {
ticker := time.NewTicker(30 * time.Second)
defer func() {
ticker.Stop()
c.conn.Close()
}()
for {
select {
case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return
}
if err := c.conn.WriteMessage(websocket.TextMessage, message); err != nil {
return
}
case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return
}
}
}
}
// Poller watches for new transmissions in SQLite and broadcasts them.
type Poller struct {
db *DB
hub *Hub
store *PacketStore // optional: if set, new transmissions are ingested into memory
interval time.Duration
stop chan struct{}
}
func NewPoller(db *DB, hub *Hub, interval time.Duration) *Poller {
return &Poller{db: db, hub: hub, interval: interval, stop: make(chan struct{})}
}
func (p *Poller) Start() {
lastID := p.db.GetMaxTransmissionID()
lastObsID := p.db.GetMaxObservationID()
log.Printf("[poller] starting from transmission ID %d, obs ID %d, interval %v", lastID, lastObsID, p.interval)
ticker := time.NewTicker(p.interval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
if p.store != nil {
// Ingest new transmissions into in-memory store and broadcast
newTxs, newMax := p.store.IngestNewFromDB(lastID, 100)
if newMax > lastID {
lastID = newMax
}
// Ingest new observations for existing transmissions (fixes #174)
nextObsID := lastObsID
if err := p.db.conn.QueryRow(`
SELECT COALESCE(MAX(id), ?) FROM (
SELECT id FROM observations
WHERE id > ?
ORDER BY id ASC
LIMIT 500
)`, lastObsID, lastObsID).Scan(&nextObsID); err != nil {
nextObsID = lastObsID
}
newObs := p.store.IngestNewObservations(lastObsID, 500)
if nextObsID > lastObsID {
lastObsID = nextObsID
}
if len(newTxs) > 0 {
log.Printf("[broadcast] sending %d packets to %d clients (lastID now %d)", len(newTxs), p.hub.ClientCount(), lastID)
}
for _, tx := range newTxs {
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: tx,
})
}
for _, obs := range newObs {
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: obs,
})
}
} else {
// Fallback: direct DB query (used when store is nil, e.g. tests)
newTxs, err := p.db.GetNewTransmissionsSince(lastID, 100)
if err != nil {
log.Printf("[poller] error: %v", err)
continue
}
for _, tx := range newTxs {
id, _ := tx["id"].(int)
if id > lastID {
lastID = id
}
// Copy packet fields for the nested packet (avoids circular ref)
pkt := make(map[string]interface{}, len(tx))
for k, v := range tx {
pkt[k] = v
}
tx["packet"] = pkt
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: tx,
})
}
}
case <-p.stop:
return
}
}
}
func (p *Poller) Stop() {
close(p.stop)
}

View File

@@ -1,275 +1,415 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gorilla/websocket"
)
func TestHubBroadcast(t *testing.T) {
hub := NewHub()
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
// Create a test server with WebSocket endpoint
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hub.ServeWS(w, r)
}))
defer srv.Close()
// Connect a WebSocket client
wsURL := "ws" + srv.URL[4:] // replace http with ws
conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn.Close()
// Wait for registration
time.Sleep(50 * time.Millisecond)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client, got %d", hub.ClientCount())
}
// Broadcast a message
hub.Broadcast(map[string]interface{}{
"type": "packet",
"data": map[string]interface{}{"id": 1, "hash": "test123"},
})
// Read the message
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg, err := conn.ReadMessage()
if err != nil {
t.Fatalf("read error: %v", err)
}
if len(msg) == 0 {
t.Error("expected non-empty message")
}
// Disconnect
conn.Close()
time.Sleep(100 * time.Millisecond)
}
func TestPollerCreation(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
poller := NewPoller(db, hub, 100*time.Millisecond)
if poller == nil {
t.Fatal("expected poller")
}
// Start and stop
go poller.Start()
time.Sleep(200 * time.Millisecond)
poller.Stop()
}
func TestHubMultipleClients(t *testing.T) {
hub := NewHub()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hub.ServeWS(w, r)
}))
defer srv.Close()
wsURL := "ws" + srv.URL[4:]
// Connect two clients
conn1, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn1.Close()
conn2, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn2.Close()
time.Sleep(100 * time.Millisecond)
if hub.ClientCount() != 2 {
t.Errorf("expected 2 clients, got %d", hub.ClientCount())
}
// Broadcast and both should receive
hub.Broadcast(map[string]interface{}{"type": "test", "data": "hello"})
conn1.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg1, err := conn1.ReadMessage()
if err != nil {
t.Fatalf("conn1 read error: %v", err)
}
if len(msg1) == 0 {
t.Error("expected non-empty message on conn1")
}
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg2, err := conn2.ReadMessage()
if err != nil {
t.Fatalf("conn2 read error: %v", err)
}
if len(msg2) == 0 {
t.Error("expected non-empty message on conn2")
}
// Disconnect one
conn1.Close()
time.Sleep(100 * time.Millisecond)
// Remaining client should still work
hub.Broadcast(map[string]interface{}{"type": "test2"})
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg3, err := conn2.ReadMessage()
if err != nil {
t.Fatalf("conn2 read error after disconnect: %v", err)
}
if len(msg3) == 0 {
t.Error("expected non-empty message")
}
}
func TestBroadcastFullBuffer(t *testing.T) {
hub := NewHub()
// Create a client with tiny buffer (1)
client := &Client{
send: make(chan []byte, 1),
}
hub.mu.Lock()
hub.clients[client] = true
hub.mu.Unlock()
// Fill the buffer
client.send <- []byte("first")
// This broadcast should drop the message (buffer full)
hub.Broadcast(map[string]interface{}{"type": "dropped"})
// Channel should still only have the first message
select {
case msg := <-client.send:
if string(msg) != "first" {
t.Errorf("expected 'first', got %s", string(msg))
}
default:
t.Error("expected message in channel")
}
// Clean up
hub.mu.Lock()
delete(hub.clients, client)
hub.mu.Unlock()
}
func TestBroadcastMarshalError(t *testing.T) {
hub := NewHub()
// Marshal error: functions can't be marshaled to JSON
hub.Broadcast(map[string]interface{}{"bad": func() {}})
// Should not panic — just log and return
}
func TestPollerBroadcastsNewData(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
// Create a client to receive broadcasts
client := &Client{
send: make(chan []byte, 256),
}
hub.mu.Lock()
hub.clients[client] = true
hub.mu.Unlock()
poller := NewPoller(db, hub, 50*time.Millisecond)
go poller.Start()
// Insert new data to trigger broadcast
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
VALUES ('EEFF', 'newhash123456789', '2026-01-16T10:00:00Z', 1, 4)`)
time.Sleep(200 * time.Millisecond)
poller.Stop()
// Check if client received broadcast with packet field (fixes #162)
select {
case msg := <-client.send:
if len(msg) == 0 {
t.Error("expected non-empty broadcast message")
}
var parsed map[string]interface{}
if err := json.Unmarshal(msg, &parsed); err != nil {
t.Fatalf("failed to parse broadcast: %v", err)
}
if parsed["type"] != "packet" {
t.Errorf("expected type=packet, got %v", parsed["type"])
}
data, ok := parsed["data"].(map[string]interface{})
if !ok {
t.Fatal("expected data to be an object")
}
// packets.js filters on m.data.packet — must exist
pkt, ok := data["packet"]
if !ok || pkt == nil {
t.Error("expected data.packet to exist (required by packets.js WS handler)")
}
pktMap, ok := pkt.(map[string]interface{})
if !ok {
t.Fatal("expected data.packet to be an object")
}
// Verify key fields exist in nested packet (timestamp required by packets.js)
for _, field := range []string{"id", "hash", "payload_type", "timestamp"} {
if _, exists := pktMap[field]; !exists {
t.Errorf("expected data.packet.%s to exist", field)
}
}
default:
// Might not have received due to timing
}
// Clean up
hub.mu.Lock()
delete(hub.clients, client)
hub.mu.Unlock()
}
func TestHubRegisterUnregister(t *testing.T) {
hub := NewHub()
client := &Client{
send: make(chan []byte, 256),
}
hub.Register(client)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client after register, got %d", hub.ClientCount())
}
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients after unregister, got %d", hub.ClientCount())
}
// Unregister again should be safe
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
}
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"sort"
"testing"
"time"
"github.com/gorilla/websocket"
)
func TestHubBroadcast(t *testing.T) {
hub := NewHub()
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
// Create a test server with WebSocket endpoint
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hub.ServeWS(w, r)
}))
defer srv.Close()
// Connect a WebSocket client
wsURL := "ws" + srv.URL[4:] // replace http with ws
conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn.Close()
// Wait for registration
time.Sleep(50 * time.Millisecond)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client, got %d", hub.ClientCount())
}
// Broadcast a message
hub.Broadcast(map[string]interface{}{
"type": "packet",
"data": map[string]interface{}{"id": 1, "hash": "test123"},
})
// Read the message
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg, err := conn.ReadMessage()
if err != nil {
t.Fatalf("read error: %v", err)
}
if len(msg) == 0 {
t.Error("expected non-empty message")
}
// Disconnect
conn.Close()
time.Sleep(100 * time.Millisecond)
}
func TestPollerCreation(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
poller := NewPoller(db, hub, 100*time.Millisecond)
if poller == nil {
t.Fatal("expected poller")
}
// Start and stop
go poller.Start()
time.Sleep(200 * time.Millisecond)
poller.Stop()
}
func TestHubMultipleClients(t *testing.T) {
hub := NewHub()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hub.ServeWS(w, r)
}))
defer srv.Close()
wsURL := "ws" + srv.URL[4:]
// Connect two clients
conn1, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn1.Close()
conn2, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil {
t.Fatalf("dial error: %v", err)
}
defer conn2.Close()
time.Sleep(100 * time.Millisecond)
if hub.ClientCount() != 2 {
t.Errorf("expected 2 clients, got %d", hub.ClientCount())
}
// Broadcast and both should receive
hub.Broadcast(map[string]interface{}{"type": "test", "data": "hello"})
conn1.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg1, err := conn1.ReadMessage()
if err != nil {
t.Fatalf("conn1 read error: %v", err)
}
if len(msg1) == 0 {
t.Error("expected non-empty message on conn1")
}
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg2, err := conn2.ReadMessage()
if err != nil {
t.Fatalf("conn2 read error: %v", err)
}
if len(msg2) == 0 {
t.Error("expected non-empty message on conn2")
}
// Disconnect one
conn1.Close()
time.Sleep(100 * time.Millisecond)
// Remaining client should still work
hub.Broadcast(map[string]interface{}{"type": "test2"})
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg3, err := conn2.ReadMessage()
if err != nil {
t.Fatalf("conn2 read error after disconnect: %v", err)
}
if len(msg3) == 0 {
t.Error("expected non-empty message")
}
}
func TestBroadcastFullBuffer(t *testing.T) {
hub := NewHub()
// Create a client with tiny buffer (1)
client := &Client{
send: make(chan []byte, 1),
}
hub.mu.Lock()
hub.clients[client] = true
hub.mu.Unlock()
// Fill the buffer
client.send <- []byte("first")
// This broadcast should drop the message (buffer full)
hub.Broadcast(map[string]interface{}{"type": "dropped"})
// Channel should still only have the first message
select {
case msg := <-client.send:
if string(msg) != "first" {
t.Errorf("expected 'first', got %s", string(msg))
}
default:
t.Error("expected message in channel")
}
// Clean up
hub.mu.Lock()
delete(hub.clients, client)
hub.mu.Unlock()
}
func TestBroadcastMarshalError(t *testing.T) {
hub := NewHub()
// Marshal error: functions can't be marshaled to JSON
hub.Broadcast(map[string]interface{}{"bad": func() {}})
// Should not panic — just log and return
}
func TestPollerBroadcastsNewData(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
// Create a client to receive broadcasts
client := &Client{
send: make(chan []byte, 256),
}
hub.mu.Lock()
hub.clients[client] = true
hub.mu.Unlock()
poller := NewPoller(db, hub, 50*time.Millisecond)
go poller.Start()
// Insert new data to trigger broadcast
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
VALUES ('EEFF', 'newhash123456789', '2026-01-16T10:00:00Z', 1, 4)`)
time.Sleep(200 * time.Millisecond)
poller.Stop()
// Check if client received broadcast with packet field (fixes #162)
select {
case msg := <-client.send:
if len(msg) == 0 {
t.Error("expected non-empty broadcast message")
}
var parsed map[string]interface{}
if err := json.Unmarshal(msg, &parsed); err != nil {
t.Fatalf("failed to parse broadcast: %v", err)
}
if parsed["type"] != "packet" {
t.Errorf("expected type=packet, got %v", parsed["type"])
}
data, ok := parsed["data"].(map[string]interface{})
if !ok {
t.Fatal("expected data to be an object")
}
// packets.js filters on m.data.packet — must exist
pkt, ok := data["packet"]
if !ok || pkt == nil {
t.Error("expected data.packet to exist (required by packets.js WS handler)")
}
pktMap, ok := pkt.(map[string]interface{})
if !ok {
t.Fatal("expected data.packet to be an object")
}
// Verify key fields exist in nested packet (timestamp required by packets.js)
for _, field := range []string{"id", "hash", "payload_type", "timestamp"} {
if _, exists := pktMap[field]; !exists {
t.Errorf("expected data.packet.%s to exist", field)
}
}
default:
// Might not have received due to timing
}
// Clean up
hub.mu.Lock()
delete(hub.clients, client)
hub.mu.Unlock()
}
func TestPollerBroadcastsMultipleObservations(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
client := &Client{
send: make(chan []byte, 256),
}
hub.mu.Lock()
hub.clients[client] = true
hub.mu.Unlock()
defer func() {
hub.mu.Lock()
delete(hub.clients, client)
hub.mu.Unlock()
}()
poller := NewPoller(db, hub, 50*time.Millisecond)
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store load failed: %v", err)
}
poller.store = store
go poller.Start()
defer poller.Stop()
// Wait for poller to initialize its lastID/lastObsID cursors before
// inserting new data; otherwise the poller may snapshot a lastID that
// already includes the test data and never broadcast it.
time.Sleep(100 * time.Millisecond)
now := time.Now().UTC().Format(time.RFC3339)
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('FACE', 'starbursthash237a', ?, 1, 4, '{"pubKey":"aabbccdd11223344","type":"ADVERT"}')`, now); err != nil {
t.Fatalf("insert tx failed: %v", err)
}
var txID int
if err := db.conn.QueryRow(`SELECT id FROM transmissions WHERE hash='starbursthash237a'`).Scan(&txID); err != nil {
t.Fatalf("query tx id failed: %v", err)
}
ts := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (?, 1, 14.0, -82, '["aa"]', ?),
(?, 2, 10.5, -90, '["aa","bb"]', ?),
(?, 1, 7.0, -96, '["aa","bb","cc"]', ?)`,
txID, ts, txID, ts+1, txID, ts+2); err != nil {
t.Fatalf("insert observations failed: %v", err)
}
deadline := time.After(2 * time.Second)
var dataMsgs []map[string]interface{}
for len(dataMsgs) < 3 {
select {
case raw := <-client.send:
var parsed map[string]interface{}
if err := json.Unmarshal(raw, &parsed); err != nil {
t.Fatalf("unmarshal ws msg failed: %v", err)
}
if parsed["type"] != "packet" {
continue
}
data, ok := parsed["data"].(map[string]interface{})
if !ok {
continue
}
if data["hash"] == "starbursthash237a" {
dataMsgs = append(dataMsgs, data)
}
case <-deadline:
t.Fatalf("timed out waiting for 3 observation broadcasts, got %d", len(dataMsgs))
}
}
if len(dataMsgs) != 3 {
t.Fatalf("expected 3 messages, got %d", len(dataMsgs))
}
paths := make([]string, 0, 3)
observers := make(map[string]bool)
for _, m := range dataMsgs {
hash, _ := m["hash"].(string)
if hash != "starbursthash237a" {
t.Fatalf("unexpected hash %q", hash)
}
p, _ := m["path_json"].(string)
paths = append(paths, p)
if oid, ok := m["observer_id"].(string); ok && oid != "" {
observers[oid] = true
}
}
sort.Strings(paths)
wantPaths := []string{`["aa","bb","cc"]`, `["aa","bb"]`, `["aa"]`}
sort.Strings(wantPaths)
for i := range wantPaths {
if paths[i] != wantPaths[i] {
t.Fatalf("path mismatch at %d: got %q want %q", i, paths[i], wantPaths[i])
}
}
if len(observers) < 2 {
t.Fatalf("expected observations from >=2 observers, got %d", len(observers))
}
}
func TestIngestNewObservationsBroadcast(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store load failed: %v", err)
}
maxObs := db.GetMaxObservationID()
now := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 2, 6.0, -100, '["aa","zz"]', ?),
(1, 1, 5.0, -101, '["aa","yy"]', ?)`, now, now+1); err != nil {
t.Fatalf("insert new observations failed: %v", err)
}
maps := store.IngestNewObservations(maxObs, 500)
if len(maps) != 2 {
t.Fatalf("expected 2 broadcast maps, got %d", len(maps))
}
for _, m := range maps {
if m["hash"] != "abc123def4567890" {
t.Fatalf("unexpected hash in map: %v", m["hash"])
}
path, ok := m["path_json"].(string)
if !ok || path == "" {
t.Fatalf("missing path_json in map: %#v", m)
}
if _, ok := m["observer_id"]; !ok {
t.Fatalf("missing observer_id in map: %#v", m)
}
}
}
func TestHubRegisterUnregister(t *testing.T) {
hub := NewHub()
client := &Client{
send: make(chan []byte, 256),
}
hub.Register(client)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client after register, got %d", hub.ClientCount())
}
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients after unregister, got %d", hub.ClientCount())
}
// Unregister again should be safe
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
}

View File

@@ -98,6 +98,13 @@
"#bookclub",
"#shtf"
],
"healthThresholds": {
"infraDegradedHours": 24,
"infraSilentHours": 72,
"nodeDegradedHours": 1,
"nodeSilentHours": 24,
"_comment": "How long (hours) before nodes show as degraded/silent. 'infra' = repeaters & rooms, 'node' = companions & others."
},
"defaultRegion": "SJC",
"mapDefaults": {
"center": [

View File

@@ -2,8 +2,8 @@
* MeshCore Packet Decoder
* Custom implementation — does NOT use meshcore-decoder library (known path_length bug).
*
* Packet layout:
* [header(1)] [pathLength(1)] [transportCodes?] [path hops] [payload...]
* Packet layout (per firmware docs/packet_format.md):
* [header(1)] [transportCodes?(4)] [pathLength(1)] [path hops] [payload...]
*
* Header byte (LSB first):
* bits 1-0: routeType (0=TRANSPORT_FLOOD, 1=FLOOD, 2=DIRECT, 3=TRANSPORT_DIRECT)
@@ -42,7 +42,7 @@ const PAYLOAD_TYPES = {
0x0F: 'RAW_CUSTOM',
};
// Route types that carry transport codes (nextHop + lastHop, 2 bytes each)
// Route types that carry transport codes (2x uint16_t, 4 bytes total)
const TRANSPORT_ROUTES = new Set([0, 3]); // TRANSPORT_FLOOD, TRANSPORT_DIRECT
// --- Header parsing ---
@@ -94,13 +94,11 @@ function decodeEncryptedPayload(buf) {
};
}
/** ACK: dest(1) + src(1) + ack_hash(4) (per Mesh.cpp) */
/** ACK: checksum(4) — CRC of message timestamp + text + sender pubkey (per Mesh.cpp createAck) */
function decodeAck(buf) {
if (buf.length < 6) return { error: 'too short', raw: buf.toString('hex') };
if (buf.length < 4) return { error: 'too short', raw: buf.toString('hex') };
return {
destHash: buf.subarray(0, 1).toString('hex'),
srcHash: buf.subarray(1, 2).toString('hex'),
extraHash: buf.subarray(2, 6).toString('hex'),
ackChecksum: buf.subarray(0, 4).toString('hex'),
};
}
@@ -125,6 +123,8 @@ function decodeAdvert(buf) {
room: advType === 3,
sensor: advType === 4,
hasLocation: !!(flags & 0x10),
hasFeat1: !!(flags & 0x20),
hasFeat2: !!(flags & 0x40),
hasName: !!(flags & 0x80),
};
@@ -134,6 +134,14 @@ function decodeAdvert(buf) {
result.lon = appdata.readInt32LE(off + 4) / 1e6;
off += 8;
}
if (result.flags.hasFeat1 && appdata.length >= off + 2) {
result.feat1 = appdata.readUInt16LE(off);
off += 2;
}
if (result.flags.hasFeat2 && appdata.length >= off + 2) {
result.feat2 = appdata.readUInt16LE(off);
off += 2;
}
if (result.flags.hasName) {
// Find null terminator to separate name from trailing telemetry bytes
let nameEnd = appdata.length;
@@ -231,7 +239,7 @@ function decodeGrpTxt(buf, channelKeys) {
return { type: 'GRP_TXT', channelHash, channelHashHex, decryptionStatus: 'no_key', mac, encryptedData };
}
/** ANON_REQ: dest(6) + ephemeral_pubkey(32) + MAC(4) + encrypted */
/** ANON_REQ: dest(1) + ephemeral_pubkey(32) + MAC(2) + encrypted */
function decodeAnonReq(buf) {
if (buf.length < 35) return { error: 'too short', raw: buf.toString('hex') };
return {
@@ -242,7 +250,7 @@ function decodeAnonReq(buf) {
};
}
/** PATH: dest(6) + src(6) + MAC(4) + path_data */
/** PATH: dest(1) + src(1) + MAC(2) + path_data */
function decodePath_payload(buf) {
if (buf.length < 4) return { error: 'too short', raw: buf.toString('hex') };
return {
@@ -253,14 +261,14 @@ function decodePath_payload(buf) {
};
}
/** TRACE: flags(1) + tag(4) + dest(6) + src(1) */
/** TRACE: tag(4) + authCode(4) + flags(1) + pathData (per Mesh.cpp onRecvPacket TRACE) */
function decodeTrace(buf) {
if (buf.length < 12) return { error: 'too short', raw: buf.toString('hex') };
if (buf.length < 9) return { error: 'too short', raw: buf.toString('hex') };
return {
flags: buf[0],
tag: buf.readUInt32LE(1),
destHash: buf.subarray(5, 11).toString('hex'),
srcHash: buf.subarray(11, 12).toString('hex'),
tag: buf.readUInt32LE(0),
authCode: buf.subarray(4, 8).toString('hex'),
flags: buf[8],
pathData: buf.subarray(9).toString('hex'),
};
}
@@ -289,20 +297,22 @@ function decodePacket(hexString, channelKeys) {
if (buf.length < 2) throw new Error('Packet too short (need at least header + pathLength)');
const header = decodeHeader(buf[0]);
const pathByte = buf[1];
let offset = 2;
let offset = 1;
// Transport codes for TRANSPORT_FLOOD / TRANSPORT_DIRECT
// Transport codes for TRANSPORT_FLOOD / TRANSPORT_DIRECT — BEFORE path_length per spec
let transportCodes = null;
if (TRANSPORT_ROUTES.has(header.routeType)) {
if (buf.length < offset + 4) throw new Error('Packet too short for transport codes');
transportCodes = {
nextHop: buf.subarray(offset, offset + 2).toString('hex').toUpperCase(),
lastHop: buf.subarray(offset + 2, offset + 4).toString('hex').toUpperCase(),
code1: buf.subarray(offset, offset + 2).toString('hex').toUpperCase(),
code2: buf.subarray(offset + 2, offset + 4).toString('hex').toUpperCase(),
};
offset += 4;
}
// Path length byte — AFTER transport codes per spec
const pathByte = buf[offset++];
// Path
const path = decodePath(pathByte, buf, offset);
offset += path.bytesConsumed;
@@ -386,7 +396,7 @@ module.exports = { decodePacket, validateAdvert, hasNonPrintableChars, ROUTE_TYP
// --- Tests ---
if (require.main === module) {
console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Test Repeater" ===');
console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Kpa Roof Solar" ===');
const pkt1 = decodePacket(
'11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172'
);
@@ -402,7 +412,7 @@ if (require.main === module) {
assert(pkt1.path.hops[0] === '1000', 'first hop should be 1000');
assert(pkt1.path.hops[1] === 'D818', 'second hop should be D818');
assert(pkt1.transportCodes === null, 'FLOOD has no transport codes');
assert(pkt1.payload.name === 'Test Repeater', 'name should be "Test Repeater"');
assert(pkt1.payload.name === 'Kpa Roof Solar', 'name should be "Kpa Roof Solar"');
console.log('✅ Test 1 passed\n');
console.log('=== Test 2: ADVERT, FLOOD, 0 hops (zero-path) ===');

View File

@@ -5,9 +5,12 @@
services:
prod:
build: .
image: corescope:latest
container_name: corescope-prod
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${PROD_HTTP_PORT:-80}:${PROD_HTTP_PORT:-80}"
- "${PROD_HTTPS_PORT:-443}:${PROD_HTTPS_PORT:-443}"
@@ -26,9 +29,12 @@ services:
retries: 3
staging:
build: .
image: corescope:latest
container_name: corescope-staging
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${STAGING_HTTP_PORT:-81}:${STAGING_HTTP_PORT:-81}"
- "${STAGING_MQTT_PORT:-1884}:1883"
@@ -57,6 +63,8 @@ services:
image: corescope-go:latest
container_name: corescope-staging-go
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "${STAGING_GO_HTTP_PORT:-82}:80"
- "${STAGING_GO_MQTT_PORT:-1885}:1883"

View File

@@ -22,9 +22,9 @@
<meta name="twitter:title" content="CoreScope">
<meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis.">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1774731523">
<link rel="stylesheet" href="home.css?v=1774731523">
<link rel="stylesheet" href="live.css?v=1774731523">
<link rel="stylesheet" href="style.css?v=1774786038">
<link rel="stylesheet" href="home.css?v=1774786038">
<link rel="stylesheet" href="live.css?v=1774786038">
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
crossorigin="anonymous">
@@ -81,29 +81,29 @@
<main id="app" role="main"></main>
<script src="vendor/qrcode.js"></script>
<script src="roles.js?v=1774731523"></script>
<script src="customize.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="region-filter.js?v=1774731523"></script>
<script src="hop-resolver.js?v=1774731523"></script>
<script src="hop-display.js?v=1774731523"></script>
<script src="app.js?v=1774731523"></script>
<script src="home.js?v=1774731523"></script>
<script src="packet-filter.js?v=1774731523"></script>
<script src="packets.js?v=1774731523"></script>
<script src="map.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v1-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="compare.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script>
<script src="roles.js?v=1774786038"></script>
<script src="customize.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="region-filter.js?v=1774786038"></script>
<script src="hop-resolver.js?v=1774786038"></script>
<script src="hop-display.js?v=1774786038"></script>
<script src="app.js?v=1774786038"></script>
<script src="home.js?v=1774786038"></script>
<script src="packet-filter.js?v=1774786038"></script>
<script src="packets.js?v=1774786038"></script>
<script src="map.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v1-constellation.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="compare.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
</body>
</html>

View File

@@ -89,7 +89,8 @@
function getStatusTooltip(role, status) {
const isInfra = role === 'repeater' || role === 'room';
const threshold = isInfra ? '72h' : '24h';
const threshMs = isInfra ? HEALTH_THRESHOLDS.infraSilentMs : HEALTH_THRESHOLDS.nodeSilentMs;
const threshold = threshMs >= 3600000 ? Math.round(threshMs / 3600000) + 'h' : Math.round(threshMs / 60000) + 'm';
if (status === 'active') {
return 'Active \u2014 heard within the last ' + threshold + '.' + (isInfra ? ' Repeaters typically advertise every 12-24h.' : '');
}

View File

@@ -1512,14 +1512,12 @@
rows += fieldRow(off + 1, 'Sender', decoded.sender || '—', '');
if (decoded.sender_timestamp) rows += fieldRow(off + 2, 'Sender Time', decoded.sender_timestamp, '');
} else if (decoded.type === 'ACK') {
rows += fieldRow(off, 'Dest Hash (6B)', decoded.destHash || '', '');
rows += fieldRow(off + 6, 'Src Hash (6B)', decoded.srcHash || '', '');
rows += fieldRow(off + 12, 'Extra (6B)', decoded.extraHash || '', '');
rows += fieldRow(off, 'Checksum (4B)', decoded.ackChecksum || '', '');
} else if (decoded.destHash !== undefined) {
rows += fieldRow(off, 'Dest Hash (6B)', decoded.destHash || '', '');
rows += fieldRow(off + 6, 'Src Hash (6B)', decoded.srcHash || '', '');
rows += fieldRow(off + 12, 'MAC (4B)', decoded.mac || '', '');
rows += fieldRow(off + 16, 'Encrypted Data', truncate(decoded.encryptedData || '', 30), '');
rows += fieldRow(off, 'Dest Hash (1B)', decoded.destHash || '', '');
rows += fieldRow(off + 1, 'Src Hash (1B)', decoded.srcHash || '', '');
rows += fieldRow(off + 2, 'MAC (2B)', decoded.mac || '', '');
rows += fieldRow(off + 4, 'Encrypted Data', truncate(decoded.encryptedData || '', 30), '');
} else {
rows += fieldRow(off, 'Raw', truncate(buf.slice(off * 2), 40), '');
}

View File

@@ -40,12 +40,12 @@
html += `<h3>🔧 Go Runtime</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${gr.goroutines}</div><div class="perf-label">Goroutines</div></div>
<div class="perf-card"><div class="perf-num">${gr.numGC}</div><div class="perf-label">GC Collections</div></div>
<div class="perf-card"><div class="perf-num" style="color:${gcColor}">${gr.pauseTotalMs}ms</div><div class="perf-label">GC Pause Total</div></div>
<div class="perf-card"><div class="perf-num">${gr.lastPauseMs}ms</div><div class="perf-label">Last GC Pause</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapAllocMB}MB</div><div class="perf-label">Heap Alloc</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapSysMB}MB</div><div class="perf-label">Heap Sys</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapInuseMB}MB</div><div class="perf-label">Heap Inuse</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapIdleMB}MB</div><div class="perf-label">Heap Idle</div></div>
<div class="perf-card"><div class="perf-num" style="color:${gcColor}">${(+gr.pauseTotalMs).toFixed(1)}ms</div><div class="perf-label">GC Pause Total</div></div>
<div class="perf-card"><div class="perf-num">${(+gr.lastPauseMs).toFixed(1)}ms</div><div class="perf-label">Last GC Pause</div></div>
<div class="perf-card"><div class="perf-num">${(+gr.heapAllocMB).toFixed(1)}MB</div><div class="perf-label">Heap Alloc</div></div>
<div class="perf-card"><div class="perf-num">${(+gr.heapSysMB).toFixed(1)}MB</div><div class="perf-label">Heap Sys</div></div>
<div class="perf-card"><div class="perf-num">${(+gr.heapInuseMB).toFixed(1)}MB</div><div class="perf-label">Heap Inuse</div></div>
<div class="perf-card"><div class="perf-num">${(+gr.heapIdleMB).toFixed(1)}MB</div><div class="perf-label">Heap Idle</div></div>
<div class="perf-card"><div class="perf-num">${gr.numCPU}</div><div class="perf-label">CPUs</div></div>
<div class="perf-card"><div class="perf-num">${health.websocket.clients}</div><div class="perf-label">WS Clients</div></div>
</div>`;

View File

@@ -155,7 +155,7 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
/* === Nav Stats === */
.nav-stats {
display: flex; gap: 12px; align-items: center; font-size: 12px; color: var(--nav-text-muted);
font-family: var(--mono); margin-right: 4px;
font-family: var(--mono); margin-right: 4px; white-space: nowrap;
}
.nav-stats .stat-val { color: var(--nav-text); font-weight: 600; transition: color 0.3s ease; }
.nav-stats .stat-val.updated { color: var(--accent); }

142
scripts/capture-fixture.sh Executable file
View File

@@ -0,0 +1,142 @@
#!/bin/bash
# Capture a fixture DB from staging for E2E tests
# Usage: ./scripts/capture-fixture.sh [source_url]
#
# Downloads nodes, observers, and recent packets from the staging API
# and creates a SQLite database suitable for E2E testing.
set -e
SOURCE_URL="${1:-https://analyzer.00id.net}"
DB_PATH="test-fixtures/e2e-fixture.db"
echo "Capturing fixture from $SOURCE_URL..."
mkdir -p test-fixtures
rm -f "$DB_PATH"
# Create schema
sqlite3 "$DB_PATH" <<'SQL'
CREATE TABLE nodes (
public_key TEXT PRIMARY KEY,
name TEXT,
role TEXT,
lat REAL,
lon REAL,
last_seen TEXT,
first_seen TEXT,
advert_count INTEGER DEFAULT 0,
battery_mv INTEGER,
temperature_c REAL
);
CREATE TABLE observers (
id TEXT PRIMARY KEY,
name TEXT,
iata TEXT,
last_seen TEXT,
first_seen TEXT,
packet_count INTEGER DEFAULT 0,
model TEXT,
firmware TEXT,
client_version TEXT,
radio TEXT,
battery_mv INTEGER,
uptime_secs INTEGER,
noise_floor REAL
);
CREATE TABLE transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
observer_idx INTEGER,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp INTEGER NOT NULL
);
SQL
# Fetch nodes
echo "Fetching nodes..."
curl -sf "$SOURCE_URL/api/nodes?limit=200" | python3 -c "
import json, sys, sqlite3
data = json.load(sys.stdin)
nodes = data.get('nodes', data) if isinstance(data, dict) else data
db = sqlite3.connect('$DB_PATH')
for n in nodes[:200]:
db.execute('INSERT OR IGNORE INTO nodes VALUES (?,?,?,?,?,?,?,?,?,?)',
(n.get('public_key',''), n.get('name',''), n.get('role',''),
n.get('lat'), n.get('lon'), n.get('last_seen',''), n.get('first_seen',''),
n.get('advert_count',0), n.get('battery_mv'), n.get('temperature_c')))
db.commit()
print(f' Inserted {min(len(nodes), 200)} nodes')
db.close()
"
# Fetch observers
echo "Fetching observers..."
curl -sf "$SOURCE_URL/api/observers" | python3 -c "
import json, sys, sqlite3
data = json.load(sys.stdin)
observers = data.get('observers', data) if isinstance(data, dict) else data
db = sqlite3.connect('$DB_PATH')
for o in observers:
db.execute('INSERT OR IGNORE INTO observers VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)',
(o.get('id',''), o.get('name',''), o.get('iata',''),
o.get('last_seen',''), o.get('first_seen',''),
o.get('packet_count',0), o.get('model',''), o.get('firmware',''),
o.get('client_version',''), o.get('radio',''),
o.get('battery_mv'), o.get('uptime_secs'), o.get('noise_floor')))
db.commit()
print(f' Inserted {len(observers)} observers')
db.close()
"
# Fetch recent packets
echo "Fetching recent packets..."
curl -sf "$SOURCE_URL/api/packets?limit=500" | python3 -c "
import json, sys, sqlite3
data = json.load(sys.stdin)
packets = data.get('packets', data) if isinstance(data, dict) else data
db = sqlite3.connect('$DB_PATH')
for p in packets:
try:
cur = db.execute('INSERT OR IGNORE INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json) VALUES (?,?,?,?,?,?,?)',
(p.get('raw_hex',''), p.get('hash',''), p.get('first_seen',''),
p.get('route_type'), p.get('payload_type'), p.get('payload_version'),
p.get('decoded_json')))
tid = cur.lastrowid
if tid and p.get('observer_id'):
db.execute('INSERT INTO observations (transmission_id, observer_idx, direction, snr, rssi, score, path_json, timestamp) VALUES (?,?,?,?,?,?,?,?)',
(tid, p.get('observer_id'), p.get('direction'),
p.get('snr'), p.get('rssi'), None,
p.get('path_json'),
int(p.get('timestamp','0')) if p.get('timestamp','').isdigit() else 0))
except Exception as e:
pass # Skip duplicates
db.commit()
print(f' Inserted {len(packets)} transmissions')
db.close()
"
SIZE=$(du -h "$DB_PATH" | cut -f1)
echo "✅ Fixture DB created: $DB_PATH ($SIZE)"
echo " Nodes: $(sqlite3 "$DB_PATH" 'SELECT COUNT(*) FROM nodes')"
echo " Observers: $(sqlite3 "$DB_PATH" 'SELECT COUNT(*) FROM observers')"
echo " Transmissions: $(sqlite3 "$DB_PATH" 'SELECT COUNT(*) FROM transmissions')"
echo " Observations: $(sqlite3 "$DB_PATH" 'SELECT COUNT(*) FROM observations')"

View File

@@ -18,10 +18,16 @@ async function collectCoverage() {
page.setDefaultTimeout(10000);
const BASE = process.env.BASE_URL || 'http://localhost:13581';
// Helper: safe click
// Helper: navigate via hash (SPA — no full page reload needed after initial load)
async function navHash(hash, wait = 150) {
await page.evaluate((h) => { location.hash = h; }, hash);
await new Promise(r => setTimeout(r, wait));
}
// Helper: safe click — 500ms timeout (elements exist immediately or not at all)
async function safeClick(selector, timeout) {
try {
await page.click(selector, { timeout: timeout || 3000 });
await page.click(selector, { timeout: timeout || 500 });
} catch {}
}
@@ -64,9 +70,9 @@ async function collectCoverage() {
// ══════════════════════════════════════════════
console.log(' [coverage] Home page — chooser...');
// Clear localStorage to get chooser
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(BASE, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click "I'm new"
await safeClick('#chooseNew');
@@ -105,7 +111,7 @@ async function collectCoverage() {
// Switch to experienced mode
await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#chooseExp');
// Interact with experienced home page
@@ -120,7 +126,7 @@ async function collectCoverage() {
// NODES PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Nodes page...');
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/nodes');
// Sort by EVERY column
for (const col of ['name', 'public_key', 'role', 'last_seen', 'advert_count']) {
@@ -156,7 +162,7 @@ async function collectCoverage() {
}
// In side pane — click detail/analytics links
await safeClick('a[href*="/nodes/"]', 2000);
await safeClick('a[href*="/nodes/"]');
// Click fav star
await clickAll('.fav-star', 2);
@@ -168,7 +174,7 @@ async function collectCoverage() {
try {
const firstNodeKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim());
if (firstNodeKey) {
await page.goto(`${BASE}/#/nodes/${firstNodeKey}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/nodes/' + firstNodeKey);
// Click tabs on detail page
await clickAll('.tab-btn, [data-tab]', 10);
@@ -191,7 +197,7 @@ async function collectCoverage() {
try {
const firstKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim()).catch(() => null);
if (firstKey) {
await page.goto(`${BASE}/#/nodes/${firstKey}?scroll=paths`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/nodes/' + firstKey + '?scroll=paths');
}
} catch {}
@@ -199,7 +205,7 @@ async function collectCoverage() {
// PACKETS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Packets page...');
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/packets');
// Open filter bar
await safeClick('#filterToggleBtn');
@@ -285,13 +291,13 @@ async function collectCoverage() {
} catch {}
// Navigate to specific packet by hash
await page.goto(`${BASE}/#/packets/deadbeef`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/packets/deadbeef');
// ══════════════════════════════════════════════
// MAP PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Map page...');
await page.goto(`${BASE}/#/map`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/map');
// Toggle controls panel
await safeClick('#mapControlsToggle');
@@ -345,7 +351,7 @@ async function collectCoverage() {
// ANALYTICS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Analytics page...');
await page.goto(`${BASE}/#/analytics`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/analytics');
// Click EVERY analytics tab
const analyticsTabs = ['overview', 'rf', 'topology', 'channels', 'hashsizes', 'collisions', 'subpaths', 'nodes', 'distance'];
@@ -381,9 +387,12 @@ async function collectCoverage() {
await clickAll('.analytics-table th', 8);
} catch {}
// Deep-link to each analytics tab via URL
// Deep-link to each analytics tab via hash (avoid full page.goto)
for (const tab of analyticsTabs) {
await page.goto(`${BASE}/#/analytics?tab=${tab}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
try {
await page.evaluate((t) => { location.hash = '#/analytics?tab=' + t; }, tab);
await new Promise(r => setTimeout(r, 100));
} catch {}
}
// Region filter on analytics
@@ -396,7 +405,7 @@ async function collectCoverage() {
// CUSTOMIZE
// ══════════════════════════════════════════════
console.log(' [coverage] Customizer...');
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/home');
await safeClick('#customizeToggle');
// Click EVERY customizer tab
@@ -503,7 +512,7 @@ async function collectCoverage() {
// CHANNELS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Channels page...');
await page.goto(`${BASE}/#/channels`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/channels');
// Click channel rows/items
await clickAll('.channel-item, .channel-row, .channel-card', 3);
await clickAll('table tbody tr', 3);
@@ -512,7 +521,7 @@ async function collectCoverage() {
try {
const channelHash = await page.$eval('table tbody tr td:first-child', el => el.textContent.trim()).catch(() => null);
if (channelHash) {
await page.goto(`${BASE}/#/channels/${channelHash}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/channels/' + channelHash);
}
} catch {}
@@ -520,7 +529,7 @@ async function collectCoverage() {
// LIVE PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Live page...');
await page.goto(`${BASE}/#/live`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/live');
// VCR controls
await safeClick('#vcrPauseBtn');
@@ -603,14 +612,14 @@ async function collectCoverage() {
// TRACES PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Traces page...');
await page.goto(`${BASE}/#/traces`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/traces');
await clickAll('table tbody tr', 3);
// ══════════════════════════════════════════════
// OBSERVERS PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Observers page...');
await page.goto(`${BASE}/#/observers`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/observers');
// Click observer rows
const obsRows = await page.$$('table tbody tr, .observer-card, .observer-row');
for (let i = 0; i < Math.min(obsRows.length, 3); i++) {
@@ -631,7 +640,7 @@ async function collectCoverage() {
// PERF PAGE
// ══════════════════════════════════════════════
console.log(' [coverage] Perf page...');
await page.goto(`${BASE}/#/perf`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/perf');
await safeClick('#perfRefresh');
await safeClick('#perfReset');
@@ -641,14 +650,14 @@ async function collectCoverage() {
console.log(' [coverage] App.js — router + global...');
// Navigate to bad route to trigger error/404
await page.goto(`${BASE}/#/nonexistent-route`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await navHash('#/nonexistent-route');
// Navigate to every route via hash
// Navigate to every route via hash (50ms is enough for SPA hash routing)
const allRoutes = ['home', 'nodes', 'packets', 'map', 'live', 'channels', 'traces', 'observers', 'analytics', 'perf'];
for (const route of allRoutes) {
try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {});
await new Promise(r => setTimeout(r, 50));
} catch {}
}
@@ -714,10 +723,11 @@ async function collectCoverage() {
await page.evaluate(() => { if (window.apiPerf) window.apiPerf(); });
} catch {}
// Exercise utility functions
// Exercise utility functions + packet filter parser in one evaluate call
console.log(' [coverage] Utility functions + packet filter...');
try {
await page.evaluate(() => {
// timeAgo with various inputs
// Utility functions
if (typeof timeAgo === 'function') {
timeAgo(null);
timeAgo(new Date().toISOString());
@@ -725,13 +735,11 @@ async function collectCoverage() {
timeAgo(new Date(Date.now() - 3600000).toISOString());
timeAgo(new Date(Date.now() - 86400000 * 2).toISOString());
}
// truncate
if (typeof truncate === 'function') {
truncate('hello world', 5);
truncate(null, 5);
truncate('hi', 10);
}
// routeTypeName, payloadTypeName, payloadTypeColor
if (typeof routeTypeName === 'function') {
for (let i = 0; i <= 4; i++) routeTypeName(i);
}
@@ -741,23 +749,14 @@ async function collectCoverage() {
if (typeof payloadTypeColor === 'function') {
for (let i = 0; i <= 15; i++) payloadTypeColor(i);
}
// invalidateApiCache
if (typeof invalidateApiCache === 'function') {
invalidateApiCache();
invalidateApiCache('/test');
}
});
} catch {}
// ══════════════════════════════════════════════
// PACKET FILTER — exercise the filter parser
// ══════════════════════════════════════════════
console.log(' [coverage] Packet filter parser...');
try {
await page.evaluate(() => {
// Packet filter parser
if (window.PacketFilter && window.PacketFilter.compile) {
const PF = window.PacketFilter;
// Valid expressions
const exprs = [
'type == ADVERT', 'type == GRP_TXT', 'type != ACK',
'snr > 0', 'snr < -5', 'snr >= 10', 'snr <= 3',
@@ -773,7 +772,6 @@ async function collectCoverage() {
for (const e of exprs) {
try { PF.compile(e); } catch {}
}
// Bad expressions
const bad = ['@@@', '== ==', '(((', 'type ==', ''];
for (const e of bad) {
try { PF.compile(e); } catch {}
@@ -787,29 +785,24 @@ async function collectCoverage() {
// ══════════════════════════════════════════════
console.log(' [coverage] Region filter...');
try {
// Open region filter on nodes page
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
// Open region filter on nodes page (use hash nav, already visited)
await page.evaluate(() => { location.hash = '#/nodes'; });
await new Promise(r => setTimeout(r, 100));
await safeClick('#nodesRegionFilter');
await clickAll('#nodesRegionFilter input[type="checkbox"]', 3);
} catch {}
// Region filter on packets
try {
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {});
await page.evaluate(() => { location.hash = '#/packets'; });
await new Promise(r => setTimeout(r, 100));
await safeClick('#packetsRegionFilter');
await clickAll('#packetsRegionFilter input[type="checkbox"]', 3);
} catch {}
// ══════════════════════════════════════════════
// FINAL — navigate through all routes once more
// FINAL — extract coverage (all routes already visited above)
// ══════════════════════════════════════════════
console.log(' [coverage] Final route sweep...');
for (const route of allRoutes) {
try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {});
} catch {}
}
// Extract coverage
const coverage = await page.evaluate(() => window.__coverage__);

View File

@@ -36,18 +36,19 @@ function loadThemeFile(themePaths) {
function buildHealthConfig(config) {
const _ht = (config && config.healthThresholds) || {};
return {
infraDegradedMs: _ht.infraDegradedMs || 86400000,
infraSilentMs: _ht.infraSilentMs || 259200000,
nodeDegradedMs: _ht.nodeDegradedMs || 3600000,
nodeSilentMs: _ht.nodeSilentMs || 86400000
infraDegraded: _ht.infraDegradedHours || 24,
infraSilent: _ht.infraSilentHours || 72,
nodeDegraded: _ht.nodeDegradedHours || 1,
nodeSilent: _ht.nodeSilentHours || 24
};
}
function getHealthMs(role, HEALTH) {
const H = 3600000;
const isInfra = role === 'repeater' || role === 'room';
return {
degradedMs: isInfra ? HEALTH.infraDegradedMs : HEALTH.nodeDegradedMs,
silentMs: isInfra ? HEALTH.infraSilentMs : HEALTH.nodeSilentMs
degradedMs: (isInfra ? HEALTH.infraDegraded : HEALTH.nodeDegraded) * H,
silentMs: (isInfra ? HEALTH.infraSilent : HEALTH.nodeSilent) * H
};
}

View File

@@ -207,6 +207,13 @@ class TTLCache {
if (key.startsWith(prefix)) this.store.delete(key);
}
}
debouncedInvalidateBulkHealth() {
if (this._bulkHealthTimer) return;
this._bulkHealthTimer = setTimeout(() => {
this._bulkHealthTimer = null;
this.invalidate('bulk-health');
}, 30000);
}
debouncedInvalidateAll() {
if (this._debounceTimer) return;
this._debounceTimer = setTimeout(() => {
@@ -300,7 +307,12 @@ app.get('/api/config/cache', (req, res) => {
app.get('/api/config/client', (req, res) => {
res.json({
roles: config.roles || null,
healthThresholds: config.healthThresholds || null,
healthThresholds: {
infraDegradedMs: HEALTH.infraDegraded * 3600000,
infraSilentMs: HEALTH.infraSilent * 3600000,
nodeDegradedMs: HEALTH.nodeDegraded * 3600000,
nodeSilentMs: HEALTH.nodeSilent * 3600000
},
tiles: config.tiles || null,
snrThresholds: config.snrThresholds || null,
distThresholds: config.distThresholds || null,
@@ -410,7 +422,7 @@ app.get('/api/perf', (req, res) => {
avgMs: perfStats.requests ? Math.round(perfStats.totalMs / perfStats.requests * 10) / 10 : 0,
endpoints: Object.fromEntries(sorted),
slowQueries: perfStats.slowQueries.slice(-20),
cache: { size: cache.size, hits: cache.hits, misses: cache.misses, staleHits: cache.staleHits, recomputes: cache.recomputes, hitRate: cache.hits + cache.misses > 0 ? Math.round(cache.hits / (cache.hits + cache.misses) * 1000) / 10 : 0 },
cache: { size: cache.size, hits: cache.hits, misses: cache.misses, staleHits: cache.staleHits, recomputes: cache.recomputes, hitRate: cache.hits + cache.staleHits + cache.misses > 0 ? Math.round((cache.hits + cache.staleHits) / (cache.hits + cache.staleHits + cache.misses) * 1000) / 10 : 0 },
packetStore: pktStore.getStats(),
sqlite: (() => {
try {
@@ -519,7 +531,7 @@ app.get('/api/health', (req, res) => {
misses: cache.misses,
staleHits: cache.staleHits,
recomputes: cache.recomputes,
hitRate: cache.hits + cache.misses > 0 ? Math.round(cache.hits / (cache.hits + cache.misses) * 1000) / 10 : 0,
hitRate: cache.hits + cache.staleHits + cache.misses > 0 ? Math.round((cache.hits + cache.staleHits) / (cache.hits + cache.staleHits + cache.misses) * 1000) / 10 : 0,
},
websocket: {
clients: wsClients,
@@ -723,7 +735,7 @@ for (const source of mqttSources) {
// Invalidate this node's caches on advert
cache.invalidate('node:' + p.pubKey);
cache.invalidate('health:' + p.pubKey);
cache.invalidate('bulk-health');
cache.debouncedInvalidateBulkHealth();
// Cross-reference: if this node's pubkey matches an existing observer, backfill observer name
if (p.name && p.pubKey) {

View File

@@ -122,13 +122,14 @@ console.log('── Spec Tests: Transport Codes ──');
{
// Route type 0 (TRANSPORT_FLOOD) and 3 (TRANSPORT_DIRECT) should have 4-byte transport codes
// Route type 0: header byte = 0bPPPPPP00, e.g. 0x14 = payloadType 5 (GRP_TXT), routeType 0
const hex = '1400' + 'AABB' + 'CCDD' + '1A' + '00'.repeat(10); // transport codes + GRP_TXT payload
// Route type 0: header=0x14 = payloadType 5 (GRP_TXT), routeType 0 (TRANSPORT_FLOOD)
// Format: header(1) + transportCodes(4) + pathByte(1) + payload
const hex = '14' + 'AABB' + 'CCDD' + '00' + '1A' + '00'.repeat(10); // transport codes + pathByte + GRP_TXT payload
const p = decodePacket(hex);
assertEq(p.header.routeType, 0, 'transport: routeType=0 (TRANSPORT_FLOOD)');
assert(p.transportCodes !== null, 'transport: transportCodes present for TRANSPORT_FLOOD');
assertEq(p.transportCodes.nextHop, 'AABB', 'transport: nextHop');
assertEq(p.transportCodes.lastHop, 'CCDD', 'transport: lastHop');
assertEq(p.transportCodes.code1, 'AABB', 'transport: code1');
assertEq(p.transportCodes.code2, 'CCDD', 'transport: code2');
}
{
@@ -257,13 +258,13 @@ console.log('── Spec Tests: Advert Payload ──');
console.log('── Spec Tests: Encrypted Payload Format ──');
// NOTE: Spec says v1 encrypted payloads have dest(1) + src(1) + MAC(2) + ciphertext
// But decoder reads dest(6) + src(6) + MAC(4) + ciphertext
// This is a known discrepancy — the decoder matches production behavior, not the spec.
// The spec may describe the firmware's internal addressing while the OTA format differs,
// or the decoder may be parsing the fields differently. Production data validates the decoder.
// Spec says v1 encrypted payloads: dest(1)+src(1)+MAC(2)+cipher — decoder matches this.
{
note('Spec says v1 encrypted payloads: dest(1)+src(1)+MAC(2)+cipher, but decoder reads dest(6)+src(6)+MAC(4)+cipher — decoder matches prod data');
const hex = '0100' + 'AA' + 'BB' + 'CCDD' + '00'.repeat(10);
const p = decodePacket(hex);
assertEq(p.payload.destHash, 'aa', 'encrypted payload: dest is 1 byte');
assertEq(p.payload.srcHash, 'bb', 'encrypted payload: src is 1 byte');
assertEq(p.payload.mac, 'ccdd', 'encrypted payload: MAC is 2 bytes');
}
console.log('── Spec Tests: validateAdvert ──');

View File

@@ -28,22 +28,22 @@ test('FLOOD + ADVERT = 0x11', () => {
});
test('TRANSPORT_FLOOD = routeType 0', () => {
// 0x00 = TRANSPORT_FLOOD + REQ(0), needs transport codes + 16 byte payload
const hex = '0000' + 'AABB' + 'CCDD' + '00'.repeat(16);
// header=0x00 (TRANSPORT_FLOOD + REQ), transportCodes=AABB+CCDD, pathByte=0x00, payload
const hex = '00' + 'AABB' + 'CCDD' + '00' + '00'.repeat(16);
const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 0);
assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_FLOOD');
assert.notStrictEqual(p.transportCodes, null);
assert.strictEqual(p.transportCodes.nextHop, 'AABB');
assert.strictEqual(p.transportCodes.lastHop, 'CCDD');
assert.strictEqual(p.transportCodes.code1, 'AABB');
assert.strictEqual(p.transportCodes.code2, 'CCDD');
});
test('TRANSPORT_DIRECT = routeType 3', () => {
const hex = '0300' + '1122' + '3344' + '00'.repeat(16);
const hex = '03' + '1122' + '3344' + '00' + '00'.repeat(16);
const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 3);
assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_DIRECT');
assert.strictEqual(p.transportCodes.nextHop, '1122');
assert.strictEqual(p.transportCodes.code1, '1122');
});
test('DIRECT = routeType 2, no transport codes', () => {
@@ -358,9 +358,7 @@ test('ACK decode', () => {
const hex = '0D00' + '00'.repeat(18);
const p = decodePacket(hex);
assert.strictEqual(p.payload.type, 'ACK');
assert(p.payload.destHash);
assert(p.payload.srcHash);
assert(p.payload.extraHash);
assert(p.payload.ackChecksum);
});
test('ACK too short', () => {
@@ -424,9 +422,9 @@ test('TRACE decode', () => {
const hex = '2500' + '00'.repeat(12);
const p = decodePacket(hex);
assert.strictEqual(p.payload.type, 'TRACE');
assert.strictEqual(p.payload.flags, 0);
assert(p.payload.tag !== undefined);
assert(p.payload.destHash);
assert(p.payload.authCode !== undefined);
assert.strictEqual(p.payload.flags, 0);
});
test('TRACE too short', () => {
@@ -460,16 +458,18 @@ test('Transport route too short throws', () => {
assert.throws(() => decodePacket('0000'), /too short for transport/);
});
test('Corrupt packet #183 — path overflow capped to buffer', () => {
test('Corrupt packet #183 — TRANSPORT_DIRECT with correct field order', () => {
const hex = 'BBAD6797EC8751D500BF95A1A776EF580E665BCBF6A0BBE03B5E730707C53489B8C728FD3FB902397197E1263CEC21E52465362243685DBBAD6797EC8751C90A75D9FD8213155D';
const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 3, 'routeType should be TRANSPORT_DIRECT');
assert.strictEqual(p.header.payloadTypeName, 'UNKNOWN');
// pathByte 0xAD claims 45 hops × 3 bytes = 135, but only 65 bytes available
// transport codes are bytes 1-4, pathByte=0x87 at byte 5
assert.strictEqual(p.transportCodes.code1, 'AD67');
assert.strictEqual(p.transportCodes.code2, '97EC');
// pathByte 0x87: hashSize=3, hashCount=7
assert.strictEqual(p.path.hashSize, 3);
assert.strictEqual(p.path.hashCount, 21, 'hashCount capped to fit buffer');
assert.strictEqual(p.path.hops.length, 21);
assert.strictEqual(p.path.truncated, true);
assert.strictEqual(p.path.hashCount, 7);
assert.strictEqual(p.path.hops.length, 7);
// No empty strings in hops
assert(p.path.hops.every(h => h.length > 0), 'no empty hops');
});

View File

@@ -17,6 +17,8 @@ async function test(name, fn) {
} catch (err) {
results.push({ name, pass: false, error: err.message });
console.log(` \u274c ${name}: ${err.message}`);
console.log(`\nFail-fast: stopping after first failure.`);
process.exit(1);
}
}
@@ -351,13 +353,22 @@ async function run() {
});
// Test: Clicking a packet row opens detail pane
await test('Packets clicking row shows detail pane', async () => {
// SKIPPED: flaky test — see https://github.com/Kpa-clawbot/CoreScope/issues/257
console.log(' ⏭️ Packets clicking row shows detail pane (SKIPPED — flaky)');
/*await test('Packets clicking row shows detail pane', async () => {
// Fresh navigation to avoid stale row references from previous test
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' });
// Wait for table rows AND initial API data to settle
await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 });
await page.waitForLoadState('networkidle');
const firstRow = await page.$('table tbody tr[data-action]');
assert(firstRow, 'No clickable packet rows found');
await firstRow.click();
// Click the row and wait for the /packets/{hash} API response
const [response] = await Promise.all([
page.waitForResponse(resp => resp.url().includes('/packets/') && resp.status() === 200, { timeout: 15000 }),
firstRow.click(),
]);
assert(response, 'API response for packet detail not received');
await page.waitForFunction(() => {
const panel = document.getElementById('pktRight');
return panel && !panel.classList.contains('empty');
@@ -375,12 +386,16 @@ async function run() {
if (!pktRight) {
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 });
await page.waitForLoadState('networkidle');
}
const panelOpen = await page.$eval('#pktRight', el => !el.classList.contains('empty'));
if (!panelOpen) {
const firstRow = await page.$('table tbody tr[data-action]');
if (!firstRow) { console.log(' ⏭️ Skipped (no clickable rows)'); return; }
await firstRow.click();
await Promise.all([
page.waitForResponse(resp => resp.url().includes('/packets/') && resp.status() === 200, { timeout: 15000 }),
firstRow.click(),
]);
await page.waitForFunction(() => {
const panel = document.getElementById('pktRight');
return panel && !panel.classList.contains('empty');
@@ -395,7 +410,8 @@ async function run() {
}, { timeout: 3000 });
const panelHidden = await page.$eval('#pktRight', el => el.classList.contains('empty'));
assert(panelHidden, 'Detail pane should be hidden after clicking ✕');
});
});*/
console.log(' ⏭️ Packet detail pane closes on ✕ click (SKIPPED — depends on flaky test above)');
// Test: GRP_TXT packet detail shows Channel Hash (#123)
await test('GRP_TXT packet detail shows Channel Hash', async () => {
@@ -829,17 +845,7 @@ async function run() {
assert(content.length > 10, 'Perf content should still be present after refresh');
});
// Test: Node.js perf page shows Event Loop metrics (not Go Runtime)
await test('Perf page shows Event Loop on Node server', async () => {
const perfText = await page.$eval('#perfContent', el => el.textContent);
// Node.js server should show Event Loop metrics
const hasEventLoop = perfText.includes('Event Loop') || perfText.includes('event loop');
const hasMemory = perfText.includes('Memory') || perfText.includes('RSS');
assert(hasEventLoop || hasMemory, 'Node perf page should show Event Loop or Memory metrics');
// Should NOT show Go Runtime section on Node.js server
const hasGoRuntime = perfText.includes('Go Runtime');
assert(!hasGoRuntime, 'Node perf page should NOT show Go Runtime section');
});
// Test: Go perf page shows Go Runtime section (goroutines, GC)
// NOTE: This test requires GO_BASE_URL pointing to Go staging (port 82)
@@ -909,6 +915,19 @@ async function run() {
assert(hexDump, 'Hex dump should be visible after selecting a packet');
});
// Extract frontend coverage if instrumented server is running
try {
const coverage = await page.evaluate(() => window.__coverage__);
if (coverage) {
const fs = require('fs');
const path = require('path');
const outDir = path.join(__dirname, '.nyc_output');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
fs.writeFileSync(path.join(outDir, 'e2e-coverage.json'), JSON.stringify(coverage));
console.log(`Frontend coverage from E2E: ${Object.keys(coverage).length} files`);
}
} catch {}
await browser.close();
// Summary

Binary file not shown.

View File

@@ -1322,7 +1322,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
assert.ok(result.includes('>v2.6.0</a>'), 'version text has v prefix');
assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit links to full hash');
assert.ok(result.includes('>abc1234</a>'), 'commit display is truncated to 7');
assert.ok(result.includes('[node]'), 'should show engine');
assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>node<'), 'should show engine name');
});
test('prod port 80: shows version', () => {
const { formatVersionBadge } = makeBadgeSandbox('80');
@@ -1348,7 +1348,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
assert.ok(!result.includes('v2.6.0'), 'staging should NOT show version');
assert.ok(result.includes('>abc1234</a>'), 'should show commit hash');
assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit is linked');
assert.ok(result.includes('[go]'), 'should show engine');
assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
});
test('staging port 81: hides version', () => {
const { formatVersionBadge } = makeBadgeSandbox('81');
@@ -1369,18 +1369,18 @@ console.log('\n=== app.js: formatVersionBadge ===');
const result = formatVersionBadge('2.6.0', 'unknown', 'node');
assert.ok(result.includes('>v2.6.0</a>'), 'should show version');
assert.ok(!result.includes('unknown'), 'should not show unknown commit');
assert.ok(result.includes('[node]'), 'should show engine');
assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>node<'), 'should show engine name');
});
test('skips commit when missing', () => {
const { formatVersionBadge } = makeBadgeSandbox('');
const result = formatVersionBadge('2.6.0', null, 'go');
assert.ok(result.includes('>v2.6.0</a>'), 'should show version');
assert.ok(result.includes('[go]'), 'should show engine');
assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
});
test('shows only engine when version/commit missing', () => {
const { formatVersionBadge } = makeBadgeSandbox('3000');
const result = formatVersionBadge(null, null, 'go');
assert.ok(result.includes('[go]'), 'should show engine');
assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
assert.ok(result.includes('version-badge'), 'should use version-badge class');
});
test('short commit not truncated in display', () => {
@@ -1398,7 +1398,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
const { formatVersionBadge } = makeBadgeSandbox('8080');
const result = formatVersionBadge('2.6.0', null, 'go');
assert.ok(!result.includes('2.6.0'), 'no version on staging');
assert.ok(result.includes('[go]'), 'engine shown');
assert.ok(result.includes('engine-badge'), 'engine badge shown'); assert.ok(result.includes('>go<'), 'engine name shown');
});
}

View File

@@ -59,17 +59,17 @@ console.log('\nloadThemeFile:');
console.log('\nbuildHealthConfig:');
{
const h = helpers.buildHealthConfig({});
assert(h.infraDegradedMs === 86400000, 'default infraDegradedMs');
assert(h.infraSilentMs === 259200000, 'default infraSilentMs');
assert(h.nodeDegradedMs === 3600000, 'default nodeDegradedMs');
assert(h.nodeSilentMs === 86400000, 'default nodeSilentMs');
assert(h.infraDegraded === 24, 'default infraDegraded');
assert(h.infraSilent === 72, 'default infraSilent');
assert(h.nodeDegraded === 1, 'default nodeDegraded');
assert(h.nodeSilent === 24, 'default nodeSilent');
const h2 = helpers.buildHealthConfig({ healthThresholds: { infraDegradedMs: 1000 } });
assert(h2.infraDegradedMs === 1000, 'custom infraDegradedMs');
assert(h2.nodeDegradedMs === 3600000, 'other defaults preserved');
const h2 = helpers.buildHealthConfig({ healthThresholds: { infraDegradedHours: 2 } });
assert(h2.infraDegraded === 2, 'custom infraDegraded');
assert(h2.nodeDegraded === 1, 'other defaults preserved');
const h3 = helpers.buildHealthConfig(null);
assert(h3.infraDegradedMs === 86400000, 'handles null config');
assert(h3.infraDegraded === 24, 'handles null config');
}
// --- getHealthMs ---
@@ -78,21 +78,21 @@ console.log('\ngetHealthMs:');
const HEALTH = helpers.buildHealthConfig({});
const rep = helpers.getHealthMs('repeater', HEALTH);
assert(rep.degradedMs === 86400000, 'repeater uses infra degraded');
assert(rep.silentMs === 259200000, 'repeater uses infra silent');
assert(rep.degradedMs === 24 * 3600000, 'repeater uses infra degraded');
assert(rep.silentMs === 72 * 3600000, 'repeater uses infra silent');
const room = helpers.getHealthMs('room', HEALTH);
assert(room.degradedMs === 86400000, 'room uses infra degraded');
assert(room.degradedMs === 24 * 3600000, 'room uses infra degraded');
const comp = helpers.getHealthMs('companion', HEALTH);
assert(comp.degradedMs === 3600000, 'companion uses node degraded');
assert(comp.silentMs === 86400000, 'companion uses node silent');
assert(comp.degradedMs === 1 * 3600000, 'companion uses node degraded');
assert(comp.silentMs === 24 * 3600000, 'companion uses node silent');
const sensor = helpers.getHealthMs('sensor', HEALTH);
assert(sensor.degradedMs === 3600000, 'sensor uses node degraded');
assert(sensor.degradedMs === 1 * 3600000, 'sensor uses node degraded');
const undef = helpers.getHealthMs(undefined, HEALTH);
assert(undef.degradedMs === 3600000, 'undefined role uses node degraded');
assert(undef.degradedMs === 1 * 3600000, 'undefined role uses node degraded');
}
// --- isHashSizeFlipFlop ---

View File

@@ -1254,6 +1254,24 @@ seedTestData();
lastPathSeenMap.delete(liveNode);
});
// ── Cache hit rate includes stale hits ──
await t('Cache hitRate includes staleHits in formula', async () => {
cache.clear();
cache.hits = 0;
cache.misses = 0;
cache.staleHits = 0;
// Simulate: 3 hits, 2 stale hits, 5 misses => rate = (3+2)/(3+2+5) = 50%
cache.hits = 3;
cache.staleHits = 2;
cache.misses = 5;
const r = await request(app).get('/api/health').expect(200);
assert(r.body.cache.hitRate === 50, 'hitRate should be (hits+staleHits)/(hits+staleHits+misses) = 50%, got ' + r.body.cache.hitRate);
// Reset
cache.hits = 0;
cache.misses = 0;
cache.staleHits = 0;
});
// ── Summary ──
console.log(`\n═══ Server Route Tests: ${passed} passed, ${failed} failed ═══`);
if (failed > 0) process.exit(1);