Compare commits

...
Author SHA1 Message Date
Kpa-clawbot 0c1c0b4849 fix: remove networkidle waits and timeout bumps that regressed #248 speed
PR #248 specifically removed networkidle waits and kept tight timeouts
for speed in an SPA where they're unnecessary. This commit removes the
networkidle and timeout increases added in the previous commit while
keeping the actual flakiness fixes:

Kept:
- Deterministic seed data script (tools/seed-test-data.js)
- Retry logic (1 retry per test)
- Fresh navigation for groupByHash test
- Compare page explicit waitForFunction/waitForSelector waits
- Observers page waitForFunction for table rows

Reverted:
- Default timeout 10s→15s (back to 10s)
- All added networkidle waits (6 instances)
- Map marker timeout 3s→8s (back to 3s)
- Node detail timeout bumps
- Nodes page timeout bumps

Principle: fix flakiness with deterministic data and targeted waits,
not with slower timeouts and networkidle.
2026-03-29 16:57:52 +00:00
Kpa-clawbot b47571c7f0 fix: make E2E Playwright tests more resilient to flaky CI
Key changes:
- Add retry logic (1 retry) to test runner for transient failures
- Increase default timeout from 10s to 15s for slow CI runners
- Add networkidle waits after navigation on nodes, map, packets,
  channels, and observers pages
- Add explicit waitForSelector timeouts on node detail, map markers,
  and observer table rows
- Fresh page navigation for groupByHash test to avoid stale state
- Wait for compare results to render before asserting on cards
- Increase map marker wait timeout from 3s to 8s
- Add seed-test-data.js to CI pipeline to ensure deterministic test
  data exists (nodes with locations, packets, channels, observers)
- Seed data creates 8 nodes, 3 observers, channel messages, and
  various packet types for full page coverage
2026-03-29 16:53:12 +00:00
5bb9bc146e docs: remove letsmesh.net reference from README (#233)
* docs: remove letsmesh.net reference from README

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* ci: remove paths-ignore from pull_request trigger

PR #233 only touches .md files, which were excluded by paths-ignore,
causing CI to be skipped entirely. Remove paths-ignore from the
pull_request trigger so all PRs get validated. Keep paths-ignore on
push to avoid unnecessary deploys for docs-only changes to master.

* ci: skip heavy CI jobs for docs-only PRs

Instead of using paths-ignore (which skips the entire workflow and
blocks required status checks), detect docs-only changes at the start
of each job and skip heavy steps while still reporting success.

This allows doc-only PRs to merge without waiting for Go builds,
Node.js tests, or Playwright E2E runs.

Reverts the approach from 7546ece (removing paths-ignore entirely)
in favor of a proper conditional skip within the jobs themselves.

* fix: update engine tests to match engine-badge HTML format

Tests expected [go]/[node] text but formatVersionBadge now renders
<span class="engine-badge">go</span>. Updated 6 assertions to
check for engine-badge class and engine name in HTML output.

---------

Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com>
Co-authored-by: you <you@example.com>
2026-03-29 16:25:51 +00:00
youandKpaBap 12d1174e39 perf: speed up frontend coverage tests (~3x faster)
Three optimizations to the CI frontend test pipeline:

1. Run E2E tests and coverage collection concurrently
   - Previously sequential (E2E ~1.5min, then coverage ~5.75min)
   - Now both run in parallel against the same instrumented server
   - Expected savings: ~5 min (coverage runs alongside E2E instead of after)

2. Replace networkidle with domcontentloaded in coverage collector
   - SPA uses hash routing — networkidle waits 500ms for network silence
     on every navigation, adding ~10-15s of dead time across 23 navigations
   - domcontentloaded fires immediately once HTML is parsed; JS initializes
     the route handler synchronously
   - For in-page hash changes, use 200ms setTimeout instead of
     waitForLoadState (which would never re-fire for same-document nav)

3. Extract coverage from E2E tests too
   - E2E tests already exercise the app against the instrumented server
   - Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
   - nyc merges both coverage files for higher total coverage

Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
2026-03-29 09:12:23 -07:00
youandKpaBap 3bbd986d41 fix: add sleep before poller data insert to prevent race condition in tests
The poller's Start() calls GetMaxTransmissionID() to initialize its cursor.
When the test goroutine inserts data between go poller.Start() and the
actual GetMaxTransmissionID() call, the poller's cursor skips past the
test data and never broadcasts it, causing a timeout.

Adding a 100ms sleep after go poller.Start() ensures the poller has
initialized its cursors before the test inserts new data.
2026-03-29 08:32:37 -07:00
youandKpaBap 712fa15a8c fix: force single SQLite connection in test DBs to prevent in-memory table visibility issues
SQLite :memory: databases create separate databases per connection.
When the connection pool opens multiple connections (e.g. poller goroutine
vs main test goroutine), tables created on one connection are invisible
to others. Setting MaxOpenConns(1) ensures all queries use the same
in-memory database, fixing TestPollerBroadcastsMultipleObservations.
2026-03-29 08:32:37 -07:00
ab03b142f5 fix: per-observation WS broadcast for live view starburst — fixes #237
IngestNewFromDB now broadcasts one message per observation (not per
transmission). IngestNewObservations also broadcasts late arrivals.
Tests verify multi-observer packets produce multiple WS messages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-29 08:32:37 -07:00
youandKpaBap def95aae64 fix: align packet decoder with MeshCore firmware spec
Compared decoder.js against the MeshCore firmware source (Dispatcher.cpp,
Packet.h, Mesh.cpp, AdvertDataHelpers.h) and fixed all mismatches:

1. Field order: transport codes now parsed BEFORE path_length byte,
   matching the spec: [header][transport_codes?][path_length][path][payload]

2. ACK payload: was incorrectly decoded as dest(1)+src(1)+ackHash(4).
   Firmware shows ACK is just checksum(4) — no dest/src hashes.

3. TRACE payload: was incorrectly decoded as flags(1)+tag(4)+dest(6)+src(1).
   Firmware shows tag(4)+authCode(4)+flags(1)+pathData.

4. ADVERT appdata: added missing feature1 (0x20 flag) and feature2
   (0x40 flag) parsing — 2-byte fields between location and name.

5. Transport code field naming: renamed nextHop/lastHop to code1/code2
   to match spec terminology (transport_code_1/transport_code_2).

6. Fixed incorrect field size labels in packets.js hex breakdown:
   dest/src are 1 byte, MAC is 2 bytes (not 6B/6B/4B).

7. Fixed ANON_REQ/PATH comment typos (dest was listed as 6 bytes,
   MAC as 4 bytes — both wrong, code was already correct).

All 329 tests pass (66 decoder + 263 spec/golden).
2026-03-29 08:32:16 -07:00
you 1b09c733f5 ci: restrict self-hosted jobs to Linux runners
The Windows self-hosted runner picks up jobs and fails because bash
scripts run in PowerShell. Node.js tests need Chromium/Playwright
(Linux-only), and build/deploy/publish use Docker (Linux-only).

Changes:
- node-test: runs-on: [self-hosted, Linux]
- build: runs-on: [self-hosted, Linux]
- deploy: runs-on: [self-hosted, Linux]
- publish: runs-on: [self-hosted, Linux]
- go-test: unchanged (ubuntu-latest)
2026-03-29 14:58:15 +00:00
553c0e4963 ci: bump GitHub Actions to Node 24 compatible versions
checkout v4→v5, setup-go v5→v6, setup-node v4→v5,
upload-artifact v4→v5, download-artifact v4→v5

Fixes the Node.js 20 deprecation warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-29 07:51:48 -07:00
8ede8427c8 fix: round Go Runtime floats to 1dp, prevent nav stats dot wrapping
- perf.js: toFixed(1) on all ms/MB values in Go Runtime section
- style.css: white-space: nowrap on .nav-stats to prevent the · separator
  from wrapping onto its own line

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 07:51:26 -07:00
youandKpaBap 8e66c68d6f fix: cache hit rate excludes stale hits + debounce bulk-health invalidation
Two cache bugs fixed:

1. Hit rate formula excluded stale hits — reported rate was artificially low
   because stale-while-revalidate responses (which ARE cache hits from the
   caller's perspective) were not counted. Changed formula from
   hits/(hits+misses) to (hits+staleHits)/(hits+staleHits+misses).

2. Bulk-health cache invalidated on every advert packet — in a mesh with
   dozens of nodes advertising every few seconds, this caused the expensive
   bulk-health query to be recomputed on nearly every request, defeating
   the cache entirely. Switched to 30s debounced invalidation via
   debouncedInvalidateBulkHealth().

Added regression test for hit rate formula in test-server-routes.js.
2026-03-29 07:51:08 -07:00
youandKpaBap 37396823ad fix: align Go packet decoder with MeshCore firmware spec
Match the C++ firmware wire format (Packet::writeTo/readFrom):

1. Field order: transport codes are parsed BEFORE path_length byte,
   matching firmware's header → transport_codes → path_len → path → payload

2. ACK payload: just 4-byte CRC checksum, not dest+src+ackHash.
   Firmware createAck() writes only ack_crc (4 bytes).

3. TRACE payload: tag(4) + authCode(4) + flags(1) + pathData,
   matching firmware createTrace() and onRecvPacket() TRACE handler.

4. ADVERT features: parse feat1 (0x20) and feat2 (0x40) optional
   2-byte fields between location and name, matching AdvertDataBuilder
   and AdvertDataParser in the firmware.

5. Transport code naming: code1/code2 instead of nextHop/lastHop,
   matching firmware's transport_codes[0]/transport_codes[1] naming.

Fixes applied to both cmd/ingestor/decoder.go and cmd/server/decoder.go.
Tests updated to match new behavior.
2026-03-29 07:50:51 -07:00
24 changed files with 5654 additions and 4899 deletions
+97 -40
View File
@@ -10,11 +10,6 @@ on:
- 'docs/**' - 'docs/**'
pull_request: pull_request:
branches: [master] branches: [master]
paths-ignore:
- '**.md'
- 'LICENSE'
- '.gitignore'
- 'docs/**'
concurrency: concurrency:
group: deploy-${{ github.event.pull_request.number || github.ref }} group: deploy-${{ github.event.pull_request.number || github.ref }}
@@ -41,10 +36,25 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Skip if docs-only change
id: docs-check
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
CHANGED=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
NON_DOCS=$(echo "$CHANGED" | grep -cvE '\.(md)$|^LICENSE$|^\.gitignore$|^docs/' || true)
if [ "$NON_DOCS" -eq 0 ]; then
echo "docs_only=true" >> $GITHUB_OUTPUT
echo "📄 Docs-only PR — skipping heavy CI"
fi
fi
- name: Set up Go 1.22 - name: Set up Go 1.22
uses: actions/setup-go@v5 if: steps.docs-check.outputs.docs_only != 'true'
uses: actions/setup-go@v6
with: with:
go-version: '1.22' go-version: '1.22'
cache-dependency-path: | cache-dependency-path: |
@@ -52,6 +62,7 @@ jobs:
cmd/ingestor/go.sum cmd/ingestor/go.sum
- name: Build and test Go server (with coverage) - name: Build and test Go server (with coverage)
if: steps.docs-check.outputs.docs_only != 'true'
run: | run: |
set -e -o pipefail set -e -o pipefail
cd cmd/server cd cmd/server
@@ -61,6 +72,7 @@ jobs:
go tool cover -func=server-coverage.out | tail -1 go tool cover -func=server-coverage.out | tail -1
- name: Build and test Go ingestor (with coverage) - name: Build and test Go ingestor (with coverage)
if: steps.docs-check.outputs.docs_only != 'true'
run: | run: |
set -e -o pipefail set -e -o pipefail
cd cmd/ingestor cd cmd/ingestor
@@ -70,6 +82,7 @@ jobs:
go tool cover -func=ingestor-coverage.out | tail -1 go tool cover -func=ingestor-coverage.out | tail -1
- name: Verify proto syntax (all .proto files compile) - name: Verify proto syntax (all .proto files compile)
if: steps.docs-check.outputs.docs_only != 'true'
run: | run: |
set -e set -e
echo "Installing protoc..." echo "Installing protoc..."
@@ -84,7 +97,7 @@ jobs:
echo "✅ All .proto files are syntactically valid" echo "✅ All .proto files are syntactically valid"
- name: Generate Go coverage badges - name: Generate Go coverage badges
if: always() if: always() && steps.docs-check.outputs.docs_only != 'true'
run: | run: |
mkdir -p .badges mkdir -p .badges
@@ -131,7 +144,7 @@ jobs:
- name: Upload Go coverage badges - name: Upload Go coverage badges
if: always() if: always()
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v5
with: with:
name: go-badges name: go-badges
path: .badges/go-*.json path: .badges/go-*.json
@@ -143,22 +156,40 @@ jobs:
# ─────────────────────────────────────────────────────────────── # ───────────────────────────────────────────────────────────────
node-test: node-test:
name: "🧪 Node.js Tests" name: "🧪 Node.js Tests"
runs-on: self-hosted runs-on: [self-hosted, Linux]
defaults:
run:
shell: bash
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v5
with: with:
fetch-depth: 2 fetch-depth: 0
- name: Skip if docs-only change
id: docs-check
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
CHANGED=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
NON_DOCS=$(echo "$CHANGED" | grep -cvE '\.(md)$|^LICENSE$|^\.gitignore$|^docs/' || true)
if [ "$NON_DOCS" -eq 0 ]; then
echo "docs_only=true" >> $GITHUB_OUTPUT
echo "📄 Docs-only PR — skipping heavy CI"
fi
fi
- name: Set up Node.js 22 - name: Set up Node.js 22
uses: actions/setup-node@v4 if: steps.docs-check.outputs.docs_only != 'true'
uses: actions/setup-node@v5
with: with:
node-version: '22' node-version: '22'
- name: Install npm dependencies - name: Install npm dependencies
if: steps.docs-check.outputs.docs_only != 'true'
run: npm ci --production=false run: npm ci --production=false
- name: Detect changed files - name: Detect changed files
if: steps.docs-check.outputs.docs_only != 'true'
id: changes id: changes
run: | run: |
BACKEND=$(git diff --name-only HEAD~1 | grep -cE '^(server|db|decoder|packet-store|server-helpers|iata-coords)\.js$' || true) BACKEND=$(git diff --name-only HEAD~1 | grep -cE '^(server|db|decoder|packet-store|server-helpers|iata-coords)\.js$' || true)
@@ -174,7 +205,7 @@ jobs:
echo "Changes: backend=$BACKEND frontend=$FRONTEND tests=$TESTS ci=$CI" echo "Changes: backend=$BACKEND frontend=$FRONTEND tests=$TESTS ci=$CI"
- name: Run backend tests with coverage - name: Run backend tests with coverage
if: steps.changes.outputs.backend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.backend == 'true'
run: | run: |
npx c8 --reporter=text-summary --reporter=text sh test-all.sh 2>&1 | tee test-output.txt npx c8 --reporter=text-summary --reporter=text sh test-all.sh 2>&1 | tee test-output.txt
@@ -192,19 +223,23 @@ jobs:
echo "## Backend: ${TOTAL_PASS} tests, ${BE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY echo "## Backend: ${TOTAL_PASS} tests, ${BE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY
- name: Run backend tests (quick, no coverage) - name: Run backend tests (quick, no coverage)
if: steps.changes.outputs.backend == 'false' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.backend == 'false'
run: npm run test:unit run: npm run test:unit
- name: Install Playwright browser - name: Install Playwright browser
if: steps.changes.outputs.frontend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: npx playwright install chromium --with-deps 2>/dev/null || true run: |
# Install chromium (skips download if already cached on self-hosted runner)
npx playwright install chromium 2>/dev/null || true
# Install system deps only if missing (apt-get is slow)
npx playwright install-deps chromium 2>/dev/null || true
- name: Instrument frontend JS for coverage - name: Instrument frontend JS for coverage
if: steps.changes.outputs.frontend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: sh scripts/instrument-frontend.sh run: sh scripts/instrument-frontend.sh
- name: Start instrumented test server on port 13581 - name: Start instrumented test server on port 13581
if: steps.changes.outputs.frontend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: | run: |
# Kill any stale server on 13581 # Kill any stale server on 13581
fuser -k 13581/tcp 2>/dev/null || true fuser -k 13581/tcp 2>/dev/null || true
@@ -227,19 +262,36 @@ jobs:
sleep 1 sleep 1
done done
- name: Run Playwright E2E tests - name: Seed test data for Playwright
if: steps.changes.outputs.frontend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt run: BASE_URL=http://localhost:13581 node tools/seed-test-data.js
- name: Collect frontend coverage report - name: Run Playwright E2E + coverage collection concurrently
if: always() && steps.changes.outputs.frontend == 'true' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: |
# Run E2E tests and coverage collection in parallel — both use the same server
BASE_URL=http://localhost:13581 node test-e2e-playwright.js 2>&1 | tee e2e-output.txt &
E2E_PID=$!
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt &
COV_PID=$!
# Wait for both — E2E must pass, coverage is best-effort
E2E_EXIT=0
wait $E2E_PID || E2E_EXIT=$?
wait $COV_PID || true
# Fail if E2E failed
[ $E2E_EXIT -ne 0 ] && exit $E2E_EXIT
true
- name: Generate frontend coverage badges
if: always() && steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: | run: |
BASE_URL=http://localhost:13581 node scripts/collect-frontend-coverage.js 2>&1 | tee fe-coverage-output.txt
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1) E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1)
mkdir -p .badges mkdir -p .badges
if [ -f .nyc_output/frontend-coverage.json ]; then # Merge E2E + coverage collector data if both exist
if [ -f .nyc_output/frontend-coverage.json ] || [ -f .nyc_output/e2e-coverage.json ]; then
npx nyc report --reporter=text-summary --reporter=text 2>&1 | tee fe-report.txt npx nyc report --reporter=text-summary --reporter=text 2>&1 | tee fe-report.txt
FE_COVERAGE=$(grep 'Statements' fe-report.txt | head -1 | grep -oP '[\d.]+(?=%)' || echo "0") FE_COVERAGE=$(grep 'Statements' fe-report.txt | head -1 | grep -oP '[\d.]+(?=%)' || echo "0")
FE_COVERAGE=${FE_COVERAGE:-0} FE_COVERAGE=${FE_COVERAGE:-0}
@@ -252,7 +304,7 @@ jobs:
echo "{\"schemaVersion\":1,\"label\":\"frontend tests\",\"message\":\"${E2E_PASS:-0} E2E passed\",\"color\":\"brightgreen\"}" > .badges/frontend-tests.json echo "{\"schemaVersion\":1,\"label\":\"frontend tests\",\"message\":\"${E2E_PASS:-0} E2E passed\",\"color\":\"brightgreen\"}" > .badges/frontend-tests.json
- name: Stop test server - name: Stop test server
if: always() && steps.changes.outputs.frontend == 'true' if: always() && steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'true'
run: | run: |
if [ -f .server.pid ]; then if [ -f .server.pid ]; then
kill $(cat .server.pid) 2>/dev/null || true kill $(cat .server.pid) 2>/dev/null || true
@@ -261,12 +313,17 @@ jobs:
fi fi
- name: Run frontend E2E (quick, no coverage) - name: Run frontend E2E (quick, no coverage)
if: steps.changes.outputs.frontend == 'false' if: steps.docs-check.outputs.docs_only != 'true' && steps.changes.outputs.frontend == 'false'
run: | run: |
fuser -k 13581/tcp 2>/dev/null || true fuser -k 13581/tcp 2>/dev/null || true
PORT=13581 node server.js & PORT=13581 node server.js &
SERVER_PID=$! SERVER_PID=$!
sleep 5 # Wait for server to be ready (up to 15s)
for i in $(seq 1 15); do
curl -sf http://localhost:13581/api/stats > /dev/null 2>&1 && break
sleep 1
done
BASE_URL=http://localhost:13581 node tools/seed-test-data.js || true
BASE_URL=http://localhost:13581 node test-e2e-playwright.js || true BASE_URL=http://localhost:13581 node test-e2e-playwright.js || true
kill $SERVER_PID 2>/dev/null || true kill $SERVER_PID 2>/dev/null || true
@@ -279,7 +336,7 @@ jobs:
- name: Upload Node.js test badges - name: Upload Node.js test badges
if: always() if: always()
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v5
with: with:
name: node-badges name: node-badges
path: .badges/ path: .badges/
@@ -293,13 +350,13 @@ jobs:
name: "🏗️ Build Docker Image" name: "🏗️ Build Docker Image"
if: github.event_name == 'push' if: github.event_name == 'push'
needs: [go-test, node-test] needs: [go-test, node-test]
runs-on: self-hosted runs-on: [self-hosted, Linux]
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v5
- name: Set up Node.js 22 - name: Set up Node.js 22
uses: actions/setup-node@v4 uses: actions/setup-node@v5
with: with:
node-version: '22' node-version: '22'
@@ -318,10 +375,10 @@ jobs:
name: "🚀 Deploy Staging" name: "🚀 Deploy Staging"
if: github.event_name == 'push' if: github.event_name == 'push'
needs: [build] needs: [build]
runs-on: self-hosted runs-on: [self-hosted, Linux]
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v5
- name: Start staging on port 82 - name: Start staging on port 82
run: | run: |
@@ -363,21 +420,21 @@ jobs:
name: "📝 Publish Badges & Summary" name: "📝 Publish Badges & Summary"
if: github.event_name == 'push' if: github.event_name == 'push'
needs: [deploy] needs: [deploy]
runs-on: self-hosted runs-on: [self-hosted, Linux]
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v5
- name: Download Go coverage badges - name: Download Go coverage badges
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@v4 uses: actions/download-artifact@v5
with: with:
name: go-badges name: go-badges
path: .badges/ path: .badges/
- name: Download Node.js test badges - name: Download Node.js test badges
continue-on-error: true continue-on-error: true
uses: actions/download-artifact@v4 uses: actions/download-artifact@v5
with: with:
name: node-badges name: node-badges
path: .badges/ path: .badges/
+1 -1
View File
@@ -8,7 +8,7 @@
> High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption. > High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption.
Self-hosted, open-source MeshCore packet analyzer — a community alternative to the closed-source `analyzer.letsmesh.net`. Collects MeshCore packets via MQTT, decodes them in real time, and presents a full web UI with live packet feed, interactive maps, channel chat, packet tracing, and per-node analytics. Self-hosted, open-source MeshCore packet analyzer. Collects MeshCore packets via MQTT, decodes them in real time, and presents a full web UI with live packet feed, interactive maps, channel chat, packet tracing, and per-node analytics.
## ⚡ Performance ## ⚡ Performance
+61 -22
View File
@@ -72,8 +72,8 @@ type Header struct {
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes. // TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
type TransportCodes struct { type TransportCodes struct {
NextHop string `json:"nextHop"` Code1 string `json:"code1"`
LastHop string `json:"lastHop"` Code2 string `json:"code2"`
} }
// Path holds decoded path/hop information. // Path holds decoded path/hop information.
@@ -92,6 +92,8 @@ type AdvertFlags struct {
Room bool `json:"room"` Room bool `json:"room"`
Sensor bool `json:"sensor"` Sensor bool `json:"sensor"`
HasLocation bool `json:"hasLocation"` HasLocation bool `json:"hasLocation"`
HasFeat1 bool `json:"hasFeat1"`
HasFeat2 bool `json:"hasFeat2"`
HasName bool `json:"hasName"` HasName bool `json:"hasName"`
} }
@@ -111,6 +113,8 @@ type Payload struct {
Lat *float64 `json:"lat,omitempty"` Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"` Lon *float64 `json:"lon,omitempty"`
Name string `json:"name,omitempty"` Name string `json:"name,omitempty"`
Feat1 *int `json:"feat1,omitempty"`
Feat2 *int `json:"feat2,omitempty"`
BatteryMv *int `json:"battery_mv,omitempty"` BatteryMv *int `json:"battery_mv,omitempty"`
TemperatureC *float64 `json:"temperature_c,omitempty"` TemperatureC *float64 `json:"temperature_c,omitempty"`
ChannelHash int `json:"channelHash,omitempty"` ChannelHash int `json:"channelHash,omitempty"`
@@ -123,6 +127,8 @@ type Payload struct {
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"` EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
PathData string `json:"pathData,omitempty"` PathData string `json:"pathData,omitempty"`
Tag uint32 `json:"tag,omitempty"` Tag uint32 `json:"tag,omitempty"`
AuthCode uint32 `json:"authCode,omitempty"`
TraceFlags *int `json:"traceFlags,omitempty"`
RawHex string `json:"raw,omitempty"` RawHex string `json:"raw,omitempty"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
} }
@@ -199,14 +205,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
} }
func decodeAck(buf []byte) Payload { func decodeAck(buf []byte) Payload {
if len(buf) < 6 { if len(buf) < 4 {
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)} return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
} }
checksum := binary.LittleEndian.Uint32(buf[0:4])
return Payload{ return Payload{
Type: "ACK", Type: "ACK",
DestHash: hex.EncodeToString(buf[0:1]), ExtraHash: fmt.Sprintf("%08x", checksum),
SrcHash: hex.EncodeToString(buf[1:2]),
ExtraHash: hex.EncodeToString(buf[2:6]),
} }
} }
@@ -231,6 +236,8 @@ func decodeAdvert(buf []byte) Payload {
if len(appdata) > 0 { if len(appdata) > 0 {
flags := appdata[0] flags := appdata[0]
advType := int(flags & 0x0F) advType := int(flags & 0x0F)
hasFeat1 := flags&0x20 != 0
hasFeat2 := flags&0x40 != 0
p.Flags = &AdvertFlags{ p.Flags = &AdvertFlags{
Raw: int(flags), Raw: int(flags),
Type: advType, Type: advType,
@@ -239,6 +246,8 @@ func decodeAdvert(buf []byte) Payload {
Room: advType == 3, Room: advType == 3,
Sensor: advType == 4, Sensor: advType == 4,
HasLocation: flags&0x10 != 0, HasLocation: flags&0x10 != 0,
HasFeat1: hasFeat1,
HasFeat2: hasFeat2,
HasName: flags&0x80 != 0, HasName: flags&0x80 != 0,
} }
@@ -252,6 +261,16 @@ func decodeAdvert(buf []byte) Payload {
p.Lon = &lon p.Lon = &lon
off += 8 off += 8
} }
if hasFeat1 && len(appdata) >= off+2 {
feat1 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
p.Feat1 = &feat1
off += 2
}
if hasFeat2 && len(appdata) >= off+2 {
feat2 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
p.Feat2 = &feat2
off += 2
}
if p.Flags.HasName { if p.Flags.HasName {
// Find null terminator to separate name from trailing telemetry bytes // Find null terminator to separate name from trailing telemetry bytes
nameEnd := len(appdata) nameEnd := len(appdata)
@@ -469,15 +488,22 @@ func decodePathPayload(buf []byte) Payload {
} }
func decodeTrace(buf []byte) Payload { func decodeTrace(buf []byte) Payload {
if len(buf) < 12 { if len(buf) < 9 {
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)} return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
} }
return Payload{ tag := binary.LittleEndian.Uint32(buf[0:4])
Type: "TRACE", authCode := binary.LittleEndian.Uint32(buf[4:8])
DestHash: hex.EncodeToString(buf[5:11]), flags := int(buf[8])
SrcHash: hex.EncodeToString(buf[11:12]), p := Payload{
Tag: binary.LittleEndian.Uint32(buf[1:5]), Type: "TRACE",
Tag: tag,
AuthCode: authCode,
TraceFlags: &flags,
} }
if len(buf) > 9 {
p.PathData = hex.EncodeToString(buf[9:])
}
return p
} }
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload { func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload {
@@ -520,8 +546,7 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
} }
header := decodeHeader(buf[0]) header := decodeHeader(buf[0])
pathByte := buf[1] offset := 1
offset := 2
var tc *TransportCodes var tc *TransportCodes
if isTransportRoute(header.RouteType) { if isTransportRoute(header.RouteType) {
@@ -529,12 +554,18 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
return nil, fmt.Errorf("packet too short for transport codes") return nil, fmt.Errorf("packet too short for transport codes")
} }
tc = &TransportCodes{ tc = &TransportCodes{
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])), Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])), Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
} }
offset += 4 offset += 4
} }
if offset >= len(buf) {
return nil, fmt.Errorf("packet too short (no path byte)")
}
pathByte := buf[offset]
offset++
path, bytesConsumed := decodePath(pathByte, buf, offset) path, bytesConsumed := decodePath(pathByte, buf, offset)
offset += bytesConsumed offset += bytesConsumed
@@ -562,16 +593,24 @@ func ComputeContentHash(rawHex string) string {
return rawHex return rawHex
} }
pathByte := buf[1] headerByte := buf[0]
offset := 1
if isTransportRoute(int(headerByte & 0x03)) {
offset += 4
}
if offset >= len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]
}
return rawHex
}
pathByte := buf[offset]
offset++
hashSize := int((pathByte>>6)&0x3) + 1 hashSize := int((pathByte>>6)&0x3) + 1
hashCount := int(pathByte & 0x3F) hashCount := int(pathByte & 0x3F)
pathBytes := hashSize * hashCount pathBytes := hashSize * hashCount
headerByte := buf[0] payloadStart := offset + pathBytes
payloadStart := 2 + pathBytes
if isTransportRoute(int(headerByte & 0x03)) {
payloadStart += 4
}
if payloadStart > len(buf) { if payloadStart > len(buf) {
if len(rawHex) >= 16 { if len(rawHex) >= 16 {
return rawHex[:16] return rawHex[:16]
+36 -25
View File
@@ -129,7 +129,8 @@ func TestDecodePath3ByteHashes(t *testing.T) {
func TestTransportCodes(t *testing.T) { func TestTransportCodes(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) should have transport codes // Route type 0 (TRANSPORT_FLOOD) should have transport codes
hex := "1400" + "AABB" + "CCDD" + "1A" + strings.Repeat("00", 10) // Firmware order: header + transport_codes(4) + path_len + path + payload
hex := "14" + "AABB" + "CCDD" + "00" + strings.Repeat("00", 10)
pkt, err := DecodePacket(hex, nil) pkt, err := DecodePacket(hex, nil)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -140,11 +141,11 @@ func TestTransportCodes(t *testing.T) {
if pkt.TransportCodes == nil { if pkt.TransportCodes == nil {
t.Fatal("transportCodes should not be nil for TRANSPORT_FLOOD") t.Fatal("transportCodes should not be nil for TRANSPORT_FLOOD")
} }
if pkt.TransportCodes.NextHop != "AABB" { if pkt.TransportCodes.Code1 != "AABB" {
t.Errorf("nextHop=%s, want AABB", pkt.TransportCodes.NextHop) t.Errorf("code1=%s, want AABB", pkt.TransportCodes.Code1)
} }
if pkt.TransportCodes.LastHop != "CCDD" { if pkt.TransportCodes.Code2 != "CCDD" {
t.Errorf("lastHop=%s, want CCDD", pkt.TransportCodes.LastHop) t.Errorf("code2=%s, want CCDD", pkt.TransportCodes.Code2)
} }
// Route type 1 (FLOOD) should NOT have transport codes // Route type 1 (FLOOD) should NOT have transport codes
@@ -537,10 +538,11 @@ func TestDecodeTraceShort(t *testing.T) {
func TestDecodeTraceValid(t *testing.T) { func TestDecodeTraceValid(t *testing.T) {
buf := make([]byte, 16) buf := make([]byte, 16)
buf[0] = 0x00 // tag(4) + authCode(4) + flags(1) + pathData
buf[1] = 0x01 // tag LE uint32 = 1 binary.LittleEndian.PutUint32(buf[0:4], 1) // tag = 1
buf[5] = 0xAA // destHash start binary.LittleEndian.PutUint32(buf[4:8], 0xDEADBEEF) // authCode
buf[11] = 0xBB buf[8] = 0x02 // flags
buf[9] = 0xAA // path data
p := decodeTrace(buf) p := decodeTrace(buf)
if p.Error != "" { if p.Error != "" {
t.Errorf("unexpected error: %s", p.Error) t.Errorf("unexpected error: %s", p.Error)
@@ -548,9 +550,18 @@ func TestDecodeTraceValid(t *testing.T) {
if p.Tag != 1 { if p.Tag != 1 {
t.Errorf("tag=%d, want 1", p.Tag) t.Errorf("tag=%d, want 1", p.Tag)
} }
if p.AuthCode != 0xDEADBEEF {
t.Errorf("authCode=%d, want 0xDEADBEEF", p.AuthCode)
}
if p.TraceFlags == nil || *p.TraceFlags != 2 {
t.Errorf("traceFlags=%v, want 2", p.TraceFlags)
}
if p.Type != "TRACE" { if p.Type != "TRACE" {
t.Errorf("type=%s, want TRACE", p.Type) t.Errorf("type=%s, want TRACE", p.Type)
} }
if p.PathData == "" {
t.Error("pathData should not be empty")
}
} }
func TestDecodeAdvertShort(t *testing.T) { func TestDecodeAdvertShort(t *testing.T) {
@@ -833,10 +844,9 @@ func TestComputeContentHashShortHex(t *testing.T) {
} }
func TestComputeContentHashTransportRoute(t *testing.T) { func TestComputeContentHashTransportRoute(t *testing.T) {
// Route type 0 (TRANSPORT_FLOOD) with no path hops + 4 transport code bytes // Route type 0 (TRANSPORT_FLOOD) with transport codes then path=0x00 (0 hops)
// header=0x14 (TRANSPORT_FLOOD, ADVERT), path=0x00 (0 hops) // header=0x14 (TRANSPORT_FLOOD, ADVERT), transport(4), path=0x00
// transport codes = 4 bytes, then payload hex := "14" + "AABBCCDD" + "00" + strings.Repeat("EE", 10)
hex := "1400" + "AABBCCDD" + strings.Repeat("EE", 10)
hash := ComputeContentHash(hex) hash := ComputeContentHash(hex)
if len(hash) != 16 { if len(hash) != 16 {
t.Errorf("hash length=%d, want 16", len(hash)) t.Errorf("hash length=%d, want 16", len(hash))
@@ -870,12 +880,10 @@ func TestComputeContentHashPayloadBeyondBufferLongHex(t *testing.T) {
func TestComputeContentHashTransportBeyondBuffer(t *testing.T) { func TestComputeContentHashTransportBeyondBuffer(t *testing.T) {
// Transport route (0x00 = TRANSPORT_FLOOD) with path claiming some bytes // Transport route (0x00 = TRANSPORT_FLOOD) with path claiming some bytes
// total buffer too short for transport codes + path // header=0x00, transport(4), pathByte=0x02 (2 hops, 1-byte hash)
// header=0x00, pathByte=0x02 (2 hops, 1-byte hash), then only 2 more bytes // offset=1+4+1+2=8, buffer needs to be >= 8
// payloadStart = 2 + 2 + 4(transport) = 8, but buffer only 6 bytes hex := "00" + "AABB" + "CCDD" + "02" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
hex := "0002" + "AABB" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
hash := ComputeContentHash(hex) hash := ComputeContentHash(hex)
// payloadStart = 2 + 2 + 4 = 8, buffer is 10 bytes → should work
if len(hash) != 16 { if len(hash) != 16 {
t.Errorf("hash length=%d, want 16", len(hash)) t.Errorf("hash length=%d, want 16", len(hash))
} }
@@ -913,8 +921,8 @@ func TestDecodePacketWithNewlines(t *testing.T) {
} }
func TestDecodePacketTransportRouteTooShort(t *testing.T) { func TestDecodePacketTransportRouteTooShort(t *testing.T) {
// TRANSPORT_FLOOD (route=0) but only 3 bytes total → too short for transport codes // TRANSPORT_FLOOD (route=0) but only 2 bytes total → too short for transport codes
_, err := DecodePacket("140011", nil) _, err := DecodePacket("1400", nil)
if err == nil { if err == nil {
t.Error("expected error for transport route with too-short buffer") t.Error("expected error for transport route with too-short buffer")
} }
@@ -931,16 +939,19 @@ func TestDecodeAckShort(t *testing.T) {
} }
func TestDecodeAckValid(t *testing.T) { func TestDecodeAckValid(t *testing.T) {
buf := []byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF} buf := []byte{0xAA, 0xBB, 0xCC, 0xDD}
p := decodeAck(buf) p := decodeAck(buf)
if p.Error != "" { if p.Error != "" {
t.Errorf("unexpected error: %s", p.Error) t.Errorf("unexpected error: %s", p.Error)
} }
if p.DestHash != "aa" { if p.ExtraHash != "ddccbbaa" {
t.Errorf("destHash=%s, want aa", p.DestHash) t.Errorf("extraHash=%s, want ddccbbaa", p.ExtraHash)
} }
if p.ExtraHash != "ccddeeff" { if p.DestHash != "" {
t.Errorf("extraHash=%s, want ccddeeff", p.ExtraHash) t.Errorf("destHash should be empty, got %s", p.DestHash)
}
if p.SrcHash != "" {
t.Errorf("srcHash should be empty, got %s", p.SrcHash)
} }
} }
+3717 -3714
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -17,6 +17,8 @@ func setupTestDB(t *testing.T) *DB {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
// Force single connection so all goroutines share the same in-memory DB
conn.SetMaxOpenConns(1)
// Create schema matching MeshCore Analyzer v3 // Create schema matching MeshCore Analyzer v3
schema := ` schema := `
+55 -22
View File
@@ -54,8 +54,8 @@ type Header struct {
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes. // TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
type TransportCodes struct { type TransportCodes struct {
NextHop string `json:"nextHop"` Code1 string `json:"code1"`
LastHop string `json:"lastHop"` Code2 string `json:"code2"`
} }
// Path holds decoded path/hop information. // Path holds decoded path/hop information.
@@ -74,6 +74,8 @@ type AdvertFlags struct {
Room bool `json:"room"` Room bool `json:"room"`
Sensor bool `json:"sensor"` Sensor bool `json:"sensor"`
HasLocation bool `json:"hasLocation"` HasLocation bool `json:"hasLocation"`
HasFeat1 bool `json:"hasFeat1"`
HasFeat2 bool `json:"hasFeat2"`
HasName bool `json:"hasName"` HasName bool `json:"hasName"`
} }
@@ -97,6 +99,8 @@ type Payload struct {
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"` EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
PathData string `json:"pathData,omitempty"` PathData string `json:"pathData,omitempty"`
Tag uint32 `json:"tag,omitempty"` Tag uint32 `json:"tag,omitempty"`
AuthCode uint32 `json:"authCode,omitempty"`
TraceFlags *int `json:"traceFlags,omitempty"`
RawHex string `json:"raw,omitempty"` RawHex string `json:"raw,omitempty"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
} }
@@ -173,14 +177,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
} }
func decodeAck(buf []byte) Payload { func decodeAck(buf []byte) Payload {
if len(buf) < 6 { if len(buf) < 4 {
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)} return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
} }
checksum := binary.LittleEndian.Uint32(buf[0:4])
return Payload{ return Payload{
Type: "ACK", Type: "ACK",
DestHash: hex.EncodeToString(buf[0:1]), ExtraHash: fmt.Sprintf("%08x", checksum),
SrcHash: hex.EncodeToString(buf[1:2]),
ExtraHash: hex.EncodeToString(buf[2:6]),
} }
} }
@@ -205,6 +208,8 @@ func decodeAdvert(buf []byte) Payload {
if len(appdata) > 0 { if len(appdata) > 0 {
flags := appdata[0] flags := appdata[0]
advType := int(flags & 0x0F) advType := int(flags & 0x0F)
hasFeat1 := flags&0x20 != 0
hasFeat2 := flags&0x40 != 0
p.Flags = &AdvertFlags{ p.Flags = &AdvertFlags{
Raw: int(flags), Raw: int(flags),
Type: advType, Type: advType,
@@ -213,6 +218,8 @@ func decodeAdvert(buf []byte) Payload {
Room: advType == 3, Room: advType == 3,
Sensor: advType == 4, Sensor: advType == 4,
HasLocation: flags&0x10 != 0, HasLocation: flags&0x10 != 0,
HasFeat1: hasFeat1,
HasFeat2: hasFeat2,
HasName: flags&0x80 != 0, HasName: flags&0x80 != 0,
} }
@@ -226,6 +233,12 @@ func decodeAdvert(buf []byte) Payload {
p.Lon = &lon p.Lon = &lon
off += 8 off += 8
} }
if hasFeat1 && len(appdata) >= off+2 {
off += 2 // skip feat1 bytes (reserved for future use)
}
if hasFeat2 && len(appdata) >= off+2 {
off += 2 // skip feat2 bytes (reserved for future use)
}
if p.Flags.HasName { if p.Flags.HasName {
name := string(appdata[off:]) name := string(appdata[off:])
name = strings.TrimRight(name, "\x00") name = strings.TrimRight(name, "\x00")
@@ -276,15 +289,22 @@ func decodePathPayload(buf []byte) Payload {
} }
func decodeTrace(buf []byte) Payload { func decodeTrace(buf []byte) Payload {
if len(buf) < 12 { if len(buf) < 9 {
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)} return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
} }
return Payload{ tag := binary.LittleEndian.Uint32(buf[0:4])
Type: "TRACE", authCode := binary.LittleEndian.Uint32(buf[4:8])
DestHash: hex.EncodeToString(buf[5:11]), flags := int(buf[8])
SrcHash: hex.EncodeToString(buf[11:12]), p := Payload{
Tag: binary.LittleEndian.Uint32(buf[1:5]), Type: "TRACE",
Tag: tag,
AuthCode: authCode,
TraceFlags: &flags,
} }
if len(buf) > 9 {
p.PathData = hex.EncodeToString(buf[9:])
}
return p
} }
func decodePayload(payloadType int, buf []byte) Payload { func decodePayload(payloadType int, buf []byte) Payload {
@@ -327,8 +347,7 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
} }
header := decodeHeader(buf[0]) header := decodeHeader(buf[0])
pathByte := buf[1] offset := 1
offset := 2
var tc *TransportCodes var tc *TransportCodes
if isTransportRoute(header.RouteType) { if isTransportRoute(header.RouteType) {
@@ -336,12 +355,18 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
return nil, fmt.Errorf("packet too short for transport codes") return nil, fmt.Errorf("packet too short for transport codes")
} }
tc = &TransportCodes{ tc = &TransportCodes{
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])), Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])), Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
} }
offset += 4 offset += 4
} }
if offset >= len(buf) {
return nil, fmt.Errorf("packet too short (no path byte)")
}
pathByte := buf[offset]
offset++
path, bytesConsumed := decodePath(pathByte, buf, offset) path, bytesConsumed := decodePath(pathByte, buf, offset)
offset += bytesConsumed offset += bytesConsumed
@@ -367,16 +392,24 @@ func ComputeContentHash(rawHex string) string {
return rawHex return rawHex
} }
pathByte := buf[1] headerByte := buf[0]
offset := 1
if isTransportRoute(int(headerByte & 0x03)) {
offset += 4
}
if offset >= len(buf) {
if len(rawHex) >= 16 {
return rawHex[:16]
}
return rawHex
}
pathByte := buf[offset]
offset++
hashSize := int((pathByte>>6)&0x3) + 1 hashSize := int((pathByte>>6)&0x3) + 1
hashCount := int(pathByte & 0x3F) hashCount := int(pathByte & 0x3F)
pathBytes := hashSize * hashCount pathBytes := hashSize * hashCount
headerByte := buf[0] payloadStart := offset + pathBytes
payloadStart := 2 + pathBytes
if isTransportRoute(int(headerByte & 0x03)) {
payloadStart += 4
}
if payloadStart > len(buf) { if payloadStart > len(buf) {
if len(rawHex) >= 16 { if len(rawHex) >= 16 {
return rawHex[:16] return rawHex[:16]
+506 -403
View File
@@ -1,403 +1,506 @@
package main package main
// parity_test.go — Golden fixture shape tests. // parity_test.go — Golden fixture shape tests.
// Validates that Go API responses match the shape of Node.js API responses. // Validates that Go API responses match the shape of Node.js API responses.
// Shapes were captured from the production Node.js server and stored in // Shapes were captured from the production Node.js server and stored in
// testdata/golden/shapes.json. // testdata/golden/shapes.json.
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"net/http/httptest" "net/http/httptest"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings" "sort"
"testing" "strings"
) "testing"
"time"
// shapeSpec describes the expected JSON structure from the Node.js server. )
type shapeSpec struct {
Type string `json:"type"` // shapeSpec describes the expected JSON structure from the Node.js server.
Keys map[string]shapeSpec `json:"keys,omitempty"` type shapeSpec struct {
ElementShape *shapeSpec `json:"elementShape,omitempty"` Type string `json:"type"`
DynamicKeys bool `json:"dynamicKeys,omitempty"` Keys map[string]shapeSpec `json:"keys,omitempty"`
ValueShape *shapeSpec `json:"valueShape,omitempty"` ElementShape *shapeSpec `json:"elementShape,omitempty"`
RequiredKeys map[string]shapeSpec `json:"requiredKeys,omitempty"` DynamicKeys bool `json:"dynamicKeys,omitempty"`
} ValueShape *shapeSpec `json:"valueShape,omitempty"`
RequiredKeys map[string]shapeSpec `json:"requiredKeys,omitempty"`
// loadShapes reads testdata/golden/shapes.json relative to this source file. }
func loadShapes(t *testing.T) map[string]shapeSpec {
t.Helper() // loadShapes reads testdata/golden/shapes.json relative to this source file.
_, thisFile, _, _ := runtime.Caller(0) func loadShapes(t *testing.T) map[string]shapeSpec {
dir := filepath.Dir(thisFile) t.Helper()
data, err := os.ReadFile(filepath.Join(dir, "testdata", "golden", "shapes.json")) _, thisFile, _, _ := runtime.Caller(0)
if err != nil { dir := filepath.Dir(thisFile)
t.Fatalf("cannot load shapes.json: %v", err) data, err := os.ReadFile(filepath.Join(dir, "testdata", "golden", "shapes.json"))
} if err != nil {
var shapes map[string]shapeSpec t.Fatalf("cannot load shapes.json: %v", err)
if err := json.Unmarshal(data, &shapes); err != nil { }
t.Fatalf("cannot parse shapes.json: %v", err) var shapes map[string]shapeSpec
} if err := json.Unmarshal(data, &shapes); err != nil {
return shapes t.Fatalf("cannot parse shapes.json: %v", err)
} }
return shapes
// validateShape recursively checks that `actual` matches the expected `spec`. }
// `path` tracks the JSON path for error messages.
// Returns a list of mismatch descriptions. // validateShape recursively checks that `actual` matches the expected `spec`.
func validateShape(actual interface{}, spec shapeSpec, path string) []string { // `path` tracks the JSON path for error messages.
var errs []string // Returns a list of mismatch descriptions.
func validateShape(actual interface{}, spec shapeSpec, path string) []string {
switch spec.Type { var errs []string
case "null", "nullable":
// nullable means: value can be null OR matching type. Accept anything. switch spec.Type {
return nil case "null", "nullable":
case "nullable_number": // nullable means: value can be null OR matching type. Accept anything.
// Can be null or number return nil
if actual != nil { case "nullable_number":
if _, ok := actual.(float64); !ok { // Can be null or number
errs = append(errs, fmt.Sprintf("%s: expected number or null, got %T", path, actual)) if actual != nil {
} if _, ok := actual.(float64); !ok {
} errs = append(errs, fmt.Sprintf("%s: expected number or null, got %T", path, actual))
return errs }
case "string": }
if actual == nil { return errs
errs = append(errs, fmt.Sprintf("%s: expected string, got null", path)) case "string":
} else if _, ok := actual.(string); !ok { if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected string, got %T", path, actual)) errs = append(errs, fmt.Sprintf("%s: expected string, got null", path))
} } else if _, ok := actual.(string); !ok {
case "number": errs = append(errs, fmt.Sprintf("%s: expected string, got %T", path, actual))
if actual == nil { }
errs = append(errs, fmt.Sprintf("%s: expected number, got null", path)) case "number":
} else if _, ok := actual.(float64); !ok { if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected number, got %T (%v)", path, actual, actual)) errs = append(errs, fmt.Sprintf("%s: expected number, got null", path))
} } else if _, ok := actual.(float64); !ok {
case "boolean": errs = append(errs, fmt.Sprintf("%s: expected number, got %T (%v)", path, actual, actual))
if actual == nil { }
errs = append(errs, fmt.Sprintf("%s: expected boolean, got null", path)) case "boolean":
} else if _, ok := actual.(bool); !ok { if actual == nil {
errs = append(errs, fmt.Sprintf("%s: expected boolean, got %T", path, actual)) errs = append(errs, fmt.Sprintf("%s: expected boolean, got null", path))
} } else if _, ok := actual.(bool); !ok {
case "array": errs = append(errs, fmt.Sprintf("%s: expected boolean, got %T", path, actual))
if actual == nil { }
errs = append(errs, fmt.Sprintf("%s: expected array, got null (arrays must be [] not null)", path)) case "array":
return errs if actual == nil {
} errs = append(errs, fmt.Sprintf("%s: expected array, got null (arrays must be [] not null)", path))
arr, ok := actual.([]interface{}) return errs
if !ok { }
errs = append(errs, fmt.Sprintf("%s: expected array, got %T", path, actual)) arr, ok := actual.([]interface{})
return errs if !ok {
} errs = append(errs, fmt.Sprintf("%s: expected array, got %T", path, actual))
if spec.ElementShape != nil && len(arr) > 0 { return errs
errs = append(errs, validateShape(arr[0], *spec.ElementShape, path+"[0]")...) }
} if spec.ElementShape != nil && len(arr) > 0 {
case "object": errs = append(errs, validateShape(arr[0], *spec.ElementShape, path+"[0]")...)
if actual == nil { }
errs = append(errs, fmt.Sprintf("%s: expected object, got null", path)) case "object":
return errs if actual == nil {
} errs = append(errs, fmt.Sprintf("%s: expected object, got null", path))
obj, ok := actual.(map[string]interface{}) return errs
if !ok { }
errs = append(errs, fmt.Sprintf("%s: expected object, got %T", path, actual)) obj, ok := actual.(map[string]interface{})
return errs if !ok {
} errs = append(errs, fmt.Sprintf("%s: expected object, got %T", path, actual))
return errs
if spec.DynamicKeys { }
// Object with dynamic keys — validate value shapes
if spec.ValueShape != nil && len(obj) > 0 { if spec.DynamicKeys {
for k, v := range obj { // Object with dynamic keys — validate value shapes
errs = append(errs, validateShape(v, *spec.ValueShape, path+"."+k)...) if spec.ValueShape != nil && len(obj) > 0 {
break // check just one sample for k, v := range obj {
} errs = append(errs, validateShape(v, *spec.ValueShape, path+"."+k)...)
} break // check just one sample
if spec.RequiredKeys != nil { }
for rk, rs := range spec.RequiredKeys { }
v, exists := obj[rk] if spec.RequiredKeys != nil {
if !exists { for rk, rs := range spec.RequiredKeys {
errs = append(errs, fmt.Sprintf("%s: missing required key %q in dynamic-key object", path, rk)) v, exists := obj[rk]
} else { if !exists {
errs = append(errs, validateShape(v, rs, path+"."+rk)...) errs = append(errs, fmt.Sprintf("%s: missing required key %q in dynamic-key object", path, rk))
} } else {
} errs = append(errs, validateShape(v, rs, path+"."+rk)...)
} }
} else if spec.Keys != nil { }
// Object with known keys — check each expected key exists and has correct type }
for key, keySpec := range spec.Keys { } else if spec.Keys != nil {
val, exists := obj[key] // Object with known keys — check each expected key exists and has correct type
if !exists { for key, keySpec := range spec.Keys {
errs = append(errs, fmt.Sprintf("%s: missing field %q (expected %s)", path, key, keySpec.Type)) val, exists := obj[key]
} else { if !exists {
errs = append(errs, validateShape(val, keySpec, path+"."+key)...) errs = append(errs, fmt.Sprintf("%s: missing field %q (expected %s)", path, key, keySpec.Type))
} } else {
} errs = append(errs, validateShape(val, keySpec, path+"."+key)...)
} }
} }
}
return errs }
}
return errs
// parityEndpoint defines one endpoint to test for parity. }
type parityEndpoint struct {
name string // key in shapes.json // parityEndpoint defines one endpoint to test for parity.
path string // HTTP path to request type parityEndpoint struct {
} name string // key in shapes.json
path string // HTTP path to request
func TestParityShapes(t *testing.T) { }
shapes := loadShapes(t)
_, router := setupTestServer(t) func TestParityShapes(t *testing.T) {
shapes := loadShapes(t)
endpoints := []parityEndpoint{ _, router := setupTestServer(t)
{"stats", "/api/stats"},
{"nodes", "/api/nodes?limit=5"}, endpoints := []parityEndpoint{
{"packets", "/api/packets?limit=5"}, {"stats", "/api/stats"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"}, {"nodes", "/api/nodes?limit=5"},
{"observers", "/api/observers"}, {"packets", "/api/packets?limit=5"},
{"channels", "/api/channels"}, {"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"channel_messages", "/api/channels/0000000000000000/messages?limit=5"}, {"observers", "/api/observers"},
{"analytics_rf", "/api/analytics/rf?days=7"}, {"channels", "/api/channels"},
{"analytics_topology", "/api/analytics/topology?days=7"}, {"channel_messages", "/api/channels/0000000000000000/messages?limit=5"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"}, {"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"}, {"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"}, {"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
{"bulk_health", "/api/nodes/bulk-health"}, {"analytics_distance", "/api/analytics/distance?days=7"},
{"health", "/api/health"}, {"analytics_subpaths", "/api/analytics/subpaths?days=7"},
{"perf", "/api/perf"}, {"bulk_health", "/api/nodes/bulk-health"},
} {"health", "/api/health"},
{"perf", "/api/perf"},
for _, ep := range endpoints { }
t.Run("Parity_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name] for _, ep := range endpoints {
if !ok { t.Run("Parity_"+ep.name, func(t *testing.T) {
t.Fatalf("no shape spec found for %q in shapes.json", ep.name) spec, ok := shapes[ep.name]
} if !ok {
t.Fatalf("no shape spec found for %q in shapes.json", ep.name)
req := httptest.NewRequest("GET", ep.path, nil) }
w := httptest.NewRecorder()
router.ServeHTTP(w, req) req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
if w.Code != 200 { router.ServeHTTP(w, req)
t.Fatalf("GET %s returned %d, expected 200. Body: %s",
ep.path, w.Code, w.Body.String()) if w.Code != 200 {
} t.Fatalf("GET %s returned %d, expected 200. Body: %s",
ep.path, w.Code, w.Body.String())
var body interface{} }
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("GET %s returned invalid JSON: %v\nBody: %s", var body interface{}
ep.path, err, w.Body.String()) if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
} t.Fatalf("GET %s returned invalid JSON: %v\nBody: %s",
ep.path, err, w.Body.String())
mismatches := validateShape(body, spec, ep.path) }
if len(mismatches) > 0 {
t.Errorf("Go %s has %d shape mismatches vs Node.js golden:\n %s", mismatches := validateShape(body, spec, ep.path)
ep.path, len(mismatches), strings.Join(mismatches, "\n ")) if len(mismatches) > 0 {
} t.Errorf("Go %s has %d shape mismatches vs Node.js golden:\n %s",
}) ep.path, len(mismatches), strings.Join(mismatches, "\n "))
} }
} })
}
// TestParityNodeDetail tests node detail endpoint shape. }
// Uses a known test node public key from seeded data.
func TestParityNodeDetail(t *testing.T) { // TestParityNodeDetail tests node detail endpoint shape.
shapes := loadShapes(t) // Uses a known test node public key from seeded data.
_, router := setupTestServer(t) func TestParityNodeDetail(t *testing.T) {
shapes := loadShapes(t)
spec, ok := shapes["node_detail"] _, router := setupTestServer(t)
if !ok {
t.Fatal("no shape spec for node_detail in shapes.json") spec, ok := shapes["node_detail"]
} if !ok {
t.Fatal("no shape spec for node_detail in shapes.json")
req := httptest.NewRequest("GET", "/api/nodes/aabbccdd11223344", nil) }
w := httptest.NewRecorder()
router.ServeHTTP(w, req) req := httptest.NewRequest("GET", "/api/nodes/aabbccdd11223344", nil)
w := httptest.NewRecorder()
if w.Code != 200 { router.ServeHTTP(w, req)
t.Fatalf("node detail returned %d: %s", w.Code, w.Body.String())
} if w.Code != 200 {
t.Fatalf("node detail returned %d: %s", w.Code, w.Body.String())
var body interface{} }
json.Unmarshal(w.Body.Bytes(), &body)
var body interface{}
mismatches := validateShape(body, spec, "/api/nodes/{pubkey}") json.Unmarshal(w.Body.Bytes(), &body)
if len(mismatches) > 0 {
t.Errorf("Go node detail has %d shape mismatches vs Node.js golden:\n %s", mismatches := validateShape(body, spec, "/api/nodes/{pubkey}")
len(mismatches), strings.Join(mismatches, "\n ")) if len(mismatches) > 0 {
} t.Errorf("Go node detail has %d shape mismatches vs Node.js golden:\n %s",
} len(mismatches), strings.Join(mismatches, "\n "))
}
// TestParityArraysNotNull verifies that array-typed fields in Go responses are }
// [] (empty array) rather than null. This is a common Go/JSON pitfall where
// nil slices marshal as null instead of []. // TestParityArraysNotNull verifies that array-typed fields in Go responses are
// Uses shapes.json to know which fields SHOULD be arrays. // [] (empty array) rather than null. This is a common Go/JSON pitfall where
func TestParityArraysNotNull(t *testing.T) { // nil slices marshal as null instead of [].
shapes := loadShapes(t) // Uses shapes.json to know which fields SHOULD be arrays.
_, router := setupTestServer(t) func TestParityArraysNotNull(t *testing.T) {
shapes := loadShapes(t)
endpoints := []struct { _, router := setupTestServer(t)
name string
path string endpoints := []struct {
}{ name string
{"stats", "/api/stats"}, path string
{"nodes", "/api/nodes?limit=5"}, }{
{"packets", "/api/packets?limit=5"}, {"stats", "/api/stats"},
{"packets_grouped", "/api/packets?limit=5&groupByHash=true"}, {"nodes", "/api/nodes?limit=5"},
{"observers", "/api/observers"}, {"packets", "/api/packets?limit=5"},
{"channels", "/api/channels"}, {"packets_grouped", "/api/packets?limit=5&groupByHash=true"},
{"bulk_health", "/api/nodes/bulk-health"}, {"observers", "/api/observers"},
{"analytics_rf", "/api/analytics/rf?days=7"}, {"channels", "/api/channels"},
{"analytics_topology", "/api/analytics/topology?days=7"}, {"bulk_health", "/api/nodes/bulk-health"},
{"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"}, {"analytics_rf", "/api/analytics/rf?days=7"},
{"analytics_distance", "/api/analytics/distance?days=7"}, {"analytics_topology", "/api/analytics/topology?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"}, {"analytics_hash_sizes", "/api/analytics/hash-sizes?days=7"},
} {"analytics_distance", "/api/analytics/distance?days=7"},
{"analytics_subpaths", "/api/analytics/subpaths?days=7"},
for _, ep := range endpoints { }
t.Run("NullArrayCheck_"+ep.name, func(t *testing.T) {
spec, ok := shapes[ep.name] for _, ep := range endpoints {
if !ok { t.Run("NullArrayCheck_"+ep.name, func(t *testing.T) {
t.Skipf("no shape spec for %s", ep.name) spec, ok := shapes[ep.name]
} if !ok {
t.Skipf("no shape spec for %s", ep.name)
req := httptest.NewRequest("GET", ep.path, nil) }
w := httptest.NewRecorder()
router.ServeHTTP(w, req) req := httptest.NewRequest("GET", ep.path, nil)
w := httptest.NewRecorder()
if w.Code != 200 { router.ServeHTTP(w, req)
t.Skipf("GET %s returned %d, skipping null-array check", ep.path, w.Code)
} if w.Code != 200 {
t.Skipf("GET %s returned %d, skipping null-array check", ep.path, w.Code)
var body interface{} }
json.Unmarshal(w.Body.Bytes(), &body)
var body interface{}
nullArrays := findNullArrays(body, spec, ep.path) json.Unmarshal(w.Body.Bytes(), &body)
if len(nullArrays) > 0 {
t.Errorf("Go %s has null where [] expected:\n %s\n"+ nullArrays := findNullArrays(body, spec, ep.path)
"Go nil slices marshal as null — initialize with make() or literal", if len(nullArrays) > 0 {
ep.path, strings.Join(nullArrays, "\n ")) t.Errorf("Go %s has null where [] expected:\n %s\n"+
} "Go nil slices marshal as null — initialize with make() or literal",
}) ep.path, strings.Join(nullArrays, "\n "))
} }
} })
}
// findNullArrays walks JSON data alongside a shape spec and returns paths }
// where the spec says the field should be an array but Go returned null.
func findNullArrays(actual interface{}, spec shapeSpec, path string) []string { // findNullArrays walks JSON data alongside a shape spec and returns paths
var nulls []string // where the spec says the field should be an array but Go returned null.
func findNullArrays(actual interface{}, spec shapeSpec, path string) []string {
switch spec.Type { var nulls []string
case "array":
if actual == nil { switch spec.Type {
nulls = append(nulls, fmt.Sprintf("%s: null (should be [])", path)) case "array":
} else if arr, ok := actual.([]interface{}); ok && spec.ElementShape != nil { if actual == nil {
for i, elem := range arr { nulls = append(nulls, fmt.Sprintf("%s: null (should be [])", path))
nulls = append(nulls, findNullArrays(elem, *spec.ElementShape, fmt.Sprintf("%s[%d]", path, i))...) } else if arr, ok := actual.([]interface{}); ok && spec.ElementShape != nil {
} for i, elem := range arr {
} nulls = append(nulls, findNullArrays(elem, *spec.ElementShape, fmt.Sprintf("%s[%d]", path, i))...)
case "object": }
obj, ok := actual.(map[string]interface{}) }
if !ok || obj == nil { case "object":
return nulls obj, ok := actual.(map[string]interface{})
} if !ok || obj == nil {
if spec.Keys != nil { return nulls
for key, keySpec := range spec.Keys { }
if val, exists := obj[key]; exists { if spec.Keys != nil {
nulls = append(nulls, findNullArrays(val, keySpec, path+"."+key)...) for key, keySpec := range spec.Keys {
} else if keySpec.Type == "array" { if val, exists := obj[key]; exists {
// Key missing entirely — also a null-array problem nulls = append(nulls, findNullArrays(val, keySpec, path+"."+key)...)
nulls = append(nulls, fmt.Sprintf("%s.%s: missing (should be [])", path, key)) } else if keySpec.Type == "array" {
} // Key missing entirely — also a null-array problem
} nulls = append(nulls, fmt.Sprintf("%s.%s: missing (should be [])", path, key))
} }
if spec.DynamicKeys && spec.ValueShape != nil { }
for k, v := range obj { }
nulls = append(nulls, findNullArrays(v, *spec.ValueShape, path+"."+k)...) if spec.DynamicKeys && spec.ValueShape != nil {
break // sample one for k, v := range obj {
} nulls = append(nulls, findNullArrays(v, *spec.ValueShape, path+"."+k)...)
} break // sample one
} }
}
return nulls }
}
return nulls
// TestParityHealthEngine verifies Go health endpoint declares engine=go }
// while Node declares engine=node (or omits it). The Go server must always
// identify itself. // TestParityHealthEngine verifies Go health endpoint declares engine=go
func TestParityHealthEngine(t *testing.T) { // while Node declares engine=node (or omits it). The Go server must always
_, router := setupTestServer(t) // identify itself.
func TestParityHealthEngine(t *testing.T) {
req := httptest.NewRequest("GET", "/api/health", nil) _, router := setupTestServer(t)
w := httptest.NewRecorder()
router.ServeHTTP(w, req) req := httptest.NewRequest("GET", "/api/health", nil)
w := httptest.NewRecorder()
var body map[string]interface{} router.ServeHTTP(w, req)
json.Unmarshal(w.Body.Bytes(), &body)
var body map[string]interface{}
engine, ok := body["engine"] json.Unmarshal(w.Body.Bytes(), &body)
if !ok {
t.Error("health response missing 'engine' field (Go server must include engine=go)") engine, ok := body["engine"]
} else if engine != "go" { if !ok {
t.Errorf("health engine=%v, expected 'go'", engine) t.Error("health response missing 'engine' field (Go server must include engine=go)")
} } else if engine != "go" {
} t.Errorf("health engine=%v, expected 'go'", engine)
}
// TestValidateShapeFunction directly tests the shape validator itself. }
func TestValidateShapeFunction(t *testing.T) {
t.Run("string match", func(t *testing.T) { // TestValidateShapeFunction directly tests the shape validator itself.
errs := validateShape("hello", shapeSpec{Type: "string"}, "$.x") func TestValidateShapeFunction(t *testing.T) {
if len(errs) != 0 { t.Run("string match", func(t *testing.T) {
t.Errorf("unexpected errors: %v", errs) errs := validateShape("hello", shapeSpec{Type: "string"}, "$.x")
} if len(errs) != 0 {
}) t.Errorf("unexpected errors: %v", errs)
}
t.Run("string mismatch", func(t *testing.T) { })
errs := validateShape(42.0, shapeSpec{Type: "string"}, "$.x")
if len(errs) != 1 { t.Run("string mismatch", func(t *testing.T) {
t.Errorf("expected 1 error, got %d: %v", len(errs), errs) errs := validateShape(42.0, shapeSpec{Type: "string"}, "$.x")
} if len(errs) != 1 {
}) t.Errorf("expected 1 error, got %d: %v", len(errs), errs)
}
t.Run("null array rejected", func(t *testing.T) { })
errs := validateShape(nil, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 1 || !strings.Contains(errs[0], "null") { t.Run("null array rejected", func(t *testing.T) {
t.Errorf("expected null-array error, got: %v", errs) errs := validateShape(nil, shapeSpec{Type: "array"}, "$.arr")
} if len(errs) != 1 || !strings.Contains(errs[0], "null") {
}) t.Errorf("expected null-array error, got: %v", errs)
}
t.Run("empty array OK", func(t *testing.T) { })
errs := validateShape([]interface{}{}, shapeSpec{Type: "array"}, "$.arr")
if len(errs) != 0 { t.Run("empty array OK", func(t *testing.T) {
t.Errorf("unexpected errors for empty array: %v", errs) errs := validateShape([]interface{}{}, shapeSpec{Type: "array"}, "$.arr")
} if len(errs) != 0 {
}) t.Errorf("unexpected errors for empty array: %v", errs)
}
t.Run("missing object key", func(t *testing.T) { })
spec := shapeSpec{Type: "object", Keys: map[string]shapeSpec{
"name": {Type: "string"}, t.Run("missing object key", func(t *testing.T) {
"age": {Type: "number"}, spec := shapeSpec{Type: "object", Keys: map[string]shapeSpec{
}} "name": {Type: "string"},
obj := map[string]interface{}{"name": "test"} "age": {Type: "number"},
errs := validateShape(obj, spec, "$.user") }}
if len(errs) != 1 || !strings.Contains(errs[0], "age") { obj := map[string]interface{}{"name": "test"}
t.Errorf("expected missing age error, got: %v", errs) errs := validateShape(obj, spec, "$.user")
} if len(errs) != 1 || !strings.Contains(errs[0], "age") {
}) t.Errorf("expected missing age error, got: %v", errs)
}
t.Run("nullable allows null", func(t *testing.T) { })
errs := validateShape(nil, shapeSpec{Type: "nullable"}, "$.x")
if len(errs) != 0 { t.Run("nullable allows null", func(t *testing.T) {
t.Errorf("nullable should accept null: %v", errs) errs := validateShape(nil, shapeSpec{Type: "nullable"}, "$.x")
} if len(errs) != 0 {
}) t.Errorf("nullable should accept null: %v", errs)
}
t.Run("dynamic keys validates value shape", func(t *testing.T) { })
spec := shapeSpec{
Type: "object", t.Run("dynamic keys validates value shape", func(t *testing.T) {
DynamicKeys: true, spec := shapeSpec{
ValueShape: &shapeSpec{Type: "number"}, Type: "object",
} DynamicKeys: true,
obj := map[string]interface{}{"a": 1.0, "b": 2.0} ValueShape: &shapeSpec{Type: "number"},
errs := validateShape(obj, spec, "$.dyn") }
if len(errs) != 0 { obj := map[string]interface{}{"a": 1.0, "b": 2.0}
t.Errorf("unexpected errors: %v", errs) errs := validateShape(obj, spec, "$.dyn")
} if len(errs) != 0 {
}) t.Errorf("unexpected errors: %v", errs)
} }
})
}
func TestParityWSMultiObserverGolden(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
hub := NewHub()
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store load failed: %v", err)
}
poller := NewPoller(db, hub, 50*time.Millisecond)
poller.store = store
client := &Client{send: make(chan []byte, 256)}
hub.Register(client)
defer hub.Unregister(client)
go poller.Start()
defer poller.Stop()
// Wait for poller to initialize its lastID/lastObsID cursors before
// inserting new data; otherwise the poller may snapshot a lastID that
// already includes the test data and never broadcast it.
time.Sleep(100 * time.Millisecond)
now := time.Now().UTC().Format(time.RFC3339)
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('BEEF', 'goldenstarburst237', ?, 1, 4, '{"pubKey":"aabbccdd11223344","type":"ADVERT"}')`, now); err != nil {
t.Fatalf("insert tx failed: %v", err)
}
var txID int
if err := db.conn.QueryRow(`SELECT id FROM transmissions WHERE hash='goldenstarburst237'`).Scan(&txID); err != nil {
t.Fatalf("query tx id failed: %v", err)
}
ts := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (?, 1, 11.0, -88, '["p1"]', ?),
(?, 2, 9.0, -92, '["p1","p2"]', ?),
(?, 1, 7.0, -96, '["p1","p2","p3"]', ?)`,
txID, ts, txID, ts+1, txID, ts+2); err != nil {
t.Fatalf("insert obs failed: %v", err)
}
type golden struct {
Hash string
Count int
Paths []string
ObserverIDs []string
}
expected := golden{
Hash: "goldenstarburst237",
Count: 3,
Paths: []string{`["p1"]`, `["p1","p2"]`, `["p1","p2","p3"]`},
ObserverIDs: []string{"obs1", "obs2"},
}
gotPaths := make([]string, 0, expected.Count)
gotObservers := make(map[string]bool)
deadline := time.After(2 * time.Second)
for len(gotPaths) < expected.Count {
select {
case raw := <-client.send:
var msg map[string]interface{}
if err := json.Unmarshal(raw, &msg); err != nil {
t.Fatalf("unmarshal ws message failed: %v", err)
}
if msg["type"] != "packet" {
continue
}
data, _ := msg["data"].(map[string]interface{})
if data == nil || data["hash"] != expected.Hash {
continue
}
if path, ok := data["path_json"].(string); ok {
gotPaths = append(gotPaths, path)
}
if oid, ok := data["observer_id"].(string); ok && oid != "" {
gotObservers[oid] = true
}
case <-deadline:
t.Fatalf("timed out waiting for %d ws messages, got %d", expected.Count, len(gotPaths))
}
}
sort.Strings(gotPaths)
sort.Strings(expected.Paths)
if len(gotPaths) != len(expected.Paths) {
t.Fatalf("path count mismatch: got %d want %d", len(gotPaths), len(expected.Paths))
}
for i := range expected.Paths {
if gotPaths[i] != expected.Paths[i] {
t.Fatalf("path mismatch at %d: got %q want %q", i, gotPaths[i], expected.Paths[i])
}
}
for _, oid := range expected.ObserverIDs {
if !gotObservers[oid] {
t.Fatalf("missing expected observer %q in ws messages", oid)
}
}
}
+70 -35
View File
@@ -1039,7 +1039,7 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
} }
} }
// Build broadcast maps (same shape as Node.js WS broadcast) // Build broadcast maps (same shape as Node.js WS broadcast), one per observation.
result := make([]map[string]interface{}, 0, len(broadcastOrder)) result := make([]map[string]interface{}, 0, len(broadcastOrder))
for _, txID := range broadcastOrder { for _, txID := range broadcastOrder {
tx := broadcastTxs[txID] tx := broadcastTxs[txID]
@@ -1055,32 +1055,34 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
decoded["payload"] = payload decoded["payload"] = payload
} }
} }
// Build the nested packet object (packets.js checks m.data.packet) for _, obs := range tx.Observations {
pkt := map[string]interface{}{ // Build the nested packet object (packets.js checks m.data.packet)
"id": tx.ID, pkt := map[string]interface{}{
"raw_hex": strOrNil(tx.RawHex), "id": tx.ID,
"hash": strOrNil(tx.Hash), "raw_hex": strOrNil(tx.RawHex),
"first_seen": strOrNil(tx.FirstSeen), "hash": strOrNil(tx.Hash),
"timestamp": strOrNil(tx.FirstSeen), "first_seen": strOrNil(tx.FirstSeen),
"route_type": intPtrOrNil(tx.RouteType), "timestamp": strOrNil(tx.FirstSeen),
"payload_type": intPtrOrNil(tx.PayloadType), "route_type": intPtrOrNil(tx.RouteType),
"decoded_json": strOrNil(tx.DecodedJSON), "payload_type": intPtrOrNil(tx.PayloadType),
"observer_id": strOrNil(tx.ObserverID), "decoded_json": strOrNil(tx.DecodedJSON),
"observer_name": strOrNil(tx.ObserverName), "observer_id": strOrNil(obs.ObserverID),
"snr": floatPtrOrNil(tx.SNR), "observer_name": strOrNil(obs.ObserverName),
"rssi": floatPtrOrNil(tx.RSSI), "snr": floatPtrOrNil(obs.SNR),
"path_json": strOrNil(tx.PathJSON), "rssi": floatPtrOrNil(obs.RSSI),
"direction": strOrNil(tx.Direction), "path_json": strOrNil(obs.PathJSON),
"observation_count": tx.ObservationCount, "direction": strOrNil(obs.Direction),
"observation_count": tx.ObservationCount,
}
// Broadcast map: top-level fields for live.js + nested packet for packets.js
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
result = append(result, broadcastMap)
} }
// Broadcast map: top-level fields for live.js + nested packet for packets.js
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
result = append(result, broadcastMap)
} }
// Invalidate analytics caches since new data was ingested // Invalidate analytics caches since new data was ingested
@@ -1101,7 +1103,7 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
// IngestNewObservations loads new observations for transmissions already in the // IngestNewObservations loads new observations for transmissions already in the
// store. This catches observations that arrive after IngestNewFromDB has already // store. This catches observations that arrive after IngestNewFromDB has already
// advanced past the transmission's ID (fixes #174). // advanced past the transmission's ID (fixes #174).
func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int { func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) []map[string]interface{} {
if limit <= 0 { if limit <= 0 {
limit = 500 limit = 500
} }
@@ -1127,7 +1129,7 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
rows, err := s.db.conn.Query(querySQL, sinceObsID, limit) rows, err := s.db.conn.Query(querySQL, sinceObsID, limit)
if err != nil { if err != nil {
log.Printf("[store] ingest observations query error: %v", err) log.Printf("[store] ingest observations query error: %v", err)
return sinceObsID return nil
} }
defer rows.Close() defer rows.Close()
@@ -1170,20 +1172,16 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
} }
if len(obsRows) == 0 { if len(obsRows) == 0 {
return sinceObsID return nil
} }
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
newMaxObsID := sinceObsID
updatedTxs := make(map[int]*StoreTx) updatedTxs := make(map[int]*StoreTx)
broadcastMaps := make([]map[string]interface{}, 0, len(obsRows))
for _, r := range obsRows { for _, r := range obsRows {
if r.obsID > newMaxObsID {
newMaxObsID = r.obsID
}
// Already ingested (e.g. by IngestNewFromDB in same cycle) // Already ingested (e.g. by IngestNewFromDB in same cycle)
if _, exists := s.byObsID[r.obsID]; exists { if _, exists := s.byObsID[r.obsID]; exists {
continue continue
@@ -1226,6 +1224,43 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
} }
s.totalObs++ s.totalObs++
updatedTxs[r.txID] = tx updatedTxs[r.txID] = tx
decoded := map[string]interface{}{
"header": map[string]interface{}{
"payloadTypeName": resolvePayloadTypeName(tx.PayloadType),
},
}
if tx.DecodedJSON != "" {
var payload map[string]interface{}
if json.Unmarshal([]byte(tx.DecodedJSON), &payload) == nil {
decoded["payload"] = payload
}
}
pkt := map[string]interface{}{
"id": tx.ID,
"raw_hex": strOrNil(tx.RawHex),
"hash": strOrNil(tx.Hash),
"first_seen": strOrNil(tx.FirstSeen),
"timestamp": strOrNil(tx.FirstSeen),
"route_type": intPtrOrNil(tx.RouteType),
"payload_type": intPtrOrNil(tx.PayloadType),
"decoded_json": strOrNil(tx.DecodedJSON),
"observer_id": strOrNil(obs.ObserverID),
"observer_name": strOrNil(obs.ObserverName),
"snr": floatPtrOrNil(obs.SNR),
"rssi": floatPtrOrNil(obs.RSSI),
"path_json": strOrNil(obs.PathJSON),
"direction": strOrNil(obs.Direction),
"observation_count": tx.ObservationCount,
}
broadcastMap := make(map[string]interface{}, len(pkt)+2)
for k, v := range pkt {
broadcastMap[k] = v
}
broadcastMap["decoded"] = decoded
broadcastMap["packet"] = pkt
broadcastMaps = append(broadcastMaps, broadcastMap)
} }
// Re-pick best observation for updated transmissions and update subpath index // Re-pick best observation for updated transmissions and update subpath index
@@ -1280,7 +1315,7 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
// analytics caches cleared; no per-cycle log to avoid stdout overhead // analytics caches cleared; no per-cycle log to avoid stdout overhead
} }
return newMaxObsID return broadcastMaps
} }
// MaxTransmissionID returns the highest transmission ID in the store. // MaxTransmissionID returns the highest transmission ID in the store.
+245 -229
View File
@@ -1,229 +1,245 @@
package main package main
import ( import (
"encoding/json" "encoding/json"
"log" "log"
"net/http" "net/http"
"strings" "strings"
"sync" "sync"
"time" "time"
"github.com/gorilla/websocket" "github.com/gorilla/websocket"
) )
var upgrader = websocket.Upgrader{ var upgrader = websocket.Upgrader{
ReadBufferSize: 1024, ReadBufferSize: 1024,
WriteBufferSize: 4096, WriteBufferSize: 4096,
CheckOrigin: func(r *http.Request) bool { return true }, CheckOrigin: func(r *http.Request) bool { return true },
} }
// Hub manages WebSocket clients and broadcasts. // Hub manages WebSocket clients and broadcasts.
type Hub struct { type Hub struct {
mu sync.RWMutex mu sync.RWMutex
clients map[*Client]bool clients map[*Client]bool
} }
// Client is a single WebSocket connection. // Client is a single WebSocket connection.
type Client struct { type Client struct {
conn *websocket.Conn conn *websocket.Conn
send chan []byte send chan []byte
} }
func NewHub() *Hub { func NewHub() *Hub {
return &Hub{ return &Hub{
clients: make(map[*Client]bool), clients: make(map[*Client]bool),
} }
} }
func (h *Hub) ClientCount() int { func (h *Hub) ClientCount() int {
h.mu.RLock() h.mu.RLock()
defer h.mu.RUnlock() defer h.mu.RUnlock()
return len(h.clients) return len(h.clients)
} }
func (h *Hub) Register(c *Client) { func (h *Hub) Register(c *Client) {
h.mu.Lock() h.mu.Lock()
h.clients[c] = true h.clients[c] = true
h.mu.Unlock() h.mu.Unlock()
log.Printf("[ws] client connected (%d total)", h.ClientCount()) log.Printf("[ws] client connected (%d total)", h.ClientCount())
} }
func (h *Hub) Unregister(c *Client) { func (h *Hub) Unregister(c *Client) {
h.mu.Lock() h.mu.Lock()
if _, ok := h.clients[c]; ok { if _, ok := h.clients[c]; ok {
delete(h.clients, c) delete(h.clients, c)
close(c.send) close(c.send)
} }
h.mu.Unlock() h.mu.Unlock()
log.Printf("[ws] client disconnected (%d total)", h.ClientCount()) log.Printf("[ws] client disconnected (%d total)", h.ClientCount())
} }
// Broadcast sends a message to all connected clients. // Broadcast sends a message to all connected clients.
func (h *Hub) Broadcast(msg interface{}) { func (h *Hub) Broadcast(msg interface{}) {
data, err := json.Marshal(msg) data, err := json.Marshal(msg)
if err != nil { if err != nil {
log.Printf("[ws] marshal error: %v", err) log.Printf("[ws] marshal error: %v", err)
return return
} }
h.mu.RLock() h.mu.RLock()
defer h.mu.RUnlock() defer h.mu.RUnlock()
for c := range h.clients { for c := range h.clients {
select { select {
case c.send <- data: case c.send <- data:
default: default:
// Client buffer full — drop // Client buffer full — drop
} }
} }
} }
// ServeWS handles the WebSocket upgrade and runs the client. // ServeWS handles the WebSocket upgrade and runs the client.
func (h *Hub) ServeWS(w http.ResponseWriter, r *http.Request) { func (h *Hub) ServeWS(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil) conn, err := upgrader.Upgrade(w, r, nil)
if err != nil { if err != nil {
log.Printf("[ws] upgrade error: %v", err) log.Printf("[ws] upgrade error: %v", err)
return return
} }
client := &Client{ client := &Client{
conn: conn, conn: conn,
send: make(chan []byte, 256), send: make(chan []byte, 256),
} }
h.Register(client) h.Register(client)
go client.writePump() go client.writePump()
go client.readPump(h) go client.readPump(h)
} }
// wsOrStatic upgrades WebSocket requests at any path, serves static files otherwise. // wsOrStatic upgrades WebSocket requests at any path, serves static files otherwise.
func wsOrStatic(hub *Hub, static http.Handler) http.Handler { func wsOrStatic(hub *Hub, static http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") { if strings.EqualFold(r.Header.Get("Upgrade"), "websocket") {
hub.ServeWS(w, r) hub.ServeWS(w, r)
return return
} }
static.ServeHTTP(w, r) static.ServeHTTP(w, r)
}) })
} }
func (c *Client) readPump(hub *Hub) { func (c *Client) readPump(hub *Hub) {
defer func() { defer func() {
hub.Unregister(c) hub.Unregister(c)
c.conn.Close() c.conn.Close()
}() }()
c.conn.SetReadLimit(512) c.conn.SetReadLimit(512)
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second)) c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
c.conn.SetPongHandler(func(string) error { c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second)) c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
return nil return nil
}) })
for { for {
_, _, err := c.conn.ReadMessage() _, _, err := c.conn.ReadMessage()
if err != nil { if err != nil {
break break
} }
} }
} }
func (c *Client) writePump() { func (c *Client) writePump() {
ticker := time.NewTicker(30 * time.Second) ticker := time.NewTicker(30 * time.Second)
defer func() { defer func() {
ticker.Stop() ticker.Stop()
c.conn.Close() c.conn.Close()
}() }()
for { for {
select { select {
case message, ok := <-c.send: case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second)) c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if !ok { if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{}) c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return return
} }
if err := c.conn.WriteMessage(websocket.TextMessage, message); err != nil { if err := c.conn.WriteMessage(websocket.TextMessage, message); err != nil {
return return
} }
case <-ticker.C: case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second)) c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil { if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return return
} }
} }
} }
} }
// Poller watches for new transmissions in SQLite and broadcasts them. // Poller watches for new transmissions in SQLite and broadcasts them.
type Poller struct { type Poller struct {
db *DB db *DB
hub *Hub hub *Hub
store *PacketStore // optional: if set, new transmissions are ingested into memory store *PacketStore // optional: if set, new transmissions are ingested into memory
interval time.Duration interval time.Duration
stop chan struct{} stop chan struct{}
} }
func NewPoller(db *DB, hub *Hub, interval time.Duration) *Poller { func NewPoller(db *DB, hub *Hub, interval time.Duration) *Poller {
return &Poller{db: db, hub: hub, interval: interval, stop: make(chan struct{})} return &Poller{db: db, hub: hub, interval: interval, stop: make(chan struct{})}
} }
func (p *Poller) Start() { func (p *Poller) Start() {
lastID := p.db.GetMaxTransmissionID() lastID := p.db.GetMaxTransmissionID()
lastObsID := p.db.GetMaxObservationID() lastObsID := p.db.GetMaxObservationID()
log.Printf("[poller] starting from transmission ID %d, obs ID %d, interval %v", lastID, lastObsID, p.interval) log.Printf("[poller] starting from transmission ID %d, obs ID %d, interval %v", lastID, lastObsID, p.interval)
ticker := time.NewTicker(p.interval) ticker := time.NewTicker(p.interval)
defer ticker.Stop() defer ticker.Stop()
for { for {
select { select {
case <-ticker.C: case <-ticker.C:
if p.store != nil { if p.store != nil {
// Ingest new transmissions into in-memory store and broadcast // Ingest new transmissions into in-memory store and broadcast
newTxs, newMax := p.store.IngestNewFromDB(lastID, 100) newTxs, newMax := p.store.IngestNewFromDB(lastID, 100)
if newMax > lastID { if newMax > lastID {
lastID = newMax lastID = newMax
} }
// Ingest new observations for existing transmissions (fixes #174) // Ingest new observations for existing transmissions (fixes #174)
newObsMax := p.store.IngestNewObservations(lastObsID, 500) nextObsID := lastObsID
if newObsMax > lastObsID { if err := p.db.conn.QueryRow(`
lastObsID = newObsMax SELECT COALESCE(MAX(id), ?) FROM (
} SELECT id FROM observations
if len(newTxs) > 0 { WHERE id > ?
log.Printf("[broadcast] sending %d packets to %d clients (lastID now %d)", len(newTxs), p.hub.ClientCount(), lastID) ORDER BY id ASC
} LIMIT 500
for _, tx := range newTxs { )`, lastObsID, lastObsID).Scan(&nextObsID); err != nil {
p.hub.Broadcast(WSMessage{ nextObsID = lastObsID
Type: "packet", }
Data: tx, newObs := p.store.IngestNewObservations(lastObsID, 500)
}) if nextObsID > lastObsID {
} lastObsID = nextObsID
} else { }
// Fallback: direct DB query (used when store is nil, e.g. tests) if len(newTxs) > 0 {
newTxs, err := p.db.GetNewTransmissionsSince(lastID, 100) log.Printf("[broadcast] sending %d packets to %d clients (lastID now %d)", len(newTxs), p.hub.ClientCount(), lastID)
if err != nil { }
log.Printf("[poller] error: %v", err) for _, tx := range newTxs {
continue p.hub.Broadcast(WSMessage{
} Type: "packet",
for _, tx := range newTxs { Data: tx,
id, _ := tx["id"].(int) })
if id > lastID { }
lastID = id for _, obs := range newObs {
} p.hub.Broadcast(WSMessage{
// Copy packet fields for the nested packet (avoids circular ref) Type: "packet",
pkt := make(map[string]interface{}, len(tx)) Data: obs,
for k, v := range tx { })
pkt[k] = v }
} } else {
tx["packet"] = pkt // Fallback: direct DB query (used when store is nil, e.g. tests)
p.hub.Broadcast(WSMessage{ newTxs, err := p.db.GetNewTransmissionsSince(lastID, 100)
Type: "packet", if err != nil {
Data: tx, log.Printf("[poller] error: %v", err)
}) continue
} }
} for _, tx := range newTxs {
case <-p.stop: id, _ := tx["id"].(int)
return if id > lastID {
} lastID = id
} }
} // Copy packet fields for the nested packet (avoids circular ref)
pkt := make(map[string]interface{}, len(tx))
func (p *Poller) Stop() { for k, v := range tx {
close(p.stop) pkt[k] = v
} }
tx["packet"] = pkt
p.hub.Broadcast(WSMessage{
Type: "packet",
Data: tx,
})
}
}
case <-p.stop:
return
}
}
}
func (p *Poller) Stop() {
close(p.stop)
}
+415 -275
View File
@@ -1,275 +1,415 @@
package main package main
import ( import (
"encoding/json" "encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"testing" "sort"
"time" "testing"
"time"
"github.com/gorilla/websocket"
) "github.com/gorilla/websocket"
)
func TestHubBroadcast(t *testing.T) {
hub := NewHub() func TestHubBroadcast(t *testing.T) {
hub := NewHub()
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount()) if hub.ClientCount() != 0 {
} t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
// Create a test server with WebSocket endpoint
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Create a test server with WebSocket endpoint
hub.ServeWS(w, r) srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})) hub.ServeWS(w, r)
defer srv.Close() }))
defer srv.Close()
// Connect a WebSocket client
wsURL := "ws" + srv.URL[4:] // replace http with ws // Connect a WebSocket client
conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil) wsURL := "ws" + srv.URL[4:] // replace http with ws
if err != nil { conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
t.Fatalf("dial error: %v", err) if err != nil {
} t.Fatalf("dial error: %v", err)
defer conn.Close() }
defer conn.Close()
// Wait for registration
time.Sleep(50 * time.Millisecond) // Wait for registration
time.Sleep(50 * time.Millisecond)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client, got %d", hub.ClientCount()) if hub.ClientCount() != 1 {
} t.Errorf("expected 1 client, got %d", hub.ClientCount())
}
// Broadcast a message
hub.Broadcast(map[string]interface{}{ // Broadcast a message
"type": "packet", hub.Broadcast(map[string]interface{}{
"data": map[string]interface{}{"id": 1, "hash": "test123"}, "type": "packet",
}) "data": map[string]interface{}{"id": 1, "hash": "test123"},
})
// Read the message
conn.SetReadDeadline(time.Now().Add(2 * time.Second)) // Read the message
_, msg, err := conn.ReadMessage() conn.SetReadDeadline(time.Now().Add(2 * time.Second))
if err != nil { _, msg, err := conn.ReadMessage()
t.Fatalf("read error: %v", err) if err != nil {
} t.Fatalf("read error: %v", err)
if len(msg) == 0 { }
t.Error("expected non-empty message") if len(msg) == 0 {
} t.Error("expected non-empty message")
}
// Disconnect
conn.Close() // Disconnect
time.Sleep(100 * time.Millisecond) conn.Close()
} time.Sleep(100 * time.Millisecond)
}
func TestPollerCreation(t *testing.T) {
db := setupTestDB(t) func TestPollerCreation(t *testing.T) {
defer db.Close() db := setupTestDB(t)
seedTestData(t, db) defer db.Close()
hub := NewHub() seedTestData(t, db)
hub := NewHub()
poller := NewPoller(db, hub, 100*time.Millisecond)
if poller == nil { poller := NewPoller(db, hub, 100*time.Millisecond)
t.Fatal("expected poller") if poller == nil {
} t.Fatal("expected poller")
}
// Start and stop
go poller.Start() // Start and stop
time.Sleep(200 * time.Millisecond) go poller.Start()
poller.Stop() time.Sleep(200 * time.Millisecond)
} poller.Stop()
}
func TestHubMultipleClients(t *testing.T) {
hub := NewHub() func TestHubMultipleClients(t *testing.T) {
hub := NewHub()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hub.ServeWS(w, r) srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})) hub.ServeWS(w, r)
defer srv.Close() }))
defer srv.Close()
wsURL := "ws" + srv.URL[4:]
wsURL := "ws" + srv.URL[4:]
// Connect two clients
conn1, _, err := websocket.DefaultDialer.Dial(wsURL, nil) // Connect two clients
if err != nil { conn1, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
t.Fatalf("dial error: %v", err) if err != nil {
} t.Fatalf("dial error: %v", err)
defer conn1.Close() }
defer conn1.Close()
conn2, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
if err != nil { conn2, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
t.Fatalf("dial error: %v", err) if err != nil {
} t.Fatalf("dial error: %v", err)
defer conn2.Close() }
defer conn2.Close()
time.Sleep(100 * time.Millisecond)
time.Sleep(100 * time.Millisecond)
if hub.ClientCount() != 2 {
t.Errorf("expected 2 clients, got %d", hub.ClientCount()) if hub.ClientCount() != 2 {
} t.Errorf("expected 2 clients, got %d", hub.ClientCount())
}
// Broadcast and both should receive
hub.Broadcast(map[string]interface{}{"type": "test", "data": "hello"}) // Broadcast and both should receive
hub.Broadcast(map[string]interface{}{"type": "test", "data": "hello"})
conn1.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg1, err := conn1.ReadMessage() conn1.SetReadDeadline(time.Now().Add(2 * time.Second))
if err != nil { _, msg1, err := conn1.ReadMessage()
t.Fatalf("conn1 read error: %v", err) if err != nil {
} t.Fatalf("conn1 read error: %v", err)
if len(msg1) == 0 { }
t.Error("expected non-empty message on conn1") if len(msg1) == 0 {
} t.Error("expected non-empty message on conn1")
}
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg2, err := conn2.ReadMessage() conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
if err != nil { _, msg2, err := conn2.ReadMessage()
t.Fatalf("conn2 read error: %v", err) if err != nil {
} t.Fatalf("conn2 read error: %v", err)
if len(msg2) == 0 { }
t.Error("expected non-empty message on conn2") if len(msg2) == 0 {
} t.Error("expected non-empty message on conn2")
}
// Disconnect one
conn1.Close() // Disconnect one
time.Sleep(100 * time.Millisecond) conn1.Close()
time.Sleep(100 * time.Millisecond)
// Remaining client should still work
hub.Broadcast(map[string]interface{}{"type": "test2"}) // Remaining client should still work
hub.Broadcast(map[string]interface{}{"type": "test2"})
conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
_, msg3, err := conn2.ReadMessage() conn2.SetReadDeadline(time.Now().Add(2 * time.Second))
if err != nil { _, msg3, err := conn2.ReadMessage()
t.Fatalf("conn2 read error after disconnect: %v", err) if err != nil {
} t.Fatalf("conn2 read error after disconnect: %v", err)
if len(msg3) == 0 { }
t.Error("expected non-empty message") if len(msg3) == 0 {
} t.Error("expected non-empty message")
} }
}
func TestBroadcastFullBuffer(t *testing.T) {
hub := NewHub() func TestBroadcastFullBuffer(t *testing.T) {
hub := NewHub()
// Create a client with tiny buffer (1)
client := &Client{ // Create a client with tiny buffer (1)
send: make(chan []byte, 1), client := &Client{
} send: make(chan []byte, 1),
hub.mu.Lock() }
hub.clients[client] = true hub.mu.Lock()
hub.mu.Unlock() hub.clients[client] = true
hub.mu.Unlock()
// Fill the buffer
client.send <- []byte("first") // Fill the buffer
client.send <- []byte("first")
// This broadcast should drop the message (buffer full)
hub.Broadcast(map[string]interface{}{"type": "dropped"}) // This broadcast should drop the message (buffer full)
hub.Broadcast(map[string]interface{}{"type": "dropped"})
// Channel should still only have the first message
select { // Channel should still only have the first message
case msg := <-client.send: select {
if string(msg) != "first" { case msg := <-client.send:
t.Errorf("expected 'first', got %s", string(msg)) if string(msg) != "first" {
} t.Errorf("expected 'first', got %s", string(msg))
default: }
t.Error("expected message in channel") default:
} t.Error("expected message in channel")
}
// Clean up
hub.mu.Lock() // Clean up
delete(hub.clients, client) hub.mu.Lock()
hub.mu.Unlock() delete(hub.clients, client)
} hub.mu.Unlock()
}
func TestBroadcastMarshalError(t *testing.T) {
hub := NewHub() func TestBroadcastMarshalError(t *testing.T) {
hub := NewHub()
// Marshal error: functions can't be marshaled to JSON
hub.Broadcast(map[string]interface{}{"bad": func() {}}) // Marshal error: functions can't be marshaled to JSON
// Should not panic — just log and return hub.Broadcast(map[string]interface{}{"bad": func() {}})
} // Should not panic — just log and return
}
func TestPollerBroadcastsNewData(t *testing.T) {
db := setupTestDB(t) func TestPollerBroadcastsNewData(t *testing.T) {
defer db.Close() db := setupTestDB(t)
seedTestData(t, db) defer db.Close()
hub := NewHub() seedTestData(t, db)
hub := NewHub()
// Create a client to receive broadcasts
client := &Client{ // Create a client to receive broadcasts
send: make(chan []byte, 256), client := &Client{
} send: make(chan []byte, 256),
hub.mu.Lock() }
hub.clients[client] = true hub.mu.Lock()
hub.mu.Unlock() hub.clients[client] = true
hub.mu.Unlock()
poller := NewPoller(db, hub, 50*time.Millisecond)
go poller.Start() poller := NewPoller(db, hub, 50*time.Millisecond)
go poller.Start()
// Insert new data to trigger broadcast
db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type) // Insert new data to trigger broadcast
VALUES ('EEFF', 'newhash123456789', '2026-01-16T10:00:00Z', 1, 4)`) db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
VALUES ('EEFF', 'newhash123456789', '2026-01-16T10:00:00Z', 1, 4)`)
time.Sleep(200 * time.Millisecond)
poller.Stop() time.Sleep(200 * time.Millisecond)
poller.Stop()
// Check if client received broadcast with packet field (fixes #162)
select { // Check if client received broadcast with packet field (fixes #162)
case msg := <-client.send: select {
if len(msg) == 0 { case msg := <-client.send:
t.Error("expected non-empty broadcast message") if len(msg) == 0 {
} t.Error("expected non-empty broadcast message")
var parsed map[string]interface{} }
if err := json.Unmarshal(msg, &parsed); err != nil { var parsed map[string]interface{}
t.Fatalf("failed to parse broadcast: %v", err) if err := json.Unmarshal(msg, &parsed); err != nil {
} t.Fatalf("failed to parse broadcast: %v", err)
if parsed["type"] != "packet" { }
t.Errorf("expected type=packet, got %v", parsed["type"]) if parsed["type"] != "packet" {
} t.Errorf("expected type=packet, got %v", parsed["type"])
data, ok := parsed["data"].(map[string]interface{}) }
if !ok { data, ok := parsed["data"].(map[string]interface{})
t.Fatal("expected data to be an object") if !ok {
} t.Fatal("expected data to be an object")
// packets.js filters on m.data.packet — must exist }
pkt, ok := data["packet"] // packets.js filters on m.data.packet — must exist
if !ok || pkt == nil { pkt, ok := data["packet"]
t.Error("expected data.packet to exist (required by packets.js WS handler)") if !ok || pkt == nil {
} t.Error("expected data.packet to exist (required by packets.js WS handler)")
pktMap, ok := pkt.(map[string]interface{}) }
if !ok { pktMap, ok := pkt.(map[string]interface{})
t.Fatal("expected data.packet to be an object") if !ok {
} t.Fatal("expected data.packet to be an object")
// Verify key fields exist in nested packet (timestamp required by packets.js) }
for _, field := range []string{"id", "hash", "payload_type", "timestamp"} { // Verify key fields exist in nested packet (timestamp required by packets.js)
if _, exists := pktMap[field]; !exists { for _, field := range []string{"id", "hash", "payload_type", "timestamp"} {
t.Errorf("expected data.packet.%s to exist", field) if _, exists := pktMap[field]; !exists {
} t.Errorf("expected data.packet.%s to exist", field)
} }
default: }
// Might not have received due to timing default:
} // Might not have received due to timing
}
// Clean up
hub.mu.Lock() // Clean up
delete(hub.clients, client) hub.mu.Lock()
hub.mu.Unlock() delete(hub.clients, client)
} hub.mu.Unlock()
}
func TestHubRegisterUnregister(t *testing.T) {
hub := NewHub() func TestPollerBroadcastsMultipleObservations(t *testing.T) {
db := setupTestDB(t)
client := &Client{ defer db.Close()
send: make(chan []byte, 256), seedTestData(t, db)
} hub := NewHub()
hub.Register(client) client := &Client{
if hub.ClientCount() != 1 { send: make(chan []byte, 256),
t.Errorf("expected 1 client after register, got %d", hub.ClientCount()) }
} hub.mu.Lock()
hub.clients[client] = true
hub.Unregister(client) hub.mu.Unlock()
if hub.ClientCount() != 0 { defer func() {
t.Errorf("expected 0 clients after unregister, got %d", hub.ClientCount()) hub.mu.Lock()
} delete(hub.clients, client)
hub.mu.Unlock()
// Unregister again should be safe }()
hub.Unregister(client)
if hub.ClientCount() != 0 { poller := NewPoller(db, hub, 50*time.Millisecond)
t.Errorf("expected 0 clients, got %d", hub.ClientCount()) store := NewPacketStore(db)
} if err := store.Load(); err != nil {
} t.Fatalf("store load failed: %v", err)
}
poller.store = store
go poller.Start()
defer poller.Stop()
// Wait for poller to initialize its lastID/lastObsID cursors before
// inserting new data; otherwise the poller may snapshot a lastID that
// already includes the test data and never broadcast it.
time.Sleep(100 * time.Millisecond)
now := time.Now().UTC().Format(time.RFC3339)
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, decoded_json)
VALUES ('FACE', 'starbursthash237a', ?, 1, 4, '{"pubKey":"aabbccdd11223344","type":"ADVERT"}')`, now); err != nil {
t.Fatalf("insert tx failed: %v", err)
}
var txID int
if err := db.conn.QueryRow(`SELECT id FROM transmissions WHERE hash='starbursthash237a'`).Scan(&txID); err != nil {
t.Fatalf("query tx id failed: %v", err)
}
ts := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (?, 1, 14.0, -82, '["aa"]', ?),
(?, 2, 10.5, -90, '["aa","bb"]', ?),
(?, 1, 7.0, -96, '["aa","bb","cc"]', ?)`,
txID, ts, txID, ts+1, txID, ts+2); err != nil {
t.Fatalf("insert observations failed: %v", err)
}
deadline := time.After(2 * time.Second)
var dataMsgs []map[string]interface{}
for len(dataMsgs) < 3 {
select {
case raw := <-client.send:
var parsed map[string]interface{}
if err := json.Unmarshal(raw, &parsed); err != nil {
t.Fatalf("unmarshal ws msg failed: %v", err)
}
if parsed["type"] != "packet" {
continue
}
data, ok := parsed["data"].(map[string]interface{})
if !ok {
continue
}
if data["hash"] == "starbursthash237a" {
dataMsgs = append(dataMsgs, data)
}
case <-deadline:
t.Fatalf("timed out waiting for 3 observation broadcasts, got %d", len(dataMsgs))
}
}
if len(dataMsgs) != 3 {
t.Fatalf("expected 3 messages, got %d", len(dataMsgs))
}
paths := make([]string, 0, 3)
observers := make(map[string]bool)
for _, m := range dataMsgs {
hash, _ := m["hash"].(string)
if hash != "starbursthash237a" {
t.Fatalf("unexpected hash %q", hash)
}
p, _ := m["path_json"].(string)
paths = append(paths, p)
if oid, ok := m["observer_id"].(string); ok && oid != "" {
observers[oid] = true
}
}
sort.Strings(paths)
wantPaths := []string{`["aa","bb","cc"]`, `["aa","bb"]`, `["aa"]`}
sort.Strings(wantPaths)
for i := range wantPaths {
if paths[i] != wantPaths[i] {
t.Fatalf("path mismatch at %d: got %q want %q", i, paths[i], wantPaths[i])
}
}
if len(observers) < 2 {
t.Fatalf("expected observations from >=2 observers, got %d", len(observers))
}
}
func TestIngestNewObservationsBroadcast(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
seedTestData(t, db)
store := NewPacketStore(db)
if err := store.Load(); err != nil {
t.Fatalf("store load failed: %v", err)
}
maxObs := db.GetMaxObservationID()
now := time.Now().Unix()
if _, err := db.conn.Exec(`INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp)
VALUES (1, 2, 6.0, -100, '["aa","zz"]', ?),
(1, 1, 5.0, -101, '["aa","yy"]', ?)`, now, now+1); err != nil {
t.Fatalf("insert new observations failed: %v", err)
}
maps := store.IngestNewObservations(maxObs, 500)
if len(maps) != 2 {
t.Fatalf("expected 2 broadcast maps, got %d", len(maps))
}
for _, m := range maps {
if m["hash"] != "abc123def4567890" {
t.Fatalf("unexpected hash in map: %v", m["hash"])
}
path, ok := m["path_json"].(string)
if !ok || path == "" {
t.Fatalf("missing path_json in map: %#v", m)
}
if _, ok := m["observer_id"]; !ok {
t.Fatalf("missing observer_id in map: %#v", m)
}
}
}
func TestHubRegisterUnregister(t *testing.T) {
hub := NewHub()
client := &Client{
send: make(chan []byte, 256),
}
hub.Register(client)
if hub.ClientCount() != 1 {
t.Errorf("expected 1 client after register, got %d", hub.ClientCount())
}
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients after unregister, got %d", hub.ClientCount())
}
// Unregister again should be safe
hub.Unregister(client)
if hub.ClientCount() != 0 {
t.Errorf("expected 0 clients, got %d", hub.ClientCount())
}
}
+33 -23
View File
@@ -2,8 +2,8 @@
* MeshCore Packet Decoder * MeshCore Packet Decoder
* Custom implementation does NOT use meshcore-decoder library (known path_length bug). * Custom implementation does NOT use meshcore-decoder library (known path_length bug).
* *
* Packet layout: * Packet layout (per firmware docs/packet_format.md):
* [header(1)] [pathLength(1)] [transportCodes?] [path hops] [payload...] * [header(1)] [transportCodes?(4)] [pathLength(1)] [path hops] [payload...]
* *
* Header byte (LSB first): * Header byte (LSB first):
* bits 1-0: routeType (0=TRANSPORT_FLOOD, 1=FLOOD, 2=DIRECT, 3=TRANSPORT_DIRECT) * bits 1-0: routeType (0=TRANSPORT_FLOOD, 1=FLOOD, 2=DIRECT, 3=TRANSPORT_DIRECT)
@@ -42,7 +42,7 @@ const PAYLOAD_TYPES = {
0x0F: 'RAW_CUSTOM', 0x0F: 'RAW_CUSTOM',
}; };
// Route types that carry transport codes (nextHop + lastHop, 2 bytes each) // Route types that carry transport codes (2x uint16_t, 4 bytes total)
const TRANSPORT_ROUTES = new Set([0, 3]); // TRANSPORT_FLOOD, TRANSPORT_DIRECT const TRANSPORT_ROUTES = new Set([0, 3]); // TRANSPORT_FLOOD, TRANSPORT_DIRECT
// --- Header parsing --- // --- Header parsing ---
@@ -94,13 +94,11 @@ function decodeEncryptedPayload(buf) {
}; };
} }
/** ACK: dest(1) + src(1) + ack_hash(4) (per Mesh.cpp) */ /** ACK: checksum(4) — CRC of message timestamp + text + sender pubkey (per Mesh.cpp createAck) */
function decodeAck(buf) { function decodeAck(buf) {
if (buf.length < 6) return { error: 'too short', raw: buf.toString('hex') }; if (buf.length < 4) return { error: 'too short', raw: buf.toString('hex') };
return { return {
destHash: buf.subarray(0, 1).toString('hex'), ackChecksum: buf.subarray(0, 4).toString('hex'),
srcHash: buf.subarray(1, 2).toString('hex'),
extraHash: buf.subarray(2, 6).toString('hex'),
}; };
} }
@@ -125,6 +123,8 @@ function decodeAdvert(buf) {
room: advType === 3, room: advType === 3,
sensor: advType === 4, sensor: advType === 4,
hasLocation: !!(flags & 0x10), hasLocation: !!(flags & 0x10),
hasFeat1: !!(flags & 0x20),
hasFeat2: !!(flags & 0x40),
hasName: !!(flags & 0x80), hasName: !!(flags & 0x80),
}; };
@@ -134,6 +134,14 @@ function decodeAdvert(buf) {
result.lon = appdata.readInt32LE(off + 4) / 1e6; result.lon = appdata.readInt32LE(off + 4) / 1e6;
off += 8; off += 8;
} }
if (result.flags.hasFeat1 && appdata.length >= off + 2) {
result.feat1 = appdata.readUInt16LE(off);
off += 2;
}
if (result.flags.hasFeat2 && appdata.length >= off + 2) {
result.feat2 = appdata.readUInt16LE(off);
off += 2;
}
if (result.flags.hasName) { if (result.flags.hasName) {
// Find null terminator to separate name from trailing telemetry bytes // Find null terminator to separate name from trailing telemetry bytes
let nameEnd = appdata.length; let nameEnd = appdata.length;
@@ -231,7 +239,7 @@ function decodeGrpTxt(buf, channelKeys) {
return { type: 'GRP_TXT', channelHash, channelHashHex, decryptionStatus: 'no_key', mac, encryptedData }; return { type: 'GRP_TXT', channelHash, channelHashHex, decryptionStatus: 'no_key', mac, encryptedData };
} }
/** ANON_REQ: dest(6) + ephemeral_pubkey(32) + MAC(4) + encrypted */ /** ANON_REQ: dest(1) + ephemeral_pubkey(32) + MAC(2) + encrypted */
function decodeAnonReq(buf) { function decodeAnonReq(buf) {
if (buf.length < 35) return { error: 'too short', raw: buf.toString('hex') }; if (buf.length < 35) return { error: 'too short', raw: buf.toString('hex') };
return { return {
@@ -242,7 +250,7 @@ function decodeAnonReq(buf) {
}; };
} }
/** PATH: dest(6) + src(6) + MAC(4) + path_data */ /** PATH: dest(1) + src(1) + MAC(2) + path_data */
function decodePath_payload(buf) { function decodePath_payload(buf) {
if (buf.length < 4) return { error: 'too short', raw: buf.toString('hex') }; if (buf.length < 4) return { error: 'too short', raw: buf.toString('hex') };
return { return {
@@ -253,14 +261,14 @@ function decodePath_payload(buf) {
}; };
} }
/** TRACE: flags(1) + tag(4) + dest(6) + src(1) */ /** TRACE: tag(4) + authCode(4) + flags(1) + pathData (per Mesh.cpp onRecvPacket TRACE) */
function decodeTrace(buf) { function decodeTrace(buf) {
if (buf.length < 12) return { error: 'too short', raw: buf.toString('hex') }; if (buf.length < 9) return { error: 'too short', raw: buf.toString('hex') };
return { return {
flags: buf[0], tag: buf.readUInt32LE(0),
tag: buf.readUInt32LE(1), authCode: buf.subarray(4, 8).toString('hex'),
destHash: buf.subarray(5, 11).toString('hex'), flags: buf[8],
srcHash: buf.subarray(11, 12).toString('hex'), pathData: buf.subarray(9).toString('hex'),
}; };
} }
@@ -289,20 +297,22 @@ function decodePacket(hexString, channelKeys) {
if (buf.length < 2) throw new Error('Packet too short (need at least header + pathLength)'); if (buf.length < 2) throw new Error('Packet too short (need at least header + pathLength)');
const header = decodeHeader(buf[0]); const header = decodeHeader(buf[0]);
const pathByte = buf[1]; let offset = 1;
let offset = 2;
// Transport codes for TRANSPORT_FLOOD / TRANSPORT_DIRECT // Transport codes for TRANSPORT_FLOOD / TRANSPORT_DIRECT — BEFORE path_length per spec
let transportCodes = null; let transportCodes = null;
if (TRANSPORT_ROUTES.has(header.routeType)) { if (TRANSPORT_ROUTES.has(header.routeType)) {
if (buf.length < offset + 4) throw new Error('Packet too short for transport codes'); if (buf.length < offset + 4) throw new Error('Packet too short for transport codes');
transportCodes = { transportCodes = {
nextHop: buf.subarray(offset, offset + 2).toString('hex').toUpperCase(), code1: buf.subarray(offset, offset + 2).toString('hex').toUpperCase(),
lastHop: buf.subarray(offset + 2, offset + 4).toString('hex').toUpperCase(), code2: buf.subarray(offset + 2, offset + 4).toString('hex').toUpperCase(),
}; };
offset += 4; offset += 4;
} }
// Path length byte — AFTER transport codes per spec
const pathByte = buf[offset++];
// Path // Path
const path = decodePath(pathByte, buf, offset); const path = decodePath(pathByte, buf, offset);
offset += path.bytesConsumed; offset += path.bytesConsumed;
@@ -386,7 +396,7 @@ module.exports = { decodePacket, validateAdvert, hasNonPrintableChars, ROUTE_TYP
// --- Tests --- // --- Tests ---
if (require.main === module) { if (require.main === module) {
console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Test Repeater" ==='); console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Kpa Roof Solar" ===');
const pkt1 = decodePacket( const pkt1 = decodePacket(
'11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172' '11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172'
); );
@@ -402,7 +412,7 @@ if (require.main === module) {
assert(pkt1.path.hops[0] === '1000', 'first hop should be 1000'); assert(pkt1.path.hops[0] === '1000', 'first hop should be 1000');
assert(pkt1.path.hops[1] === 'D818', 'second hop should be D818'); assert(pkt1.path.hops[1] === 'D818', 'second hop should be D818');
assert(pkt1.transportCodes === null, 'FLOOD has no transport codes'); assert(pkt1.transportCodes === null, 'FLOOD has no transport codes');
assert(pkt1.payload.name === 'Test Repeater', 'name should be "Test Repeater"'); assert(pkt1.payload.name === 'Kpa Roof Solar', 'name should be "Kpa Roof Solar"');
console.log('✅ Test 1 passed\n'); console.log('✅ Test 1 passed\n');
console.log('=== Test 2: ADVERT, FLOOD, 0 hops (zero-path) ==='); console.log('=== Test 2: ADVERT, FLOOD, 0 hops (zero-path) ===');
+27 -27
View File
@@ -22,9 +22,9 @@
<meta name="twitter:title" content="CoreScope"> <meta name="twitter:title" content="CoreScope">
<meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis."> <meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis.">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/public/og-image.png"> <meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/corescope/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1774731523"> <link rel="stylesheet" href="style.css?v=1774786038">
<link rel="stylesheet" href="home.css?v=1774731523"> <link rel="stylesheet" href="home.css?v=1774786038">
<link rel="stylesheet" href="live.css?v=1774731523"> <link rel="stylesheet" href="live.css?v=1774786038">
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css" <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY=" integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
crossorigin="anonymous"> crossorigin="anonymous">
@@ -81,29 +81,29 @@
<main id="app" role="main"></main> <main id="app" role="main"></main>
<script src="vendor/qrcode.js"></script> <script src="vendor/qrcode.js"></script>
<script src="roles.js?v=1774731523"></script> <script src="roles.js?v=1774786038"></script>
<script src="customize.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="customize.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="region-filter.js?v=1774731523"></script> <script src="region-filter.js?v=1774786038"></script>
<script src="hop-resolver.js?v=1774731523"></script> <script src="hop-resolver.js?v=1774786038"></script>
<script src="hop-display.js?v=1774731523"></script> <script src="hop-display.js?v=1774786038"></script>
<script src="app.js?v=1774731523"></script> <script src="app.js?v=1774786038"></script>
<script src="home.js?v=1774731523"></script> <script src="home.js?v=1774786038"></script>
<script src="packet-filter.js?v=1774731523"></script> <script src="packet-filter.js?v=1774786038"></script>
<script src="packets.js?v=1774731523"></script> <script src="packets.js?v=1774786038"></script>
<script src="map.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="map.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="channels.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="nodes.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="traces.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="analytics.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="audio.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v1-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="audio-v1-constellation.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-v2-constellation.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="audio-v2-constellation.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="audio-lab.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="audio-lab.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="live.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="observers.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="observer-detail.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="compare.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="compare.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="node-analytics.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1774731523" onerror="console.error('Failed to load:', this.src)"></script> <script src="perf.js?v=1774786038" onerror="console.error('Failed to load:', this.src)"></script>
</body> </body>
</html> </html>
+5 -7
View File
@@ -1512,14 +1512,12 @@
rows += fieldRow(off + 1, 'Sender', decoded.sender || '—', ''); rows += fieldRow(off + 1, 'Sender', decoded.sender || '—', '');
if (decoded.sender_timestamp) rows += fieldRow(off + 2, 'Sender Time', decoded.sender_timestamp, ''); if (decoded.sender_timestamp) rows += fieldRow(off + 2, 'Sender Time', decoded.sender_timestamp, '');
} else if (decoded.type === 'ACK') { } else if (decoded.type === 'ACK') {
rows += fieldRow(off, 'Dest Hash (6B)', decoded.destHash || '', ''); rows += fieldRow(off, 'Checksum (4B)', decoded.ackChecksum || '', '');
rows += fieldRow(off + 6, 'Src Hash (6B)', decoded.srcHash || '', '');
rows += fieldRow(off + 12, 'Extra (6B)', decoded.extraHash || '', '');
} else if (decoded.destHash !== undefined) { } else if (decoded.destHash !== undefined) {
rows += fieldRow(off, 'Dest Hash (6B)', decoded.destHash || '', ''); rows += fieldRow(off, 'Dest Hash (1B)', decoded.destHash || '', '');
rows += fieldRow(off + 6, 'Src Hash (6B)', decoded.srcHash || '', ''); rows += fieldRow(off + 1, 'Src Hash (1B)', decoded.srcHash || '', '');
rows += fieldRow(off + 12, 'MAC (4B)', decoded.mac || '', ''); rows += fieldRow(off + 2, 'MAC (2B)', decoded.mac || '', '');
rows += fieldRow(off + 16, 'Encrypted Data', truncate(decoded.encryptedData || '', 30), ''); rows += fieldRow(off + 4, 'Encrypted Data', truncate(decoded.encryptedData || '', 30), '');
} else { } else {
rows += fieldRow(off, 'Raw', truncate(buf.slice(off * 2), 40), ''); rows += fieldRow(off, 'Raw', truncate(buf.slice(off * 2), 40), '');
} }
+6 -6
View File
@@ -40,12 +40,12 @@
html += `<h3>🔧 Go Runtime</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;"> html += `<h3>🔧 Go Runtime</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${gr.goroutines}</div><div class="perf-label">Goroutines</div></div> <div class="perf-card"><div class="perf-num">${gr.goroutines}</div><div class="perf-label">Goroutines</div></div>
<div class="perf-card"><div class="perf-num">${gr.numGC}</div><div class="perf-label">GC Collections</div></div> <div class="perf-card"><div class="perf-num">${gr.numGC}</div><div class="perf-label">GC Collections</div></div>
<div class="perf-card"><div class="perf-num" style="color:${gcColor}">${gr.pauseTotalMs}ms</div><div class="perf-label">GC Pause Total</div></div> <div class="perf-card"><div class="perf-num" style="color:${gcColor}">${(+gr.pauseTotalMs).toFixed(1)}ms</div><div class="perf-label">GC Pause Total</div></div>
<div class="perf-card"><div class="perf-num">${gr.lastPauseMs}ms</div><div class="perf-label">Last GC Pause</div></div> <div class="perf-card"><div class="perf-num">${(+gr.lastPauseMs).toFixed(1)}ms</div><div class="perf-label">Last GC Pause</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapAllocMB}MB</div><div class="perf-label">Heap Alloc</div></div> <div class="perf-card"><div class="perf-num">${(+gr.heapAllocMB).toFixed(1)}MB</div><div class="perf-label">Heap Alloc</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapSysMB}MB</div><div class="perf-label">Heap Sys</div></div> <div class="perf-card"><div class="perf-num">${(+gr.heapSysMB).toFixed(1)}MB</div><div class="perf-label">Heap Sys</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapInuseMB}MB</div><div class="perf-label">Heap Inuse</div></div> <div class="perf-card"><div class="perf-num">${(+gr.heapInuseMB).toFixed(1)}MB</div><div class="perf-label">Heap Inuse</div></div>
<div class="perf-card"><div class="perf-num">${gr.heapIdleMB}MB</div><div class="perf-label">Heap Idle</div></div> <div class="perf-card"><div class="perf-num">${(+gr.heapIdleMB).toFixed(1)}MB</div><div class="perf-label">Heap Idle</div></div>
<div class="perf-card"><div class="perf-num">${gr.numCPU}</div><div class="perf-label">CPUs</div></div> <div class="perf-card"><div class="perf-num">${gr.numCPU}</div><div class="perf-label">CPUs</div></div>
<div class="perf-card"><div class="perf-num">${health.websocket.clients}</div><div class="perf-label">WS Clients</div></div> <div class="perf-card"><div class="perf-num">${health.websocket.clients}</div><div class="perf-label">WS Clients</div></div>
</div>`; </div>`;
+1 -1
View File
@@ -155,7 +155,7 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
/* === Nav Stats === */ /* === Nav Stats === */
.nav-stats { .nav-stats {
display: flex; gap: 12px; align-items: center; font-size: 12px; color: var(--nav-text-muted); display: flex; gap: 12px; align-items: center; font-size: 12px; color: var(--nav-text-muted);
font-family: var(--mono); margin-right: 4px; font-family: var(--mono); margin-right: 4px; white-space: nowrap;
} }
.nav-stats .stat-val { color: var(--nav-text); font-weight: 600; transition: color 0.3s ease; } .nav-stats .stat-val { color: var(--nav-text); font-weight: 600; transition: color 0.3s ease; }
.nav-stats .stat-val.updated { color: var(--accent); } .nav-stats .stat-val.updated { color: var(--accent); }
+23 -23
View File
@@ -64,9 +64,9 @@ async function collectCoverage() {
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Home page — chooser...'); console.log(' [coverage] Home page — chooser...');
// Clear localStorage to get chooser // Clear localStorage to get chooser
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(BASE, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await page.evaluate(() => localStorage.clear()).catch(() => {}); await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click "I'm new" // Click "I'm new"
await safeClick('#chooseNew'); await safeClick('#chooseNew');
@@ -105,7 +105,7 @@ async function collectCoverage() {
// Switch to experienced mode // Switch to experienced mode
await page.evaluate(() => localStorage.clear()).catch(() => {}); await page.evaluate(() => localStorage.clear()).catch(() => {});
await page.goto(`${BASE}/#/home`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/home`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#chooseExp'); await safeClick('#chooseExp');
// Interact with experienced home page // Interact with experienced home page
@@ -120,7 +120,7 @@ async function collectCoverage() {
// NODES PAGE // NODES PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Nodes page...'); console.log(' [coverage] Nodes page...');
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Sort by EVERY column // Sort by EVERY column
for (const col of ['name', 'public_key', 'role', 'last_seen', 'advert_count']) { for (const col of ['name', 'public_key', 'role', 'last_seen', 'advert_count']) {
@@ -168,7 +168,7 @@ async function collectCoverage() {
try { try {
const firstNodeKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim()); const firstNodeKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim());
if (firstNodeKey) { if (firstNodeKey) {
await page.goto(`${BASE}/#/nodes/${firstNodeKey}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/nodes/${firstNodeKey}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click tabs on detail page // Click tabs on detail page
await clickAll('.tab-btn, [data-tab]', 10); await clickAll('.tab-btn, [data-tab]', 10);
@@ -191,7 +191,7 @@ async function collectCoverage() {
try { try {
const firstKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim()).catch(() => null); const firstKey = await page.$eval('#nodesBody tr td:nth-child(2)', el => el.textContent.trim()).catch(() => null);
if (firstKey) { if (firstKey) {
await page.goto(`${BASE}/#/nodes/${firstKey}?scroll=paths`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/nodes/${firstKey}?scroll=paths`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
} }
} catch {} } catch {}
@@ -199,7 +199,7 @@ async function collectCoverage() {
// PACKETS PAGE // PACKETS PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Packets page...'); console.log(' [coverage] Packets page...');
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Open filter bar // Open filter bar
await safeClick('#filterToggleBtn'); await safeClick('#filterToggleBtn');
@@ -285,13 +285,13 @@ async function collectCoverage() {
} catch {} } catch {}
// Navigate to specific packet by hash // Navigate to specific packet by hash
await page.goto(`${BASE}/#/packets/deadbeef`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/packets/deadbeef`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
// MAP PAGE // MAP PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Map page...'); console.log(' [coverage] Map page...');
await page.goto(`${BASE}/#/map`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/map`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Toggle controls panel // Toggle controls panel
await safeClick('#mapControlsToggle'); await safeClick('#mapControlsToggle');
@@ -345,7 +345,7 @@ async function collectCoverage() {
// ANALYTICS PAGE // ANALYTICS PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Analytics page...'); console.log(' [coverage] Analytics page...');
await page.goto(`${BASE}/#/analytics`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/analytics`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click EVERY analytics tab // Click EVERY analytics tab
const analyticsTabs = ['overview', 'rf', 'topology', 'channels', 'hashsizes', 'collisions', 'subpaths', 'nodes', 'distance']; const analyticsTabs = ['overview', 'rf', 'topology', 'channels', 'hashsizes', 'collisions', 'subpaths', 'nodes', 'distance'];
@@ -383,7 +383,7 @@ async function collectCoverage() {
// Deep-link to each analytics tab via URL // Deep-link to each analytics tab via URL
for (const tab of analyticsTabs) { for (const tab of analyticsTabs) {
await page.goto(`${BASE}/#/analytics?tab=${tab}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/analytics?tab=${tab}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
} }
// Region filter on analytics // Region filter on analytics
@@ -396,7 +396,7 @@ async function collectCoverage() {
// CUSTOMIZE // CUSTOMIZE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Customizer...'); console.log(' [coverage] Customizer...');
await page.goto(BASE, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(BASE, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#customizeToggle'); await safeClick('#customizeToggle');
// Click EVERY customizer tab // Click EVERY customizer tab
@@ -503,7 +503,7 @@ async function collectCoverage() {
// CHANNELS PAGE // CHANNELS PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Channels page...'); console.log(' [coverage] Channels page...');
await page.goto(`${BASE}/#/channels`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/channels`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click channel rows/items // Click channel rows/items
await clickAll('.channel-item, .channel-row, .channel-card', 3); await clickAll('.channel-item, .channel-row, .channel-card', 3);
await clickAll('table tbody tr', 3); await clickAll('table tbody tr', 3);
@@ -512,7 +512,7 @@ async function collectCoverage() {
try { try {
const channelHash = await page.$eval('table tbody tr td:first-child', el => el.textContent.trim()).catch(() => null); const channelHash = await page.$eval('table tbody tr td:first-child', el => el.textContent.trim()).catch(() => null);
if (channelHash) { if (channelHash) {
await page.goto(`${BASE}/#/channels/${channelHash}`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/channels/${channelHash}`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
} }
} catch {} } catch {}
@@ -520,7 +520,7 @@ async function collectCoverage() {
// LIVE PAGE // LIVE PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Live page...'); console.log(' [coverage] Live page...');
await page.goto(`${BASE}/#/live`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/live`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// VCR controls // VCR controls
await safeClick('#vcrPauseBtn'); await safeClick('#vcrPauseBtn');
@@ -603,14 +603,14 @@ async function collectCoverage() {
// TRACES PAGE // TRACES PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Traces page...'); console.log(' [coverage] Traces page...');
await page.goto(`${BASE}/#/traces`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/traces`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await clickAll('table tbody tr', 3); await clickAll('table tbody tr', 3);
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
// OBSERVERS PAGE // OBSERVERS PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Observers page...'); console.log(' [coverage] Observers page...');
await page.goto(`${BASE}/#/observers`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/observers`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Click observer rows // Click observer rows
const obsRows = await page.$$('table tbody tr, .observer-card, .observer-row'); const obsRows = await page.$$('table tbody tr, .observer-card, .observer-row');
for (let i = 0; i < Math.min(obsRows.length, 3); i++) { for (let i = 0; i < Math.min(obsRows.length, 3); i++) {
@@ -631,7 +631,7 @@ async function collectCoverage() {
// PERF PAGE // PERF PAGE
// ══════════════════════════════════════════════ // ══════════════════════════════════════════════
console.log(' [coverage] Perf page...'); console.log(' [coverage] Perf page...');
await page.goto(`${BASE}/#/perf`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/perf`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#perfRefresh'); await safeClick('#perfRefresh');
await safeClick('#perfReset'); await safeClick('#perfReset');
@@ -641,14 +641,14 @@ async function collectCoverage() {
console.log(' [coverage] App.js — router + global...'); console.log(' [coverage] App.js — router + global...');
// Navigate to bad route to trigger error/404 // Navigate to bad route to trigger error/404
await page.goto(`${BASE}/#/nonexistent-route`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/nonexistent-route`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
// Navigate to every route via hash // Navigate to every route via hash
const allRoutes = ['home', 'nodes', 'packets', 'map', 'live', 'channels', 'traces', 'observers', 'analytics', 'perf']; const allRoutes = ['home', 'nodes', 'packets', 'map', 'live', 'channels', 'traces', 'observers', 'analytics', 'perf'];
for (const route of allRoutes) { for (const route of allRoutes) {
try { try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route); await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {}); await new Promise(r => setTimeout(r, 200));
} catch {} } catch {}
} }
@@ -788,14 +788,14 @@ async function collectCoverage() {
console.log(' [coverage] Region filter...'); console.log(' [coverage] Region filter...');
try { try {
// Open region filter on nodes page // Open region filter on nodes page
await page.goto(`${BASE}/#/nodes`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/nodes`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#nodesRegionFilter'); await safeClick('#nodesRegionFilter');
await clickAll('#nodesRegionFilter input[type="checkbox"]', 3); await clickAll('#nodesRegionFilter input[type="checkbox"]', 3);
} catch {} } catch {}
// Region filter on packets // Region filter on packets
try { try {
await page.goto(`${BASE}/#/packets`, { waitUntil: 'networkidle', timeout: 15000 }).catch(() => {}); await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {});
await safeClick('#packetsRegionFilter'); await safeClick('#packetsRegionFilter');
await clickAll('#packetsRegionFilter input[type="checkbox"]', 3); await clickAll('#packetsRegionFilter input[type="checkbox"]', 3);
} catch {} } catch {}
@@ -807,7 +807,7 @@ async function collectCoverage() {
for (const route of allRoutes) { for (const route of allRoutes) {
try { try {
await page.evaluate((r) => { location.hash = '#/' + r; }, route); await page.evaluate((r) => { location.hash = '#/' + r; }, route);
await page.waitForLoadState('networkidle').catch(() => {}); await new Promise(r => setTimeout(r, 200));
} catch {} } catch {}
} }
+10 -3
View File
@@ -207,6 +207,13 @@ class TTLCache {
if (key.startsWith(prefix)) this.store.delete(key); if (key.startsWith(prefix)) this.store.delete(key);
} }
} }
debouncedInvalidateBulkHealth() {
if (this._bulkHealthTimer) return;
this._bulkHealthTimer = setTimeout(() => {
this._bulkHealthTimer = null;
this.invalidate('bulk-health');
}, 30000);
}
debouncedInvalidateAll() { debouncedInvalidateAll() {
if (this._debounceTimer) return; if (this._debounceTimer) return;
this._debounceTimer = setTimeout(() => { this._debounceTimer = setTimeout(() => {
@@ -410,7 +417,7 @@ app.get('/api/perf', (req, res) => {
avgMs: perfStats.requests ? Math.round(perfStats.totalMs / perfStats.requests * 10) / 10 : 0, avgMs: perfStats.requests ? Math.round(perfStats.totalMs / perfStats.requests * 10) / 10 : 0,
endpoints: Object.fromEntries(sorted), endpoints: Object.fromEntries(sorted),
slowQueries: perfStats.slowQueries.slice(-20), slowQueries: perfStats.slowQueries.slice(-20),
cache: { size: cache.size, hits: cache.hits, misses: cache.misses, staleHits: cache.staleHits, recomputes: cache.recomputes, hitRate: cache.hits + cache.misses > 0 ? Math.round(cache.hits / (cache.hits + cache.misses) * 1000) / 10 : 0 }, cache: { size: cache.size, hits: cache.hits, misses: cache.misses, staleHits: cache.staleHits, recomputes: cache.recomputes, hitRate: cache.hits + cache.staleHits + cache.misses > 0 ? Math.round((cache.hits + cache.staleHits) / (cache.hits + cache.staleHits + cache.misses) * 1000) / 10 : 0 },
packetStore: pktStore.getStats(), packetStore: pktStore.getStats(),
sqlite: (() => { sqlite: (() => {
try { try {
@@ -519,7 +526,7 @@ app.get('/api/health', (req, res) => {
misses: cache.misses, misses: cache.misses,
staleHits: cache.staleHits, staleHits: cache.staleHits,
recomputes: cache.recomputes, recomputes: cache.recomputes,
hitRate: cache.hits + cache.misses > 0 ? Math.round(cache.hits / (cache.hits + cache.misses) * 1000) / 10 : 0, hitRate: cache.hits + cache.staleHits + cache.misses > 0 ? Math.round((cache.hits + cache.staleHits) / (cache.hits + cache.staleHits + cache.misses) * 1000) / 10 : 0,
}, },
websocket: { websocket: {
clients: wsClients, clients: wsClients,
@@ -723,7 +730,7 @@ for (const source of mqttSources) {
// Invalidate this node's caches on advert // Invalidate this node's caches on advert
cache.invalidate('node:' + p.pubKey); cache.invalidate('node:' + p.pubKey);
cache.invalidate('health:' + p.pubKey); cache.invalidate('health:' + p.pubKey);
cache.invalidate('bulk-health'); cache.debouncedInvalidateBulkHealth();
// Cross-reference: if this node's pubkey matches an existing observer, backfill observer name // Cross-reference: if this node's pubkey matches an existing observer, backfill observer name
if (p.name && p.pubKey) { if (p.name && p.pubKey) {
+11 -10
View File
@@ -122,13 +122,14 @@ console.log('── Spec Tests: Transport Codes ──');
{ {
// Route type 0 (TRANSPORT_FLOOD) and 3 (TRANSPORT_DIRECT) should have 4-byte transport codes // Route type 0 (TRANSPORT_FLOOD) and 3 (TRANSPORT_DIRECT) should have 4-byte transport codes
// Route type 0: header byte = 0bPPPPPP00, e.g. 0x14 = payloadType 5 (GRP_TXT), routeType 0 // Route type 0: header=0x14 = payloadType 5 (GRP_TXT), routeType 0 (TRANSPORT_FLOOD)
const hex = '1400' + 'AABB' + 'CCDD' + '1A' + '00'.repeat(10); // transport codes + GRP_TXT payload // Format: header(1) + transportCodes(4) + pathByte(1) + payload
const hex = '14' + 'AABB' + 'CCDD' + '00' + '1A' + '00'.repeat(10); // transport codes + pathByte + GRP_TXT payload
const p = decodePacket(hex); const p = decodePacket(hex);
assertEq(p.header.routeType, 0, 'transport: routeType=0 (TRANSPORT_FLOOD)'); assertEq(p.header.routeType, 0, 'transport: routeType=0 (TRANSPORT_FLOOD)');
assert(p.transportCodes !== null, 'transport: transportCodes present for TRANSPORT_FLOOD'); assert(p.transportCodes !== null, 'transport: transportCodes present for TRANSPORT_FLOOD');
assertEq(p.transportCodes.nextHop, 'AABB', 'transport: nextHop'); assertEq(p.transportCodes.code1, 'AABB', 'transport: code1');
assertEq(p.transportCodes.lastHop, 'CCDD', 'transport: lastHop'); assertEq(p.transportCodes.code2, 'CCDD', 'transport: code2');
} }
{ {
@@ -257,13 +258,13 @@ console.log('── Spec Tests: Advert Payload ──');
console.log('── Spec Tests: Encrypted Payload Format ──'); console.log('── Spec Tests: Encrypted Payload Format ──');
// NOTE: Spec says v1 encrypted payloads have dest(1) + src(1) + MAC(2) + ciphertext // Spec says v1 encrypted payloads: dest(1)+src(1)+MAC(2)+cipher — decoder matches this.
// But decoder reads dest(6) + src(6) + MAC(4) + ciphertext
// This is a known discrepancy — the decoder matches production behavior, not the spec.
// The spec may describe the firmware's internal addressing while the OTA format differs,
// or the decoder may be parsing the fields differently. Production data validates the decoder.
{ {
note('Spec says v1 encrypted payloads: dest(1)+src(1)+MAC(2)+cipher, but decoder reads dest(6)+src(6)+MAC(4)+cipher — decoder matches prod data'); const hex = '0100' + 'AA' + 'BB' + 'CCDD' + '00'.repeat(10);
const p = decodePacket(hex);
assertEq(p.payload.destHash, 'aa', 'encrypted payload: dest is 1 byte');
assertEq(p.payload.srcHash, 'bb', 'encrypted payload: src is 1 byte');
assertEq(p.payload.mac, 'ccdd', 'encrypted payload: MAC is 2 bytes');
} }
console.log('── Spec Tests: validateAdvert ──'); console.log('── Spec Tests: validateAdvert ──');
+16 -16
View File
@@ -28,22 +28,22 @@ test('FLOOD + ADVERT = 0x11', () => {
}); });
test('TRANSPORT_FLOOD = routeType 0', () => { test('TRANSPORT_FLOOD = routeType 0', () => {
// 0x00 = TRANSPORT_FLOOD + REQ(0), needs transport codes + 16 byte payload // header=0x00 (TRANSPORT_FLOOD + REQ), transportCodes=AABB+CCDD, pathByte=0x00, payload
const hex = '0000' + 'AABB' + 'CCDD' + '00'.repeat(16); const hex = '00' + 'AABB' + 'CCDD' + '00' + '00'.repeat(16);
const p = decodePacket(hex); const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 0); assert.strictEqual(p.header.routeType, 0);
assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_FLOOD'); assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_FLOOD');
assert.notStrictEqual(p.transportCodes, null); assert.notStrictEqual(p.transportCodes, null);
assert.strictEqual(p.transportCodes.nextHop, 'AABB'); assert.strictEqual(p.transportCodes.code1, 'AABB');
assert.strictEqual(p.transportCodes.lastHop, 'CCDD'); assert.strictEqual(p.transportCodes.code2, 'CCDD');
}); });
test('TRANSPORT_DIRECT = routeType 3', () => { test('TRANSPORT_DIRECT = routeType 3', () => {
const hex = '0300' + '1122' + '3344' + '00'.repeat(16); const hex = '03' + '1122' + '3344' + '00' + '00'.repeat(16);
const p = decodePacket(hex); const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 3); assert.strictEqual(p.header.routeType, 3);
assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_DIRECT'); assert.strictEqual(p.header.routeTypeName, 'TRANSPORT_DIRECT');
assert.strictEqual(p.transportCodes.nextHop, '1122'); assert.strictEqual(p.transportCodes.code1, '1122');
}); });
test('DIRECT = routeType 2, no transport codes', () => { test('DIRECT = routeType 2, no transport codes', () => {
@@ -358,9 +358,7 @@ test('ACK decode', () => {
const hex = '0D00' + '00'.repeat(18); const hex = '0D00' + '00'.repeat(18);
const p = decodePacket(hex); const p = decodePacket(hex);
assert.strictEqual(p.payload.type, 'ACK'); assert.strictEqual(p.payload.type, 'ACK');
assert(p.payload.destHash); assert(p.payload.ackChecksum);
assert(p.payload.srcHash);
assert(p.payload.extraHash);
}); });
test('ACK too short', () => { test('ACK too short', () => {
@@ -424,9 +422,9 @@ test('TRACE decode', () => {
const hex = '2500' + '00'.repeat(12); const hex = '2500' + '00'.repeat(12);
const p = decodePacket(hex); const p = decodePacket(hex);
assert.strictEqual(p.payload.type, 'TRACE'); assert.strictEqual(p.payload.type, 'TRACE');
assert.strictEqual(p.payload.flags, 0);
assert(p.payload.tag !== undefined); assert(p.payload.tag !== undefined);
assert(p.payload.destHash); assert(p.payload.authCode !== undefined);
assert.strictEqual(p.payload.flags, 0);
}); });
test('TRACE too short', () => { test('TRACE too short', () => {
@@ -460,16 +458,18 @@ test('Transport route too short throws', () => {
assert.throws(() => decodePacket('0000'), /too short for transport/); assert.throws(() => decodePacket('0000'), /too short for transport/);
}); });
test('Corrupt packet #183 — path overflow capped to buffer', () => { test('Corrupt packet #183 — TRANSPORT_DIRECT with correct field order', () => {
const hex = 'BBAD6797EC8751D500BF95A1A776EF580E665BCBF6A0BBE03B5E730707C53489B8C728FD3FB902397197E1263CEC21E52465362243685DBBAD6797EC8751C90A75D9FD8213155D'; const hex = 'BBAD6797EC8751D500BF95A1A776EF580E665BCBF6A0BBE03B5E730707C53489B8C728FD3FB902397197E1263CEC21E52465362243685DBBAD6797EC8751C90A75D9FD8213155D';
const p = decodePacket(hex); const p = decodePacket(hex);
assert.strictEqual(p.header.routeType, 3, 'routeType should be TRANSPORT_DIRECT'); assert.strictEqual(p.header.routeType, 3, 'routeType should be TRANSPORT_DIRECT');
assert.strictEqual(p.header.payloadTypeName, 'UNKNOWN'); assert.strictEqual(p.header.payloadTypeName, 'UNKNOWN');
// pathByte 0xAD claims 45 hops × 3 bytes = 135, but only 65 bytes available // transport codes are bytes 1-4, pathByte=0x87 at byte 5
assert.strictEqual(p.transportCodes.code1, 'AD67');
assert.strictEqual(p.transportCodes.code2, '97EC');
// pathByte 0x87: hashSize=3, hashCount=7
assert.strictEqual(p.path.hashSize, 3); assert.strictEqual(p.path.hashSize, 3);
assert.strictEqual(p.path.hashCount, 21, 'hashCount capped to fit buffer'); assert.strictEqual(p.path.hashCount, 7);
assert.strictEqual(p.path.hops.length, 21); assert.strictEqual(p.path.hops.length, 7);
assert.strictEqual(p.path.truncated, true);
// No empty strings in hops // No empty strings in hops
assert(p.path.hops.every(h => h.length > 0), 'no empty hops'); assert(p.path.hops.every(h => h.length > 0), 'no empty hops');
}); });
+65 -11
View File
@@ -10,13 +10,21 @@ const GO_BASE = process.env.GO_BASE_URL || ''; // e.g. https://analyzer.00id.ne
const results = []; const results = [];
async function test(name, fn) { async function test(name, fn) {
try { const MAX_RETRIES = 2;
await fn(); for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
results.push({ name, pass: true }); try {
console.log(` \u2705 ${name}`); await fn();
} catch (err) { results.push({ name, pass: true });
results.push({ name, pass: false, error: err.message }); console.log(` \u2705 ${name}${attempt > 1 ? ` (retry ${attempt - 1})` : ''}`);
console.log(` \u274c ${name}: ${err.message}`); return;
} catch (err) {
if (attempt < MAX_RETRIES) {
console.log(` \u26a0\ufe0f ${name}: ${err.message} (retrying...)`);
continue;
}
results.push({ name, pass: false, error: err.message });
console.log(` \u274c ${name}: ${err.message}`);
}
} }
} }
@@ -324,7 +332,9 @@ async function run() {
// Test: Packets groupByHash toggle changes view // Test: Packets groupByHash toggle changes view
await test('Packets groupByHash toggle works', async () => { await test('Packets groupByHash toggle works', async () => {
await page.waitForSelector('table tbody tr'); // Fresh navigation to ensure clean state
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('table tbody tr', { timeout: 15000 });
const groupBtn = await page.$('#fGroup'); const groupBtn = await page.$('#fGroup');
assert(groupBtn, 'Group by hash button (#fGroup) not found'); assert(groupBtn, 'Group by hash button (#fGroup) not found');
// Check initial state (default is grouped/active) // Check initial state (default is grouped/active)
@@ -354,10 +364,17 @@ async function run() {
await test('Packets clicking row shows detail pane', async () => { await test('Packets clicking row shows detail pane', async () => {
// Fresh navigation to avoid stale row references from previous test // Fresh navigation to avoid stale row references from previous test
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' }); await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' });
// Wait for table rows AND initial API data to settle
await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 }); await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 });
await page.waitForLoadState('networkidle');
const firstRow = await page.$('table tbody tr[data-action]'); const firstRow = await page.$('table tbody tr[data-action]');
assert(firstRow, 'No clickable packet rows found'); assert(firstRow, 'No clickable packet rows found');
await firstRow.click(); // Click the row and wait for the /packets/{hash} API response
const [response] = await Promise.all([
page.waitForResponse(resp => resp.url().includes('/packets/') && resp.status() === 200, { timeout: 15000 }),
firstRow.click(),
]);
assert(response, 'API response for packet detail not received');
await page.waitForFunction(() => { await page.waitForFunction(() => {
const panel = document.getElementById('pktRight'); const panel = document.getElementById('pktRight');
return panel && !panel.classList.contains('empty'); return panel && !panel.classList.contains('empty');
@@ -375,12 +392,16 @@ async function run() {
if (!pktRight) { if (!pktRight) {
await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' }); await page.goto(`${BASE}/#/packets`, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 }); await page.waitForSelector('table tbody tr[data-action]', { timeout: 15000 });
await page.waitForLoadState('networkidle');
} }
const panelOpen = await page.$eval('#pktRight', el => !el.classList.contains('empty')); const panelOpen = await page.$eval('#pktRight', el => !el.classList.contains('empty'));
if (!panelOpen) { if (!panelOpen) {
const firstRow = await page.$('table tbody tr[data-action]'); const firstRow = await page.$('table tbody tr[data-action]');
if (!firstRow) { console.log(' ⏭️ Skipped (no clickable rows)'); return; } if (!firstRow) { console.log(' ⏭️ Skipped (no clickable rows)'); return; }
await firstRow.click(); await Promise.all([
page.waitForResponse(resp => resp.url().includes('/packets/') && resp.status() === 200, { timeout: 15000 }),
firstRow.click(),
]);
await page.waitForFunction(() => { await page.waitForFunction(() => {
const panel = document.getElementById('pktRight'); const panel = document.getElementById('pktRight');
return panel && !panel.classList.contains('empty'); return panel && !panel.classList.contains('empty');
@@ -523,6 +544,11 @@ async function run() {
}); });
await test('Compare page runs comparison', async () => { await test('Compare page runs comparison', async () => {
// Wait for dropdowns to be populated (may still be loading from previous test)
await page.waitForFunction(() => {
const selA = document.getElementById('compareObsA');
return selA && selA.options.length > 2;
}, { timeout: 10000 });
const options = await page.$$eval('#compareObsA option', opts => const options = await page.$$eval('#compareObsA option', opts =>
opts.filter(o => o.value).map(o => o.value) opts.filter(o => o.value).map(o => o.value)
); );
@@ -544,6 +570,12 @@ async function run() {
// Test: Compare results show shared/unique breakdown (#129) // Test: Compare results show shared/unique breakdown (#129)
await test('Compare results show shared/unique cards', async () => { await test('Compare results show shared/unique cards', async () => {
// Wait for comparison results to fully render (depends on previous test)
await page.waitForFunction(() => {
return document.querySelector('.compare-card-both') &&
document.querySelector('.compare-card-a') &&
document.querySelector('.compare-card-b');
}, { timeout: 10000 });
// Results should be visible from previous test // Results should be visible from previous test
const cardBoth = await page.$('.compare-card-both'); const cardBoth = await page.$('.compare-card-both');
assert(cardBoth, 'Should have "shared" card (.compare-card-both)'); assert(cardBoth, 'Should have "shared" card (.compare-card-both)');
@@ -566,6 +598,11 @@ async function run() {
// Test: Compare "both" tab shows table with shared packets // Test: Compare "both" tab shows table with shared packets
await test('Compare both tab shows shared packets table', async () => { await test('Compare both tab shows shared packets table', async () => {
// Ensure compare results are present
await page.waitForFunction(() => {
const c = document.getElementById('compareContent');
return c && c.textContent.trim().length > 20;
}, { timeout: 10000 });
const bothTab = await page.$('[data-cview="both"]'); const bothTab = await page.$('[data-cview="both"]');
assert(bothTab, '"both" tab button not found'); assert(bothTab, '"both" tab button not found');
await bothTab.click(); await bothTab.click();
@@ -789,7 +826,11 @@ async function run() {
// Check for summary stats // Check for summary stats
const summary = await page.$('.obs-summary'); const summary = await page.$('.obs-summary');
assert(summary, 'Observer summary stats not found'); assert(summary, 'Observer summary stats not found');
// Verify table has rows // Wait for table rows to populate
await page.waitForFunction(() => {
const rows = document.querySelectorAll('#obsTable tbody tr');
return rows.length > 0;
}, { timeout: 10000 });
const rows = await page.$$('#obsTable tbody tr'); const rows = await page.$$('#obsTable tbody tr');
assert(rows.length > 0, `Expected >=1 observer rows, got ${rows.length}`); assert(rows.length > 0, `Expected >=1 observer rows, got ${rows.length}`);
}); });
@@ -909,6 +950,19 @@ async function run() {
assert(hexDump, 'Hex dump should be visible after selecting a packet'); assert(hexDump, 'Hex dump should be visible after selecting a packet');
}); });
// Extract frontend coverage if instrumented server is running
try {
const coverage = await page.evaluate(() => window.__coverage__);
if (coverage) {
const fs = require('fs');
const path = require('path');
const outDir = path.join(__dirname, '.nyc_output');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
fs.writeFileSync(path.join(outDir, 'e2e-coverage.json'), JSON.stringify(coverage));
console.log(`Frontend coverage from E2E: ${Object.keys(coverage).length} files`);
}
} catch {}
await browser.close(); await browser.close();
// Summary // Summary
+6 -6
View File
@@ -1322,7 +1322,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
assert.ok(result.includes('>v2.6.0</a>'), 'version text has v prefix'); assert.ok(result.includes('>v2.6.0</a>'), 'version text has v prefix');
assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit links to full hash'); assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit links to full hash');
assert.ok(result.includes('>abc1234</a>'), 'commit display is truncated to 7'); assert.ok(result.includes('>abc1234</a>'), 'commit display is truncated to 7');
assert.ok(result.includes('[node]'), 'should show engine'); assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>node<'), 'should show engine name');
}); });
test('prod port 80: shows version', () => { test('prod port 80: shows version', () => {
const { formatVersionBadge } = makeBadgeSandbox('80'); const { formatVersionBadge } = makeBadgeSandbox('80');
@@ -1348,7 +1348,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
assert.ok(!result.includes('v2.6.0'), 'staging should NOT show version'); assert.ok(!result.includes('v2.6.0'), 'staging should NOT show version');
assert.ok(result.includes('>abc1234</a>'), 'should show commit hash'); assert.ok(result.includes('>abc1234</a>'), 'should show commit hash');
assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit is linked'); assert.ok(result.includes(`href="${GH}/commit/abc1234def5678"`), 'commit is linked');
assert.ok(result.includes('[go]'), 'should show engine'); assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
}); });
test('staging port 81: hides version', () => { test('staging port 81: hides version', () => {
const { formatVersionBadge } = makeBadgeSandbox('81'); const { formatVersionBadge } = makeBadgeSandbox('81');
@@ -1369,18 +1369,18 @@ console.log('\n=== app.js: formatVersionBadge ===');
const result = formatVersionBadge('2.6.0', 'unknown', 'node'); const result = formatVersionBadge('2.6.0', 'unknown', 'node');
assert.ok(result.includes('>v2.6.0</a>'), 'should show version'); assert.ok(result.includes('>v2.6.0</a>'), 'should show version');
assert.ok(!result.includes('unknown'), 'should not show unknown commit'); assert.ok(!result.includes('unknown'), 'should not show unknown commit');
assert.ok(result.includes('[node]'), 'should show engine'); assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>node<'), 'should show engine name');
}); });
test('skips commit when missing', () => { test('skips commit when missing', () => {
const { formatVersionBadge } = makeBadgeSandbox(''); const { formatVersionBadge } = makeBadgeSandbox('');
const result = formatVersionBadge('2.6.0', null, 'go'); const result = formatVersionBadge('2.6.0', null, 'go');
assert.ok(result.includes('>v2.6.0</a>'), 'should show version'); assert.ok(result.includes('>v2.6.0</a>'), 'should show version');
assert.ok(result.includes('[go]'), 'should show engine'); assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
}); });
test('shows only engine when version/commit missing', () => { test('shows only engine when version/commit missing', () => {
const { formatVersionBadge } = makeBadgeSandbox('3000'); const { formatVersionBadge } = makeBadgeSandbox('3000');
const result = formatVersionBadge(null, null, 'go'); const result = formatVersionBadge(null, null, 'go');
assert.ok(result.includes('[go]'), 'should show engine'); assert.ok(result.includes('engine-badge'), 'should show engine badge'); assert.ok(result.includes('>go<'), 'should show engine name');
assert.ok(result.includes('version-badge'), 'should use version-badge class'); assert.ok(result.includes('version-badge'), 'should use version-badge class');
}); });
test('short commit not truncated in display', () => { test('short commit not truncated in display', () => {
@@ -1398,7 +1398,7 @@ console.log('\n=== app.js: formatVersionBadge ===');
const { formatVersionBadge } = makeBadgeSandbox('8080'); const { formatVersionBadge } = makeBadgeSandbox('8080');
const result = formatVersionBadge('2.6.0', null, 'go'); const result = formatVersionBadge('2.6.0', null, 'go');
assert.ok(!result.includes('2.6.0'), 'no version on staging'); assert.ok(!result.includes('2.6.0'), 'no version on staging');
assert.ok(result.includes('[go]'), 'engine shown'); assert.ok(result.includes('engine-badge'), 'engine badge shown'); assert.ok(result.includes('>go<'), 'engine name shown');
}); });
} }
+18
View File
@@ -1254,6 +1254,24 @@ seedTestData();
lastPathSeenMap.delete(liveNode); lastPathSeenMap.delete(liveNode);
}); });
// ── Cache hit rate includes stale hits ──
await t('Cache hitRate includes staleHits in formula', async () => {
cache.clear();
cache.hits = 0;
cache.misses = 0;
cache.staleHits = 0;
// Simulate: 3 hits, 2 stale hits, 5 misses => rate = (3+2)/(3+2+5) = 50%
cache.hits = 3;
cache.staleHits = 2;
cache.misses = 5;
const r = await request(app).get('/api/health').expect(200);
assert(r.body.cache.hitRate === 50, 'hitRate should be (hits+staleHits)/(hits+staleHits+misses) = 50%, got ' + r.body.cache.hitRate);
// Reset
cache.hits = 0;
cache.misses = 0;
cache.staleHits = 0;
});
// ── Summary ── // ── Summary ──
console.log(`\n═══ Server Route Tests: ${passed} passed, ${failed} failed ═══`); console.log(`\n═══ Server Route Tests: ${passed} passed, ${failed} failed ═══`);
if (failed > 0) process.exit(1); if (failed > 0) process.exit(1);
+228
View File
@@ -0,0 +1,228 @@
#!/usr/bin/env node
'use strict';
/**
* Seed synthetic test data into a running CoreScope server.
* Usage: node tools/seed-test-data.js [baseUrl]
* Default: http://localhost:13581
*/
const crypto = require('crypto');
const BASE = process.argv[2] || process.env.BASE_URL || 'http://localhost:13581';
const OBSERVERS = [
{ id: 'E2E-SJC-1', iata: 'SJC' },
{ id: 'E2E-SFO-2', iata: 'SFO' },
{ id: 'E2E-OAK-3', iata: 'OAK' },
];
const NODE_NAMES = [
'TestNode Alpha', 'TestNode Beta', 'TestNode Gamma', 'TestNode Delta',
'TestNode Epsilon', 'TestNode Zeta', 'TestNode Eta', 'TestNode Theta',
];
function rand(a, b) { return Math.random() * (b - a) + a; }
function randInt(a, b) { return Math.floor(rand(a, b + 1)); }
function pick(a) { return a[randInt(0, a.length - 1)]; }
function randomBytes(n) { return crypto.randomBytes(n); }
function pubkeyFor(name) { return crypto.createHash('sha256').update(name).digest(); }
function encodeHeader(routeType, payloadType, ver = 0) {
return (routeType & 0x03) | ((payloadType & 0x0F) << 2) | ((ver & 0x03) << 6);
}
function buildPath(hopCount, hashSize = 2) {
const pathByte = ((hashSize - 1) << 6) | (hopCount & 0x3F);
const hops = crypto.randomBytes(hashSize * hopCount);
return { pathByte, hops };
}
function buildAdvert(name, role) {
const pubKey = pubkeyFor(name);
const ts = Buffer.alloc(4); ts.writeUInt32LE(Math.floor(Date.now() / 1000));
const sig = randomBytes(64);
let flags = 0x80 | 0x10; // hasName + hasLocation
if (role === 'repeater') flags |= 0x02;
else if (role === 'room') flags |= 0x04;
else if (role === 'sensor') flags |= 0x08;
else flags |= 0x01;
const nameBuf = Buffer.from(name, 'utf8');
const appdata = Buffer.alloc(9 + nameBuf.length);
appdata[0] = flags;
appdata.writeInt32LE(Math.round(37.34 * 1e6), 1);
appdata.writeInt32LE(Math.round(-121.89 * 1e6), 5);
nameBuf.copy(appdata, 9);
const payload = Buffer.concat([pubKey, ts, sig, appdata]);
const header = encodeHeader(1, 0x04, 0); // FLOOD + ADVERT
const { pathByte, hops } = buildPath(randInt(0, 3));
return Buffer.concat([Buffer.from([header, pathByte]), hops, payload]);
}
function buildGrpTxt(channelHash = 0) {
const mac = randomBytes(2);
const enc = randomBytes(randInt(10, 40));
const payload = Buffer.concat([Buffer.from([channelHash]), mac, enc]);
const header = encodeHeader(1, 0x05, 0); // FLOOD + GRP_TXT
const { pathByte, hops } = buildPath(randInt(0, 3));
return Buffer.concat([Buffer.from([header, pathByte]), hops, payload]);
}
/**
* Build a properly encrypted GRP_TXT packet that decrypts to a CHAN message.
* Uses #LongFast channel key from channel-rainbow.json.
*/
function buildEncryptedGrpTxt(sender, message) {
try {
const CryptoJS = require('crypto-js');
const { ChannelCrypto } = require('@michaelhart/meshcore-decoder/dist/crypto/channel-crypto');
const channelKey = '2cc3d22840e086105ad73443da2cacb8'; // #LongFast
const text = `${sender}: ${message}`;
const buf = Buffer.alloc(5 + text.length + 1);
buf.writeUInt32LE(Math.floor(Date.now() / 1000), 0);
buf[4] = 0;
buf.write(text + '\0', 5, 'utf8');
const padded = Buffer.alloc(Math.ceil(buf.length / 16) * 16);
buf.copy(padded);
const keyWords = CryptoJS.enc.Hex.parse(channelKey);
const plaintextWords = CryptoJS.enc.Hex.parse(padded.toString('hex'));
const encrypted = CryptoJS.AES.encrypt(plaintextWords, keyWords, {
mode: CryptoJS.mode.ECB, padding: CryptoJS.pad.NoPadding
});
const cipherHex = encrypted.ciphertext.toString(CryptoJS.enc.Hex);
const channelSecret = Buffer.alloc(32);
Buffer.from(channelKey, 'hex').copy(channelSecret);
const mac = CryptoJS.HmacSHA256(
CryptoJS.enc.Hex.parse(cipherHex),
CryptoJS.enc.Hex.parse(channelSecret.toString('hex'))
);
const macHex = mac.toString(CryptoJS.enc.Hex).substring(0, 4);
const chHash = ChannelCrypto.calculateChannelHash('#LongFast');
const grpPayload = Buffer.from(
chHash.toString(16).padStart(2, '0') + macHex + cipherHex, 'hex'
);
const header = encodeHeader(1, 0x05, 0);
const { pathByte, hops } = buildPath(randInt(0, 2));
return Buffer.concat([Buffer.from([header, pathByte]), hops, grpPayload]);
} catch (e) {
// Fallback to unencrypted if crypto libs unavailable
return buildGrpTxt(0);
}
}
function buildAck() {
const payload = randomBytes(18);
const header = encodeHeader(2, 0x03, 0);
const { pathByte, hops } = buildPath(randInt(0, 2));
return Buffer.concat([Buffer.from([header, pathByte]), hops, payload]);
}
function buildTxtMsg() {
const payload = Buffer.concat([randomBytes(6), randomBytes(6), randomBytes(4), randomBytes(20)]);
const header = encodeHeader(2, 0x02, 0);
const { pathByte, hops } = buildPath(randInt(0, 2));
return Buffer.concat([Buffer.from([header, pathByte]), hops, payload]);
}
function computeContentHash(hex) {
return crypto.createHash('sha256').update(hex.toUpperCase()).digest('hex').substring(0, 16);
}
async function post(path, body) {
const r = await fetch(`${BASE}${path}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
return { status: r.status, data: await r.json() };
}
async function main() {
console.log(`Seeding test data into ${BASE}...`);
const packets = [];
// 1. ADVERTs for each node (creates nodes with location for map)
const roles = ['repeater', 'repeater', 'room', 'companion', 'repeater', 'companion', 'sensor', 'repeater'];
for (let i = 0; i < NODE_NAMES.length; i++) {
const obs = pick(OBSERVERS);
const hex = buildAdvert(NODE_NAMES[i], roles[i]).toString('hex').toUpperCase();
const hash = computeContentHash(hex);
packets.push({ hex, observer: obs.id, region: obs.iata, hash, snr: 5.0, rssi: -80 });
// Send same advert from multiple observers for compare page
for (const otherObs of OBSERVERS) {
if (otherObs.id !== obs.id) {
packets.push({ hex, observer: otherObs.id, region: otherObs.iata, hash, snr: rand(-2, 10), rssi: rand(-110, -60) });
}
}
}
// 2. Encrypted GRP_TXT packets (creates channel messages for channels page)
const chatMessages = [
['Alice', 'Hello everyone!'], ['Bob', 'Hey Alice!'], ['Charlie', 'Good morning'],
['Alice', 'How is the mesh today?'], ['Bob', 'Looking great, 8 nodes online'],
['Charlie', 'I just set up a new repeater'], ['Alice', 'Nice! Where is it?'],
['Bob', 'Signal looks strong from here'], ['Charlie', 'On top of the hill'],
['Alice', 'Perfect location!'],
];
for (const [sender, message] of chatMessages) {
const obs = pick(OBSERVERS);
const hex = buildEncryptedGrpTxt(sender, message).toString('hex').toUpperCase();
const hash = computeContentHash(hex);
packets.push({ hex, observer: obs.id, region: obs.iata, hash, snr: rand(-2, 10), rssi: rand(-110, -60) });
}
// 3. Unencrypted GRP_TXT packets (won't create channel entries but add packet variety)
for (let i = 0; i < 10; i++) {
const obs = pick(OBSERVERS);
const hex = buildGrpTxt(randInt(0, 3)).toString('hex').toUpperCase();
const hash = computeContentHash(hex);
packets.push({ hex, observer: obs.id, region: obs.iata, hash, snr: rand(-2, 10), rssi: rand(-110, -60) });
}
// 3. ACK packets
for (let i = 0; i < 15; i++) {
const obs = pick(OBSERVERS);
const hex = buildAck().toString('hex').toUpperCase();
const hash = computeContentHash(hex);
packets.push({ hex, observer: obs.id, region: obs.iata, hash, snr: rand(-2, 10), rssi: rand(-110, -60) });
}
// 4. TXT_MSG packets
for (let i = 0; i < 15; i++) {
const obs = pick(OBSERVERS);
const hex = buildTxtMsg().toString('hex').toUpperCase();
const hash = computeContentHash(hex);
packets.push({ hex, observer: obs.id, region: obs.iata, hash, snr: rand(-2, 10), rssi: rand(-110, -60) });
}
// 5. Extra packets with shared hashes (for trace/compare)
for (let t = 0; t < 5; t++) {
const hex = buildGrpTxt(0).toString('hex').toUpperCase();
const traceHash = computeContentHash(hex);
for (const obs of OBSERVERS) {
packets.push({ hex, observer: obs.id, region: obs.iata, hash: traceHash, snr: 5, rssi: -80 });
}
}
console.log(`Injecting ${packets.length} packets...`);
let ok = 0, fail = 0;
for (const pkt of packets) {
const r = await post('/api/packets', pkt);
if (r.status === 200) ok++;
else { fail++; if (fail <= 3) console.error(' Inject fail:', r.data); }
}
console.log(`Done: ${ok} ok, ${fail} fail`);
if (fail > 0) {
process.exit(1);
}
}
main().catch(err => { console.error(err); process.exit(1); });