## Bug
Master CI failing on `Map page loads with markers: No map
markers/overlays found` since #954 (observer filter) merged.
## Root cause chain
1. Fixture has 31 observers, all dated `2026-03-26` to `2026-03-29` (33+
days old)
2. PR #957's `tools/freshen-fixture.sh` shifts `nodes`, `transmissions`,
`neighbor_edges` timestamps but NOT `observers.last_seen`
3. Server startup runs `RemoveStaleObservers(14)` per
`cmd/server/main.go:382` — marks all 33-day-old observers `inactive=1`
4. PR #954's `GetObservers` filter then excludes them
5. `/api/observers` returns 0 → map has no observer markers → test
asserts >0 → fails
Server log line confirms: `[db] transmissions=499 observations=500
nodes=200 observers=0`
## Fix
Extend `freshen-fixture.sh` to also shift `observers.last_seen` (same
algorithm — preserve relative ordering, max anchored to now). Also
defensively clear any stale `inactive=1` flags from prior failed runs.
The `inactive` column may not exist on a fresh fixture (server adds via
migration); script silently no-ops if column absent.
## Verification
```
$ bash tools/freshen-fixture.sh /tmp/test.db
nodes: min=2026-05-01T11:07:29Z max=2026-05-01T18:49:02Z
observers: count=31 max=2026-05-01T18:49:02Z
```
After: 31 observers, oldest 3 days old, within the 14d retention window.
Server's startup prune won't touch them.
Co-authored-by: Kpa-clawbot <bot@example.invalid>
## Problem
The E2E fixture DB (`test-fixtures/e2e-fixture.db`) has static
timestamps from March 29, 2026. The map page applies a default
`lastHeard=30d` filter, so once the fixture ages past 30 days all nodes
are excluded from `/api/nodes?lastHeard=30d` — causing the "Map page
loads with markers" test to fail deterministically.
This started blocking all CI on ~April 28, 2026 (30 days after March
29).
Closes#955 (RCA #1: time-based fixture rot)
## Fix
Added `tools/freshen-fixture.sh` — a small script that shifts all
`last_seen`/`first_seen` timestamps forward so the newest is near
`now()`, preserving relative ordering between nodes. Runs in CI before
the Go server starts. Does **not** modify the checked-in fixture (no
binary blob churn).
## Verification
```
$ cp test-fixtures/e2e-fixture.db /tmp/fix4.db
$ bash tools/freshen-fixture.sh /tmp/fix4.db
Fixture timestamps freshened in /tmp/fix4.db
nodes: min=2026-05-01T07:10:00Z max=2026-05-01T14:51:33Z
$ ./corescope-server -port 13585 -db /tmp/fix4.db -public public &
$ curl -s "http://localhost:13585/api/nodes?limit=200&lastHeard=30d" | jq '{total, count: (.nodes | length)}'
{
"total": 200,
"count": 200
}
```
All 200 nodes returned with the 30-day filter after freshening (vs 0
without the fix).
Co-authored-by: you <you@example.com>
## Summary
- Fixes#912 — geofilter-builder generates out-of-range longitudes for
southern hemisphere locations
- Root cause: Leaflet's `latlng.lng` is unbounded; panning from Europe
to Australia produces values like `-210` instead of `150`
- Fix: call `latlng.wrap()` in `latLonPair()` to normalise longitude to
`[-180, 180]` before writing the config JSON
## Details
When the user opens the builder (default view: Europe, `[50.5, 4.4]`)
and pans east to Australia, Leaflet tracks the cumulative pan offset and
returns `lng = 150 - 360 = -210` to keep the path continuous. The
builder was passing that raw value straight into the output JSON,
producing coordinates that fall outside any valid bounding box.
`L.LatLng.wrap()` is Leaflet's built-in normalisation method — collapses
any longitude to `[-180, 180]` with no loss of precision.
## Test plan
- [x] Open the builder, navigate to NSW Australia, place a polygon —
confirm longitudes are `~141`–`154`, not `~-219`–`-206`
- [x] Repeat for a northern hemisphere location (e.g. Belgium) — confirm
output is unchanged
- [x] Paste the generated config into CoreScope — confirm nodes appear
on Maps and Live view
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
## Summary
Several features and fixes from a live deployment of the Go v3.0.0
backend.
### geo_filter — full enforcement
- **Go backend config** (`cmd/server/config.go`,
`cmd/ingestor/config.go`): added `GeoFilterConfig` struct so
`geo_filter.polygon` and `bufferKm` from `config.json` are parsed by
both the server and ingestor
- **Ingestor** (`cmd/ingestor/geo_filter.go`, `cmd/ingestor/main.go`):
ADVERT packets from nodes outside the configured polygon + buffer are
dropped *before* any DB write — no transmission, node, or observation
data is stored
- **Server API** (`cmd/server/geo_filter.go`, `cmd/server/routes.go`):
`GET /api/config/geo-filter` endpoint returns the polygon + bufferKm to
the frontend; `/api/nodes` responses filter out any out-of-area nodes
already in the DB
- **Frontend** (`public/map.js`, `public/live.js`): blue polygon overlay
(solid inner + dashed buffer zone) on Map and Live pages, toggled via
"Mesh live area" checkbox, state shared via localStorage
### Automatic DB pruning
- Add `retention.packetDays` to `config.json` to delete transmissions +
observations older than N days on a daily schedule (1 min after startup,
then every 24h). Nodes and observers are never pruned.
- `POST /api/admin/prune?days=N` for manual runs (requires `X-API-Key`
header if `apiKey` is set)
```json
"retention": {
"nodeDays": 7,
"packetDays": 30
}
```
### tools/geofilter-builder.html
Standalone HTML tool (no server needed) — open in browser, click to
place polygon points on a Leaflet map, set `bufferKm`, copy the
generated `geo_filter` JSON block into `config.json`.
### scripts/prune-nodes-outside-geo-filter.py
Utility script to clean existing out-of-area nodes from the database
(dry-run + confirm). Useful after first enabling geo_filter on a
populated DB.
### HB column in packets table
Shows the hop hash size in bytes (1–4) decoded from the path byte of
each packet's raw hex. Displayed as **HB** between Size and Type
columns, hidden on small screens.
## Test plan
- [x] ADVERT from node outside polygon is not stored (no new row in
nodes or transmissions)
- [x] `GET /api/config/geo-filter` returns polygon + bufferKm when
configured, `{polygon: null, bufferKm: 0}` when not
- [x] `/api/nodes` excludes nodes outside polygon even if present in DB
- [x] Map and Live pages show blue polygon overlay when configured;
checkbox toggles it
- [x] `retention.packetDays: 30` deletes old transmissions/observations
on startup and daily
- [x] `POST /api/admin/prune?days=30` returns `{deleted: N, days: 30}`
- [x] `tools/geofilter-builder.html` opens standalone, draws polygon,
copies valid JSON
- [x] HB column shows 1–4 for all packets in grouped and flat view
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add per-payload-type packet detail fixtures captured from production:
- packet-type-advert.json (payload_type=4, ADVERT)
- packet-type-grptxt-decrypted.json (payload_type=5, decrypted GRP_TXT)
- packet-type-grptxt-undecrypted.json (payload_type=5, decryption_failed GRP_TXT)
- packet-type-txtmsg.json (payload_type=1, TXT_MSG)
- packet-type-req.json (payload_type=0, REQ)
Update validate-protos.py to validate all 5 new fixtures against
PacketDetailResponse proto message.
Update CI deploy workflow to automatically capture per-type fixtures
on each deploy, including both decrypted and undecrypted GRP_TXT.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Automated script that compares all 13 major API endpoints between
Go staging (meshcore-staging-go) and Node prod (meshcore-prod)
containers. Uses python3 for JSON field diffing and reports
MATCH/PARTIAL/MISMATCH per endpoint.
Usage: scp to server then run, or pipe via ssh.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validates all 33 captured Node fixtures against the 10 .proto files.
Parses proto message definitions, maps each fixture to its response
message, and checks field presence, types, and structural shape.
Mismatches found (for Hicks to fix):
1. analytics-channels.json: ChannelAnalyticsSummary.hash is int in
fixture but proto says string
2. analytics-rf.json: PayloadTypeEntry.type is null in fixture but
proto says non-optional int32
3. bulk-health.json: API returns bare array with flat node fields;
proto nests them in BulkHealthEntry.node (structural mismatch)
4. node-analytics.json: activityTimeline uses 'bucket' key but
TimeBucket proto expects 'label'
5. observer-analytics.json: recentPackets are Observation-shaped
(have transmission_id) but proto says repeated Transmission
6. packet-detail.json: ByteRange has 'color' field not in proto
7. websocket-message.json: DecodedResult missing transportCodes,
raw fields; DecodedHeader missing routeTypeName; DecodedPayload
is flat (not oneof-wrapped); WSPacketData.packet is Observation-
shaped, not Transmission-shaped
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
npm test: all tests + coverage summary
npm run test:unit: fast unit tests only
npm run test:coverage: full suite + HTML report in coverage/
Baseline: 37% statements, 42% branches, 54% functions
Fixed e2e channels crash (undefined .length on null)
Kpa-clawbot