AdamandAdam c83499763d Allow mobile clients to manage repeater claims and details (#2429)
Draft for feedback: a working version of native repeater claims and
edits from mobile clients, built on the repeater info feature from
#2415. Four open questions are at the end.

## Situation

Repeater info (#2415) can only be claimed and edited from a browser
session. Device tokens from `POST /api/auth/device-token` are limited to
`rx` (CoreDrive RX: settings and companion linking), and `GET
/api/repeaters/{pubkey}/info` treats a bearer token as anonymous. A
mobile client can show a repeater's details but can't offer "claim" or
"edit" natively.

`PUT /api/repeaters/{pubkey}/info` is also a full replace with no
precondition. Two people editing from the same copy silently overwrite
each other. That's rare in the browser, but likelier from an app that
keeps a cached copy.

## Change

Three commits, one per piece.

**1. Device-token scopes for the repeater routes** (`8aba3715`)

- `POST /api/auth/device-token` takes an optional `scopes` list. Omitted
(or `null`) still means `["rx"]`, so CoreDrive RX and every existing
token keep exactly their current permissions. An empty list or an
unknown scope answers 400, checked before the password.
- New scopes:
- `repeater:claim`: `POST` and `DELETE /api/repeaters/{pubkey}/claim`.
These already only add or remove the caller's own claim. Several
claimers per repeater, idempotent `POST`, unchanged.
- `repeater:edit`: `PUT /api/repeaters/{pubkey}/info`, still subject to
`CanEditRepeater`.
- Every device token may use `/api/auth/me` and `/api/auth/logout`, so a
repeater-only token can check and end its own login.
- `GET /api/repeaters/{pubkey}/info` with a token holding either
repeater scope returns `canEdit` and `claimedByMe`. An unknown or
expired token answers 401 rather than passing as anonymous, so a client
learns its login is gone. A valid token without a repeater scope still
gets the public view. A cookie still wins over a bearer header, as
elsewhere.
- `GET /api/account/repeaters` (My repeaters, from #2430) also takes a
token holding either repeater scope, so an app gets the account's claims
in one request. It can't build that list from the public catalog, where
claimers are display names, and reading repeaters one at a time would
miss a claimed repeater that has left the node list. An `rx`-only token
still answers 403.
- Scope checks are method-aware (`bearerScopesFor(method, path)`). The
repeater paths are added to the bearer CORS paths.
- Same handlers, validation, rate limit (`repeater`, 120/h per user and
per IP), audit actions and admin claim removal as the browser. Token
writes add `via: device` to the `repeater.claim`, `repeater.edit` and
`repeater.unclaim` rows, and the `user.login` row records the granted
scopes. The admin route keeps refusing bearer tokens.

**2. Conditional edits with a revision** (`9f9bf47b`)

- `users.db` schema v10 adds `repeater_info.revision`. Existing rows
start at 1; a repeater with no saved details is at 0. Claims don't
change it.
- `GET` (single and list) and `PUT` responses include `revision`.
- `PUT` accepts an optional `baseRevision`. The check is part of the
write statement, so of two saves from the same base exactly one lands:
- `baseRevision: 0` (first save): `INSERT … ON CONFLICT(pubkey) DO
NOTHING`.
- `baseRevision: n`: `UPDATE … SET revision = revision + 1 … WHERE
pubkey = ? AND revision = ?`.
- No match answers `409` with the stored details
(`canEdit`/`claimedByMe` included) and writes nothing. That's the same
shape `PUT /api/account/settings` uses for its conflicts.
  - A stale base is refused even when the body would change nothing.
- Permission is checked first, so a 409 never shows details to someone
who can't edit.
- Without `baseRevision` the save is unconditional and bumps the
revision, so `public/repeater-info.js` works unchanged.
- `/api/config/client` adds `userManagement.repeaterTokens: true`, so a
client can tell this server from one without the scopes or
`baseRevision` (older servers answer the device-token request with the
SPA page, as noted for `companionLinking`).

**Discovery and conflict recovery, for a client:**

1. Check `userManagement.repeaterTokens` in `/api/config/client`.
Without it, keep the claim and edit actions hidden.
2. Log in with `scopes: ["repeater:claim", "repeater:edit"]` (or just
what the client offers). The response echoes the granted `scopes`.
3. Read `GET /api/repeaters/{pubkey}/info` with the token. Show
Claim/Unclaim from `claimedByMe` and Edit from `canEdit`, and keep
`revision`.
4. Save with `PUT …/info` and `baseRevision` set to that revision. On
200, keep the new `revision` from the response. On 409, the body is the
current record: show or merge it, then retry with its `revision`. On
401, log in again. On 403, the account can no longer edit (for example
someone claimed it), so refresh from `canEdit`.

**3. Account page device list** (`d28374aa`)

My account, Devices labelled every device token "CoreDrive RX".
`/api/account/sessions` now includes a device token's `scopes`. The page
keeps the CoreDrive RX label for `rx` tokens (and rows without scopes).
It shows other tokens under the device name the app sent, with a line
such as "can claim repeaters and edit repeater details".

Docs: `openapi.go`, `docs/api-spec.md` and the Repeater info section of
`docs/user-guide/accounts.md`.

## Questions

1. **One scope or two?** This uses separate `repeater:claim` and
`repeater:edit`, so a client can ask for less, and a token that only
edits can't put the user's name on a repeater publicly. One `repeater`
scope would be simpler if you'd rather not split it.
2. **Editing unclaimed repeaters from mobile.** This keeps web parity:
any active user may edit a repeater nobody has claimed, and once it has
claimers only they (and admins) can. The alternative is to require a
claim before a token may edit. That's stricter, but then app and browser
rules differ.
3. **Operator switch.** There's no new setting: the scopes are available
whenever `userManagement` is on, and the user opts in when the app asks
for them at login. Would you want an explicit opt-in such as
`userManagement.mobileRepeaterWrites`?
4. **Stale-edit contract.** Does `revision` plus `baseRevision` in the
body, with 409 returning the current record, fit? I matched account
settings rather than `ETag`/`If-Match`, to keep one pattern in the API.
The browser editor doesn't send `baseRevision` yet. Making it send one
would be a small follow-up, if you want browser saves protected too.

## Tests

- `cmd/server/repeater_device_token_test.go`:
- default and requested scopes, rejected scopes, and the method-aware
route table;
- per-scope allowed and denied routes, logout with a repeater-only
token;
  - claim and unclaim of own claim only, alongside another claimer;
- edit permission (unclaimed repeater, claimer, non-claimer, admin) and
a disabled account;
  - the admin route refusing tokens;
  - authenticated, public and expired-token reads;
- `via: device` audit rows, cookie writes still needing CSRF, and
session scopes.
- My repeaters with a claim or edit token (own claims only), and 403 for
an `rx`-only token.
- `cmd/server/repeater_revision_test.go`:
- revision on reads, a first edit with no saved row, and a stale base
returning 409 with the current record and no audit row;
  - unconditional browser saves, validation and permission order;
- two concurrent saves from the same revision (one 200, one 409), and
the client config flag.
- `internal/users/repeater_revision_test.go`:
  - revision counting and conditional first and later saves;
  - an 8-way race for both cases (`-race`);
  - the v9 to v10 upgrade.
- `tests/unit/test-user-management-ui.js`: app token labels and scope
line.

Run locally:

- `make test`: all 16 modules OK.
- `sh test-all.sh`: all suites pass, run in `node:24-bookworm`. On
macOS, `test-manage-workflow.js` fails 2 cases on unchanged master
because of the BSD tools.
- `npx eslint public/*.js`: 0 errors, the same 87 warnings as master.
- `node tests/e2e/test-lint-network-calls.js`
- `tests/e2e/test-user-management-e2e.js` against local e2etest servers
with the CI config: 44/44.
- Manually against a local instance:
  - in the browser: claim, edit, unclaim, and the Devices list;
  - with curl: the token flow, including a 409 after a browser save.

Not run: the full Playwright shard matrix from CI.

---------

Co-authored-by: Adam <196335+anieto@users.noreply.github.com>
2026-11-21 22:27:48 +01:00
2026-04-05 06:36:03 +00:00
2026-03-20 05:38:23 +00:00

CoreScope

Go Server Coverage Go Ingestor Coverage E2E Tests Frontend Coverage Deploy

High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption.

Self-hosted, open-source MeshCore packet analyzer. Collects MeshCore packets via MQTT, decodes them in real time, and presents a full web UI with live packet feed, interactive maps, channel chat, packet tracing, and per-node analytics.

⚡ Performance

The Go backend serves its packet and analytics API from an in-memory packet store with 7 indexes (hash, txID, obsID, observer, node, path hop, payload type). SQLite is for persistence only — reads never touch disk.

Metric Value
Packet queries < 1 ms (in-memory)
All API endpoints < 100 ms
Memory (56K packets) ~300 MB (vs 1.3 GB on Node.js)
WebSocket broadcast Real-time to all connected browsers
Channel decryption AES-128-ECB with rainbow table
GOMEMLIMIT (memory-constrained hosts) set to ≥1.5× working set (e.g. 1536 MiB on a 2 GB Pi for a ~1 GB store). Lower values trigger a GC death-spiral. Configure via the GOMEMLIMIT env var or runtime.maxMemoryMB in config.json; env wins. Applies to both server and ingestor. See #1010.

See PERFORMANCE.md for full benchmarks.

✨ Features

📡 Live Trace Map

Real-time animated map with packet route visualization, VCR-style playback controls, and a retro LCD clock. Replay the last 24 hours of mesh activity, scrub through the timeline, or watch packets flow live at up to 8× speed.

Live VCR playback — watch packets flow across the Bay Area mesh

📦 Packet Feed

Filterable real-time packet stream with byte-level breakdown, Excel-like resizable columns, and a detail pane. Toggle "My Nodes" to focus on your mesh.

Packets view

🗺️ Network Overview

At-a-glance mesh stats — node counts, packet volume, observer coverage.

Network overview

📊 Node Analytics

Per-node deep dive with interactive charts: activity timeline, packet type breakdown, SNR distribution, hop count analysis, peer network graph, and hourly heatmap.

Node analytics

💬 Channel Chat

Decoded group messages with sender names, @mentions, timestamps — like reading a Discord channel for your mesh.

Channels

📱 Mobile Ready

Full experience on your phone — proper touch controls, iOS safe area support, and a compact VCR bar.

Live view on iOS

And More

  • 17 Analytics Tabs — RF, topology, channels, hash stats, distance, route patterns, scopes, and more
  • Node Directory — searchable list with role tabs, detail panel, QR codes, advert timeline
  • Packet Tracing — follow individual packets across observers with SNR/RSSI timeline
  • Observer Status — health monitoring, packet counts, uptime, per-observer analytics
  • Hash Collision Matrix — detect address collisions across the mesh
  • Scope Audit: per repeater, which declared regions show no forwarding and which scopes it forwards without declaring them
  • Channel Key Auto-Derivation — hashtag channels (#channel) keys derived via SHA256
  • Multi-Broker MQTT — connect to multiple brokers with per-source IATA filtering
  • Dark / Light Mode — auto-detects system preference, map tiles swap too
  • Theme Customizer — design your theme in-browser, export as theme.json
  • Global Search — search packets, nodes, and channels (Ctrl+K)
  • Shareable URLs — deep links to packets, channels, and observer detail pages
  • Protobuf API Contract — typed API definitions in proto/
  • Accessible — ARIA patterns, keyboard navigation, screen reader support

Optional Features

Off by default, enabled in config.json:

  • User Accounts: registration with email activation, settings sync, an admin area, hashtag channel proposals, mail notifications for watched nodes, and a personal data export. The dashboard stays public. See docs/user-guide/accounts.md
  • Mobile RX Coverage: crowdsourced coverage from phones connected to a companion radio, drawn as hex bins on the RX Coverage page and on each node's Reach page. See docs/client-rx-coverage.md

Quick Start

No build step required — just run:

docker run -d --name corescope \
  --restart=unless-stopped \
  -p 80:80 -p 1883:1883 \
  -v /your/data:/app/data \
  ghcr.io/kpa-clawbot/corescope:latest

Open http://localhost — done. No config file needed; CoreScope starts with sensible defaults.

For HTTPS with a custom domain, add -p 443:443 and mount your Caddyfile:

docker run -d --name corescope \
  --restart=unless-stopped \
  -p 80:80 -p 443:443 -p 1883:1883 \
  -v /your/data:/app/data \
  -v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
  -v /your/caddy-data:/data/caddy \
  ghcr.io/kpa-clawbot/corescope:latest

Disable built-in services with -e DISABLE_MOSQUITTO=true or -e DISABLE_CADDY=true, or drop a .env file in your data volume. See docs/deployment.md for the full reference.

Build from Source

git clone https://github.com/Kpa-clawbot/CoreScope.git
cd CoreScope
./manage.sh setup

The setup wizard walks you through config, domain, HTTPS, build, and run.

./manage.sh status       # Health check + packet/node counts
./manage.sh logs         # Follow logs
./manage.sh backup       # Backup database
./manage.sh update       # Pull latest + rebuild + restart
./manage.sh mqtt-test    # Check if observer data is flowing
./manage.sh help         # All commands

Configure

Copy config.example.json to config.json and edit:

{
  "port": 3000,
  "mqtt": {
    "broker": "mqtt://localhost:1883",
    "topic": "meshcore/+/+/packets"
  },
  "mqttSources": [
    {
      "name": "remote-feed",
      "broker": "mqtts://remote-broker:8883",
      "topics": ["meshcore/+/+/packets"],
      "username": "user",
      "password": "pass",
      "iataFilter": ["SJC", "SFO", "OAK"]
    }
  ],
  "channelKeys": {
    "public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
  },
  "defaultRegion": "SJC"
}
Field Description
port HTTP server port (default: 3000)
mqtt.broker Local MQTT broker URL ("" to disable)
mqttSources External MQTT broker connections (optional)
channelKeys Channel decryption keys (hex). Hashtag channels auto-derived via SHA256
defaultRegion Default IATA region code for the UI
dbPath SQLite database path (default: data/meshcore.db)

Environment Variables

Variable Description
DB_PATH Override SQLite database path
GOMEMLIMIT Go soft memory limit, wins over runtime.maxMemoryMB

The HTTP port is set with port in config.json or the server's -port flag.

Architecture

                           ┌─────────────────────────────────────────────┐
                           │              Docker Container               │
                           │                                             │
Observer → USB →           │  Mosquitto ──→ Go Ingestor ──→ SQLite DB   │
  meshcoretomqtt → MQTT ──→│                    │                        │
                           │              Go HTTP Server ──→ WebSocket   │
                           │                    │               │        │
                           │              Caddy (HTTPS) ←───────┘        │
                           └────────────────────┼────────────────────────┘
                                                │
                                             Browser

Two-process model: The Go ingestor handles MQTT ingestion and packet decoding. The Go HTTP server loads all packets into an in-memory store on startup (5 indexes for fast lookups) and serves the REST API + WebSocket broadcast. Both are managed by supervisord inside a single container with Caddy for HTTPS and Mosquitto for local MQTT.

MQTT Setup

  1. Flash an observer node with MESH_PACKET_LOGGING=1 build flag
  2. Connect via USB to a host running meshcoretomqtt
  3. Configure meshcoretomqtt with your IATA region code and MQTT broker address
  4. Packets appear on topic meshcore/{IATA}/{PUBKEY}/packets

To decode a single raw hex packet without storing it, use POST /api/decode.

Project Structure

corescope/
├── cmd/
│   ├── server/              # Go HTTP server + WebSocket + REST API
│   │   ├── main.go          # Entry point
│   │   ├── routes.go        # 40+ API endpoint handlers
│   │   ├── store.go         # In-memory packet store (5 indexes)
│   │   ├── db.go            # SQLite persistence layer
│   │   ├── decoder.go       # MeshCore packet decoder
│   │   ├── websocket.go     # WebSocket broadcast
│   │   └── *_test.go        # Server tests
│   └── ingestor/            # Go MQTT ingestor
│       ├── main.go          # MQTT subscription + packet processing
│       ├── decoder.go       # Packet decoder (shared logic)
│       ├── db.go            # SQLite write path
│       └── *_test.go        # Ingestor tests
├── proto/                   # Protobuf API definitions
├── public/                  # Vanilla JS frontend (no build step)
│   ├── index.html           # SPA shell
│   ├── app.js               # Router, WebSocket, utilities
│   ├── packets.js           # Packet feed + hex breakdown
│   ├── map.js               # Leaflet map + route visualization
│   ├── live.js              # Live trace + VCR playback
│   ├── channels.js          # Channel chat
│   ├── nodes.js             # Node directory + detail views
│   ├── analytics.js         # Analytics dashboard
│   └── style.css            # CSS variable theming (light/dark)
├── docker/
│   ├── supervisord-go.conf  # Process manager (server + ingestor)
│   ├── mosquitto.conf       # MQTT broker config
│   ├── Caddyfile            # Reverse proxy + HTTPS
│   └── entrypoint-go.sh     # Container entrypoint
├── Dockerfile               # Multi-stage Go build + Alpine runtime
├── config.example.json      # Example configuration
├── tests/                   # Node.js test suite: unit/ (test-all.sh) and e2e/ (Playwright)
├── test-all.sh              # Runs every suite in tests/unit
└── tools/                   # Generators, E2E tests, utilities

For Developers

Building

make build        # all four binaries for your machine, into dist/
make build-server # just one
make crossbuild   # static linux/amd64 + linux/arm64 binaries

The SQLite driver is mattn/go-sqlite3, which is cgo, so a plain GOOS=linux go build from a Mac will not work: cross-compiling needs a C compiler that can target the other platform. make crossbuild uses zig as that compiler (install it and it just works) and links statically against musl, so the result is one self-contained file that runs on Alpine or scratch. The container build does the same thing — see Dockerfile.

Test Suite

Go tests cover the backend, Node.js unit tests in tests/unit/ cover the frontend logic, and Playwright tests in tests/e2e/ validate the UI in a browser.

# Go backend tests
cd cmd/server && go test ./... -v
cd cmd/ingestor && go test ./... -v

# Or across all 16 modules at once
make test

# Node.js frontend + integration tests
npm test

# Playwright E2E (requires running server on localhost:3000)
node tests/e2e/test-e2e-playwright.js

Generate Test Data

node tools/generate-packets.js --api --count 200

Migrating from Node.js

If you're running an existing Node.js deployment, see docs/go-migration.md for a step-by-step guide. The Go engine reads the same SQLite database and config.json — no data migration needed.

Contributing

Contributions welcome. Please read AGENTS.md for coding conventions, testing requirements, and engineering principles before submitting a PR.

Live instance: analyzer.on8ar.eu: all read endpoints are public, no auth required.

API Documentation: CoreScope auto-generates an OpenAPI 3.0 spec. Browse the interactive Swagger UI at /api/docs or fetch the machine-readable spec at /api/spec.

License

GPL-3.0-or-later

S
Description
No description provided
Readme GPL-3.0
138 MiB
0 Stars 1 Watchers 0 Forks
Languages
JavaScript 49.1%
Go 45.7%
CSS 2.8%
Shell 1.5%
HTML 0.5%
Other 0.2%