MrAlders0n 16fb706d17 Packet sites: a read model that serves region packet pages while trusted (#247)
* feat(db): packet_sites table and indexes

* feat(ingest): keep each packet's site row in the observation insert

* feat(db): packet_sites backfill, reconcile and audit

* feat(db): region packet pages from packet_sites when trusted, with an exact page cursor

* feat(db): packet_sites front sweeper with plan and liveness metrics

* feat: boot repair, reconcile, audit, sweep and first-time verification for packet_sites

* fix(db): read no site range when no packet was first heard in the window

* fix(db): read the window guard's maxima through indexes

* test(planguard): packet_sites statements active, data-derived cases, diff and storediff

* docs(api): pageCursor applies to region packet pages

* fix(db): drop packet_sites trust again after the unsited fallback write

* fix(db): the packet_sites audit refills a pair at its newest hearing

* fix(db): bounded region reads fill the page within the request, else today's statement answers

* fix(planguard): tie_cut needs both pages at the limit and members of other identities

* fix: packet_sites sweep without JIT, one verification VACUUM, read-only checks to the newest id

* docs(api): cursor and pageCursor on region pages served without nextPageCursor

* test: drop local baselines of entries the copy skips; plain race schedule names

* test(planguard): storediff walks of the bounded region shapes; local baselines after the overlay rebuild

* fix(db): drop message-site trust again after the unseeded insert

* fix(db): a region page cursor's fallback to today's statement continues from its exact position
2026-10-20 00:18:33 -04:00
2026-07-24 12:59:48 -07:00
2026-06-12 11:15:01 -07:00
2026-10-05 18:35:41 -04:00
2026-06-08 15:46:01 -07:00

MeshCore Beacon

MeshCore Beacon is a MeshCore network observation backend. It connects to one or more MeshCore MQTT brokers, ingests LoRa packet traffic in real time, stores it in PostgreSQL, and streams live events to WebSocket clients.

CI CodeQL Coverage Docker

What it does

  • Subscribes to MeshCore MQTT brokers and decodes incoming LoRa packets using meshcore-go
  • Stores packets, observations, nodes, observers, traces, routes and channel messages in PostgreSQL
  • Deduplicates observations across multiple brokers (the same packet heard by two brokers is one observation per observer)
  • Decrypts group text messages for known channel keys
  • Detects firmware capability flags from path hash sizes
  • Streams live events to WebSocket clients with subscription filtering by IATA, region, payload type, and event type
  • Serves a REST API for querying stored data
  • Seeds regions, IATA display names, and channel keys from a YAML config file on startup

This repo is the code. Deploying, configuring and operating Beacon is documented in beacon-docs; see Documentation below.

Stack

Component Technology
Language Go 1.26
Router Chi v5
Database PostgreSQL 16
Caching Redis 7
DB queries sqlc + pgx/v5
MQTT paho.mqtt.golang
WebSocket coder/websocket
Packet decode meshcore-go
Config YAML via gopkg.in/yaml.v3
Env godotenv

Running it locally

You need Go 1.26+ and a PostgreSQL 16 database. Docker is the easy way to get the database.

git clone https://github.com/MeshCore-Beacon/beacon-server.git && cd beacon-server
cp env.example .env
cp config.yaml.example config.yaml
docker run -d --name beacon-postgres -p 5432:5432 \
  -e POSTGRES_USER=beacon -e POSTGRES_PASSWORD=beacon -e POSTGRES_DB=beacon postgres:16-alpine
go run ./cmd/beacon

Fill in POSTGRES_DSN and your MQTT broker credentials in .env. Every variable is described in Configuration, and the fully annotated config.yaml lives in beacon-docs as well; the copy here is a working starter. Migrations run on startup. The API listens on LISTEN_ADDR (default :8080) and Swagger is at http://localhost:8080/swagger/index.html.

MeshMapper authentication

MeshMapper APIs require either a regional API key or a grouped-region API key covering multiple regions. Your local MeshMapper regional or grouped-region admin can generate these keys.

Set MESHMAPPER_API_KEY in each deployment's private environment or gitignored .env; it overrides meshmapper.api_key in config.yaml, including when empty. The examples leave the key empty. Never put it in browser settings or use the mobile app's App key. Restart Beacon after changing the key.

A regional deployment needs access to its IATA; a group deployment needs access to every member IATA, including members added by import_groups. API keys do not remove rate limits or change the get_zones.php?country=... request shape. IP exemptions are not authentication.

Beacon sends X-API-Key to get_zones.php, get_geojson.php, get_scopes.php, get_channels.php, and get_repeaters.php. Requests are restricted to those HTTPS MeshMapper endpoints and never follow redirects. Coverage consumers keep their separate keys, scopes, quotas, and supported ?key= authentication; this client does not call coverage.php.

Missing keys are reported as unconfigured without stopping Beacon or deleting cached imports. Failed refreshes record last_error, retain the last successful data and freshness timestamp, and report 401 as an authentication failure or 403 as a permission failure. Inspect the meshmapper.zones, meshmapper.scopes, meshmapper.channels, and meshmapper.repeaters logs for freshness and the next attempt. No anonymous fallback occurs. Existing refresh limits still apply, including longer Retry-After delays on 429/503.

Use the environment setting when backups are enabled. Backup exports preserve saved YAML verbatim and refuse a nonempty meshmapper.api_key in that file so the credential cannot enter a backup download. Clear that YAML value after moving the key to the environment; environment secrets are excluded from backups.

Deployment coordination is required with MeshMapper Server before enforcement: confirm X-API-Key support, obtain and verify the integration key's API and IATA permissions, and confirm these requests do not consume Coverage quota. Header support must not be assumed deployed. The first four endpoints can enforce authentication before the app rollout; repeater enforcement waits for app 1.4.1 and its forced update. Key issuance, permissions, quotas, and enforcement switches remain MeshMapper Server work, tracked in MeshMapper_Server#431.

To run the web frontend against it, see Running the full stack locally.

The published image is ghcr.io/meshcore-beacon/beacon-server. latest tracks stable releases, dev follows the development branch, and each release is also tagged X.Y.Z and X.Y.

What you see on an empty database

Path resolution, capability detection and known routes depend on nodes having advertised to a local observer. On a fresh database every hop shows "confidence": "none" and supportsMultibytePaths is false until adverts populate node_short_ids. That is expected and fills in as the mesh is observed.

GRP_TXT packets whose channel key is not known yet are stored hash-only. After adding the key to config.yaml, the next restart decrypts the matching history; look for config: backfilled N previously-undecrypted channel message(s) in the log.

Documentation

In beacon-docs:

In this repo: Historical stats explains the hourly rollups behind the stats endpoints, for anyone changing them. docs/swagger.yaml is the generated OpenAPI description.

Contributing

CONTRIBUTING.md covers code style, tests, and database and API changes. Branches, commits and the release flow are shared across the Beacon repos and live in beacon-docs/CONTRIBUTING.md. Please read the Code of Conduct. Security reports go through SECURITY.md.

Acknowledgements

See CONTRIBUTORS.md for the people who have helped build Beacon, and SHOULDERS.md for the open source projects it stands on.

Languages
Go 98.7%
PLpgSQL 1.2%