docs(meshmapper): document API key setup (#14)

2026-10-23 00:41:24 +00:00 · 2026-10-04 23:22:36 -04:00
1 parent 5a60f1e00b
commit ccf67b1eea
@@ -71,6 +71,11 @@ Everything else in the file is optional and explained in
 [Configuration](docs/configuration.md): the admin API key, log settings, Redis, and the
 web app's map view, tabs, themes, name and banner.

+If you enable MeshMapper imports, also set `MESHMAPPER_API_KEY`. Your local MeshMapper
+regional or grouped-region admin can generate a regional key or a key covering multiple
+regions. See [MeshMapper API key](docs/configuration.md#meshmapper-api-key) for setup and
+the server rollout requirements. Keep this key out of `VITE_*` settings.
+
 ### 3. Describe your network in `config.yaml`

 ```bash
@@ -15,6 +15,13 @@ LISTEN_ADDR=:8080
 # At least 16 characters, no whitespace; e.g. `openssl rand -hex 32`.
 # BEACON_API_KEY=

+# MeshMapper regional or grouped-region API key for enabled imports.
+# Your local MeshMapper regional or grouped-region admin can generate the key;
+# grouped-region keys cover multiple regions. Never use the mobile App key.
+# Overrides meshmapper.api_key, even when empty. Keep this out of VITE_* settings.
+# Use the environment setting with backups; saved YAML keys prevent exports.
+# MESHMAPPER_API_KEY=
+
 # Override log.level / log.format from config.yaml (defaults: info / text).
 # LOG_LEVEL=info
 # LOG_FORMAT=json
@@ -45,7 +45,14 @@ ratelimit:
  # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds).
  burst: 300

-# MeshMapper integrations (optional, no API key). Import transport scopes and
+# MeshMapper integrations (optional, require a regional or grouped-region API key).
+# Your local MeshMapper regional or grouped-region admin can generate the key.
+# A grouped-region key covers multiple regions, including imported group members.
+# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the
+# environment for secrets, especially with backups; never use the mobile App key.
+# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain.
+# 401/403 failures retain cached data and report authentication/permission errors.
+# Import transport scopes and
 # IATA border outlines from MeshMapper instead of maintaining scopes: and
 # iatas.*.borderFile by hand.
 #
@@ -63,6 +70,7 @@ ratelimit:
 # Region-filtered scope lists and scope stats show imported names only under the
 # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only.
 #meshmapper:
+#  api_key: ""
 #  scopes:
 #    enabled: false
 #    refresh_interval: 1h  # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick
@@ -46,7 +46,14 @@ ratelimit:
  # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds).
  burst: 300

-# MeshMapper integrations (optional, no API key). Import transport scopes and
+# MeshMapper integrations (optional, require a regional or grouped-region API key).
+# Your local MeshMapper regional or grouped-region admin can generate the key.
+# A grouped-region key covers multiple regions, including imported group members.
+# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the
+# environment for secrets, especially with backups; never use the mobile App key.
+# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain.
+# 401/403 failures retain cached data and report authentication/permission errors.
+# Import transport scopes and
 # IATA border outlines from MeshMapper instead of maintaining scopes: and
 # iatas.*.borderFile by hand.
 #
@@ -64,6 +71,7 @@ ratelimit:
 # Region-filtered scope lists and scope stats show imported names only under the
 # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only.
 #meshmapper:
+#  api_key: ""
 #  scopes:
 #    enabled: false
 #    refresh_interval: 1h  # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick
@@ -45,6 +45,11 @@ docker-deployment-type2/
   Caddy redirects anything that is not `/api/*` or `/ws` to that host. The password inside
   `POSTGRES_DSN` must match `POSTGRES_PASSWORD` in `docker-compose.yml`.

+   For MeshMapper imports, set `MESHMAPPER_API_KEY` in this server host's `.env`.
+   Your local MeshMapper regional or grouped-region admin can generate the regional or
+   grouped-region key. Do not copy it to the web host or a `VITE_*` setting. See
+   [MeshMapper API key](../docs/configuration.md#meshmapper-api-key).
+
 3. Edit `data/app/config.yaml` for your network (see
   [Configuration](../docs/configuration.md#configyaml)). Because the web app will be served
   from another origin, browsers may only open the WebSocket if that origin is listed:
@@ -46,7 +46,14 @@ ratelimit:
  # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds).
  burst: 300

-# MeshMapper integrations (optional, no API key). Import transport scopes and
+# MeshMapper integrations (optional, require a regional or grouped-region API key).
+# Your local MeshMapper regional or grouped-region admin can generate the key.
+# A grouped-region key covers multiple regions, including imported group members.
+# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the
+# environment for secrets, especially with backups; never use the mobile App key.
+# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain.
+# 401/403 failures retain cached data and report authentication/permission errors.
+# Import transport scopes and
 # IATA border outlines from MeshMapper instead of maintaining scopes: and
 # iatas.*.borderFile by hand.
 #
@@ -64,6 +71,7 @@ ratelimit:
 # Region-filtered scope lists and scope stats show imported names only under the
 # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only.
 #meshmapper:
+#  api_key: ""
 #  scopes:
 #    enabled: false
 #    refresh_interval: 1h  # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick
@@ -22,6 +22,10 @@ changes, PostgreSQL roles and cluster settings, Redis and deployment files are n

 ## Requirements and limits

+- Keep the MeshMapper API key in `MESHMAPPER_API_KEY`, with `meshmapper.api_key` empty or
+  omitted in the saved YAML. Both CLI exports and admin downloads refuse a nonempty saved
+  MeshMapper key because the YAML is included verbatim. Environment secrets remain excluded;
+  preserve them separately for recovery. See [MeshMapper API key](configuration.md#meshmapper-api-key).
 - `pg_dump` must be in the **same runtime** as the exporter (the CLI's environment, or the
  server's PATH for the download), with a major version at least as new as the database's. A client on
  the Docker host or in another container doesn't count.
@@ -23,6 +23,7 @@ to open).
 | `MQTT_BROKER_1_PASSWORD` | unset | Subscriber password for broker 1. |
 | `MQTT_BROKER_2_URL`, `_USERNAME`, `_PASSWORD` | unset | The same for an optional second broker. A packet heard through both brokers is stored once. |
 | `BEACON_API_KEY` | unset | Bearer key for `/api/v1/admin/*`. When set, even to an empty string, it replaces `auth.api_key` from the config file. See [Admin API key](#admin-api-key). |
+| `MESHMAPPER_API_KEY` | unset | Regional or grouped-region key for enabled MeshMapper imports. When set, even to an empty string, it replaces `meshmapper.api_key`. Missing keys leave imports unconfigured while Beacon keeps running. See [MeshMapper API key](#meshmapper-api-key). |
 | `LOG_LEVEL` | `info` | `debug`, `info`, `warn` or `error`. Overrides `log.level`. Anything else stops startup. |
 | `LOG_FORMAT` | `text` | `text` or `json`. Overrides `log.format`. Anything else stops startup. |
 | `BEACON_CPU_PROFILE_DIR`, `BEACON_CPU_PROFILE_UNTIL` | unset | Enable a bounded CPU capture in production. See [Profiling](profiling.md). |
@@ -68,7 +69,7 @@ which ones to read.
 | `log` | Log level and format. | You want JSON logs for a collector. |
 | `server` | `trusted_proxies`: the proxy addresses allowed to tell Beacon the real client IP. | Always, if Beacon sits behind a proxy. See [Reverse proxy](reverse-proxy.md). |
 | `ratelimit` | Per-client REST limits for `/api/v1/*`. On by default at 300 requests a minute. | You get 429s you did not expect. |
-| `meshmapper` | Import transport scopes, IATA borders, region groups and public channels from MeshMapper instead of listing them by hand. | You are in a region MeshMapper covers. |
+| `meshmapper` | Import transport scopes, IATA borders, region groups and public channels from MeshMapper instead of listing them by hand. Requires a [MeshMapper API key](#meshmapper-api-key). | You are in a region MeshMapper covers. |
 | `iatas` | Display names, coordinates and optional border files for airport codes. IATAs are created automatically when traffic arrives; this block only decorates them. A `borderFile` path is relative to `config.yaml`; in Docker the compose file mounts only `config.yaml`, so add a mount for the folder too (`- ./data/app/borders:/app/borders:ro` under the `app` service). | You want names on the map or a border drawn. |
 | `regions` | Groups of IATAs with a name, map centre and zoom. | Always, unless MeshMapper imports them. |
 | `channel_keys` | Hashtag channels and explicit keys for decrypting group messages. | Always. Without keys, channel messages are stored as hashes only. |
@@ -106,6 +107,58 @@ cannot honour. These are the ones people hit:
  A negative `websocket.max_connections_per_ip` is not caught at startup and rejects every
  connection instead, so leave it at a positive number.

+## MeshMapper API key
+
+MeshMapper APIs require either a regional API key or a grouped-region API key covering
+multiple regions. Your local MeshMapper regional or grouped-region admin can generate
+these keys. Request a key for the APIs Beacon uses, covering your IATA or every member IATA
+in your group, including members added by `meshmapper.zones.import_groups`.
+
+Set `MESHMAPPER_API_KEY` in the deployment's private `.env` or backend environment. Each
+Beacon deployment can use a different key. It overrides `meshmapper.api_key` in
+`config.yaml`, even when the environment value is empty. Both examples leave the key empty.
+Do not use the mobile app's App key, `BEACON_API_KEY`, or a Coverage API key for this setting.
+Never put it in a `VITE_*` value, browser configuration, logs, or git.
+
+After changing `.env`, recreate the backend container from the deployment folder
+(`docker-deployment-type2/server` for the split deployment):
+
+```bash
+docker compose up -d --force-recreate app
+```
+
+A plain `docker compose restart app` does not load changed container environment variables.
+If you use the YAML setting instead, restarting Beacon is enough. Prefer the environment
+setting when using [backups](backup-export.md): exports refuse a nonempty
+`meshmapper.api_key` in the saved YAML so the key cannot enter a download.
+
+Beacon sends `X-API-Key` to `get_zones.php`, `get_geojson.php`, `get_scopes.php`, and
+`get_channels.php`. Its shared client supports the same header for `get_repeaters.php`,
+although Beacon does not currently fetch repeaters. Requests use trusted HTTPS MeshMapper
+endpoints and do not follow redirects. `get_zones.php` still needs its `country` parameter;
+a key covering multiple regions does not change the request shape or remove rate limits.
+IP exemptions are not authentication.
+
+With no key, MeshMapper imports report unconfigured and make no requests. Beacon keeps
+running and restores saved imports. Failed refreshes keep the last successful cached data
+and its freshness timestamp; 401 reports authentication failure and 403 reports permission
+failure, without falling back to anonymous requests. Existing backoff and longer
+`Retry-After` delays on 429/503 remain in effect. See
+[MeshMapper troubleshooting](troubleshooting.md#meshmapper-imports-are-unconfigured-or-stale).
+
+### Coordinate the MeshMapper rollout
+
+Before enforcement, confirm with the MeshMapper admin that Server supports `X-API-Key`,
+that the key covers the required APIs and IATAs, and that these calls do not consume Coverage
+quota. Header support must not be assumed deployed. The first four endpoints can require
+keys before the app rollout; repeater enforcement waits for app 1.4.1 and its forced update.
+Key issuance, permissions, quotas, and enforcement switches remain MeshMapper Server work,
+tracked in [MeshMapper_Server#431](https://github.com/MeshMapper/MeshMapper_Server/issues/431).
+
+Existing `coverage.php` consumers keep their own keys, scopes, quotas, and supported
+`?key=` authentication. Do not move them to header-only authentication without confirmed
+server support; Beacon's importer does not call `coverage.php`.
+
 ## Admin API key

 The `/api/v1/admin/*` endpoints need `Authorization: Bearer <key>`. Everything else, including
@@ -1,7 +1,8 @@
 # Securing a deployment

 Beacon is a read-mostly public service. The things worth protecting are the admin key, the API
-listener, your broker credentials, and the database, which holds decrypted channel messages.
+listener, your broker and MeshMapper credentials, and the database, which holds decrypted
+channel messages.

 ## The admin key

@@ -14,6 +15,14 @@ environment wins over the file) are in [Configuration](configuration.md#admin-ap
 If you do not need the admin endpoints, do not set a key. They answer `503` and nothing else
 changes.

+## The MeshMapper API key
+
+Obtain a regional or grouped-region API key from your local MeshMapper admin and keep it
+in `MESHMAPPER_API_KEY` in the backend environment. Never use the mobile App key, publish
+the key through a `VITE_*` setting, or commit it. Beacon sends it only as `X-API-Key` to
+the supported HTTPS MeshMapper API endpoints and refuses redirects. This is separate
+from Beacon's admin authentication. See [MeshMapper API key](configuration.md#meshmapper-api-key).
+
 ## Keep the API listener private

 beacon-server listens on `:8080` with no TLS. In the Docker deployments it is only reachable
@@ -49,6 +58,6 @@ if you enable it, contains everything the database holds, so keep archives priva

 ## Secrets in files

-`.env` holds the database password, broker credentials and the admin key. It is gitignored in
+`.env` holds the database password, broker credentials, admin key, and MeshMapper API key. It is gitignored in
 the deployment folders; keep it that way, and keep its permissions tight on the host.
 `data/app/config.yaml` holds channel keys. Neither belongs in a public backup.
@@ -93,6 +93,37 @@ The channel's key is not configured, or it was added without a restart. Add it u
 the stored messages for the new key and logs
 `config: backfilled N previously-undecrypted channel message(s)`.

+## MeshMapper imports are unconfigured or stale
+
+Read `docker compose logs app` for `component=meshmapper`, `meshmapper.zones`,
+`meshmapper.scopes`, or `meshmapper.channels`. Refresh records include `last_error`,
+`next_attempt`, and, for regional imports, `checked_at`.
+
+| Message | Fix |
+|---|---|
+| `unconfigured: MeshMapper API key is missing` | Set `MESHMAPPER_API_KEY` in the backend environment. An explicitly empty environment value overrides a YAML key. |
+| `unconfigured: MeshMapper API key contains invalid characters` | Re-enter the key supplied by your local MeshMapper regional or grouped-region admin; remove embedded whitespace or control characters. |
+| `authentication failed (HTTP 401)` | Check that you are using the correct regional or grouped-region API key and that MeshMapper Server supports `X-API-Key`. Do not use the mobile App key. |
+| `permission denied (HTTP 403)` | Ask the MeshMapper admin to check access to the requested API and IATA. A group key needs every member IATA, including members added by `import_groups`. |
+| `HTTP 429` or `HTTP 503` | Wait until `next_attempt`. Beacon honours longer `Retry-After` delays and its existing refresh limits. Do not shorten polling intervals or repeatedly restart to force a request. |
+
+After changing `.env`, run `docker compose up -d --force-recreate app` from the backend
+deployment folder. A plain restart does not reload container environment variables.
+Persisted backoff still applies after a restart. Never paste the key into logs, URLs, or
+support messages.
+
+Beacon keeps the last successful cached data and freshness timestamp after a failed
+refresh, including an authentication failure. It does not replace imports with empty
+lists or retry anonymously. See [MeshMapper API key](configuration.md#meshmapper-api-key)
+for setup and rollout requirements.
+
+## Backup export refuses a saved MeshMapper key
+
+Move the key from `meshmapper.api_key` to `MESHMAPPER_API_KEY` in the backend environment,
+clear the saved YAML value, and recreate `app`. Backups include saved YAML verbatim, so
+exports refuse a nonempty MeshMapper key there to keep it out of download responses.
+See [Backup and export](backup-export.md#requirements-and-limits).
+
 ## I changed a `VITE_*` value and nothing changed

 Run `docker compose up -d web` (`docker compose up -d beacon-web` in the split deployment). The web container writes the values to `/config.js` when it
@@ -93,6 +93,26 @@ docker compose up -d
 Database migrations run when the server starts. Read the release notes first; a release that
 needs a config change says so there. Keep server and web on the same `X.Y`.

+## MeshMapper API authentication rollout
+
+When upgrading to a Beacon build with MeshMapper API authentication, configure the key
+before relying on imports. Ask your local MeshMapper regional or grouped-region admin
+for a key covering the required APIs and your region or all member regions. Confirm
+MeshMapper Server supports `X-API-Key` before enforcement; see
+[MeshMapper API key](configuration.md#meshmapper-api-key) for the rollout details.
+
+Set `MESHMAPPER_API_KEY` in the backend `.env`, then recreate the `app` container:
+
+```bash
+docker compose up -d --force-recreate app
+```
+
+Without a key, Beacon keeps running and retains saved imports, but MeshMapper refreshes
+are unconfigured. Existing IP exemptions are not a substitute for a key. After configuring
+one, check the MeshMapper logs for successful refreshes; existing backoff still applies.
+If you use backups, leave `meshmapper.api_key` empty in the saved YAML and keep the key in
+the environment. Exports refuse a nonempty saved YAML key.
+
 ## Changes that need a restart

 Beacon reads `config.yaml` once at startup. After editing it, run
@@ -101,7 +121,10 @@ Beacon reads `config.yaml` once at startup. After editing it, run
 - A newly added channel key decrypts the matching stored messages. Look for
  `config: backfilled N previously-undecrypted channel message(s)` in the log.
 - Border file and MeshMapper import changes take effect.
- A changed admin key is picked up.
+- Changed admin or MeshMapper keys in the YAML file are picked up.
+
+For environment changes in `.env`, use `docker compose up -d --force-recreate app` instead.
+A container restart reuses its old environment.

 The one runtime-only setting is CORS origins: `PUT /api/v1/admin/config` changes them without a
 restart, and nothing is written to the file, so a restart reloads whatever the file says.