diff --git a/README.md b/README.md index 7404752..f7d9a4b 100644 --- a/README.md +++ b/README.md @@ -71,6 +71,11 @@ Everything else in the file is optional and explained in [Configuration](docs/configuration.md): the admin API key, log settings, Redis, and the web app's map view, tabs, themes, name and banner. +If you enable MeshMapper imports, also set `MESHMAPPER_API_KEY`. Your local MeshMapper +regional or grouped-region admin can generate a regional key or a key covering multiple +regions. See [MeshMapper API key](docs/configuration.md#meshmapper-api-key) for setup and +the server rollout requirements. Keep this key out of `VITE_*` settings. + ### 3. Describe your network in `config.yaml` ```bash diff --git a/app_config/.env.example b/app_config/.env.example index 41353e2..d85bd62 100644 --- a/app_config/.env.example +++ b/app_config/.env.example @@ -15,6 +15,13 @@ LISTEN_ADDR=:8080 # At least 16 characters, no whitespace; e.g. `openssl rand -hex 32`. # BEACON_API_KEY= +# MeshMapper regional or grouped-region API key for enabled imports. +# Your local MeshMapper regional or grouped-region admin can generate the key; +# grouped-region keys cover multiple regions. Never use the mobile App key. +# Overrides meshmapper.api_key, even when empty. Keep this out of VITE_* settings. +# Use the environment setting with backups; saved YAML keys prevent exports. +# MESHMAPPER_API_KEY= + # Override log.level / log.format from config.yaml (defaults: info / text). # LOG_LEVEL=info # LOG_FORMAT=json diff --git a/app_config/config.yaml.example b/app_config/config.yaml.example index 6491285..c30fe48 100644 --- a/app_config/config.yaml.example +++ b/app_config/config.yaml.example @@ -45,7 +45,14 @@ ratelimit: # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds). burst: 300 -# MeshMapper integrations (optional, no API key). Import transport scopes and +# MeshMapper integrations (optional, require a regional or grouped-region API key). +# Your local MeshMapper regional or grouped-region admin can generate the key. +# A grouped-region key covers multiple regions, including imported group members. +# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the +# environment for secrets, especially with backups; never use the mobile App key. +# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain. +# 401/403 failures retain cached data and report authentication/permission errors. +# Import transport scopes and # IATA border outlines from MeshMapper instead of maintaining scopes: and # iatas.*.borderFile by hand. # @@ -63,6 +70,7 @@ ratelimit: # Region-filtered scope lists and scope stats show imported names only under the # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only. #meshmapper: +# api_key: "" # scopes: # enabled: false # refresh_interval: 1h # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick diff --git a/docker-deployment-type1/data/app/config.yaml b/docker-deployment-type1/data/app/config.yaml index 6dfc478..59f3592 100644 --- a/docker-deployment-type1/data/app/config.yaml +++ b/docker-deployment-type1/data/app/config.yaml @@ -46,7 +46,14 @@ ratelimit: # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds). burst: 300 -# MeshMapper integrations (optional, no API key). Import transport scopes and +# MeshMapper integrations (optional, require a regional or grouped-region API key). +# Your local MeshMapper regional or grouped-region admin can generate the key. +# A grouped-region key covers multiple regions, including imported group members. +# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the +# environment for secrets, especially with backups; never use the mobile App key. +# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain. +# 401/403 failures retain cached data and report authentication/permission errors. +# Import transport scopes and # IATA border outlines from MeshMapper instead of maintaining scopes: and # iatas.*.borderFile by hand. # @@ -64,6 +71,7 @@ ratelimit: # Region-filtered scope lists and scope stats show imported names only under the # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only. #meshmapper: +# api_key: "" # scopes: # enabled: false # refresh_interval: 1h # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick diff --git a/docker-deployment-type2/README.md b/docker-deployment-type2/README.md index de6a7ff..ec620a3 100644 --- a/docker-deployment-type2/README.md +++ b/docker-deployment-type2/README.md @@ -45,6 +45,11 @@ docker-deployment-type2/ Caddy redirects anything that is not `/api/*` or `/ws` to that host. The password inside `POSTGRES_DSN` must match `POSTGRES_PASSWORD` in `docker-compose.yml`. + For MeshMapper imports, set `MESHMAPPER_API_KEY` in this server host's `.env`. + Your local MeshMapper regional or grouped-region admin can generate the regional or + grouped-region key. Do not copy it to the web host or a `VITE_*` setting. See + [MeshMapper API key](../docs/configuration.md#meshmapper-api-key). + 3. Edit `data/app/config.yaml` for your network (see [Configuration](../docs/configuration.md#configyaml)). Because the web app will be served from another origin, browsers may only open the WebSocket if that origin is listed: diff --git a/docker-deployment-type2/server/data/app/config.yaml b/docker-deployment-type2/server/data/app/config.yaml index 6dfc478..59f3592 100644 --- a/docker-deployment-type2/server/data/app/config.yaml +++ b/docker-deployment-type2/server/data/app/config.yaml @@ -46,7 +46,14 @@ ratelimit: # A 429 returns error.code=rate_limited and Retry-After (1 or 60 seconds). burst: 300 -# MeshMapper integrations (optional, no API key). Import transport scopes and +# MeshMapper integrations (optional, require a regional or grouped-region API key). +# Your local MeshMapper regional or grouped-region admin can generate the key. +# A grouped-region key covers multiple regions, including imported group members. +# MESHMAPPER_API_KEY overrides meshmapper.api_key, even when empty. Use the +# environment for secrets, especially with backups; never use the mobile App key. +# Missing keys leave imports unconfigured; cached data and the rest of Beacon remain. +# 401/403 failures retain cached data and report authentication/permission errors. +# Import transport scopes and # IATA border outlines from MeshMapper instead of maintaining scopes: and # iatas.*.borderFile by hand. # @@ -64,6 +71,7 @@ ratelimit: # Region-filtered scope lists and scope stats show imported names only under the # IATAs whose catalogue lists them; IATAs without a catalogue show manual scopes only. #meshmapper: +# api_key: "" # scopes: # enabled: false # refresh_interval: 1h # 1h-24h (MeshMapper allows one get_scopes.php per region per 55m); one source per 15s tick diff --git a/docs/backup-export.md b/docs/backup-export.md index 355ef37..0bea075 100644 --- a/docs/backup-export.md +++ b/docs/backup-export.md @@ -22,6 +22,10 @@ changes, PostgreSQL roles and cluster settings, Redis and deployment files are n ## Requirements and limits +- Keep the MeshMapper API key in `MESHMAPPER_API_KEY`, with `meshmapper.api_key` empty or + omitted in the saved YAML. Both CLI exports and admin downloads refuse a nonempty saved + MeshMapper key because the YAML is included verbatim. Environment secrets remain excluded; + preserve them separately for recovery. See [MeshMapper API key](configuration.md#meshmapper-api-key). - `pg_dump` must be in the **same runtime** as the exporter (the CLI's environment, or the server's PATH for the download), with a major version at least as new as the database's. A client on the Docker host or in another container doesn't count. diff --git a/docs/configuration.md b/docs/configuration.md index 9d12ac0..24c0690 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -23,6 +23,7 @@ to open). | `MQTT_BROKER_1_PASSWORD` | unset | Subscriber password for broker 1. | | `MQTT_BROKER_2_URL`, `_USERNAME`, `_PASSWORD` | unset | The same for an optional second broker. A packet heard through both brokers is stored once. | | `BEACON_API_KEY` | unset | Bearer key for `/api/v1/admin/*`. When set, even to an empty string, it replaces `auth.api_key` from the config file. See [Admin API key](#admin-api-key). | +| `MESHMAPPER_API_KEY` | unset | Regional or grouped-region key for enabled MeshMapper imports. When set, even to an empty string, it replaces `meshmapper.api_key`. Missing keys leave imports unconfigured while Beacon keeps running. See [MeshMapper API key](#meshmapper-api-key). | | `LOG_LEVEL` | `info` | `debug`, `info`, `warn` or `error`. Overrides `log.level`. Anything else stops startup. | | `LOG_FORMAT` | `text` | `text` or `json`. Overrides `log.format`. Anything else stops startup. | | `BEACON_CPU_PROFILE_DIR`, `BEACON_CPU_PROFILE_UNTIL` | unset | Enable a bounded CPU capture in production. See [Profiling](profiling.md). | @@ -68,7 +69,7 @@ which ones to read. | `log` | Log level and format. | You want JSON logs for a collector. | | `server` | `trusted_proxies`: the proxy addresses allowed to tell Beacon the real client IP. | Always, if Beacon sits behind a proxy. See [Reverse proxy](reverse-proxy.md). | | `ratelimit` | Per-client REST limits for `/api/v1/*`. On by default at 300 requests a minute. | You get 429s you did not expect. | -| `meshmapper` | Import transport scopes, IATA borders, region groups and public channels from MeshMapper instead of listing them by hand. | You are in a region MeshMapper covers. | +| `meshmapper` | Import transport scopes, IATA borders, region groups and public channels from MeshMapper instead of listing them by hand. Requires a [MeshMapper API key](#meshmapper-api-key). | You are in a region MeshMapper covers. | | `iatas` | Display names, coordinates and optional border files for airport codes. IATAs are created automatically when traffic arrives; this block only decorates them. A `borderFile` path is relative to `config.yaml`; in Docker the compose file mounts only `config.yaml`, so add a mount for the folder too (`- ./data/app/borders:/app/borders:ro` under the `app` service). | You want names on the map or a border drawn. | | `regions` | Groups of IATAs with a name, map centre and zoom. | Always, unless MeshMapper imports them. | | `channel_keys` | Hashtag channels and explicit keys for decrypting group messages. | Always. Without keys, channel messages are stored as hashes only. | @@ -106,6 +107,58 @@ cannot honour. These are the ones people hit: A negative `websocket.max_connections_per_ip` is not caught at startup and rejects every connection instead, so leave it at a positive number. +## MeshMapper API key + +MeshMapper APIs require either a regional API key or a grouped-region API key covering +multiple regions. Your local MeshMapper regional or grouped-region admin can generate +these keys. Request a key for the APIs Beacon uses, covering your IATA or every member IATA +in your group, including members added by `meshmapper.zones.import_groups`. + +Set `MESHMAPPER_API_KEY` in the deployment's private `.env` or backend environment. Each +Beacon deployment can use a different key. It overrides `meshmapper.api_key` in +`config.yaml`, even when the environment value is empty. Both examples leave the key empty. +Do not use the mobile app's App key, `BEACON_API_KEY`, or a Coverage API key for this setting. +Never put it in a `VITE_*` value, browser configuration, logs, or git. + +After changing `.env`, recreate the backend container from the deployment folder +(`docker-deployment-type2/server` for the split deployment): + +```bash +docker compose up -d --force-recreate app +``` + +A plain `docker compose restart app` does not load changed container environment variables. +If you use the YAML setting instead, restarting Beacon is enough. Prefer the environment +setting when using [backups](backup-export.md): exports refuse a nonempty +`meshmapper.api_key` in the saved YAML so the key cannot enter a download. + +Beacon sends `X-API-Key` to `get_zones.php`, `get_geojson.php`, `get_scopes.php`, and +`get_channels.php`. Its shared client supports the same header for `get_repeaters.php`, +although Beacon does not currently fetch repeaters. Requests use trusted HTTPS MeshMapper +endpoints and do not follow redirects. `get_zones.php` still needs its `country` parameter; +a key covering multiple regions does not change the request shape or remove rate limits. +IP exemptions are not authentication. + +With no key, MeshMapper imports report unconfigured and make no requests. Beacon keeps +running and restores saved imports. Failed refreshes keep the last successful cached data +and its freshness timestamp; 401 reports authentication failure and 403 reports permission +failure, without falling back to anonymous requests. Existing backoff and longer +`Retry-After` delays on 429/503 remain in effect. See +[MeshMapper troubleshooting](troubleshooting.md#meshmapper-imports-are-unconfigured-or-stale). + +### Coordinate the MeshMapper rollout + +Before enforcement, confirm with the MeshMapper admin that Server supports `X-API-Key`, +that the key covers the required APIs and IATAs, and that these calls do not consume Coverage +quota. Header support must not be assumed deployed. The first four endpoints can require +keys before the app rollout; repeater enforcement waits for app 1.4.1 and its forced update. +Key issuance, permissions, quotas, and enforcement switches remain MeshMapper Server work, +tracked in [MeshMapper_Server#431](https://github.com/MeshMapper/MeshMapper_Server/issues/431). + +Existing `coverage.php` consumers keep their own keys, scopes, quotas, and supported +`?key=` authentication. Do not move them to header-only authentication without confirmed +server support; Beacon's importer does not call `coverage.php`. + ## Admin API key The `/api/v1/admin/*` endpoints need `Authorization: Bearer `. Everything else, including diff --git a/docs/security.md b/docs/security.md index 05c68fb..85b6bfc 100644 --- a/docs/security.md +++ b/docs/security.md @@ -1,7 +1,8 @@ # Securing a deployment Beacon is a read-mostly public service. The things worth protecting are the admin key, the API -listener, your broker credentials, and the database, which holds decrypted channel messages. +listener, your broker and MeshMapper credentials, and the database, which holds decrypted +channel messages. ## The admin key @@ -14,6 +15,14 @@ environment wins over the file) are in [Configuration](configuration.md#admin-ap If you do not need the admin endpoints, do not set a key. They answer `503` and nothing else changes. +## The MeshMapper API key + +Obtain a regional or grouped-region API key from your local MeshMapper admin and keep it +in `MESHMAPPER_API_KEY` in the backend environment. Never use the mobile App key, publish +the key through a `VITE_*` setting, or commit it. Beacon sends it only as `X-API-Key` to +the supported HTTPS MeshMapper API endpoints and refuses redirects. This is separate +from Beacon's admin authentication. See [MeshMapper API key](configuration.md#meshmapper-api-key). + ## Keep the API listener private beacon-server listens on `:8080` with no TLS. In the Docker deployments it is only reachable @@ -49,6 +58,6 @@ if you enable it, contains everything the database holds, so keep archives priva ## Secrets in files -`.env` holds the database password, broker credentials and the admin key. It is gitignored in +`.env` holds the database password, broker credentials, admin key, and MeshMapper API key. It is gitignored in the deployment folders; keep it that way, and keep its permissions tight on the host. `data/app/config.yaml` holds channel keys. Neither belongs in a public backup. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 4d29677..f4840a5 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -93,6 +93,37 @@ The channel's key is not configured, or it was added without a restart. Add it u the stored messages for the new key and logs `config: backfilled N previously-undecrypted channel message(s)`. +## MeshMapper imports are unconfigured or stale + +Read `docker compose logs app` for `component=meshmapper`, `meshmapper.zones`, +`meshmapper.scopes`, or `meshmapper.channels`. Refresh records include `last_error`, +`next_attempt`, and, for regional imports, `checked_at`. + +| Message | Fix | +|---|---| +| `unconfigured: MeshMapper API key is missing` | Set `MESHMAPPER_API_KEY` in the backend environment. An explicitly empty environment value overrides a YAML key. | +| `unconfigured: MeshMapper API key contains invalid characters` | Re-enter the key supplied by your local MeshMapper regional or grouped-region admin; remove embedded whitespace or control characters. | +| `authentication failed (HTTP 401)` | Check that you are using the correct regional or grouped-region API key and that MeshMapper Server supports `X-API-Key`. Do not use the mobile App key. | +| `permission denied (HTTP 403)` | Ask the MeshMapper admin to check access to the requested API and IATA. A group key needs every member IATA, including members added by `import_groups`. | +| `HTTP 429` or `HTTP 503` | Wait until `next_attempt`. Beacon honours longer `Retry-After` delays and its existing refresh limits. Do not shorten polling intervals or repeatedly restart to force a request. | + +After changing `.env`, run `docker compose up -d --force-recreate app` from the backend +deployment folder. A plain restart does not reload container environment variables. +Persisted backoff still applies after a restart. Never paste the key into logs, URLs, or +support messages. + +Beacon keeps the last successful cached data and freshness timestamp after a failed +refresh, including an authentication failure. It does not replace imports with empty +lists or retry anonymously. See [MeshMapper API key](configuration.md#meshmapper-api-key) +for setup and rollout requirements. + +## Backup export refuses a saved MeshMapper key + +Move the key from `meshmapper.api_key` to `MESHMAPPER_API_KEY` in the backend environment, +clear the saved YAML value, and recreate `app`. Backups include saved YAML verbatim, so +exports refuse a nonempty MeshMapper key there to keep it out of download responses. +See [Backup and export](backup-export.md#requirements-and-limits). + ## I changed a `VITE_*` value and nothing changed Run `docker compose up -d web` (`docker compose up -d beacon-web` in the split deployment). The web container writes the values to `/config.js` when it diff --git a/docs/upgrading.md b/docs/upgrading.md index 7b46061..3f4c5f9 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -93,6 +93,26 @@ docker compose up -d Database migrations run when the server starts. Read the release notes first; a release that needs a config change says so there. Keep server and web on the same `X.Y`. +## MeshMapper API authentication rollout + +When upgrading to a Beacon build with MeshMapper API authentication, configure the key +before relying on imports. Ask your local MeshMapper regional or grouped-region admin +for a key covering the required APIs and your region or all member regions. Confirm +MeshMapper Server supports `X-API-Key` before enforcement; see +[MeshMapper API key](configuration.md#meshmapper-api-key) for the rollout details. + +Set `MESHMAPPER_API_KEY` in the backend `.env`, then recreate the `app` container: + +```bash +docker compose up -d --force-recreate app +``` + +Without a key, Beacon keeps running and retains saved imports, but MeshMapper refreshes +are unconfigured. Existing IP exemptions are not a substitute for a key. After configuring +one, check the MeshMapper logs for successful refreshes; existing backoff still applies. +If you use backups, leave `meshmapper.api_key` empty in the saved YAML and keep the key in +the environment. Exports refuse a nonempty saved YAML key. + ## Changes that need a restart Beacon reads `config.yaml` once at startup. After editing it, run @@ -101,7 +121,10 @@ Beacon reads `config.yaml` once at startup. After editing it, run - A newly added channel key decrypts the matching stored messages. Look for `config: backfilled N previously-undecrypted channel message(s)` in the log. - Border file and MeshMapper import changes take effect. -- A changed admin key is picked up. +- Changed admin or MeshMapper keys in the YAML file are picked up. + +For environment changes in `.env`, use `docker compose up -d --force-recreate app` instead. +A container restart reuses its old environment. The one runtime-only setting is CORS origins: `PUT /api/v1/admin/config` changes them without a restart, and nothing is written to the file, so a restart reloads whatever the file says.