mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-22 17:38:00 +00:00
## What this does The frontend half of serving the UI apart from the server. The server half was #2362. It gives CoreScope three deployment modes. The server container is the same image in all three, and it still serves everything, the classic UI included. | Mode | The UI is served by | API | Server CORS setting | Logins | |---|---|---|---|---| | All-in-one container (today) | the server container | same origin | none | yes | | Split containers (new) | a web container that forwards the API | same origin | none | yes | | Static host or CDN (new) | Cloudflare Pages, a CDN or any web server | another origin | `CORS_ALLOWED_ORIGINS` | no, links to the classic UI | ### Commit 1: the UI as static files, for any host - **One place for API URLs:** `public/deployment.js` says where the API is, and `public/api-base.js` (`CSApi`) builds every API and WebSocket URL. A lint rule keeps raw `/api` fetches out of the other files. The embedded UI keeps `apiBase: ''`, so it behaves as before, except that its WebSocket now uses `/ws` instead of `/`. - **Build:** `make web` (`scripts/static/build.sh`) writes `dist/web`, which is `public/` byte for byte except for the cache buster in `index.html` and `deployment.js`. - It is `dist/web` rather than `dist/` because the Go binaries are built into `dist/` and `make clean` removes that folder. - `API_BASE=same-origin` is for a host that forwards `/api` and `/ws` itself, as the web container does. - **Anonymous build (`AUTH=false`):** for another origin. It has no account features: login, account, admin and other writes link to the same page on the classic UI. It sends no cookies, and its three JSON POSTs go as `text/plain`, so there is no preflight. - **Error notices:** a blocked API answer or an HTML answer each show a notice. - **Channel badge:** `beta`, `next` and `dev` builds show a badge that opens a pre-filled issue, plus an "Open in classic" link. - **Runbook:** `deploy/static/README.md` covers Cloudflare Pages, previews, server CORS and rollback. ### Commit 2: the web container and the three modes - **`Dockerfile.web`:** Caddy on `alpine:3.24`, like the main image. It packages the UI with `build.sh` when the image is built, serves it, and forwards `/api/*` and `/ws` to `BACKEND` (default `corescope:3000`). `SITE_ADDRESS` set to a domain name turns on automatic HTTPS. - **Cache rules:** `docker/web/Caddyfile` sends the same no-cache rules as `deploy/static/_headers`, and compresses responses with zstd or gzip. - **`docker-compose.split.yml`:** the stock server image (its own Caddy off) plus the web container: `CORESCOPE_IMAGE=ghcr.io/kpa-clawbot/corescope:edge docker compose -f docker-compose.split.yml up -d --build`. - `CORESCOPE_IMAGE` is required and must match the checkout's version, so a release server is never paired with a newer UI by default. - The server's built-in broker, which accepts anonymous clients, is published on `127.0.0.1` unless `MQTT_BIND` says otherwise. - **Docs:** `docs/deployment.md` has a new "Deployment modes" section, and `DEPLOY.md` points to it. ### Keeping new code on `CSApi` - **Lint rule:** `.eslintrc.json` now rejects every direct network call in `public/*.js`. That covers `fetch`, `XMLHttpRequest`, `EventSource`, `sendBeacon` and `WebSocket` without `CSApi.wsUrl`, plus `/api` paths in `href`, `src` and `action` (assigned, set with `setAttribute` or written into markup), `window.open`, `location.assign`, `location.replace` and `location =`. `api-base.js` is the one file exempt. - **HTML pages:** `eslint public/*.js` never sees the inline scripts and markup of `public/*.html`, so `test-lint-network-calls.js` applies the same rule to them, at their page lines. `area-map.html`'s two fetches carry an `eslint-disable` with the reason: its Server field can name another server. - **Why it changed:** the earlier rule only caught a literal `/api…` passed straight to `fetch`. - **Check:** CI already fails on lint errors, and `tests/e2e/test-lint-network-calls.js` pins exactly what the rule reports: 25 bad statements reported, 15 good ones not, `api-base.js` exempt. - **AGENTS.md:** a new "API calls" section covers: - which helper to use for each need; - the lint rule; - how anonymous builds gate login-only features; - loading `api-base.js` in vm and jsdom tests (the region quick-picks suite missed that, and failed shard 1 on the first CI run). ### Review follow-ups - **Account tokens:** the channel badge's pre-filled issue drops the query on `#/account/…` routes, so mail-link tokens never reach a GitHub URL. "Open in classic" keeps the full route, since the classic UI needs the token. - **HTML answers:** on a packaged UI (static host, web container), `api()` rejects an HTML answer before status and retry handling. An HTML 404 or 503 (a wrong `API_BASE`, a proxy error page) shows "Server returned an unexpected response" after one request. It used to give a generic error, or six "warming up" retries for a 503. JSON errors and the server's JSON 503 warm-up behave as before. The embedded UI keeps its old handling (`CSApi.embedded`). - **Logins:** `docs/deployment.md`, `Dockerfile.web` and `docker-compose.split.yml` say `userManagement.publicBaseUrl` must be the web container's public origin. ### Audit fixes - **No broken logins build:** `build.sh` refuses `AUTH=true` with a cross-origin `API_BASE`. The server allows no credentialed CORS, so every API request of such a build was blocked; in Chromium all 12 were, and the pages stayed empty. - **No accidental deletes:** `build.sh` refuses an `OUT` that is neither empty nor a previous build. It used to wipe whatever folder or file `OUT` named. - **Lint:** the rule now also reports `/api/` paths in `href`, `src` and `action` attributes in single-quoted strings and template literals. `tests/e2e/test-lint-network-calls.js` pins exactly what it reports. - **Compose:** `CORESCOPE_IMAGE` has no default, and MQTT is published on localhost only. - **Runbook:** the deploy and rollback examples use `vX.Y.Z`, since v3.16.0 and v3.15.0 predate the static build. It also names the server version the static build needs: before #2362 the warm-up headers stay hidden and wildcard previews are refused. ### Audit notes, fixed - **Cache busting on every page:** the tool pages (`area-map.html`, `geofilter-builder.html`) load their scripts with `?v=__BUST__`. The server and `build.sh` replace it in every top-level page, and `_headers` and the Caddyfile make every page revalidate. Behind Caddy, a returning visitor could otherwise get a new tool page with an old `api-base.js`. - **HTTP/3:** the split compose publishes 443 over UDP too, for the HTTP/3 Caddy advertises once `SITE_ADDRESS` is a domain. Checked with Chromium forced onto QUIC: the page and `/api/healthz` loaded over `h3`. - **WebSocket on `/`:** a Go test pins that an upgrade on `/` still works, for UIs cached from before `/ws`. ### Second review - **Trusted proxies:** the docs said to trust the compose subnet, and also to publish port 3000 if wanted. Together they let a client spoof its address: the subnet holds the gateway, where a client reaching a published 3000 through Docker's proxy arrives from. The compose file now fixes the subnet (`SPLIT_NET`, default `172.30.83.0/24`) and the web container's address (`172.30.83.200`), and the docs say to trust that address alone. - **Classic UI unchanged:** the HTML-answer notice above applies to packaged UIs only. On the embedded UI a proxy's HTML 503 is retried as before, and an unknown `/api` path fails to parse as before, with no notice. - **Lint coverage:** the HTML pages and the extra sinks above. AGENTS.md says what the rule cannot see: a URL assembled elsewhere and passed in a variable. - **AGENTS.md:** `build.sh` is described as packaging (a copy, `deployment.js` and `__BUST__`, never a code transform), and rule 3 says `__BUST__` is filled in every top-level page. - **CI builds the web image:** the image-check job (no publish) builds `Dockerfile.web`, runs it, and checks the busted, no-cache `index.html`, the same-origin `deployment.js` and that `/api` reaches `BACKEND`. It also validates the split compose file. - **Cross-origin logins:** a hand-edited `deployment.js` with `auth: true` and another origin now logs a warning naming the setting, instead of failing with CORS errors alone. ## Upgrade notes - **WebSocket path:** the UI's live feed connects to `/ws` instead of `/`. The server has answered `/ws` since #1793 and still upgrades `/`, so browsers holding the previous UI keep working. - A reverse proxy that forwards WebSocket upgrades only for an exact `/` must forward `/ws` too. - The nginx example in `docs/deployment.md` (`location /`) already covers it. - **Static UI from another origin:** - The server needs `CORS_ALLOWED_ORIGINS`, and a version with #2362 for the warm-up banner and wildcard preview origins. - `build.sh` refuses `AUTH=true` there, since the server allows no credentialed CORS. - **Split containers:** - `CORESCOPE_IMAGE` is required. - With user accounts on, `userManagement.publicBaseUrl` must be the web container's public origin. - The built-in MQTT broker is published on `127.0.0.1` unless `MQTT_BIND` says otherwise. - The compose network is `172.30.83.0/24`, with the web container at `.200`. If Docker reports that the pool overlaps another network, set `SPLIT_NET`. ## Testing - `go test ./...`, `go vet` and `gofmt` are clean in `cmd/server`, with new tests for busting every page and the WebSocket upgrade on `/`. - `sh test-all.sh` passes. New unit suites: - `test-static-build.js`: the build and its input checks, `same-origin` included, cross-origin logins refused, and an unrelated `OUT` left intact. - `test-web-container.js`: the Dockerfile, Caddyfile and compose file agree on packaging, forwarded paths, cache rules and the server port; the server image has no default and MQTT stays on localhost. - The `api-base`, anonymous mode, badge and `dev-static` suites. - `eslint public/*.js` has 0 errors. `test-lint-network-calls.js` passes 47/47, the four pages included; it needs eslint, so CI runs it on E2E shard 1. - **Static host or CDN mode:** `test-split-anonymous-e2e.js` passes 11/11. That is the static build on one origin against an API-only server on another, which CI runs on E2E shard 1. - **Split containers mode:** - The image builds, and `caddy validate` passes (Caddy 2.11.4). The new CI step passes locally, and fails when the Caddyfile stops forwarding `/api`. - Trusted proxies, with the stack up: the web container gets `172.30.83.200`. With only that address trusted, a spoofed `X-Forwarded-For` sent straight to a published 3000 is ignored, and one sent through Caddy is replaced with the client's address. With the subnet trusted, as the docs used to say, the spoofed address was believed: a deny entry for it rejected the upgrade. - With `docker-compose.split.yml` on the E2E fixture, seeded the way CI seeds it, `test-e2e-playwright.js` passes 149 of 152 (3 skipped) both through the web container and against the server directly. - Headers, checked with curl: - `no-cache` on `/`, `/index.html` and `/deployment.js`; - zstd on static assets; - `/api/*` proxied (`Via: 1.1 Caddy`); - a `101` WebSocket upgrade on `/ws`. - In Chromium: - every API request and the WebSocket stay on the web container's origin; - there are no page errors and no horizontal overflow at 390 px; - no badge shows on the `static` channel. - Logins through the web container: with user management on and `userManagement.publicBaseUrl` set to the web container's origin, a Chromium run passed every step, all on the web container's origin with no page errors: - registration, and activation from the mail link (which uses `publicBaseUrl`); - the session surviving a reload; - a CSRF-protected `PATCH /api/account`; - logout, then login with the form. - The same login with the backend's origin as `Origin` is refused with 403. The server checks `Origin` against `publicBaseUrl`, not `Host`, which is why the docs ask for it. ## Performance There are no hot-path changes. API calls go through `CSApi.url()`, a string concatenation. In the web container, Caddy serves files from disk and compresses them; API responses pass through with one proxy hop, the same as the all-in-one image's built-in Caddy. ## Notes - **Per-IP limits:** behind the web container, as behind the all-in-one image's Caddy, requests reach the server from the proxy's address. The per-IP limits stay off until `trustedProxies` lists it: `172.30.83.200`, the web container's fixed address. The docs say so. - **`depends_on`:** the web container starts with the server, not after its healthcheck, like the all-in-one image's Caddy. Gating on health would make every `up` wait for the first check (30 s) and fail outright when a large database loads for longer than the healthcheck allows; until the server listens, Caddy answers `/api` with 502. - **Not in this PR:** - publishing a `corescope-web` image (CI builds and smokes it, without publishing); - Cross-origin login—including a static frontend and API on sibling custom domains—is outside this PR. In the deployments supported by this PR, account support requires the frontend and API to share one browser origin, either served together or through a reverse proxy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
872 lines
32 KiB
Markdown
872 lines
32 KiB
Markdown
# CoreScope Deployment Guide
|
|
|
|
Comprehensive guide to deploying and operating CoreScope. For a quick start, see [DEPLOY.md](../DEPLOY.md).
|
|
|
|
## Table of Contents
|
|
|
|
- [System Requirements](#system-requirements)
|
|
- [Deployment modes](#deployment-modes)
|
|
- [Docker Deployment](#docker-deployment)
|
|
- [Configuration Reference](#configuration-reference)
|
|
- [MQTT Setup](#mqtt-setup)
|
|
- [TLS / HTTPS](#tls--https)
|
|
- [Behind a CDN (Cloudflare, Fastly)](#behind-a-cdn-cloudflare-fastly)
|
|
- [Monitoring & Health Checks](#monitoring--health-checks)
|
|
- [Backup & Restore](#backup--restore)
|
|
- [Troubleshooting](#troubleshooting)
|
|
|
|
---
|
|
|
|
## System Requirements
|
|
|
|
| Resource | Minimum | Recommended |
|
|
|----------|---------|-------------|
|
|
| RAM | 256 MB | 512 MB+ |
|
|
| Disk | 500 MB (image + DB) | 2 GB+ for long-term data |
|
|
| CPU | 1 core | 2+ cores |
|
|
| Architecture | `linux/amd64`, `linux/arm64` | — |
|
|
| Docker (run the published image) | 20.10+ | Latest stable |
|
|
| Docker (build the image yourself) | 23.0+, or 20.10+ with `DOCKER_BUILDKIT=1` | Latest stable |
|
|
|
|
The Dockerfile uses BuildKit features (`# syntax=` and `RUN --mount`). BuildKit is the default builder from Docker 23.0; on 20.10 to 22.x the legacy builder is used unless `DOCKER_BUILDKIT=1` is set, and it rejects `RUN --mount`.
|
|
|
|
CoreScope runs well on Raspberry Pi 4/5 (ARM64). The Go server uses ~300 MB RAM for 56K+ packets.
|
|
|
|
---
|
|
|
|
## Deployment modes
|
|
|
|
The UI is static files (`public/`) that call the server's API. The three modes
|
|
differ only in what serves those files. The server container is the same image
|
|
in all three, and it still serves the classic UI itself.
|
|
|
|
| Mode | The UI is served by | API | Server CORS setting | Logins |
|
|
|---|---|---|---|---|
|
|
| [All-in-one container](#all-in-one-container) | the server container | same origin | none | yes |
|
|
| [Split containers](#split-containers) | a web container (`Dockerfile.web`) that forwards the API | same origin | none | yes |
|
|
| [Static host or CDN](#static-host-or-cdn) | Cloudflare Pages, a CDN or any web server | another origin | `CORS_ALLOWED_ORIGINS` | no, links to the classic UI |
|
|
|
|
### All-in-one container
|
|
|
|
The default, and what the rest of this guide describes: one image runs the
|
|
server, the ingestor and, optionally, Mosquitto and Caddy. The server serves the
|
|
UI from `/app/public` and the API on one origin. See
|
|
[Docker Deployment](#docker-deployment).
|
|
|
|
### Split containers
|
|
|
|
`docker-compose.split.yml` runs the server image unchanged next to a web
|
|
container built from `Dockerfile.web`. The web container is Caddy: it serves the
|
|
UI, packaged by `scripts/static/build.sh` when the image is built, and forwards
|
|
`/api/*` and `/ws` to the server. The browser sees one origin, so the server
|
|
needs no CORS setting. Logins work as on the classic UI when account
|
|
configuration uses the public web origin, as described below.
|
|
|
|
```bash
|
|
CORESCOPE_IMAGE=ghcr.io/kpa-clawbot/corescope:edge \
|
|
docker compose -f docker-compose.split.yml up -d --build
|
|
```
|
|
|
|
| Variable | Default | |
|
|
|---|---|---|
|
|
| `HTTP_PORT`, `HTTPS_PORT` | `80`, `443` | Published by the web container; 443 over TCP and UDP (HTTP/3) |
|
|
| `SITE_ADDRESS` | `:80` | A domain name turns on Caddy's automatic HTTPS and HTTP/3. Keep the default ports then: certificate issuance and the HTTP/3 advertisement both assume 80 and 443 |
|
|
| `CORESCOPE_IMAGE` | none, required | The server image of the same version as the checkout the web image is built from: `:edge` on `master`, `:vX.Y.Z` on a release tag |
|
|
| `DATA_DIR` | `./data` | The server's data directory |
|
|
| `GIT_COMMIT` | the build time | The UI's cache buster |
|
|
| `CHANNEL` | `static` | `beta`, `next` or `dev` shows a badge |
|
|
| `MQTT_BIND`, `MQTT_PORT` | `127.0.0.1`, `1883` | Where the server's built-in broker is published. It accepts anonymous clients, so set `MQTT_BIND=0.0.0.0` only to let gateways on a network you trust publish to it |
|
|
| `SPLIT_NET` | `172.30.83` | The compose network's /24, as its first three octets; the web container is `.200` in it. Change it only if Docker reports that the pool overlaps another network, and update `trustedProxies` to match |
|
|
|
|
- When `userManagement.enabled` is true, set `userManagement.publicBaseUrl` in
|
|
the server's config to the public web origin, including its scheme and port
|
|
(for example, `http://localhost:8080` for a web container published on port
|
|
8080). Login and other state-changing requests compare the browser's `Origin`
|
|
or `Referer` with this configured origin; forwarding the public `Host` header
|
|
alone is not enough. This value also determines mail links. See
|
|
[account configuration](user-guide/accounts.md#2-configure).
|
|
- The web container finds the server through `BACKEND` (`corescope:3000` in the
|
|
compose file). Port 3000 stays inside the compose network, and the server
|
|
still serves everything there, the classic UI included; publish it if you want
|
|
to reach that directly.
|
|
- The web image is built from your checkout, so keep the checkout and
|
|
`CORESCOPE_IMAGE` on the same version: a newer UI may call endpoints an older
|
|
server lacks. To update only the UI, rebuild the web container:
|
|
`docker compose -f docker-compose.split.yml up -d --build web`.
|
|
- Behind the web container, as behind the all-in-one image's Caddy, every
|
|
request reaches the server from the proxy's address, so per-IP limits are off
|
|
until `webSocket.trustedProxies` and `userManagement.trustedProxies` list it.
|
|
The compose file fixes that address at `172.30.83.200`: list it alone. Never
|
|
list the compose subnet. It also holds the gateway address, and a client that
|
|
reaches a published port 3000 through Docker's proxy (from the host itself or
|
|
over IPv6, for example) arrives from there, so its own `X-Forwarded-For`
|
|
would be believed.
|
|
|
|
### Static host or CDN
|
|
|
|
`make web` (or `scripts/static/build.sh`) writes the anonymous build to
|
|
`dist/web` for Cloudflare Pages or any static host. It calls the server
|
|
cross-origin, so the server lists the UI's origin in `CORS_ALLOWED_ORIGINS`.
|
|
It sends no cookies; logins and admin pages link to the classic UI on the
|
|
server. Setup, previews, server settings and rollback:
|
|
[deploy/static/README.md](../deploy/static/README.md).
|
|
|
|
---
|
|
|
|
## Docker Deployment
|
|
|
|
### Quick Start (one command)
|
|
|
|
```bash
|
|
docker run -d --name corescope \
|
|
-p 80:80 \
|
|
-v corescope-data:/app/data \
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
Open `http://localhost` — you'll see an empty dashboard ready to receive packets.
|
|
|
|
No `config.json` is required. The server starts with sensible defaults:
|
|
- HTTP on port 3000 (Caddy proxies port 80 → 3000 internally)
|
|
- Internal Mosquitto MQTT broker on port 1883
|
|
- Ingestor connects to `mqtt://localhost:1883` automatically
|
|
- SQLite database at `/app/data/meshcore.db`
|
|
|
|
### Full `docker run` Reference (recommended)
|
|
|
|
The bare `docker run` command is the primary deployment method. One image, documented parameters — run it however you want.
|
|
|
|
```bash
|
|
docker run -d --name corescope \
|
|
--restart=unless-stopped \
|
|
-p 80:80 -p 443:443 -p 1883:1883 \
|
|
-e DISABLE_MOSQUITTO=false \
|
|
-e DISABLE_CADDY=false \
|
|
-v /your/data:/app/data \
|
|
-v /your/Caddyfile:/etc/caddy/Caddyfile:ro \
|
|
-v /your/caddy-data:/data/caddy \
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
#### Parameters
|
|
|
|
| Parameter | Required | Description |
|
|
|-----------|----------|-------------|
|
|
| `-p 80:80` | Yes | HTTP web UI |
|
|
| `-p 443:443` | No | HTTPS (only if using built-in Caddy with a domain) |
|
|
| `-p 1883:1883` | No | MQTT broker (expose if external gateways connect directly) |
|
|
| `-v /your/data:/app/data` | Yes | Persistent data: SQLite DB, config.json, theme.json |
|
|
| `-v /your/Caddyfile:/etc/caddy/Caddyfile:ro` | No | Custom Caddyfile for HTTPS |
|
|
| `-v /your/caddy-data:/data/caddy` | No | Caddy TLS certificate storage |
|
|
| `-e DISABLE_MOSQUITTO=true` | No | Skip the internal Mosquitto broker (use your own) |
|
|
| `-e DISABLE_CADDY=true` | No | Skip the built-in Caddy reverse proxy |
|
|
| `-e MQTT_BROKER=mqtt://host:1883` | No | Override MQTT broker URL |
|
|
| `-e RUN_AS_UID=1000` | No | Run the Go services as this uid instead of root — see [Running the Go services unprivileged](#running-the-go-services-unprivileged) |
|
|
| `-e RUN_AS_GID=1000` | No | Group for `RUN_AS_UID` (defaults to it) |
|
|
|
|
#### `/app/data/.env` convenience file
|
|
|
|
Instead of passing `-e` flags, you can drop a `.env` file in your data volume:
|
|
|
|
```bash
|
|
# /your/data/.env
|
|
DISABLE_MOSQUITTO=true
|
|
DISABLE_CADDY=true
|
|
MQTT_BROKER=mqtt://my-broker:1883
|
|
```
|
|
|
|
The entrypoint sources this file before starting services. This works with any launch method (`docker run`, compose, or manage.sh).
|
|
|
|
Two exceptions, both because the entrypoint runs this file as root from a
|
|
directory the Go services can write once they run unprivileged:
|
|
|
|
- `RUN_AS_UID` and `RUN_AS_GID` are never taken from `.env`. Set them with
|
|
`-e` or in compose's `environment:`.
|
|
- With `RUN_AS_UID` set, `.env` is read only if it is owned by root and
|
|
writable by no one else (`sudo chown root:root .env && sudo chmod 644 .env`).
|
|
Otherwise the log says it was skipped, and why. "Owned by root" means uid 0
|
|
inside the container. With rootless Docker or a user-namespace remap
|
|
(`userns-remap`), host root is not that uid: look at the owner the container
|
|
reports (`docker exec <container> stat -c '%u' /app/data/.env`) and give the host
|
|
file whichever owner id maps to container uid 0.
|
|
|
|
### Docker Compose (legacy alternative)
|
|
|
|
Docker Compose files are maintained for backward compatibility but are no longer the recommended approach.
|
|
|
|
```bash
|
|
curl -sL https://raw.githubusercontent.com/Kpa-clawbot/CoreScope/master/docker-compose.example.yml \
|
|
-o docker-compose.yml
|
|
docker compose up -d
|
|
```
|
|
|
|
#### Compose environment variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `HTTP_PORT` | `80` | Host port for the web UI |
|
|
| `DATA_DIR` | `./data` | Host path for persistent data |
|
|
| `DISABLE_MOSQUITTO` | `false` | Set `true` to use an external MQTT broker |
|
|
| `DISABLE_CADDY` | `false` | Set `true` to skip the built-in Caddy proxy |
|
|
|
|
### manage.sh (legacy alternative)
|
|
|
|
The `manage.sh` wrapper script provides a setup wizard and convenience commands. It uses Docker Compose internally. See [DEPLOY.md](../DEPLOY.md) for usage. New deployments should prefer bare `docker run`.
|
|
|
|
### Image tags
|
|
|
|
| Tag | Use case |
|
|
|-----|----------|
|
|
| `v3.4.1` | Pinned release — recommended for production |
|
|
| `v3.4` | Latest patch in the v3.4.x series |
|
|
| `v3` | Latest minor+patch in v3.x |
|
|
| `latest` | Latest release tag |
|
|
| `edge` | Built from master on every push — unstable |
|
|
|
|
### Updating
|
|
|
|
```bash
|
|
docker compose pull
|
|
docker compose up -d
|
|
```
|
|
|
|
For `docker run` users:
|
|
|
|
```bash
|
|
docker pull ghcr.io/kpa-clawbot/corescope:latest
|
|
docker stop corescope && docker rm corescope
|
|
docker run -d --name corescope ... # same flags as before
|
|
```
|
|
|
|
Data is preserved in the volume — updates are non-destructive.
|
|
|
|
### Running the Go services unprivileged
|
|
|
|
`corescope-server` and `corescope-ingestor` can run as a normal user instead of
|
|
root, so that a code-execution bug in either one does not land as root in a
|
|
container that has your data directory and `config.json` mounted. `supervisord`
|
|
itself stays root, because mosquitto and caddy still need to bind their ports.
|
|
|
|
It is opt-in, with `RUN_AS_UID`:
|
|
|
|
| `RUN_AS_UID` | Services run as |
|
|
|--------------|-----------------|
|
|
| unset (the default) | root — as in earlier releases |
|
|
| set | `RUN_AS_UID` / `RUN_AS_GID` |
|
|
|
|
Nothing about an upgrade changes until you set it: the container does not pick a
|
|
uid for you, and does not change the owner of anything on your host. That is
|
|
deliberate rather than cautious. Every release before this one ran as root, so
|
|
the `meshcore.db` in your data directory — with its `-wal` and `-shm`, `users.db`
|
|
and the backups beside them — is root-owned. Switching user without also handing
|
|
those files over leaves the services unable to write the database, so the switch
|
|
has to be something you ask for.
|
|
|
|
**To turn it on**, pick the uid you want and restart. Setting `RUN_AS_UID` is
|
|
taken as permission to `chown` the data directory, and what an earlier root-run
|
|
container left in it, to that uid. This happens at every start, so a file
|
|
created there later as root (with `docker exec`, say) is handed over at the
|
|
next restart:
|
|
|
|
```bash
|
|
docker run -d --name corescope \
|
|
-e RUN_AS_UID=1000 -e RUN_AS_GID=1000 \
|
|
-v /your/data:/app/data \
|
|
... # same flags as before
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
Check which user it picked:
|
|
|
|
```bash
|
|
docker logs corescope 2>&1 | grep '\[entrypoint\] Go services run as'
|
|
# [entrypoint] Go services run as corescope (uid=1000 gid=1000)
|
|
```
|
|
|
|
If you would rather the container never `chown` anything, do the hand-over
|
|
yourself first. Then there is nothing left for it to change:
|
|
|
|
```bash
|
|
docker stop corescope
|
|
sudo chown -R 1000:1000 /your/data # uid of your own account is usually 1000
|
|
docker start corescope # with RUN_AS_UID=1000 set as above
|
|
```
|
|
|
|
To go back, unset `RUN_AS_UID` and `chown -R root:root /your/data`.
|
|
|
|
Notes:
|
|
|
|
- `RUN_AS_GID` defaults to `RUN_AS_UID` when omitted. `RUN_AS_UID=0` is the
|
|
same as leaving it unset.
|
|
- If the uid already belongs to another account inside the image (`caddy` is
|
|
uid 100, `mosquitto` 101), that account is used as it is, keeping its own
|
|
group, and the log says so. Pick a different uid if you need `RUN_AS_GID`
|
|
honoured.
|
|
- With `RUN_AS_UID` set to a uid the container cannot use, startup fails rather
|
|
than quietly falling back to root.
|
|
- What the hand-over leaves alone: anything mounted into the data directory on
|
|
its own, such as `-v ./config.json:/app/data/config.json:ro`, which keeps the
|
|
owner it has on your host (make sure the uid can read it), and `.env` (see
|
|
above). Each mount it skips is named in the log.
|
|
- If the hand-over cannot change a file, for example on NFS with
|
|
`root_squash`, the log says how many and which. Fix their owner on the host.
|
|
- `/app` stays root's. The services write only to the data directory and
|
|
`/tmp`.
|
|
- A fresh install is root-owned and so starts as root, like any other. Set
|
|
`RUN_AS_UID` from the first run to avoid a hand-over later.
|
|
|
|
The uid selection lives in `docker/select-service-user.sh`, with its branches
|
|
covered by `docker/test-select-service-user.sh`.
|
|
|
|
---
|
|
|
|
## Configuration Reference
|
|
|
|
CoreScope uses a layered configuration system (highest priority wins):
|
|
|
|
1. **Environment variables** — `MQTT_BROKER`, `DB_PATH`, etc.
|
|
2. **`/app/data/config.json`** — full config file (volume-mounted)
|
|
3. **Built-in defaults** — work out of the box with no config
|
|
|
|
### Environment variable overrides
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `MQTT_BROKER` | `mqtt://localhost:1883` | MQTT broker URL (overrides config file) |
|
|
| `MQTT_TOPIC` | `meshcore/#` | MQTT topic subscription pattern |
|
|
| `DB_PATH` | `data/meshcore.db` | SQLite database path |
|
|
| `DISABLE_MOSQUITTO` | `false` | Skip the internal Mosquitto broker |
|
|
| `DISABLE_CADDY` | `false` | Skip the built-in Caddy reverse proxy |
|
|
|
|
### config.json
|
|
|
|
For advanced configuration, create a `config.json` and mount it at `/app/data/config.json`:
|
|
|
|
```bash
|
|
docker run -d --name corescope \
|
|
-p 80:80 \
|
|
-v corescope-data:/app/data \
|
|
-v ./config.json:/app/data/config.json:ro \
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
See `config.example.json` in the repository for all available options including:
|
|
- MQTT sources (multiple brokers)
|
|
- Channel encryption keys
|
|
- Branding and theming
|
|
- Health thresholds
|
|
- Region filters
|
|
- Retention policies
|
|
- Geo-filtering
|
|
- Map tile providers (OSM, Stamen, Carto, etc.)
|
|
|
|
### Map Tile Providers
|
|
|
|
Map tile providers are enabled and configured via the `config.json` file. You can provide your custom API credentials (e.g. `osm_url`, `stamen_api_key`, `mapbox_api_key`) to activate external tile services. Once configured on the server, users can select their preferred tile provider from the Customizer UI on the client, and their choice will be persisted automatically.
|
|
|
|
#### CARTO now requires an API key
|
|
|
|
CARTO's raster basemaps (`carto-dark`, `carto-light`, `carto-voyager`,
|
|
`carto-voyager-dark`, `positron-dark`) require an API key. Without one, tiles
|
|
are still returned with a **200 status** but every one of them is stamped
|
|
`API KEY REQUIRED` — so nothing errors, no healthcheck fires, and the map just
|
|
quietly looks broken.
|
|
|
|
Get a free key at <https://carto.com/basemaps/apikey> — no CARTO account, no
|
|
credit card, emailed back immediately, 5 million tile requests per calendar
|
|
month across raster and vector. Then set it as `key`:
|
|
|
|
```json
|
|
"map": {
|
|
"tiles": {
|
|
"providers": {
|
|
"carto": { "enabled": true, "key": "YOUR_CARTO_KEY" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Notes:
|
|
|
|
- The query parameter is `key`. **`api_key` is silently ignored** and still
|
|
serves the watermarked tile.
|
|
- The key is sent to the browser, so treat it as public. CARTO asks for a domain
|
|
when the key is issued, but whether that restriction is enforced on every
|
|
request has not been verified here. Assume anyone can read the key from
|
|
devtools and spend your quota. The 5M/month fair-use ceiling makes that
|
|
low-impact for most deployments, but it is worth knowing.
|
|
- A wrong or expired key fails the same silent way as no key at all. Verify by
|
|
**looking at a tile**, not by checking for a 200.
|
|
- CARTO has said it is considering freezing data updates to the raster
|
|
basemaps. If you would rather not depend on them, set
|
|
`"carto": { "enabled": false }` and use another provider — `esri` needs no
|
|
key at all, and `osm`/`stamen` accept their own tokens.
|
|
|
|
---
|
|
|
|
## MQTT Setup
|
|
|
|
CoreScope receives MeshCore packets via MQTT. The container ships with an internal Mosquitto broker — no setup needed for basic use.
|
|
|
|
### Internal broker (default)
|
|
|
|
The built-in Mosquitto broker listens on port 1883 inside the container. Point your MeshCore gateways at it:
|
|
|
|
```bash
|
|
# Expose MQTT port for external gateways
|
|
docker run -d --name corescope \
|
|
-p 80:80 -p 1883:1883 \
|
|
-v corescope-data:/app/data \
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
### External broker
|
|
|
|
To use your own MQTT broker (Mosquitto, EMQX, HiveMQ, etc.):
|
|
|
|
1. Disable the internal broker:
|
|
```bash
|
|
-e DISABLE_MOSQUITTO=true
|
|
```
|
|
|
|
2. Point the ingestor at your broker:
|
|
```bash
|
|
-e MQTT_BROKER=mqtt://your-broker:1883
|
|
```
|
|
|
|
Or via `config.json`:
|
|
```json
|
|
{
|
|
"mqttSources": [
|
|
{
|
|
"name": "my-broker",
|
|
"broker": "mqtt://your-broker:1883",
|
|
"username": "user",
|
|
"password": "pass",
|
|
"topics": ["meshcore/#"]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Multiple brokers
|
|
|
|
CoreScope can connect to multiple MQTT brokers simultaneously:
|
|
|
|
```json
|
|
{
|
|
"mqttSources": [
|
|
{
|
|
"name": "local",
|
|
"broker": "mqtt://localhost:1883",
|
|
"topics": ["meshcore/#"]
|
|
},
|
|
{
|
|
"name": "remote",
|
|
"broker": "mqtts://remote-broker:8883",
|
|
"username": "reader",
|
|
"password": "secret",
|
|
"topics": ["meshcore/+/+/packets"]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### MQTT topic format
|
|
|
|
MeshCore gateways typically publish to `meshcore/<gateway>/<region>/packets`. The default subscription `meshcore/#` catches all of them.
|
|
|
|
---
|
|
|
|
## TLS / HTTPS
|
|
|
|
### Option 1: External reverse proxy (recommended)
|
|
|
|
Run CoreScope behind nginx, Traefik, or Cloudflare Tunnel for TLS termination:
|
|
|
|
```nginx
|
|
# nginx example
|
|
server {
|
|
listen 443 ssl;
|
|
server_name corescope.example.com;
|
|
|
|
ssl_certificate /etc/ssl/certs/corescope.pem;
|
|
ssl_certificate_key /etc/ssl/private/corescope.key;
|
|
|
|
location / {
|
|
proxy_pass http://localhost:80;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
proxy_set_header Host $host;
|
|
}
|
|
}
|
|
```
|
|
|
|
The `Upgrade` and `Connection` headers are required for WebSocket support.
|
|
|
|
### Option 2: Built-in Caddy (auto-TLS)
|
|
|
|
The container includes Caddy for automatic Let's Encrypt certificates:
|
|
|
|
1. Create a Caddyfile:
|
|
```
|
|
corescope.example.com {
|
|
reverse_proxy localhost:3000
|
|
}
|
|
```
|
|
|
|
2. Mount it and expose TLS ports:
|
|
```bash
|
|
docker run -d --name corescope \
|
|
-p 80:80 -p 443:443 \
|
|
-v corescope-data:/app/data \
|
|
-v caddy-certs:/data/caddy \
|
|
-v ./Caddyfile:/etc/caddy/Caddyfile:ro \
|
|
ghcr.io/kpa-clawbot/corescope:latest
|
|
```
|
|
|
|
Caddy handles certificate issuance and renewal automatically.
|
|
|
|
---
|
|
|
|
## API Documentation
|
|
|
|
CoreScope auto-generates an OpenAPI 3.0 specification from its route definitions. The spec is always in sync with the running server — no manual maintenance required.
|
|
|
|
### Endpoints
|
|
|
|
| URL | Description |
|
|
|-----|-------------|
|
|
| `/api/spec` | OpenAPI 3.0 JSON schema — machine-readable API definition |
|
|
| `/api/docs` | Interactive Swagger UI — browse and test all 40+ endpoints |
|
|
|
|
### Usage
|
|
|
|
**Browse the API interactively:**
|
|
```
|
|
http://your-instance/api/docs
|
|
```
|
|
|
|
**Fetch the spec programmatically:**
|
|
```bash
|
|
curl http://your-instance/api/spec | jq .
|
|
```
|
|
|
|
**For bot/integration developers:** The spec includes all request parameters, response schemas, and example values. Import it into Postman, Insomnia, or any OpenAPI-compatible tool.
|
|
|
|
### Public instance
|
|
The live instance at [analyzer.00id.net](https://analyzer.00id.net) has all API endpoints publicly accessible:
|
|
- Spec: [analyzer.00id.net/api/spec](https://analyzer.00id.net/api/spec)
|
|
- Docs: [analyzer.00id.net/api/docs](https://analyzer.00id.net/api/docs)
|
|
|
|
---
|
|
|
|
## Behind a CDN (Cloudflare, Fastly)
|
|
|
|
If you front CoreScope with a CDN — Cloudflare, Fastly, Akamai, or
|
|
similar — you **must** configure the CDN to bypass cache for `/api/*`.
|
|
The server emits `Cache-Control: no-store` on every API response
|
|
(see #1551), but Cloudflare's zone-level Cache Rules and legacy Page
|
|
Rules can override origin headers. When that happens, observers, packets,
|
|
stats and other API responses get cached at the edge for minutes to hours,
|
|
producing observer-flap, stale dashboards and inconsistent state across
|
|
viewers.
|
|
|
|
### 1. Verify whether your CDN is caching `/api/*`
|
|
|
|
From **outside** the CDN (a different network than your origin), run:
|
|
|
|
```sh
|
|
curl -sI 'https://<your-domain>/api/observers' | grep -iE 'cf-cache|age|cache-control'
|
|
```
|
|
|
|
Healthy output (cache is bypassed):
|
|
|
|
```
|
|
cache-control: no-store
|
|
cf-cache-status: BYPASS
|
|
age: 0
|
|
```
|
|
|
|
Unhealthy output (CDN is caching despite `no-store`):
|
|
|
|
```
|
|
cache-control: no-store
|
|
cf-cache-status: HIT
|
|
age: 4732
|
|
```
|
|
|
|
`HIT` or `age > 0` means an intermediary is serving cached JSON. Fix it
|
|
before relying on the dashboard.
|
|
|
|
You can also run the bundled helper, which exits non-zero with a precise
|
|
diagnostic when caching is detected:
|
|
|
|
```sh
|
|
scripts/check-cdn-bypass.sh https://<your-domain>
|
|
```
|
|
|
|
### 2. Cloudflare: add a Cache Rule (recommended)
|
|
|
|
Cloudflare Dashboard → your zone → **Caching → Cache Rules → Create rule**:
|
|
|
|
- **When incoming requests match:** Field = `URI Path`, Operator = `starts with`, Value = `/api/`
|
|
- **Then:** Cache eligibility → **Bypass cache**
|
|
|
|
Save and deploy. Re-run the curl above; you should now see
|
|
`cf-cache-status: BYPASS`.
|
|
|
|
Legacy Page Rules equivalent (if your zone has no Cache Rules):
|
|
|
|
- URL pattern: `*your-domain*/api/*`
|
|
- Setting: **Cache Level → Bypass**
|
|
|
|
### 3. Fastly / other CDNs
|
|
|
|
Apply the equivalent bypass-cache rule for the `/api/` path prefix.
|
|
The key invariant is: any response from `/api/*` must reach the
|
|
browser uncached (no shared-cache HIT, no positive `Age` header).
|
|
|
|
### 4. Re-verify
|
|
|
|
After applying the rule, run step 1's curl from outside the CDN again
|
|
and confirm `cf-cache-status: BYPASS` (or absence of HIT) and `age: 0`.
|
|
|
|
### 5. Watch the server log
|
|
|
|
The server logs a one-shot warning at the first request bearing a
|
|
CDN-specific header (`CF-Connecting-IP`, `CF-Ray`, `Fastly-Client-IP`,
|
|
or `True-Client-IP`):
|
|
|
|
Generic reverse-proxy headers (`X-Forwarded-For`, `X-Real-IP`) are
|
|
deliberately NOT used as the signal — every nginx/Caddy/Traefik/k8s
|
|
deploy sets them, so they'd produce false positives on every
|
|
reverse-proxied install.
|
|
|
|
```
|
|
[security] WARNING: detected request via CDN (CF-Ray header present).
|
|
Ensure /api/* is bypassed in your CDN config — see docs/deployment-behind-cdn.md.
|
|
Cached API responses cause observer-flap and incorrect dashboards.
|
|
```
|
|
|
|
This is informational — the request is not blocked. Treat it as a
|
|
prompt to run the verification curl above. The warning logs at most
|
|
once per process boot, regardless of how many CDN-fronted requests
|
|
arrive.
|
|
|
|
### Why this can't be fixed server-side
|
|
|
|
CDN cache policy is operator-controlled. The application emits the
|
|
most conservative cache header it can (`Cache-Control: no-store`),
|
|
but Cloudflare Cache Rules / Page Rules have higher precedence than
|
|
origin headers in many zone configurations. The only durable fix is
|
|
the operator-side bypass rule.
|
|
|
|
---
|
|
|
|
## Monitoring & Health Checks
|
|
|
|
### Docker health check
|
|
|
|
The container includes a built-in health check that hits `/api/stats`:
|
|
|
|
```bash
|
|
docker inspect --format='{{.State.Health.Status}}' corescope
|
|
```
|
|
|
|
Docker reports `healthy` or `unhealthy` automatically. The check runs every 30 seconds.
|
|
|
|
### Manual health check
|
|
|
|
```bash
|
|
curl -f http://localhost/api/stats
|
|
```
|
|
|
|
Returns JSON with packet counts, node counts, and version info:
|
|
|
|
```json
|
|
{
|
|
"totalPackets": 56234,
|
|
"totalNodes": 142,
|
|
"totalObservers": 12,
|
|
"packetsLastHour": 830,
|
|
"packetsLast24h": 19644,
|
|
"engine": "go",
|
|
"version": "v3.4.1"
|
|
}
|
|
```
|
|
|
|
### Log monitoring
|
|
|
|
```bash
|
|
# All logs
|
|
docker compose logs -f
|
|
|
|
# Server only
|
|
docker compose logs -f | grep '\[server\]'
|
|
|
|
# Ingestor only
|
|
docker compose logs -f | grep '\[ingestor\]'
|
|
```
|
|
|
|
### Resource monitoring
|
|
|
|
```bash
|
|
docker stats corescope
|
|
```
|
|
|
|
---
|
|
|
|
## Backup & Restore
|
|
|
|
### Backup
|
|
|
|
Use **`GET /api/backup`** for the telemetry database. It uses SQLite's
|
|
[`VACUUM INTO`](https://www.sqlite.org/lang_vacuum.html) to produce a consistent
|
|
snapshot while the ingestor is running, including committed data in the WAL.
|
|
Do not copy the live `meshcore.db` file: a checkpoint followed by a file copy
|
|
also races new writes. Allow room for a full snapshot in the server's temporary
|
|
directory and at the download destination; large snapshots cost CPU and disk I/O.
|
|
|
|
When accounts are enabled, sign in as an administrator and open `/api/backup`
|
|
on the same site to download a snapshot. For command-line or scheduled backups,
|
|
configure a randomly generated `apiKey` in `config.json` (at least 16 characters).
|
|
Known weak/default keys are rejected even when they meet the length requirement.
|
|
Create a private header file, `~/.config/corescope/backup.header`, containing one line:
|
|
`X-API-Key: <your configured key>`. Set its permissions to `600`; keep the key out
|
|
of shell history, URLs, and scripts checked into Git.
|
|
|
|
On the backup host, install `curl` and the `sqlite3` CLI. Replace the example
|
|
HTTPS origin below with your instance. The download is published as a `.db`
|
|
only after it is complete, nonempty, and passes SQLite's consistency check:
|
|
|
|
```bash
|
|
(
|
|
set -eu
|
|
umask 077
|
|
backup_dir="$HOME/corescope-backups"
|
|
mkdir -p "$backup_dir"
|
|
tmp=$(mktemp "$backup_dir/corescope-$(date -u +%Y%m%dT%H%M%SZ).XXXXXXXX")
|
|
trap 'rm -f "$tmp"' EXIT
|
|
curl -q --fail --silent --show-error \
|
|
--connect-timeout 30 --max-time 1800 \
|
|
--header "@$HOME/.config/corescope/backup.header" \
|
|
--output "$tmp" https://corescope.example.org/api/backup
|
|
test -s "$tmp"
|
|
test "$(sqlite3 -readonly "$tmp" 'PRAGMA quick_check;')" = ok
|
|
mv "$tmp" "$tmp.db"
|
|
)
|
|
```
|
|
|
|
The temporary-file template ends in `X` characters for both GNU and BSD/macOS
|
|
`mktemp`. The 30-minute limit is curl's client-side ceiling for generating and
|
|
downloading the snapshot. CoreScope separately sets a 60-second HTTP write
|
|
timeout, which the backup handler does not override; reverse proxies can impose
|
|
their own deadlines. Raising `--max-time` does not extend those server or proxy limits.
|
|
Timeouts fail the script and remove the incomplete download without replacing a
|
|
previous completed backup.
|
|
|
|
This is a point-in-time telemetry snapshot, not continuous replication or a full
|
|
instance backup. Keep protected copies off the CoreScope host, and also preserve:
|
|
- `config.json`, `theme.json`, deployment settings, and any configured keys/secrets.
|
|
- When accounts are enabled, **`users.db` separately**: use the
|
|
[account backup and restore guide](user-guide/accounts.md#backups), which covers
|
|
its own snapshot endpoint and scheduled backups. Do not copy that database live.
|
|
|
|
### Restore
|
|
|
|
1. Verify the downloaded file is nonempty and run
|
|
`sqlite3 -readonly backup.db 'PRAGMA quick_check;'`; the result must be `ok`.
|
|
Keep an untouched copy of this snapshot.
|
|
2. Stop **every server and ingestor using the target database**, including any
|
|
separate instances. For the single-container example, use `docker stop corescope`;
|
|
for `docker-compose.example.yml`, use
|
|
`docker compose -f docker-compose.example.yml stop corescope`.
|
|
3. Locate the host directory or named volume mounted at `/app/data`. For the
|
|
example Compose file, this is `DATA_DIR` (default `./data`). While all database
|
|
users remain stopped, move the old `meshcore.db` **and any matching `-wal`,
|
|
`-shm`, and `-journal` files together** into a separate recovery directory.
|
|
Keep that set intact; never leave an old WAL beside the replacement database.
|
|
4. Copy the checked snapshot into that data mount as `meshcore.db`. Preserve the
|
|
previous database's owner and permissions so the configured service user can
|
|
read and write it. Leave configuration, keys, and `users.db` in place.
|
|
5. Start the stopped services (`docker start corescope`, or the matching Compose
|
|
`start corescope` command). Check the logs, `/api/healthz`, and expected historical
|
|
packets in the dashboard. Keep the recovery directory until these checks pass.
|
|
|
|
To roll back, stop all database users again, move the replacement database and
|
|
its sidecars aside, then restore the entire original file set before restarting.
|
|
|
|
### Automated backups
|
|
|
|
Run the checked-download block above from a private script using cron or a
|
|
systemd timer. Use absolute paths and the backup user's protected header file;
|
|
schedule one run at a time during quieter hours. Alert on nonzero exit status.
|
|
Prune old **completed `.db` snapshots only after a new backup succeeds**, according
|
|
to your retention policy, and periodically test a restore in an isolated instance.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Container starts but dashboard is empty
|
|
|
|
This is normal on first start with no MQTT sources configured. The dashboard shows data once packets arrive via MQTT. Either:
|
|
- Point a MeshCore gateway at the container's MQTT broker (port 1883)
|
|
- Configure an external MQTT source in `config.json`
|
|
|
|
### "no MQTT connections established" in logs
|
|
|
|
The ingestor couldn't connect to any MQTT broker. Check:
|
|
1. Is the internal Mosquitto running? (`DISABLE_MOSQUITTO` should be `false`)
|
|
2. Is the external broker reachable? Test with `mosquitto_sub -h broker -t meshcore/#`
|
|
3. Are credentials correct in `config.json`?
|
|
|
|
### WebSocket disconnects / real-time updates stop
|
|
|
|
If behind a reverse proxy, ensure WebSocket upgrade headers are forwarded:
|
|
```nginx
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
```
|
|
|
|
Also check proxy timeouts — set them to at least 300s for long-lived WebSocket connections.
|
|
|
|
### High memory usage
|
|
|
|
The in-memory packet store grows with retained packets. Configure retention limits in `config.json`:
|
|
|
|
```json
|
|
{
|
|
"packetStore": {
|
|
"retentionHours": 24,
|
|
"maxMemoryMB": 512
|
|
},
|
|
"retention": {
|
|
"nodeDays": 7,
|
|
"packetDays": 30
|
|
}
|
|
}
|
|
```
|
|
|
|
`packetStore.maxMemoryMB` bounds the store **and the caches that belong to it** — the decoded-packet cache and per-packet index entries, not just the stored rows. It is enforced in two places: the startup load stops at the budget, and the store evicts oldest-first when it exceeds it. Leaving it unset means no limit. Actual usage is on `/api/perf` as `packetStore.trackedMB`, next to `maxMB`.
|
|
|
|
### Database locked errors
|
|
|
|
SQLite doesn't support concurrent writers well. Ensure only one CoreScope instance accesses the database file. If running multiple containers, each needs its own database.
|
|
|
|
### Container unhealthy
|
|
|
|
Check logs: `docker compose logs --tail 50`. Common causes:
|
|
- Port 3000 already in use inside the container
|
|
- Database file permissions (must be writable by the container user)
|
|
- Corrupted database — restore from backup
|
|
|
|
### ARM / Raspberry Pi issues
|
|
|
|
- Use `linux/arm64` images (Pi 4 and 5). Pi 3 (armv7) is not supported.
|
|
- First pull may be slow — the multi-arch manifest selects the right image automatically.
|
|
- If memory is tight, set `packetStore.maxMemoryMB` to limit RAM usage.
|