Follow-up to #2125: the six non-blocking notes on its approval, with
note 3 as the one that changes behaviour.
**What changes**
- `.env` is now read last. The hand-over of the data directory moves
before it, and the only code that runs after the file is sourced reads
`prepare_service_user`'s positional parameters: the account, the
container's `RUN_AS_UID`/`RUN_AS_GID` (set or unset, and value) and the
file's path are parked in `$1..$6` before the `.`.
- The comment gives the real reason the account was already safe: the
`.` happens inside `load_env_file`, so a `set -- root` in `.env` lands
in that function's `$1`, not in `prepare_service_user`'s (note 1). It
also says that a trusted `.env` is root's code and this does not defend
against it (it can redefine functions or set `PATH`); what it cannot do
by assigning variables is change the account, where the hand-over goes,
or `RUN_AS_UID`/`RUN_AS_GID` as the services see them.
- `svc_log` uses `printf` (note 5); the entrypoint check also pins that
it does not assign `CORESCOPE_SERVICE_USER` (note 4); the test header is
reflowed (note 6); `TEST_SHELL` picks the inner shell so the suite runs
under dash, bash or busybox without editing.
**Why**
A sourced file shares the shell's variables: every global, the locals of
`load_env_file`, and, by dynamic scoping, the locals of its caller too
(checked under dash, bash 5.3 and busybox ash 1.37). So a trusted `.env`
that assigned `_uid_before` or `_uid_was` skipped the restore and left
`RUN_AS_UID=0` in the services' environment, as note 3 says. The same
class reached the hand-over: with `SERVICE_DATA_DIR=/` in `.env`, the
walk ran over `/`. Nothing but the entrypoint reads
`RUN_AS_UID`/`RUN_AS_GID`, so the effect was a wrong value in the
services' environment and a log line that said "ignoring" when it had
not.
**What I checked**
- `docker/test-select-service-user.sh`: 116 assertions, 0 failed, under
dash and bash. The new cases fail on master for the stated reasons and
nothing else does.
- New cases: `.env` running `set -- root` (note 1; this is the case that
catches sourcing the file inline, the names case alone does not); `.env`
assigning the restore's names, `_env_file`, `SERVICE_ENV_FILE` and
`RUN_AS_*`, with uid 1000 and gid 1001 so a swapped parameter shows;
`RUN_AS_GID` alone; `.env` pointing the three `SERVICE_*` paths
elsewhere; a logged path containing `\012`.
- Mutations, each applied to a copy and confirmed to make the suite
fail: hand-over moved back after `.env`; restore from `load_env_file`'s
locals (master); restore from `prepare_service_user`'s locals; file
sourced inline in `prepare_service_user` with the refusal log kept;
restore dropped; only the uid compared; gid restored from the uid's
parameter; set/unset flipped; the log reading `SERVICE_ENV_FILE` after
the `.`; the entrypoint reassigning the account. `svc_log` back to
`echo` fails under dash only, which is where note 5 applies.
- The image, built from this branch (linux/amd64), with a root-owned,
root-only-writable `.env` containing all of the above plus
`DISABLE_MOSQUITTO=true`, `DISABLE_CADDY=true` and `set -- root`: (A)
`RUN_AS_UID` unset: root, nothing re-owned, `DISABLE_*` honoured,
`RUN_AS_*` not in the services' environment. (B) `RUN_AS_UID=1000
RUN_AS_GID=1001` on A's data directory, root-owned `meshcore.db` inside:
services run as 1000, their environment has `RUN_AS_UID=1000
RUN_AS_GID=1001`, `meshcore.db` and the directory are 1000:1001, `.env`
stays root's, `/etc/passwd` stays root's. (C) `.env` owned by uid 1000:
refused with the reason, services run as 1000. The same image with
master's two scripts mounted over it, in B's shape: services'
environment `RUN_AS_UID=0 RUN_AS_GID=0`, `/etc/passwd` owned by 1000,
`.env` owned by 1000:1001.
- `sh test-all.sh` passes; no JS or Go change.
**Not checked**
- Windows: the suite stops at the symlink fixture, as before.
- busybox as `TEST_SHELL` on my machine: Ubuntu's busybox prefers its
own applets over `PATH`, so the stubs are bypassed and 28 cases fail, on
master and on this branch alike. Your WSL Debian run (103 passed) and
the image itself, which runs Alpine's ash with real binaries, are what
cover ash.
- Rootless Docker and `userns-remap`: unchanged, docs gap already noted.
Note 2: kept the names case as a second guard; the `set -- root` case is
the one that pins the design.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: nullrouten <201262769+nullrouten0@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Runs `corescope-server` and `corescope-ingestor` unprivileged, opt-in
with `RUN_AS_UID`. A code-execution bug in either one then no longer
lands as root next to the mounted data and `config.json`. `supervisord`
stays root, for mosquitto and caddy.
| `RUN_AS_UID` | Go services run as |
|---|---|
| unset (default) | root, as before. Upgrades change nothing |
| set | `RUN_AS_UID` / `RUN_AS_GID` |
With `RUN_AS_UID` set:
- Each start hands the data directory to that uid (`chown -h`), except
`.env` and anything mounted into it on its own. Failures are logged.
- `.env` is read only if it is root-owned and only root can write it.
`RUN_AS_UID` and `RUN_AS_GID` are never taken from it.
- `/app` stays root's. **Depends on #2137.**
- A uid with no account in the image gets one at start; `caddy` and
`mosquitto` are reused. A uid that can't be provided fails startup
instead of falling back to root.
The logic is in `docker/select-service-user.sh`, tested by
`docker/test-select-service-user.sh`, which needs no Docker or root and
runs in CI. See also "Running the Go services unprivileged" in
`docs/deployment.md`.
Verified in the built image (master + this + #2137) across 14 cases.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com>
Co-authored-by: nullrouten <201262769+nullrouten0@users.noreply.github.com>
## Summary
Fixes#702 — `.env` file `DISABLE_MOSQUITTO`/`DISABLE_CADDY` ignored
when using `docker run`.
## Changes
### Entrypoint sources `/app/data/.env`
The entrypoint now sources `/app/data/.env` (if present) before the
`DISABLE_*` checks. This works regardless of how the container is
started — `docker run`, compose, or `manage.sh`.
```bash
if [ -f /app/data/.env ]; then
set -a
. /app/data/.env
set +a
fi
```
### `DISABLE_CADDY` added to compose files
Both `docker-compose.yml` and `docker-compose.staging.yml` now forward
`DISABLE_CADDY` to the container environment (was missing — only
`DISABLE_MOSQUITTO` was wired).
### Deployment docs updated
- `docs/deployment.md`: bare `docker run` is now the primary/recommended
approach with a full parameter reference table
- Documents the `/app/data/.env` convenience feature
- Compose and `manage.sh` marked as legacy alternatives
- `DISABLE_CADDY` added to the environment variable reference
### README quick start updated
Shows the full `docker run` command with `--restart`, ports, and
volumes. Includes HTTPS variant. Documents `-e` flags and `.env` file.
### v3.5.0 release notes
Updated the env var documentation to mention the `.env` file support.
## Testing
- All Go server tests pass
- All Go ingestor tests pass
- No logic changes to Go code — entrypoint shell script + docs only
---------
Co-authored-by: you <you@example.com>
## Summary
Implements the `DISABLE_CADDY` environment variable in the Docker
entrypoint, fixing #629.
## Problem
The `DISABLE_CADDY` env var was documented but had no effect — the
entrypoint only handled `DISABLE_MOSQUITTO`.
## Changes
### New supervisord configs
- **`supervisord-go-no-caddy.conf`** — mosquitto + ingestor + server (no
Caddy)
- **`supervisord-go-no-mosquitto-no-caddy.conf`** — ingestor + server
only
### Updated entrypoint (`docker/entrypoint-go.sh`)
Handles all 4 combinations:
| DISABLE_MOSQUITTO | DISABLE_CADDY | Config used |
|---|---|---|
| false | false | `supervisord.conf` (default) |
| true | false | `supervisord-no-mosquitto.conf` |
| false | true | `supervisord-no-caddy.conf` |
| true | true | `supervisord-no-mosquitto-no-caddy.conf` |
### Dockerfiles
Added COPY lines for the new configs in both `Dockerfile` and
`Dockerfile.go`.
## Testing
```bash
# Verify correct config selection
docker run -e DISABLE_CADDY=true corescope
# Should log: [config] Caddy reverse proxy disabled (DISABLE_CADDY=true)
docker run -e DISABLE_CADDY=true -e DISABLE_MOSQUITTO=true corescope
# Should log both disabled messages
```
Fixes#629
Co-authored-by: you <you@example.com>
## Summary
Fixes#450 — staging deployment flaky due to container not shutting down
cleanly.
## Root Causes
1. **Server never closed DB on shutdown** — SQLite WAL lock held
indefinitely, blocking new container startup
2. **`httpServer.Close()` instead of `Shutdown()`** — abruptly kills
connections instead of draining them
3. **No `stop_grace_period` in compose configs** — Docker sends SIGTERM
then immediately SIGKILL (default 10s is often not enough for WAL
checkpoint)
4. **Supervisor didn't forward SIGTERM** — missing
`stopsignal`/`stopwaitsecs` meant Go processes got SIGKILL instead of
graceful shutdown
5. **Deploy scripts used default `docker stop` timeout** — only 10s
grace period
## Changes
### Go Server (`cmd/server/`)
- **Graceful HTTP shutdown**: `httpServer.Shutdown(ctx)` with 15s
context timeout — drains in-flight requests before closing
- **WebSocket cleanup**: New `Hub.Close()` method sends `CloseGoingAway`
frames to all connected clients
- **DB close on shutdown**: Explicitly closes DB after HTTP server stops
(was never closed before)
- **WAL checkpoint**: `PRAGMA wal_checkpoint(TRUNCATE)` before DB close
— flushes WAL to main DB file and removes WAL/SHM lock files
### Go Ingestor (`cmd/ingestor/`)
- **WAL checkpoint on shutdown**: New `Store.Checkpoint()` method,
called before `Close()`
- **Longer MQTT disconnect timeout**: 5s (was 1s) to allow in-flight
messages to drain
### Docker Compose (all 4 variants)
- Added `stop_grace_period: 30s` and `stop_signal: SIGTERM`
### Supervisor Configs (both variants)
- Added `stopsignal=TERM` and `stopwaitsecs=20` to server and ingestor
programs
### Deploy Scripts
- `deploy-staging.sh`: `docker stop -t 30` with explicit grace period
- `deploy-live.sh`: `docker stop -t 30` with explicit grace period
## Shutdown Sequence (after fix)
1. Docker sends SIGTERM to supervisord (PID 1)
2. Supervisord forwards SIGTERM to server + ingestor (waits up to 20s
each)
3. Server: stops poller → drains HTTP (15s) → closes WS clients →
checkpoints WAL → closes DB
4. Ingestor: stops tickers → disconnects MQTT (5s) → checkpoints WAL →
closes DB
5. Docker waits up to 30s total before SIGKILL
## Tests
All existing tests pass:
- `cd cmd/server && go test ./...` ✅
- `cd cmd/ingestor && go test ./...` ✅
---------
Co-authored-by: you <you@example.com>
Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com>
## Summary
- add `DISABLE_MOSQUITTO` support in container startup by switching
supervisord config when disabled
- add a no-mosquitto supervisord config
(`docker/supervisord-go-no-mosquitto.conf`)
- fix Compose port mapping regression so host ports map to fixed
internal listener ports (`80`, `443`, `1883`)
- add compose variants without MQTT port publishing
(`docker-compose.no-mosquitto.yml`,
`docker-compose.staging.no-mosquitto.yml`)
- update `manage.sh` setup flow to ask `Use built-in MQTT broker?
[Y/n]`, skip MQTT port prompt when disabled, persist
`DISABLE_MOSQUITTO`, and use no-mosquitto compose files when
starting/stopping/restarting
- align `.env.example` staging keys with compose
(`STAGING_GO_HTTP_PORT`, `STAGING_GO_MQTT_PORT`)
- fix staging Caddyfile generation to use `STAGING_GO_HTTP_PORT`
- fix `.env.example` staging default comments to match actual values
(82/1885)
## Validation performed
- ✅ `bash -n manage.sh` passes.
- ✅ With `DISABLE_MOSQUITTO=true`, no-mosquitto compose overrides are
selected, Mosquitto is not started, and MQTT port is not published.
- ✅ With `DISABLE_MOSQUITTO=false`, standard compose files are used,
Mosquitto starts, and MQTT port mapping is present.
- ℹ️ Runtime Docker validation requires a running Docker host.
Fixes#267
---------
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes the separate config.json file bind mount from both compose
files. The data directory mount already covers it, and the Go server
searches /app/data/config.json via LoadConfig.
- Entrypoint symlinks /app/data/config.json for ingestor compatibility
- manage.sh setup creates config in data dir, prompts admin if missing
- manage.sh start checks config exists before starting, offers to create
- deploy.yml simplified — no more sudo rm or directory cleanup
- Backup/restore updated to use data dir path
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes the separate config.json file bind mount from both compose
files. The data directory mount already covers it, and the Go server
searches /app/data/config.json via LoadConfig.
- Entrypoint symlinks /app/data/config.json for ingestor compatibility
- manage.sh setup creates config in data dir, prompts admin if missing
- manage.sh start checks config exists before starting, offers to create
- deploy.yml simplified — no more sudo rm or directory cleanup
- Backup/restore updated to use data dir path
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root causes from CI logs:
1. 'read /app/config.json: is a directory' — Docker creates a directory
when bind-mounting a non-existent file. The entrypoint now detects
and removes directory config.json before falling back to example.
2. 'unable to open database file: out of memory (14)' — old container
(3GB) not fully exited when new one starts. Deploy now uses
'docker compose down' with timeout and waits for memory reclaim.
3. Supervisor gave up after 3 fast retries (FATAL in ~6s). Increased
startretries to 10 and startsecs to 2 for server and ingestor.
Additional:
- Deploy step ensures staging config.json exists before starting
- Healthcheck: added start_period=60s, increased timeout and retries
- No longer uses manage.sh (CI working dir != repo checkout dir)
- Multi-stage Dockerfile builds Go server + ingestor (pure Go SQLite, no CGO)
- supervisord-go.conf runs meshcore-server + meshcore-ingestor + mosquitto + caddy
- staging-go compose service on port 81/1884 with staging-go profile
- Identical volume/config structure to Node.js deployment
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Milestone 1 of #132. Adds docker-compose.yml with prod + staging services,
.env.example for port/data configuration, and Caddyfile.staging for HTTP-only
staging proxy. No changes to Dockerfile or server.js — same image, different
config.
Fixes#132 (partially)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- entrypoint copies example config to /app/data/config.json on first run
- symlinks /app/config.json → /app/data/config.json so app code unchanged
- theme.json also symlinked from /app/data/ if present
- config persists across container rebuilds without extra bind mounts
- updated README with new config/theme instructions
