Commit Graph

2 Commits

Author SHA1 Message Date
Kpa-clawbot 750b8742a7 fix(staging-compose): decouple in-container mosquitto from standalone broker (#1813)
Red commit: `3898dbc5` (verified locally — CI run URL pending)

## Problem

A standalone `mqtt-broker` container (`eclipse-mosquitto:2`) was
provisioned out-of-band on the staging VM. It now owns MQTT, is attached
to external docker network `meshcore-net`, and binds host port `8883`.
The current `docker-compose.staging.yml` still:

- Publishes `1883:1883` on the host (dead weight; conflicts the moment
the broker moves to that port).
- Defaults `DISABLE_MOSQUITTO=false`, so the in-container mosquitto
burns RAM and briefly contests the `mqtt-broker` docker DNS name on cold
start.
- Doesn't join `meshcore-net`, so the ingestor can't resolve
`mqtt-broker:1883` via docker DNS without manual surgery.

## Fix (`docker-compose.staging.yml` only)

1. Remove the `1883:1883` host port publish from `staging-go`.
2. Flip `DISABLE_MOSQUITTO` default from `false` to `true`. Operators
can opt back in with the env var.
3. Attach `staging-go` to both `default` and `meshcore-net`; declare
`meshcore-net` as `external: true` so the file never tries to
create/destroy operator state.

Healthcheck and Caddy/443 plumbing untouched (out of scope).

## Test added (TDD framing: Option A — Go shape-asserts)

`cmd/server/staging_compose_broker_test.go:1` adds four regex-based
assertions on the compose file shape:

- staging-go does **not** bind port `1883` in ANY form (quoted/unquoted
short form, or long-form `target: 1883` / `published: 1883`).
- `DISABLE_MOSQUITTO` uses the interpolated default form
`${DISABLE_MOSQUITTO:-true}` (preserves operator override). Bare literal
`true`, or a later `=false` override in the same env block, is rejected.
- Top-level `networks:` declares `meshcore-net` as `external: true`.
- `staging-go` attaches to `meshcore-net` via a real
`services.staging-go.networks:` sub-key (comment-stripped so an
in-comment example can't masquerade).

Regex (not YAML byte-equality) so cosmetic edits don't break the guard.
No new go module deps. Red commit `3898dbc5` fails all 4 assertions on
master. Green commit `38297ff4` makes them pass. Round-1 hardening
commit `9f7155e2` tightens the regexes (per adversarial + kent-beck
must-fixes) and was verified against master's YAML shape — all 4 tests
fail on `origin/master`'s compose, pass on branch, proving the tightened
regexes still gate a real regression.

## Risk

Low, with one intentional semantic change.

- **Semantic change (v3.7+):** `DISABLE_MOSQUITTO` in
`docker-compose.staging.yml` now defaults to `true`. This is a
**deliberate flip** — the standalone `mqtt-broker` container is now
authoritative on the staging host, and running the in-container
mosquitto alongside it wastes RAM and races the docker DNS name
`mqtt-broker` on cold start. Operators who want the pre-v3.7 shape
(in-container mosquitto + host-published `1883`) must explicitly opt
back in via env override AND re-add the `1883:1883` port mapping
(concrete snippet is inline in the compose file and in `DEPLOY.md` under
"Standalone MQTT broker (staging)"). This intent is called out in a
`SEMANTIC CHANGE (v3.7+)` header comment at the top of
`docker-compose.staging.yml`.
- **Deploy prereq:** the external `meshcore-net` docker network MUST
already exist on the host before `docker compose up`. If it doesn't,
compose refuses to start `staging-go`. This is documented inline in the
compose file (with the `docker network create meshcore-net` one-liner)
and in `DEPLOY.md`.
- **Only takes effect where the standalone broker is deployed** — which
it already is on staging today. The legacy `DISABLE_MOSQUITTO=false`
path remains reachable via env override; the ingestor's upstream config
is untouched.

Partial fix — no tracking issue; follow-up to operator-side broker
provisioning.

---------

Co-authored-by: corescope-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-06-30 18:21:21 -07:00
Kpa-clawbot f59b4629b0 feat: publish Docker images to GHCR + simplified deploy (#610) (#627)
## Summary

Implements M1-M2 of the deployment simplification spec (#610). Adds
pre-built multi-arch Docker images published to GHCR, plus a simplified
deploy experience for operators.

**Spec:**
[docs/specs/deployment-simplification.md](https://github.com/Kpa-clawbot/CoreScope/blob/master/docs/specs/deployment-simplification.md)

## Files Added (no existing files modified)

### 1. `.github/workflows/publish.yml`
Multi-arch Docker publish workflow:
- Triggers on `v*` tags (releases) → produces `vX.Y.Z`, `vX.Y`, `vX`,
`latest`
- Triggers on master push → produces `edge` (unstable)
- `workflow_dispatch` for manual runs
- QEMU + buildx for `linux/amd64` + `linux/arm64`
- GHCR auth via `GITHUB_TOKEN`
- GHA layer caching for fast rebuilds

### 2. `docker-compose.example.yml`
20-line compose file that pulls from GHCR (no local build required):
- Env var overrides: `HTTP_PORT`, `DATA_DIR`, `DISABLE_CADDY`,
`DISABLE_MOSQUITTO`
- Health check included
- Volume mount for data persistence

### 3. `DEPLOY.md`
Operator documentation:
- One-line `docker run` deploy
- Tag reference (pinned vs latest vs edge)
- Environment variables table
- Update path (`docker compose pull && docker compose up -d`)
- TLS options (Caddy auto-TLS vs reverse proxy)
- **Migration guide for existing manage.sh users** — both paths
documented with command equivalency table

## Review Status

-  Self-review: Actions syntax, GHCR auth, multi-arch, tag strategy,
security — all verified
-  Torvalds: Deploy UX is clean, one-liner works, right level of
simplicity
-  BUILD_TIME fixed: uses `date` command instead of fragile
`head_commit.timestamp`
-  Migration guide added for existing manage.sh admins
- ⚠️ `DISABLE_CADDY` env var documented but not implemented in
entrypoint — pre-existing bug, filed as #629

Fixes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 14:33:57 -07:00