Commit Graph
3 Commits
Author SHA1 Message Date
n30nex 643a53e98d fix(operators): refresh update metadata and wait for readiness (#2285)
Red commit: `7277ca23` ([failing
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40190626951)).

Fixes #2229. Management startup now defines logging helpers before
normalizing CRLF environment files. Updates refresh version, revision
and build time after checking out the target. Deployment verification
and all five Compose healthchecks use `/api/healthz`, so an available
statistics endpoint cannot report an unfinished startup as healthy.

- The actual-script regression passes 15 checks: LF/CRLF/no environment
file, repeat startup, all update targets, failed checkout,
delayed/failed readiness, HTTPS readiness and unchanged informational
statistics.
- RED CI reproduced all 11 intended failures; GREEN `7ff9050b` passes
the full standalone frontend suite and shell syntax checks.
- Browser verified: an isolated Go fixture at http://127.0.0.1:18851
renders Home and returns ready=true from `/api/healthz`.
- Git, Docker and network boundaries are mocked in management tests. No
real container update was performed locally; CI validates the
Compose/build path.
- No new configuration, dependencies, schema changes or
packet-processing work.

Privacy, whitespace and scope checks passed. The external home-directory
preflight bundle is unavailable; applicable checks were performed
locally. Independent review found no outstanding fixes. [Final
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40191647297)
passed on `7ff9050b`, including all three browser shards and both
architecture builds.
2026-10-26 15:50:36 +01:00
750b8742a7 fix(staging-compose): decouple in-container mosquitto from standalone broker (#1813)
Red commit: `3898dbc5` (verified locally — CI run URL pending)

## Problem

A standalone `mqtt-broker` container (`eclipse-mosquitto:2`) was
provisioned out-of-band on the staging VM. It now owns MQTT, is attached
to external docker network `meshcore-net`, and binds host port `8883`.
The current `docker-compose.staging.yml` still:

- Publishes `1883:1883` on the host (dead weight; conflicts the moment
the broker moves to that port).
- Defaults `DISABLE_MOSQUITTO=false`, so the in-container mosquitto
burns RAM and briefly contests the `mqtt-broker` docker DNS name on cold
start.
- Doesn't join `meshcore-net`, so the ingestor can't resolve
`mqtt-broker:1883` via docker DNS without manual surgery.

## Fix (`docker-compose.staging.yml` only)

1. Remove the `1883:1883` host port publish from `staging-go`.
2. Flip `DISABLE_MOSQUITTO` default from `false` to `true`. Operators
can opt back in with the env var.
3. Attach `staging-go` to both `default` and `meshcore-net`; declare
`meshcore-net` as `external: true` so the file never tries to
create/destroy operator state.

Healthcheck and Caddy/443 plumbing untouched (out of scope).

## Test added (TDD framing: Option A — Go shape-asserts)

`cmd/server/staging_compose_broker_test.go:1` adds four regex-based
assertions on the compose file shape:

- staging-go does **not** bind port `1883` in ANY form (quoted/unquoted
short form, or long-form `target: 1883` / `published: 1883`).
- `DISABLE_MOSQUITTO` uses the interpolated default form
`${DISABLE_MOSQUITTO:-true}` (preserves operator override). Bare literal
`true`, or a later `=false` override in the same env block, is rejected.
- Top-level `networks:` declares `meshcore-net` as `external: true`.
- `staging-go` attaches to `meshcore-net` via a real
`services.staging-go.networks:` sub-key (comment-stripped so an
in-comment example can't masquerade).

Regex (not YAML byte-equality) so cosmetic edits don't break the guard.
No new go module deps. Red commit `3898dbc5` fails all 4 assertions on
master. Green commit `38297ff4` makes them pass. Round-1 hardening
commit `9f7155e2` tightens the regexes (per adversarial + kent-beck
must-fixes) and was verified against master's YAML shape — all 4 tests
fail on `origin/master`'s compose, pass on branch, proving the tightened
regexes still gate a real regression.

## Risk

Low, with one intentional semantic change.

- **Semantic change (v3.7+):** `DISABLE_MOSQUITTO` in
`docker-compose.staging.yml` now defaults to `true`. This is a
**deliberate flip** — the standalone `mqtt-broker` container is now
authoritative on the staging host, and running the in-container
mosquitto alongside it wastes RAM and races the docker DNS name
`mqtt-broker` on cold start. Operators who want the pre-v3.7 shape
(in-container mosquitto + host-published `1883`) must explicitly opt
back in via env override AND re-add the `1883:1883` port mapping
(concrete snippet is inline in the compose file and in `DEPLOY.md` under
"Standalone MQTT broker (staging)"). This intent is called out in a
`SEMANTIC CHANGE (v3.7+)` header comment at the top of
`docker-compose.staging.yml`.
- **Deploy prereq:** the external `meshcore-net` docker network MUST
already exist on the host before `docker compose up`. If it doesn't,
compose refuses to start `staging-go`. This is documented inline in the
compose file (with the `docker network create meshcore-net` one-liner)
and in `DEPLOY.md`.
- **Only takes effect where the standalone broker is deployed** — which
it already is on staging today. The legacy `DISABLE_MOSQUITTO=false`
path remains reachable via env override; the ingestor's upstream config
is untouched.

Partial fix — no tracking issue; follow-up to operator-side broker
provisioning.

---------

Co-authored-by: corescope-bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-06-30 18:21:21 -07:00
Kpa-clawbotandyou f59b4629b0 feat: publish Docker images to GHCR + simplified deploy (#610) (#627)
## Summary

Implements M1-M2 of the deployment simplification spec (#610). Adds
pre-built multi-arch Docker images published to GHCR, plus a simplified
deploy experience for operators.

**Spec:**
[docs/specs/deployment-simplification.md](https://github.com/Kpa-clawbot/CoreScope/blob/master/docs/specs/deployment-simplification.md)

## Files Added (no existing files modified)

### 1. `.github/workflows/publish.yml`
Multi-arch Docker publish workflow:
- Triggers on `v*` tags (releases) → produces `vX.Y.Z`, `vX.Y`, `vX`,
`latest`
- Triggers on master push → produces `edge` (unstable)
- `workflow_dispatch` for manual runs
- QEMU + buildx for `linux/amd64` + `linux/arm64`
- GHCR auth via `GITHUB_TOKEN`
- GHA layer caching for fast rebuilds

### 2. `docker-compose.example.yml`
20-line compose file that pulls from GHCR (no local build required):
- Env var overrides: `HTTP_PORT`, `DATA_DIR`, `DISABLE_CADDY`,
`DISABLE_MOSQUITTO`
- Health check included
- Volume mount for data persistence

### 3. `DEPLOY.md`
Operator documentation:
- One-line `docker run` deploy
- Tag reference (pinned vs latest vs edge)
- Environment variables table
- Update path (`docker compose pull && docker compose up -d`)
- TLS options (Caddy auto-TLS vs reverse proxy)
- **Migration guide for existing manage.sh users** — both paths
documented with command equivalency table

## Review Status

- ✅ Self-review: Actions syntax, GHCR auth, multi-arch, tag strategy,
security — all verified
- ✅ Torvalds: Deploy UX is clean, one-liner works, right level of
simplicity
- ✅ BUILD_TIME fixed: uses `date` command instead of fragile
`head_commit.timestamp`
- ✅ Migration guide added for existing manage.sh admins
- ⚠️ `DISABLE_CADDY` env var documented but not implemented in
entrypoint — pre-existing bug, filed as #629

Fixes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 14:33:57 -07:00