Files
ukmesh/docker-compose.ci.yml
hermes-gadgetandgadgethd ab6372b8a9 fix(ci): pin gitleaks-action to valid commit after upstream force-push (#63)
* fix(ci): pin gitleaks-action to valid commit after upstream force-push

Upstream deleted dcedce43 (force-push), so every ci.yml run fails at
workflow-parse with 0 jobs. The dependabot actions-group bump (#30)
contains this fix but cannot merge whole: its docker/* actions require
node24, which GitHub runners do not support yet. Pin gitleaks alone
(ff98106e is node20).

* fix(ci): downgrade v7 actions to node20-compatible versions

actions/checkout@v7, setup-node@v7, setup-python@v7 require node24,
which GitHub-hosted runners do not support yet — every ci.yml run has
failed at workflow-parse (0 jobs) since the Aug 3 v7 bump. Pin to the
latest node20 versions (checkout v4, setup-node v4, setup-python v5).
Also pins gitleaks-action to a valid commit (upstream force-pushed away
dcedce43).

* ci: noop retrigger

* fix(ci): escape literal ${{ in bash string so GitHub parser accepts workflow

The 'Validate tracked build and Compose inventory' step matched literal
${{ inside run-block strings, which GitHub's expression parser reads as
the start of an expression (${{'* is invalid) — the workflow fails at
parse time with 0 jobs. Split the literal as '$''{{' (bash concatenates
adjacent quoted strings at runtime; the source no longer contains a
contiguous ${{).

* fix(ci): resolve all three pre-existing ukmesh CI failures

Backend (packetBatchWriteCoalesce):
- 045: restore nodes_public_visibility_generation trigger — 015 is
  superseded by 044 on fresh DBs so the trigger was never created,
  leaving generation/visibility_generation out of sync and public
  packet reads returning empty.
- 046: restore packet privacy classification — the 042 fence design
  assumes sync_private_node_prefixes rewrites packets on privacy
  change, but 026's version only maintains prefixes. Node flips to
  private left old packets visible; direct SQL inserts were never
  classified. Restore the packet rewrite in the sync trigger and add
  a BEFORE INSERT trigger mirroring the batch path's is_private /
  visibility_ok computation.
- packetBatch.integration.test.ts: exclude the prefix-cache refresh
  query from statementCount (matches unit-test convention).

Frontend e2e:
- public.spec.ts: assert the TopologyMap component's actual labels
  ('Geographic repeater topology map', '2 mapped repeaters · 2
  observed relationships') instead of the pre-map SVG graph labels.

Workers (pip install):
- viewshed-worker: bump numpy 1.26.4→2.3.5, scipy 1.13.1→1.16.3,
  psycopg2-binary 2.9.10→2.9.11 (cp314 wheels); the base gdal image
  ships Python 3.14 so the old pins had no wheels. Inherit shapely
  from the image's python3-shapely apt package (no cp314 wheel
  exists; source builds are GEOS 3.14-incompatible).

* fix(ci): restore data-plane services for compose validation and smoke test

Commit 8c5e1c8 split the long-lived data-plane services (timescaledb,
mosquitto, redis, mosquitto-reloader) into the external meshcore-infra
project on the live host, but the Workers-and-Compose CI job still
asserted their presence in the app compose project (max_worker_processes
check, inventory check, smoke-test execs). CI had been broken since
Aug 3 so the mismatch was never caught.

Add docker-compose.ci.yml — a CI-only overlay restoring the four
data-plane services from their pre-split definitions — and point every
compose invocation in the Workers-and-Compose job at
'-f docker-compose.yml -f docker-compose.ci.yml'.

Validated locally: merged config parses, the jq assertions and the full
inventory (db-migrate backend app-ukmesh website-ukmesh website-dev
mesh-health-check mosquitto-reloader link-worker link-backfill-worker
hopreach timescaledb mosquitto) all pass.

* fix(ci): include dev profile in compose inventory check

website-dev is a dev-profile service; the inventory gate compared the
base-config service list, so the merged stack never matched. This gate
has been red since the workflow was first written (CI parse-broken from
Aug 3 until the gitleaks pin landed).

* fix(ci): db-migrate must wait for timescaledb health

The Aug 9 data-plane split removed db-migrate's depends_on along with
the timescaledb service definition, so the migration runner raced the
fresh database init in the empty-volume smoke test and failed on the
not-yet-created base schema. Restore the pre-split condition:
timescaledb service_healthy.

* fix(ci): restore internal TCP MQTT listener for smoke test + exporter

Commit 40e843f dropped the 1883 TCP listener, breaking the CI smoke
(mosquitto_pub targets 1883) and the Prometheus mosquitto exporter.
The listener is internal-only: compose never publishes 1883 on the
host, and the password/ACL policy still applies.

---------

Co-authored-by: gadgethd <111318106+gadgethd@users.noreply.github.com>
2026-08-11 19:55:29 +01:00

120 lines
3.9 KiB
YAML

# CI-only data-plane overlay.
#
# The live host runs the long-lived data-plane services (timescaledb, mosquitto,
# redis, mosquitto-reloader) as a separate meshcore-infra project on the shared
# bridge network. GitHub-hosted runners have no such project, so the compose
# validation and empty-volume smoke test need the definitions here.
#
# Never use this file for a real deployment; it exists purely so `ci.yml` can
# merge a self-contained stack with `docker compose -f docker-compose.yml
# -f docker-compose.ci.yml`.
services:
timescaledb:
image: timescale/timescaledb@sha256:22e8a5ae7aef121d1537afe946dd7cc5deeeb63ab36ce19849d671bd3b663509
logging: &json-log-limits
options:
max-size: "10m"
max-file: "3"
mem_limit: "4g"
memswap_limit: "4g"
stop_grace_period: 2m
# The image currently tunes timescaledb.max_background_workers to 16.
# Leave headroom for the Timescale launcher/schedulers, logical replication,
# and a policy worker so scheduled jobs do not fail their first launch.
command:
- postgres
- -c
- max_connections=300
- -c
- max_worker_processes=${POSTGRES_MAX_WORKER_PROCESSES:-24}
- -c
- max_parallel_workers_per_gather=0
environment:
POSTGRES_DB: ${POSTGRES_DB:-meshcore}
POSTGRES_USER: ${POSTGRES_USER:-meshcore}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD required}
volumes:
- ./backend/src/db/schema/base.sql:/docker-entrypoint-initdb.d/001_schema.sql:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-meshcore} -d ${POSTGRES_DB:-meshcore}"]
interval: 30s
timeout: 10s
retries: 10
start_period: 30s
mosquitto:
image: eclipse-mosquitto@sha256:9cfdd46ad59f3e3e5f592f6baf57ab23e1ad00605509d0f5c1e9b179c5314d87
logging: *json-log-limits
mem_limit: "128m"
memswap_limit: "128m"
healthcheck:
test: ["CMD-SHELL", "pgrep mosquitto >/dev/null && nc -z 127.0.0.1 9001"]
interval: 30s
timeout: 10s
retries: 5
start_period: 10s
volumes:
- ./mosquitto:/mosquitto/config
- mosquitto_data:/mosquitto/data
- mosquitto_log:/mosquitto/log
mosquitto-reloader:
image: ${MOSQUITTO_RELOADER_IMAGE:-meshcore-analytics-mosquitto-reloader:local}
build:
context: .
dockerfile: Dockerfile.mosquitto-reloader
args:
SOURCE_REVISION: ${SOURCE_REVISION:-local}
restart: "no"
logging: *json-log-limits
mem_limit: "64m"
memswap_limit: "64m"
pid: "service:mosquitto"
# Match the broker UID inside the shared PID namespace so SIGHUP can be
# delivered without granting CAP_KILL or root.
user: "1883:1883"
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
read_only: true
environment:
OWNER_ACL_RELOAD_TOKEN: ${JWT_SECRET:?JWT_SECRET required}
MOSQUITTO_LOG_PATH: /mosquitto/log/mosquitto.log
volumes:
- mosquitto_log:/mosquitto/log
depends_on:
mosquitto:
condition: service_healthy
healthcheck:
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/healthz', timeout=2).read()"]
interval: 30s
timeout: 5s
retries: 5
redis:
image: redis@sha256:8b81dd37ff027bec4e516d41acfbe9fe2460070dc6d4a4570a2ac5b9d59df065
logging: *json-log-limits
mem_limit: "512m"
memswap_limit: "512m"
command:
- redis-server
- --requirepass
- ${REDIS_PASSWORD:?REDIS_PASSWORD required}
- --maxmemory
- 448mb
- --maxmemory-policy
- noeviction
- --appendonly
- "yes"
- --appendfsync
- everysec
environment:
REDIS_PASSWORD: ${REDIS_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "redis-cli -a $REDIS_PASSWORD --no-auth-warning ping"]
interval: 30s
timeout: 10s
retries: 5