## Summary
Fixes#485 — the app version was derived from `package.json` via
Node.js, which is a meaningless artifact for this Go project. This
caused version mismatches (e.g., v3.3.0 release showing "3.2.0") when
someone forgot to bump `package.json`.
## Changes
### `manage.sh`
- **Line 43**: Replace `node -p "require('./package.json').version"`
with `git describe --tags --match "v*"` — version is now derived
automatically from git tags
- **Line 515**: Add `--force` to `git fetch origin --tags` in setup
command
- **Line 1320**: Add `--force` to `git fetch origin --tags` in update
command — prevents "would clobber existing tag" errors when tags are
moved
### `package.json`
- Version field set to `0.0.0-use-git-tags` to make it clear this is not
the source of truth. File kept because npm scripts and devDependencies
are still used for testing.
## How it works
`git describe --tags --match "v*"` produces:
- `v3.3.0` — when on an exact tag
- `v3.3.0-3-gabcdef1` — when 3 commits after a tag (useful for
debugging)
- Falls back to `unknown` if no tags exist
## Testing
- All Go tests pass (`cmd/server`, `cmd/ingestor`)
- All frontend unit tests pass (254/254)
- No changes to application logic — only build-time version derivation
Co-authored-by: you <you@example.com>
## Summary
Fixes#466 — staging config was not refreshed from prod due to a stale
`-nt` timestamp guard.
## Root Cause
`prepare_staging_config()` only copied prod config when staging was
missing or prod was newer by mtime. However, the `sed -i` that applies
the STAGING siteName updated staging's mtime, making it appear newer
than prod. Subsequent runs skipped the copy entirely.
## Changes
- **`manage.sh`**: Removed the `-nt` timestamp conditional in
`prepare_staging_config()`. Staging config is now always copied fresh
from prod with the STAGING siteName applied.
Note: `prepare_staging_db()` already copies unconditionally — no change
needed there.
Co-authored-by: you <you@example.com>
## Summary
`manage.sh update` now supports pinning to specific release tags instead
of always pulling tip of master.
Fixes#455
## Changes
### `cmd_update` — accepts optional version argument
- **No argument**: fetches tags, checks out latest release tag (`git tag
-l 'v*' --sort=-v:refname | head -1`)
- **`latest`**: explicit opt-in to tip of master (bleeding edge)
- **Specific tag** (e.g. `v3.1.0`): checks out that exact tag, with
error message + available tags if not found
### `cmd_setup` — defaults to latest tag
- After Docker check, fetches tags and pins to latest release tag
- Skips if already on the latest tag
- Uses state tracking (`version_pin`) so re-runs don't repeat
### `cmd_status` — shows version
- Displays current version (exact tag name or short commit hash) at the
top of status output
### Help text
- Updated to reflect new `update [version]` syntax
## Usage
```bash
./manage.sh update # checkout latest release tag (e.g. v3.2.0)
./manage.sh update v3.1.0 # pin to specific version
./manage.sh update latest # explicit tip of master (bleeding edge)
./manage.sh status # now shows "Version: v3.2.0"
```
## Testing
- `bash -n manage.sh` passes (syntax valid)
- Logic follows existing patterns (git fetch, checkout, rebuild,
restart)
---------
Co-authored-by: you <you@example.com>
## Summary
- fix safe .env parser in manage.sh to expand a leading ~ before export
- ensure setup-time PROD_DATA_DIR read from .env also expands ~
- keep behavior unchanged for non-tilde values
## Why
xport "=" does not perform tilde expansion, so values like
PROD_DATA_DIR=~/meshcore-data stayed literal and broke path-based
operations in manage.sh.
## Validation
- ash -n manage.sh
- manual reasoning: PROD_DATA_DIR=~/meshcore-data now resolves to
$HOME/meshcore-data
## Notes
Docker Compose handling is unchanged (compose already expands ~); this
PR only fixes manage.sh runtime parsing.
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
## Summary
- add `DISABLE_MOSQUITTO` support in container startup by switching
supervisord config when disabled
- add a no-mosquitto supervisord config
(`docker/supervisord-go-no-mosquitto.conf`)
- fix Compose port mapping regression so host ports map to fixed
internal listener ports (`80`, `443`, `1883`)
- add compose variants without MQTT port publishing
(`docker-compose.no-mosquitto.yml`,
`docker-compose.staging.no-mosquitto.yml`)
- update `manage.sh` setup flow to ask `Use built-in MQTT broker?
[Y/n]`, skip MQTT port prompt when disabled, persist
`DISABLE_MOSQUITTO`, and use no-mosquitto compose files when
starting/stopping/restarting
- align `.env.example` staging keys with compose
(`STAGING_GO_HTTP_PORT`, `STAGING_GO_MQTT_PORT`)
- fix staging Caddyfile generation to use `STAGING_GO_HTTP_PORT`
- fix `.env.example` staging default comments to match actual values
(82/1885)
## Validation performed
- ✅ `bash -n manage.sh` passes.
- ✅ With `DISABLE_MOSQUITTO=true`, no-mosquitto compose overrides are
selected, Mosquitto is not started, and MQTT port is not published.
- ✅ With `DISABLE_MOSQUITTO=false`, standard compose files are used,
Mosquitto starts, and MQTT port mapping is present.
- ℹ️ Runtime Docker validation requires a running Docker host.
Fixes#267
---------
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes the separate config.json file bind mount from both compose
files. The data directory mount already covers it, and the Go server
searches /app/data/config.json via LoadConfig.
- Entrypoint symlinks /app/data/config.json for ingestor compatibility
- manage.sh setup creates config in data dir, prompts admin if missing
- manage.sh start checks config exists before starting, offers to create
- deploy.yml simplified — no more sudo rm or directory cleanup
- Backup/restore updated to use data dir path
Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes the separate config.json file bind mount from both compose
files. The data directory mount already covers it, and the Go server
searches /app/data/config.json via LoadConfig.
- Entrypoint symlinks /app/data/config.json for ingestor compatibility
- manage.sh setup creates config in data dir, prompts admin if missing
- manage.sh start checks config exists before starting, offers to create
- deploy.yml simplified — no more sudo rm or directory cleanup
- Backup/restore updated to use data dir path
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root cause: on the 8GB VM, both prod (~2.5GB) and staging (~2GB) containers
run simultaneously. During deploy, manage.sh would rm the old staging container
and immediately start a new one. The old container's memory wasn't reclaimed
yet, so the new one got 'unable to open database file: out of memory (14)'
from SQLite and both corescope-server and corescope-ingestor entered FATAL.
Fix:
- manage.sh restart staging: wait up to 15s for old container to fully exit,
plus 3s for OS memory reclamation before starting new container
- manage.sh restart staging: verify config.json exists before starting
- docker-compose.staging.yml: add deploy.resources.limits.memory=3g to
prevent staging from consuming unbounded memory
Version/Commit/BuildTime now populated from package.json, git, and
date. Exported as env vars so docker compose build picks them up.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add preflight check for 'docker compose' in manage.sh (catches plugin missing)
- Document named Caddy volumes as cert storage, not user data
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove all legacy docker run code paths. manage.sh is now a pure
docker compose wrapper with no dual-mode branching.
Removed:
- COMPOSE_MODE flag and all if/else branches
- get_docker_run_args(), get_data_mount_args(), recreate_container()
- get_required_ports(), get_current_ports(), check_port_match()
- CONTAINER_NAME, DATA_VOLUME, CADDY_VOLUME variables
- All direct docker run/stop/start/rm invocations
All commands now delegate to docker compose:
- start → docker compose up -d prod
- stop → docker compose down / docker compose stop
- restart → docker compose up -d --force-recreate
- update → docker compose build prod + up -d --force-recreate
- reset → docker compose down --rmi local
- backup/restore use bind mount path from .env (PROD_DATA_DIR)
- verify_health, mqtt-test, status all use corescope-prod
Net result: -248 lines, zero dual-mode logic, identical behavior
to running docker compose directly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace all hardcoded \C:\Users\KpaBap/meshcore-data with \ variable
- \ resolves from \ in .env or defaults to ~/meshcore-data
- Updated get_data_mount_args(), cmd_backup(), cmd_restore(), cmd_reset()
- Enhanced .env.example with detailed comments for each variable
- Both docker compose and manage.sh now read same .env file
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PROBLEM:
manage.sh was using named Docker volumes (meshcore-data) as the default,
which hides the database and theme files inside Docker's internal storage.
Users couldn't find their DB on the filesystem for backups or inspection.
The function get_data_mount_args() had conditional logic that only used
bind mounts IF it detected an existing ~/meshcore-data with a DB file.
For new installs, it fell through to the named volume — silently hiding
all data in /var/lib/docker/volumes/.
FIXES:
1. get_data_mount_args() — Always use bind mount to ~/meshcore-data
- Creates the directory if it doesn't exist
- Removes all conditional logic and the named volume fallback
2. cmd_backup() — Use direct path C:\Users\KpaBap/meshcore-data/meshcore.db
- No longer tries to inspect the named volume
- Consistent with the bind mount approach
3. cmd_restore() — Use direct path for restore operations
- Ensures directory exists before restoring files
- No fallback to docker cp
4. cmd_reset() — Updated message to reflect bind mount location
- Changed from 'docker volume rm' to '~/meshcore-data (not removed)'
5. docker-compose.yml — Added documentation comment
- Clarifies that bind mounts are intentional, not named volumes
- Ensures future changes maintain this pattern
VALIDATION:
- docker-compose.yml already used bind mounts correctly (\)
- Legacy 'docker run' mode now matches compose behavior
- All backup/restore operations reference the same bind mount path
DATABASE LOCATION:
- Always: ~/meshcore-data/meshcore.db
- Never: Hidden in Docker's volume storage
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Requested-by: Kpa-clawbot
Build args ensure version badge shows correctly. Health timeout
bumped from 20s to 90s for Go store loading time.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Changed all 'Node.js' references to generic 'Server' in:
- verify_health() - health check messages
- show_container_status() - stats display comment
- cmd_status() - service health output
The Go backend runs behind Caddy just like the Node version did,
so the health checks via docker exec localhost:3000 remain correct.
Only the messaging needed updating.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Milestone 2 of #132. Updates manage.sh to use docker-compose.yml when present:
- start/start --with-staging (copies prod DB + config to staging)
- stop [prod|staging|all]
- status shows both containers
- logs [prod|staging]
- promote (backup prod, restart with latest image)
Legacy single-container mode preserved as fallback.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Config protection: never overwrite existing config.json, warn on placeholder values
- Port mapping validation: start/restart check if container ports match Caddyfile,
offer to recreate if mismatched
- Data volume detection: detect existing DB in $HOME/meshcore-data/ or ./data/,
use bind mount instead of named volume (never hardcodes paths)
- Real health verification: wait for /api/stats response, check HTTPS if domain
configured, scan logs for MQTT errors
- Restart recreates container with correct ports when mappings changed
- Status command: shows MQTT errors, port mismatch warnings
- Update command: uses shared recreate_container helper
- Extracted helpers: get_data_mount_args, get_required_ports, check_port_match,
recreate_container, verify_health, check_config_placeholders
Backup creates a directory with meshcore.db + config.json + Caddyfile +
theme.json (if present). Restore accepts a backup directory (restores
all files) or a single .db file (DB only). Lists available backups
when called without arguments.
- State tracking (.setup-state) — resume from where you left off
- Every step checks what's already done and skips it
- Safe to Ctrl+C and re-run at any point
- Auto-generates random API key on first config creation
- HTTPS choice menu: domain (auto), Cloudflare/proxy (HTTP), or later
- DNS validation with IP comparison
- Port 80 conflict detection
- Status shows packet/node counts, DB size, per-service health
- Backup auto-creates ./backups/ dir, restore backs up current first
- Reset command (keeps data + config, removes container + image)
- .setup-state in .gitignore
Kpa-clawbot