Commit Graph

5 Commits

Author SHA1 Message Date
Eldoon Nemar d7cd9203ca Fixes #1165: add OSM/Stamen tile providers with per-provider Leaflet layer control. (#1533)
List of changes too long to describe, so I'll hit high level.

- Config now supports the json map tiles that were suggested by
@Kpa-clawbot.
- Leaflet map layer button appears in the top right of live.js and
map.js (because all the work was already done on live.js... Added bonus)
- Allows users to enter creds for OSM and Stamen to get enterprise
related perks, in the config file
- Added a default light map under customizer. Still suggest removing
them all together and relying on the config
- You can enable OSM and Stamen in the config without a license, but at
your own risk!!!
- Config comment explains where to register and the providers for osm,
as well as the general limits per X interval
- Updated tests (28) to address the changes made to the maps

### TDD Exemption

**Reason**: Net-new UI surfaces (per `AGENTS.md`)

This PR introduces a net-new UI surface (the multi-provider map tile
selector). Under the `AGENTS.md` exemption for net-new UI surfaces, the
absence of an initial failing (red) commit is permitted, as the UI was
built first. However, the underlying public APIs are fully covered.

The following tests serve as the first assertions for these new APIs:
- `window.MC_createLayerControl`: Asserted in `MC_createLayerControl
handles Auto mode and explicit layers correctly`
- `window.MC_setDarkTileProvider` & `window.MC_getDarkTileProvider`:
Asserted in `MC_setDarkTileProvider persists to localStorage...`
- `window.MC_setLightTileProvider` & `window.MC_getLightTileProvider`:
Asserted in `MC_setLightTileProvider persists to localStorage...`
- `window.MC_initTileRegistry`: Asserted in `MC_initTileRegistry(true)
dispatches mc-tile-provider-changed`
- `applyTileFilter`: Asserted in `applyTileFilter sets invert CSS for
inverted dark provider...`
- Cross-tab synchronization: Asserted in `Cross-tab storage event
re-dispatches mc-tile-provider-changed`
2026-06-04 06:53:30 -07:00
Kpa-clawbot 63bfa3d910 feat(security): detect CDN-fronted deployment + document bypass requirement (closes #1561) (#1564)
Closes #1561. Follow-up to #1551.

## Why

#1551 added `Cache-Control: no-store` to all `/api/*` responses. That's
sufficient for CDNs that honour origin headers (Varnish, nginx). It is
**not** sufficient for Cloudflare zones where Cache Rules / Page Rules
override origin Cache-Control.

Field evidence from the meshat.se diagnosis (2026-06-04): observers
behind Cloudflare were returning `cf-cache-status: HIT` with `age` up to
~6 hours despite the origin emitting `no-store`. The CDN was caching per
zone policy and ignoring the upstream directive — exactly the failure
mode #1551 cannot reach. The application has no way to inject CDN rules;
the only durable fix is operator-side.

This PR makes that operator step discoverable and verifiable.

## What

### Server-side detection (log-only)

`cmd/server/cdn_detection.go` adds a middleware wired into the `/api/*`
chain after `noStoreAPIMiddleware`. On the **first** request bearing any
CDN-typical header (`CF-Connecting-IP`, `CF-Ray`, `X-Forwarded-For`,
`X-Real-IP`, `Fastly-Client-IP`, `True-Client-IP`) it logs:

```
[security] WARNING: detected request via CDN (CF-Ray header present).
Ensure /api/* is bypassed in your CDN config — see docs/deployment-behind-cdn.md.
Cached API responses cause observer-flap and incorrect dashboards.
```

`sync.Once` guarantees the warning fires at most once per process boot.
The middleware never blocks, never modifies the response, never adds
headers. Detection is observational only — operators who run behind a
CDN without bypass have a real bug; the warning is appropriate.

### Operator documentation

`docs/deployment.md` gains a new **"Behind a CDN (Cloudflare, Fastly)"**
section covering:

1. Curl verification command + healthy vs unhealthy output examples
2. Cloudflare Cache Rule creation (URI Path starts-with `/api/` → Bypass
cache)
3. Legacy Page Rules equivalent
4. Fastly note
5. Re-verification
6. Meaning of the startup log warning
7. Why we can't fix this server-side

`docs/deployment-behind-cdn.md` is the canonical path the log message
references — it's a short TL;DR that links back to the full section.

### Healthcheck script

`scripts/check-cdn-bypass.sh` — POSIX sh, no dependencies beyond curl +
grep + awk. Operators run:

```sh
scripts/check-cdn-bypass.sh https://your-domain.example.com
```

Exits `0` with `OK: no CDN caching detected ...` or `1` with a precise
diagnostic naming the offending header (`cf-cache-status: HIT` or stale
`age`).

## TDD

- **Red commit `e90ccaba`** (`test(security): RED ...`) —
`cmd/server/cdn_detection_test.go` (4 Go tests + 6 subtests for each
header) and `scripts/test-check-cdn-bypass.sh` (3 shell harness cases).
Middleware stub returns `next` unchanged so tests compile and fail on
assertions, not build errors.
- **Green commit `5e6a60b5`** (`feat(security): GREEN ...`) — real
middleware, wiring in `routes.go`, healthcheck script, doc.

## Deliverables

| File | Status | Purpose |
|------|--------|---------|
| `cmd/server/cdn_detection.go` | new | middleware + sync.Once warning |
| `cmd/server/cdn_detection_test.go` | new | 4 Go tests (1 stand-alone +
1 silence + 1 once + 1 table-driven over 6 headers) |
| `cmd/server/routes.go` | modified | `r.Use(cdnDetectionMiddleware)`
after no-store |
| `docs/deployment.md` | modified | TOC entry + "Behind a CDN" section |
| `docs/deployment-behind-cdn.md` | new | canonical path referenced by
log message + script output |
| `scripts/check-cdn-bypass.sh` | new | operator-runnable healthcheck |
| `scripts/test-check-cdn-bypass.sh` | new | shell harness with fake
curl |

## What this PR explicitly does NOT do

- Does not block requests based on CDN detection (log-only).
- Does not enforce CDN bypass (impossible — operator-controlled).
- Does not spoof, strip or modify CDN headers.
- Does not add CSP / HSTS / other security headers (out of scope).
- Warning is not configurable — operators behind a CDN without bypass
have a real bug, surfacing it is correct.

## Verification

- `go test ./...` in `cmd/server/` — full suite green.
- `sh scripts/test-check-cdn-bypass.sh` — 3/3 pass.
- Preflight checklist — all 11 gates clean (PII, branch scope, red
commit, CSS vars, CSS self-fallback, LIKE-on-JSON, sync migration,
async-migration annotation, XSS sinks, img/SVG ratio, themed-img/SVG,
fixture coverage).

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: clawbot <bot@clawbot.invalid>
2026-06-04 13:14:09 +00:00
Kpa-clawbot 26c47df814 fix: entrypoint .env support + deployment docs for bare docker run (#704)
## Summary

Fixes #702 — `.env` file `DISABLE_MOSQUITTO`/`DISABLE_CADDY` ignored
when using `docker run`.

## Changes

### Entrypoint sources `/app/data/.env`
The entrypoint now sources `/app/data/.env` (if present) before the
`DISABLE_*` checks. This works regardless of how the container is
started — `docker run`, compose, or `manage.sh`.

```bash
if [ -f /app/data/.env ]; then
  set -a
  . /app/data/.env
  set +a
fi
```

### `DISABLE_CADDY` added to compose files
Both `docker-compose.yml` and `docker-compose.staging.yml` now forward
`DISABLE_CADDY` to the container environment (was missing — only
`DISABLE_MOSQUITTO` was wired).

### Deployment docs updated
- `docs/deployment.md`: bare `docker run` is now the primary/recommended
approach with a full parameter reference table
- Documents the `/app/data/.env` convenience feature
- Compose and `manage.sh` marked as legacy alternatives
- `DISABLE_CADDY` added to the environment variable reference

### README quick start updated
Shows the full `docker run` command with `--restart`, ports, and
volumes. Includes HTTPS variant. Documents `-e` flags and `.env` file.

### v3.5.0 release notes
Updated the env var documentation to mention the `.env` file support.

## Testing
- All Go server tests pass
- All Go ingestor tests pass
- No logic changes to Go code — entrypoint shell script + docs only

---------

Co-authored-by: you <you@example.com>
2026-04-11 20:43:16 -07:00
you cde62166cb docs: v3.5.0 release notes + API documentation across README, deployment guide, FAQ
- Release notes for 95 commits since v3.4.1
- OpenAPI/Swagger docs: /api/spec and /api/docs called out everywhere
- Deployment guide: new API Documentation section
- README: API docs link added
- FAQ: 'Where is the API documentation?' entry
- Test plans for v3.4.2 validation
2026-04-08 06:55:25 +00:00
Kpa-clawbot a068e3e086 feat: zero-config defaults + deployment docs (M3-M4, #610) (#631)
## Zero-Config Defaults + Deployment Docs

Make CoreScope start with zero configuration — no `config.json`
required. The ingestor falls back to sensible defaults (local MQTT
broker, standard topics, default DB path) when no config file exists.

### What changed

**`cmd/ingestor/config.go`** — `LoadConfig` no longer errors on missing
config file. Instead it logs a message and uses defaults. If no MQTT
sources are configured (from file or env), defaults to
`mqtt://localhost:1883` with `meshcore/#` topic.

**`cmd/ingestor/main.go`** — Removed redundant "no MQTT sources" fatal
(now handled in config layer). Improved the "no connections established"
fatal with actionable hints.

**`README.md`** — Replaced "Docker (Recommended)" section with a
one-command quickstart using the pre-built image. No build step, no
config file, just `docker run`.

**`docs/deployment.md`** — New comprehensive deployment guide covering
Docker, Compose, config reference, MQTT setup, TLS/HTTPS, monitoring,
backup, and troubleshooting.

### Zero-config flow

```
docker run -d -p 80:80 -v corescope-data:/app/data ghcr.io/kpa-clawbot/corescope:latest
```

1. No config.json found → defaults used, log message printed
2. No MQTT sources → defaults to `mqtt://localhost:1883`
3. Internal Mosquitto broker already running in container → connection
succeeds
4. Dashboard shows empty, ready for packets

### Review fixes (commit 13b89bb)

- Removed `DISABLE_CADDY` references from all docs — this env var was
never implemented in the entrypoint
- Fixed `/api/stats` example in deployment guide — showed nonexistent
fields (`mqttConnected`, `uptimeSeconds`, `activeNodes`)
- Improved MQTT connection failure message with actionable
troubleshooting hints

Closes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 15:04:49 -07:00