Files
meshcore-analyzer/scripts/test-check-cdn-bypass.sh
T
63bfa3d910 feat(security): detect CDN-fronted deployment + document bypass requirement (closes #1561) (#1564)
Closes #1561. Follow-up to #1551.

## Why

#1551 added `Cache-Control: no-store` to all `/api/*` responses. That's
sufficient for CDNs that honour origin headers (Varnish, nginx). It is
**not** sufficient for Cloudflare zones where Cache Rules / Page Rules
override origin Cache-Control.

Field evidence from the meshat.se diagnosis (2026-06-04): observers
behind Cloudflare were returning `cf-cache-status: HIT` with `age` up to
~6 hours despite the origin emitting `no-store`. The CDN was caching per
zone policy and ignoring the upstream directive — exactly the failure
mode #1551 cannot reach. The application has no way to inject CDN rules;
the only durable fix is operator-side.

This PR makes that operator step discoverable and verifiable.

## What

### Server-side detection (log-only)

`cmd/server/cdn_detection.go` adds a middleware wired into the `/api/*`
chain after `noStoreAPIMiddleware`. On the **first** request bearing any
CDN-typical header (`CF-Connecting-IP`, `CF-Ray`, `X-Forwarded-For`,
`X-Real-IP`, `Fastly-Client-IP`, `True-Client-IP`) it logs:

```
[security] WARNING: detected request via CDN (CF-Ray header present).
Ensure /api/* is bypassed in your CDN config — see docs/deployment-behind-cdn.md.
Cached API responses cause observer-flap and incorrect dashboards.
```

`sync.Once` guarantees the warning fires at most once per process boot.
The middleware never blocks, never modifies the response, never adds
headers. Detection is observational only — operators who run behind a
CDN without bypass have a real bug; the warning is appropriate.

### Operator documentation

`docs/deployment.md` gains a new **"Behind a CDN (Cloudflare, Fastly)"**
section covering:

1. Curl verification command + healthy vs unhealthy output examples
2. Cloudflare Cache Rule creation (URI Path starts-with `/api/` → Bypass
cache)
3. Legacy Page Rules equivalent
4. Fastly note
5. Re-verification
6. Meaning of the startup log warning
7. Why we can't fix this server-side

`docs/deployment-behind-cdn.md` is the canonical path the log message
references — it's a short TL;DR that links back to the full section.

### Healthcheck script

`scripts/check-cdn-bypass.sh` — POSIX sh, no dependencies beyond curl +
grep + awk. Operators run:

```sh
scripts/check-cdn-bypass.sh https://your-domain.example.com
```

Exits `0` with `OK: no CDN caching detected ...` or `1` with a precise
diagnostic naming the offending header (`cf-cache-status: HIT` or stale
`age`).

## TDD

- **Red commit `e90ccaba`** (`test(security): RED ...`) —
`cmd/server/cdn_detection_test.go` (4 Go tests + 6 subtests for each
header) and `scripts/test-check-cdn-bypass.sh` (3 shell harness cases).
Middleware stub returns `next` unchanged so tests compile and fail on
assertions, not build errors.
- **Green commit `5e6a60b5`** (`feat(security): GREEN ...`) — real
middleware, wiring in `routes.go`, healthcheck script, doc.

## Deliverables

| File | Status | Purpose |
|------|--------|---------|
| `cmd/server/cdn_detection.go` | new | middleware + sync.Once warning |
| `cmd/server/cdn_detection_test.go` | new | 4 Go tests (1 stand-alone +
1 silence + 1 once + 1 table-driven over 6 headers) |
| `cmd/server/routes.go` | modified | `r.Use(cdnDetectionMiddleware)`
after no-store |
| `docs/deployment.md` | modified | TOC entry + "Behind a CDN" section |
| `docs/deployment-behind-cdn.md` | new | canonical path referenced by
log message + script output |
| `scripts/check-cdn-bypass.sh` | new | operator-runnable healthcheck |
| `scripts/test-check-cdn-bypass.sh` | new | shell harness with fake
curl |

## What this PR explicitly does NOT do

- Does not block requests based on CDN detection (log-only).
- Does not enforce CDN bypass (impossible — operator-controlled).
- Does not spoof, strip or modify CDN headers.
- Does not add CSP / HSTS / other security headers (out of scope).
- Warning is not configurable — operators behind a CDN without bypass
have a real bug, surfacing it is correct.

## Verification

- `go test ./...` in `cmd/server/` — full suite green.
- `sh scripts/test-check-cdn-bypass.sh` — 3/3 pass.
- Preflight checklist — all 11 gates clean (PII, branch scope, red
commit, CSS vars, CSS self-fallback, LIKE-on-JSON, sync migration,
async-migration annotation, XSS sinks, img/SVG ratio, themed-img/SVG,
fixture coverage).

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: clawbot <bot@clawbot.invalid>
2026-06-04 13:14:09 +00:00

126 lines
3.7 KiB
Bash
Executable File

#!/bin/sh
# Test harness for scripts/check-cdn-bypass.sh — issue #1561.
# Substitutes a fake `curl` on PATH so we can simulate CDN responses
# without network access. The fake curl honors `-o <file>` (writes
# the mocked headers there) and `-w '%{http_code}'` (writes the
# mocked HTTP status to stdout), matching the real curl interface
# the production script depends on.
set -u
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TARGET="$SCRIPT_DIR/check-cdn-bypass.sh"
PASS=0
FAIL=0
# mk_fake_curl HEADERS HTTP_STATUS
# Builds a tmpdir containing a `curl` shim that:
# - parses -o <file> from its args and writes HEADERS there
# - parses -w <fmt> and prints HTTP_STATUS to stdout (the script
# only ever uses -w '%{http_code}')
# - exits 0 (curl considers a non-2xx still a successful transfer
# unless -f is passed; production script does not pass -f, so
# this matches reality).
mk_fake_curl() {
headers="$1"
status="$2"
tmpdir="$(mktemp -d)"
# Embed the values via heredoc with single-quoted delimiter so
# nothing is expanded inside the generated script except by the
# outer shell now.
cat >"$tmpdir/curl" <<EOF
#!/bin/sh
out=""
while [ \$# -gt 0 ]; do
case "\$1" in
-o) out="\$2"; shift 2 ;;
-w) shift 2 ;; # always %{http_code} in this script
-*) shift ;; # -s, -S, -I, -L, etc.
*) shift ;; # URL
esac
done
if [ -n "\$out" ]; then
cat >"\$out" <<'BODY'
$headers
BODY
fi
printf '%s' '$status'
exit 0
EOF
chmod +x "$tmpdir/curl"
echo "$tmpdir"
}
run_case() {
name="$1"
headers="$2"
status="$3"
want_exit="$4"
want_substr="$5"
if [ ! -f "$TARGET" ]; then
echo "FAIL: $name$TARGET missing"
FAIL=$((FAIL+1))
return
fi
fakedir="$(mk_fake_curl "$headers" "$status")"
out="$(PATH="$fakedir:$PATH" sh "$TARGET" https://example.test 2>&1)"
rc=$?
rm -rf "$fakedir"
if [ "$rc" != "$want_exit" ]; then
echo "FAIL: $name — exit code $rc, want $want_exit; output: $out"
FAIL=$((FAIL+1))
return
fi
case "$out" in
*"$want_substr"*) ;;
*)
printf 'FAIL: %s — output missing %s; got: %s\n' "$name" "$want_substr" "$out"
FAIL=$((FAIL+1))
return
;;
esac
echo "PASS: $name"
PASS=$((PASS+1))
}
# Case 1: HTTP 200, bypass — no cf-cache HIT, age:0 → exit 0
run_case "bypass-ok-200" "cache-control: no-store
cf-cache-status: BYPASS
age: 0" "200" 0 "OK"
# Case 2: HTTP 200, CDN HIT — exit 1, mention HIT
run_case "cf-hit-200" "cache-control: no-store
cf-cache-status: HIT
age: 47" "200" 1 "HIT"
# Case 3: HTTP 200, stale by age — no cf-cache header but age > 0 → exit 1
run_case "stale-age-200" "cache-control: no-store
age: 120" "200" 1 "stale"
# Case 4: HTTP 200, no cache headers at all → exit 0 (existing behavior preserved)
run_case "bare-200-no-cache-headers" "content-type: application/json" "200" 0 "OK"
# Round-1 regression: a non-200 endpoint had been silently reported
# as "OK: no CDN caching detected" because no cache headers were
# present. Script must now refuse to draw a conclusion.
# Case 5: HTTP 404 → exit 1, status-related error
run_case "http-404-fails-with-status-error" "" "404" 1 "HTTP 404"
# Case 6: HTTP 401 → exit 1, status-related error
run_case "http-401-fails-with-status-error" "" "401" 1 "HTTP 401"
# Case 7: HTTP 403 → exit 1, status-related error
run_case "http-403-fails-with-status-error" "" "403" 1 "HTTP 403"
# Case 8: HTTP 500 → exit 1, status-related error
run_case "http-500-fails-with-status-error" "" "500" 1 "HTTP 500"
echo
echo "Results: $PASS passed, $FAIL failed"
[ "$FAIL" = "0" ] || exit 1