mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-08-28 22:08:25 +00:00
Kpa-clawbot
2a07716885
GREEN for the invariant test added in the prior commit. Resolves the dead-on- arrival blocker introduced by PR #738: the server opens SQLite mode=ro after #1283/#1289, so the `confirm=true` path of `/api/admin/prune-geo-filter` would fail in production with "attempt to write a readonly database". The author's tests only passed because setupTestDB uses `:memory:` without mode=ro. ## Architecture: file-marker queue (chosen over IPC/HTTP proxy) Server → marker file → ingestor → result file → server. New package `internal/prunequeue` defines the on-disk protocol: <dir(dbPath)>/prune-requests/request-<id>.json (written by server) <dir(dbPath)>/prune-requests/result-<id>.json (written by ingestor) Atomic via os.Rename. The ingestor removes the request file as part of writing the result, so the directory naturally drains. Why file markers over an internal localhost HTTP socket: * No new listener / port / auth surface to manage. * Inherits the same backup/permissions story as the SQLite file — they share a directory and a deployment. * Survives ingestor restarts: pending requests are processed on next boot. * Trivial to inspect / clear with `ls` / `trash` during ops. * Matches the existing maintenance-ticker pattern in cmd/ingestor/main.go. ## HTTP surface (unchanged for clients in spirit) POST /api/admin/prune-geo-filter * dry-run (default): same response shape as before — preview list. * confirm=true: now returns 202 Accepted + {requestId, statusUrl, count, nodes}. Body must still carry the pubkeys snapshot from the preview (TOCTOU guard preserved — the ingestor honors the list verbatim, does NOT re-evaluate geo_filter membership). GET /api/admin/prune-geo-filter/status?id=<id> (NEW, requires API key) * pending: 200 {requestId, status:"pending"} * done: 200 {requestId, status:"done", deleted, requestedAt, completedAt} * error: 200 {requestId, status:"error", error, ...} * 404 when neither marker nor result exist. * 400 for invalid ids (path-traversal guard: hex/[0-9a-f] only, ≤64 chars). ## TOCTOU + cascade-cleanup from prior CR * TOCTOU: server snapshots the preview list at confirm time and writes the PRUNED list (intersection of preview + still-outside) into the marker. The ingestor does NOT re-check geo_filter — it executes exactly what the operator confirmed. * Cascade cleanup: ingestor's DeleteNodesByPubkeys chunks under SQLite's variable limit (500 per stmt). It deliberately limits scope to the `nodes` table; the schema has no FKs, so observation/transmission rows are left for the regular packet-retention ticker. Documented in-line. ## Read-only invariant test Extends cmd/server/readonly_invariant_test.go (TestServerDBHasNoWriteMethods) to forbid `DeleteNodesByPubkeys` on the server `*DB`. With this commit the method is gone, so the test passes: $ go test -run TestServerDBHasNoWriteMethods ./... ok github.com/corescope/server 0.190s ## Tests * cmd/server/routes_test.go: rewrote the confirm-test to assert 202 + marker file existence; added pending→done status-roundtrip test; added 404 + path-traversal-id rejection tests. Old TestDeleteNodesByPubkeys removed (the method now lives in cmd/ingestor). * cmd/ingestor/prune_geofilter_test.go: end-to-end queue test (request file on disk → RunPendingPruneRequests → result file + DB rows actually deleted), plus empty-queue no-op. * internal/prunequeue/prunequeue_test.go: round-trip, path-traversal rejection, missing-result returns (nil, nil). ## Files * internal/prunequeue/{prunequeue.go,prunequeue_test.go,go.mod} — new package * cmd/ingestor/prune_geofilter.go — Store.DeleteNodesByPubkeys + RunPendingPruneRequests * cmd/ingestor/prune_geofilter_test.go — queue end-to-end test * cmd/ingestor/main.go — startup pass + 15s ticker for the queue, Stop on shutdown * cmd/server/db.go — DeleteNodesByPubkeys removed; only GetNodesForGeoPrune (read) remains * cmd/server/routes.go — handlePruneGeoFilter rewritten to enqueue; handlePruneGeoFilterStatus added; route registered * cmd/server/routes_test.go — new tests; old direct-DELETE coverage removed
48 lines
1.5 KiB
AMPL
48 lines
1.5 KiB
AMPL
module github.com/corescope/server
|
|
|
|
go 1.22
|
|
|
|
require (
|
|
github.com/gorilla/mux v1.8.1
|
|
github.com/gorilla/websocket v1.5.3
|
|
github.com/meshcore-analyzer/geofilter v0.0.0
|
|
github.com/meshcore-analyzer/sigvalidate v0.0.0
|
|
modernc.org/sqlite v1.34.5
|
|
)
|
|
|
|
replace github.com/meshcore-analyzer/geofilter => ../../internal/geofilter
|
|
|
|
replace github.com/meshcore-analyzer/sigvalidate => ../../internal/sigvalidate
|
|
|
|
require github.com/meshcore-analyzer/packetpath v0.0.0
|
|
|
|
replace github.com/meshcore-analyzer/packetpath => ../../internal/packetpath
|
|
|
|
require github.com/meshcore-analyzer/dbconfig v0.0.0
|
|
|
|
replace github.com/meshcore-analyzer/dbconfig => ../../internal/dbconfig
|
|
|
|
require github.com/meshcore-analyzer/perfio v0.0.0
|
|
|
|
replace github.com/meshcore-analyzer/perfio => ../../internal/perfio
|
|
|
|
require github.com/meshcore-analyzer/dbschema v0.0.0
|
|
|
|
replace github.com/meshcore-analyzer/dbschema => ../../internal/dbschema
|
|
|
|
require (
|
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
|
github.com/google/uuid v1.6.0 // indirect
|
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
|
github.com/ncruces/go-strftime v0.1.9 // indirect
|
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
|
golang.org/x/sys v0.22.0 // indirect
|
|
modernc.org/libc v1.55.3 // indirect
|
|
modernc.org/mathutil v1.6.0 // indirect
|
|
modernc.org/memory v1.8.0 // indirect
|
|
)
|
|
|
|
require github.com/meshcore-analyzer/prunequeue v0.0.0
|
|
|
|
replace github.com/meshcore-analyzer/prunequeue => ../../internal/prunequeue
|