mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-26 00:15:24 +00:00
liquidraverandClaude Opus 5
b695b979a2
## Problem `handleNodes` runs the geo-filter, `nodeBlacklist`, `hiddenNamePrefixes` and area passes **after** the SQL `LIMIT/OFFSET`, and rewrites `total` to the filtered length. A page that loses a row is therefore short **without being the last page**, and neither the page length nor `total` can tell a client whether to ask for another page. #1606 added the pagination loop and chose the page length as the canonical stop. That is correct only where nothing is ever filtered. Everywhere else the list truncates at the first filtered page boundary and strands every node behind it — the #1598 symptom reached by a different route: a node that is relaying right now simply stops being in the list. The comment at `app.js:240` rejects `total` for exactly the right reason, then picks the signal the same code path also breaks. ## Measured on a live 2346-node deployment Page sizes for the query the map issues: ``` offset=0 returned=500 ← full, loop continues offset=500 returned=499 ← one row filtered AFTER the LIMIT → loop STOPS offset=1000 returned=500 ← never requested offset=1500 returned=500 ← never requested offset=2000 returned=345 ← never requested ``` | stop rule | requests | nodes reached | |---|---:|---:| | short page (master) | 2 | **999** | | `has_more`, else empty page | 6 | **2344** | **1341 nodes, 57%, unreachable through the UI.** ### One hidden node truncates the whole list The deployment this came from has no `geoFilter` (`/api/config/geo-filter` returns `polygon: null`) and no `nodeBlacklist`. It has a single `hiddenNamePrefixes` entry — a deliberate operator choice — and exactly one node whose name starts with it: ``` public_key d4a46ea2…1054 (64 clean hex chars) name 🚫🔥☀️ role repeater last_seen 2026-09-22T10:09:38Z ``` `handleNodes` drops that row in the `IsNameHidden` pass, which runs after the SQL `LIMIT`. The row is counted by the `LIMIT` and by `COUNT(*)`, so the page it lands in comes back exactly one short — and stops every client that treats a short page as the end. Isolated against SQL on the same database, seconds apart: ``` SELECT lower(public_key) FROM nodes ORDER BY last_seen DESC LIMIT 500 OFFSET 500 -> 500 rows GET /api/nodes?limit=500&offset=500 -> 499 rows comm -23 sql.txt api.txt -> d4a46ea2e99cab132a3286ef3d9cce9099318790af7f25671fe83de453721054 ``` Deterministic — `offset=500` returned 499 on three consecutive requests. Not a CDN artifact either: `cf-cache-status: DYNAMIC`, origin `cache-control: no-store`, no `age` header, and four requests with deliberately unique cache keys all returned 499. So **one deliberately hidden node makes 1341 of 2344 nodes unreachable.** The hiding feature does exactly what it was asked to do for that one node, and takes 57% of the network with it, silently. A single `hiddenNamePrefixes` entry is enough; no geo-filter, blacklist or area filter is needed to reach this state. ### The cutoff moves, which is why this reads as intermittent The visible set is the sum of the pages up to and including the first short one, so the boundary sits wherever the unreturnable row currently sorts by `last_seen`, and jumps a whole page as ingest reorders the list. Same deployment, same code, same config, ~2h apart: | dropped row's rank | first short page | nodes visible | |---|---|---:| | inside 0–499 | page 1 | 499 | | inside 500–999 | page 2 | 999 | A node is visible or invisible purely by where it lands relative to that moving line, so affected nodes appear to vanish and return on their own. Two operators on this deployment reported exactly that, independently, while I was measuring. ### A named reproduction `HU-ZA-Lentihegy` (`5287a33f…`), reported missing from the map by an operator whose companion had logged its advert at 04:20 local the same morning. Ingest was fine. The row is in `nodes` with `last_seen` `2026-09-22T02:20:35Z` — the same advert, to the second — valid GPS, role `repeater`, 1033 adverts, and `/api/nodes/search?q=lentihegy` returns it. ``` rank by last_seen : 1081 cutoff at the time: 999 ``` It missed by 82 positions. Walking the same live endpoint, same moment: | stop rule | requests | nodes reached | Lentihegy | |---|---:|---:|---| | short page (master) | 2 | 999 | **not reached** | | `has_more`, else empty page | 6 | 2340 | reached | The practical shape of this on a busy mesh: 1081 nodes had been heard more recently than 9.4 hours, so on that deployment **anything last heard more than ~9 hours ago was invisible**, alive or not. `#/nodes` compounds it — its search box filters client-side over the truncated set, so the server-side `?search=` never runs and an operator cannot find the node by searching for it either, even though the endpoint would return it. ## Change **Server** — `NodeListResponse` gains `has_more`, computed from the raw SQL page against the real `COUNT(*)` before the filter passes run, so it survives them: ```go hasMore := offset+len(nodes) < total ``` Always emitted (no `omitempty`) so a client can tell `false` from an old server. No extra request in the fixed path: `has_more` ends the loop exactly, where the old rule needed a probe page. **Clients** — `app.js` `fetchAllNodes`, `nodes.js` `loadNodes` and `area-map.html`'s inline helper stop on `has_more`, falling back to a zero-length page against a server that predates it. An empty page always ends the loop, so a `has_more` against a concurrently-shrinking table cannot spin to `safetyCap`. Left alone: the three loops are still three copies. Collapsing them onto `fetchAllNodes` is a bigger change than this fix needs, and `nodes.js` has its own inter-page progress UI. Happy to do it separately if you want it. ## Testing - **Unit** (`tests/unit/test-fetch-all-nodes-pagination.js`): the fixture now models the real handler — a row counted by the LIMIT and by `COUNT(*)`, then removed from the page. Three new cases. Fails on the old rule at 499 of 1199. - **E2E** (`tests/e2e/test-map-nodes-pagination-e2e.js`, already wired into `deploy.yml`): the mock drops a page-1 row and emits `has_more`. Mutation-checked — restoring master's stop rule fails 3 of its steps. - **Go** (`cmd/server/nodes_pagination_has_more_test.go`): asserts `has_more` stays true on a page filtering shortened. Mutation-checked — recomputing it after the filter block fails the test. - Full server suite `go test -race`: ok, 41.4s. `gofmt` clean, `go vet` passes. - **Against a real binary**, not just mocks: fixture DB migrated with `corescope-migrate`, `hiddenNamePrefixes: ["SKCE"]`, `limit=3`. Page 1 returns 2 of 3 with `total` rewritten to 2 and `has_more=true`. Walking the real server with master's rule reaches 2 nodes; with `has_more`, all 199 visible of 200, the hidden one still hidden. The real frontend against that server loads 199 with no JS errors. Two existing expectations changed, both deliberate: 1. `surfaces ALL nodes past the 500 server cap` — 3 → 4 requests. That mock emits no `has_more`, so the 200-row final page can no longer end the loop (a short page is exactly what a filtered page looks like) and a zero-length probe follows. Against a current server `has_more` still ends it at 3. 2. `rows missing public_key are NOT collapsed into one` — its stub returned a constant body, which would now be paged to `safetyCap`. It serves one page then empties. Local `test-all.sh` exits 1 on two XSS-gate self-tests (`good-2-tested.js`, `good-4-tested.js`) via a `UnicodeEncodeError` printing an emoji under Windows cp1252. Identical on clean `origin/master` in a scratch worktree, so it is pre-existing and platform-local, not this branch. There is a second identical filter block further down `routes.go` on another list endpoint. Likely the same class; not touched here. If you would rather land your own version of this, say so and I will close mine. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
527 lines
29 KiB
Go
527 lines
29 KiB
Go
package main
|
||
|
||
import (
|
||
"encoding/json"
|
||
"fmt"
|
||
"net/http"
|
||
"sort"
|
||
"strings"
|
||
|
||
"github.com/gorilla/mux"
|
||
)
|
||
|
||
// routeMeta holds metadata for a single API route.
|
||
type routeMeta struct {
|
||
Summary string `json:"summary"`
|
||
Description string `json:"description,omitempty"`
|
||
Tag string `json:"tag"`
|
||
Auth bool `json:"auth,omitempty"`
|
||
QueryParams []paramMeta `json:"queryParams,omitempty"`
|
||
// Response, when non-nil, is the OpenAPI schema object for the 200
|
||
// application/json response body. Routes without it fall back to the
|
||
// generic {"type":"object"} placeholder. Use schemaRef(...) to point
|
||
// at a named entry in components/schemas (see componentSchemas).
|
||
Response map[string]interface{} `json:"-"`
|
||
}
|
||
|
||
type paramMeta struct {
|
||
Name string `json:"name"`
|
||
Description string `json:"description"`
|
||
Required bool `json:"required,omitempty"`
|
||
Type string `json:"type"` // "string", "integer", "boolean"
|
||
}
|
||
|
||
// routeDescriptions returns metadata for all known API routes.
|
||
// Key format: "METHOD /path/pattern"
|
||
func routeDescriptions() map[string]routeMeta {
|
||
return map[string]routeMeta{
|
||
// Config
|
||
"GET /api/config/cache": {Summary: "Get cache configuration", Tag: "config"},
|
||
"GET /api/config/client": {Summary: "Get client configuration", Tag: "config"},
|
||
"GET /api/config/regions": {Summary: "Get configured regions", Tag: "config"},
|
||
"GET /api/config/theme": {Summary: "Get theme configuration", Description: "Returns color maps, CSS variables, and theme defaults.", Tag: "config"},
|
||
"GET /api/config/map": {Summary: "Get map configuration", Tag: "config"},
|
||
"GET /api/config/geo-filter": {Summary: "Get geo-filter configuration", Tag: "config"},
|
||
|
||
// Admin / system
|
||
"GET /api/health": {Summary: "Health check", Description: "Returns server health, uptime, and memory stats.", Tag: "admin"},
|
||
"GET /api/stats": {Summary: "Network statistics", Description: "Returns aggregate stats (node counts, packet counts, observer counts). Cached for 10s.", Tag: "admin"},
|
||
"GET /api/perf": {Summary: "Performance statistics", Description: "Returns per-endpoint request timing and slow query log.", Tag: "admin"},
|
||
"GET /api/mqtt/status": {Summary: "MQTT source status", Description: "Returns per-MQTT-source connection state and counters (lastConnectUnix, lastPacketUnix, packetsTotal, etc.). Broker URL passwords are masked. Sourced from the ingestor stats file; empty list when unavailable. (#1043)", Tag: "admin"},
|
||
"POST /api/perf/reset": {Summary: "Reset performance stats", Tag: "admin", Auth: true},
|
||
// "POST /api/admin/prune" removed in #1283 (ingestor owns prune).
|
||
"GET /api/debug/affinity": {Summary: "Debug neighbor affinity scores", Tag: "admin", Auth: true},
|
||
"GET /api/backup": {Summary: "Download SQLite backup", Description: "Streams a consistent SQLite snapshot of the analyzer DB (VACUUM INTO). Response is application/octet-stream with attachment filename corescope-backup-<unix>.db.", Tag: "admin", Auth: true},
|
||
|
||
// Packets
|
||
"GET /api/packets": {Summary: "List packets", Description: "Returns decoded packets with filtering, sorting, and pagination.", Tag: "packets",
|
||
QueryParams: []paramMeta{
|
||
{Name: "limit", Description: "Max packets to return", Type: "integer"},
|
||
{Name: "offset", Description: "Pagination offset", Type: "integer"},
|
||
{Name: "sort", Description: "Sort field", Type: "string"},
|
||
{Name: "order", Description: "Sort order (asc/desc)", Type: "string"},
|
||
{Name: "type", Description: "Filter by packet type", Type: "string"},
|
||
{Name: "observer", Description: "Filter by observer ID", Type: "string"},
|
||
{Name: "timeRange", Description: "Time range filter (e.g. 1h, 24h, 7d)", Type: "string"},
|
||
{Name: "search", Description: "Full-text search", Type: "string"},
|
||
{Name: "groupByHash", Description: "Group duplicate packets by hash", Type: "boolean"},
|
||
}},
|
||
"GET /api/packets/{id}": {Summary: "Get packet detail", Tag: "packets"},
|
||
"GET /api/packets/timestamps": {Summary: "Get packet timestamp ranges", Tag: "packets"},
|
||
"POST /api/packets/observations": {Summary: "Batch submit observations", Description: "Submit multiple observer sightings for existing packets.", Tag: "packets"},
|
||
|
||
// Decode
|
||
"POST /api/decode": {Summary: "Decode a raw packet", Description: "Decodes a hex-encoded packet without storing it.", Tag: "packets"},
|
||
|
||
// Nodes
|
||
"GET /api/nodes": {Summary: "List nodes", Description: "Returns all known mesh nodes with status and metadata. Repeater/room rows carry the issue #672 usefulness metrics (traffic_share_score, bridge_score, coverage_score, redundancy_score), the composite usefulness_score + usefulness_grade, and relay-activity counters. See the Node schema.", Tag: "nodes",
|
||
Response: schemaRef("NodeListResponse"),
|
||
QueryParams: []paramMeta{
|
||
{Name: "role", Description: "Filter by node role", Type: "string"},
|
||
{Name: "status", Description: "Filter by status (active/stale/offline)", Type: "string"},
|
||
}},
|
||
"GET /api/nodes/search": {Summary: "Search nodes", Description: "Search nodes by name or public key prefix.", Tag: "nodes", QueryParams: []paramMeta{{Name: "q", Description: "Search query", Type: "string", Required: true}}},
|
||
"GET /api/nodes/bulk-health": {Summary: "Bulk node health", Description: "Returns health status for all nodes in one call.", Tag: "nodes"},
|
||
"GET /api/nodes/network-status": {Summary: "Network status summary", Description: "Returns counts of active, stale, and offline nodes.", Tag: "nodes"},
|
||
"GET /api/nodes/{pubkey}": {Summary: "Get node detail", Description: "Returns full detail for a single node by public key. For repeater/room nodes this includes the issue #672 usefulness axes + composite score/grade (see the Node schema).", Tag: "nodes", Response: schemaRef("NodeDetailResponse")},
|
||
"GET /api/nodes/{pubkey}/health": {Summary: "Get node health", Tag: "nodes"},
|
||
"GET /api/nodes/{pubkey}/paths": {Summary: "Get node routing paths", Tag: "nodes"},
|
||
"GET /api/nodes/{pubkey}/analytics": {Summary: "Get node analytics", Description: "Per-node packet counts, timing, and RF stats.", Tag: "nodes"},
|
||
"GET /api/nodes/{pubkey}/hop_analytics": {Summary: "Get node hop counts", Description: "One entry per flood packet the node forwarded in the window, with the hop count its flood.max check saw (the node's zero-based index in the path) and tags (flood, scoped or unscoped, advert). DIRECT packets are excluded. Every observation in the window is read. A colliding path prefix is attributed only when the previous hop's neighbor_edges neighbors leave this node as the one candidate; the server's resolved-path pick is not used. Packets carrying this node's prefix that cannot be attributed are counted in `ambiguous`.", Tag: "nodes",
|
||
QueryParams: []paramMeta{
|
||
{Name: "days", Description: "Lookback window in days (default 7, clamped 1-365)", Type: "integer"},
|
||
}},
|
||
"GET /api/nodes/{pubkey}/neighbors": {Summary: "Get node neighbors", Description: "Returns the queried node's first-hop neighbors with affinity scores and observation metadata (count, SNR, distance, observers). Ambiguous edges carry candidate pubkeys.", Tag: "nodes", Response: schemaRef("NodeNeighborsResponse")},
|
||
|
||
"GET /api/scope-audit": {Summary: "Network-wide scope audit", Description: "For every repeater that has answered a declared-regions request: the regions it declares, which of those it has NOT been observed forwarding in the window, which scopes it forwards without declaring, and whether it forwards unscoped floods while omitting the '*' wildcard. '*' is never listed as a region — it governs unscoped floods, not a scope. Repeaters never successfully asked are absent rather than shown as declaring nothing. Rows with missing regions sort first; a short window is weak evidence, since a quiet region simply has no traffic.", Tag: "analytics",
|
||
QueryParams: []paramMeta{
|
||
{Name: "window", Description: "Time window: 1h, 24h, or 7d (default 24h)", Type: "string"},
|
||
}},
|
||
|
||
// Analytics
|
||
"GET /api/analytics/rf": {Summary: "RF analytics", Description: "SNR/RSSI distributions and statistics.", Tag: "analytics"},
|
||
"GET /api/analytics/topology": {Summary: "Network topology", Description: "Hop-count distribution and route analysis.", Tag: "analytics"},
|
||
"GET /api/analytics/channels": {Summary: "Channel analytics", Description: "Message counts and activity per channel.", Tag: "analytics"},
|
||
"GET /api/analytics/distance": {Summary: "Distance analytics", Description: "Geographic distance calculations between nodes.", Tag: "analytics"},
|
||
"GET /api/analytics/hash-sizes": {Summary: "Hash size analysis", Description: "Distribution of hash prefix sizes across the network.", Tag: "analytics"},
|
||
"GET /api/analytics/hash-collisions": {Summary: "Hash collision detection", Description: "Identifies nodes sharing hash prefixes.", Tag: "analytics"},
|
||
"GET /api/analytics/subpaths": {Summary: "Subpath analysis", Description: "Common routing subpaths through the mesh.", Tag: "analytics"},
|
||
"GET /api/analytics/subpaths-bulk": {Summary: "Bulk subpath analysis", Tag: "analytics"},
|
||
"GET /api/analytics/subpath-detail": {Summary: "Subpath detail", Tag: "analytics"},
|
||
"GET /api/analytics/neighbor-graph": {Summary: "Neighbor graph", Description: "Full neighbor affinity graph for visualization.", Tag: "analytics"},
|
||
"GET /api/analytics/retransmissions": {Summary: "Retransmission pressure over time", Description: "Collision-pressure proxy (#1699): per time bucket, the average number of distinct repeaters in the union of all observed paths of each flood event (route types 0/1, TRACE excluded). A transmission's observations are split into flood events at gaps of more than 5 minutes; each event is bucketed by its first observation, and events before the store retention floor are left out. Hop prefixes are not resolved: a prefix counts once per event, so colliding 1-byte prefixes make this a lower bound. Only repeaters some observer heard are counted, so the value also follows observer coverage; each bucket carries its observer count.", Tag: "analytics",
|
||
QueryParams: []paramMeta{
|
||
{Name: "region", Description: "Comma-separated IATA codes; only observations from observers in the region are counted. A region with no known observers is not filtered", Type: "string"},
|
||
{Name: "window", Description: "Relative window: 1h, 24h, 3d, 7d or 30d", Type: "string"},
|
||
{Name: "from", Description: "Absolute window start (RFC3339)", Type: "string"},
|
||
{Name: "to", Description: "Absolute window end (RFC3339)", Type: "string"},
|
||
{Name: "bucket", Description: "Bucket size: 5m, 15m, 1h, 6h or 1d (default 1h)", Type: "string"},
|
||
}},
|
||
|
||
// Channels
|
||
"GET /api/channels": {Summary: "List channels", Description: "Returns known mesh channels with message counts.", Tag: "channels"},
|
||
"GET /api/channels/{hash}/messages": {Summary: "Get channel messages", Description: "Returns messages for a specific channel.", Tag: "channels"},
|
||
|
||
// Observers
|
||
"GET /api/observers": {Summary: "List observers", Description: "Returns all known packet observers/gateways.", Tag: "observers"},
|
||
"GET /api/observers/{id}": {Summary: "Get observer detail", Tag: "observers"},
|
||
"GET /api/observers/{id}/metrics": {Summary: "Get observer metrics", Description: "Packet rates, uptime, and performance metrics.", Tag: "observers"},
|
||
"GET /api/observers/{id}/analytics": {Summary: "Get observer analytics", Tag: "observers"},
|
||
"GET /api/observers/metrics/summary": {Summary: "Observer metrics summary", Description: "Aggregate metrics across all observers.", Tag: "observers"},
|
||
|
||
// Misc
|
||
"GET /api/resolve-hops": {Summary: "Resolve hop path", Description: "Resolves hash prefixes in a hop path to node names. Returns affinity scores and best candidates.", Tag: "nodes", QueryParams: []paramMeta{{Name: "hops", Description: "Comma-separated hop hash prefixes", Type: "string", Required: true}}},
|
||
"GET /api/traces/{hash}": {Summary: "Get packet traces", Description: "Returns all observer sightings for a packet hash.", Tag: "packets"},
|
||
"GET /api/iata-coords": {Summary: "Get IATA airport coordinates", Description: "Returns lat/lon for known airport codes (used for observer positioning).", Tag: "config"},
|
||
"GET /api/audio-lab/buckets": {Summary: "Audio lab frequency buckets", Description: "Returns frequency bucket data for audio analysis.", Tag: "analytics"},
|
||
}
|
||
}
|
||
|
||
// schemaRef returns an OpenAPI $ref pointing at a named component schema.
|
||
func schemaRef(name string) map[string]interface{} {
|
||
return map[string]interface{}{"$ref": "#/components/schemas/" + name}
|
||
}
|
||
|
||
// componentSchemas returns the reusable OpenAPI schemas surfaced under
|
||
// components/schemas. The Node schema documents the per-node usefulness
|
||
// metrics (issue #672) that the /api/nodes handlers attach to repeater/room
|
||
// rows — previously these were set on the wire but undocumented (issue
|
||
// #672 / E). Score axes are bounded [0,1]; usefulness_score is the weighted
|
||
// composite and usefulness_grade its A–F letter.
|
||
func componentSchemas() map[string]interface{} {
|
||
score01 := func(desc string) map[string]interface{} {
|
||
// "double" matches the Go float64 wire type (some linters flag "float").
|
||
return map[string]interface{}{
|
||
"type": "number", "format": "double", "minimum": 0, "maximum": 1,
|
||
"description": desc,
|
||
}
|
||
}
|
||
str := func(desc string) map[string]interface{} {
|
||
m := map[string]interface{}{"type": "string"}
|
||
if desc != "" {
|
||
m["description"] = desc
|
||
}
|
||
return m
|
||
}
|
||
return map[string]interface{}{
|
||
"Node": map[string]interface{}{
|
||
"type": "object",
|
||
// additionalProperties:true — the node object carries more fields
|
||
// than documented here (e.g. foreign, default_scope, hash-size and
|
||
// multi-byte enrichment); only the stable + #672 fields are spelled
|
||
// out. The #672 usefulness fields are emitted only by a server that
|
||
// has shipped issue #672 (PR #1762); on an older server they are
|
||
// simply absent.
|
||
"additionalProperties": true,
|
||
"description": "A mesh node. Repeater and room nodes additionally carry the issue #672 usefulness metrics and relay-activity fields below; those fields are absent on other roles. NOTE: coverage_score, redundancy_score and usefulness_grade ship only with the #672 4-axis scorer (PR #1762) and are absent on every build without it; until that lands usefulness_score is aliased to traffic_share_score. Only traffic_share_score and bridge_score ship today.",
|
||
"properties": map[string]interface{}{
|
||
"public_key": str("Node public key (hex)."),
|
||
"name": str("Node display name (most recent advert name)."),
|
||
"role": str("Node role (e.g. repeater, room, client, sensor)."),
|
||
"lat": map[string]interface{}{"type": "number", "nullable": true},
|
||
"lon": map[string]interface{}{"type": "number", "nullable": true},
|
||
"last_seen": str("RFC3339 timestamp of the most recent observation."),
|
||
"first_seen": str("RFC3339 timestamp of the first observation."),
|
||
"advert_count": map[string]interface{}{"type": "integer"},
|
||
"flood_advert_count_7d": map[string]interface{}{"type": "integer", "description": "Distinct FLOOD adverts originated in the last 7 days (zero-hop adverts excluded). Present on the node detail endpoint."},
|
||
"battery_mv": map[string]interface{}{"type": "integer", "nullable": true},
|
||
"temperature_c": map[string]interface{}{"type": "number", "nullable": true},
|
||
"relay_active": map[string]interface{}{"type": "boolean", "description": "Repeater/room only: relayed traffic within the active window."},
|
||
"relay_count_1h": map[string]interface{}{"type": "integer", "description": "Repeater/room only: relay-hop appearances in the last hour."},
|
||
"relay_count_24h": map[string]interface{}{"type": "integer", "description": "Repeater/room only: relay-hop appearances in the last 24 hours."},
|
||
"unscoped_relay_count_24h": map[string]interface{}{"type": "integer", "description": "Repeater/room only: subset of relay_count_24h that were unscoped floods (route_type FLOOD). A well-configured repeater sets flood.max.unscoped 0, so a non-trivial count flags a base-config problem."},
|
||
"last_relayed": str("Repeater/room only: RFC3339 time this node last appeared as a relay hop."),
|
||
"declared_regions": map[string]interface{}{"type": "array", "items": map[string]interface{}{"type": "string"}, "description": "Repeater/room only (#1862): named regions from this node's newest declared-regions answer, spelled as GET /api/scope-audit declaredRegions spells them (leading '#' stripped, '*' wildcard excluded). Empty array: it answered and named no region. Absent: it never answered, this database has no declared-regions source, or the declared-regions lookup failed. Absence is not evidence the node lacks a region."},
|
||
"relay_window_hours": map[string]interface{}{"type": "integer", "description": "Repeater/room only, /api/nodes/{pubkey} detail endpoint only: width (hours) of the relay-activity window the relay_count_* values cover."},
|
||
"traffic_share_score": score01("#672 Traffic axis: share of non-advert traffic relayed through this repeater. Repeater/room only."),
|
||
"bridge_score": score01("#672 Bridge axis: normalized betweenness centrality (chokepoint importance). Repeater/room only."),
|
||
"coverage_score": score01("#672 Coverage axis: normalized harmonic reach centrality (how much of the mesh the node can reach). Repeater/room only."),
|
||
"redundancy_score": score01("#672 Redundancy axis: normalized articulation-point criticality — 1 means removing the node fragments the mesh, 0 means alternate paths exist. Repeater/room only."),
|
||
"usefulness_score": score01("#672 composite usefulness = 0.30·bridge + 0.25·coverage + 0.25·redundancy + 0.20·traffic. Until the 4-axis scorer ships (PR #1762) this is aliased to traffic_share_score. Repeater/room only."),
|
||
"usefulness_grade": map[string]interface{}{
|
||
"type": "string", "enum": []string{"A", "B", "C", "D", "F"},
|
||
"description": "Letter grade derived from usefulness_score. Repeater/room only.",
|
||
},
|
||
"declared_regions_truncated": map[string]interface{}{"type": "boolean", "description": "Repeater/room only (#1862): present, and true, only when the answer behind declared_regions was flagged as truncated, so that list is partial. GET /api/scope-audit shows the same flag as truncated. Absent otherwise, including for a source that does not record truncation, so absence does not mean the list is complete."},
|
||
},
|
||
},
|
||
"NodeListResponse": map[string]interface{}{
|
||
"type": "object",
|
||
"properties": map[string]interface{}{
|
||
"nodes": map[string]interface{}{"type": "array", "items": schemaRef("Node")},
|
||
"total": map[string]interface{}{"type": "integer", "description": "Total nodes matching the query after filtering."},
|
||
"counts": map[string]interface{}{"type": "object", "additionalProperties": map[string]interface{}{"type": "integer"}, "description": "Per-role node counts."},
|
||
"has_more": map[string]interface{}{"type": "boolean", "description": "True when rows exist past this page. Decided before the blacklist / hidden-prefix / geo-filter / area passes, which drop rows from the page and rewrite total — so neither the page length nor total can be used to stop paginating. Paginate until this is false."},
|
||
},
|
||
},
|
||
"NodeDetailResponse": map[string]interface{}{
|
||
"type": "object",
|
||
"properties": map[string]interface{}{
|
||
"node": schemaRef("Node"),
|
||
"recentAdverts": map[string]interface{}{"type": "array", "items": schemaRef("NodeAdvert"), "description": "Up to 20 most recent transmissions from this node (newest first)."},
|
||
},
|
||
},
|
||
"NodeAdvert": map[string]interface{}{
|
||
"type": "object",
|
||
"description": "A recent transmission/advert from a node (the /api/packets transmission shape). Only the commonly-used fields are documented.",
|
||
"additionalProperties": true,
|
||
"properties": map[string]interface{}{
|
||
"id": map[string]interface{}{"type": "integer"},
|
||
"hash": str("Transmission content hash."),
|
||
"payload_type": map[string]interface{}{"type": "integer", "description": "MeshCore payload type."},
|
||
"first_seen": str("RFC3339 time the transmission was first observed."),
|
||
"from_pubkey": str("Originating node public key."),
|
||
},
|
||
},
|
||
"CandidateEntry": map[string]interface{}{
|
||
"type": "object",
|
||
"description": "A candidate pubkey offered when a neighbor edge is ambiguous.",
|
||
"properties": map[string]interface{}{
|
||
"pubkey": str("Candidate node public key (hex)."), "name": str("Candidate node display name."), "role": str("Candidate node role (e.g. repeater, room)."),
|
||
},
|
||
},
|
||
"NeighborEntry": map[string]interface{}{
|
||
"type": "object",
|
||
"description": "One neighbor of the queried node, with affinity score and observation metadata.",
|
||
"properties": map[string]interface{}{
|
||
"pubkey": map[string]interface{}{"type": "string", "nullable": true, "description": "Resolved neighbor public key, or null when only a hop prefix is known."},
|
||
"prefix": str("Raw hop hash prefix that established this edge."),
|
||
"name": map[string]interface{}{"type": "string", "nullable": true},
|
||
"role": map[string]interface{}{"type": "string", "nullable": true},
|
||
"count": map[string]interface{}{"type": "integer", "description": "Total observations supporting this neighborship."},
|
||
"score": score01("Affinity score: count saturation × recency decay × observer-diversity confidence."),
|
||
"counts_by_mode": map[string]interface{}{"type": "object", "additionalProperties": map[string]interface{}{"type": "integer"}, "description": "#1638: observation counts keyed by hash-prefix mode in bytes (1/2/3; 0 = legacy/unknown)."},
|
||
"first_seen": str(""),
|
||
"last_seen": str(""),
|
||
"avg_snr": map[string]interface{}{"type": "number", "nullable": true},
|
||
"distance_km": map[string]interface{}{"type": "number", "nullable": true},
|
||
"observers": map[string]interface{}{"type": "array", "items": map[string]interface{}{"type": "string"}},
|
||
"ambiguous": map[string]interface{}{"type": "boolean"},
|
||
"unresolved": map[string]interface{}{"type": "boolean"},
|
||
"candidates": map[string]interface{}{"type": "array", "items": schemaRef("CandidateEntry")},
|
||
},
|
||
},
|
||
"NodeNeighborsResponse": map[string]interface{}{
|
||
"type": "object",
|
||
"properties": map[string]interface{}{
|
||
"node": str("The queried node's public key."),
|
||
"neighbors": map[string]interface{}{"type": "array", "items": schemaRef("NeighborEntry")},
|
||
"total_observations": map[string]interface{}{"type": "integer"},
|
||
},
|
||
},
|
||
}
|
||
}
|
||
|
||
// buildOpenAPISpec constructs an OpenAPI 3.0 spec by walking the mux router.
|
||
func buildOpenAPISpec(router *mux.Router, version string) map[string]interface{} {
|
||
descriptions := routeDescriptions()
|
||
|
||
// Collect routes from the router
|
||
type routeInfo struct {
|
||
path string
|
||
method string
|
||
authReq bool
|
||
}
|
||
var routes []routeInfo
|
||
|
||
router.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error {
|
||
path, err := route.GetPathTemplate()
|
||
if err != nil {
|
||
return nil
|
||
}
|
||
if !strings.HasPrefix(path, "/api/") {
|
||
return nil
|
||
}
|
||
// Skip the spec/docs endpoints themselves
|
||
if path == "/api/spec" || path == "/api/docs" {
|
||
return nil
|
||
}
|
||
methods, err := route.GetMethods()
|
||
if err != nil {
|
||
return nil
|
||
}
|
||
for _, m := range methods {
|
||
routes = append(routes, routeInfo{path: path, method: m})
|
||
}
|
||
return nil
|
||
})
|
||
|
||
// Sort routes for deterministic output
|
||
sort.Slice(routes, func(i, j int) bool {
|
||
if routes[i].path != routes[j].path {
|
||
return routes[i].path < routes[j].path
|
||
}
|
||
return routes[i].method < routes[j].method
|
||
})
|
||
|
||
// Build paths object
|
||
paths := make(map[string]interface{})
|
||
tagSet := make(map[string]bool)
|
||
|
||
for _, ri := range routes {
|
||
key := ri.method + " " + ri.path
|
||
meta, hasMeta := descriptions[key]
|
||
|
||
// Convert mux path params {name} to OpenAPI {name} (same format, convenient)
|
||
openAPIPath := ri.path
|
||
|
||
// Documented routes can declare a concrete 200 response schema;
|
||
// everything else falls back to the generic object placeholder.
|
||
respSchema := map[string]interface{}{"type": "object"}
|
||
if hasMeta && meta.Response != nil {
|
||
respSchema = meta.Response
|
||
}
|
||
|
||
// Build operation
|
||
op := map[string]interface{}{
|
||
"summary": func() string {
|
||
if hasMeta {
|
||
return meta.Summary
|
||
}
|
||
return ri.path
|
||
}(),
|
||
"responses": map[string]interface{}{
|
||
"200": map[string]interface{}{
|
||
"description": "Success",
|
||
"content": map[string]interface{}{
|
||
"application/json": map[string]interface{}{
|
||
"schema": respSchema,
|
||
},
|
||
},
|
||
},
|
||
},
|
||
}
|
||
|
||
if hasMeta {
|
||
if meta.Description != "" {
|
||
op["description"] = meta.Description
|
||
}
|
||
if meta.Tag != "" {
|
||
op["tags"] = []string{meta.Tag}
|
||
tagSet[meta.Tag] = true
|
||
}
|
||
if meta.Auth {
|
||
op["security"] = []map[string]interface{}{
|
||
{"ApiKeyAuth": []string{}},
|
||
}
|
||
}
|
||
|
||
// Add query parameters
|
||
if len(meta.QueryParams) > 0 {
|
||
params := make([]interface{}, 0, len(meta.QueryParams))
|
||
for _, qp := range meta.QueryParams {
|
||
p := map[string]interface{}{
|
||
"name": qp.Name,
|
||
"in": "query",
|
||
"required": qp.Required,
|
||
"schema": map[string]interface{}{"type": qp.Type},
|
||
}
|
||
if qp.Description != "" {
|
||
p["description"] = qp.Description
|
||
}
|
||
params = append(params, p)
|
||
}
|
||
op["parameters"] = params
|
||
}
|
||
}
|
||
|
||
// Extract path parameters from {name} patterns
|
||
pathParams := extractPathParams(openAPIPath)
|
||
if len(pathParams) > 0 {
|
||
existing, _ := op["parameters"].([]interface{})
|
||
for _, pp := range pathParams {
|
||
existing = append(existing, map[string]interface{}{
|
||
"name": pp,
|
||
"in": "path",
|
||
"required": true,
|
||
"schema": map[string]interface{}{"type": "string"},
|
||
})
|
||
}
|
||
op["parameters"] = existing
|
||
}
|
||
|
||
// Add to paths
|
||
methodLower := strings.ToLower(ri.method)
|
||
if _, ok := paths[openAPIPath]; !ok {
|
||
paths[openAPIPath] = make(map[string]interface{})
|
||
}
|
||
paths[openAPIPath].(map[string]interface{})[methodLower] = op
|
||
}
|
||
|
||
// Build tags array (sorted)
|
||
tagOrder := []string{"admin", "analytics", "channels", "config", "nodes", "observers", "packets"}
|
||
tagDescriptions := map[string]string{
|
||
"admin": "Server administration and diagnostics",
|
||
"analytics": "Network analytics and statistics",
|
||
"channels": "Mesh channel operations",
|
||
"config": "Server configuration",
|
||
"nodes": "Mesh node operations",
|
||
"observers": "Packet observer/gateway operations",
|
||
"packets": "Packet capture and decoding",
|
||
}
|
||
var tags []interface{}
|
||
for _, t := range tagOrder {
|
||
if tagSet[t] {
|
||
tags = append(tags, map[string]interface{}{
|
||
"name": t,
|
||
"description": tagDescriptions[t],
|
||
})
|
||
}
|
||
}
|
||
|
||
spec := map[string]interface{}{
|
||
"openapi": "3.0.3",
|
||
"info": map[string]interface{}{
|
||
"title": "CoreScope API",
|
||
"description": "MeshCore network analyzer — packet capture, node tracking, and mesh analytics.",
|
||
"version": version,
|
||
"license": map[string]interface{}{
|
||
"name": "MIT",
|
||
},
|
||
},
|
||
"paths": paths,
|
||
"tags": tags,
|
||
"components": map[string]interface{}{
|
||
"securitySchemes": map[string]interface{}{
|
||
"ApiKeyAuth": map[string]interface{}{
|
||
"type": "apiKey",
|
||
"in": "header",
|
||
"name": "X-API-Key",
|
||
},
|
||
},
|
||
"schemas": componentSchemas(),
|
||
},
|
||
}
|
||
|
||
return spec
|
||
}
|
||
|
||
// extractPathParams returns parameter names from a mux-style path like /api/nodes/{pubkey}.
|
||
func extractPathParams(path string) []string {
|
||
var params []string
|
||
for {
|
||
start := strings.Index(path, "{")
|
||
if start == -1 {
|
||
break
|
||
}
|
||
end := strings.Index(path[start:], "}")
|
||
if end == -1 {
|
||
break
|
||
}
|
||
params = append(params, path[start+1:start+end])
|
||
path = path[start+end+1:]
|
||
}
|
||
return params
|
||
}
|
||
|
||
// handleOpenAPISpec serves the OpenAPI 3.0 spec as JSON.
|
||
// The router is injected via RegisterRoutes storing it on the Server.
|
||
func (s *Server) handleOpenAPISpec(w http.ResponseWriter, r *http.Request) {
|
||
spec := buildOpenAPISpec(s.router, s.version)
|
||
|
||
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||
w.Header().Set("Access-Control-Allow-Origin", "*")
|
||
enc := json.NewEncoder(w)
|
||
enc.SetIndent("", " ")
|
||
if err := enc.Encode(spec); err != nil {
|
||
http.Error(w, fmt.Sprintf("failed to encode spec: %v", err), http.StatusInternalServerError)
|
||
}
|
||
}
|
||
|
||
// handleSwaggerUI serves a minimal Swagger UI page.
|
||
func (s *Server) handleSwaggerUI(w http.ResponseWriter, r *http.Request) {
|
||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||
fmt.Fprint(w, swaggerUIHTML)
|
||
}
|
||
|
||
const swaggerUIHTML = `<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<title>CoreScope API — Swagger UI</title>
|
||
<link rel="stylesheet" href="https://unpkg.com/swagger-ui-dist@5/swagger-ui.css">
|
||
<style>
|
||
html { box-sizing: border-box; overflow-y: scroll; }
|
||
*, *:before, *:after { box-sizing: inherit; }
|
||
body { margin: 0; background: #fafafa; }
|
||
.topbar { display: none; }
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<div id="swagger-ui"></div>
|
||
<script src="https://unpkg.com/swagger-ui-dist@5/swagger-ui-bundle.js"></script>
|
||
<script>
|
||
SwaggerUIBundle({
|
||
url: '/api/spec',
|
||
dom_id: '#swagger-ui',
|
||
deepLinking: true,
|
||
presets: [
|
||
SwaggerUIBundle.presets.apis,
|
||
SwaggerUIBundle.SwaggerUIStandalonePreset
|
||
],
|
||
layout: 'BaseLayout'
|
||
});
|
||
</script>
|
||
</body>
|
||
</html>`
|