feat(scope-audit): transport view of every repeater seen forwarding (#2142, API) (#2167)

## Summary

Backend half of #2142 (the page is the stacked PR #2169).

`GET /api/scope-audit` lists only repeaters that have answered a
declared-regions request: 17 of almost 200 nodes on the instance the
request came from. This adds a transport view, `GET
/api/scope-audit?mode=transport&window=…[&scope=X]`, with one row per
repeater **seen forwarding** in the window:

- `transported`: the named region scopes it was seen carrying, each with
packets and first/last seen (most packets first), plus unscoped,
unmatched and ambiguous-hop counts;
- `asked`, `declaredRegions`, `notObserved`, `configState`, `declaredAt`
where a declared answer exists; `asked: false` with null declared fields
where it does not ("not asked" is not "declares nothing");
- with `scope=X` (spelled `be`, `#be` or `BE`): `carriesScope` on every
row and `carrying` / `notCarrying` counts, which is the rollout report
the issue asks for.

`mode=declared` (or no `mode`) is unchanged. An unknown `mode` is a 400.

Partial fix for #2142: this PR provides the API; the page is #2169 and
the observer region filter #2171. The issue should only be considered
done once all three are in.

## How it follows the issue's notes

- **The scan does not change.** It is the same single
`ScopeAuditForwarding` pass, with every repeater and room in `nodes`
(plus any declared target) as a target instead of only the declared
ones. Same attribution rules: FLOOD-family route types only,
`minForwarderHopHexLen` (no 1-byte hops), and a hop whose prefix matches
more than one repeater is credited to none of them.
- **Identity lookup with 1,000+ pubkeys:** `scopeAuditNodeIdentities`
runs one `IN` query; exercised at 1,200 keys in the benchmark below.
- **Row count and response size:** measured below.
- **Cache:** the transport view is cached per window with the declared
view's TTLs (30 s; 5 min for 7d), under the same singleflight group with
a `transport|` key. The scope filter is applied to a copy of the cached
result, so one scan serves every scope. (`scope=`, not `region=`: across
the API `region=` is the observer IATA filter.)
- **Wording:** "transported" is documented as *seen carrying*, not
configured for; the declared side stays separate.

## Performance

`BenchmarkComputeScopeTransport`: 1,200 repeaters (#1975 quotes an
instance with 1,179), 30k flood transmissions in the last 24h on 4-hop
paths of 3-byte prefixes, a third region-scoped:

```
BenchmarkComputeScopeTransport-12   5   219253727 ns/op   425.4 response-KB   1200 rows
```

219 ms per compute, served from cache for 30 s (24h) or 5 min (7d); a
425 KB uncompressed response for 1,200 rows. This is new code with no
"before"; the declared view's numbers are untouched (same scan, same
cache).

### On real data (a staging copy of a production database and its live
feed)

| request | rows | result | time |
|---|---|---|---|
| declared view, 24h | 16 | (unchanged) | |
| `mode=transport`, 24h | **385** repeaters seen forwarding, 16 of them
with a declared answer | | 1.5 s cold |
| `mode=transport&scope=<scope A>`, 7d | 664 | **51 carrying it, 613
not** | 14.4 s cold |
| `mode=transport&scope=%23<SCOPE B>` (`#` and capitals), 7d, same
window now cached | 664 | 237 carrying, 427 not | 0.35 s |

The 7d cold compute is the same full-window hop scan the declared view
already pays for 7d (its own TTL comment measures 16.7 s on a
live-shaped database); like the declared view it is then cached for 5
minutes, and every scope filter is served from that one scan.
- Commit `ceaeb56b` renamed the filter from `region=` to `scope=` after
opening; the real-data figures above were taken before the rename and
are otherwise the same code.

## Tests

- Red commit `2a9b1286`: response types and a `mode=transport` stub that
returns no rows, plus tests. Four fail on the stub:
- repeaters that never answered are listed with `asked: false` and null
declared fields (and the answered one with its declared list and
`notObserved`);
- `scope=fr` / `%23fr` / `FR` marks carriers and non-carriers and counts
them; no scope means no `carriesScope` or counts;
  - an unknown mode is a 400;
- the transport and declared views are cached separately (requesting one
does not serve the other).
- Also added, passing on the stub by construction: a repeater with no
traffic in the window is not listed; the node blacklist applies.
- Green commit `60c669f4`: the implementation, `docs/api-spec.md` (new
`mode` and `scope` parameters, a Transport view section with the
response shape), and the benchmark.
- `make test`: 16 of 16 modules pass, including every existing
scope-audit test.

## Review notes

- Hidden-name and blacklist filtering use the same rules as the declared
view.
- `newTestStoreWithDB` now takes `testing.TB` so the benchmark can use
it (callers unchanged).
- No new `map[string]interface{}`; the response is a named struct.
- No config added.

## SonarLint

SonarQube Community rules (the SonarLint engine) were run on the files
this PR touches, and only issues on lines it adds are listed.

- **Fixed** in `060668b6`: S3776 (cognitive complexity 41 > 15) in the
new `computeScopeTransport`. It is split into target, visibility, row
and sort helpers; the response is the same and the transport tests are
unchanged.

## Checked against the repo rules

- Rebased onto `master` (`673442cc`). Each red commit was re-run and
still fails on its assertions; each green commit passes (TDD.md).
- `make test`: all 16 modules pass. Frontend suites: all pass except
`test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME`
and fails the same way on `master` locally (#2160 addresses it).
- No private data: test fixtures and examples use the codes the existing
tests use (SJC/SFO/LAX, `#be`-style scopes).
- No new `map[string]interface{}`, no hard-coded colours. `git add` was
explicit, and there is one logical change per commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Alex BandClaude Opus 5.5 authored and GitHub committed 2026-10-10 21:22:20 +02:00
1 parent 8a3b923e81
commit 696906315a
5 files changed
+594 -4

No files matched your search

+1 -1
View File
@@ -74,7 +74,7 @@ func TestGetNodePubkeysInArea_Polygon(t *testing.T) {
}
// newTestStoreWithDB builds a minimal PacketStore wired to the given DB and config.
func newTestStoreWithDB(t *testing.T, db *DB, cfg *Config) *PacketStore {
func newTestStoreWithDB(t testing.TB, db *DB, cfg *Config) *PacketStore {
t.Helper()
return &PacketStore{
db: db,
+4
View File
@@ -101,6 +101,10 @@ type Server struct {
scopeAuditCache map[string]*ScopeAuditResponse
scopeAuditCachedAt map[string]time.Time
scopeAuditSF singleflight.Group
// #2142: the transport view (?mode=transport), cached per window with the
// same TTLs; computed under scopeAuditSF with a "transport|" key.
scopeTransportCache map[string]*ScopeTransportResponse
scopeTransportCachedAt map[string]time.Time
// #1975: /api/scope-audit window cache and its single-flight guard, so a
// burst of viewers on a cold cache recomputes the network-wide scan once
+288
View File
@@ -849,6 +849,50 @@ type ScopeAuditResponse struct {
Repeaters []ScopeAuditRow `json:"repeaters"`
}
// ScopeTransportRow is one repeater seen forwarding in the window, in the
// transport view of the scope audit (#2142, ?mode=transport). Unlike
// ScopeAuditRow it does not require a declared-regions answer: "transported"
// means seen carrying, not configured for, and the declared side is reported
// where it exists (asked) and left null where it does not.
type ScopeTransportRow struct {
PublicKey string `json:"publicKey"`
Name *string `json:"name"` // null = no nodes row held for this key
Role *string `json:"role"`
// Transported lists the named region scopes this repeater was seen
// forwarding in the window, most packets first.
Transported []ScopeObservation `json:"transported"`
UnscopedPackets int64 `json:"unscopedPackets"` // plain floods forwarded
UnmatchedPackets int64 `json:"unmatchedPackets"` // scoped, but no region key held here
AmbiguousHops int64 `json:"ambiguousHops"` // hops shared with another repeater's prefix, credited to neither
// Asked is whether a declared-regions answer exists for this repeater.
// DeclaredRegions is null when it does not, [] when it answered with no
// named region. NotObserved (declared, not seen carried) is null when not
// asked.
Asked bool `json:"asked"`
DeclaredRegions []string `json:"declaredRegions"`
DeclaredWildcard bool `json:"declaredWildcard"`
ConfigState string `json:"configState,omitempty"`
DeclaredAt string `json:"declaredAt,omitempty"`
NotObserved []string `json:"notObserved"`
// CarriesScope is set only when the request named a scope (?scope=):
// whether this repeater was seen carrying it in the window.
CarriesScope *bool `json:"carriesScope,omitempty"`
}
// ScopeTransportResponse is GET /api/scope-audit?mode=transport.
type ScopeTransportResponse struct {
Mode string `json:"mode"` // "transport"
Window string `json:"window"`
Since string `json:"since"`
Scope string `json:"scope,omitempty"` // normalised ?scope= filter, when given
Carrying *int `json:"carrying,omitempty"`
NotCarry *int `json:"notCarrying,omitempty"`
Repeaters []ScopeTransportRow `json:"repeaters"`
}
// NodeScopesResponse is the payload for GET /api/nodes/{pubkey}/scopes: the
// observed-forwarding side (ScopeConformance, embedded BY VALUE so its three
// scope states sit at the JSON top level — unmatched and unscoped are
@@ -998,6 +1042,16 @@ func (s *Server) handleScopeAudit(w http.ResponseWriter, r *http.Request) {
sinceISO := time.Now().Add(-lookback).UTC().Format(time.RFC3339)
switch r.URL.Query().Get("mode") {
case "", "declared":
case "transport":
s.serveScopeTransport(w, r, window, sinceISO)
return
default:
writeError(w, 400, "mode must be declared or transport")
return
}
if cached, ok := s.scopeAuditCached(window); ok {
writeJSON(w, cached)
return
@@ -1259,3 +1313,237 @@ func (s *Server) computeScopeAudit(window, sinceISO string) (*ScopeAuditResponse
return resp, nil
}
// serveScopeTransport answers GET /api/scope-audit?mode=transport (#2142):
// the cached per-window transport view, with the optional region filter
// applied to a copy so the cached response is never changed.
func (s *Server) serveScopeTransport(w http.ResponseWriter, r *http.Request, window, sinceISO string) {
resp, ok := s.scopeTransportCached(window)
if !ok {
v, err, _ := s.scopeAuditSF.Do("transport|"+window, func() (interface{}, error) {
if cached, ok := s.scopeTransportCached(window); ok {
return cached, nil
}
fresh, cErr := s.computeScopeTransport(window, sinceISO)
if cErr != nil {
return nil, cErr
}
s.scopeTransportStore(window, fresh)
return fresh, nil
})
if err != nil {
writeError(w, 500, err.Error())
return
}
resp = v.(*ScopeTransportResponse)
}
// ?scope= names a region SCOPE (be, be-van). ?region= is kept for the
// observer IATA filter, as on every other endpoint.
if scope := strings.ToLower(normScope(strings.TrimSpace(r.URL.Query().Get("scope")))); scope != "" {
resp = withTransportScope(resp, scope)
}
writeJSON(w, resp)
}
func (s *Server) scopeTransportCached(window string) (*ScopeTransportResponse, bool) {
s.scopeAuditMu.Lock()
defer s.scopeAuditMu.Unlock()
cached, ok := s.scopeTransportCache[window]
if !ok || time.Since(s.scopeTransportCachedAt[window]) >= scopeAuditTTLFor(window) {
return nil, false
}
return cached, true
}
func (s *Server) scopeTransportStore(window string, resp *ScopeTransportResponse) {
s.scopeAuditMu.Lock()
defer s.scopeAuditMu.Unlock()
if s.scopeTransportCache == nil {
s.scopeTransportCache = make(map[string]*ScopeTransportResponse)
s.scopeTransportCachedAt = make(map[string]time.Time)
}
s.scopeTransportCache[window] = resp
s.scopeTransportCachedAt[window] = time.Now()
}
// withTransportScope returns a copy of resp in which every row says whether
// it was seen carrying scope, with the two counts. scope is already
// normalised (no '#', lower case).
func withTransportScope(resp *ScopeTransportResponse, scope string) *ScopeTransportResponse {
out := *resp
out.Scope = scope
out.Repeaters = make([]ScopeTransportRow, len(resp.Repeaters))
carrying, notCarrying := 0, 0
for i, row := range resp.Repeaters {
carries := false
for _, so := range row.Transported {
if strings.ToLower(so.Scope) == scope {
carries = true
break
}
}
if carries {
carrying++
} else {
notCarrying++
}
c := carries
row.CarriesScope = &c
out.Repeaters[i] = row
}
out.Carrying, out.NotCarry = &carrying, &notCarrying
return &out
}
// repeaterPubkeys lists every repeater and room this instance holds a nodes
// row for: the candidates the transport view attributes forwarding hops to.
func (db *DB) repeaterPubkeys() ([]string, error) {
rows, err := db.conn.Query(`SELECT public_key FROM nodes WHERE role IN ('repeater', 'room')`)
if err != nil {
return nil, fmt.Errorf("scope transport repeaters: %w", err)
}
defer rows.Close()
var out []string
for rows.Next() {
var pk string
if rows.Scan(&pk) == nil {
out = append(out, strings.ToLower(pk))
}
}
return out, rows.Err()
}
// computeScopeTransport builds one window's transport view (#2142): the same
// single forwarding scan as the declared view (ScopeAuditForwarding, same
// FLOOD-family route types and minimum hop length, ambiguous hops credited to
// nobody), with every known repeater as a target instead of only the ones that
// answered a declared-regions request. A repeater is listed when the scan
// attributed any forwarding to it in the window.
func (s *Server) computeScopeTransport(window, sinceISO string) (*ScopeTransportResponse, error) {
declaredByPK, targets, err := s.scopeTransportTargets()
if err != nil {
return nil, err
}
forwarding := map[string]*scopeAuditTargetAgg{}
if s.store != nil {
forwarding, err = s.store.ScopeAuditForwarding(sinceISO, targets)
if err != nil {
return nil, err
}
}
seen := make([]string, 0, len(forwarding))
for pk, agg := range forwarding {
if len(agg.scopes) > 0 || agg.unscopedPackets > 0 || agg.unmatchedPackets > 0 {
seen = append(seen, pk)
}
}
identities := s.db.scopeAuditNodeIdentities(seen)
resp := &ScopeTransportResponse{Mode: "transport", Window: window, Since: sinceISO, Repeaters: []ScopeTransportRow{}}
for _, pk := range seen {
id := identities[pk]
if s.scopeTransportHidden(pk, id) {
continue
}
resp.Repeaters = append(resp.Repeaters, scopeTransportRow(pk, id, forwarding[pk], declaredByPK))
}
sortScopeTransportRows(resp.Repeaters)
return resp, nil
}
// scopeTransportTargets is every repeater the scan should attribute
// forwarding to: those that answered a declared-regions request and every
// known repeater, sorted, with the declared answers by key.
func (s *Server) scopeTransportTargets() (map[string]DeclaredRegionsRow, []string, error) {
declared, err := s.db.AllCurrentDeclaredRegions()
if err != nil {
return nil, nil, err
}
declaredByPK := make(map[string]DeclaredRegionsRow, len(declared))
targetSet := make(map[string]bool, len(declared))
for _, d := range declared {
pk := strings.ToLower(d.Target)
declaredByPK[pk] = d
targetSet[pk] = true
}
repeaters, err := s.db.repeaterPubkeys()
if err != nil {
return nil, nil, err
}
for _, pk := range repeaters {
targetSet[pk] = true
}
targets := make([]string, 0, len(targetSet))
for pk := range targetSet {
targets = append(targets, pk)
}
sort.Strings(targets)
return declaredByPK, targets, nil
}
// scopeTransportHidden applies the blacklist and hidden name prefixes.
func (s *Server) scopeTransportHidden(pk string, id scopeAuditNodeIdentity) bool {
if s.cfg == nil {
return false
}
return s.cfg.IsBlacklisted(pk) || (id.Name != nil && s.cfg.IsNameHidden(*id.Name))
}
// scopeTransportRow is one repeater's row: the scopes it was seen carrying,
// busiest first, and, when it answered a declared-regions request, what it
// declares and which of those it was not seen carrying.
func scopeTransportRow(pk string, id scopeAuditNodeIdentity, agg *scopeAuditTargetAgg, declaredByPK map[string]DeclaredRegionsRow) ScopeTransportRow {
transported := make([]ScopeObservation, 0, len(agg.scopes))
for _, so := range agg.scopes {
transported = append(transported, *so)
}
sort.Slice(transported, func(i, j int) bool {
if transported[i].Packets != transported[j].Packets {
return transported[i].Packets > transported[j].Packets
}
return transported[i].Scope < transported[j].Scope
})
row := ScopeTransportRow{
PublicKey: pk,
Name: id.Name,
Role: id.Role,
Transported: transported,
UnscopedPackets: agg.unscopedPackets,
UnmatchedPackets: agg.unmatchedPackets,
AmbiguousHops: agg.ambiguousHops,
}
d, ok := declaredByPK[pk]
if !ok {
return row
}
named, wildcard := splitDeclaredRegions(d.RegionsCSV)
row.Asked = true
row.DeclaredRegions = named
row.DeclaredWildcard = wildcard
row.ConfigState = scopeAuditConfigState(named, wildcard)
row.DeclaredAt = d.ObservedAt
row.NotObserved = []string{}
for _, rgn := range named {
if agg.scopes[rgn] == nil {
row.NotObserved = append(row.NotObserved, rgn)
}
}
return row
}
// sortScopeTransportRows orders rows by name, falling back to the key.
func sortScopeTransportRows(rows []ScopeTransportRow) {
label := func(r ScopeTransportRow) string {
if r.Name != nil && *r.Name != "" {
return *r.Name
}
return r.PublicKey
}
sort.Slice(rows, func(i, j int) bool {
if an, bn := label(rows[i]), label(rows[j]); an != bn {
return an < bn
}
return rows[i].PublicKey < rows[j].PublicKey
})
}
+244
View File
@@ -0,0 +1,244 @@
package main
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"github.com/gorilla/mux"
)
// #2142: the scope audit lists only repeaters that answered a declared-regions
// request (17 of ~200 nodes on the instance that asked). The transport view
// lists every repeater seen forwarding in the window, with the region scopes
// it carried, its declared regions where known, and a scope filter (?scope=)
// that splits the fleet into carriers and non-carriers for rollout tracking.
var testFullPubkeyC = "cccc" + strings.Repeat("33", 30)
func getScopeTransport(t *testing.T, router *mux.Router, query string) ScopeTransportResponse {
t.Helper()
req := httptest.NewRequest("GET", "/api/scope-audit?mode=transport"+query, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d, want 200: %s", w.Code, w.Body.String())
}
var got ScopeTransportResponse
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
return got
}
func transportRow(t *testing.T, rows []ScopeTransportRow, pk string) ScopeTransportRow {
t.Helper()
for _, r := range rows {
if r.PublicKey == pk {
return r
}
}
t.Fatalf("no row for %s in %d rows", pk[:8], len(rows))
return ScopeTransportRow{}
}
func addRepeater(t *testing.T, srv *Server, pk, name string) {
t.Helper()
if _, err := srv.db.conn.Exec(`INSERT INTO nodes (public_key, name, role) VALUES (?, ?, 'repeater')
ON CONFLICT(public_key) DO UPDATE SET name = excluded.name, role = 'repeater'`, pk, name); err != nil {
t.Fatal(err)
}
}
// seedTwoRepeaters: A declared "be" and carries #be; B never answered and
// carries #fr twice; C is a repeater with no traffic in the window.
func seedTwoRepeaters(t *testing.T, srv *Server) {
t.Helper()
now := time.Now().UTC()
insertDeclared(t, srv, testFullPubkeyA, now.Format(time.RFC3339), "be", 0)
addRepeater(t, srv, testFullPubkeyA, "Alpha")
addRepeater(t, srv, testFullPubkeyB, "Bravo")
addRepeater(t, srv, testFullPubkeyC, "Charlie")
recent := now.Add(-time.Minute).Format(time.RFC3339)
seedTransmissionRouteAt(t, srv.store, testFullPubkeyA[:4], scopeMatched("#be"), RouteFlood, recent)
seedTransmissionRouteAt(t, srv.store, testFullPubkeyB[:4], scopeMatched("#fr"), RouteFlood, recent)
seedTransmissionRouteAt(t, srv.store, testFullPubkeyB[:4], scopeMatched("#fr"), RouteFlood, recent)
}
func TestScopeTransport_ListsRepeatersThatNeverAnswered(t *testing.T) {
srv, router := setupScopeAuditServer(t)
seedTwoRepeaters(t, srv)
got := getScopeTransport(t, router, "")
if got.Mode != "transport" {
t.Fatalf("mode = %q, want transport", got.Mode)
}
a := transportRow(t, got.Repeaters, testFullPubkeyA)
if !a.Asked || len(a.DeclaredRegions) != 1 || a.DeclaredRegions[0] != "be" {
t.Errorf("A: asked=%v declared=%v, want asked with [be]", a.Asked, a.DeclaredRegions)
}
if len(a.Transported) != 1 || a.Transported[0].Scope != "be" || a.Transported[0].Packets != 1 {
t.Errorf("A: transported = %+v, want be x1", a.Transported)
}
if len(a.NotObserved) != 0 {
t.Errorf("A: notObserved = %v, want none", a.NotObserved)
}
b := transportRow(t, got.Repeaters, testFullPubkeyB)
if b.Asked || b.DeclaredRegions != nil || b.NotObserved != nil {
t.Errorf("B: asked=%v declared=%v notObserved=%v, want not asked with null declared fields", b.Asked, b.DeclaredRegions, b.NotObserved)
}
if len(b.Transported) != 1 || b.Transported[0].Scope != "fr" || b.Transported[0].Packets != 2 {
t.Errorf("B: transported = %+v, want fr x2", b.Transported)
}
if b.Name == nil || *b.Name != "Bravo" {
t.Errorf("B: name = %v, want Bravo", b.Name)
}
}
func TestScopeTransport_OmitsRepeatersNotSeenForwarding(t *testing.T) {
srv, router := setupScopeAuditServer(t)
seedTwoRepeaters(t, srv)
for _, r := range getScopeTransport(t, router, "").Repeaters {
if r.PublicKey == testFullPubkeyC {
t.Fatalf("C forwarded nothing in the window and must not be listed: %+v", r)
}
}
}
func TestScopeTransport_ScopeFilterSplitsCarriers(t *testing.T) {
srv, router := setupScopeAuditServer(t)
seedTwoRepeaters(t, srv)
for _, q := range []string{"&scope=fr", "&scope=%23fr", "&scope=FR"} {
got := getScopeTransport(t, router, q)
if got.Scope != "fr" {
t.Errorf("%s: scope = %q, want normalised \"fr\"", q, got.Scope)
}
a := transportRow(t, got.Repeaters, testFullPubkeyA)
b := transportRow(t, got.Repeaters, testFullPubkeyB)
if a.CarriesScope == nil || *a.CarriesScope || b.CarriesScope == nil || !*b.CarriesScope {
t.Errorf("%s: carriesScope A=%v B=%v, want A false, B true", q, a.CarriesScope, b.CarriesScope)
}
if got.Carrying == nil || *got.Carrying != 1 || got.NotCarry == nil || *got.NotCarry != 1 {
t.Errorf("%s: carrying=%v notCarrying=%v, want 1 and 1", q, got.Carrying, got.NotCarry)
}
}
plain := getScopeTransport(t, router, "")
if plain.Scope != "" || plain.Carrying != nil || transportRow(t, plain.Repeaters, testFullPubkeyB).CarriesScope != nil {
t.Error("without a scope filter, no carriesScope or counts may be reported")
}
}
func TestScopeTransport_FiltersBlacklistedRepeater(t *testing.T) {
srv, router := setupScopeAuditServer(t)
seedTwoRepeaters(t, srv)
srv.cfg.NodeBlacklist = []string{testFullPubkeyB}
for _, r := range getScopeTransport(t, router, "").Repeaters {
if r.PublicKey == testFullPubkeyB {
t.Fatal("blacklisted repeater must be excluded")
}
}
}
func TestScopeAudit_RejectsUnknownMode(t *testing.T) {
_, router := setupScopeAuditServer(t)
req := httptest.NewRequest("GET", "/api/scope-audit?mode=bogus", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400 for an unknown mode", w.Code)
}
}
func TestScopeTransport_CachedSeparatelyFromDeclaredView(t *testing.T) {
srv, router := setupScopeAuditServer(t)
seedTwoRepeaters(t, srv)
declared := getScopeAudit(t, router, "")
transport := getScopeTransport(t, router, "")
if len(declared.Repeaters) != 1 {
t.Fatalf("declared view = %d rows, want only the repeater that answered", len(declared.Repeaters))
}
if len(transport.Repeaters) != 2 {
t.Fatalf("transport view = %d rows after the declared view was cached, want 2", len(transport.Repeaters))
}
}
// BenchmarkComputeScopeTransport: a fleet of 1,200 repeaters (#1975 quotes an
// instance with 1,179) and 30k flood transmissions in the last 24h, each
// forwarded along a 4-hop path of 3-byte prefixes, a third of them scoped to
// one of eight regions. Reports the response size too, as #2142 asks.
func BenchmarkComputeScopeTransport(b *testing.B) {
log.SetOutput(io.Discard)
b.Cleanup(func() { log.SetOutput(os.Stderr) })
conn, err := sql.Open("sqlite3", ":memory:")
if err != nil {
b.Fatal(err)
}
conn.SetMaxOpenConns(1)
for _, ddl := range []string{
`CREATE TABLE transmissions (id INTEGER PRIMARY KEY AUTOINCREMENT, raw_hex TEXT NOT NULL, hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL, route_type INTEGER, payload_type INTEGER, code1 TEXT, code2 TEXT, scope_name TEXT)`,
`CREATE INDEX idx_transmissions_first_seen ON transmissions(first_seen)`,
`CREATE TABLE observations (id INTEGER PRIMARY KEY AUTOINCREMENT, transmission_id INTEGER NOT NULL, path_json TEXT, timestamp INTEGER NOT NULL)`,
`CREATE INDEX idx_obs_tx ON observations(transmission_id)`,
`CREATE TABLE nodes (public_key TEXT PRIMARY KEY, name TEXT, role TEXT, configured_scope TEXT, configured_scope_at TEXT)`,
} {
if _, err := conn.Exec(ddl); err != nil {
b.Fatal(err)
}
}
db := &DB{conn: conn}
if err := db.detectSchema(context.Background(), conn); err != nil {
b.Fatal(err)
}
const fleet = 1200
pks := make([]string, fleet)
conn.Exec("BEGIN")
for i := range pks {
pks[i] = fmt.Sprintf("%06x", i*13+0x100000) + strings.Repeat("ab", 29)
scope := sql.NullString{}
if i%7 == 0 {
scope = sql.NullString{String: "be,be-van,*", Valid: true}
}
conn.Exec(`INSERT INTO nodes (public_key, name, role, configured_scope, configured_scope_at) VALUES (?, ?, 'repeater', ?, ?)`,
pks[i], fmt.Sprintf("Repeater %d", i), scope, time.Now().UTC().Format(time.RFC3339))
}
regions := []string{"#be", "#be-van", "#be-ant", "#be-gnt", "#be-lge", "#nl", "#nl-ams", "#de"}
now := time.Now().UTC()
for i := 0; i < 30000; i++ {
var scope interface{}
route := RouteFlood
if i%3 == 0 {
route, scope = 0, regions[i%len(regions)]
}
res, _ := conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, code1, code2, scope_name)
VALUES ('AA', ?, ?, ?, 1, '1234', '00', ?)`, fmt.Sprintf("bench%d", i), now.Add(-time.Duration(i)*2*time.Second).Format(time.RFC3339), route, scope)
id, _ := res.LastInsertId()
path := fmt.Sprintf(`["%s","%s","%s","%s"]`, strings.ToUpper(pks[i%fleet][:6]), strings.ToUpper(pks[(i*7)%fleet][:6]),
strings.ToUpper(pks[(i*11)%fleet][:6]), strings.ToUpper(pks[(i*17)%fleet][:6]))
conn.Exec(`INSERT INTO observations (transmission_id, path_json, timestamp) VALUES (?, ?, ?)`, id, path, now.Unix())
}
conn.Exec("COMMIT")
cfg := &Config{Port: 3000}
srv := NewServer(db, cfg, NewHub())
srv.store = newTestStoreWithDB(b, db, cfg)
since := now.Add(-24 * time.Hour).Format(time.RFC3339)
b.ResetTimer()
var resp *ScopeTransportResponse
for i := 0; i < b.N; i++ {
if resp, err = srv.computeScopeTransport("24h", since); err != nil {
b.Fatal(err)
}
}
b.StopTimer()
body, _ := json.Marshal(resp)
b.ReportMetric(float64(len(resp.Repeaters)), "rows")
b.ReportMetric(float64(len(body))/1024, "response-KB")
}
+57 -3
View File
@@ -1890,9 +1890,11 @@ not being the same as "declared nothing"), which apply here identically.
### Query Parameters
| Param | Type | Default | Description |
|----------|--------|---------|---------------------------------|
| `window` | string | `24h` | Time window: `1h`, `24h`, `7d` |
| Param | Type | Default | Description |
|----------|--------|------------|---------------------------------|
| `window` | string | `24h` | Time window: `1h`, `24h`, `7d` |
| `mode` | string | `declared` | `declared` (this section) or `transport` (see [Transport view](#transport-view-modetransport)) |
| `scope` | string | — | `mode=transport` only: region scope to split the fleet by (`be`, `be-van`), with or without `#`, any case |
### Response `200`
@@ -1932,6 +1934,12 @@ not being the same as "declared nothing"), which apply here identically.
{ "error": "window must be 1h, 24h, or 7d" }
```
or, for an unknown `mode`:
```json
{ "error": "mode must be declared or transport" }
```
**Notes:**
- Only repeaters with **at least one** declared-regions answer appear in `repeaters`. A
repeater that was never successfully asked is absent, not shown as a row that declares
@@ -2038,6 +2046,52 @@ not being the same as "declared nothing"), which apply here identically.
---
### Transport view (`mode=transport`)
Issue #2142. One row per repeater **seen forwarding** in the window, whether or not it has
ever answered a declared-regions request. Built from the same single forwarding scan as the
declared view, with the same attribution rules (FLOOD-family route types only, no 1-byte
hops, a hop whose prefix matches several repeaters is credited to none of them), but with
every repeater and room this instance knows as a target instead of only the declared ones.
"Transported" means **seen carrying**, not configured for; the declared side is reported
separately where it exists. Cached per window with the same TTLs as the declared view; the
`scope` filter is applied to the cached result.
```jsonc
{
"mode": "transport",
"window": string,
"since": string (ISO),
"scope": string | undefined, // normalised ?scope= filter ("#FR" -> "fr"); present only when given
"carrying": number | undefined, // rows with carriesScope: true; only with scope
"notCarrying": number | undefined, // rows with carriesScope: false; only with scope
"repeaters": [
{
"publicKey": string,
"name": string | null, // null = no nodes row for this key
"role": string | null,
"transported": [ // named region scopes seen forwarded, most packets first
{ "scope": string, "packets": number, "firstSeen": string (ISO), "lastSeen": string (ISO) }
],
"unscopedPackets": number, // plain floods forwarded
"unmatchedPackets": number, // scoped packets whose region this instance holds no key for
"ambiguousHops": number, // hops shared with another repeater's prefix, credited to neither
"asked": boolean, // a declared-regions answer exists
"declaredRegions": [string] | null,// null when not asked; [] when it answered with no named region
"declaredWildcard": boolean,
"configState": string | undefined, // as in the declared view; only when asked
"declaredAt": string | undefined, // only when asked
"notObserved": [string] | null, // declared regions not seen carried; null when not asked
"carriesScope": boolean | undefined // only with scope
}
]
}
```
A repeater with no attributed forwarding in the window (only ambiguous hops, or no traffic)
is not listed. Rows are sorted by name, then public key.
## GET /api/scope-stats
Scope-based packet statistics over a time window. Requires ingestor `scope_name_v1` migration to have run.