diff --git a/AREAS.md b/AREAS.md new file mode 100644 index 00000000..2b24eb42 --- /dev/null +++ b/AREAS.md @@ -0,0 +1,143 @@ +# Areas + +`config.json`'s `areas` map lets you draw named geographic regions — cities, +regions, whole countries, even continents — and have CoreScope automatically +count which nodes fall inside each one, roll counts up through nested areas +(a city inside a country inside a continent), and (optionally) classify +foreign vs. domestic traffic from the same boundaries. + +There is no per-deployment code involved. Everything below is `config.json` +content — the areas feature itself (`AreaEntry`, `AreaForPoint`, +`AreaKeyForPoint`, `AreaKeysForPoint`, `HomeArea`, `computeScopeAdoptionByArea` +in `cmd/server/`) is generic and works the same regardless of what regions you +draw or which country you run CoreScope in. + +## Defining an area + +```json +"areas": { + "DK": { + "label": "Danmark (alle)", + "regionScopes": ["dk"], + "polygon": [[54.85, 8.65], [55.50, 8.10], [57.10, 8.20], ...] + }, + "AAR": { + "label": "Aarhus by", + "regionScopes": ["dk-aarhus"], + "polygon": [[56.35, 10.33], [56.31, 10.45], ...] + }, + "EU": { + "label": "Europa (alle)", + "regionScopes": ["eu", "europe"], + "latMin": 34.0, "latMax": 71.5, "lonMin": -25.0, "lonMax": 45.0 + } +} +``` + +Each entry has three parts: + +- **`label`** — the human-readable name shown in the UI. +- **Geometry** — either: + - `polygon`: a list of `[lat, lon]` points tracing a real boundary + (coastline, border). Use this when you care about precision — the + boundary between two adjacent countries/regions especially, since a + simple box will bleed across it. + - `latMin` / `latMax` / `lonMin` / `lonMax`: a plain bounding box. Good + enough for a rough first pass, or for areas with no close neighbor to + worry about overlapping (e.g. a whole continent). + + If `polygon` has at least 3 points, it's used; otherwise the code falls + back to the box. An area can have one or the other, not both meaningfully + at once. +- **`regionScopes`** (optional) — links this area to one or more hashRegions + channel scopes (e.g. `["dk-aarhus"]`, each stored *without* the leading + `#`). This powers the Scopes tab's "Scope Adoption by Area" section: which + nodes physically in this area actually use (via their own `default_scope`, + or by relaying it) *any* of the regions this area represents. Most areas + only need one scope, but a broad umbrella area can link several — e.g. + Europa linking both `"eu"` and `"europe"` if both names see real traffic — + and a node matching any one of them counts as supporting the area. Leave + it unset/empty if there's no matching hashRegion — the area still works + for everything else (badges, node counts, the area filter), it just won't + have anything to compare scope-adoption against. + +## Hierarchy: draw it, don't declare it + +**There is no `"parent"` field.** An area doesn't know it's "inside" another +area — that's worked out purely from geometry, every time a node needs to be +counted: for each area, does the node's `(lat, lon)` fall inside its +geometry? If yes, the node counts toward that area. A node in Aarhus falls +inside `AAR`'s polygon *and* a broader `JYL` (Jylland) polygon *and* `DK` +*and* `EU`, simultaneously — no code anywhere needs to know those areas are +related, and none of them need to enumerate their members. + +Practically, this means: + +- To make a country-level area (e.g. "Danmark") show the *whole country's* + totals rather than just the leftover nodes no smaller area already + claimed, its geometry must actually contain those smaller areas' + geometry. Draw the country boundary generously enough to cover all its + regions/cities and it just works. +- Adding a new area — a new city, region, or country — never requires + touching any other area's config, or any code. Draw its boundary, add the + entry, restart. If it geographically sits inside an existing broader + area, it's automatically included in that area's totals from the next + restart on. +- The same applies at any scale: adding e.g. "Finland" as a new country + area automatically starts contributing to "Europa (alle)" the moment its + polygon is added, with zero changes to the Europe entry. + +Two different lookups use this geometry, for different purposes: + +- **Single most-specific match** (`AreaForPoint` / `AreaKeyForPoint`) — used + for per-node badges (Wardriving tab's GPS-share/session area tags): picks + the *smallest* matching area, so a node in Aarhus is labeled "Aarhus by", + not "Danmark". +- **All containing areas** (`AreaKeysForPoint`) — used for aggregate counts + (Scope Adoption by Area): a node counts toward *every* area it + geographically sits inside, so country/continent totals genuinely + aggregate their sub-areas instead of only showing leftovers. + +## `homeArea`: linking foreign/domestic classification to an area + +```json +"homeArea": "DK" +``` + +`homeArea` names an entry in `areas` whose geometry becomes the effective +`geo_filter` — the boundary the Foreign Traffic tab, the Nodes page +All/Domestic/Foreign filter, and the live map's declutter logic all use to +decide "is this node ours or foreign". + +Before `homeArea` existed, `geo_filter` was a second, independently-drawn +boundary — easy to let drift out of sync with whatever the "home" area +actually looked like (this happened in practice: a too-loose home-country +box quietly claimed a neighboring country's nodes as domestic, and fixing +the area's polygon didn't fix `geo_filter` until this field existed). + +Set `homeArea` to the key of whichever area represents "home" for your +deployment. Leave it unset (or pointing at a key that doesn't exist in +`areas`) to keep using a standalone `geo_filter` value exactly as before — +this is fully backward compatible. + +## Adding a new area + +1. Get a boundary. A rough bounding box is a fine starting point + (`latMin`/`latMax`/`lonMin`/`lonMax`); upgrade to a `polygon` later if it + turns out to overlap a neighbor. +2. Add it under `areas` in `config.json`. +3. Restart CoreScope — config is only read at startup, there's no hot-reload. +4. Done. It's picked up everywhere automatically: the area filter dropdown, + per-node badges, Scope Adoption by Area, and (if it's nested inside a + broader area) that broader area's totals. + +If you draw a `polygon`, verify it before deploying: fetch +`/api/nodes?limit=5000` and `/api/config/areas/polygons`, and run every +node you know belongs on each side of the new boundary through a +point-in-polygon check (ray-casting — see `AreaForPoint` in +`cmd/server/config.go` for the exact algorithm CoreScope uses) to catch +bleed across a shared border before it ships. A box is more forgiving of +imprecision than a polygon since there's usually nothing on the other side +of an unclaimed edge to misclassify — but two adjacent countries sharing a +long border will bleed into each other badly with simple boxes, which is +why Denmark/Sweden/Norway/Germany all ended up as polygons. diff --git a/cmd/server/area_filter_test.go b/cmd/server/area_filter_test.go index e08082c0..989c8790 100644 --- a/cmd/server/area_filter_test.go +++ b/cmd/server/area_filter_test.go @@ -296,6 +296,38 @@ func TestHandleConfigAreas(t *testing.T) { } } +func TestHandleConfigAreas_RegionScopes(t *testing.T) { + db := setupTestDBv2(t) + cfg := &Config{Areas: map[string]AreaEntry{ + "AAR": {Label: "Aarhus by", RegionScopes: []string{"dk-aarhus"}}, + "MST": {Label: "Maastricht"}, + }} + + r := mux.NewRouter() + srv := &Server{db: db, cfg: cfg} + r.HandleFunc("/api/config/areas", srv.handleConfigAreas).Methods("GET") + + req := httptest.NewRequest(http.MethodGet, "/api/config/areas", nil) + w := httptest.NewRecorder() + r.ServeHTTP(w, req) + + var result []map[string]interface{} + if err := json.NewDecoder(w.Body).Decode(&result); err != nil { + t.Fatalf("decode: %v", err) + } + byKey := map[string]map[string]interface{}{} + for _, entry := range result { + byKey[entry["key"].(string)] = entry + } + aarScopes, _ := byKey["AAR"]["regionScopes"].([]interface{}) + if len(aarScopes) != 1 || aarScopes[0] != "dk-aarhus" { + t.Errorf("AAR regionScopes = %v, want [\"dk-aarhus\"]", byKey["AAR"]["regionScopes"]) + } + if _, present := byKey["MST"]["regionScopes"]; present { + t.Errorf("MST should omit regionScopes entirely (unset), got %v", byKey["MST"]["regionScopes"]) + } +} + func TestHandleConfigAreasEmpty(t *testing.T) { db := setupTestDBv2(t) cfg := &Config{} diff --git a/cmd/server/channel_message_area_test.go b/cmd/server/channel_message_area_test.go new file mode 100644 index 00000000..e4612d49 --- /dev/null +++ b/cmd/server/channel_message_area_test.go @@ -0,0 +1,157 @@ +package main + +import ( + "encoding/json" + "net/http/httptest" + "testing" + "time" +) + +// TestHandleChannelMessages_EntryPointArea covers dborup's request: a +// channel message needs BOTH the scope it was sent with (already shown as +// "Scope: #dk (Danmark (alle))") AND the area the sender was actually +// physically in, resolved from the message's own path[0] entry-point +// repeater -- distinct because someone in Aarhus can still send with the +// broad #dk scope. Also confirms the internal "entryPrefix" field used to +// carry path[0] through to the resolution step never reaches the client. +func TestHandleChannelMessages_EntryPointArea(t *testing.T) { + srv, router := setupTestServer(t) + if _, err := srv.db.conn.Exec(`DELETE FROM transmissions`); err != nil { + t.Fatalf("clear transmissions: %v", err) + } + if _, err := srv.db.conn.Exec(`DELETE FROM observations`); err != nil { + t.Fatalf("clear observations: %v", err) + } + if !srv.db.hasScopeName { + if _, err := srv.db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add scope_name column: %v", err) + } + srv.db.hasScopeName = true + } + + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "AAR": {Label: "Aarhus by", LatMin: f(56.05), LatMax: f(56.25), LonMin: f(9.95), LonMax: f(10.35)}, + } + + // A repeater physically in Aarhus, resolvable only via a unique + // full-length prefix match (mirrors TestResolveHopsAPI_UniquePrefix). + if _, err := srv.db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, lat, lon, role) VALUES (?, ?, ?, ?, ?)", + "aar0011223344", "AarhusRepeater", 56.1503, 10.1965, "repeater"); err != nil { + t.Fatalf("insert node: %v", err) + } + srv.store.InvalidateNodeCache() + + now := time.Now().UTC().Format(time.RFC3339) + res, err := srv.db.conn.Exec( + `INSERT INTO transmissions (raw_hex,hash,first_seen,route_type,payload_type,channel_hash,decoded_json,scope_name) VALUES (?,?,?,0,5,'#dk',?,'#dk')`, + "aa", "chmsg1", now, `{"sender":"AarhusSender","text":"AarhusSender: hey from #dk"}`, + ) + if err != nil { + t.Fatalf("insert tx: %v", err) + } + txID, _ := res.LastInsertId() + + obsRes, err := srv.db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES (?,?,?)`, "obsAAR", "AarhusObs", "AAR") + if err != nil { + t.Fatalf("insert observer: %v", err) + } + obsIdx, _ := obsRes.LastInsertId() + if _, err := srv.db.conn.Exec( + `INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp) VALUES (?,?,?,?,?,?)`, + txID, obsIdx, 1.0, -90.0, `["aar0011223344"]`, time.Now().Unix(), + ); err != nil { + t.Fatalf("insert observation: %v", err) + } + + req := httptest.NewRequest("GET", "/api/channels/%23dk/messages?limit=10", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + if w.Code != 200 { + t.Fatalf("status=%d body=%s", w.Code, w.Body.String()) + } + + var body map[string]interface{} + if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil { + t.Fatalf("decode: %v", err) + } + messages, _ := body["messages"].([]interface{}) + if len(messages) != 1 { + t.Fatalf("messages = %+v, want 1", messages) + } + msg, _ := messages[0].(map[string]interface{}) + + if msg["scope"] != "#dk" { + t.Errorf("scope = %v, want #dk", msg["scope"]) + } + if msg["area"] != "Aarhus by" { + t.Errorf("area = %v, want \"Aarhus by\" (resolved from path[0], not the #dk scope)", msg["area"]) + } + if _, present := msg["entryPrefix"]; present { + t.Error("entryPrefix must never reach the client -- it's an internal-only intermediate field") + } +} + +// TestHandleChannelMessages_EntryPointArea_Unresolved confirms "area" is +// simply omitted (never guessed) when the entry-point prefix doesn't +// resolve unambiguously, or when no areas are configured at all. +func TestHandleChannelMessages_EntryPointArea_Unresolved(t *testing.T) { + srv, router := setupTestServer(t) + if _, err := srv.db.conn.Exec(`DELETE FROM transmissions`); err != nil { + t.Fatalf("clear transmissions: %v", err) + } + if _, err := srv.db.conn.Exec(`DELETE FROM observations`); err != nil { + t.Fatalf("clear observations: %v", err) + } + if !srv.db.hasScopeName { + if _, err := srv.db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add scope_name column: %v", err) + } + srv.db.hasScopeName = true + } + // No areas configured at all. + srv.cfg.Areas = nil + + now := time.Now().UTC().Format(time.RFC3339) + res, err := srv.db.conn.Exec( + `INSERT INTO transmissions (raw_hex,hash,first_seen,route_type,payload_type,channel_hash,decoded_json,scope_name) VALUES (?,?,?,0,5,'#dk',?,'#dk')`, + "aa", "chmsg2", now, `{"sender":"Someone","text":"Someone: hi"}`, + ) + if err != nil { + t.Fatalf("insert tx: %v", err) + } + txID, _ := res.LastInsertId() + obsRes, err := srv.db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES (?,?,?)`, "obsX", "ObsX", "XXX") + if err != nil { + t.Fatalf("insert observer: %v", err) + } + obsIdx, _ := obsRes.LastInsertId() + if _, err := srv.db.conn.Exec( + `INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp) VALUES (?,?,?,?,?,?)`, + txID, obsIdx, 1.0, -90.0, `["deadbeef99"]`, time.Now().Unix(), + ); err != nil { + t.Fatalf("insert observation: %v", err) + } + + req := httptest.NewRequest("GET", "/api/channels/%23dk/messages?limit=10", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + if w.Code != 200 { + t.Fatalf("status=%d body=%s", w.Code, w.Body.String()) + } + var body map[string]interface{} + if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil { + t.Fatalf("decode: %v", err) + } + messages, _ := body["messages"].([]interface{}) + if len(messages) != 1 { + t.Fatalf("messages = %+v, want 1", messages) + } + msg, _ := messages[0].(map[string]interface{}) + if _, present := msg["area"]; present { + t.Errorf("area = %v, want absent (no areas configured)", msg["area"]) + } + if _, present := msg["entryPrefix"]; present { + t.Error("entryPrefix must never reach the client") + } +} diff --git a/cmd/server/config.go b/cmd/server/config.go index e259a31e..af3c139a 100644 --- a/cmd/server/config.go +++ b/cmd/server/config.go @@ -4,6 +4,7 @@ import ( "encoding/json" "fmt" "log" + "math" "os" "path/filepath" "strings" @@ -23,6 +24,95 @@ type AreaEntry struct { LatMax *float64 `json:"latMax,omitempty"` LonMin *float64 `json:"lonMin,omitempty"` LonMax *float64 `json:"lonMax,omitempty"` + + // RegionScopes links this area to one or more hashRegions channel + // scopes (e.g. "dk-aarhus"; a broad umbrella area can have several, + // e.g. Europa linking both "eu" and "europe"), stored without the + // leading "#" — callers should run each through regions.Normalize + // before comparing against a scope_name. Left empty when no confident + // area<->scope mapping exists. + RegionScopes []string `json:"regionScopes,omitempty"` +} + +// AreaForPoint returns the label of the most specific configured area that +// contains (lat, lon), preferring the smallest matching area when several +// nested areas overlap (e.g. a point inside both "Odense by" and "Fyn"). +// Returns ok=false for (0,0)/no-fix points or when no area matches. +func AreaForPoint(lat, lon float64, areas map[string]AreaEntry) (label string, ok bool) { + _, label, ok = areaMatchForPoint(lat, lon, areas) + return label, ok +} + +// AreaKeyForPoint is AreaForPoint but returns the area's config key (e.g. +// "ODE") instead of its display label — for callers that need to look up +// other fields on the matched AreaEntry (e.g. RegionScopes), not just show +// the name. +func AreaKeyForPoint(lat, lon float64, areas map[string]AreaEntry) (key string, ok bool) { + key, _, ok = areaMatchForPoint(lat, lon, areas) + return key, ok +} + +// AreaKeysForPoint returns every configured area's key whose geometry +// contains (lat, lon) — unlike AreaForPoint/AreaKeyForPoint (single +// most-specific match, for per-node badges), this returns ALL of them, so +// a point inside both "Aarhus by" and the broader "Jylland"/"Danmark +// (alle)" counts toward all three. For aggregate reporting +// (computeScopeAdoptionByArea) where a country-level area should roll up +// its sub-areas' totals rather than only catching leftovers no smaller +// area claimed. Returns nil for (0,0)/no-fix points. +func AreaKeysForPoint(lat, lon float64, areas map[string]AreaEntry) []string { + if lat == 0 && lon == 0 { + return nil + } + var keys []string + for k, a := range areas { + gf := &geofilter.Config{Polygon: a.Polygon, LatMin: a.LatMin, LatMax: a.LatMax, LonMin: a.LonMin, LonMax: a.LonMax} + if geofilter.PassesFilter(lat, lon, gf) { + keys = append(keys, k) + } + } + return keys +} + +func areaMatchForPoint(lat, lon float64, areas map[string]AreaEntry) (key, label string, ok bool) { + if lat == 0 && lon == 0 { + return "", "", false + } + bestSpan := math.MaxFloat64 + for k, a := range areas { + gf := &geofilter.Config{Polygon: a.Polygon, LatMin: a.LatMin, LatMax: a.LatMax, LonMin: a.LonMin, LonMax: a.LonMax} + if !geofilter.PassesFilter(lat, lon, gf) { + continue + } + span := areaSpan(a) + if span < bestSpan { + bestSpan = span + key = k + label = a.Label + ok = true + } + } + return key, label, ok +} + +// areaSpan approximates an area's size as its bounding-box extent in +// degrees², used only to rank overlapping areas from most to least specific. +func areaSpan(a AreaEntry) float64 { + var latMin, latMax, lonMin, lonMax float64 + switch { + case len(a.Polygon) > 0: + latMin, latMax = a.Polygon[0][0], a.Polygon[0][0] + lonMin, lonMax = a.Polygon[0][1], a.Polygon[0][1] + for _, p := range a.Polygon { + latMin, latMax = math.Min(latMin, p[0]), math.Max(latMax, p[0]) + lonMin, lonMax = math.Min(lonMin, p[1]), math.Max(lonMax, p[1]) + } + case a.LatMin != nil && a.LatMax != nil && a.LonMin != nil && a.LonMax != nil: + latMin, latMax, lonMin, lonMax = *a.LatMin, *a.LatMax, *a.LonMin, *a.LonMax + default: + return math.MaxFloat64 + } + return (latMax - latMin) * (lonMax - lonMin) } // ListLimitsConfig defines maximum row limits for list endpoints to prevent DoS. @@ -156,6 +246,15 @@ type Config struct { Areas map[string]AreaEntry `json:"areas,omitempty"` + // HomeArea names an entry in Areas whose geometry defines "home" for + // the foreign/domestic classification (geo_filter above), instead of + // maintaining a second, separately-drawn boundary that can drift out + // of sync with the area map (see AreaForPoint/AreaKeysForPoint). + // When set and the named area exists, it takes priority over any + // standalone GeoFilter value — see (*Server).getGeoFilter. Leave + // unset to keep using GeoFilter exactly as before. + HomeArea string `json:"homeArea,omitempty"` + Timestamps *TimestampConfig `json:"timestamps,omitempty"` // CORSAllowedOrigins is the list of origins permitted to make cross-origin diff --git a/cmd/server/config_client_geofilter_test.go b/cmd/server/config_client_geofilter_test.go index 7cb7fc8d..188c5794 100644 --- a/cmd/server/config_client_geofilter_test.go +++ b/cmd/server/config_client_geofilter_test.go @@ -76,3 +76,49 @@ func TestConfigClientOmitsGeoFilterWhenUnconfigured(t *testing.T) { t.Error("expected geoFilter to be omitted from /api/config/client when unconfigured") } } + +// TestGetGeoFilter_HomeAreaTakesPriority covers the new HomeArea linkage: +// when cfg.HomeArea names an existing area, its geometry is the effective +// geo_filter -- a single boundary shared with the areas system instead of +// a second copy that can drift out of sync (exactly what happened with +// Germany's box vs. Denmark's area earlier this session). +func TestGetGeoFilter_HomeAreaTakesPriority(t *testing.T) { + srv, _ := setupTestServer(t) + standaloneLat := 10.0 + srv.cfg.GeoFilter = &GeoFilterConfig{LatMin: &standaloneLat} // deliberately different from the area + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "DK": {Label: "Danmark", LatMin: f(54.5), LatMax: f(57.8), LonMin: f(8.0), LonMax: f(15.25)}, + } + srv.cfg.HomeArea = "DK" + + gf := srv.getGeoFilter() + if gf == nil || gf.LatMin == nil || *gf.LatMin != 54.5 { + t.Fatalf("getGeoFilter() = %+v, want DK area's LatMin=54.5 (HomeArea should win over the standalone GeoFilter)", gf) + } +} + +// TestGetGeoFilter_FallsBackWhenHomeAreaUnresolved confirms existing +// deployments (no HomeArea configured, or one naming a since-removed area) +// see no behavior change -- the standalone GeoFilter still applies. +func TestGetGeoFilter_FallsBackWhenHomeAreaUnresolved(t *testing.T) { + srv, _ := setupTestServer(t) + standaloneLat := 10.0 + srv.cfg.GeoFilter = &GeoFilterConfig{LatMin: &standaloneLat} + + t.Run("HomeArea unset", func(t *testing.T) { + srv.cfg.HomeArea = "" + gf := srv.getGeoFilter() + if gf == nil || gf.LatMin == nil || *gf.LatMin != 10.0 { + t.Errorf("getGeoFilter() = %+v, want the standalone GeoFilter (LatMin=10.0)", gf) + } + }) + + t.Run("HomeArea names a nonexistent area", func(t *testing.T) { + srv.cfg.HomeArea = "NOPE" + gf := srv.getGeoFilter() + if gf == nil || gf.LatMin == nil || *gf.LatMin != 10.0 { + t.Errorf("getGeoFilter() = %+v, want the standalone GeoFilter (LatMin=10.0)", gf) + } + }) +} diff --git a/cmd/server/config_test.go b/cmd/server/config_test.go index b78968ab..27b1fc44 100644 --- a/cmd/server/config_test.go +++ b/cmd/server/config_test.go @@ -515,3 +515,85 @@ func TestApplyListLimitsDefaults(t *testing.T) { } }) } + +func TestAreaForPoint(t *testing.T) { + f := func(v float64) *float64 { return &v } + + areas := map[string]AreaEntry{ + "DK": { + Label: "Danmark (alle)", + LatMin: f(54.5), LatMax: f(57.8), + LonMin: f(8.0), LonMax: f(15.25), + }, + "FYN": { + Label: "Fyn", + LatMin: f(54.9), LatMax: f(55.65), + LonMin: f(9.85), LonMax: f(11.0), + }, + "ODE": { + Label: "Odense by", + LatMin: f(55.32), LatMax: f(55.45), + LonMin: f(10.3), LonMax: f(10.5), + }, + } + + t.Run("picks the most specific nested area", func(t *testing.T) { + label, ok := AreaForPoint(55.4047, 10.381, areas) // central Odense + if !ok || label != "Odense by" { + t.Errorf("expected Odense by, got %q (ok=%v)", label, ok) + } + }) + + t.Run("falls back to a broader area when no narrower one matches", func(t *testing.T) { + label, ok := AreaForPoint(55.0, 10.6, areas) // Fyn but not Odense + if !ok || label != "Fyn" { + t.Errorf("expected Fyn, got %q (ok=%v)", label, ok) + } + }) + + t.Run("no match outside every area", func(t *testing.T) { + _, ok := AreaForPoint(60.0, 20.0, areas) + if ok { + t.Error("expected no match far outside Denmark") + } + }) + + t.Run("zero coordinates never match", func(t *testing.T) { + _, ok := AreaForPoint(0, 0, areas) + if ok { + t.Error("expected (0,0) to never match an area") + } + }) + + t.Run("empty areas map", func(t *testing.T) { + _, ok := AreaForPoint(55.4, 10.4, map[string]AreaEntry{}) + if ok { + t.Error("expected no match with no configured areas") + } + }) + + t.Run("AreaKeysForPoint returns every containing area, not just the most specific", func(t *testing.T) { + keys := AreaKeysForPoint(55.4047, 10.381, areas) // central Odense -- also inside Fyn and DK + want := map[string]bool{"DK": true, "FYN": true, "ODE": true} + if len(keys) != len(want) { + t.Fatalf("got %v, want all 3 of %v", keys, want) + } + for _, k := range keys { + if !want[k] { + t.Errorf("unexpected key %q in %v", k, keys) + } + } + }) + + t.Run("AreaKeysForPoint returns nil outside every area", func(t *testing.T) { + if keys := AreaKeysForPoint(60.0, 20.0, areas); keys != nil { + t.Errorf("expected nil, got %v", keys) + } + }) + + t.Run("AreaKeysForPoint returns nil for zero coordinates", func(t *testing.T) { + if keys := AreaKeysForPoint(0, 0, areas); keys != nil { + t.Errorf("expected nil, got %v", keys) + } + }) +} diff --git a/cmd/server/db.go b/cmd/server/db.go index 97d29455..34079c79 100644 --- a/cmd/server/db.go +++ b/cmd/server/db.go @@ -1945,10 +1945,14 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region . } } var hops int + var entryPrefix string if pathJSON.Valid { - var h []interface{} + var h []string if json.Unmarshal([]byte(pathJSON.String), &h) == nil { hops = len(h) + if len(h) > 0 { + entryPrefix = h[0] + } } } senderTs := decoded["sender_timestamp"] @@ -1967,6 +1971,7 @@ func (db *DB) GetChannelMessages(channelHash string, limit, offset int, region . "snr": nullFloat(snr), "scope": nullStr(scopeName), "routeType": nullInt(routeType), + "entryPrefix": entryPrefix, }, Repeats: 1, } @@ -2239,6 +2244,151 @@ func (db *DB) GetNodesByDefaultScope() (map[string][]RepeaterRef, error) { return result, rows.Err() } +// nodeAreaScopeInput is one node's position + name + default_scope + +// pubkey — the raw input to computeScopeAdoptionByArea. DefaultScope is "" +// when unset or when this DB predates #899 (no default_scope column at +// all). PublicKey is lowercase, for looking a node up in a +// RepeaterRelayInfo map. +type nodeAreaScopeInput struct { + PublicKey string + Name string + Lat, Lon float64 + DefaultScope string +} + +// GetNodesForScopeAdoption returns every node with a real GPS fix (0,0 +// excluded, same convention as geofilter.PassesFilter) and its +// default_scope, for computeScopeAdoptionByArea to bucket by configured +// area. Unlike GetNodesByDefaultScope, this includes nodes with NO scope +// too — the whole point is measuring adoption, not just listing who has one. +func (db *DB) GetNodesForScopeAdoption() ([]nodeAreaScopeInput, error) { + query := "SELECT public_key, name, lat, lon" + if db.hasDefaultScope { + query += ", default_scope" + } + query += " FROM nodes WHERE lat IS NOT NULL AND lon IS NOT NULL AND lat != 0 AND lon != 0" + rows, err := db.conn.Query(query) + if err != nil { + return nil, fmt.Errorf("nodes for scope adoption query: %w", err) + } + defer rows.Close() + var out []nodeAreaScopeInput + for rows.Next() { + var pk string + var name sql.NullString + var lat, lon float64 + var scope sql.NullString + var scanErr error + if db.hasDefaultScope { + scanErr = rows.Scan(&pk, &name, &lat, &lon, &scope) + } else { + scanErr = rows.Scan(&pk, &name, &lat, &lon) + } + if scanErr != nil { + continue + } + displayName := pk + if name.Valid && name.String != "" { + displayName = name.String + } + out = append(out, nodeAreaScopeInput{PublicKey: strings.ToLower(pk), Name: displayName, Lat: lat, Lon: lon, DefaultScope: scope.String}) + } + return out, rows.Err() +} + +// computeScopeAdoptionByArea buckets nodes by their most specific +// configured area and tallies, per area: how many nodes sit there at all, +// how many "use scope" in ANY sense, and (when the area itself has a +// RegionScopes link) how many specifically use THAT region — i.e. does +// this geographic community actually engage with the scope the area is +// nominally tied to, or something else entirely (or nothing at all). A +// node outside every configured area is excluded. +// +// Unlike the per-node area *badges* (AreaForPoint/AreaKeyForPoint, which +// pick a single most-specific area), this uses AreaKeysForPoint so a node +// counts toward EVERY containing area — a node in "Aarhus by" also counts +// toward "Jylland" and "Danmark (alle)". Without this, a broad roll-up +// area like "Danmark (alle)" would only ever show the handful of nodes not +// claimed by any smaller, more specific area (dborup flagged this: DK +// showed almost nothing because nearly every real node already belonged +// to a narrower sub-area), instead of the whole country's actual adoption. +// +// "Uses scope" counts two distinct signals, same runs-this-region vs +// carried-this-region's-traffic distinction as OriginatingNodesByRegion vs +// RepeatersByRegion above: (1) the node's own default_scope, and (2) any +// region it has ever RELAYED (relayInfo/TransportedScopes) — a repeater +// can carry dk-horsens traffic and thereby support the Horsens area +// without ever configuring dk-horsens as its own default_scope. relayInfo +// may be nil (in-memory store unavailable), in which case matching falls +// back to default_scope only. +// +// For areas with a RegionScopes link, also returns the actual node lists +// (Matching/NotMatching) — dborup wanted to see which specific nodes in +// e.g. Østjylland relay dk-oj (correctly "support" the area) and which +// don't, not just an aggregate count. +func computeScopeAdoptionByArea(nodes []nodeAreaScopeInput, areas map[string]AreaEntry, relayInfo map[string]RepeaterRelayInfo) []AreaScopeAdoption { + counts := make(map[string]*AreaScopeAdoption) + for _, n := range nodes { + keys := AreaKeysForPoint(n.Lat, n.Lon, areas) + if len(keys) == 0 { + continue + } + + ownScope := strings.ToLower(strings.TrimPrefix(n.DefaultScope, "#")) + relayedRegions := make(map[string]bool) + if info, ok := relayInfo[n.PublicKey]; ok { + for _, r := range info.TransportedScopes { + relayedRegions[strings.ToLower(strings.TrimPrefix(r, "#"))] = true + } + } + hasAnyScope := ownScope != "" || len(relayedRegions) > 0 + + for _, key := range keys { + c, exists := counts[key] + if !exists { + a := areas[key] + c = &AreaScopeAdoption{AreaKey: key, Label: a.Label, RegionScopes: a.RegionScopes} + counts[key] = c + } + c.TotalNodes++ + if hasAnyScope { + c.NodesWithAnyScope++ + } + // Matching/NotMatching per-node lists only make sense when the + // area actually has region(s) to compare against — an area + // with no RegionScopes link has nothing to be "not matching". + if len(c.RegionScopes) > 0 { + var matchedScopes []string + for _, rs := range c.RegionScopes { + normalizedRegion := strings.ToLower(rs) + if ownScope == normalizedRegion || relayedRegions[normalizedRegion] { + matchedScopes = append(matchedScopes, rs) + } + } + if len(matchedScopes) > 0 { + c.NodesMatchingArea++ + c.Matching = append(c.Matching, AreaScopeMatch{Name: n.Name, PublicKey: n.PublicKey, MatchedScopes: matchedScopes}) + } else { + c.NotMatching = append(c.NotMatching, RepeaterRef{Name: n.Name, PublicKey: n.PublicKey}) + } + } + } + } + result := make([]AreaScopeAdoption, 0, len(counts)) + for _, c := range counts { + sort.Slice(c.Matching, func(i, j int) bool { return c.Matching[i].Name < c.Matching[j].Name }) + sort.Slice(c.NotMatching, func(i, j int) bool { return c.NotMatching[i].Name < c.NotMatching[j].Name }) + result = append(result, *c) + } + sort.Slice(result, func(i, j int) bool { + if result[i].TotalNodes != result[j].TotalNodes { + return result[i].TotalNodes > result[j].TotalNodes + } + return result[i].Label < result[j].Label + }) + return result +} + // QueryMultiNodePackets returns transmissions referencing any of the given pubkeys. func (db *DB) QueryMultiNodePackets(pubkeys []string, limit, offset int, order, since, until string) (*PacketResult, error) { if len(pubkeys) == 0 { @@ -3494,6 +3644,10 @@ func (db *DB) buildWardrivingSessions(channel, since string) ([]WardrivingSessio } cur.EntryPointCount = len(curPrefixes) cur.ObserverCount = len(curObservers) + for p := range curPrefixes { + cur.EntryPointPrefixes = append(cur.EntryPointPrefixes, p) + } + sort.Strings(cur.EntryPointPrefixes) start, errS := time.Parse(time.RFC3339, cur.StartTime) end, errE := time.Parse(time.RFC3339, cur.EndTime) if errS == nil && errE == nil { diff --git a/cmd/server/db_test.go b/cmd/server/db_test.go index 5ac18c08..20af417d 100644 --- a/cmd/server/db_test.go +++ b/cmd/server/db_test.go @@ -2327,3 +2327,217 @@ func TestLoadIndexesRelayHopsFromResolvedPath(t *testing.T) { t.Errorf("relay byNode entry has wrong hash: %s", store.byNode[relayPubkey][0].Hash) } } + +func TestComputeScopeAdoptionByArea(t *testing.T) { + f := func(v float64) *float64 { return &v } + areas := map[string]AreaEntry{ + "ODE": {Label: "Odense by", RegionScopes: []string{"dk-fyn-odense"}, LatMin: f(55.32), LatMax: f(55.45), LonMin: f(10.3), LonMax: f(10.5)}, + "GOT": {Label: "Göteborg, SE", LatMin: f(57.35), LatMax: f(57.90), LonMin: f(11.85), LonMax: f(12.85)}, // no RegionScopes link + } + + nodes := []nodeAreaScopeInput{ + {Lat: 55.4047, Lon: 10.3810, DefaultScope: "#dk-fyn-odense"}, // Odense, matches area's own region + {Lat: 55.40, Lon: 10.40, DefaultScope: "#dk-aarhus"}, // Odense, but a DIFFERENT region + {Lat: 55.41, Lon: 10.41, DefaultScope: ""}, // Odense, no scope at all + {Lat: 57.68, Lon: 11.97, DefaultScope: "#dk-aarhus"}, // Göteborg, has a scope, but area has no RegionScopes link + {Lat: 57.70, Lon: 11.98, DefaultScope: ""}, // Göteborg, no scope + {Lat: 0, Lon: 0, DefaultScope: "#dk-aarhus"}, // no-fix, must be excluded entirely + {Lat: 51.0, Lon: 4.0, DefaultScope: "#belgium"}, // outside every configured area, excluded + } + + got := computeScopeAdoptionByArea(nodes, areas, nil) + if len(got) != 2 { + t.Fatalf("got %d areas, want 2 (ODE and GOT) -- result: %+v", len(got), got) + } + byKey := map[string]AreaScopeAdoption{} + for _, a := range got { + byKey[a.AreaKey] = a + } + + ode := byKey["ODE"] + if ode.TotalNodes != 3 { + t.Errorf("ODE.TotalNodes = %d, want 3", ode.TotalNodes) + } + if ode.NodesWithAnyScope != 2 { + t.Errorf("ODE.NodesWithAnyScope = %d, want 2 (one has no scope at all)", ode.NodesWithAnyScope) + } + if ode.NodesMatchingArea != 1 { + t.Errorf("ODE.NodesMatchingArea = %d, want 1 (only the dk-fyn-odense one matches, the dk-aarhus one doesn't)", ode.NodesMatchingArea) + } + if len(ode.Matching) != 1 || len(ode.NotMatching) != 2 { + t.Errorf("ODE.Matching=%v NotMatching=%v, want 1 matching + 2 not-matching (RegionScopes is set)", ode.Matching, ode.NotMatching) + } + + got2 := byKey["GOT"] + if got2.TotalNodes != 2 { + t.Errorf("GOT.TotalNodes = %d, want 2", got2.TotalNodes) + } + if got2.NodesWithAnyScope != 1 { + t.Errorf("GOT.NodesWithAnyScope = %d, want 1", got2.NodesWithAnyScope) + } + if got2.NodesMatchingArea != 0 { + t.Errorf("GOT.NodesMatchingArea = %d, want 0 (area has no RegionScopes link to match against)", got2.NodesMatchingArea) + } + if len(got2.Matching) != 0 || len(got2.NotMatching) != 0 { + t.Errorf("GOT.Matching=%v NotMatching=%v, want both empty (no RegionScopes, nothing to split into two groups)", got2.Matching, got2.NotMatching) + } +} + +// TestComputeScopeAdoptionByArea_RelayedRegionCounts covers the case +// dborup flagged directly: a repeater sitting inside the Horsens area that +// has RELAYED dk-horsens traffic supports that region, even if its own +// default_scope is something else (or unset entirely) — matching must not +// be limited to default_scope, same runs-this-region vs +// carried-this-region's-traffic distinction as RepeatersByRegion vs +// OriginatingNodesByRegion elsewhere in this file. +func TestComputeScopeAdoptionByArea_RelayedRegionCounts(t *testing.T) { + f := func(v float64) *float64 { return &v } + areas := map[string]AreaEntry{ + "HORSENS": {Label: "Horsens", RegionScopes: []string{"dk-horsens"}, LatMin: f(55.76), LatMax: f(55.94), LonMin: f(9.6), LonMax: f(9.96)}, + } + nodes := []nodeAreaScopeInput{ + {PublicKey: "relayer01", Lat: 55.85, Lon: 9.85, DefaultScope: "#dk"}, // own scope is the generic #dk, NOT dk-horsens + {PublicKey: "plainnode1", Lat: 55.86, Lon: 9.86, DefaultScope: ""}, // no scope, no relay activity either + {PublicKey: "relayerother", Lat: 55.87, Lon: 9.87, DefaultScope: ""}, // relays something, but not dk-horsens + } + relayInfo := map[string]RepeaterRelayInfo{ + "relayer01": {TransportedScopes: []string{"#dk-horsens"}}, + "relayerother": {TransportedScopes: []string{"#dk-aarhus"}}, + } + + got := computeScopeAdoptionByArea(nodes, areas, relayInfo) + if len(got) != 1 { + t.Fatalf("got %d areas, want 1", len(got)) + } + h := got[0] + if h.TotalNodes != 3 { + t.Errorf("TotalNodes = %d, want 3", h.TotalNodes) + } + // relayer01 (relays dk-horsens) and relayerother (relays something, + // just not dk-horsens) both "use scope" in some sense; plainnode1 does + // nothing at all. + if h.NodesWithAnyScope != 2 { + t.Errorf("NodesWithAnyScope = %d, want 2", h.NodesWithAnyScope) + } + // Only relayer01 specifically relays THIS area's own region + // (dk-horsens) -- despite its own default_scope being the unrelated, + // generic #dk. + if h.NodesMatchingArea != 1 { + t.Errorf("NodesMatchingArea = %d, want 1 (relayer01 relays dk-horsens even though its default_scope is #dk)", h.NodesMatchingArea) + } + if len(h.Matching) != 1 || h.Matching[0].PublicKey != "relayer01" { + t.Errorf("Matching = %v, want just relayer01", h.Matching) + } + if len(h.NotMatching) != 2 { + t.Errorf("NotMatching = %v, want 2 (plainnode1 and relayerother)", h.NotMatching) + } +} + +// TestComputeScopeAdoptionByArea_RollsUpIntoBroaderAreas is a regression +// test for exactly what dborup flagged live: "Danmark (alle)" showed only +// a handful of nodes because AreaKeyForPoint's single-most-specific-match +// meant every node already claimed by a smaller sub-area (e.g. "Odense +// by") never counted toward the broader containing area at all. A node +// inside a nested area must now count toward EVERY containing area, so +// DK's totals genuinely reflect the whole country, not just leftovers. +func TestComputeScopeAdoptionByArea_RollsUpIntoBroaderAreas(t *testing.T) { + f := func(v float64) *float64 { return &v } + areas := map[string]AreaEntry{ + "DK": {Label: "Danmark (alle)", RegionScopes: []string{"dk"}, LatMin: f(54.5), LatMax: f(57.8), LonMin: f(8.0), LonMax: f(15.25)}, + "ODE": {Label: "Odense by", RegionScopes: []string{"dk-fyn-odense"}, LatMin: f(55.32), LatMax: f(55.45), LonMin: f(10.3), LonMax: f(10.5)}, + } + nodes := []nodeAreaScopeInput{ + {PublicKey: "odenode01", Name: "OdenseNode", Lat: 55.4047, Lon: 10.3810, DefaultScope: "#dk-fyn-odense"}, // inside BOTH DK and ODE + {PublicKey: "dkonly01", Name: "SomewhereElseInDK", Lat: 56.0, Lon: 9.0, DefaultScope: "#dk"}, // inside DK only, not ODE + } + + got := computeScopeAdoptionByArea(nodes, areas, nil) + byKey := map[string]AreaScopeAdoption{} + for _, a := range got { + byKey[a.AreaKey] = a + } + + dk := byKey["DK"] + if dk.TotalNodes != 2 { + t.Errorf("DK.TotalNodes = %d, want 2 (both nodes fall inside DK's box, including the one also inside ODE)", dk.TotalNodes) + } + // The Odense node's own scope is dk-fyn-odense, not dk -- so it does + // NOT match DK's own region even though it geographically counts + // toward DK's totals. Only dkonly01 (#dk) matches. + if dk.NodesMatchingArea != 1 { + t.Errorf("DK.NodesMatchingArea = %d, want 1 (only dkonly01 actually uses #dk)", dk.NodesMatchingArea) + } + + ode := byKey["ODE"] + if ode.TotalNodes != 1 { + t.Errorf("ODE.TotalNodes = %d, want 1 (only the Odense-positioned node)", ode.TotalNodes) + } + if ode.NodesMatchingArea != 1 { + t.Errorf("ODE.NodesMatchingArea = %d, want 1", ode.NodesMatchingArea) + } +} + +func TestComputeScopeAdoptionByArea_Empty(t *testing.T) { + got := computeScopeAdoptionByArea(nil, map[string]AreaEntry{"DK": {Label: "Danmark"}}, nil) + if len(got) != 0 { + t.Errorf("expected no areas with 0 nodes, got %+v", got) + } +} + +// TestComputeScopeAdoptionByArea_MultipleRegionScopes covers dborup's exact +// request: a broad umbrella area (Europa) can be linked to more than one +// hashRegions scope at once (e.g. both "eu" and "europe"), and a node using +// EITHER counts as matching -- not just the first-configured one. +func TestComputeScopeAdoptionByArea_MultipleRegionScopes(t *testing.T) { + f := func(v float64) *float64 { return &v } + areas := map[string]AreaEntry{ + "EU": {Label: "Europa", RegionScopes: []string{"eu", "europe"}, LatMin: f(34.0), LatMax: f(71.5), LonMin: f(-25.0), LonMax: f(45.0)}, + } + nodes := []nodeAreaScopeInput{ + {PublicKey: "usesEu", Lat: 48.0, Lon: 10.0, DefaultScope: "#eu"}, + {PublicKey: "usesEurope", Lat: 48.0, Lon: 11.0, DefaultScope: "#europe"}, + {PublicKey: "relaysEurope", Lat: 48.0, Lon: 12.0, DefaultScope: ""}, + {PublicKey: "usesNeither", Lat: 48.0, Lon: 13.0, DefaultScope: "#dk"}, + // Own scope is #eu AND it also relays #europe -- must be + // reported as matching BOTH, not just the first one found. + {PublicKey: "usesBoth", Lat: 48.0, Lon: 14.0, DefaultScope: "#eu"}, + } + relayInfo := map[string]RepeaterRelayInfo{ + "relaysEurope": {TransportedScopes: []string{"#europe"}}, + "usesBoth": {TransportedScopes: []string{"#europe"}}, + } + + got := computeScopeAdoptionByArea(nodes, areas, relayInfo) + if len(got) != 1 { + t.Fatalf("got %d areas, want 1", len(got)) + } + eu := got[0] + if eu.TotalNodes != 5 { + t.Errorf("TotalNodes = %d, want 5", eu.TotalNodes) + } + // usesEu (#eu), usesEurope (#europe), relaysEurope (relays #europe), + // and usesBoth (both) all match one of Europa's two linked scopes; + // usesNeither (#dk) matches neither. + if eu.NodesMatchingArea != 4 { + t.Errorf("NodesMatchingArea = %d, want 4 (any of #eu or #europe should count)", eu.NodesMatchingArea) + } + matchedScopesByKey := map[string][]string{} + for _, m := range eu.Matching { + matchedScopesByKey[m.PublicKey] = m.MatchedScopes + } + if diff := matchedScopesByKey["usesEu"]; len(diff) != 1 || diff[0] != "eu" { + t.Errorf("usesEu.MatchedScopes = %v, want [\"eu\"]", diff) + } + if diff := matchedScopesByKey["usesEurope"]; len(diff) != 1 || diff[0] != "europe" { + t.Errorf("usesEurope.MatchedScopes = %v, want [\"europe\"]", diff) + } + if diff := matchedScopesByKey["relaysEurope"]; len(diff) != 1 || diff[0] != "europe" { + t.Errorf("relaysEurope.MatchedScopes = %v, want [\"europe\"]", diff) + } + if diff := matchedScopesByKey["usesBoth"]; len(diff) != 2 || diff[0] != "eu" || diff[1] != "europe" { + t.Errorf("usesBoth.MatchedScopes = %v, want [\"eu\" \"europe\"] (node uses one scope AND relays the other)", diff) + } + if len(eu.NotMatching) != 1 || eu.NotMatching[0].PublicKey != "usesNeither" { + t.Errorf("NotMatching = %v, want just usesNeither", eu.NotMatching) + } +} diff --git a/cmd/server/routes.go b/cmd/server/routes.go index 9172363a..f23ba854 100644 --- a/cmd/server/routes.go +++ b/cmd/server/routes.go @@ -18,6 +18,7 @@ import ( "time" "github.com/gorilla/mux" + "github.com/meshcore-analyzer/geofilter" "github.com/meshcore-analyzer/packetpath" "github.com/meshcore-analyzer/prunequeue" regionutil "github.com/meshcore-analyzer/regions" @@ -180,9 +181,26 @@ func (s *Server) isPubkeyHidden(pubkey string) bool { return s.cfg.IsNameHidden(name) } +// getGeoFilter returns the effective home-boundary geometry. If +// cfg.HomeArea names an existing entry in cfg.Areas, that area's geometry +// wins — a single boundary shared with the areas system, instead of a +// second copy that can silently drift (see cfg.HomeArea doc comment). +// Falls back to the standalone GeoFilter field when HomeArea is unset or +// doesn't resolve, so existing deployments see no behavior change. func (s *Server) getGeoFilter() *GeoFilterConfig { s.cfgMu.RLock() defer s.cfgMu.RUnlock() + if s.cfg != nil && s.cfg.HomeArea != "" { + if area, ok := s.cfg.Areas[s.cfg.HomeArea]; ok { + return &geofilter.Config{ + Polygon: area.Polygon, + LatMin: area.LatMin, + LatMax: area.LatMax, + LonMin: area.LonMin, + LonMax: area.LonMax, + } + } + } return s.cfg.GeoFilter } @@ -471,15 +489,16 @@ func (s *Server) handleConfigClient(w http.ResponseWriter, r *http.Request) { func (s *Server) handleConfigAreas(w http.ResponseWriter, r *http.Request) { type areaListEntry struct { - Key string `json:"key"` - Label string `json:"label"` + Key string `json:"key"` + Label string `json:"label"` + RegionScopes []string `json:"regionScopes,omitempty"` } result := make([]areaListEntry, 0, len(s.cfg.Areas)) for k, v := range s.cfg.Areas { if v.Label == "" { continue // skip comment/invalid entries (e.g. "_comment" keys in config) } - result = append(result, areaListEntry{Key: k, Label: v.Label}) + result = append(result, areaListEntry{Key: k, Label: v.Label, RegionScopes: v.RegionScopes}) } writeJSON(w, result) } @@ -2716,6 +2735,44 @@ func (s *Server) handleResolveHops(w http.ResponseWriter, r *http.Request) { writeJSON(w, ResolveHopsResponse{Resolved: resolved}) } +// resolveEntryPointArea approximates a wardriving session's area from its +// entry-point repeater(s): for each candidate path[0] prefix, only a +// unique_prefix match (exactly one node in the prefix map, same discipline +// as handleResolveHops) with a known position is trusted — an ambiguous +// prefix is skipped rather than guessed. Returns ok=false if no prefix +// resolves this way, or the resolved node has no position, or areas aren't +// configured. +func (s *Server) resolveEntryPointArea(prefixes []string) (label string, ok bool) { + if s.store == nil { + return "", false + } + return s.store.resolveEntryPointArea(prefixes) +} + +// annotateMessageAreas resolves each message's entry-point repeater (its +// path[0], captured as "entryPrefix" by GetChannelMessages) to a +// configured area — same unique_prefix-only discipline as +// resolveEntryPointArea/handleWardrivingStats: only a truly unambiguous +// prefix match with a known position sets "area", so this shows where the +// SENDER actually was, distinct from (and often more specific than) the +// area linked to the channel scope they sent with — dborup's own example: +// sitting in Aarhus but sending with the broad #dk scope should still show +// "Aarhus by" here. The raw entryPrefix never reaches the client, whether +// or not it resolved. +func (s *Server) annotateMessageAreas(messages []map[string]interface{}) { + hasAreas := s.cfg != nil && len(s.cfg.Areas) > 0 + for _, m := range messages { + prefix, _ := m["entryPrefix"].(string) + delete(m, "entryPrefix") + if !hasAreas || prefix == "" { + continue + } + if label, ok := s.resolveEntryPointArea([]string{prefix}); ok { + m["area"] = label + } + } +} + func (s *Server) handleChannels(w http.ResponseWriter, r *http.Request) { region := r.URL.Query().Get("region") includeEncrypted := r.URL.Query().Get("includeEncrypted") == "true" @@ -2760,11 +2817,13 @@ func (s *Server) handleChannelMessages(w http.ResponseWriter, r *http.Request) { writeError(w, 500, err.Error()) return } + s.annotateMessageAreas(messages) writeJSON(w, ChannelMessagesResponse{Messages: messages, Total: total}) return } if s.store != nil { messages, total := s.store.GetChannelMessages(hash, limit, offset, region) + s.annotateMessageAreas(messages) writeJSON(w, ChannelMessagesResponse{Messages: messages, Total: total}) return } @@ -3710,6 +3769,22 @@ func (s *Server) handleScopeStats(w http.ResponseWriter, r *http.Request) { log.Printf("WARN GetChannelMessageScopeStats: %v", err) } + if s.cfg != nil && len(s.cfg.Areas) > 0 { + if nodes, err := s.db.GetNodesForScopeAdoption(); err == nil { + // Reuses the same cached relay-info map as RepeatersByRegion + // above (GetRepeaterRelayInfoMap never rebuilds inline on a + // populated cache) so a node relaying an area's region counts + // as using it, not just one with a matching default_scope. + var relayInfo map[string]RepeaterRelayInfo + if s.store != nil { + relayInfo = s.store.GetRepeaterRelayInfoMap(s.cfg.GetHealthThresholds().RelayActiveHours) + } + resp.ScopeAdoptionByArea = computeScopeAdoptionByArea(nodes, s.cfg.Areas, relayInfo) + } else { + log.Printf("WARN GetNodesForScopeAdoption: %v", err) + } + } + if adoption, err := s.db.GetChannelScopeAdoption(window); err == nil { resp.ChannelScopeAdoption = adoption } else { @@ -3776,6 +3851,21 @@ func (s *Server) handleWardrivingStats(w http.ResponseWriter, r *http.Request) { } } + if s.cfg != nil && len(s.cfg.Areas) > 0 { + for i := range resp.GPSShares { + if label, ok := AreaForPoint(resp.GPSShares[i].Lat, resp.GPSShares[i].Lon, s.cfg.Areas); ok { + resp.GPSShares[i].Area = &label + } + } + if s.store != nil { + for i := range resp.Sessions { + if label, ok := s.resolveEntryPointArea(resp.Sessions[i].EntryPointPrefixes); ok { + resp.Sessions[i].Area = &label + } + } + } + } + s.wardrivingStatsMu.Lock() if s.wardrivingStatsCache == nil { s.wardrivingStatsCache = make(map[string]*WardrivingStatsResponse) diff --git a/cmd/server/routes_test.go b/cmd/server/routes_test.go index e6619976..0ace90ff 100644 --- a/cmd/server/routes_test.go +++ b/cmd/server/routes_test.go @@ -34,6 +34,7 @@ func setupTestServer(t *testing.T) (*Server, *mux.Router) { if !store.WaitIndexesReady(5 * time.Second) { t.Fatalf("background indexes never became ready") } + store.config = cfg // mirrors main.go's real wiring -- store.resolveEntryPointArea/resolveAreaNodes need it srv.store = store router := mux.NewRouter() srv.RegisterRoutes(router) @@ -55,6 +56,7 @@ func setupTestServerWithAPIKey(t *testing.T, apiKey string) (*Server, *mux.Route if !store.WaitIndexesReady(5 * time.Second) { t.Fatalf("background indexes never became ready") } + store.config = cfg srv.store = store router := mux.NewRouter() srv.RegisterRoutes(router) @@ -4786,6 +4788,114 @@ func TestHandleScopeStats_OriginatingNodesByRegion(t *testing.T) { } } +func TestHandleScopeStats_ScopeAdoptionByArea(t *testing.T) { + srv, _ := setupTestServer(t) + if _, err := srv.db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add scope_name column: %v", err) + } + srv.db.hasScopeName = true + if !srv.db.hasDefaultScope { + if _, err := srv.db.conn.Exec(`ALTER TABLE nodes ADD COLUMN default_scope TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add default_scope column: %v", err) + } + srv.db.hasDefaultScope = true + } + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "ODE": {Label: "Odense by", RegionScopes: []string{"dk-fyn-odense"}, LatMin: f(55.32), LatMax: f(55.45), LonMin: f(10.3), LonMax: f(10.5)}, + } + + insertNode := func(pk, defaultScope string, lat, lon float64) { + if _, err := srv.db.conn.Exec( + `INSERT INTO nodes (public_key, name, role, default_scope, lat, lon) VALUES (?, ?, 'repeater', ?, ?, ?)`, + pk, pk, defaultScope, lat, lon, + ); err != nil { + t.Fatalf("seed node %s: %v", pk, err) + } + } + insertNode("odematch01", "#dk-fyn-odense", 55.4047, 10.3810) + insertNode("odewrong01", "#dk-aarhus", 55.40, 10.40) + insertNode("odenoscope1", "", 55.41, 10.41) + + req := httptest.NewRequest("GET", "/api/scope-stats?window=24h", nil) + w := httptest.NewRecorder() + srv.handleScopeStats(w, req) + + if w.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body: %s", w.Code, w.Body.String()) + } + var resp ScopeStatsResponse + if err := json.NewDecoder(w.Body).Decode(&resp); err != nil { + t.Fatalf("decode: %v", err) + } + if len(resp.ScopeAdoptionByArea) != 1 { + t.Fatalf("scopeAdoptionByArea = %+v, want 1 entry (ODE)", resp.ScopeAdoptionByArea) + } + ode := resp.ScopeAdoptionByArea[0] + if ode.AreaKey != "ODE" || ode.TotalNodes != 3 || ode.NodesWithAnyScope != 2 || ode.NodesMatchingArea != 1 { + t.Errorf("ScopeAdoptionByArea[0] = %+v, want AreaKey=ODE TotalNodes=3 NodesWithAnyScope=2 NodesMatchingArea=1", ode) + } +} + +// TestHandleScopeStats_ScopeAdoptionByArea_ZeroMatchKeyPresent is a +// regression test: NodesMatchingArea must NOT have `omitempty`, or a +// genuine 0 count (a real, meaningful value — "this area has a linked +// region but nobody uses it") silently disappears from the raw JSON. The +// frontend reads a.nodesMatchingArea directly and calls .toLocaleString() +// on it — an absent key deserializes to `undefined` in JS, not 0, which +// throws and breaks the whole Scopes tab render. Decodes into a raw map +// (not the typed struct, which would hide this by defaulting to the zero +// value regardless of whether the key was present). +func TestHandleScopeStats_ScopeAdoptionByArea_ZeroMatchKeyPresent(t *testing.T) { + srv, _ := setupTestServer(t) + if _, err := srv.db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add scope_name column: %v", err) + } + srv.db.hasScopeName = true + if !srv.db.hasDefaultScope { + if _, err := srv.db.conn.Exec(`ALTER TABLE nodes ADD COLUMN default_scope TEXT DEFAULT NULL`); err != nil { + t.Fatalf("add default_scope column: %v", err) + } + srv.db.hasDefaultScope = true + } + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "AAR": {Label: "Aarhus by", RegionScopes: []string{"dk-aarhus"}, LatMin: f(56.05), LatMax: f(56.25), LonMin: f(9.95), LonMax: f(10.35)}, + } + // A node with a scope, but NOT the area's own region — NodesMatchingArea + // must come out as a real, present 0, same as the live #dk-vs-#dk-aarhus + // case this feature was built to surface. + if _, err := srv.db.conn.Exec( + `INSERT INTO nodes (public_key, name, role, default_scope, lat, lon) VALUES ('aar00000001', 'AarhusNode', 'repeater', '#dk', 56.15, 10.15)`, + ); err != nil { + t.Fatalf("seed node: %v", err) + } + + req := httptest.NewRequest("GET", "/api/scope-stats?window=24h", nil) + w := httptest.NewRecorder() + srv.handleScopeStats(w, req) + if w.Code != http.StatusOK { + t.Fatalf("status = %d, want 200; body: %s", w.Code, w.Body.String()) + } + + var raw map[string]interface{} + if err := json.NewDecoder(w.Body).Decode(&raw); err != nil { + t.Fatalf("decode: %v", err) + } + areas, ok := raw["scopeAdoptionByArea"].([]interface{}) + if !ok || len(areas) != 1 { + t.Fatalf("scopeAdoptionByArea = %v, want 1 entry", raw["scopeAdoptionByArea"]) + } + area := areas[0].(map[string]interface{}) + val, present := area["nodesMatchingArea"] + if !present { + t.Fatal("nodesMatchingArea key is missing from the JSON entirely — omitempty regression, this crashes the frontend") + } + if val != float64(0) { + t.Errorf("nodesMatchingArea = %v, want 0", val) + } +} + func TestHandleScopeStatsInvalidWindow(t *testing.T) { srv, _ := setupTestServer(t) if _, err := srv.db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil { diff --git a/cmd/server/store.go b/cmd/server/store.go index da066405..a66d9cb3 100644 --- a/cmd/server/store.go +++ b/cmd/server/store.go @@ -1675,6 +1675,19 @@ func pathLen(pathJSON string) int { return len(hops) } +// pathFirstHop returns path[0] (the entry-point repeater's hex prefix), or +// "" when pathJSON is empty/invalid/has no hops. +func pathFirstHop(pathJSON string) string { + if pathJSON == "" { + return "" + } + var hops []string + if json.Unmarshal([]byte(pathJSON), &hops) != nil || len(hops) == 0 { + return "" + } + return hops[0] +} + // indexResolvedPathHops indexes a transmission under every relay-hop pubkey // extracted from an observation's resolved_path, and refreshes the dependent // resolved-pubkey + path-hop indexes. This is the single point of truth for @@ -2949,6 +2962,16 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac "observation_count": tx.ObservationCount, "scope_name": strOrNil(tx.ScopeName), } + // Same entry-point area resolution as the REST channel message + // list (annotateMessageAreas), applied live so a message + // appended via WebSocket shows "Area:" immediately instead of + // only after the next full reload. Only a unique_prefix match + // with a known position sets it — never a guess. + if entryPrefix := pathFirstHop(obs.PathJSON); entryPrefix != "" { + if label, ok := s.resolveEntryPointArea([]string{entryPrefix}); ok { + pkt["area"] = label + } + } // Use decode-window resolved path for broadcast (never from struct) if broadcastRP != nil { if rp, ok := broadcastRP[obs.ID]; ok && rp != nil { @@ -3222,6 +3245,13 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) []map[string] "observation_count": tx.ObservationCount, "scope_name": strOrNil(tx.ScopeName), } + // Same live entry-point area resolution as IngestNewFromDB above -- + // see the comment there. + if entryPrefix := pathFirstHop(obs.PathJSON); entryPrefix != "" { + if label, ok := s.resolveEntryPointArea([]string{entryPrefix}); ok { + pkt["area"] = label + } + } // Use decode-window resolved path for broadcast if obsRPMap != nil { if rp, ok := obsRPMap[obs.ID]; ok && rp != nil { @@ -3600,6 +3630,46 @@ func (s *PacketStore) fetchAndCacheRegionObs(region string) map[string]bool { return m } +// resolveEntryPointArea approximates a sender's area from their entry-point +// repeater(s): for each candidate path[0] prefix, only a unique_prefix +// match (exactly one node in the prefix map, same discipline as +// handleResolveHops) with a known position is trusted — an ambiguous +// prefix is skipped rather than guessed. Returns ok=false if no prefix +// resolves this way, the resolved node has no position, or areas aren't +// configured. Lives on PacketStore (not Server) so both the REST message +// list (via (*Server).resolveEntryPointArea, which just delegates here) +// and the live WebSocket broadcast path can reuse the exact same +// resolution — a store method has everything it needs (config, prefix +// map) without reaching back out to *Server. +func (s *PacketStore) resolveEntryPointArea(prefixes []string) (label string, ok bool) { + if s == nil || s.config == nil || len(s.config.Areas) == 0 || len(prefixes) == 0 { + return "", false + } + // getCachedNodesAndPM guards itself with its own cacheMu -- it never + // touches s.mu. Do not wrap this call in s.mu.RLock(): callers like + // IngestNewFromDB/IngestNewObservations invoke this while already + // holding s.mu.Lock(), and Go's RWMutex is not reentrant, so an + // RLock here would deadlock against the caller's own write lock. + _, pm := s.getCachedNodesAndPM() + if pm == nil { + return "", false + } + for _, prefix := range prefixes { + candidates, found := pm.m[strings.ToLower(prefix)] + if !found || len(candidates) != 1 { + continue + } + ni := candidates[0] + if !ni.HasGPS { + continue + } + if label, ok := AreaForPoint(ni.Lat, ni.Lon, s.config.Areas); ok { + return label, true + } + } + return "", false +} + // resolveAreaNodes returns a set of node pubkeys whose GPS coordinates fall // inside the named area polygon. Returns nil if the area key is not in config. // Results are cached per-key for 30 seconds. Uses its own RWMutex so callers @@ -5521,6 +5591,7 @@ func (s *PacketStore) GetChannelMessages(channelHash string, limit, offset int, "snr": snrVal, "scope": strOrNil(tx.ScopeName), "routeType": intPtrOrNil(tx.RouteType), + "entryPrefix": pathFirstHop(tx.PathJSON), }, Repeats: 1, Observers: observers, diff --git a/cmd/server/types.go b/cmd/server/types.go index 918bfc4e..0f0e8195 100644 --- a/cmd/server/types.go +++ b/cmd/server/types.go @@ -191,6 +191,49 @@ type ScopeStatsResponse struct { // HourlyActivityByRegion is window-scoped like Summary/TimeSeries // above — see ScopeHourlyActivity doc. HourlyActivityByRegion []ScopeHourlyActivity `json:"hourlyActivityByRegion,omitempty"` + // ScopeAdoptionByArea buckets every positioned node by its configured + // geographic area (config.Areas, AreaKeyForPoint) and tallies scope + // adoption within that area — independent of whether the raw + // hashRegion codes above are "used" at all. Surfaces gaps like "34 + // real nodes here, 0 have ever configured a scope" that + // UnusedRegions/RepeatersByRegion can't see, since those only know + // about region strings that already appeared in traffic. All-time, + // like the other Regions-tab sections. Omitted when no areas are + // configured. + ScopeAdoptionByArea []AreaScopeAdoption `json:"scopeAdoptionByArea,omitempty"` +} + +// AreaScopeAdoption is one configured area's node count and scope adoption +// — see ScopeStatsResponse.ScopeAdoptionByArea and computeScopeAdoptionByArea. +type AreaScopeAdoption struct { + AreaKey string `json:"areaKey"` + Label string `json:"label"` + RegionScopes []string `json:"regionScopes,omitempty"` + TotalNodes int `json:"totalNodes"` + // NodesWithAnyScope is how many of TotalNodes "use scope" in any + // sense: either their own default_scope is set, or they've ever + // relayed traffic carrying ANY region's scope (a repeater can support + // a region purely by relaying it, without configuring that region as + // its own — see computeScopeAdoptionByArea). + NodesWithAnyScope int `json:"nodesWithAnyScope"` + // NodesMatchingArea is the subset of NodesWithAnyScope that + // specifically use one of THIS area's own RegionScopes — via + // default_scope OR by having relayed it. Only meaningful when + // RegionScopes is non-empty — 0 otherwise (not the same as "0 of them + // match", there's simply nothing configured to match against). No + // omitempty: a real 0 count must still serialize, or the frontend has + // no way to distinguish it from "field absent". + NodesMatchingArea int `json:"nodesMatchingArea"` + // Matching/NotMatching are the actual nodes behind NodesMatchingArea — + // which specific nodes in this area relay/configure any of the area's + // own regions (correctly "support" it) and which sit here but don't. + // Only populated when RegionScopes is non-empty (nothing to split + // into two groups otherwise). Matching entries also carry WHICH of the + // area's regions each node matched (an area with several linked + // scopes, e.g. Europa's "eu"/"europe", needs this to answer "which + // nodes support which scope" — not just an aggregate yes/no). + Matching []AreaScopeMatch `json:"matching,omitempty"` + NotMatching []RepeaterRef `json:"notMatching,omitempty"` } type RepeaterRef struct { @@ -198,6 +241,16 @@ type RepeaterRef struct { PublicKey string `json:"publicKey"` } +// AreaScopeMatch is a RepeaterRef plus which of the area's own +// RegionScopes this node actually matched (via default_scope or by having +// relayed it) — a node can match more than one when an area links several +// scopes and the node uses/relays more than one of them. +type AreaScopeMatch struct { + Name string `json:"name"` + PublicKey string `json:"publicKey"` + MatchedScopes []string `json:"matchedScopes"` +} + type ScopeRegionRepeaters struct { Region string `json:"region"` Count int `json:"count"` @@ -285,6 +338,19 @@ type WardrivingSession struct { // TransmissionIDs is internal — the session's own transmission IDs, // used by the route handler to compute AirtimeMs. Never serialized. TransmissionIDs []int64 `json:"-"` + // EntryPointPrefixes is internal — the session's distinct path[0] + // hex prefixes, used by the route handler to resolve Area. Never + // serialized directly. + EntryPointPrefixes []string `json:"-"` + // Area is the most specific configured area containing the session's + // entry-point repeater — always approximate (the repeater's known + // position, not the sender's own; wardriving sessions never carry a + // literal shared GPS fix, unlike WardrivingGPSShare). Resolved by the + // route handler from EntryPointPrefixes when exactly one candidate + // node matches a prefix (same unique_prefix-only discipline as + // /api/resolve-hops) and that node has a known position. Nil when no + // prefix resolves unambiguously or areas aren't configured. + Area *string `json:"area,omitempty"` } // WardrivingGPSShare is one sender who has explicitly shared their own @@ -299,6 +365,9 @@ type WardrivingGPSShare struct { Lon float64 `json:"lon"` MessageCount int `json:"messageCount"` // how many times this sender shared a position in this window LastSeen string `json:"lastSeen"` + // Area is the most specific configured area containing (Lat, Lon), set + // by the handler from config.Areas — omitted when no area matches. + Area *string `json:"area,omitempty"` } type WardrivingStatsResponse struct { diff --git a/cmd/server/wardriving_stats_test.go b/cmd/server/wardriving_stats_test.go index 17ac787c..fe6429b3 100644 --- a/cmd/server/wardriving_stats_test.go +++ b/cmd/server/wardriving_stats_test.go @@ -449,6 +449,171 @@ func TestHandleWardrivingStats_GPSShares(t *testing.T) { } } +// TestHandleWardrivingStats_GPSShareArea confirms a shared position gets +// tagged with the most specific configured area (config.Areas), and is left +// unset when no area matches or none are configured. +func TestHandleWardrivingStats_GPSShareArea(t *testing.T) { + srv, router := setupTestServer(t) + if _, err := srv.db.conn.Exec(`DELETE FROM transmissions`); err != nil { + t.Fatalf("clear transmissions: %v", err) + } + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "DK": {Label: "Danmark (alle)", LatMin: f(54.5), LatMax: f(57.8), LonMin: f(8.0), LonMax: f(15.25)}, + "ODE": {Label: "Odense by", LatMin: f(55.32), LatMax: f(55.45), LonMin: f(10.3), LonMax: f(10.5)}, + } + + insertTx := func(hash, sender, mmPayload string) { + ts := time.Now().UTC().Add(-30 * time.Minute).Format(time.RFC3339) + text := sender + ": " + mmPayload + if _, err := srv.db.conn.Exec( + `INSERT INTO transmissions (raw_hex,hash,first_seen,route_type,payload_type,channel_hash,decoded_json) VALUES (?,?,?,1,5,'#wardriving',?)`, + "aa", hash, ts, `{"sender":"`+sender+`","text":"`+text+`"}`, + ); err != nil { + t.Fatalf("insert tx %s: %v", hash, err) + } + } + insertTx("in-ode", "InOdense", "MM:c3e_zJ1rUA:55.4047,10.3810") // inside Odense (and DK) + insertTx("outside", "Elsewhere", "MM:c3e_zJ1rUA:40.0000,-74.0000") // outside every configured area + + req := httptest.NewRequest("GET", "/api/analytics/wardriving?window=24h", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + if w.Code != 200 { + t.Fatalf("status=%d body=%s", w.Code, w.Body.String()) + } + var resp WardrivingStatsResponse + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { + t.Fatalf("decode: %v body=%s", err, w.Body.String()) + } + + byShare := map[string]WardrivingGPSShare{} + for _, s := range resp.GPSShares { + byShare[s.Sender] = s + } + inOdense, ok := byShare["InOdense"] + if !ok { + t.Fatal("InOdense missing from GPSShares") + } + if inOdense.Area == nil || *inOdense.Area != "Odense by" { + t.Errorf("InOdense.Area = %v, want \"Odense by\" (most specific match, not \"Danmark (alle)\")", inOdense.Area) + } + elsewhere, ok := byShare["Elsewhere"] + if !ok { + t.Fatal("Elsewhere missing from GPSShares") + } + if elsewhere.Area != nil { + t.Errorf("Elsewhere.Area = %v, want nil (outside every configured area)", *elsewhere.Area) + } +} + +// TestHandleWardrivingStats_SessionArea covers approximating a session's +// area from its entry-point repeater (path[0]) when the sender never shares +// a literal GPS fix — must only trust a unique_prefix match, same discipline +// as /api/resolve-hops, and must never guess when a prefix is ambiguous or +// unresolvable. +func TestHandleWardrivingStats_SessionArea(t *testing.T) { + srv, router := setupTestServer(t) + if _, err := srv.db.conn.Exec(`DELETE FROM transmissions`); err != nil { + t.Fatalf("clear transmissions: %v", err) + } + if _, err := srv.db.conn.Exec(`DELETE FROM observations`); err != nil { + t.Fatalf("clear observations: %v", err) + } + + f := func(v float64) *float64 { return &v } + srv.cfg.Areas = map[string]AreaEntry{ + "ODE": {Label: "Odense by", LatMin: f(55.32), LatMax: f(55.45), LonMin: f(10.3), LonMax: f(10.5)}, + } + + // A repeater inside the configured area, resolvable only via a unique + // full-length prefix match (mirrors TestResolveHopsAPI_UniquePrefix). + if _, err := srv.db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, lat, lon, role) VALUES (?, ?, ?, ?, ?)", + "ff11223344", "OdenseRepeater", 55.4047, 10.3810, "repeater"); err != nil { + t.Fatalf("insert node: %v", err) + } + // A second, ambiguous prefix: two nodes share it, so it must never + // resolve to an area even though one of them has a position. + srv.db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, lat, lon, role) VALUES (?, ?, ?, ?, ?)", + "ee1aaaaaaa", "AmbigA", 55.40, 10.38, "repeater") + srv.db.conn.Exec("INSERT OR IGNORE INTO nodes (public_key, name, lat, lon, role) VALUES (?, ?, ?, ?, ?)", + "ee1bbbbbbb", "AmbigB", 55.41, 10.39, "repeater") + srv.store.InvalidateNodeCache() + + now := time.Now().UTC() + insertTx := func(hash, sender string, ts time.Time) int64 { + res, err := srv.db.conn.Exec( + `INSERT INTO transmissions (raw_hex,hash,first_seen,route_type,payload_type,channel_hash,decoded_json) VALUES (?,?,?,1,5,'#wardriving',?)`, + "aa", hash, ts.Format(time.RFC3339), `{"sender":"`+sender+`","text":"`+sender+`: MM:x"}`, + ) + if err != nil { + t.Fatalf("insert tx %s: %v", hash, err) + } + id, _ := res.LastInsertId() + return id + } + seaIdx := int64(0) + res, err := srv.db.conn.Exec(`INSERT INTO observers (id, name, iata) VALUES (?,?,?)`, "obsSEA", "SeattleObs", "SEA") + if err != nil { + t.Fatalf("insert observer: %v", err) + } + seaIdx, _ = res.LastInsertId() + insertObs := func(txID int64, pathJSON string) { + if _, err := srv.db.conn.Exec( + `INSERT INTO observations (transmission_id, observer_idx, snr, rssi, path_json, timestamp) VALUES (?,?,?,?,?,?)`, + txID, seaIdx, 1.0, -90.0, pathJSON, time.Now().Unix(), + ); err != nil { + t.Fatalf("insert observation: %v", err) + } + } + + txResolvable := insertTx("area1", "InOdense", now.Add(-10*time.Minute)) + insertObs(txResolvable, `["ff11223344"]`) + + txAmbiguous := insertTx("area2", "Ambiguous", now.Add(-10*time.Minute)) + insertObs(txAmbiguous, `["ee1"]`) // shared prefix of ee1aaaaaaa and ee1bbbbbbb -- 2 candidates + + txNoMatch := insertTx("area3", "NoMatch", now.Add(-10*time.Minute)) + insertObs(txNoMatch, `["deadbeef99"]`) + + req := httptest.NewRequest("GET", "/api/analytics/wardriving?window=24h", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + if w.Code != 200 { + t.Fatalf("status=%d body=%s", w.Code, w.Body.String()) + } + var resp WardrivingStatsResponse + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { + t.Fatalf("decode: %v body=%s", err, w.Body.String()) + } + + bySender := map[string]WardrivingSession{} + for _, s := range resp.Sessions { + bySender[s.Sender] = s + } + inOdense, ok := bySender["InOdense"] + if !ok { + t.Fatal("InOdense session missing") + } + if inOdense.Area == nil || *inOdense.Area != "Odense by" { + t.Errorf("InOdense.Area = %v, want \"Odense by\" (unique_prefix match with a known position)", inOdense.Area) + } + ambiguous, ok := bySender["Ambiguous"] + if !ok { + t.Fatal("Ambiguous session missing") + } + if ambiguous.Area != nil { + t.Errorf("Ambiguous.Area = %v, want nil (prefix matches 2 candidates, must not guess)", *ambiguous.Area) + } + noMatch, ok := bySender["NoMatch"] + if !ok { + t.Fatal("NoMatch session missing") + } + if noMatch.Area != nil { + t.Errorf("NoMatch.Area = %v, want nil (prefix matches no known node)", *noMatch.Area) + } +} + // TestHandleWardrivingStats_InvalidWindow mirrors the existing scope-stats // window validation. func TestHandleWardrivingStats_InvalidWindow(t *testing.T) { diff --git a/public/analytics.js b/public/analytics.js index c56d2fa8..2144c869 100644 --- a/public/analytics.js +++ b/public/analytics.js @@ -4504,12 +4504,18 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf var subtabKey = 'scopes_subtab'; var selectedSubtab = (typeof sessionStorage !== 'undefined' && sessionStorage.getItem(subtabKey)) || 'overview'; - // Role/text filter for the "Nodes Without a Default Scope" section + // Role/text/geo filter for the "Nodes Without a Default Scope" section // below. Lives at this scope (not inside updateData) so it survives // the 60s auto-refresh re-render — same reasoning as selectedWindow // above, just kept in memory rather than sessionStorage since it's a // finer-grained, more transient filter. - var noScopeFilter = { role: '', q: '' }; + var noScopeFilter = { role: '', q: '', geo: '' }; + + // Geo filter for "Repeaters Never Relaying Any Scope" below — same + // domestic/foreign split as noScopeFilter.geo, kept separate since the + // two sections' result sets are independent (see + // computeRepeatersNeverRelayingScope's doc comment). + var neverRelayFilter = { geo: '' }; // Encrypted/unencrypted filter for "Scope Adoption by Channel" below — // same persistence reasoning as noScopeFilter above. '' = all, @@ -4556,6 +4562,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf '' + '' + '
' + + '
' + '
' + '
' + '
' + @@ -4656,7 +4663,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf var searchFocused = document.activeElement && document.activeElement.id === 'noScopeSearch'; var caretPos = searchFocused ? document.activeElement.selectionStart : null; - var noScope = computeNodesWithoutScope(allNodes, 100, { role: noScopeFilter.role, q: noScopeFilter.q }); + var noScope = computeNodesWithoutScope(allNodes, 100, { role: noScopeFilter.role, q: noScopeFilter.q, geo: noScopeFilter.geo }); var roleSummaryText = noScope.roleSummary.map(function(r) { return r.count.toLocaleString() + ' ' + esc(r.role); }).join(', '); var roleButtons = ''; }).join(''); - var controlsHtml = '
' + + var controlsHtml = geoFilterButtons(noScopeFilter, 'geo-filter') + + '
' + roleButtons + '' + @@ -4683,7 +4691,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf 'NodeRoleLast Seen' + '' + noScopeRows + '' + ''; - } else if (noScopeFilter.role || noScopeFilter.q) { + } else if (noScopeFilter.role || noScopeFilter.q || noScopeFilter.geo) { resultsBody = '

No nodes without a default scope match this filter.

'; } else { resultsBody = '

Every known node has a configured default scope.

'; @@ -4709,6 +4717,13 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf }); }); + sectionEl.querySelectorAll('[data-geo-filter]').forEach(function(btn) { + btn.addEventListener('click', function() { + noScopeFilter.geo = btn.dataset.geoFilter || ''; + renderNoScopeSection(allNodes); + }); + }); + var searchInput = sectionEl.querySelector('#noScopeSearch'); if (searchInput) { searchInput.addEventListener('input', debounce(function() { @@ -4812,7 +4827,41 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf } } + // regionScope -> area label lookup, e.g. "dk-aarhus" -> "Aarhus by". + // An area can link more than one scope (e.g. Europa: both "eu" and + // "europe"), so each one gets its own entry pointing back to the same + // label. Areas rarely change, so this is fetched once and reused + // across every region-code section below rather than re-fetched per + // render. + var regionAreaLabels = null; + async function loadRegionAreaLabels() { + if (regionAreaLabels) return regionAreaLabels; + regionAreaLabels = {}; + try { + var areas = await api('/config/areas', { ttl: CLIENT_TTL.nodeDetail }); + (areas || []).forEach(function(a) { + (a.regionScopes || []).forEach(function(rs) { + regionAreaLabels[rs.toLowerCase()] = a.label; + }); + }); + } catch (e) { /* leave empty -- region codes render unlabeled */ } + return regionAreaLabels; + } + // A configured hashRegions name is always "#"-prefixed (see + // internal/regions.Normalize); AreaEntry.RegionScopes entries are + // stored without it, so strip before looking up. + function regionAreaLabel(rawRegionName) { + if (!regionAreaLabels || !rawRegionName) return null; + var key = String(rawRegionName).replace(/^#/, '').toLowerCase(); + return regionAreaLabels[key] || null; + } + function regionCodeHtml(rawRegionName) { + var label = regionAreaLabel(rawRegionName); + return '' + esc(rawRegionName) + '' + (label ? ' (' + esc(label) + ')' : ''); + } + async function updateData(d, w) { + await loadRegionAreaLabels(); var s = d.summary; // #1838: denominator = transport-carrying transmissions (route_type 0,3). // Unscoped now includes non-transport routes (1,2) which are inherently @@ -5035,6 +5084,77 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf } } + // Scope Adoption by Area: for each configured geographic area, which + // SPECIFIC nodes actually support the region that area is linked to + // (own default_scope OR ever relayed it — a repeater can carry + // dk-oj traffic and thereby support Østjylland without configuring + // dk-oj as its own scope) and which sit there but don't. Independent + // of Region Utilization below, which only knows about region + // strings that already appeared in a message — a real area with + // real nodes that never produced one is invisible there. + var areaAdoptEl = document.getElementById('scopes-area-adoption'); + if (areaAdoptEl) { + var byArea = d.scopeAdoptionByArea || []; + if (byArea.length > 0) { + function nodeLinks(refs) { + return (refs || []).map(function(r) { + return '' + esc(r.name) + ''; + }).join(', '); + } + // For areas linking more than one scope (e.g. Europa: "eu" and + // "europe"), a flat Supporting list can't say WHICH scope each + // node actually uses/relays — split it into one sub-list per + // scope instead. A node matching more than one scope (its own + // default_scope is one, and it relays another) appears in every + // group it matched, not just the first. Single-scope areas keep + // the plain flat list (grouping by one group is just noise). + function matchingByScope(matching, regionScopes) { + return regionScopes.map(function(rs) { + var inGroup = (matching || []).filter(function(m) { + return (m.matchedScopes || []).indexOf(rs) !== -1; + }); + return '
#' + esc(rs) + ' (' + inGroup.length + '): ' + + (inGroup.length ? nodeLinks(inGroup) : 'none') + + '
'; + }).join(''); + } + var areaGroups = byArea.map(function(a) { + var summary, body; + if (a.regionScopes && a.regionScopes.length) { + var matchCount = a.nodesMatchingArea || 0; + var scopeCodes = a.regionScopes.map(function(rs) { return '#' + esc(rs) + ''; }).join(' or '); + summary = esc(a.label) + ' — ' + matchCount.toLocaleString() + ' of ' + a.totalNodes.toLocaleString() + + ' support ' + scopeCodes + ' (' + pct(matchCount, a.totalNodes) + ')'; + var supportingBody = a.regionScopes.length > 1 + ? matchingByScope(a.matching, a.regionScopes) + : (a.matching && a.matching.length ? nodeLinks(a.matching) : 'none'); + body = + '
Supporting (' + (a.matching || []).length + '): ' + supportingBody + + '
' + + '
Not supporting (' + (a.notMatching || []).length + '): ' + + (a.notMatching && a.notMatching.length ? nodeLinks(a.notMatching) : 'none') + + '
'; + } else { + summary = esc(a.label) + ' — ' + a.totalNodes.toLocaleString() + ' node' + (a.totalNodes === 1 ? '' : 's') + + ', no region linked to this area'; + body = '

This area has no regionScopes configured, so there\'s nothing to check adoption against.

'; + } + return '
' + + '' + summary + '' + + '
' + body + '
' + + '
'; + }).join(''); + setSectionHtml(areaAdoptEl, detailsSection( + 'Scope Adoption by Area (' + byArea.length.toLocaleString() + ' areas)', + null, + areaGroups, + 'scope-adoption-by-area' + )); + } else { + areaAdoptEl.innerHTML = ''; + } + } + // Region utilization: how much of the configured hashRegions list // has never actually matched anything — all-time (not window-scoped), // so it doesn't fluctuate with the 1h/24h/7d selector above. Only @@ -5046,7 +5166,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf var unused = d.unusedRegions || []; var usedCount = configured - unused.length; var unusedPct = (unused.length / configured * 100).toFixed(1); - var listHtml = unused.map(function(name) { return esc(name); }).join(', '); + var listHtml = unused.map(function(name) { return regionCodeHtml(name); }).join(', '); setSectionHtml(utilEl, detailsSection( 'Region Utilization (' + usedCount.toLocaleString() + ' of ' + configured.toLocaleString() + ' used)', 'All-time, not limited to the window above — has this configured region ever matched a message still in retention?', @@ -5084,7 +5204,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf return '' + esc(rp.name) + ''; }).join(', '); return '
' + - '' + esc(g.region) + ' — ' + g.count.toLocaleString() + ' ' + unitLabel + (g.count === 1 ? '' : 's') + '' + + '' + regionCodeHtml(g.region) + ' — ' + g.count.toLocaleString() + ' ' + unitLabel + (g.count === 1 ? '' : 's') + '' + '
' + links + '
' + '
'; }).join('') @@ -5110,7 +5230,7 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf var bridgeBody; if (bridges.length > 0) { var bridgeRows = bridges.map(function(b) { - var regionList = b.regions.map(function(r) { return '' + esc(r) + ''; }).join(', '); + var regionList = b.regions.map(function(r) { return regionCodeHtml(r); }).join(', '); return '' + '' + esc(b.name) + '' + '' + b.count + '' + @@ -5165,39 +5285,64 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf // computeRepeatersNeverRelayingScope doc comment). Reuses the same // `allNodes` fetched above. var neverRelayEl = document.getElementById('scopes-never-relay-scope'); - if (neverRelayEl && !allNodes) { - setSectionHtml(neverRelayEl, detailsSection('Repeaters Never Relaying Any Scope', null, '

Failed to load.

', 'never-relay')); - } else if (neverRelayEl) { - try { - var neverRelay = computeRepeatersNeverRelayingScope(allNodes, 100); - var neverRelayBody; - if (neverRelay.sortedCapped.length > 0) { - var neverRelayRows = neverRelay.sortedCapped.map(function(n) { - return '' + esc(n.name || n.public_key) + '' + - '' + esc(n.role || '—') + '' + - '' + (n.relay_count_24h || 0).toLocaleString() + '' + - '' + timeAgo(n.last_seen) + ''; - }).join(''); - neverRelayBody = '' + - '' + - '' + neverRelayRows + '' + - '
RepeaterRoleRelays (24h)Last Seen
'; - } else { - neverRelayBody = '

Every known repeater/room has relayed at least one region-scoped packet.

'; - } - setSectionHtml(neverRelayEl, detailsSection( - 'Repeaters Never Relaying Any Scope (' + neverRelay.total.toLocaleString() + ')', - 'Repeater/room nodes that have never carried a single region-scoped (TRANSPORT_FLOOD/DIRECT) packet, ever — not the same set as "no default_scope" above: a repeater\'s hashRegions config can let it relay for others even when its own adverts never carry a matching transport code. ' + - 'Sorted by current relay volume — the busiest ones are the most consequential to configure first' + (neverRelay.truncated ? ' (showing the top ' + neverRelay.sortedCapped.length + ')' : '') + '.', - neverRelayBody, - 'never-relay' - )); - } catch (e) { + if (neverRelayEl) { + if (allNodes) { + renderNeverRelaySection(allNodes); + } else { setSectionHtml(neverRelayEl, detailsSection('Repeaters Never Relaying Any Scope', null, '

Failed to load.

', 'never-relay')); } } } + // Renders (and re-renders, on geo filter change) the "Repeaters Never + // Relaying Any Scope" section from the already-fetched node list — same + // reactive-filter pattern as renderNoScopeSection above. + function renderNeverRelaySection(allNodes) { + var neverRelayElInner = document.getElementById('scopes-never-relay-scope'); + if (!neverRelayElInner) return; + try { + var neverRelay = computeRepeatersNeverRelayingScope(allNodes, 100, { geo: neverRelayFilter.geo }); + var neverRelayBody; + if (neverRelay.sortedCapped.length > 0) { + var neverRelayRows = neverRelay.sortedCapped.map(function(n) { + return '' + esc(n.name || n.public_key) + '' + + '' + esc(n.role || '—') + '' + + '' + (n.relay_count_24h || 0).toLocaleString() + '' + + '' + timeAgo(n.last_seen) + ''; + }).join(''); + neverRelayBody = '' + + '' + + '' + neverRelayRows + '' + + '
RepeaterRoleRelays (24h)Last Seen
'; + } else if (neverRelayFilter.geo) { + neverRelayBody = '

No repeaters match this filter.

'; + } else { + neverRelayBody = '

Every known repeater/room has relayed at least one region-scoped packet.

'; + } + setSectionHtml(neverRelayElInner, detailsSection( + 'Repeaters Never Relaying Any Scope (' + neverRelay.total.toLocaleString() + ')', + 'Repeater/room nodes that have never carried a single region-scoped (TRANSPORT_FLOOD/DIRECT) packet, ever — not the same set as "no default_scope" above: a repeater\'s hashRegions config can let it relay for others even when its own adverts never carry a matching transport code. ' + + 'Sorted by current relay volume — the busiest ones are the most consequential to configure first' + + (neverRelay.filteredTotal !== neverRelay.total ? ', filtered to ' + neverRelay.filteredTotal.toLocaleString() + ' matching below' : '') + + (neverRelay.truncated ? ' (showing the top ' + neverRelay.sortedCapped.length + ')' : '') + '.', + geoFilterButtons(neverRelayFilter, 'never-relay-geo-filter') + neverRelayBody, + 'never-relay' + )); + } catch (e) { + setSectionHtml(neverRelayElInner, detailsSection('Repeaters Never Relaying Any Scope', null, '

Failed to load.

', 'never-relay')); + return; + } + + var sectionEl = document.getElementById('scopes-never-relay-scope'); + if (!sectionEl) return; + sectionEl.querySelectorAll('[data-never-relay-geo-filter]').forEach(function(btn) { + btn.addEventListener('click', function() { + neverRelayFilter.geo = btn.dataset.neverRelayGeoFilter || ''; + renderNeverRelaySection(allNodes); + }); + }); + } + load(selectedWindow); @@ -5220,6 +5365,27 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf // returns a most-recently-active-first slice capped at `cap` rows. // `total` is always the full unfiltered count; `filteredTotal` reflects // opts. + + // Classifies live from the node's own lat/lon against the currently + // configured geo_filter (window.MC_GEO_FILTER, set from + // /api/config/client — see public/roles.js), mirroring the Nodes tab's + // Domestic/Foreign filter (public/nodes.js) and renderForeignTrafficTab's + // isForeignNode below — NOT the node's one-way, never-cleared `foreign` + // DB flag. + function nodeMatchesGeo(n, mode) { + if (!mode) return true; + var domestic = nodePassesGeoFilter(n.lat, n.lon, window.MC_GEO_FILTER); + return mode === 'domestic' ? domestic : !domestic; + } + + function geoFilterButtons(filter, dataAttr) { + return '
' + + '' + + '' + + '' + + '
'; + } + function computeNodesWithoutScope(allNodes, cap, opts) { opts = opts || {}; var noScopeNodes = allNodes.filter(function(n) { return !n.default_scope; }); @@ -5233,6 +5399,9 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf .map(function(r) { return { role: r, count: roleCounts[r] }; }); var filtered = noScopeNodes; + if (opts.geo) { + filtered = filtered.filter(function(n) { return nodeMatchesGeo(n, opts.geo); }); + } if (opts.role) { filtered = filtered.filter(function(n) { return (n.role || 'unknown') === opts.role; }); } @@ -5268,16 +5437,19 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf // out rather than silently 0-counted alongside them. Sorted by // relay_count_24h descending: the busiest unconfigured repeaters are the // most consequential ones to fix first. - function computeRepeatersNeverRelayingScope(allNodes, cap) { + function computeRepeatersNeverRelayingScope(allNodes, cap, opts) { + opts = opts || {}; var candidates = allNodes.filter(function(n) { return (n.role === 'repeater' || n.role === 'room') && (!n.transported_scopes || n.transported_scopes.length === 0); }); - var sorted = candidates.slice().sort(function(a, b) { + var filtered = opts.geo ? candidates.filter(function(n) { return nodeMatchesGeo(n, opts.geo); }) : candidates; + var sorted = filtered.slice().sort(function(a, b) { return Number(b.relay_count_24h || 0) - Number(a.relay_count_24h || 0); }); return { total: candidates.length, + filteredTotal: filtered.length, sortedCapped: sorted.slice(0, cap), truncated: sorted.length > cap, }; @@ -5545,16 +5717,20 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf } var shown = expanded ? sessions : sessions.slice(0, TOP_N_LIMIT); var rows = shown.map(function(s) { + var areaBadge = s.area + ? '' + esc(s.area) + '' + : '—'; return '' + senderTriggerHtml(s.sender, s.startTime, s.endTime) + '' + '' + (typeof timeAgo === 'function' ? timeAgo(s.startTime) : s.startTime) + '' + '' + formatSessionDuration(s.durationMinutes) + '' + '' + s.messageCount.toLocaleString() + '' + '' + s.entryPointCount.toLocaleString() + '' + '' + s.observerCount.toLocaleString() + '' + + '' + areaBadge + '' + '' + formatAirtimeMs(s.airtimeMs) + ''; }).join(''); - return '' + - '' + + return '
SenderStartedDurationMessagesEntry PointsObserversAirtime
' + + '' + '' + rows + '' + '
SenderStartedDurationMessagesEntry PointsObserversAreaAirtime
' + topNToggleHtml(sessions.length, expanded, 'sessions'); } @@ -5684,13 +5860,17 @@ function destroy() { _stopRolesRefresh(); _stopScopesRefresh(); _stopForeignTraf return '

No sender has shared an explicit position in this window.

'; } var rows = shares.map(function(s) { + var areaBadge = s.area + ? '' + esc(s.area) + '' + : '—'; return '' + esc(s.sender) + '' + '' + mapLinkHtml(s.lat, s.lon) + '' + + '' + areaBadge + '' + '' + s.messageCount.toLocaleString() + '' + '' + (typeof timeAgo === 'function' ? timeAgo(s.lastSeen) : s.lastSeen) + ''; }).join(''); return '' + - '' + + '' + '' + rows + '' + '
SenderPosition (most recent)Times SharedLast Seen
SenderPosition (most recent)AreaTimes SharedLast Seen
'; } diff --git a/public/channels.js b/public/channels.js index 40abb9db..2dea7662 100644 --- a/public/channels.js +++ b/public/channels.js @@ -1423,6 +1423,10 @@ var observer = m.data?.packet?.observer_name || m.data?.observer || null; var scope = m.data?.scope_name || m.data?.packet?.scope_name || null; var routeType = m.data?.route_type ?? m.data?.packet?.route_type ?? null; + // Same path[0]-resolved area as the REST message list (server-side + // resolveEntryPointArea, see store.go) -- already computed at + // broadcast time, just read it here. + var area = m.data?.area || m.data?.packet?.area || null; // Update channel list entry — only once per unique packet hash var isFirstObservation = pktHash && !seenHashes.has(pktHash + ':' + channelKey); @@ -1476,6 +1480,7 @@ snr: snr, scope: scope, routeType: routeType, + area: area, // #1498: mark as WS-pushed so a later REST replacement // (selectChannel / refreshMessages) can merge instead of // stomp. Without this flag the REST response wipes any @@ -2273,8 +2278,16 @@ // HMAC collision made the match ambiguous) — show it as unknown // rather than silently omitting the tag. const isTransportRoute = msg.routeType === 0 || msg.routeType === 3; - if (msg.scope) meta.push(`Scope: ${escapeHtml(msg.scope)}`); - else if (isTransportRoute) meta.push('Scope: unknown'); + if (msg.scope) meta.push(`scope: ${escapeHtml(msg.scope)}`); + else if (isTransportRoute) meta.push('scope: unknown'); + // msg.area (set server-side from the message's own path[0] + // entry-point repeater, not from the scope string) is where the + // SENDER physically was — distinct from the scope-linked area + // above, since e.g. a sender in Aarhus can still send with the + // broad #dk scope. Only present when path[0] resolved unambiguously + // (unique_prefix) to a positioned node; omitted otherwise, not + // guessed. + if (msg.area) meta.push(`area: ${escapeHtml(msg.area)}`); const safeId = btoa(encodeURIComponent(sender)); // #1367: emit BOTH the new chat-app class names (.ch-message / diff --git a/test-analytics-nodes-without-scope.js b/test-analytics-nodes-without-scope.js index f60b173f..160b0df4 100644 --- a/test-analytics-nodes-without-scope.js +++ b/test-analytics-nodes-without-scope.js @@ -219,6 +219,29 @@ test('no opts (or empty opts) behaves exactly as before — filteredTotal equals assert.strictEqual(result.filteredTotal, result.total); }); +test('opts.geo narrows to domestic/foreign via window.MC_GEO_FILTER, same box the Nodes tab uses', () => { + // Denmark-ish box: lat 54-58, lon 8-13. A node outside it is "foreign". + ctx.window.MC_GEO_FILTER = { latMin: 54, latMax: 58, lonMin: 8, lonMax: 13 }; + try { + const nodes = [ + { public_key: 'pk1', name: 'Domestic1', role: 'repeater', default_scope: null, lat: 56.0, lon: 10.0 }, + { public_key: 'pk2', name: 'Foreign1', role: 'repeater', default_scope: null, lat: 40.0, lon: 20.0 }, + ]; + const domestic = computeNodesWithoutScope(nodes, 100, { geo: 'domestic' }); + assert.strictEqual(domestic.filteredTotal, 1); + assert.strictEqual(domestic.sortedCapped[0].name, 'Domestic1'); + + const foreign = computeNodesWithoutScope(nodes, 100, { geo: 'foreign' }); + assert.strictEqual(foreign.filteredTotal, 1); + assert.strictEqual(foreign.sortedCapped[0].name, 'Foreign1'); + + const all = computeNodesWithoutScope(nodes, 100); + assert.strictEqual(all.filteredTotal, 2, 'no geo opt leaves both nodes'); + } finally { + ctx.window.MC_GEO_FILTER = null; + } +}); + console.log('\n=== analytics.js: computeRepeatersNeverRelayingScope ==='); test('is exported for testing', () => { @@ -285,6 +308,26 @@ test('returns zero total when every repeater/room has relayed at least one scope assert.deepStrictEqual(result.sortedCapped, []); }); +test('opts.geo narrows to domestic/foreign, leaving total (unfiltered) unchanged', () => { + ctx.window.MC_GEO_FILTER = { latMin: 54, latMax: 58, lonMin: 8, lonMax: 13 }; + try { + const nodes = [ + { public_key: 'pk1', name: 'DomesticRepeater', role: 'repeater', transported_scopes: null, lat: 56.0, lon: 10.0 }, + { public_key: 'pk2', name: 'ForeignRepeater', role: 'repeater', transported_scopes: null, lat: 40.0, lon: 20.0 }, + ]; + const domestic = computeRepeatersNeverRelayingScope(nodes, 100, { geo: 'domestic' }); + assert.strictEqual(domestic.total, 2, 'total stays the full unfiltered count'); + assert.strictEqual(domestic.filteredTotal, 1); + assert.strictEqual(domestic.sortedCapped[0].name, 'DomesticRepeater'); + + const foreign = computeRepeatersNeverRelayingScope(nodes, 100, { geo: 'foreign' }); + assert.strictEqual(foreign.filteredTotal, 1); + assert.strictEqual(foreign.sortedCapped[0].name, 'ForeignRepeater'); + } finally { + ctx.window.MC_GEO_FILTER = null; + } +}); + console.log('\n════════════════════════════════════════'); console.log(` Nodes Without Scope: ${passed} passed, ${failed} failed`); console.log('════════════════════════════════════════'); diff --git a/test-analytics-wardriving-tab.js b/test-analytics-wardriving-tab.js index 131a740d..d281ed9f 100644 --- a/test-analytics-wardriving-tab.js +++ b/test-analytics-wardriving-tab.js @@ -188,6 +188,23 @@ function makeApiStub(wardrivingResp, resolveHopsResp) { assert.ok(section.includes('href="#/map?lat=55.59743&lon=13.00128&zoom=15"'), 'the position should link to the live map centered on it'); }); + await testAsync('GPS Sharing shows the area badge when the API resolved one, and a dash otherwise', async () => { + const ctx = makeAnalyticsSandbox(makeApiStub(makeWardrivingResponse({ + gpsShares: [ + { sender: 'InOdense', lat: 55.4047, lon: 10.381, messageCount: 3, lastSeen: '2026-07-20T09:00:00Z', area: 'Odense by' }, + { sender: 'NoAreaMatch', lat: 40.0, lon: -74.0, messageCount: 1, lastSeen: '2026-07-20T09:00:00Z' }, + ], + }))); + const el = fakeEl(); + await ctx.window._analyticsRenderWardrivingTab(el); + const startIdx = el.innerHTML.indexOf('id="wardrivingGPSShares"'); + const section = el.innerHTML.slice(startIdx); + assert.ok(section.includes('Area'), 'GPS Sharing table should have an Area column'); + assert.ok(section.includes('>Odense by<'), 'a resolved area should render as a badge with its label'); + const noAreaRowIdx = section.indexOf('NoAreaMatch'); + assert.ok(noAreaRowIdx > -1, 'the unresolved-area sender should still be listed'); + }); + await testAsync('GPS Sharing shows a neutral message when nobody has shared a position', async () => { const ctx = makeAnalyticsSandbox(makeApiStub(makeWardrivingResponse({ gpsShares: [] }))); const el = fakeEl(); @@ -334,6 +351,22 @@ function makeApiStub(wardrivingResp, resolveHopsResp) { assert.ok(section.includes('—'), 'a session with no airtime data (DB-only mode) should show a dash, not blank/null'); }); + await testAsync('Sessions table shows the approximate entry-point area, and a dash when unresolved', async () => { + const ctx = makeAnalyticsSandbox(makeApiStub(makeWardrivingResponse({ + sessions: [ + { sender: 'Alice', startTime: '2026-07-20T09:00:00Z', endTime: '2026-07-20T09:00:00Z', durationMinutes: 0, messageCount: 1, entryPointCount: 1, observerCount: 1, airtimeMs: 245, area: 'Odense by' }, + { sender: 'Bob', startTime: '2026-07-20T08:30:00Z', endTime: '2026-07-20T08:30:00Z', durationMinutes: 0, messageCount: 1, entryPointCount: 1, observerCount: 1, airtimeMs: null }, + ], + }))); + const el = fakeEl(); + await ctx.window._analyticsRenderWardrivingTab(el); + const startIdx = el.innerHTML.indexOf('id="wardrivingSessions"'); + const endIdx = el.innerHTML.indexOf('id="wardrivingEntryPoints"'); + const section = el.innerHTML.slice(startIdx, endIdx); + assert.ok(section.includes('Area'), 'Sessions table should have an Area column'); + assert.ok(section.includes('>Odense by<'), 'a resolved area should render as a badge'); + }); + await testAsync('Entry Points resolves unique_prefix repeaters and folds ambiguous into one bucket', async () => { const ctx = makeAnalyticsSandbox(makeApiStub(makeWardrivingResponse(), { resolved: {