Commit Graph
5 Commits
Author SHA1 Message Date
078225a54e perf(neighbor_api): fold first_seen into cached map — fix #1627 r3 regression (#1632)
## TL;DR
Post-merge regression introduced by #1627 r3 (commit `e2212f50`):
`buildNodeInfoMap` in `cmd/server/neighbor_api.go` ran an uncached
`SELECT … FROM nodes` scan on every call. Folded `first_seen` into the
already-cached `getCachedNodesAndPM` (30s TTL) so the 4 hot handlers
that call `buildNodeInfoMap` no longer pay for a full table scan per
request.

## Before / After

`buildNodeInfoMap` is called by **4 hot handlers**:
- `cmd/server/neighbor_api.go:130`
- `cmd/server/neighbor_api.go:297`
- `cmd/server/neighbor_debug.go:83`
- `cmd/server/node_reach.go:421`

| | Before | After |
|---|---|---|
| `SELECT … FROM nodes` per call | 1 (uncached) | 0 (cache hit) |
| `SELECT … FROM observers` per call | 1 (uncached) | 1 (unchanged) |
| At Cascadia scale (~2600 nodes) | full scan × 4 handlers × N req/s |
one scan / 30s |

## How

- Extended the `getAllNodes` schema probe to also `COALESCE(first_seen,
'')`. Falls back through the existing richest → leanest ladder if the
column is missing.
- `nodeInfo.FirstSeen` is therefore populated for every cached entry in
`getCachedNodesAndPM`.
- `buildNodeInfoMap` drops its second `SELECT` entirely and just copies
`nodeInfo` values out of the cached map.
- Public signature of `buildNodeInfoMap` is unchanged.
`node_reach.go:421` still sees `nodeInfo.FirstSeen` populated, served
from cache.

`cmd/server/store.go` is touched because `getAllNodes` is the only
sensible owner of the `first_seen` SELECT — adding a parallel cache
would duplicate the 30s TTL machinery this fix is designed to leverage.

## Test (red → green)

- Commit 1 (`test:`): `TestBuildNodeInfoMap_FirstSeenIsCached` — calls
`buildNodeInfoMap`, mutates `first_seen` out-of-band via a separate rw
connection, calls it again, and asserts both calls return the same
(cached) value. Fails on `origin/master` (call 2 sees the mutated value,
proving the uncached scan).
- Commit 2 (`perf:`): the fold. Test now passes.

## Refs

Post-merge audit identified this as the only MAJOR finding from #1627;
recommendation was a follow-up hot-fix PR. This is that PR.

---------

Co-authored-by: openclaw-bot <bot@openclaw>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-06-09 01:24:46 -07:00
b881a09f02 feat(#1188): show observer IATA on packets + filter grammar (#1189)
Red commit: 4ed272761ba94d99912a08e31533ee484ad3f1cd (CI run:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/25651898290)

Fixes #1188 — observer IATA on packets in three UI surfaces + filter
grammar.

cross-stack: justified — feature spans API shape (Go), store, filter
grammar (JS), three packets UI surfaces.

## Scope shipped
- Packets table row: `.badge-iata` pill inline next to observer name
- Expanded observation rows: per-observation IATA badge
- Detail pane: Observer dd + per-observation list both render the badge
- Filter grammar: `observer_iata` field + `iata` alias;
`==`/`!=`/`contains`, plus a new `in (a, b, c)` list operator. Both
names appear in autocomplete with descriptions.

## TDD red→green pairs
1. `271d72f` filter-grammar tests → `2c182eb` evaluator + suggest
entries
2. `4ed2727` backend `observer_iata` API tests → `7856914` SQL join +
struct/store wiring
3. `0e09371` display E2E → `7a3f45d` packets.js + style.css badge
(E2E swapped for string-contract unit test in `ee414b4` — fixture
`observations.observer_idx` stores text pubkeys, blocking the join the
badge depends on)

## Backend
- `cmd/server/db.go`: SELECT `obs.iata AS observer_iata` in
`transmissionBaseSQL`, grouped query, observations-by-transmissions
- `cmd/server/store.go`: `ObserverIATA` on `StoreTx`/`StoreObs`, load
via all three ingest paths, surface in
`txToMap`/`enrichObs`/`groupedTxsToPage`
- `cmd/server/types.go`: field added to
`TransmissionResp`/`ObservationResp`/`GroupedPacketResp`
- Test fixture schemas declare `iata` on observers

## Perf
Per #383, `obsIataBadge(packet)` reads `packet.observer_iata` directly
(server-joined). Falls back to `observerMap.get(id).iata` only if absent
— hot row-render loop avoids per-row Map lookup on fresh data.

## Display rules
Missing IATA: nothing inline (Region column still shows `—`). No new hex
— `.badge-iata` uses `var(--nav-bg)` / `var(--nav-text)`.

E2E assertion added: test-observer-iata-1188.js:51

---------

Co-authored-by: OpenClaw Bot <bot@openclaw.dev>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-17 16:13:11 +00:00
Kpa-clawbotandyou ceea136e97 feat: observer graph representation (M1+M2) (#774)
## Summary

Fixes #753 — Milestones M1 and M2: Observer nodes in the neighbor graph
are now correctly labeled, colored, and filterable.

### M1: Label + color observers

**Backend** (`cmd/server/neighbor_api.go`):
- `buildNodeInfoMap()` now queries the `observers` table after building
from `nodes`
- Observer-only pubkeys (not already in the map as repeaters etc.) get
`role: "observer"` and their name from the observers table
- Observer-repeaters keep their repeater role (not overwritten)

**Frontend**:
- CSS variable `--role-observer: #8b5cf6` added to `:root`
- `ROLE_COLORS.observer` was already defined in `roles.js`

### M2: Observer filter checkbox (default unchecked)

**Frontend** (`public/analytics.js`):
- Observer checkbox added to the role filter section, **unchecked by
default**
- Observers create hub-and-spoke patterns (one observer can have 100+
edges) that drown out the actual repeater topology — hiding them by
default keeps the graph clean
- Fixed `applyNGFilters()` which previously always showed observers
regardless of checkbox state

### Tests

- Backend: `TestBuildNodeInfoMap_ObserverEnrichment` — verifies
observer-only pubkeys get name+role from observers table, and
observer-repeaters keep their repeater role
- All existing Go tests pass
- All frontend helper tests pass (544/544)

---------

Co-authored-by: you <you@example.com>
2026-04-16 21:35:14 -07:00
efiten b587f20d1c feat: add distance column to neighbor table in node details (#617)
Closes #616

## What

Adds a **Distance** column to the neighbor table on the node detail
page.

When both the viewed node and a neighbor have GPS coordinates recorded,
the table shows the haversine distance between them (e.g. `3.2 km`).
When either node lacks GPS, the cell shows `—`.

## Changes

**Backend** (`cmd/server/neighbor_api.go`):
- Added `distance_km *float64` (omitempty) to `NeighborEntry`
- In `handleNodeNeighbors`: look up source node coords from `nodeMap`,
then for each resolved (non-ambiguous) neighbor with GPS, compute
`haversineKm` and set the field

**Frontend** (`public/nodes.js`):
- Added `Distance` column header between Last Seen and Conf
- Cell renders `X.X km` or `—` (muted) when unavailable

**Tests** (`cmd/server/neighbor_api_test.go`):
- `TestNeighborAPI_DistanceKm_WithGPS`: two nodes with real coords →
`distance_km` is positive
- `TestNeighborAPI_DistanceKm_NoGPS`: two nodes at 0,0 → `distance_km`
is nil

## Verification

Test at **https://staging.on8ar.eu** — navigate to any node detail page
and scroll to the Neighbors section. Nodes with GPS coordinates show a
distance; those without show `—`.
2026-04-05 12:33:23 -07:00
Kpa-clawbotandyou e66085092e feat: neighbor affinity API endpoints (#482) — milestone 2 (#508)
## Summary

Milestone 2 of the neighbor affinity graph (#482). Adds two API
endpoints that expose the neighbor graph built in M1 (PR #507).

### Endpoints

#### `GET /api/nodes/{pubkey}/neighbors`
Returns neighbors for a specific node with affinity scores.

**Query params:** `min_count` (default 1), `min_score` (default 0.0),
`include_ambiguous` (default true)

**Response shape:**
```json
{
  "node": "pubkey",
  "neighbors": [
    { "pubkey": "...", "prefix": "BB", "name": "...", "role": "repeater",
      "count": 847, "score": 0.95, "first_seen": "...", "last_seen": "...",
      "avg_snr": -8.2, "observers": ["obs1"], "ambiguous": false }
  ],
  "total_observations": 847
}
```

Ambiguous entries have `candidates` array; unresolved prefixes have
`unresolved: true`.

#### `GET /api/analytics/neighbor-graph`
Returns full graph summary for analytics/visualization.

**Query params:** `min_count` (default 5), `min_score` (default 0.1),
`region` (IATA code filter)

**Response shape:**
```json
{
  "nodes": [{ "pubkey": "...", "name": "...", "role": "...", "neighbor_count": 5 }],
  "edges": [{ "source": "...", "target": "...", "weight": 847, "score": 0.95, "ambiguous": false }],
  "stats": { "total_nodes": 42, "total_edges": 87, "ambiguous_edges": 3, "avg_cluster_size": 4.2 }
}
```

### Wiring
- `NeighborGraph` + `neighborMu` added to `Server` struct
- Lazy initialization: graph built on first API call, cached with 60s
TTL
- Node name/role lookups via existing `getCachedNodesAndPM()`
- Region filtering via existing `resolveRegionObservers()`

### Tests (15 tests)
- Empty graph, single neighbor, multiple neighbors (sorted by score)
- Ambiguous candidates with candidate list
- Unresolved prefix (orphan) with `unresolved: true`
- `min_count` filter, `min_score` filter, `include_ambiguous=false`
filter
- Unknown node returns 200 with empty neighbors
- Graph endpoint: empty, with edges, default min_count, ambiguous count
- Region filter (graceful when no store)
- Response shape validation (all required keys present)

All existing tests continue to pass.

Part of #482

---------

Co-authored-by: you <you@example.com>
2026-04-02 21:30:23 -07:00