dandri/meshcore-bot
1
0
Fork 0
mirror of https://github.com/agessaman/meshcore-bot.git synced 2026-08-22 18:40:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(web-viewer): derive neighbours and hop distance from path evidence

Browse Source 
307 direct neighbours was not plausible, and it was not real.

complete_contact_tracking.hop_count claims 800 zero-hop contacts. Only
68 of them have any one-hop path in observed_paths to corroborate that.
Their stored SNR piles up in a 1.5 dB band — 655 of 800 between 11.25
and 12.75 dB — and their RSSI clusters at -39..-48 dBm. Hundreds of
radios at different distances and terrain cannot land in a 10 dB window.
That is the signature of one strong local link being recorded against
every node whose traffic happened to arrive through it. Their return
paths agree: these "direct" contacts have out_path_len of 3 to 11.

The writer's intent is sound — repeater_manager only stores RSSI/SNR
when signal_info reports hops == 0 — so the field being fed to it does
not mean what the surrounding code assumes. Left as is; this change
stops the dashboard depending on it.

Neighbour membership now comes from path evidence: an advert whose
path_length equals its bytes_per_hop travelled exactly one hop. That
yields 38 nodes in 24h and 124 in 7d, with a plausible spread. Signal is
shown only where the path evidence and the stored hop count agree, which
is 5 and 12 nodes respectively; the rest read "no signal reading" rather
than borrowing a measurement taken on somebody else's link. A 24h/7d
selector bounds the window, capped well under observed_paths' 90-day
retention because a month-old link says nothing about today.

Separately, this fixes a bug I introduced. path_length is a BYTE count,
and with 2- or 3-byte hop encoding a 3-hop path is 6 or 9 bytes long. The
path-length histogram plotted that raw value on an axis readers would
take as hops, overstating distance two- to threefold on a mesh that is
~95% multibyte. It is replaced by a single hops-away chart computed as
path_length / bytes_per_hop, which also retires the histogram built on
the untrustworthy stored hop count. The result is unimodal, peaking at 3
hops and decaying — the shape a mesh should have, and not the bimodal
one the old chart drew.
This commit is contained in:
agessaman
2026-07-29 22:23:43 -07:00
parent c94a3f861e
commit f68da5287c
7 changed files with 370 additions and 363 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+17 -7
View File
@@ -23,13 +23,23 @@ semantic versioning.
- New dashboard tiles and charts: routing mix (flood vs direct), hop-count and
path-length histograms, a 30-day multibyte adoption trend, busiest repeaters,
and a role mix.
- Direct-neighbour signal panel: the nodes heard with no repeater in between,
their SNR distribution, and the weakest links named. Scoped to `hop_count = 0`
because that is the only population where SNR means anything — a relayed
packet's SNR measures the last hop into this radio, not the link to whoever
sent it, so a network-wide average describes nothing in particular.
`complete_contact_tracking` records signal for exactly the zero-hop contacts,
so this is a complete census of the neighbours rather than a sample.
- One-hop neighbours panel, with a 24-hour / 7-day selector: the nodes whose
advert reached this radio in a single hop, weakest measured link first.
Membership is derived from observed path evidence rather than from
`complete_contact_tracking.hop_count`, which is not trustworthy — it claims
800 zero-hop contacts on the live database while only 68 have any one-hop
path to corroborate it, their stored SNR piles up in a 1.5 dB band (655 of
800 between 11.25 and 12.75 dB), and their RSSI clusters near -45 dBm. That
is the signature of one strong local link being recorded against every node
whose traffic arrived through it, not of hundreds of separate radios. SNR is
therefore shown only where the path evidence and the stored hop count agree;
the rest read "no signal reading" rather than borrowing another link's
measurement.
- Hop-distance chart derived as `path_length / bytes_per_hop`. `path_length` is
a byte count, so with 2- or 3-byte hop encoding — about 95% of adverts on the
live mesh — reading it directly as a hop count overstates distance two- to
threefold. This replaces both the raw path-length chart and the chart built
on the untrustworthy stored hop count.
- New `[Web_Viewer]` settings: `dashboard_snapshot_enabled`,
`dashboard_snapshot_interval_seconds`, `dashboard_snapshot_history_days`, and
`dashboard_packet_backfill_rows`.
docs/web-viewer.md
+19 -9
View File
@@ -141,17 +141,27 @@ proxy_set_header X-Forwarded-Proto $scheme;
and snapshot age with a manual refresh control
- **Mesh**: nodes heard, adverts, new nodes, nodes gone quiet, and geographic
coverage — the count tiles carry a 30-day sparkline and a change chip
- Routing mix (flood vs direct), hop-count and path-length histograms, and the
role mix
- Routing mix (flood vs direct), a hop-distance chart, and the role mix
- Path encoding: multibyte share among contacts and among incoming packets,
plus a 30-day multibyte adoption trend
- Busiest repeaters, and **direct neighbours** — the nodes heard with no
repeater in between, their SNR distribution, and the weakest links named.
Restricted to `hop_count = 0` deliberately: a relayed packet's SNR measures
the last hop into this radio, not the link to whoever originated it, so
mixing hop counts together yields a number that describes nothing. The bot's
contact records store signal for exactly the zero-hop contacts, which makes
this a complete census of the neighbours rather than a sample.
- Busiest repeaters, and **one-hop neighbours** (24-hour or 7-day window)
Two measurement notes for the mesh charts:
**Hops are derived, not read.** `observed_paths.path_length` is a *byte* count,
and with 2- or 3-byte hop encoding a three-hop path is six or nine bytes long.
The dashboard divides by `bytes_per_hop`; charting the raw value would overstate
distance two- to threefold on a mesh that is ~95% multibyte.
**Neighbour signal is reported only where two sources agree.**
`complete_contact_tracking.hop_count` is not a reliable direct-neighbour marker:
on a representative database it claims 800 zero-hop contacts while only 68 have
any one-hop path to corroborate it, and the SNR stored against them clusters in
a ~1.5 dB band with RSSI near -45 dBm — one strong local link recorded against
every node whose traffic arrived through it. Neighbour membership therefore
comes from path evidence, and SNR/RSSI appear only when the stored hop count
agrees; otherwise the row reads "no signal reading". A relayed packet's SNR
measures the last hop into this radio, never the link to whoever sent it.
- **Bot**: messages, commands, reply rate, and unique users, plus the top
commands/users/channels and longest paths
- Live activity feed
modules/web_viewer/dashboard_stats.py
+120 -107
View File
@@ -33,7 +33,6 @@ Two conventions matter throughout:
from __future__ import annotations
import json
import math
import os
import socket
import sqlite3
@@ -85,16 +84,21 @@ SERIES_METRICS: dict[str, str] = {
SUMMARY_SERIES_POINTS = 30
# Direct-neighbour signal panel: how many of the weakest links to list, and the
# width of the SNR histogram buckets in dB.
WEAKEST_NEIGHBOURS = 8
SNR_BIN_DB = 2
# Metrics that are already a ratio: a period "total" has to be the mean of the
# daily values, not their sum — adding percentages together means nothing.
RATIO_METRICS = frozenset({"multibyte_share"})
TOP_KINDS = ("users", "commands", "channels", "paths", "repeaters")
TOP_KINDS = ("users", "commands", "channels", "paths", "repeaters", "neighbors")
# Neighbour windows are capped well below observed_paths' 90-day retention: a
# link last exercised a month ago says nothing about whether it works today.
NEIGHBOR_WINDOWS = ("24h", "7d")
# A path is one hop when its byte length equals the per-hop encoding width.
# path_length is measured in BYTES, and with 2- or 3-byte hop encoding a 3-hop
# path is 6 or 9 bytes long — reading the raw value as a hop count inflates
# every multibyte path by 2-3x.
ONE_HOP_PATH = "op.bytes_per_hop > 0 AND op.path_length = op.bytes_per_hop"
# Roles the firmware reports as an unmapped enum ordinal. They are real
# contacts, so they belong in the mix — just not as sixteen singleton slices.
@@ -124,22 +128,6 @@ def _table_exists(conn: sqlite3.Connection, table: str) -> bool:
return row is not None
def _percentile(sorted_values: list[float], fraction: float) -> float | None:
"""Linear-interpolated percentile over a pre-sorted list."""
if not sorted_values:
return None
if len(sorted_values) == 1:
return round(float(sorted_values[0]), 1)
position = fraction * (len(sorted_values) - 1)
low = math.floor(position)
high = math.ceil(position)
if low == high:
return round(float(sorted_values[low]), 1)
weight = position - low
value = sorted_values[low] * (1 - weight) + sorted_values[high] * weight
return round(float(value), 1)
def _change_pct(current: float | None, previous: float | None) -> float | None:
"""Percent change, or None when the baseline cannot support one.
@@ -727,32 +715,14 @@ class DashboardStatsService:
mesh["states"] = row[1] or 0
mesh["cities"] = row[2] or 0
mesh["hop_histogram"] = [
[int(hop), int(count)]
for hop, count in conn.execute(
"""
SELECT hop_count, COUNT(*) FROM complete_contact_tracking
WHERE hop_count IS NOT NULL AND hop_count BETWEEN 0 AND 32
GROUP BY hop_count ORDER BY hop_count
"""
)
]
mesh["role_mix"] = self._mix(conn, "role")
mesh["neighbors"] = self._direct_neighbor_signal(conn)
if sources & SOURCE_OBSERVED_PATHS:
mesh["path_len_histogram"] = [
[int(length), int(count)]
for length, count in conn.execute(
"""
SELECT path_length, COUNT(*) FROM observed_paths
WHERE packet_type = 'advert' AND path_length IS NOT NULL
AND path_length BETWEEN 0 AND 32
AND last_seen >= datetime('now','localtime','-7 days')
GROUP BY path_length ORDER BY path_length
"""
)
]
mesh["hops_histogram"] = self._hops_histogram(conn)
mesh["neighbors"] = {
window: self._count_one_hop_nodes(conn, window)
for window in NEIGHBOR_WINDOWS
}
if sources & SOURCE_PACKET_STREAM:
row = conn.execute(
@@ -795,74 +765,100 @@ class DashboardStatsService:
buckets[key] = buckets.get(key, 0) + (count or 0)
return [[name, count] for name, count in sorted(buckets.items(), key=lambda kv: -kv[1])]
def _direct_neighbor_signal(self, conn: sqlite3.Connection) -> dict[str, Any]:
"""Per-neighbour signal for nodes heard directly, weakest link first.
def _hops_histogram(self, conn: sqlite3.Connection) -> list[list[int]]:
"""Nodes by how many hops away their closest observed advert path was.
Scoped to ``hop_count = 0`` because that is the only population where
SNR means anything: a relayed packet's SNR describes the last hop into
this radio, not the link to the node that originated it. Averaging
those together produces a number that describes nothing in particular.
This is also why the data is complete rather than sampled —
complete_contact_tracking records snr and signal_strength for exactly
the zero-hop contacts and leaves them NULL for everything further away.
Recency-scoped as well: a neighbour last heard two months ago says
nothing about the link today.
Derived from ``path_length / bytes_per_hop`` rather than read from
``complete_contact_tracking.hop_count``. Two reasons: path_length is a
byte count, so with multibyte encoding the raw value overstates hops by
2-3x; and the stored hop_count disagrees with the path evidence badly
enough at the low end to be unusable (see _one_hop_rows).
"""
window_days = 7
rows = conn.execute(
"""
SELECT public_key, name, role, snr, signal_strength, hop_count, last_heard
FROM complete_contact_tracking
WHERE hop_count = 0 AND snr IS NOT NULL
AND last_heard > datetime('now', 'localtime', ?)
ORDER BY snr ASC
""",
(f"-{window_days} days",),
SELECT MIN(path_length / bytes_per_hop) AS hops
FROM observed_paths
WHERE packet_type = 'advert' AND bytes_per_hop > 0
AND last_seen >= datetime('now','localtime','-7 days')
GROUP BY public_key
"""
).fetchall()
counts: dict[int, int] = {}
for row in rows:
hops = row["hops"]
if hops is None or not 0 <= hops <= 32:
continue
counts[int(hops)] = counts.get(int(hops), 0) + 1
if not counts:
return []
return [[hop, counts.get(hop, 0)] for hop in range(min(counts), max(counts) + 1)]
values = sorted(float(row["snr"]) for row in rows)
rssi_values = sorted(
float(row["signal_strength"]) for row in rows if row["signal_strength"] is not None
)
return {
"count": len(rows),
"window_days": window_days,
"snr": {
"min": round(values[0], 1) if values else None,
"p50": _percentile(values, 0.50),
"max": round(values[-1], 1) if values else None,
},
"rssi": {"p50": _percentile(rssi_values, 0.50)},
"histogram": self._snr_histogram(values),
"weakest": [
{
"name": row["name"] or (row["public_key"] or "")[:12],
"role": normalize_role(row["role"]),
"snr": round(float(row["snr"]), 1),
"rssi": (
round(float(row["signal_strength"]), 0)
if row["signal_strength"] is not None
else None
),
"last_heard": row["last_heard"],
}
for row in rows[:WEAKEST_NEIGHBOURS]
],
}
def _count_one_hop_nodes(self, conn: sqlite3.Connection, window: str) -> int:
return conn.execute(
f"""
SELECT COUNT(DISTINCT op.public_key) FROM observed_paths op
WHERE op.packet_type = 'advert' AND {ONE_HOP_PATH}
AND op.last_seen > datetime('now','localtime', ?)
""", # noqa: S608 - ONE_HOP_PATH is a module constant, not input
(self._window_offset(window),),
).fetchone()[0] or 0
@staticmethod
def _snr_histogram(sorted_values: list[float]) -> list[list[Any]]:
"""Bin neighbour SNR into fixed dB buckets so the shape is comparable over time."""
if not sorted_values:
return []
bins: dict[int, int] = {}
for value in sorted_values:
edge = int(math.floor(value / SNR_BIN_DB) * SNR_BIN_DB)
bins[edge] = bins.get(edge, 0) + 1
low, high = min(bins), max(bins)
return [[edge, bins.get(edge, 0)] for edge in range(low, high + SNR_BIN_DB, SNR_BIN_DB)]
def _window_offset(window: str) -> str:
return {"24h": "-1 days", "7d": "-7 days", "30d": "-30 days"}.get(window, "-7 days")
def _one_hop_rows(self, conn: sqlite3.Connection, window: str, limit: int) -> list[dict]:
"""Nodes whose advert reached this radio in a single hop, weakest first.
Membership comes from path evidence, not from
``complete_contact_tracking.hop_count``. That column claims 800 zero-hop
contacts, but only 68 of them have any one-hop path to corroborate it,
their stored SNR piles up in a 1.5 dB band (655 of 800 between 11.25 and
12.75), and their RSSI clusters around -45 dBm — the signature of one
strong local link being recorded against every node whose traffic came
through it, not of hundreds of separate radios.
Signal is therefore reported only where the path evidence and the stored
hop_count agree; everything else lists as unknown rather than being given
a number that belongs to somebody else's link.
"""
rows = conn.execute(
f"""
SELECT op.public_key,
MAX(op.last_seen) AS last_seen,
c.name, c.role, c.snr, c.signal_strength, c.hop_count
FROM observed_paths op
LEFT JOIN complete_contact_tracking c ON c.public_key = op.public_key
WHERE op.packet_type = 'advert' AND {ONE_HOP_PATH}
AND op.last_seen > datetime('now','localtime', ?)
GROUP BY op.public_key
""", # noqa: S608 - ONE_HOP_PATH is a module constant, not input
(self._window_offset(window),),
).fetchall()
measured: list[dict[str, Any]] = []
unmeasured: list[dict[str, Any]] = []
for row in rows:
corroborated = row["hop_count"] == 0 and row["snr"] is not None
item = {
"name": row["name"] or (row["public_key"] or "")[:12],
"public_key": row["public_key"],
"role": normalize_role(row["role"]),
"snr": round(float(row["snr"]), 1) if corroborated else None,
"rssi": (
round(float(row["signal_strength"]))
if corroborated and row["signal_strength"] is not None
else None
),
"signal_corroborated": corroborated,
"last_seen": row["last_seen"],
}
(measured if corroborated else unmeasured).append(item)
# Weakest measured links first — those are the ones worth acting on.
measured.sort(key=lambda item: item["snr"])
unmeasured.sort(key=lambda item: item["last_seen"] or "", reverse=True)
return (measured + unmeasured)[:limit]
def _bot_snapshot(self, conn: sqlite3.Connection, sources: int) -> dict[str, Any]:
"""Rolling 24-hour bot activity (distinct from the calendar-day rollup)."""
@@ -1200,7 +1196,16 @@ class DashboardStatsService:
retention = self.stats_retention_days
items: list[dict[str, Any]] = []
if kind == "repeaters":
total: int | None = None
if kind == "neighbors":
if window not in NEIGHBOR_WINDOWS:
window = NEIGHBOR_WINDOWS[0]
retention = self.adverts_retention_days
if _table_exists(conn, "observed_paths"):
items = self._one_hop_rows(conn, window, limit)
total = self._count_one_hop_nodes(conn, window)
elif kind == "repeaters":
retention = self.adverts_retention_days
days = {"24h": 1, "7d": 7, "30d": 30, "90d": 90}.get(window, retention)
since = (datetime.now() - timedelta(days=days - 1)).strftime("%Y-%m-%d")
@@ -1288,6 +1293,7 @@ class DashboardStatsService:
"truncated_by_retention": bool(
requested_days is None or requested_days > retention
),
"total": total if total is not None else len(items),
"items": items,
}
@@ -1332,5 +1338,12 @@ class DashboardStatsService:
"channels": stats_options,
"paths": stats_options,
"repeaters": options(self.adverts_retention_days),
# Deliberately not retention-derived: observed_paths keeps 90
# days, but a link last used a month ago tells you nothing about
# whether it works now.
"neighbors": [
{"value": value, "label": self._window_label(value, self.adverts_retention_days)}
for value in NEIGHBOR_WINDOWS
],
},
}
modules/web_viewer/static/css/dashboard.css
+12 -23
View File
@@ -287,6 +287,18 @@
border-radius: 5px;
}
/* Hatched, not empty: "we have no reading" reads differently from "zero". */
.neighbor-row__track--unknown {
background-image: repeating-linear-gradient(
45deg,
var(--bg-tertiary),
var(--bg-tertiary) 3px,
transparent 3px,
transparent 6px
);
opacity: 0.6;
}
.neighbor-row__values {
color: var(--text-muted);
font-variant-numeric: tabular-nums;
@@ -298,29 +310,6 @@
font-weight: 600;
}
/* ── Signal percentile readout ─────────────────────────────────────────── */
.signal-grid {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 0.5rem;
text-align: center;
}
.signal-grid__value {
font-size: 1.1rem;
font-weight: 700;
color: var(--text-color);
font-variant-numeric: tabular-nums;
}
.signal-grid__label {
font-size: 0.68rem;
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--text-muted);
}
/* ── Leaderboard rows ──────────────────────────────────────────────────── */
.top-list-row {
modules/web_viewer/static/js/dashboard.js
+67 -112
View File
@@ -316,6 +316,7 @@
);
if (!response.ok) throw new Error('HTTP ' + response.status);
const data = await response.json();
if (config.onLoad) config.onLoad(data);
if (!data.items || data.items.length === 0) {
container.replaceChildren(
makeTextElement('div', config.empty, 'dashboard-empty')
@@ -365,9 +366,7 @@
// Routing, encoding, distributions
this.renderRouteMix(mesh.route_mix, coverage.packets_window_label);
this.renderHistogram('hopHistogramChart', mesh.hop_histogram, 'Contacts', COLOR.accent);
this.renderHistogram('pathLenHistogramChart', mesh.path_len_histogram, 'Advert paths', COLOR.flood);
this.renderNeighbors(mesh.neighbors);
this.renderHopsHistogram(mesh.hops_histogram);
this.renderDoughnut('contactsEncodingChart', 'contacts-encoding-summary',
(mesh.encoding || {}).contacts_7d, 'contacts');
this.renderDoughnut('packetsEncodingChart', 'packets-encoding-summary',
@@ -501,85 +500,11 @@
);
}
renderHistogram(canvasId, points, label, color) {
renderHopsHistogram(points) {
const canvasId = 'hopsHistogramChart';
const canvas = el(canvasId);
if (!canvas || typeof Chart === 'undefined') return;
const list = Array.isArray(points) ? points : [];
const colors = themeColors();
if (this.charts[canvasId]) {
this.charts[canvasId].destroy();
delete this.charts[canvasId];
}
if (list.length === 0) return;
this.charts[canvasId] = new Chart(canvas.getContext('2d'), {
type: 'bar',
data: {
labels: list.map((p) => p[0]),
datasets: [{
label: label,
data: list.map((p) => p[1]),
backgroundColor: color,
borderRadius: 3,
}],
},
options: noAnimation({
responsive: true,
maintainAspectRatio: false,
plugins: { legend: { display: false } },
scales: {
x: {
ticks: { color: colors.muted, font: { size: 10 } },
grid: { display: false },
},
y: {
beginAtZero: true,
ticks: { color: colors.muted, font: { size: 10 }, precision: 0 },
grid: { color: colors.grid },
},
},
}),
});
}
/**
* Direct (zero-hop) neighbours: the distribution plus the weakest links.
* Only these have a meaningful SNR — a relayed packet's SNR describes
* the last hop, not the node that sent it.
*/
renderNeighbors(neighbors) {
const data = neighbors || {};
const snr = data.snr || {};
const count = data.count || 0;
setText('neighbors-count', formatNumber(count));
setText('neighbors-window', 'heard in the last ' + (data.window_days || 7) + 'd');
setText('neighbors-snr-min', formatSigned(snr.min));
setText('neighbors-snr-p50', formatSigned(snr.p50));
setText('neighbors-snr-max', formatSigned(snr.max));
setText('neighbors-rssi', formatSigned((data.rssi || {}).p50));
this.renderNeighborHistogram(data.histogram);
const list = el('neighbors-weakest');
if (!list) return;
const weakest = Array.isArray(data.weakest) ? data.weakest : [];
if (weakest.length === 0) {
list.replaceChildren(
makeTextElement('div', 'No direct neighbours heard recently.', 'dashboard-empty')
);
return;
}
list.replaceChildren();
weakest.forEach((item) => list.appendChild(neighborRow(item)));
}
renderNeighborHistogram(histogram) {
const canvasId = 'neighborSnrChart';
const canvas = el(canvasId);
if (!canvas || typeof Chart === 'undefined') return;
const bins = Array.isArray(histogram) ? histogram : [];
const bins = Array.isArray(points) ? points : [];
const colors = themeColors();
if (this.charts[canvasId]) {
@@ -593,9 +518,9 @@
data: {
labels: bins.map((b) => b[0]),
datasets: [{
label: 'Neighbours',
label: 'Nodes',
data: bins.map((b) => b[1]),
backgroundColor: bins.map((b) => snrColor(b[0])),
backgroundColor: COLOR.accent,
borderRadius: 3,
}],
},
@@ -606,17 +531,14 @@
legend: { display: false },
tooltip: {
callbacks: {
title: (items) => {
const low = Number(items[0].label);
return low + ' to ' + (low + 2) + ' dB SNR';
},
label: (ctx) => formatNumber(ctx.parsed.y) + ' neighbours',
title: (items) => items[0].label + ' hops away',
label: (ctx) => formatNumber(ctx.parsed.y) + ' nodes',
},
},
},
scales: {
x: {
title: { display: true, text: 'SNR (dB)', color: colors.muted,
title: { display: true, text: 'Hops away', color: colors.muted,
font: { size: 10 } },
ticks: { color: colors.muted, font: { size: 10 } },
grid: { display: false },
@@ -883,7 +805,13 @@
return '#198754';
}
/** One weakest-link row: name, SNR bar positioned on a fixed scale, values. */
/**
* One one-hop neighbour: name, SNR bar on a fixed scale, values.
*
* Signal is shown only where the stored hop count corroborates the path
* evidence. For the rest the row says "no signal reading" rather than
* borrowing a number measured on somebody else's link.
*/
function neighborRow(item) {
const row = document.createElement('div');
row.className = 'neighbor-row';
@@ -891,27 +819,35 @@
const name = makeTextElement('div', item.name, 'neighbor-row__name');
name.title = String(item.name ?? '') + ' (' + String(item.role ?? '') + ')';
// Fixed -12..+14 dB scale so bars are comparable between refreshes
// rather than rescaling to whatever the current worst link happens to be.
const track = document.createElement('div');
track.className = 'neighbor-row__track';
const fill = document.createElement('div');
fill.className = 'neighbor-row__fill';
const ratio = Math.max(0, Math.min(1, ((Number(item.snr) + 12) / 26)));
fill.style.width = (ratio * 100).toFixed(1) + '%';
fill.style.backgroundColor = snrColor(item.snr);
track.appendChild(fill);
const values = document.createElement('div');
values.className = 'neighbor-row__values';
values.append(
makeTextElement('span', formatSigned(item.snr) + ' dB', 'neighbor-row__snr'),
makeTextElement('span',
item.rssi === null || item.rssi === undefined
? ''
: ' / ' + Math.round(item.rssi) + ' dBm',
'neighbor-row__rssi')
);
if (item.signal_corroborated) {
// Fixed -12..+14 dB scale so bars are comparable between refreshes
// rather than rescaling to whatever the current worst link happens to be.
const fill = document.createElement('div');
fill.className = 'neighbor-row__fill';
fill.style.width = (Math.max(0, Math.min(1, (Number(item.snr) + 12) / 26)) * 100)
.toFixed(1) + '%';
fill.style.backgroundColor = snrColor(item.snr);
track.appendChild(fill);
values.append(
makeTextElement('span', formatSigned(item.snr) + ' dB', 'neighbor-row__snr'),
makeTextElement('span',
item.rssi === null || item.rssi === undefined
? ''
: ' / ' + Math.round(item.rssi) + ' dBm',
'neighbor-row__rssi')
);
} else {
track.classList.add('neighbor-row__track--unknown');
const unknown = makeTextElement('span', 'no signal reading', 'neighbor-row__rssi');
unknown.title = 'Heard one hop away, but no corroborated direct-reception '
+ 'measurement is stored for this node.';
values.appendChild(unknown);
}
row.append(name, track, values);
return row;
@@ -987,6 +923,19 @@
empty: 'No repeater adverts recorded.',
render: (item, i) => rankedRow(i + 1, item.name, formatNumber(item.count) + ' adverts', 'bg-success'),
},
neighbors: {
select: 'neighbors-window',
container: 'neighbors-list',
limit: 10,
empty: 'No one-hop adverts heard in this window.',
render: (item) => neighborRow(item),
onLoad: (data) => {
setText('neighbors-count', formatNumber(data.total));
const shown = Math.min(data.total || 0, (data.items || []).length);
setText('neighbors-shown',
data.total > shown ? 'showing ' + shown + ' of ' + formatNumber(data.total) : '');
},
},
};
// ── live activity feed ───────────────────────────────────────────────
@@ -1130,12 +1079,18 @@
'Change compares the last complete calendar day against the day before it — ' +
'not a rolling 24 hours. The headline number above it is a rolling 24-hour count.',
'neighbors-info':
'Nodes heard directly, with no repeater in between, in the last 7 days. ' +
'Only these have a meaningful SNR: a relayed packet\'s SNR measures the last ' +
'hop into this radio, not the link to whoever sent it, so averaging across all ' +
'hop counts describes nothing in particular. Bars use a fixed -12 to +14 dB ' +
'scale; below 0 dB the signal is under the noise floor and only the spreading ' +
'factor is recovering it.',
'Nodes whose advert reached this radio in a single hop, newest evidence first, ' +
'with the weakest measured links promoted to the top. Membership comes from the ' +
'observed path length divided by its bytes-per-hop encoding, not from the stored ' +
'hop count — that field claims far more direct neighbours than the path evidence ' +
'supports. SNR is shown only where both agree; a relayed packet\'s SNR measures ' +
'the last hop into this radio rather than the link to whoever sent it, so the ' +
'rest are left blank instead of borrowing another link\'s number.',
'hops-info':
'Nodes by the fewest hops any of their adverts took to reach this radio, over the ' +
'last 7 days. Hops are derived as path length divided by bytes per hop: path ' +
'length is a byte count, so with 2- or 3-byte hop encoding the raw value would ' +
'overstate the distance by two to three times.',
};
function initTooltips() {
modules/web_viewer/templates/index.html
+28 -39
View File
@@ -121,21 +121,25 @@
</p>
<div class="stacked-bar" id="route-mix-bar"></div>
<div class="stacked-bar__legend" id="route-mix-legend"></div>
<hr class="my-3">
<p class="dashboard-note mb-2">Hop count across known contacts</p>
<div class="chart-box" style="height:150px">
<canvas id="hopHistogramChart" aria-label="Contacts by hop count"></canvas>
</div>
</div>
</div>
</div>
<div class="col-12 col-lg-4">
<div class="card h-100">
<div class="card-header"><i class="fas fa-ruler-horizontal"></i> Path length</div>
<div class="card-header d-flex justify-content-between align-items-center">
<span><i class="fas fa-ruler-horizontal"></i> Hops away</span>
<button type="button" class="dashboard-info-btn" id="hops-info"
aria-label="How hop distance is derived"
data-bs-toggle="tooltip" data-bs-placement="top">
<i class="fas fa-info-circle" aria-hidden="true"></i>
</button>
</div>
<div class="card-body">
<p class="dashboard-note mb-2">Advert paths observed in the last 7 days</p>
<p class="dashboard-note mb-2">
Nodes by their closest advert path, last 7 days
</p>
<div class="chart-box chart-box--tall">
<canvas id="pathLenHistogramChart" aria-label="Advert paths by path length"></canvas>
<canvas id="hopsHistogramChart" aria-label="Nodes by hop distance"></canvas>
</div>
</div>
</div>
@@ -228,41 +232,26 @@
<div class="col-12 col-lg-6">
<div class="card h-100">
<div class="card-header d-flex justify-content-between align-items-center">
<span><i class="fas fa-signal"></i> Direct neighbours &amp; signal</span>
<button type="button" class="dashboard-info-btn" id="neighbors-info"
aria-label="What counts as a direct neighbour"
data-bs-toggle="tooltip" data-bs-placement="top">
<i class="fas fa-info-circle" aria-hidden="true"></i>
</button>
<span>
<i class="fas fa-signal"></i> One-hop neighbours
<button type="button" class="dashboard-info-btn" id="neighbors-info"
aria-label="What counts as a one-hop neighbour"
data-bs-toggle="tooltip" data-bs-placement="top">
<i class="fas fa-info-circle" aria-hidden="true"></i>
</button>
</span>
<select id="neighbors-window" class="form-select form-select-sm"
style="width:auto;font-size:0.75rem" aria-label="Neighbour time window">
<option value="24h">Last 24 hours</option>
</select>
</div>
<div class="card-body">
<div class="d-flex align-items-baseline gap-2 flex-wrap mb-2">
<div class="d-flex align-items-baseline gap-2 flex-wrap mb-1">
<span class="stat-tile__value" id="neighbors-count">—</span>
<span class="dashboard-note">nodes heard with no repeater in between,
<span id="neighbors-window">—</span></span>
<span class="dashboard-note">nodes reached this radio in a single hop</span>
</div>
<div class="signal-grid mb-3">
<div>
<div class="signal-grid__value" id="neighbors-snr-min">—</div>
<div class="signal-grid__label">worst SNR</div>
</div>
<div>
<div class="signal-grid__value" id="neighbors-snr-p50">—</div>
<div class="signal-grid__label">median SNR</div>
</div>
<div>
<div class="signal-grid__value" id="neighbors-snr-max">—</div>
<div class="signal-grid__label">best SNR</div>
</div>
</div>
<div class="chart-box" style="height:130px">
<canvas id="neighborSnrChart"
aria-label="Direct neighbours by signal-to-noise ratio"></canvas>
</div>
<p class="dashboard-note mt-3 mb-2">
Weakest links (median RSSI <span id="neighbors-rssi">—</span> dBm)
</p>
<div id="neighbors-weakest"><div class="dashboard-empty">Loading…</div></div>
<p class="dashboard-note mb-3" id="neighbors-shown"></p>
<div id="neighbors-list"><div class="dashboard-empty">Loading…</div></div>
</div>
</div>
</div>
tests/test_dashboard_stats.py
+107 -66
View File
@@ -587,84 +587,125 @@ class TestDerivedWindows:
assert windows["windows"]["users"][-1]["label"] == f"All retained ({retention}d)"
class TestDirectNeighbourSignal:
"""SNR is only meaningful for nodes heard with no repeater in between."""
class TestOneHopNeighbours:
"""Neighbour membership comes from path evidence, not the stored hop_count.
def _seed_neighbours(self, viewer, rows):
with sqlite3.connect(viewer.db_path) as conn:
conn.executemany(
"""
INSERT INTO complete_contact_tracking
(public_key, name, role, hop_count, snr, signal_strength, last_heard)
VALUES (?, ?, ?, ?, ?, ?, datetime('now','localtime', ?))
""",
rows,
)
On the live database hop_count claims 800 zero-hop contacts while only 68
have any one-hop path to corroborate it, and their stored SNR piles up in a
1.5 dB band — one strong local link recorded against every node whose
traffic came through it.
"""
def _signal(self, viewer):
def _seed_contact(self, conn, pk, name, role, hop_count, snr, rssi):
conn.execute(
"""
INSERT INTO complete_contact_tracking
(public_key, name, role, hop_count, snr, signal_strength, last_heard)
VALUES (?, ?, ?, ?, ?, ?, datetime('now','localtime','-1 hours'))
""",
(pk, name, role, hop_count, snr, rssi),
)
def _seed_path(self, conn, pk, path_length, bytes_per_hop, age="-1 hours"):
conn.execute(
"""
INSERT INTO observed_paths
(public_key, from_prefix, to_prefix, path_hex, path_length,
bytes_per_hop, packet_type, last_seen)
VALUES (?, 'aa', 'bb', ?, ?, ?, 'advert', datetime('now','localtime', ?))
""",
(pk, "ab" * path_length, path_length, bytes_per_hop, age),
)
def _top(self, viewer, window="24h", limit=10):
with closing(viewer._dashboard_connection()) as conn:
return viewer.dashboard_stats._direct_neighbor_signal(conn)
return viewer.dashboard_stats.read_top(conn, "neighbors", window, limit)
def test_relayed_contacts_are_excluded(self, viewer):
"""A relayed packet's SNR describes the last hop, not the sender."""
self._seed_neighbours(viewer, [
(_pk(1), "direct", "repeater", 0, 5.0, -70.0, "-1 hours"),
(_pk(2), "one-hop", "repeater", 1, -9.0, -110.0, "-1 hours"),
(_pk(3), "far", "companion", 5, 12.0, -50.0, "-1 hours"),
])
signal = self._signal(viewer)
assert signal["count"] == 1
assert [n["name"] for n in signal["weakest"]] == ["direct"]
assert signal["snr"]["p50"] == 5.0
def test_multibyte_paths_are_not_read_as_extra_hops(self, viewer):
"""path_length is bytes: 3 bytes at 3 bytes/hop is ONE hop, not three."""
with sqlite3.connect(viewer.db_path) as conn:
self._seed_contact(conn, _pk(1), "onebyte-1hop", "repeater", 0, 5.0, -70.0)
self._seed_path(conn, _pk(1), path_length=1, bytes_per_hop=1)
self._seed_contact(conn, _pk(2), "multibyte-1hop", "repeater", 0, 6.0, -60.0)
self._seed_path(conn, _pk(2), path_length=3, bytes_per_hop=3)
self._seed_contact(conn, _pk(3), "multibyte-2hop", "repeater", 0, 7.0, -50.0)
self._seed_path(conn, _pk(3), path_length=6, bytes_per_hop=3)
def test_weakest_links_come_first(self, viewer):
self._seed_neighbours(viewer, [
(_pk(i), f"n{i}", "repeater", 0, snr, -60.0 - i, "-2 hours")
for i, snr in enumerate([8.0, -9.5, 2.0, -3.0, 13.0])
])
signal = self._signal(viewer)
assert [n["snr"] for n in signal["weakest"]] == [-9.5, -3.0, 2.0, 8.0, 13.0]
assert signal["snr"]["min"] == -9.5
assert signal["snr"]["max"] == 13.0
names = {item["name"] for item in self._top(viewer)["items"]}
assert names == {"onebyte-1hop", "multibyte-1hop"}
def test_stale_neighbours_are_excluded(self, viewer):
"""A link not exercised in a week says nothing about the link today."""
self._seed_neighbours(viewer, [
(_pk(1), "recent", "repeater", 0, 4.0, -80.0, "-2 days"),
(_pk(2), "stale", "repeater", 0, -11.0, -115.0, "-40 days"),
])
signal = self._signal(viewer)
assert signal["count"] == 1
assert [n["name"] for n in signal["weakest"]] == ["recent"]
def test_uncorroborated_signal_is_withheld(self, viewer):
"""Path evidence without a matching hop_count gets no SNR figure."""
with sqlite3.connect(viewer.db_path) as conn:
self._seed_contact(conn, _pk(1), "agrees", "repeater", 0, 4.0, -80.0)
self._seed_path(conn, _pk(1), 1, 1)
# hop_count says 4 hops but a one-hop path exists: the stored signal
# belongs to some other link, so it must not be shown.
self._seed_contact(conn, _pk(2), "disagrees", "repeater", 4, 12.0, -45.0)
self._seed_path(conn, _pk(2), 1, 1)
def test_histogram_bins_are_fixed_width_and_contiguous(self, viewer):
self._seed_neighbours(viewer, [
(_pk(i), f"n{i}", "repeater", 0, snr, -70.0, "-1 hours")
for i, snr in enumerate([-5.0, -4.5, 0.5, 5.0])
])
histogram = self._signal(viewer)["histogram"]
edges = [b[0] for b in histogram]
assert edges == list(range(-6, 6, 2)), histogram
assert sum(b[1] for b in histogram) == 4
# An empty bucket in the middle is present with a zero, not skipped.
assert dict(histogram)[-2] == 0
items = {item["name"]: item for item in self._top(viewer)["items"]}
assert items["agrees"]["signal_corroborated"] is True
assert items["agrees"]["snr"] == 4.0
assert items["disagrees"]["signal_corroborated"] is False
assert items["disagrees"]["snr"] is None
assert items["disagrees"]["rssi"] is None
def test_no_neighbours_degrades_cleanly(self, viewer):
signal = self._signal(viewer)
assert signal["count"] == 0
assert signal["weakest"] == []
assert signal["histogram"] == []
assert signal["snr"]["p50"] is None
def test_contacts_with_no_one_hop_path_are_excluded(self, viewer):
"""hop_count = 0 alone does not make something a neighbour."""
with sqlite3.connect(viewer.db_path) as conn:
self._seed_contact(conn, _pk(1), "claims-direct", "repeater", 0, 12.0, -45.0)
self._seed_path(conn, _pk(1), path_length=5, bytes_per_hop=1)
assert self._top(viewer)["items"] == []
assert self._top(viewer)["total"] == 0
def test_snapshot_exposes_neighbours_not_device_mix(self, viewer):
"""role and device_type are the same field twice — only role is charted."""
self._seed_neighbours(viewer, [(_pk(1), "n", "repeater", 0, 3.0, -75.0, "-1 hours")])
def test_weakest_measured_links_are_promoted(self, viewer):
with sqlite3.connect(viewer.db_path) as conn:
for i, snr in enumerate([9.0, -8.0, 2.0]):
self._seed_contact(conn, _pk(i), f"m{i}", "repeater", 0, snr, -70.0)
self._seed_path(conn, _pk(i), 1, 1)
self._seed_contact(conn, _pk(9), "unmeasured", "repeater", 3, None, None)
self._seed_path(conn, _pk(9), 1, 1)
items = self._top(viewer)["items"]
assert [i["name"] for i in items[:3]] == ["m1", "m2", "m0"]
assert items[-1]["name"] == "unmeasured"
def test_window_bounds_membership(self, viewer):
with sqlite3.connect(viewer.db_path) as conn:
self._seed_contact(conn, _pk(1), "today", "repeater", 0, 5.0, -70.0)
self._seed_path(conn, _pk(1), 1, 1, age="-2 hours")
self._seed_contact(conn, _pk(2), "last-week", "repeater", 0, 5.0, -70.0)
self._seed_path(conn, _pk(2), 1, 1, age="-4 days")
assert {i["name"] for i in self._top(viewer, "24h")["items"]} == {"today"}
assert {i["name"] for i in self._top(viewer, "7d")["items"]} == {"today", "last-week"}
def test_windows_are_capped_below_retention(self, viewer):
"""observed_paths keeps 90 days; a month-old link says nothing about today."""
options = viewer.dashboard_stats.derive_windows()["windows"]["neighbors"]
assert [o["value"] for o in options] == ["24h", "7d"]
def test_snapshot_reports_hops_not_path_bytes(self, viewer):
with sqlite3.connect(viewer.db_path) as conn:
self._seed_contact(conn, _pk(1), "near", "repeater", 0, 5.0, -70.0)
self._seed_path(conn, _pk(1), path_length=6, bytes_per_hop=3) # 2 hops
self._seed_contact(conn, _pk(2), "far", "repeater", 4, None, None)
self._seed_path(conn, _pk(2), path_length=4, bytes_per_hop=1) # 4 hops
_refresh(viewer)
with viewer.app.test_client() as client:
mesh = client.get("/api/dashboard/summary").get_json()["mesh"]
assert "device_mix" not in mesh
assert "role_mix" in mesh
assert mesh["neighbors"]["count"] == 1
assert "device_mix" not in mesh # same field as role
assert "hop_histogram" not in mesh # replaced by path-derived hops
assert "path_len_histogram" not in mesh # was byte length labelled as hops
assert dict(mesh["hops_histogram"]) == {2: 1, 3: 0, 4: 1}
assert mesh["neighbors"] == {"24h": 0, "7d": 0}
def test_no_neighbours_degrades_cleanly(self, viewer):
payload = self._top(viewer)
assert payload["items"] == []
assert payload["total"] == 0
class TestRoleBucketing: