- Real-time rows put SNR before the hop count. On a phone the end of the line is cut off, and SNR is the reading a quick check wants; it used to show as "SNR …".
- A repeat heard again only lights its "heard ×N" count instead of re-running the whole row's 1.6 s flash, so a busy tape no longer shimmers. In 20 s of live traffic: 26 count bumps, 8 flashes for genuinely new rows. Reduced motion turns both off.
- The health strip says "1 client", not "1 clients".
- American spelling throughout the user-visible text: "One-hop neighbors" and its tooltip and empty message.
- "Multibyte share of packets" in sentence case like the other card titles.
- The hops chart's y-axis reads "% of nodes or packets" instead of chart-library "% of series".
- The route bar's label rounds to one decimal like its legend, so it no longer shows 98% beside 98.4%.
- The note that bot statistics need collect_stats is configuration guidance, not a degraded state, so it drops the warning color.
The full suite passes (5878), and axe-core still finds nothing on any page in either theme.
The dashboard's payload mix colored its bars by rank, so a packet type's color changed whenever the traffic order did, and disagreed with the multibyte chart further down, which colors the same types by a fixed slot. The sixth rank was also #198754, the status green. Payload types now take their fixed slot from the chart's SERIES_PALETTE, per theme, so each type has one color everywhere on the page. The role mix gives each role a fixed color matching the Mesh page's node fills: repeaters Signal Blue, room servers Route Teal, then companion orange, sensor violet, bot pink, gateway cyan. Unknown and Other stay neutral, any other role takes a spare, and none of these colors is a lamp.
Real-time hashed channel names onto the whole hue wheel, so a channel could come out status green or alarm red; Public was green. Channel dots now draw from 20 colors per theme, ten hues kept clear of the lamp red, amber and green at two lightness steps, each holding 3:1 on the card (4.2:1 light, 5.2:1 dark). A channel starts at its hashed slot, so its color stays the same between visits. If another channel on the page already holds that slot it steps 7 slots on, a clearly different hue, so up to 20 channels never share a color; with a plain hash, two of three channels on the test mesh collided.
DESIGN.md's data-palette section describes both.
Real-time's receiver strip gains an RX LED beside the station state, borrowed from the radio's own front panel. It lights for 140 ms on each live reception, and a burst keeps it lit instead of strobing. It is dark when nothing is arriving, which honestly answers "is anything coming in?" on a quiet station. It uses the connected lamp's green because it means receiving, sits behind aria-hidden next to "Last heard", which carries the same information in words, and stays dark under prefers-reduced-motion. Replayed history doesn't light it; it's one class toggle per packet, with no polling.
The dashboard's New nodes tile names the newest arrival from the last day under its count, e.g. "newest: N7JMV-BEACON-RP · 7 hops · 1h ago", so the number has a neighbor behind it. Long names truncate so the tile keeps its height. The snapshot computes it with one indexed query alongside the other contact counts, so nothing new is polled.
On a fresh install the dashboard was about fifteen identical "No data." boxes and empty donut rings, which read as broken at exactly the moment an operator wants to know the new bot is working. Real-time said "Nothing heard yet" forever, with no sense of whether the silence was normal.
Until the snapshot has seen any node or packet, the dashboard now shows one panel in place of the grid:
- What the station is doing, from the same verdict as the strip: listening, starting, not running (start the bot), or the radio problem in the way.
- What fills in first: nodes from adverts, then paths and hop counts, then one-hop neighbors.
- That timing depends on the mesh.
- For admins, buttons to watch Real-time and check the radio settings; both pages are admin-only, so anonymous visitors get the panel without them.
The grid comes back on its own once anything is heard.
Real-time's empty tape now says "Listening since 2:28 PM (12 min) · nothing heard yet", counting each second from the bot's start time, which /api/banner-status now reports while the bot runs. After ten minutes with nothing, it adds one line of diagnosis: that can be normal on a quiet mesh, otherwise check the frequency and preset in Radio settings, or the Logs. When the bot isn't running or the radio can't receive, it says that instead.
The dashboard's health strip and Real-time's receiver strip both said all was well whenever the viewer itself answered. On a database the bot had never run against, the strip showed a green "Online · radio ok" beside "bot down?", and Real-time a green "Listening": /api/health reported "healthy" with no bot at all, and "radio ok" only meant the zombie flag was unset.
/api/banner-status, which every page already polls every 10 seconds, now also says whether the bot is running and whether its radio is connected. The stored start time survives the bot stopping, so "running" comes from the system_health heartbeat the bot rewrites every 30 seconds: older than two minutes, or missing, and the bot isn't there. radio_connected is only reported while it is, since it's the bot's last word on the link. The poll also runs once on load instead of waiting 10 seconds.
base.html turns that into one verdict for both pages: Bot starting, Bot not running (with when it was last seen), Radio not responding, Radio disconnected, Radio can't transmit, or Listening. The lamp colors follow DESIGN.md: red for not working, amber for degraded (sends failing while receiving may still work), green only when the radio is listening. Real-time says "Checking the bot…" until the first answer instead of assuming. Uptime shows a dash while the bot isn't running, in place of "bot down?".
The change chips compared the last two complete days even when one of them had recorded nothing, so a day the bot was down showed as a 100% collapse on every tile. When either day has no packets, adverts or messages, the chips stay neutral and the note under the tiles names the day: "No change shown: nothing was recorded on Oct 23." A fresh install, where both days are empty, gets the neutral chips without the note.
My fork of meshcore-decoder is now at 0.4.0, which brings it in line with current MeshCore firmware: encrypted PATH payloads, GRP_DATA and MULTIPART, region transport codes, the new request and text types, and the firmware's packet hash. 0.4.0 isn't published to npm, so build.sh now fetches the fork at a pinned commit (60733c0) and installs its runtime dependencies from the lockfile, still with install scripts disabled. Two builds produce byte-identical bundles.
The browser entry exports the same additions as the fork's lite entry (AnonRequestType, TextType and their name functions, PathPayloadDecoder) plus the region transport-code helpers.
The byte breakdown's hash fallback now uses the decoder's packetHash, which is the firmware hash the server sends. The old fallback hashed the hex text and never matched.
All 72,828 packets in the bot's database decode to the same payload type and path as the bot's own decoder, and packetHash matches the stored packet_hash on every one. Advert names and Public decryption are unchanged from 0.3.0. In the browser, Real-time loads without console errors, and 2-byte-hop packets, including a decrypted Public message, show the right header, path and payload in the byte breakdown.
- base.html adds shared readout colors (blue, teal, violet, orange), a row-action component (per-action color with a light outline, as on Contacts) and tinted tag chips
- Radio and Feeds stat tiles read in their own colors; Feeds' Active Errors turns red only when there are errors
- Greeter's 311 solid red "Mark Ungreeted" buttons are red-outline row actions, Manual/Rollout are tinted chips instead of lamp green and amber, and names are plain text instead of pink code
- Feeds' four solid row buttons use the row-action style
- Radio's writes are primary blue; amber stays only on the frequency/bandwidth write, which changes the channel for every node on it
- the dashboard's ranked lists (commands, users, channels, repeaters, paths) show counts as tinted chips in each list's own color with a muted rank, instead of lamp-colored pills
The web viewer's static/js/meshcore-browser.mjs was an esbuild bundle of @michaelhart/meshcore-decoder at 0.2.7 or older, added without a record of its version or how it was built. Those versions read the path-length byte as a plain byte count, so every packet with 2- or 3-byte hops was misparsed from the path onward: wrong header and type, a path split into single bytes, garbage payload fields, garbled advert names and group text that never decrypted.
The bundle is rebuilt from 0.3.0, which takes the hop count from bits 0-5 and the hash size from bits 6-7. Twelve real packets from the bot's database (adverts, group text, direct messages and returned paths at 1, 2 and 3 bytes per hop) decode to the same hop count, path and payload as the bot's own decoder; multibyte adverts get their real names and a 2-byte-hop Public message decrypts, on the tape and in the byte breakdown.
scripts/meshcore-decoder-browser/ holds the browser entry the old bundle was built from (recovered from the bundle: it re-exports the library and routes Ed25519's SHA-512 through crypto-js so signatures verify on a plain-HTTP page) and build.sh, which fetches the published package, installs its runtime dependencies with install scripts disabled and bundles it with a pinned esbuild in a temporary directory. It reproduces the committed bundle byte for byte. The unused meshcore-browser.min.mjs is removed, and the multibyte workarounds from the byte breakdown polish (skipping the browser decode and saying so) are taken out again.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
- Added a new feature to track and display the multibyte share of packets by payload type in the dashboard.
- Introduced a new database migration to store per-payload-type multibyte encoding data in the daily rollup.
- Updated the dashboard to visualize the multibyte share as a stacked bar chart, reflecting the share of each day's packets that took a multibyte path.
- Enhanced the API to provide raw counts for each payload type, ensuring accurate representation in the dashboard.
- Adjusted the frontend to maintain consistent color coding for payload types and improve the overall user experience.
- Updated tests to validate the new multibyte share functionality and ensure data integrity.
The flood tail decays over roughly twenty hops in bars under a pixel
tall. Buckets holding less than 0.1% of the series are no longer drawn,
which on the live mesh takes the axis from 64 buckets to 44.
What is left out is reported, not dropped: a line under the chart reads
"1,467 further flood packets (0.9%) sit in 20 hop buckets below 0.1%
each, and are not drawn." An unannounced cut would be the same lie as a
silently truncated category list.
Two details that matter for honesty:
Percentages still divide by the whole series, never by the drawn subset,
so removing the tail cannot inflate the bars that remain. The smallest
surviving bar on live data is 0.118%, which is what it was before.
A withheld bucket inside the axis is null rather than zero. Zero would
claim no packets travelled that far, which is a different and false
statement; null draws nothing and says nothing. Padding gaps stay zero,
because there it is true.
The threshold applies to the flood series only — the node series is
small enough to draw in full — and the underlying computation still
covers the whole 64-hop protocol range.
Three layout and cost changes.
Hops away and Roles now grow into their cards instead of leaving dead
space under a fixed-height canvas. Chart.js needs a positioned parent
with a real height, so the body becomes a flex column and the chart takes
the slack via flex-basis 0 — height: 100% would resolve against an
auto-height parent and collapse.
The hop axis now ends at the last hop that carries an observation rather
than at the last non-empty bucket of the padded union. On a quiet window
that collapses 64 buckets to 13; on a full one it changes nothing,
because the flood series really does have packets at every hop out to 63
— 14 of them at hop 63, and 14.1% of all flood traffic beyond hop 20.
That tail is real data, so it is drawn rather than truncated.
Dropped the Live Activity card. It opened three SocketIO subscriptions
and re-rendered on every packet to duplicate /realtime, which is a page
that already does it better. The dashboard now costs one snapshot read
per poll and holds no streaming subscriptions. A test asserts it stays
that way rather than merely that the markup is gone.
MeshCore carries up to a 64-byte path, so a route can be 64 hops long at
one byte per hop — and proportionally fewer with 2- or 3-byte hashes,
which is where the observed ceilings of 32 and 21 come from.
The chart capped at 32. That number was inherited from the old
dashboard's `BETWEEN 0 AND 32` filters and carried forward without
checking it against the protocol. On the live database it silently
discarded 5,654 flood packets arriving from as far as 63 hops.
Worse for the node series: the cap runs after the per-node MIN(), so a
node whose *closest* route exceeded 32 hops disappeared from the chart
altogether instead of appearing at the far end. The live history holds
one such node, reachable only by a 48-hop path.
The axis can now span 64 grouped categories, so bars drop their rounded
corners and padding above 24 buckets and the x ticks auto-skip. 64 is
kept as a hard ceiling: beyond it the path field could not have held the
route, so the value is corrupt rather than distant.
Keeps the advert series — nodes by their closest observed path — and
adds arriving flood packets by how far they had already travelled.
The two answer different questions and, on the live mesh, disagree
usefully: nodes peak at 2-3 hops and fall away quickly, while flood
traffic peaks at 5 and holds a long tail past 16. A close-in
neighbourhood absorbing flood from well beyond it.
One series counts nodes (2.8k) and the other packets (156k), so raw
counts on a shared axis would flatten the node series into the baseline.
Both are drawn as a share of their own total, with absolute counts in
the tooltip, and padded onto one contiguous hop range so the bars line
up. They also cover different spans — 7 days of adverts against
whatever packet_stream retains — so each is labelled with its own
window instead of being presented as one period.
Watch the units. observed_paths.path_length is a BYTE count, so hops are
path_length / bytes_per_hop. packet_stream.path_len is already a HOP
count, with the byte length carried separately as path_byte_length. A
17-hop 3-byte path is path_length 51 in one table and path_len 17 in the
other. Applying either rule to the other silently rescales the axis and
the only symptom is a chart that looks a bit off, so both conventions
are now pinned by tests against the shapes real rows take.
Flood packets carry no sender identity and observed_paths holds only
adverts, so the flood series cannot be reduced to a shortest path per
node the way the advert series is. It is a per-packet distribution, and
the tooltip says so.
Moving the hop histogram out into its own card left the routing mix as a
26px bar alone in a col-lg-5, next to a full-height chart. Fill it with
something that belongs there: what the traffic is, beside how it is
routed, over exactly the same packets — the totals agree because both
read the dimensioned rows.
This also gives payload_type_name a reason to exist. Migration 0019
added the column, integration.py writes it at capture time, the
refresher backfills it and an index covers it, and until now nothing
read it.
Category lists now roll their tail into "Other" instead of being
truncated at eight rows in the client. Silently dropping the tail left
bars that no longer summed to the total printed beside them; on the live
database that would have hidden 1,015 packets across four payload types.
307 direct neighbours was not plausible, and it was not real.
complete_contact_tracking.hop_count claims 800 zero-hop contacts. Only
68 of them have any one-hop path in observed_paths to corroborate that.
Their stored SNR piles up in a 1.5 dB band — 655 of 800 between 11.25
and 12.75 dB — and their RSSI clusters at -39..-48 dBm. Hundreds of
radios at different distances and terrain cannot land in a 10 dB window.
That is the signature of one strong local link being recorded against
every node whose traffic happened to arrive through it. Their return
paths agree: these "direct" contacts have out_path_len of 3 to 11.
The writer's intent is sound — repeater_manager only stores RSSI/SNR
when signal_info reports hops == 0 — so the field being fed to it does
not mean what the surrounding code assumes. Left as is; this change
stops the dashboard depending on it.
Neighbour membership now comes from path evidence: an advert whose
path_length equals its bytes_per_hop travelled exactly one hop. That
yields 38 nodes in 24h and 124 in 7d, with a plausible spread. Signal is
shown only where the path evidence and the stored hop count agree, which
is 5 and 12 nodes respectively; the rest read "no signal reading" rather
than borrowing a measurement taken on somebody else's link. A 24h/7d
selector bounds the window, capped well under observed_paths' 90-day
retention because a month-old link says nothing about today.
Separately, this fixes a bug I introduced. path_length is a BYTE count,
and with 2- or 3-byte hop encoding a 3-hop path is 6 or 9 bytes long. The
path-length histogram plotted that raw value on an axis readers would
take as hops, overstating distance two- to threefold on a mesh that is
~95% multibyte. It is replaced by a single hops-away chart computed as
path_length / bytes_per_hop, which also retires the histogram built on
the untrustworthy stored hop count. The result is unimodal, peaking at 3
hops and decaying — the shape a mesh should have, and not the bimodal
one the old chart drew.
Three adjustments from review.
Role and device type are the same field twice. Measured on the live
database they disagree on 16 of 11,028 contacts (ten roomservers and a
handful of bots and gateways reporting device Companion); every other
row is repeater/Repeater, companion/Companion, type11/Type11 and so on.
Charting both filled half a card with a copy of the other half. Keep the
role mix, which also carries the type0..type15 bucketing, and move it
into the routing row where the old signal card was.
Drop the tracked-contacts tile. is_currently_tracked does not describe
anything a reader can act on, and node activity is already covered by
nodes-heard and gone-quiet. The known-contacts total moves onto the
coverage tile, which leaves five tiles splitting the row evenly.
Rebuild the signal panel around zero-hop neighbours. Percentiles over
every received message answered no question anyone has: SNR on a relayed
packet measures the last hop into this radio, not the link to the node
that sent it, so averaging across hop counts describes nothing in
particular.
The panel now shows the nodes heard with no repeater in between — how
many, their SNR distribution, and the weakest links named, worst first,
on a fixed -12..+14 dB scale so bars mean the same thing between
refreshes. On the live database that is 307 neighbours, median 12.0 dB,
with eight marginal links surfaced from -9.0 dB down.
This also corrects the source. The plan rejected
complete_contact_tracking.snr as a badly biased 7% sample; in fact it is
populated on exactly the 800 hop_count=0 contacts and NULL on all 10,228
others. It is not a sample of the network, it is a complete census of
the neighbours — which is precisely the population the metric applies
to. message_stats.hops=0 covers only 34 senders by comparison.
The landing page re-ran ~50 aggregate queries five times per load, then
repeated the whole sequence every 30 seconds forever — including in
backgrounded tabs. Against the live 1.44 GB database that was roughly 20
seconds of SQLite work per page load.
Move the work off the request path. A refresher thread in the viewer
process (which already runs migrations, so it works for a split-DB
install) writes two tables: daily_rollup, one row per local date, and
dashboard_snapshot, a single JSON row. A page load now reads one row.
Measured on the live database: first paint 6 requests -> 2,
/api/dashboard/summary p50 1.1 ms (304 in 0.8 ms), /api/stats 130 ms,
and a 0.32 s refresh once a minute in the background.
Make the numbers mean what they say:
- Window selectors are built from each source's retention. The page
offered "30d" and "All" against tables pruned at 7 days, so three of
four choices returned the same figure under a label that denied it.
- The incoming-packet chart reports its measured window instead of
claiming 7 days for a table pruned at 3 — it sat beside a genuine
7-day contacts chart inviting an invalid comparison.
- Days with no source data store NULL and render as gaps. Writing 0
would put a fake cliff at every retention boundary.
- Signal metrics are stored as sums and counts, never means, so any
window re-aggregates correctly.
- Delta chips compare the last two complete calendar days and say so;
the headline above them is a rolling 24 hours.
- Unmapped role ordinals (type0..type15) bucket into "Unknown".
- SNR comes from message_stats, where it is populated on every row, not
from complete_contact_tracking, where it is populated on 7%.
Kill the json_extract scans: packet_stream gains denormalized
route_type_name, payload_type_name, path_len and bytes_per_hop, written
at capture time. Aggregating those from JSON cost 3-6 s per query.
Existing rows convert a bounded batch per tick rather than in one
migration that would rewrite ~180 MB into the WAL and stall bot startup.
A partial index serves as the backfill worklist — without it the "any
rows left?" probe is a full scan costing 4.6 s per tick, and it costs
that after the backfill finishes, because finding nothing still means
reading everything.
Also: replace the per-contact hop-prefix scan with the existing bucketed
matcher and memoize the 7-day chunk set (264 ms -> 35 ms on a synthetic
100k-row database, regression-locked by a test); move the dashboard's JS
and CSS to static files, which removes the CSP nonce requirement for the
bulk of the page; and give cleanup_old_stats a future-timestamp guard,
without which rows dated 2103 are never older than the cutoff and so
live forever.
Deletes the orphaned /stats page, unreachable from the nav and rendering
stub charts that never populated. /api/stats stays as a shim with every
key name intact plus Deprecation and Sunset headers.
All schema changes are additive, so a downgraded codebase can read the
data; it would however need the new schema_version rows removed, since
MigrationRunner rejects versions it does not know.
- Added 'X-Requested-With' header to various API requests in channel_operations.js, cache.html, config.html, contacts.html, feeds.html, greeter.html, mesh.html, radio.html, and other templates to improve request handling and prevent potential issues with cross-origin requests.
- Ensured consistent header usage across all relevant fetch calls to enhance security and compatibility.
These changes improve the robustness of API interactions within the web viewer.
- Updated `pyproject.toml` to include JavaScript files for the web viewer.
- Added a new script reference in `base.html` for channel operations.
- Improved the channel creation process in `feeds.html` with enhanced UI elements and error handling.
- Refactored channel index retrieval in `radio.html` to utilize a centralized method for better maintainability.
- Implemented asynchronous channel statistics loading to improve responsiveness during channel operations.
- Updated `config.ini.example` to include a new option for additional hashtag channels to decode in the packet stream.
- Modified `BotDataViewer` to retrieve and display additional decode-only channels from the configuration.
- Improved packet handling in `message_handler.py` to capture full packet data for web viewer integration.
- Enhanced the web viewer's JavaScript to support detailed packet analysis and display, including color-coded hex breakdowns and improved user interface elements.
- Added new styles and scripts to the web viewer templates for better visual representation of packet data and improved user experience.
agessaman