Correctness:
- {path_distance} was always blank in production. The resolution code that
builds repeater_info dropped latitude/longitude in every branch, so the
calculator could never find a coordinate. My tests passed hand-built
dicts straight to the calculator and never exercised the builder, which
is why they stayed green. Coordinates are now carried through all four
construction sites, and the new tests drive _lookup_repeater_names via
its lookup_func hook so the real builder runs.
- The #80 route guard was defeated two ways in the channel handler. When
the RF data was an uncorrelated fallback, control fell through to the
raw-hex and routing_info fallbacks below, which took the route from the
unrelated packet anyway; the guard had actually made that path
reachable. message.routing_info was also assigned unconditionally, and
the path command reads it. "Not attributable" is now a terminal branch
and the routing_info hand-off checks provenance.
- Same fix was incomplete for DMs: routing_info was captured and turned
into path_info before the provenance check ran, so the later check only
declined to overwrite an already-wrong value. Guarded at the source.
- Rendering could transmit for real. Capture only intercepts
send_response, but advert calls send_advert() directly and
send_response_chunked never checked capture_sink. Chunked sends are now
captured, and rendering is opt-in via BaseCommand.render_safe (default
False) instead of a denylist that cannot be complete. This also closes
the DM-only leak: schedule is not marked safe, so {cmd:schedule} can no
longer broadcast configuration to a channel.
- Multi-part rendered output was rejoined into one oversized send.
Scheduled messages are now split to the RF body budget and sent through
send_channel_messages_chunked, on character boundaries so multi-byte
text is not corrupted.
- _last_path_distance_km is instance state that was only set on success,
so an invalid path request could show the previous request's distance.
Reset at the start of every execute().
- Stale-contact retries were only counted on non-OK results, so timeouts
and exceptions left a contact eligible forever and could recreate the
storm. All failed attempts count now.
Web viewer:
- A failed config reload was reported as a successful save. The API and
UI now distinguish "saved and active" from "saved, restart needed".
- Preview count was unbounded; clamped to 1-20.
- update_ini_values is a read-modify-replace with no locking, so two
concurrent viewer saves could lose one. Serialised behind a lock.
Adds a Schedule page that lists every [Scheduled_Messages] entry with its
next run time and supports add, edit and delete. Writes go to config.ini
and queue a config reload, so schedules change without restarting the
bot, which was the actual request.
It edits the config section the bot already reads rather than
introducing a database table. reload_config() already re-runs
setup_scheduled_messages() with rollback, so there is nothing to keep in
sync and the schedule command lists exactly what the page shows.
Validation runs through the same parsers the scheduler uses, so the UI
cannot accept a schedule the bot would later reject. The builder
composes cron from plain-language options and previews the next five
runs; entries the bot cannot run are listed as "Not scheduled" with the
reason instead of being hidden, since a typo that silences a message is
what an operator most needs to see. The 15-minute floor for {cmd:...}
messages is enforced at save time too.
Also relabels the radio Disconnect button to "Stop Bot" behind a
confirmation (#240). It was never a radio-only disconnect: the main loop
runs while self.connected is true, so disconnecting exits the process.
That surprised an operator running under tmux with nothing to restart
it. disconnect_radio()'s docstring now says so as well.
Two documentation gaps behind open questions.
The [External_Data] repeater_prefix_api_url comment claimed that leaving
it empty "disables prefix command functionality" (#70). That is not what
happens: the prefix command answers from the bot's own database of heard
repeaters, and the API only augments it with node counts from a wider
dataset. The wrong comment is a plausible reason the question was asked
at all. Corrected, and the JSON contract is now documented in the command
reference for anyone serving their own endpoint, since map.w0z.is is
defunct and has no drop-in replacement.
Also documented the pipx path (#222). The installer already grew a
virtualenv in July, which covered the PEP 668 half of that report, but
the unanswered part was where config.ini, the database and local/ live
under a pipx install. They are all resolved relative to the directory
containing config.ini, which means a bare `meshcore-bot` picks up
whatever is in the current directory — so the guidance is to pass an
absolute --config. Both console scripts are already smoke-tested from an
installed wheel in CI, so this path is supported rather than incidental.
Two guards on command placeholders in scheduled messages, neither
configurable, because both exist to protect a shared medium:
A 15-minute floor. A schedule containing {cmd:...} that fires more often
is rejected at startup with an error rather than quietly running slower.
The interval is sampled and measured by the tightest gap between
firings, so "0,1 * * * *" is correctly treated as every 60 seconds and
not as hourly. Schedules without a command placeholder are unaffected.
The command's own cooldown_seconds now applies to a render. Scheduling
is not a way around the rate a command was configured to run at. The
execution is recorded before the command runs, matching execute_commands,
so a slow or failing render cannot be retried straight past the cooldown.
Also fills documentation gaps from the preceding commits:
- --install-extras was only in the script's own --help; documented in
service-installation.md (including alongside --update-venv) and
upgrade.md.
- weather-service.md documented weather_alarm's once-a-day scheduling
with no route to more than one forecast a day, which is exactly what
people go there looking for. It now points at {cmd:wx ...} and notes
that sunrise/sunset still belong to weather_alarm.
A scheduled message can now embed the reply of any bot command:
[Scheduled_Messages]
0 6,12,18 * * * = Public:{cmd:wx Seattle}
The schedule side was never the missing piece — [Scheduled_Messages]
keys have been 5-field cron with @presets for a while, so multiple
times, intervals and @hourly were all already expressible. What was
missing was any way to get a service's output into a scheduled message:
placeholders were limited to mesh info (contact counts). That gap is
why services were growing their own schedule parsers.
CommandManager.render_command_output runs a command for its text alone.
MeshMessage.capture_sink makes that safe: when set, send_response
collects the reply and transmits nothing, checked before the
_last_response bookkeeping so a background render cannot overwrite the
response captured for a real user's command.
Degrades quietly in every failure mode — unknown, disabled, admin-only,
non-renderable, timed out or silent commands expand to nothing, so raw
{cmd:...} text is never put on the air, and a message left empty is not
sent at all. Command output is not re-scanned, so a reply containing
{cmd:...} cannot recurse. Commands that transmit directly rather than
returning text (announcements) are refused outright, since rendering
them would broadcast for real.
Bounded by [Bot] scheduled_command_timeout_seconds (default 30).
Total path distance was already a solved problem here: the test command
computes it and exposes {path_distance} through the piped-template
engine, with pipe filters and docs. The path command just had no way to
reach it, since reply_prefix went through plain str.format.
Rather than add a second distance implementation, this reuses what
exists. BaseCommand.get_standard_placeholder_fields now returns the
shared placeholder set so subclasses can extend it, the path command
renders reply_prefix through format_piped_template with an added
{path_distance}, and the sum itself uses utils.calculate_distance over
the nodes the path command has already resolved.
Distance covers sender -> each hop -> bot and is deliberately blank when
the chain cannot be measured end to end (unresolved hop, prefix
collision, missing or 0,0 coordinates, unknown sender, no configured
bot position), so a partial sum is never reported as the real distance.
Because the prefix now supports pipe filters, an empty value takes its
label with it: {path_distance|prefix_if_nonempty:📏 }
Supersedes #198, which added a parallel haversine, a show_path_distance
flag, and a distance_traveled key across 10 locales.
Add configurable PacketCapture observer name
Adds an optional [PacketCapture] observer_name used as the MQTT `origin`
for packet and status reporting, letting the observer identity differ
from the connected MeshCore device name. Unset keeps the previous
behavior. The public key backing origin_id, authentication, and topic
resolution is unchanged.
CHANGELOG conflict resolved by keeping both Unreleased entries.
This commit enhances the handling of zero-hop neighbors in the dashboard and database. It ensures that the **One-hop neighbours** section accurately reflects radios heard directly (MeshCore hop count 0) rather than originators of relayed adverts. The `observed_paths` table now includes nullable `snr` and `rssi` columns for zero-hop advert rows, allowing for better signal reporting. Additionally, a one-time backfill process copies recent zero-hop ADVERTs from the `packet_stream` to `observed_paths`. Documentation and tests have been updated to reflect these changes.
This update ensures that console scripts in the virtual environment do not retain outdated shebangs pointing to temporary build paths. The `--update-venv` command now rewrites shebangs in place, allowing broken installs to recover without a full rebuild. Additionally, all pip invocations now use `python -m pip` to prevent issues with stale shebangs. This change addresses issue #229 and improves the overall reliability of the service installation process.
solar-conditions-provenance.md was in the tree but absent from the
mkdocs nav and unlinked from anywhere, so it never appeared on the docs
site. Added under a Project section and linked from the solar command
entry, where a reader would look for it.
Removes the kg7qin PR integration log — an internal development record
that was excluded from the docs build but still shipped in the repo —
along with the now-dead exclude glob.
Move the 15-minute gap between cycles from the DM command to the service, as
MIN_CYCLE_GAP_SECONDS. The command was one caller among several: the scheduler
retried a failed cycle on its own 300s backoff, keyed off last_neighbors_publish
which a failed cycle never stamps, so a discover round whose acknowledgement was
lost went back on the air every five minutes. run_neighbors_cycle now refuses a
cycle inside the gap whoever asks, and the scheduler's retry backoff waits out
whatever remains of it. Failures that never reached the radio still stamp
nothing, so re-checking a disconnected radio stays on the short backoff.
The command asks the service for the remaining wait instead of computing its own,
so the two cannot drift, and rewinds the sender's per-user cooldown to expire
with the shared one. The command manager records an execution before calling
execute(), so a refusal had been consuming the sender's full 15 minutes: told to
wait one more minute, they would retry and be refused for another fourteen.
Rewound rather than cleared — a refusal reply is airtime too.
Follow-up to 7cffe1d; all three confirmed against meshcore 2.3.8.
Charge the node-wide cooldown for the transmission, not the result.
last_neighbors_publish is stamped only by a cycle that completes, so a discover
request whose acknowledgement was lost spent the airtime and left the clock at
zero -- another sender could immediately start a second round. The service now
stamps last_neighbors_attempt before the request, and the command rations on
whichever stamp is later. A cycle that bails out before touching the radio still
records nothing, so retries stay possible.
Restore the contact path when req_regions_sync returns None. send_anon_req gives
up early when change_contact_path reports an error -- which is also what a lost
acknowledgement for an applied path change looks like -- and that return skips
the library's own reset_path. req_regions_sync collapses it to None, previously
treated as a plain no-response. On the common "neighbour did not answer" path
the extra reset is a redundant device command: no airtime, idempotent, and it
re-syncs the contact cache.
Stop reporting a rejected restore as a success. reset_path returns an ERROR
event for a device rejection or its own response timeout rather than raising, so
the helper logged "restored flood path" either way and hid a contact left pinned
to zero-hop. It now inspects the event and warns with the reason.
Four review findings on this branch, all confirmed against the code and the
installed meshcore 2.3.8:
Single-flight the discovery cycle. The command guarded only its own task, so
the scheduler's independent call could overlap a manual cycle and each round
would collect into the other's discover window. run_neighbors_cycle is now a
guard around the cycle body, refusing whichever trigger arrives second.
Make the 15-minute cooldown per node rather than per sender. The base class
rations per user, but the cost here is mesh airtime: users could take turns and
keep the radio discovering continuously. Measured from the last cycle that
produced a result, so the scheduler's cycles count and a cycle that bailed out
without transmitting does not start the clock.
Window the neighbours evidence label. The combined viewer applied `days` only
to mesh_connections while reading every lifetime row from neighbor_links, which
is deliberately never pruned — so a link last heard years ago kept claiming a
recent path-derived edge was a current direct neighbour.
Match that label on full public keys too. MeshGraph.add_edge does not promote a
1-byte edge that has no public key, but still fills in the keys discovery
supplied, so the 3-byte prefix comparison alone left confirmed neighbours
labelled singlebyte. Truncating our keys to 2 chars instead would relabel every
other node sharing that byte.
Restore a contact's flood path after an interrupted scope request. send_anon_req
pins a path-less contact to zero-hop and restores it after the send with no
try/finally, so our own budget cancelling the request left the contact pinned
and every later message to it sent direct-only.
The meshcore>=2.3.8 pin needed no change: PyPI publishes 2.3.8 now.
- Updated `config.ini.example`, `command-reference.md`, and `configuration.md` to clarify the use of APScheduler for cron scheduling, emphasizing the difference in day-of-week numbering compared to Vixie cron.
- Added examples and notes to prevent confusion regarding the interpretation of cron expressions, particularly for users transitioning from Vixie cron conventions.
- Enhanced documentation to guide users in using the correct syntax for scheduling messages, ensuring better understanding and usability.
Port the observer firmware's neighbours feature into the bot's packet capture
service, by way of meshcore-packet-capture (upstream PRs #42/#43). On a long
interval the bot asks which repeaters it hears directly and records each
confirmed link with its measured SNR.
This is the strongest link evidence the bot collects: a first-party RF
measurement between two full 32-byte public keys. Path inference works from
1-3 byte prefixes with no keys, and complete_contact_tracking.hop_count
over-claims zero-hop (800 claimed vs 68 corroborated on the live database).
modules/neighbors_discovery.py keeps upstream's public names so its fixes and
tests stay portable. Two deliberate divergences:
- No command_lock plumbing. _SerializedCommands in modules/core.py already
serialises and paces every radio command, strictly more than upstream's
reentrant lock did.
- neighbors_collect_scopes defaults off. Upstream's zero-hop scope probe
relies on a neighbour not being a known contact; this bot tracks contacts,
and for a repeater with no stored path the library reaches zero-hop by
calling change_contact_path() then reset_path() -- mutating the device's
contact table per neighbour. Scope requests also hold the radio lock for
their whole round trip (~25s), stalling bot replies. The default cycle is
one command plus a passive listen window, during which the bot stays
responsive.
Evidence lands in neighbor_links and neighbor_observations (migration 22)
rather than mesh_connections, which cannot persist provenance. The viewer
exposes it as evidence=neighbors on /api/mesh/edges and a Neighbours Only
mode on the mesh page, with populated public keys and real SNR; confirmed
neighbours also relabel edges in the combined view and count as
provenance-trusted when framing the initial map.
neighbors_enabled is the single switch. Every enabled broker publishes once
it is on (mqttN_neighbors defaults true; set false to hold one back). The
topic derives from each broker's packets topic with the last segment swapped,
so a templated broker gets meshcore/{IATA}/{PUBLIC_KEY}/neighbors -- the
topic the firmware uses -- instead of an unrelated flat one. A derived
location-routed topic is skipped with a warning when no iata is set, rather
than publishing into meshcore/XYZ/... on a shared namespace. Snapshots are
non-retained: heard_secs_ago is relative to publish time, so a retained copy
would read as current days later.
Also adds a DM-gated `neighbors` command (the 12h interval floor makes
waiting for the scheduler impractical), which acks immediately and reports in
a second message once the window closes.
Requires meshcore >= 2.3.8 for send_node_discover_req / req_regions_sync.
- Added a new feature to track and display the multibyte share of packets by payload type in the dashboard.
- Introduced a new database migration to store per-payload-type multibyte encoding data in the daily rollup.
- Updated the dashboard to visualize the multibyte share as a stacked bar chart, reflecting the share of each day's packets that took a multibyte path.
- Enhanced the API to provide raw counts for each payload type, ensuring accurate representation in the dashboard.
- Adjusted the frontend to maintain consistent color coding for payload types and improve the overall user experience.
- Updated tests to validate the new multibyte share functionality and ensure data integrity.
- Implemented caching for multi-byte mesh aggregation, allowing concurrent requests to share computation and reducing CPU load.
- Updated the web viewer to coalesce live edge events and refresh the mesh graph every 30 seconds, improving responsiveness on busy networks.
- Adjusted logging behavior in the web viewer to respect the configured log level, ensuring efficient logging without excessive duplicate entries.
- Added configuration options for mesh graph caching duration, enhancing user control over performance settings.
- Implemented chunked deletion for data retention, allowing for smoother cleanup processes without monopolizing SQLite's writer lock.
- Configured retention settings to delete in batches with pauses, improving performance on SD-card installations.
- Updated Linux service installers to allocate 1GB of memory and 200% CPU, providing better resource management for Raspberry Pi workloads.
- Added detailed configuration examples for Raspberry Pi in documentation to guide users on optimal settings.
- Updated mesh graph path splitting and aggregation to run in SQLite, improving performance by avoiding Python materialization.
- Defaulted graph persistence to batched writes for new installations, reducing WAL churn and SD-card writes.
- Enhanced data retention to execute shortly after startup, independent of the nightly maintenance schedule.
- Added a table-specific index for `mesh_connections` to support window and retention queries, ensuring efficient data access.
The flood tail decays over roughly twenty hops in bars under a pixel
tall. Buckets holding less than 0.1% of the series are no longer drawn,
which on the live mesh takes the axis from 64 buckets to 44.
What is left out is reported, not dropped: a line under the chart reads
"1,467 further flood packets (0.9%) sit in 20 hop buckets below 0.1%
each, and are not drawn." An unannounced cut would be the same lie as a
silently truncated category list.
Two details that matter for honesty:
Percentages still divide by the whole series, never by the drawn subset,
so removing the tail cannot inflate the bars that remain. The smallest
surviving bar on live data is 0.118%, which is what it was before.
A withheld bucket inside the axis is null rather than zero. Zero would
claim no packets travelled that far, which is a different and false
statement; null draws nothing and says nothing. Padding gaps stay zero,
because there it is true.
The threshold applies to the flood series only — the node series is
small enough to draw in full — and the underlying computation still
covers the whole 64-hop protocol range.
Three layout and cost changes.
Hops away and Roles now grow into their cards instead of leaving dead
space under a fixed-height canvas. Chart.js needs a positioned parent
with a real height, so the body becomes a flex column and the chart takes
the slack via flex-basis 0 — height: 100% would resolve against an
auto-height parent and collapse.
The hop axis now ends at the last hop that carries an observation rather
than at the last non-empty bucket of the padded union. On a quiet window
that collapses 64 buckets to 13; on a full one it changes nothing,
because the flood series really does have packets at every hop out to 63
— 14 of them at hop 63, and 14.1% of all flood traffic beyond hop 20.
That tail is real data, so it is drawn rather than truncated.
Dropped the Live Activity card. It opened three SocketIO subscriptions
and re-rendered on every packet to duplicate /realtime, which is a page
that already does it better. The dashboard now costs one snapshot read
per poll and holds no streaming subscriptions. A test asserts it stays
that way rather than merely that the markup is gone.
Keeps the advert series — nodes by their closest observed path — and
adds arriving flood packets by how far they had already travelled.
The two answer different questions and, on the live mesh, disagree
usefully: nodes peak at 2-3 hops and fall away quickly, while flood
traffic peaks at 5 and holds a long tail past 16. A close-in
neighbourhood absorbing flood from well beyond it.
One series counts nodes (2.8k) and the other packets (156k), so raw
counts on a shared axis would flatten the node series into the baseline.
Both are drawn as a share of their own total, with absolute counts in
the tooltip, and padded onto one contiguous hop range so the bars line
up. They also cover different spans — 7 days of adverts against
whatever packet_stream retains — so each is labelled with its own
window instead of being presented as one period.
Watch the units. observed_paths.path_length is a BYTE count, so hops are
path_length / bytes_per_hop. packet_stream.path_len is already a HOP
count, with the byte length carried separately as path_byte_length. A
17-hop 3-byte path is path_length 51 in one table and path_len 17 in the
other. Applying either rule to the other silently rescales the axis and
the only symptom is a chart that looks a bit off, so both conventions
are now pinned by tests against the shapes real rows take.
Flood packets carry no sender identity and observed_paths holds only
adverts, so the flood series cannot be reduced to a shortest path per
node the way the advert series is. It is a per-packet distribution, and
the tooltip says so.
307 direct neighbours was not plausible, and it was not real.
complete_contact_tracking.hop_count claims 800 zero-hop contacts. Only
68 of them have any one-hop path in observed_paths to corroborate that.
Their stored SNR piles up in a 1.5 dB band — 655 of 800 between 11.25
and 12.75 dB — and their RSSI clusters at -39..-48 dBm. Hundreds of
radios at different distances and terrain cannot land in a 10 dB window.
That is the signature of one strong local link being recorded against
every node whose traffic happened to arrive through it. Their return
paths agree: these "direct" contacts have out_path_len of 3 to 11.
The writer's intent is sound — repeater_manager only stores RSSI/SNR
when signal_info reports hops == 0 — so the field being fed to it does
not mean what the surrounding code assumes. Left as is; this change
stops the dashboard depending on it.
Neighbour membership now comes from path evidence: an advert whose
path_length equals its bytes_per_hop travelled exactly one hop. That
yields 38 nodes in 24h and 124 in 7d, with a plausible spread. Signal is
shown only where the path evidence and the stored hop count agree, which
is 5 and 12 nodes respectively; the rest read "no signal reading" rather
than borrowing a measurement taken on somebody else's link. A 24h/7d
selector bounds the window, capped well under observed_paths' 90-day
retention because a month-old link says nothing about today.
Separately, this fixes a bug I introduced. path_length is a BYTE count,
and with 2- or 3-byte hop encoding a 3-hop path is 6 or 9 bytes long. The
path-length histogram plotted that raw value on an axis readers would
take as hops, overstating distance two- to threefold on a mesh that is
~95% multibyte. It is replaced by a single hops-away chart computed as
path_length / bytes_per_hop, which also retires the histogram built on
the untrustworthy stored hop count. The result is unimodal, peaking at 3
hops and decaying — the shape a mesh should have, and not the bimodal
one the old chart drew.
Three adjustments from review.
Role and device type are the same field twice. Measured on the live
database they disagree on 16 of 11,028 contacts (ten roomservers and a
handful of bots and gateways reporting device Companion); every other
row is repeater/Repeater, companion/Companion, type11/Type11 and so on.
Charting both filled half a card with a copy of the other half. Keep the
role mix, which also carries the type0..type15 bucketing, and move it
into the routing row where the old signal card was.
Drop the tracked-contacts tile. is_currently_tracked does not describe
anything a reader can act on, and node activity is already covered by
nodes-heard and gone-quiet. The known-contacts total moves onto the
coverage tile, which leaves five tiles splitting the row evenly.
Rebuild the signal panel around zero-hop neighbours. Percentiles over
every received message answered no question anyone has: SNR on a relayed
packet measures the last hop into this radio, not the link to the node
that sent it, so averaging across hop counts describes nothing in
particular.
The panel now shows the nodes heard with no repeater in between — how
many, their SNR distribution, and the weakest links named, worst first,
on a fixed -12..+14 dB scale so bars mean the same thing between
refreshes. On the live database that is 307 neighbours, median 12.0 dB,
with eight marginal links surfaced from -9.0 dB down.
This also corrects the source. The plan rejected
complete_contact_tracking.snr as a badly biased 7% sample; in fact it is
populated on exactly the 800 hop_count=0 contacts and NULL on all 10,228
others. It is not a sample of the network, it is a complete census of
the neighbours — which is precisely the population the metric applies
to. message_stats.hops=0 covers only 34 senders by comparison.
The landing page re-ran ~50 aggregate queries five times per load, then
repeated the whole sequence every 30 seconds forever — including in
backgrounded tabs. Against the live 1.44 GB database that was roughly 20
seconds of SQLite work per page load.
Move the work off the request path. A refresher thread in the viewer
process (which already runs migrations, so it works for a split-DB
install) writes two tables: daily_rollup, one row per local date, and
dashboard_snapshot, a single JSON row. A page load now reads one row.
Measured on the live database: first paint 6 requests -> 2,
/api/dashboard/summary p50 1.1 ms (304 in 0.8 ms), /api/stats 130 ms,
and a 0.32 s refresh once a minute in the background.
Make the numbers mean what they say:
- Window selectors are built from each source's retention. The page
offered "30d" and "All" against tables pruned at 7 days, so three of
four choices returned the same figure under a label that denied it.
- The incoming-packet chart reports its measured window instead of
claiming 7 days for a table pruned at 3 — it sat beside a genuine
7-day contacts chart inviting an invalid comparison.
- Days with no source data store NULL and render as gaps. Writing 0
would put a fake cliff at every retention boundary.
- Signal metrics are stored as sums and counts, never means, so any
window re-aggregates correctly.
- Delta chips compare the last two complete calendar days and say so;
the headline above them is a rolling 24 hours.
- Unmapped role ordinals (type0..type15) bucket into "Unknown".
- SNR comes from message_stats, where it is populated on every row, not
from complete_contact_tracking, where it is populated on 7%.
Kill the json_extract scans: packet_stream gains denormalized
route_type_name, payload_type_name, path_len and bytes_per_hop, written
at capture time. Aggregating those from JSON cost 3-6 s per query.
Existing rows convert a bounded batch per tick rather than in one
migration that would rewrite ~180 MB into the WAL and stall bot startup.
A partial index serves as the backfill worklist — without it the "any
rows left?" probe is a full scan costing 4.6 s per tick, and it costs
that after the backfill finishes, because finding nothing still means
reading everything.
Also: replace the per-contact hop-prefix scan with the existing bucketed
matcher and memoize the 7-day chunk set (264 ms -> 35 ms on a synthetic
100k-row database, regression-locked by a test); move the dashboard's JS
and CSS to static files, which removes the CSP nonce requirement for the
bulk of the page; and give cleanup_old_stats a future-timestamp guard,
without which rows dated 2103 are never older than the cutoff and so
live forever.
Deletes the orphaned /stats page, unreachable from the nav and rendering
stub charts that never populated. /api/stats stays as a shim with every
key name intact plus Deprecation and Sunset headers.
All schema changes are additive, so a downgraded codebase can read the
data; it would however need the new schema_version rows removed, since
MigrationRunner rejects versions it does not know.
Changed the default setting for `auto_manage_contacts` from `false` to `device` across configuration files and updated related documentation. This ensures that the device handles auto-addition of contacts while the bot manages capacity. Adjusted comments and documentation to reflect this change for clarity and consistency.
- Added a tracked `LICENSE` file and updated `pyproject.toml` to include license metadata.
- Enhanced `CHANGELOG.md` with recent changes and clarifications.
- Updated `config.ini.example` and related documentation to reflect clamping behavior for numeric limits in `[Feed_Manager]`.
- Improved startup validation to suggest corrections for unknown and misspelled keys.
- Refactored geocoding and HTTP request handling to run off-thread, preventing event loop stalls.
- Added thread safety to cache management in geocoding functions to avoid race conditions.
Consistency pass ahead of the 1.0.0 release. `mkdocs build --strict`
now completes with no warnings; it previously failed.
- mkdocs.yml: the nav referenced `feeds.md` but the file is `FEEDS.md`.
This resolved on case-insensitive macOS and broke on the Linux CI
runner, so the live site shipped a dead Feed Management link.
- mkdocs.yml: add eight pages that existed and were linked from
docs/index.md but were absent from the nav, so they were unreachable
by site navigation: World Cup, Earthquake, Telegram Bridge, Repeater
Prefix Collision, Repeater Commands, and the custom command website.
Exclude docs/integration/, which is an internal development log.
- faq.md: the recovery snippets pointed at
/opt/meshcore-bot/meshcore_bot.db. Since the service-layout hardening
the database lives in /var/lib/meshcore-bot, and /opt is root-owned,
so `sudo -u meshcore` could not write there. sqlite3.connect() would
have created an empty database and failed with a confusing
"no such table" instead of a clear permissions error.
- repeater-commands.md: section headings containing "&" generated
different anchors under GitHub and MkDocs, so the table of contents
worked in one renderer and broke in the other. Use "and" so both
produce the same slug. Point the Auto-Purge entry at the subsection
that exists; it previously matched no heading in either renderer.
- command-reference.md: document the `webviewer` command, the only one
of 44 command modules with no entry.
- checkin-api.md: link the reference receiver on GitHub rather than by
a relative path that escapes the docs tree.
- index.md: add the upgrade guide, FAQ, data retention, local plugins,
check-in API, and custom command website.
Promote the pending 0.9.4 release to 1.0.0 and polish the release
metadata before tagging.
- Bump the version in pyproject.toml and the packaging docs assertion.
- Reframe the CHANGELOG entry as the first stable release, and add the
user-visible changes that shipped without entries: flexible command
prefixes, per-channel flood scope, packet-capture payload decoding,
startup config validation, MOWAS region scoping, hops/RSSI
placeholders, help-command channel overrides, webhook readiness, and
NWS coverage detection.
- Backfill CHANGELOG entries for 0.9.1, 0.9.2, and 0.9.3, which were
tagged but never documented, and add compare links for each.
- Credit external contributors for 0.9.1 and 1.0.0.
- Document the opt-in sender-language detection in the upgrade guide,
alongside the existing optional-extras sections.
- build-deb.sh: fail loudly when the version cannot be read from
pyproject.toml instead of falling back to a hardcoded 0.9.1, which
would stamp a release build with an apparent downgrade.
Reworked implementation of the multilingual-response feature (issue #218)
as a reusable, framework-level capability instead of logic bolted onto
HelloCommand.
- modules/lang_detector.py: keyword-first detection (reliable for the short
greetings typical on mesh) with optional langdetect for longer text;
results are constrained to languages that actually have a translation file.
- core.py: cache Translator instances by language (get_translator) so
per-message switching never re-reads files, and expose available_languages()
derived from translations/*.json. Store translation_path; keep cache/path
consistent on config reload.
- base_command.py: task-local translator override via a ContextVar (safe under
asyncio concurrency, no global mutation), plus detect_response_language() and
a respond_in_sender_language() context manager any command can opt into.
- hello_command.py: build the reply inside respond_in_sender_language(); the
only awaited call (send_response) stays outside the override window.
- Gated by [Localization] auto_detect_language (default false); langdetect is
an optional [lang] extra, not a hard dependency.
Tests: unit coverage for the detector and the base-command wiring.
- Enhanced the `/api/contacts` endpoint to support pagination, allowing users to specify `page` and `page_size` parameters.
- Added search functionality to filter contacts based on a search term.
- Implemented sorting options for the contacts list, enabling users to sort by various fields.
- Updated the frontend to reflect these changes, including a new pagination UI and improved loading states.
- Introduced caching for multibyte hop chunks to optimize performance when retrieving contact badge evidence.
- Added tests to ensure the new features work as expected and maintain existing functionality.
- Introduced a shared API for location resolution using `modules.location`, allowing for better handling of place lookups (coordinates, ZIP codes, city names).
- Updated geocoding functions to accept both strings and structured dictionaries, improving flexibility in location queries.
- Refactored existing commands (AQI, Rain, Wx) to utilize the new location resolution methods, streamlining the codebase and enhancing maintainability.
- Removed redundant location handling logic from commands, centralizing functionality in the new location module.
- Added tests to ensure proper classification and resolution of various location formats, including multi-word international cities and ZIP codes with surrounding whitespace.
- Added max_posts_per_check to limit the number of items posted per check, defaulting to max_items_per_check for backward compatibility.
- Updated logic in FeedManager to process new items, ensuring that only the specified number of posts are made while examining a larger set of items.
- Enhanced emoji selection to prioritize per-item emojis from the API, falling back to heuristics based on feed names when absent.
- Introduced new truncation and substring functions for formatting messages, improving text handling in feed outputs.
- Added API endpoints to reset feed error counts globally or for individual feeds, enhancing error management capabilities in the web viewer.
Adds device-level companion settings to /radio, headlined by the response
path hash size (mode 0-2 = 1-3 bytes per hop) wired to the existing
firmware config endpoints. New sections: Identity & Adverts (name, advert
lat/lon, location policy, zero-hop/flood advert buttons), Mesh Behavior
(extra ACKs, telemetry permissions), and write-only Advanced Tuning
(RX delay base, airtime factor, sent x1000 per the wire format).
Backend: GET /api/radio/params now returns the full SELF_INFO node fields;
POST accepts the new fields with validation; new POST /api/radio/advert;
scheduler ops handle the writes (other-params group as one read-modify-write
frame so partial updates never clobber device state).
Config-managed settings are not writable from the panel: new-contact mode
is owned by [Bot] auto_manage_contacts and shown read-only; the node name
is locked while bot_name + auto_update_device_name manage it. loop.detect
is removed from the firmware endpoints entirely - it is repeater/room-server
CLI config, and companion custom vars map to sensor settings.
Also: dark-mode styling for disabled form fields (base.html), TASK-01 guard
test updated for the deliberate reintroduction, 23 endpoint/template tests.
- Updated `config.ini.example`, `config.ini.minimal-example`, and `config.ini.quickstart` to clarify connection type precedence and required keys for serial, BLE, and TCP connections.
- Added detailed comments and examples for each connection type, ensuring users understand how to configure their connection settings effectively.
- Introduced a new `[Service_Overrides]` section in `config.ini.example` for alternative service plugin implementations, improving extensibility.
- Enhanced documentation in `configuration.md` to reflect these changes and provide clearer guidance on connection options and templates.
Add optional payload decoding to the packet capture service. GRP_TXT
channel messages are decrypted (sender/text), ADVERTs are parsed
(name/role/lat-lon), and a nested "decoded" object is attached to each
packet alongside the unchanged raw fields.
- Comprehensive channel key store: bot's configured radio channels plus
decode_hashtag_channels, [Channels_List], decode_channel_keys, and the
built-in default Public key.
- Publishing the decoded object to MQTT is off by default and
configurable per broker via mqttN_include_decoded.
- Configurable packet-log rotation (off/size/time) for historical dumps.
The decoder lives in a standalone, dependency-free module
(modules/meshcore_payload_decode.py) so it can be shared verbatim with
the meshcore-packet-capture project.
Closes#197Closes#35
- Updated `config.ini.example` to support single, multiple, and decorative command prefixes, improving flexibility in command invocation.
- Refactored `CommandManager` and `BaseCommand` to utilize new command prefix parsing logic, allowing for better handling of multiple prefixes and optional bare commands.
- Enhanced documentation to clarify command prefix configuration options and behavior, ensuring users understand how to set and use prefixes effectively.
- Added unit tests to verify the correct functionality of command prefix handling, including support for permissive and strict prefix modes.
- Updated configuration examples to include new `{hops}` and `{hops_label}` placeholders for path command replies, enhancing response detail.
- Modified `BaseCommand` to support hop count retrieval and formatting, ensuring accurate display of hop information in responses.
- Enhanced unit tests to verify correct formatting of hop-related placeholders in response messages, improving overall command functionality.
- Updated the `[Rain_Command]` section in `config.ini.example` to include support for snow alongside rain, improving the command's functionality.
- Enhanced documentation for the rain command to reflect the new snow alias and clarify response behavior based on the selected keyword.
- Added a new `collect_stats` option in the `[Stats_Command]` section, allowing stats collection to be enabled independently of the user-facing command, with updated documentation to explain its behavior.
- Improved the web viewer documentation to clarify how stats are collected and displayed, ensuring users understand the configuration options.
- Introduced a new section in the web viewer documentation detailing how to set up a reverse proxy using Nginx with basic authentication for enhanced security when exposing the web viewer outside the local network.
- Updated instructions for network access to recommend setting a password or using a reverse proxy for authentication.
- Included example Nginx server block configuration and necessary proxy parameters for proper functionality.
Add a rain nowcast (Open-Meteo 15-minutely precipitation — worldwide, no API
key) as both an on-demand `rain`/`nowcast` command and an opt-in Weather_Service
push that announces rain starting and stopping at the bot's position.
- modules/commands/rain_command.py — command plus pure, unit-tested
fetch/analyze/dedup/label helpers (also reused by the service)
- modules/service_plugins/weather_service.py — background poller mirroring the
existing weather-alert poll pattern; ships disabled (opt-in)
- Location labels resolve to "City, ST" (US) / "City, Country" (non-US)
- 34 unit tests; rain_command added to the strict-mypy module list
agessaman