Channel-delivered region warnings were hard-coded to scope='*', on the belief that a sender with no region could not hear a scoped message. That is not how MeshCore works: the companion firmware's filterRecvFloodPacket drops nothing, so a client accepts a regional flood whatever region it has set itself, and only repeaters decide how far it goes. The override only meant that a bot configured with outgoing_flood_scope_override flooded the whole mesh to tell someone not to, which is exactly what #306 reports (the log shows the override 'was not applied (explicit scope="*")').
The warning now resolves its scope like any other proactive channel send: [Region_Warnings] flood_scope, then flood_scope.<channel>, then outgoing_flood_scope_override, then global. Its body is sized with channel_body_budget so the regional overhead is accounted for, and the web viewer's byte limit reserves that overhead whenever a regional scope could apply. flood_scope = * restores the old behavior.
An oversized webhook or channel body was handed to the firmware whole. The device dropped it without an ack, retries stalled the transport, and the radio looked dead. Split to the frame budget, keep links intact, and tag multi-part sends so they can be reassembled.
Reserve the wx location prefix from that same budget, and let NOAA alert polling be turned off without disabling the rest of the weather service.
* feat(website): Include local commands when generating website
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* feat(website): Disabled commands are excluded from generated HTML
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* Updated CHANGELOG.md
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
* fix(website): resolve enablement through the bot's own config helpers
The generated page is meant to list what the bot actually answers, so it
has to read the config the same way the bot does. Three places where it
did not:
The local `config.ini` overlay was never read. `read_config` parsed only
the base file, while the bot overlays `<local_dir_path>/config.ini` on
top of it, and that overlay is exactly where the web viewer saves a local
plugin's settings. A local command disabled from the settings page still
showed up on the site.
`is_command_enabled` derived the section name and the legacy `enabled`
spellings by hand, duplicating `BaseCommand._derive_config_section_name`
and guessing that a legacy key lives in the command's own section. Half
of them do not: `[Jokes] joke_enabled` disables `Joke_Command`. It now
calls `command_section_name` and `read_enabled`, which share the alias
table with the runtime and the settings UI.
That table was missing the same-section `[Joke_Command] joke_enabled` and
`[DadJoke_Command] dadjoke_enabled` spellings, which both commands accept
through their own fallback. The settings page read only the `[Jokes]`
form, so a bot disabled the same-section way displayed as enabled.
Also fold the duplicated local-commands-dir resolution into
`resolve_local_commands_dir`, take `bot_root` as a `MinimalBot` argument
instead of assigning it post-construction twice, and restore the `hidden`
attribute check that `generate_samples` lost when it moved to
`filter_commands`.
---------
Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
Co-authored-by: agessaman <adam@gessaman.com>
Closes#279.
A MeshCore client with no region configured sends every channel message as
a plain FLOOD, which every repeater on the mesh rebroadcasts. This adds two
things: free observation of how much of that the bot hears, and an opt-in
warning to the senders.
Observation classifies each channel message as scoped, global or unknown and
tallies it per channel per local day. It costs one upsert and no airtime, and
it is what lets an operator see the size of the problem before deciding to
spend airtime on it. The classification runs ahead of the flood_scopes
allowlist, because an unscoped message is exactly what that allowlist drops.
Warnings only ever fire on positive RF evidence of an unscoped FLOOD. Absence
of correlation is not proof that a sender omitted a region, so it classifies
as unknown and stays quiet. They are off by default, start in dry run, and are
fenced by min_unscoped_messages, a per-sender cooldown, a mesh-wide cooldown
and a daily cap on attempts. Dry run consumes the same budget it previews, so
the log is what going live would put on the air, not an upper bound. All three
limits read from the database, so a restart cannot release a burst.
Channel-delivered warnings go out at global scope on purpose: the recipient is
by definition outside any region the bot replies under.
New Settings -> Region Warnings page in the web viewer covers all of it. Its
dark-mode striped rows exposed a pre-existing base.html bug where Bootstrap's
light-theme text color survived on a dark row background (~1.3:1); fixed there
for every table in the app.
The local translation path defaulted to a bare `local/translations/`, resolved
against the process cwd and unrelated to `[Bot] local_dir_path`. Since
`local_dir_path` already selects where an operator's commands, service plugins
and config overlay live, the catalog belongs in that same tree. It now defaults
to `<local_dir_path>/translations`, resolved absolute against the bot root, so
relocating `local_dir_path` moves the catalog with it and the lookup no longer
depends on the working directory. An explicit `local_translation_path` still
wins.
Only the constructor at startup was passing the local path. `get_translator()`
builds and caches a translator per detected language, and `reload_config()`
builds a fresh one, and both were still constructing `Translator` with the
distributed path alone. So local overrides silently vanished from any reply that
used the sender's language, and did not survive a config reload. Both now pass
it, `reload_config()` republishes it alongside `translation_path`, and it is
snapshotted and restored on rollback. The DummyTranslator fallback sets it too,
since `get_translator()` would otherwise raise AttributeError there.
Collapsed `_deep_merge_translations` into the existing `_merge_translations`,
which already merged deeply with the primary winning. Rewrote that one to stop
mutating its input: it shallow-copied `fallback` and then recursed into the
shared sub-dicts, so merging a base language over English corrupted the cached
English catalog at every level below the first. Also replaced the two `print()`
calls on the error paths with logger calls.
Registered `local_translation_path` in the config schema next to
`translation_path`, and reverted a trailing-whitespace reflow that touched 84
lines of config.ini.example for a one-key addition.
Tests: local overlay (single-string override, added keys, local-only catalog,
missing directory), the merge no longer mutating its fallback, the default
following `local_dir_path`, an explicit setting overriding it, the overlay
reaching `bot.translator` end to end, and a reload picking up a moved path.
- Implemented caching for multi-byte mesh aggregation, allowing concurrent requests to share computation and reducing CPU load.
- Updated the web viewer to coalesce live edge events and refresh the mesh graph every 30 seconds, improving responsiveness on busy networks.
- Adjusted logging behavior in the web viewer to respect the configured log level, ensuring efficient logging without excessive duplicate entries.
- Added configuration options for mesh graph caching duration, enhancing user control over performance settings.
The landing page re-ran ~50 aggregate queries five times per load, then
repeated the whole sequence every 30 seconds forever — including in
backgrounded tabs. Against the live 1.44 GB database that was roughly 20
seconds of SQLite work per page load.
Move the work off the request path. A refresher thread in the viewer
process (which already runs migrations, so it works for a split-DB
install) writes two tables: daily_rollup, one row per local date, and
dashboard_snapshot, a single JSON row. A page load now reads one row.
Measured on the live database: first paint 6 requests -> 2,
/api/dashboard/summary p50 1.1 ms (304 in 0.8 ms), /api/stats 130 ms,
and a 0.32 s refresh once a minute in the background.
Make the numbers mean what they say:
- Window selectors are built from each source's retention. The page
offered "30d" and "All" against tables pruned at 7 days, so three of
four choices returned the same figure under a label that denied it.
- The incoming-packet chart reports its measured window instead of
claiming 7 days for a table pruned at 3 — it sat beside a genuine
7-day contacts chart inviting an invalid comparison.
- Days with no source data store NULL and render as gaps. Writing 0
would put a fake cliff at every retention boundary.
- Signal metrics are stored as sums and counts, never means, so any
window re-aggregates correctly.
- Delta chips compare the last two complete calendar days and say so;
the headline above them is a rolling 24 hours.
- Unmapped role ordinals (type0..type15) bucket into "Unknown".
- SNR comes from message_stats, where it is populated on every row, not
from complete_contact_tracking, where it is populated on 7%.
Kill the json_extract scans: packet_stream gains denormalized
route_type_name, payload_type_name, path_len and bytes_per_hop, written
at capture time. Aggregating those from JSON cost 3-6 s per query.
Existing rows convert a bounded batch per tick rather than in one
migration that would rewrite ~180 MB into the WAL and stall bot startup.
A partial index serves as the backfill worklist — without it the "any
rows left?" probe is a full scan costing 4.6 s per tick, and it costs
that after the backfill finishes, because finding nothing still means
reading everything.
Also: replace the per-contact hop-prefix scan with the existing bucketed
matcher and memoize the 7-day chunk set (264 ms -> 35 ms on a synthetic
100k-row database, regression-locked by a test); move the dashboard's JS
and CSS to static files, which removes the CSP nonce requirement for the
bulk of the page; and give cleanup_old_stats a future-timestamp guard,
without which rows dated 2103 are never older than the cutoff and so
live forever.
Deletes the orphaned /stats page, unreachable from the nav and rendering
stub charts that never populated. /api/stats stays as a shim with every
key name intact plus Deprecation and Sunset headers.
All schema changes are additive, so a downgraded codebase can read the
data; it would however need the new schema_version rows removed, since
MigrationRunner rejects versions it does not know.
- Added a tracked `LICENSE` file and updated `pyproject.toml` to include license metadata.
- Enhanced `CHANGELOG.md` with recent changes and clarifications.
- Updated `config.ini.example` and related documentation to reflect clamping behavior for numeric limits in `[Feed_Manager]`.
- Improved startup validation to suggest corrections for unknown and misspelled keys.
- Refactored geocoding and HTTP request handling to run off-thread, preventing event loop stalls.
- Added thread safety to cache management in geocoding functions to avoid race conditions.
Reworked implementation of the multilingual-response feature (issue #218)
as a reusable, framework-level capability instead of logic bolted onto
HelloCommand.
- modules/lang_detector.py: keyword-first detection (reliable for the short
greetings typical on mesh) with optional langdetect for longer text;
results are constrained to languages that actually have a translation file.
- core.py: cache Translator instances by language (get_translator) so
per-message switching never re-reads files, and expose available_languages()
derived from translations/*.json. Store translation_path; keep cache/path
consistent on config reload.
- base_command.py: task-local translator override via a ContextVar (safe under
asyncio concurrency, no global mutation), plus detect_response_language() and
a respond_in_sender_language() context manager any command can opt into.
- hello_command.py: build the reply inside respond_in_sender_language(); the
only awaited call (send_response) stays outside the override window.
- Gated by [Localization] auto_detect_language (default false); langdetect is
an optional [lang] extra, not a hard dependency.
Tests: unit coverage for the detector and the base-command wiring.
- Added a type ignore comment for the assignment of optionxform in config_schema.py to suppress type checker warnings.
- Updated path inference logic to return None when bot latitude or longitude is not set, ensuring safer handling of missing values.
- Enhanced the decoding of path nodes to check for None before appending repeater details, preventing potential errors.
The legacy [Section] enabled alias map (e.g. [Jokes] joke_enabled,
[Stats] stats_enabled) was duplicated in BaseCommand.get_config_value and
modules/settings_schema.py, so runtime behavior and the web settings UI
could drift apart. Both now read config_schema.LEGACY_ENABLED_ALIASES.
- Updated `config.ini.example`, `config.ini.minimal-example`, and `config.ini.quickstart` to clarify connection type precedence and required keys for serial, BLE, and TCP connections.
- Added detailed comments and examples for each connection type, ensuring users understand how to configure their connection settings effectively.
- Introduced a new `[Service_Overrides]` section in `config.ini.example` for alternative service plugin implementations, improving extensibility.
- Enhanced documentation in `configuration.md` to reflect these changes and provide clearer guidance on connection options and templates.
agessaman