Commit Graph
152 Commits
Author SHA1 Message Date
agessaman 6cbe4e1ca8 Merge dev (#345) into refactor/maintainability
Conflict in modules/core.py: #345 changed _validate_config_snapshot, which this branch moved to config_reload.py. The method there now matches dev's version; reload_config's warning merged cleanly.
2026-10-04 11:06:43 -07:00
agessaman 388540cbc7 fix(config): a bare '%' in a value no longer blocks config reload
_validate_config_snapshot expanded '%' interpolation on every value, so a value read raw with a literal '%' (an [MqttWeather] json_template with "% RH", which config.ini.example documents as fine) failed every reload, although startup accepts the same file. Such values are now returned and named in a warning, and the reload goes ahead. Typed schema keys are still checked as before.
2026-10-04 09:04:43 -07:00
agessaman ba8e3f0229 fix(refactor): restore 24 module-level names the moves stopped binding, and guard all of them
Codex's final review found that moving code out of modules dropped imports that local plugins could import from them, for example `from modules.service_plugins.discord_bridge_service import EventType`, which would make such a plugin fail to load. Comparing every top-level name bound on dev against the branch (both trees imported, so names under TYPE_CHECKING are excluded) found 24 in 11 modules: EventType, copy and contextlib in both bridges; sha256, Iterable and hmac_mod in message_handler; RotatingFileHandler, colorlog, contextlib, contextvars and functools in core; hashlib and re in utils; contextlib in feed_manager, earthquake_service and weather_service; Any in cmd and help; time in path. Each is re-imported exactly as dev imported it.

The public-import guard only covered `from modules... import` statements, so it missed these. It now records every unconditional top-level import and assignment, regenerated from dev's tree (461 → 1,597 names), and fails if the discord bridge stops exposing EventType.
2026-10-03 22:13:19 -07:00
agessaman bdb59221da refactor(core): move reload_config's helpers into ConfigReloadMixin
_config_section_values, _restart_only_config_changes, _validate_config_snapshot, _command_config_state and _apply_command_config_state move verbatim (AST-identical) to modules/config_reload.py, mixed into MeshCoreBot. reload_config itself stays in core: the reload-rollback characterization tests patch modules.core.CommandManager, which it calls. _COMMAND_CONFIG_STATE stays on MeshCoreBot (annotated on the mixin for mypy). core.py 1,269 → 1,154 lines.
2026-10-03 14:27:59 -07:00
agessaman e6c9a74e71 refactor(core): move radio device setup into DeviceSetupMixin
set_radio_clock, set_device_name and send_startup_advert move verbatim (AST-identical) to modules/device_setup.py, mixed into MeshCoreBot after the existing mixins; callers, the web viewer and tests reach them on the bot as before, and no test patches a name they use. core.py 1,428 → 1,269 lines.
2026-10-03 14:24:49 -07:00
agessaman 505058cc90 refactor(message_handler): remove the pending-message correlation list
When a channel message had no RF log row yet, the handler stored it in pending_messages, slept 100 ms, then looked up find_recent_rf_data(payload["pubkey_prefix"]) through correlate_message_with_rf_data. Nothing else read the list: try_correlate_pending_messages only set a "processed" flag nobody checked, and cleanup_old_messages pruned it, both on every received packet.

The wait and the second lookup stay, now written out directly. Results only differ where the list misbehaved (found by codex review): two messages entering the wait in the same millisecond with the same packet prefix shared one entry, so both got the second one's row, and now each gets its own; with message_correlation_timeout under 0.1 s the entry could be pruned during the wait, skipping the second lookup; and a non-string pubkey_prefix made the per-packet scan raise, logged as "Error handling RF log data". The list's DEBUG lines are gone with it.

The list, its four methods and PendingMessageEntry were public names, so this goes in as its own change rather than a refactor.

message_correlation_timeout is still read: it bounds how old an RF row can be and still match a channel message by its contents (#255). The example configs described it as a wait time; the comment now says what it does.

New tests cover the wait (a row logged during it is found; no wait without enhanced correlation or for scope lookups). They pass against the previous code and one fails without the wait.
2026-10-03 12:14:25 -07:00
agessaman e359d8d01f Merge origin/dev (#335, #336, #338, #341) into refactor/maintainability
command_manager: _record_transmission passes extra keyword arguments through, so #336's recipient_key (DMs) and channel_idx (channels) reach record_transmission as on dev.

graph_trace_helper: dev's rewrite from #336, with its two repeater-key lookups (immediate neighbor and _unique_repeater_key) going through contacts_repo.unique_recent_repeater_key, whose queries are identical.
2026-10-03 09:41:56 -07:00
agessaman 3c76c5bf73 fix(transmission_tracker): require our radio name on a channel match and drain writes at shutdown
Codex review follow-up. A channel message from another node with the same text claimed our record, and names the sender-prefix heuristic cannot split ("Bot [RF]", "Bot: RF") never matched; the match now compares the full decrypted message with "<radio name>: <text>", falling back to the text alone before the name is known. Bot shutdown closes the tracker after disconnecting, which waits for queued repeat-count writes; any later write runs inline.
2026-10-02 17:33:18 -07:00
agessaman 308e95f159 refactor: restore AqiCommand.geolocator and MeshCoreBot.last_clock_sync_time
The S1 dead-code slice dropped both because nothing in the tree reads them, but local plugins may (codex review of the dev merge). set_radio_clock records the sync time again after a successful update. Two DEBUG diagnostics S1 removed with dead code stay removed: parse_advert's flags line behind handler.debug and the overwritten contact-path lookup lines.
2026-10-02 16:44:22 -07:00
agessaman 0853739f30 fix(weather): review follow-ups: warning units, wx localization, help text
A space bunny review of this branch's commits found:
- gwx's high-wind warning converted the speed to mph for every catalog, so on a km/h bot it read 49 next to a reply saying 80, and Polish (whose template said km/h) regressed in the merge. The threshold stays 30 mph; the warning shows the speed in the configured unit with a {unit} placeholder in every catalog.
- tomorrow_not_available, tomorrow_error, multiday_not_available and multiday_error were still English in seven catalogs; they are translated, and the catalog test now fails on any weather reply left in English.
- wx kept NOAA's English wind letters and English weekday letters while gwx's are translated, so one German bot showed NE and NO for the same wind. wx now translates both through the catalog (commands.wx.day_abbrev added); English output is byte-for-byte unchanged, which the goldens show (their stub translator gives English for these short labels, via tests.helpers.key_name_translate).
- Polish feels_like was ungrammatical (odczuwalna -> odczuwalnie), Russian's used a nonstandard abbreviation, and Russian gwx's MQTT subscriber error lacked wx's hint.
- The help text (commands.wx.description/usage, commands.gwx.usage) described wx as ZIP-only; it now lists cities, coordinates and the options, under 130 bytes in every language.
- The generated default config, the docs' default_state wording, _fit_tomorrow_parts' docstring and the CHANGELOG's stats claim were inaccurate.
2026-10-02 00:48:31 -07:00
agessaman e71f91adda refactor(core): read the probe interval through _radio_probe_interval_seconds in the main loop
The main loop computed the same clamped [Connection]/[Bot] radio_probe_interval_seconds expression inline that the health probe already reads through the helper.
2026-10-01 18:55:54 -07:00
agessaman fe912299d5 refactor(core): move service supervision into ServiceSupervisorMixin
`modules/service_supervisor.py` holds starting services at boot, the backoff-gated restart of unhealthy services and `get_system_health`: six methods moved verbatim, still reachable on `MeshCoreBot`. Stopping services stays in `_stop_inner`, where it is one step of the shutdown order.
2026-10-01 18:53:21 -07:00
agessaman 3fc3bbf8fb refactor(core): move the radio link and health probe into RadioLinkMixin
`modules/radio_link.py` now holds connecting and reconnecting the transport, the host->radio command serializer and `radio_session`, zombie state and the periodic health probe: 20 methods moved verbatim, so every name is still on `MeshCoreBot`. `_serialize_command_frames` and `_radio_session_held` moved with them and are re-exported from `modules.core`; the serializer's `bot` annotation is now the mixin. `core.py` keeps `import meshcore` because tests patch `modules.core.meshcore.MeshCore`. core.py: 2,164 → 1,599 lines.
2026-10-01 18:50:20 -07:00
agessaman b09340fdf3 Merge remote-tracking branch 'origin/dev' into refactor/maintainability 2026-10-01 15:11:53 -07:00
Adam Gessaman 9027c2aef5 Merge pull request #318 from agessaman/fix/service-restart-loop
fix(services): back off services that don't start, and unsubscribe on stop
2026-10-01 15:11:44 -07:00
agessaman 4bdb1ef186 fix(services): keep restarted services logging, and stop a taken port from killing the bot
A second review of the restart fix found more of the same class. The map uploader's stop() removes its file log handler and start() never added it back, so after one restart it logged nothing to file; start() now restores it. DARC MoWaS set _running before binding its port, so a port in use left it dead but healthy, and werkzeug's make_server calls sys.exit when the port is taken, which got past every except Exception and stopped the bot; start() now binds first and turns that exit into an OSError.

Boot-time starts go through _start_service_at_boot, which records a restart failure when start() raises, so that service also waits out the backoff, and no longer reports a disabled service as a configuration problem. The health-loop step is _restart_unhealthy_services. New tests cover both, the bridges' session cleanup and the map uploader restart; all 13 lifecycle tests fail on dev.
2026-10-01 14:33:56 -07:00
agessaman 726e1381cb refactor(core): move the radio-offline circuit breaker into a RadioOfflineBreaker mixin
The 17 breaker methods #313 added or rewrote, and the trial context variable, move verbatim from core.py into modules/radio_offline.py; MeshCoreBot inherits them, so every method name and instance attribute stays where callers and tests use it. The only textual change is that the moved annotations are unquoted under from __future__ import annotations. core.py drops to 2,117 lines.
2026-10-01 11:57:07 -07:00
agessaman b750ad43f0 Merge origin/dev (#313) into refactor/maintainability
core.py conflicts were S3's simplifications of is_radio_offline, _record_send_failure and _record_send_success colliding with #313's rewrite of the same methods; dev's versions are kept.
2026-10-01 11:53:01 -07:00
agessaman 3ff24a93f6 fix(services): back off a service that does not start instead of retiring it
Review found that marking a not-running service inactive for good also retired services that only returned early for a transient reason: packet capture and the map uploader give up after waiting 30 s for the radio, and the bridges return when meshcore is momentarily unset, which is exactly when the health loop restarts them. Such a service now records a restart failure, so the next attempt waits out service_restart_backoff_seconds; a transient failure recovers, a missing configuration costs one attempt per backoff instead of one every 5 s. The restart decision is _service_restart_due, which the tests now exercise directly.

Also from the review: packet capture and the map uploader never cleared the should_exit flag stop() sets, so after a restart their background loops exited at once while the service reported healthy; start() now clears it. The Discord and Telegram bridges close the HTTP session they open when they return early because meshcore is not available.
2026-10-01 11:47:47 -07:00
agessaman 4c4f65d73f fix(core): a trial that does not succeed waits for the next probe, and its settlement survives cancellation
A fifth review found two gaps. A scheduled trial that returned False was re-armed at once, but False also follows exhausted no_event_received retries, so the radio could be tried again and again without another answered probe. And a reply's success settlement ran in a worker thread that a cancelled caller could cancel while it was still queued, stranding the trial in flight.

_end_offline_trial ends a trial that went out without succeeding (False or an exception, from a reply or a scheduled send) under the state lock alone; only paths where nothing reached the send still re-arm. The success settlement is shielded, so it completes even if the caller is cancelled. The cancellation test queues the settlement behind a busy one-worker executor and fails without the shield.
2026-10-01 11:29:44 -07:00
agessaman a3b0f56639 fix(services): stop restarting unconfigured services, and unsubscribe on stop
The health loop restarted every enabled service whose is_healthy() was False. A service that returns from start() without running because it has nothing to do (Discord or Telegram with no channels, the map uploader with no key) is never healthy, and its restart raised nothing, so it counted as a success and cleared the backoff: the service was restarted every 5 seconds, logging a warning each time.

Services whose start() returns without running are now recorded as inactive, logged once, and skipped by the health loop. A restart that leaves a service running but unhealthy records a failure, so the next attempt waits out service_restart_backoff_seconds.

Restarts also double-subscribed: services subscribed to meshcore events in start() and never unsubscribed (the comments said meshcore cleans up; it does not). BaseServicePlugin gains _subscribe and _unsubscribe_all; the Discord, Telegram, map uploader, packet capture and prefix-collision services subscribe through it, unsubscribe in stop(), and drop the old instance's subscription before re-subscribing after a transport reconnect. Tests use meshcore's real EventDispatcher and fail on dev.
2026-10-01 11:25:18 -07:00
agessaman 49960e50bf fix(core): let a reply take the offline trial, so bots without schedules recover too
Only scheduled messages and interval adverts could claim the trial armed by an answered health probe, so a bot with neither stayed offline however healthy the radio became. send_dm and send_channel_message now claim an armed trial when offline and run the send through _send_as_offline_trial, under the trial's identity; its result settles the trial (True clears the outage, False or an exception fails it until the next answered probe), so a dead radio still gets at most one attempt per probe. Settlement runs in a worker thread because it can write bot_metadata. Tests drive the real CommandManager guard, and the alert-thread counting in the recovery tests no longer stops asyncio.to_thread's workers.
2026-10-01 11:18:21 -07:00
agessaman 43ca23b30f fix(core): only the current trial can clear an outage, and an expired trial is cancelled
A third review found four more gaps. A trial send that hit the 60 s wait kept running and could transmit later, during a newer trial; _record_send_success cleared any outage without checking whose success it was, so an old send finishing late could clear a newer one; the wrappers claimed the trial before logging and time lookup, outside the code that settles it; and the startup advert's timeout path recorded its failure on the event loop, where tripping the outage writes bot_metadata.

_record_send_success now takes the trial id and clears only when that trial is still in flight, deciding and resetting under the state lock (_reset_offline_state_locked); success of an ordinary send while offline leaves the outage alone. A trial whose wait fails is cancelled. Each wrapper settles the trial if anything after admission raises. The startup advert records its failure through asyncio.to_thread.
2026-10-01 10:57:46 -07:00
agessaman cf2c2f47ee Merge remote-tracking branch 'origin/dev' into fix/radio-offline-latch
# Conflicts:
#	CHANGELOG.md
2026-10-01 09:36:23 -07:00
agessaman 5e35098a36 fix(core): give each offline trial an identity and settle it on every exit
A second review found five gaps. A loop that closed between is_running() and submission stranded the trial in flight; fallback sends (no running main loop) settled every trial as inconclusive; a timed-out send that kept running passed the guard during a later trial, because the context flag was a bare True; the viewer clear checked the outage generation, released the lock, then cleared, so it could clear a newer outage; and the health loop ran the metadata sync, with its database retries, on the event loop.

Trials now carry an id: _admit_measured_send returns it, _as_offline_trial sets it in the context, and is_radio_offline and the settle methods only honor the trial currently in flight. The scheduler's two wrappers share _run_measured_send, which submits inside its error handling (closing the coroutine if submission fails), settles a fallback trial by its real outcome, and settles any trial left open in a finally. _clear_radio_offline_state takes an expected generation and checks it under the state lock. The health loop checks the 30 s throttle on the loop and runs the sync itself in a worker thread. The delayed-write test now waits for the newer outage to trip before releasing the stalled writer, so it fails without publish serialization.
2026-10-01 09:33:22 -07:00
agessaman 473d0d6a22 fix(core): admit exactly one trial send while radio-offline and settle it by its real outcome
Review of the circuit breaker found four problems. The trial was a shared flag that let every sender through, interactive sends never settled it, and it could stay open forever. The scheduler threw away send_channel_message's result, so a send the radio rejected counted as a success and cleared the outage. Metadata writes happened outside the state lock, so a stalled clear could overwrite a newer outage's 'true' and the viewer check would then cancel it. And one failed read-back of the persisted 'true' disabled the viewer clear for the rest of the outage.

Now a probe arms one trial, which the scheduler's next measured send (scheduled message or interval advert) claims atomically through _admit_measured_send; that send runs under a context flag so only it passes the send-path offline guards. Its real outcome settles it: success clears the outage, a timeout ends the trial until the next answered probe, and a False result or nothing sent re-arms it. _send_scheduled_message_async now returns whether the message went out. Metadata is published under its own lock from the current state, confirmed per outage generation, and retried from the health loop until confirmed. The in-process viewer clear calls the full clear. Tests go through the real scheduler on a live loop, race admission across threads, stall a clear write while a new outage trips, and count alerts from the first trip.
2026-10-01 09:15:00 -07:00
agessaman 857d0ccb02 fix(core): treat radio-offline recovery as a circuit breaker
Review follow-up. A get_time probe is a host-to-companion frame with no RF transmit, so it answers in exactly the failure mode the offline state exists for; clearing on it would flap offline and online every probe interval and send an alert email each time. A probe success now only opens a trial: sends are let through, and the next send the scheduler measures either clears the state or closes it again, without another alert. The alert goes out once per outage.

The reconnect clear is gone (a connect can still fail after the socket is up, and the probe covers recovery after it). The viewer-clear check runs only after this process has read back its own 'true', so a read that races the trip cannot cancel it; it is throttled to every 30 s and runs before the transport check, so it works while disconnected. State changes take a lock, since the scheduler thread and the event loop both write them.
2026-10-01 06:58:42 -07:00
agessaman efe074b151 refactor: restore behavior the branch had changed (codex review)
A codex review of the branch found seven places where earlier refactor commits changed behavior. Each is restored here with a test:

- base_service imported flood_scope relatively; the local-service loader also runs that file as local_services.base_service, where the import cannot resolve. Now absolute.
- Radio-health fields were initialized after commands and services were built, so a plugin constructor reading is_radio_offline raised. They are now set before any plugin.
- Removing the per-command can_execute overrides made 31 commands honor skip_channel_check, which their overrides had dropped. BaseCommand.honors_skip_channel_check (False on exactly those commands) keeps them enforcing their channel list.
- CommandManager.get_plugin_metadata, WxCommand.get_weather_for_zipcode and PluginLoader.validate_plugin were public compatibility entry points; they are back.
- feed_manager stopped exporting DEFAULT_MAX_FEED_RESPONSE_BYTES and DEFAULT_MAX_PARSED_FEED_ITEMS despite the commit saying it re-exported them; it does now.
- wx and gwx logged companion-location lookup failures at WARNING; the shared helper logged at debug. It now takes the level.
- The INFO 'MESSAGE ROUTING' / 'DIRECT MESSAGE' lines for pubkey-correlated DMs were removed with the dead path_info computation; they are logged again by a log-only helper.
2026-10-01 06:53:14 -07:00
agessaman 009dae8a05 fix(core): let radio-offline state clear while sends are suppressed
After radio_offline_threshold consecutive send timeouts the bot enters radio-offline state and every send path refuses to transmit. The only thing that cleared it was _record_send_success, which runs after a send goes out, so once offline the bot stayed offline until restarted. The web viewer's Clear Offline Flag could not help either: the viewer runs in its own process, so it only wrote bot_metadata, which the bot never read back.

Three paths now clear the state: a health probe the radio answers, a successful (re)connect, and the health loop noticing that bot_metadata says the flag was cleared. The new tests for the probe and viewer paths fail without the change.
2026-10-01 06:35:37 -07:00
agessaman 15195eccf9 refactor(core): drive reload's publish and rollback from one candidate map
reload_config named the same thirteen attributes four times: building the candidates, snapshotting the old values, publishing, and rolling back. The candidates now go in one dict whose keys are the attribute names; the snapshot, the publish loop and the rollback loop all come from it. Command config state, max_channels, solar config, the region-warning monitor and the scheduler keep their explicit rollback steps, since a reference swap cannot undo them. The reload failure-injection tests pass unchanged.
2026-10-01 06:17:39 -07:00
agessaman 727909ba8e refactor(core): move logging configuration into modules/logging_setup.py
setup_logging's formatter, handler and library-logger wiring is now configure_bot_logging(config, bot_root); MeshCoreBot.setup_logging calls it and still installs the signal handlers. The meshcore logger list that setup_logging and _configure_meshcore_debug_logging each spelled out is one MESHCORE_LOGGER_NAMES, and both use configure_meshcore_loggers. _JsonFormatter moves along as JsonFormatter. Statement order inside setup is unchanged, so the startup log lines come out as before.

Also fixes mypy on the admin_server module from the previous commit: as a new module it is type-checked, and its Flask view annotations needed loosening.
2026-10-01 06:14:25 -07:00
agessaman afea655b75 refactor(core): ship the first-run config as modules/templates/default_config.ini
create_default_config held a 340-line config.ini as a string literal in core.py. The text now lives in a packaged template (added to package-data, which the packaging test checks) and the method writes it the same way. The file written on first run is byte-identical to before, checked against the literal's value from the previous commit.
2026-10-01 06:07:54 -07:00
agessaman e3a6e91878 refactor(core): move the admin HTTP server into modules/admin_server.py
_BotAdminServer (the loopback Flask thread behind [Admin] with the reload and health endpoints) is unrelated to the rest of core.py and moves unchanged to admin_server.BotAdminServer.
2026-10-01 06:04:37 -07:00
agessaman 0db6d857fb refactor(core): initialize radio-health state in __init__
_radio_zombie_detected, _radio_fail_count, _tcp_probe_fail_count, _radio_offline, _send_consecutive_failures, _last_radio_probe and _last_health_update were created on first write, so every read went through getattr(self, name, default) or hasattr. They are now set in __init__ and read directly. The probe timestamp starts as None and is still stamped on the first health-loop pass, so the first probe waits a full interval as before.
2026-10-01 02:05:37 -07:00
agessaman 06a4db114c refactor: remove branches for APIs that always exist
meshcore >= 2.3.14 is required and installed, and these guards protected against its absence or against first-party modules failing to import:

- channel_manager: the meshcore_cli get_channel fallback beside commands.get_channel
- core: the set_name hasattr check (an absent method still lands in the existing AttributeError handler and returns False), the wait_for_contacts polling loop after an always-true hasattr(meshcore, 'contacts'), and the hasattr check before _start_queue_processor
- wx and gwx: try/except ImportError around wx_international and wxsim_parser, which import only required dependencies, and the 'parser not available' branches that could never run

Branches that also catch runtime errors raised inside an existing API stay, as do the DM retry and set_flood_scope paths, which open PRs are changing. The inspect.signature checks in execute_commands become one _call_with_optional_user_id helper, keeping zero-argument cooldown hooks working for local plugin commands.
2026-10-01 01:15:19 -07:00
agessaman d3a7bf95a4 refactor: delete functions and attributes nothing calls
Every name removed here has zero references across modules, tests, docs, local plugins, scripts and generate_website.py, and none is part of the documented plugin surface (db_manager, command_manager send methods, channel_manager stay untouched). About 1,160 lines:

- multitest: four formatter helpers that only referenced each other (the flat and nested variants are the ones in use)
- path: _get_api_cache_data, _filter_recent_repeaters
- repeater command: _handle_contact_stats, _get_geocoding_status
- airplanes: _send_split_response; greeter: get_recent_greeted_users; plugin_loader: validate_plugin
- wx: get_weather_for_zipcode, abbreviate_alert_title, get_current_conditions
- message_handler: discover_message_path, and an ADVERT debug line behind a self.debug attribute that is never set
- utils: _get_node_location_and_key_from_db
- repeater_manager: test_meshcore_cli_commands, toggle_auto_add, restore_repeater, get_purging_stats, populate_missing_geolocation_data, get_daily_advertisement_stats, get_nodes_per_day_stats
- command_manager: get_available_commands_list and get_plugin_metadata (HelpCommand and PluginLoader have the live versions)
- base_command: _load_command_prefix, and a no-op .replace('_', '_')
- scheduler: the first of two _run_data_retention definitions, shadowed by the second since it was added
- core: the write-only last_clock_sync_time, and _setup_routing_capture, which only logged a line

Full suite, ruff and mypy pass, and the characterization golden files are unchanged.
2026-10-01 01:07:29 -07:00
agessaman 5452fd58ee fix(dm): wait at least dm_min_ack_timeout for each DM attempt's ACK
send_msg_with_retry was called with timeout=0, so each attempt waited 1.2 times the radio's suggested_timeout. The companion firmware derives that from airtime alone (500 + (6 x airtime + 250) x (hops + 1) ms direct, 500 + 16 x airtime flood), but the round trip also carries fixed costs that do not shrink with airtime: the recipient's ACK delay, repeater retransmit and rx delays, the radio's TX queue and the host link. On a fast preset a zero-hop attempt gave up after about 1.3 seconds, the ACK of a delivered message arrived after the last attempt, and the send reported failure. That is the residual #290 symptom: webhook DMs erroring within a couple of seconds for messages the recipient had.

New [Bot] dm_min_ack_timeout (default 8 seconds) is passed as min_timeout. An ACK still ends the wait at once, so successful sends are no slower, and since the radio lock is per-frame the longer wait on a failing DM does not hold up other sends. 0 restores the old behavior. The new test drives the real meshcore retry loop with an ACK slower than the radio's estimate and fails without the floor.
2026-09-30 22:20:10 -07:00
agessaman b5e61306b2 Merge remote-tracking branch 'origin/dev' into feat/region-code-warnings
# Conflicts:
#	CHANGELOG.md
2026-09-20 10:43:24 -07:00
Adam Gessaman 38b259156e chore(deps): require meshcore 2.3.14 for the send_msg_with_retry ACK fix (#294)
The ACK fix I wrote for the radio-lock work is now upstream and released
(meshcore_py#108, v2.3.13). Before it, send_msg_with_retry reported ACKed DMs as
failures: the ACK subscription was registered only after send_msg returned, so
an ACK queued right behind MSG_SENT was dispatched with no listener, and each
attempt accepted only its own ACK code, so a late ACK answering an earlier
attempt was ignored. The bot logged "no ACK received after retries" and skipped
the delivery bookkeeping for messages the recipient had in fact received. Pin
2.3.14, which also fixes send_cmd's destination-type handling.

Two fixes from re-reading the merged radio-lock change:

A region-scoped channel message now restores global flood even when
set_flood_scope raises. The restore only ran in the finally around the send, so
a set that raised left the device pinned to that region and every later send
went out under it.

Drop the __setattr__ left behind when _SerializedCommands became
_serialize_command_frames; it sat after the function's return, unreachable, and
still referenced the proxy's _commands slot.
2026-09-19 17:28:14 -07:00
Adam Gessaman 9939e22575 fix(core): release the radio lock while waiting for ACKs (#291)
Radio commands were serialized per method call, so a composite command held the lock for everything it awaited. A DM's send_msg_with_retry kept every other command waiting through all of its ACK timeouts (up to ~36 s with three attempts), stalling channel replies, other DMs, scheduled sends, and automatic message fetching. req_regions_sync did the same for each neighbor scope reply.

Serialize at the frame instead: wrap CommandHandler.send() on the handler instance, which is where every meshcore command writes its frame and waits for the radio's immediate reply. There is still at most one in-flight companion frame, paced as before, and library methods that call self.send are covered without a proxy. Waits for ACKs and remote responses now run outside the lock.

Add MeshCoreBot.radio_session() for short sequences that must not interleave, and use it for region-scoped channel sends so set scope, send, and restore stay together. The per-call lock never guaranteed that: a DM retry loop queued between set_flood_scope and send_chan_msg sent its flood attempts under the channel's scope.
2026-09-19 09:12:07 -07:00
agessaman c566fa97c8 fix(channel): improve channel fetching with retries and handle empty results cleanly (#266) 2026-09-16 17:32:54 -07:00
agessaman bb3bfe321b feat(region): warn senders whose channel messages carry no region code
Closes #279.

A MeshCore client with no region configured sends every channel message as
a plain FLOOD, which every repeater on the mesh rebroadcasts. This adds two
things: free observation of how much of that the bot hears, and an opt-in
warning to the senders.

Observation classifies each channel message as scoped, global or unknown and
tallies it per channel per local day. It costs one upsert and no airtime, and
it is what lets an operator see the size of the problem before deciding to
spend airtime on it. The classification runs ahead of the flood_scopes
allowlist, because an unscoped message is exactly what that allowlist drops.

Warnings only ever fire on positive RF evidence of an unscoped FLOOD. Absence
of correlation is not proof that a sender omitted a region, so it classifies
as unknown and stays quiet. They are off by default, start in dry run, and are
fenced by min_unscoped_messages, a per-sender cooldown, a mesh-wide cooldown
and a daily cap on attempts. Dry run consumes the same budget it previews, so
the log is what going live would put on the air, not an upper bound. All three
limits read from the database, so a restart cannot release a burst.

Channel-delivered warnings go out at global scope on purpose: the recipient is
by definition outside any region the bot replies under.

New Settings -> Region Warnings page in the web viewer covers all of it. Its
dark-mode striped rows exposed a pre-existing base.html bug where Bootstrap's
light-theme text color survived on a dark row background (~1.3:1); fixed there
for every table in the app.
2026-09-15 22:34:46 -07:00
agessaman 681f0dcdcc fix(core): keep services alive during radio reconnects 2026-09-15 22:03:09 -07:00
agessaman 698f18dc6e fix(i18n): anchor the local catalog on local_dir_path and wire it everywhere
The local translation path defaulted to a bare `local/translations/`, resolved
against the process cwd and unrelated to `[Bot] local_dir_path`. Since
`local_dir_path` already selects where an operator's commands, service plugins
and config overlay live, the catalog belongs in that same tree. It now defaults
to `<local_dir_path>/translations`, resolved absolute against the bot root, so
relocating `local_dir_path` moves the catalog with it and the lookup no longer
depends on the working directory. An explicit `local_translation_path` still
wins.

Only the constructor at startup was passing the local path. `get_translator()`
builds and caches a translator per detected language, and `reload_config()`
builds a fresh one, and both were still constructing `Translator` with the
distributed path alone. So local overrides silently vanished from any reply that
used the sender's language, and did not survive a config reload. Both now pass
it, `reload_config()` republishes it alongside `translation_path`, and it is
snapshotted and restored on rollback. The DummyTranslator fallback sets it too,
since `get_translator()` would otherwise raise AttributeError there.

Collapsed `_deep_merge_translations` into the existing `_merge_translations`,
which already merged deeply with the primary winning. Rewrote that one to stop
mutating its input: it shallow-copied `fallback` and then recursed into the
shared sub-dicts, so merging a base language over English corrupted the cached
English catalog at every level below the first. Also replaced the two `print()`
calls on the error paths with logger calls.

Registered `local_translation_path` in the config schema next to
`translation_path`, and reverted a trailing-whitespace reflow that touched 84
lines of config.ini.example for a one-key addition.

Tests: local overlay (single-string override, added keys, local-only catalog,
missing directory), the merge no longer mutating its fallback, the default
following `local_dir_path`, an explicit setting overriding it, the overlay
reaching `bot.translator` end to end, and a reload picking up a moved path.
2026-09-07 20:28:45 -07:00
Gerard Hickey 9f807c9260 feat(i18n): Add support for local translation files
Add the `local_tranlation_path` to the `Localization` section  of
the configuration file to allow one to specify a local translation
file outside the distributed translation files. This allows
translation files to be built for local commands without the
need of altering the distributed ones.

Signed-off-by: Gerard Hickey <hickey@kinetic-compute.com>
2026-09-04 21:50:02 -04:00
agessaman d2b83481f7 feat(placeholders): expose {packet_hash} through the placeholder service
Adds the 16-char MeshCore packet identity hash to the three formatters that
render user-facing templates: [Keywords] responses, the test command's
response_format, and the path command's reply_prefix. It is read only from
routing_info, which is attached solely from an RF packet correlated to the
message, so a hash from an unrelated transmission is never shown. Missing,
empty, and the 0000000000000000 error sentinel all render empty.

get_packet_hash_placeholder returns "" rather than a False sentinel. Empty
gives byte-identical output on every path—prefix_if_nonempty already tests
falsiness and str.format renders "" as nothing—whereas False would print
the literal string "False" into a mesh reply from any consumer of
get_standard_placeholder_fields that formatted it directly.

TestCommand.format_response now builds on get_standard_placeholder_fields
instead of duplicating it. That also honors the extra argument its docstring
already promised but silently ignored, and picks up the path-string hop
fallback, so {hops} reads "0" rather than "?" for a message whose hops and
routing_info are both unset but whose path says "Direct (0 hops)".

Documents the placeholder in config.ini.example, the default config that
core.py writes on a fresh install, the README and docs/path-command-config.md.
The generated config was also missing {elapsed}; added that too.
2026-08-25 09:16:09 -07:00
agessaman 29a2e7b412 feat(web): manage scheduled messages from the viewer (#174)
Adds a Schedule page that lists every [Scheduled_Messages] entry with its
next run time and supports add, edit and delete. Writes go to config.ini
and queue a config reload, so schedules change without restarting the
bot, which was the actual request.

It edits the config section the bot already reads rather than
introducing a database table. reload_config() already re-runs
setup_scheduled_messages() with rollback, so there is nothing to keep in
sync and the schedule command lists exactly what the page shows.

Validation runs through the same parsers the scheduler uses, so the UI
cannot accept a schedule the bot would later reject. The builder
composes cron from plain-language options and previews the next five
runs; entries the bot cannot run are listed as "Not scheduled" with the
reason instead of being hidden, since a typo that silences a message is
what an operator most needs to see. The 15-minute floor for {cmd:...}
messages is enforced at save time too.

Also relabels the radio Disconnect button to "Stop Bot" behind a
confirmation (#240). It was never a radio-only disconnect: the main loop
runs while self.connected is true, so disconnecting exits the process.
That surprised an operator running under tmux with nothing to restart
it. disconnect_radio()'s docstring now says so as well.
2026-08-21 23:32:45 -07:00
agessaman f4587392db fix(config): update auto_manage_contacts default behavior and documentation
Changed the default setting for `auto_manage_contacts` from `false` to `device` across configuration files and updated related documentation. This ensures that the device handles auto-addition of contacts while the bot manages capacity. Adjusted comments and documentation to reflect this change for clarity and consistency.
2026-07-29 19:49:50 -07:00
agessaman 717f9a8ef3 feat(i18n): opt-in auto-detect sender language for replies
Reworked implementation of the multilingual-response feature (issue #218)
as a reusable, framework-level capability instead of logic bolted onto
HelloCommand.

- modules/lang_detector.py: keyword-first detection (reliable for the short
  greetings typical on mesh) with optional langdetect for longer text;
  results are constrained to languages that actually have a translation file.
- core.py: cache Translator instances by language (get_translator) so
  per-message switching never re-reads files, and expose available_languages()
  derived from translations/*.json. Store translation_path; keep cache/path
  consistent on config reload.
- base_command.py: task-local translator override via a ContextVar (safe under
  asyncio concurrency, no global mutation), plus detect_response_language() and
  a respond_in_sender_language() context manager any command can opt into.
- hello_command.py: build the reply inside respond_in_sender_language(); the
  only awaited call (send_response) stays outside the override window.
- Gated by [Localization] auto_detect_language (default false); langdetect is
  an optional [lang] extra, not a hard dependency.

Tests: unit coverage for the detector and the base-command wiring.
2026-07-28 12:44:53 -07:00
agessaman 1f0ec48448 Harden feed, config reload, and wx async paths 2026-07-16 15:40:20 -07:00