dandri/meshcore-bot
1
0 Fork 0
mirror of https://github.com/agessaman/meshcore-bot.git synced 2026-10-22 22:11:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update README, config example, and tracking files for v0.9.0

Browse Source 
- README: reflect all new features, installation steps, Python 3.9
  minimum requirement, and updated configuration reference
- config.ini.example: add [Aliases], [Webhook], [Rate_Limits],
  [Logging] json_logging, [Path_Command] geographic_scoring_enabled,
  [Web_Viewer] web_viewer_password, and all new scheduler/backup options
- BUGS.md, TESTING.md, TODO.md: updated to reflect current state
- scripts/update_todos.py: updated scan output
This commit is contained in:
Stacy Olivas committed 2026-03-17 17:45:10 -07:00
1 parent 4979687299
commit 6d9d01ee4f
6 files changed
+605 -129
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

BUGS.md
+22 -9
View File
@@ -6,10 +6,22 @@ Tracking of known bugs, fixed issues, and outstanding defects in meshcore-bot.
## Fixed Bugs
### v0.8.3 and recent history
### v0.9.0 and recent history
| Commit | Summary |
|--------|---------|
| (dev TASK-02 2026-03-15) | Fixed BUG-023: Realtime monitoring command stream blank on load — added 50-row history replay to `subscribe_commands` handler; fixed `last_timestamp = 0` → `time.time() - 300` in polling thread |
| (dev TASK-00 2026-03-15) | Fixed BUG-022: `IndexError` from meshcore parser silently discarded in asyncio task — installed custom loop exception handler in `core.py:start()` to suppress at DEBUG level |
| (dev) | Fixed BUG-020: Config TUI `[Scheduled_Messages]` keys showed `?` marker — dynamic sections (no fixed example keys) now suppress unknown-key marker |
| (dev) | Fixed BUG-021: Config TUI had no way to edit the time portion of a scheduled message — added `r` (rename key), `a` (add key+value), `d`/Delete (delete key with confirmation) bindings in keys pane |
| (dev) | Fixed BUG-019: `addPacketEntry` in `realtime.html` crashed with TypeError when `data.path` is a string — guarded with `Array.isArray()` |
| (dev) | Fixed BUG-018: Real-time monitoring SocketIO connections dropped due to Werkzeug 3.1 WebSocket teardown assertion (see BUG-012 fix); DB polling and subscription replay confirmed correct |
| (dev) | Fixed BUG-017: `disconnect_radio()` now uses `asyncio.wait_for(..., timeout=10)` — no longer hangs indefinitely |
| (dev) | Fixed BUG-016: `reboot_radio()` now sends `meshcore.commands.reboot()` firmware command before disconnecting/reconnecting |
| (dev) | Fixed BUG-015: scheduler thread blocked on `future.result(timeout=X)` causing `TimeoutError` spam and stalling the loop — replaced all four blocking waits with `add_done_callback` (fire-and-forget) in `run_scheduler` |
| `0ef4424` | Fixed BUG-001: web viewer now supports optional password authentication via `web_viewer_password` in `[Web_Viewer]` config |
| `0ef4424` | Fixed BUG-002: `db_manager` now runs ALTER TABLE migrations for `channel_operations` (`result_data`, `processed_at`) and `feed_message_queue` (`item_id`, `item_title`, `priority`) on startup |
| `0ef4424` | Fixed BUG-003: geocoding rate-limit skip in `repeater_manager` now logs at INFO level instead of DEBUG so it is visible in production logs |
| `1264f49` | Fixed repeater manager auto-purge ignoring `auto_manage_contacts` config — purge ran unconditionally regardless of setting (issue #50) |
| `1264f49` | Fixed web viewer responses returning stale or incorrect repeater data |
| `5c8ee35` | Fixed timezone handling in `format_elapsed_display` — elapsed times displayed incorrectly in non-UTC timezones (issue #75) |
@@ -32,19 +44,20 @@ Tracking of known bugs, fixed issues, and outstanding defects in meshcore-bot.
### High Priority
| ID | Module | Description | Workaround |
|----|--------|-------------|------------|
| BUG-001 | `web_viewer` | Web viewer has **no authentication** — exposes all contact data and packet history on the configured host/port | Set `host = 127.0.0.1` in `[Web_Viewer]` config; use firewall rules or SSH tunnel for remote access |
| BUG-002 | `db_manager` | Moving a database from an older install can cause `"no such column"` errors at startup if schema migrations haven't run yet | Start bot once to trigger migrations; if errors persist, clear stale `feed_message_queue` and `channel_operations` rows |
| BUG-003 | `repeater_manager` | Geocoding runs at most once per packet hash within a 60-second window, but concurrent adverts from the same node within that window skip geocoding silently | None; by design for rate limiting, but location may lag for rapid-fire adverts |
| ID | Task | Module | Description | Workaround |
|----|------|--------|-------------|------------|
| BUG-022 | TASK-00 | `core.py` / `meshcore_parser` | `IndexError: index out of range` in `meshcore/meshcore_parser.py:parsePacketPayload` surfaces as "Task exception was never retrieved" WARNING spam — malformed/truncated packet payload; no custom exception handler installed | None; fix: install `loop.set_exception_handler()` to suppress `IndexError`/`struct.error` at DEBUG level |
### Medium Priority
| ID | Module | Description | Workaround |
|----|--------|-------------|------------|
| ID | Task | Module | Description | Workaround |
|----|------|--------|-------------|------------|
| BUG-024 | TASK-05 | `scheduler.py` | DB backup scheduler fires every second after the 5-minute startup window — `last_db_backup_run` is never updated in `run_scheduler()` loop body after calling `_maybe_run_db_backup()` | None; fix: assign `self.last_db_backup_run = time.time()` after the call |
| BUG-025 | TASK-10 | `message_handler.py` | Channel message send fails with `{'reason': 'no_event_received'}` and is not retried — seen in logs as `❌ Channel message failed to #testing` | Restart bot; fix: wrap send in retry loop (max 2 retries, 2s delay) |
| BUG-026 | TASK-11 | `command_manager.py` | Help and long bot responses are cut off — chunking logic does not send all parts | None; fix: audit `split_message`/chunking; ensure all chunks sent sequentially |
| BUG-004 | `message_handler` | RF data correlation (SNR/RSSI) can miss messages if the RF log event arrives more than `rf_data_timeout` (default 15s) after the message | Increase `rf_data_timeout` in `[Bot]` config |
| BUG-005 | `scheduler` | On Raspberry Pi Zero 2 W, bot + web viewer together use ~300 MB RAM, leaving little headroom under load | Disable web viewer (`[Web_Viewer] enabled = false`) or tune mesh graph settings (`graph_startup_load_days = 7`) |
| BUG-006 | `feed_manager` | Stale rows in `feed_message_queue` from an old install can cause repeated scheduler errors after a database migration | Clear pending queue: `DELETE FROM feed_message_queue WHERE sent_at IS NULL` |
| BUG-006 | `feed_manager` | Stale rows in `feed_message_queue` from an old install can cause repeated queue-processing errors after a database migration (note: scheduler `TimeoutError` spam from the same area is fixed — see BUG-015) | Clear pending queue: `DELETE FROM feed_message_queue WHERE sent_at IS NULL` |
| BUG-007 | `discord_bridge_service` | Discord webhook rate limit is 30 requests/minute; bot warns at 20% exhaustion but does not queue excess messages — they are dropped | Keep bridged channels low-traffic; consider rate-limiting at mesh level |
| BUG-008 | `telegram_bridge_service` | Telegram `message_thread_id` (forum/topic support) is not implemented — messages go to the main group channel only | Manual: add thread ID mapping in a future plugin iteration |
README.md
+291 -32
View File
@@ -1,54 +1,75 @@
# MeshCore Bot
A Python bot that connects to MeshCore mesh networks via serial port, BLE, or TCP/IP. The bot responds to messages containing configured keywords, executes commands, and provides various data services including weather, solar conditions, and satellite pass information.
A Python bot that connects to MeshCore mesh networks via serial port, BLE, or TCP/IP. The bot responds to messages containing configured keywords, executes commands, and provides various data services including weather, solar conditions, and satellite pass information. A web viewer provides a browser-based dashboard for monitoring and managing the bot.
## Features
- **Connection Methods**: Serial port, BLE (Bluetooth Low Energy), or TCP/IP
- **Keyword Responses**: Configurable keyword-response pairs with template variables
- **Command System**: Plugin-based command architecture with built-in commands
- **Rate Limiting**: Global, per-user (by pubkey or name), and bot transmission rate limits to prevent spam
- **Command Aliases**: Define shorthand aliases for any command via `[Aliases]` config section
- **Rate Limiting**: Global, per-user (by pubkey or name), and per-channel rate limits to prevent spam
- **User Management**: Ban/unban users with persistent storage
- **Scheduled Messages**: Send messages at configured times
- **Direct Message Support**: Respond to private messages
- **Logging**: Console and file logging with configurable levels
- **Inbound Webhook**: Accept HTTP POST payloads and relay to MeshCore channels or DMs
- **Web Viewer**: Browser-based dashboard for monitoring contacts, mesh graph, radio settings, feeds, live packets, and logs
- **Radio Control**: Reboot or connect/disconnect the radio connection from the web viewer
- **Database Migrations**: Versioned schema migrations via `MigrationRunner` — safe upgrades across versions
- **DB Backup Scheduling**: Automated daily/weekly database backups with configurable retention
- **Nightly Maintenance Email**: Daily digest with uptime, network activity, DB stats, and error counts
- **Log Rotation**: Configurable log rotation via `[Logging]` section
- **Logging**: Console and file logging with configurable levels; optional structured JSON mode for log aggregation (Loki, Elasticsearch, Splunk)
### Service Plugins
- **Discord Bridge**: One-way webhook bridge to post mesh messages to Discord ([docs](docs/discord-bridge.md))
- **Telegram Bridge**: One-way bridge to post mesh messages to Telegram ([docs](docs/telegram-bridge.md))
- **Packet Capture**: Capture and publish packets to MQTT brokers ([docs](docs/packet-capture.md))
- **Map Uploader**: Upload node adverts to map.meshcore.dev ([docs](docs/map-uploader.md))
- **Weather Service**: Scheduled forecasts, alerts, and lightning detection ([docs](docs/weather-service.md))
- **Webhook Service**: Accept inbound HTTP POST payloads and relay to channels or DMs
## Requirements
- Python 3.7+
- Python 3.9+
- MeshCore-compatible device (Heltec V3, RAK Wireless, etc.)
- USB cable or BLE capability
## Installation
### Quick Start (Development)
1. Clone the repository:
```bash
git clone <repository-url>
cd meshcore-bot
```
2. Install dependencies:
2. Create a virtual environment and install dependencies via Makefile:
```bash
pip install -r requirements.txt
make dev # creates .venv, installs all deps including test tools
```
3. Copy and configure the bot:
Or for production dependencies only:
```bash
make install # creates .venv, installs runtime + optional deps
```
**Full Configuration Option**: This config.ini example enables all bot commands and provides full configuration options.
3. Configure the bot:
**Interactive TUI (recommended):** Launch the ncurses config editor — it reads an existing `config.ini` or lets you start from `config.ini.example`:
```bash
make config
```
**Manual option (full config):** Enables all bot commands and provides all configuration options:
```bash
cp config.ini.example config.ini
# Edit config.ini with your settings
```
**Minimal Configuration Option**: For users who only want core testing commands (ping, test, path, prefix, multitest), you can use the minimal configuration instead:
**Manual option (minimal config):** For users who only want core testing commands (ping, test, path, prefix, multitest):
```bash
cp config.ini.minimal-example config.ini
# Edit config.ini with your connection and bot settings
@@ -56,7 +77,15 @@ cp config.ini.minimal-example config.ini
4. Run the bot:
```bash
python3 meshcore_bot.py
.venv/bin/python meshcore_bot.py
```
5. Run tests and linting:
```bash
make test # pytest with coverage
make test-no-cov # pytest without coverage (faster)
make lint # ruff check + mypy
make fix # auto-fix ruff lint errors
```
### Production Installation (Systemd Service)
@@ -84,6 +113,17 @@ sudo systemctl status meshcore-bot
See [Service installation](docs/service-installation.md) for detailed service installation instructions.
### Debian Package (.deb)
Build and install a `.deb` package for Debian/Ubuntu systems:
```bash
make deb
sudo dpkg -i dist/meshcore-bot_*.deb
```
The package installs the bot to `/opt/meshcore-bot/`, installs a systemd unit, and creates a `meshcore-bot` system user.
### Docker Deployment
For containerized deployment using Docker:
@@ -98,25 +138,19 @@ For containerized deployment using Docker:
```ini
[Bot]
db_path = /data/databases/meshcore_bot.db
[Logging]
log_file = /data/logs/meshcore_bot.log
```
3. **Build and start with Docker Compose**:
```bash
docker compose build
docker compose up -d
```
Or build and start in one command:
```bash
docker compose up -d --build
```
4. **View logs**:
```bash
docker-compose logs -f
docker compose logs -f
```
See [Docker deployment](docs/docker.md) for detailed Docker deployment instructions, including serial port access, web viewer configuration, and troubleshooting.
@@ -144,9 +178,122 @@ And in your system config
}
```
## Web Viewer
The web viewer provides a browser-based dashboard for monitoring and managing the bot. Enable it in `config.ini`:
```ini
[Web_Viewer]
enabled = true
host = 0.0.0.0
port = 8080
web_viewer_password = yourpassword # optional; omit to disable auth
```
Features:
- **Contacts** — live contact list with signal, path, and location data; star any contact; purge inactive contacts by age threshold; export to CSV/JSON
- **Mesh Graph** — interactive node graph of the mesh network
- **Radio Settings** — manage channels, reboot or connect/disconnect the radio
- **Feeds** — manage RSS/API feed subscriptions per channel
- **Packets** — raw packet monitor
- **Live Activity** — real-time color-coded packet/command/message feed with pause and clear controls
- **Live Channel Messages** — real-time channel message monitor via SocketIO
- **Logs** — real-time log viewer at `/logs`; level-based coloring, filter, pause, and clear
### Configuration Tab
The `/config` page exposes bot settings in-browser — no `config.ini` edit required.
**Email & Notifications** — configure SMTP and opt in to a nightly maintenance digest:
| Field | Description |
|-------|-------------|
| Server hostname | SMTP host (e.g. `smtp.gmail.com`) |
| Port | 587 (STARTTLS), 465 (SSL), or 25 (plain) |
| Security | STARTTLS / SSL / None |
| Username / Password | SMTP credentials (app-specific passwords recommended) |
| Sender display name | Name shown in the From field |
| Sender email | Address shown in the From field |
| Recipients | Comma-separated list of addresses |
| Nightly email toggle | Enable / disable the nightly maintenance digest |
All settings are stored in the bot database (`bot_metadata` table) and take effect immediately. Use **Send test email** to verify SMTP settings before enabling the digest.
When **Nightly maintenance email** is enabled, the scheduler sends a digest every 24 hours:
```
MeshCore Bot — Nightly Maintenance Report
============================================
Period : 2026-03-14 06:00 UTC → 2026-03-15 06:00 UTC
BOT STATUS
──────────────────────────────
Uptime : 2d 4h 32m
Connected : yes
NETWORK ACTIVITY (past 24 h)
──────────────────────────────
Active contacts : 12
New contacts : 3
Total tracked : 47
DATABASE
──────────────────────────────
Size : 14.2 MB
Last retention run : 2026-03-15T06:00:00
ERRORS (past 24 h)
──────────────────────────────
ERROR : 2
CRITICAL : 0
LOG FILES
──────────────────────────────
Current : meshcore_bot.log (2.1 MB)
Rotated : no
```
**Log Rotation** — configure via the Config tab or `[Logging]` in `config.ini`:
```ini
[Logging]
log_max_bytes = 5242880 # 5 MB per file
log_backup_count = 3 # number of rotated files to keep
```
**Database Backup** — schedule automatic backups from the Config tab:
```ini
[Maintenance]
db_backup_enabled = true
db_backup_schedule = daily # daily | weekly | manual
db_backup_time = 02:00 # HH:MM local time
db_backup_retention_count = 7
db_backup_dir = /data/backups
```
The **Maintenance Status** card in the Config tab shows the last backup time, next scheduled run, and log rotation status.
### Radio Control
The Radio Settings page includes two control buttons next to the page heading:
- **Reboot Radio** — disconnects and reconnects the bot's radio connection (requires confirmation)
- **Connect / Disconnect** — toggles the bot's connection state; button color and label update live
These operations are queued via the database and processed by the bot's scheduler within ~5 seconds.
## Configuration
The bot uses `config.ini` for all settings. Key configuration sections:
The bot uses `config.ini` for all settings. The quickest way to create and edit `config.ini` is:
```bash
make config
```
This launches an interactive ncurses TUI that lets you browse sections, edit values, validate, and save. It can also migrate from `config.ini.example` if no `config.ini` exists. In the keys pane, `r` renames the selected key (useful for changing a scheduled-message time), `a` adds a new key+value, and `d`/Delete removes a key.
Key configuration sections:
### Connection
```ini
@@ -175,7 +322,9 @@ startup_advert = flood # Send advert on startup
```ini
[Keywords]
# Format: keyword = response_template
# Variables: {sender}, {connection_info}, {snr}, {timestamp}, {path}
# Variables: {sender}, {connection_info}, {snr}, {rssi}, {timestamp}, {path},
# {hops}, {hops_label}, {elapsed}, {path_distance}, {firstlast_distance},
# {total_contacts}, {total_repeaters}, {total_companions}, ...
test = "Message received from {sender} | {connection_info}"
help = "Bot Help: test, ping, help, hello, cmd, wx, aqi, sun, moon, solar, hfcond, satpass, dice, roll, joke, dadjoke, sports, channels, path, prefix, repeater, stats, alert"
```
@@ -189,6 +338,51 @@ respond_to_dms = true # Enable DM responses
# channel_keywords = help,ping,test,hello
```
### Per-Channel Rate Limiting
```ini
[Rate_Limits]
# Format: channel.<name>_seconds = <float>
# Overrides the global rate_limit_seconds for a specific channel.
channel.general_seconds = 5.0
channel.emergency_seconds = 0.0 # no rate limit on emergency channel
```
### Command Aliases
```ini
[Aliases]
# Format: alias = target_command
# Injects the alias string into the target command's keyword list.
w = wx
p = ping
```
### Inbound Webhook
```ini
[Webhook]
enabled = false
host = 0.0.0.0
port = 8765
secret_token = # Bearer token; leave blank to disable auth
max_message_length = 200
# allowed_channels = # Comma-separated allowlist; blank = all channels
```
Send a message to a channel:
```bash
curl -X POST http://localhost:8765/webhook \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"channel": "general", "message": "Hello from webhook!"}'
```
Send a DM:
```bash
curl -X POST http://localhost:8765/webhook \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"dm_to": "NodeName", "message": "Private message"}'
```
### External Data APIs
```ini
[External_Data]
@@ -211,8 +405,41 @@ agency.county.<county_name> = <agency_ids> # County-specific agency IDs (aggrega
```ini
[Logging]
log_level = INFO # DEBUG, INFO, WARNING, ERROR, CRITICAL
log_file = meshcore_bot.log # Log file path
log_file = meshcore_bot.log # Log file path (empty = console only)
colored_output = true # Enable colored console output
json_logging = false # Emit one JSON object per log line (for Loki/Elasticsearch/Splunk)
# When true, colored_output is ignored
log_max_bytes = 5242880 # Max log file size before rotation (bytes; default 5 MB)
log_backup_count = 3 # Number of rotated log files to keep
```
When `json_logging = true` each log line is a JSON object:
```json
{"timestamp":"2026-03-14T12:00:00.000Z","level":"INFO","logger":"MeshCoreBot","message":"Connected to radio"}
```
### Maintenance
```ini
[Maintenance]
db_backup_enabled = false
db_backup_schedule = daily # daily | weekly | manual
db_backup_time = 02:00 # HH:MM local time
db_backup_retention_count = 7
db_backup_dir = /data/backups
email_attach_log = false # attach current log file (≤ 5 MB) to nightly email before rotation
```
### Notifications
```ini
[Notifications]
email_enabled = false
smtp_host =
smtp_port = 587
smtp_user =
smtp_password =
smtp_from =
email_recipients = # comma-separated
email_send_time = 06:00 # nightly digest send time (HH:MM local)
```
## Usage
@@ -220,7 +447,12 @@ colored_output = true # Enable colored console output
### Running the Bot
```bash
python meshcore_bot.py
.venv/bin/python meshcore_bot.py
```
Or if installed as a package entry point:
```bash
.venv/bin/meshcore-bot
```
### Available Commands
@@ -235,17 +467,35 @@ Quick reference:
- **Entertainment:** `joke`, `dadjoke`, `hacker`, `catfact`
- **Sports:** `sports`
- **MeshCore Utility:** `path`, `prefix`, `stats`, `multitest`, `webviewer`
- **Management (DM only):** `repeater`, `advert`, `feed`, `announcements`, `greeter`
- **Management (DM only):** `repeater`, `advert`, `feed`, `announcements`, `greeter`, `schedule`
### `schedule` Command (DM only)
The `!schedule` command (DM only by default) shows all upcoming scheduled messages and the current advert interval:
```
Scheduled Messages (2 configured):
06:00 → #general: "Good morning, mesh!"
18:00 → #general: "Evening check-in"
Advert interval: every 30 min
```
## Message Response Templates
Keyword responses support these template variables:
- `{sender}` - Sender's node ID
- `{connection_info}` - Connection details (direct/routed)
- `{snr}` - Signal-to-noise ratio
- `{timestamp}` - Message timestamp
- `{connection_info}` - Connection details (path | SNR | RSSI)
- `{snr}` - Signal-to-noise ratio in dB
- `{rssi}` - Received signal strength in dBm
- `{timestamp}` - Current time (HH:MM:SS in configured timezone)
- `{elapsed}` - Time since message was sent
- `{path}` - Message routing path
- `{hops}` - Hop count (integer or `?`)
- `{hops_label}` - Hop count with label (`"1 hop"`, `"3 hops"`, `"?"`)
- `{path_distance}` - Estimated total path distance in km
- `{firstlast_distance}` - First-to-last repeater distance in km
- `{total_contacts}`, `{total_repeaters}`, `{total_companions}` - Mesh network counts (for scheduled messages)
### Adding Newlines
@@ -263,7 +513,7 @@ Line 2
Line 3
```
To use a literal backslash + n, use `\\n` (double backslash + n).
To use a literal backslash + n, use `\\n` (double backslash + n).
Other escape sequences: `\t` (tab), `\r` (carriage return), `\\` (literal backslash)
Example:
@@ -337,7 +587,8 @@ help = "Bot Help: test, ping, help, hello, cmd, wx, gwx, aqi, sun, moon, solar,
5. **Rate Limiting**:
- **Global**: `rate_limit_seconds` — minimum time between any two bot replies
- **Per-user**: `per_user_rate_limit_seconds` and `per_user_rate_limit_enabled` — minimum time between replies to the same user (user identified by public key when available, else sender name; channel senders often matched by name)
- **Per-user**: `per_user_rate_limit_seconds` and `per_user_rate_limit_enabled` — minimum time between replies to the same user (user identified by public key when available, else sender name)
- **Per-channel**: `[Rate_Limits] channel.<name>_seconds` — override rate limit for a specific channel
- **Bot TX**: `bot_tx_rate_limit_seconds` — minimum time between bot transmissions on the mesh
- Check logs for rate limiting messages
@@ -355,9 +606,12 @@ The bot uses a modular plugin architecture:
- **Core modules** (`modules/`): Shared utilities and core functionality
- **Command plugins** (`modules/commands/`): Individual command implementations
- **Service plugins** (`modules/service_plugins/`): Background services (Discord bridge, packet capture, etc.)
- **Service plugins** (`modules/service_plugins/`): Background services (Discord/Telegram bridges, webhook, packet capture, etc.)
- **Web viewer** (`modules/web_viewer/`): Flask + SocketIO browser dashboard
- **Plugin loaders**: Dynamic discovery and loading of command and service plugins
- **Message handler**: Processes incoming messages and routes to appropriate handlers
- **Scheduler**: Background thread for timed tasks; dispatches async ops into the bot's event loop
- **Database migrations**: `modules/db_migrations.py` — versioned `MigrationRunner` applied once on startup
### Adding New Plugins
@@ -383,9 +637,14 @@ class MyCommand(BaseCommand):
**Service Plugin:**
1. Create a new file in `modules/service_plugins/`
2. Inherit from `BaseServicePlugin`
3. Implement `start()` and `stop()` methods
4. Add configuration section to `config.ini.example`
2. Inherit from `BaseServicePlugin`, set `config_section = 'My_Section'`
3. Implement `async start()` and `async stop()` methods
4. Add `[My_Section] enabled = true` to `config.ini.example`
**Database Migration:**
1. Write `_mNNNN_short_desc(cursor)` in `modules/db_migrations.py`
2. Append `(NNNN, "description", _mNNNN_...)` to `MIGRATIONS`
3. Never modify or remove existing migrations — add a new one instead
## Contributing
TESTING.md
+39 -25
View File
@@ -7,25 +7,24 @@ Complete reference for the meshcore-bot test suite: how to run tests, what each
## Quick Start
```bash
# Create and activate a virtual environment (first time only)
python3 -m venv .venv
source .venv/bin/activate
# Install the package with test dependencies
pip install -e ".[test]"
# Create venv and install all dependencies (first time only)
make dev
# Run the full test suite
pytest
make test
# Run without coverage (faster)
pytest --no-cov
make test-no-cov
# Run a specific file
pytest tests/test_enums.py -v
.venv/bin/pytest tests/test_enums.py -v
# Run a specific test class or function
pytest tests/test_message_handler.py::TestShouldProcessMessage -v
pytest tests/test_enums.py::TestPayloadType::test_lookup_by_value -v
.venv/bin/pytest tests/test_message_handler.py::TestShouldProcessMessage -v
.venv/bin/pytest tests/test_enums.py::TestPayloadType::test_lookup_by_value -v
# Run and stop on first failure
.venv/bin/pytest -x
```
---
@@ -49,7 +48,7 @@ Test configuration lives in two files:
|---------|-------|
| `source` | `modules/` |
| `omit` | `tests/`, `.venv/` |
| `fail_under` | 70% |
| `fail_under` | 20% (structural ceiling; `web_viewer/app.py` at 2% without Flask test client tests) |
---
@@ -92,9 +91,6 @@ pytest
# HTML report — open htmlcov/index.html in a browser
pytest --cov=modules --cov-report=html
# Check that coverage meets the 70% threshold
pytest --cov=modules --cov-fail-under=70
# Coverage for a single module
pytest tests/test_message_handler.py --cov=modules.message_handler --cov-report=term-missing
```
@@ -103,17 +99,24 @@ pytest tests/test_message_handler.py --cov=modules.message_handler --cov-report=
## Linting (configured alongside tests)
```bash
pip install ruff mypy # one-time, or use pip install -e ".[test]" + pip install ruff mypy
All lint and type-check commands are available via the Makefile (preferred):
```bash
make lint # ruff check + mypy
make fix # auto-fix safe ruff issues
```
Or run directly:
```bash
# Check for style/lint issues
ruff check .
.venv/bin/ruff check modules/ tests/
# Auto-fix safe issues
ruff check . --fix
.venv/bin/ruff check --fix modules/ tests/
# Type checking
mypy modules/ --ignore-missing-imports
.venv/bin/mypy modules/
```
---
@@ -302,6 +305,14 @@ Tests `modules.utils` — utility functions.
| `TestParsePathString` | Comma/space/continuous hex, hop-count suffix, 4-char nodes, legacy fallback |
| `TestCalculatePacketHashPathLength` | Single/multi-byte path hashes, different sizes produce different hashes |
| `TestMultiBytePathDisplayContract` | Format contract for 1-byte and 2-byte node paths |
| `TestIsValidTimezone` | Valid IANA zones, invalid zone, empty/whitespace, leading/trailing whitespace |
| `TestGetConfigTimezone` | Valid zone returned, invalid falls back to UTC, empty falls back, logger warning |
| `TestFormatLocationForDisplay` | `None`/empty city, city-only, city+state, no duplicate parts, max_length respected |
| `TestGetMajorCityQueries` | Known city returns queries, unknown city returns empty, case-insensitive, multiple results |
| `TestResolvePath` | Absolute path unchanged, relative resolved to base_dir, Path objects, `"."` base uses cwd |
| `TestCheckInternetConnectivity` | Returns True on successful socket, False when all fail, HTTP fallback on socket failure |
| `TestCalculatePathDistances` | Empty/direct path, no db_manager, single node, two nodes with/without locations |
| `TestFormatKeywordResponseWithPlaceholders` | `{sender}`, `{hops_label}`, `{connection_info}`, `{total_contacts}`, no-message defaults, bad placeholder fallback |
---
@@ -534,12 +545,15 @@ class TestMyFeature:
## CI Integration
Tests run automatically on push/PR to `main`, `master`, and `dev` via `.github/workflows/test.yml`:
Tests run automatically on push/PR via GitHub Actions. Jobs include:
1. **Install deps** — `pip install -e ".[test]"` + `pip install ruff mypy`
2. **Lint** — `ruff check .`
3. **Type check** — `mypy modules/ --ignore-missing-imports`
4. **Tests** — `pytest tests/ -v --tb=short`
| Job | Command |
|-----|---------|
| `lint` | `ruff check modules/ tests/` |
| `typecheck` | `mypy modules/` |
| `lint-frontend` | ESLint + HTMLHint on `modules/web_viewer/templates/` |
| `lint-shell` | ShellCheck `--severity=warning` on all `.sh` files |
| `test` | `pytest tests/ -v --tb=short` with coverage |
To keep `TODO.md` in sync locally, run:
TODO.md
+130 -48
View File
@@ -3,23 +3,27 @@
Task list for meshcore-bot development. Auto-updated sections are regenerated
by running `python scripts/update_todos.py` (see [Auto-Update](#auto-update)).
**Last updated:** 2026-03-14
**Last updated:** 2026-03-15 (rev 3)
---
## In Progress
- [ ] Expand test coverage to ≥70% across all modules (Phase 5 of dev setup plan)
- [x] `tests/test_enums.py` — enum values and flag combinations
- [x] `tests/test_models.py` — MeshMessage dataclass
- [x] `tests/test_transmission_tracker.py` — full TransmissionTracker
- [x] `tests/test_message_handler.py` — path parsing, cache, message routing
- [x] `tests/test_repeater_manager.py` — role detection, ACL, device type
- [x] `tests/test_core.py` — config loading, radio settings, reload
- [ ] `tests/test_feed_manager.py` — feed polling, deduplication, queue logic
- [ ] `tests/test_scheduler.py` — scheduled message dispatch, interval advertising
- [ ] `tests/test_command_manager.py` — full command dispatch, keyword matching
- [ ] `tests/test_channel_manager.py` — channel fetch, cache lifecycle
- [ ] Expand test coverage to ≥35% (currently **27.62%**, 1,129 passed / 29 skipped; `fail_under=27`; hardware-dependent modules cap realistic ceiling at ~35–40%)
- [x] (2026-03-15) `tests/test_enums.py` — enum values and flag combinations
- [x] (2026-03-15) `tests/test_models.py` — MeshMessage dataclass
- [x] (2026-03-15) `tests/test_transmission_tracker.py` — full TransmissionTracker
- [x] (2026-03-15) `tests/test_message_handler.py` — path parsing, cache, message routing
- [x] (2026-03-15) `tests/test_repeater_manager.py` — role detection, ACL, device type
- [x] (2026-03-15) `tests/test_core.py` — config loading, radio settings, reload
- [x] (2026-03-15) `tests/test_feed_manager.py` — queue insert, deduplication via feed_activity, interval due-check
- [x] (2026-03-15) `tests/test_scheduler_logic.py` — scheduled message dispatch, interval advertising setup
- [x] (2026-03-15) `tests/test_command_manager.py` — full command dispatch, keyword matching
- [x] (2026-03-15) `tests/test_channel_manager_logic.py` — cache lifecycle, fetch-all, sorted cache, connectivity guard
- [ ] `tests/test_path_geo_toggle.py` — `!path` geographic scoring toggle (in progress)
- [ ] `tests/test_utils_geocoding.py` — geocoding helpers (in progress)
- [ ] `tests/test_web_viewer.py` — web viewer routes (in progress)
- [ ] Next targets: `security_utils.py`, `web_viewer/integration.py`, remaining `app.py` routes
---
@@ -34,62 +38,138 @@ by running `python scripts/update_todos.py` (see [Auto-Update](#auto-update)).
### Web Viewer
- [ ] **Authentication** — add token or basic-auth to protect the web dashboard (see BUG-001)
- [ ] **Live packet streaming** — SSE/WebSocket stream of incoming packets on dashboard
- [ ] **Real-time message monitoring** — live channel message feed in browser
- [ ] **Interactive contact management** — star/purge contacts from the UI
- [ ] **Export functionality** — CSV/JSON export of contact and path data
- [x] (2026-03-15) **Authentication** — `web_viewer_password` in `[Web_Viewer]`; login page + session auth + SocketIO guard
- [x] (2026-03-15) **Radio reboot button** — disconnect + reconnect bot-to-radio from web UI (confirmation modal, operation queue)
- [x] (2026-03-15) **Radio connect/disconnect button** — toggle bot connection from web UI (live status polling via `bot_metadata`)
- [x] (2026-03-15) **Live packet streaming** — Live Activity panel on dashboard; SocketIO packet/command/message feed; pause/clear
- [x] (2026-03-15) **Real-time message monitoring** — `capture_channel_message()` → `packet_stream`; `message_data` SocketIO event; Live Channel Messages panel
- [x] (2026-03-15) **Interactive contact management** — star any contact type; Purge Inactive modal with threshold selector + preview
- [x] (2026-03-15) **Export functionality** — `GET /api/export/contacts` and `/api/export/paths`; CSV/JSON with time-range; Export dropdown in toolbar
- [x] (2026-03-15) **Configuration tab** — `/config` page; SMTP + nightly email toggle; log rotation; DB backup; stored in `bot_metadata`
- [x] (2026-03-15) **Real-time log viewer** — `/logs` page; SocketIO `subscribe_logs`/`log_line`; level-based coloring; pause/clear/filter; log tail thread; "Logs" nav link
- [ ] **Mobile-responsive improvements** — optimize layout for small screens
- [x] (TASK-01 2026-03-15) **Remove firmware config + reboot UI** — radio.html: Firmware Configuration card and Reboot Radio button removed; JS handlers removed; 4 tests added
- [x] (TASK-02 2026-03-15) **Fix realtime stream blank on load** — added 50-row history replay to `subscribe_commands`; fixed `last_timestamp = 0` → `time.time() - 300` in polling thread; 5 tests added (BUG-023 fixed)
- [ ] (TASK-03) **Dashboard: connected agents popup** — click connected-clients count → modal with agent list; `GET /api/connected_clients` ⏸ paused 2026-03-15 20:10 — see SESSION_RESUME.md
- [ ] (TASK-04) **DB backup dir validation on save** — reject `POST /api/config/maintenance` if backup directory does not exist
- [ ] (TASK-06) **DB backup: Backup Now button** — `POST /api/maintenance/backup_now`; spinner + toast in Config tab
- [ ] (TASK-07) **DB backup: Restore button** — `POST /api/maintenance/restore`; modal with path input; lists backups from backup dir
- [ ] (TASK-08) **Database Operations: purge by age** — `POST /api/maintenance/purge`; keep all/1/7/14/30/60/90 days; confirmation dialog
- [ ] (TASK-12) **Dashboard live activity controls** — scroll top/bottom buttons; type-filter checkboxes; `[#channel]` prefix on messages
- [ ] (TASK-13) **Realtime page scroll/filter** — scroll top/bottom on each stream panel; `[#channel] message` format
### Maintenance and Notifications
- [x] (2026-03-15) **Log rotation configuration** — `log_max_bytes`/`log_backup_count` in `[Logging]`; Config tab Log Rotation card; live-apply via scheduler
- [x] (2026-03-15) **Nightly maintenance email dispatch** — digest every 24h; uptime, contact counts, DB size, log error counts, rotation detection
- [x] (2026-03-15) **Pre-rotation email hook** — `maint.email_attach_log = true` attaches log file (≤ 5 MB) to nightly email
- [x] (2026-03-15) **DB backup scheduling** — `sqlite3.Connection.backup()`; daily/weekly/manual; retention pruning; Config tab Database Backup card
- [x] (2026-03-15) **Maintenance task status API** — `GET /api/maintenance/status`; Maintenance Status card in Config tab
### Commands and Features
- [ ] **Inbound webhook command** — trigger bot responses from external HTTP POST
- [ ] **Per-channel rate limiting** — different rate limits for different channels
- [ ] **Command aliases** — allow users to define shorthand aliases for commands
- [ ] **Scheduled message preview** — `!schedule list` command to show upcoming scheduled messages
- [x] (2026-03-15) **Inbound webhook** — `POST /webhook` relays HTTP payloads to MeshCore channels/DMs; bearer token auth
- [x] (2026-03-15) **Per-channel rate limiting** — `ChannelRateLimiter` in `rate_limiter.py`; `[Rate_Limits] channel.<name>_seconds`; checked in `_check_rate_limits(channel=)`
- [x] (2026-03-15) **Command aliases** — `[Aliases]` config section injects shorthands into command keywords
- [x] (2026-03-15) **Scheduled message preview** — `!schedule` command (DM-only); shows times, channels, message previews, advert interval
- [ ] **`!wx` non-US improvement** — promote `wx_international.py` to default with US fallback
- [ ] **`!path` geographic scoring toggle** — runtime config flag without restart
- [ ] (TASK-11) **Fix help + long response truncation** — audit chunking logic; ensure all parts sent (BUG-026)
- [x] (2026-03-15) **`!path` geographic scoring toggle** — `[Path_Command] geographic_scoring_enabled = true/false` config flag; no restart required
### Infrastructure
- [ ] **Virtual environment setup automation** — `make install` / `make dev` targets
- [ ] **`ruff check` CI gate** — block merges when linting fails (test.yml updated; needs clean pass)
- [ ] **`mypy` strict mode** — incrementally add type annotations and raise strictness
- [ ] **Coverage threshold enforcement** — currently set to 70%; raise to 80% once baseline is met
- [ ] **Database migration versioning** — replace ad-hoc `ALTER TABLE IF NOT EXISTS` with numbered migrations
- [ ] **Docker multi-arch build** — publish `linux/arm64` image for Raspberry Pi natively
- [x] (2026-03-15) **Virtual environment / Makefile** — `make install/dev/test/test-no-cov/lint/fix/deb/config/clean`
- [x] (2026-03-15) **`ruff check` CI gate** — `lint` job in CI; 9262 auto-fixed, legacy patterns in ignore list
- [x] (2026-03-15) **`mypy` strict mode** — incremental: global safe options + per-module `disallow_untyped_defs`; `typecheck` CI job
- [x] (2026-03-15) **HTML/JS test framework** — `package.json` + ESLint (`eslint-plugin-html`) + HTMLHint; `lint-frontend` CI job
- [x] (2026-03-15) **ShellCheck CI gate** — `lint-shell` job checks all `.sh` files at `--severity=warning`
- [x] (2026-03-15) **Database migration versioning** — `MigrationRunner`; 5 numbered migrations; `schema_version` table
- [x] (2026-03-15) **Docker multi-arch build** — `linux/amd64` + `linux/arm64` + `linux/arm/v7`; SBOM + provenance
- [x] (2026-03-15) **Structured JSON logging** — `json_logging = true` in `[Logging]`; `_JsonFormatter`; Loki/ES/Splunk compatible
- [x] (2026-03-15) **aiosqlite async DB** — `AsyncDBManager` in `db_manager.py`; `bot.async_db_manager` in core; `aiosqlite>=0.19.0` dep
- [x] (2026-03-15) **.deb packaging** — `scripts/build-deb.sh`; `DEBIAN/control/postinst/prerm/postrm`; systemd unit; `make deb`
- [x] (2026-03-15) **ncurses config TUI** — `scripts/config_tui.py`; browse/edit/save; validate; migrate from example; `make config`; `r` rename key, `a` add key, `d`/Delete remove key; dynamic sections suppress `?` marker
- [x] (2026-03-15) **APScheduler migration** — `BackgroundScheduler` + `CronTrigger`; replaces `schedule` lib
- [x] (2026-03-15) **Rate-limiter observability** — `GET /api/stats/rate_limiters`; exposes stats for all 4 limiter types
- [x] (2026-03-15) **Map uploader configurable interval** — `min_reupload_interval` in config (fallback 3600 s)
- [x] (2026-03-15) **Per-channel greeter messages** — `channel_greetings`/`per_channel_greetings` config keys
- [x] (2026-03-15) **Radio firmware config UI** — Migration 6 (`payload_data`); `firmware_read`/`firmware_write` op types; `POST /api/radio/firmware/config/read|write`; Firmware Configuration card in web UI
- [x] (2026-03-15) **Werkzeug WebSocket fix** — `_apply_werkzeug_websocket_fix()` patches `SimpleWebSocketWSGI.__call__` at import time; 5 tests
- [x] (2026-03-15) **pytest-timeout runaway prevention** — `pytest-timeout>=2.1.0`; `timeout=30` per test; `asyncio_mode="auto"`
- [x] (2026-03-15) **SMTP timeout** — `SMTP`/`SMTP_SSL` constructed with `timeout=30`; nightly email never hangs
- [x] (2026-03-15) **Real-time monitoring history replay** — `subscribe_packets`/`subscribe_messages`/`subscribe_logs` replay last 50/50/200 items on connect
- [ ] **Coverage threshold enforcement** — `fail_under=27` (current); raise to 30 once 30% confirmed; target 40% (TASK-14)
- [ ] (TASK-09) **Message processing performance** — batch `packet_stream` inserts; reduce per-packet `sqlite3.connect()` round-trips
- [ ] (TASK-05) **Fix DB backup scheduler interval guard** — `last_db_backup_run` never updated (BUG-024)
- [ ] (TASK-00) **Fix meshcore IndexError crash** — asyncio exception handler for `IndexError` from meshcore parser (BUG-022) ⏸ paused 2026-03-15 19:19 — see SESSION_RESUME.md
- [ ] (TASK-10) **Retry `no_event_received` channel sends** — up to 2 retries with 2s delay (BUG-025)
- [x] (TASK-INFRA 2026-03-15) **Context checkpoint system** — `scripts/context_checkpoint.sh`, `scripts/post_tool_counter.sh`, `.claude/hooks.json`; cron every 15 min
---
## Backlog
- [ ] Investigate replacing `schedule` library with `APScheduler` for more flexible recurring jobs
- [ ] Evaluate moving web viewer to a separate installable package
- [ ] Add structured logging (JSON) mode for log aggregation pipelines
- [ ] Rate-limiter observability — expose current limiter state via web viewer API
- [ ] Consider async-safe SQLite wrapper (e.g. `aiosqlite`) for `db_manager`
- [ ] Repeater auto-purge dry-run mode — log what would be purged without acting
- [ ] Feed manager: add support for JSON API feeds (not just RSS/Atom)
- [ ] Map uploader: configurable upload interval (currently hardcoded)
- [ ] Support custom greeting messages per-channel (greeter command)
- [ ] Mobile-responsive web viewer improvements — optimize layout for small screens
---
## Recently Completed
- [x] Added `ruff` and `mypy` linting configuration to `pyproject.toml`
- [x] Added lint steps to `.github/workflows/test.yml`
- [x] Fixed `meshcore>=2.2.14` → `meshcore>=2.2.31` version pin in `pyproject.toml`
- [x] Added `profanity`, `geo`, `pytest-mock`, `pytest-cov` optional dependency groups
- [x] Added coverage reporting to `pytest.ini`
- [x] Created 6 new test modules (enums, models, transmission_tracker, message_handler, repeater_manager, core)
- [x] Fixed `RepeaterManager` ignoring `auto_manage_contacts = false` (commit `58deb12`)
- [x] Fixed timezone handling in `format_elapsed_display` — issue #75 (commit `5c8ee35`)
- [x] Fixed `TraceCommand` reversed path nodes and truncated return paths
- [x] Fixed shutdown log spam after streams closed (commit `2178a80`)
- [x] Added Discord and Telegram one-way bridges
- [x] Added chunked message sending for rate-limit-aware large responses
- [x] Multi-byte prefix (2-byte) support throughout codebase
- [x] (2026-03-15) Radio firmware config UI — Migration 6 (`payload_data`); `firmware_read`/`firmware_write` op types; `POST /api/radio/firmware/config/read|write`; Firmware Configuration card (path.hash.mode + loop.detect)
- [x] (2026-03-15) APScheduler migration — `BackgroundScheduler` + `CronTrigger`; removes `schedule` lib dependency
- [x] (2026-03-15) Rate-limiter observability — `GET /api/stats/rate_limiters`; all 4 limiter types exposed
- [x] (2026-03-15) Map uploader configurable interval — `min_reupload_interval` config key (fallback 3600 s)
- [x] (2026-03-15) Per-channel greeter messages — `channel_greetings`/`per_channel_greetings` config keys
- [x] (2026-03-15) `!path` geographic scoring toggle — `[Path_Command] geographic_scoring_enabled = true/false`; tests in `test_path_geo_toggle.py`
- [x] (2026-03-15) Real-time monitoring history replay — last 50/50/200 items replayed on `subscribe_packets`/`subscribe_messages`/`subscribe_logs`
- [x] (2026-03-15) Werkzeug WebSocket fix — `_apply_werkzeug_websocket_fix()` patches `SimpleWebSocketWSGI.__call__`; 5 tests
- [x] (2026-03-15) Radio reboot firmware command — sends `meshcore.commands.reboot()` before disconnect; 8 s wait; 10 s disconnect timeout
- [x] (2026-03-15) pytest-timeout — `pytest-timeout>=2.1.0`; `timeout=30` per test; `asyncio_mode="auto"`
- [x] (2026-03-15) SMTP timeout — `timeout=30` on all `SMTP`/`SMTP_SSL` constructors; nightly email no longer hangs
- [x] (2026-03-15) Per-channel rate limiting — `ChannelRateLimiter`; `[Rate_Limits]` config section; integrated into `_check_rate_limits` and `send_channel_message`
- [x] (2026-03-15) Real-time log viewer — `/logs` page; SocketIO `subscribe_logs`/`log_line`; log tail background thread; "Logs" nav link; toggle from `/realtime`
- [x] (2026-03-15) HTML/JS test framework — `package.json`, ESLint + `eslint-plugin-html`, HTMLHint; `lint-frontend` CI job
- [x] (2026-03-15) ShellCheck CI gate — `lint-shell` job; all `.sh` files checked at `--severity=warning`
- [x] (2026-03-15) .deb packaging — `scripts/build-deb.sh`; DEBIAN control/postinst/prerm/postrm; systemd unit; `make deb`
- [x] (2026-03-15) aiosqlite `AsyncDBManager` — `db_manager.py`; `aiosqlite>=0.19.0`; `bot.async_db_manager` in core
- [x] (2026-03-15) ncurses config TUI — `scripts/config_tui.py`; read/create/edit/save/validate/migrate; `make config`; `r`/`a`/`d` key management; dynamic-section `?` fix
- [x] (2026-03-15) Makefile — added `make deb` and `make config` targets; `clean` now removes `dist/deb-build/`
- [x] (2026-03-15) .gitignore — added `node_modules/`, `.npm`, `package-lock.json`, `dist/deb-build/`
- [x] (2026-03-15) Export functionality — `GET /api/export/contacts` + `/api/export/paths`; CSV/JSON with time-range; Export dropdown in contacts.html
- [x] (2026-03-15) Live packet streaming — Live Activity panel in `index.html`; SocketIO color-coded feed with pause/clear
- [x] (2026-03-15) Real-time message monitoring — `capture_channel_message()` → `packet_stream`; `message_data` SocketIO event; Live Channel Messages panel
- [x] (2026-03-15) Maintenance status API — `GET /api/maintenance/status`; Maintenance Status card in Config tab
- [x] (2026-03-15) DB backup scheduling — `scheduler._run_db_backup()`; daily/weekly/manual; retention pruning; Config tab card; status in `maint.status.*`
- [x] (2026-03-15) Pre-rotation email hook — `maint.email_attach_log = true` attaches log file (≤ 5 MB) to nightly digest
- [x] (2026-03-15) Log rotation configuration — `log_max_bytes`/`log_backup_count` in `[Logging]`; Config tab card; live-apply via scheduler
- [x] (2026-03-15) Nightly email dispatch — `_send_nightly_email()` every 24h; uptime, contacts, DB size, log errors; STARTTLS/SSL/plain
- [x] (2026-03-15) Configuration tab — `/config` page; `GET/POST /api/config/notifications`; SMTP settings stored as `notif.*` in `bot_metadata`
- [x] (2026-03-15) Interactive contact management — star all contact types; `GET /api/contacts/purge-preview` + `POST /api/contacts/purge`; Purge Inactive modal
- [x] (2026-03-15) Structured JSON logging — `json_logging = true`; `_JsonFormatter`; Loki/Elasticsearch/Splunk compatible
- [x] (2026-03-15) Radio connect/disconnect button — `GET /api/radio/status`; `POST /api/radio/connect`; live status from `bot_metadata`
- [x] (2026-03-15) Radio reboot button — `POST /api/radio/reboot` queues `radio_reboot` op; scheduler calls `reconnect_radio()`
- [x] (2026-03-15) Docker multi-arch — QEMU; `linux/amd64` + `linux/arm64` + `linux/arm/v7`; SBOM + provenance
- [x] (2026-03-15) mypy incremental strict mode — global safe options + per-module `disallow_untyped_defs`; `typecheck` CI job
- [x] (2026-03-15) ruff CI gate — clean pass; `lint` CI job; 9262 auto-fixed
- [x] (2026-03-15) Database migration versioning — `MigrationRunner`; 5 numbered migrations; `schema_version` table
- [x] (2026-03-15) Command aliases — `[Aliases]` config section injects shorthands into keywords
- [x] (2026-03-15) Inbound webhook service — `POST /webhook`; bearer token auth; relay to channel or DM
- [x] (2026-03-15) Makefile — `make install/dev/test/test-no-cov/lint/fix/clean`
- [x] (2026-03-15) Fixed BUG-001 web viewer authentication (Flask session auth, login/logout, SocketIO guard)
- [x] (2026-03-15) Fixed BUG-002 DB migration missing columns (channel_operations, feed_message_queue)
- [x] (2026-03-15) Fixed BUG-003 geocoding rate-limit skip logged at INFO with full context
- [x] (2026-03-15) Fixed `RepeaterManager` ignoring `auto_manage_contacts = false`
- [x] (2026-03-15) Fixed timezone handling in `format_elapsed_display` (issue #75)
- [x] (2026-03-15) Fixed `TraceCommand` reversed path nodes and truncated return paths
- [x] (2026-03-15) Fixed shutdown log spam after streams closed
- [x] (2026-03-15) Added Discord and Telegram one-way bridges
- [x] (2026-03-15) Added chunked message sending for rate-limit-aware large responses
- [x] (2026-03-15) Multi-byte prefix (2-byte) support throughout codebase
- [x] (2026-03-15) Added `ScheduleCommand` — lists scheduled messages and advert interval (DM-only by default)
- [x] (2026-03-15) Created 10 test modules covering enums, models, transmission_tracker, message_handler, repeater_manager, core, feed_manager, scheduler_logic, command_manager, channel_manager_logic
---
@@ -102,6 +182,8 @@ The **Inline TODOs** section below is auto-generated by scanning source files fo
python scripts/update_todos.py
```
The script also updates the `**Last updated:**` date at the top of this file.
Or run it as part of a pre-commit hook by adding to `.pre-commit-config.yaml`:
```yaml
@@ -118,6 +200,6 @@ Or run it as part of a pre-commit hook by adding to `.pre-commit-config.yaml`:
## Inline TODOs (auto-generated)
> _Last scanned: 2026-03-14. No `# TODO`, `# FIXME`, or `# HACK` markers
> _Last scanned: 2026-03-15. No `# TODO`, `# FIXME`, or `# HACK` markers
> found in `modules/` or `tests/`. Run `python scripts/update_todos.py` to refresh._
config.ini.example
+103 -7
View File
@@ -378,6 +378,35 @@ prefix.funfact = 💡
#1200 = Public:Midday status check - all systems operational.
#1800 = Public:Evening update - bot status: Good
[Schedule_Command]
# Show scheduled messages and advert interval via the 'schedule' command
# Responds with a compact list of scheduled times, channels, and message previews
# Default: enabled, DM-only (exposes bot configuration — restrict access)
enabled = true
# dm_only = true: only respond to direct messages (recommended)
# dm_only = false: allow in monitored channels
dm_only = true
# ---------------------------------------------------------------------------
# Command aliases
# ---------------------------------------------------------------------------
# Map short aliases to canonical command names.
# Format: <alias> = <command_name>
# Both the alias and command name are case-insensitive.
# The alias is injected into the command's keyword list at startup, so it
# responds exactly the same as typing the full command name.
# Unknown aliases (command not loaded) are logged and silently ignored.
#
# Examples:
# s = schedule
# wx = weather
# h = help
# e = earthquake
#
# [Aliases]
# s = schedule
# wx = weather
[Logging]
# Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
# DEBUG: Most verbose, shows all details
@@ -402,6 +431,18 @@ colored_output = true
# Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
meshcore_log_level = INFO
# Structured JSON logging mode for log aggregation pipelines (Loki, Elasticsearch, Splunk, etc.)
# When enabled, each log line is a JSON object:
# {"timestamp":"2026-03-14T12:00:00.000Z","level":"INFO","logger":"MeshCoreBot","message":"..."}
# colored_output is ignored when json_logging = true.
json_logging = false
# Log rotation settings (can also be changed at runtime via the web viewer Configuration tab)
# log_max_bytes: rotate when the log file exceeds this size (default 5 MB = 5242880)
log_max_bytes = 5242880
# log_backup_count: number of rotated backup files to keep (e.g. meshcore_bot.log.1 … .3)
log_backup_count = 3
[Custom_Syntax]
# Custom syntax patterns for special message formats
# Format: pattern = "response_format"
@@ -1201,22 +1242,25 @@ enabled = false
[Web_Viewer]
# Enable or disable the web data viewer
# SECURITY NOTE: Web viewer has NO AUTHENTICATION built-in
# Enable web viewer (true/false). SECURITY: only enable if you understand implications; false recommended for production
# Enable web viewer (true/false). false recommended unless you have configured web_viewer_password.
enabled = false
# Password to protect the web viewer UI (BUG-001 fix)
# If set, a login page is shown and all routes + SocketIO connections require authentication.
# If left empty, the web viewer is accessible without any authentication.
# Recommendation: always set a password when host != 127.0.0.1
# web_viewer_password = changeme
# Web viewer host address
# SECURITY WARNING:
# - 127.0.0.1: Only accessible from localhost (SECURE - recommended)
# - 0.0.0.0: Accessible from any network interface (INSECURE - exposes all bot data to network)
#
# Using 0.0.0.0 will expose:
# - 0.0.0.0: Accessible from any network interface (use only with web_viewer_password set)
#
# Using 0.0.0.0 without web_viewer_password will expose:
# - All messages and their contents
# - Contact list and repeater information
# - Real-time packet stream
# - Bot configuration details
#
# WITHOUT ANY AUTHENTICATION OR ENCRYPTION
host = 127.0.0.1
# Web viewer port
@@ -1490,6 +1534,58 @@ maxlongitude = -114.0
# Default: true
send_link = true
[Rate_Limits]
# Per-channel rate limiting: minimum seconds between bot messages on a specific channel.
# Format: channel.<channel_name>_seconds = <seconds>
# Channels without an explicit entry are unrestricted.
# Example: limit "BotCmds" channel to one bot response every 15 seconds:
#channel.BotCmds_seconds = 15
# Example: limit the default channel to 10 seconds between responses:
#channel.BroadcastCh_seconds = 10
[Webhook]
# Inbound webhook receiver — accept HTTP POST requests and relay them as MeshCore messages.
# Useful for integrating external systems (alerts, monitoring, scripts) with the mesh.
#
# Enable the webhook HTTP server (true/false). Default: false
enabled = false
# Bind address. Use 127.0.0.1 to accept connections only from localhost (recommended).
# Use 0.0.0.0 to accept from any interface (ensure your firewall restricts access).
host = 127.0.0.1
# Listen port (must not conflict with the web viewer port, default 8080).
port = 8765
# Optional shared secret. If set, every request must include one of:
# Authorization: Bearer <secret_token>
# X-Webhook-Token: <secret_token>
# Leave empty to disable authentication (NOT recommended for production).
# secret_token = changeme
# Maximum message length in characters (excess is silently truncated). Default: 200
# max_message_length = 200
# Comma-separated channel whitelist. If set, only these channels may be posted to.
# Leave empty to allow any channel.
# allowed_channels = general,alerts
# ---------------------------------------------------------------------------
# HTTP API
# ---------------------------------------------------------------------------
# POST http://<host>:<port>/webhook
# Content-Type: application/json
# Authorization: Bearer <secret_token>
#
# Channel message:
# {"channel": "general", "message": "Hello from webhook!"}
#
# Direct message:
# {"dm_to": "SomeUser", "message": "Private message"}
#
# Response: {"ok": true} or {"error": "..."}
# ---------------------------------------------------------------------------
[DiscordBridge]
# Enable Discord bridge service
# Enable Discord bridge (true/false). One-way, read-only webhooks
scripts/update_todos.py
+20 -8
View File
@@ -1,15 +1,20 @@
#!/usr/bin/env python3
"""
update_todos.py — Scans source files for TODO/FIXME/HACK markers and rewrites
the "Inline TODOs" section of TODO.md.
the "Inline TODOs" section of TODO.md. Also updates the "Last updated:" date
at the top of the file.
Usage:
python scripts/update_todos.py # from project root
python scripts/update_todos.py --check # exit 1 if TODO.md would change (CI use)
The script only touches the content between the two sentinel comments:
<!-- inline-todos-start -->
<!-- inline-todos-end -->
Completed item date format in TODO.md:
- [x] (YYYY-MM-DD) description of completed item
The script manages two things in TODO.md:
1. The "**Last updated:**" line near the top — set to today's date.
2. The "## Inline TODOs (auto-generated)" section at the bottom — replaced
wholesale with a fresh scan of # TODO / # FIXME / # HACK markers.
Everything else in TODO.md is left exactly as-is.
"""
@@ -94,9 +99,9 @@ def build_section(todos, today: str) -> str:
return "\n".join(lines)
def rewrite_todo_md(new_section: str, check_only: bool = False) -> bool:
def rewrite_todo_md(new_section: str, today: str, check_only: bool = False) -> bool:
"""
Replace the Inline TODOs section in TODO.md.
Replace the Inline TODOs section and update the Last updated date in TODO.md.
Returns True if the file was (or would be) changed.
"""
@@ -106,6 +111,13 @@ def rewrite_todo_md(new_section: str, check_only: bool = False) -> bool:
content = TODO_FILE.read_text(encoding="utf-8")
# Update "**Last updated:**" line
content = re.sub(
r"(\*\*Last updated:\*\*\s*)[\d-]+",
rf"\g<1>{today}",
content,
)
# Find the section heading and replace everything from it to EOF
idx = content.find(f"\n{SECTION_START}")
if idx == -1:
@@ -114,7 +126,7 @@ def rewrite_todo_md(new_section: str, check_only: bool = False) -> bool:
else:
new_content = content[: idx + 1] + new_section + "\n"
changed = new_content != content
changed = new_content != TODO_FILE.read_text(encoding="utf-8")
if check_only:
if changed:
@@ -142,7 +154,7 @@ def main():
today = datetime.date.today().isoformat()
todos = scan_todos()
section = build_section(todos, today)
changed = rewrite_todo_md(section, check_only=args.check)
changed = rewrite_todo_md(section, today, check_only=args.check)
if args.check and changed:
sys.exit(1)