meshcore-bot/docs/packet-capture.md

Packet Capture Service

Captures packets from the MeshCore network and publishes them to MQTT brokers.

---

Quick Start

1. Configure Bot - Edit config.ini:

[PacketCapture]
enabled = true

# Owner info for JWT auth -- these are optional
owner_public_key = YOUR_COMPANION_PUBLIC_KEY_HERE
owner_email = your.email@example.com

# IATA code for topic routing (XYZ is invalid set it to a real IATA)
iata = XYZ

# MQTT Broker (Let's Mesh Analyzer)
mqtt1_enabled = true
mqtt1_server = mqtt-us-v1.letsmesh.net
mqtt1_port = 443
mqtt1_transport = websockets
mqtt1_use_tls = true
mqtt1_use_auth_token = true
mqtt1_token_audience = mqtt-us-v1.letsmesh.net
mqtt1_topic_status = meshcore/{IATA}/{PUBLIC_KEY}/status
mqtt1_topic_packets = meshcore/{IATA}/{PUBLIC_KEY}/packets

2. Restart Bot - The service starts automatically

---

Configuration

Basic Settings

[PacketCapture]
enabled = true                    # Enable packet capture
output_file = packets.json        # Optional: save to file
verbose = false                   # Detailed packet logging
debug = false                     # Debug mode
mqtt_skip_unparseable_packets = true   # Skip MQTT when content hash is all zeros (strict path reject / short buffer)

# Optional: skip MQTT for ADVERT packets whose Ed25519 signature does not verify (damaged or spoofed mesh payload).
# Does not affect file/JSONL capture.
advert_require_valid_signature = false

Authentication

Option 1: On-Device Signing (Recommended)

auth_token_method = device        # Use device's built-in signing
# No private key file needed

Option 2: Python Signing

auth_token_method = python        # Use Python signing
private_key_path = /path/to/key.txt  # Path to private key file

MQTT Brokers

Configure multiple brokers using mqttN_* pattern:

# Broker 1
mqtt1_enabled = true
mqtt1_server = mqtt-us-v1.letsmesh.net
mqtt1_port = 443
mqtt1_transport = websockets      # tcp or websockets
mqtt1_use_tls = true
mqtt1_use_auth_token = true
mqtt1_topic_status = meshcore/{IATA}/{PUBLIC_KEY}/status
mqtt1_topic_packets = meshcore/{IATA}/{PUBLIC_KEY}/packets

# Broker 2
mqtt2_enabled = true
mqtt2_server = your.broker.com
mqtt2_port = 1883
mqtt2_transport = tcp
mqtt2_username = user
mqtt2_password = pass

Filtering by packet type

You can limit which packet types are uploaded to each broker with mqttN_upload_packet_types. Use a comma-separated list of type numbers; if unset or empty, all packet types are uploaded.

# Only upload text messages and adverts to this broker
mqtt1_upload_packet_types = 2, 4

# Broker 2 gets everything (default)
# mqtt2_upload_packet_types =

Packet type reference:

Type	Name	Description
0	REQ	Request
1	RESPONSE	Response
2	TXT_MSG	Text message
3	ACK	Acknowledgment
4	ADVERT	Advertisement
5	GRP_TXT	Group text
6	GRP_DATA	Group data
7	ANON_REQ	Anonymous request
8	PATH	Path
9	TRACE	Trace
10	MULTIPART	Multipart
11–15	Type11–RAW_CUSTOM	Other types

Packets that are excluded by this filter are still written to the output file (if configured) and still counted; they are only skipped for MQTT upload to that broker. Debug logs will show "Skipping" for those packets.

Topic Templates

Placeholders:

• {IATA} - Your IATA code (e.g., SEA)
• {iata} - Lowercase IATA code
• {PUBLIC_KEY} - Device public key (uppercase)
• {public_key} - Device public key (lowercase)

Status Publishing and MQTT auth (JWT)

Two separate settings:

• jwt_ttl_seconds (global) / mqttN_jwt_ttl_seconds (per broker): lifetime of the JWT in the exp claim (exp = iat + ttl). Use this when the broker enforces a maximum token lifetime (e.g. 60 minutes → 3600).
• jwt_renewal_interval (global) / mqttN_jwt_renewal_interval (per broker): how often the bot refreshes the MQTT password for that broker. Set less than the TTL (e.g. TTL 3600s and renewal every 1800s) so the connection does not outlive the token.

Per-broker keys override the global values for that broker only. Omit them to inherit globals.

stats_in_status_enabled = true    # Include device stats in status
stats_refresh_interval = 300      # Publish status every 5 minutes

jwt_ttl_seconds = 86400           # Default JWT exp − iat (24 hours) for all brokers unless overridden
jwt_renewal_interval = 43200      # Default proactive refresh cadence (12 hours); 0 = no renewal task

# Example on a broker that requires 60-minute tokens and refresh halfway through:
# mqtt1_jwt_ttl_seconds = 3600
# mqtt1_jwt_renewal_interval = 1800

---

Packet Format

Packet Message

{
  "origin": "MyBot",
  "origin_id": "ABCD1234...",
  "timestamp": "2026-01-04T12:34:56",
  "type": "PACKET",
  "direction": "rx",
  "len": "42",
  "packet_type": "2",
  "route": "D",
  "payload_len": "32",
  "raw": "DEADBEEF...",
  "SNR": "8.5",
  "RSSI": "-42",
  "hash": "ABC123..."
}

Decoded Payloads

When decode_payloads = true, each packet gains a nested decoded object with plain-text / structured fields, in addition to the unchanged raw fields above. This makes dumps easy to process with tools like jq (e.g. jq 'select(.decoded.kind=="GRP_TXT") | .decoded.text').

{
  "type": "PACKET",
  "packet_type": "5",
  "route": "F",
  "raw": "1540CAB3...",
  "decoded": {
    "kind": "GRP_TXT",
    "channel_hash": "ca",
    "channel": "#bot",
    "sender": "Alice",
    "text": "hello mesh",
    "msg_timestamp": "2026-07-08T21:22:31Z",
    "decrypted": true,
    "path": ["A1", "B2"]
  }
}

The decoded object holds only payload-specific content — it does not restate header fields (packet_type, route) that already exist at the top level.

What can be decoded:

• GRP_TXT (channel messages) are decrypted when a matching channel key is available. Keys come from the bot's own configured radio channels automatically, plus decode_hashtag_channels (keys derived from the #name), decode_channel_keys (name=hexOrBase64 pairs), and the built-in default Public channel key (decode_include_public = true).
• ADVERT packets are parsed into name, mode (role), lat/lon, and public_key.
• The decoded path hop list is included in decoded.path when it isn't already present at the top level (the top-level path is only added for route=D), so flood-route paths are captured without duplication.
• Direct messages (TXT_MSG) are ECDH-encrypted between two nodes and cannot be decrypted by a passive observer — they appear as {"kind": "TXT_MSG", "encrypted": true}.

Publishing of the decoded object to MQTT is off by default (include_decoded = false) — opt in per broker with mqttN_include_decoded = true, or set include_decoded = true to publish it to all brokers. This lets you, e.g., send decoded text to a private broker while public brokers receive only raw packets. The log file always includes the decoded object when decode_payloads = true, independent of this setting.

Status Message

{
  "status": "online",
  "timestamp": "2026-01-04T12:34:56",
  "origin": "MyBot",
  "origin_id": "ABCD1234...",
  "model": "Heltec V3",
  "firmware_version": "v3.1.2",
  "radio": "915000000,250,9,8",
  "client_version": "meshcore-bot/v1.0.0",
  "stats": {
    "rx_packets": 1234,
    "tx_packets": 567
  }
}

---

Troubleshooting

Service Not Starting

Check logs:

tail -f meshcore_bot.log | grep PacketCapture

Common issues:

• enabled = false in config
• Missing paho-mqtt library: pip install paho-mqtt

MQTT Not Connecting

1. Check broker settings - Verify hostname and port
2. Test connection manually:

   mosquitto_pub -h mqtt-us-v1.letsmesh.net -p 443 -t test -m "test"
   

3. Check authentication - Verify JWT token generation
4. Check logs - Look for connection errors

No Packets Being Published

1. Verify MQTT connection - Check logs for "Connected to MQTT broker"
2. Check packet count - Service logs "Captured packet #N" (or "Skipping packet #N" when filtered) for each packet
3. Verify topics - Ensure topics match broker expectations
4. Check upload filter - If mqttN_upload_packet_types is set, only those types are uploaded. DEBUG Logs show "packet type X not in [Y, Z]" when a packet is skipped

---

Advanced

Multiple Brokers

Configure up to 10 brokers (mqtt1_* through mqtt10_*). Each broker has independent connection tracking and auto-reconnection.

Health Monitoring

health_check_interval = 30        # Check connection every 30s
health_check_grace_period = 2     # Allow 2 failures before warning

Log Rotation

By default output_file is a single file that is appended to forever. To keep historical dumps manageable, enable rotation:

# Size-based: roll at 50 MB, keep 5 backups (packets.jsonl.1 ... .5)
log_rotation = size
log_max_bytes = 50MB
log_backup_count = 5

# Or time-based: roll daily at midnight, keep 14 days
log_rotation = time
log_rotation_when = midnight
log_backup_count = 14

log_rotation = off (default) keeps the original single-file behavior. log_max_bytes accepts plain bytes or suffixes like 10M / 1G. log_rotation_when uses Python's TimedRotatingFileHandler values (midnight, H, D, W0–W6).

JWT Authentication

Tokens are valid for 24 hours and auto-renewed. The service tries on-device signing first (if auth_token_method = device), then falls back to Python signing.

Token Format:

{
  "iat": 1234567890,
  "exp": 1234654290,
  "aud": "mqtt-us-v1.letsmesh.net",
  "publicKey": "DEVICE_PUBLIC_KEY",
  "owner": "OWNER_PUBLIC_KEY",
  "email": "your@email.com",
  "iata": "SEA"
}

---

Neighbour Discovery (zero-hop)

Periodically asks which repeaters this node can hear directly, and records each confirmed link with its measured SNR. Ported from the observer firmware's neighbours feature by way of meshcore-packet-capture. Off by default.

This is the strongest link evidence the bot collects. Everything else is weaker: path inference works from 1–3 byte prefixes with no public keys, and complete_contact_tracking.hop_count over-claims zero-hop (it asserts ~800 zero-hop contacts where only ~68 have corroborating path evidence). A discover response is a first-party RF measurement between two full 32-byte public keys.

[PacketCapture]
enabled = true
neighbors_enabled = true          # the only switch you need
neighbors_interval_hours = 24     # clamped to 12-336

That one setting turns the whole feature on. Every enabled broker publishes the snapshot by default (mqttN_neighbors defaults to true) — set it false on any broker you want to hold back:

mqtt2_neighbors = false
# mqtt1_topic_neighbors = meshcore/{IATA}/{PUBLIC_KEY}/neighbors   # optional override

The neighbours topic is derived from the broker's packets topic by swapping its last segment, so a broker configured with meshcore/{IATA}/{PUBLIC_KEY}/packets publishes to meshcore/{IATA}/{PUBLIC_KEY}/neighbors — the same topic the firmware uses. Brokers with only a topic_prefix get <prefix>/neighbors. If a derived topic is location-routed but no iata is set, that broker is skipped with a warning rather than publishing into meshcore/XYZ/... on a shared namespace.

Each cycle sends one zero-hop node-discover request, then listens for neighbors_discover_window seconds (60 by default). The bot stays fully responsive during the window — it is a passive listen, and only the single discover command touches the radio.

Results go three places, independently of each other:

• neighbor_links — current adjacency (full public keys, observation count, best/last/mean SNR). This is the source of truth.

• neighbor_observations — one row per neighbour per cycle, for signal history. Pruned by [Data_Retention] neighbor_observations_retention_days (default 365).

• The mesh graph — as edges, when neighbors_feed_mesh_graph = true (default), plus a dedicated Neighbours Only evidence mode on the mesh page and GET /api/mesh/edges?evidence=neighbors. Confirmed neighbours render as heavier lines and show their SNR.

  mesh_connections cannot record why an edge exists, so the combined view re-derives the neighbors label from neighbor_links — matching on the 3-byte prefix pair or the full public-key pair, since the graph deliberately keeps some edges at a 1-byte prefix while still filling in the keys discovery gave it. The label honours the view's days window: neighbor_links is never pruned, so without that a link last heard years ago would keep claiming a recent path-derived edge is a current direct neighbour.

Snapshots are published non-retained. heard_secs_ago is relative to publish time, so a retained copy replayed days later would still claim the neighbour was heard seconds ago. Consumers that want the current picture should subscribe and wait for the next cycle, or read timestamp and correct for the age. No broker is required at all — the database is a perfectly good consumer on its own.

Triggering a cycle manually

The minimum interval is 12 hours, so use the DM command to test:

neighbors

It acknowledges immediately and reports the result in a second DM once the window closes. Enabled via [Neighbors_Command]; add neighbors to [Admin_ACL] admin_commands to restrict it, since a cycle spends airtime.

Two guards keep the airtime bounded. Both live in the service rather than in the command, because what is being rationed belongs to the whole mesh and every trigger reaches the same radio — the scheduler included:

• Only one cycle at a time. An overlapping cycle is refused whichever trigger asks, so two discover rounds can never collect into each other's window.
• At most one cycle every 15 minutes. Measured from the last cycle that reached the radio, including one that failed after the discover request went out, since a lost acknowledgement spends the airtime just the same. Users cannot take turns and keep the radio discovering continuously, and the scheduler's own retry-after-failure backoff waits this out rather than re-transmitting every five minutes. A cycle that bailed out before transmitting (radio down, unsupported build) does not start the clock, so re-checking those stays quick.

The DM command reports the wait instead of failing opaquely, and rewinds the sender's personal cooldown to expire with the shared one — the command manager records an execution before the command runs, so otherwise being told "wait one more minute" would be followed by fourteen more minutes of personal cooldown.

Region scopes are opt-in, and why

neighbors_collect_scopes additionally asks each neighbour for its region scopes. It defaults to false for two reasons specific to running inside the bot:

1. It stalls bot replies. Every bot radio command is serialised through one lock (modules/core.py _SerializedCommands), and req_regions_sync waits for its reply inside the call — so one request holds the radio for up to ~25 s. With 32 neighbours the bot's own messages stall in bursts for minutes.
2. It mutates device contact state. The zero-hop probe relies on the neighbour not being a known contact. The bot does track contacts, and for a repeater with no stored path the meshcore library reaches zero-hop by calling change_contact_path() and then reset_path() — temporarily rewriting that contact's path on the device. Those two calls are not paired by a try/finally upstream, and one error path returns between them, so a request cut short — or one whose path change was applied but not acknowledged — would leave the contact pinned to zero-hop and every later message to it sent direct-only. modules/neighbors_discovery.py restores the path itself in each of those cases, and warns if the device rejects the restore (which it reports as an error event rather than an exception), since that contact's routing is then wrong until something else fixes it.

With it off, the snapshot reports every neighbour it heard with empty scopes and status: responded. Enable it on a bench radio first.

Published payload

{
  "timestamp": "2026-08-04T12:00:00.000000+00:00",
  "origin": "MeshCore-HOWL",
  "origin_id": "A1B2C3D4E5F67890...",
  "total_neighbors": 6,
  "queried_neighbors": 6,
  "truncated": false,
  "self": { "scopes": "" },
  "neighbors": [
    {
      "pubkey": "0011223344556677...",
      "snr": 9.75,
      "heard_secs_ago": 42,
      "scopes": "",
      "status": "responded"
    }
  ]
}

total_neighbors is how many were discovered, queried_neighbors how many were kept after the neighbors_max cap, and truncated is true when either that cap or the 10 KB payload budget dropped entries. Entries are ordered most- to least-useful (most recently heard, then stronger SNR). status is responded, timeout, or send_failed.

Requires meshcore >= 2.3.8 and a firmware build exposing CMD_SEND_CONTROL_DATA; the service logs once and skips the feature if either is missing.

---

FAQ

Q: Do I need to provide a private key? A: Not if using on-device signing (auth_token_method = device). The service will fetch the key from your device automatically.

Q: Can I publish to my own MQTT broker? A: Yes. Set mqtt1_use_auth_token = false and provide mqtt1_username and mqtt1_password.

Q: What's the difference between TCP and WebSockets? A: WebSockets work through firewalls better (uses HTTPS port 443). TCP is lighter but may be blocked.

Q: How do I disable packet capture but keep status publishing? A: You can't disable just packet capture - it's all or nothing. Consider filtering on the broker side.

Q: Can I capture TX (outgoing) packets? A: Currently only RX (incoming) packets are captured.