Share high-contact RAK offline queue with held one-key DMs

docs/companion_contact_cache.md

@@ -1,6 +1,7 @@
-# Full Companion contact caches
+# Companion contact caches
 
-Full Companions on ESP32 and nRF52 without PSRAM now keep 16 outgoing paths
+Full Companions on ESP32 and nRF52 without PSRAM, plus the high-contact RAK4631
+USB/BLE profiles, keep 16 outgoing paths
 and 16 shared secrets in RAM. Every contact remains in the contact table;
 selecting a different contact loads its saved path as needed. This recovers
 about 25.4 KiB of internal RAM in the 350-contact ESP32 qualification builds.
@@ -9,8 +10,10 @@ See the [qualification results](companion_contact_cache_results.md) for the
 six-board RAM comparison, physical tests, and shared-secret timings.
 
 PSRAM boards retain their existing complete contact table in external RAM.
-Legacy USB/BLE/WiFi Companion profiles also retain their inline paths and
-secrets. The build flag `MESH_CONTACT_CACHE=0` or `1` overrides this policy for
+Most legacy USB/BLE/WiFi Companion profiles retain inline paths and secrets.
+The high-contact RAK4631 USB/BLE profiles also enable the same cache,
+keeping all 350 contacts while making room for the shared 256-frame offline/DM
+pool. The build flag `MESH_CONTACT_CACHE=0` or `1` overrides this policy for
 qualification; it is not a runtime feature switch.
 
 ## Paths and persistence

docs/companion_offline_queue.md

@@ -12,7 +12,8 @@ A reboot clears it.
 | ESP32 with configured PSRAM | 512 |
 | ESP32 without PSRAM | 256 |
 | nRF52840 with at most 300 contact slots | 256 |
-| nRF52840 with more than 300 contact slots (non-shared queue) | 208 |
+| nRF52840 with more than 300 contacts and the contact cache (non-mOTA-shared queue) | 256 total: up to 15 held DMs use slots until accepted |
+| nRF52840 with more than 300 contacts but no contact cache | 208 ordinary frames plus a separate 15-DM queue |
 | nRF52 Full Companion with the memory correction | 256 normally; 128 while mOTA owns shared storage |
 | RP2040 | 256 |
 | STM32 | 16 |
@@ -45,9 +46,15 @@ Full profiles require at least 72 KiB of heap space at link time; OLED and
 headless profiles use their own [memory budgets](firmware_memory_budget.md).
 
 Standard, logging, MQTT, and Cascade build overlays retain the selected target
-capacity. The high-contact nRF52 default above is the exception: it reserves
-RAM for the separate 15-entry consent-pending DM queue. A target's explicit
-`OFFLINE_QUEUE_SIZE` still takes precedence.
+capacity. On cache-enabled high-contact nRF52 Companions, including the RAK4631
+USB and BLE profiles, ordinary messages and consent-pending DMs share one
+256-frame RAM pool. With no held DMs, all 256 slots are available to ordinary
+messages; with 15 held DMs, up to 241 are. The app sees only ordinary messages
+until a sender is accepted. The oldest held DM rolls off when a sixteenth
+decryptable DM arrives. The shared pool is still volatile and clears on reboot.
+Uncached high-contact profiles keep the conservative 208-frame default and
+separate held queue. An explicit target `OFFLINE_QUEUE_SIZE` still takes
+precedence.
 
 Each queue slot currently costs 177 bytes. A 256-frame queue reserves 45,312
 bytes, while a 512-frame queue reserves 90,624 bytes. There is no 256-frame

docs/companion_protocol.md

@@ -260,9 +260,12 @@ The receiving Companion verifies the introduction signature. By default,
 `PUSH_CODE_NEW_ADVERT` (`0x8A`) but is not added as a contact. The app/user
 accepts through the normal add-contact command. Up to 15 verified, decryptable
 plain DMs are held in RAM and delivered after acceptance; the oldest rolls
-off first. The queue is lost on reboot. On high-contact nRF52 builds, the
-ordinary offline queue is 208 frames (previously 256) to retain the runtime
-heap margin for this separate 15-DM queue. The device sends a signed `DMR1`
+off first. The queue is lost on reboot. Cache-enabled high-contact nRF52
+Companions share a 256-frame pool between ordinary offline messages and up to
+15 held DMs, so ordinary capacity is 256 when none are held and 241 when all
+15 are held. The held frames are not exposed by ordinary message sync before
+acceptance. Uncached high-contact nRF52 builds retain 208 ordinary frames and
+a separate held queue. The device sends a signed `DMR1`
 refusal, and the sender pushes `0x91` plus the full rejecting key to a
 connected app, then stops further introductions for that peer until reboot.
 Directly attached clients may use `get dm.one_key` or `set dm.one_key on|off`;

examples/companion_radio/MyMesh.cpp

@@ -459,11 +459,27 @@ int MyMesh::getOfflineQueueCapacity() const {
 MyMesh::Frame& MyMesh::offlineQueueFrameAt(int logical_index) {
 #if defined(OTA_SHARED_COMPANION_QUEUE)
   return offline_queue.at((offline_queue_head + logical_index) % getOfflineQueueCapacity());
+#elif ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+  return offline_queue[logical_index];
 #else
   return offline_queue[(offline_queue_head + logical_index) % getOfflineQueueCapacity()];
 #endif
 }
 
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+MyMesh::Frame& MyMesh::heldDMFrameAt(uint8_t index) {
+  return offline_queue[getOfflineQueueCapacity() - 1 - index];
+}
+
+void MyMesh::removeHeldOneKeyDM(uint8_t index) {
+  for (uint8_t j = index + 1; j < held_dm_count; ++j) {
+    held_dms[j - 1] = held_dms[j];
+    heldDMFrameAt(j - 1) = heldDMFrameAt(j);
+  }
+  --held_dm_count;
+}
+#endif
+
 #if defined(OTA_SHARED_COMPANION_QUEUE)
 mesh::ota::OtaContext* MyMesh::acquireOfflineQueueForOta(void* owner) {
   MyMesh* mesh = static_cast<MyMesh*>(owner);
@@ -518,7 +534,10 @@ void MyMesh::initializeOfflineQueue() {
 }
 
 bool MyMesh::addToOfflineQueue(const uint8_t frame[], int len) {
-  const int capacity = getOfflineQueueCapacity();
+  int capacity = getOfflineQueueCapacity();
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+  capacity -= held_dm_count;
+#endif
   if (!frame || len <= 0 || len > MAX_FRAME_SIZE || capacity <= 0) {
     MESH_DEBUG_PRINTLN("WARN: invalid offline queue frame length: %d", len);
     return false;
@@ -564,11 +583,17 @@ int MyMesh::getFromOfflineQueue(uint8_t frame[]) {
     memcpy(frame, head.buf, len);
 
     offline_queue_len--;
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+    for (int i = 0; i < offline_queue_len; ++i) {
+      offlineQueueFrameAt(i) = offlineQueueFrameAt(i + 1);
+    }
+#else
     if (offline_queue_len == 0) {
       offline_queue_head = 0;
     } else {
       offline_queue_head = (offline_queue_head + 1) % getOfflineQueueCapacity();
     }
+#endif
 #ifdef DISPLAY_CLASS
     if (_ui) _ui->syncMessageQueue(offline_queue_len, 0);
 #endif
@@ -1251,6 +1276,36 @@ bool MyMesh::onAddressedTextPacket(mesh::Packet* packet, uint8_t src_hash,
     }
 
     // Only verified and decryptable messages count toward the FIFO of 15.
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+    if (held_dm_count == MAX_HELD_ONE_KEY_DMS) removeHeldOneKeyDM(0);
+    if (offline_queue_len + held_dm_count >= getOfflineQueueCapacity()) {
+      // Preserve private messages already waiting for the app. As with the
+      // ordinary queue, an old channel message may be displaced instead.
+      int pos = 0;
+      while (pos < offline_queue_len && !offlineQueueFrameAt(pos).isChannelMsg()) ++pos;
+      if (pos == offline_queue_len) return false;
+      for (int j = pos; j < offline_queue_len - 1; ++j) {
+        offlineQueueFrameAt(j) = offlineQueueFrameAt(j + 1);
+      }
+      --offline_queue_len;
+#ifdef DISPLAY_CLASS
+      if (_ui) _ui->syncMessageQueue(offline_queue_len, pos);
+#endif
+    }
+    HeldOneKeyDM& entry = held_dms[held_dm_count];
+    memcpy(entry.sender_key, pub_key, PUB_KEY_SIZE);
+    memcpy(entry.id, id, ONE_KEY_DM_ID_SIZE);
+    Frame& held = heldDMFrameAt(held_dm_count++);
+    memcpy(held.buf, &timestamp, sizeof(timestamp));
+    mesh::Utils::sha256(&held.buf[4], 4, data, 5 + strlen(text),
+                        pub_key, PUB_KEY_SIZE);
+    held.buf[8] = packet->isRouteFlood() ? packet->path_len : 0xFF;
+    held.buf[9] = packet->_snr;
+    const uint8_t text_len = strlen(text);
+    held.buf[10] = text_len;
+    memcpy(&held.buf[11], text, text_len + 1);
+    held.len = 12 + text_len;
+#else
     if (held_dm_count == MAX_HELD_ONE_KEY_DMS) {
       for (uint8_t j = 1; j < held_dm_count; ++j) held_dms[j - 1] = held_dms[j];
       --held_dm_count;
@@ -1259,6 +1314,7 @@ bool MyMesh::onAddressedTextPacket(mesh::Packet* packet, uint8_t src_hash,
     memcpy(entry.sender_key, pub_key, PUB_KEY_SIZE);
     memcpy(entry.id, id, ONE_KEY_DM_ID_SIZE);
     entry.packet = *packet;
+#endif
     return true;
   }
   return false;
@@ -1279,6 +1335,39 @@ void MyMesh::releaseHeldOneKeyDMs() {
     HeldOneKeyDM& entry = held_dms[i];
     ContactInfo* contact = lookupPersistentContactByPubKey(entry.sender_key, PUB_KEY_SIZE);
     if (contact == NULL) { ++i; continue; }
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+    // The held frame itself occupies the slot needed for ordinary delivery.
+    // Copy its small payload before removing it from the tail of the pool.
+    Frame& held = heldDMFrameAt(i);
+    uint32_t timestamp;
+    memcpy(&timestamp, held.buf, sizeof(timestamp));
+    uint8_t ack_hash[4];
+    memcpy(ack_hash, &held.buf[4], sizeof(ack_hash));
+    uint8_t path_len = held.buf[8];
+    int8_t snr = static_cast<int8_t>(held.buf[9]);
+    uint8_t text_len = held.buf[10];
+    char text[MAX_TEXT_LEN + 1];
+    if (held.len != 12 + text_len || text_len > MAX_TEXT_LEN ||
+        held.buf[11 + text_len] != 0) {
+      removeHeldOneKeyDM(i);
+      continue;
+    }
+    memcpy(text, &held.buf[11], text_len + 1);
+    uint8_t sender_key[PUB_KEY_SIZE];
+    uint8_t id[ONE_KEY_DM_ID_SIZE];
+    memcpy(sender_key, entry.sender_key, sizeof(sender_key));
+    memcpy(id, entry.id, sizeof(id));
+    removeHeldOneKeyDM(i);
+    if (!wasDeliveredOneKeyDM(sender_key, id)) {
+      mesh::Packet packet;
+      packet.header = path_len == 0xFF ? ROUTE_TYPE_DIRECT : ROUTE_TYPE_FLOOD;
+      packet.path_len = path_len == 0xFF ? 0 : path_len;
+      packet._snr = snr;
+      onMessageRecv(*contact, &packet, timestamp, text);
+      rememberDeliveredOneKeyDM(sender_key, id);
+    }
+    sendAckTo(*contact, ack_hash, sizeof(ack_hash));
+#else
     if (offline_queue_len >= getOfflineQueueCapacity()) break;
 
     uint8_t data[MAX_PACKET_PAYLOAD + 1];
@@ -1305,6 +1394,7 @@ void MyMesh::releaseHeldOneKeyDMs() {
     }
     for (uint8_t j = i + 1; j < held_dm_count; ++j) held_dms[j - 1] = held_dms[j];
     --held_dm_count;
+#endif
   }
 }
 
@@ -1507,14 +1597,23 @@ void MyMesh::onMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t
   uint8_t dm_id[ONE_KEY_DM_ID_SIZE];
   makeOneKeyDMId(dm_id, sender_timestamp, text);
   if (wasDeliveredOneKeyDM(from.id.pub_key, dm_id)) return;
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+  bool replaced_held_dm = false;
+#endif
   for (uint8_t i = 0; i < held_dm_count; ++i) {
     if (memcmp(held_dms[i].sender_key, from.id.pub_key, PUB_KEY_SIZE) == 0 &&
         memcmp(held_dms[i].id, dm_id, ONE_KEY_DM_ID_SIZE) == 0) {
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+      // Removing the held frame frees exactly one ordinary slot for this retry.
+      removeHeldOneKeyDM(i);
+      replaced_held_dm = true;
+#else
       // Keep the held copy if the ordinary delivery path cannot queue this
       // just-arrived retry; otherwise accepting a peer with a full app queue
       // could silently discard the original when the queue drains.
       if (offline_queue_len >= getOfflineQueueCapacity()) return;
       rememberDeliveredOneKeyDM(from.id.pub_key, dm_id);
+#endif
       break;
     }
   }
@@ -1522,6 +1621,9 @@ void MyMesh::onMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t
   // BaseChatMesh updates lastmod immediately before this callback.
   scheduleContactWrite(from);
   queueMessage(from, TXT_TYPE_PLAIN, pkt, sender_timestamp, NULL, 0, text);
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+  if (replaced_held_dm) rememberDeliveredOneKeyDM(from.id.pub_key, dm_id);
+#endif
 }
 
 void MyMesh::onCommandDataRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, const char *text) {

examples/companion_radio/MyMesh.h

@@ -84,13 +84,18 @@
 #define MAX_CONTACTS 100
 #endif
 
+#if defined(NRF52_PLATFORM) && MAX_CONTACTS > 300 \
+    && !defined(OTA_SHARED_COMPANION_QUEUE) && MESH_CONTACT_CACHE
+#define ONE_KEY_DM_SHARED_OFFLINE_QUEUE 1
+#else
+#define ONE_KEY_DM_SHARED_OFFLINE_QUEUE 0
+#endif
+
 #ifndef OFFLINE_QUEUE_SIZE
 #if defined(ESP32_PLATFORM) && defined(BOARD_HAS_PSRAM)
 #define OFFLINE_QUEUE_SIZE 512
 #elif defined(NRF52_PLATFORM) && MAX_CONTACTS > 300 \
-    && !defined(OTA_SHARED_COMPANION_QUEUE)
-// Large nRF52 contact tables and the 15-DM consent queue need a smaller
-// ordinary offline queue to preserve the runtime heap safety margin.
+    && !defined(OTA_SHARED_COMPANION_QUEUE) && !ONE_KEY_DM_SHARED_OFFLINE_QUEUE
 #define OFFLINE_QUEUE_SIZE 208
 #elif defined(ESP32_PLATFORM) || defined(NRF52_PLATFORM) \
     || defined(RP2040_PLATFORM)
@@ -101,6 +106,9 @@
 #endif
 
 static_assert(OFFLINE_QUEUE_SIZE > 0, "OFFLINE_QUEUE_SIZE must be positive");
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+static_assert(OFFLINE_QUEUE_SIZE >= 15, "Shared DM queue needs 15 frame slots");
+#endif
 
 #ifndef ROOM_MESSAGE_TIMESTAMP_CACHE_SIZE
 #define ROOM_MESSAGE_TIMESTAMP_CACHE_SIZE 16
@@ -113,6 +121,11 @@ static_assert(OFFLINE_QUEUE_SIZE > 0, "OFFLINE_QUEUE_SIZE must be positive");
 #include <helpers/BaseChatMesh.h>
 #include <helpers/TransportKeyStore.h>
 
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+static_assert(MAX_TEXT_LEN + 12 <= MAX_FRAME_SIZE,
+              "A held plain DM must fit in one offline frame");
+#endif
+
 /* -------------------------------------------------------------------------------------- */
 
 #define REQ_TYPE_GET_STATUS             0x01 // same as _GET_STATS
@@ -655,6 +668,10 @@ private:
     bool isChannelMsg() const;
   };
   Frame& offlineQueueFrameAt(int logical_index);
+#if ONE_KEY_DM_SHARED_OFFLINE_QUEUE
+  Frame& heldDMFrameAt(uint8_t index);
+  void removeHeldOneKeyDM(uint8_t index);
+#endif
   void initializeOfflineQueue();
   int offline_queue_len;
   int offline_queue_head;
@@ -716,7 +733,9 @@ private:
   struct HeldOneKeyDM {
     uint8_t sender_key[PUB_KEY_SIZE];
     uint8_t id[ONE_KEY_DM_ID_SIZE];
+#if !ONE_KEY_DM_SHARED_OFFLINE_QUEUE
     mesh::Packet packet;
+#endif
   };
   HeldOneKeyDM held_dms[MAX_HELD_ONE_KEY_DMS];
   uint8_t held_dm_count = 0;

test/test_one_key_dm_shared_queue.py

@@ -0,0 +1,315 @@
+#!/usr/bin/env python3
+"""Exercise the real shared Companion queue methods on the host."""
+
+from pathlib import Path
+import os
+import subprocess
+import tempfile
+import unittest
+
+ROOT = Path(__file__).resolve().parents[1]
+SOURCE = ROOT / "examples/companion_radio/MyMesh.cpp"
+
+
+def method(source: str, signature: str) -> str:
+    start = source.index(signature)
+    end = source.index("{", start) + 1
+    depth = 1
+    while depth:
+        depth += (source[end] == "{") - (source[end] == "}")
+        end += 1
+    return source[start:end]
+
+
+class OneKeyDMSharedQueueTest(unittest.TestCase):
+    def compile_and_run(self, source: str):
+        with tempfile.TemporaryDirectory(prefix="meshcore-shared-dm-") as temp:
+            path = Path(temp) / "shared_dm.cpp"
+            binary = Path(temp) / "shared_dm.exe"
+            path.write_text(source)
+            sanitizer_flags = ["-fsanitize=address,undefined"] if os.name != "nt" else []
+            built = subprocess.run(
+                ["c++", "-std=c++17", "-O0", *sanitizer_flags,
+                 str(path), "-o", str(binary)],
+                text=True, capture_output=True,
+            )
+            self.assertEqual(built.returncode, 0, built.stderr)
+            result = subprocess.run([str(binary)], text=True, capture_output=True)
+            self.assertEqual(result.returncode, 0, result.stderr)
+
+    def test_shared_pool_keeps_ordinary_fifo_and_held_tail_separate(self):
+        source = SOURCE.read_text()
+        methods = "\n\n".join(method(source, signature) for signature in (
+            "bool MyMesh::Frame::isChannelMsg() const",
+            "int MyMesh::getOfflineQueueCapacity() const",
+            "MyMesh::Frame& MyMesh::offlineQueueFrameAt(int logical_index)",
+            "MyMesh::Frame& MyMesh::heldDMFrameAt(uint8_t index)",
+            "void MyMesh::removeHeldOneKeyDM(uint8_t index)",
+            "bool MyMesh::addToOfflineQueue(const uint8_t frame[], int len)",
+            "int MyMesh::getFromOfflineQueue(uint8_t frame[])",
+        ))
+        harness = r'''
+#include <cassert>
+#include <cstdint>
+#include <cstring>
+#define ONE_KEY_DM_SHARED_OFFLINE_QUEUE 1
+#define OFFLINE_QUEUE_SIZE 256
+#define MAX_FRAME_SIZE 176
+#define RESP_CODE_CHANNEL_MSG_RECV 0x8c
+#define RESP_CODE_CHANNEL_MSG_RECV_V3 0x90
+#define RESP_CODE_CHANNEL_DATA_RECV 0x8d
+#define MESH_DEBUG_PRINTLN(...) ((void)0)
+class MyMesh {
+public:
+  struct Frame {
+    uint8_t len = 0;
+    uint8_t buf[MAX_FRAME_SIZE] = {};
+    bool isChannelMsg() const;
+  };
+  struct Held { uint8_t marker = 0; };
+  Frame offline_queue[OFFLINE_QUEUE_SIZE] = {};
+  Held held_dms[15] = {};
+  uint8_t held_dm_count = 0;
+  int offline_queue_len = 0;
+  int offline_queue_head = 0;
+  int getOfflineQueueCapacity() const;
+  Frame& offlineQueueFrameAt(int index);
+  Frame& heldDMFrameAt(uint8_t index);
+  void removeHeldOneKeyDM(uint8_t index);
+  bool addToOfflineQueue(const uint8_t frame[], int len);
+  int getFromOfflineQueue(uint8_t frame[]);
+};
+static_assert(sizeof(MyMesh::Frame) == 177);
+'''
+        exercise = r'''
+int main() {
+  MyMesh mesh;
+  uint8_t out[176] = {};
+  // All 256 slots are ordinary messages when there is no held DM.
+  for (int i = 0; i < 256; ++i) {
+    uint8_t frame[2] = {42, static_cast<uint8_t>(i)};
+    assert(mesh.addToOfflineQueue(frame, sizeof(frame)));
+  }
+  const uint8_t overflow = 42;
+  assert(!mesh.addToOfflineQueue(&overflow, 1));
+  for (int i = 0; i < 256; ++i) {
+    assert(mesh.getFromOfflineQueue(out) == 2);
+    assert(out[1] == static_cast<uint8_t>(i));
+  }
+
+  MyMesh channel_mesh;
+  const uint8_t channel = RESP_CODE_CHANNEL_MSG_RECV;
+  assert(channel_mesh.addToOfflineQueue(&channel, 1));
+  for (int i = 0; i < 255; ++i) {
+    uint8_t frame[2] = {42, static_cast<uint8_t>(i)};
+    assert(channel_mesh.addToOfflineQueue(frame, sizeof(frame)));
+  }
+  assert(channel_mesh.addToOfflineQueue(&overflow, 1));
+  assert(channel_mesh.offline_queue_len == 256);
+  assert(channel_mesh.getFromOfflineQueue(out) == 2 && out[1] == 0);
+  for (int i = 1; i < 255; ++i) {
+    assert(channel_mesh.getFromOfflineQueue(out) == 2);
+    assert(out[1] == static_cast<uint8_t>(i));
+  }
+  assert(channel_mesh.getFromOfflineQueue(out) == 1 && out[0] == overflow);
+
+  // Held entries occupy the opposite end; sync must never expose them.
+  for (int i = 0; i < 15; ++i) {
+    mesh.held_dms[i].marker = static_cast<uint8_t>(i + 1);
+    mesh.heldDMFrameAt(mesh.held_dm_count).buf[0] = static_cast<uint8_t>(i + 1);
+    ++mesh.held_dm_count;
+  }
+  for (int i = 0; i < 241; ++i) {
+    uint8_t frame[2] = {42, static_cast<uint8_t>(i)};
+    assert(mesh.addToOfflineQueue(frame, sizeof(frame)));
+  }
+  assert(!mesh.addToOfflineQueue(&overflow, 1));
+  for (int i = 0; i < 241; ++i) {
+    assert(mesh.getFromOfflineQueue(out) == 2);
+    assert(out[1] == static_cast<uint8_t>(i));
+  }
+  assert(mesh.getFromOfflineQueue(out) == 0);
+  for (int i = 0; i < 15; ++i) {
+    assert(mesh.held_dms[i].marker == i + 1);
+    assert(mesh.heldDMFrameAt(i).buf[0] == i + 1);
+  }
+
+  // Removing any held DM shifts only the held tail and frees one normal slot.
+  mesh.removeHeldOneKeyDM(0);
+  assert(mesh.held_dm_count == 14);
+  assert(mesh.held_dms[0].marker == 2);
+  assert(mesh.heldDMFrameAt(0).buf[0] == 2);
+  assert(mesh.addToOfflineQueue(&overflow, 1));
+  assert(mesh.getFromOfflineQueue(out) == 1 && out[0] == overflow);
+  for (int i = 0; i < 14; ++i) mesh.removeHeldOneKeyDM(0);
+  assert(mesh.held_dm_count == 0);
+  for (int i = 0; i < 256; ++i) {
+    uint8_t frame[2] = {42, static_cast<uint8_t>(i)};
+    assert(mesh.addToOfflineQueue(frame, sizeof(frame)));
+  }
+  return 0;
+}
+'''
+        self.compile_and_run(harness + methods + exercise)
+
+    def test_sixteenth_dm_rolls_off_and_acceptance_releases_only_last_fifteen(self):
+        source = SOURCE.read_text()
+        methods = "\n\n".join(method(source, signature) for signature in (
+            "bool MyMesh::Frame::isChannelMsg() const",
+            "int MyMesh::getOfflineQueueCapacity() const",
+            "MyMesh::Frame& MyMesh::offlineQueueFrameAt(int logical_index)",
+            "MyMesh::Frame& MyMesh::heldDMFrameAt(uint8_t index)",
+            "void MyMesh::removeHeldOneKeyDM(uint8_t index)",
+            "bool MyMesh::addToOfflineQueue(const uint8_t frame[], int len)",
+            "int MyMesh::getFromOfflineQueue(uint8_t frame[])",
+            "bool MyMesh::onAddressedTextPacket(mesh::Packet* packet, uint8_t src_hash,",
+            "void MyMesh::releaseHeldOneKeyDMs()",
+        ))
+        harness = r'''
+#include <cassert>
+#include <cstdint>
+#include <cstdio>
+#include <cstring>
+#define ONE_KEY_DM_SHARED_OFFLINE_QUEUE 1
+#define OFFLINE_QUEUE_SIZE 256
+#define MAX_FRAME_SIZE 176
+#define MAX_PACKET_PAYLOAD 184
+#define MAX_TEXT_LEN 160
+#define PUB_KEY_SIZE 32
+#define TXT_TYPE_PLAIN 0
+#define ROUTE_TYPE_DIRECT 2
+#define ROUTE_TYPE_FLOOD 1
+#define RESP_CODE_CHANNEL_MSG_RECV 0x8c
+#define RESP_CODE_CHANNEL_MSG_RECV_V3 0x90
+#define RESP_CODE_CHANNEL_DATA_RECV 0x8d
+#define MESH_DEBUG_PRINTLN(...) ((void)0)
+namespace mesh {
+struct Packet {
+  uint8_t header = ROUTE_TYPE_FLOOD;
+  uint8_t path_len = 0;
+  int8_t _snr = 0;
+  bool isRouteFlood() const { return header == ROUTE_TYPE_FLOOD; }
+};
+struct Utils {
+  static int MACThenDecrypt(const uint8_t*, uint8_t* out,
+                            const uint8_t* in, size_t len) {
+    memcpy(out, in, len);
+    return static_cast<int>(len);
+  }
+  static void sha256(uint8_t* out, size_t out_len, const uint8_t* in,
+                     size_t in_len, const uint8_t*, size_t) {
+    for (size_t i = 0; i < out_len; ++i) out[i] = in[i % in_len];
+  }
+};
+}
+struct ContactInfo { struct { uint8_t pub_key[PUB_KEY_SIZE] = {}; } id; };
+struct SelfIdentity {
+  void calcSharedSecret(uint8_t* secret, const uint8_t*) { memset(secret, 0, PUB_KEY_SIZE); }
+};
+class MyMesh {
+public:
+  static constexpr uint8_t MAX_HELD_ONE_KEY_DMS = 15;
+  static constexpr uint8_t ONE_KEY_DM_ID_SIZE = 8;
+  struct Frame {
+    uint8_t len = 0;
+    uint8_t buf[MAX_FRAME_SIZE] = {};
+    bool isChannelMsg() const;
+  };
+  struct HeldOneKeyDM {
+    uint8_t sender_key[PUB_KEY_SIZE];
+    uint8_t id[ONE_KEY_DM_ID_SIZE];
+  };
+  Frame offline_queue[OFFLINE_QUEUE_SIZE] = {};
+  int offline_queue_len = 0;
+  int offline_queue_head = 0;
+  HeldOneKeyDM held_dms[MAX_HELD_ONE_KEY_DMS] = {};
+  uint8_t held_dm_count = 0;
+  uint8_t verified_pending_keys[MAX_HELD_ONE_KEY_DMS][PUB_KEY_SIZE] = {};
+  uint8_t verified_pending_count = 0;
+  uint8_t delivered_ids[MAX_HELD_ONE_KEY_DMS][ONE_KEY_DM_ID_SIZE] = {};
+  uint8_t delivered_count = 0;
+  int ack_count = 0;
+  bool accepted = false;
+  ContactInfo contact;
+  SelfIdentity self_id;
+  int getOfflineQueueCapacity() const;
+  Frame& offlineQueueFrameAt(int index);
+  Frame& heldDMFrameAt(uint8_t index);
+  void removeHeldOneKeyDM(uint8_t index);
+  bool addToOfflineQueue(const uint8_t frame[], int len);
+  int getFromOfflineQueue(uint8_t frame[]);
+  bool onAddressedTextPacket(mesh::Packet*, uint8_t, const uint8_t*, size_t);
+  void releaseHeldOneKeyDMs();
+  ContactInfo* lookupPersistentContactByPubKey(const uint8_t* key, size_t) {
+    return accepted && memcmp(contact.id.pub_key, key, PUB_KEY_SIZE) == 0
+        ? &contact : nullptr;
+  }
+  void forgetVerifiedPendingSender(const uint8_t*) { verified_pending_count = 0; }
+  static void makeOneKeyDMId(uint8_t id[ONE_KEY_DM_ID_SIZE], uint32_t timestamp,
+                             const char*) {
+    memset(id, 0, ONE_KEY_DM_ID_SIZE);
+    memcpy(id, &timestamp, sizeof(timestamp));
+  }
+  bool wasDeliveredOneKeyDM(const uint8_t*, const uint8_t id[ONE_KEY_DM_ID_SIZE]) {
+    for (int i = 0; i < delivered_count; ++i)
+      if (memcmp(delivered_ids[i], id, ONE_KEY_DM_ID_SIZE) == 0) return true;
+    return false;
+  }
+  void rememberDeliveredOneKeyDM(const uint8_t*, const uint8_t id[ONE_KEY_DM_ID_SIZE]) {
+    memcpy(delivered_ids[delivered_count++], id, ONE_KEY_DM_ID_SIZE);
+  }
+  void onMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t timestamp,
+                     const char* text) {
+    uint8_t frame[MAX_FRAME_SIZE] = {};
+    memcpy(frame, &timestamp, 4);
+    memcpy(frame + 4, text, strlen(text) + 1);
+    assert(addToOfflineQueue(frame, 5 + strlen(text)));
+  }
+  void sendAckTo(const ContactInfo&, const uint8_t*, uint8_t) { ++ack_count; }
+};
+'''
+        exercise = r'''
+int main() {
+  MyMesh receiver;
+  receiver.contact.id.pub_key[0] = 0x42;
+  receiver.verified_pending_count = 1;
+  receiver.verified_pending_keys[0][0] = 0x42;
+  mesh::Packet packet;
+  packet._snr = 12;
+  for (uint32_t timestamp = 1; timestamp <= 16; ++timestamp) {
+    uint8_t encrypted[32] = {};
+    memcpy(encrypted, &timestamp, 4);
+    encrypted[4] = TXT_TYPE_PLAIN << 2;
+    snprintf(reinterpret_cast<char*>(encrypted + 5), 20, "msg%02lu",
+             static_cast<unsigned long>(timestamp));
+    assert(receiver.onAddressedTextPacket(&packet, 0x42, encrypted,
+                                           5 + strlen(reinterpret_cast<char*>(encrypted + 5))));
+  }
+  assert(receiver.held_dm_count == 15);
+  uint8_t out[MAX_FRAME_SIZE] = {};
+  assert(receiver.getFromOfflineQueue(out) == 0);
+  receiver.accepted = true;
+  receiver.releaseHeldOneKeyDMs();
+  assert(receiver.held_dm_count == 0);
+  assert(receiver.verified_pending_count == 0);
+  assert(receiver.offline_queue_len == 15);
+  assert(receiver.ack_count == 15);
+  for (uint32_t timestamp = 2; timestamp <= 16; ++timestamp) {
+    int len = receiver.getFromOfflineQueue(out);
+    assert(len > 5);
+    uint32_t delivered_timestamp;
+    memcpy(&delivered_timestamp, out, 4);
+    assert(delivered_timestamp == timestamp);
+    char expected[20];
+    snprintf(expected, sizeof(expected), "msg%02lu", static_cast<unsigned long>(timestamp));
+    assert(strcmp(reinterpret_cast<char*>(out + 4), expected) == 0);
+  }
+  assert(receiver.getFromOfflineQueue(out) == 0);
+  return 0;
+}
+'''
+        self.compile_and_run(harness + methods + exercise)
+
+
+if __name__ == "__main__":
+    unittest.main()

variants/rak4631/platformio.ini

@@ -369,6 +369,7 @@ build_flags =
   -D DISPLAY_CLASS=SSD1306Display
   -D MAX_CONTACTS=350
   -D MAX_GROUP_CHANNELS=40
+  -D MESH_CONTACT_CACHE=1
   -D ENABLE_USB_INTERFACE
 ; NOTE: DO NOT ENABLE -->  -D MESH_PACKET_LOGGING=1
 ; NOTE: DO NOT ENABLE -->  -D MESH_DEBUG=1
@@ -440,6 +441,7 @@ build_flags =
   -D DISPLAY_CLASS=SSD1306Display
   -D MAX_CONTACTS=350
   -D MAX_GROUP_CHANNELS=40
+  -D MESH_CONTACT_CACHE=1
   -D BLE_PIN_CODE=123456
 ;  -D BLE_DEBUG_LOGGING=1
   -D OFFLINE_QUEUE_SIZE=256