diff --git a/docs/cli_commands.md b/docs/cli_commands.md index d46f155a..73ce4206 100644 --- a/docs/cli_commands.md +++ b/docs/cli_commands.md @@ -2605,6 +2605,73 @@ eviction like administrators. --- +#### Recover a repeater's future-dated replay timestamp + +This is an explicit recovery operation after correcting a bad clock, not a +contact deletion. It lowers selected replay timestamps to the repeater's +current UTC epoch only when they are later than that epoch. Earlier values, +public keys, permissions, stored paths, and historical identity records remain. +The replay file is committed before the live table changes. No extra 60-second +login reservation is added by this command. + +First set/synchronize and verify the repeater's clock (`clock`); a build-default +clock without a manual or observed synchronization is not accepted. Also fix +the companion's clock before its next login. + +USB console: + +```text +replay reset +replay reset all CONFIRM +``` + +Authenticated LoRa admin, including a resumed admin session: + +```text +replay reset +``` + +The reply shows `now=`, `ttl=s`, and a confirmation command containing the same +full key and a one-use 32-hex-character token. Verify the displayed time and send +that command before the original 300-second deadline. During the first 120 +seconds, repeated requests for the same key by the same admin return the same +token without restarting either timer. From 120 through 300 seconds, the token +is retained for confirmation only: requests do not resend or replace it. This +leaves at least 180 seconds to deliver a confirmation after the last permitted +token response is generated (radio transit time still counts toward expiry). +At 300 seconds it expires and a new request can receive a new token. Confirmation +can succeed immediately; there is no requirement to wait for the resend window +to close. The displayed TTL decreases on retries and is measured when the reply +is generated, not when it reaches the companion. + +It can target the caller's own key or another +exact key; prefixes, `self`, wildcards, and `all` are not allowed over LoRa. +Tokens are bound to both the requesting admin and target, expire on reboot, +and are invalid after use or a clock correction outside the five-second +confirmation tolerance. A failed write requires a new confirmation. Challenge +requests consult live token state instead of replaying cached challenge text; +completed confirmation results remain cacheable without executing the reset +again. Normal packet freshness checks still apply to retries. + +Only the physical serial console grants `all` access. Ethernet, browser and +internal command callbacks do not count as USB. Guest, read-only, region-manager +and filter-manager roles cannot reset replay state. This command is implemented +in repeater firmware; room-server and sensor CLI are unchanged. + +Normal login and command admission checks remain in force: a fully locked-out +caller that cannot send an accepted admin command needs another working admin +or USB access. A corrupt/unreadable replay file fails closed and is not erased +or formatted by this operation. Unknown keys do not create records. + +Security trade-off: lowering a replay boundary can admit previously captured +future-dated login/command packets above the new boundary, including packets +that could raise it again. The one-use token prevents the recovery command +itself from being repeatedly executed; it does not replace the protocol's +timestamp-based replay protection. Use recovery only after verifying clocks. +Setting a clock alone never automatically resets this table. + +--- + #### View or change this room server's 'read-only' flag **Usage:** - `get allow.read.only` diff --git a/docs/spiffs_regular_file_reads.md b/docs/spiffs_regular_file_reads.md new file mode 100644 index 00000000..6965efc4 --- /dev/null +++ b/docs/spiffs_regular_file_reads.md @@ -0,0 +1,63 @@ +# SPIFFS regular-file reads and login replay state + +Arduino-ESP32 SPIFFS can return a truthy directory handle from a read-open of a +nonexistent filename. `File::operator bool()` alone does not prove that a +regular file exists. A directory's `size()` and `read()` are zero. + +The first privileged login on an upgraded G2 encountered this in +`ClientACL::writeClientLoginReplayCeiling()`: opening the not-yet-created +`/s_login_replay` appeared successful, then subtracting the eight-byte trailer +from size zero underflowed. Copying the supposed records failed and login was +rejected. Waiting or changing the repeater clock cannot fix that file-open bug. + +## Fix + +Use `mesh::openFileRead()` for regular-file reads. It checks existence and +rejects directory handles, while preserving real empty files. Replay record +counts also validate the minimum trailer size, record alignment, and maximum +count before subtraction. An existing replay file that becomes unreadable is +not treated as a new store. + +No replay records are cleared and authentication is not weakened. Invalid or +unwritable replay state still fails closed. The first successful privileged +login creates a 44-byte file: one 36-byte identity/ceiling record plus its +eight-byte integrity trailer. + +Setting the repeater clock must not clear this file. Admission compares the +sender's timestamp with that sender's saved boundary, not with the repeater's +current time. Clearing the boundary could make captured requests reusable. A +sender clock rollback is a separate condition and remains subject to the saved +boundary after this fix. + +Companion directory enumeration uses the separate `openDirectory()` API, +which deliberately permits SPIFFS virtual directories. Regular-file reads are +also enforced for companion data, repeater/room logs, flood-rule verification, +and HTTP packet-log downloads (missing logs return 404; real empty logs 200). + +## Audit boundary + +The audit covered file opens and size arithmetic in `src` and `examples`. +Identity, region, clock, and common preference loaders already gate reads with +filesystem existence checks; SPIFFS's `exists()` explicitly excludes directory +handles. Relevant MQTT length subtraction follows validated headers and exact +reads. ESP32 OTA staging uses partition APIs, not these SPIFFS file handles. +Intentional directory enumeration must not be changed to a regular-file read. + +## Regression checks + +- `test/test_client_acl_spiffs.py` compiles the actual `ClientACL.cpp` against + a filesystem that reproduces the misleading missing-file directory handle. + It covers first creation, fresh/stale login, reboot ceilings, corrupt and + unreadable state, short writes, failed publication, and truncated sources. +- `test/test_regular_file_reads.py` executes production companion readers, + flood-file verification, and HTTP log delivery with the same filesystem + behavior, including platform-specific file API differences. +- `test/test_esp32_tinyusb_cooperative_output.py` exercises both real role log + pumps, including missing logs and bounded output. +- Native ACL transaction, login persistence, and client-path persistence suites + retain the existing security and recovery coverage. + +The pre-fix actual ACL code was reproduced as `first_login=rejected`, one +missing read-open, and no replay file. The fixed code accepted that identical +scenario without a missing read-open and created the valid 44-byte record. +Host simulations do not replace a post-flash LoRa login test on the G2. diff --git a/docs/usb_serial_backpressure.md b/docs/usb_serial_backpressure.md new file mode 100644 index 00000000..f2562644 --- /dev/null +++ b/docs/usb_serial_backpressure.md @@ -0,0 +1,74 @@ +# Native USB backpressure and radio liveness + +ESP32-S2/S3 builds using native TinyUSB CDC (`ARDUINO_USB_MODE=0` and +CDC-on-boot) must not wait for a computer to read USB output. In the bundled +Arduino-ESP32 2.0.17 core, `USBCDC::write()` can wait indefinitely for transmit +space; its configured timeout bounds a mutex, not that wait. This can stop the +same loop that services LoRa, even while the USB connection still appears open. + +## Firmware behavior + +- Native CDC output uses a single FIFO attempt, not Arduino's wait-for-space + write or flush. TinyUSB's internal short mutex operations still apply; this + is a no-host-progress-wait guarantee, not a lock-free driver replacement. +- Functional text and diagnostics share one ordered 4 KiB queue. Diagnostics + leave 3 KiB reserved for commands. A diagnostic backlog alone does not prevent + command input. A stalled host may lose diagnostic records; radio work continues. +- A connected terminal that exceeds its bounded output capacity gets an explicit + dropped-byte notice when transmission resumes. Large local file dumps and + recent-repeater listings advance between radio service passes instead. +- File dumps stop at their initial file size and emit `-> EOF` only after the + final accepted record. Corrupt stored lines exceeding 640 bytes are omitted + with a visible notice. Recent-repeater output is a live, bounded-cursor view; + incoming packets can change its ordering while it is being printed. +- Disconnects discard pending old-session text and reset the role's partial + command/listing state. Binary/mOTA transitions suppress pending text notices. + Bytes already transmitted cannot be recalled. +- Companion frames retain and retry short writes. An mOTA request is admitted + only when its complete, at-most-11-byte record fits. +- nRF52, UART, and USB-Serial-JTAG behavior is not changed by this native-CDC path. + +Host software must also keep reading independently of command writes. A serial +relay should start its reader before the first command, use finite write and +response deadlines, avoid discarding received packet logs, and cancel I/O during +shutdown. A response timeout must not allow a late reply to satisfy another +command: the CLI has no transaction identifiers. Updating firmware alone does +not correct an indefinite host-side serial write. + +## Regression checks + +Run only one PlatformIO process in the checkout at a time. + +```sh +python test/test_esp32_tinyusb_nonblocking.py +python test/test_esp32_tinyusb_role_hygiene.py +python test/test_esp32_tinyusb_cooperative_output.py +python test/test_esp32_usb_serial_hygiene.py +python test/test_nrf52_usb_logging_contract.py +pio test -e native -f test_nrf52_debug_output -f test_serial_packet_log -f test_serial_mode_switch -f test_mesh_tables +``` + +The first test compiles the real USB facade against a simulated 64-byte FIFO, +including stopped readers, reconnects, protocol transitions, and other-platform +fallbacks. Host simulations cannot establish that every real USB driver or +endpoint failure has recovered. + +A Full Station G2 validation build with USA Cascadia radio settings and the Cascade +profile can be made using the normal build entry point. The portable `standard` +recipe preserves the deployed partition layout but omits LoRa OTA; it still has +the browser firmware uploader. The `auto`/`full` recipe instead enables the +expanded feature set and partition layout: its merged image is **not** an +app-only update for a device with the legacy layout. + +```sh +MESHDEBUG_OVERRIDE=on PACKET_LOGGING_OVERRIDE=on \ + bash build.sh build-firmware Station_G2_repeater --build-profile full \ + --radio-preset usa-cascadia --profile cascade +``` + +Before installing on hardware, preserve the device identity, preferences, and +existing partition layout. Then verify USB command responses and repeated LoRa +logins with the relay running, paused, and stopped, including a host that leaves +USB open without reading. Check LoRa recovery separately from USB OUT recovery; +fixing transmit backpressure does not prove an unrelated OUT endpoint fault is +resolved. Do not erase or repartition the radio as part of this test. diff --git a/examples/companion_radio/DataStore.cpp b/examples/companion_radio/DataStore.cpp index f988450d..b55223a9 100644 --- a/examples/companion_radio/DataStore.cpp +++ b/examples/companion_radio/DataStore.cpp @@ -1,6 +1,7 @@ #include #include #include "DataStore.h" +#include #if defined(NRF52_PLATFORM) #include @@ -381,23 +382,33 @@ uint32_t DataStore::getStorageTotalKb() const { } File DataStore::openRead(const char* filename) { -#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) - return _fs->open(filename, FILE_O_READ); -#elif defined(RP2040_PLATFORM) - return _fs->open(filename, "r"); -#else - return _fs->open(filename, "r", false); -#endif + return openRead(_fs, filename); } File DataStore::openRead(FILESYSTEM* fs, const char* filename) { + return mesh::openFileRead(fs, filename); +} + +File DataStore::openDirectory(const char* path) { + return openDirectory(_fs, path); +} + +File DataStore::openDirectory(FILESYSTEM* fs, const char* path) { + if (fs == nullptr || path == nullptr) return mesh::emptyFile(fs); + // SPIFFS has virtual directories: do not use the regular-file existence + // check here. Only the explicit listing API may return directory handles. #if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) - return fs->open(filename, FILE_O_READ); + File directory = fs->open(path, FILE_O_READ); #elif defined(RP2040_PLATFORM) - return fs->open(filename, "r"); + File directory = fs->open(path, "r"); #else - return fs->open(filename, "r", false); + File directory = fs->open(path, "r", false); #endif + if (directory && !directory.isDirectory()) { + directory.close(); + return mesh::emptyFile(fs); + } + return directory; } bool DataStore::removeFile(const char* filename) { diff --git a/examples/companion_radio/DataStore.h b/examples/companion_radio/DataStore.h index aeefc775..d4167794 100644 --- a/examples/companion_radio/DataStore.h +++ b/examples/companion_radio/DataStore.h @@ -91,6 +91,8 @@ public: bool deleteBlobByKey(const uint8_t key[], int key_len); File openRead(const char* filename); File openRead(FILESYSTEM* fs, const char* filename); + File openDirectory(const char* path); + File openDirectory(FILESYSTEM* fs, const char* path); bool removeFile(const char* filename); bool removeFile(FILESYSTEM* fs, const char* filename); uint32_t getStorageUsedKb() const; diff --git a/examples/companion_radio/MyMesh.cpp b/examples/companion_radio/MyMesh.cpp index 25ff1d79..2e27a199 100644 --- a/examples/companion_radio/MyMesh.cpp +++ b/examples/companion_radio/MyMesh.cpp @@ -7465,7 +7465,7 @@ void MyMesh::checkCLIRescueCmd() { output.printf("Listing files in %s\n", path); // log each file and directory - File root = _store->openRead(path); + File root = _store->openDirectory(path); if (is_fs2 == false) { if (root) { File file = root.openNextFile(); @@ -7485,7 +7485,7 @@ void MyMesh::checkCLIRescueCmd() { if (is_fs2 == true || strlen(path) == 0 || strcmp(path, "/") == 0) { if (_store->getSecondaryFS() != nullptr) { - File root2 = _store->openRead(_store->getSecondaryFS(), path); + File root2 = _store->openDirectory(_store->getSecondaryFS(), path); File file = root2.openNextFile(); while (file) { if (file.isDirectory()) { diff --git a/examples/simple_repeater/MyMesh.cpp b/examples/simple_repeater/MyMesh.cpp index e4d70ccf..28760f48 100644 --- a/examples/simple_repeater/MyMesh.cpp +++ b/examples/simple_repeater/MyMesh.cpp @@ -1,4 +1,6 @@ #include "MyMesh.h" +#include +#include #include #include #include @@ -403,11 +405,7 @@ static bool buildRepeatersChannel(mesh::GroupChannel& channel) { } static File openFloodSettingsRead(FILESYSTEM* fs, const char* filename) { -#if defined(RP2040_PLATFORM) - return fs->open(filename, "r"); -#else - return fs->open(filename); -#endif + return mesh::openFileRead(fs, filename); } static File openFloodSettingsWrite(FILESYSTEM* fs, const char* filename) { @@ -2400,14 +2398,25 @@ void MyMesh::formatRecentRepeatersReply(char *reply, int page, void MyMesh::printRecentRepeatersSerial() { const SimpleMeshTables* tables = static_cast(getTables()); if (tables == NULL) { - Serial.println("Error: unsupported"); + mesh::usbConsolePort().printf("Error: unsupported\r\n"); return; } +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (hasPendingSerialOutput()) { + mesh::usbConsolePort().printf("Err - USB output busy\r\n"); + return; + } + serial_recent_count = tables->getRecentRepeaterCount(); + serial_recent_next = 0; + serial_recent_header = true; + serial_recent_has_cursor = false; + servicePendingSerialOutput(); +#else int count = tables->getRecentRepeaterCount(); - Serial.printf("Recent repeaters (%d):\n", count); + mesh::usbConsolePort().printf("Recent repeaters (%d):\n", count); if (count <= 0) { - Serial.println("-none-"); + mesh::usbConsolePort().printf("-none-\r\n"); return; } @@ -2419,8 +2428,9 @@ void MyMesh::printRecentRepeatersSerial() { mesh::Utils::toHex(prefix, info->prefix, info->prefix_len); prefix[info->prefix_len * 2] = 0; formatLocalSnrX4(snr, sizeof(snr), info->snr_x4); - Serial.printf("%s,%s%s\n", prefix, snr[0] == '-' ? "" : " ", snr); + mesh::usbConsolePort().printf("%s,%s%s\n", prefix, snr[0] == '-' ? "" : " ", snr); } +#endif } bool MyMesh::setRecentRepeater(const uint8_t* prefix, uint8_t prefix_len, int8_t snr_x4) { @@ -2768,12 +2778,19 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, char *command = (char *)&data[5]; size_t command_len = strlen(command); + mesh::ReplayResetRequest replay_request; + const bool replay_command = mesh::parseReplayResetCommand(command, replay_request) + != mesh::ReplayResetKind::NotReplay; + const bool replay_prepare = replay_request.kind == mesh::ReplayResetKind::ExactKey; uint32_t request_id = sender_timestamp; mesh::RemoteCliRequest::parse(data, len, 5, request_id); uint32_t command_fingerprint = mesh::RemoteCliReplyCache::fingerprint(command, command_len); const char* cached_response = NULL; - const bool cached_retry = remote_cli_reply_cache.lookup( + // A cached challenge response can outlive its disclosure window or name + // a token from an earlier attempt. Re-evaluate preparation against the + // live nonce policy instead; confirmation results remain cacheable. + const bool cached_retry = !replay_prepare && remote_cli_reply_cache.lookup( client->id.pub_key, request_id, command_fingerprint, &cached_response); @@ -2783,7 +2800,11 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, MESH_DEBUG_PRINTLN("onPeerDataRecv: possible replay attack detected"); } else { const bool repeated_timestamp = sender_timestamp == client->last_timestamp; - if (sender_timestamp > client->last_timestamp) { + // Recovery uses a one-time challenge, not its packet's future clock. + // Never let a consumed/malformed/cached recovery packet re-poison the + // sender's floor after their own entry was clamped. Normal admission + // and current-role authentication still apply above and in the handler. + if (!replay_command && sender_timestamp > client->last_timestamp) { client->last_timestamp = sender_timestamp; } client->last_activity = getRTCClock()->getCurrentTime(); @@ -3059,7 +3080,7 @@ void __attribute__((noinline)) MyMesh::processDeferredCliCommand() { record[6U + signed_content_len] = ' '; mesh::Utils::toHex(record + 6U + signed_content_len + 1U, signature, sizeof(signature)); - Serial.println(record); + mesh::usbConsolePort().printf("%s\r\n", record); host_cli_claim_emit = false; host_cli_claim_emit_at = 0; host_cli_claimed = true; @@ -3107,7 +3128,7 @@ void __attribute__((noinline)) MyMesh::processDeferredCliCommand() { record[6U + signed_content_len] = ' '; mesh::Utils::toHex(record + 6U + signed_content_len + 1U, signature, sizeof(signature)); - Serial.println(record); + mesh::usbConsolePort().printf("%s\r\n", record); host_cli_waiting = true; host_cli_claimed = false; host_cli_claim_emit = false; @@ -3682,7 +3703,7 @@ void MyMesh::begin(FILESYSTEM *fs) { if (start_webui) { char wc_reply[160]; startWebConfig(false, wc_reply); - Serial.println(wc_reply); + mesh::usbConsolePort().printf("%s\r\n", wc_reply); } #endif @@ -4824,21 +4845,181 @@ void MyMesh::updateFloodAdvertTimer() { } void MyMesh::dumpLogFile() { -#if defined(RP2040_PLATFORM) - File f = _fs->open(PACKET_LOG_FILE, "r"); +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (hasPendingSerialOutput()) { + mesh::usbConsolePort().printf("Err - USB output busy\r\n"); + return; + } + serial_log_dump = mesh::openFileRead(_fs, PACKET_LOG_FILE); + serial_log_active = static_cast(serial_log_dump); + serial_log_remaining = serial_log_active ? serial_log_dump.size() : 0; + serial_log_pending_size = 0; + serial_log_eof_pending = true; + serial_log_skip_line = false; #else - File f = _fs->open(PACKET_LOG_FILE); -#endif + File f = mesh::openFileRead(_fs, PACKET_LOG_FILE); if (f) { while (f.available()) { int c = f.read(); if (c < 0) break; - Serial.print((char)c); + mesh::usbConsolePort().print((char)c); } f.close(); } +#endif } +#if MESH_ESP32_TINYUSB_NONBLOCKING +bool MyMesh::hasPendingSerialOutput() const { + return serial_log_active || serial_log_eof_pending || serial_recent_next >= 0; +} + +void MyMesh::cancelPendingSerialOutput() { + if (serial_log_active) serial_log_dump.close(); + serial_log_active = false; + serial_log_eof_pending = false; + serial_log_skip_line = false; + serial_log_remaining = 0; + serial_log_pending_size = 0; + serial_recent_next = -1; + serial_recent_count = 0; + serial_recent_header = false; + serial_recent_has_cursor = false; + serial_recent_cursor_index = -1; +} + +void MyMesh::servicePendingSerialOutput() { + Stream& console = mesh::usbConsolePort(); + + if (serial_recent_next >= 0) { + char record[64]; + int length = 0; + const SimpleMeshTables::RecentRepeaterInfo* next_info = nullptr; + int next_index = -1; + if (serial_recent_header) { + length = snprintf(record, sizeof(record), "Recent repeaters (%d):\n", + serial_recent_count); + } else if (serial_recent_count == 0) { + length = snprintf(record, sizeof(record), "-none-\r\n"); + } else if (serial_recent_next < serial_recent_count) { + const auto* tables = static_cast(getTables()); + const auto* info = tables ? tables->getNextRecentRepeaterBySortKey( + serial_recent_has_cursor ? &serial_recent_cursor : nullptr, + serial_recent_cursor_index, next_index) : nullptr; + if (info == nullptr) { + serial_recent_next = -1; + return; + } + next_info = info; + char prefix[MAX_ROUTE_HASH_BYTES * 2 + 1]; + char snr[12]; + mesh::Utils::toHex(prefix, info->prefix, info->prefix_len); + formatLocalSnrX4(snr, sizeof(snr), info->snr_x4); + length = snprintf(record, sizeof(record), "%s,%s%s\n", prefix, + snr[0] == '-' ? "" : " ", snr); + } else { + serial_recent_next = -1; + return; + } + // One complete row per pass, admitted atomically only when it fits. + if (length <= 0 || static_cast(length) >= sizeof(record)) { + serial_recent_next = -1; + return; + } + if (console.availableForWrite() < length + || console.write(reinterpret_cast(record), length) + != static_cast(length)) return; + if (serial_recent_header) { + serial_recent_header = false; + } else { + if (next_info != nullptr) { + serial_recent_cursor = *next_info; + serial_recent_cursor_index = next_index; + serial_recent_has_cursor = true; + } + if (serial_recent_count == 0 || ++serial_recent_next >= serial_recent_count) { + serial_recent_next = -1; + } + } + return; + } + if (!serial_log_active) { + // CommonCLI's synchronous EOF is suppressed until the queued dump ends. + static const char eof[] = " -> EOF\r\n"; + if (serial_log_eof_pending + && console.availableForWrite() >= static_cast(sizeof(eof) - 1) + && console.write(reinterpret_cast(eof), sizeof(eof) - 1) + == sizeof(eof) - 1) { + serial_log_eof_pending = false; + } + return; + } + + // Read at most one bounded record per mesh pass. Snapshotting the original + // file size prevents a busy radio's newly appended log from extending this + // command forever. A retained suffix survives temporary USB backpressure. + if (serial_log_skip_line) { + // Do not split a malformed overlong stored line around live packet logs. + // Skip it in bounded passes and substitute one explicit complete record. + size_t budget = sizeof(serial_log_pending); + while (budget-- > 0 && serial_log_remaining > 0) { + const int value = serial_log_dump.read(); + if (value < 0) { + serial_log_remaining = 0; + break; + } + --serial_log_remaining; + if (value == '\n') { + serial_log_skip_line = false; + break; + } + } + if (serial_log_remaining == 0) serial_log_skip_line = false; + if (serial_log_skip_line) return; + static const char omitted[] = "[USB log line omitted: exceeds 640 bytes]\r\n"; + memcpy(serial_log_pending, omitted, sizeof(omitted) - 1); + serial_log_pending_size = sizeof(omitted) - 1; + } else if (serial_log_pending_size == 0) { + while (serial_log_remaining > 0 + && serial_log_pending_size < sizeof(serial_log_pending)) { + const int value = serial_log_dump.read(); + if (value < 0) { + serial_log_remaining = 0; + break; + } + --serial_log_remaining; + serial_log_pending[serial_log_pending_size++] = static_cast(value); + if (value == '\n') break; + } + if (serial_log_pending_size == sizeof(serial_log_pending) + && serial_log_pending[serial_log_pending_size - 1] != '\n') { + serial_log_pending_size = 0; + serial_log_skip_line = true; + return; + } + if (serial_log_remaining == 0 && serial_log_pending_size > 0 + && serial_log_pending[serial_log_pending_size - 1] != '\n') { + serial_log_pending[serial_log_pending_size++] = '\n'; + } + } + if (serial_log_pending_size > 0 + && console.availableForWrite() >= static_cast(serial_log_pending_size)) { + size_t written = console.write( + reinterpret_cast(serial_log_pending), serial_log_pending_size); + if (written > serial_log_pending_size) written = serial_log_pending_size; + serial_log_pending_size -= written; + if (written > 0 && serial_log_pending_size > 0) { + memmove(serial_log_pending, serial_log_pending + written, serial_log_pending_size); + } + } + if (serial_log_remaining == 0 && serial_log_pending_size == 0) { + serial_log_dump.close(); + serial_log_active = false; + } +} +#endif + + bool MyMesh::setTxPower(int8_t power_dbm) { return radio_driver.setTxPower(power_dbm); } @@ -9624,6 +9805,8 @@ void MyMesh::suppressMeshClockSyncForBoot(uint8_t source) { } void MyMesh::onManualClockSet() { + replay_clock_set = true; + replay_reset_nonce.clear(); suppressMeshClockSyncForBoot(CLOCK_SYNC_MESH_SUPPRESS_CLI); } @@ -10168,7 +10351,10 @@ static const char* clockSyncMeshSuppressionName(uint8_t source) { return "unavailable"; } -void MyMesh::onManualClockSet() {} +void MyMesh::onManualClockSet() { + replay_clock_set = true; + replay_reset_nonce.clear(); +} #endif @@ -10794,14 +10980,130 @@ static bool isFilterMgrAllowed(const char* cmd) { || commandFamilyMatches(cmd, "set repeat"); } +bool MyMesh::handleReplayResetCommand(ClientInfo* sender, const char* command, + char* reply, bool usb_origin) { + mesh::ReplayResetRequest request; + const mesh::ReplayResetKind kind = mesh::parseReplayResetCommand(command, request); + if (kind == mesh::ReplayResetKind::NotReplay) return false; + const char* prefix = mesh::replay_reset_detail::skipSpace(command); + if (prefix[0] != 0 && prefix[1] != 0 && prefix[2] == '|') { + memcpy(reply, prefix, 3); + reply += 3; + } + + // A null sender also identifies web/Ethernet/internal callbacks: it is not + // proof of physical-console access. Non-admin authenticated peers cannot + // mint or consume a recovery challenge, including for their own identity. + if ((!usb_origin && sender == NULL) || (sender != NULL && !sender->isAdmin())) { + strcpy(reply, "Err - replay recovery requires USB or LoRa admin"); + return true; + } + if (kind == mesh::ReplayResetKind::Invalid) { + strcpy(reply, "Err - use: replay reset <64-hex-public-key> [token]; USB: replay reset all CONFIRM"); + return true; + } + const bool all = kind == mesh::ReplayResetKind::AllConfirm; + if (all && !usb_origin) { + strcpy(reply, "Err - replay reset all is USB-only"); + return true; + } + + const uint32_t now = getRTCClock()->getCurrentTime(); + const bool clock_observed = replay_clock_set + || clock_sync_mesh_suppressed_by != CLOCK_SYNC_MESH_SUPPRESS_NONE + || (clock_sync_last_result >= CLOCK_SYNC_RESULT_WITHIN_DRIFT + && clock_sync_last_result <= CLOCK_SYNC_RESULT_CORRECTED_BACKWARD); + if (!clock_observed || !clockSyncEpochIsValid(now)) { + strcpy(reply, "Err - set/sync and verify the repeater clock before replay recovery"); + return true; + } + + if (!usb_origin) { + const uint32_t now_ms = millis(); + if (kind == mesh::ReplayResetKind::ExactKey) { + uint8_t random_token[16]; + getRNG()->random(random_token, sizeof(random_token)); + const auto issued = replay_reset_nonce.issue( + sender->id.pub_key, request.key, random_token, now_ms, now); + if (issued == mesh::ReplayResetNonce::IssueResult::Busy) { + strcpy(reply, "Err - another replay confirmation is pending (up to 300 seconds)"); + } else if (issued == mesh::ReplayResetNonce::IssueResult::AwaitingConfirmation) { + snprintf(reply, 156, "Err - token is confirmation-only; use earlier reply or wait %lus for expiry", + (unsigned long)replay_reset_nonce.remainingSeconds(now_ms, now)); + } else if (issued == mesh::ReplayResetNonce::IssueResult::Invalid) { + strcpy(reply, "Err - could not create replay confirmation"); + } else { + char key_hex[65], token_hex[33]; + mesh::Utils::toHex(key_hex, request.key, sizeof(request.key)); + mesh::Utils::toHex(token_hex, replay_reset_nonce.token(), sizeof(random_token)); + snprintf(reply, 156, "now=%lu ttl=%lus; confirm: replay reset %s %s", + (unsigned long)now, + (unsigned long)replay_reset_nonce.remainingSeconds(now_ms, now), + key_hex, token_hex); + } + return true; + } + // Consume before writing. A failed write needs a new challenge, and a + // reboot forgets all challenges, so no durable replay-command exception + // or growing nonce history is necessary. + if (!replay_reset_nonce.consume(sender->id.pub_key, request.key, + request.token, now_ms, now)) { + strcpy(reply, "Err - expired/used replay token or clock changed; request a new reset"); + return true; + } + } else if (kind == mesh::ReplayResetKind::ExactKeyConfirm) { + strcpy(reply, "Err - USB uses: replay reset <64-hex-public-key> (no token)"); + return true; + } + + ClientLoginReplayClampResult result; + const uint8_t* target = all ? NULL : request.key; + if (!acl.clampLoginReplayTimestamps(target, now, result)) { + strcpy(reply, "Err - replay storage unavailable; no live timestamps changed"); + return true; + } + replay_reset_nonce.clear(); + // A USB recovery may interrupt a previously admitted host/remote command + // for the affected identity. Do not execute that stale mailbox afterwards. + // The executing LoRa reset owns the mailbox until its reply is sent. + if (usb_origin && deferred_cli_command.pending) { + const int index = deferred_cli_command.client_index; + if (all || (index >= 0 && index < acl.getNumClients() + && memcmp(acl.getClientByIdx(index)->id.pub_key, target, PUB_KEY_SIZE) == 0)) { + clearDeferredCliCommand(); + } + } + for (int index = 0; index < acl.getNumClients(); ++index) { + ClientInfo* client = acl.getClientByIdx(index); + if (all || memcmp(client->id.pub_key, target, PUB_KEY_SIZE) == 0) { + client->observed_path_pending = false; + } + } + // Preserve the response cache: exact cached retries may resend their result + // but must not execute again. The receive path never advances replay-command + // timestamps, even if a reset reply survives in this cache. + if (result.stored_matched == 0 && result.live_matched == 0) { + strcpy(reply, "Err - no matching replay entries; nothing created"); + } else { + snprintf(reply, 156, "OK - clamped to %lu: stored=%u live=%u; records retained", + (unsigned long)now, (unsigned)result.stored_changed, + (unsigned)result.live_changed); + } + return true; +} + void MyMesh::handleCommand(uint32_t sender_timestamp, ClientInfo* sender, char *command, char *reply, int gpio_client_index, - uint8_t gpio_path_hash_size) { + uint8_t gpio_path_hash_size, bool usb_origin) { #if defined(ESP32_PLATFORM) || defined(USER_GPIO_CONTROL) _gpio_reply_tracker.beginCommand(gpio_client_index, gpio_path_hash_size, sender == NULL ? NULL : sender->id.pub_key); #endif char* reply_start = reply; + // Parse the original wire text exactly once, also before region-load mode. + // Parsing again after stripping a prefix would let nested prefixes bypass + // the receive path's recovery-family timestamp guard. + if (handleReplayResetCommand(sender, command, reply, usb_origin)) return; // Remote admin clients may include a line ending in the command payload. // Normalize it here so exact-match commands such as `get outpath` behave the @@ -11234,6 +11536,10 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, ClientInfo* sender, char * // Compatibility path for manually defined legacy portable builds. Current // release builds use FULL and do not enter this branch. _cli.handleCommand(sender_timestamp, command, reply); +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (sender_timestamp == 0 && serial_log_eof_pending + && strcmp(reply, " EOF") == 0) reply[0] = 0; +#endif return; #endif @@ -11473,14 +11779,16 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, ClientInfo* sender, char * printRecentRepeatersSerial(); reply_start[0] = 0; } else if (sender_timestamp == 0 && strcmp(command, "get acl") == 0) { - Serial.println("ACL:"); + mesh::usbConsolePort().printf("ACL:\r\n"); for (int i = 0; i < acl.getNumClients(); i++) { auto c = acl.getClientByIdx(i); if (c->permissions == 0) continue; // skip deleted (or guest) entries - Serial.printf("%02X ", c->permissions); - mesh::Utils::printHex(Serial, c->id.pub_key, PUB_KEY_SIZE); - Serial.printf("\n"); + // Admit each line together so concurrent USB diagnostics cannot split + // a public key or insert text between its permission prefix and value. + char public_key[PUB_KEY_SIZE * 2 + 1]; + mesh::Utils::toHex(public_key, c->id.pub_key, PUB_KEY_SIZE); + mesh::usbConsolePort().printf("%02X %s\n", c->permissions, public_key); } reply[0] = 0; } else if (handleClientPathCommand(sender, command, reply)) { @@ -11634,6 +11942,10 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, ClientInfo* sender, char * #endif } else{ _cli.handleCommand(sender_timestamp, command, reply); // common CLI commands +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (sender_timestamp == 0 && serial_log_eof_pending + && strcmp(reply, " EOF") == 0) reply[0] = 0; +#endif } } @@ -11644,6 +11956,9 @@ void MyMesh::loop() { // Check radio FIRST to ensure we don't miss incoming packets // MQTT processing runs in a separate FreeRTOS task on Core 0, so we don't call bridge.loop() here mesh::Mesh::loop(); +#if MESH_ESP32_TINYUSB_NONBLOCKING + servicePendingSerialOutput(); +#endif _cli.loop(); processDeferredCliCommand(); servicePostMeshLoop(); @@ -11992,7 +12307,7 @@ void __attribute__((noinline)) MyMesh::servicePostMeshLoop() { // headroom, then flash: otaFromManifest reboots into the new image on success // (so this never returns); on any abort (already up to date, partition change, // download error) it returns and we resume the bridge. - Serial.println("OTA: starting update"); + mesh::usbConsolePort().printf("OTA: starting update\r\n"); // Flush the START alert (and CLI reply) out the radio BEFORE teardown blocks // the loop until reboot - otherwise a packet still queued here (busy / // duty-limited channel) is lost when the flash spins the loop and reboots. @@ -12007,11 +12322,11 @@ void __attribute__((noinline)) MyMesh::servicePostMeshLoop() { // firmware then is the observed teardown heap-panic path - so abort and // resume the bridge instead of flashing under uncertain ownership. if (mqtt_bridge && !mqtt_bridge->canFlashAfterStop()) { - Serial.println("OTA: aborted, MQTT stop did not complete cleanly - resuming bridge"); + mesh::usbConsolePort().printf("OTA: aborted, MQTT stop did not complete cleanly - resuming bridge\r\n"); otaAlert("OTA aborted: MQTT stop unclean, bridge resumed"); setBridgeState(true); } else if (!_cli.getBoard()->otaFromManifest(getFirmwareVer(), false, ota_reply)) { - Serial.print("OTA: aborted, resuming bridge - "); Serial.println(ota_reply); + mesh::usbConsolePort().printf("OTA: aborted, resuming bridge - %s\r\n", ota_reply); char ota_alert_msg[160]; snprintf(ota_alert_msg, sizeof(ota_alert_msg), "OTA aborted: %s", ota_reply); otaAlert(ota_alert_msg); @@ -12027,7 +12342,7 @@ void __attribute__((noinline)) MyMesh::servicePostMeshLoop() { if (WebConfigServer::takeButtonToggleRequest()) { char wc_reply[160]; setWebUIEnabled(!WebConfigServer::loadEnabled(false), wc_reply); - Serial.println(wc_reply); + mesh::usbConsolePort().printf("%s\r\n", wc_reply); } if (_webconfig) { _webconfig->tick(millis()); diff --git a/examples/simple_repeater/MyMesh.h b/examples/simple_repeater/MyMesh.h index 4de3cbec..861a03f9 100644 --- a/examples/simple_repeater/MyMesh.h +++ b/examples/simple_repeater/MyMesh.h @@ -94,6 +94,7 @@ #include #endif #include +#include #include #include #include @@ -287,11 +288,28 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks }; FILESYSTEM* _fs; +#if MESH_ESP32_TINYUSB_NONBLOCKING + File serial_log_dump; + size_t serial_log_remaining = 0; + size_t serial_log_pending_size = 0; + char serial_log_pending[640]; + bool serial_log_active = false; + bool serial_log_eof_pending = false; + bool serial_log_skip_line = false; + int serial_recent_next = -1; + int serial_recent_count = 0; + bool serial_recent_header = false; + bool serial_recent_has_cursor = false; + SimpleMeshTables::RecentRepeaterInfo serial_recent_cursor; + int serial_recent_cursor_index = -1; +#endif uint32_t last_millis; uint64_t uptime_millis; unsigned long next_local_advert, next_flood_advert; mesh::DeferredCliCommand deferred_cli_command; mesh::RemoteCliReplyCache remote_cli_reply_cache; + mesh::ReplayResetNonce replay_reset_nonce; + bool replay_clock_set = false; mesh::TempRadioReplyBarrier temp_radio_reply_barrier; TransportKey deferred_cli_reply_scope; bool deferred_cli_reply_scoped; @@ -964,6 +982,12 @@ public: } void dumpLogFile() override; +#if MESH_ESP32_TINYUSB_NONBLOCKING + // Large local-only replies advance between radio service passes. + bool hasPendingSerialOutput() const; + void servicePendingSerialOutput(); + void cancelPendingSerialOutput(); +#endif bool setTxPower(int8_t power_dbm) override; bool setRxPowerSaving(bool enable, uint32_t rx_us, uint32_t sleep_us) override; bool supportsRxPowerSavingRfRxDisable() const override; @@ -991,10 +1015,17 @@ public: void handleCommand(uint32_t sender_timestamp, ClientInfo* sender, char* command, char* reply, int gpio_client_index = -1, - uint8_t gpio_path_hash_size = 1); + uint8_t gpio_path_hash_size = 1, bool usb_origin = false); void handleCommand(uint32_t sender_timestamp, char* command, char* reply) { handleCommand(sender_timestamp, NULL, command, reply); } + // Only the physical console input loop may grant USB-only recovery access. + // Web, Ethernet and execCommand callbacks deliberately keep the default. + void handleUsbCommand(char* command, char* reply) { + handleCommand(0, NULL, command, reply, -1, 1, true); + } + bool handleReplayResetCommand(ClientInfo* sender, const char* command, + char* reply, bool usb_origin); #if MESH_ENABLE_HOST_CLI bool handleHostCliSerialReply(const char* command, char* reply); #endif diff --git a/examples/simple_repeater/UITask.cpp b/examples/simple_repeater/UITask.cpp index 66e3b011..25a39a9d 100644 --- a/examples/simple_repeater/UITask.cpp +++ b/examples/simple_repeater/UITask.cpp @@ -1,6 +1,7 @@ #include "UITask.h" #include "target.h" #include +#include #include #include @@ -42,7 +43,7 @@ void UITask::applyDisplayFlip() { _display->setFlipped(_flip_seen != 0); // Logged unconditionally: this is persisted config, so it survives a reflash // and is otherwise invisible when someone is chasing a wrong orientation. - Serial.printf("Display: flip %s\n", _flip_seen ? "on (rotated 180)" : "off"); + mesh::usbConsolePort().printf("Display: flip %s\n", _flip_seen ? "on (rotated 180)" : "off"); #ifdef DISPLAY_REDRAW_ON_CHANGE _frame_valid = false; #endif @@ -368,7 +369,7 @@ void UITask::toggleDisplay(const char* source) { _display->turnOn(); } #ifdef DISPLAY_TOUCH_DEBUG - Serial.printf("Display: %s -> %s\n", source, _display->isOn() ? "on" : "off"); + mesh::usbConsolePort().printf("Display: %s -> %s\n", source, _display->isOn() ? "on" : "off"); #else (void)source; #endif @@ -408,7 +409,7 @@ void UITask::loop() { #endif } else if (ev == BUTTON_EVENT_LONG_PRESS) { _display->turnOn(); - Serial.println("Powering Off"); + mesh::usbConsolePort().printf("Powering Off\r\n"); _powering_off_at = millis() + POWEROFF_DELAY; #ifdef DISPLAY_REDRAW_ON_CHANGE _frame_valid = false; diff --git a/examples/simple_repeater/main.cpp b/examples/simple_repeater/main.cpp index 88e4a987..05496f02 100644 --- a/examples/simple_repeater/main.cpp +++ b/examples/simple_repeater/main.cpp @@ -72,6 +72,9 @@ static unsigned long userBtnDownAt = 0; void setup() { Serial.begin(115200); +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::beginUsbLoggingPort(); +#endif #if MESH_PACKET_LOGGING mesh::serialLogBegin(); #endif @@ -109,10 +112,14 @@ void setup() { // power cut merely because its radio is temporarily unavailable. Keep // the MCU alive and retry in place; target radio_init() performs the // board-specific regulator/reset/wake recovery on each attempt. - Serial.println("Radio unavailable; retrying in 60 seconds"); + mesh::usbConsolePort().println("Radio unavailable; retrying in 60 seconds"); radioinit_attempts = 0; const uint32_t retry_started = millis(); while (millis() - retry_started < 60000UL) { +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbLoggingPort(); + mesh::serviceUsbTerminalPort(); +#endif #if defined(NRF52_PLATFORM) board.feedWatchdog(); #endif @@ -168,11 +175,12 @@ void setup() { // Print the running firmware version at boot so it's visible after an OTA // reboot without having to issue `ver` manually. - Serial.print("Firmware: "); Serial.print(FIRMWARE_VERSION); - Serial.print(" (built "); Serial.print(FIRMWARE_BUILD_DATE); Serial.println(")"); + Stream& console = mesh::usbConsolePort(); + console.print("Firmware: "); console.print(FIRMWARE_VERSION); + console.print(" (built "); console.print(FIRMWARE_BUILD_DATE); console.println(")"); - Serial.print("Repeater ID: "); - mesh::Utils::printHex(Serial, the_mesh.self_id.pub_key, PUB_KEY_SIZE); Serial.println(); + console.print("Repeater ID: "); + mesh::Utils::printHex(console, the_mesh.self_id.pub_key, PUB_KEY_SIZE); console.println(); command[0] = 0; #ifdef ETHERNET_ENABLED @@ -215,14 +223,31 @@ void setup() { } static void __attribute__((noinline)) serviceCommandInterfaces() { + bool usb_ready = true; +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbLoggingPort(); + mesh::serviceUsbTerminalPort(); + if (mesh::takeUsbTerminalSessionReset()) { + command[0] = 0; + command_overflow = false; + the_mesh.cancelPendingSerialOutput(); + } + // A busy USB session cleanup may defer the CLI, never the radio loop. + usb_ready = mesh::tryCompleteUsbTerminalSessionReset(); + // Large listings advance from MyMesh::loop without blocking radio service. + usb_ready = usb_ready && !the_mesh.hasPendingSerialOutput() + && mesh::canAcceptUsbConsoleCommand(); +#endif + Stream& console = mesh::usbConsolePort(); // Handle Serial CLI int len = strlen(command); bool line_complete = false; bool overlong_line_complete = false; - while (Serial.available()) { - char c = Serial.read(); + size_t read_budget = 256; + while (usb_ready && read_budget-- > 0 && console.available()) { + char c = console.read(); if (c == '\n') continue; - Serial.print(c); + console.print(c); if (command_overflow) { if (c == '\r') { @@ -249,37 +274,37 @@ static void __attribute__((noinline)) serviceCommandInterfaces() { } if (overlong_line_complete) { - Serial.print('\n'); - Serial.println(" -> Err - command too long"); + console.print('\n'); + console.println(" -> Err - command too long"); command[0] = 0; return; } if (line_complete) { - Serial.print('\n'); + console.print('\n'); char reply[160]; reply[0] = 0; #ifdef ETHERNET_ENABLED if (!ethernet_handle_command(command, reply)) { #if MESH_ENABLE_HOST_CLI if (!the_mesh.handleHostCliSerialReply(command, reply)) { - the_mesh.handleCommand(0, command, reply); + the_mesh.handleUsbCommand(command, reply); } #else - the_mesh.handleCommand(0, command, reply); + the_mesh.handleUsbCommand(command, reply); #endif } #else #if MESH_ENABLE_HOST_CLI if (!the_mesh.handleHostCliSerialReply(command, reply)) { - the_mesh.handleCommand(0, command, reply); // NOTE: there is no sender_timestamp via serial! + the_mesh.handleUsbCommand(command, reply); } #else - the_mesh.handleCommand(0, command, reply); // NOTE: there is no sender_timestamp via serial! + the_mesh.handleUsbCommand(command, reply); #endif #endif if (reply[0]) { - Serial.print(" -> "); Serial.println(reply); + console.printf(" -> %s\r\n", reply); } command[0] = 0; // reset command buffer @@ -312,7 +337,7 @@ void loop() { if (userBtnDownAt == 0) { userBtnDownAt = millis(); } else if ((unsigned long)(millis() - userBtnDownAt) >= USER_BTN_HOLD_OFF_MILLIS) { - Serial.println("Powering off..."); + mesh::usbConsolePort().println("Powering off..."); board.powerOff(); // does not return } } else { @@ -326,6 +351,9 @@ void loop() { if (display_ready) ui_task.loop(); #endif rtc_clock.tick(); +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbTerminalPort(); +#endif #ifdef TBEAM_1W board.updateFanControl(); diff --git a/examples/simple_room_server/FloodRuleEngine.cpp b/examples/simple_room_server/FloodRuleEngine.cpp index 3d9753ae..1ae16c8a 100644 --- a/examples/simple_room_server/FloodRuleEngine.cpp +++ b/examples/simple_room_server/FloodRuleEngine.cpp @@ -6,6 +6,7 @@ #include #include +#include #include namespace { @@ -350,7 +351,7 @@ static bool commandMatches(const char* command, const char* base) { } static File openRead(FILESYSTEM* fs, const char* path) { - return fs->open(path); + return mesh::openFileRead(fs, path); } static File openWrite(FILESYSTEM* fs, const char* path) { diff --git a/examples/simple_room_server/MyMesh.cpp b/examples/simple_room_server/MyMesh.cpp index aebf1a90..263524f0 100644 --- a/examples/simple_room_server/MyMesh.cpp +++ b/examples/simple_room_server/MyMesh.cpp @@ -1,4 +1,6 @@ #include "MyMesh.h" +#include +#include #include #include #include @@ -1429,7 +1431,7 @@ void MyMesh::begin(FILESYSTEM *fs) { if (start_webui) { char wc_reply[160]; startWebConfig(false, wc_reply); - Serial.println(wc_reply); + mesh::usbConsolePort().printf("%s\r\n", wc_reply); } #endif @@ -1575,21 +1577,123 @@ void MyMesh::updateFloodAdvertTimer() { } void MyMesh::dumpLogFile() { -#if defined(RP2040_PLATFORM) - File f = _fs->open(PACKET_LOG_FILE, "r"); +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (hasPendingSerialOutput()) { + mesh::usbConsolePort().printf("Err - USB output busy\r\n"); + return; + } + serial_log_dump = mesh::openFileRead(_fs, PACKET_LOG_FILE); + serial_log_active = static_cast(serial_log_dump); + serial_log_remaining = serial_log_active ? serial_log_dump.size() : 0; + serial_log_pending_size = 0; + serial_log_eof_pending = true; + serial_log_skip_line = false; #else - File f = _fs->open(PACKET_LOG_FILE); -#endif + File f = mesh::openFileRead(_fs, PACKET_LOG_FILE); if (f) { while (f.available()) { int c = f.read(); if (c < 0) break; - Serial.print((char)c); + mesh::usbConsolePort().print((char)c); } f.close(); } +#endif } +#if MESH_ESP32_TINYUSB_NONBLOCKING +bool MyMesh::hasPendingSerialOutput() const { + return serial_log_active || serial_log_eof_pending; +} + +void MyMesh::cancelPendingSerialOutput() { + if (serial_log_active) serial_log_dump.close(); + serial_log_active = false; + serial_log_eof_pending = false; + serial_log_skip_line = false; + serial_log_remaining = 0; + serial_log_pending_size = 0; +} + +void MyMesh::servicePendingSerialOutput() { + Stream& console = mesh::usbConsolePort(); + if (!serial_log_active) { + // CommonCLI's synchronous EOF is suppressed until the queued dump ends. + static const char eof[] = " -> EOF\r\n"; + if (serial_log_eof_pending + && console.availableForWrite() >= static_cast(sizeof(eof) - 1) + && console.write(reinterpret_cast(eof), sizeof(eof) - 1) + == sizeof(eof) - 1) { + serial_log_eof_pending = false; + } + return; + } + + // Read at most one bounded record per mesh pass. Snapshotting the original + // file size prevents a busy radio's newly appended log from extending this + // command forever. A retained suffix survives temporary USB backpressure. + if (serial_log_skip_line) { + // Do not split a malformed overlong stored line around live packet logs. + // Skip it in bounded passes and substitute one explicit complete record. + size_t budget = sizeof(serial_log_pending); + while (budget-- > 0 && serial_log_remaining > 0) { + const int value = serial_log_dump.read(); + if (value < 0) { + serial_log_remaining = 0; + break; + } + --serial_log_remaining; + if (value == '\n') { + serial_log_skip_line = false; + break; + } + } + if (serial_log_remaining == 0) serial_log_skip_line = false; + if (serial_log_skip_line) return; + static const char omitted[] = "[USB log line omitted: exceeds 640 bytes]\r\n"; + memcpy(serial_log_pending, omitted, sizeof(omitted) - 1); + serial_log_pending_size = sizeof(omitted) - 1; + } else if (serial_log_pending_size == 0) { + while (serial_log_remaining > 0 + && serial_log_pending_size < sizeof(serial_log_pending)) { + const int value = serial_log_dump.read(); + if (value < 0) { + serial_log_remaining = 0; + break; + } + --serial_log_remaining; + serial_log_pending[serial_log_pending_size++] = static_cast(value); + if (value == '\n') break; + } + if (serial_log_pending_size == sizeof(serial_log_pending) + && serial_log_pending[serial_log_pending_size - 1] != '\n') { + serial_log_pending_size = 0; + serial_log_skip_line = true; + return; + } + if (serial_log_remaining == 0 && serial_log_pending_size > 0 + && serial_log_pending[serial_log_pending_size - 1] != '\n') { + serial_log_pending[serial_log_pending_size++] = '\n'; + } + } + if (serial_log_pending_size > 0 + && console.availableForWrite() >= static_cast(serial_log_pending_size)) { + size_t written = console.write( + reinterpret_cast(serial_log_pending), serial_log_pending_size); + if (written > serial_log_pending_size) written = serial_log_pending_size; + serial_log_pending_size -= written; + if (written > 0 && serial_log_pending_size > 0) { + memmove(serial_log_pending, serial_log_pending + written, serial_log_pending_size); + } + } + if (serial_log_remaining == 0 && serial_log_pending_size == 0) { + serial_log_dump.close(); + serial_log_active = false; + } +} +#endif + + bool MyMesh::setTxPower(int8_t power_dbm) { return radio_driver.setTxPower(power_dbm); } @@ -2241,14 +2345,16 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply } } } else if (sender_timestamp == 0 && strcmp(command, "get acl") == 0) { - Serial.println("ACL:"); + mesh::usbConsolePort().printf("ACL:\r\n"); for (int i = 0; i < acl.getNumClients(); i++) { auto c = acl.getClientByIdx(i); if (c->permissions == 0) continue; // skip deleted (or guest) entries - Serial.printf("%02X ", c->permissions); - mesh::Utils::printHex(Serial, c->id.pub_key, PUB_KEY_SIZE); - Serial.printf("\n"); + // Admit each line together so concurrent USB diagnostics cannot split + // a public key or insert text between its permission prefix and value. + char public_key[PUB_KEY_SIZE * 2 + 1]; + mesh::Utils::toHex(public_key, c->id.pub_key, PUB_KEY_SIZE); + mesh::usbConsolePort().printf("%02X %s\n", c->permissions, public_key); } reply[0] = 0; #if defined(WITH_MQTT_NEIGHBORS) @@ -2309,6 +2415,10 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply // handled by the role-independent mesh clock synchronizer } else { _cli.handleCommand(sender_timestamp, command, reply); // common CLI commands +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (sender_timestamp == 0 && serial_log_eof_pending + && strcmp(reply, " EOF") == 0) reply[0] = 0; +#endif } } @@ -2323,6 +2433,9 @@ void MyMesh::loop() { // Check radio FIRST to ensure we don't miss incoming packets // MQTT processing can take time, so we prioritize radio reception mesh::Mesh::loop(); +#if MESH_ESP32_TINYUSB_NONBLOCKING + servicePendingSerialOutput(); +#endif _cli.loop(); _clock_sync.loop(); #if MESH_ENABLE_TELEMETRY_HISTORY @@ -2477,7 +2590,7 @@ void MyMesh::loop() { // blocks the loop until reboot, then free a running bridge for heap headroom. // Remember its state: an OTA request must not enable MQTT that an operator // had deliberately stopped. - Serial.println("OTA: starting update"); + mesh::usbConsolePort().printf("OTA: starting update\r\n"); const bool bridge_was_running = bridge && bridge->isRunning(); drainOutbound(OTA_TX_DRAIN_TIMEOUT_MS); @@ -2492,18 +2605,18 @@ void MyMesh::loop() { // ownership is uncertain until a subsequent clean start/stop cycle. may_flash = bridge && bridge->canFlashAfterStop(); if (!may_flash) { - Serial.println("OTA: aborted, MQTT stop did not complete cleanly"); + mesh::usbConsolePort().printf("OTA: aborted, MQTT stop did not complete cleanly\r\n"); } else { // TODO: Replace this mitigation with a real MQTT task-exit/join barrier. delay(OTA_MQTT_STOP_SETTLE_MS); } } else if (!may_flash) { - Serial.println("OTA: aborted, prior MQTT stop did not complete cleanly"); + mesh::usbConsolePort().printf("OTA: aborted, prior MQTT stop did not complete cleanly\r\n"); } char ota_reply[160]; if (may_flash && !_cli.getBoard()->otaFromManifest(getFirmwareVer(), false, ota_reply)) { - Serial.print("OTA: aborted - "); Serial.println(ota_reply); + mesh::usbConsolePort().printf("OTA: aborted - %s\r\n", ota_reply); may_flash = false; } @@ -2511,7 +2624,7 @@ void MyMesh::loop() { // bridge that was running before this attempt; leave an intentionally // stopped bridge stopped after any OTA refusal or download failure. if (!may_flash && bridge_was_running) { - Serial.println("OTA: resuming bridge"); + mesh::usbConsolePort().printf("OTA: resuming bridge\r\n"); setBridgeState(true); } } diff --git a/examples/simple_room_server/MyMesh.h b/examples/simple_room_server/MyMesh.h index 6146ac68..c04b729a 100644 --- a/examples/simple_room_server/MyMesh.h +++ b/examples/simple_room_server/MyMesh.h @@ -158,6 +158,15 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks, #endif { FILESYSTEM* _fs; +#if MESH_ESP32_TINYUSB_NONBLOCKING + File serial_log_dump; + size_t serial_log_remaining = 0; + size_t serial_log_pending_size = 0; + char serial_log_pending[640]; + bool serial_log_active = false; + bool serial_log_eof_pending = false; + bool serial_log_skip_line = false; +#endif uint32_t last_millis; uint64_t uptime_millis; unsigned long next_local_advert, next_flood_advert; @@ -454,6 +463,12 @@ public: } void dumpLogFile() override; +#if MESH_ESP32_TINYUSB_NONBLOCKING + // Large local-only replies advance between radio service passes. + bool hasPendingSerialOutput() const; + void servicePendingSerialOutput(); + void cancelPendingSerialOutput(); +#endif bool setTxPower(int8_t power_dbm) override; bool setRxPowerSaving(bool enable, uint32_t rx_us, uint32_t sleep_us) override; void recalibrateNoiseFloor() override { _radio->recalibrateNoiseFloor(); } diff --git a/examples/simple_room_server/UITask.cpp b/examples/simple_room_server/UITask.cpp index 1a1efc56..9fdbce21 100644 --- a/examples/simple_room_server/UITask.cpp +++ b/examples/simple_room_server/UITask.cpp @@ -1,6 +1,7 @@ #include "UITask.h" #include "target.h" #include +#include #include #include @@ -42,7 +43,7 @@ void UITask::applyDisplayFlip() { _display->setFlipped(_flip_seen != 0); // Logged unconditionally: this is persisted config, so it survives a reflash // and is otherwise invisible when someone is chasing a wrong orientation. - Serial.printf("Display: flip %s\n", _flip_seen ? "on (rotated 180)" : "off"); + mesh::usbConsolePort().printf("Display: flip %s\n", _flip_seen ? "on (rotated 180)" : "off"); #ifdef DISPLAY_REDRAW_ON_CHANGE _frame_valid = false; #endif @@ -347,7 +348,7 @@ void UITask::toggleDisplay(const char* source) { _display->turnOn(); } #ifdef DISPLAY_TOUCH_DEBUG - Serial.printf("Display: %s -> %s\n", source, _display->isOn() ? "on" : "off"); + mesh::usbConsolePort().printf("Display: %s -> %s\n", source, _display->isOn() ? "on" : "off"); #else (void)source; #endif @@ -369,7 +370,7 @@ void UITask::loop() { toggleDisplay("button"); } else if (ev == BUTTON_EVENT_LONG_PRESS) { _display->turnOn(); - Serial.println("Powering Off"); + mesh::usbConsolePort().printf("Powering Off\r\n"); _powering_off_at = millis() + POWEROFF_DELAY; #ifdef DISPLAY_REDRAW_ON_CHANGE _frame_valid = false; diff --git a/examples/simple_room_server/main.cpp b/examples/simple_room_server/main.cpp index 7f982b8d..d5b58b77 100644 --- a/examples/simple_room_server/main.cpp +++ b/examples/simple_room_server/main.cpp @@ -44,6 +44,9 @@ unsigned long POWERSAVING_FIRSTSLEEP_SECS = 120; // The first sleep (if enabled) void setup() { Serial.begin(115200); +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::beginUsbLoggingPort(); +#endif #if MESH_PACKET_LOGGING mesh::serialLogBegin(); #endif @@ -112,8 +115,9 @@ void setup() { return; } - Serial.print("Room ID: "); - mesh::Utils::printHex(Serial, the_mesh.self_id.pub_key, PUB_KEY_SIZE); Serial.println(); + Stream& console = mesh::usbConsolePort(); + console.print("Room ID: "); + mesh::Utils::printHex(console, the_mesh.self_id.pub_key, PUB_KEY_SIZE); console.println(); command[0] = 0; #ifdef ETHERNET_ENABLED @@ -152,14 +156,27 @@ void loop() { #if defined(NRF52_PLATFORM) board.feedWatchdog(the_mesh.getNodePrefs()->system_watchdog_enabled != 0); #endif + bool usb_ready = true; +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbLoggingPort(); + mesh::serviceUsbTerminalPort(); + if (mesh::takeUsbTerminalSessionReset()) { + command[0] = 0; + the_mesh.cancelPendingSerialOutput(); + } + usb_ready = mesh::tryCompleteUsbTerminalSessionReset(); + usb_ready = usb_ready && !the_mesh.hasPendingSerialOutput() + && mesh::canAcceptUsbConsoleCommand(); +#endif + Stream& console = mesh::usbConsolePort(); int len = strlen(command); - while (Serial.available() && len < sizeof(command)-1) { - char c = Serial.read(); + while (usb_ready && console.available() && len < sizeof(command)-1) { + char c = console.read(); if (c != '\n') { command[len++] = c; command[len] = 0; } - Serial.print(c); + console.print(c); } if (len == sizeof(command)-1) { // command buffer full command[sizeof(command)-1] = '\r'; @@ -177,7 +194,7 @@ void loop() { the_mesh.handleCommand(0, command, reply); // NOTE: there is no sender_timestamp via serial! #endif if (reply[0]) { - Serial.print(" -> "); Serial.println(reply); + console.printf(" -> %s\r\n", reply); } command[0] = 0; // reset command buffer @@ -202,6 +219,9 @@ void loop() { if (display_ready) ui_task.loop(); #endif rtc_clock.tick(); +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbTerminalPort(); +#endif #ifdef TBEAM_1W board.updateFanControl(); #endif diff --git a/src/MeshCore.h b/src/MeshCore.h index 8b45f3d6..3279b2db 100644 --- a/src/MeshCore.h +++ b/src/MeshCore.h @@ -23,7 +23,8 @@ #define MAX_PATH_SIZE 64 #define MAX_TRANS_UNIT 255 -#if defined(ARDUINO) && defined(NRF52_PLATFORM) && \ +#if defined(ARDUINO) && \ + (defined(NRF52_PLATFORM) || MESH_ESP32_TINYUSB_NONBLOCKING) && \ ((defined(MESH_DEBUG) && MESH_DEBUG) || \ (defined(BRIDGE_DEBUG) && BRIDGE_DEBUG) || \ (defined(POWERSAVING_DEBUG) && POWERSAVING_DEBUG)) @@ -39,10 +40,11 @@ namespace mesh { // Adafruit_USBD_CDC::write() waits until the complete buffer has entered the // TinyUSB FIFO. That is normally convenient, but it can wait forever when a // host has opened the dedicated logging CDC without draining it. Keep debug -// output best-effort on nRF52: format into one bounded record, preserve a +// output best-effort on native TinyUSB: format into one bounded record, preserve a // visible truncation marker, and submit it only when the whole record fits in -// the FIFO snapshot. The atomic flag also prevents two debug callers from both -// relying on the same availableForWrite() result. +// the available FIFO or ESP32 software queue. The atomic flag also prevents +// overlapping debug formatter calls. The historical helper name +// is retained for callers; ESP32 TinyUSB uses the same bounded formatter. inline size_t nrf52DebugPrintf(const char* format, ...) { if (format == nullptr || !isUsbLoggingEnabled()) return 0; @@ -84,7 +86,7 @@ inline size_t nrf52DebugPrintf(const char* format, ...) { #if MESH_DEBUG && ARDUINO #include - #if defined(NRF52_PLATFORM) + #if defined(NRF52_PLATFORM) || MESH_ESP32_TINYUSB_NONBLOCKING #define MESH_DEBUG_PRINT(F, ...) do { mesh::nrf52DebugPrintf("DEBUG: " F, ##__VA_ARGS__); } while(0) #define MESH_DEBUG_PRINTLN(F, ...) do { mesh::nrf52DebugPrintf("DEBUG: " F "\n", ##__VA_ARGS__); } while(0) #else @@ -97,7 +99,7 @@ inline size_t nrf52DebugPrintf(const char* format, ...) { #endif #if BRIDGE_DEBUG && ARDUINO - #if defined(NRF52_PLATFORM) + #if defined(NRF52_PLATFORM) || MESH_ESP32_TINYUSB_NONBLOCKING #define BRIDGE_DEBUG_PRINTLN(F, ...) do { mesh::nrf52DebugPrintf("%s BRIDGE: " F, getLogDateTime(), ##__VA_ARGS__); } while(0) #else #define BRIDGE_DEBUG_PRINTLN(F, ...) do { if (mesh::isUsbLoggingEnabled() && mesh::usbLoggingPort().availableForWrite() > 0) { mesh::usbLoggingPort().printf("%s BRIDGE: " F, getLogDateTime(), ##__VA_ARGS__); } } while(0) @@ -108,7 +110,7 @@ inline size_t nrf52DebugPrintf(const char* format, ...) { #if POWERSAVING_DEBUG && ARDUINO #include - #if defined(NRF52_PLATFORM) + #if defined(NRF52_PLATFORM) || MESH_ESP32_TINYUSB_NONBLOCKING #define POWERSAVING_DEBUG_PRINT(F, ...) do { mesh::nrf52DebugPrintf("POWERSAVING: " F, ##__VA_ARGS__); } while(0) #define POWERSAVING_DEBUG_PRINTLN(F, ...) do { mesh::nrf52DebugPrintf("POWERSAVING: " F "\n", ##__VA_ARGS__); } while(0) #else diff --git a/src/helpers/ClientACL.cpp b/src/helpers/ClientACL.cpp index 98f195f7..c571d2eb 100644 --- a/src/helpers/ClientACL.cpp +++ b/src/helpers/ClientACL.cpp @@ -3,6 +3,7 @@ #include "ClientACLFileIntegrity.h" #include "ClientLoginPersistence.h" #include "ClientPathPersistence.h" +#include "FileRead.h" #if defined(NRF52_PLATFORM) #include "AtomicFileWriter.h" #endif @@ -11,11 +12,7 @@ static const uint8_t CONTACT_RECORD_VERSION_ALT_PATH = 1; static const uint8_t EMPTY_OUT_PATH[MAX_PATH_SIZE] = {}; static File openRead(FILESYSTEM* fs, const char* filename) { -#if defined(RP2040_PLATFORM) - return fs->open(filename, "r"); -#else - return fs->open(filename); -#endif + return mesh::openFileRead(fs, filename); } #if !defined(NRF52_PLATFORM) @@ -106,14 +103,17 @@ static mesh::StoredClientPathView storedClientPathForSave( #if !defined(NRF52_PLATFORM) static File openWrite(FILESYSTEM* _fs, const char* filename) { + if (_fs == NULL) return mesh::emptyFile(_fs); #if defined(STM32_PLATFORM) _fs->remove(filename); - return _fs->open(filename, FILE_O_WRITE); + File file = _fs->open(filename, FILE_O_WRITE); #elif defined(RP2040_PLATFORM) - return _fs->open(filename, "w"); + File file = _fs->open(filename, "w"); #else - return _fs->open(filename, "w", true); + File file = _fs->open(filename, "w", true); #endif + if (file && file.isDirectory()) file.close(); + return file; } static bool readMatches(File& file, const uint8_t* expected, size_t length) { @@ -220,21 +220,24 @@ static bool verifyContactsFile( } #endif +static bool loginReplayRecordCount(File& file, size_t* count) { + if (!file || file.isDirectory()) return false; + const size_t size = file.size(); + if (size < mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE) return false; + const size_t payload_size = + size - mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE; + if (payload_size % mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE != 0 + || payload_size / mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE + > mesh::MAX_CLIENT_LOGIN_REPLAY_IDENTITIES) return false; + *count = payload_size / mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE; + return true; +} + static bool validateLoginReplayFileIntegrity(FILESYSTEM* fs, const char* filename) { File file = openRead(fs, filename); - if (!file) return false; - const size_t size = file.size(); - if (size < mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE) { - file.close(); - return false; - } - const size_t payload_size = - size - mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE; - const size_t record_count = - payload_size / mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE; - if (payload_size % mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE != 0 - || record_count > mesh::MAX_CLIENT_LOGIN_REPLAY_IDENTITIES) { + size_t record_count = 0; + if (!loginReplayRecordCount(file, &record_count)) { file.close(); return false; } @@ -284,10 +287,11 @@ static bool readClientLoginReplayCeiling( return false; } File file = openRead(fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); - if (!file) return false; - const size_t record_count = - (file.size() - mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE) - / mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE; + size_t record_count = 0; + if (!loginReplayRecordCount(file, &record_count)) { + file.close(); + return false; + } bool success = true; for (size_t i = 0; success && i < record_count; i++) { uint8_t record_pubkey[PUB_KEY_SIZE]; @@ -344,11 +348,18 @@ static bool writeClientLoginReplayCeiling( } #endif - File source = openRead(fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); - const size_t record_count = source - ? (source.size() - mesh::CLIENT_LOGIN_REPLAY_TRAILER_SIZE) - / mesh::CLIENT_LOGIN_REPLAY_RECORD_SIZE - : 0; + File source = mesh::emptyFile(fs); + size_t record_count = 0; + if (fs->exists(mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH)) { + source = openRead(fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); + // A previously present image becoming unreadable is not a first login. + // Check again after opening so disappearance/truncation cannot underflow + // the trailer subtraction or replace historical replay boundaries. + if (!loginReplayRecordCount(source, &record_count)) { + source.close(); + return false; + } + } #if defined(NRF52_PLATFORM) mesh::AtomicFileWriter destination( fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); @@ -419,9 +430,147 @@ static bool writeClientLoginReplayCeiling( #endif } +// Scan the complete source, including its CRC, even when no record is selected. +// The optional destination preserves every record in its original order. This +// also handles duplicate historical identities without dropping tombstones. +template +static bool copyClampedLoginReplay( + File& source, Writer* destination, const uint8_t* selected_pubkey, + uint32_t now, ClientLoginReplayClampResult& result, uint32_t& source_crc) { + size_t record_count = 0; + if (!loginReplayRecordCount(source, &record_count)) return false; + uint32_t original_crc = 0xFFFFFFFFUL; + uint32_t output_crc = 0xFFFFFFFFUL; + for (size_t i = 0; i < record_count; i++) { + uint8_t pubkey[PUB_KEY_SIZE]; + uint32_t ceiling; + if (source.read(pubkey, sizeof(pubkey)) != (int)sizeof(pubkey) + || source.read((uint8_t*)&ceiling, sizeof(ceiling)) + != (int)sizeof(ceiling) + || ceiling == 0) return false; + original_crc = mesh::updateClientLoginReplayCRC( + original_crc, pubkey, sizeof(pubkey)); + original_crc = mesh::updateClientLoginReplayCRC( + original_crc, (const uint8_t*)&ceiling, sizeof(ceiling)); + if (selected_pubkey == NULL + || memcmp(pubkey, selected_pubkey, PUB_KEY_SIZE) == 0) { + result.stored_matched++; + if (ceiling > now) { + ceiling = now; + result.stored_changed++; + } + } + if (destination != NULL && !writeClientLoginReplayRecord( + *destination, pubkey, ceiling, &output_crc)) return false; + } + uint8_t magic[sizeof(mesh::CLIENT_LOGIN_REPLAY_MAGIC)]; + uint32_t stored_crc; + if (source.read(magic, sizeof(magic)) != (int)sizeof(magic) + || source.read((uint8_t*)&stored_crc, sizeof(stored_crc)) + != (int)sizeof(stored_crc) + || memcmp(magic, mesh::CLIENT_LOGIN_REPLAY_MAGIC, sizeof(magic)) != 0 + || stored_crc != (original_crc ^ 0xFFFFFFFFUL)) return false; + source_crc = stored_crc; + if (destination == NULL) return true; + const uint32_t final_crc = output_crc ^ 0xFFFFFFFFUL; + return destination->write(magic, sizeof(magic)) == sizeof(magic) + && destination->write((const uint8_t*)&final_crc, sizeof(final_crc)) + == sizeof(final_crc); +} + +bool ClientACL::clampLoginReplayTimestamps( + const uint8_t* pubkey, uint32_t now, + ClientLoginReplayClampResult& result) { + result = {}; + if (_fs == NULL || !login_replay_store_available || now == 0) return false; + + ClientLoginReplayClampResult pending = {}; + if (_fs->exists(mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH)) { + File source = openRead(_fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); + uint32_t original_crc = 0; + const bool valid = copyClampedLoginReplay( + source, (File*)NULL, pubkey, now, pending, original_crc); + source.close(); + if (!valid) { + login_replay_store_available = false; + return false; + } + if (pending.stored_changed != 0) { +#if !defined(NRF52_PLATFORM) + // A validated live image is authoritative. Only discard a stale backup + // after validation, and never enter publication with an uncleared one. + if (!mesh::removeClientLoginReplayArtifact( + _fs, mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH)) return false; +#endif + source = openRead(_fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); +#if defined(NRF52_PLATFORM) + mesh::AtomicFileWriter destination( + _fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); +#else + File destination = openWrite(_fs, mesh::CLIENT_LOGIN_REPLAY_TEMP_PATH); +#endif + if (!destination) { + source.close(); + return false; + } + ClientLoginReplayClampResult copied = {}; + uint32_t copied_crc = 0; + const bool copied_ok = copyClampedLoginReplay( + source, &destination, pubkey, now, copied, copied_crc) + && copied_crc == original_crc + && copied.stored_matched == pending.stored_matched + && copied.stored_changed == pending.stored_changed; + source.close(); +#if defined(NRF52_PLATFORM) + const bool published = destination.commit(copied_ok); +#else + destination.close(); + const bool verified = copied_ok && validateLoginReplayFileIntegrity( + _fs, mesh::CLIENT_LOGIN_REPLAY_TEMP_PATH); + const bool published = mesh::publishClientLoginReplayTemp( + _fs, verified, validateLoginReplayFileIntegrity); +#endif + if (!published) { + // Ordinary failed writes retain the old store. If rollback or a read + // also failed, do not let a missing/corrupt image become a first login. + File retained = openRead(_fs, mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH); + ClientLoginReplayClampResult ignored = {}; + uint32_t retained_crc = 0; + const bool retained_ok = copyClampedLoginReplay( + retained, (File*)NULL, pubkey, now, ignored, retained_crc) + && retained_crc == original_crc; + retained.close(); + if (!retained_ok) login_replay_store_available = false; + return false; + } + } + } else if (_fs->exists(mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH) + || _fs->exists(mesh::CLIENT_LOGIN_REPLAY_TEMP_PATH)) { + // Recovery is performed by load(). Do not mistake an interrupted/corrupt + // transaction for a never-created store during this explicit operation. + login_replay_store_available = false; + return false; + } + + for (int i = 0; i < num_clients; i++) { + ClientInfo& client = clients[i]; + if (pubkey != NULL && memcmp(client.id.pub_key, pubkey, PUB_KEY_SIZE) != 0) + continue; + pending.live_matched++; + if (client.last_timestamp > now) { + client.last_timestamp = now; + pending.live_changed++; + } + } + result = pending; + return true; +} + void ClientACL::load(FILESYSTEM* fs, const mesh::LocalIdentity& self_id) { _fs = fs; num_clients = 0; + login_replay_store_available = false; + if (_fs == NULL) return; #if defined(NRF52_PLATFORM) // AtomicFileWriter may leave only a harmless temp image when reset before // rename. The live image remains authoritative. @@ -447,11 +596,7 @@ void ClientACL::load(FILESYSTEM* fs, const mesh::LocalIdentity& self_id) { } #endif if (_fs->exists("/s_contacts")) { - #if defined(RP2040_PLATFORM) - File file = _fs->open("/s_contacts", "r"); - #else - File file = _fs->open("/s_contacts"); - #endif + File file = openRead(_fs, "/s_contacts"); if (file) { bool full = false; while (!full) { diff --git a/src/helpers/ClientACL.h b/src/helpers/ClientACL.h index 9f18e580..38b3342a 100644 --- a/src/helpers/ClientACL.h +++ b/src/helpers/ClientACL.h @@ -51,6 +51,13 @@ struct ClientInfo { #define MAX_CLIENTS 32 #endif +struct ClientLoginReplayClampResult { + uint16_t stored_matched; + uint16_t stored_changed; + uint16_t live_matched; + uint16_t live_changed; +}; + class ClientACL { FILESYSTEM* _fs; ClientInfo clients[MAX_CLIENTS]; @@ -81,6 +88,15 @@ public: uint32_t runtime_last_timestamp, uint8_t login_permissions); + // Explicitly authorized recovery only: the caller validates its transport, + // permissions and clock. A non-null selector is a complete PUB_KEY_SIZE key; + // null selects all records, including historical identities outside the ACL. + // Only lower existing values to now. Never insert/delete records or raise a + // floor. Publish durable changes before live changes; false returns zero + // counts and leaves live state untouched. Counts include duplicate records. + bool clampLoginReplayTimestamps(const uint8_t* pubkey, uint32_t now, + ClientLoginReplayClampResult& result); + ClientInfo* getClient(const uint8_t* pubkey, int key_len); ClientInfo* putClient(const mesh::Identity& id, uint8_t init_perms); bool applyPermissions(const mesh::LocalIdentity& self_id, const uint8_t* pubkey, int key_len, uint8_t perms); diff --git a/src/helpers/ESP32Board.cpp b/src/helpers/ESP32Board.cpp index f488ad17..2857c425 100644 --- a/src/helpers/ESP32Board.cpp +++ b/src/helpers/ESP32Board.cpp @@ -3,6 +3,7 @@ #include "ESP32Board.h" #include #include "UsbLogging.h" +#include "FileRead.h" #include "UserGpioPinPolicy.h" namespace mesh { @@ -116,7 +117,7 @@ class LightweightOTAServer { } void sendLog(WiFiClient& client) { - File log = SPIFFS.open("/packet_log", FILE_READ); + File log = mesh::openFileRead(&SPIFFS, "/packet_log"); if (!log) { static const char missing[] = "packet log not found"; sendResponse(client, 404, "Not Found", "text/plain", missing, sizeof(missing) - 1); @@ -1123,8 +1124,13 @@ void ESP32Board::enterDeepSleep(uint32_t secs) { sensors.getLocationProvider()->stop(); } - // Flush serial buffers + // Native TinyUSB must not wait for a host which has stopped reading. + // Keep the original flush behavior for UART and USB-Serial-JTAG targets. +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::serviceUsbTerminalPort(); +#else Serial.flush(); +#endif delay(100); // Clear stale wakeup sources to avoid ghost wakeup diff --git a/src/helpers/FileRead.h b/src/helpers/FileRead.h new file mode 100644 index 00000000..3d518735 --- /dev/null +++ b/src/helpers/FileRead.h @@ -0,0 +1,53 @@ +#pragma once + +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) +#include +#endif + +namespace mesh { + +// Adafruit LittleFS has no default File constructor: even a closed handle +// needs a valid filesystem owner. Its owner-only constructor does no I/O, so +// InternalFS is a safe fallback when the requested filesystem is unavailable. +template +auto emptyFile(Filesystem* fs) +#if defined(RP2040_PLATFORM) + -> decltype(fs->open(static_cast(nullptr), "r")) { + using FileType = decltype(fs->open(static_cast(nullptr), "r")); +#else + -> decltype(fs->open(static_cast(nullptr))) { + using FileType = decltype(fs->open(static_cast(nullptr))); +#endif +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + if (fs != nullptr) return FileType(*fs); + return FileType(InternalFS); +#else + (void)fs; + return FileType(); +#endif +} + +// Arduino-ESP32 SPIFFS can return a truthy directory handle when a missing +// pathname is opened for reading. A state-file reader must not interpret that +// zero-length directory as a valid empty file. Keep real directory enumeration +// on the filesystem's ordinary open() API. +template +auto openFileRead(Filesystem* fs, const char* path) +#if defined(RP2040_PLATFORM) + -> decltype(fs->open(path, "r")) { + using FileType = decltype(fs->open(path, "r")); +#else + -> decltype(fs->open(path)) { + using FileType = decltype(fs->open(path)); +#endif + if (fs == nullptr || path == nullptr || !fs->exists(path)) return emptyFile(fs); +#if defined(RP2040_PLATFORM) + FileType file = fs->open(path, "r"); +#else + FileType file = fs->open(path); +#endif + if (file && file.isDirectory()) file.close(); + return file; +} + +} // namespace mesh diff --git a/src/helpers/ReplayResetCommand.h b/src/helpers/ReplayResetCommand.h new file mode 100644 index 00000000..381ee5c6 --- /dev/null +++ b/src/helpers/ReplayResetCommand.h @@ -0,0 +1,233 @@ +#pragma once + +#include +#include +#include + +namespace mesh { + +enum class ReplayResetKind : uint8_t { + NotReplay, + Invalid, + ExactKey, + ExactKeyConfirm, + AllConfirm, +}; + +struct ReplayResetRequest { + ReplayResetKind kind; + uint8_t key[32]; + uint8_t token[16]; +}; + +namespace replay_reset_detail { + +inline bool space(char value) { + return value == ' ' || value == '\t' || value == '\r' || value == '\n'; +} + +inline const char* skipSpace(const char* text) { + while (space(*text)) ++text; + return text; +} + +inline char lower(char value) { + return value >= 'A' && value <= 'Z' ? (char)(value + ('a' - 'A')) : value; +} + +inline bool word(const char* text, size_t length, const char* expected) { + size_t i = 0; + while (i < length && expected[i] != 0) { + if (lower(text[i]) != expected[i]) return false; + ++i; + } + return i == length && expected[i] == 0; +} + +inline size_t wordLength(const char* text) { + size_t length = 0; + while (text[length] != 0 && !space(text[length])) ++length; + return length; +} + +inline int hexDigit(char value) { + if (value >= '0' && value <= '9') return value - '0'; + value = lower(value); + return value >= 'a' && value <= 'f' ? value - 'a' + 10 : -1; +} + +inline bool hex(const char* text, size_t length, uint8_t* output, + size_t output_length) { + if (length != output_length * 2U) return false; + for (size_t i = 0; i < output_length; ++i) { + const int high = hexDigit(text[2U * i]); + const int low = hexDigit(text[2U * i + 1U]); + if (high < 0 || low < 0) return false; + output[i] = (uint8_t)((high << 4) | low); + } + return true; +} + +} // namespace replay_reset_detail + +// Use this same parser before the receiver's timestamp mutation and in the +// command handler. Even malformed replay commands must not advance a sender's +// live floor: an old consumed confirmation could otherwise poison it again. +// This parser supplies no authentication or transport permission by itself. +inline ReplayResetKind parseReplayResetCommand(const char* command, + ReplayResetRequest& request) { + memset(&request, 0, sizeof(request)); + request.kind = ReplayResetKind::NotReplay; + if (command == nullptr) return request.kind; + using namespace replay_reset_detail; + const char* text = skipSpace(command); + // Match the optional two-character companion CLI response prefix. + if (text[0] != 0 && text[1] != 0 && text[2] == '|') { + text = skipSpace(text + 3); + } + const size_t verb_length = wordLength(text); + if (!word(text, verb_length, "replay")) { + // Reserve dotted replay subcommands as malformed members of this family. + if (verb_length > 6U && word(text, 6U, "replay") && text[6] == '.') { + request.kind = ReplayResetKind::Invalid; + } + return request.kind; + } + request.kind = ReplayResetKind::Invalid; + text = skipSpace(text + verb_length); + const size_t action_length = wordLength(text); + if (!word(text, action_length, "reset")) return request.kind; + text = skipSpace(text + action_length); + const size_t key_length = wordLength(text); + if (word(text, key_length, "all")) { + text = skipSpace(text + key_length); + const size_t confirmation_length = wordLength(text); + if (word(text, confirmation_length, "confirm") + && *skipSpace(text + confirmation_length) == 0) { + request.kind = ReplayResetKind::AllConfirm; + } + return request.kind; + } + if (!hex(text, key_length, request.key, sizeof(request.key))) return request.kind; + text = skipSpace(text + key_length); + if (*text == 0) { + request.kind = ReplayResetKind::ExactKey; + return request.kind; + } + const size_t token_length = wordLength(text); + if (!hex(text, token_length, request.token, sizeof(request.token)) + || *skipSpace(text + token_length) != 0) return request.kind; + request.kind = ReplayResetKind::ExactKeyConfirm; + return request.kind; +} + +// One short-lived challenge binds a reset to both the authenticated issuer and +// exact target. It is deliberately RAM-only: a reboot invalidates all captured +// confirmations. Disclose the same token on retries for the first two minutes; +// then retain it for confirmation only until the original five-minute deadline. +// Retries never replace the token or restart either window. +// Call consume() before attempting durable state publication; +// a failed write requires a new challenge rather than making an old one usable. +class ReplayResetNonce { +public: + static constexpr size_t KEY_SIZE = 32; + static constexpr size_t TOKEN_SIZE = 16; + static constexpr uint32_t RESEND_WINDOW_MILLIS = 120000; + static constexpr uint32_t LIFETIME_MILLIS = 300000; + static constexpr int64_t CLOCK_TOLERANCE_SECONDS = 5; + + enum class IssueResult : uint8_t { + Issued, Reused, AwaitingConfirmation, Busy, Invalid + }; + + ReplayResetNonce() { clear(); } + + IssueResult issue(const uint8_t* issuer, const uint8_t* target, + const uint8_t* random_token, uint32_t now_millis, + uint32_t now_epoch) { + if (issuer == nullptr || target == nullptr || now_epoch == 0) { + return IssueResult::Invalid; + } + if (isCurrent(now_millis, now_epoch)) { + if (!sameIdentity(issuer, target)) return IssueResult::Busy; + return now_millis - issued_millis_ < RESEND_WINDOW_MILLIS + ? IssueResult::Reused : IssueResult::AwaitingConfirmation; + } + clear(); + if (random_token == nullptr) return IssueResult::Invalid; + uint8_t nonzero = 0; + for (size_t i = 0; i < TOKEN_SIZE; ++i) nonzero |= random_token[i]; + if (nonzero == 0) return IssueResult::Invalid; + memcpy(issuer_, issuer, KEY_SIZE); + memcpy(target_, target, KEY_SIZE); + memcpy(token_, random_token, TOKEN_SIZE); + issued_millis_ = now_millis; + issued_epoch_ = now_epoch; + active_ = true; + return IssueResult::Issued; + } + + bool matches(const uint8_t* issuer, const uint8_t* target, + const uint8_t* supplied_token, uint32_t now_millis, + uint32_t now_epoch) const { + if (issuer == nullptr || target == nullptr || supplied_token == nullptr + || !isCurrent(now_millis, now_epoch) || !sameIdentity(issuer, target)) { + return false; + } + uint8_t difference = 0; + for (size_t i = 0; i < TOKEN_SIZE; ++i) { + difference |= token_[i] ^ supplied_token[i]; + } + return difference == 0; + } + + bool consume(const uint8_t* issuer, const uint8_t* target, + const uint8_t* supplied_token, uint32_t now_millis, + uint32_t now_epoch) { + if (!matches(issuer, target, supplied_token, now_millis, now_epoch)) return false; + clear(); + return true; + } + + const uint8_t* token() const { return token_; } + + uint32_t remainingSeconds(uint32_t now_millis, uint32_t now_epoch) const { + if (!isCurrent(now_millis, now_epoch)) return 0; + return (LIFETIME_MILLIS - (now_millis - issued_millis_)) / 1000U; + } + + void clear() { + active_ = false; + memset(issuer_, 0, sizeof(issuer_)); + memset(target_, 0, sizeof(target_)); + memset(token_, 0, sizeof(token_)); + issued_millis_ = 0; + issued_epoch_ = 0; + } + +private: + bool sameIdentity(const uint8_t* issuer, const uint8_t* target) const { + return memcmp(issuer_, issuer, KEY_SIZE) == 0 + && memcmp(target_, target, KEY_SIZE) == 0; + } + + bool isCurrent(uint32_t now_millis, uint32_t now_epoch) const { + if (!active_ || now_epoch == 0) return false; + const uint32_t elapsed = now_millis - issued_millis_; + if (elapsed >= LIFETIME_MILLIS) return false; + const uint64_t expected_epoch = (uint64_t)issued_epoch_ + elapsed / 1000U; + if (expected_epoch > UINT32_MAX) return false; + const int64_t clock_difference = (int64_t)now_epoch - (int64_t)expected_epoch; + return clock_difference >= -CLOCK_TOLERANCE_SECONDS + && clock_difference <= CLOCK_TOLERANCE_SECONDS; + } + + uint8_t issuer_[KEY_SIZE]; + uint8_t target_[KEY_SIZE]; + uint8_t token_[TOKEN_SIZE]; + uint32_t issued_millis_; + uint32_t issued_epoch_; + bool active_; +}; + +} // namespace mesh diff --git a/src/helpers/SimpleMeshTables.h b/src/helpers/SimpleMeshTables.h index f358fdfe..fc3c5435 100644 --- a/src/helpers/SimpleMeshTables.h +++ b/src/helpers/SimpleMeshTables.h @@ -465,6 +465,27 @@ public: const RecentRepeaterInfo* getRecentRepeaterBySortedIdx(int idx_wanted) const { return getRecentRepeaterBySortedIdxFiltered(idx_wanted, NULL, 0); } + // Advance a cooperative live view in one bounded table scan. Unlike a + // rank lookup, this never replays every earlier rank as the list grows. + // The caller copies the last accepted row: intervening RX may change the + // live ordering, but cannot invalidate that saved sort key or force a + // snapshot allocation. The existing paginated/rank APIs are unchanged. + const RecentRepeaterInfo* getNextRecentRepeaterBySortKey( + const RecentRepeaterInfo* after, int after_index, int& result_index) const { + const RecentRepeaterInfo* best = NULL; + result_index = -1; + for (int i = 0; i < _recent_repeater_count; ++i) { + const RecentRepeaterInfo* info = &_recent_repeaters[i]; + if (after != NULL && !recentRepeaterComesBefore(*after, after_index, *info, i)) { + continue; + } + if (best == NULL || recentRepeaterComesBefore(*info, i, *best, result_index)) { + best = info; + result_index = i; + } + } + return best; + } int getRecentRepeaterMatchingCount(const uint8_t* search_prefix, uint8_t search_prefix_len) const { if (_max_recent_repeaters == 0 || search_prefix == NULL diff --git a/src/helpers/UsbLogging.cpp b/src/helpers/UsbLogging.cpp index 6a384f1a..2f79dd79 100644 --- a/src/helpers/UsbLogging.cpp +++ b/src/helpers/UsbLogging.cpp @@ -3,6 +3,7 @@ #if defined(ARDUINO) #include #include +#include #if defined(ESP32) && defined(ARDUINO_USB_MODE) && ARDUINO_USB_MODE == 1 \ && defined(ARDUINO_USB_CDC_ON_BOOT) && ARDUINO_USB_CDC_ON_BOOT \ @@ -14,9 +15,13 @@ #define MESH_ESP32_HWCDC_SESSION_GUARD 0 #endif -#if defined(NRF52_PLATFORM) || MESH_ESP32_HWCDC_SESSION_GUARD +#if defined(NRF52_PLATFORM) || MESH_ESP32_HWCDC_SESSION_GUARD \ + || MESH_ESP32_TINYUSB_NONBLOCKING #include "NonBlockingWriteStream.h" #endif +#if MESH_ESP32_TINYUSB_NONBLOCKING + #include "esp32-hal-tinyusb.h" +#endif #if MESH_ESP32_HWCDC_SESSION_GUARD #include "UsbAsciiBinarySwitch.h" #endif @@ -72,6 +77,208 @@ class NullUsbLoggingStream : public Stream { static NullUsbLoggingStream null_usb_logging_stream; +#if MESH_ESP32_TINYUSB_NONBLOCKING +// Arduino-ESP32 2.0.17 constructs Serial as USBCDC(0). Keep its RX queue and +// existing descriptors/callbacks, but never call its potentially unbounded +// write()/flush() or its mutex-taking availableForWrite(). The native CDC +// application API makes one FIFO attempt and can safely return short; its +// flush starts an available endpoint transfer without waiting for the host. +// TinyUSB still takes short RTOS FIFO/endpoint mutexes internally: this is a +// no-host-progress-wait contract, not a claim that the USB stack is lock-free. +static std::atomic esp32_tinyusb_reset_generation{0}; +static std::atomic esp32_tinyusb_clean_generation{0}; +static std::atomic_flag esp32_tinyusb_queue_busy = ATOMIC_FLAG_INIT; +static std::atomic esp32_tinyusb_terminal_discard_pending{false}; +static std::atomic esp32_tinyusb_terminal_dropped_bytes{0}; +static uint32_t esp32_tinyusb_terminal_reported_dropped_bytes = 0; +static uint32_t esp32_tinyusb_taken_reset_generation = 0; +static bool esp32_tinyusb_event_handler_registered = false; +static bool esp32_tinyusb_was_connected = false; + +static bool canAccessEsp32TinyUsb(void*) { + return !xPortInIsrContext() && tud_cdc_n_connected(0) + && esp32_tinyusb_clean_generation.load(std::memory_order_acquire) + == esp32_tinyusb_reset_generation.load(std::memory_order_acquire); +} + +static void handleEsp32TinyUsbEvent(void*, esp_event_base_t, int32_t event_id, + void* event_data) { + // This is the Arduino event task, not TinyUSB's owner. Only publish an + // epoch: queue cleanup and endpoint access remain in application service. + bool closed = event_id == ARDUINO_USB_CDC_DISCONNECTED_EVENT; + if (event_id == ARDUINO_USB_CDC_LINE_STATE_EVENT && event_data != nullptr) { + const auto* event = static_cast(event_data); + closed = !event->line_state.dtr; + } + if (closed) { + esp32_tinyusb_reset_generation.fetch_add(1, std::memory_order_acq_rel); + } +} + +class Esp32TinyUsbFifoStream : public Stream { + public: + int available() override { + const int count = Serial.available(); + return count > 0 ? count : 0; + } + int read() override { return Serial.read(); } + int peek() override { return Serial.peek(); } + void flush() override {} + int availableForWrite() override { + return canAccessEsp32TinyUsb(nullptr) + ? static_cast(tud_cdc_n_write_available(0)) : 0; + } + size_t write(uint8_t value) override { return write(&value, 1); } + size_t write(const uint8_t* data, + size_t size) override { + if (data == nullptr || size == 0 || !canAccessEsp32TinyUsb(nullptr)) return 0; + const size_t available = tud_cdc_n_write_available(0); + const size_t attempt = size < available ? size : available; + if (attempt == 0) return 0; + const size_t written = tud_cdc_n_write(0, data, attempt); + (void)tud_cdc_n_write_flush(0); + return written; + } +}; + +static Esp32TinyUsbFifoStream esp32_tinyusb_fifo_port; +static size_t writeEsp32TinyUsbOnce(void*, const uint8_t* data, size_t size) { + return esp32_tinyusb_fifo_port.write(data, size); +} +static SingleAttemptNonBlockingStream nonblocking_esp32_tinyusb_port( + esp32_tinyusb_fifo_port, writeEsp32TinyUsbOnce, nullptr, + canAccessEsp32TinyUsb); +static AtomicWholeRecordNonBlockingStream<11> nonblocking_esp32_tinyusb_mota_port( + nonblocking_esp32_tinyusb_port); + +// The ESP32 CDC FIFO is just 64 bytes. Retain complete producer writes in ONE +// chronological queue: independent log/reply queues would interleave their +// partial lines as that small FIFO drains. Diagnostics leave most of the bounded +// queue reserved for functional replies; congestion drops records, not LoRa. +static constexpr size_t esp32_tinyusb_text_capacity = 4096; +static constexpr size_t esp32_tinyusb_functional_reserve = 3072; +static constexpr size_t esp32_tinyusb_log_record_capacity = 640; +// ESP32 Print::printf has no Adafruit 256-byte scratch-length bug. Disable +// that nRF52-specific sentinel without changing the existing nRF52 facade. +static BufferedNonBlockingWriteStream esp32_tinyusb_text_queue( + nonblocking_esp32_tinyusb_port); + +template +class Esp32TinyUsbBufferedStream : public Stream { + public: + int available() override { return nonblocking_esp32_tinyusb_port.available(); } + int read() override { return nonblocking_esp32_tinyusb_port.read(); } + int peek() override { return nonblocking_esp32_tinyusb_port.peek(); } + void flush() override { serviceUsbTerminalPort(); } + int availableForWrite() override { + if (esp32_tinyusb_queue_busy.test_and_set(std::memory_order_acquire)) return 0; + const int available = canQueue() + ? esp32_tinyusb_text_queue.availableForWrite() : 0; + esp32_tinyusb_queue_busy.clear(std::memory_order_release); + if (!Diagnostic) return available; + // SerialLogLine must admit its entire <=640-byte record instead of + // splitting it when only a few bytes of diagnostic capacity remain. + return available >= static_cast(esp32_tinyusb_functional_reserve + + esp32_tinyusb_log_record_capacity) + ? available - esp32_tinyusb_functional_reserve : 0; + } + size_t write(uint8_t value) override { return write(&value, 1); } + size_t write(const uint8_t* data, + size_t size) override { + if (data == nullptr || size == 0) return 0; + if (esp32_tinyusb_queue_busy.test_and_set(std::memory_order_acquire)) { + noteDropped(size); + return 0; + } + size_t written = 0; + if (canQueue()) { + const size_t available = esp32_tinyusb_text_queue.availableForWrite(); + if (!Diagnostic || (available >= esp32_tinyusb_functional_reserve + && size <= available - esp32_tinyusb_functional_reserve)) { + written = esp32_tinyusb_text_queue.write(data, size); + } + } + if (written != size) noteDropped(size - written); + esp32_tinyusb_queue_busy.clear(std::memory_order_release); + return written; + } + + private: + bool canQueue() const { + return canAccessEsp32TinyUsb(nullptr) + && (Diagnostic ? isUsbLoggingEnabled() + : !esp32_tinyusb_terminal_discard_pending.load( + std::memory_order_acquire)); + } + void noteDropped(size_t size) { + if (!Diagnostic && !xPortInIsrContext() && tud_cdc_n_connected(0)) { + esp32_tinyusb_terminal_dropped_bytes.fetch_add( + static_cast(size), std::memory_order_relaxed); + } + } +}; + +static Esp32TinyUsbBufferedStream buffered_esp32_tinyusb_logging_port; +static Esp32TinyUsbBufferedStream buffered_esp32_tinyusb_terminal_port; + +static void clearEsp32TinyUsbTx(void*) { + (void)tud_cdc_n_write_clear(0); +} + +static void serviceEsp32TinyUsbPorts() { + if (xPortInIsrContext()) return; + if (esp32_tinyusb_queue_busy.test_and_set(std::memory_order_acquire)) return; + // Polling also catches a physical disconnect without a CDC line-state + // event. Events capture quick close/reopen pairs between service calls. + const bool connected = tud_cdc_n_connected(0); + if (esp32_tinyusb_was_connected && !connected) { + esp32_tinyusb_reset_generation.fetch_add(1, std::memory_order_acq_rel); + } + esp32_tinyusb_was_connected = connected; + const uint32_t generation = + esp32_tinyusb_reset_generation.load(std::memory_order_acquire); + if (generation != esp32_tinyusb_clean_generation.load(std::memory_order_acquire)) { + if (!nonblocking_esp32_tinyusb_port.tryRunExclusive(clearEsp32TinyUsbTx)) { + esp32_tinyusb_queue_busy.clear(std::memory_order_release); + return; + } + esp32_tinyusb_text_queue.discardPending(); + esp32_tinyusb_terminal_reported_dropped_bytes = + esp32_tinyusb_terminal_dropped_bytes.load(std::memory_order_relaxed); + // Do not purge Serial's RX queue here: the new host may already have sent + // its first query. Protocol owners reset their partial parser separately. + esp32_tinyusb_clean_generation.store(generation, std::memory_order_release); + } + if (esp32_tinyusb_terminal_discard_pending.exchange(false, + std::memory_order_acq_rel)) { + esp32_tinyusb_text_queue.discardPending(); + esp32_tinyusb_terminal_reported_dropped_bytes = + esp32_tinyusb_terminal_dropped_bytes.load(std::memory_order_relaxed); + } + if (canAccessEsp32TinyUsb(nullptr)) { + esp32_tinyusb_text_queue.service(); + const uint32_t dropped = + esp32_tinyusb_terminal_dropped_bytes.load(std::memory_order_relaxed); + if (dropped != esp32_tinyusb_terminal_reported_dropped_bytes + && esp32_tinyusb_text_queue.availableForWrite() >= 96) { + char marker[96]; + const int length = snprintf(marker, sizeof(marker), + "\r\n[USB terminal output dropped %lu bytes]\r\n", + static_cast( + dropped - esp32_tinyusb_terminal_reported_dropped_bytes)); + if (length > 0 && static_cast(length) < sizeof(marker) + && esp32_tinyusb_text_queue.write( + reinterpret_cast(marker), length) + == static_cast(length)) { + esp32_tinyusb_terminal_reported_dropped_bytes = dropped; + } + } + } + esp32_tinyusb_queue_busy.clear(std::memory_order_release); +} +#endif + #if MESH_ESP32_HWCDC_SESSION_GUARD // Every primary HWCDC role shares one producer gate. In particular, returning // this facade from usbLoggingPort() means a task which cached its Stream& before @@ -317,7 +524,13 @@ static uint32_t primary_usb_terminal_taken_reset_generation = 0; #endif static void setPlatformDebugOutputEnabled(bool enabled) { -#if defined(ESP32_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + // The framework putc hook bypasses the common producer gate. MeshCore + // diagnostics remain enabled through usbLoggingPort(); keep raw framework + // bytes from racing a binary mOTA/Companion record's capacity preflight. + (void)enabled; + Serial.setDebugOutput(false); +#elif defined(ESP32_PLATFORM) && defined(ENABLE_USB_INTERFACE) // Arduino-ESP32 log_e()/ESP-IDF diagnostics otherwise write straight to // the same UART/CDC stream used by Binary Companion. #if MESH_ESP32_HWCDC_SESSION_GUARD @@ -549,9 +762,18 @@ bool isUsbLoggingEnabled() { } void setUsbLoggingEnabled(bool enabled) { +#if MESH_ESP32_TINYUSB_NONBLOCKING + const bool was_enabled = isUsbLoggingEnabled(); +#endif usb_logging_enabled.store(enabled, std::memory_order_relaxed); usb_logging_preference_known.store(true, std::memory_order_relaxed); setPlatformDebugOutputEnabled(enabled); +#if MESH_ESP32_TINYUSB_NONBLOCKING + // Text shares one chronological queue. When leaving logging mode, discard + // its residual application bytes before later Binary/mOTA traffic can start. + // An in-flight producer is gated immediately and cleaned by the next service. + if (was_enabled && !enabled) discardUsbTerminalOutput(); +#endif } bool saveUsbLoggingBootPreference(bool enabled) { @@ -573,6 +795,12 @@ void beginUsbLoggingPort() { // ESP32 Companion stream; setUsbLoggingEnabled() restores them only when the // saved setting explicitly enables logging. setPlatformDebugOutputEnabled(isUsbLoggingEnabled()); +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (!esp32_tinyusb_event_handler_registered) { + Serial.onEvent(ARDUINO_USB_CDC_ANY_EVENT, handleEsp32TinyUsbEvent); + esp32_tinyusb_event_handler_registered = true; + } +#endif #if MESH_ESP32_HWCDC_SESSION_GUARD if (!esp32_hwcdc_event_handler_registered) { Serial.onEvent(ARDUINO_HW_CDC_ANY_EVENT, handleEsp32HwcdcEvent); @@ -603,6 +831,9 @@ void beginUsbLoggingPort() { } void serviceUsbLoggingPort() { +#if MESH_ESP32_TINYUSB_NONBLOCKING + serviceEsp32TinyUsbPorts(); +#endif #if defined(MESH_DUAL_CDC_LOGGING) const bool connected = dedicated_usb_logging_port_started && dedicated_usb_logging_port.dtr(); @@ -702,7 +933,12 @@ static void purgeEsp32HwcdcQueues(void* opaque) { #endif bool resetUsbCompanionTransport() { -#if MESH_ESP32_HWCDC_SESSION_GUARD +#if MESH_ESP32_TINYUSB_NONBLOCKING + esp32_tinyusb_reset_generation.fetch_add(1, std::memory_order_acq_rel); + serviceEsp32TinyUsbPorts(); + return esp32_tinyusb_clean_generation.load(std::memory_order_acquire) + == esp32_tinyusb_reset_generation.load(std::memory_order_acquire); +#elif MESH_ESP32_HWCDC_SESSION_GUARD // HWCDC owns RTOS queues, a TX mutex, an ISR, and an event task. Calling // end()/begin() here can delete those objects while a WiFi/MQTT/diagnostic // producer is writing. Close the independent transport gate first. Runtime @@ -776,7 +1012,9 @@ Stream& usbLoggingPort() { return null_usb_logging_stream; #else if (!isUsbLoggingEnabled()) return null_usb_logging_stream; - #if MESH_ESP32_HWCDC_SESSION_GUARD + #if MESH_ESP32_TINYUSB_NONBLOCKING + return buffered_esp32_tinyusb_logging_port; + #elif MESH_ESP32_HWCDC_SESSION_GUARD return guarded_esp32_hwcdc_port; #elif defined(NRF52_PLATFORM) return nonblocking_primary_usb_logging_port; @@ -787,7 +1025,9 @@ Stream& usbLoggingPort() { } Stream& usbCompanionPort() { -#if MESH_ESP32_HWCDC_SESSION_GUARD +#if MESH_ESP32_TINYUSB_NONBLOCKING + return nonblocking_esp32_tinyusb_port; +#elif MESH_ESP32_HWCDC_SESSION_GUARD return guarded_esp32_hwcdc_port; #elif MESH_NRF52_PRIMARY_USB_NONBLOCKING return nonblocking_primary_usb_companion_port; @@ -797,7 +1037,9 @@ Stream& usbCompanionPort() { } Stream& usbMotaPort() { -#if MESH_ESP32_HWCDC_SESSION_GUARD +#if MESH_ESP32_TINYUSB_NONBLOCKING + return nonblocking_esp32_tinyusb_mota_port; +#elif MESH_ESP32_HWCDC_SESSION_GUARD return guarded_esp32_hwcdc_mota_port; #elif MESH_NRF52_PRIMARY_USB_NONBLOCKING return nonblocking_primary_usb_mota_port; @@ -807,15 +1049,36 @@ Stream& usbMotaPort() { } Stream& usbTerminalPort() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + return buffered_esp32_tinyusb_terminal_port; +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) return buffered_primary_usb_terminal_port; #else return usbCompanionPort(); #endif } +Stream& usbConsolePort() { +#if MESH_ESP32_TINYUSB_NONBLOCKING + return buffered_esp32_tinyusb_terminal_port; +#else + return Serial; +#endif +} + +bool canAcceptUsbConsoleCommand() { +#if MESH_ESP32_TINYUSB_NONBLOCKING + return buffered_esp32_tinyusb_terminal_port.availableForWrite() + >= static_cast(esp32_tinyusb_functional_reserve); +#else + return true; +#endif +} + void serviceUsbTerminalPort() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + serviceEsp32TinyUsbPorts(); +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) const uint32_t reset_generation = primaryUsbSessionGeneration(); if (reset_generation != primary_usb_terminal_seen_reset_generation) { @@ -829,21 +1092,48 @@ void serviceUsbTerminalPort() { } void discardUsbTerminalOutput() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + esp32_tinyusb_terminal_discard_pending.store(true, std::memory_order_release); + if (esp32_tinyusb_queue_busy.test_and_set(std::memory_order_acquire)) return; + esp32_tinyusb_text_queue.discardPending(); + esp32_tinyusb_terminal_reported_dropped_bytes = + esp32_tinyusb_terminal_dropped_bytes.load(std::memory_order_relaxed); + esp32_tinyusb_terminal_discard_pending.store(false, std::memory_order_release); + esp32_tinyusb_queue_busy.clear(std::memory_order_release); +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) buffered_primary_usb_terminal_port.discardPending(); #endif } bool hasPendingUsbTerminalOutput() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + if (esp32_tinyusb_queue_busy.test_and_set(std::memory_order_acquire)) return true; + const bool pending = esp32_tinyusb_text_queue.queuedByteCount() != 0; + esp32_tinyusb_queue_busy.clear(std::memory_order_release); + return pending; +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) return buffered_primary_usb_terminal_port.queuedByteCount() != 0; #else return false; #endif } +uint32_t usbTerminalDroppedBytes() { +#if MESH_ESP32_TINYUSB_NONBLOCKING + return esp32_tinyusb_terminal_dropped_bytes.load(std::memory_order_relaxed); +#else + return 0; +#endif +} + bool takeUsbTerminalSessionReset() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + const uint32_t reset_generation = + esp32_tinyusb_reset_generation.load(std::memory_order_acquire); + if (reset_generation == esp32_tinyusb_taken_reset_generation) return false; + esp32_tinyusb_taken_reset_generation = reset_generation; + return true; +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) const uint32_t reset_generation = primaryUsbSessionGeneration(); if (reset_generation == primary_usb_terminal_taken_reset_generation) { @@ -881,7 +1171,11 @@ static void completePrimaryUsbSessionReset(void*) { #endif bool tryCompleteUsbTerminalSessionReset() { -#if defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) +#if MESH_ESP32_TINYUSB_NONBLOCKING + serviceEsp32TinyUsbPorts(); + return esp32_tinyusb_clean_generation.load(std::memory_order_acquire) + == esp32_tinyusb_reset_generation.load(std::memory_order_acquire); +#elif defined(NRF52_PLATFORM) && defined(ENABLE_USB_INTERFACE) const uint32_t settle_until = primary_usb_reset_settle_until.load(std::memory_order_acquire); if ((int32_t)(millis() - settle_until) < 0) return false; diff --git a/src/helpers/UsbLogging.h b/src/helpers/UsbLogging.h index a11b786e..3de340ae 100644 --- a/src/helpers/UsbLogging.h +++ b/src/helpers/UsbLogging.h @@ -14,6 +14,16 @@ #if defined(ARDUINO) #include +// ESP32-S2/S3 native USB CDC uses USBCDC, not the hardware USB-Serial-JTAG +// driver. Its write timeout only bounds a mutex, not waiting for FIFO space. +// All roles sharing this CDC therefore need the single-attempt transport. +#if defined(ESP32) && defined(ARDUINO_USB_MODE) && ARDUINO_USB_MODE == 0 \ + && defined(ARDUINO_USB_CDC_ON_BOOT) && ARDUINO_USB_CDC_ON_BOOT + #define MESH_ESP32_TINYUSB_NONBLOCKING 1 +#else + #define MESH_ESP32_TINYUSB_NONBLOCKING 0 +#endif + namespace mesh { #ifndef MESH_ESP32_USB_TX_BUFFER_SIZE @@ -59,11 +69,12 @@ void beginUsbLoggingPort(); // such as /dev/ttyACM1 or COM7. void serviceUsbLoggingPort(); Stream& usbLoggingPort(); -// Primary USB Companion data stream. On nRF52 this uses one direct TinyUSB -// FIFO attempt per write so a host-side open/close race cannot trap the main -// loop in Adafruit_USBD_CDC::write(). Other platforms retain Serial. +// Primary USB Companion data stream. Native TinyUSB on nRF52 and ESP32 uses +// one FIFO attempt per write, never the framework's wait-for-space loop. +// Callers retain and retry unwritten suffixes. Other platforms retain Serial +// or their hardware-CDC session facade. Stream& usbCompanionPort(); -// Serial mOTA requests are binary records of at most 11 bytes. On nRF52 this +// Serial mOTA requests are binary records of at most 11 bytes. On TinyUSB this // facade admits a request only when the complete record fits in CDC0's current // TX capacity, so a retry can never append to a prefix from the prior attempt. Stream& usbMotaPort(); @@ -72,11 +83,21 @@ Stream& usbMotaPort(); // it from the application loop; discard it before changing the CDC protocol or // after a host disconnect so stale text cannot prefix a later Binary session. Stream& usbTerminalPort(); +// Repeater/room-server console: protect ESP32 TinyUSB while preserving the +// historical raw Serial behavior of other platforms and roles. +Stream& usbConsolePort(); +// Diagnostic backlog alone must not starve console input. Native ESP32 CDC +// admits another command when its reserved functional capacity is available; +// other platforms preserve their existing command-processing policy. +bool canAcceptUsbConsoleCommand(); void serviceUsbTerminalPort(); void discardUsbTerminalOutput(); bool hasPendingUsbTerminalOutput(); +// Bytes refused while an ESP32 TinyUSB terminal host was connected. A visible +// overflow marker is emitted once queue capacity recovers; other ports return 0. +uint32_t usbTerminalDroppedBytes(); // Consume a primary-USB session boundary reported by the USB owner task: -// CDC0 DTR-low on nRF52, or a hardware CDC bus reset on ESP32. This is +// CDC0 DTR-low on TinyUSB, or a hardware CDC bus reset on ESP32. This is // independent of polling current line/SOF state; ESP32 retains that poll as a // fallback because its bundled framework event queue is finite. bool takeUsbTerminalSessionReset(); diff --git a/src/helpers/ota/OtaContext.h b/src/helpers/ota/OtaContext.h index bcf5163b..cc13cfad 100644 --- a/src/helpers/ota/OtaContext.h +++ b/src/helpers/ota/OtaContext.h @@ -23,8 +23,10 @@ #endif #if defined(OTA_FOLDER_SERIAL) #include "MotaSourceSerial.h" // relay an external folder served by a host daemon over the USB serial + #include "../UsbLogging.h" #ifndef OTA_FOLDER_SERIAL_STREAM #if defined(NRF52_PLATFORM) \ + || MESH_ESP32_TINYUSB_NONBLOCKING \ || (defined(ESP32) && defined(ARDUINO_USB_MODE) \ && ARDUINO_USB_MODE == 1 \ && defined(ARDUINO_USB_CDC_ON_BOOT) \ @@ -33,7 +35,6 @@ // Native USB serial-folder requests share the primary session facade. // It prevents a host reset from racing a cached mOTA Stream reference; // dedicated-UART overrides retain their normal stream. - #include "../UsbLogging.h" #define OTA_FOLDER_SERIAL_STREAM ::mesh::usbMotaPort() #define MESH_OTA_FOLDER_SERIAL_NONBLOCKING_USB 1 #else diff --git a/test/README.md b/test/README.md index 9bd6dd0d..c61f55e9 100644 --- a/test/README.md +++ b/test/README.md @@ -21,8 +21,17 @@ python3 test/test_color_theme.py # shared color-display dark-pale python3 test/test_indicator_font_recovery.py # Indicator TLS/SD font recovery contract python3 test/test_companion_terminal_profile.py # Companion CLI capability gates python3 test/test_client_login_profile_contract.py # ACL login ordering/role contract +python3 test/test_client_acl_spiffs.py # Actual ACL: first login, replay/reboot, failed storage +python3 test/test_replay_reset_command.py # Strict full keys and one-use recovery confirmations +python3 test/test_replay_reset_integration.py # Actual repeater handler: USB/LoRa permissions and persistence ordering +python3 test/test_regular_file_reads.py # SPIFFS phantom directories, listings, log HTTP status python3 test/test_esp32_full_partition.py # Full partition-preservation policy python3 test/test_esp32_usb_serial_hygiene.py # Single-TTY diagnostics/NVS contract +python3 test/test_esp32_tinyusb_role_hygiene.py # G2/room USB write coverage and bounded-list contracts +python3 test/test_esp32_tinyusb_cooperative_output.py # Real role dump/list pumps with host C++ stubs +python3 test/test_esp32_tinyusb_nonblocking.py # Native CDC stalled-host/64-byte-FIFO simulation (C++17 compiler) +python3 test/test_esp32_tinyusb_role_hygiene.py # Repeater/room nonblocking console and paced large replies +python3 test/test_esp32_tinyusb_cooperative_output.py # Real role pumps: large logs/listings, EOF, backpressure python3 test/test_temp_radio_reply_delivery_contract.py # TempRadio ACK path/barrier integration python3 test/test_tls_download_clock_gates.py # Fresh-NTP/TLS download integration contract ``` diff --git a/test/fixtures/client_acl_spiffs/mocks/Arduino.h b/test/fixtures/client_acl_spiffs/mocks/Arduino.h new file mode 100644 index 00000000..a796f3e0 --- /dev/null +++ b/test/fixtures/client_acl_spiffs/mocks/Arduino.h @@ -0,0 +1,125 @@ +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define MESH_DEBUG_PRINTLN(...) ((void)0) + +class FakeFilesystem; +struct FakeFileHandle { + FakeFilesystem* fs; + std::string path; + bool directory; + bool writable; + bool open = true; + size_t position = 0; +}; + +class File { + std::shared_ptr handle; +public: + File() = default; + File(FakeFilesystem* fs, const std::string& path, bool directory, bool writable) + : handle(std::make_shared(FakeFileHandle{fs, path, directory, writable})) {} + explicit operator bool() const { return handle && handle->open; } + bool isDirectory() const { return *this && handle->directory; } + size_t size() const; + int read(uint8_t* output, size_t length); + size_t write(const uint8_t* input, size_t length); + void close(); +}; + +class FakeFilesystem { +public: + std::map> files; + std::set unreadable; + std::set directories_on_read; + std::set directories_on_write; + std::map read_open_count; + std::string truncate_path; + size_t truncate_on_read_open = 0; + std::string fail_rename_from; + std::set fail_rename_from_paths; + std::string fail_remove; + std::string fail_open_write; + std::string corrupt_on_close; + size_t write_budget = std::numeric_limits::max(); + size_t bytes_written = 0; + size_t directory_closes = 0; + size_t missing_read_opens = 0; + + bool exists(const char* path) const { return files.count(path) != 0; } + File open(const char* path, const char* mode = "r", bool = false) { + const std::string name(path); + if (mode[0] == 'r') { + ++read_open_count[name]; + if (unreadable.count(name)) return File(); + if (!exists(path)) { + ++missing_read_opens; + // The real Arduino-ESP32 SPIFFS opendir fallback for missing paths. + return File(this, name, true, false); + } + if (truncate_path == name && read_open_count[name] == truncate_on_read_open) + files[name].resize(1); + return File(this, name, directories_on_read.count(name) != 0, false); + } + if (fail_open_write == name) return File(); + if (directories_on_write.count(name)) return File(this, name, true, true); + files[name].clear(); + return File(this, name, false, true); + } + bool remove(const char* path) { + return fail_remove != path && files.erase(path) != 0; + } + bool rename(const char* from, const char* to) { + if (fail_rename_from == from || fail_rename_from_paths.count(from) + || !exists(from) || exists(to)) return false; + files[to] = files[from]; + files.erase(from); + return true; + } +}; + +inline size_t File::size() const { + if (!*this || handle->directory) return 0; + auto found = handle->fs->files.find(handle->path); + return found == handle->fs->files.end() ? 0 : found->second.size(); +} +inline int File::read(uint8_t* output, size_t length) { + if (!*this || handle->directory) return 0; + auto found = handle->fs->files.find(handle->path); + if (found == handle->fs->files.end() || handle->position >= found->second.size()) return 0; + length = std::min(length, found->second.size() - handle->position); + std::memcpy(output, found->second.data() + handle->position, length); + handle->position += length; + return static_cast(length); +} +inline size_t File::write(const uint8_t* input, size_t length) { + if (!*this || handle->directory || !handle->writable) return 0; + auto& fs = *handle->fs; + length = std::min(length, fs.write_budget); + if (length == 0) return 0; + fs.write_budget -= length; + fs.bytes_written += length; + auto& data = fs.files[handle->path]; + data.resize(handle->position + length); + std::memcpy(data.data() + handle->position, input, length); + handle->position += length; + return length; +} +inline void File::close() { + if (!*this) return; + if (handle->directory) ++handle->fs->directory_closes; + if (handle->writable && handle->fs->corrupt_on_close == handle->path) { + auto& data = handle->fs->files[handle->path]; + if (!data.empty()) data.back() ^= 0x80; + } + handle->open = false; +} diff --git a/test/fixtures/client_acl_spiffs/mocks/Mesh.h b/test/fixtures/client_acl_spiffs/mocks/Mesh.h new file mode 100644 index 00000000..b92d45b7 --- /dev/null +++ b/test/fixtures/client_acl_spiffs/mocks/Mesh.h @@ -0,0 +1,17 @@ +#pragma once +#include "Arduino.h" +#define PUB_KEY_SIZE 32 +#define MAX_PATH_SIZE 64 +namespace mesh { +struct Identity { + uint8_t pub_key[PUB_KEY_SIZE]; + Identity() { std::memset(pub_key, 0, sizeof(pub_key)); } + explicit Identity(const uint8_t* key) { std::memcpy(pub_key, key, sizeof(pub_key)); } + bool matches(const Identity& other) const { return std::memcmp(pub_key, other.pub_key, sizeof(pub_key)) == 0; } +}; +struct LocalIdentity : Identity { + void calcSharedSecret(uint8_t* output, const uint8_t* key) const { + std::memcpy(output, key, PUB_KEY_SIZE); + } +}; +} diff --git a/test/fixtures/client_acl_spiffs/mocks/helpers/IdentityStore.h b/test/fixtures/client_acl_spiffs/mocks/helpers/IdentityStore.h new file mode 100644 index 00000000..76471485 --- /dev/null +++ b/test/fixtures/client_acl_spiffs/mocks/helpers/IdentityStore.h @@ -0,0 +1,3 @@ +#pragma once +#include +#define FILESYSTEM FakeFilesystem diff --git a/test/fixtures/client_acl_spiffs/test_client_acl_spiffs.cpp b/test/fixtures/client_acl_spiffs/test_client_acl_spiffs.cpp new file mode 100644 index 00000000..10654927 --- /dev/null +++ b/test/fixtures/client_acl_spiffs/test_client_acl_spiffs.cpp @@ -0,0 +1,482 @@ +// Include the production implementation so its static validators and the full +// ClientACL load/admission/publication path execute against the SPIFFS model. +#include "../../../src/helpers/ClientACL.cpp" +#include +#include +#include +#include + +#define CHECK(condition) do { if (!(condition)) { \ + std::fprintf(stderr, "FAIL line %d: %s\n", __LINE__, #condition); std::exit(1); \ +} } while (0) + +static const uint8_t KEY[PUB_KEY_SIZE] = {0x12, 0x57, 0xae, 0xe5}; +static const char* PRIMARY = mesh::CLIENT_LOGIN_REPLAY_PRIMARY_PATH; +static const char* TEMP = mesh::CLIENT_LOGIN_REPLAY_TEMP_PATH; +static mesh::LocalIdentity SELF; +static const uint8_t SECOND_KEY[PUB_KEY_SIZE] = {0x12, 0x57, 0xae, 0xe5, 0x02}; +static const uint8_t ARCHIVED_KEY[PUB_KEY_SIZE] = {0x77, 0x03}; + +static std::vector replay_records( + std::initializer_list> records) { + std::vector image; + for (const auto& record : records) { + image.insert(image.end(), record.first, record.first + PUB_KEY_SIZE); + const auto* value = reinterpret_cast(&record.second); + image.insert(image.end(), value, value + sizeof(record.second)); + } + const uint32_t crc = mesh::updateClientLoginReplayCRC( + 0xffffffffu, image.data(), image.size()) ^ 0xffffffffu; + image.insert(image.end(), mesh::CLIENT_LOGIN_REPLAY_MAGIC, + mesh::CLIENT_LOGIN_REPLAY_MAGIC + 4); + const auto* value = reinterpret_cast(&crc); + image.insert(image.end(), value, value + sizeof(crc)); + return image; +} + +static void check_empty_result(const ClientLoginReplayClampResult& result) { + CHECK(result.stored_matched == 0 && result.stored_changed == 0 + && result.live_matched == 0 && result.live_changed == 0); +} + +static std::vector replay(uint32_t ceiling) { + std::vector image(KEY, KEY + PUB_KEY_SIZE); + const auto* bytes = reinterpret_cast(&ceiling); + image.insert(image.end(), bytes, bytes + sizeof(ceiling)); + const uint32_t crc = mesh::updateClientLoginReplayCRC(0xffffffffu, image.data(), image.size()) ^ 0xffffffffu; + image.insert(image.end(), mesh::CLIENT_LOGIN_REPLAY_MAGIC, mesh::CLIENT_LOGIN_REPLAY_MAGIC + 4); + bytes = reinterpret_cast(&crc); + image.insert(image.end(), bytes, bytes + sizeof(crc)); + return image; +} + +static void missing_read_is_not_empty_file() { + FakeFilesystem fs; + auto raw = fs.open(PRIMARY); + CHECK(raw && raw.isDirectory() && raw.size() == 0); + raw.close(); + auto checked = mesh::openFileRead(&fs, PRIMARY); + CHECK(!checked && fs.missing_read_opens == 1); + CHECK(!validateContactsFileIntegrity(&fs, "/s_contacts")); + CHECK(!validateLoginReplayFileIntegrity(&fs, PRIMARY)); + CHECK(!mesh::openFileRead(static_cast(nullptr), PRIMARY)); + CHECK(!mesh::openFileRead(&fs, nullptr)); +} + +static void first_admin_and_retries() { + FakeFilesystem fs; + ClientACL acl; + acl.load(&fs, SELF); + CHECK(acl.authorizeLoginTimestamp(KEY, 100, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(160)); + CHECK(fs.files[PRIMARY].size() == 44 && !fs.exists(TEMP)); + CHECK(fs.missing_read_opens == 0); + const auto writes = fs.bytes_written; + CHECK(acl.authorizeLoginTimestamp(KEY, 101, 100, PERM_ACL_ADMIN)); + CHECK(fs.bytes_written == writes); + CHECK(!acl.authorizeLoginTimestamp(KEY, 101, 101, PERM_ACL_ADMIN)); + CHECK(!acl.authorizeLoginTimestamp(KEY, 0, 101, PERM_ACL_ADMIN)); + CHECK(acl.authorizeLoginTimestamp(KEY, 161, 101, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(221)); +} + +static void reboot_preserves_ceiling() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + ClientACL acl; + acl.load(&fs, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 160, 0, PERM_ACL_ADMIN)); + CHECK(acl.authorizeLoginTimestamp(KEY, 161, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(221)); +} + +static void corrupt_state_is_preserved() { + for (size_t length : {size_t(0), size_t(1), size_t(7), size_t(9), size_t(43)}) { + FakeFilesystem fs; + fs.files[PRIMARY] = std::vector(length, 0x99); + const auto original = fs.files[PRIMARY]; + ClientACL acl; + acl.load(&fs, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 1000, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == original && fs.bytes_written == 0); + } + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + fs.files[PRIMARY].back() ^= 1; + const auto original = fs.files[PRIMARY]; + ClientACL acl; + acl.load(&fs, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 1000, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == original); +} + +static void write_failures_keep_boundary() { + for (int failure = 0; failure < 5; ++failure) { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + ClientACL acl; + acl.load(&fs, SELF); + if (failure == 0) fs.write_budget = 12; + if (failure == 1) fs.fail_open_write = TEMP; + if (failure == 2) fs.fail_rename_from = TEMP; + if (failure == 3) fs.corrupt_on_close = TEMP; + if (failure == 4) fs.directories_on_write.insert(TEMP); + CHECK(!acl.authorizeLoginTimestamp(KEY, 200, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(160)); + CHECK(!fs.exists(mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH)); + } +} + +static void first_write_failure_is_retriable() { + FakeFilesystem fs; + ClientACL acl; + acl.load(&fs, SELF); + fs.write_budget = 5; + CHECK(!acl.authorizeLoginTimestamp(KEY, 100, 0, PERM_ACL_ADMIN)); + CHECK(!fs.exists(PRIMARY) && !fs.exists(TEMP)); + fs.write_budget = std::numeric_limits::max(); + CHECK(acl.authorizeLoginTimestamp(KEY, 101, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(161)); +} + +static void directory_reads_are_rejected() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + fs.directories_on_read.insert(PRIMARY); + ClientACL acl; + acl.load(&fs, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 1000, 0, PERM_ACL_ADMIN)); + CHECK(fs.directory_closes > 0 && fs.files[PRIMARY] == replay(160)); + fs.files["/s_contacts"] = {}; + fs.directories_on_read.insert("/s_contacts"); + CHECK(!validateContactsFileIntegrity(&fs, "/s_contacts")); +} + +static void regular_empty_contacts_remain_valid() { + FakeFilesystem fs; + fs.files["/s_contacts"] = {}; + CHECK(validateContactsFileIntegrity(&fs, "/s_contacts")); + ClientACL acl; + acl.load(&fs, SELF); + CHECK(acl.getNumClients() == 0); + CHECK(acl.authorizeLoginTimestamp(KEY, 100, 0, PERM_ACL_ADMIN)); +} + +static void contacts_save_load_uses_regular_files() { + FakeFilesystem fs; + ClientACL acl; + acl.load(&fs, SELF); + CHECK(acl.applyPermissions(SELF, KEY, PUB_KEY_SIZE, PERM_ACL_ADMIN)); + CHECK(acl.save(&fs)); + CHECK(fs.files["/s_contacts"].size() == 209); + ClientACL reloaded; + reloaded.load(&fs, SELF); + CHECK(reloaded.getNumClients() == 1); + CHECK(reloaded.getClient(KEY, PUB_KEY_SIZE)->isAdmin()); + CHECK(reloaded.authorizeLoginTimestamp(KEY, 100, 0, PERM_ACL_ADMIN)); +} + +static void replay_source_reopen_failure_is_not_missing() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + fs.truncate_path = PRIMARY; + fs.truncate_on_read_open = 2; // validation succeeds, source reopened truncated + CHECK(!writeClientLoginReplayCeiling(&fs, KEY, 260, + mesh::ClientLoginReplayReservationAction::UpdateExisting)); + CHECK(fs.bytes_written == 0 && !fs.exists(TEMP)); +} + +static void unreadable_primary_is_not_first_login() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(160); + fs.unreadable.insert(PRIMARY); + ClientACL acl; + acl.load(&fs, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 1000, 0, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(160) && fs.bytes_written == 0); +} + +static void null_load_fails_closed() { + ClientACL acl; + acl.load(nullptr, SELF); + CHECK(!acl.authorizeLoginTimestamp(KEY, 100, 0, PERM_ACL_ADMIN)); +} + +static void clamp_exact_key_preserves_other_state() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay_records({{KEY, 900}, {SECOND_KEY, 800}}); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + client->last_activity = 777; + client->out_path_len = 1; + client->out_path[0] = 0x77; + client->alt_path_len = 1; + client->alt_path[0] = 0x12; + client->shared_secret[3] = 0x88; + client->extra.room.sync_since = 600; + ClientInfo* other = acl.putClient(mesh::Identity(SECOND_KEY), PERM_ACL_REGION_MGR); + other->last_timestamp = 800; + CHECK(acl.save(&fs)); + const auto saved_contacts = fs.files["/s_contacts"]; + ClientInfo expected = *client; + expected.last_timestamp = 500; + const ClientInfo expected_other = *other; + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + CHECK(result.stored_matched == 1 && result.stored_changed == 1); + CHECK(result.live_matched == 1 && result.live_changed == 1); + CHECK(fs.files[PRIMARY] == replay_records({{KEY, 500}, {SECOND_KEY, 800}})); + CHECK(fs.files["/s_contacts"] == saved_contacts); + CHECK(std::memcmp(client, &expected, sizeof(expected)) == 0); + CHECK(std::memcmp(other, &expected_other, sizeof(expected_other)) == 0); +} + +static void clamp_all_keeps_duplicates_and_tombstones() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay_records( + {{KEY, 100}, {ARCHIVED_KEY, 900}, {KEY, 700}, {SECOND_KEY, 500}}); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + ClientInfo* other = acl.putClient(mesh::Identity(SECOND_KEY), PERM_ACL_READ_ONLY); + other->last_timestamp = 400; + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(nullptr, 500, result)); + CHECK(result.stored_matched == 4 && result.stored_changed == 2); + CHECK(result.live_matched == 2 && result.live_changed == 1); + CHECK(fs.files[PRIMARY] == replay_records( + {{KEY, 100}, {ARCHIVED_KEY, 500}, {KEY, 500}, {SECOND_KEY, 500}})); + CHECK(acl.getNumClients() == 2 && acl.getClient(ARCHIVED_KEY, PUB_KEY_SIZE) == nullptr); + CHECK(client->last_timestamp == 500 && other->last_timestamp == 400); + CHECK(validateLoginReplayFileIntegrity(&fs, PRIMARY)); +} + +static void clamp_missing_key_or_store_never_creates() { + FakeFilesystem fs; + ClientACL acl; + acl.load(&fs, SELF); + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + CHECK(fs.files.empty() && fs.bytes_written == 0 && acl.getNumClients() == 0); + fs.files[PRIMARY] = replay(900); + const auto before = fs.files; + CHECK(acl.clampLoginReplayTimestamps(SECOND_KEY, 500, result)); + check_empty_result(result); + CHECK(fs.files == before && fs.bytes_written == 0 && acl.getNumClients() == 0); + fs.files[PRIMARY] = replay_records({}); + CHECK(acl.clampLoginReplayTimestamps(nullptr, 500, result)); + check_empty_result(result); + CHECK(fs.files[PRIMARY] == replay_records({}) && fs.bytes_written == 0); +} + +static void clamp_noop_never_writes_or_raises() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay_records({{KEY, 100}, {SECOND_KEY, 500}}); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 100; + const auto before = fs.files; + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(nullptr, 500, result)); + CHECK(result.stored_matched == 2 && result.stored_changed == 0); + CHECK(result.live_matched == 1 && result.live_changed == 0); + CHECK(acl.clampLoginReplayTimestamps(nullptr, UINT32_MAX, result)); + CHECK(result.stored_changed == 0 && result.live_changed == 0); + CHECK(fs.files == before && fs.bytes_written == 0 && client->last_timestamp == 100); +} + +static void clamp_live_only_does_not_create_or_raise_store() { + for (bool has_store : {false, true}) { + FakeFilesystem fs; + if (has_store) fs.files[PRIMARY] = replay(100); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 900; + const auto before = fs.files; + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + CHECK(result.stored_matched == (has_store ? 1 : 0) && result.stored_changed == 0); + CHECK(result.live_matched == 1 && result.live_changed == 1); + CHECK(fs.files == before && fs.bytes_written == 0 && client->last_timestamp == 500); + } +} + +static void clamp_stored_only_does_not_raise_live() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 100; + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + CHECK(result.stored_changed == 1 && result.live_changed == 0); + CHECK(fs.files[PRIMARY] == replay(500) && client->last_timestamp == 100); + const auto writes = fs.bytes_written; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + CHECK(result.stored_changed == 0 && result.live_changed == 0); + CHECK(fs.bytes_written == writes); +} + +static void clamp_invalid_clock_or_unavailable_fails_closed() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + ClientLoginReplayClampResult result = {9, 9, 9, 9}; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 900; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 0, result)); + check_empty_result(result); + CHECK(fs.files[PRIMARY] == replay(900) && fs.bytes_written == 0); + CHECK(client->last_timestamp == 900); +} + +static void clamp_write_failures_leave_live_and_store_unchanged() { + for (int failure = 0; failure < 6; ++failure) { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + if (failure == 0) fs.write_budget = 12; + if (failure == 1) fs.fail_open_write = TEMP; + if (failure == 2) fs.fail_rename_from = TEMP; + if (failure == 3) fs.corrupt_on_close = TEMP; + if (failure == 4) fs.directories_on_write.insert(TEMP); + if (failure == 5) fs.fail_rename_from = PRIMARY; + ClientLoginReplayClampResult result = {9, 9, 9, 9}; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + CHECK(fs.files[PRIMARY] == replay(900) && client->last_timestamp == 1000); + } +} + +static void clamp_corrupt_or_unreadable_source_never_repairs() { + for (int failure = 0; failure < 5; ++failure) { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + if (failure == 0) fs.files[PRIMARY].back() ^= 1; + if (failure == 1) fs.files[PRIMARY].resize(1); + if (failure == 2) fs.unreadable.insert(PRIMARY); + if (failure == 3) fs.directories_on_read.insert(PRIMARY); + if (failure == 4) { + fs.files[mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH] = replay(900); + fs.files.erase(PRIMARY); + } + const auto before = fs.files; + ClientLoginReplayClampResult result = {}; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + CHECK(fs.files == before && fs.bytes_written == 0 && client->last_timestamp == 1000); + CHECK(!acl.authorizeLoginTimestamp(SECOND_KEY, 2000, 0, PERM_ACL_ADMIN)); + } +} + +static void clamp_reopen_and_rollback_failures_block_new_admission() { + for (bool rollback_failure : {false, true}) { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + if (rollback_failure) { + fs.fail_rename_from_paths.insert(TEMP); + fs.fail_rename_from_paths.insert(mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH); + } else { + fs.truncate_path = PRIMARY; + fs.truncate_on_read_open = fs.read_open_count[PRIMARY] + 2; + } + ClientLoginReplayClampResult result = {}; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + CHECK(client->last_timestamp == 1000); + CHECK(!acl.authorizeLoginTimestamp(SECOND_KEY, 2000, 0, PERM_ACL_ADMIN)); + if (rollback_failure) { + CHECK(!fs.exists(PRIMARY)); + CHECK(fs.files[mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH] == replay(900)); + } + } +} + +static void clamp_reboot_keeps_lowered_boundary() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + CHECK(acl.save(&fs)); + ClientLoginReplayClampResult result = {}; + CHECK(acl.clampLoginReplayTimestamps(KEY, 500, result)); + ClientACL rebooted; + rebooted.load(&fs, SELF); + CHECK(rebooted.getClient(KEY, PUB_KEY_SIZE)->last_timestamp == 500); + CHECK(!rebooted.authorizeLoginTimestamp(KEY, 500, 500, PERM_ACL_ADMIN)); + CHECK(rebooted.authorizeLoginTimestamp(KEY, 501, 500, PERM_ACL_ADMIN)); + CHECK(fs.files[PRIMARY] == replay(561)); +} + +static void clamp_backup_cleanup_failure_does_not_mutate_live() { + FakeFilesystem fs; + fs.files[PRIMARY] = replay(900); + ClientACL acl; + acl.load(&fs, SELF); + ClientInfo* client = acl.putClient(mesh::Identity(KEY), PERM_ACL_ADMIN); + client->last_timestamp = 1000; + fs.files[mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH] = replay(950); + fs.fail_remove = mesh::CLIENT_LOGIN_REPLAY_BACKUP_PATH; + const auto before = fs.files; + ClientLoginReplayClampResult result = {}; + CHECK(!acl.clampLoginReplayTimestamps(KEY, 500, result)); + check_empty_result(result); + CHECK(fs.files == before && fs.bytes_written == 0 && client->last_timestamp == 1000); +} + +int main() { + const struct { const char* name; void (*run)(); } tests[] = { + {"missing read differs from empty file", missing_read_is_not_empty_file}, + {"first admin and monotonic retries", first_admin_and_retries}, + {"reboot preserves ceiling", reboot_preserves_ceiling}, + {"corrupt state retained", corrupt_state_is_preserved}, + {"write failures retain boundary", write_failures_keep_boundary}, + {"failed first write can retry", first_write_failure_is_retriable}, + {"truthy directories rejected", directory_reads_are_rejected}, + {"regular empty legacy ACL accepted", regular_empty_contacts_remain_valid}, + {"contacts save and reload", contacts_save_load_uses_regular_files}, + {"reopen truncation fails closed", replay_source_reopen_failure_is_not_missing}, + {"unreadable history preserved", unreadable_primary_is_not_first_login}, + {"uninitialized storage fails closed", null_load_fails_closed}, + {"clamp exact key preserves all other state", clamp_exact_key_preserves_other_state}, + {"clamp all preserves duplicate and historical records", clamp_all_keeps_duplicates_and_tombstones}, + {"clamp missing selection creates nothing", clamp_missing_key_or_store_never_creates}, + {"clamp noop never writes or raises", clamp_noop_never_writes_or_raises}, + {"clamp live only leaves storage alone", clamp_live_only_does_not_create_or_raise_store}, + {"clamp stored only never raises live", clamp_stored_only_does_not_raise_live}, + {"clamp zero clock and unavailable fail closed", clamp_invalid_clock_or_unavailable_fails_closed}, + {"clamp write errors preserve live and disk", clamp_write_failures_leave_live_and_store_unchanged}, + {"clamp corrupt source is not a repair", clamp_corrupt_or_unreadable_source_never_repairs}, + {"clamp reopen and rollback failures fail closed", clamp_reopen_and_rollback_failures_block_new_admission}, + {"clamp survives reboot", clamp_reboot_keeps_lowered_boundary}, + {"clamp backup cleanup failure preserves state", clamp_backup_cleanup_failure_does_not_mutate_live}, + }; + for (const auto& test : tests) { + test.run(); + std::printf("PASS: %s\n", test.name); + } + std::puts("24 ClientACL SPIFFS checks passed"); +} diff --git a/test/fixtures/esp32_tinyusb_nonblocking/mocks/Arduino.h b/test/fixtures/esp32_tinyusb_nonblocking/mocks/Arduino.h new file mode 100644 index 00000000..5a8fb4e4 --- /dev/null +++ b/test/fixtures/esp32_tinyusb_nonblocking/mocks/Arduino.h @@ -0,0 +1,50 @@ +#pragma once + +#include "../../../mocks/Arduino.h" +#include + +using esp_event_base_t = const char*; +using esp_event_handler_t = void (*)(void*, esp_event_base_t, int32_t, void*); +constexpr int ARDUINO_USB_CDC_ANY_EVENT = -1; +constexpr int ARDUINO_USB_CDC_DISCONNECTED_EVENT = 1; +constexpr int ARDUINO_USB_CDC_LINE_STATE_EVENT = 2; +struct arduino_usb_cdc_event_data_t { + struct { bool dtr; bool rts; } line_state{}; +}; + +inline bool mock_isr = false; +inline bool xPortInIsrContext() { return mock_isr; } + +class MockSerial : public Stream { + public: + int available() override { return rx_count; } + int read() override { return rx_count > 0 ? (--rx_count, 'v') : -1; } + int peek() override { return rx_count > 0 ? 'v' : -1; } + int availableForWrite() override { + ++blocking_calls; + assert(false && "mode0 must not acquire the USBCDC TX mutex"); + return 0; + } + size_t write(const uint8_t*, size_t) override { + ++blocking_calls; + assert(false && "mode0 must never call USBCDC::write"); + return 0; + } + void flush() override { + ++blocking_calls; + assert(false && "mode0 must never call USBCDC::flush"); + } + void setDebugOutput(bool enabled) { debug_enabled = enabled; } + void onEvent(int event, esp_event_handler_t handler) { + assert(event == ARDUINO_USB_CDC_ANY_EVENT); + callback = handler; + ++registrations; + } + int blocking_calls = 0; + int rx_count = 0; + int registrations = 0; + bool debug_enabled = true; + esp_event_handler_t callback = nullptr; +}; + +extern MockSerial Serial; diff --git a/test/fixtures/esp32_tinyusb_nonblocking/mocks/esp32-hal-tinyusb.h b/test/fixtures/esp32_tinyusb_nonblocking/mocks/esp32-hal-tinyusb.h new file mode 100644 index 00000000..5ef060ee --- /dev/null +++ b/test/fixtures/esp32_tinyusb_nonblocking/mocks/esp32-hal-tinyusb.h @@ -0,0 +1,9 @@ +#pragma once + +#include + +bool tud_cdc_n_connected(uint8_t instance); +uint32_t tud_cdc_n_write_available(uint8_t instance); +uint32_t tud_cdc_n_write(uint8_t instance, const void* data, uint32_t size); +uint32_t tud_cdc_n_write_flush(uint8_t instance); +bool tud_cdc_n_write_clear(uint8_t instance); diff --git a/test/fixtures/esp32_tinyusb_nonblocking/test_esp32_tinyusb_nonblocking.cpp b/test/fixtures/esp32_tinyusb_nonblocking/test_esp32_tinyusb_nonblocking.cpp new file mode 100644 index 00000000..16db4d47 --- /dev/null +++ b/test/fixtures/esp32_tinyusb_nonblocking/test_esp32_tinyusb_nonblocking.cpp @@ -0,0 +1,232 @@ +#include +#include +#include +#include +#include "helpers/UsbLogging.h" +#include "MeshCore.h" + +MockSerial Serial; +static bool connected = true; +static bool auto_drain = false; +static std::string fifo; +static std::string host; +static unsigned write_calls = 0; +static unsigned flush_calls = 0; +static unsigned clear_calls = 0; +static void (*during_write)() = nullptr; + +bool tud_cdc_n_connected(uint8_t instance) { + assert(!mock_isr); + assert(instance == 0); + return connected; +} +uint32_t tud_cdc_n_write_available(uint8_t instance) { + assert(!mock_isr); + assert(instance == 0); + return 64 - fifo.size(); +} +uint32_t tud_cdc_n_write(uint8_t instance, const void* data, uint32_t size) { + assert(!mock_isr); + assert(instance == 0); + assert(size <= 64 - fifo.size()); + ++write_calls; + if (during_write) during_write(); + fifo.append(static_cast(data), size); + return size; +} +uint32_t tud_cdc_n_write_flush(uint8_t instance) { + assert(!mock_isr); + assert(instance == 0); + ++flush_calls; + if (!auto_drain) return 0; + const auto size = fifo.size(); + host += fifo; + fifo.clear(); + return size; +} +bool tud_cdc_n_write_clear(uint8_t instance) { + assert(!mock_isr); + assert(instance == 0); + ++clear_calls; + fifo.clear(); + return true; +} + +static size_t put(Stream& port, const std::string& value) { + return port.write(reinterpret_cast(value.data()), value.size()); +} + +#if MESH_ESP32_TINYUSB_NONBLOCKING +static void drain_all() { + for (unsigned turn = 0; turn != 200; ++turn) { + host += fifo; + fifo.clear(); + mesh::serviceUsbTerminalPort(); + if (fifo.empty() && !mesh::hasPendingUsbTerminalOutput()) return; + } + assert(false && "finite queue should drain in bounded service calls"); +} + +static void fresh_session() { + connected = false; + arduino_usb_cdc_event_data_t event; + event.line_state.dtr = false; + Serial.callback(nullptr, nullptr, ARDUINO_USB_CDC_LINE_STATE_EVENT, &event); + connected = true; // fast close/reopen, without an intervening service poll + mesh::serviceUsbLoggingPort(); + assert(mesh::takeUsbTerminalSessionReset()); + assert(!mesh::takeUsbTerminalSessionReset()); + assert(mesh::tryCompleteUsbTerminalSessionReset()); + fifo.clear(); + host.clear(); + auto_drain = false; +} + +static void check_native_short_writes_and_mota() { + const unsigned before = write_calls; + assert(put(mesh::usbCompanionPort(), std::string(200, 'B')) == 64); + assert(write_calls == before + 1); + assert(put(mesh::usbCompanionPort(), "more") == 0); + const unsigned flushed = flush_calls; + mesh::usbCompanionPort().flush(); + assert(flush_calls == flushed); + fifo.resize(59); // only 5 bytes free: an 11-byte mOTA record must not split + assert(put(mesh::usbMotaPort(), std::string(11, 'M')) == 0); + assert(write_calls == before + 1); + fifo.resize(50); + assert(put(mesh::usbMotaPort(), std::string(11, 'M')) == 11); + assert(write_calls == before + 2); + fresh_session(); +} + +static void check_ordered_text_and_functional_reserve() { + const std::string log = "RAW: " + std::string(545, 'L') + "\r\n"; + const std::string reply = " -> " + std::string(2177, 'R') + "\r\n"; + assert(put(mesh::usbLoggingPort(), log) == log.size()); + assert(fifo.size() == 64); + assert(mesh::hasPendingUsbTerminalOutput()); + assert(mesh::canAcceptUsbConsoleCommand()); // logs cannot starve CLI input + assert(put(mesh::usbConsolePort(), reply) == reply.size()); + assert(!mesh::canAcceptUsbConsoleCommand()); // previous reply needs draining + assert(mesh::usbLoggingPort().availableForWrite() == 0); + assert(put(mesh::usbLoggingPort(), std::string(900, 'X')) == 0); + drain_all(); + assert(host == log + reply); // no 64-byte log/reply interleaving + assert(mesh::canAcceptUsbConsoleCommand()); + assert(mesh::usbTerminalDroppedBytes() == 0); + fresh_session(); +} + +static void check_stalled_host_and_visible_overflow() { + const std::string fill(4096, 'F'); + assert(put(mesh::usbConsolePort(), fill) == fill.size()); + assert(put(mesh::usbConsolePort(), std::string(64, 'T')) == 64); + assert(!mesh::canAcceptUsbConsoleCommand()); + const unsigned before = write_calls; + for (unsigned i = 0; i != 1000; ++i) mesh::serviceUsbLoggingPort(); + assert(write_calls == before); // no retry loop enters a full USB FIFO + assert(put(mesh::usbConsolePort(), "!") == 0); + assert(mesh::usbTerminalDroppedBytes() == 1); + drain_all(); + assert(host.find(fill + std::string(64, 'T')) == 0); + assert(host.find("[USB terminal output dropped 1 bytes]") != std::string::npos); + fresh_session(); +} + +static void check_disconnect_cleanup_keeps_new_host_input() { + assert(put(mesh::usbConsolePort(), std::string(1000, 'O')) == 1000); + const auto before = clear_calls; + Serial.rx_count = 4; + fresh_session(); + assert(clear_calls > before); + assert(!mesh::hasPendingUsbTerminalOutput()); + assert(mesh::usbConsolePort().available() == 4); + assert(mesh::usbConsolePort().peek() == 'v'); + assert(mesh::usbConsolePort().read() == 'v'); + assert(put(mesh::usbConsolePort(), "new\r\n") == 5); + drain_all(); + assert(host == "new\r\n"); + fresh_session(); +} + +static void check_debug_formatter_and_reentrancy() { + auto_drain = true; + const std::string long_text(1000, 'D'); + assert(mesh::nrf52DebugPrintf("%s\n", long_text.c_str()) == 255); + drain_all(); + assert(host.size() == 255); + assert(host.substr(host.size() - 4) == "...\n"); + host.clear(); + during_write = [] { + assert(mesh::nrf52DebugPrintf("must not recurse\n") == 0); + }; + assert(mesh::nrf52DebugPrintf("outer\n") == 6); + during_write = nullptr; + drain_all(); + assert(host == "outer\n"); + fresh_session(); +} + +static void check_cached_logging_gate_and_isr() { + Stream& cached = mesh::usbLoggingPort(); + mesh::setUsbLoggingEnabled(false); + assert(put(cached, "off") == 0); + mesh::setUsbLoggingEnabled(true); + mock_isr = true; + const auto before = write_calls; + assert(put(mesh::usbCompanionPort(), "isr") == 0); + assert(put(mesh::usbConsolePort(), "isr") == 0); + mesh::serviceUsbTerminalPort(); + assert(write_calls == before); + mock_isr = false; +} + +static void check_protocol_switch_cancels_text_and_overflow_marker() { + fresh_session(); + assert(put(mesh::usbConsolePort(), std::string(5000, 'X')) == 0); + assert(put(mesh::usbLoggingPort(), std::string(550, 'L')) == 550); + mesh::setUsbLoggingEnabled(false); + assert(!mesh::hasPendingUsbTerminalOutput()); + // A protocol owner may discard application text without resetting USB. The + // already accepted FIFO prefix stays ordered before the new binary frame. + mesh::discardUsbTerminalOutput(); + host += fifo; + fifo.clear(); + assert(put(mesh::usbCompanionPort(), "binary") == 6); + drain_all(); + assert(host == std::string(64, 'L') + "binary"); + // In particular, service must not inject a delayed ASCII overflow notice + // into Binary or mOTA after the owner has discarded the terminal epoch. + host.clear(); + assert(put(mesh::usbMotaPort(), std::string(11, 'M')) == 11); + drain_all(); + assert(host == std::string(11, 'M')); + mesh::setUsbLoggingEnabled(true); + fresh_session(); +} +#endif + +int main() { +#if MESH_ESP32_TINYUSB_NONBLOCKING + mesh::beginUsbLoggingPort(); + mesh::beginUsbLoggingPort(); + mesh::setUsbLoggingEnabled(true); + assert(Serial.registrations == 1); + assert(!Serial.debug_enabled); + mesh::serviceUsbLoggingPort(); + assert(&mesh::usbConsolePort() == &mesh::usbTerminalPort()); + check_native_short_writes_and_mota(); + check_ordered_text_and_functional_reserve(); + check_stalled_host_and_visible_overflow(); + check_disconnect_cleanup_keeps_new_host_input(); + check_debug_formatter_and_reentrancy(); + check_cached_logging_gate_and_isr(); + check_protocol_switch_cancels_text_and_overflow_marker(); + assert(Serial.blocking_calls == 0); +#else + assert(&mesh::usbConsolePort() == &Serial); + assert(mesh::canAcceptUsbConsoleCommand()); + assert(mesh::usbTerminalDroppedBytes() == 0); +#endif + std::cout << "ESP32 TinyUSB transport checks passed\n"; +} diff --git a/test/fixtures/regular_file_reads/InternalFileSystem.h b/test/fixtures/regular_file_reads/InternalFileSystem.h new file mode 100644 index 00000000..ec5b0105 --- /dev/null +++ b/test/fixtures/regular_file_reads/InternalFileSystem.h @@ -0,0 +1,6 @@ +#pragma once + +// The production helper only needs a valid owner for a closed LittleFS file. +// The harness defines this object without any filesystem or hardware access. +struct FakeFS; +extern FakeFS InternalFS; diff --git a/test/fixtures/replay_reset_command/main.cpp b/test/fixtures/replay_reset_command/main.cpp new file mode 100644 index 00000000..4d41a9d7 --- /dev/null +++ b/test/fixtures/replay_reset_command/main.cpp @@ -0,0 +1,251 @@ +#include + +#include +#include +#include +#include +#include + +using mesh::ReplayResetKind; +using mesh::ReplayResetNonce; +using mesh::ReplayResetRequest; +using Result = mesh::ReplayResetNonce::IssueResult; + +static void check(bool condition, const char* message) { + if (!condition) throw std::runtime_error(message); +} + +static ReplayResetKind parse(const std::string& command) { + ReplayResetRequest request; + const auto result = mesh::parseReplayResetCommand(command.c_str(), request); + check(result == request.kind, "parser result disagrees with request"); + return result; +} + +static void parserTests() { + const std::string key = + "0123456789abcdef0123456789ABCDEF0123456789abcdef0123456789ABCDEF"; + const std::string token = "0123456789abcdefFEDCBA9876543210"; + const std::string base = "replay reset " + key; + check(parse(base) == ReplayResetKind::ExactKey, "exact key"); + check(parse(base + " " + token) == ReplayResetKind::ExactKeyConfirm, "exact confirmation"); + check(parse(" \tQ7| RePlAy\tRESET\t" + key + "\r\n") == ReplayResetKind::ExactKey, + "normalization and companion prefix"); + check(parse("ab|replay reset " + key + " " + token + " \t\r\n") + == ReplayResetKind::ExactKeyConfirm, "prefixed confirmation"); + check(parse("replay reset all CONFIRM") == ReplayResetKind::AllConfirm, "all confirmation"); + check(parse(" REPLAY reset ALL confirm\r\n") == ReplayResetKind::AllConfirm, "all case fold"); + + ReplayResetRequest request; + memset(&request, 0xff, sizeof(request)); + check(mesh::parseReplayResetCommand(nullptr, request) == ReplayResetKind::NotReplay, + "null command"); + for (size_t i = 0; i < sizeof(request.key); ++i) check(request.key[i] == 0, "request key initialized"); + for (size_t i = 0; i < sizeof(request.token); ++i) check(request.token[i] == 0, "request token initialized"); + mesh::parseReplayResetCommand((base + " " + token).c_str(), request); + check(request.key[0] == 0x01 && request.key[7] == 0xef && request.key[31] == 0xef, + "decoded exact full key"); + check(request.token[0] == 0x01 && request.token[8] == 0xfe && request.token[15] == 0x10, + "decoded full nonce"); + + const std::string invalid[] = { + "replay", "replay ", "replay reset", "replay reset all", "replay reset all 123", + "replay reset all CONFIRM extra", "replay RESET " + key.substr(2), + "replay reset " + key + "00", "replay reset g" + key.substr(1), + "replay reset " + key.substr(0, 20) + " " + key.substr(20), + base + " " + token.substr(2), base + " " + token + "00", + base + " " + token + " extra", base + " g" + token.substr(1), + base + " CONFIRM", "replay reset0 " + key, "replay bogus", "replay.reset " + key, + " ab|replay reset all", "replay reset all\nCONFIRM extra", + }; + for (const auto& command : invalid) { + check(parse(command) == ReplayResetKind::Invalid, command.c_str()); + } + const std::string unrelated[] = {"", " ", "a", "ab", "ab|", "replayx", "replay-reset", "get replay", "reboot"}; + for (const auto& command : unrelated) { + check(parse(command) == ReplayResetKind::NotReplay, command.c_str()); + } +} + +struct Identities { + uint8_t issuer[32] = {1}; + uint8_t target[32] = {2}; + uint8_t other[32] = {3}; + uint8_t random[16] = {4}; + uint8_t later_random[16] = {5}; + uint8_t zero[16] = {}; +}; + +static void lifecycleTests() { + Identities ids; + ReplayResetNonce nonce; + check(!nonce.matches(ids.issuer, ids.target, ids.random, 100, 2000000000), "unissued nonce"); + check(nonce.issue(ids.issuer, ids.target, ids.random, 100, 2000000000) == Result::Issued, + "issue nonce"); + check(memcmp(nonce.token(), ids.random, 16) == 0, "exposes original token"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 1100, 2000000001), "matches issuer and target"); + check(!nonce.matches(ids.other, ids.target, ids.random, 1100, 2000000001), "different issuer"); + check(!nonce.matches(ids.issuer, ids.other, ids.random, 1100, 2000000001), "different target"); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 1100, 2000000001), "wrong token"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 20100, 2000000020) == Result::Reused, + "prepare retry reuses existing token"); + check(memcmp(nonce.token(), ids.random, 16) == 0, "retry cannot replace token"); + check(nonce.issue(ids.other, ids.target, ids.later_random, 20100, 2000000020) == Result::Busy, + "other issuer cannot replace challenge"); + check(nonce.issue(ids.issuer, ids.other, ids.later_random, 20100, 2000000020) == Result::Busy, + "other target cannot replace challenge"); + check(!nonce.consume(ids.issuer, ids.target, ids.later_random, 20100, 2000000020), + "bad token does not consume challenge"); + check(nonce.consume(ids.issuer, ids.target, ids.random, 20100, 2000000020), "valid token consumed"); + check(!nonce.consume(ids.issuer, ids.target, ids.random, 20100, 2000000020), "cannot consume twice"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 21100, 2000000021), "captured confirmation rejected"); + for (size_t i = 0; i < 16; ++i) check(nonce.token()[i] == 0, "consumption clears token"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 21100, 2000000021) == Result::Issued, + "reissue after consumption"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 21100, 2000000021), "old token rejected after reissue"); + nonce.clear(); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 21100, 2000000021), "USB invalidation"); + ReplayResetNonce rebooted; + check(!rebooted.matches(ids.issuer, ids.target, ids.random, 100, 2000000000), "reboot invalidates captured token"); + check(rebooted.issue(ids.issuer, ids.issuer, ids.random, 100, 2000000000) == Result::Issued, + "self reset allowed by nonce layer"); + check(rebooted.consume(ids.issuer, ids.issuer, ids.random, 100, 2000000000), "self reset consumed once"); +} + +static void timeTests() { + Identities ids; + ReplayResetNonce nonce; + check(ReplayResetNonce::RESEND_WINDOW_MILLIS == 120000, "resend window is exactly 120 seconds"); + check(ReplayResetNonce::LIFETIME_MILLIS == 300000, "confirmation expires at exactly 300 seconds"); + check(nonce.issue(ids.issuer, ids.target, ids.random, 100, 2000000000) == Result::Issued, "time fixture"); + check(nonce.remainingSeconds(100, 2000000000) == 300, "initial TTL is 300 seconds"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 120099, 2000000119) == Result::Reused, + "119999ms remains in the resend window"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 120099, 2000000119), "119999ms original token valid"); + check(nonce.remainingSeconds(120099, 2000000119) == 180, "last resend advertises original 180-second TTL"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 120100, 2000000120) + == Result::AwaitingConfirmation, "120000ms starts confirmation-only window"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 120100, 2000000120), "120000ms token still valid"); + check(nonce.remainingSeconds(120100, 2000000120) == 180, "confirmation-only boundary retains original TTL"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 300099, 2000000299) + == Result::AwaitingConfirmation, "299999ms cannot reissue or resend original token"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 300099, 2000000299), "299999ms original token valid"); + check(nonce.remainingSeconds(300099, 2000000299) == 0, "TTL rounds down during valid final subsecond"); + check(memcmp(nonce.token(), ids.random, 16) == 0, "confirmation-only retries cannot replace token"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 300100, 2000000300), "300000ms expiration boundary"); + check(nonce.remainingSeconds(300100, 2000000300) == 0, "expired TTL is zero"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 300100, 2000000300) == Result::Issued, + "expired challenge replaceable"); + check(memcmp(nonce.token(), ids.later_random, 16) == 0, "new lifetime uses fresh token"); + check(nonce.matches(ids.issuer, ids.target, ids.later_random, 301100, 2000000306), "positive clock tolerance"); + check(nonce.matches(ids.issuer, ids.target, ids.later_random, 301100, 2000000296), "negative clock tolerance"); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 301100, 2000000307), "forward clock jump"); + check(nonce.remainingSeconds(301100, 2000000307) == 0, "clock-invalid TTL is zero"); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 301100, 2000000295), "backward clock jump"); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 301100, 0), "zero clock invalid"); + check(!nonce.matches(ids.issuer, ids.target, ids.later_random, 300099, 2000000300), "monotonic backwards fails closed"); + + nonce.clear(); + nonce.issue(ids.issuer, ids.target, ids.random, 0, 2000000000); + for (uint32_t age : {119999U, 120000U, 299999U}) { + check(nonce.issue(ids.other, ids.target, ids.later_random, age, 2000000000 + age / 1000) + == Result::Busy, "another issuer stays blocked through confirmation-only window"); + check(nonce.issue(ids.issuer, ids.other, ids.later_random, age, 2000000000 + age / 1000) + == Result::Busy, "another target stays blocked through confirmation-only window"); + } + check(nonce.issue(ids.other, ids.target, ids.later_random, 300000, 2000000300) + == Result::Issued, "another identity can start only after 300000ms"); + + nonce.clear(); + const uint32_t started = UINT32_MAX - 500; + check(nonce.issue(ids.issuer, ids.target, ids.random, started, 2000000000) == Result::Issued, "rollover issue"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 499, 2000000001), "millis rollover valid"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, started + 119999U, 2000000119) + == Result::Reused, "rollover 119999ms resend"); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, started + 120000U, 2000000120) + == Result::AwaitingConfirmation, "rollover 120000ms confirmation-only"); + check(nonce.matches(ids.issuer, ids.target, ids.random, started + 299999U, 2000000299), + "rollover 299999ms original confirmation valid"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, started + 300000U, 2000000300), + "rollover 300000ms expiration"); + + nonce.clear(); + check(nonce.issue(ids.issuer, ids.target, ids.random, 0, UINT32_MAX) == Result::Issued, "epoch boundary issue"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 1000, UINT32_MAX), "epoch overflow rejected"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 1000, 1), "epoch wrap rejected"); + + nonce.clear(); + nonce.issue(ids.issuer, ids.target, ids.random, 0, 2000000000); + check(nonce.issue(ids.issuer, ids.target, ids.later_random, 119999, 2000000119) == Result::Reused, + "last possible resend still reuses"); + check(nonce.consume(ids.issuer, ids.target, ids.random, 299999, 2000000299), + "original confirmation works exactly 180 seconds after last possible resend"); + check(nonce.remainingSeconds(299999, 2000000299) == 0, "consumed token TTL is zero"); + nonce.issue(ids.issuer, ids.target, ids.random, 0, 2000000000); + for (uint32_t age : {119000U, 119999U, 120000U, 200000U, 299999U}) { + const auto expected = age < 120000 ? Result::Reused : Result::AwaitingConfirmation; + check(nonce.issue(ids.issuer, ids.target, ids.later_random, age, 2000000000 + age / 1000) + == expected, "repeated requests obey original window"); + } + check(!nonce.matches(ids.issuer, ids.target, ids.random, 300000, 2000000300), "retries never extend lifetime"); + + nonce.clear(); + nonce.issue(ids.issuer, ids.target, ids.random, 0, 2000000000); + check(nonce.matches(ids.issuer, ids.target, ids.random, 180000, 2000000185), + "confirmation-only phase allows positive clock tolerance"); + check(nonce.matches(ids.issuer, ids.target, ids.random, 180000, 2000000175), + "confirmation-only phase allows negative clock tolerance"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 180000, 2000000186), + "forward correction invalidates confirmation-only token"); + check(!nonce.matches(ids.issuer, ids.target, ids.random, 180000, 2000000174), + "backward correction invalidates confirmation-only token"); +} + +static void invalidTests() { + Identities ids; + ReplayResetNonce nonce; + check(nonce.issue(nullptr, ids.target, ids.random, 0, 1) == Result::Invalid, "null issuer"); + check(nonce.issue(ids.issuer, nullptr, ids.random, 0, 1) == Result::Invalid, "null target"); + check(nonce.issue(ids.issuer, ids.target, nullptr, 0, 1) == Result::Invalid, "null random"); + check(nonce.issue(ids.issuer, ids.target, ids.zero, 0, 1) == Result::Invalid, "zero random"); + check(nonce.issue(ids.issuer, ids.target, ids.random, 0, 0) == Result::Invalid, "zero epoch"); + check(nonce.issue(ids.issuer, ids.target, ids.random, 0, 2000000000) == Result::Issued, "valid after invalid attempts"); + check(!nonce.matches(nullptr, ids.target, ids.random, 0, 2000000000), "null matching issuer"); + check(!nonce.matches(ids.issuer, nullptr, ids.random, 0, 2000000000), "null matching target"); + check(!nonce.matches(ids.issuer, ids.target, nullptr, 0, 2000000000), "null matching token"); + check(nonce.issue(ids.issuer, ids.target, nullptr, 0, 2000000000) == Result::Reused, + "existing challenge does not require replacement randomness"); + check(nonce.issue(ids.issuer, ids.target, nullptr, 120000, 2000000120) + == Result::AwaitingConfirmation, "confirmation-only phase needs no replacement randomness"); + for (size_t i = 0; i < 32; ++i) { + uint8_t changed[32]; + memcpy(changed, ids.issuer, 32); + changed[i] ^= 1; + check(!nonce.matches(changed, ids.target, ids.random, 0, 2000000000), "every issuer byte bound"); + memcpy(changed, ids.target, 32); + changed[i] ^= 1; + check(!nonce.matches(ids.issuer, changed, ids.random, 0, 2000000000), "every target byte bound"); + } + for (size_t i = 0; i < 16; ++i) { + uint8_t changed[16]; + memcpy(changed, ids.random, 16); + changed[i] ^= 1; + check(!nonce.matches(ids.issuer, ids.target, changed, 0, 2000000000), "every token byte bound"); + } +} + +int main(int argc, char** argv) { + try { + if (argc != 2) throw std::runtime_error("expected test case"); + const std::string test = argv[1]; + if (test == "parser") parserTests(); + else if (test == "lifecycle") lifecycleTests(); + else if (test == "time") timeTests(); + else if (test == "invalid") invalidTests(); + else throw std::runtime_error("unknown test case"); + } catch (const std::exception& error) { + std::cerr << error.what() << std::endl; + return EXIT_FAILURE; + } + return EXIT_SUCCESS; +} diff --git a/test/fixtures/replay_reset_integration/test_replay_reset_integration.cpp b/test/fixtures/replay_reset_integration/test_replay_reset_integration.cpp new file mode 100644 index 00000000..50cb3f0d --- /dev/null +++ b/test/fixtures/replay_reset_integration/test_replay_reset_integration.cpp @@ -0,0 +1,433 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +static constexpr size_t PUB_KEY_SIZE = 32; +static constexpr uint32_t NOW = 1760000000; +static constexpr int CLOCK_SYNC_MESH_SUPPRESS_NONE = 0; +static constexpr int CLOCK_SYNC_RESULT_WITHIN_DRIFT = 5; +static constexpr int CLOCK_SYNC_RESULT_CORRECTED_BACKWARD = 7; +static uint32_t fake_millis = 1000; +uint32_t millis() { return fake_millis; } +static uint32_t clockSyncMinimumValidEpoch() { return 1700000000; } +static uint32_t clockSyncMaximumValidEpoch() { return 1900000000; } + +#define CHECK(expression) do { if (!(expression)) { \ + std::fprintf(stderr, "FAIL line %d: %s\n", __LINE__, #expression); std::exit(1); \ +} } while (0) + +namespace mesh { +struct Utils { + static void toHex(char* out, const uint8_t* data, size_t length) { + const char digits[] = "0123456789ABCDEF"; + for (size_t i = 0; i < length; ++i) { + out[i * 2] = digits[data[i] >> 4]; + out[i * 2 + 1] = digits[data[i] & 15]; + } + out[length * 2] = 0; + } +}; +} + +struct ClientInfo { + struct { uint8_t pub_key[PUB_KEY_SIZE]; } id{}; + bool admin = false; + bool observed_path_pending = true; + uint32_t last_timestamp = NOW + 10000; + bool isAdmin() const { return admin; } +}; + +struct ClientLoginReplayClampResult { + uint16_t stored_matched, stored_changed, live_matched, live_changed; +}; + +struct FakeACL { + std::vector clients; + std::vector stored; + bool fail_persist = false; + int calls = 0; + bool selected_all = false; + std::array selected_key{}; + uint32_t selected_now = 0; + + int getNumClients() const { return static_cast(clients.size()); } + ClientInfo* getClientByIdx(int index) { return &clients.at(index); } + bool clampLoginReplayTimestamps(const uint8_t* key, uint32_t now, + ClientLoginReplayClampResult& result) { + ++calls; + selected_all = key == nullptr; + if (key) std::copy(key, key + 32, selected_key.begin()); + selected_now = now; + result = {}; + if (fail_persist) return false; + for (auto& record : stored) { + if (!key || std::memcmp(key, record.id.pub_key, 32) == 0) { + ++result.stored_matched; + if (record.last_timestamp > now) { record.last_timestamp = now; ++result.stored_changed; } + } + } + for (auto& client : clients) { + if (!key || std::memcmp(key, client.id.pub_key, 32) == 0) { + ++result.live_matched; + if (client.last_timestamp > now) { client.last_timestamp = now; ++result.live_changed; } + } + } + return true; + } +}; + +struct FakeClock { uint32_t now = NOW; uint32_t getCurrentTime() const { return now; } }; +struct FakeRNG { + uint8_t next = 1; + int calls = 0; + void random(uint8_t* output, size_t length) { ++calls; std::memset(output, next++, length); } +}; + +struct CountingReplyCache { + bool hit; + int& calls; + bool lookup(const uint8_t*, uint32_t, uint32_t, const char**) { + ++calls; + return hit; + } +}; + +class MyMesh { +public: + FakeACL acl; + FakeClock clock; + FakeRNG rng; + bool replay_clock_set = true; + int clock_sync_mesh_suppressed_by = CLOCK_SYNC_MESH_SUPPRESS_NONE; + int clock_sync_last_result = 0; + mesh::ReplayResetNonce replay_reset_nonce; + struct { bool pending = false; int client_index = -1; } deferred_cli_command; + int mailbox_clears = 0; + + FakeClock* getRTCClock() { return &clock; } + FakeRNG* getRNG() { return &rng; } + void clearDeferredCliCommand() { deferred_cli_command.pending = false; ++mailbox_clears; } + bool handleReplayResetCommand(ClientInfo*, const char*, char*, bool); +}; + +#include "production_handler.inc" + +static ClientInfo client(uint8_t key_byte, bool admin = false) { + ClientInfo value; + std::memset(value.id.pub_key, key_byte, sizeof(value.id.pub_key)); + value.admin = admin; + return value; +} + +static std::string key(const ClientInfo& value) { + char output[65]; + mesh::Utils::toHex(output, value.id.pub_key, 32); + return output; +} + +static MyMesh populated() { + fake_millis = 1000; + MyMesh value; + value.acl.clients = {client(0x12, true), client(0x77), client(0x99)}; + value.acl.stored = value.acl.clients; + value.acl.stored.push_back(client(0x88)); // Historical identity, no live ACL entry. + return value; +} + +static std::string run(MyMesh& value, ClientInfo* sender, const std::string& command, + bool usb = false, bool expected_handled = true) { + std::array guarded; + guarded.fill('#'); + guarded[4] = 0; + CHECK(value.handleReplayResetCommand(sender, command.c_str(), guarded.data() + 4, usb) + == expected_handled); + for (size_t i = 0; i < 4; ++i) CHECK(guarded[i] == '#'); + for (size_t i = 164; i < guarded.size(); ++i) CHECK(guarded[i] == '#'); + return std::string(guarded.data() + 4); +} + +static std::string request(const ClientInfo& target) { return "replay reset " + key(target); } +static bool contains(const std::string& text, const char* part) { return text.find(part) != std::string::npos; } +static std::string challenge(MyMesh& value, ClientInfo* admin, const ClientInfo& target, + std::string* full_reply = nullptr) { + const std::string response = run(value, admin, request(target)); + if (full_reply) *full_reply = response; + const auto where = response.find("confirm: "); + CHECK(where != std::string::npos); + return response.substr(where + std::strlen("confirm: ")); +} + +static void authorization() { + auto value = populated(); + const auto command = request(value.acl.clients[1]); + CHECK(contains(run(value, nullptr, command), "requires USB or LoRa admin")); + CHECK(contains(run(value, &value.acl.clients[1], command), "requires USB or LoRa admin")); + CHECK(contains(run(value, &value.acl.clients[1], command, true), "requires USB or LoRa admin")); + CHECK(value.acl.calls == 0 && value.rng.calls == 0); + CHECK(run(value, nullptr, "get name", false, false).empty()); +} + +static void usb_exact_and_all() { + auto value = populated(); + CHECK(contains(run(value, nullptr, request(value.acl.clients[1]), true), "OK - clamped")); + CHECK(value.acl.calls == 1 && !value.acl.selected_all && value.acl.selected_now == NOW); + CHECK(value.acl.clients[1].last_timestamp == NOW); + CHECK(value.acl.clients[0].last_timestamp > NOW); + CHECK(!value.acl.clients[1].observed_path_pending && value.acl.clients[0].observed_path_pending); + CHECK(contains(run(value, nullptr, "replay reset all CONFIRM", true), "stored=3 live=2")); + CHECK(value.acl.selected_all && value.acl.stored[3].last_timestamp == NOW); + CHECK(value.acl.clients.size() == 3 && value.acl.stored.size() == 4); +} + +static void all_usb_only() { + auto value = populated(); + CHECK(contains(run(value, &value.acl.clients[0], "replay reset all CONFIRM"), "USB-only")); + CHECK(contains(run(value, nullptr, "replay reset all CONFIRM"), "requires USB")); + CHECK(contains(run(value, nullptr, "replay reset all", true), "64-hex")); + CHECK(value.acl.calls == 0 && value.rng.calls == 0); +} + +static void strict_full_key() { + auto value = populated(); + const std::string full = key(value.acl.clients[1]); + for (size_t size : {size_t(2), size_t(4), size_t(6), size_t(63)}) { + CHECK(contains(run(value, nullptr, "replay reset " + full.substr(0, size), true), "64-hex")); + } + CHECK(contains(run(value, nullptr, "replay reset " + full + "00", true), "64-hex")); + CHECK(contains(run(value, nullptr, "replay reset " + full + " trailing", true), "64-hex")); + CHECK(value.acl.calls == 0); +} + +static void clock_must_be_observed() { + auto value = populated(); + value.replay_clock_set = false; + CHECK(contains(run(value, nullptr, request(value.acl.clients[1]), true), "set/sync")); + CHECK(value.acl.calls == 0); + value.clock_sync_last_result = CLOCK_SYNC_RESULT_WITHIN_DRIFT; + CHECK(contains(run(value, nullptr, request(value.acl.clients[1]), true), "OK")); + value.clock_sync_last_result = 0; + value.clock_sync_mesh_suppressed_by = 1; + CHECK(contains(run(value, nullptr, request(value.acl.clients[2]), true), "OK")); +} + +static void clock_must_be_sane() { + auto value = populated(); + for (uint32_t now : {0U, clockSyncMinimumValidEpoch() - 1, clockSyncMaximumValidEpoch() + 1}) { + value.clock.now = now; + CHECK(contains(run(value, nullptr, request(value.acl.clients[1]), true), "set/sync")); + } + CHECK(value.acl.calls == 0 && value.rng.calls == 0); +} + +static void remote_challenge_then_commit() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto confirm = challenge(value, admin, value.acl.clients[1]); + CHECK(value.acl.calls == 0 && value.acl.clients[1].last_timestamp > NOW); + CHECK(challenge(value, admin, value.acl.clients[1]) == confirm); + CHECK(contains(run(value, admin, confirm), "OK - clamped")); + CHECK(value.acl.calls == 1 && value.acl.clients[1].last_timestamp == NOW); +} + +static void nonce_binds_sender_and_target() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + value.acl.clients[2].admin = true; + const auto confirm = challenge(value, admin, value.acl.clients[1]); + CHECK(contains(run(value, &value.acl.clients[2], confirm), "expired/used")); + std::string swapped = confirm; + swapped.replace(std::strlen("replay reset "), 64, key(value.acl.clients[2])); + CHECK(contains(run(value, admin, swapped), "expired/used")); + CHECK(value.acl.calls == 0); + CHECK(contains(run(value, admin, confirm), "OK")); +} + +static void failure_is_not_live_and_token_is_consumed() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto confirm = challenge(value, admin, value.acl.clients[1]); + value.acl.fail_persist = true; + value.deferred_cli_command = {true, 1}; + CHECK(contains(run(value, admin, confirm), "no live timestamps changed")); + CHECK(value.acl.calls == 1 && value.acl.clients[1].last_timestamp > NOW); + CHECK(value.acl.clients[1].observed_path_pending && value.deferred_cli_command.pending); + value.acl.fail_persist = false; + CHECK(contains(run(value, admin, confirm), "expired/used")); + CHECK(value.acl.calls == 1); +} + +static void self_reset_replay_never_reexecutes_or_raises_floor() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto confirm = challenge(value, admin, *admin); + CHECK(contains(run(value, admin, confirm), "OK")); + CHECK(admin->last_timestamp == NOW && value.acl.calls == 1); + apply_actual_receive_guard(admin, confirm.c_str(), NOW + 100000); + CHECK(admin->last_timestamp == NOW); + CHECK(contains(run(value, admin, confirm), "expired/used")); + CHECK(value.acl.calls == 1 && admin->last_timestamp == NOW); +} + +static void every_replay_family_preserves_receive_floor() { + auto value = populated(); + auto* sender = &value.acl.clients[0]; + sender->last_timestamp = NOW; + std::vector commands = {"replay", "replay anything", "replay.reset", "REPLAY RESET 12", + "replay reset all", "replay reset all CONFIRM", request(*sender), + "aa| REPLAY RESET " + key(*sender) + " " + std::string(32, 'a'), + "\tRePlAy\treset\tbad", "replay reset " + key(*sender) + " bad"}; + for (const auto& command : commands) { + apply_actual_receive_guard(sender, command.c_str(), NOW + 10000); + CHECK(sender->last_timestamp == NOW); + } + apply_actual_receive_guard(sender, "get name", NOW + 10); + CHECK(sender->last_timestamp == NOW + 10); + apply_actual_receive_guard(sender, "replayable", NOW + 20); + CHECK(sender->last_timestamp == NOW + 20); +} + +static void pending_usb_affected_and_unrelated_mailboxes() { + auto value = populated(); + value.deferred_cli_command = {true, 2}; + run(value, nullptr, request(value.acl.clients[1]), true); + CHECK(value.deferred_cli_command.pending && value.mailbox_clears == 0); + value.deferred_cli_command = {true, 1}; + run(value, nullptr, request(value.acl.clients[1]), true); + CHECK(!value.deferred_cli_command.pending && value.mailbox_clears == 1); + value.deferred_cli_command = {true, -1}; + run(value, nullptr, "replay reset all CONFIRM", true); + CHECK(!value.deferred_cli_command.pending && value.mailbox_clears == 2); +} + +static void remote_keeps_its_executing_mailbox() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto confirm = challenge(value, admin, *admin); + value.deferred_cli_command = {true, 0}; + CHECK(contains(run(value, admin, confirm), "OK")); + CHECK(value.deferred_cli_command.pending && value.mailbox_clears == 0); +} + +static void expiry_clock_change_and_usb_token_rejected() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + auto confirm = challenge(value, admin, value.acl.clients[1]); + CHECK(contains(run(value, nullptr, confirm, true), "no token")); + fake_millis += mesh::ReplayResetNonce::LIFETIME_MILLIS; + value.clock.now += 300; + CHECK(contains(run(value, admin, confirm), "expired/used")); + confirm = challenge(value, admin, value.acl.clients[1]); + value.clock.now += 60; + CHECK(contains(run(value, admin, confirm), "expired/used")); + CHECK(value.acl.calls == 0); +} + +static void missing_identity_does_not_create_records() { + auto value = populated(); + const auto unknown = client(0xee); + CHECK(contains(run(value, nullptr, request(unknown), true), "nothing created")); + CHECK(value.acl.clients.size() == 3 && value.acl.stored.size() == 4); + CHECK(value.acl.calls == 1); + for (const auto& entry : value.acl.clients) CHECK(entry.last_timestamp > NOW); +} + +static void confirmation_only_window_preserves_last_resend() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto prepare = request(value.acl.clients[1]); + std::string first_reply; + const auto original = challenge(value, admin, value.acl.clients[1], &first_reply); + CHECK(contains(first_reply, "ttl=300s; confirm:")); + const auto token = original.substr(original.rfind(' ') + 1); + fake_millis = 1000 + 119999; + value.clock.now = NOW + 119; + std::string last_resend_reply; + CHECK(challenge(value, admin, value.acl.clients[1], &last_resend_reply) == original); + CHECK(contains(last_resend_reply, "ttl=180s; confirm:")); + + for (uint32_t age : {120000U, 180000U, 299999U}) { + fake_millis = 1000 + age; + value.clock.now = NOW + age / 1000; + const auto response = run(value, admin, prepare); + CHECK(contains(response, "confirmation-only")); + CHECK(!contains(response, "confirm: ")); + CHECK(response.find(token) == std::string::npos); + CHECK(response.find(key(value.acl.clients[1])) == std::string::npos); + CHECK(value.acl.calls == 0); + CHECK(contains(run(value, admin, request(value.acl.clients[2])), "another replay confirmation")); + CHECK(contains(run(value, &value.acl.clients[1], original), "requires USB or LoRa admin")); + } + // This is exactly 180000ms after the last allowed token resend at119999ms. + CHECK(contains(run(value, admin, original), "OK - clamped")); + CHECK(value.acl.calls == 1 && value.acl.clients[1].last_timestamp == NOW + 299); + CHECK(contains(run(value, admin, original), "expired/used")); + CHECK(value.acl.calls == 1); +} + +static void repeated_requests_cannot_extend_original_deadline() { + auto value = populated(); + auto* admin = &value.acl.clients[0]; + const auto original = challenge(value, admin, value.acl.clients[1]); + for (uint32_t age : {119999U, 120000U, 200000U, 299999U}) { + fake_millis = 1000 + age; + value.clock.now = NOW + age / 1000; + run(value, admin, request(value.acl.clients[1])); + } + fake_millis = 1000 + 300000; + value.clock.now = NOW + 300; + CHECK(contains(run(value, admin, original), "expired/used")); + CHECK(value.acl.calls == 0); + const auto next = challenge(value, admin, value.acl.clients[1]); + CHECK(next != original); + CHECK(contains(run(value, admin, original), "expired/used")); + CHECK(contains(run(value, admin, next), "OK")); + CHECK(value.acl.calls == 1); +} + +static void preparation_never_uses_cached_token_reply() { + auto value = populated(); + const auto prepare = request(value.acl.clients[1]); + const auto confirm = prepare + " " + std::string(32, 'a'); + for (const auto& command : {prepare, "aa| " + prepare, "\tREPLAY RESET " + key(value.acl.clients[1])}) { + int calls = 0; + CHECK(!apply_actual_receive_cache_gate(command.c_str(), true, calls)); + CHECK(calls == 0); + } + for (const auto& command : {confirm, std::string("get name"), std::string("replay reset 12")}) { + int calls = 0; + CHECK(apply_actual_receive_cache_gate(command.c_str(), true, calls)); + CHECK(calls == 1); + CHECK(!apply_actual_receive_cache_gate(command.c_str(), false, calls)); + CHECK(calls == 2); + } +} + +int main() { +#define RUN(test) do { test(); std::printf("PASS: %s\n", #test); } while (0) + RUN(authorization); + RUN(usb_exact_and_all); + RUN(all_usb_only); + RUN(strict_full_key); + RUN(clock_must_be_observed); + RUN(clock_must_be_sane); + RUN(remote_challenge_then_commit); + RUN(nonce_binds_sender_and_target); + RUN(failure_is_not_live_and_token_is_consumed); + RUN(self_reset_replay_never_reexecutes_or_raises_floor); + RUN(every_replay_family_preserves_receive_floor); + RUN(pending_usb_affected_and_unrelated_mailboxes); + RUN(remote_keeps_its_executing_mailbox); + RUN(expiry_clock_change_and_usb_token_rejected); + RUN(missing_identity_does_not_create_records); + RUN(confirmation_only_window_preserves_last_resend); + RUN(repeated_requests_cannot_extend_original_deadline); + RUN(preparation_never_uses_cached_token_reply); + std::puts("18 replay-reset integration checks passed"); +} diff --git a/test/test_client_acl_spiffs.py b/test/test_client_acl_spiffs.py new file mode 100644 index 00000000..b291f6d5 --- /dev/null +++ b/test/test_client_acl_spiffs.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +"""Run real ClientACL persistence against ESP32's missing-file directory quirk.""" +from pathlib import Path +import shutil +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/client_acl_spiffs" + + +class ClientAclSpiffsTest(unittest.TestCase): + def test_real_acl_with_spiffs_file_semantics(self): + compiler = shutil.which("g++") or shutil.which("clang++") + if compiler is None: + self.skipTest("a host C++17 compiler is required") + with tempfile.TemporaryDirectory(prefix=".tmp-client-acl-", dir=ROOT) as directory: + binary = Path(directory) / "client-acl-spiffs.exe" + compiled = subprocess.run([ + compiler, "-std=c++17", "-Wall", "-Wextra", "-DESP32=1", + "-DESP32_PLATFORM=1", f"-I{FIXTURE / 'mocks'}", f"-I{ROOT / 'src'}", + str(FIXTURE / "test_client_acl_spiffs.cpp"), "-o", str(binary), + ], capture_output=True, text=True, timeout=60) + self.assertEqual(compiled.returncode, 0, compiled.stdout + compiled.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=10) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("24 ClientACL SPIFFS checks passed", checked.stdout) + self.assertEqual(checked.stdout.count("PASS:"), 24) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_esp32_tinyusb_cooperative_output.py b/test/test_esp32_tinyusb_cooperative_output.py new file mode 100644 index 00000000..3861c5ca --- /dev/null +++ b/test/test_esp32_tinyusb_cooperative_output.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Execute the real native-TinyUSB role output pumps with small host stubs.""" + +from pathlib import Path +import os +import shutil +import subprocess +import tempfile +import unittest + + +ROOT = Path(__file__).resolve().parents[1] + +HARNESS = r''' +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#define MESH_ESP32_TINYUSB_NONBLOCKING 1 +#define MAX_ROUTE_HASH_BYTES 3 +#define PACKET_LOG_FILE "/packet_log" +static void require(bool ok, const char* what) { + if (!ok) throw std::runtime_error(what); +} +struct FileData { std::string text; size_t reads = 0; bool exists = true; }; +struct File { + std::shared_ptr data; + size_t pos = 0; + bool closed = false; + bool directory = false; + explicit operator bool() const { return data && (data->exists || directory) && !closed; } + bool isDirectory() const { return directory; } + size_t size() const { return data ? data->text.size() : 0; } + int read() { + if (!static_cast(*this) || pos >= data->text.size()) return -1; + ++data->reads; + return static_cast(data->text[pos++]); + } + void close() { closed = true; } +}; +struct FakeFS { + std::shared_ptr data = std::make_shared(); + bool exists(const char*) const { return data->exists; } + // ESP32 SPIFFS can return a truthy directory for a missing regular file. + File open(const char*) { return File{data, 0, false, !data->exists}; } +} fs; +struct Stream { + std::string output; + std::vector records; + size_t capacity = 4096; + size_t short_limit = 0; + bool fail_once = false; + int availableForWrite() const { return static_cast(capacity); } + size_t write(const uint8_t* data, size_t len) { + if (fail_once) { fail_once = false; return 0; } + if (len > capacity) return 0; + if (short_limit != 0) len = std::min(len, short_limit); + output.append(reinterpret_cast(data), len); + records.emplace_back(reinterpret_cast(data), len); + capacity -= len; + return len; + } + size_t printf(const char* format, ...) { + char record[1024]; + va_list args; + va_start(args, format); + int len = vsnprintf(record, sizeof(record), format, args); + va_end(args); + return len > 0 ? write(reinterpret_cast(record), len) : 0; + } +} console; +namespace mesh { +Stream& usbConsolePort() { return console; } +namespace Utils { +void toHex(char* dest, const uint8_t* src, size_t len) { + static const char digits[] = "0123456789ABCDEF"; + for (size_t i = 0; i < len; ++i) { + *dest++ = digits[src[i] >> 4]; + *dest++ = digits[src[i] & 15]; + } + *dest = 0; +} +} +} +struct SimpleMeshTables { + struct RecentRepeaterInfo { + uint8_t prefix[3]; + uint8_t prefix_len; + int8_t snr_x4; + }; + std::vector rows; + int getRecentRepeaterCount() const { return static_cast(rows.size()); } + const RecentRepeaterInfo* getNextRecentRepeaterBySortKey( + const RecentRepeaterInfo*, int previous, int& result) const { + result = previous + 1; + return result < static_cast(rows.size()) ? &rows[result] : nullptr; + } +}; +class MyMesh { + public: + FakeFS* _fs = &fs; + File serial_log_dump; + size_t serial_log_remaining = 0; + size_t serial_log_pending_size = 0; + char serial_log_pending[640]; + bool serial_log_active = false; + bool serial_log_eof_pending = false; + bool serial_log_skip_line = false; + int serial_recent_next = -1; + int serial_recent_count = 0; + bool serial_recent_header = false; + bool serial_recent_has_cursor = false; + SimpleMeshTables::RecentRepeaterInfo serial_recent_cursor{}; + int serial_recent_cursor_index = -1; + SimpleMeshTables tables; + const SimpleMeshTables* getTables() const { return &tables; } + void dumpLogFile(); + bool hasPendingSerialOutput() const; + void servicePendingSerialOutput(); + void cancelPendingSerialOutput(); + void printRecentRepeatersSerial(); +}; +@METHODS@ +static void setupFile(const std::string& text, bool exists = true) { + fs.data = std::make_shared(); + fs.data->text = text; + fs.data->exists = exists; + console = Stream(); +} +static void drain(MyMesh& radio, size_t capacity = 4096) { + int passes = 0; + while (radio.hasPendingSerialOutput() && passes++ < 10000) { + console.capacity = capacity; + const size_t reads = fs.data->reads; + radio.servicePendingSerialOutput(); + require(fs.data->reads - reads <= 640, "unbounded file read pass"); + } + require(!radio.hasPendingSerialOutput(), "output pump failed to finish"); +} +int main() { + try { + const std::string eof = " -> EOF\r\n"; + for (bool exists : {false, true}) { + setupFile("", exists); + MyMesh radio; + radio.dumpLogFile(); + require(exists || !radio.serial_log_active, "missing log accepted as a directory"); + require(console.output.empty(), "synchronous premature EOF"); + drain(radio); + require(console.output == eof, "missing or duplicate empty-file EOF"); + } + setupFile("hello\nworld\n"); + { + MyMesh radio; + radio.dumpLogFile(); + console.capacity = 0; + radio.servicePendingSerialOutput(); + const size_t reads = fs.data->reads; + radio.servicePendingSerialOutput(); + require(reads == fs.data->reads, "backpressure lost pending line"); + require(console.output.empty(), "wrote without capacity"); + console.short_limit = 3; + console.capacity = 4096; + radio.servicePendingSerialOutput(); + console.short_limit = 0; + drain(radio); + require(console.output == "hello\nworld\n" + eof, "short-write suffix lost"); + } + std::string large; + for (int i = 0; i < 1000; ++i) large += "stored packet\n"; + setupFile(large); + { + MyMesh radio; + radio.dumpLogFile(); + fs.data->text += "arrived later\n"; + drain(radio); + require(console.output == large + eof, "large dump lost bytes or ignored snapshot"); + for (const auto& record : console.records) + require(!record.empty() && record.back() == '\n', "split stored record"); + } + const std::string boundary(639, 'a'); + const std::string too_long(640, 'b'); + setupFile(boundary + "\n" + too_long + "\nend\ntail"); + { + MyMesh radio; + radio.dumpLogFile(); + drain(radio); + require(console.output == boundary + "\n" + "[USB log line omitted: exceeds 640 bytes]\r\nend\ntail\n" + eof, + "long line boundary or final partial line is wrong"); + } + setupFile(std::string(2000, 'x') + "\n"); + { + MyMesh radio; + radio.dumpLogFile(); + radio.servicePendingSerialOutput(); + require(radio.hasPendingSerialOutput(), "overlong line not pending"); + radio.cancelPendingSerialOutput(); + drain(radio); + require(console.output.empty(), "canceled old-session output leaked"); + } + @RECENT_TEST@ + std::cout << "cooperative output checks passed\n"; + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << "\n"; + return 1; + } +} +''' + +RECENT_TEST = r''' + setupFile(""); + { + MyMesh radio; + for (int i = 0; i < 2048; ++i) { + radio.tables.rows.push_back({{0, static_cast(i >> 8), + static_cast(i)}, 3, -127}); + } + console.fail_once = true; + radio.printRecentRepeatersSerial(); + require(radio.serial_recent_header, "failed header write advanced cursor"); + drain(radio, 64); + require(console.output.find("Recent repeaters (2048):\n") == 0, "missing header"); + require(std::count(console.output.begin(), console.output.end(), '\n') == 2049, + "2048-row listing truncated"); + require(console.output.substr(console.output.size() - 14) == "0007FF,-31.75\n", + "last recent repeater missing"); + for (const auto& record : console.records) + require(!record.empty() && record.back() == '\n', "split recent row"); + } +''' + + +class CooperativeOutputTest(unittest.TestCase): + def test_real_role_pumps(self): + compiler = shutil.which("g++") or shutil.which("clang++") + if not compiler: + self.skipTest("A host C++ compiler is required for pump execution") + for role in ("simple_repeater", "simple_room_server"): + with self.subTest(role=role), tempfile.TemporaryDirectory() as temporary: + text = (ROOT / f"examples/{role}/MyMesh.cpp").read_text() + dump_start = text.index("void MyMesh::dumpLogFile()") + dump_end = text.index("\n#if MESH_ESP32_TINYUSB_NONBLOCKING\nbool MyMesh::hasPendingSerialOutput", dump_start) + pump_start = text.index("bool MyMesh::hasPendingSerialOutput()", dump_end) + pump_end = text.index("\n#endif", pump_start) + methods = text[dump_start:dump_end] + "\n" + text[pump_start:pump_end] + if role == "simple_repeater": + format_start = text.index("static void formatLocalSnrX4(") + format_end = text.index("\nvoid MyMesh::formatRecentRepeatersReply", format_start) + recent_start = text.index("void MyMesh::printRecentRepeatersSerial()") + recent_end = text.index("\nbool MyMesh::setRecentRepeater(", recent_start) + methods = text[format_start:format_end] + "\n" + text[recent_start:recent_end] + "\n" + methods + program = HARNESS.replace("@METHODS@", methods).replace( + "@RECENT_TEST@", RECENT_TEST if role == "simple_repeater" else "" + ) + executable = Path(temporary) / ("pump.exe" if os.name == "nt" else "pump") + build = subprocess.run( + [compiler, "-std=c++17", "-O0", "-I", str(ROOT / "src"), + "-x", "c++", "-", "-o", str(executable)], + input=program, text=True, capture_output=True, timeout=60, + ) + self.assertEqual(build.returncode, 0, build.stdout + build.stderr) + run = subprocess.run( + [str(executable)], text=True, capture_output=True, timeout=30, + ) + self.assertEqual(run.returncode, 0, run.stdout + run.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_esp32_tinyusb_nonblocking.py b/test/test_esp32_tinyusb_nonblocking.py new file mode 100644 index 00000000..de89598d --- /dev/null +++ b/test/test_esp32_tinyusb_nonblocking.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python3 +"""Compile the real USB facade against a stalled, 64-byte ESP32 CDC FIFO.""" + +from pathlib import Path +import shutil +import subprocess +import tempfile +import unittest + + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/esp32_tinyusb_nonblocking" + + +class Esp32TinyUsbNonblockingTest(unittest.TestCase): + def test_native_fifo_and_other_role_fallback(self): + compiler = shutil.which("g++") or shutil.which("clang++") + if compiler is None: + self.skipTest("a host C++17 compiler is required") + with tempfile.TemporaryDirectory(prefix=".tmp-usb-mode0-", dir=ROOT) as temporary: + for mode, companion in ((0, False), (0, True), (1, False)): + with self.subTest(usb_mode=mode, companion=companion): + binary = Path(temporary) / f"usb-mode-{mode}-{companion}.exe" + command = [ + compiler, "-std=c++17", "-DARDUINO=1", "-DESP32=1", + "-DESP32_PLATFORM=1", "-DMESH_DEBUG=1", + "-DARDUINO_USB_CDC_ON_BOOT=1", f"-DARDUINO_USB_MODE={mode}", + f"-I{FIXTURE / 'mocks'}", f"-I{ROOT / 'src'}", + str(FIXTURE / "test_esp32_tinyusb_nonblocking.cpp"), + str(ROOT / "src/helpers/UsbLogging.cpp"), + "-o", str(binary), + ] + if companion: + command.insert(1, "-DENABLE_USB_INTERFACE=1") + built = subprocess.run(command, capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=10) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("transport checks passed", checked.stdout) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_esp32_tinyusb_role_hygiene.py b/test/test_esp32_tinyusb_role_hygiene.py new file mode 100644 index 00000000..2bc746f0 --- /dev/null +++ b/test/test_esp32_tinyusb_role_hygiene.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +"""Keep native-TinyUSB repeater/room output away from blocking Arduino writes.""" + +from pathlib import Path +import re +import unittest + + +ROOT = Path(__file__).resolve().parents[1] +ROLES = ("simple_repeater", "simple_room_server") + + +def source(relative: str) -> str: + return (ROOT / relative).read_text() + + +class Esp32TinyUsbRoleHygieneTest(unittest.TestCase): + def test_console_helper_preserves_other_platform_semantics(self): + text = source("src/helpers/UsbLogging.cpp") + self.assertRegex( + text, + r"Stream& usbConsolePort\(\)\s*\{\s*" + r"#if MESH_ESP32_TINYUSB_NONBLOCKING\s+" + r"return buffered_esp32_tinyusb_terminal_port;\s+" + r"#else\s+return Serial;\s+#endif", + ) + + def test_role_output_never_bypasses_the_usb_facade(self): + # A capacity check outside Serial.write() cannot make Arduino's + # native USBCDC writer bounded. Cover direct writes and Print/Stream + # references, including printHex's less obvious indirect writes. + for role in ROLES: + for filename in ("main.cpp", "MyMesh.cpp", "UITask.cpp"): + relative = f"examples/{role}/{filename}" + with self.subTest(path=relative): + text = source(relative) + self.assertNotRegex( + text, + r"\bSerial\s*\.\s*(?:print|println|printf|write|flush)\s*\(", + ) + self.assertNotRegex(text, r"\bprintHex\s*\(\s*Serial\b") + self.assertNotRegex( + text, + r"\b(?:Print|Stream)\s*&\s*\w+\s*=\s*Serial\b", + ) + self.assertIn("mesh::usbConsolePort()", text) + + def test_functional_replies_do_not_use_the_logging_gate(self): + for role in ROLES: + with self.subTest(role=role): + text = source(f"examples/{role}/MyMesh.cpp") + self.assertIn('mesh::usbConsolePort().printf("ACL:\\r\\n");', text) + self.assertIn( + 'mesh::usbConsolePort().printf("%02X %s\\n", ' + "c->permissions, public_key);", + text, + ) + self.assertIn( + 'mesh::usbConsolePort().printf("OTA: starting update\\r\\n");', + text, + ) + self.assertIn('mesh::usbConsolePort().printf("%s\\r\\n", wc_reply);', text) + start = text.index("void MyMesh::dumpLogFile()") + end = text.index("bool MyMesh::setTxPower(", start) + dump = text[start:end] + self.assertIn("mesh::usbConsolePort().print((char)c);", dump) + self.assertNotIn("isUsbLoggingEnabled", dump) + self.assertNotIn("usbLoggingPort", dump) + + repeater = source("examples/simple_repeater/MyMesh.cpp") + self.assertIn( + 'mesh::usbConsolePort().printf("Recent repeaters (%d):\\n", count);', + repeater, + ) + self.assertIn('mesh::usbConsolePort().printf("%s\\r\\n", record);', repeater) + + def test_unconditional_display_diagnostics_keep_their_visibility(self): + for role in ROLES: + with self.subTest(role=role): + text = source(f"examples/{role}/UITask.cpp") + self.assertIn("#include ", text) + self.assertIn("// Logged unconditionally:", text) + self.assertIn( + 'mesh::usbConsolePort().printf("Display: flip %s\\n",', + text, + ) + self.assertIn( + 'mesh::usbConsolePort().printf("Display: %s -> %s\\n",', + text, + ) + self.assertIn( + 'mesh::usbConsolePort().printf("Powering Off\\r\\n");', text + ) + + def test_both_roles_initialize_and_service_queued_terminal_output(self): + for role in ROLES: + with self.subTest(role=role): + text = source(f"examples/{role}/main.cpp") + setup = text[text.index("void setup()") : text.index("void loop()")] + loop = text[text.index("void loop()") :] + self.assertIn("mesh::beginUsbLoggingPort();", setup) + self.assertIn("mesh::serviceUsbTerminalPort();", loop) + + def test_sleep_keeps_raw_flush_only_for_other_esp32_transports(self): + text = source("src/helpers/ESP32Board.cpp") + sleep = text[text.index("void ESP32Board::enterDeepSleep(") :] + guard = re.search( + r"#if MESH_ESP32_TINYUSB_NONBLOCKING\s+" + r"mesh::serviceUsbTerminalPort\(\);\s+" + r"#else\s+Serial\.flush\(\);\s+#endif", + sleep, + ) + self.assertIsNotNone(guard) + self.assertEqual(sleep.count("Serial.flush();"), 1) + + def test_large_file_dumps_are_bounded_and_cancelable(self): + for role in ROLES: + with self.subTest(role=role): + text = source(f"examples/{role}/MyMesh.cpp") + header = source(f"examples/{role}/MyMesh.h") + self.assertIn("char serial_log_pending[640];", header) + self.assertIn("serial_log_active ? serial_log_dump.size() : 0", text) + start = text.index("void MyMesh::servicePendingSerialOutput()") + end = text.index("bool MyMesh::setTxPower(", start) + service = text[start:end] + self.assertIn("serial_log_pending_size < sizeof(serial_log_pending)", service) + self.assertIn("--serial_log_remaining;", service) + self.assertIn("serial_log_pending_size -= written;", service) + self.assertIn("memmove(serial_log_pending", service) + self.assertNotIn("delay(", service) + self.assertNotIn("flush(", service) + cancel = text[ + text.index("void MyMesh::cancelPendingSerialOutput()") : start + ] + self.assertIn("serial_log_dump.close();", cancel) + self.assertIn("serial_log_pending_size = 0;", cancel) + loop = text[text.index("void MyMesh::loop()") :] + self.assertLess( + loop.index("mesh::Mesh::loop();"), + loop.index("servicePendingSerialOutput();"), + ) + + def test_recent_list_advances_only_after_whole_row_admission(self): + text = source("examples/simple_repeater/MyMesh.cpp") + start = text.index("void MyMesh::servicePendingSerialOutput()") + service = text[start : text.index("if (!serial_log_active)", start)] + self.assertIn("char record[64];", service) + self.assertIn("serial_recent_next < serial_recent_count", service) + self.assertIn("getNextRecentRepeaterBySortKey", service) + self.assertNotIn("getRecentRepeaterBySortedIdx", service) + self.assertNotRegex(service, r"\b(?:while|for)\s*\(") + self.assertLess( + service.index("console.availableForWrite() < length"), + service.index("++serial_recent_next"), + ) + self.assertIn("!= static_cast(length)) return;", service) + + def test_functional_reserve_fits_default_full_acl_plus_command_echo(self): + logging = source("src/helpers/UsbLogging.cpp") + reserve = int(re.search( + r"esp32_tinyusb_functional_reserve\s*=\s*(\d+)", logging + ).group(1)) + acl = source("src/helpers/ClientACL.h") + clients = int(re.search(r"#define MAX_CLIENTS\s+(\d+)", acl).group(1)) + acl_bytes = len("ACL:\r\n") + clients * (2 + 1 + 64 + 1) + self.assertGreaterEqual(reserve, acl_bytes + len("get acl\r\n") + 160) + + def test_file_eof_and_malformed_line_marker_are_deferred(self): + for role in ROLES: + with self.subTest(role=role): + text = source(f"examples/{role}/MyMesh.cpp") + self.assertIn('strcmp(reply, " EOF") == 0) reply[0] = 0;', text) + self.assertIn('static const char eof[] = " -> EOF\\r\\n";', text) + self.assertIn("if (!serial_log_active) {", text) + self.assertIn("serial_log_eof_pending = false;", text) + self.assertIn("while (budget-- > 0 && serial_log_remaining > 0)", text) + self.assertIn("[USB log line omitted: exceeds 640 bytes]", text) + self.assertIn("if (serial_log_skip_line) return;", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_mesh_tables/test_simple_mesh_tables.cpp b/test/test_mesh_tables/test_simple_mesh_tables.cpp index 7b7b7414..78976733 100644 --- a/test/test_mesh_tables/test_simple_mesh_tables.cpp +++ b/test/test_mesh_tables/test_simple_mesh_tables.cpp @@ -347,6 +347,76 @@ TEST(SimpleMeshTables, ExpirationKeepsOccupiedEntriesPacked) { EXPECT_EQ(0, storage[3].prefix_len); } +TEST(SimpleMeshTables, CooperativeRecentCursorMatchesUnchangedRankOrder) { + SimpleMeshTables::RecentRepeaterInfo storage[8]; + SimpleMeshTables t(storage, 8); + for (int i = 0; i < 8; ++i) { + const uint8_t prefix[] = {static_cast(0x10 + i), 0x22, 0x33}; + ASSERT_TRUE(t.setRecentRepeater(prefix, 1 + i % 3, (i / 2) * 4)); + } + SimpleMeshTables::RecentRepeaterInfo cursor{}; + int cursor_index = -1; + for (int rank = 0; rank < 8; ++rank) { + int found_index = -1; + const auto* found = t.getNextRecentRepeaterBySortKey( + rank == 0 ? nullptr : &cursor, cursor_index, found_index); + ASSERT_NE(nullptr, found); + EXPECT_EQ(t.getRecentRepeaterBySortedIdx(rank), found); + cursor = *found; + cursor_index = found_index; + } + int found_index = 123; + EXPECT_EQ(nullptr, t.getNextRecentRepeaterBySortKey(&cursor, cursor_index, found_index)); + EXPECT_EQ(-1, found_index); +} + +TEST(SimpleMeshTables, CooperativeRecentCursorSurvivesLiveStorageCompaction) { + SimpleMeshTables::RecentRepeaterInfo storage[3]; + SimpleMeshTables t(storage, 3); + const uint8_t first[] = {0x10}; + const uint8_t second[] = {0x20}; + const uint8_t third[] = {0x30}; + ASSERT_TRUE(t.setRecentRepeater(first, 1, 12)); + ASSERT_TRUE(t.setRecentRepeater(second, 1, 8)); + ASSERT_TRUE(t.setRecentRepeater(third, 1, 4)); + int cursor_index = -1; + const auto* first_row = t.getNextRecentRepeaterBySortKey(nullptr, -1, cursor_index); + ASSERT_NE(nullptr, first_row); + const auto cursor = *first_row; + storage[0].last_heard_millis = 0; + storage[1].last_heard_millis = 100; + storage[2].last_heard_millis = 100; + ASSERT_EQ(1, t.expireRecentRepeaters(101, 50)); + int next_index = -1; + const auto* next = t.getNextRecentRepeaterBySortKey(&cursor, cursor_index, next_index); + ASSERT_NE(nullptr, next); + EXPECT_EQ(0x20, next->prefix[0]); + EXPECT_EQ(0x10, cursor.prefix[0]); +} + +TEST(SimpleMeshTables, CooperativeRecentCursorTraverses2048Rows) { + SimpleMeshTables::RecentRepeaterInfo storage[2048]; + SimpleMeshTables t(storage, 2048); + for (int i = 0; i < 2048; ++i) { + const uint8_t prefix[] = {0x80, static_cast(i >> 8), + static_cast(i)}; + ASSERT_TRUE(t.setRecentRepeater(prefix, 3, 4)); + } + SimpleMeshTables::RecentRepeaterInfo cursor{}; + int cursor_index = -1; + for (int i = 0; i < 2048; ++i) { + int found_index = -1; + const auto* found = t.getNextRecentRepeaterBySortKey( + i == 0 ? nullptr : &cursor, cursor_index, found_index); + ASSERT_NE(nullptr, found); + EXPECT_EQ(static_cast(i >> 8), found->prefix[1]); + EXPECT_EQ(static_cast(i), found->prefix[2]); + cursor = *found; + cursor_index = found_index; + } + EXPECT_EQ(nullptr, t.getNextRecentRepeaterBySortKey(&cursor, cursor_index, cursor_index)); +} + int main(int argc, char** argv) { ::testing::InitGoogleTest(&argc, argv); return RUN_ALL_TESTS(); diff --git a/test/test_regular_file_reads.py b/test/test_regular_file_reads.py new file mode 100644 index 00000000..c0a34deb --- /dev/null +++ b/test/test_regular_file_reads.py @@ -0,0 +1,259 @@ +#!/usr/bin/env python3 +"""Execute production file readers against SPIFFS-style phantom directories.""" + +from pathlib import Path +import os +import shutil +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] + + +def section(path, start, end): + text = (ROOT / path).read_text(encoding="utf-8") + begin = text.index(start) + return text[begin:text.index(end, begin)] + + +HARNESS = r''' +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#define FILE_O_READ 0 +static void check(bool ok, const char* why) { + if (!ok) throw std::runtime_error(why); +} +struct Entry { std::string text; bool directory = false; }; +struct FakeFS; +struct File { + std::shared_ptr entry; + size_t position = 0; + FakeFS* owner = nullptr; +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + // Actual Adafruit LittleFS requires an owner even for a closed File. + File() = delete; +#else + File() = default; +#endif + explicit File(FakeFS& fs) : owner(&fs) {} + File(FakeFS& fs, std::shared_ptr data) : entry(data), owner(&fs) {} + explicit operator bool() const { return bool(entry); } + bool isDirectory() const { return entry && entry->directory; } + size_t size() const { return entry && !isDirectory() ? entry->text.size() : 0; } + size_t available() const { return size() - position; } + size_t read(uint8_t* dest, size_t count) { + if (!entry || isDirectory()) return 0; + count = std::min(count, available()); + memcpy(dest, entry->text.data() + position, count); + position += count; + return count; + } + void close() { entry.reset(); } +}; +struct FakeFS { + std::map> entries; + bool fail_open = false; + bool report_directory_exists = false; + unsigned missing_opens = 0; + unsigned open_calls = 0; + bool exists(const char* path) { + auto it = entries.find(path); + return it != entries.end() && + (!it->second->directory || report_directory_exists); + } + File fixtureOpen(const char* path) { + ++open_calls; + if (fail_open) return File(*this); + auto it = entries.find(path); + if (it != entries.end()) return File(*this, it->second); + ++missing_opens; + return File(*this, std::make_shared(Entry{"", true})); + } +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + File open(const char* path, uint8_t = FILE_O_READ) { return fixtureOpen(path); } +#elif defined(RP2040_PLATFORM) + // Philhower FS requires the mode argument, unlike Arduino ESP32. + File open(const char* path, const char*) { return fixtureOpen(path); } +#else + File open(const char* path, const char* = "r", bool = false) { return fixtureOpen(path); } +#endif + void put(const char* path, const std::string& content, bool directory = false) { + entries[path] = std::make_shared(Entry{content, directory}); + } +} SPIFFS; +FakeFS InternalFS; +using FILESYSTEM = FakeFS; +class DataStore { + public: + FILESYSTEM* _fs; + explicit DataStore(FILESYSTEM* fs) : _fs(fs) {} + File openRead(const char*); + File openRead(FILESYSTEM*, const char*); + File openDirectory(const char*); + File openDirectory(FILESYSTEM*, const char*); +}; +@DATASTORE@ +struct WiFiClient { + std::string output; + bool connected() const { return true; } + size_t write(const uint8_t* data, size_t size) { + output.append(reinterpret_cast(data), size); + return size; + } + void printf(const char* format, ...) { + char buffer[512]; + va_list args; + va_start(args, format); + int size = vsnprintf(buffer, sizeof(buffer), format, args); + va_end(args); + check(size >= 0 && static_cast(size) < sizeof(buffer), "HTTP overflow"); + output.append(buffer, size); + } +}; +struct LogServer { + bool running = true; + @RESPONSE@ + @SENDLOG@ +}; +@FLOOD_VERIFY@ +int main() { + try { + FakeFS fs; + DataStore store(&fs); + File unopened = mesh::emptyFile(&fs); + File unavailable = mesh::emptyFile(static_cast(nullptr)); + check(!unopened && !unavailable, "empty helper returned an open file"); + check(fs.open_calls == 0 && InternalFS.open_calls == 0, + "closed file construction must never access either filesystem"); +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + check(unopened.owner == &fs, "closed LittleFS file lost its selected owner"); + check(unavailable.owner == &InternalFS, "null filesystem lacks a valid fallback owner"); +#endif + File phantom = fs.fixtureOpen("/missing"); + check(bool(phantom) && phantom.isDirectory(), "fixture must reproduce SPIFFS"); + unsigned before = fs.missing_opens; + check(!store.openRead("/missing"), "missing file accepted"); + check(fs.missing_opens == before, "missing regular file should not be opened"); + check(!store.openRead(nullptr, "/file"), "unavailable FS accepted"); + check(!store.openRead(nullptr), "null path accepted"); + check(!store.openDirectory(nullptr, "/"), "unavailable directory FS accepted"); + check(!store.openDirectory(nullptr), "null directory path accepted"); + fs.put("/empty", ""); + fs.put("/real", "data"); + check(bool(store.openRead("/empty")), "real empty file rejected"); + File real = store.openRead("/real"); + uint8_t bytes[4]; + check(real.read(bytes, sizeof(bytes)) == 4 && memcmp(bytes, "data", 4) == 0, + "real file changed"); + check(!store.openDirectory("/real"), "regular file accepted as directory"); + check(store.openDirectory("/").isDirectory(), "SPIFFS root listing broken"); + fs.put("/directory", "", true); + fs.report_directory_exists = true; + check(!store.openRead("/directory"), "directory accepted when exists is true"); + check(store.openDirectory("/directory").isDirectory(), "real listing broken"); + fs.fail_open = true; + check(!store.openRead("/real"), "failed read-open accepted"); + fs.fail_open = false; + // Empty expected images must not validate a missing file/directory as data. + check(!verifyFloodSettingsWrite(&fs, "/missing", 0, 2166136261UL), + "missing flood file passed empty verification"); + check(!verifyFloodSettingsWrite(&fs, "/directory", 0, 2166136261UL), + "directory passed empty verification"); + check(verifyFloodSettingsWrite(&fs, "/empty", 0, 2166136261UL), + "real empty image should verify"); + check(verifyFloodSettingsWrite(&fs, "/real", 4, + updateFloodSettingsHash(2166136261UL, reinterpret_cast("data"), 4)), + "valid flood image rejected"); + LogServer server; + WiFiClient missing; + server.sendLog(missing); + check(missing.output.find("HTTP/1.1 404 Not Found\r\n") == 0, + "missing packet log must return HTTP404"); + SPIFFS.put("/packet_log", ""); + WiFiClient empty; + server.sendLog(empty); + check(empty.output.find("HTTP/1.1 200 OK\r\n") == 0 && + empty.output.find("Content-Length: 0\r\n") != std::string::npos, + "existing empty packet log must remain HTTP200"); + SPIFFS.put("/packet_log", "packet\n"); + WiFiClient log; + server.sendLog(log); + check(log.output.find("Content-Length: 7\r\n") != std::string::npos && + log.output.substr(log.output.size() - 7) == "packet\n", "log download changed"); + SPIFFS.put("/packet_log", "", true); + SPIFFS.report_directory_exists = true; + WiFiClient directory; + server.sendLog(directory); + check(directory.output.find("HTTP/1.1 404 Not Found\r\n") == 0, + "directory log must return HTTP404"); + std::cout << "regular-file readers: PASS\n"; + return 0; + } catch (const std::exception& e) { std::cerr << e.what() << '\n'; return 1; } +} +''' + + +class RegularFileReadTest(unittest.TestCase): + def test_production_readers(self): + compiler = shutil.which("g++") or shutil.which("clang++") + if not compiler: + self.skipTest("A host C++ compiler is required") + program = HARNESS.replace("@DATASTORE@", section( + "examples/companion_radio/DataStore.cpp", "File DataStore::openRead(", + "bool DataStore::removeFile(")) + program = program.replace("@RESPONSE@", section( + "src/helpers/ESP32Board.cpp", " static void sendResponse(", " bool readLine(")) + program = program.replace("@SENDLOG@", section( + "src/helpers/ESP32Board.cpp", " void sendLog(", " void sendUpdateError(")) + read_helper = section("examples/simple_repeater/MyMesh.cpp", + "static File openFloodSettingsRead(", + "static File openFloodSettingsWrite(") + verification = section("examples/simple_repeater/MyMesh.cpp", + "static uint32_t updateFloodSettingsHash(", + "static uint8_t batteryPercentFromMilliVolts(") + program = program.replace("@FLOOD_VERIFY@", read_helper + verification) + for platform in ("ESP32_PLATFORM", "NRF52_PLATFORM", "RP2040_PLATFORM", "STM32_PLATFORM"): + with self.subTest(platform=platform), tempfile.TemporaryDirectory() as tmp: + executable = Path(tmp) / ("readers.exe" if os.name == "nt" else "readers") + built = subprocess.run( + [compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", "-O0", + f"-D{platform}", "-I", str(ROOT / "test/fixtures/regular_file_reads"), + "-I", str(ROOT / "src"), "-x", "c++", "-", + "-o", str(executable)], + input=program, text=True, capture_output=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + run = subprocess.run([str(executable)], capture_output=True, text=True, timeout=15) + self.assertEqual(run.returncode, 0, run.stdout + run.stderr) + + def test_directory_listing_uses_explicit_directory_api(self): + cli = (ROOT / "examples/companion_radio/MyMesh.cpp").read_text() + self.assertIn("File root = _store->openDirectory(path);", cli) + self.assertIn("File root2 = _store->openDirectory(_store->getSecondaryFS(), path);", cli) + self.assertIn("File file = _store->openRead(selected_fs, path);", cli) + + def test_audited_read_helpers_remain_regular_file_only(self): + for path, marker, end in ( + ("src/helpers/ClientACL.cpp", "static File openRead(", "#if !defined(NRF52_PLATFORM)"), + ("examples/simple_room_server/FloodRuleEngine.cpp", "static File openRead(", "static File openWrite("), + ): + with self.subTest(path=path): + self.assertIn("mesh::openFileRead(", section(path, marker, end)) + for role in ("simple_repeater", "simple_room_server"): + dump = section(f"examples/{role}/MyMesh.cpp", "void MyMesh::dumpLogFile()", + "bool MyMesh::hasPendingSerialOutput()") + self.assertEqual(dump.count("mesh::openFileRead(_fs, PACKET_LOG_FILE)"), 2) + self.assertNotIn("_fs->open(", dump) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_replay_reset_command.py b/test/test_replay_reset_command.py new file mode 100644 index 00000000..5442e0cd --- /dev/null +++ b/test/test_replay_reset_command.py @@ -0,0 +1,50 @@ +#!/usr/bin/env python3 +"""Native, hardware-free tests of replay-reset parsing and one-use challenges.""" + +from pathlib import Path +import os +import shutil +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] + + +class ReplayResetCommandTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + compiler = shutil.which("g++") or shutil.which("clang++") + if compiler is None: + raise unittest.SkipTest("a host C++ compiler is required") + cls.directory = tempfile.TemporaryDirectory(prefix="mesh-replay-reset-") + cls.addClassCleanup(cls.directory.cleanup) + cls.executable = Path(cls.directory.name) / ( + "replay-reset.exe" if os.name == "nt" else "replay-reset" + ) + subprocess.run( + [compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", "-pedantic", + "-I", str(ROOT / "src"), + str(ROOT / "test/fixtures/replay_reset_command/main.cpp"), + "-o", str(cls.executable)], + check=True, + ) + + def run_case(self, name): + subprocess.run([str(self.executable), name], check=True) + + def test_strict_parser_and_family_classification(self): + self.run_case("parser") + + def test_one_use_identity_bound_lifecycle(self): + self.run_case("lifecycle") + + def test_expiration_rollover_and_clock_changes(self): + self.run_case("time") + + def test_invalid_arguments_and_complete_binding(self): + self.run_case("invalid") + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_replay_reset_integration.py b/test/test_replay_reset_integration.py new file mode 100644 index 00000000..9babc3e0 --- /dev/null +++ b/test/test_replay_reset_integration.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +"""Compile the production replay-reset handler and verify its transport wiring.""" +from pathlib import Path +import re +import shutil +import subprocess +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/replay_reset_integration" +SOURCE = ROOT / "examples/simple_repeater/MyMesh.cpp" +HEADER = ROOT / "examples/simple_repeater/MyMesh.h" +MAIN = ROOT / "examples/simple_repeater/main.cpp" + + +def extract_braced(source, signature): + """Return an actual C++ definition/block, ignoring quoted/comment braces.""" + start = source.index(signature) + opening = source.index("{", start) + depth = 0 + token = re.compile(r'//[^\n]*|/\*[\s\S]*?\*/|"(?:\\.|[^"\\])*"|\'(?:\\.|[^\'\\])*\'|[{}]') + for match in token.finditer(source, opening): + if match.group() == "{": + depth += 1 + elif match.group() == "}": + depth -= 1 + if depth == 0: + return source[start:match.end()] + raise AssertionError(f"unterminated C++ block: {signature}") + + +class ReplayResetIntegrationTest(unittest.TestCase): + def test_executable_production_handler(self): + compiler = shutil.which("g++") or shutil.which("clang++") + if compiler is None: + self.skipTest("a host C++17 compiler is required") + source = SOURCE.read_text(encoding="utf-8") + handler = extract_braced(source, "bool MyMesh::handleReplayResetCommand(") + epoch_check = extract_braced(source, "static bool clockSyncEpochIsValid(") + guard = extract_braced(source, "if (!replay_command && sender_timestamp > client->last_timestamp)") + prepare = re.search(r"const bool replay_prepare\s*=[\s\S]*?;", source).group() + cache_gate = re.search(r"const bool cached_retry\s*=\s*!replay_prepare[\s\S]*?;", source).group() + generated = epoch_check + "\n" + handler + "\n" + """ +static void apply_actual_receive_guard(ClientInfo* client, const char* command, + uint32_t sender_timestamp) { + mesh::ReplayResetRequest replay_request; + const bool replay_command = mesh::parseReplayResetCommand(command, replay_request) + != mesh::ReplayResetKind::NotReplay; +""" + guard + "\n}\n" + """ +static bool apply_actual_receive_cache_gate(const char* command, bool cache_hit, + int& lookup_calls) { + mesh::ReplayResetRequest replay_request; + mesh::parseReplayResetCommand(command, replay_request); + ClientInfo sender; + ClientInfo* client = &sender; + uint32_t request_id = 1, command_fingerprint = 2; + const char* cached_response = nullptr; + CountingReplyCache remote_cli_reply_cache{cache_hit, lookup_calls}; +""" + prepare + "\n" + cache_gate + "\nreturn cached_retry;\n}\n" + with tempfile.TemporaryDirectory(prefix=".tmp-replay-integration-", dir=ROOT) as directory: + work = Path(directory) + (work / "production_handler.inc").write_text(generated, encoding="utf-8") + binary = work / "replay-reset-integration.exe" + compiled = subprocess.run([ + compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", + f"-I{work}", f"-I{ROOT / 'src'}", + str(FIXTURE / "test_replay_reset_integration.cpp"), "-o", str(binary), + ], capture_output=True, text=True, timeout=60) + self.assertEqual(compiled.returncode, 0, compiled.stdout + compiled.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=10) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("18 replay-reset integration checks passed", checked.stdout) + self.assertEqual(checked.stdout.count("PASS:"), 18) + + def test_only_physical_console_grants_usb_origin(self): + header = HEADER.read_text(encoding="utf-8") + main = MAIN.read_text(encoding="utf-8") + self.assertIn("bool usb_origin = false", header) + usb = extract_braced(header, "void handleUsbCommand(") + self.assertIn("handleCommand(0, NULL, command, reply, -1, 1, true)", usb) + null_sender = extract_braced(header, "void handleCommand(uint32_t sender_timestamp, char* command, char* reply)") + self.assertIn("handleCommand(sender_timestamp, NULL, command, reply)", null_sender) + console_start = main.index("if (line_complete)") + ethernet_start = main.index("if (ethernet_read_line(", console_start) + self.assertIn("the_mesh.handleUsbCommand(command, reply)", main[console_start:ethernet_start]) + ethernet = extract_braced(main, "if (ethernet_read_line(") + self.assertNotIn("handleUsbCommand", ethernet) + self.assertIn("the_mesh.handleCommand(0, ethernet_command, reply)", ethernet) + # No web/internal callback is allowed to adopt the physical entry point. + cpp = SOURCE.read_text(encoding="utf-8") + self.assertNotIn("handleUsbCommand(", cpp) + + def test_receiver_keeps_authentication_and_blocks_replay_floor_mutation(self): + source = SOURCE.read_text(encoding="utf-8") + cli_start = source.index("else if (type == PAYLOAD_TYPE_TXT_MSG && len > 5") + cli_end = source.index("void MyMesh::", cli_start) + cli = source[cli_start:cli_end] + self.assertIn("client->isAdmin() || client->isRegionMgr() || client->isFilterMgr()", cli) + parser = cli.index("mesh::parseReplayResetCommand(command, replay_request)") + stale = cli.index("sender_timestamp < client->last_timestamp && !cached_retry") + assignment = cli.index("client->last_timestamp = sender_timestamp;") + self.assertLess(parser, stale) + self.assertLess(stale, assignment) + guarded = extract_braced(cli, "if (!replay_command && sender_timestamp > client->last_timestamp)") + self.assertIn("client->last_timestamp = sender_timestamp;", guarded) + self.assertEqual(cli.count("client->last_timestamp = sender_timestamp;"), 1) + self.assertIn("const bool replay_prepare = replay_request.kind == mesh::ReplayResetKind::ExactKey;", cli) + self.assertRegex(cli, r"const bool cached_retry\s*=\s*!replay_prepare\s*&&\s*remote_cli_reply_cache\.lookup\(") + self.assertLess(cli.index("const bool cached_retry"), stale) + handler = extract_braced(source, "bool MyMesh::handleReplayResetCommand(") + self.assertNotIn("remote_cli_reply_cache.clear", handler) + self.assertLess(handler.index("replay_reset_nonce.consume("), + handler.index("acl.clampLoginReplayTimestamps(")) + + def test_recovery_dispatch_precedes_other_command_handlers(self): + source = SOURCE.read_text(encoding="utf-8") + handler = extract_braced(source, "void MyMesh::handleCommand(uint32_t sender_timestamp, ClientInfo* sender,") + recovery = handler.index("handleReplayResetCommand(sender, command, reply, usb_origin)") + self.assertLess(recovery, handler.index("_cli.handleCommand(")) + self.assertNotIn("sender_timestamp == 0", handler[:recovery]) + + +if __name__ == "__main__": + unittest.main()