diff --git a/.github/workflows/run-unit-tests.yml b/.github/workflows/run-unit-tests.yml index 638cb81f0..ab361c2ed 100644 --- a/.github/workflows/run-unit-tests.yml +++ b/.github/workflows/run-unit-tests.yml @@ -111,12 +111,22 @@ jobs: run: | python3 -B test/test_protocol_parser_bounds.py -v python3 -B test/test_companion_offline_delivery.py -v + python3 -B test/test_room_receive_admission.py -v python3 -B test/test_companion_lfs_traversal.py -v python3 -B test/test_room_post_permissions.py -v python3 -B test/test_room_login_authorization.py -v python3 -B test/test_room_login_integration.py -v python3 -B test/test_room_topic_store.py -v python3 -B test/test_room_topic_delivery.py -v + python3 -B test/test_room_acl_persistence.py -v + python3 -B test/test_room_access_policy.py -v + python3 -B test/test_room_history_store.py -v + python3 -B test/test_room_history_integration.py -v + python3 -B test/test_room_board_store.py -v + python3 -B test/test_room_board_protocol.py -v + python3 -B test/test_room_web_mailbox.py -v + python3 -B test/test_room_web_http.py -v + python3 -B test/test_room_web_service.py -v python3 -B test/test_contact_clock_bootstrap.py -v python3 -B test/test_rtc_discovery.py -v python3 -B test/test_radio_interrupt_recovery.py -v @@ -537,6 +547,7 @@ jobs: python3 -B test/test_heltec_v4_wifi_setup_page.py -v python3 -B test/test_sensor_webconfig_capabilities.py -v python3 -B test/test_webconfig_ui_runtime.py + MESHCORE_REQUIRE_ROOM_BROWSER=1 python3 -B test/test_room_web_ui.py -v - name: Upload Test Results # Upload test results even if the test step failed. diff --git a/README.md b/README.md index b3eed4508..cd31e6013 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ * **[Flood retries that account for network topology](https://mikecarper.github.io/MeshCore/halo_keymind_settings/#flood-and-advert-settings).** Configure retry counts, path limits, packet-type limits, target prefixes, ignored repeaters, and channel-specific eligibility. Bridge buckets keep retrying until traffic is heard on the intended sides of a relay or the retry budget expires. Qualifying echoes cancel unnecessary attempts. * **[Enhanced direct messages and direct-route retries](https://mikecarper.github.io/MeshCore/halo_keymind_settings/#direct-retry-settings).** Adapt the coding rate across bounded retry attempts and cancel unnecessary retries when a qualifying echo is heard. [Single Key DMs](https://mikecarper.github.io/MeshCore/companion_protocol/#private-text-when-only-the-sender-knows-the-recipient) let compatible Companions send private text when only the sender knows the recipient's key, using a signed, encrypted introduction. The recipient accepts the sender through the app or opts into automatic acceptance; compact STM32WL builds omit this feature. * **[Enhanced telemetry history](https://mikecarper.github.io/MeshCore/telemetry_decoder/).** Send compact temperature and battery-voltage snapshots of up to 165 samples per packet, plus external-voltage history in chunks. Read retained GPS history through the CLI; GPS history is not included in the automatic telemetry packets. Decode snapshots and CLI history in the browser. -* **[Room topics and access controls](https://mikecarper.github.io/MeshCore/cli_commands/#view-or-change-the-room-topic).** Save a room announcement across reboots and deliver it through the existing app message format while preserving unread chat catch-up. Public read-only login and explicit ACL roles are respected; guest-password login cannot demote an admin. +* **[Room topics, history, and access controls](https://mikecarper.github.io/MeshCore/room_services/).** Save an announcement and optionally retain the newest 32 chat posts across reboots. Explicit ACL roles persist; full-key bans and per-user post/request limits apply to room sessions. ESP32 images with WebConfig provide local Wi-Fi chat and a persistent information board with selected, resumable article downloads. The existing app's chat and catch-up formats remain supported. * **[Full firmware profiles, Companion transports, and ESP32 bridges](https://mikecarper.github.io/MeshCore/full_profiles/).** Full combines the exact board and role's supported features with runtime controls: Companion transports and firmware serving, or qualified ESP32 infrastructure services and MQTT/ESP-NOW/RS-232 bridges. Includes capacity exceptions, matching application slots, exact-board/role partition migration packages, and nRF52 Full/Reduced supported-sensor packages with LoRa OTA. The picker links supported migration routes; USB layout changes use the matching merged image, since an app-only image cannot change the partition table. * **[Operational reporting and alerts](https://mikecarper.github.io/MeshCore/management_reports/).** Adds optional management reports with firmware version, bootloader version, LoRa update capabilities, uptime, and historical voltage and temperature ranges. * **[Set time using the LoRa network](https://mikecarper.github.io/MeshCore/cli_commands/#estimate-and-correct-infrastructure-node-time-after-startup).** Infrastructure nodes can estimate time from fresh mesh evidence and correct their clocks when a consensus is available. Successful manual, GPS, or NTP synchronization takes precedence until reboot. diff --git a/docs/cli_commands.md b/docs/cli_commands.md index c7888c3d0..e22e0e1c1 100644 --- a/docs/cli_commands.md +++ b/docs/cli_commands.md @@ -3813,8 +3813,12 @@ a packet already queued for transmission can still arrive. Topic saves are verified before publication. A failed save reports an error and retains the previous live topic. Corrupt, unreadable, or unsupported stored topic files are preserved and block replacement until their storage issue is -repaired. Topics do not make ordinary room history persistent: retained posts -still use the existing bounded RAM ring and are lost on reboot. +repaired. Ordinary room history uses the bounded RAM ring by default. Enable +optional retention of the newest 32 posts with `set room.history on`. + +See [Room history, moderation, and information board](room_services.md) for +persistent history, saved ACL roles, full-key bans, per-user post/request +limits, the ESP32 `/room` browser page, and versioned article downloads. --- diff --git a/docs/index.md b/docs/index.md index acc635ed5..907cc58e8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,6 +17,7 @@ Choose firmware with the [firmware picker](firmware_picker.md). For help with a - [Firmware update guide](ota_user_guide.md), [easy LoRa OTA](ota_easy.md), and [scripted LoRa OTA](lora_ota_automation.md) - [CLI commands](cli_commands.md), [availability by build](cli_build_matrix.md), and [radio profiles](radio_profiles.md) - [Private fleet management](fleet_management.md): signed filtering and secondary-radio commands for a fleet or one node +- [Room history, moderation, and information board](room_services.md): optional persistent chat and local Wi-Fi room access - [Companion protocol](companion_protocol.md), [packet format](packet_format.md), and [QR codes](qr_codes.md) The navigation groups the remaining guides under Documentation, including board-specific updates, networking, power, and protocol references. diff --git a/docs/room_services.md b/docs/room_services.md new file mode 100644 index 000000000..974ff4590 --- /dev/null +++ b/docs/room_services.md @@ -0,0 +1,220 @@ + + + + +# Room history, moderation, and information board + +Room Server firmware keeps one room per radio. Existing app login, chat, +catch-up, and Access Control formats remain supported. The information board +and local Wi-Fi chat use the browser page below; the stock mobile app does not +have an information-board screen. + +## Optional persistent chat history + +History uses RAM by default. From a local console or authenticated remote +administrator session: + +~~~text +get room.history +set room.history on +set room.history off +room.history.clear +~~~ + +Enabling persistence first saves the current RAM ring. The newest **32 posts** +then survive reboot. Disabling retains the archive and stops disk writes; the +next enable replaces it with the current RAM ring. Clearing removes disk and +RAM posts, leaving the topic and information board alone. + +Accepted posts append 192-byte records. At 64 records the journal compacts to +the newest 32. Its maximum size is 12,300 bytes; compaction temporarily also +needs a verified snapshot of about 6.2 KB. This is bounded recent history, +and its filesystem records are not encrypted at rest. + +A post is saved before the server sends its ACK or advances the live ring, +retry cache, or sender replay timestamp. Full storage and unsuccessful writes +retain the previous chat and withhold the ACK. A recoverable incomplete or +corrupt final append keeps its verified earlier records. Corruption before +later complete records preserves the entire journal and blocks replacement. +Unreadable, unsupported, or corrupt primary metadata is also preserved until +storage is repaired. + +A reboot after storage completes but before the sender receives the ACK can +recover a valid post without confirming it to the sender. Exactly-once delivery +across that crash boundary is not guaranteed. + +Catch-up timestamps stay increasing across restored history and backward clock +corrections without setting the radio's RTC from archived posts. New posts use +a six-second monotonic delivery delay; restored posts can be delivered +immediately. Set the clock correctly for useful displayed dates. + +The Companion also withholds a room-message ACK when its receive queue cannot +retain the message. Rejected delivery does not advance catch-up, schedule +receipt persistence, or emit a new-message notification. + +## Saved ACL roles and moderation + +An operator's nonzero `setperm` assignment survives reboot and is protected +from automatic room-client eviction. Assigned read-only and read/write users +appear in Access Control alongside administrators. Automatic guest-password +read/write sessions remain transient; password-created administrators remain +retained. The existing permission byte is unchanged. + +ACL updates use the existing delayed settings save. Allow at least five seconds +after `setperm` before removing power so the assignment reaches storage. + +~~~text +setperm 1 +setperm 2 +setperm 3 +setperm 0 +room.ban +room.unban +get room.bans +get room.bans 2 +set room.post.rate 12 +set room.poll.rate 60 +get room.post.rate +get room.poll.rate +~~~ + +Roles are `1` read-only, `2` read/write, and `3` administrator; `0` removes +the assignment. Guest-password login cannot demote an explicit administrator +or upgrade an explicit read-only user. Empty passwords are not enabled +credentials; public passwordless read-only login still requires its existing +setting. + +The persistent ban list accepts up to 32 complete identities, never names or +prefixes. It excludes new logins and existing sessions, including posts, +requests, path updates, ACKs, and pushes. It leaves existing posts and ACL +assignments intact. Use another authorized administrator or a local console +to undo a ban on your own identity. + +Rates are per identity per shared monotonic minute. `0` disables a limit +(the default); the maximum is 65,535. The post limit covers new chat posts. +The poll limit covers room requests, including keep-alive, status, telemetry, +ACL and board reads, and authenticated browser operations. +Login does not reset counts. Exact accepted retries are not charged again. +Old keep-alive retries cannot rewind catch-up or cancel a newer pending push. +Rejected posts do not enter the retry cache. + +Changing to a different identity creates a different user. These controls +complement forwarding and airtime policies. + +## Local Wi-Fi chat on ESP32 + +ESP32 Room Server images with WebConfig have an offline browser room page. +It is **disabled by default** and uses the existing WebConfig listener: + +~~~text +set room.web on +start webconfig +~~~ + +Open `http:///room`. Without a usable router, use +`start webconfig ap`, connect the phone to its access point, and append +`/room` to the reported address. Existing setup AP time limits and Wi-Fi/OTA +ownership rules apply. Starting OTA shuts down this listener. Compact images +and boards without WebConfig retain LoRa chat and CLI access. + +Enter the guest or administrator room password, or leave it empty for +explicitly enabled public read-only access. Readers cannot post or edit +notices; only administrators can publish or delete them. Each operation +rechecks credentials and bans. Use private Wi-Fi: the local page uses HTTP, +and network access alone does not grant administrator permission. + +The browser generates a secret token and its own moderation identity. It +does not import a radio private key or inherit a claimed radio key's ACL role. +The Access and history panel shows the complete browser identity for banning. +Posts share the LoRa chat history and include the display name; combined name +and text must fit 151 UTF-8 bytes. + +Passwords stay in page memory. The token, sequence and display name are saved +locally; use one active room tab per browser identity. Interrupted submissions +retry the same sequence and body, then poll the owned result. Retry floors +are bounded to twenty minutes and the current boot. The page stops retrying +after twenty seconds and preserves an unconfirmed draft. After a restart or +uncertain result, refresh and check before submitting again. + +Authentication, storage and radio work run on the mesh loop. HTTP callbacks +only admit a bounded request and return its owned result. No arbitrary CLI +operation is exposed. Disable room access with `set room.web off`; stop the +portal with `stop webconfig`. + +## Information board + +Eight persistent article slots have titles up to 63 UTF-8 bytes and bodies up +to 2,048 bytes. Index revisions and article versions survive reboot. Bodies +use two bounded file banks per slot, streamed verification, and atomic index +publication. + +Available filesystem space determines how much fits. Small internal nRF52 +filesystems cannot hold every article at maximum size alongside persistent +history and settings. Leave space for the history snapshot and article/index +replacement; a full filesystem rejects the change and retains existing data. + +In `/room`, refresh the notice index and select an article to download. +Unselected bodies are not pushed or downloaded. Chunks are at most 128 bytes; +completed chunks remain available within the page after a lost connection. +A new article version starts a new transfer. The administrator editor checks +the version of the loaded article before saving to prevent lost edits. + +Small notices also support administrator/local CLI management: + +~~~text +room.board.put 1 Trail conditions|Bridge closed until Saturday. +get room.board +get room.board 1 +get room.board.read 1 0 +room.board.del 1 +~~~ + +Index arguments are zero-based entry cursors; follow the returned `next=`. +Long titles can reduce a reply to one entry. Put uses `TITLE|BODY` and still +obeys the command transport's total length limit; use the browser for a full +2 KB article. Read replies contain bounded Base64 and ID, version, offset, +total length and byte count. Continue at offset plus count. Deletion requires +a current nonzero version; reusing a deleted slot assigns a newer version. + +### LoRa board read protocol + +Logged-in readers, including read-only guests, can use encrypted +`PAYLOAD_TYPE_REQ` requests to the existing single room. The usual four-byte +tag precedes these payloads and is echoed before responses. Multi-byte +integers are little-endian. + +| Operation | Payload after tag | +| --- | --- | +| Index | `0a 00` + expected revision `u32` + entry cursor `u8` | +| Article | `0a 01` + article ID `u8` + expected version `u32` + offset `u16` | + +First index page: revision `0`, cursor `0`. Subsequent pages require the +returned revision. Response: subtype, operation, status, revision `u32`, +total count, start cursor, next cursor, then up to two complete entries. +Each entry is ID `u8`, version `u32`, body length `u16`, title length `u8` +and title bytes. Next cursor `255` means complete. + +Article response: subtype, operation, status, ID `u8`, version `u32`, +offset `u16`, total body length `u16`, chunk length `u8`, then raw bytes. +The route's encrypted reply capacity may reduce the chunk. Continue at offset +plus length; offset equal to total returns successful EOF. Never combine +different versions. + +Statuses: `0` success, `1` invalid, `2` storage unavailable, `3` write +failure, `4` not found, `5` stale version. A capacity too small for a full +header may return only subtype, operation, and invalid status. On stale +revision/version, reload the index and restart the affected transfer. + +## Regression coverage + +Production handlers and stores are tested across ESP32, nRF52, STM32, Pico +and generic filesystem APIs, including full storage, corruption, interrupted +publication, queue rejection, ACL reboot/eviction, quotas, clock correction +and allocation failure. ARM checks verify the new counters fit the existing +client union. + +The pinned official app JavaScript consumes real firmware room frames and +login responses. Chromium executes the bundled browser page with real backend +response fixtures, testing untrusted text, lost replies, draft preservation, +selective reads, resume and UTF-8 limits. These browser fixtures do not claim +a physical LoRa/Wi-Fi transfer. diff --git a/examples/companion_radio/MyMesh.cpp b/examples/companion_radio/MyMesh.cpp index 4f0f02522..4ec99ecab 100644 --- a/examples/companion_radio/MyMesh.cpp +++ b/examples/companion_radio/MyMesh.cpp @@ -1517,7 +1517,7 @@ void MyMesh::releaseHeldOneKeyDMs() { } #endif // MESH_ENABLE_ONE_KEY_DM -void MyMesh::queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packet *pkt, +bool MyMesh::queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packet *pkt, uint32_t sender_timestamp, const uint8_t *extra, int extra_len, const char *text, bool terminal_command_reply, @@ -1570,9 +1570,12 @@ void MyMesh::queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packe memset(private_key_backup_nonce, 0, sizeof(private_key_backup_nonce)); memset(private_key_backup_sender, 0, sizeof(private_key_backup_sender)); memset(out_frame, 0, i); - return; + return false; } const bool queued = addToOfflineQueue(out_frame, i); + // Signed room posts are retried by the server until retained. Do not make + // an unretained post visible as delivered or notify the user about it. + if (txt_type == TXT_TYPE_SIGNED_PLAIN && !queued) return false; if (_serial->isConnected()) { uint8_t frame[1]; @@ -1618,6 +1621,7 @@ void MyMesh::queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packe } #endif if (_ui) _ui->setMessageNotificationOverride(false); + return queued; } static uint16_t emergencyClientRepeatKey(const mesh::Packet* packet) { @@ -1845,12 +1849,13 @@ void MyMesh::onCLICommandRecv(const ContactInfo &from, mesh::Packet *pkt, uint32 } } -void MyMesh::onSignedMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, +bool MyMesh::onSignedMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) { + if (!queueMessage(from, TXT_TYPE_SIGNED_PLAIN, pkt, sender_timestamp, sender_prefix, 4, text)) return false; markConnectionActive(from); // from.sync_since change needs to be persisted scheduleContactWrite(from); - queueMessage(from, TXT_TYPE_SIGNED_PLAIN, pkt, sender_timestamp, sender_prefix, 4, text); + return true; } void MyMesh::onChannelMessageRecv(const mesh::GroupChannel &channel, mesh::Packet *pkt, uint32_t timestamp, diff --git a/examples/companion_radio/MyMesh.h b/examples/companion_radio/MyMesh.h index da497eb5d..329ad9807 100644 --- a/examples/companion_radio/MyMesh.h +++ b/examples/companion_radio/MyMesh.h @@ -395,7 +395,7 @@ protected: bool onAddressedTextPacket(mesh::Packet* packet, uint8_t src_hash, const uint8_t* mac_and_data, size_t len) override; #endif - void queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packet *pkt, uint32_t sender_timestamp, + bool queueMessage(const ContactInfo &from, uint8_t txt_type, mesh::Packet *pkt, uint32_t sender_timestamp, const uint8_t *extra, int extra_len, const char *text, bool terminal_command_reply=false, uint32_t terminal_command_elapsed_millis=0); @@ -406,7 +406,7 @@ protected: const char *text) override; void onCLICommandRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, const char *text, char* reply) override; - void onSignedMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, + bool onSignedMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) override; void onChannelMessageRecv(const mesh::GroupChannel &channel, mesh::Packet *pkt, uint32_t timestamp, const char *text) override; diff --git a/examples/simple_room_server/MyMesh.cpp b/examples/simple_room_server/MyMesh.cpp index 9d9c67f74..bf6c5e8b3 100644 --- a/examples/simple_room_server/MyMesh.cpp +++ b/examples/simple_room_server/MyMesh.cpp @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #if MESH_PACKET_LOGGING @@ -36,6 +37,9 @@ #include #ifdef WITH_WEBCONFIG #include +#include +#include +#include #endif #include @@ -97,25 +101,42 @@ struct ServerStats { uint16_t n_posted, n_post_push; }; -void MyMesh::addPost(ClientInfo *client, const char *postData) { - storePost(client->id, postData); +bool MyMesh::addPost(ClientInfo *client, const char *postData) { + return storePost(client->id, postData); } -void MyMesh::addSystemPost(const char *postData) { - if (!postData || postData[0] == 0) return; +bool MyMesh::addSystemPost(const char *postData) { + if (!postData || postData[0] == 0) return false; MESH_DEBUG_PRINTLN("room.post: addSystemPost: %s", postData); - storePost(self_id, postData); + return storePost(self_id, postData); } -void MyMesh::storePost(const mesh::Identity &author, const char *postData) { +bool MyMesh::storePost(const mesh::Identity &author, const char *postData) { + if (!postData || !postData[0] || strlen(postData) > MAX_POST_TEXT_LEN + || !room_history_available || last_room_timestamp == UINT32_MAX) return false; + uint32_t timestamp = getRTCClock()->getCurrentTimeUnique(); + // Keep catch-up cursors increasing after reboot and backward RTC corrections, + // without setting the radio's wall clock from archived messages. + if (timestamp <= last_room_timestamp) timestamp = last_room_timestamp + 1; + if (timestamp == 0) return false; + if (room_history_enabled) { + if ((room_history_state.needs_repair + || room_history_state.total_records >= mesh::ROOM_HISTORY_JOURNAL_RECORDS) + && !snapshotRoomHistory()) return false; + if (!mesh::appendRoomHistory(_fs, room_history_state, author.pub_key, + timestamp, postData)) return false; + } int idx = next_post_idx; // TODO: suggested postData format: /<descrption> posts[idx].author = author; // add to cyclic queue - StrHelper::strncpy(posts[idx].text, postData, MAX_POST_TEXT_LEN); + StrHelper::strncpy(posts[idx].text, postData, sizeof(posts[idx].text)); - posts[idx].post_timestamp = getRTCClock()->getCurrentTimeUnique(); + posts[idx].post_timestamp = timestamp; + last_room_timestamp = timestamp; + post_ready_at[idx] = futureMillis(POST_SYNC_DELAY_SECS * 1000UL); + post_ready_mask &= ~(UINT32_C(1) << idx); MESH_DEBUG_PRINTLN("room.post: storePost idx=%d text=%s", idx, posts[idx].text); MESH_DEBUG_PRINTLN("room.post: timestamp=%u", posts[idx].post_timestamp); next_post_idx = (next_post_idx + 1) % MAX_UNSYNCED_POSTS; @@ -123,6 +144,94 @@ void MyMesh::storePost(const mesh::Identity &author, const char *postData) { next_push = futureMillis(PUSH_NOTIFY_DELAY_MILLIS); _num_posted++; // stats MESH_DEBUG_PRINTLN("room.post: next_post_idx=%d num_posted=%d push scheduled", next_post_idx, _num_posted); + return true; +} + +bool MyMesh::snapshotRoomHistory() { + uint8_t count = 0; + for (const auto& post : posts) if (post.post_timestamp != 0) ++count; + return mesh::saveRoomHistorySnapshot(_fs, room_history_state, count, + [this](uint8_t ordinal, mesh::RoomHistoryRecord& record) { + for (int k = 0, idx = next_post_idx; k < MAX_UNSYNCED_POSTS; ++k, + idx = (idx + 1) % MAX_UNSYNCED_POSTS) { + if (posts[idx].post_timestamp == 0) continue; + if (ordinal-- != 0) continue; + memcpy(record.author, posts[idx].author.pub_key, PUB_KEY_SIZE); + record.timestamp = posts[idx].post_timestamp; + memcpy(record.text, posts[idx].text, sizeof(record.text)); + return true; + } + return false; + }); +} + +void MyMesh::loadRoomHistory() { + room_history_available = mesh::loadRoomHistoryConfig(_fs, room_history_enabled); + if (!room_history_available || !room_history_enabled) return; + room_history_available = mesh::loadRoomHistory(_fs, room_history_state, + [this](uint8_t ordinal, const mesh::RoomHistoryRecord& record) { + if (ordinal >= MAX_UNSYNCED_POSTS || record.timestamp <= last_room_timestamp) return false; + auto& post = posts[ordinal]; + memcpy(post.author.pub_key, record.author, PUB_KEY_SIZE); + post.post_timestamp = record.timestamp; + memcpy(post.text, record.text, sizeof(post.text)); + last_room_timestamp = record.timestamp; + next_post_idx = (ordinal + 1) % MAX_UNSYNCED_POSTS; + return true; + }); + if (!room_history_available) { + for (auto& post : posts) post.post_timestamp = 0; + next_post_idx = 0; + last_room_timestamp = 0; + MESH_DEBUG_PRINTLN("Room history unavailable; stored journal preserved"); + } +} + +bool MyMesh::handleRoomHistoryCommand(const char* command, char* reply) { + if (strcmp(command, "get room.history") == 0) { + snprintf(reply, 157, "> %s; storage %s; retained %u/32", + room_history_enabled ? "on" : "off", room_history_available ? "ready" : "unavailable", + unsigned(room_history_state.count)); + return true; + } + if (strcmp(command, "room.history.clear") == 0) { + if (!room_history_available || !mesh::clearRoomHistory(_fs, room_history_state)) { + strcpy(reply, "Err - history clear failed; previous history retained"); return true; + } + for (auto& post : posts) post.post_timestamp = 0; + next_post_idx = 0; + // Do not lower cursors already sent to companions in this boot. + recent_room_posts.clear(); + strcpy(reply, "OK - room history cleared"); + return true; + } + if (strncmp(command, "set room.history ", 17) != 0) return false; + const char* value = command + 17; + bool enabled = strcmp(value, "on") == 0 || strcmp(value, "1") == 0; + if (!enabled && strcmp(value, "off") != 0 && strcmp(value, "0") != 0) { + strcpy(reply, "Err - room.history requires on/off"); return true; + } + if (!room_history_available || (enabled && !snapshotRoomHistory()) + || !mesh::saveRoomHistoryConfig(_fs, enabled)) { + strcpy(reply, "Err - history save failed; previous setting retained"); return true; + } + room_history_enabled = enabled; + strcpy(reply, enabled ? "OK - persistent room history enabled (32 posts)" + : "OK - persistent room history disabled; archive retained"); + return true; +} + +void MyMesh::serviceRoomQuotas() { + room_access.serviceQuotaWindow(millis(), [this]() { + for (int i = 0; i < acl.getNumClients(); ++i) { + auto client = acl.getClientByIdx(i); + client->extra.room.post_quota_used = 0; + client->extra.room.poll_quota_used = 0; + } +#ifdef WITH_WEBCONFIG + for (auto& user : room_web_users) { user.posts = 0; user.polls = 0; } +#endif + }); } bool MyMesh::pushPostToClient(ClientInfo *client, PostInfo &post) { @@ -130,6 +239,219 @@ bool MyMesh::pushPostToClient(ClientInfo *client, PostInfo &post) { return pushRoomTextToClient(client, post.post_timestamp, post.author, post.text); } +#ifdef WITH_WEBCONFIG +bool MyMesh::handleRoomWebCommand(const char* command, char* reply) { + if (strcmp(command, "get room.web") == 0) { + snprintf(reply, 157, "> %s; browser path /room", room_web_enabled ? "on" : "off"); + return true; + } + if (strncmp(command, "set room.web ", 13) != 0) return false; + const char* value = command + 13; + const bool enabled = strcmp(value, "on") == 0 || strcmp(value, "1") == 0; + if (!enabled && strcmp(value, "off") != 0 && strcmp(value, "0") != 0) { + strcpy(reply, "Err - room.web requires on/off"); return true; + } + Preferences settings; + if (!settings.begin("mesh-room", false)) { + strcpy(reply, "Err - room browser setting unavailable"); return true; + } + const bool saved = settings.putBool("web", enabled) == sizeof(bool) + && settings.getBool("web", !enabled) == enabled; + settings.end(); + if (!saved) { strcpy(reply, "Err - room browser setting save failed"); return true; } + room_web_enabled = enabled; + strcpy(reply, enabled ? "OK - room browser enabled; start webconfig, then open /room" + : "OK - room browser disabled"); + return true; +} + +static const char* roomWebString(JsonVariantConst value, size_t maximum) { + if (!value.is<const char*>()) return nullptr; + const JsonString text = value.as<JsonString>(); + if (text.size() > maximum || strlen(text.c_str()) != text.size()) return nullptr; + return text.c_str(); +} + +void MyMesh::processRoomRequest(const uint8_t* token, uint32_t sequence, char* request, + char* response, size_t capacity) { + room_web_request_authenticated = false; + auto error = [response, capacity](const char* message) { + mesh::RoomJsonWriter json(response, capacity); + json.raw("{\"error\":"); json.string(message); json.raw("}"); json.finish(); + }; + if (!room_web_enabled) { error("room browser disabled; use set room.web on"); return; } + if (!token || !sequence || !request || strlen(request) > 4096) { error("invalid request"); return; } + uint8_t nonzero = 0; + for (size_t i = 0; i < 32; ++i) nonzero |= token[i]; + if (!nonzero) { error("invalid browser identity"); return; } + mesh::RoomJsonAllocator allocator; + JsonDocument input(&allocator); + if (deserializeJson(input, request, DeserializationOption::NestingLimit(4)) + != DeserializationError::Ok || !input.is<JsonObject>()) { error("invalid JSON"); return; } + const char* operation = roomWebString(input["op"], 20); + const char* password = roomWebString(input["password"], 64); + if (!operation || !password) { error("missing operation or password"); return; } + + // A browser proves possession of its own random web token. It cannot claim + // a radio public key or inherit that key's ACL permission without its secret. + mesh::Identity author; + mesh::Utils::sha256(author.pub_key, PUB_KEY_SIZE, token, 32, self_id.pub_key, PUB_KEY_SIZE); + if (!room_access.allowsIdentity(author.pub_key)) { error("room access denied"); return; } + const auto authorization = mesh::authorizeRoomLogin(password, _prefs.password, + _prefs.guest_password, false, 0, _prefs.allow_read_only, PERM_ACL_ROLE_MASK, + PERM_ACL_GUEST, PERM_ACL_READ_WRITE, PERM_ACL_ADMIN); + if (!authorization.accepted) { error("incorrect room password"); return; } + room_web_request_authenticated = true; + const uint8_t role = authorization.role & PERM_ACL_ROLE_MASK; + const bool writing = strcmp(operation, "post") == 0 + || strcmp(operation, "board.save") == 0 || strcmp(operation, "board.delete") == 0; + if (writing && (!input["boot"].is<uint32_t>() + || input["boot"].as<uint32_t>() != room_web_boot_id)) { + error("radio restarted; refresh before sending"); return; + } + if (strcmp(operation, "post") == 0 && role != PERM_ACL_READ_WRITE && role != PERM_ACL_ADMIN) { + error("read-only room access"); return; + } + if ((strcmp(operation, "board.save") == 0 || strcmp(operation, "board.delete") == 0) + && role != PERM_ACL_ADMIN) { error("admin permission required"); return; } + if (strcmp(operation, "status") != 0 && strcmp(operation, "posts") != 0 + && strcmp(operation, "post") != 0 && strcmp(operation, "board.index") != 0 + && strcmp(operation, "board.read") != 0 && strcmp(operation, "board.save") != 0 + && strcmp(operation, "board.delete") != 0) { error("unknown room operation"); return; } + + const uint32_t now = millis(); + RoomWebUser* user = nullptr; + RoomWebUser* available = nullptr; + for (auto& entry : room_web_users) { + if (entry.id.matches(author) && entry.last_sequence != 0) user = &entry; + if (entry.last_sequence == 0 || uint32_t(now - entry.last_seen) >= 1200000UL) available = &entry; + } + if (!user) { + if (!available) { error("room browser busy; try later"); return; } + user = available; *user = RoomWebUser{}; user->id = author; + } else if (sequence <= user->last_sequence) { error("request already completed; refresh to check its result"); return; } + // Preserve the floor even if WebConfig is stopped and restarted. Failed + // requests also have a result; a retry must reuse the mailbox's cached result. + user->last_sequence = sequence; user->last_seen = now; + serviceRoomQuotas(); + const uint16_t prior_posts = user->posts; + if (!(strcmp(operation, "post") == 0 ? room_access.consumePost(user->posts) + : room_access.consumeKeepAlive(user->polls))) { + error("room rate limit reached; try after the next minute"); return; + } + char key[65]; mesh::Utils::toHex(key, author.pub_key, PUB_KEY_SIZE); + mesh::RoomJsonWriter json(response, capacity); + if (strcmp(operation, "status") == 0) { + json.raw("{\"name\":"); json.string(_prefs.node_name); + json.raw(",\"topic\":"); json.string(room_topic); + json.raw(",\"boot\":"); json.number(room_web_boot_id); + json.raw(",\"role\":"); json.number(role); + json.raw(",\"web_key\":"); json.string(key); + json.raw(",\"persistent_history\":"); json.raw(room_history_enabled ? "true" : "false"); + json.raw("}"); json.finish(); return; + } + if (strcmp(operation, "post") == 0) { + const char* name = roomWebString(input["name"], 24); + const char* text = roomWebString(input["text"], MAX_POST_TEXT_LEN); + if (!name || !*name || !text || !*text) { user->posts = prior_posts; error("name and message required"); return; } + for (const unsigned char* p = (const unsigned char*)name; *p; ++p) { + if (*p < 0x20 || *p == 0x7f || *p == ':') { user->posts = prior_posts; error("invalid display name"); return; } + } + char post[MAX_POST_TEXT_LEN + 1]; + const int length = snprintf(post, sizeof(post), "%s: %s", name, text); + if (length < 0 || size_t(length) >= sizeof(post)) { user->posts = prior_posts; error("message with name exceeds 151 UTF-8 bytes"); return; } + if (!storePost(author, post)) { user->posts = prior_posts; error("post not retained; storage unavailable"); return; } + json.raw("{\"ok\":true,\"timestamp\":"); json.number(last_room_timestamp); + json.raw("}"); json.finish(); return; + } + if (strcmp(operation, "posts") == 0) { + if (!input["after"].is<uint32_t>()) { error("invalid message cursor"); return; } + const uint32_t after = input["after"].as<uint32_t>(); + const PostInfo* selected = nullptr; + for (int k = 0, idx = next_post_idx; k < MAX_UNSYNCED_POSTS; ++k, idx = (idx + 1) % MAX_UNSYNCED_POSTS) { + if (posts[idx].post_timestamp > after) { selected = &posts[idx]; break; } + } + json.raw("{\"post\":"); + if (!selected) json.raw("null"); + else { + mesh::Utils::toHex(key, selected->author.pub_key, PUB_KEY_SIZE); + json.raw("{\"timestamp\":"); json.number(selected->post_timestamp); + json.raw(",\"author\":"); json.string(key); + json.raw(",\"text\":"); json.string(selected->text); json.raw("}"); + } + json.raw("}"); json.finish(); return; + } + + mesh::RoomBoardIndex board; + if (!mesh::loadRoomBoard(_fs, board)) { error("information board storage unavailable"); return; } + if (strcmp(operation, "board.index") == 0) { + if (!input["revision"].is<uint32_t>() || !input["cursor"].is<uint8_t>()) { error("invalid board cursor"); return; } + const uint32_t revision = input["revision"].as<uint32_t>(); + const uint8_t cursor = input["cursor"].as<uint8_t>(); + if (cursor > board.count || (revision != board.revision && (revision != 0 || cursor != 0))) { + error("board changed; reload index"); return; + } + json.raw("{\"revision\":"); json.number(board.revision); + json.raw(",\"articles\":["); + uint8_t next = cursor; + for (; next < board.count && next < cursor + 2; ++next) { + if (next != cursor) json.raw(","); + const auto& article = board.articles[next]; + json.raw("{\"id\":"); json.number(article.id); + json.raw(",\"version\":"); json.number(article.version); + json.raw(",\"length\":"); json.number(article.body_length); + json.raw(",\"title\":"); json.string(article.title); json.raw("}"); + } + json.raw("],\"next\":"); json.number(next < board.count ? next : 255); + json.raw("}"); json.finish(); return; + } + if (!input["id"].is<uint8_t>() || !input["version"].is<uint32_t>()) { error("invalid article identity/version"); return; } + const uint8_t id = input["id"].as<uint8_t>(); + const uint32_t version = input["version"].as<uint32_t>(); + const mesh::RoomBoardArticle* article = nullptr; + for (uint8_t i = 0; i < board.count; ++i) if (board.articles[i].id == id) article = &board.articles[i]; + if (strcmp(operation, "board.read") == 0) { + if (!input["offset"].is<uint16_t>()) { error("invalid article offset"); return; } + const uint16_t offset = input["offset"].as<uint16_t>(); + uint8_t chunk[mesh::ROOM_BOARD_READ_MAX_CHUNK]; + size_t length = 0; + const auto result = mesh::readRoomBoardArticle(_fs, id, version, offset, chunk, sizeof(chunk), length); + if (result != mesh::RoomBoardResult::Success || !article) { error("article changed or unavailable; reload index"); return; } + char encoded[4 * ((sizeof(chunk) + 2) / 3) + 1]; mesh::roomEncodeBase64(chunk, length, encoded); + json.raw("{\"version\":"); json.number(version); + json.raw(",\"offset\":"); json.number(offset); + json.raw(",\"next\":"); json.number(uint32_t(offset) + length); + json.raw(",\"total\":"); json.number(article->body_length); + json.raw(",\"data64\":"); json.string(encoded); json.raw("}"); json.finish(); return; + } + mesh::RoomBoardResult result; + if (strcmp(operation, "board.delete") == 0) { + if (version == 0) { error("deletion requires the current article version"); return; } + result = mesh::deleteRoomBoardArticle(_fs, id, version); + } else { + const char* title = roomWebString(input["title"], 63); + const char* encoded = roomWebString(input["body64"], 2732); + if (!title || !encoded || (article ? article->version != version : version != 0)) { + error("article changed; reload index before editing"); return; + } + auto* body = new (std::nothrow) uint8_t[mesh::ROOM_BOARD_MAX_BODY_LENGTH]; + if (!body) { error("insufficient memory; article retained"); return; } + size_t length = 0; + if (!mesh::roomDecodeBase64(encoded, body, mesh::ROOM_BOARD_MAX_BODY_LENGTH, length)) { + delete[] body; error("invalid article encoding"); return; + } + result = mesh::saveRoomBoardArticle(_fs, id, title, length, + [body](size_t offset, uint8_t* out, size_t count) { + memcpy(out, body + offset, count); return count; + }, version); + delete[] body; + } + if (result != mesh::RoomBoardResult::Success) { error("article save failed or version changed; previous article retained"); return; } + json.raw("{\"ok\":true,\"version\":"); json.number(board.revision + 1); + json.raw("}"); json.finish(); +} +#endif + bool MyMesh::pushRoomTextToClient(ClientInfo *client, uint32_t timestamp, const mesh::Identity& author, const char* text, uint32_t topic_revision) { @@ -209,6 +531,7 @@ uint8_t MyMesh::getUnsyncedCount(ClientInfo *client) { bool MyMesh::processAck(const uint8_t *data) { for (int i = 0; i < acl.getNumClients(); i++) { auto client = acl.getClientByIdx(i); + if (!room_access.allowsIdentity(client->id.pub_key)) continue; if (client->extra.room.pending_ack && memcmp(data, &client->extra.room.pending_ack, 4) == 0) { // got an ACK from Client! client->extra.room.pending_ack = 0; // clear this, so next push can happen if (client->extra.room.pending_topic_revision != 0) { @@ -268,6 +591,10 @@ bool MyMesh::handleRoomTopicCommand(const char* command, char* reply) { } void MyMesh::serviceRoomPush() { + for (int idx = 0; idx < MAX_UNSYNCED_POSTS; ++idx) { + if (!(post_ready_mask & (UINT32_C(1) << idx)) + && millisHasNowPassed(post_ready_at[idx])) post_ready_mask |= UINT32_C(1) << idx; + } // Latch this even without subscribers. An unchanged topic remains ready // beyond the signed timer comparison's half-range (~25 days). if (!room_topic_ready && millisHasNowPassed(room_topic_ready_at)) room_topic_ready = true; @@ -298,7 +625,8 @@ void MyMesh::serviceRoomPush() { auto client = acl.getClientByIdx(next_client_idx); bool did_push = false; if (client->extra.room.pending_ack == 0 && client->last_activity != 0 - && client->extra.room.push_failures < 3) { + && client->extra.room.push_failures < 3 + && room_access.allowsIdentity(client->id.pub_key)) { const uint32_t now = getRTCClock()->getCurrentTime(); if (room_topic[0] != 0 && room_topic_ready && client->extra.room.topic_seen_revision != room_topic_revision @@ -316,8 +644,7 @@ void MyMesh::serviceRoomPush() { // Companions echo the latest received timestamp as keep-alive forceSince. // Even an unaged older post must precede the topic on the wire. if (post->post_timestamp <= room_topic_timestamp) has_older_topic_posts = true; - if (next_post == NULL && now >= post->post_timestamp - && now - post->post_timestamp >= POST_SYNC_DELAY_SECS) next_post = post; + if (next_post == NULL && (post_ready_mask & (UINT32_C(1) << idx))) next_post = post; } idx = (idx + 1) % MAX_UNSYNCED_POSTS; } @@ -360,6 +687,11 @@ int MyMesh::handleRequest(ClientInfo *sender, uint32_t sender_timestamp, uint8_t // uint32_t now = getRTCClock()->getCurrentTimeUnique(); // memcpy(reply_data, &now, 4); // response packets always prefixed with timestamp memcpy(reply_data, &sender_timestamp, 4); // reflect sender_timestamp back in response packet (kind of like a 'tag') + if (payload[0] == mesh::ROOM_BOARD_REQUEST_SUBTYPE) { + const size_t length = mesh::handleRoomBoardRequest(_fs, payload, payload_len, + reply_data + 4, reply_capacity - 4); + return length ? 4 + length : 0; + } if (payload[0] == REQ_TYPE_GET_STATUS) { ServerStats stats; @@ -417,7 +749,7 @@ int MyMesh::handleRequest(ClientInfo *sender, uint32_t sender_timestamp, uint8_t // Preserve the legacy seven-byte entries within the encrypted route budget. for (int i = 0; i < acl.getNumClients() && ofs + 7 <= reply_capacity; i++) { auto c = acl.getClientByIdx(i); - if (!c->isAdmin()) continue; // skip non-Admin entries + if (!saveFilter(c)) continue; // operator assignments plus password-created Admins memcpy(&reply_data[ofs], c->id.pub_key, 6); ofs += 6; // just 6-byte pub_key prefix reply_data[ofs++] = c->permissions; } @@ -831,6 +1163,7 @@ mesh::DispatcherAction MyMesh::onRecvPacket(mesh::Packet* pkt) { void MyMesh::onAnonDataRecv(mesh::Packet *packet, const uint8_t *secret, const mesh::Identity &sender, uint8_t *data, size_t len) { + if (!room_access.allowsIdentity(sender.pub_key)) return; if (packet->getPayloadType() == PAYLOAD_TYPE_ANON_REQ) { // received an initial request by a possible admin // client (unknown at this stage) if (len < 8) { @@ -996,6 +1329,7 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, return; } auto client = acl.getClientByIdx(i); + if (!room_access.allowsIdentity(client->id.pub_key)) return; #if defined(WITH_MQTT_NEIGHBORS) // A neighbour that IS an ACL client resolves to a normal index above, so a // scope-query response from it lands here -- match it against the overlay. @@ -1037,18 +1371,17 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, mesh::Utils::sha256(message_fingerprint, sizeof(message_fingerprint), client->id.pub_key, PUB_KEY_SIZE, (const uint8_t*)text, text_len); - const auto replay_decision = recent_room_posts.classifyAndRemember( - message_fingerprint, sender_timestamp, - client->extra.room.last_post_timestamp); - - using ReplayDecision = - mesh::LogicalMessageCache<ROOM_MESSAGE_CACHE_SIZE>::ReplayDecision; - if (replay_decision == ReplayDecision::StaleOrMismatched) { - MESH_DEBUG_PRINTLN("onPeerDataRecv: stale or mismatched room post detected"); - return; - } - if (replay_decision == ReplayDecision::NewMessage) { - addPost(client, text); + if (!recent_room_posts.find(message_fingerprint, sender_timestamp)) { + if (sender_timestamp <= client->extra.room.last_post_timestamp) return; + serviceRoomQuotas(); + const uint16_t prior_quota = client->extra.room.post_quota_used; + if (!room_access.consumePost(client->extra.room.post_quota_used)) return; + if (!addPost(client, text)) { + client->extra.room.post_quota_used = prior_quota; + return; + } + client->extra.room.last_post_timestamp = sender_timestamp; + recent_room_posts.remember(message_fingerprint, sender_timestamp); } // Exact retries are ACKed again regardless of whether newer posts have // advanced this client's room-post replay timestamp. @@ -1161,10 +1494,45 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, } else if (type == PAYLOAD_TYPE_REQ && len >= 5) { uint32_t sender_timestamp; memcpy(&sender_timestamp, data, 4); // timestamp (by sender's RTC clock - which could be wrong) - if (sender_timestamp < client->last_timestamp) { // prevent replay attacks + const bool keep_alive = data[4] == REQ_TYPE_KEEP_ALIVE; + uint32_t since = 0; + uint8_t canonical_poll[5]; + const uint8_t* request_bytes = data + 4; + size_t request_length = len - 4; + if (keep_alive) { + if (!packet->isRouteDirect()) return; + if (len >= 9) memcpy(&since, data + 5, 4); + canonical_poll[0] = data[4]; memcpy(canonical_poll + 1, &since, 4); + request_bytes = canonical_poll; request_length = sizeof(canonical_poll); + } + uint8_t fingerprint[MAX_HASH_SIZE]; + mesh::Utils::sha256(fingerprint, sizeof(fingerprint), client->id.pub_key, PUB_KEY_SIZE, + request_bytes, request_length); + const bool request_retry = recent_room_polls.find(fingerprint, sender_timestamp); + if (sender_timestamp < client->last_timestamp && !request_retry) return; + serviceRoomQuotas(); + if (!request_retry && !room_access.consumeKeepAlive(client->extra.room.poll_quota_used)) return; + recent_room_polls.remember(fingerprint, sender_timestamp); + // All accepted keep-alive retries get an ACK without rewinding catch-up or + // cancelling a newer pending delivery, even if no new request advanced RTC. + if (keep_alive && request_retry) { + uint32_t ack_hash; + uint8_t ack_data[9]; + memcpy(ack_data, data, 5); memcpy(ack_data + 5, &since, 4); + mesh::Utils::sha256((uint8_t*)&ack_hash, 4, ack_data, 9, client->id.pub_key, PUB_KEY_SIZE); + if (mesh::Packet::isValidPathLen(client->out_path_len)) { + auto reply = createAck(ack_hash); + if (reply) { + reply->payload[reply->payload_len++] = getUnsyncedCount(client); + sendDirect(reply, client->out_path, client->out_path_len, SERVER_RESPONSE_DELAY); + } + } + return; + } + if (sender_timestamp < client->last_timestamp && !request_retry) { // prevent replay attacks MESH_DEBUG_PRINTLN("onPeerDataRecv: possible replay attack detected"); } else { - client->last_timestamp = sender_timestamp; + if (sender_timestamp > client->last_timestamp) client->last_timestamp = sender_timestamp; uint32_t now = getRTCClock()->getCurrentTime(); client->last_activity = now; // <-- THIS will keep client connection alive @@ -1184,9 +1552,6 @@ void MyMesh::onPeerDataRecv(mesh::Packet *packet, uint8_t type, int sender_idx, client->extra.room.pending_ack = 0; client->extra.room.pending_topic_revision = 0; - // TODO: Throttle KEEP_ALIVE requests! - // if client sends too quickly, evict() - // RULE: only send keep_alive response DIRECT! if (mesh::Packet::isValidPathLen(client->out_path_len)) { uint32_t ack_hash; // calc ACK to prove to sender that we got request @@ -1232,6 +1597,7 @@ bool MyMesh::onPeerPathRecv(mesh::Packet *packet, int sender_idx, const uint8_t if (i >= 0 && i < acl.getNumClients()) { // get from our known_clients table (sender SHOULD already be known in this context) MESH_DEBUG_PRINTLN("PATH to client, path_len=%d", (uint32_t)path_len); auto client = acl.getClientByIdx(i); + if (!room_access.allowsIdentity(client->id.pub_key)) return false; // PATH has no durable replay freshness signal. Keep it RAM-only and do not // let a replay replace an operator-selected force-flood route. const bool persistence_allowed = mesh::clientPathPersistenceAllowed( @@ -1529,6 +1895,18 @@ void MyMesh::begin(FILESYSTEM *fs) { _cli.beginManagement(*this, _fs); acl.load(_fs, self_id); + acl.protectExplicitPermissions(); + room_access.load(_fs); + loadRoomHistory(); +#ifdef WITH_WEBCONFIG + Preferences room_settings; + if (room_settings.begin("mesh-room", true)) { + room_web_enabled = room_settings.getBool("web", false); + room_settings.end(); + } + getRNG()->random((uint8_t*)&room_web_boot_id, sizeof(room_web_boot_id)); + if (room_web_boot_id == 0) room_web_boot_id = 1; +#endif region_map.load(_fs); _clock_sync.begin(_fs); if (mesh::loadRoomTopic(_fs, room_topic)) { @@ -2689,6 +3067,12 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply mesh::cli::normalizeCommandVerb(command); if (handleRoomTopicCommand(command, reply)) return; + if (handleRoomHistoryCommand(command, reply)) return; + if (room_access.handleConfig(_fs, command, reply, 157)) return; + if (mesh::handleRoomBoardCommand(_fs, command, reply, 157)) return; +#ifdef WITH_WEBCONFIG + if (handleRoomWebCommand(command, reply)) return; +#endif #if MESH_ENABLE_TELEMETRY_HISTORY if (handleTelemetryHistoryCommand(command, reply)) return; @@ -2791,8 +3175,7 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply if (*msg == 0) { snprintf(reply, MAX_POST_TEXT_LEN, "ERR empty message"); } else { - addSystemPost(msg); - snprintf(reply, MAX_POST_TEXT_LEN, "OK"); + snprintf(reply, MAX_POST_TEXT_LEN, addSystemPost(msg) ? "OK" : "Err - post not retained"); } } #if MESH_ENABLE_ROOM_FLOOD_RULE_ENGINE @@ -2815,7 +3198,8 @@ void MyMesh::handleCommand(uint32_t sender_timestamp, char *command, char *reply } bool MyMesh::saveFilter(ClientInfo* client) { - return client->isAdmin(); // only save Admins + return mesh::roomAclShouldPersist(client->permissions, client->permissions_are_explicit, + PERM_ACL_ROLE_MASK, PERM_ACL_ADMIN); } void MyMesh::loop() { diff --git a/examples/simple_room_server/MyMesh.h b/examples/simple_room_server/MyMesh.h index c177f4d37..d942d4bde 100644 --- a/examples/simple_room_server/MyMesh.h +++ b/examples/simple_room_server/MyMesh.h @@ -30,6 +30,9 @@ #include <helpers/StatsFormatHelper.h> #include <helpers/LocalCliOutput.h> #include <helpers/ClientACL.h> +#include <helpers/RoomAccessPolicy.h> +#include <helpers/RoomHistoryStore.h> +#include <helpers/RoomBoardStore.h> #include <helpers/LogicalMessageCache.h> #include <helpers/RemoteCliReplyCache.h> #include <helpers/RemoteCliRequest.h> @@ -211,6 +214,8 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks, #endif ClientACL acl; mesh::LogicalMessageCache<ROOM_MESSAGE_CACHE_SIZE> recent_room_posts; + mesh::LogicalMessageCache<16> recent_room_polls; + mesh::RoomAccessPolicy room_access; mesh::RemoteCliReplyCache remote_cli_reply_cache; CommonCLI _cli; mesh::MeshClockSync _clock_sync; @@ -225,6 +230,13 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks, int next_client_idx; // for round-robin polling int next_post_idx; PostInfo posts[MAX_UNSYNCED_POSTS]; // cyclic queue + static_assert(MAX_UNSYNCED_POSTS <= 32, "Room readiness bitmap holds 32 posts"); + uint32_t post_ready_at[MAX_UNSYNCED_POSTS] = {}; + uint32_t post_ready_mask = UINT32_MAX; + uint32_t last_room_timestamp = 0; + mesh::RoomHistoryState room_history_state; + bool room_history_enabled = false; + bool room_history_available = true; char room_topic[MAX_POST_TEXT_LEN + 1] = {}; uint32_t room_topic_revision = 0; uint32_t room_topic_timestamp = 0; @@ -325,6 +337,15 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks, #endif #ifdef WITH_WEBCONFIG WebConfigServer* _webconfig = nullptr; + bool room_web_enabled = false; + bool room_web_request_authenticated = false; + uint32_t room_web_boot_id = 0; + struct RoomWebUser { + mesh::Identity id; + uint32_t last_seen = 0, last_sequence = 0; + uint16_t posts = 0, polls = 0; + } room_web_users[8] = {}; + bool handleRoomWebCommand(const char* command, char* reply); bool _unconfigured_setup_espnow_suspended = false; void suspendUnconfiguredSetupBridges(); bool startWebConfigImpl(bool force_ap, char* reply, bool automatic_setup); @@ -336,9 +357,13 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks, void serviceIdleWiFi(); #endif - void addPost(ClientInfo* client, const char* postData); + bool addPost(ClientInfo* client, const char* postData); bool applySavedRadioParams(); - void storePost(const mesh::Identity& author, const char* postData); + bool storePost(const mesh::Identity& author, const char* postData); + bool snapshotRoomHistory(); + void loadRoomHistory(); + bool handleRoomHistoryCommand(const char* command, char* reply); + void serviceRoomQuotas(); bool pushPostToClient(ClientInfo* client, PostInfo& post); bool pushRoomTextToClient(ClientInfo* client, uint32_t timestamp, const mesh::Identity& author, const char* text, @@ -455,7 +480,7 @@ public: MyMesh(mesh::MainBoard& board, mesh::Radio& radio, mesh::MillisecondClock& ms, mesh::RNG& rng, mesh::RTCClock& rtc, mesh::MeshTables& tables); void begin(FILESYSTEM* fs); - void addSystemPost(const char* postData); + bool addSystemPost(const char* postData); const char* getFirmwareVer() override { return FIRMWARE_VERSION; } const char* getBuildDate() override { return FIRMWARE_BUILD_DATE; } @@ -950,6 +975,10 @@ public: } void getNodeSnapshot(WebConfigServer::NodeSnapshot& snapshot) override; void execCommand(char* cmd, char* reply) override { handleCommand(0, cmd, reply); } + bool supportsRoomService() const override { return true; } + bool roomRequestAuthenticated() const override { return room_web_request_authenticated; } + void processRoomRequest(const uint8_t* token, uint32_t sequence, char* request, + char* response, size_t capacity) override; bool supportsCliTerminal() const override { return true; } void execAdminCommand(char* cmd, char* reply) override { if (strcmp(cmd, "get acl") == 0 || strcmp(cmd, "log") == 0 diff --git a/examples/simple_secure_chat/main.cpp b/examples/simple_secure_chat/main.cpp index e5937e429..781361938 100644 --- a/examples/simple_secure_chat/main.cpp +++ b/examples/simple_secure_chat/main.cpp @@ -271,7 +271,8 @@ protected: void onCLICommandRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const char *text, char* reply) override { } - void onSignedMessageRecv(const ContactInfo& from, mesh::Packet* pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) override { + bool onSignedMessageRecv(const ContactInfo& from, mesh::Packet* pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) override { + return false; // This minimal example does not retain room posts. } void onChannelMessageRecv(const mesh::GroupChannel& channel, mesh::Packet* pkt, uint32_t timestamp, const char *text) override { diff --git a/mkdocs.yml b/mkdocs.yml index d9d843355..1ed5d851e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -75,6 +75,7 @@ nav: - Radio, networking, and power: - radio_profiles.md - fleet_management.md + - room_services.md - carrier_wave.md - espnow_bridge_setup.md - WiFi.md diff --git a/scripts/check_firmware_ram.py b/scripts/check_firmware_ram.py index 3eaad6f2b..bc4586a05 100644 --- a/scripts/check_firmware_ram.py +++ b/scripts/check_firmware_ram.py @@ -112,6 +112,12 @@ def requirements(platform, defines, target): # Larger replies grow on demand only while preserving 32 KiB # of free internal heap, and shrink after the browser reads. parts["browser_terminal_scrollback"] = 4096 + if "room_server" in target.lower(): + # The mailbox is exclusive to Room Server callbacks. JSON's + # real allocator cap and one decoded maximum article coexist; + # ordinary HTTP bodies/responses use the transient margin. + parts["room_browser_mailbox"] = 6656 + 16 + parts["room_browser_request_workspace"] = 8192 + 2048 + 16 if repeater: # Match MyMesh's platform defaults. The post-hook supplies the # MCU-specific classic ESP32 default before sdkconfig is included. @@ -232,6 +238,8 @@ def requirements(platform, defines, target): "ota_context", "screen_objects_and_history", "ota_encoder_workspace", "ota_generic_source_buffers", "ethernet_or_external_ota_workspaces"): largest = max(largest, parts.get(name, 0)) + if "room_browser_mailbox" in parts: + largest = max(largest, 8192, parts["room_browser_mailbox"]) return {"required_heap_bytes": required, "required_contiguous_bytes": largest, "components": parts, "display": display, "full_companion": full} diff --git a/src/helpers/BaseChatMesh.cpp b/src/helpers/BaseChatMesh.cpp index 0f9d42489..571ffca22 100644 --- a/src/helpers/BaseChatMesh.cpp +++ b/src/helpers/BaseChatMesh.cpp @@ -428,13 +428,26 @@ void BaseChatMesh::onPeerDataRecv(mesh::Packet* packet, uint8_t type, int sender } } } else if (flags == TXT_TYPE_SIGNED_PLAIN) { - if (canMutateContacts()) { + if (len < 9) return; // timestamp, flags and complete author prefix + const uint32_t previous_sync_since = from.sync_since; + const uint32_t previous_lastmod = from.lastmod; + const bool mutate_contact = canMutateContacts(); + if (mutate_contact) { if (sender_timestamp > from.sync_since) { // make sure 'sync_since' is up-to-date from.sync_since = sender_timestamp; } from.lastmod = getRTCClock()->getCurrentTime(); // update last heard time } - onSignedMessageRecv(from, packet, sender_timestamp, &data[5], (const char *) &data[9]); // let UI know + // The accepted callback can persist the updated live contact. Rejected + // admission must not persist it or report delivery; restore its cursor + // so a subsequent keep-alive still requests this post. + if (!onSignedMessageRecv(from, packet, sender_timestamp, &data[5], (const char *) &data[9])) { + if (mutate_contact) { + from.sync_since = previous_sync_since; + from.lastmod = previous_lastmod; + } + return; + } uint32_t ack_hash; // calc truncated hash of the message timestamp + text + OUR pub_key, to prove to sender that we got it mesh::Utils::sha256((uint8_t *) &ack_hash, 4, data, 9 + strlen((char *)&data[9]), self_id.pub_key, PUB_KEY_SIZE); diff --git a/src/helpers/BaseChatMesh.h b/src/helpers/BaseChatMesh.h index 64cc0c16f..7cb8f5645 100644 --- a/src/helpers/BaseChatMesh.h +++ b/src/helpers/BaseChatMesh.h @@ -171,7 +171,9 @@ protected: virtual void onMessageRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const char *text) = 0; virtual void onCommandDataRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const char *text) = 0; virtual void onCLICommandRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const char *text, char* reply) = 0; - virtual void onSignedMessageRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) = 0; + // Return true only after retaining/delivering the room post. A false result + // leaves the catch-up cursor unchanged and suppresses the delivery ACK. + virtual bool onSignedMessageRecv(const ContactInfo& contact, mesh::Packet* pkt, uint32_t sender_timestamp, const uint8_t *sender_prefix, const char *text) = 0; virtual uint32_t calcFloodTimeoutMillisFor(uint32_t pkt_airtime_millis) const = 0; virtual uint32_t calcDirectTimeoutMillisFor(uint32_t pkt_airtime_millis, uint8_t path_len) const = 0; virtual void onSendTimeout() = 0; diff --git a/src/helpers/ClientACL.cpp b/src/helpers/ClientACL.cpp index 4bdecf37e..6b42f3b2e 100644 --- a/src/helpers/ClientACL.cpp +++ b/src/helpers/ClientACL.cpp @@ -715,6 +715,7 @@ void ClientACL::load(FILESYSTEM* fs, const mesh::LocalIdentity& self_id) { c.last_timestamp = UINT32_MAX; } self_id.calcSharedSecret(c.shared_secret, pub_key); // recalculate shared secrets in case our private key changed + c.permissions_are_explicit = true; // this record was deliberately retained by the role's save policy clients[num_clients++] = c; } file.close(); @@ -908,7 +909,9 @@ ClientInfo* ClientACL::putClient(const mesh::Identity& id, uint8_t init_perms) { ClientInfo* oldest = NULL; for (int i = 0; i < num_clients; i++) { if (id.matches(clients[i].id)) return &clients[i]; // already known - if (!clients[i].isProtectedManager() && clients[i].last_activity < min_time) { + if (!clients[i].isProtectedManager() + && !(protect_explicit_permissions && clients[i].permissions_are_explicit) + && clients[i].last_activity < min_time) { oldest = &clients[i]; min_time = oldest->last_activity; } @@ -953,6 +956,7 @@ bool ClientACL::applyPermissions(const mesh::LocalIdentity& self_id, const uint8 if (c == NULL) return false; c->permissions = perms; // update their permissions + c->permissions_are_explicit = true; self_id.calcSharedSecret(c->shared_secret, pubkey); } return true; diff --git a/src/helpers/ClientACL.h b/src/helpers/ClientACL.h index 85a1b7e74..dc64e71bd 100644 --- a/src/helpers/ClientACL.h +++ b/src/helpers/ClientACL.h @@ -32,6 +32,7 @@ struct ClientInfo { uint8_t out_path_len; uint8_t out_path[MAX_PATH_SIZE]; bool out_path_is_persistable; // live route may replace, but not erase, durable route + bool permissions_are_explicit; // operator assignment; room login caches remain transient uint8_t alt_path_len; uint8_t alt_path[MAX_PATH_SIZE]; uint8_t observed_path_len; // reciprocal login PATH (transient) @@ -55,6 +56,8 @@ struct ClientInfo { uint32_t topic_seen_revision; uint32_t pending_topic_revision; uint8_t topic_failures; + uint16_t post_quota_used; + uint16_t poll_quota_used; #endif } room; #if !MESH_CLIENT_REPEATER_ONLY @@ -96,6 +99,7 @@ class ClientACL { int num_clients; bool login_replay_store_available; bool acl_load_complete; + bool protect_explicit_permissions = false; public: // MAX_CLIENTS entries run to several kilobytes. Classic ESP32's link-time @@ -117,6 +121,7 @@ public: ClientACL(const ClientACL&) = delete; ClientACL& operator=(const ClientACL&) = delete; void load(FILESYSTEM* _fs, const mesh::LocalIdentity& self_id); + void protectExplicitPermissions(bool protect = true) { protect_explicit_permissions = protect; } bool save(FILESYSTEM* _fs, bool (*filter)(ClientInfo*)=NULL); bool clear(); diff --git a/src/helpers/RoomAccessPolicy.h b/src/helpers/RoomAccessPolicy.h new file mode 100644 index 000000000..3dae58e6c --- /dev/null +++ b/src/helpers/RoomAccessPolicy.h @@ -0,0 +1,226 @@ +#pragma once + +#include <stdio.h> +#include "RoomAccessPolicyStore.h" + +namespace mesh { + +inline bool roomAclShouldPersist(uint8_t permissions, bool explicitly_assigned, + uint8_t role_mask, uint8_t admin_role) { + return explicitly_assigned || (permissions & role_mask) == (admin_role & role_mask); +} + +// The two counts live beside their actual ClientInfo identity, not in a second +// table keyed by a short hash. The role owns resetting all counts when the +// policy's one shared monotonic minute expires. Re-login must not reset them. +class RoomAccessPolicy { + RoomAccessSettings settings_; + uint32_t quota_window_started_ = 0; + bool quota_window_active_ = false; + bool healthy_ = false; + + int findBan(const uint8_t* key) const { + if (!key) return -1; + for (uint8_t i = 0; i < settings_.ban_count; ++i) { + if (memcmp(settings_.bans[i], key, ROOM_ACCESS_KEY_SIZE) == 0) return i; + } + return -1; + } + template <typename Filesystem> + bool commit(Filesystem* fs, const RoomAccessSettings& next) { + if (!healthy_) return false; + const auto result = room_access_detail::save(fs, next); + if (result == room_access_detail::SaveResult::Unavailable) healthy_ = false; + if (result != room_access_detail::SaveResult::Saved) return false; + if (settings_.posts_per_minute != next.posts_per_minute + || settings_.polls_per_minute != next.polls_per_minute) quota_window_active_ = false; + settings_ = next; + return true; + } + bool consume(uint16_t& used, uint16_t limit) const { + if (!healthy_) return false; + if (limit == 0) return true; + if (used >= limit) return false; + ++used; + return true; + } + static bool prefix(const char* command, const char* expected, const char*& args) { + const size_t length = strlen(expected); + if (strncmp(command, expected, length) != 0 + || (command[length] != 0 && command[length] != ' ')) return false; + args = command + length; + while (*args == ' ') ++args; + return true; + } + static bool parseUnsigned(const char* value, uint16_t& result) { + if (!value || *value < '0' || *value > '9') return false; + uint32_t number = 0; + do { + number = number * 10 + unsigned(*value++ - '0'); + if (number > UINT16_MAX) return false; + } while (*value >= '0' && *value <= '9'); + if (*value != 0) return false; + result = static_cast<uint16_t>(number); + return true; + } + +public: + enum class BanResult : uint8_t { Saved, Unchanged, InvalidKey, Full, StorageFailure }; + + bool healthy() const { return healthy_; } + uint16_t postRate() const { return settings_.posts_per_minute; } + uint16_t pollRate() const { return settings_.polls_per_minute; } + uint8_t banCount() const { return settings_.ban_count; } + const uint8_t* banAt(size_t index) const { + return index < settings_.ban_count ? settings_.bans[index] : nullptr; + } + bool isBanned(const uint8_t* key) const { return findBan(key) >= 0; } + // Apply before password checks and to existing sessions too. Unknown policy + // storage cannot silently turn a banned identity into an admitted identity. + bool allowsIdentity(const uint8_t* key) const { + return healthy_ && room_access_detail::nonzeroKey(key) && !isBanned(key); + } + template <typename Filesystem> + bool load(Filesystem* fs) { + RoomAccessSettings staged; + healthy_ = room_access_detail::recover(fs, staged, false); + if (healthy_) settings_ = staged; + quota_window_active_ = false; + return healthy_; + } + template <typename Filesystem> + BanResult addBan(Filesystem* fs, const uint8_t* key) { + if (!room_access_detail::nonzeroKey(key)) return BanResult::InvalidKey; + if (!healthy_) return BanResult::StorageFailure; + if (isBanned(key)) return BanResult::Unchanged; + if (settings_.ban_count == ROOM_ACCESS_MAX_BANS) return BanResult::Full; + RoomAccessSettings next = settings_; + memcpy(next.bans[next.ban_count++], key, ROOM_ACCESS_KEY_SIZE); + return commit(fs, next) ? BanResult::Saved : BanResult::StorageFailure; + } + template <typename Filesystem> + BanResult removeBan(Filesystem* fs, const uint8_t* key) { + if (!room_access_detail::nonzeroKey(key)) return BanResult::InvalidKey; + if (!healthy_) return BanResult::StorageFailure; + const int at = findBan(key); + if (at < 0) return BanResult::Unchanged; + RoomAccessSettings next = settings_; + --next.ban_count; + if (at < next.ban_count) { + memmove(next.bans[at], next.bans[at + 1], (next.ban_count - at) * ROOM_ACCESS_KEY_SIZE); + } + memset(next.bans[next.ban_count], 0, ROOM_ACCESS_KEY_SIZE); + return commit(fs, next) ? BanResult::Saved : BanResult::StorageFailure; + } + template <typename Filesystem> + bool setRates(Filesystem* fs, uint16_t post_rate, uint16_t poll_rate) { + if (!healthy_) return false; + if (post_rate == postRate() && poll_rate == pollRate()) return true; + RoomAccessSettings next = settings_; + next.posts_per_minute = post_rate; + next.polls_per_minute = poll_rate; + return commit(fs, next); + } + template <typename ResetCounts> + void serviceQuotaWindow(uint32_t now, ResetCounts reset_counts) { + if (!quota_window_active_ || uint32_t(now - quota_window_started_) >= 60000UL) { + reset_counts(); + quota_window_started_ = now; + quota_window_active_ = true; + } + } + bool consumePost(uint16_t& used) const { return consume(used, postRate()); } + bool consumeKeepAlive(uint16_t& used) const { return consume(used, pollRate()); } + + static bool parseFullKey(const char* text, uint8_t (&key)[ROOM_ACCESS_KEY_SIZE]) { + if (!text || strlen(text) != ROOM_ACCESS_KEY_SIZE * 2) return false; + uint8_t parsed[ROOM_ACCESS_KEY_SIZE]; + for (size_t i = 0; i < sizeof(parsed); ++i) { + uint8_t byte = 0; + for (unsigned nibble = 0; nibble < 2; ++nibble) { + const char digit = text[i * 2 + nibble]; + unsigned value; + if (digit >= '0' && digit <= '9') value = digit - '0'; + else if (digit >= 'a' && digit <= 'f') value = digit - 'a' + 10; + else if (digit >= 'A' && digit <= 'F') value = digit - 'A' + 10; + else return false; + byte = static_cast<uint8_t>((byte << 4) | value); + } + parsed[i] = byte; + } + if (!room_access_detail::nonzeroKey(parsed)) return false; + memcpy(key, parsed, sizeof(parsed)); + return true; + } + void formatBanPage(unsigned page, char* reply, size_t capacity) const { + if (!reply || capacity == 0) return; + if (!healthy_) { snprintf(reply, capacity, "Err - room access storage unavailable"); return; } + const unsigned pages = settings_.ban_count == 0 ? 1 : (settings_.ban_count + 1) / 2; + if (page == 0 || page > pages) { + snprintf(reply, capacity, "Err - ban page range: 1-%u", pages); return; + } + if (settings_.ban_count == 0) { snprintf(reply, capacity, "Room bans: empty"); return; } + const unsigned offset = (page - 1) * 2; + const unsigned rows = settings_.ban_count - offset < 2 ? settings_.ban_count - offset : 2; + const int header = snprintf(reply, capacity, "Room bans %u/%u", page, pages); + if (header < 0 || size_t(header) >= capacity + || rows > (capacity - size_t(header) - 1) / (ROOM_ACCESS_KEY_SIZE * 2 + 1)) { + snprintf(reply, capacity, "Err - ban reply too small"); return; + } + static const char hex_digits[] = "0123456789abcdef"; + size_t used = static_cast<size_t>(header); + for (unsigned row = 0; row < rows; ++row) { + reply[used++] = '\n'; + const uint8_t* key = settings_.bans[offset + row]; + for (size_t i = 0; i < ROOM_ACCESS_KEY_SIZE; ++i) { + reply[used++] = hex_digits[key[i] >> 4]; + reply[used++] = hex_digits[key[i] & 15]; + } + } + reply[used] = 0; + } + + // The role must authorize Admin/local access before invoking this backend. + template <typename Filesystem> + bool handleConfig(Filesystem* fs, const char* command, char* reply, size_t capacity) { + if (!command || !reply || capacity == 0) return false; + const char* args; + bool remove = false; + if (prefix(command, "room.ban", args) || (remove = prefix(command, "room.unban", args))) { + uint8_t key[ROOM_ACCESS_KEY_SIZE]; + if (!parseFullKey(args, key)) { + snprintf(reply, capacity, "Err - use a complete 64-hex public key"); return true; + } + const BanResult result = remove ? removeBan(fs, key) : addBan(fs, key); + if (result == BanResult::Full) snprintf(reply, capacity, "Err - room ban list full (32)"); + else if (result == BanResult::StorageFailure) snprintf(reply, capacity, "Err - room access save failed"); + else snprintf(reply, capacity, remove ? "OK - room ban removed" : "OK - room ban saved"); + return true; + } + if (prefix(command, "get room.bans", args)) { + uint16_t page = 1; + if (*args && !parseUnsigned(args, page)) snprintf(reply, capacity, "Err - usage: get room.bans [page]"); + else formatBanPage(page, reply, capacity); + return true; + } + bool post = false; + if ((post = prefix(command, "get room.post.rate", args)) || prefix(command, "get room.poll.rate", args)) { + if (*args) return false; + if (!healthy_) snprintf(reply, capacity, "Err - room access storage unavailable"); + else snprintf(reply, capacity, "> %u/min (0=off)", unsigned(post ? postRate() : pollRate())); + return true; + } + if ((post = prefix(command, "set room.post.rate", args)) || prefix(command, "set room.poll.rate", args)) { + uint16_t value; + if (!parseUnsigned(args, value)) snprintf(reply, capacity, "Err - rate must be 0-65535/min"); + else if (!setRates(fs, post ? value : postRate(), post ? pollRate() : value)) + snprintf(reply, capacity, "Err - room access save failed"); + else snprintf(reply, capacity, "OK - room rate saved"); + return true; + } + return false; + } +}; +static_assert(sizeof(RoomAccessPolicy) <= 1040, + "Room access policy must not allocate a duplicate identity table"); +} // namespace mesh diff --git a/src/helpers/RoomAccessPolicyStore.h b/src/helpers/RoomAccessPolicyStore.h new file mode 100644 index 000000000..ecf046a15 --- /dev/null +++ b/src/helpers/RoomAccessPolicyStore.h @@ -0,0 +1,204 @@ +#pragma once + +#include <stddef.h> +#include <stdint.h> +#include <string.h> +#include "FilePresence.h" + +namespace mesh { + +static constexpr size_t ROOM_ACCESS_KEY_SIZE = 32; +static constexpr size_t ROOM_ACCESS_MAX_BANS = 32; +static const char ROOM_ACCESS_PRIMARY_PATH[] = "/room_access"; +static const char ROOM_ACCESS_TEMP_PATH[] = "/room_access.tmp"; +static const char ROOM_ACCESS_BACKUP_PATH[] = "/room_access.bak"; + +struct RoomAccessSettings { + uint16_t posts_per_minute = 0; + uint16_t polls_per_minute = 0; + uint8_t ban_count = 0; + uint8_t bans[ROOM_ACCESS_MAX_BANS][ROOM_ACCESS_KEY_SIZE] = {}; +}; +static_assert(sizeof(RoomAccessSettings) <= 1032, + "Room access settings must remain bounded"); + +namespace room_access_detail { +static const uint8_t MAGIC[4] = {'R', 'A', 'P', 1}; +static constexpr size_t HEADER_SIZE = 12; +static constexpr size_t TRAILER_SIZE = 4; + +inline uint32_t crc(uint32_t value, const uint8_t* data, size_t length) { + while (length-- != 0) { + value ^= *data++; + for (uint8_t bit = 0; bit < 8; ++bit) { + value = (value >> 1) ^ ((value & 1) ? UINT32_C(0xedb88320) : 0); + } + } + return value; +} + +inline bool nonzeroKey(const uint8_t* key) { + if (!key) return false; + uint8_t any = 0; + for (size_t i = 0; i < ROOM_ACCESS_KEY_SIZE; ++i) any |= key[i]; + return any != 0; +} + +inline bool validSettings(const RoomAccessSettings& settings) { + if (settings.ban_count > ROOM_ACCESS_MAX_BANS) return false; + for (size_t i = 0; i < settings.ban_count; ++i) { + if (!nonzeroKey(settings.bans[i])) return false; + for (size_t prior = 0; prior < i; ++prior) { + if (memcmp(settings.bans[prior], settings.bans[i], ROOM_ACCESS_KEY_SIZE) == 0) return false; + } + } + return true; +} + +template <typename Filesystem> +bool removeArtifact(Filesystem* fs, const char* path) { + bool present; + if (!filePresence(fs, path, present)) return false; + if (!present) return true; + fs->remove(path); + return filePresence(fs, path, present) && !present; +} + +// Decode into a staging object. Callers publish it only after full validation; +// a failed read can never replace the active policy with a partial ban list. +template <typename Filesystem> +bool readState(Filesystem* fs, const char* path, RoomAccessSettings& staged, + const RoomAccessSettings* expected = nullptr) { +#if defined(RP2040_PLATFORM) + auto file = fs->open(path, "r"); +#else + auto file = fs->open(path); +#endif + if (!file) return false; + uint8_t header[HEADER_SIZE]; + bool valid = !file.isDirectory() + && static_cast<size_t>(file.read(header, sizeof(header))) == sizeof(header); + if (valid) { + const size_t count = header[8]; + valid = memcmp(header, MAGIC, sizeof(MAGIC)) == 0 + && header[9] == 0 && header[10] == 0 && header[11] == 0 + && count <= ROOM_ACCESS_MAX_BANS + && file.size() == HEADER_SIZE + count * ROOM_ACCESS_KEY_SIZE + TRAILER_SIZE; + if (valid) { + staged.posts_per_minute = uint16_t(header[4]) | (uint16_t(header[5]) << 8); + staged.polls_per_minute = uint16_t(header[6]) | (uint16_t(header[7]) << 8); + staged.ban_count = static_cast<uint8_t>(count); + memset(staged.bans, 0, sizeof(staged.bans)); + } + } + uint32_t checksum = valid ? crc(UINT32_C(0xffffffff), header, sizeof(header)) : 0; + for (size_t i = 0; valid && i < staged.ban_count; ++i) { + uint8_t* key = staged.bans[i]; + valid = static_cast<size_t>(file.read(key, ROOM_ACCESS_KEY_SIZE)) == ROOM_ACCESS_KEY_SIZE + && nonzeroKey(key); + if (valid) { + checksum = crc(checksum, key, ROOM_ACCESS_KEY_SIZE); + for (size_t prior = 0; prior < i; ++prior) { + if (memcmp(staged.bans[prior], key, ROOM_ACCESS_KEY_SIZE) == 0) valid = false; + } + } + } + uint8_t trailer[TRAILER_SIZE] = {}; + if (valid) { + valid = static_cast<size_t>(file.read(trailer, sizeof(trailer))) == sizeof(trailer); + const uint32_t stored = uint32_t(trailer[0]) | (uint32_t(trailer[1]) << 8) + | (uint32_t(trailer[2]) << 16) | (uint32_t(trailer[3]) << 24); + valid = valid && (checksum ^ UINT32_C(0xffffffff)) == stored; + } + file.close(); + if (valid && expected) { + valid = staged.posts_per_minute == expected->posts_per_minute + && staged.polls_per_minute == expected->polls_per_minute + && staged.ban_count == expected->ban_count + && memcmp(staged.bans, expected->bans, + staged.ban_count * ROOM_ACCESS_KEY_SIZE) == 0; + } + return valid; +} + +// A present primary is authoritative even when corrupt/newer/unreadable. +// Restore only a verified backup when the primary is genuinely absent. A temp +// file is uncommitted and can never be promoted after reboot. +template <typename Filesystem> +bool recover(Filesystem* fs, RoomAccessSettings& staged, bool require_cleanup) { + bool present; + if (!fs || !filePresence(fs, ROOM_ACCESS_PRIMARY_PATH, present)) return false; + if (present) { + if (!readState(fs, ROOM_ACCESS_PRIMARY_PATH, staged)) return false; + const bool removed_temp = removeArtifact(fs, ROOM_ACCESS_TEMP_PATH); + const bool removed_backup = removeArtifact(fs, ROOM_ACCESS_BACKUP_PATH); + return !require_cleanup || (removed_temp && removed_backup); + } + if (!filePresence(fs, ROOM_ACCESS_BACKUP_PATH, present)) return false; + if (present) { + if (!readState(fs, ROOM_ACCESS_BACKUP_PATH, staged) + || !fs->rename(ROOM_ACCESS_BACKUP_PATH, ROOM_ACCESS_PRIMARY_PATH)) return false; + } else { + staged = RoomAccessSettings{}; + } + const bool removed_temp = removeArtifact(fs, ROOM_ACCESS_TEMP_PATH); + return !require_cleanup || removed_temp; +} + +enum class SaveResult : uint8_t { Saved, Unavailable, WriteFailure }; + +template <typename Filesystem> +SaveResult save(Filesystem* fs, const RoomAccessSettings& next) { + if (!validSettings(next)) return SaveResult::WriteFailure; + RoomAccessSettings checked; + if (!recover(fs, checked, true)) return SaveResult::Unavailable; + uint8_t header[HEADER_SIZE] = {}; + memcpy(header, MAGIC, sizeof(MAGIC)); + header[4] = static_cast<uint8_t>(next.posts_per_minute); + header[5] = static_cast<uint8_t>(next.posts_per_minute >> 8); + header[6] = static_cast<uint8_t>(next.polls_per_minute); + header[7] = static_cast<uint8_t>(next.polls_per_minute >> 8); + header[8] = next.ban_count; +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + auto file = fs->open(ROOM_ACCESS_TEMP_PATH, FILE_O_WRITE); +#elif defined(ESP32_PLATFORM) + auto file = fs->open(ROOM_ACCESS_TEMP_PATH, "w", true); +#else + auto file = fs->open(ROOM_ACCESS_TEMP_PATH, "w"); +#endif + if (!file) return SaveResult::WriteFailure; + bool written = !file.isDirectory() + && file.write(header, sizeof(header)) == sizeof(header); + uint32_t checksum = crc(UINT32_C(0xffffffff), header, sizeof(header)); + for (size_t i = 0; written && i < next.ban_count; ++i) { + written = file.write(next.bans[i], ROOM_ACCESS_KEY_SIZE) == ROOM_ACCESS_KEY_SIZE; + checksum = crc(checksum, next.bans[i], ROOM_ACCESS_KEY_SIZE); + } + checksum ^= UINT32_C(0xffffffff); + uint8_t trailer[TRAILER_SIZE]; + for (size_t i = 0; i < sizeof(trailer); ++i) trailer[i] = static_cast<uint8_t>(checksum >> (8 * i)); + written = written && file.write(trailer, sizeof(trailer)) == sizeof(trailer); + file.flush(); + file.close(); + if (!written || !readState(fs, ROOM_ACCESS_TEMP_PATH, checked, &next)) { + removeArtifact(fs, ROOM_ACCESS_TEMP_PATH); + return SaveResult::WriteFailure; + } + bool had_primary; + if (!filePresence(fs, ROOM_ACCESS_PRIMARY_PATH, had_primary) + || (had_primary && !fs->rename(ROOM_ACCESS_PRIMARY_PATH, ROOM_ACCESS_BACKUP_PATH))) { + removeArtifact(fs, ROOM_ACCESS_TEMP_PATH); + return SaveResult::WriteFailure; + } + if (!fs->rename(ROOM_ACCESS_TEMP_PATH, ROOM_ACCESS_PRIMARY_PATH)) { + if (had_primary) fs->rename(ROOM_ACCESS_BACKUP_PATH, ROOM_ACCESS_PRIMARY_PATH); + removeArtifact(fs, ROOM_ACCESS_TEMP_PATH); + return SaveResult::WriteFailure; + } + // Commit occurred at rename. Failed backup housekeeping cannot reject a + // policy that is already durable, nor publish an older in-memory policy. + removeArtifact(fs, ROOM_ACCESS_BACKUP_PATH); + return SaveResult::Saved; +} +} // namespace room_access_detail +} // namespace mesh diff --git a/src/helpers/RoomBoardProtocol.h b/src/helpers/RoomBoardProtocol.h new file mode 100644 index 000000000..0387b9d9d --- /dev/null +++ b/src/helpers/RoomBoardProtocol.h @@ -0,0 +1,250 @@ +#pragma once + +#include <stddef.h> +#include <stdint.h> +#include <stdio.h> +#include <string.h> +#include "RoomBoardStore.h" + +namespace mesh { + +static constexpr uint8_t ROOM_BOARD_REQUEST_SUBTYPE = 0x0a; +static constexpr size_t ROOM_BOARD_MAX_COMMAND_LENGTH = 512; + +namespace room_board_protocol_detail { +inline uint8_t status(RoomBoardResult result) { + switch (result) { + case RoomBoardResult::Success: return 0; + case RoomBoardResult::Invalid: return 1; + case RoomBoardResult::Unavailable: return 2; + case RoomBoardResult::WriteFailure: return 3; + case RoomBoardResult::NotFound: return 4; + case RoomBoardResult::StaleVersion: return 5; + } + return 1; +} +inline size_t shortError(uint8_t op, RoomBoardResult result, uint8_t* reply, size_t capacity) { + if (!reply || capacity < 3) return 0; + reply[0] = ROOM_BOARD_REQUEST_SUBTYPE; reply[1] = op; reply[2] = status(result); return 3; +} +inline void textResult(char* reply, size_t capacity, RoomBoardResult result, uint32_t revision = 0) { + if (!reply || capacity == 0) return; + const char* message; + switch (result) { + case RoomBoardResult::Success: snprintf(reply, capacity, "OK rev=%lu", (unsigned long)revision); return; + case RoomBoardResult::Unavailable: message = "Error unavailable"; break; + case RoomBoardResult::WriteFailure: message = "Error write failure"; break; + case RoomBoardResult::NotFound: message = "Error not found"; break; + case RoomBoardResult::StaleVersion: message = "Error stale version"; break; + default: message = "Error invalid"; break; + } + snprintf(reply, capacity, "%s", message); +} +inline bool prefix(const char* command, const char* expected) { + const size_t length = strlen(expected); + return strncmp(command, expected, length) == 0 && (command[length] == 0 || command[length] == ' '); +} +inline const char* spaces(const char* cursor) { while (*cursor == ' ') ++cursor; return cursor; } +inline bool number(const char*& cursor, uint32_t maximum, uint32_t& value) { + cursor = spaces(cursor); value = 0; + if (*cursor < '0' || *cursor > '9') return false; + do { + const uint8_t digit = uint8_t(*cursor++ - '0'); + if (value > maximum / 10 || (value == maximum / 10 && digit > maximum % 10)) return false; + value = value * 10 + digit; + } while (*cursor >= '0' && *cursor <= '9'); + return *cursor == 0 || *cursor == ' '; +} +inline size_t base64(const uint8_t* data, size_t length, char* output) { + static const char alphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + size_t position = 0; + for (size_t i = 0; i < length; i += 3) { + const uint32_t bits = (uint32_t(data[i]) << 16) + | (i + 1 < length ? uint32_t(data[i + 1]) << 8 : 0) + | (i + 2 < length ? uint32_t(data[i + 2]) : 0); + output[position++] = alphabet[(bits >> 18) & 63]; + output[position++] = alphabet[(bits >> 12) & 63]; + output[position++] = i + 1 < length ? alphabet[(bits >> 6) & 63] : '='; + output[position++] = i + 2 < length ? alphabet[bits & 63] : '='; + } + output[position] = 0; return position; +} +} // namespace room_board_protocol_detail + +// Read-only requests, callable for any authorized room reader, including +// guests. The caller reserves its transport tag outside this reply capacity. +// All integer fields are little-endian. No partial index entry is emitted. +template <typename Filesystem> +size_t handleRoomBoardRequest(Filesystem* fs, const uint8_t* payload, size_t length, + uint8_t* reply, size_t capacity) { + using namespace room_board_protocol_detail; + const uint8_t op = payload && length > 1 ? payload[1] : 0xff; + if (!payload || length < 2 || payload[0] != ROOM_BOARD_REQUEST_SUBTYPE + || !reply || capacity < 3 || op > 1) return shortError(op, RoomBoardResult::Invalid, reply, capacity); + if (op == 0) { + if (length < 7 || capacity < 10) return shortError(op, RoomBoardResult::Invalid, reply, capacity); + const uint32_t expected = room_board_detail::get32(payload + 2); + const uint8_t start = payload[6]; + RoomBoardIndex index; + RoomBoardResult result = RoomBoardResult::Success; + if (!loadRoomBoard(fs, index)) result = RoomBoardResult::Unavailable; + else if ((!expected && start) || (expected && expected != index.revision)) result = RoomBoardResult::StaleVersion; + else if (start > index.count) result = RoomBoardResult::Invalid; + reply[0] = ROOM_BOARD_REQUEST_SUBTYPE; reply[1] = op; + room_board_detail::put32(reply + 3, index.revision); + reply[7] = index.count; reply[8] = start; reply[9] = start; + size_t used = 10; + if (result == RoomBoardResult::Success) { + uint8_t position = start; + while (position < index.count && position < size_t(start) + 2) { + const RoomBoardArticle& entry = index.articles[position]; + const size_t title_length = strlen(entry.title); + const size_t entry_length = 8 + title_length; + if (entry_length > capacity - used) { + if (position == start) result = RoomBoardResult::Invalid; + break; + } + uint8_t* encoded = reply + used; + encoded[0] = entry.id; room_board_detail::put32(encoded + 1, entry.version); + room_board_detail::put16(encoded + 5, entry.body_length); encoded[7] = uint8_t(title_length); + memcpy(encoded + 8, entry.title, title_length); used += entry_length; ++position; + } + reply[9] = position; + } + reply[2] = status(result); return used; + } + if (length < 9 || capacity < 13) return shortError(op, RoomBoardResult::Invalid, reply, capacity); + const uint8_t id = payload[2]; + const uint32_t version = room_board_detail::get32(payload + 3); + const uint16_t offset = room_board_detail::get16(payload + 7); + reply[0] = ROOM_BOARD_REQUEST_SUBTYPE; reply[1] = op; reply[3] = id; + room_board_detail::put32(reply + 4, version); room_board_detail::put16(reply + 8, offset); + room_board_detail::put16(reply + 10, 0); reply[12] = 0; + RoomBoardIndex index; + RoomBoardResult result = RoomBoardResult::Success; + const RoomBoardArticle* article = nullptr; + if (id == 0 || id > ROOM_BOARD_MAX_ARTICLES || version == 0) result = RoomBoardResult::Invalid; + else if (!loadRoomBoard(fs, index)) result = RoomBoardResult::Unavailable; + else { + for (uint8_t i = 0; i < index.count; ++i) if (index.articles[i].id == id) article = &index.articles[i]; + if (!article) result = RoomBoardResult::NotFound; + else { + room_board_detail::put32(reply + 4, article->version); + room_board_detail::put16(reply + 10, article->body_length); + if (article->version != version) result = RoomBoardResult::StaleVersion; + else if (offset > article->body_length || (capacity == 13 && offset < article->body_length)) result = RoomBoardResult::Invalid; + } + } + size_t copied = 0; + if (result == RoomBoardResult::Success && offset < article->body_length) { + const size_t available = capacity - 13; + const size_t chunk = available < ROOM_BOARD_READ_MAX_CHUNK ? available : ROOM_BOARD_READ_MAX_CHUNK; + result = readRoomBoardArticle(fs, id, version, offset, reply + 13, chunk, copied); + } + reply[2] = status(result); reply[12] = uint8_t(copied); return 13 + copied; +} + +// Caller owns administrative authorization for put/del (and may grant get +// to ordinary room readers). The command body is borrowed, never buffered. +// `get room.board [cursor]` returns `next=` for the next whole-entry page. +template <typename Filesystem> +bool handleRoomBoardCommand(Filesystem* fs, const char* command, char* reply, size_t capacity) { + using namespace room_board_protocol_detail; + if (!command) return false; + const bool put = prefix(command, "room.board.put"); + const bool del = prefix(command, "room.board.del"); + const bool read = prefix(command, "get room.board.read"); + const bool list = prefix(command, "get room.board"); + if (!put && !del && !read && !list) return false; + if (!reply || capacity == 0) return true; + reply[0] = 0; + size_t length = 0; + while (length <= ROOM_BOARD_MAX_COMMAND_LENGTH && command[length]) ++length; + if (length > ROOM_BOARD_MAX_COMMAND_LENGTH) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + if (put) { + const char* cursor = command + strlen("room.board.put"); uint32_t id; + if (!number(cursor, ROOM_BOARD_MAX_ARTICLES, id) || id == 0) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + cursor = spaces(cursor); const char* separator = strchr(cursor, '|'); + size_t title_length = separator ? size_t(separator - cursor) : 0; + while (title_length && cursor[title_length - 1] == ' ') --title_length; + if (!separator || title_length == 0 || title_length >= ROOM_BOARD_TITLE_CAPACITY) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + char title[ROOM_BOARD_TITLE_CAPACITY] = {}; memcpy(title, cursor, title_length); + const char* body = separator + 1; const size_t body_length = strlen(body); + const RoomBoardResult result = saveRoomBoardArticle(fs, uint8_t(id), title, body_length, + [body](size_t offset, uint8_t* output, size_t count) -> size_t { + memcpy(output, body + offset, count); return count; + }); + RoomBoardIndex index; + if (result == RoomBoardResult::Success) loadRoomBoard(fs, index); + textResult(reply, capacity, result, index.revision); return true; + } + if (del) { + const char* cursor = command + strlen("room.board.del"); uint32_t id, version; + if (!number(cursor, ROOM_BOARD_MAX_ARTICLES, id) || id == 0 || !number(cursor, UINT32_MAX, version) + || version == 0 || *spaces(cursor)) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + const RoomBoardResult result = deleteRoomBoardArticle(fs, uint8_t(id), version); + RoomBoardIndex index; + if (result == RoomBoardResult::Success) loadRoomBoard(fs, index); + textResult(reply, capacity, result, index.revision); return true; + } + RoomBoardIndex index; + if (!loadRoomBoard(fs, index)) { textResult(reply, capacity, RoomBoardResult::Unavailable); return true; } + if (read) { + const char* cursor = command + strlen("get room.board.read"); uint32_t id, version, offset; + if (!number(cursor, ROOM_BOARD_MAX_ARTICLES, id) || id == 0 || !number(cursor, UINT32_MAX, version) + || version == 0 || !number(cursor, ROOM_BOARD_MAX_BODY_LENGTH, offset) || *spaces(cursor)) { + textResult(reply, capacity, RoomBoardResult::Invalid); return true; + } + const RoomBoardArticle* article = nullptr; + for (uint8_t i = 0; i < index.count; ++i) if (index.articles[i].id == id) article = &index.articles[i]; + if (!article) { textResult(reply, capacity, RoomBoardResult::NotFound); return true; } + if (article->version != version) { textResult(reply, capacity, RoomBoardResult::StaleVersion); return true; } + if (offset > article->body_length) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + const size_t remaining = article->body_length - offset; + size_t count = remaining < 80 ? remaining : 80; + int prefix_length = -1; + while (true) { + prefix_length = snprintf(reply, capacity, "id=%lu v=%lu off=%lu total=%u n=%u data=", + (unsigned long)id, (unsigned long)version, (unsigned long)offset, + unsigned(article->body_length), unsigned(count)); + if (prefix_length >= 0 && size_t(prefix_length) + ((count + 2) / 3) * 4 < capacity) break; + if (count == 0) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + --count; + } + if (count == 0 && remaining) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + uint8_t data[80]; size_t copied = 0; + if (count) { + const RoomBoardResult result = readRoomBoardArticle(fs, uint8_t(id), version, offset, data, count, copied); + if (result != RoomBoardResult::Success) { textResult(reply, capacity, result); return true; } + } + base64(data, copied, reply + prefix_length); return true; + } + const char* cursor = spaces(command + strlen("get room.board")); uint32_t start = 0; + if ((*cursor && !number(cursor, ROOM_BOARD_MAX_ARTICLES, start)) || *spaces(cursor) || start > index.count) { + textResult(reply, capacity, RoomBoardResult::Invalid); return true; + } + // Format entries first, then the fixed prefix with the final next cursor. + char entries[2 * (ROOM_BOARD_TITLE_CAPACITY + 35)] = {}; + size_t used = 0; uint32_t next = start; + while (next < index.count && next < start + 2) { + const RoomBoardArticle& article = index.articles[next]; + const int written = snprintf(entries + used, sizeof(entries) - used, " %u@%lu/%u:%s", + unsigned(article.id), (unsigned long)article.version, unsigned(article.body_length), article.title); + char header[64]; + const int header_length = snprintf(header, sizeof(header), "rev=%lu total=%u start=%lu next=%lu", + (unsigned long)index.revision, unsigned(index.count), (unsigned long)start, (unsigned long)(next + 1)); + if (written < 0 || header_length < 0 || size_t(written) >= sizeof(entries) - used + || size_t(header_length) + used + size_t(written) >= capacity) { + entries[used] = 0; + if (next == start) { textResult(reply, capacity, RoomBoardResult::Invalid); return true; } + break; + } + used += size_t(written); ++next; + } + const int written = snprintf(reply, capacity, "rev=%lu total=%u start=%lu next=%lu%s", + (unsigned long)index.revision, unsigned(index.count), (unsigned long)start, (unsigned long)next, entries); + if (written < 0 || size_t(written) >= capacity) textResult(reply, capacity, RoomBoardResult::Invalid); + return true; +} + +} // namespace mesh diff --git a/src/helpers/RoomBoardStore.h b/src/helpers/RoomBoardStore.h new file mode 100644 index 000000000..b5fc02706 --- /dev/null +++ b/src/helpers/RoomBoardStore.h @@ -0,0 +1,403 @@ +#pragma once + +#include <stddef.h> +#include <stdint.h> +#include <stdio.h> +#include <string.h> +#include "FilePresence.h" + +namespace mesh { + +static constexpr uint8_t ROOM_BOARD_MAX_ARTICLES = 8; +static constexpr size_t ROOM_BOARD_TITLE_CAPACITY = 64; +static constexpr size_t ROOM_BOARD_MAX_BODY_LENGTH = 2048; +static constexpr size_t ROOM_BOARD_READ_MAX_CHUNK = 128; +static const char ROOM_BOARD_PRIMARY_PATH[] = "/room_board"; +static const char ROOM_BOARD_TEMP_PATH[] = "/room_board.tmp"; +static const char ROOM_BOARD_BACKUP_PATH[] = "/room_board.bak"; + +struct RoomBoardArticle { + uint8_t id = 0; // Stable slots, 1 through 8. + uint8_t bank = 0; // Internal immutable-body file selection. + uint16_t body_length = 0; + uint32_t version = 0; + char title[ROOM_BOARD_TITLE_CAPACITY] = {}; +}; + +struct RoomBoardIndex { + uint32_t revision = 0; + uint8_t count = 0; + RoomBoardArticle articles[ROOM_BOARD_MAX_ARTICLES]; // Sorted by article ID. +}; +static_assert(sizeof(RoomBoardIndex) <= 640, "Board metadata must remain bounded"); + +enum class RoomBoardResult : uint8_t { + Success, Invalid, Unavailable, WriteFailure, NotFound, StaleVersion +}; + +namespace room_board_detail { +static constexpr size_t HEADER_SIZE = 16; +static constexpr size_t ENTRY_SIZE = 72; +static constexpr size_t STREAM_CHUNK = 64; +static const uint8_t INDEX_MAGIC[4] = {'R', 'B', 'I', 1}; +static const uint8_t ARTICLE_MAGIC[4] = {'R', 'B', 'A', 1}; + +inline uint32_t crc(uint32_t value, const uint8_t* data, size_t length) { + while (length-- != 0) { + value ^= *data++; + for (uint8_t bit = 0; bit < 8; ++bit) { + value = (value >> 1) ^ ((value & 1) ? UINT32_C(0xedb88320) : 0); + } + } + return value; +} +inline uint32_t get32(const uint8_t* bytes) { + return uint32_t(bytes[0]) | (uint32_t(bytes[1]) << 8) + | (uint32_t(bytes[2]) << 16) | (uint32_t(bytes[3]) << 24); +} +inline void put32(uint8_t* bytes, uint32_t value) { + for (uint8_t i = 0; i < 4; ++i) bytes[i] = uint8_t(value >> (8 * i)); +} +inline uint16_t get16(const uint8_t* bytes) { + return uint16_t(bytes[0]) | (uint16_t(bytes[1]) << 8); +} +inline void put16(uint8_t* bytes, uint16_t value) { + bytes[0] = uint8_t(value); bytes[1] = uint8_t(value >> 8); +} + +// Titles are complete UTF-8 strings; reject overlong encodings, surrogate +// code points, embedded controls, and values beyond Unicode's maximum. +inline bool validTitle(const char* title) { + if (!title) return false; + size_t length = 0; + while (length < ROOM_BOARD_TITLE_CAPACITY && title[length] != 0) ++length; + if (length == 0 || length == ROOM_BOARD_TITLE_CAPACITY) return false; + for (size_t i = 0; i < length;) { + const uint8_t first = uint8_t(title[i++]); + if (first < 0x80) { + if (first < 0x20 || first == 0x7f) return false; + continue; + } + uint8_t count; + uint32_t point, minimum; + if (first >= 0xc2 && first <= 0xdf) { count = 1; point = first & 0x1f; minimum = 0x80; } + else if (first >= 0xe0 && first <= 0xef) { count = 2; point = first & 0x0f; minimum = 0x800; } + else if (first >= 0xf0 && first <= 0xf4) { count = 3; point = first & 0x07; minimum = 0x10000; } + else return false; + if (i + count > length) return false; + while (count-- != 0) { + const uint8_t next = uint8_t(title[i++]); + if ((next & 0xc0) != 0x80) return false; + point = (point << 6) | (next & 0x3f); + } + if (point < minimum || point > 0x10ffff || (point >= 0xd800 && point <= 0xdfff)) return false; + } + return true; +} + +inline void articlePath(const RoomBoardArticle& entry, char (&path)[24]) { + snprintf(path, sizeof(path), "/room_board_%u.%c", unsigned(entry.id), entry.bank ? 'b' : 'a'); +} +inline void encodeEntry(const RoomBoardArticle& entry, uint8_t (&bytes)[ENTRY_SIZE]) { + memset(bytes, 0, sizeof(bytes)); + bytes[0] = entry.id; bytes[1] = entry.bank; + put16(bytes + 2, entry.body_length); put32(bytes + 4, entry.version); + memcpy(bytes + 8, entry.title, sizeof(entry.title)); +} +inline void articleHeader(const RoomBoardArticle& entry, uint8_t (&bytes)[HEADER_SIZE]) { + memset(bytes, 0, sizeof(bytes)); + memcpy(bytes, ARTICLE_MAGIC, sizeof(ARTICLE_MAGIC)); + bytes[4] = entry.id; bytes[5] = entry.bank; + put16(bytes + 6, entry.body_length); put32(bytes + 8, entry.version); +} + +template <typename Filesystem> +auto openRead(Filesystem* fs, const char* path) +#if defined(RP2040_PLATFORM) + -> decltype(fs->open(path, static_cast<const char*>(nullptr))) { return fs->open(path, "r"); } +#else + -> decltype(fs->open(path)) { return fs->open(path); } +#endif + +template <typename Filesystem> +auto openWrite(Filesystem* fs, const char* path) +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + -> decltype(fs->open(path, FILE_O_WRITE)) { return fs->open(path, FILE_O_WRITE); } +#elif defined(ESP32_PLATFORM) + -> decltype(fs->open(path, static_cast<const char*>(nullptr), true)) { return fs->open(path, "w", true); } +#else + -> decltype(fs->open(path, static_cast<const char*>(nullptr))) { return fs->open(path, "w"); } +#endif + +template <typename Filesystem> +bool removeArtifact(Filesystem* fs, const char* path) { + bool present; + if (!filePresence(fs, path, present)) return false; + if (!present) return true; + fs->remove(path); + return filePresence(fs, path, present) && !present; +} + +// Validate the whole selected body before any caller may send the copied +// slice. The only internal body buffer is 64 bytes, including readback. +template <typename Filesystem> +bool readBody(Filesystem* fs, const RoomBoardArticle& entry, + uint8_t* output = nullptr, size_t offset = 0, size_t capacity = 0, + const uint32_t* expected_checksum = nullptr) { + char path[24]; articlePath(entry, path); + auto file = openRead(fs, path); + if (!file) return false; + uint8_t header[HEADER_SIZE], expected[HEADER_SIZE]; + articleHeader(entry, expected); + bool valid = !file.isDirectory() + && file.size() == HEADER_SIZE + entry.body_length + 4 + && static_cast<size_t>(file.read(header, sizeof(header))) == sizeof(header) + && memcmp(header, expected, sizeof(header)) == 0; + uint32_t checksum = valid ? crc(UINT32_C(0xffffffff), header, sizeof(header)) : 0; + uint8_t chunk[STREAM_CHUNK]; + for (size_t position = 0; valid && position < entry.body_length;) { + const size_t remaining = entry.body_length - position; + const size_t length = remaining < sizeof(chunk) ? remaining : sizeof(chunk); + valid = static_cast<size_t>(file.read(chunk, length)) == length; + if (!valid) break; + checksum = crc(checksum, chunk, length); + if (output && position + length > offset && position < offset + capacity) { + const size_t start = position < offset ? offset - position : 0; + size_t end = length; + if (position + end > offset + capacity) end = offset + capacity - position; + memcpy(output + position + start - offset, chunk + start, end - start); + } + position += length; + } + uint8_t trailer[4]; + if (valid) { + checksum ^= UINT32_C(0xffffffff); + valid = static_cast<size_t>(file.read(trailer, sizeof(trailer))) == sizeof(trailer) + && get32(trailer) == checksum + && (!expected_checksum || *expected_checksum == checksum); + } + file.close(); + if (!valid && output) memset(output, 0, capacity); + return valid; +} + +inline bool sameIndex(const RoomBoardIndex& first, const RoomBoardIndex& second) { + if (first.revision != second.revision || first.count != second.count) return false; + for (uint8_t i = 0; i < first.count; ++i) { + uint8_t a[ENTRY_SIZE], b[ENTRY_SIZE]; + encodeEntry(first.articles[i], a); encodeEntry(second.articles[i], b); + if (memcmp(a, b, sizeof(a)) != 0) return false; + } + return true; +} + +template <typename Filesystem> +bool readIndex(Filesystem* fs, const char* path, RoomBoardIndex& staged, + const RoomBoardIndex* expected = nullptr) { + auto file = openRead(fs, path); + if (!file) return false; + uint8_t header[HEADER_SIZE]; + bool valid = !file.isDirectory() + && static_cast<size_t>(file.read(header, sizeof(header))) == sizeof(header); + if (valid) { + valid = memcmp(header, INDEX_MAGIC, sizeof(INDEX_MAGIC)) == 0 + && get32(header + 4) != 0 && header[8] <= ROOM_BOARD_MAX_ARTICLES + && file.size() == HEADER_SIZE + header[8] * ENTRY_SIZE + 4; + for (size_t i = 9; i < sizeof(header); ++i) valid = valid && header[i] == 0; + } + uint32_t checksum = valid ? crc(UINT32_C(0xffffffff), header, sizeof(header)) : 0; + staged = RoomBoardIndex{}; + if (valid) { staged.revision = get32(header + 4); staged.count = header[8]; } + for (uint8_t i = 0; valid && i < staged.count; ++i) { + uint8_t bytes[ENTRY_SIZE]; + valid = static_cast<size_t>(file.read(bytes, sizeof(bytes))) == sizeof(bytes); + if (!valid) break; + checksum = crc(checksum, bytes, sizeof(bytes)); + RoomBoardArticle& entry = staged.articles[i]; + entry.id = bytes[0]; entry.bank = bytes[1]; entry.body_length = get16(bytes + 2); + entry.version = get32(bytes + 4); memcpy(entry.title, bytes + 8, sizeof(entry.title)); + valid = entry.id >= 1 && entry.id <= ROOM_BOARD_MAX_ARTICLES && entry.bank <= 1 + && entry.body_length <= ROOM_BOARD_MAX_BODY_LENGTH && entry.version != 0 + && entry.version <= staged.revision && validTitle(entry.title) + && (i == 0 || staged.articles[i - 1].id < entry.id); + bool terminated = false; + for (size_t j = 0; j < sizeof(entry.title); ++j) { + if (terminated && entry.title[j] != 0) valid = false; + if (entry.title[j] == 0) terminated = true; + } + } + uint8_t trailer[4]; + if (valid) valid = static_cast<size_t>(file.read(trailer, sizeof(trailer))) == sizeof(trailer) + && get32(trailer) == (checksum ^ UINT32_C(0xffffffff)); + file.close(); + if (valid && expected) valid = sameIndex(staged, *expected); + for (uint8_t i = 0; valid && i < staged.count; ++i) valid = readBody(fs, staged.articles[i]); + return valid; +} + +// A present primary, including corrupt/future/unreadable metadata or bodies, +// is authoritative. Never replace it with an older backup or delete evidence. +// A temp index is uncommitted; only a verified backup may restore absence. +template <typename Filesystem> +bool recover(Filesystem* fs, RoomBoardIndex& staged, bool require_cleanup) { + bool present; + if (!fs || !filePresence(fs, ROOM_BOARD_PRIMARY_PATH, present)) return false; + if (present) { + if (!readIndex(fs, ROOM_BOARD_PRIMARY_PATH, staged)) return false; + const bool temp_removed = removeArtifact(fs, ROOM_BOARD_TEMP_PATH); + const bool backup_removed = removeArtifact(fs, ROOM_BOARD_BACKUP_PATH); + return !require_cleanup || (temp_removed && backup_removed); + } + if (!filePresence(fs, ROOM_BOARD_BACKUP_PATH, present)) return false; + if (present) { + if (!readIndex(fs, ROOM_BOARD_BACKUP_PATH, staged) + || !fs->rename(ROOM_BOARD_BACKUP_PATH, ROOM_BOARD_PRIMARY_PATH)) return false; + } else staged = RoomBoardIndex{}; + const bool temp_removed = removeArtifact(fs, ROOM_BOARD_TEMP_PATH); + return !require_cleanup || temp_removed; +} + +template <typename Filesystem> +RoomBoardResult commitIndex(Filesystem* fs, const RoomBoardIndex& next) { + uint8_t header[HEADER_SIZE] = {}; + memcpy(header, INDEX_MAGIC, sizeof(INDEX_MAGIC)); + put32(header + 4, next.revision); header[8] = next.count; + auto file = openWrite(fs, ROOM_BOARD_TEMP_PATH); + if (!file) return RoomBoardResult::WriteFailure; + bool written = !file.isDirectory() && file.write(header, sizeof(header)) == sizeof(header); + uint32_t checksum = crc(UINT32_C(0xffffffff), header, sizeof(header)); + for (uint8_t i = 0; written && i < next.count; ++i) { + uint8_t bytes[ENTRY_SIZE]; encodeEntry(next.articles[i], bytes); + written = file.write(bytes, sizeof(bytes)) == sizeof(bytes); + checksum = crc(checksum, bytes, sizeof(bytes)); + } + uint8_t trailer[4]; put32(trailer, checksum ^ UINT32_C(0xffffffff)); + written = written && file.write(trailer, sizeof(trailer)) == sizeof(trailer); + file.flush(); file.close(); + RoomBoardIndex checked; + if (!written || !readIndex(fs, ROOM_BOARD_TEMP_PATH, checked, &next)) { + removeArtifact(fs, ROOM_BOARD_TEMP_PATH); return RoomBoardResult::WriteFailure; + } + bool had_primary; + if (!filePresence(fs, ROOM_BOARD_PRIMARY_PATH, had_primary) + || (had_primary && !fs->rename(ROOM_BOARD_PRIMARY_PATH, ROOM_BOARD_BACKUP_PATH))) { + removeArtifact(fs, ROOM_BOARD_TEMP_PATH); return RoomBoardResult::WriteFailure; + } + if (!fs->rename(ROOM_BOARD_TEMP_PATH, ROOM_BOARD_PRIMARY_PATH)) { + if (had_primary) fs->rename(ROOM_BOARD_BACKUP_PATH, ROOM_BOARD_PRIMARY_PATH); + removeArtifact(fs, ROOM_BOARD_TEMP_PATH); return RoomBoardResult::WriteFailure; + } + // The rename is the commit point. Failed housekeeping cannot undo success. + removeArtifact(fs, ROOM_BOARD_BACKUP_PATH); + return RoomBoardResult::Success; +} +} // namespace room_board_detail + +template <typename Filesystem> +bool loadRoomBoard(Filesystem* fs, RoomBoardIndex& index) { + RoomBoardIndex staged; + index = RoomBoardIndex{}; + if (!room_board_detail::recover(fs, staged, false)) return false; + index = staged; return true; +} + +// Main-loop single writer. The reader is called as reader(offset, buffer, +// requested_length), returning exactly requested_length, in <=64-byte pieces. +// expected_version == 0 permits an administrative create/overwrite; nonzero +// versions perform compare-and-swap and reject an article changed since load. +template <typename Filesystem, typename Reader> +RoomBoardResult saveRoomBoardArticle(Filesystem* fs, uint8_t id, const char* title, + size_t body_length, Reader reader, + uint32_t expected_version = 0) { + using namespace room_board_detail; + if (id == 0 || id > ROOM_BOARD_MAX_ARTICLES || !validTitle(title) + || body_length > ROOM_BOARD_MAX_BODY_LENGTH) return RoomBoardResult::Invalid; + RoomBoardIndex next; + if (!recover(fs, next, true)) return RoomBoardResult::Unavailable; + uint8_t position = 0; + while (position < next.count && next.articles[position].id < id) ++position; + const bool existing = position < next.count && next.articles[position].id == id; + if (expected_version && (!existing || next.articles[position].version != expected_version)) { + return RoomBoardResult::StaleVersion; + } + if (next.revision == UINT32_MAX) return RoomBoardResult::Unavailable; + RoomBoardArticle entry; + entry.id = id; entry.bank = existing ? 1 - next.articles[position].bank : 0; + entry.version = ++next.revision; entry.body_length = uint16_t(body_length); + memcpy(entry.title, title, strlen(title)); + char path[24]; articlePath(entry, path); + if (!removeArtifact(fs, path)) return RoomBoardResult::WriteFailure; + auto file = openWrite(fs, path); + if (!file) return RoomBoardResult::WriteFailure; + uint8_t header[HEADER_SIZE]; articleHeader(entry, header); + bool written = !file.isDirectory() && file.write(header, sizeof(header)) == sizeof(header); + uint32_t checksum = crc(UINT32_C(0xffffffff), header, sizeof(header)); + uint8_t chunk[STREAM_CHUNK]; + for (size_t offset = 0; written && offset < body_length;) { + const size_t remaining = body_length - offset; + const size_t length = remaining < sizeof(chunk) ? remaining : sizeof(chunk); + written = reader(offset, chunk, length) == length + && file.write(chunk, length) == length; + if (written) checksum = crc(checksum, chunk, length); + offset += length; + } + checksum ^= UINT32_C(0xffffffff); + uint8_t trailer[4]; put32(trailer, checksum); + written = written && file.write(trailer, sizeof(trailer)) == sizeof(trailer); + file.flush(); file.close(); + if (!written || !readBody(fs, entry, nullptr, 0, 0, &checksum)) { + removeArtifact(fs, path); return RoomBoardResult::WriteFailure; + } + if (!existing) { + for (uint8_t i = next.count; i > position; --i) next.articles[i] = next.articles[i - 1]; + ++next.count; + } + next.articles[position] = entry; + return commitIndex(fs, next); +} + +template <typename Filesystem> +RoomBoardResult deleteRoomBoardArticle(Filesystem* fs, uint8_t id, + uint32_t expected_version = 0) { + using namespace room_board_detail; + if (id == 0 || id > ROOM_BOARD_MAX_ARTICLES) return RoomBoardResult::Invalid; + RoomBoardIndex next; + if (!recover(fs, next, true)) return RoomBoardResult::Unavailable; + uint8_t position = 0; + while (position < next.count && next.articles[position].id != id) ++position; + if (position == next.count) return RoomBoardResult::NotFound; + if (expected_version && next.articles[position].version != expected_version) return RoomBoardResult::StaleVersion; + if (next.revision == UINT32_MAX) return RoomBoardResult::Unavailable; + ++next.revision; + for (uint8_t i = position; i + 1 < next.count; ++i) next.articles[i] = next.articles[i + 1]; + next.articles[--next.count] = RoomBoardArticle{}; + // Bodies are bounded to two banks per ID. Retaining unreferenced banks + // avoids deleting a body still needed by recovery after a power cut. + return commitIndex(fs, next); +} + +// Every page must identify the exact published article version. The caller +// supplies a <=128-byte capacity; offset==body_length returns successful EOF. +template <typename Filesystem> +RoomBoardResult readRoomBoardArticle(Filesystem* fs, uint8_t id, + uint32_t expected_version, size_t offset, + uint8_t* output, size_t capacity, size_t& bytes_read) { + using namespace room_board_detail; + bytes_read = 0; + if (id == 0 || id > ROOM_BOARD_MAX_ARTICLES || expected_version == 0 + || !output || capacity == 0 || capacity > ROOM_BOARD_READ_MAX_CHUNK) return RoomBoardResult::Invalid; + RoomBoardIndex index; + if (!recover(fs, index, false)) return RoomBoardResult::Unavailable; + uint8_t position = 0; + while (position < index.count && index.articles[position].id != id) ++position; + if (position == index.count) return RoomBoardResult::NotFound; + const RoomBoardArticle& entry = index.articles[position]; + if (entry.version != expected_version) return RoomBoardResult::StaleVersion; + if (offset > entry.body_length) return RoomBoardResult::Invalid; + if (!readBody(fs, entry, output, offset, capacity)) return RoomBoardResult::Unavailable; + const size_t remaining = entry.body_length - offset; + bytes_read = remaining < capacity ? remaining : capacity; + return RoomBoardResult::Success; +} + +} // namespace mesh diff --git a/src/helpers/RoomHistoryStore.h b/src/helpers/RoomHistoryStore.h new file mode 100644 index 000000000..5f8fe7997 --- /dev/null +++ b/src/helpers/RoomHistoryStore.h @@ -0,0 +1,443 @@ +#pragma once + +#include <stddef.h> +#include <stdint.h> +#include <string.h> +#include "FilePresence.h" + +namespace mesh { + +static constexpr uint8_t ROOM_HISTORY_MAX_RECORDS = 32; +static constexpr uint8_t ROOM_HISTORY_JOURNAL_RECORDS = 64; +static constexpr size_t ROOM_HISTORY_TEXT_CAPACITY = 152; +static constexpr size_t ROOM_HISTORY_RECORD_SIZE = 192; +static constexpr size_t ROOM_HISTORY_HEADER_SIZE = 12; +static constexpr size_t ROOM_HISTORY_MAX_FILE_SIZE = ROOM_HISTORY_HEADER_SIZE + + ROOM_HISTORY_JOURNAL_RECORDS * ROOM_HISTORY_RECORD_SIZE; +static const char ROOM_HISTORY_PATH[] = "/room_history"; +static const char ROOM_HISTORY_TEMP_PATH[] = "/room_history.tmp"; +static const char ROOM_HISTORY_BACKUP_PATH[] = "/room_history.bak"; +static const char ROOM_HISTORY_CONFIG_PATH[] = "/room_hist_cfg"; +static const char ROOM_HISTORY_CONFIG_TEMP_PATH[] = "/room_hist_cfg.tmp"; +static const char ROOM_HISTORY_CONFIG_BACKUP_PATH[] = "/room_hist_cfg.bak"; + +struct RoomHistoryRecord { + uint8_t author[32]; + uint32_t timestamp; + char text[ROOM_HISTORY_TEXT_CAPACITY]; +}; + +struct RoomHistoryState { + uint8_t count = 0; + uint8_t total_records = 0; + bool needs_repair = false; + bool present = false; +}; + +namespace room_history_detail { + +inline uint32_t crcBytes(const uint8_t* data, size_t length) { + uint32_t crc = UINT32_C(0xffffffff); + for (size_t index = 0; index < length; ++index) { + crc ^= data[index]; + for (uint8_t bit = 0; bit < 8; ++bit) { + crc = (crc >> 1) ^ ((crc & 1) ? UINT32_C(0xedb88320) : 0); + } + } + return crc ^ UINT32_C(0xffffffff); +} + +inline void put32(uint8_t* data, uint32_t value) { + for (uint8_t index = 0; index < 4; ++index) data[index] = value >> (8 * index); +} + +inline uint32_t get32(const uint8_t* data) { + uint32_t value = 0; + for (uint8_t index = 0; index < 4; ++index) value |= uint32_t(data[index]) << (8 * index); + return value; +} + +template <typename Filesystem> +bool presence(Filesystem* fs, const char* path, bool& present) { + return filePresence(fs, path, present); +} + +template <typename Filesystem> +bool removeArtifact(Filesystem* fs, const char* path) { + bool present; + if (!presence(fs, path, present)) return false; + if (!present) return true; + fs->remove(path); + return presence(fs, path, present) && !present; +} + +template <typename Filesystem> +auto openRead(Filesystem* fs, const char* path) +#if defined(RP2040_PLATFORM) + -> decltype(fs->open(path, static_cast<const char*>(nullptr))) { return fs->open(path, "r"); } +#else + -> decltype(fs->open(path)) { return fs->open(path); } +#endif + +template <typename Filesystem> +auto openWrite(Filesystem* fs, const char* path, bool append) +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + -> decltype(fs->open(path, FILE_O_WRITE)) { + (void)append; + return fs->open(path, FILE_O_WRITE); // verified temp absence makes this a fresh file +} +#elif defined(ESP32_PLATFORM) + -> decltype(fs->open(path, static_cast<const char*>(nullptr), true)) { return fs->open(path, append ? "a" : "w", true); } +#else + -> decltype(fs->open(path, static_cast<const char*>(nullptr))) { return fs->open(path, append ? "a" : "w"); } +#endif + +inline void makeHeader(uint8_t (&image)[ROOM_HISTORY_HEADER_SIZE]) { + const uint8_t prefix[] = { 'R', 'H', 'J', 1, 192, 0, + ROOM_HISTORY_MAX_RECORDS, ROOM_HISTORY_JOURNAL_RECORDS }; + memcpy(image, prefix, sizeof(prefix)); + put32(image + 8, crcBytes(image, 8)); +} + +template <typename File> +bool readHeader(File& file) { + uint8_t actual[ROOM_HISTORY_HEADER_SIZE], expected[ROOM_HISTORY_HEADER_SIZE]; + makeHeader(expected); + return static_cast<size_t>(file.read(actual, sizeof(actual))) == sizeof(actual) + && memcmp(actual, expected, sizeof(actual)) == 0; +} + +inline bool encodeRecord(const RoomHistoryRecord& record, + uint8_t (&image)[ROOM_HISTORY_RECORD_SIZE]) { + size_t length = 0; + while (length < ROOM_HISTORY_TEXT_CAPACITY && record.text[length] != 0) ++length; + bool author_nonzero = false; + for (uint8_t byte : record.author) author_nonzero |= byte != 0; + if (length == ROOM_HISTORY_TEXT_CAPACITY || record.timestamp == 0 || !author_nonzero) return false; + memset(image, 0, sizeof(image)); + memcpy(image, record.author, sizeof(record.author)); + put32(image + 32, record.timestamp); + memcpy(image + 36, record.text, length); + put32(image + 188, crcBytes(image, 188)); + return true; +} + +enum class RecordRead { Valid, Corrupt, Unreadable }; + +// An incomplete physical tail is detected from file size by scanJournal. A +// short read inside a physically present record is an I/O failure, not proof +// that the record is corrupt and may safely be discarded. +template <typename File> +RecordRead readRecord(File& file, RoomHistoryRecord* record = nullptr, + const uint8_t* expected = nullptr) { + uint8_t image[ROOM_HISTORY_RECORD_SIZE]; + if (static_cast<size_t>(file.read(image, sizeof(image))) != sizeof(image)) return RecordRead::Unreadable; + bool author_nonzero = false; + for (size_t index = 0; index < 32; ++index) author_nonzero |= image[index] != 0; + bool terminated = false; + for (size_t index = 36; index < 188; ++index) { + if (terminated && image[index] != 0) return RecordRead::Corrupt; + if (image[index] == 0) terminated = true; + } + if (!author_nonzero || get32(image + 32) == 0 || !terminated + || get32(image + 188) != crcBytes(image, 188)) return RecordRead::Corrupt; + if (expected && memcmp(image, expected, sizeof(image)) != 0) return RecordRead::Corrupt; + if (record) { + memcpy(record->author, image, sizeof(record->author)); + record->timestamp = get32(image + 32); + memcpy(record->text, image + 36, sizeof(record->text)); + } + return RecordRead::Valid; +} + +template <typename Filesystem> +bool scanJournal(Filesystem* fs, const char* path, RoomHistoryState& result) { + auto file = openRead(fs, path); + if (!file) return false; + const size_t size = file.size(); + bool valid = !file.isDirectory() && size >= ROOM_HISTORY_HEADER_SIZE + && size <= ROOM_HISTORY_MAX_FILE_SIZE && readHeader(file); + RoomHistoryState staged; + staged.present = true; + const size_t physical_records = size >= ROOM_HISTORY_HEADER_SIZE + ? (size - ROOM_HISTORY_HEADER_SIZE) / ROOM_HISTORY_RECORD_SIZE : 0; + for (size_t index = 0; valid && index < physical_records; ++index) { + const RecordRead read = readRecord(file); + if (read == RecordRead::Unreadable) valid = false; + else if (read == RecordRead::Corrupt) { + // A failed append can leave only the final physical record corrupt. + // Later complete records prove this is older storage corruption: keep + // the entire primary and do not publish a prefix for automatic repair. + valid = index + 1 == physical_records; + break; + } + else ++staged.total_records; + } + staged.count = staged.total_records > ROOM_HISTORY_MAX_RECORDS + ? ROOM_HISTORY_MAX_RECORDS : staged.total_records; + staged.needs_repair = size != ROOM_HISTORY_HEADER_SIZE + + staged.total_records * ROOM_HISTORY_RECORD_SIZE; + file.close(); + if (valid) result = staged; + return valid; +} + +template <typename Filesystem> +bool readConfig(Filesystem* fs, const char* path, bool& enabled) { + auto file = openRead(fs, path); + if (!file) return false; + uint8_t image[16]; + bool valid = !file.isDirectory() && file.size() == sizeof(image) + && static_cast<size_t>(file.read(image, sizeof(image))) == sizeof(image); + file.close(); + const uint8_t magic[] = { 'R', 'H', 'C', 1 }; + valid = valid && memcmp(image, magic, sizeof(magic)) == 0 && image[4] <= 1 + && get32(image + 12) == crcBytes(image, 12); + for (size_t index = 5; valid && index < 12; ++index) valid = image[index] == 0; + if (valid) enabled = image[4] != 0; + return valid; +} + +// Primary authority is preserved even when corrupt or from a future format. +// Only an absent primary permits restoring backup; temp is never promoted. +template <typename Filesystem, typename Validator> +bool recover(Filesystem* fs, const char* primary, const char* temp, + const char* backup, Validator validate, bool strict, bool& found) { + if (!presence(fs, primary, found)) return false; + if (found) { + if (!validate(primary)) return false; + const bool cleaned_temp = removeArtifact(fs, temp); + const bool cleaned_backup = removeArtifact(fs, backup); + return !strict || (cleaned_temp && cleaned_backup); + } + bool had_backup; + if (!presence(fs, backup, had_backup)) return false; + if (had_backup) { + if (!validate(backup) || !fs->rename(backup, primary)) return false; + found = true; + } + const bool cleaned_temp = removeArtifact(fs, temp); + return !strict || cleaned_temp; +} + +template <typename Filesystem> +bool publish(Filesystem* fs, const char* primary, const char* temp, const char* backup) { + bool had_primary; + if (!presence(fs, primary, had_primary) + || (had_primary && !fs->rename(primary, backup))) { + removeArtifact(fs, temp); + return false; + } + if (!fs->rename(temp, primary)) { + if (had_primary) fs->rename(backup, primary); + removeArtifact(fs, temp); + return false; + } + removeArtifact(fs, backup); // post-commit cleanup cannot turn success into failure + return true; +} + +struct EmptyProvider { + bool operator()(uint8_t, RoomHistoryRecord&) const { return false; } +}; + +} // namespace room_history_detail + +template <typename Filesystem> +bool loadRoomHistoryConfig(Filesystem* fs, bool& enabled) { + if (!fs) return false; + bool found, staged = false; + const auto validate = [&](const char* path) { return room_history_detail::readConfig(fs, path, staged); }; + if (!room_history_detail::recover(fs, ROOM_HISTORY_CONFIG_PATH, + ROOM_HISTORY_CONFIG_TEMP_PATH, ROOM_HISTORY_CONFIG_BACKUP_PATH, + validate, false, found)) return false; + enabled = staged; + return true; +} + +template <typename Filesystem> +bool saveRoomHistoryConfig(Filesystem* fs, bool enabled) { + if (!fs) return false; + bool found, previous = false; + const auto validate = [&](const char* path) { return room_history_detail::readConfig(fs, path, previous); }; + if (!room_history_detail::recover(fs, ROOM_HISTORY_CONFIG_PATH, + ROOM_HISTORY_CONFIG_TEMP_PATH, ROOM_HISTORY_CONFIG_BACKUP_PATH, + validate, true, found)) return false; + uint8_t image[16] = { 'R', 'H', 'C', 1, uint8_t(enabled ? 1 : 0) }; + room_history_detail::put32(image + 12, room_history_detail::crcBytes(image, 12)); + auto file = room_history_detail::openWrite(fs, ROOM_HISTORY_CONFIG_TEMP_PATH, false); + if (!file) return false; + if (file.isDirectory()) { + file.close(); + room_history_detail::removeArtifact(fs, ROOM_HISTORY_CONFIG_TEMP_PATH); + return false; + } + const bool written = file.write(image, sizeof(image)) == sizeof(image); + file.flush(); file.close(); + bool readback = !enabled; + if (!written || !room_history_detail::readConfig(fs, ROOM_HISTORY_CONFIG_TEMP_PATH, readback) + || readback != enabled) { + room_history_detail::removeArtifact(fs, ROOM_HISTORY_CONFIG_TEMP_PATH); + return false; + } + return room_history_detail::publish(fs, ROOM_HISTORY_CONFIG_PATH, + ROOM_HISTORY_CONFIG_TEMP_PATH, ROOM_HISTORY_CONFIG_BACKUP_PATH); +} + +// Validate the complete readable prefix before calling the consumer. The +// consumer receives only the newest 32 records, oldest first. A later read or +// consumer failure leaves state unchanged; boot callers must clear any partial +// destination ring. No live ring reload is needed for ordinary appends. +template <typename Filesystem, typename Consumer> +bool loadRoomHistory(Filesystem* fs, RoomHistoryState& state, Consumer consume) { + if (!fs) return false; + RoomHistoryState staged; + bool found; + const auto validate = [&](const char* path) { return room_history_detail::scanJournal(fs, path, staged); }; + if (!room_history_detail::recover(fs, ROOM_HISTORY_PATH, ROOM_HISTORY_TEMP_PATH, + ROOM_HISTORY_BACKUP_PATH, validate, false, found)) return false; + if (found && staged.count != 0) { + auto file = room_history_detail::openRead(fs, ROOM_HISTORY_PATH); + if (!file) return false; + bool valid = !file.isDirectory() && room_history_detail::readHeader(file) + && file.seek(ROOM_HISTORY_HEADER_SIZE + + (staged.total_records - staged.count) * ROOM_HISTORY_RECORD_SIZE); + RoomHistoryRecord record; + for (uint8_t index = 0; valid && index < staged.count; ++index) { + valid = room_history_detail::readRecord(file, &record) == room_history_detail::RecordRead::Valid + && consume(index, record); + } + file.close(); + if (!valid) return false; + } + state = staged; + return true; +} + +// Provider(index, record) supplies an immutable, oldest-first RAM snapshot. +// One record at a time is rendered and read back; there is no second 6 KB ring. +// Used to enable history, compact 64 journal entries, repair a failed tail, or +// clear with count zero. State changes only after atomic publication succeeds. +template <typename Filesystem, typename Provider> +bool saveRoomHistorySnapshot(Filesystem* fs, RoomHistoryState& state, + uint8_t count, Provider provide) { + if (!fs || count > ROOM_HISTORY_MAX_RECORDS) return false; + RoomHistoryState old; + bool found; + const auto validate = [&](const char* path) { return room_history_detail::scanJournal(fs, path, old); }; + if (!room_history_detail::recover(fs, ROOM_HISTORY_PATH, ROOM_HISTORY_TEMP_PATH, + ROOM_HISTORY_BACKUP_PATH, validate, true, found)) return false; + auto file = room_history_detail::openWrite(fs, ROOM_HISTORY_TEMP_PATH, false); + if (!file) return false; + if (file.isDirectory()) { + file.close(); + room_history_detail::removeArtifact(fs, ROOM_HISTORY_TEMP_PATH); + return false; + } + uint8_t header[ROOM_HISTORY_HEADER_SIZE]; room_history_detail::makeHeader(header); + bool valid = file.write(header, sizeof(header)) == sizeof(header); + RoomHistoryRecord record; + uint8_t image[ROOM_HISTORY_RECORD_SIZE]; + for (uint8_t index = 0; valid && index < count; ++index) { + valid = provide(index, record) && room_history_detail::encodeRecord(record, image) + && file.write(image, sizeof(image)) == sizeof(image); + } + file.flush(); file.close(); + if (valid) { + auto verify = room_history_detail::openRead(fs, ROOM_HISTORY_TEMP_PATH); + if (!verify) valid = false; + else { + valid = !verify.isDirectory() + && verify.size() == ROOM_HISTORY_HEADER_SIZE + count * ROOM_HISTORY_RECORD_SIZE + && room_history_detail::readHeader(verify); + for (uint8_t index = 0; valid && index < count; ++index) { + valid = provide(index, record) && room_history_detail::encodeRecord(record, image) + && room_history_detail::readRecord(verify, nullptr, image) == room_history_detail::RecordRead::Valid; + } + verify.close(); + } + } + if (!valid) { + room_history_detail::removeArtifact(fs, ROOM_HISTORY_TEMP_PATH); + return false; + } + if (!room_history_detail::publish(fs, ROOM_HISTORY_PATH, ROOM_HISTORY_TEMP_PATH, + ROOM_HISTORY_BACKUP_PATH)) return false; + state.count = state.total_records = count; + state.needs_repair = false; + state.present = true; + return true; +} + +template <typename Filesystem> +bool clearRoomHistory(Filesystem* fs, RoomHistoryState& state) { + return saveRoomHistorySnapshot(fs, state, 0, room_history_detail::EmptyProvider{}); +} + +// Single-writer main-loop use. A failed append can leave an uncommitted tail; +// mark it for snapshot repair from the unchanged trusted live ring before a +// retry. Never ACK or update the message replay cache when this returns false. +template <typename Filesystem> +bool appendRoomHistory(Filesystem* fs, RoomHistoryState& state, + const uint8_t* author, uint32_t timestamp, const char* text) { + if (!fs || !author || !text || state.needs_repair + || state.total_records >= ROOM_HISTORY_JOURNAL_RECORDS) return false; + RoomHistoryRecord record; + memcpy(record.author, author, sizeof(record.author)); + record.timestamp = timestamp; + size_t length = 0; + while (length < ROOM_HISTORY_TEXT_CAPACITY && text[length] != 0) ++length; + if (length == ROOM_HISTORY_TEXT_CAPACITY) return false; + memset(record.text, 0, sizeof(record.text)); memcpy(record.text, text, length); + uint8_t image[ROOM_HISTORY_RECORD_SIZE]; + if (!room_history_detail::encodeRecord(record, image)) return false; + RoomHistoryState disk; + bool found; + const auto validate = [&](const char* path) { return room_history_detail::scanJournal(fs, path, disk); }; + if (!room_history_detail::recover(fs, ROOM_HISTORY_PATH, ROOM_HISTORY_TEMP_PATH, + ROOM_HISTORY_BACKUP_PATH, validate, true, found)) { + state.needs_repair = true; + return false; + } + if (!state.present) { + // A caller must load an existing journal before appending. In particular, + // never turn a missed/failed load into an implicit clear of accepted posts. + if (found || state.count != 0 || state.total_records != 0) { + state.needs_repair = true; + return false; + } + if (!clearRoomHistory(fs, state)) return false; + disk = state; + found = true; + } + if (!found || disk.needs_repair || disk.total_records != state.total_records + || disk.count != state.count) { + state.needs_repair = true; + return false; + } + const size_t expected_size = ROOM_HISTORY_HEADER_SIZE + + state.total_records * ROOM_HISTORY_RECORD_SIZE; + auto file = room_history_detail::openWrite(fs, ROOM_HISTORY_PATH, true); + if (!file) { state.needs_repair = true; return false; } + if (file.isDirectory()) { file.close(); state.needs_repair = true; return false; } + const bool written = file.size() == expected_size + && file.write(image, sizeof(image)) == sizeof(image); + file.flush(); file.close(); + bool verified = false; + if (written) { + auto verify = room_history_detail::openRead(fs, ROOM_HISTORY_PATH); + if (verify) { + verified = !verify.isDirectory() + && verify.size() == expected_size + ROOM_HISTORY_RECORD_SIZE + && room_history_detail::readHeader(verify) && verify.seek(expected_size) + && room_history_detail::readRecord(verify, nullptr, image) == room_history_detail::RecordRead::Valid; + verify.close(); + } + } + if (!verified) { state.needs_repair = true; return false; } + ++state.total_records; + state.count = state.total_records > ROOM_HISTORY_MAX_RECORDS ? ROOM_HISTORY_MAX_RECORDS : state.total_records; + return true; +} + +} // namespace mesh diff --git a/src/helpers/RoomTopicStore.h b/src/helpers/RoomTopicStore.h index 56b4dcac9..a41802f34 100644 --- a/src/helpers/RoomTopicStore.h +++ b/src/helpers/RoomTopicStore.h @@ -3,9 +3,7 @@ #include <stddef.h> #include <stdint.h> #include <string.h> -#if defined(ESP32_PLATFORM) #include "FilePresence.h" -#endif namespace mesh { @@ -30,12 +28,7 @@ inline uint32_t crcByte(uint32_t crc, uint8_t value) { template <typename Filesystem> bool presence(Filesystem* fs, const char* path, bool& present) { -#if defined(ESP32_PLATFORM) return filePresence(fs, path, present); -#else - present = fs->exists(path); - return true; -#endif } template <typename Filesystem> diff --git a/src/helpers/RoomWebJson.h b/src/helpers/RoomWebJson.h new file mode 100644 index 000000000..6e86efa66 --- /dev/null +++ b/src/helpers/RoomWebJson.h @@ -0,0 +1,112 @@ +#pragma once + +#include <ArduinoJson.h> +#include <stdint.h> +#include <stddef.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +namespace mesh { + +// JsonDocument v7 is dynamically allocated. Bound actual allocations rather +// than relying on the deprecated StaticJsonDocument capacity shim. +class RoomJsonAllocator : public ArduinoJson::Allocator { + union Header { max_align_t alignment; size_t bytes; }; + size_t used_ = 0; +public: + static constexpr size_t BUDGET = 8192; + void* allocate(size_t bytes) override { + if (bytes > BUDGET - used_ || sizeof(Header) > BUDGET - used_ - bytes) return nullptr; + auto* header = static_cast<Header*>(malloc(sizeof(Header) + bytes)); + if (!header) return nullptr; + header->bytes = bytes; used_ += sizeof(Header) + bytes; return header + 1; + } + void deallocate(void* ptr) override { + if (!ptr) return; + auto* header = static_cast<Header*>(ptr) - 1; + used_ -= sizeof(Header) + header->bytes; free(header); + } + void* reallocate(void* ptr, size_t bytes) override { + if (!ptr) return allocate(bytes); + auto* header = static_cast<Header*>(ptr) - 1; + const size_t remaining = used_ - sizeof(Header) - header->bytes; + if (bytes > BUDGET - remaining || sizeof(Header) > BUDGET - remaining - bytes) return nullptr; + header = static_cast<Header*>(realloc(header, sizeof(Header) + bytes)); + if (!header) return nullptr; + header->bytes = bytes; used_ = remaining + sizeof(Header) + bytes; return header + 1; + } +}; + +// Replies use their fixed mailbox buffer, with no second JSON heap pool. +class RoomJsonWriter { + char* output_; + size_t capacity_, used_ = 0; + bool valid_ = true; +public: + RoomJsonWriter(char* output, size_t capacity) : output_(output), capacity_(capacity) { + if (!output || !capacity) valid_ = false; + else output[0] = 0; + } + void raw(const char* text) { + const size_t length = strlen(text); + if (!valid_ || length >= capacity_ - used_) { valid_ = false; return; } + memcpy(output_ + used_, text, length + 1); used_ += length; + } + void number(uint32_t value) { char text[12]; snprintf(text, sizeof(text), "%lu", (unsigned long)value); raw(text); } + void string(const char* text) { + raw("\""); + for (const auto* p = reinterpret_cast<const uint8_t*>(text); valid_ && *p; ++p) { + char escaped[7] = {}; + if (*p == '"' || *p == '\\') { escaped[0] = '\\'; escaped[1] = *p; } + else if (*p < 0x20) snprintf(escaped, sizeof(escaped), "\\u%04x", unsigned(*p)); + else escaped[0] = *p; + raw(escaped); + } + raw("\""); + } + bool finish() { + if (!valid_ && output_ && capacity_ >= 29) strcpy(output_, "{\"error\":\"reply too large\"}"); + return valid_; + } +}; + +inline void roomEncodeBase64(const uint8_t* bytes, size_t length, char* output) { + static const char alphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + size_t used = 0; + for (size_t i = 0; i < length; i += 3) { + const uint32_t value = (uint32_t(bytes[i]) << 16) + | (i + 1 < length ? uint32_t(bytes[i + 1]) << 8 : 0) + | (i + 2 < length ? bytes[i + 2] : 0); + output[used++] = alphabet[(value >> 18) & 63]; output[used++] = alphabet[(value >> 12) & 63]; + output[used++] = i + 1 < length ? alphabet[(value >> 6) & 63] : '='; + output[used++] = i + 2 < length ? alphabet[value & 63] : '='; + } + output[used] = 0; +} + +inline bool roomDecodeBase64(const char* text, uint8_t* bytes, size_t capacity, size_t& length) { + length = 0; + const size_t encoded = strlen(text); + if (encoded % 4 != 0 || encoded > ((capacity + 2) / 3) * 4) return false; + auto digit = [](char c) -> int { + if (c >= 'A' && c <= 'Z') return c - 'A'; if (c >= 'a' && c <= 'z') return c - 'a' + 26; + if (c >= '0' && c <= '9') return c - '0' + 52; if (c == '+') return 62; if (c == '/') return 63; + return -1; + }; + for (size_t i = 0; i < encoded; i += 4) { + int values[4]; + for (size_t j = 0; j < 4; ++j) values[j] = text[i + j] == '=' ? 0 : digit(text[i + j]); + if (values[0] < 0 || values[1] < 0 || values[2] < 0 || values[3] < 0 + || text[i] == '=' || text[i + 1] == '=' + || (text[i + 2] == '=' && text[i + 3] != '=') + || (i + 4 != encoded && (text[i + 2] == '=' || text[i + 3] == '='))) return false; + const size_t n = text[i + 2] == '=' ? 1 : (text[i + 3] == '=' ? 2 : 3); + if (length + n > capacity || (n == 1 && (values[1] & 15)) || (n == 2 && (values[2] & 3))) return false; + const uint32_t value = (uint32_t(values[0]) << 18) | (uint32_t(values[1]) << 12) + | (uint32_t(values[2]) << 6) | uint32_t(values[3]); + for (size_t j = 0; j < n; ++j) bytes[length++] = uint8_t(value >> (16 - 8 * j)); + } + return true; +} +} // namespace mesh diff --git a/src/helpers/RoomWebMailbox.h b/src/helpers/RoomWebMailbox.h new file mode 100644 index 000000000..62032247e --- /dev/null +++ b/src/helpers/RoomWebMailbox.h @@ -0,0 +1,111 @@ +#pragma once + +#include <stdint.h> +#include <stddef.h> +#include <string.h> + +namespace mesh { + +// One loop-owned operation and eight bounded browser sequence floors. Caller +// serializes submit/read/begin/finish with its mutex; processing runs outside +// that mutex. Running input and output cannot be recycled by an HTTP handler. +// Sequence floors expire after twenty minutes and on reboot. Clients never +// retry a write after that window or across a changed boot identifier. +class RoomWebMailbox { +public: + static constexpr size_t INPUT_CAPACITY = 4097; + static constexpr size_t OUTPUT_CAPACITY = 2048; + static constexpr uint32_t OWNER_TTL_MS = 1200000; + static constexpr uint32_t RESULT_TTL_MS = 30000; + enum class Result { Accepted, Busy, Invalid, Completed, Mismatch }; + enum class State { Idle, Pending, Running, Done }; +private: + struct Owner { uint8_t token[32]; uint32_t sequence = 0, seen = 0; } owners_[8] = {}; + uint8_t token_[32] = {}, digest_[32] = {}; + uint32_t sequence_ = 0, finished_ = 0; + State state_ = State::Idle; + bool delivered_ = false; + bool owner_was_new_ = false; + char input_[INPUT_CAPACITY] = {}, output_[OUTPUT_CAPACITY] = {}; +public: + static bool decodeToken(const char* text, uint8_t (&out)[32]) { + if (!text || strlen(text) != 64) return false; + uint8_t parsed[32], nonzero = 0; + for (size_t i = 0; i < 32; ++i) { + unsigned byte = 0; + for (size_t n = 0; n < 2; ++n) { + char c = text[i * 2 + n]; + unsigned value; + if (c >= '0' && c <= '9') value = c - '0'; + else if (c >= 'a' && c <= 'f') value = c - 'a' + 10; + else if (c >= 'A' && c <= 'F') value = c - 'A' + 10; + else return false; + byte = (byte << 4) | value; + } + parsed[i] = uint8_t(byte); nonzero |= parsed[i]; + } + if (!nonzero) return false; + memcpy(out, parsed, sizeof(parsed)); return true; + } + State state() const { return state_; } + uint32_t sequence() const { return sequence_; } + Result submit(const uint8_t (&token)[32], uint32_t sequence, + const uint8_t (&digest)[32], const char* body, size_t length, uint32_t now) { + if (!body || !sequence || !length || length >= INPUT_CAPACITY + || memchr(body, 0, length)) return Result::Invalid; + if (state_ != State::Idle && sequence == sequence_ && memcmp(token_, token, 32) == 0) { + return memcmp(digest_, digest, 32) == 0 ? Result::Accepted : Result::Mismatch; + } + Owner* owner = nullptr; + Owner* empty = nullptr; + for (auto& entry : owners_) { + if (entry.sequence && uint32_t(now - entry.seen) < OWNER_TTL_MS) { + if (memcmp(entry.token, token, 32) == 0) owner = &entry; + } else if (!empty) empty = &entry; + } + if (owner && sequence <= owner->sequence) return Result::Completed; + if (state_ == State::Pending || state_ == State::Running + || (state_ == State::Done && !delivered_ + && uint32_t(now - finished_) < RESULT_TTL_MS)) return Result::Busy; + owner_was_new_ = !owner; + if (!owner) owner = empty; + if (!owner) return Result::Busy; + memcpy(owner->token, token, 32); owner->sequence = sequence; owner->seen = now; + memcpy(token_, token, 32); memcpy(digest_, digest, 32); + memcpy(input_, body, length); input_[length] = 0; output_[0] = 0; + sequence_ = sequence; delivered_ = false; state_ = State::Pending; + return Result::Accepted; + } + bool begin() { + if (state_ != State::Pending) return false; + state_ = State::Running; return true; + } + char* input() { return input_; } + char* output() { return output_; } + const uint8_t* token() const { return token_; } + void finish(uint32_t now, bool authenticated = true) { + if (state_ != State::Running) return; + memset(input_, 0, sizeof(input_)); // passwords and article data have served their purpose + output_[OUTPUT_CAPACITY - 1] = 0; + finished_ = now; state_ = State::Done; + // Unauthenticated traffic must not occupy all eight sequence-floor slots + // for twenty minutes. Existing authenticated floors are never erased. + if (!authenticated && owner_was_new_) { + for (auto& owner : owners_) if (owner.sequence == sequence_ && memcmp(owner.token, token_, 32) == 0) owner = Owner{}; + } + } + // Output must be copied while caller holds the mutex. Wrong owners never + // learn whether another user's operation is pending or completed. + const char* read(const uint8_t (&token)[32], uint32_t sequence, bool& pending) { + pending = false; + if (state_ == State::Idle || sequence_ != sequence || memcmp(token_, token, 32) != 0) return nullptr; + if (state_ != State::Done) { pending = true; return nullptr; } + return output_; + } + void delivered(const uint8_t (&token)[32], uint32_t sequence) { + if (state_ == State::Done && sequence_ == sequence && memcmp(token_, token, 32) == 0) delivered_ = true; + } +}; +static_assert(sizeof(RoomWebMailbox) <= 6656, "Update the room WebConfig heap budget"); + +} // namespace mesh diff --git a/src/helpers/esp32/RoomWebHtml.h b/src/helpers/esp32/RoomWebHtml.h new file mode 100644 index 000000000..1bc8a9497 --- /dev/null +++ b/src/helpers/esp32/RoomWebHtml.h @@ -0,0 +1,206 @@ +#pragma once + +#include <Arduino.h> + +// Offline browser room client, served by the existing WebConfig listener. +// No radio private key is imported; a separate random browser token identifies +// posts and moderation. Keep untrusted room text out of HTML interpolation. +static const char ROOM_WEB_HTML[] PROGMEM = R"roomhtml(<!doctype html> +<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"> +<title>MeshCore Room +

MeshCore Room

+

Chat and selected notices over local Wi-Fi. This page also works without internet access.

+
+ + +
+

+

Radio configuration

+)roomhtml"; diff --git a/src/helpers/esp32/WebConfigServer.cpp b/src/helpers/esp32/WebConfigServer.cpp index fcc8cd429..aed6e541c 100644 --- a/src/helpers/esp32/WebConfigServer.cpp +++ b/src/helpers/esp32/WebConfigServer.cpp @@ -32,6 +32,8 @@ static_assert(sizeof(WEBCONFIG_AP_PREFIX) <= 28, #endif #include "WebConfigHtml.h" +#include "RoomWebHtml.h" +#include #include #include "WebTerminalStream.h" #include @@ -339,6 +341,9 @@ WebConfigServer::WebConfigServer(Callbacks* callbacks, void* mqtt_prefs, bool ow _standalone_wifi(standalone_wifi || mqtt_prefs == nullptr || !owns_wifi), _pub_key(pub_key), _fw_ver(fw_ver), _build_date(build_date), _role(role), _board_name(board_name) { _mux = xSemaphoreCreateMutex(); + if (_mux && _cb && _cb->supportsRoomService()) { + _room_mailbox = new (std::nothrow) mesh::RoomWebMailbox; + } _cli_enabled = loadCliEnabled(true); #ifdef WITH_MQTT_BRIDGE @@ -364,6 +369,7 @@ WebConfigServer::WebConfigServer(Callbacks* callbacks, void* mqtt_prefs, bool ow WebConfigServer::~WebConfigServer() { detachRoutes(); closeTerminal(); + delete _room_mailbox; if (_mux) vSemaphoreDelete(_mux); } @@ -1043,6 +1049,8 @@ bool WebConfigServer::stopForOTA(char reply[]) { void WebConfigServer::finalizeTeardown() { closeTerminal(); + delete _room_mailbox; + _room_mailbox = nullptr; // Async requests keep a pointer to their server until disconnect. Retain the // listener and route table for the firmware lifetime; only detach and reclaim // the per-session state. @@ -1127,6 +1135,7 @@ void WebConfigServer::tick(uint32_t now) { if (!_board_cmds_probed) probeBoardCommands(); serviceTerminal(now); + serviceRoom(now); // A primary ESP-NOW radio and its WiFi station cannot remain on different // channels. Reject a router-driven channel move (for example a CSA) and put @@ -1613,6 +1622,15 @@ void WebConfigServer::dispatchRequest(AsyncWebServerRequest* req, } void WebConfigServer::registerRoutes() { + _server->on("/room", HTTP_GET, [](AsyncWebServerRequest* r) { + dispatchRequest(r, &WebConfigServer::handleRoomPage); + }); + _server->on("/api/room/result", HTTP_POST, [](AsyncWebServerRequest* r) { + dispatchRequest(r, &WebConfigServer::handleRoomResult); + }); + _server->on("/api/room", HTTP_POST, [](AsyncWebServerRequest* r) { + dispatchRequest(r, &WebConfigServer::handleRoomPost); + }, NULL, collectBody); _server->on("/ui", HTTP_GET, [](AsyncWebServerRequest* r) { dispatchRequest(r, &WebConfigServer::handleUi); }); @@ -1676,6 +1694,105 @@ void WebConfigServer::registerRoutes() { }); } +void WebConfigServer::serviceRoom(uint32_t now) { + if (!_room_mailbox && _mux && _cb && _cb->supportsRoomService()) { + WCLock lock(_mux); + _room_mailbox = new (std::nothrow) mesh::RoomWebMailbox; + } + mesh::RoomWebMailbox* operation = nullptr; + { + WCLock lock(_mux); + if (_room_mailbox && _room_mailbox->begin()) operation = _room_mailbox; + } + if (!operation) return; + // Async submit/read may run here, but cannot recycle a Running operation. + // No filesystem, room client or radio mutation runs on async_tcp. + _cb->processRoomRequest(operation->token(), operation->sequence(), operation->input(), + operation->output(), mesh::RoomWebMailbox::OUTPUT_CAPACITY); + { + WCLock lock(_mux); + operation->finish(now, _cb->roomRequestAuthenticated()); + } +} + +static bool roomRequestIdentity(AsyncWebServerRequest* req, uint8_t (&token)[32], uint32_t& sequence) { + if (!req || !req->hasHeader("X-Room-Token") || !req->hasHeader("X-Room-Seq")) return false; + if (req->hasHeader("Origin")) { + if (!req->hasHeader("Host") || req->getHeader("Origin")->value() + != String("http://") + req->getHeader("Host")->value()) return false; + } + return mesh::RoomWebMailbox::decodeToken(req->getHeader("X-Room-Token")->value().c_str(), token) + && mesh::cli::parseUnsignedIntegerStrict(req->getHeader("X-Room-Seq")->value().c_str(), sequence) + && sequence != 0; +} + +void WebConfigServer::handleRoomPage(AsyncWebServerRequest* req) { + if (_mode == MODE_OFF || !_cb || !_cb->supportsRoomService()) { req->send(404); return; } + _last_activity = millis(); + auto* response = new (std::nothrow) WebConfigPacedProgmemResponse( + "text/html; charset=utf-8", reinterpret_cast(ROOM_WEB_HTML), sizeof(ROOM_WEB_HTML) - 1); + if (!response) { req->send(503); return; } + response->addHeader("Cache-Control", "no-store"); + response->addHeader("X-Content-Type-Options", "nosniff"); + response->addHeader("Referrer-Policy", "no-referrer"); + req->send(response); +} + +void WebConfigServer::handleRoomPost(AsyncWebServerRequest* req) { + uint8_t token[32]; uint32_t sequence; + if (!roomRequestIdentity(req, token, sequence)) { + req->send(400, "application/json", "{\"error\":\"invalid room identity\"}"); return; + } + if (req->contentType() != "application/json" || !req->_tempObject + || req->contentLength() == 0 || req->contentLength() > MAX_BODY) { + req->send(400, "application/json", "{\"error\":\"invalid room body\"}"); return; + } + uint8_t digest[32]; + mesh::Utils::sha256(digest, sizeof(digest), (const uint8_t*)req->_tempObject, req->contentLength()); + mesh::RoomWebMailbox::Result result; + { + WCLock lock(_mux); + if (!_room_mailbox || _stopping || _mode == MODE_OFF) { + req->send(503, "application/json", "{\"error\":\"room service unavailable\"}"); return; + } + result = _room_mailbox->submit(token, sequence, digest, (const char*)req->_tempObject, + req->contentLength(), millis()); + // Do not keep a second password/article buffer while tick parses the + // accepted copy. The request destructor tolerates a null temporary object. + memset(req->_tempObject, 0, req->contentLength()); + free(req->_tempObject); + req->_tempObject = nullptr; + } + if (result == mesh::RoomWebMailbox::Result::Accepted) { + _last_activity = millis(); + req->send(202, "application/json", "{\"accepted\":true}"); + } else if (result == mesh::RoomWebMailbox::Result::Busy) { + req->send(429, "application/json", "{\"error\":\"room service busy\"}"); + } else { + req->send(409, "application/json", "{\"error\":\"request already completed or changed; refresh to check\"}"); + } +} + +void WebConfigServer::handleRoomResult(AsyncWebServerRequest* req) { + uint8_t token[32]; uint32_t sequence; + if (!roomRequestIdentity(req, token, sequence)) { req->send(400); return; } + WCLock lock(_mux); + if (!_room_mailbox || _stopping || _mode == MODE_OFF) { req->send(503); return; } + bool pending; + const char* result = _room_mailbox->read(token, sequence, pending); + if (pending) req->send(202, "application/json", "{\"pending\":true}"); + else if (!result) req->send(404, "application/json", "{\"error\":\"room result unavailable; refresh to check\"}"); + else { + String body(result); + if (body.length() != strlen(result)) { req->send(503); return; } + auto* response = req->beginResponse(200, "application/json", body); + if (!response) { req->send(503); return; } + response->addHeader("Cache-Control", "no-store"); + req->send(response); + _room_mailbox->delivered(token, sequence); + } +} + // Accumulate a small JSON body into request->_tempObject (freed automatically // by the request destructor). Oversized bodies are left unbuffered and // rejected in the completion handler. diff --git a/src/helpers/esp32/WebConfigServer.h b/src/helpers/esp32/WebConfigServer.h index 903918288..ff91b7430 100644 --- a/src/helpers/esp32/WebConfigServer.h +++ b/src/helpers/esp32/WebConfigServer.h @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -161,6 +162,12 @@ public: virtual void buildStatsJson(char* buf, size_t buf_size) = 0; // Teardown finished (server + DNS freed, WiFi mode restored). virtual void onWebConfigStopped() {} + virtual bool supportsRoomService() const { return false; } + // Input may be parsed in place. Authentication, storage and radio work + // happen only on tick(), never on async_tcp. + virtual void processRoomRequest(const uint8_t* token, uint32_t sequence, + char* request, char* response, size_t capacity) {} + virtual bool roomRequestAuthenticated() const { return false; } }; // mqtt_prefs is an MQTTPrefs* on WITH_MQTT_BRIDGE builds and NULL otherwise. @@ -339,6 +346,7 @@ private: char _setup_wifi_handoff_ip[16] = {0}; BatchEntry _batch[MAX_BATCH]; WebTerminalStream* _terminal = NULL; + mesh::RoomWebMailbox* _room_mailbox = NULL; // LAN-mode session (single slot; new login evicts the old session) char _session_token[33] = {0}; @@ -378,6 +386,7 @@ private: void drainBatch(uint32_t now); void probeBoardCommands(); void serviceTerminal(uint32_t now); + void serviceRoom(uint32_t now); void closeTerminal(); void serviceSetupWiFiHandoff(uint32_t now); void finishBatch(uint32_t now); @@ -404,6 +413,9 @@ private: void handlePresets(AsyncWebServerRequest* req); void handleReboot(AsyncWebServerRequest* req); void handlePortalExit(AsyncWebServerRequest* req); + void handleRoomPage(AsyncWebServerRequest* req); + void handleRoomPost(AsyncWebServerRequest* req); + void handleRoomResult(AsyncWebServerRequest* req); void handleNotFound(AsyncWebServerRequest* req); }; diff --git a/test/fixtures/client_acl_infrastructure/test_client_acl_infrastructure.cpp b/test/fixtures/client_acl_infrastructure/test_client_acl_infrastructure.cpp index 9f02fe062..42fe0633c 100644 --- a/test/fixtures/client_acl_infrastructure/test_client_acl_infrastructure.cpp +++ b/test/fixtures/client_acl_infrastructure/test_client_acl_infrastructure.cpp @@ -3,6 +3,9 @@ #include #include #include +#include "filesystem.h" +#include +#include #include #include @@ -87,11 +90,13 @@ struct RegionMap { struct ClientInfo { mesh::Identity id; uint8_t permissions = 3; + bool permissions_are_explicit = false; uint8_t out_path_len = 0, out_path[MAX_PATH_SIZE] = {}; uint32_t last_timestamp = 0, last_activity = 0; struct { struct { uint32_t sync_since = 0, pending_ack = 0, pending_topic_revision = 0; uint8_t push_failures = 0; + uint16_t poll_quota_used = 0, post_quota_used = 0; } room; } extra; bool isAdmin() const { return (permissions & PERM_ACL_ROLE_MASK) == PERM_ACL_ADMIN; } }; @@ -106,6 +111,7 @@ struct Clock : mesh::RTCClock { }; #ifdef TEST_SENSOR #define TARGET_CLASS SensorMesh +struct MetadataFixtureUse { MetadataFixtureUse() { (void)metadata_filesystem; } } metadata_fixture_use; #else #define TARGET_CLASS MyMesh #endif @@ -130,6 +136,12 @@ public: int handleRequest(ClientInfo*, uint32_t, uint8_t*, size_t, size_t = mesh::CLIENT_ACL_DIRECT_REPLY_CAPACITY); int telemetryPrefix(uint8_t*, size_t); + MemoryFS fs; + mesh::RoomAccessPolicy room_access; + mesh::LogicalMessageCache<8> recent_room_polls; + TARGET_CLASS() { metadata_filesystem = &fs; assert(room_access.load(&fs)); } + static bool saveFilter(ClientInfo*); + void serviceRoomQuotas() { room_access.serviceQuotaWindow(0, [] {}); } #endif void receive(mesh::Packet*, uint8_t*, size_t); bool admit(mesh::Packet* packet) { diff --git a/test/fixtures/official_room_app_compatibility/room_messages.cpp b/test/fixtures/official_room_app_compatibility/room_messages.cpp index bf7c52740..e94c8d7c7 100644 --- a/test/fixtures/official_room_app_compatibility/room_messages.cpp +++ b/test/fixtures/official_room_app_compatibility/room_messages.cpp @@ -16,7 +16,7 @@ struct WirePresentation { int app_target_ver; uint8_t out_frame[MAX_FRAME_SIZE] = {}; std::vector frame; - void queueMessage(const RoomContact&, uint8_t, CompanionPacket*, uint32_t, + bool queueMessage(const RoomContact&, uint8_t, CompanionPacket*, uint32_t, const uint8_t*, int, const char*, bool, uint32_t); }; #include "presentation.inc" diff --git a/test/fixtures/room_history_integration/test.cpp b/test/fixtures/room_history_integration/test.cpp new file mode 100644 index 000000000..e50d2ded2 --- /dev/null +++ b/test/fixtures/room_history_integration/test.cpp @@ -0,0 +1,308 @@ +// The production methods, plain-post gate, RTCClock and PostInfo are extracted +// unchanged. Only filesystem faults, radio ACK observation and identity hashing +// are host boundaries; the actual history store, quota and replay cache run here. +#include "../room_history_store/filesystem.h" +#include +#include +#include +#define PUB_KEY_SIZE 32 +#define MAX_POST_TEXT_LEN 151 +#define MAX_UNSYNCED_POSTS 32 +#define MAX_HASH_SIZE 8 +#define POST_SYNC_DELAY_SECS 6 +#define PUSH_NOTIFY_DELAY_MILLIS 2000 +#define TXT_TYPE_PLAIN 0 +#undef MESH_DEBUG_PRINTLN +#define MESH_DEBUG_PRINTLN(...) ((void)0) +static uint32_t ticks = 100; +static unsigned long futureMillis(unsigned long delay) { return uint32_t(ticks + delay); } +namespace mesh { +struct Identity { uint8_t pub_key[32] = {}; }; +struct Utils { + static void sha256(uint8_t* out, size_t size, const uint8_t* key, size_t key_size, + const uint8_t* text, size_t text_size) { + uint64_t value = UINT64_C(14695981039346656037); + for (size_t i = 0; i < key_size; ++i) value = (value ^ key[i]) * UINT64_C(1099511628211); + for (size_t i = 0; i < text_size; ++i) value = (value ^ text[i]) * UINT64_C(1099511628211); + for (size_t i = 0; i < size; ++i) out[i] = uint8_t(value >> (8 * (i % 8))); + } +}; +} +#include "state.inc" +struct StrHelper { static void strncpy(char*, const char*, size_t); }; +struct Clock : mesh::RTCClock { + uint32_t now = 1000; + unsigned set_calls = 0; + uint32_t getCurrentTime() override { return now; } + void setCurrentTime(uint32_t value) override { now = value; ++set_calls; } +}; +struct ClientInfo { + mesh::Identity id; + uint8_t permissions = PERM_ACL_READ_WRITE; + struct { struct { uint32_t last_post_timestamp = 0; uint16_t post_quota_used = 0; } room; } extra; + ClientInfo() { id.pub_key[0] = 1; } +}; +struct MyMesh { + MemoryFS* _fs; + Clock clock; + mesh::Identity self_id; + mesh::RoomAccessPolicy room_access; + mesh::LogicalMessageCache<8> recent_room_posts; + ClientInfo* active = nullptr; + PostInfo posts[MAX_UNSYNCED_POSTS] = {}; + uint32_t post_ready_at[MAX_UNSYNCED_POSTS] = {}, post_ready_mask = UINT32_MAX; + uint32_t last_room_timestamp = 0; + mesh::RoomHistoryState room_history_state; + bool room_history_enabled = false, room_history_available = true; + int next_post_idx = 0; + unsigned long next_push = 0; + uint16_t _num_posted = 0; + unsigned acks = 0; + explicit MyMesh(MemoryFS* fs) : _fs(fs) { + metadata_filesystem = fs; + self_id.pub_key[0] = 77; + if (fs) assert(room_access.load(fs)); + } + Clock* getRTCClock() { return &clock; } + bool addPost(ClientInfo*, const char*); + bool addSystemPost(const char*); + bool storePost(const mesh::Identity&, const char*); + bool snapshotRoomHistory(); + void loadRoomHistory(); + bool handleRoomHistoryCommand(const char*, char*); + void serviceRoomQuotas() { + room_access.serviceQuotaWindow(ticks, [this] { active->extra.room.post_quota_used = 0; }); + } + void receive(ClientInfo* client, const char* text, uint32_t sender_timestamp) { + active = client; + const uint8_t flags = TXT_TYPE_PLAIN; + const size_t text_len = strlen(text); + bool send_ack = false; +#include "post_gate.inc" + if (send_ack) ++acks; + } + std::string command(const char* command) { + char reply[160] = {}; + assert(handleRoomHistoryCommand(command, reply)); + return reply; + } + std::vector retained() const { + std::vector result; + for (int k = 0, idx = next_post_idx; k < MAX_UNSYNCED_POSTS; + ++k, idx = (idx + 1) % MAX_UNSYNCED_POSTS) { + if (posts[idx].post_timestamp) result.emplace_back(posts[idx].text); + } + return result; + } +}; +#include "production.inc" +struct AdmissionState { + std::vector text; + uint32_t floor, mask, ready[32], timestamps[32]; + int index; + unsigned long next; + uint16_t stats, quota; + uint32_t replay; + unsigned acks; + AdmissionState(const MyMesh& m, const ClientInfo& c) + : text(m.retained()), floor(m.last_room_timestamp), mask(m.post_ready_mask), + index(m.next_post_idx), next(m.next_push), stats(m._num_posted), + quota(c.extra.room.post_quota_used), replay(c.extra.room.last_post_timestamp), acks(m.acks) { + memcpy(ready, m.post_ready_at, sizeof(ready)); + for (unsigned i = 0; i < 32; ++i) timestamps[i] = m.posts[i].post_timestamp; + } + void unchanged(const MyMesh& m, const ClientInfo& c) const { + assert(text == m.retained() && floor == m.last_room_timestamp && mask == m.post_ready_mask); + assert(index == m.next_post_idx && next == m.next_push && stats == m._num_posted); + assert(quota == c.extra.room.post_quota_used && replay == c.extra.room.last_post_timestamp); + assert(acks == m.acks && memcmp(ready, m.post_ready_at, sizeof(ready)) == 0); + for (unsigned i = 0; i < 32; ++i) assert(timestamps[i] == m.posts[i].post_timestamp); + } +}; +static void compareBoot(MemoryFS& fs, const MyMesh& running) { + const auto text = running.retained(); + const uint32_t floor = running.last_room_timestamp; + MyMesh boot(&fs); boot.clock.now = 5; boot.loadRoomHistory(); + assert(boot.room_history_available && boot.room_history_enabled); + assert(boot.retained() == text && boot.last_room_timestamp == floor); + assert(boot.clock.now == 5 && boot.clock.set_calls == 0 && boot.post_ready_mask == UINT32_MAX); + assert(boot._num_posted == 0); +} +int main() { + unsigned cases = 0; + { + MemoryFS fs; MyMesh m(&fs); m.loadRoomHistory(); + assert(!m.room_history_enabled && m.room_history_available && m.retained().empty()); + for (unsigned i = 0; i < 40; ++i) assert(m.addSystemPost(("volatile-" + std::to_string(i)).c_str())); + assert(!fs.files.count(mesh::ROOM_HISTORY_PATH) && fs.write_opens == 0); + assert(m.retained().size() == 32 && m.retained().front() == "volatile-8"); + assert(m.command("set room.history on").find("enabled") != std::string::npos); + assert(m.room_history_enabled && m.room_history_state.count == 32 && m.room_history_state.total_records == 32); + compareBoot(fs, m); + for (unsigned i = 0; i < 180; ++i) { + assert(m.addSystemPost(("durable-" + std::to_string(i)).c_str())); + assert(m.room_history_state.count == 32 && m.room_history_state.total_records <= 64); + assert(fs.get(mesh::ROOM_HISTORY_PATH).size() <= mesh::ROOM_HISTORY_MAX_FILE_SIZE); + if (i % 13 == 0) compareBoot(fs, m); + ++cases; + } + compareBoot(fs, m); + assert(m.retained().front() == "durable-148" && m.retained().back() == "durable-179"); + assert(m.command("get room.history") == "> on; storage ready; retained 32/32"); + ++cases; + } + for (const std::string& text : {std::string(151, 'A'), std::string("x") + std::string(50, 'z')}) { + // Second text is replaced below with exactly fifty complete UTF-8 codepoints. + std::string exact = text; + if (exact.size() != 151) { exact = "x"; for (unsigned i = 0; i < 50; ++i) exact += "\xe2\x98\x83"; } + assert(exact.size() == 151); + MemoryFS fs; MyMesh m(&fs); assert(m.command("set room.history on").find("enabled") != std::string::npos); + ClientInfo client; m.receive(&client, exact.c_str(), 123); + assert(m.acks == 1 && m._num_posted == 1 && m.retained()[0] == exact); + assert(m.snapshotRoomHistory()); compareBoot(fs, m); + m.receive(&client, exact.c_str(), 123); + assert(m.acks == 2 && m._num_posted == 1); + AdmissionState before(m, client); + m.receive(&client, (exact + "!").c_str(), 124); before.unchanged(m, client); + assert(m.addSystemPost("short")); + for (unsigned i = 0; i < 31; ++i) assert(m.addSystemPost("tiny")); + assert(m.snapshotRoomHistory()); compareBoot(fs, m); // reused long slot has canonical padding + ++cases; + } + for (const std::string failure : {"open:a:/room_history", "write:/room_history", "flush:/room_history", + "close:w:/room_history", "read:/room_history", "seek:/room_history"}) { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; + assert(m.command("set room.history on").find("enabled") != std::string::npos); + m.receive(&client, "accepted", 100); assert(m.acks == 1 && m._num_posted == 1); + ticks = 100; fs.reset(); fs.faults.insert(failure); + AdmissionState before(m, client); + m.receive(&client, "not-yet-retained", 101); before.unchanged(m, client); + fs.reset(); m.receive(&client, "not-yet-retained", 101); + assert(m.acks == 2 && m._num_posted == 2 && m.retained().back() == "not-yet-retained"); + assert(client.extra.room.last_post_timestamp == 101); + compareBoot(fs, m); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; + assert(m.command("set room.history on").find("enabled") != std::string::npos); + m.receive(&client, "trusted", 100); fs.short_write = true; + AdmissionState before(m, client); m.receive(&client, "partial", 101); before.unchanged(m, client); + assert(m.room_history_state.needs_repair); + fs.reset(); + MyMesh reboot(&fs); reboot.loadRoomHistory(); + assert(reboot.room_history_state.needs_repair && reboot.retained() == m.retained()); + m.receive(&client, "partial", 101); + assert(!m.room_history_state.needs_repair && m.acks == 2); + assert(m.retained() == std::vector({"trusted", "partial"})); + compareBoot(fs, m); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; + assert(m.command("set room.history on").find("enabled") != std::string::npos); + m.receive(&client, "durable-but-unconfirmed", 100); assert(m.acks == 1); + // Fail readback only after header+existing record validation. A whole disk + // append can survive without ACK, but trusted RAM repair must remove it. + fs.reset(); fs.fail_read_at = 3; + AdmissionState before(m, client); m.receive(&client, "readback-failed", 101); before.unchanged(m, client); + assert(m.room_history_state.needs_repair); + fs.reset(); m.receive(&client, "readback-failed", 101); + assert(m.acks == 2 && m._num_posted == 2 && m.retained().size() == 2); + compareBoot(fs, m); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); + assert(m.command("set room.history on").find("enabled") != std::string::npos); + assert(m.addSystemPost("archived")); const auto archive = fs.get(mesh::ROOM_HISTORY_PATH); + assert(m.command("set room.history off").find("archive retained") != std::string::npos); + assert(fs.get(mesh::ROOM_HISTORY_PATH) == archive); + assert(m.addSystemPost("volatile")); assert(fs.get(mesh::ROOM_HISTORY_PATH) == archive); + MyMesh boot(&fs); boot.loadRoomHistory(); + assert(!boot.room_history_enabled && boot.retained().empty() && fs.get(mesh::ROOM_HISTORY_PATH) == archive); + assert(m.command("set room.history 1").find("enabled") != std::string::npos); + compareBoot(fs, m); + const auto floor = m.last_room_timestamp; + assert(m.command("room.history.clear").find("cleared") != std::string::npos); + assert(m.retained().empty() && m.last_room_timestamp == floor); + MyMesh cleared(&fs); cleared.loadRoomHistory(); + assert(cleared.room_history_available && cleared.retained().empty() && cleared.last_room_timestamp == 0); + ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; + assert(m.command("set room.history on").find("enabled") != std::string::npos); + assert(m.addSystemPost("before-clear")); const auto archive = fs.get(mesh::ROOM_HISTORY_PATH); + fs.faults.insert("rename:/room_history.tmp:/room_history"); AdmissionState before(m, client); + assert(m.command("room.history.clear").find("failed") != std::string::npos); + before.unchanged(m, client); fs.reset(); assert(fs.get(mesh::ROOM_HISTORY_PATH) == archive); + ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; assert(m.addSystemPost("volatile-before-enable")); + fs.faults.insert("rename:/room_hist_cfg.tmp:/room_hist_cfg"); + assert(m.command("set room.history on").find("failed") != std::string::npos && !m.room_history_enabled); + fs.reset(); assert(m.command("set room.history on").find("enabled") != std::string::npos); + const auto config = fs.get(mesh::ROOM_HISTORY_CONFIG_PATH); + fs.faults.insert("rename:/room_hist_cfg.tmp:/room_hist_cfg"); + assert(m.command("set room.history 0").find("failed") != std::string::npos && m.room_history_enabled); + fs.reset(); assert(fs.get(mesh::ROOM_HISTORY_CONFIG_PATH) == config); + for (const char* invalid : {"set room.history yes", "set room.history on ", "set room.history 2"}) { + assert(m.command(invalid).find("requires on/off") != std::string::npos); + } + char reply[160] = {}; assert(!m.handleRoomHistoryCommand("get room.history.extra", reply)); + ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); m.clock.now = 2000; + assert(m.command("set room.history on").find("enabled") != std::string::npos); + assert(m.addSystemPost("first")); const auto floor = m.last_room_timestamp; + m.clock.now = 5; m.clock.resetUniqueTime(5); + assert(m.addSystemPost("backward")); assert(m.last_room_timestamp == floor + 1 && m.clock.now == 5); + assert(m.clock.set_calls == 0 && m.post_ready_at[1] == ticks + 6000); + MyMesh boot(&fs); boot.clock.now = 1; boot.loadRoomHistory(); const auto boot_floor = boot.last_room_timestamp; + assert(boot.addSystemPost("after-reboot") && boot.last_room_timestamp == boot_floor + 1); + assert(boot.clock.now == 1 && boot.clock.set_calls == 0); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); + assert(m.command("set room.history on").find("enabled") != std::string::npos); + m.clock.now = UINT32_MAX; assert(m.addSystemPost("last representable")); + ClientInfo client; AdmissionState before(m, client); assert(!m.addSystemPost("overflow")); before.unchanged(m, client); + MyMesh boot(&fs); boot.loadRoomHistory(); assert(boot.last_room_timestamp == UINT32_MAX); + assert(!boot.addSystemPost("cannot wrap")); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); ClientInfo client; + assert(!m.addSystemPost(nullptr) && !m.addSystemPost("") && !m.storePost(client.id, nullptr)); + assert(!m.addSystemPost(std::string(152, 'x').c_str()) && m._num_posted == 0); + MyMesh unavailable(nullptr); unavailable.loadRoomHistory(); + assert(!unavailable.room_history_available && !unavailable.addSystemPost("no fs")); + assert(unavailable.command("get room.history").find("unavailable") != std::string::npos); + assert(unavailable.command("set room.history on").find("failed") != std::string::npos); + assert(unavailable.command("room.history.clear").find("failed") != std::string::npos); ++cases; + } + { + MemoryFS fs; MyMesh m(&fs); + assert(m.command("set room.history on").find("enabled") != std::string::npos); + assert(m.addSystemPost("kept")); const auto original = fs.get(mesh::ROOM_HISTORY_PATH); + fs.faults.insert("open:r:/room_history"); MyMesh boot(&fs); boot.loadRoomHistory(); + assert(!boot.room_history_available && boot.retained().empty() && fs.get(mesh::ROOM_HISTORY_PATH) == original); + assert(!boot.addSystemPost("must not overwrite")); fs.reset(); + auto corrupt = original; corrupt[3] = 2; fs.put(mesh::ROOM_HISTORY_PATH, corrupt); + MyMesh future(&fs); future.loadRoomHistory(); + assert(!future.room_history_available && !future.addSystemPost("must preserve future")); + assert(fs.get(mesh::ROOM_HISTORY_PATH) == corrupt); ++cases; + } + { + MemoryFS fs; mesh::RoomHistoryState state; + metadata_filesystem = &fs; + assert(mesh::saveRoomHistoryConfig(&fs, true)); + assert(mesh::saveRoomHistorySnapshot(&fs, state, 2, [](uint8_t ordinal, mesh::RoomHistoryRecord& record) { + memset(&record, 0, sizeof(record)); record.author[0] = 1; + record.timestamp = ordinal == 0 ? 100 : 99; strcpy(record.text, "out of order"); return true; + })); + const auto original = fs.get(mesh::ROOM_HISTORY_PATH); MyMesh boot(&fs); boot.loadRoomHistory(); + assert(!boot.room_history_available && boot.retained().empty() && boot.last_room_timestamp == 0); + assert(fs.get(mesh::ROOM_HISTORY_PATH) == original && !boot.addSystemPost("no rewrite")); ++cases; + } + printf("%u production room history integration scenarios passed\n", cases); +} diff --git a/test/fixtures/room_history_store/filesystem.h b/test/fixtures/room_history_store/filesystem.h new file mode 100644 index 000000000..0431b55a9 --- /dev/null +++ b/test/fixtures/room_history_store/filesystem.h @@ -0,0 +1,155 @@ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct PowerCut {}; +class MemoryFS; +static MemoryFS* metadata_filesystem = nullptr; +class File { + MemoryFS* fs = nullptr; + std::shared_ptr> bytes; + std::string path; + size_t cursor = 0, initial_size = 0; + bool writable = false, directory = false, valid = false; +public: + File() = default; + File(MemoryFS*, std::shared_ptr>, const char*, bool, bool); + operator bool() const { return valid; } + bool isDirectory() const { return directory; } + size_t size(); + bool seek(uint32_t position); + int read(uint8_t*, size_t); + size_t write(const uint8_t*, size_t); + void flush(); + void close(); +}; +class MemoryFS { +public: + void _lockFS() {} + void _unlockFS() {} + MemoryFS* _getFS() { return this; } + std::map>> files; + std::set faults, directories; + std::vector trace; + size_t boundary = 0, cut_at = 0; + bool short_write = false, short_read = false; + std::vector replacement_on_close; + unsigned write_opens = 0; + size_t capacity = std::numeric_limits::max(); + unsigned read_calls = 0, fail_read_at = 0; + std::string replacement_path; + void point(const std::string& event) { + trace.push_back(event); + if (++boundary == cut_at) throw PowerCut{}; + } + bool begin(const std::string& op) { point("before:" + op); return !faults.count(op); } + void end(const std::string& op) { point("after:" + op); } + bool exists(const char* path) { + const std::string op = std::string("stat:") + path; + const bool okay = begin(op); + const bool present = files.count(path) || directories.count(path); + end(op); + return okay && present; + } + bool remove(const char* path) { + const std::string op = std::string("remove:") + path; + if (!begin(op)) return false; + const bool removed = files.erase(path) + directories.erase(path) != 0; + end(op); return removed; + } + bool rename(const char* from, const char* to) { + const std::string op = std::string("rename:") + from + ":" + to; + if (!begin(op) || !files.count(from) || files.count(to) || directories.count(to)) return false; + files[to] = files[from]; files.erase(from); end(op); return true; + } + File open(const char* path, const char* mode = "r", bool create = false) { + (void)create; + const bool append = mode[0] == 'a'; + const bool write = mode[0] == 'w' || append; + const std::string op = std::string("open:") + (append ? "a:" : write ? "w:" : "r:") + path; + if (!begin(op)) return {}; + if (write) { + ++write_opens; + if (!append || !files.count(path)) files[path] = std::make_shared>(); + if (faults.count(std::string("directory:") + path)) directories.insert(path); + } + if (!files.count(path) && !directories.count(path)) { end(op); return {}; } + if (!files.count(path)) files[path] = std::make_shared>(); + File file(this, files[path], path, write, directories.count(path)); + end(op); return file; + } + File open(const char* path, int mode) { assert(mode == 1); return open(path, "a"); } + void put(const char* path, const std::vector& bytes) { + files[path] = std::make_shared>(bytes); + } + std::vector get(const char* path) const { return *files.at(path); } + void reset() { trace.clear(); boundary = cut_at = 0; faults.clear(); short_write = short_read = false; write_opens = 0; read_calls = fail_read_at = 0; replacement_on_close.clear(); replacement_path.clear(); capacity = std::numeric_limits::max(); } +}; +File::File(MemoryFS* owner, std::shared_ptr> data, + const char* name, bool write, bool dir) + : fs(owner), bytes(data), path(name), writable(write), directory(dir), valid(true) { + initial_size = bytes->size(); cursor = write ? initial_size : 0; +} +size_t File::size() { + const std::string op = "size:" + path; + const bool okay = fs->begin(op); const size_t size = okay ? bytes->size() : 0; + fs->end(op); return size; +} +int File::read(uint8_t* output, size_t length) { + const std::string op = "read:" + path; + if (!valid || directory || !fs->begin(op)) return -1; + if (++fs->read_calls == fs->fail_read_at) return -1; + length = std::min(length, bytes->size() - cursor); + if (fs->short_read && length) --length; + if (length) memcpy(output, bytes->data() + cursor, length); + cursor += length; fs->end(op); return static_cast(length); +} +size_t File::write(const uint8_t* data, size_t length) { + const std::string op = "write:" + path; + if (!valid || directory || !fs->begin(op)) return 0; + if (fs->short_write && length) --length; + size_t occupied = 0; + for (const auto& entry : fs->files) occupied += entry.second->size(); + const size_t free = occupied < fs->capacity ? fs->capacity - occupied : 0; + length = std::min(length, free); + bytes->insert(bytes->end(), data, data + length); fs->end(op); return length; +} +void File::flush() { + const std::string op = "flush:" + path; + if (!fs->begin(op) && bytes->size() > initial_size) bytes->pop_back(); + fs->end(op); +} +void File::close() { + const std::string op = std::string("close:") + (writable ? "w:" : "r:") + path; + if (!fs->begin(op) && writable && bytes->size() > initial_size) (*bytes)[initial_size] ^= 1; + if (writable && !fs->replacement_on_close.empty() && fs->replacement_path == path) *bytes = fs->replacement_on_close; + valid = false; fs->end(op); +} + +bool File::seek(uint32_t position) { + const std::string op = "seek:" + path; + if (!valid || directory || !fs->begin(op) || position > bytes->size()) return false; + cursor = position; fs->end(op); return true; +} +#define FILE_O_WRITE 1 +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) +struct lfs_info {}; +static constexpr int LFS_ERR_NOENT = -2; +static int lfs_stat(MemoryFS* fs, const char* path, lfs_info*) { + const std::string op = std::string("stat:") + path; + const bool okay = fs->begin(op); + const bool present = fs->files.count(path) || fs->directories.count(path); + fs->end(op); + return !okay ? -5 : present ? 0 : LFS_ERR_NOENT; +} +#endif diff --git a/test/fixtures/room_history_store/test.cpp b/test/fixtures/room_history_store/test.cpp new file mode 100644 index 000000000..38eeb2deb --- /dev/null +++ b/test/fixtures/room_history_store/test.cpp @@ -0,0 +1,385 @@ +#include "filesystem.h" +#include +#include "vectors.inc" + +using Record = mesh::RoomHistoryRecord; +using State = mesh::RoomHistoryState; +static const char* P = mesh::ROOM_HISTORY_PATH; +static const char* T = mesh::ROOM_HISTORY_TEMP_PATH; +static const char* B = mesh::ROOM_HISTORY_BACKUP_PATH; +static const char* C = mesh::ROOM_HISTORY_CONFIG_PATH; +static const char* CT = mesh::ROOM_HISTORY_CONFIG_TEMP_PATH; +static const char* CB = mesh::ROOM_HISTORY_CONFIG_BACKUP_PATH; +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) +static const char* WRITE_MODE = "a"; +#else +static const char* WRITE_MODE = "w"; +#endif +static unsigned scenarios = 0; + +static Record record(unsigned number) { + Record result = {}; + result.author[0] = 1; result.author[31] = uint8_t(number); + result.timestamp = 1000 + number; + snprintf(result.text, sizeof(result.text), "post %u", number); + return result; +} +static bool same(const Record& a, const Record& b) { + return a.timestamp == b.timestamp && strcmp(a.text, b.text) == 0 + && memcmp(a.author, b.author, sizeof(a.author)) == 0; +} +static bool sameState(const State& a, const State& b) { + return a.count == b.count && a.total_records == b.total_records + && a.needs_repair == b.needs_repair && a.present == b.present; +} +static bool snapshot(MemoryFS& fs, State& state, const std::vector& records) { + metadata_filesystem = &fs; + return mesh::saveRoomHistorySnapshot(&fs, state, uint8_t(records.size()), + [&](uint8_t index, Record& output) { + if (index >= records.size()) return false; + output = records[index]; return true; + }); +} +static bool append(MemoryFS& fs, State& state, const Record& input) { + metadata_filesystem = &fs; + return mesh::appendRoomHistory(&fs, state, input.author, input.timestamp, input.text); +} +static bool load(MemoryFS& fs, State& state, std::vector& records) { + metadata_filesystem = &fs; records.clear(); + return mesh::loadRoomHistory(&fs, state, + [&](uint8_t index, const Record& input) { + assert(index == records.size()); records.push_back(input); return true; + }); +} +static bool config(MemoryFS& fs, bool enabled) { + metadata_filesystem = &fs; return mesh::saveRoomHistoryConfig(&fs, enabled); +} +static bool loadConfig(MemoryFS& fs, bool& enabled) { + metadata_filesystem = &fs; return mesh::loadRoomHistoryConfig(&fs, enabled); +} +static void expect(MemoryFS& fs, const std::vector& expected, bool repair = false) { + State state; std::vector actual; + assert(load(fs, state, actual) && actual.size() == expected.size()); + for (size_t index = 0; index < expected.size(); ++index) assert(same(actual[index], expected[index])); + assert(state.count == expected.size() && state.needs_repair == repair); +} +static std::vector rendered(const Record& input) { + uint8_t image[mesh::ROOM_HISTORY_RECORD_SIZE]; + assert(mesh::room_history_detail::encodeRecord(input, image)); + return {image, image + sizeof(image)}; +} +static void checkFailedSnapshot(MemoryFS& fs, State& state, + const std::vector& old) { + const State previous = state; + assert(!snapshot(fs, state, {record(10), record(11)})); + assert(sameState(state, previous)); fs.reset(); expect(fs, old); + assert(!fs.files.count(T) && !fs.files.count(B)); ++scenarios; +} + +int main() { + const std::vector old = {record(1), record(2)}; + { + MemoryFS fs; bool enabled = true; assert(loadConfig(fs, enabled) && !enabled); + assert(config(fs, true) && fs.get(C) == CONFIG_ON); + assert(loadConfig(fs, enabled) && enabled); + assert(config(fs, false) && fs.get(C) == CONFIG_OFF); + assert(loadConfig(fs, enabled) && !enabled); ++scenarios; + } + { + MemoryFS fs; State state; std::vector rows; + assert(load(fs, state, rows) && !state.present && rows.empty()); + assert(append(fs, state, record(1)) && state.present); + const auto wire = fs.get(P); + assert(wire.size() == HEADER.size() + RECORD_ONE.size()); + assert(std::equal(HEADER.begin(), HEADER.end(), wire.begin())); + assert(std::equal(RECORD_ONE.begin(), RECORD_ONE.end(), wire.begin() + HEADER.size())); + expect(fs, {record(1)}); ++scenarios; + } + { + MemoryFS fs; State loaded; assert(snapshot(fs, loaded, old)); + const auto previous = fs.get(P); State uninitialized; fs.reset(); + assert(!append(fs, uninitialized, record(3)) && uninitialized.needs_repair); + assert(fs.get(P) == previous && fs.write_opens == 0); + fs.files.erase(P); fs.put(B, previous); uninitialized = State{}; fs.reset(); + assert(!append(fs, uninitialized, record(3)) && uninitialized.needs_repair); + assert(fs.get(P) == previous && fs.write_opens == 0); ++scenarios; + } + { + MemoryFS fs; State state; std::vector retained; + assert(snapshot(fs, state, {})); + for (unsigned number = 1; number <= 64; ++number) assert(append(fs, state, record(number))); + assert(state.count == 32 && state.total_records == 64); + assert(fs.get(P).size() == mesh::ROOM_HISTORY_MAX_FILE_SIZE); + for (unsigned number = 33; number <= 64; ++number) retained.push_back(record(number)); + expect(fs, retained); const auto full = fs.get(P); fs.reset(); + assert(!append(fs, state, record(65)) && fs.trace.empty() && fs.get(P) == full); + assert(snapshot(fs, state, retained) && state.total_records == 32); + assert(fs.get(P).size() == HEADER.size() + 32 * RECORD_ONE.size()); + assert(append(fs, state, record(65))); retained.erase(retained.begin()); retained.push_back(record(65)); + expect(fs, retained); metadata_filesystem = &fs; + assert(mesh::clearRoomHistory(&fs, state) && fs.get(P) == HEADER && state.count == 0); + expect(fs, {}); assert(!fs.files.count(T) && !fs.files.count(B)); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); const auto previous = fs.get(P); + Record maximum = record(3); memset(maximum.text, 'x', 151); maximum.text[151] = 0; + assert(append(fs, state, maximum)); expect(fs, {record(1), record(2), maximum}); + fs.reset(); const auto accepted = fs.get(P); + Record bad = maximum; memset(bad.text, 'x', sizeof(bad.text)); + assert(!append(fs, state, bad) && fs.trace.empty()); + bad = record(4); bad.timestamp = 0; assert(!append(fs, state, bad) && fs.trace.empty()); + bad = record(4); memset(bad.author, 0, sizeof(bad.author)); + assert(!append(fs, state, bad) && fs.trace.empty() && fs.get(P) == accepted); + assert(previous.size() < accepted.size()); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); const auto previous = fs.get(P); + metadata_filesystem = &fs; + assert(!mesh::saveRoomHistorySnapshot(&fs, state, 33, mesh::room_history_detail::EmptyProvider{})); + assert(fs.get(P) == previous); + unsigned calls = 0; + assert(!mesh::saveRoomHistorySnapshot(&fs, state, 2, [&](uint8_t, Record& out) { + out = record(10); return ++calls == 1; + })); expect(fs, old); + calls = 0; + assert(!mesh::saveRoomHistorySnapshot(&fs, state, 2, [&](uint8_t index, Record& out) { + out = record(index + 10); return ++calls <= 2; // writer succeeds, verification provider fails + })); expect(fs, old); ++scenarios; + } + for (const std::string& fault : {std::string("open:") + WRITE_MODE + ":" + T, + std::string("write:") + T, std::string("flush:") + T, + std::string("close:w:") + T, std::string("open:r:") + T, + std::string("read:") + T, std::string("size:") + T, + std::string("directory:") + T, std::string("rename:") + P + ":" + B, + std::string("rename:") + T + ":" + P}) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); + fs.faults.insert(fault); checkFailedSnapshot(fs, state, old); + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + fs.capacity = fs.get(P).size() + 100; checkFailedSnapshot(fs, state, old); + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + fs.replacement_path = T; fs.replacement_on_close = fs.get(P); + checkFailedSnapshot(fs, state, old); // valid CRC stale image is not intended contents + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + fs.faults.insert(std::string("rename:") + T + ":" + P); + fs.faults.insert(std::string("rename:") + B + ":" + P); + fs.faults.insert(std::string("remove:") + T); + assert(!snapshot(fs, state, {record(10)})); assert(!fs.files.count(P) && fs.files.count(B)); + fs.reset(); expect(fs, old); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + fs.faults.insert(std::string("remove:") + B); + assert(snapshot(fs, state, {record(10)}) && fs.files.count(B)); + const unsigned writes = fs.write_opens; + assert(!snapshot(fs, state, old) && fs.write_opens == writes); + assert(!append(fs, state, record(11)) && fs.write_opens == writes && state.needs_repair); + fs.reset(); expect(fs, {record(10)}); assert(snapshot(fs, state, {record(10)})); + assert(append(fs, state, record(11))); ++scenarios; + } + for (const std::string& fault : {std::string("open:a:") + P, std::string("write:") + P, + std::string("flush:") + P, std::string("close:w:") + P, + std::string("seek:") + P}) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); + fs.faults.insert(fault); assert(!append(fs, state, record(3)) && state.needs_repair); + assert(state.count == 2 && state.total_records == 2); + const unsigned writes = fs.write_opens; + assert(!append(fs, state, record(3)) && fs.write_opens == writes); + fs.reset(); assert(snapshot(fs, state, old) && append(fs, state, record(3))); + expect(fs, {record(1), record(2), record(3)}); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); fs.short_write = true; + assert(!append(fs, state, record(3)) && state.needs_repair); fs.reset(); expect(fs, old, true); + assert(snapshot(fs, state, old) && append(fs, state, record(3))); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); + fs.capacity = fs.get(P).size() + 100; + assert(!append(fs, state, record(3)) && state.needs_repair); fs.reset(); expect(fs, old, true); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); + // Three old scan reads; the fourth, post-write verification header, fails. + fs.fail_read_at = 4; + assert(!append(fs, state, record(3)) && state.needs_repair && state.count == 2); + fs.reset(); expect(fs, {record(1), record(2), record(3)}); + // Runtime repair uses unchanged trusted RAM, discarding the unACKed append. + assert(snapshot(fs, state, old) && append(fs, state, record(3))); + expect(fs, {record(1), record(2), record(3)}); ++scenarios; + } + for (size_t tail_length = 1; tail_length < RECORD_ONE.size(); ++tail_length) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); auto bytes = fs.get(P); + const auto tail = rendered(record(3)); bytes.insert(bytes.end(), tail.begin(), tail.begin() + tail_length); + fs.put(P, bytes); expect(fs, old, true); std::vector rows; + assert(load(fs, state, rows) && state.needs_repair); + fs.reset(); assert(!append(fs, state, record(3)) && fs.trace.empty() && fs.get(P) == bytes); + assert(snapshot(fs, state, old) && append(fs, state, record(3))); ++scenarios; + } + for (size_t position = 0; position < RECORD_ONE.size(); ++position) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); assert(append(fs, state, record(3))); + auto bytes = fs.get(P); bytes[HEADER.size() + 2 * RECORD_ONE.size() + position] ^= 1; + fs.put(P, bytes); expect(fs, old, true); assert(fs.get(P) == bytes); ++scenarios; + } + // Corruption before another complete accepted record is never a torn + // append. Do not expose a shortened RAM ring that could replace the intact + // later records on the next post, clear, or snapshot attempt. + for (size_t damaged_record = 0; damaged_record < 2; ++damaged_record) { + for (size_t position = 0; position < RECORD_ONE.size(); ++position) { + MemoryFS fs; State state; + assert(snapshot(fs, state, {record(1), record(2), record(3)})); + auto bytes = fs.get(P); + bytes[HEADER.size() + damaged_record * RECORD_ONE.size() + position] ^= 1; + fs.put(P, bytes); fs.reset(); + const State previous = state; + std::vector rows; + assert(!load(fs, state, rows) && sameState(state, previous) && rows.empty()); + assert(fs.get(P) == bytes && fs.write_opens == 0); + // Both a truncated boot destination and the old trusted live ring must + // fail closed; administrative clear cannot discard the evidence either. + assert(!snapshot(fs, state, {record(1)})); + assert(!snapshot(fs, state, {record(1), record(2), record(3)})); + assert(!mesh::clearRoomHistory(&fs, state)); + assert(fs.get(P) == bytes && fs.write_opens == 0 && sameState(state, previous)); + assert(!append(fs, state, record(4)) && state.needs_repair); + assert(fs.get(P) == bytes && fs.write_opens == 0); + assert(!fs.files.count(T) && !fs.files.count(B)); ++scenarios; + } + } + for (int malformed = 0; malformed < 4; ++malformed) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + auto bytes = fs.get(P); auto tail = rendered(record(3)); + if (malformed == 0) std::fill(tail.begin(), tail.begin() + 32, 0); + if (malformed == 1) std::fill(tail.begin() + 32, tail.begin() + 36, 0); + if (malformed == 2) std::fill(tail.begin() + 36, tail.begin() + 188, 'x'); + if (malformed == 3) tail[150] = 'x'; + mesh::room_history_detail::put32(tail.data() + 188, + mesh::room_history_detail::crcBytes(tail.data(), 188)); + bytes.insert(bytes.end(), tail.begin(), tail.end()); fs.put(P, bytes); + expect(fs, old, true); assert(fs.get(P) == bytes); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); fs.reset(); fs.fail_read_at = 2; + const State before = state; std::vector rows; + assert(!load(fs, state, rows) && sameState(state, before) && rows.empty()); + fs.reset(); fs.fail_read_at = 6; + assert(!load(fs, state, rows) && sameState(state, before) && rows.size() == 1); + rows.clear(); fs.reset(); expect(fs, old); ++scenarios; + } + std::vector> invalid_headers = {{}, {1}, std::vector(11), HEADER, + std::vector(mesh::ROOM_HISTORY_MAX_FILE_SIZE + 1)}; + invalid_headers[3][3] = 2; + mesh::room_history_detail::put32(invalid_headers[3].data() + 8, + mesh::room_history_detail::crcBytes(invalid_headers[3].data(), 8)); + for (const auto& bytes : invalid_headers) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); const auto good = fs.get(P); + fs.put(P, bytes); fs.put(B, good); fs.put(T, good); + State before = state; std::vector rows; + assert(!load(fs, state, rows) && sameState(state, before)); + assert(!snapshot(fs, state, {}) && fs.get(P) == bytes && fs.get(B) == good && fs.get(T) == good); + MemoryFS backup; backup.put(B, bytes); backup.put(T, good); + assert(!load(backup, state, rows) && backup.get(B) == bytes && backup.get(T) == good); ++scenarios; + } + for (size_t position = 0; position < HEADER.size(); ++position) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); auto bytes = fs.get(P); + bytes[position] ^= 1; fs.put(P, bytes); std::vector rows; + assert(!load(fs, state, rows) && !snapshot(fs, state, {})); + assert(fs.get(P) == bytes); ++scenarios; + } + for (size_t position : {size_t(4), size_t(6), size_t(7)}) { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); auto bytes = fs.get(P); + ++bytes[position]; mesh::room_history_detail::put32(bytes.data() + 8, + mesh::room_history_detail::crcBytes(bytes.data(), 8)); fs.put(P, bytes); + std::vector rows; assert(!load(fs, state, rows) && !snapshot(fs, state, {})); + assert(fs.get(P) == bytes); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); const auto bytes = fs.get(P); + fs.files.erase(P); fs.put(B, bytes); fs.put(T, HEADER); expect(fs, old); + assert(fs.get(P) == bytes && !fs.files.count(T) && !fs.files.count(B)); + MemoryFS temp; temp.put(T, bytes); expect(temp, {}); assert(!temp.files.count(P)); ++scenarios; + } + { + MemoryFS fs; State state; assert(snapshot(fs, state, old)); + fs.directories.insert(P); std::vector rows; + assert(!load(fs, state, rows) && !snapshot(fs, state, {}) && fs.files.count(P)); ++scenarios; + } + for (size_t position = 0; position < CONFIG_ON.size(); ++position) { + MemoryFS fs; auto image = CONFIG_ON; image[position] ^= 1; + fs.put(C, image); fs.put(CB, CONFIG_OFF); fs.put(CT, CONFIG_OFF); + bool enabled = false; + assert(!loadConfig(fs, enabled) && !enabled && !config(fs, false)); + assert(fs.get(C) == image && fs.get(CB) == CONFIG_OFF && fs.get(CT) == CONFIG_OFF); ++scenarios; + } + { + MemoryFS fs; auto image = CONFIG_ON; image[3] = 2; + mesh::room_history_detail::put32(image.data() + 12, mesh::room_history_detail::crcBytes(image.data(), 12)); + fs.put(C, image); bool enabled = true; + assert(!loadConfig(fs, enabled) && enabled && !config(fs, false) && fs.get(C) == image); + MemoryFS backup; backup.put(CB, CONFIG_ON); backup.put(CT, CONFIG_OFF); + assert(loadConfig(backup, enabled) && enabled && backup.get(C) == CONFIG_ON); + MemoryFS temp; temp.put(CT, CONFIG_ON); assert(loadConfig(temp, enabled) && !enabled && !temp.files.count(C)); ++scenarios; + } + for (size_t position : {size_t(4), size_t(5)}) { + MemoryFS fs; auto image = CONFIG_ON; image[position] = 2; + mesh::room_history_detail::put32(image.data() + 12, + mesh::room_history_detail::crcBytes(image.data(), 12)); fs.put(C, image); + bool enabled = false; assert(!loadConfig(fs, enabled) && !config(fs, false)); + assert(fs.get(C) == image); ++scenarios; + } + for (const std::string& fault : {std::string("open:") + WRITE_MODE + ":" + CT, + std::string("write:") + CT, std::string("flush:") + CT, + std::string("close:w:") + CT, std::string("open:r:") + CT, + std::string("read:") + CT, std::string("size:") + CT, + std::string("directory:") + CT, std::string("rename:") + C + ":" + CB, + std::string("rename:") + CT + ":" + C}) { + MemoryFS fs; assert(config(fs, false)); fs.reset(); fs.faults.insert(fault); + assert(!config(fs, true)); fs.reset(); bool enabled = true; + assert(loadConfig(fs, enabled) && !enabled); ++scenarios; + } +#if defined(ESP32_PLATFORM) || defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) + for (const char* path : {P, C}) { + MemoryFS fs; State state; assert(snapshot(fs, state, old) && config(fs, false)); fs.reset(); + fs.faults.insert(std::string("stat:") + path); bool enabled = false; std::vector rows; + if (path == P) assert(!load(fs, state, rows) && !snapshot(fs, state, {})); + else assert(!loadConfig(fs, enabled) && !config(fs, true)); + assert(fs.write_opens == 0); ++scenarios; + } +#endif + // Simulate interruption before/after every filesystem operation, including + // verification and publication. Only temp->primary publication changes the + // selected snapshot/config; journal appends may recover a complete new row. + for (int action = 0; action < 3; ++action) { + MemoryFS baseline; State state; assert(snapshot(baseline, state, old) && config(baseline, false)); + baseline.reset(); + if (action == 0) assert(snapshot(baseline, state, {record(10), record(11)})); + if (action == 1) assert(append(baseline, state, record(3))); + if (action == 2) assert(config(baseline, true)); + const size_t boundaries = baseline.boundary; + for (size_t cut = 1; cut <= boundaries; ++cut) { + MemoryFS fs; State current; assert(snapshot(fs, current, old) && config(fs, false)); + fs.reset(); fs.cut_at = cut; + try { + if (action == 0) snapshot(fs, current, {record(10), record(11)}); + if (action == 1) append(fs, current, record(3)); + if (action == 2) config(fs, true); + } catch (const PowerCut&) {} + const bool new_snapshot = fs.files.count(P) && fs.get(P).size() > HEADER.size() + && fs.get(P)[HEADER.size() + 31] == 10; + const bool appended = action == 1 && fs.files.count(P) + && fs.get(P).size() == HEADER.size() + 3 * RECORD_ONE.size(); + const bool new_config = fs.files.count(C) && fs.get(C) == CONFIG_ON; + fs.reset(); + if (action == 0) expect(fs, new_snapshot ? std::vector{record(10), record(11)} : old); + if (action == 1) expect(fs, appended ? std::vector{record(1), record(2), record(3)} : old); + if (action == 2) { bool enabled; assert(loadConfig(fs, enabled) && enabled == new_config); expect(fs, old); } + assert(!fs.files.count(T) && !fs.files.count(B) && !fs.files.count(CT) && !fs.files.count(CB)); ++scenarios; + } + } + printf("PASS: %u bounded room history fault and recovery scenarios\n", scenarios); +} diff --git a/test/fixtures/room_receive_admission/test.cpp b/test/fixtures/room_receive_admission/test.cpp new file mode 100644 index 000000000..d596401e0 --- /dev/null +++ b/test/fixtures/room_receive_admission/test.cpp @@ -0,0 +1,237 @@ +// Actual receive dispatcher and Companion queue/callbacks are extracted into +// production.inc. Radio, crypto, UI and persistent-store boundaries are mocks. +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "helpers/CompanionFrameLimits.h" +#define PUB_KEY_SIZE 32 +#define PRV_KEY_SIZE 64 +#define MAX_CONTACTS 2 +#define OFFLINE_QUEUE_SIZE 4 +#define ADV_TYPE_NONE 0 +#define ADV_TYPE_ROOM 3 +#define OUT_PATH_UNKNOWN 255 +#define PAYLOAD_TYPE_TXT_MSG 2 +#define PAYLOAD_TYPE_REQ 0 +#define PAYLOAD_TYPE_RESPONSE 1 +#define PAYLOAD_TYPE_ACK 3 +#define TXT_ACK_DELAY 200 +#define CLI_REPLY_DELAY_MILLIS 200 +#define SERVER_RESPONSE_DELAY 200 +#define COMPANION_FEATURE_TEXT_TERMINAL 0 +#define COMPANION_FEATURE_NOTIFICATIONS 1 +#define MESH_ENABLE_ONE_KEY_DM 0 +#define ONE_KEY_DM_SHARED_OFFLINE_QUEUE 0 +#define DISPLAY_CLASS TestDisplay +#define MESH_DEBUG_PRINTLN(...) ((void)0) +#include "constants.inc" +static uint32_t millis() { return 100; } +static bool millisHasNowPassed(unsigned long deadline) { return millis() > deadline; } +namespace mesh { +struct Identity { uint8_t pub_key[PUB_KEY_SIZE] = {}; }; +struct Packet { + bool flood = false; + uint8_t path[2] = {7, 8}, path_len = 2; + float getSNR() const { return 10.75f; } + bool isRouteFlood() const { return flood; } +}; +struct Utils { + static void sha256(uint8_t* out, size_t size, const uint8_t*, size_t, + const uint8_t*, size_t) { memset(out, 0x55, size); } +}; +} +struct ContactInfo { + mesh::Identity id; + uint8_t type = ADV_TYPE_ROOM, out_path_len = 0; + uint32_t sync_since = 50, lastmod = 99; + char name[16] = "room"; + const uint8_t* getPath() const { return id.pub_key; } +}; +struct Clock { + uint32_t now = 1200; + uint32_t getCurrentTime() const { return now; } + uint32_t getCurrentTimeUnique() { return now++; } +}; +struct Rng { void random(uint8_t* out, size_t length) { memset(out, 0, length); } }; +struct BaseChatMesh { + ContactInfo contacts[MAX_CONTACTS]; + int matching_peer_indexes[1] = {0}, num_contacts = MAX_CONTACTS; + bool mutable_contacts = true; + Clock clock; + Rng rng; + mesh::Identity self_id; + mesh::Packet response; + unsigned direct_acks = 0, flood_acks = 0; + std::vector ack; + uint8_t temp_buf[184] = {}; + bool canMutateContacts() const { return mutable_contacts; } + Clock* getRTCClock() { return &clock; } + Rng* getRNG() { return &rng; } + virtual bool onSignedMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, const uint8_t*, const char*) = 0; + virtual void onMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*) = 0; + void onCommandDataRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*) {} + void onCLICommandRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*, char*) {} + uint8_t onContactRequest(const ContactInfo&, uint32_t, const uint8_t*, size_t, uint8_t*) { return 0; } + void onContactResponse(const ContactInfo&, const uint8_t*, size_t) {} + void handleReturnPathRetry(const ContactInfo&, const uint8_t*, uint8_t) {} + mesh::Packet* createDatagram(uint8_t, const mesh::Identity&, const uint8_t*, const uint8_t*, size_t) { return &response; } + mesh::Packet* createPathReturn(const mesh::Identity&, const uint8_t*, const uint8_t*, uint8_t, + uint8_t type, const uint8_t* bytes, size_t length) { + if (type == PAYLOAD_TYPE_ACK) { ++flood_acks; ack.assign(bytes, bytes + length); } + return &response; + } + void sendAckTo(const ContactInfo&, const uint8_t* bytes, size_t length = 4) { + ++direct_acks; ack.assign(bytes, bytes + length); + } + void sendFloodScoped(const ContactInfo&, mesh::Packet*, unsigned long = 0) {} + void sendDirect(mesh::Packet*, const uint8_t*, uint8_t, unsigned long = 0) {} + void onPeerDataRecv(mesh::Packet*, uint8_t, int, const uint8_t*, uint8_t*, size_t); +}; +enum class UIEventType { contactMessage }; +struct UI { + unsigned queue_changes = 0, messages = 0, notifications = 0, overrides = 0; + void syncMessageQueue(int, int = -1) { ++queue_changes; } + void setMessageNotificationOverride(bool, bool = false) { ++overrides; } + template void newMsg(Args...) { ++messages; } + void notify(UIEventType) { ++notifications; } +}; +struct Notifications { + unsigned remote = 0, ordinary = 0; + bool remoteMessage(const uint8_t*, const char*, uint32_t, uint32_t) { ++remote; return false; } + bool message(const uint8_t*, const uint8_t*, bool, uint32_t) { ++ordinary; return false; } + bool overridesScreen() const { return false; } +}; +struct Serial { + bool connected = false; + unsigned writes = 0; + bool isConnected() const { return connected; } + bool isReplyRouteAvailable(void*) const { return true; } + size_t writeFrameToRoute(void*, const uint8_t*, size_t length) { ++writes; return length; } + size_t writeFrame(const uint8_t*, size_t length) { ++writes; return length; } +}; +struct MyMesh : BaseChatMesh { +#include "frame.inc" + Frame offline_queue[OFFLINE_QUEUE_SIZE] = {}; + int offline_queue_len = 0, offline_queue_head = 0; + uint8_t out_frame[MAX_FRAME_SIZE + 1] = {}; + uint8_t app_target_ver = 3; + UI ui, *_ui = &ui; + Serial serial, *_serial = &serial; + Notifications _notifications; + void* private_key_backup_route = nullptr; + unsigned long private_key_backup_deadline = 1000; + uint8_t private_key_backup_sender[6] = {}; + char private_key_backup_nonce[17] = {}; + unsigned active = 0; + std::vector> saved; + MyMesh() { contacts[0].id.pub_key[0] = 77; contacts[1].id.pub_key[0] = 77; contacts[1].id.pub_key[31] = 1; } + void markConnectionActive(const ContactInfo&) { ++active; } + bool scheduleContactWrite(const ContactInfo& from) { saved.emplace_back(from.sync_since, from.lastmod); return true; } + int getOfflineQueueCapacity() const; + Frame& offlineQueueFrameAt(int); + bool addToOfflineQueue(const uint8_t*, int); + void popOfflineQueue(); + bool queueMessage(const ContactInfo&, uint8_t, mesh::Packet*, uint32_t, const uint8_t*, int, const char*, bool = false, uint32_t = 0); + bool onSignedMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, const uint8_t*, const char*) override; + void onMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*) override; + void receive(mesh::Packet& packet, uint32_t timestamp, const char* text, uint8_t type = TXT_TYPE_SIGNED_PLAIN, int sender = 0) { + std::vector payload(10 + strlen(text), 0); + memcpy(payload.data(), ×tamp, 4); payload[4] = uint8_t(type << 2); + const size_t offset = type == TXT_TYPE_SIGNED_PLAIN ? 9 : 5; + if (offset == 9) { payload[5] = 1; payload[6] = 2; payload[7] = 3; payload[8] = 4; } + memcpy(payload.data() + offset, text, strlen(text)); + matching_peer_indexes[0] = sender; + const uint8_t secret[32] = {}; + onPeerDataRecv(&packet, PAYLOAD_TYPE_TXT_MSG, 0, secret, payload.data(), offset + strlen(text)); + } + void fill(bool channel = false) { + for (int at = 0; at < OFFLINE_QUEUE_SIZE; ++at) { + const uint8_t frame[] = {uint8_t(channel && at == 1 ? RESP_CODE_CHANNEL_MSG_RECV : RESP_CODE_CONTACT_MSG_RECV), uint8_t(at)}; + assert(addToOfflineQueue(frame, sizeof(frame))); + } + } +}; +#include "production.inc" + +static unsigned effects(const MyMesh& m) { + return m.ui.messages + m.ui.notifications + m.ui.overrides + m.serial.writes + + m._notifications.remote + m._notifications.ordinary + m.active + unsigned(m.saved.size()); +} +int main() { + unsigned cases = 0; + for (int version : {2, 3}) for (bool flood : {false, true}) for (bool connected : {false, true}) { + MyMesh m; m.app_target_ver = version; m.serial.connected = connected; m.fill(); + mesh::Packet packet; packet.flood = flood; + const unsigned before = effects(m); + m.receive(packet, 100, "retained-before-ack"); + assert(m.contacts[0].sync_since == 50 && m.contacts[0].lastmod == 99); + assert(m.direct_acks == 0 && m.flood_acks == 0 && m.saved.empty()); + assert(effects(m) == before && m.offline_queue_len == OFFLINE_QUEUE_SIZE); + m.popOfflineQueue(); m.receive(packet, 100, "retained-before-ack"); + assert(m.contacts[0].sync_since == 100 && m.contacts[0].lastmod == 1200); + assert(m.saved.size() == 1 && m.saved[0] == std::make_pair(100U, 1200U)); + assert(m.direct_acks == unsigned(!flood) && m.flood_acks == unsigned(flood)); + assert(m.ack.size() == 4 && m.ui.messages == 1 && m._notifications.ordinary == 1); + const auto& frame = m.offlineQueueFrameAt(OFFLINE_QUEUE_SIZE - 1); + const size_t timestamp_offset = version >= 3 ? 12 : 9; + uint32_t stored; memcpy(&stored, frame.buf + timestamp_offset, 4); assert(stored == 100); + assert(frame.buf[timestamp_offset - 1] == TXT_TYPE_SIGNED_PLAIN); + assert(memcmp(frame.buf + timestamp_offset + 4, "\x01\x02\x03\x04", 4) == 0); + // Lost ACK with a now-full queue must not claim retention of another copy. + const unsigned accepted_effects = effects(m); + m.receive(packet, 100, "retained-before-ack"); + assert(effects(m) == accepted_effects && m.saved.size() == 1); + assert(m.direct_acks + m.flood_acks == 1 && m.contacts[0].sync_since == 100); + // Once the app drains capacity, normal retry admission and ACK resume. + m.popOfflineQueue(); m.receive(packet, 100, "retained-before-ack"); + assert(m.direct_acks + m.flood_acks == 2 && m.saved.size() == 2); + assert(m.contacts[0].sync_since == 100 && m.offline_queue_len == OFFLINE_QUEUE_SIZE); + ++cases; + } + { + MyMesh m; m.fill(true); mesh::Packet packet; + m.receive(packet, 100, "replace-channel"); + assert(m.offline_queue_len == OFFLINE_QUEUE_SIZE && m.direct_acks == 1); + assert(m.contacts[0].sync_since == 100 && m.saved.size() == 1); ++cases; + } + { + MyMesh m; mesh::Packet packet; m.mutable_contacts = false; + m.receive(packet, 100, "immutable-contact"); + assert(m.contacts[0].sync_since == 50 && m.contacts[0].lastmod == 99); + assert(m.direct_acks == 1 && m.offline_queue_len == 1); ++cases; + } + { + MyMesh m; mesh::Packet packet; + m.receive(packet, 40, "earlier-topic"); + assert(m.contacts[0].sync_since == 50 && m.direct_acks == 1); + m.receive(packet, 100, "same-text", TXT_TYPE_SIGNED_PLAIN, 0); + m.receive(packet, 100, "same-text", TXT_TYPE_SIGNED_PLAIN, 1); + assert(m.contacts[1].sync_since == 100 && m.offline_queue_len == 3); + // Distinct full room keys sharing the serialized prefix are both retained. + ++cases; + } + { + MyMesh m; m.fill(); mesh::Packet packet; + m.receive(packet, 100, "direct-message", TXT_TYPE_PLAIN); + assert(m.direct_acks == 1 && m.ack.size() == 6); + assert(m.contacts[0].sync_since == 50 && m.saved.size() == 1); + assert(m.ui.messages == 1 && m.offline_queue_len == OFFLINE_QUEUE_SIZE); ++cases; + } + for (size_t length : {6U, 7U, 8U}) { + MyMesh m; mesh::Packet packet; std::vector malformed(length + 1, 0); + malformed[4] = TXT_TYPE_SIGNED_PLAIN << 2; + const uint8_t secret[32] = {}; + m.onPeerDataRecv(&packet, PAYLOAD_TYPE_TXT_MSG, 0, secret, malformed.data(), length); + assert(m.direct_acks == 0 && m.offline_queue_len == 0 && effects(m) == 0); + assert(m.contacts[0].sync_since == 50 && m.contacts[0].lastmod == 99); ++cases; + } + assert(cases == 15); + printf("PASS: production room receive admission (%u cases, direct/flood, legacy/V3, both app states)\n", cases); + return 0; +} diff --git a/test/fixtures/room_topic_delivery/test.cpp b/test/fixtures/room_topic_delivery/test.cpp index bd3bcb9ed..d487ee308 100644 --- a/test/fixtures/room_topic_delivery/test.cpp +++ b/test/fixtures/room_topic_delivery/test.cpp @@ -4,6 +4,8 @@ #include #include #include +#include "../room_history_store/filesystem.h" +#include #define PUB_KEY_SIZE 32 #define MAX_PATH_SIZE 64 @@ -55,7 +57,7 @@ static bool saveRoomTopic(Store* store, const char* text) { } } #include "state.inc" -static_assert(sizeof(((ClientInfo*)0)->extra.room) <= sizeof(((ClientInfo*)0)->extra.sensor), +static_assert(sizeof(void*) != 4 || sizeof(((ClientInfo*)0)->extra.room) <= sizeof(((ClientInfo*)0)->extra.sensor), "topic state must fit the existing shared union"); struct Clock : mesh::RTCClock { uint32_t now = 1000; @@ -79,6 +81,8 @@ struct Sent { }; struct MyMesh { ACL acl; + MemoryFS policy_fs; + mesh::RoomAccessPolicy room_access; Clock clock; Random rng; mesh::Identity self_id; @@ -95,9 +99,13 @@ struct MyMesh { uint16_t _num_post_pushes = 0; char room_topic[MAX_POST_TEXT_LEN + 1] = {}; PostInfo posts[MAX_UNSYNCED_POSTS] = {}; + uint32_t post_ready_at[MAX_UNSYNCED_POSTS] = {}; + uint32_t post_ready_mask = UINT32_MAX; struct { uint8_t path_hash_mode = 0; } _prefs; int default_scope = 0; MyMesh() { + metadata_filesystem = &policy_fs; + assert(room_access.load(&policy_fs)); self_id.pub_key[0] = 77; acl.clients[0].id.pub_key[0] = 1; acl.clients[0].last_activity = 1000; @@ -200,9 +208,10 @@ int main() { } { MyMesh m; m.topic("topic"); m.clock.now = 1003; - m.post(0, 1000, "unaged"); m.tick(6000); + m.post(0, 1000, "unaged"); m.post_ready_mask &= ~UINT32_C(1); + m.post_ready_at[0] = futureMillis(10000); m.tick(6000); assert(m.sent.empty()); // Unaged older posts still protect app forceSince. - m.clock.now = 1007; m.tick(); assert(m.sent.back().text == "unaged"); + m.tick(4000); assert(m.sent.back().text == "unaged"); m.ack(); m.tick(); assert(m.sent.back().text == "topic"); } for (bool flood : {false, true}) for (bool pool : {false, true}) { @@ -266,5 +275,29 @@ int main() { m.tick(UINT32_C(0x80000000)); assert(m.sent.size() == 2); m.ack(); } + { + MyMesh m; m.topic("banned topic"); m.post(0, 900, "banned post"); + assert(m.room_access.addBan(&m.policy_fs, m.acl.clients[0].id.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + m.tick(6000); assert(m.sent.empty()); + assert(m.acl.clients[0].extra.room.pending_ack == 0); + assert(m.room_access.removeBan(&m.policy_fs, m.acl.clients[0].id.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + m.tick(); assert(m.sent.back().text == "banned post"); m.ack(); + m.tick(); assert(m.sent.back().text == "banned topic"); m.ack(); + } + { + // A banned first slot cannot swallow an ACK for a later allowed identity. + MyMesh m; m.acl.count = 2; + auto& banned = m.acl.clients[0]; auto& allowed = m.acl.clients[1]; + allowed.id.pub_key[0] = 2; + banned.extra.room.pending_ack = allowed.extra.room.pending_ack = 0x12345678; + banned.extra.room.push_post_timestamp = 200; + allowed.extra.room.push_post_timestamp = 300; + assert(m.room_access.addBan(&m.policy_fs, banned.id.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + const auto ban_before = banned; + const uint32_t ack = 0x12345678; + assert(m.processAck(reinterpret_cast(&ack))); + assert(memcmp(&ban_before, &banned, sizeof(banned)) == 0); + assert(allowed.extra.room.pending_ack == 0 && allowed.extra.room.sync_since == 300); + } puts("room topic delivery regressions passed"); } diff --git a/test/fixtures/room_web_service/boundary.json b/test/fixtures/room_web_service/boundary.json new file mode 100644 index 000000000..5ba59fec0 --- /dev/null +++ b/test/fixtures/room_web_service/boundary.json @@ -0,0 +1,5 @@ +{ + "status": {"name":"Test room","topic":"Topic ","boot":71,"role":2,"web_key":"5f5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a","persistent_history":false}, + "index": {"revision":1,"articles":[{"id":1,"version":1,"length":22,"title":"Instructions"}],"next":255}, + "read": {"version":1,"offset":0,"next":22,"total":22,"data64":"UmVhZCBvbmx5IHRoaXMgYXJ0aWNsZQ=="} +} diff --git a/test/fixtures/room_web_service/service.cpp b/test/fixtures/room_web_service/service.cpp new file mode 100644 index 000000000..ec72081e6 --- /dev/null +++ b/test/fixtures/room_web_service/service.cpp @@ -0,0 +1,279 @@ +#include "filesystem.h" +#include +#include +#include +#include +static bool deny_json_allocation = false; +static unsigned json_allocations = 0; +static void* trackedMalloc(size_t size) { + if (deny_json_allocation) return nullptr; + void* result = std::malloc(size); if (result) ++json_allocations; return result; +} +static void trackedFree(void* ptr) { + if (ptr) { assert(json_allocations != 0); --json_allocations; } std::free(ptr); +} +static void* trackedRealloc(void* ptr, size_t size) { + if (deny_json_allocation) return nullptr; + return std::realloc(ptr, size); +} +#define malloc trackedMalloc +#define free trackedFree +#define realloc trackedRealloc +#include +#undef malloc +#undef free +#undef realloc +#include +#include +#include +#include +#define WITH_WEBCONFIG 1 +#define PUB_KEY_SIZE 32 +#define MAX_POST_TEXT_LEN 151 +#define MAX_UNSYNCED_POSTS 32 +#define POST_SYNC_DELAY_SECS 2 +#define PUSH_NOTIFY_DELAY_MILLIS 200 +#define MESH_DEBUG_PRINTLN(...) ((void)0) +#include "roles.inc" +static uint32_t ticks = 100; +static uint32_t millis() { return ticks; } +static uint32_t futureMillis(uint32_t duration) { return millis() + duration; } +namespace mesh { +struct Identity { + uint8_t pub_key[32] = {}; + bool matches(const Identity& other) const { return memcmp(pub_key, other.pub_key, 32) == 0; } +}; +struct Utils { + // Cryptography is a boundary mock: every fixture token has a distinct key. + static void sha256(uint8_t* out, size_t length, const uint8_t* token, size_t token_length, + const uint8_t* server, size_t server_length) { + assert(length == 32 && token_length == 32 && server_length == 32); + for (size_t i = 0; i < length; ++i) out[i] = token[i] ^ server[i] ^ 0x5a; + } + static void toHex(char* output, const uint8_t* key, size_t length) { + static const char digits[] = "0123456789abcdef"; + for (size_t i = 0; i < length; ++i) { output[2 * i] = digits[key[i] >> 4]; output[2 * i + 1] = digits[key[i] & 15]; } + output[length * 2] = 0; + } +}; +} +struct StrHelper { + static void strncpy(char* out, const char* text, size_t capacity) { std::snprintf(out, capacity, "%s", text); } +}; +struct Preferences { + static bool value, fail_open, fail_write; + bool begin(const char* name, bool readonly) { assert(strcmp(name, "mesh-room") == 0 && !readonly); return !fail_open; } + size_t putBool(const char* key, bool next) { assert(strcmp(key, "web") == 0); if (fail_write) return 0; value = next; return sizeof(bool); } + bool getBool(const char* key, bool fallback) { (void)fallback; assert(strcmp(key, "web") == 0); return value; } + void end() {} +}; +bool Preferences::value = false, Preferences::fail_open = false, Preferences::fail_write = false; +struct ClientInfo { + mesh::Identity id; + uint8_t permissions = PERM_ACL_ADMIN; + struct { struct { uint16_t post_quota_used = 0, poll_quota_used = 0; uint32_t last_post_timestamp = 0; } room; } extra; +}; +struct ACL { + ClientInfo radio_admin; + unsigned lookups = 0; + int getNumClients() const { return 1; } + ClientInfo* getClientByIdx(int i) { assert(i == 0); return &radio_admin; } + ClientInfo* getClient(const mesh::Identity&) { ++lookups; return &radio_admin; } +}; +struct Clock { uint32_t now = 1000; uint32_t getCurrentTimeUnique() { return now++; } }; +#include "post.inc" +struct MyMesh { + MemoryFS storage; MemoryFS* _fs = &storage; + mesh::RoomAccessPolicy room_access; + mesh::Identity self_id; + struct { char password[65] = "admin-secret", guest_password[65] = "write-secret", node_name[64] = "Test room"; bool allow_read_only = true; } _prefs; + bool room_web_enabled = false; + bool room_web_request_authenticated = false; + uint32_t room_web_boot_id = 71; +#include "web_user.inc" + ACL acl; + Clock clock; + PostInfo posts[MAX_UNSYNCED_POSTS] = {}; + int next_post_idx = 0; + uint32_t last_room_timestamp = 0, post_ready_at[MAX_UNSYNCED_POSTS] = {}, post_ready_mask = UINT32_MAX, next_push = 0; + uint16_t _num_posted = 0; + mesh::RoomHistoryState room_history_state; + bool room_history_enabled = false, room_history_available = true; + char room_topic[152] = "Topic "; + MyMesh() { self_id.pub_key[0] = 4; metadata_filesystem = &storage; assert(room_access.load(_fs)); } + Clock* getRTCClock() { return &clock; } + bool handleRoomWebCommand(const char*, char*); + void processRoomRequest(const uint8_t*, uint32_t, char*, char*, size_t); + void serviceRoomQuotas(); + bool storePost(const mesh::Identity&, const char*); + bool snapshotRoomHistory(); +}; +#include "production.inc" + +static std::array token(uint8_t id = 1) { std::array value = {}; value[0] = id; return value; } +static std::string execute(MyMesh& m, const std::string& input, uint32_t sequence, + uint8_t owner = 1, size_t capacity = 2048) { + metadata_filesystem = &m.storage; + std::vector request(input.begin(), input.end()); request.push_back(0); + std::vector reply(capacity + 2, 0x7f); + const auto key = token(owner); + m.processRoomRequest(key.data(), sequence, request.data(), reply.data() + 1, capacity); + assert(reply.front() == 0x7f && reply.back() == 0x7f && json_allocations == 0); + if (!capacity) return ""; + assert(memchr(reply.data() + 1, 0, capacity)); return reply.data() + 1; +} +static JsonDocument result(MyMesh& m, const std::string& input, uint32_t sequence, uint8_t owner = 1) { + const auto text = execute(m, input, sequence, owner); + JsonDocument json; assert(deserializeJson(json, text) == DeserializationError::Ok); return json; +} +static std::string request(const char* op, const char* password = "write-secret", const std::string& fields = "") { + return std::string("{\"op\":\"") + op + "\",\"password\":\"" + password + "\"" + fields + "}"; +} +static void error(MyMesh& m, const std::string& input, uint32_t sequence, const char* fragment, uint8_t owner = 1) { + const auto json = result(m, input, sequence, owner); const char* text = json["error"]; + if (!text || !strstr(text, fragment)) fprintf(stderr, "Expected %s, got %s for %.120s\n", fragment, text ? text : "no error", input.c_str()); + assert(text && strstr(text, fragment)); +} +static std::string encoded(const std::string& body) { + std::vector out(4 * ((body.size() + 2) / 3) + 1); + mesh::roomEncodeBase64(reinterpret_cast(body.data()), body.size(), out.data()); return out.data(); +} +static std::string saveRequest(uint8_t id, uint32_t version, const std::string& title, const std::string& body, const char* password = "admin-secret") { + return request("board.save", password, ",\"boot\":71,\"id\":" + std::to_string(id) + ",\"version\":" + std::to_string(version) + + ",\"title\":\"" + title + "\",\"body64\":\"" + encoded(body) + "\""); +} +static void enable(MyMesh& m) { char reply[160]; assert(m.handleRoomWebCommand("set room.web on", reply)); assert(m.room_web_enabled); } +static unsigned floors(const MyMesh& m) { unsigned total = 0; for (const auto& user : m.room_web_users) total += user.last_sequence; return total; } +int main() { + unsigned scenarios = 0; + { + MyMesh m; assert(!m.room_web_enabled); + error(m, request("status"), 1, "disabled"); assert(floors(m) == 0); + char reply[160]; assert(m.handleRoomWebCommand("get room.web", reply) && strstr(reply, "off")); + Preferences::fail_write = true; assert(m.handleRoomWebCommand("set room.web on", reply) && !m.room_web_enabled); + Preferences::fail_write = false; enable(m); + assert(!m.handleRoomWebCommand("get room.webx", reply)); + auto status = result(m, request("status"), 1); assert(status["role"] == PERM_ACL_READ_WRITE && status["boot"] == 71); + assert(strcmp(status["topic"], m.room_topic) == 0 && status["persistent_history"] == false); + status = result(m, request("status", "bad-password", ",\"key\":\"radio-admin\",\"permissions\":3"), 2); + assert(status["role"] == PERM_ACL_GUEST && m.acl.lookups == 0); + const std::string web_key = status["web_key"].as(); assert(web_key.size() == 64); + error(m, request("post", "bad-password", ",\"boot\":71,\"name\":\"A\",\"text\":\"denied\""), 500, "read-only"); + assert(floors(m) == 2 && m.last_room_timestamp == 0 && m._num_posted == 0); + assert(result(m, request("status", "admin-secret"), 3)["role"] == PERM_ACL_ADMIN); ++scenarios; + } + { + MyMesh m; enable(m); m._prefs.allow_read_only = false; + error(m, request("status", "wrong"), 100, "incorrect"); assert(floors(m) == 0 && !m.room_web_request_authenticated); + error(m, request("status"), 100, "invalid browser", 0); assert(floors(m) == 0 && !m.room_web_request_authenticated); + assert(result(m, request("status"), 1)["role"] == PERM_ACL_READ_WRITE); + error(m, request("post", "write-secret", ",\"boot\":70,\"name\":\"A\",\"text\":\"no\""), 200, "restarted"); + assert(floors(m) == 1 && m._num_posted == 0); + error(m, saveRequest(1, 0, "Title", "body", "write-secret"), 300, "admin"); assert(floors(m) == 1); + const auto okay = result(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"one\""), 2); + assert(okay["ok"] == true && m.last_room_timestamp == 1000 && m._num_posted == 1); + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"one\""), 2, "completed"); + assert(m._num_posted == 1 && m.acl.radio_admin.extra.room.last_post_timestamp == 0); + auto post = result(m, request("posts", "write-secret", ",\"after\":0"), 3); + assert(strcmp(post["post"]["text"], "A: one") == 0 && post["post"]["timestamp"] == 1000); + post = result(m, request("posts", "write-secret", ",\"after\":1000"), 4); assert(post["post"].isNull()); ++scenarios; + } + { + MyMesh m; enable(m); m.room_history_enabled = true; + const std::string text(148, 'x'); + assert(result(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"" + text + "\""), 1)["ok"] == true); + assert(strlen(m.posts[0].text) == 151 && m.room_history_state.count == 1); + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"" + text + "x\""), 2, "151"); + assert(m._num_posted == 1 && m.last_room_timestamp == 1000); + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A:B\",\"text\":\"x\""), 3, "display name"); + m.storage.capacity = 0; + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"retain\""), 4, "not retained"); + assert(m._num_posted == 1 && m.last_room_timestamp == 1000 && m.room_history_state.count == 1); + m.storage.reset(); m.clock.now = 2; + assert(result(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"retry with new sequence\""), 5)["timestamp"] == 1001); + assert(m._num_posted == 2 && m.room_history_state.count == 2); ++scenarios; + } + { + MyMesh m; enable(m); assert(m.room_access.setRates(m._fs, 1, 2)); + assert(result(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"one\""), 1)["ok"] == true); + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"two\""), 2, "rate limit"); + assert(m._num_posted == 1 && m.last_room_timestamp == 1000); + result(m, request("status"), 3); result(m, request("status"), 4); error(m, request("status"), 5, "rate limit"); + ticks += 60000; result(m, request("status"), 6); + const auto key = token(); uint8_t author[32]; mesh::Utils::sha256(author, 32, key.data(), 32, m.self_id.pub_key, 32); + assert(m.room_access.addBan(m._fs, author) == mesh::RoomAccessPolicy::BanResult::Saved); + const unsigned floor = floors(m); error(m, request("status"), 1000, "denied"); assert(floors(m) == floor); + assert(m.room_access.removeBan(m._fs, author) == mesh::RoomAccessPolicy::BanResult::Saved); + result(m, request("status"), 7); ++scenarios; + } + { + MyMesh m; enable(m); const std::string title = u8"Café 地図 📡"; const std::string body(2048, 'b'); + const auto saved = result(m, saveRequest(1, 0, title, body), 1); + if (saved["ok"] != true) fprintf(stderr, "Full board save error: %s\n", saved["error"].as()); + assert(saved["ok"] == true); + assert(saved["version"] == 1); + auto page = result(m, request("board.index", "", ",\"revision\":0,\"cursor\":0"), 2); + assert(page["revision"] == 1 && page["articles"].size() == 1 && page["next"] == 255); + assert(page["articles"][0]["length"] == 2048 && strcmp(page["articles"][0]["title"], title.c_str()) == 0); + std::string assembled; + for (unsigned offset = 0, sequence = 3; offset < 2048; ++sequence) { + auto chunk = result(m, request("board.read", "", ",\"id\":1,\"version\":1,\"offset\":" + std::to_string(offset)), sequence); + assert(chunk["version"] == 1 && chunk["offset"] == offset && chunk["total"] == 2048); + uint8_t bytes[128]; size_t length; assert(mesh::roomDecodeBase64(chunk["data64"], bytes, sizeof(bytes), length) && length > 0 && length <= 128); + assembled.append(reinterpret_cast(bytes), length); offset += length; assert(chunk["next"] == offset); + } + assert(assembled == body); + const auto updated = result(m, saveRequest(1, 1, "Updated", "new body"), 30); + assert(updated["ok"] == true && updated["version"] == 2); + error(m, request("board.index", "", ",\"revision\":1,\"cursor\":0"), 31, "board changed"); + error(m, request("board.read", "", ",\"id\":1,\"version\":1,\"offset\":128"), 32, "article changed"); + error(m, saveRequest(1, 1, "Stale", "old"), 33, "article changed"); + error(m, request("board.delete", "admin-secret", ",\"boot\":71,\"id\":1,\"version\":0"), 34, "current article version"); + assert(result(m, request("board.delete", "admin-secret", ",\"boot\":71,\"id\":1,\"version\":2"), 35)["ok"] == true); + assert(result(m, saveRequest(1, 0, "Reused", "new"), 36)["ok"] == true); + error(m, request("board.read", "", ",\"id\":1,\"version\":2,\"offset\":0"), 37, "article changed"); ++scenarios; + } + { + MyMesh m; enable(m); uint32_t sequence = 1; + for (const std::string& input : {std::string("{"), std::string("[]"), std::string("null"), std::string("{\"op\":{\"a\":{\"b\":{\"c\":{\"d\":1}}}}}"), + std::string("{\"op\":\"status\",\"password\":\"write-secret\\u0000ignored\"}"), std::string("{\"op\":\"status\\u0000ignored\",\"password\":\"write-secret\"}"), + std::string("{\"op\":3,\"password\":[]}")}) { + const auto json = result(m, input, sequence++); assert(json["error"].is() && floors(m) == 0); + } + error(m, std::string(4097, ' '), sequence++, "invalid request"); + for (const std::string text : {"A===", "====", "Zg=A", "Zg==Zg==", "Zh==", "Zm9=", "*m9v", "Zg", "Zg== "}) { + error(m, request("board.save", "admin-secret", ",\"boot\":71,\"id\":1,\"version\":0,\"title\":\"T\",\"body64\":\"" + text + "\""), sequence++, "encoding"); + } + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"before\\u0000after\""), sequence++, "name and message"); + error(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\\u0000B\",\"text\":\"body\""), sequence++, "name and message"); + error(m, request("board.save", "admin-secret", ",\"boot\":71,\"id\":1,\"version\":0,\"title\":\"T\\u0000X\",\"body64\":\"Zg==\""), sequence++, "article changed"); + error(m, request("board.save", "admin-secret", ",\"boot\":71,\"id\":1,\"version\":0,\"title\":\"T\",\"body64\":\"Zg==\\u0000Zg==\""), sequence++, "article changed"); + assert(m._num_posted == 0 && m.last_room_timestamp == 0); + error(m, saveRequest(1, 0, std::string(64, 't'), "body"), sequence++, "article changed"); + error(m, saveRequest(1, 0, "Title", std::string(2049, 'b')), sequence++, "encoding"); + assert(!m.storage.files.count(mesh::ROOM_BOARD_PRIMARY_PATH)); + deny_json_allocation = true; error(m, request("status"), sequence++, "invalid JSON"); deny_json_allocation = false; + result(m, request("status"), sequence++); + // Excess unknown array members exhaust the actual bounded allocator. + std::string huge = "{\"op\":\"status\",\"password\":\"write-secret\",\"extra\":["; + for (unsigned i = 0; i < 1000; ++i) { if (i) huge += ','; huge += '0'; } huge += "]}"; + error(m, huge, sequence++, "invalid JSON"); assert(json_allocations == 0); ++scenarios; + } + { + MyMesh m; enable(m); + for (uint8_t owner = 1; owner <= 8; ++owner) result(m, request("status"), 1, owner); + error(m, request("status"), 1, "busy", 9); ticks += 1200000; + result(m, request("status"), 1, 9); + // Every small response capacity stays within its provided buffer. + for (size_t capacity = 0; capacity < 200; ++capacity) execute(m, request("status"), uint32_t(capacity + 2), 9, capacity); + for (unsigned i = 0; i < 40; ++i) assert(result(m, request("post", "write-secret", ",\"boot\":71,\"name\":\"A\",\"text\":\"bounded ring\""), 300 + i, 9)["ok"] == true); + assert(m._num_posted == 40 && m.next_post_idx == 8 && m.last_room_timestamp == 1039); ++scenarios; + } + // Deliver concrete backend responses for the browser boundary fixture. + MyMesh boundary; enable(boundary); + printf("BOUNDARY_STATUS %s\n", execute(boundary, request("status"), 1).c_str()); + assert(result(boundary, saveRequest(1, 0, "Instructions", "Read only this article"), 2)["ok"] == true); + printf("BOUNDARY_INDEX %s\n", execute(boundary, request("board.index", "write-secret", ",\"revision\":0,\"cursor\":0"), 3).c_str()); + printf("BOUNDARY_READ %s\n", execute(boundary, request("board.read", "write-secret", ",\"id\":1,\"version\":1,\"offset\":0"), 4).c_str()); + printf("PASS: %u actual room web service authorization, storage, quota, JSON and boundary scenarios\n", scenarios); +} diff --git a/test/fixtures/sensor_reply_bounds/test_sensor_reply_bounds.cpp b/test/fixtures/sensor_reply_bounds/test_sensor_reply_bounds.cpp index fc5da9837..d037fa07c 100644 --- a/test/fixtures/sensor_reply_bounds/test_sensor_reply_bounds.cpp +++ b/test/fixtures/sensor_reply_bounds/test_sensor_reply_bounds.cpp @@ -7,6 +7,8 @@ #include #include #include +#include "../room_history_store/filesystem.h" +#include #include "telemetry_capacity.inc" #include "telemetry_access.inc" @@ -127,7 +129,12 @@ public: }; class MyMesh : public SensorMesh { public: - MyMesh() { telemetry.max_len = ROOM_TELEMETRY_CAPACITY; telemetry.data.reserve(telemetry.max_len); } + MemoryFS room_fs; + MemoryFS* _fs = &room_fs; + MyMesh() { + telemetry.max_len = ROOM_TELEMETRY_CAPACITY; telemetry.data.reserve(telemetry.max_len); + metadata_filesystem = _fs; + } int handleRequest(ClientInfo*, uint32_t, uint8_t*, size_t, size_t = mesh::CLIENT_ACL_DIRECT_REPLY_CAPACITY); }; @@ -245,6 +252,30 @@ static void room_telemetry_checks() { assert(target.handleRequest(nullptr, 51, query, sizeof(query)) == 0); } +static void room_board_dispatch_checks() { + ClientInfo client; + uint8_t malformed[] = {mesh::ROOM_BOARD_REQUEST_SUBTYPE, 0}; + for (size_t capacity = 0; capacity <= MAX_PACKET_PAYLOAD; ++capacity) { + MyMesh target; + memset(target.reply_data, 0xee, sizeof(target.reply_data)); + const int length = target.handleRequest(&client, 51, malformed, sizeof(malformed), capacity); + assert(length == (capacity < 7 ? 0 : 7)); + assert(size_t(length) <= capacity); + for (size_t i = capacity; i < sizeof(target.reply_data); ++i) assert(target.reply_data[i] == 0xee); + if (length) { + uint32_t tag = 0; memcpy(&tag, target.reply_data, sizeof(tag)); + assert(tag == 51 && target.reply_data[4] == mesh::ROOM_BOARD_REQUEST_SUBTYPE); + assert(target.reply_data[5] == 0 && target.reply_data[6] == 1); + } + } + uint8_t index[] = {mesh::ROOM_BOARD_REQUEST_SUBTYPE, 0, 0, 0, 0, 0, 0}; + MyMesh target; + assert(target.handleRequest(&client, 51, index, sizeof(index), 13) == 7); + assert(target.reply_data[6] == 1); + assert(target.handleRequest(&client, 51, index, sizeof(index), 14) == 14); + assert(target.reply_data[6] == 0); +} + static void history_checks() { ClientInfo client; uint8_t query[10] = {}; @@ -313,6 +344,7 @@ static void history_checks() { int main(int argc, char** argv) { assert(argc == 2); if (!strcmp(argv[1], "room")) room_telemetry_checks(); + else if (!strcmp(argv[1], "room.board")) room_board_dispatch_checks(); else if (!strcmp(argv[1], "telemetry")) telemetry_checks(); else if (!strcmp(argv[1], "history")) history_checks(); else if (!strcmp(argv[1], "empty")) { diff --git a/test/fixtures/webconfig_ota_handoff.cpp b/test/fixtures/webconfig_ota_handoff.cpp index d96fcd644..25929c8cd 100644 --- a/test/fixtures/webconfig_ota_handoff.cpp +++ b/test/fixtures/webconfig_ota_handoff.cpp @@ -4,6 +4,7 @@ #include #include #include +#include static uint32_t millis() { return 123; } struct WebConfigBatch { @@ -34,6 +35,7 @@ struct WebConfigServer { DNS* _dns = nullptr; Callbacks callbacks; Callbacks* _cb = &callbacks; + mesh::RoomWebMailbox* _room_mailbox = new mesh::RoomWebMailbox; Mode _mode = MODE_LAN; bool _stopping = false, _keep_wifi_on_stop = false, _was_setup_ap = false; bool _owns_wifi = true, _stop_warned = false, _initial_setup = false; @@ -54,6 +56,7 @@ struct WebConfigServer { void requestStop(); bool stopForOTA(char*); void finalizeTeardown(); + ~WebConfigServer() { delete _room_mailbox; } }; @METHODS@ @@ -79,6 +82,7 @@ int main() { WiFi.ap = true; portal.requestStop(); // A duplicate stop cannot revoke the handoff. portal.finalizeTeardown(); // Existing requests have finally drained. + assert(portal._room_mailbox == nullptr); assert(WiFi.ap && WiFi.sta && WiFi.stops == 0); assert(WiFi.ap_stops == unsigned(setup)); assert(portal.callbacks.stops == 1 && !portal._keep_wifi_on_stop); @@ -88,6 +92,7 @@ int main() { WiFi = WiFiFake{}; ordinary.requestStop(); ordinary.finalizeTeardown(); + assert(ordinary._room_mailbox == nullptr); assert(WiFi.stops == 1 && !WiFi.sta); // Normal stop keeps its old behavior. WebConfigServer failed; failed._was_setup_ap = true; diff --git a/test/fixtures/webconfig_station_recovery.cpp b/test/fixtures/webconfig_station_recovery.cpp index 2f5ca1ff7..f0f5bbdaa 100644 --- a/test/fixtures/webconfig_station_recovery.cpp +++ b/test/fixtures/webconfig_station_recovery.cpp @@ -2,7 +2,9 @@ #include #include #include +#include #include +#include #include static uint32_t clock_ms = 1000; uint32_t millis() { return clock_ms; } @@ -77,7 +79,7 @@ struct DNS { void stop() {} }; class WebConfigServer { public: enum Mode { MODE_OFF, MODE_SETUP, MODE_CONNECTING, MODE_LAN }; - struct Callbacks {}; + struct Callbacks { bool supportsRoomService() const { return false; } }; Callbacks* _cb; void* _mqtt_prefs; bool _owns_wifi; @@ -86,6 +88,7 @@ public: const uint8_t* _pub_key; const char *_fw_ver, *_build_date, *_role, *_board_name; void* _mux = nullptr; + mesh::RoomWebMailbox* _room_mailbox = nullptr; bool _cli_enabled = false; char _wifi_ssid[32] = {}, _wifi_password[65] = {}; uint8_t _wifi_power_save = 0; @@ -130,6 +133,7 @@ public: void createServer() { ++server_starts; } void probeBoardCommands() { _board_cmds_probed = true; } void serviceTerminal(uint32_t) {} + void serviceRoom(uint32_t) {} bool bluetoothWiFiCoexistenceRequired() { return false; } bool acceptWiFi(Entry& e, const char* value) { @CREDENTIAL_BATCH@ diff --git a/test/test_client_acl_infrastructure.py b/test/test_client_acl_infrastructure.py index 2d4d18e58..d9878fd57 100644 --- a/test/test_client_acl_infrastructure.py +++ b/test/test_client_acl_infrastructure.py @@ -27,6 +27,11 @@ def production_methods(role, transform=None): begin = handler.index(" if (req_type == REQ_TYPE_GET_TELEMETRY_DATA" if sensor else " if (payload[0] == REQ_TYPE_GET_STATUS)") prefix = handler[:begin] + # The new board protocol has its own admission/storage regression suite. + # Keep this legacy ACL/telemetry fixture on its original request families. + if not sensor and "if (payload[0] == mesh::ROOM_BOARD_REQUEST_SUBTYPE)" in prefix: + board = extract_braced(prefix, "if (payload[0] == mesh::ROOM_BOARD_REQUEST_SUBTYPE)") + prefix = prefix.replace(board, "", 1) acl = extract_braced(handler, "if (req_type == REQ_TYPE_GET_ACCESS_LIST" if sensor else "if (payload[0] == REQ_TYPE_GET_ACCESS_LIST") generated = prefix + (" (void)perms;\n" if sensor else "") + acl + "\nreturn 0;\n}\n" @@ -36,6 +41,7 @@ def production_methods(role, transform=None): generated += "ClientInfo* " + ("from" if sensor else "client") + " = &sender;\n" generated += "const uint8_t type = PAYLOAD_TYPE_REQ; uint8_t secret[PUB_KEY_SIZE] = {};\n" + receive + "\n}\n" if not sensor: + generated += extract_braced(source, "bool MyMesh::saveFilter(") + "\n" telemetry = extract_braced(handler, "if (payload[0] == REQ_TYPE_GET_TELEMETRY_DATA)") telemetry_prefix = telemetry[:telemetry.index(" telemetry.reset();")] generated += "int MyMesh::telemetryPrefix(uint8_t* payload, size_t payload_len) {\n" @@ -67,7 +73,7 @@ class ClientAclInfrastructureTest(unittest.TestCase): (work / "production.inc").write_text(production_methods(role, transform), encoding="ascii") binary = work / "acl.exe" command = [compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", "-Wno-unused-parameter", - "-I" + str(work), "-I" + str(ROOT / "src"), str(FIXTURE), "-o", str(binary)] + "-I" + str(work), "-I" + str(ROOT / "test/fixtures/room_history_store"), "-I" + str(ROOT / "src"), str(FIXTURE), "-o", str(binary)] if role == "sensor": command.insert(1, "-DTEST_SENSOR=1") if sys.platform.startswith("linux"): command[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] diff --git a/test/test_client_acl_role.py b/test/test_client_acl_role.py index ad6de8c6f..7f23fa0dc 100644 --- a/test/test_client_acl_role.py +++ b/test/test_client_acl_role.py @@ -104,7 +104,7 @@ static_assert(sizeof(((ClientInfo*)0)->extra) == 4, "only persisted sync field r #else static_assert(sizeof(((ClientInfo*)0)->extra.sensor.min_deltas) == 14, "sensor state retained"); static_assert(sizeof(((ClientInfo*)0)->extra.room.pending_ack) == 4, "room state retained"); -static_assert(sizeof(((ClientInfo*)0)->extra.room) <= sizeof(((ClientInfo*)0)->extra.sensor), +static_assert(sizeof(void*) != 4 || sizeof(((ClientInfo*)0)->extra.room) <= sizeof(((ClientInfo*)0)->extra.sensor), "topic delivery must fit the existing shared union"); #endif int main() { diff --git a/test/test_contact_import/test_contact_import.cpp b/test/test_contact_import/test_contact_import.cpp index c5013f643..efc501e06 100644 --- a/test/test_contact_import/test_contact_import.cpp +++ b/test/test_contact_import/test_contact_import.cpp @@ -78,8 +78,8 @@ public: void onMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*) override {} void onCommandDataRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*) override {} void onCLICommandRecv(const ContactInfo&, mesh::Packet*, uint32_t, const char*, char*) override {} - void onSignedMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, - const uint8_t*, const char*) override {} + bool onSignedMessageRecv(const ContactInfo&, mesh::Packet*, uint32_t, + const uint8_t*, const char*) override { return true; } uint32_t calcFloodTimeoutMillisFor(uint32_t) const override { return 100; } uint32_t calcDirectTimeoutMillisFor(uint32_t, uint8_t) const override { return 100; } void onSendTimeout() override {} diff --git a/test/test_firmware_ram.py b/test/test_firmware_ram.py index b9db6b67b..aa0792d51 100644 --- a/test/test_firmware_ram.py +++ b/test/test_firmware_ram.py @@ -156,6 +156,19 @@ class FirmwareRamTest(unittest.TestCase): self.assertEqual(local["components"]["browser_terminal_session"], 2048) self.assertEqual(local["components"]["browser_terminal_scrollback"], 4096) + def test_room_browser_workspaces_are_bounded_and_only_budgeted_on_eligible_rooms(self): + flags = {"ADMIN_PASSWORD": "test"} + enabled = ram.requirements("ESP32_PLATFORM", flags, "v4_room_server") + disabled = ram.requirements("ESP32_PLATFORM", {**flags, "WEBCONFIG_DISABLED": 1}, "v4_room_server") + self.assertEqual(enabled["components"]["room_browser_mailbox"], 6656 + 16) + self.assertEqual(enabled["components"]["room_browser_request_workspace"], 8192 + 2048 + 16) + self.assertGreaterEqual(enabled["required_contiguous_bytes"], 8192) + for policy in (disabled, ram.requirements("ESP32_PLATFORM", flags, "v4_repeater"), + ram.requirements("ESP32_PLATFORM", flags, "v4_companion"), + ram.requirements("NRF52_PLATFORM", flags, "RAK_4631_room_server")): + self.assertNotIn("room_browser_mailbox", policy["components"]) + self.assertNotIn("room_browser_request_workspace", policy["components"]) + def test_longer_display_previews_reserve_heap_and_contiguous_history(self): defines = {"DISPLAY_CLASS": "SSD1306Display", "UI_SMALL_MESSAGE_FONT": 0} base = ram.requirements("ESP32_PLATFORM", defines, "v4_companion") diff --git a/test/test_official_room_app_compatibility.py b/test/test_official_room_app_compatibility.py index a30aa6a54..5e1a7a2de 100644 --- a/test/test_official_room_app_compatibility.py +++ b/test/test_official_room_app_compatibility.py @@ -50,11 +50,11 @@ def room_outputs(work, compiler): codes = "\n".join(line for line in companion.splitlines() if line.startswith("#define RESP_CODE_CONTACT_MSG_RECV")) (work / "frame_codes.inc").write_text(codes, encoding="ascii") - presentation = extract_braced(companion, "void MyMesh::queueMessage(") + presentation = extract_braced(companion, "bool MyMesh::queueMessage(") # The entire original framing prefix runs unchanged; the offline queue/UI # boundary is replaced by capture of those exact bytes, not a serializer. presentation = presentation[:presentation.index(" const char* sensitive_reply = text;")] - presentation += " (void)path_len;\n frame.assign(out_frame, out_frame + i);\n}\n" + presentation += " (void)path_len;\n frame.assign(out_frame, out_frame + i);\n return true;\n}\n" presentation = presentation.replace("MyMesh::queueMessage", "WirePresentation::queueMessage") presentation = presentation.replace("const ContactInfo &from", "const RoomContact &from") presentation = presentation.replace("mesh::Packet *pkt", "CompanionPacket *pkt") @@ -62,7 +62,9 @@ def room_outputs(work, compiler): binary = work / "room-messages" checked([compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", "-Wno-unused-parameter", "-Wno-missing-field-initializers", *SANITIZERS, - f"-I{work}", f"-I{ROOT / 'src'}", str(FIXTURES / "room_messages.cpp"), "-o", str(binary)]) + f"-I{work}", f"-I{ROOT / 'src'}", + f"-I{ROOT / 'test/fixtures/room_topic_delivery'}", + str(FIXTURES / "room_messages.cpp"), "-o", str(binary)]) records = [] for line in checked([str(binary)], timeout=20).splitlines(): kind, group, index, version, flood, timestamp, author, text, frame = line.split(":") diff --git a/test/test_room_access_policy.py b/test/test_room_access_policy.py new file mode 100644 index 000000000..d398247cb --- /dev/null +++ b/test/test_room_access_policy.py @@ -0,0 +1,404 @@ +#!/usr/bin/env python3 +"""Run the room ban/quota policy and its real store against faults and reboots.""" + +from pathlib import Path +import os +import re +import shutil +import struct +import subprocess +import sys +import tempfile +import unittest +import zlib + +from test_room_topic_store import HARNESS as FILESYSTEM_HARNESS, STAT_MOCK +from test_replay_reset_integration import extract_braced + +ROOT = Path(__file__).resolve().parents[1] + +LFS_MOCK = r''' +struct lfs_info {}; +static constexpr int LFS_ERR_NOENT = -2; +static int lfs_stat(MemoryFS* fs, const char* path, struct lfs_info*) { + assert(fs && fs->lock_depth == 1); + const std::string op = std::string("stat:") + path; + const bool okay = fs->begin(op); + const bool present = fs->files.count(path) || fs->directories.count(path); + fs->end(op); + return !okay ? -5 : present ? 0 : LFS_ERR_NOENT; +} +''' + +TESTS = r''' +#define FILE_O_WRITE 1 +#include +#define HEX 16 // Arduino's numeric-format macro must remain usable. +#include +static_assert(HEX == 16, "room policy must preserve Arduino HEX"); +using Policy = mesh::RoomAccessPolicy; +using BanResult = Policy::BanResult; +static const char* P = mesh::ROOM_ACCESS_PRIMARY_PATH; +static const char* T = mesh::ROOM_ACCESS_TEMP_PATH; +static const char* B = mesh::ROOM_ACCESS_BACKUP_PATH; +static const std::vector OLD = { @OLD_VECTOR@ }; +static const std::vector NEW = { @NEW_VECTOR@ }; +static unsigned scenarios = 0; +static std::array key(uint8_t suffix) { + std::array result; + result.fill(0xab); result.back() = suffix; return result; +} +static std::string hex(const uint8_t* key) { + static const char digits[] = "0123456789abcdef"; + std::string result; + for (size_t i = 0; i < 32; ++i) { result += digits[key[i] >> 4]; result += digits[key[i] & 15]; } + return result; +} +static bool load(Policy& policy, MemoryFS& fs) { + metadata_filesystem = &fs; + const bool okay = policy.load(&fs); + assert(fs.lock_depth == 0); return okay; +} +static void loaded(MemoryFS& fs, unsigned post, unsigned poll, bool banned = true) { + Policy rebooted; assert(load(rebooted, fs)); + assert(rebooted.healthy() && rebooted.postRate() == post && rebooted.pollRate() == poll); + assert(rebooted.isBanned(key(1).data()) == banned); + assert(rebooted.allowsIdentity(key(1).data()) == !banned); + assert(rebooted.allowsIdentity(key(2).data())); +} +static void rechecksum(std::vector& image) { + uint32_t sum = 0xffffffffu; + for (size_t i = 0; i + 4 < image.size(); ++i) { + sum ^= image[i]; + for (unsigned bit = 0; bit < 8; ++bit) sum = (sum >> 1) ^ ((sum & 1) ? 0xedb88320u : 0); + } + sum ^= 0xffffffffu; + for (unsigned i = 0; i < 4; ++i) image[image.size() - 4 + i] = uint8_t(sum >> (8 * i)); +} +static mesh::RoomAccessSettings nextSettings() { + mesh::RoomAccessSettings result; + result.posts_per_minute = 5; result.polls_per_minute = 7; result.ban_count = 1; + memcpy(result.bans[0], key(1).data(), 32); return result; +} +static void ordinaryFault(MemoryFS& fs) { + Policy policy; assert(load(policy, fs)); + assert(!policy.setRates(&fs, 5, 7)); + assert(policy.healthy() && policy.postRate() == 2 && policy.pollRate() == 3); + assert(!policy.allowsIdentity(key(1).data()) && policy.allowsIdentity(key(2).data())); + fs.reset(); loaded(fs, 2, 3); assert(fs.get(P) == OLD); + assert(!fs.files.count(T) && !fs.files.count(B)); ++scenarios; +} + +static void admissionAndBans() { + MemoryFS fs; Policy policy; + assert(!policy.healthy() && !policy.allowsIdentity(key(1).data())); + uint16_t count = 0; assert(!policy.consumePost(count) && count == 0); + assert(!policy.load(static_cast(nullptr))); + assert(load(policy, fs) && policy.postRate() == 0 && policy.pollRate() == 0); + assert(!policy.allowsIdentity(nullptr)); uint8_t zero[32] = {}; + assert(!policy.allowsIdentity(zero) && !policy.isBanned(nullptr)); + assert(policy.addBan(&fs, nullptr) == BanResult::InvalidKey); + assert(policy.addBan(&fs, zero) == BanResult::InvalidKey && fs.write_opens == 0); + assert(policy.addBan(&fs, key(1).data()) == BanResult::Saved); + assert(!policy.allowsIdentity(key(1).data()) && policy.allowsIdentity(key(2).data())); + // Keys share their entire first31 bytes: prefixes cannot broaden a ban. + for (uint8_t suffix = 2; suffix < 34; ++suffix) assert(policy.allowsIdentity(key(suffix).data())); + const unsigned writes = fs.write_opens; + assert(policy.addBan(&fs, key(1).data()) == BanResult::Unchanged && fs.write_opens == writes); + assert(policy.removeBan(&fs, key(2).data()) == BanResult::Unchanged && fs.write_opens == writes); + Policy rebooted; assert(load(rebooted, fs)); assert(rebooted.banCount() == 1); + assert(!rebooted.allowsIdentity(key(1).data())); + for (uint8_t suffix = 2; suffix <= 32; ++suffix) { + assert(policy.addBan(&fs, key(suffix).data()) == BanResult::Saved); + } + assert(policy.banCount() == 32); + const auto full = fs.get(P); const unsigned full_writes = fs.write_opens; + assert(policy.addBan(&fs, key(33).data()) == BanResult::Full); + assert(fs.get(P) == full && fs.write_opens == full_writes && policy.allowsIdentity(key(33).data())); + for (uint8_t suffix : {uint8_t(32), uint8_t(1), uint8_t(16)}) { + assert(policy.removeBan(&fs, key(suffix).data()) == BanResult::Saved); + assert(policy.allowsIdentity(key(suffix).data())); + } + assert(policy.banCount() == 29); + assert(policy.banAt(29) == nullptr && policy.banAt(size_t(-1)) == nullptr); + assert(policy.addBan(&fs, key(33).data()) == BanResult::Saved); + Policy final; assert(load(final, fs)); + assert(final.banCount() == 30 && !final.allowsIdentity(key(33).data())); + assert(final.allowsIdentity(key(1).data()) && final.allowsIdentity(key(16).data())); ++scenarios; +} + +static void aclPersistenceAndQuotas() { + for (unsigned permissions = 0; permissions < 256; ++permissions) { + assert(mesh::roomAclShouldPersist(permissions, true, 7, 3)); + assert(mesh::roomAclShouldPersist(permissions, false, 7, 3) == ((permissions & 7) == 3)); + } + MemoryFS fs; Policy policy; assert(load(policy, fs)); + struct Counts { uint16_t posts = 0, polls = 0; } users[2]; + unsigned resets = 0; + const auto reset = [&] { ++resets; for (auto& u : users) u.posts = u.polls = 0; }; + policy.serviceQuotaWindow(1000, reset); assert(resets == 1); + users[0].posts = users[0].polls = UINT16_MAX; + assert(policy.consumePost(users[0].posts) && policy.consumeKeepAlive(users[0].polls)); + assert(users[0].posts == UINT16_MAX && users[0].polls == UINT16_MAX); // Disabled is free. + assert(policy.setRates(&fs, 2, 3)); + policy.serviceQuotaWindow(2000, reset); assert(resets == 2); + for (unsigned n = 0; n < 2; ++n) assert(policy.consumePost(users[0].posts)); + assert(!policy.consumePost(users[0].posts) && users[0].posts == 2); + for (unsigned n = 0; n < 3; ++n) assert(policy.consumeKeepAlive(users[0].polls)); + assert(!policy.consumeKeepAlive(users[0].polls) && users[0].polls == 3); + assert(policy.consumePost(users[1].posts) && users[1].posts == 1); // Actual separate identity state. + policy.serviceQuotaWindow(61999, reset); assert(resets == 2 && users[0].posts == 2); + policy.serviceQuotaWindow(62000, reset); assert(resets == 3 && users[0].posts == 0); + assert(policy.addBan(&fs, key(1).data()) == BanResult::Saved); + assert(policy.consumePost(users[1].posts)); + policy.serviceQuotaWindow(62001, reset); assert(resets == 3 && users[1].posts == 1); // Ban doesn't reset quotas. + assert(policy.setRates(&fs, 2, 3)); policy.serviceQuotaWindow(62002, reset); + assert(resets == 3); // Idempotent config cannot renew a user's budget. + assert(policy.setRates(&fs, UINT16_MAX, UINT16_MAX)); + policy.serviceQuotaWindow(UINT32_MAX - 30000U, reset); const unsigned before_wrap = resets; + users[0].posts = users[0].polls = UINT16_MAX - 1; + assert(policy.consumePost(users[0].posts) && users[0].posts == UINT16_MAX); + assert(!policy.consumePost(users[0].posts)); + assert(policy.consumeKeepAlive(users[0].polls) && users[0].polls == UINT16_MAX); + assert(!policy.consumeKeepAlive(users[0].polls)); + policy.serviceQuotaWindow(29998, reset); assert(resets == before_wrap); + policy.serviceQuotaWindow(29999, reset); assert(resets == before_wrap + 1); + assert(users[0].posts == 0 && users[0].polls == 0); + policy.serviceQuotaWindow(UINT32_C(0x80007530), reset); assert(resets == before_wrap + 2); + loaded(fs, UINT16_MAX, UINT16_MAX); + ++scenarios; +} + +static void cliCases() { + MemoryFS fs; Policy policy; assert(load(policy, fs)); char reply[160]; + const auto run = [&](const std::string& command) { + memset(reply, 0xa5, sizeof(reply)); + assert(policy.handleConfig(&fs, command.c_str(), reply, 157)); + assert(uint8_t(reply[157]) == 0xa5); + }; + run("get room.post.rate"); assert(!strcmp(reply, "> 0/min (0=off)")); + run("set room.post.rate 12"); assert(!strncmp(reply, "OK", 2)); + run("set room.poll.rate 7"); assert(policy.postRate() == 12 && policy.pollRate() == 7); + run("get room.post.rate "); assert(!strcmp(reply, "> 12/min (0=off)")); + run("get room.poll.rate"); assert(!strcmp(reply, "> 7/min (0=off)")); + const unsigned saved = fs.write_opens; + for (const char* rate : {"", "-1", "+1", "65536", "9999999999999999999999", "1/min", "1 2", "1.0", "1;2"}) { + run(std::string("set room.post.rate ") + rate); assert(!strncmp(reply, "Err", 3)); + assert(policy.postRate() == 12 && fs.write_opens == saved); + } + uint8_t parsed[32]; memset(parsed, 0xcc, sizeof(parsed)); + const std::string first = hex(key(1).data()), second = hex(key(2).data()); + for (const std::string& bad : {first.substr(0,8), first.substr(0,12), first.substr(0,63), first + "0", std::string("all"), std::string(64,'0'), std::string(64,'g')}) { + assert(!Policy::parseFullKey(bad.c_str(), parsed)); + for (uint8_t value : parsed) assert(value == 0xcc); + run("room.ban " + bad); assert(!strncmp(reply, "Err", 3) && policy.banCount() == 0); + } + assert(!Policy::parseFullKey(nullptr, parsed)); + std::string uppercase = first; for (char& c : uppercase) if (c >= 'a' && c <= 'f') c -= 'a' - 'A'; + assert(Policy::parseFullKey(uppercase.c_str(), parsed) && memcmp(parsed,key(1).data(),32) == 0); + run("room.ban " + first); assert(policy.banCount() == 1); + run("room.ban " + second); assert(policy.banCount() == 2); + run("room.ban " + hex(key(3).data())); assert(policy.banCount() == 3); + run("get room.bans"); assert(std::string(reply) == "Room bans 1/2\n" + first + "\n" + second); + run("get room.bans 2"); assert(std::string(reply) == "Room bans 2/2\n" + hex(key(3).data())); + for (const char* page : {"0", "3", "65536", "-1", "1 2", "abc"}) { + run(std::string("get room.bans ") + page); assert(!strncmp(reply, "Err", 3)); + } + char tiny[65]; memset(tiny,0xa5,sizeof(tiny)); policy.formatBanPage(1,tiny,64); + assert(!strncmp(tiny,"Err",3) && uint8_t(tiny[64]) == 0xa5); + assert(!policy.handleConfig(&fs,"room.ban-other",reply,157)); + assert(!policy.handleConfig(&fs,"get room.bans-other",reply,157)); + assert(!policy.handleConfig(&fs,"set room.poll.rate-other",reply,157)); + assert(!policy.handleConfig(&fs,nullptr,reply,157)); + assert(!policy.handleConfig(&fs,"get room.bans",nullptr,157)); + run("room.unban " + first); assert(policy.allowsIdentity(key(1).data())); + run("room.unban " + first); assert(!strncmp(reply,"OK",2)); + ++scenarios; +} + +static void persistenceAndFaults() { + { MemoryFS fs; fs.put(P,OLD); loaded(fs,2,3); + Policy policy; assert(load(policy,fs)); assert(policy.setRates(&fs,5,7)); + assert(fs.get(P) == NEW); loaded(fs,5,7); ++scenarios; } + { mesh::RoomAccessSettings bad; bad.ban_count = 33; MemoryFS fs; + assert(mesh::room_access_detail::save(&fs,bad) == mesh::room_access_detail::SaveResult::WriteFailure); + bad.ban_count = 1; + assert(mesh::room_access_detail::save(&fs,bad) == mesh::room_access_detail::SaveResult::WriteFailure); + memcpy(bad.bans[0],key(1).data(),32); memcpy(bad.bans[1],key(1).data(),32); bad.ban_count = 2; + assert(mesh::room_access_detail::save(&fs,bad) == mesh::room_access_detail::SaveResult::WriteFailure); + assert(fs.trace.empty()); ++scenarios; } + std::vector> invalid = {{}, {1}, std::vector(15), OLD}; + invalid.back().push_back(0); + auto future = OLD; future[3] = 2; rechecksum(future); invalid.push_back(future); + auto reserved = OLD; reserved[10] = 1; rechecksum(reserved); invalid.push_back(reserved); + auto too_many = OLD; too_many[8] = 33; rechecksum(too_many); invalid.push_back(too_many); + auto zero_key = OLD; std::fill(zero_key.begin()+12,zero_key.end()-4,0); rechecksum(zero_key); invalid.push_back(zero_key); + auto duplicate = OLD; duplicate.insert(duplicate.end()-4,OLD.begin()+12,OLD.end()-4); + duplicate[8] = 2; rechecksum(duplicate); invalid.push_back(duplicate); + for (size_t byte = 0; byte < OLD.size(); ++byte) { + auto corrupt = OLD; corrupt[byte] ^= 1; invalid.push_back(corrupt); + } + for (const auto& bytes : invalid) { + MemoryFS fs; fs.put(P,bytes); fs.put(B,OLD); fs.put(T,NEW); Policy policy; + assert(!load(policy,fs) && !policy.healthy()); + assert(!policy.allowsIdentity(key(2).data())); uint16_t count = 0; + assert(!policy.consumePost(count) && !policy.consumeKeepAlive(count) && count == 0); + assert(!policy.setRates(&fs,5,7) && policy.addBan(&fs,key(2).data()) == BanResult::StorageFailure); + assert(fs.get(P) == bytes && fs.get(B) == OLD && fs.get(T) == NEW && fs.write_opens == 0); ++scenarios; + } + { MemoryFS fs; fs.put(B,OLD); fs.put(T,NEW); loaded(fs,2,3); + assert(fs.get(P) == OLD && !fs.files.count(T)); ++scenarios; } + { MemoryFS fs; fs.put(T,NEW); loaded(fs,0,0,false); + assert(!fs.files.count(P) && !fs.files.count(T)); ++scenarios; } + for (const std::string& fault : {std::string("open:w:")+T,std::string("write:")+T, + std::string("flush:")+T,std::string("close:w:")+T,std::string("open:r:")+T, + std::string("read:")+T,std::string("size:")+T,std::string("rename:")+P+":"+B, + std::string("rename:")+T+":"+P,std::string("directory:")+T}) { + MemoryFS fs; fs.put(P,OLD); fs.faults.insert(fault); ordinaryFault(fs); + } + { MemoryFS fs; fs.put(P,OLD); Policy policy; assert(load(policy,fs)); fs.short_write = true; + assert(!policy.setRates(&fs,5,7) && policy.postRate() == 2); fs.reset(); loaded(fs,2,3); ++scenarios; } + { MemoryFS fs; fs.put(P,OLD); Policy policy; assert(load(policy,fs)); fs.replacement_on_close = OLD; + assert(!policy.setRates(&fs,5,7) && policy.postRate() == 2); fs.reset(); loaded(fs,2,3); ++scenarios; } + { MemoryFS fs; fs.put(P,OLD); Policy policy; assert(load(policy,fs)); + fs.faults.insert(std::string("open:r:")+P); + assert(!policy.setRates(&fs,5,7) && !policy.healthy() && !policy.allowsIdentity(key(2).data())); + assert(fs.write_opens == 0 && fs.get(P) == OLD); fs.reset(); assert(load(policy,fs)); ++scenarios; } + { MemoryFS fs; fs.put(P,OLD); Policy policy; assert(load(policy,fs)); + fs.faults.insert(std::string("rename:")+T+":"+P); + fs.faults.insert(std::string("rename:")+B+":"+P); + fs.faults.insert(std::string("remove:")+T); + assert(!policy.setRates(&fs,5,7) && policy.postRate() == 2); + assert(!fs.files.count(P) && fs.get(B) == OLD && fs.get(T) == NEW); + fs.reset(); loaded(fs,2,3); ++scenarios; } + { MemoryFS fs; fs.put(P,OLD); Policy policy; assert(load(policy,fs)); + fs.faults.insert(std::string("remove:")+B); + assert(policy.setRates(&fs,5,7) && policy.postRate() == 5 && fs.get(P) == NEW); + loaded(fs,5,7); assert(!policy.setRates(&fs,2,3)); + fs.reset(); loaded(fs,5,7); ++scenarios; } +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) || defined(ESP32_PLATFORM) + for (bool present : {false,true}) { + MemoryFS fs; if (present) fs.put(P,OLD); + fs.faults.insert(std::string("stat:")+P); Policy policy; + assert(!load(policy,fs) && !policy.allowsIdentity(key(1).data())); + assert(!policy.setRates(&fs,5,7) && fs.write_opens == 0); + assert(fs.files.count(P) == size_t(present)); ++scenarios; + } +#endif + for (bool existing : {false,true}) { + MemoryFS baseline; if (existing) baseline.put(P,OLD); metadata_filesystem = &baseline; + assert(mesh::room_access_detail::save(&baseline,nextSettings()) == mesh::room_access_detail::SaveResult::Saved); + const size_t boundaries = baseline.boundary; + for (size_t cut = 1; cut <= boundaries; ++cut) { + MemoryFS fs; if (existing) fs.put(P,OLD); fs.cut_at = cut; metadata_filesystem = &fs; + try { mesh::room_access_detail::save(&fs,nextSettings()); } catch (const PowerCut&) {} + const bool committed = fs.files.count(P) && fs.get(P) == NEW; + fs.reset(); fs.lock_depth = 0; // Power loss also releases the abandoned mount lock. + loaded(fs,committed ? 5 : existing ? 2 : 0, committed ? 7 : existing ? 3 : 0,committed || existing); + assert(!fs.files.count(T) && !fs.files.count(B)); + if (committed || existing) assert(fs.get(P) == (committed ? NEW : OLD)); + else assert(!fs.files.count(P)); + ++scenarios; + } + } +} +int main() { + admissionAndBans(); aclPersistenceAndQuotas(); cliCases(); persistenceAndFaults(); + printf("PASS: %u room access identity/quota/storage/power-cut scenarios\n",scenarios); +} +''' + + +def image(post, poll, bans): + payload = b"RAP\x01" + struct.pack(" +#include +#define PUB_KEY_SIZE 32 +#define MAX_PATH_SIZE 64 +namespace mesh { struct Identity { uint8_t pub_key[PUB_KEY_SIZE]; }; } +""" + roles + "\n" + prior + ";\n" + current + ";\n" + """ +static_assert(sizeof(void*) == 4 && sizeof(unsigned long) == 4, "target ABI must be 32-bit"); +static_assert(sizeof(ClientInfo) == sizeof(PriorClientInfo), "room counters/explicit marker must use existing padding"); +static_assert(sizeof(ClientInfo) <= (MESH_CLIENT_REPEATER_ONLY ? 284 : 320), "existing client RAM bound must hold"); +#if !MESH_CLIENT_REPEATER_ONLY +static_assert(sizeof(((ClientInfo*)0)->extra.room) <= sizeof(((ClientInfo*)0)->extra.sensor), "room counters must fit existing sensor union"); +#endif +""" + with tempfile.TemporaryDirectory(prefix="room-access-layout-") as temporary: + work = Path(temporary) + cpp = work / "layout.cpp" + cpp.write_text(source) + for repeater_only in (0, 1): + with self.subTest(repeater_only=repeater_only): + compiled = subprocess.run([compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", + "-DMESH_CLIENT_REPEATER_ONLY=" + str(repeater_only), + "-fsyntax-only", str(cpp)], capture_output=True, text=True, timeout=30) + self.assertEqual(compiled.returncode, 0, compiled.stdout + compiled.stderr) + + def test_production_policy_store_cli_and_faults_on_all_backends(self): + compiler = os.environ.get("CXX") or shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler) + filesystem = FILESYSTEM_HARNESS[:FILESYSTEM_HARNESS.index("#define FILE_O_WRITE")] + # Reuse the existing power-cut filesystem without changing that suite. + # Native LittleFS metadata probes also prove the lock is released. + default_lock = (" void _lockFS() {}\n" + " void _unlockFS() {}\n" + " MemoryFS* _getFS() { return this; }\n") + self.assertIn(default_lock, filesystem) + filesystem = filesystem.replace(default_lock, + " int lock_depth = 0;\n" + " void _lockFS() { assert(lock_depth++ == 0); }\n" + " void _unlockFS() { assert(--lock_depth == 0); }\n" + " MemoryFS* _getFS() { return this; }\n", 1) + tests = TESTS + key = bytes([0xAB] * 31 + [1]) + for marker, settings in (("@OLD_VECTOR@", (2, 3, [key])), ("@NEW_VECTOR@", (5, 7, [key]))): + tests = tests.replace(marker, ",".join(str(byte) for byte in image(*settings))) + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="room-access-policy-") as temporary: + work = Path(temporary) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + source = work / "access.cpp" + for platform in (None, "NRF52_PLATFORM", "STM32_PLATFORM", "RP2040_PLATFORM", "ESP32_PLATFORM"): + with self.subTest(platform=platform): + source.write_text(filesystem + (LFS_MOCK if platform in ("NRF52_PLATFORM", "STM32_PLATFORM") else "") + tests) + binary = work / (platform or "generic") + defines = ["-D" + platform + "=1"] if platform else [] + built = subprocess.run([compiler, "-std=c++11", "-O1", "-Wall", "-Wextra", "-Werror", + *sanitizers, *defines, "-I" + str(work), "-I" + str(ROOT / "src"), + str(source), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=30) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("room access identity/quota/storage/power-cut scenarios", checked.stdout) + print((platform or "generic") + ": " + checked.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_acl_persistence.py b/test/test_room_acl_persistence.py new file mode 100644 index 000000000..967d11361 --- /dev/null +++ b/test/test_room_acl_persistence.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +"""Run the actual room ACL filter and ClientACL storage/eviction policy.""" +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_replay_reset_integration import extract_braced + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/client_acl_spiffs" + +HARNESS = r''' +#define main existing_acl_fixture_main +#include "test_client_acl_spiffs.cpp" +#undef main +#include +struct MyMesh { static bool saveFilter(ClientInfo*); }; +@SAVE_FILTER@ +static mesh::Identity keyed(unsigned n) { + uint8_t key[PUB_KEY_SIZE] = {}; + key[0] = 0x49; key[29] = uint8_t(n >> 16); key[30] = uint8_t(n >> 8); key[31] = uint8_t(n); + return mesh::Identity(key); +} +static void roomAssignedAclSurvivesReboot() { + FakeFilesystem fs; + ClientACL acl; acl.load(&fs, SELF); acl.protectExplicitPermissions(); + for (unsigned role = 1; role <= 7; ++role) { + const auto id = keyed(role); + CHECK(acl.applyPermissions(SELF, id.pub_key, PUB_KEY_SIZE, uint8_t(role | 0x80))); + auto* client = acl.getClient(id.pub_key, PUB_KEY_SIZE); + CHECK(client && client->permissions_are_explicit && MyMesh::saveFilter(client)); + client->last_activity = 1000 + role; + CHECK(acl.authorizeLoginTimestamp(id.pub_key, 100 + role, 0, uint8_t(role))); + client->last_timestamp = 100 + role; + client->extra.room.sync_since = 40 + role; +#if !MESH_CLIENT_REPEATER_ONLY + client->extra.room.post_quota_used = 6; client->extra.room.poll_quota_used = 5; +#endif + } + const auto guest = keyed(20), writer = keyed(21), admin = keyed(22); + CHECK(acl.putClient(guest, PERM_ACL_GUEST)); + auto* auto_rw = acl.putClient(writer, PERM_ACL_READ_WRITE); + auto* auto_admin = acl.putClient(admin, PERM_ACL_ADMIN); + CHECK(auto_rw && auto_admin); + CHECK(!auto_rw->permissions_are_explicit && !MyMesh::saveFilter(auto_rw)); + CHECK(!auto_admin->permissions_are_explicit && MyMesh::saveFilter(auto_admin)); + SELF.calcSharedSecret(auto_admin->shared_secret, admin.pub_key); + CHECK(acl.save(&fs, MyMesh::saveFilter)); + const auto contacts = fs.files["/s_contacts"]; + ClientACL restored; restored.load(&fs, SELF); restored.protectExplicitPermissions(); + CHECK(restored.getNumClients() == 8); + CHECK(!restored.getClient(guest.pub_key, PUB_KEY_SIZE)); + CHECK(!restored.getClient(writer.pub_key, PUB_KEY_SIZE)); + for (unsigned role = 1; role <= 7; ++role) { + const auto id = keyed(role); + auto* client = restored.getClient(id.pub_key, PUB_KEY_SIZE); + CHECK(client && client->permissions == uint8_t(role | 0x80)); + CHECK(client->permissions_are_explicit && client->extra.room.sync_since == 40 + role); + if (role == PERM_ACL_READ_ONLY) { + // Existing RO-only sessions deliberately do not allocate durable login ceilings. + CHECK(client->last_timestamp == 0); + CHECK(!restored.authorizeLoginTimestamp(id.pub_key, 100 + role, 100 + role, uint8_t(role))); + } else { + CHECK(client->last_timestamp >= 160 + role); + CHECK(!restored.authorizeLoginTimestamp(id.pub_key, 100 + role, client->last_timestamp, uint8_t(role))); + } +#if !MESH_CLIENT_REPEATER_ONLY + CHECK(client->extra.room.post_quota_used == 0 && client->extra.room.poll_quota_used == 0); +#endif + } + CHECK(restored.getClient(admin.pub_key, PUB_KEY_SIZE)->permissions_are_explicit); + // The marker is not a new wire/storage flag and an unchanged save is byte-identical. + CHECK(restored.save(&fs, MyMesh::saveFilter)); + CHECK(fs.files["/s_contacts"] == contacts); +} +static void roomPinsOnlyExplicitAssignments() { + FakeFilesystem fs; ClientACL acl; acl.load(&fs, SELF); acl.protectExplicitPermissions(); + for (unsigned i = 0; i < MAX_CLIENTS; ++i) { + const auto id = keyed(i + 1); + CHECK(acl.applyPermissions(SELF, id.pub_key, PUB_KEY_SIZE, i % 2 ? PERM_ACL_READ_ONLY : PERM_ACL_READ_WRITE)); + acl.getClient(id.pub_key, PUB_KEY_SIZE)->last_activity = i + 1; + } + CHECK(!acl.putClient(keyed(100), PERM_ACL_GUEST)); + CHECK(acl.getNumClients() == MAX_CLIENTS); + // setperm0 stays removal, creating one ephemeral login-cache slot. + const auto removed = keyed(1); + CHECK(acl.applyPermissions(SELF, removed.pub_key, PUB_KEY_SIZE, 0)); + CHECK(!acl.getClient(removed.pub_key, PUB_KEY_SIZE)); + auto* ephemeral = acl.putClient(keyed(100), PERM_ACL_READ_WRITE); + CHECK(ephemeral && !ephemeral->permissions_are_explicit); +#if !MESH_CLIENT_REPEATER_ONLY + ephemeral->extra.room.post_quota_used = ephemeral->extra.room.poll_quota_used = 9; +#endif + CHECK(acl.putClient(keyed(101), PERM_ACL_GUEST)); + CHECK(!acl.getClient(keyed(100).pub_key, PUB_KEY_SIZE)); + for (unsigned i = 2; i <= MAX_CLIENTS; ++i) CHECK(acl.getClient(keyed(i).pub_key, PUB_KEY_SIZE)); +#if !MESH_CLIENT_REPEATER_ONLY + CHECK(acl.getClient(keyed(101).pub_key, PUB_KEY_SIZE)->extra.room.post_quota_used == 0); +#endif + // Other roles keep the existing default policy: explicit RO is still evictable. + FakeFilesystem other_fs; ClientACL other; other.load(&other_fs, SELF); + for (unsigned i = 0; i < MAX_CLIENTS; ++i) { + const auto id = keyed(i + 1); + CHECK(other.applyPermissions(SELF, id.pub_key, PUB_KEY_SIZE, PERM_ACL_READ_ONLY)); + other.getClient(id.pub_key, PUB_KEY_SIZE)->last_activity = i + 1; + } + CHECK(other.putClient(keyed(999), PERM_ACL_GUEST)); + CHECK(!other.getClient(keyed(1).pub_key, PUB_KEY_SIZE)); + // Opt-out is deliberate and reversible; no persistence-format migration. + acl.protectExplicitPermissions(false); + CHECK(acl.putClient(keyed(102), PERM_ACL_GUEST)); +} +int main() { + roomAssignedAclSurvivesReboot(); roomPinsOnlyExplicitAssignments(); + puts("actual room assigned ACL persistence, replay, eviction, and other-role policy passed"); +} +''' + + +class RoomAclPersistenceTests(unittest.TestCase): + def test_actual_acl_filter_storage_and_opt_in_pressure_protection(self): + compiler = shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler) + room = (ROOT / "examples/simple_room_server/MyMesh.cpp").read_text() + generated = HARNESS.replace("@SAVE_FILTER@", extract_braced(room, "bool MyMesh::saveFilter(")) + with tempfile.TemporaryDirectory(prefix="room-assigned-acl-") as directory: + work = Path(directory) + (work / "test.cpp").write_text(generated) + for compact in (0, 1): + with self.subTest(repeater_only=compact): + binary = work / ("acl-" + str(compact)) + command = [compiler, "-std=c++17", "-O1", "-Wall", "-Wextra", "-Werror", + "-DESP32=1", "-DESP32_PLATFORM=1", f"-DMESH_CLIENT_REPEATER_ONLY={compact}", + f"-I{FIXTURE / 'mocks'}", f"-I{FIXTURE}", f"-I{ROOT / 'src'}", + str(work / "test.cpp"), "-o", str(binary)] + if "g++" in Path(compiler).name: + # The existing fixture identity has a test-only nontrivial constructor. + command.insert(1, "-Wno-class-memaccess") + if sys.platform.startswith("linux"): + command[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + built = subprocess.run(command, capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=20) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("actual room assigned ACL persistence, replay, eviction, and other-role policy passed", checked.stdout) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_board_protocol.py b/test/test_room_board_protocol.py new file mode 100644 index 000000000..454552d68 --- /dev/null +++ b/test/test_room_board_protocol.py @@ -0,0 +1,241 @@ +#!/usr/bin/env python3 +"""Run actual bounded board LoRa/CLI wrappers against their real file store.""" + +from pathlib import Path +import os +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_room_board_store import FILESYSTEM_HARNESS, LFS_MOCK, STAT_MOCK + +ROOT = Path(__file__).resolve().parents[1] + +TESTS = r''' +#define FILE_O_WRITE 1 +#include +using Result = mesh::RoomBoardResult; +static unsigned scenarios = 0; +static Result save(MemoryFS& fs, uint8_t id, const std::string& title, const std::string& body) { + metadata_filesystem = &fs; + return mesh::saveRoomBoardArticle(&fs, id, title.c_str(), body.size(), + [&body](size_t offset, uint8_t* out, size_t length) -> size_t { + memcpy(out, body.data() + offset, length); return length; + }); +} +static std::vector indexRequest(uint32_t revision = 0, uint8_t start = 0) { + std::vector request(7); request[0] = 10; + mesh::room_board_detail::put32(request.data() + 2, revision); request[6] = start; return request; +} +static std::vector readRequest(uint8_t id, uint32_t version, uint16_t offset = 0) { + std::vector request(9); request[0] = 10; request[1] = 1; request[2] = id; + mesh::room_board_detail::put32(request.data() + 3, version); + mesh::room_board_detail::put16(request.data() + 7, offset); return request; +} +static std::vector request(MemoryFS& fs, const std::vector& payload, size_t capacity = 160) { + metadata_filesystem = &fs; + std::vector out(capacity + 2, 0x7f); + const size_t used = mesh::handleRoomBoardRequest(&fs, payload.data(), payload.size(), out.data() + 1, capacity); + assert(used <= capacity && out.front() == 0x7f && out.back() == 0x7f); + ++scenarios; return std::vector(out.begin() + 1, out.begin() + 1 + used); +} +static std::string command(MemoryFS& fs, const std::string& text, size_t capacity = 160, bool handled = true) { + metadata_filesystem = &fs; + std::vector out(capacity + 2, 0x7f); + assert(mesh::handleRoomBoardCommand(&fs, text.c_str(), out.data() + 1, capacity) == handled); + assert(out.front() == 0x7f && out.back() == 0x7f); + ++scenarios; + if (!capacity || !handled) return ""; + const void* terminated = memchr(out.data() + 1, 0, capacity); assert(terminated); + return out.data() + 1; +} +static std::string decode(const std::string& encoded) { + static const std::string alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + unsigned bits = 0, held = 0; std::string output; + for (char byte : encoded) { + if (byte == '=') break; + const size_t digit = alphabet.find(byte); assert(digit != std::string::npos); + bits = (bits << 6) | unsigned(digit); held += 6; + if (held >= 8) { held -= 8; output += char((bits >> held) & 255); } + } + return output; +} +static void checkIndex(const std::vector& reply, uint8_t start, uint8_t count) { + assert(reply.size() >= 10 && reply[0] == 10 && reply[1] == 0 && reply[2] == 0); + assert(reply[8] == start && reply[9] >= start && reply[9] <= start + 2 && reply[7] == count); + size_t cursor = 10; + for (uint8_t i = start; i < reply[9]; ++i) { + assert(cursor + 8 <= reply.size()); + assert(reply[cursor] == i + 1 && reply[cursor + 7] <= 63); + const size_t title_length = reply[cursor + 7]; + assert(cursor + 8 + title_length <= reply.size()); cursor += 8 + title_length; + } + assert(cursor == reply.size()); +} +int main() { + MemoryFS fs; + auto packet = request(fs, indexRequest()); checkIndex(packet, 0, 0); + assert(mesh::room_board_detail::get32(packet.data() + 3) == 0 && packet[9] == 0); + assert(command(fs, "get room.board") == "rev=0 total=0 start=0 next=0"); + assert(command(fs, "room.board.put 1 Instructions|Original body") == "OK rev=1"); + assert(command(fs, "room.board.put 2 Cafe |Unicode café 📡") == "OK rev=2"); + for (uint8_t id = 3; id <= 8; ++id) assert(save(fs, id, "Article " + std::to_string(id), std::string(2048, char('a' + id))) == Result::Success); + packet = request(fs, indexRequest()); checkIndex(packet, 0, 8); assert(packet[9] == 2); + assert(mesh::room_board_detail::get32(packet.data() + 3) == 8); + for (uint8_t start = 0; start <= 8; start += 2) checkIndex(request(fs, indexRequest(8, start)), start, 8); + packet = request(fs, indexRequest(0, 2)); assert(packet[2] == 5 && packet.size() == 10 && packet[9] == 2); + packet = request(fs, indexRequest(7)); assert(packet[2] == 5 && packet.size() == 10); + packet = request(fs, indexRequest(8, 9)); assert(packet[2] == 1 && packet.size() == 10); + assert(command(fs, "get room.board 2").find("start=2 next=4 3@") != std::string::npos); + + // Every possible transport capacity checks canaries and complete entries. + for (size_t capacity = 0; capacity <= 256; ++capacity) { + packet = request(fs, indexRequest(), capacity); + if (capacity < 3) assert(packet.empty()); + else if (capacity < 10) assert(packet.size() == 3 && packet[2] == 1); + else if (packet[2] == 0) checkIndex(packet, 0, 8); + else assert(packet.size() == 10 && packet[2] == 1 && packet[9] == 0); + packet = request(fs, readRequest(3, 3), capacity); + if (capacity < 3) assert(packet.empty()); + else if (capacity < 13) assert(packet.size() == 3 && packet[2] == 1); + else if (capacity == 13) assert(packet.size() == 13 && packet[2] == 1 && packet[12] == 0); + else { + const size_t wanted = std::min(size_t(128), capacity - 13); + assert(packet.size() == 13 + wanted && packet[2] == 0 && packet[12] == wanted); + assert(std::all_of(packet.begin() + 13, packet.end(), [](uint8_t byte) { return byte == 'd'; })); + } + } + for (size_t capacity = 0; capacity <= 160; ++capacity) { + const std::string listed = command(fs, "get room.board", capacity); + if (listed.find("rev=") == 0) assert(listed.find("next=") != std::string::npos); + const std::string part = command(fs, "get room.board.read 3 3 0", capacity); + if (part.find("id=") == 0) { + unsigned id, version, offset, total, length; + assert(sscanf(part.c_str(), "id=%u v=%u off=%u total=%u n=%u data=", &id, &version, &offset, &total, &length) == 5); + const size_t data = part.find("data="); assert(data != std::string::npos); + assert(id == 3 && version == 3 && offset == 0 && total == 2048 && length > 0 && length <= 80); + assert(decode(part.substr(data + 5)) == std::string(length, 'd')); + } + } + // Maximum titles still page by the emitted cursor, without dropping an item. + MemoryFS long_titles; + assert(save(long_titles, 1, std::string(63, 'x'), "a") == Result::Success); + assert(save(long_titles, 2, std::string(63, 'y'), "b") == Result::Success); + packet = request(long_titles, indexRequest(), 81); checkIndex(packet, 0, 2); assert(packet[9] == 1 && packet.size() == 81); + checkIndex(request(long_titles, indexRequest(2, 1), 81), 1, 2); + packet = request(long_titles, indexRequest(), 151); assert(packet[9] == 1); + packet = request(long_titles, indexRequest(), 152); assert(packet[9] == 2 && packet.size() == 152); + std::string listed = command(long_titles, "get room.board"); assert(listed.find("next=1") != std::string::npos); + listed = command(long_titles, "get room.board 1"); assert(listed.find("next=2") != std::string::npos && listed.find(std::string(63, 'y')) != std::string::npos); + + // Both binary and CLI paging reconstruct a full 2-KiB article exactly. + std::string assembled; + for (uint16_t offset = 0; offset < 2048;) { + packet = request(fs, readRequest(3, 3, offset), 103); + assert(packet[2] == 0 && packet[3] == 3 && mesh::room_board_detail::get32(packet.data() + 4) == 3); + assert(mesh::room_board_detail::get16(packet.data() + 8) == offset && mesh::room_board_detail::get16(packet.data() + 10) == 2048); + assembled.append(reinterpret_cast(packet.data() + 13), packet[12]); offset += packet[12]; + } + assert(assembled == std::string(2048, 'd')); + assembled.clear(); + for (size_t offset = 0; offset < 2048;) { + const std::string part = command(fs, "get room.board.read 3 3 " + std::to_string(offset)); + const size_t data = part.find("data="); assert(data != std::string::npos); + const std::string bytes = decode(part.substr(data + 5)); assert(!bytes.empty() && bytes.size() <= 80); + assembled += bytes; offset += bytes.size(); + } + assert(assembled == std::string(2048, 'd')); + packet = request(fs, readRequest(3, 3, 2048), 13); assert(packet[2] == 0 && packet.size() == 13 && packet[12] == 0); + packet = request(fs, readRequest(3, 3, 2049)); assert(packet[2] == 1 && packet.size() == 13); + assert(command(fs, "get room.board.read 3 3 2048").find("n=0 data=") != std::string::npos); + assert(command(fs, "get room.board.read 2 2 0").find("data=VW5pY29kZSBjYWbDqSDwn5Oh") != std::string::npos); + + // A mutation between index pages or article chunks cannot mix snapshots. + assert(save(fs, 3, "Changed", "replacement") == Result::Success); + packet = request(fs, indexRequest(8, 2)); assert(packet[2] == 5 && mesh::room_board_detail::get32(packet.data() + 3) == 9); + packet = request(fs, readRequest(3, 3, 90)); assert(packet[2] == 5 && packet[12] == 0 && mesh::room_board_detail::get32(packet.data() + 4) == 9); + assert(command(fs, "get room.board.read 3 3 90") == "Error stale version"); + assert(command(fs, "room.board.del 3 3") == "Error stale version"); + assert(command(fs, "room.board.del 3 9") == "OK rev=10"); + packet = request(fs, readRequest(3, 9)); assert(packet[2] == 4); + assert(command(fs, "room.board.put 3 Reused|new") == "OK rev=11"); + packet = request(fs, readRequest(3, 9)); assert(packet[2] == 5); + + const auto malformed = readRequest(1, 1); + for (size_t length = 0; length < malformed.size(); ++length) { + packet = request(fs, std::vector(malformed.begin(), malformed.begin() + length)); + assert(packet.size() == 3 && packet[2] == 1); + } + packet = request(fs, std::vector{10, 2}); assert(packet[2] == 1); + packet = request(fs, std::vector{9, 0}); assert(packet[2] == 1); + packet = request(fs, readRequest(0, 1)); assert(packet[2] == 1); + packet = request(fs, readRequest(1, 0)); assert(packet[2] == 1); + const unsigned writes = fs.write_opens; + for (const std::string text : {"room.board.put", "room.board.put 0 Title|body", "room.board.put 9 Title|body", "room.board.put 1 No separator", + "room.board.put 1 |body", "room.board.del 1", "room.board.del 1 0", "room.board.del 1 -1", "room.board.del 1 4294967296", + "room.board.del 1 1 trailing", "get room.board -1", "get room.board 9", "get room.board.read 1 1", + "get room.board.read 1 1 -1", "get room.board.read 1 1 2049", "get room.board.read 1 0 0", "get room.board.read 1 1 0 trailing"}) { + assert(command(fs, text) == "Error invalid"); + } + assert(command(fs, "room.board.put 1 " + std::string(64, 't') + "|body") == "Error invalid"); + assert(command(fs, "room.board.put 1 Title|" + std::string(513, 'b')) == "Error invalid"); + assert(command(fs, "room.board.put 1 " + std::string("\xed\xa0\x80") + "|body") == "Error invalid"); + assert(fs.write_opens == writes); + command(fs, "unrelated", 160, false); command(fs, "get room.boardwalk", 160, false); + command(fs, "room.board.putaway 1 Title|body", 160, false); + + // Fail-closed CRC checks apply before protocol reads and command writes. + mesh::RoomBoardIndex index; metadata_filesystem = &fs; assert(mesh::loadRoomBoard(&fs, index)); + char path[24]; mesh::room_board_detail::articlePath(index.articles[0], path); + auto damaged = fs.get(path); damaged.back() ^= 1; fs.put(path, damaged); + const auto primary = fs.get(mesh::ROOM_BOARD_PRIMARY_PATH); + packet = request(fs, indexRequest()); assert(packet[2] == 2 && packet.size() == 10); + packet = request(fs, readRequest(1, 1)); assert(packet[2] == 2 && packet[12] == 0); + assert(command(fs, "get room.board") == "Error unavailable"); + assert(command(fs, "get room.board.read 1 1 0") == "Error unavailable"); + assert(command(fs, "room.board.put 1 Changed|body") == "Error unavailable"); + assert(command(fs, "room.board.del 1 1") == "Error unavailable"); + assert(fs.get(mesh::ROOM_BOARD_PRIMARY_PATH) == primary && fs.get(path) == damaged && fs.write_opens == writes); + printf("PASS: %u actual room board LoRa/CLI protocol, bounds and stale-snapshot scenarios\n", scenarios); +} +''' + + +class RoomBoardProtocolTest(unittest.TestCase): + def test_real_protocol_and_store_all_backend_modes(self): + compiler = os.environ.get("CXX") or shutil.which("g++") or shutil.which("clang++") + if compiler is None: + self.skipTest("a host C++ compiler is required") + filesystem = FILESYSTEM_HARNESS.split("#define FILE_O_WRITE 1")[0] + filesystem = filesystem.replace(" void _lockFS() {}\n void _unlockFS() {}\n MemoryFS* _getFS() { return this; }\n", "", 1) + filesystem = filesystem.replace(" void point(const std::string& event) {", """ int lock_depth = 0; + void _lockFS() { assert(lock_depth++ == 0); } + void _unlockFS() { assert(--lock_depth == 0); } + MemoryFS* _getFS() { return this; } + void point(const std::string& event) {""") + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="meshcore-room-board-protocol-") as temporary: + work = Path(temporary) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + source = work / "protocol.cpp" + for platform in (None, "NRF52_PLATFORM", "STM32_PLATFORM", "RP2040_PLATFORM", "ESP32_PLATFORM"): + with self.subTest(platform=platform): + source.write_text(filesystem + (LFS_MOCK if platform in ("NRF52_PLATFORM", "STM32_PLATFORM") else "") + TESTS) + binary = work / (platform or "generic") + defines = ["-D" + platform + "=1"] if platform else [] + built = subprocess.run([compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", *sanitizers, + *defines, "-I" + str(work), "-I" + str(ROOT / "src"), str(source), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=60) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("actual room board LoRa/CLI protocol", checked.stdout) + print(checked.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_board_store.py b/test/test_room_board_store.py new file mode 100644 index 000000000..68a4309b6 --- /dev/null +++ b/test/test_room_board_store.py @@ -0,0 +1,322 @@ +#!/usr/bin/env python3 +"""Exercise the real bounded room board with faults, paging, and power cuts.""" + +from pathlib import Path +import os +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_room_topic_store import HARNESS as FILESYSTEM_HARNESS, STAT_MOCK + +ROOT = Path(__file__).resolve().parents[1] + +LFS_MOCK = r''' +struct lfs_info {}; +static constexpr int LFS_ERR_NOENT = -2; +static int lfs_stat(MemoryFS* fs, const char* path, struct lfs_info*) { + assert(fs && fs->lock_depth == 1); + const std::string op = std::string("stat:") + path; + const bool okay = fs->begin(op); + const bool present = fs->files.count(path) || fs->directories.count(path); + fs->end(op); + return !okay ? -5 : present ? 0 : LFS_ERR_NOENT; +} +''' + +TESTS = r''' +#define FILE_O_WRITE 1 +#include +using Result = mesh::RoomBoardResult; +using Index = mesh::RoomBoardIndex; +static const char* P = mesh::ROOM_BOARD_PRIMARY_PATH; +static const char* T = mesh::ROOM_BOARD_TEMP_PATH; +static const char* B = mesh::ROOM_BOARD_BACKUP_PATH; +static unsigned scenarios = 0; +static const std::string OLD_BODY = "Original room directions"; +static const std::string NEW_BODY = std::string(2048, 'n'); +static bool load(MemoryFS& fs, Index& index) { + metadata_filesystem = &fs; return mesh::loadRoomBoard(&fs, index); +} +static Result save(MemoryFS& fs, uint8_t id, const char* title, + const std::string& body, uint32_t version = 0, bool short_reader = false) { + metadata_filesystem = &fs; + return mesh::saveRoomBoardArticle(&fs, id, title, body.size(), + [&](size_t offset, uint8_t* out, size_t length) -> size_t { + assert(length <= 64 && offset + length <= body.size()); + memcpy(out, body.data() + offset, length); + return short_reader && length ? length - 1 : length; + }, version); +} +static Result erase(MemoryFS& fs, uint8_t id, uint32_t version = 0) { + metadata_filesystem = &fs; return mesh::deleteRoomBoardArticle(&fs, id, version); +} +static Result read(MemoryFS& fs, uint8_t id, uint32_t version, size_t offset, + uint8_t* out, size_t capacity, size_t& count) { + metadata_filesystem = &fs; + return mesh::readRoomBoardArticle(&fs, id, version, offset, out, capacity, count); +} +static MemoryFS clone(const MemoryFS& source) { + MemoryFS fs; fs.directories = source.directories; + for (const auto& file : source.files) fs.put(file.first.c_str(), *file.second); + return fs; +} +static std::map> snapshot(const MemoryFS& fs) { + std::map> result; + for (const auto& file : fs.files) result[file.first] = *file.second; + return result; +} +static Index loaded(MemoryFS& fs, uint32_t revision, uint8_t count) { + Index index; assert(load(fs, index)); + assert(index.revision == revision && index.count == count); return index; +} +static std::string bodyPath(const mesh::RoomBoardArticle& article) { + char path[24]; mesh::room_board_detail::articlePath(article, path); return path; +} +static void bodyIs(MemoryFS& fs, const mesh::RoomBoardArticle& article, const std::string& expected) { + assert(article.body_length == expected.size()); + std::string assembled; + for (size_t offset = 0; offset <= expected.size();) { + uint8_t out[39]; memset(out, 0x7f, sizeof(out)); size_t length = 999; + assert(read(fs, article.id, article.version, offset, out + 1, 37, length) == Result::Success); + assert(out[0] == 0x7f && out[38] == 0x7f && length <= 37); + assembled.append(reinterpret_cast(out + 1), length); + offset += length; + if (length == 0) break; + } + assert(assembled == expected); +} +static void checksum(std::vector& image) { + const uint32_t value = mesh::room_board_detail::crc(0xffffffffu, image.data(), image.size() - 4) ^ 0xffffffffu; + mesh::room_board_detail::put32(image.data() + image.size() - 4, value); +} +static void preserveUnavailable(MemoryFS& fs) { + const auto before = snapshot(fs); const auto directories = fs.directories; + Index index; index.count = 7; index.revision = 99; + assert(!load(fs, index) && index.count == 0 && index.revision == 0); + assert(save(fs, 1, "Changed", NEW_BODY) == Result::Unavailable); + assert(erase(fs, 1) == Result::Unavailable); + uint8_t out[128]; size_t count = 99; + assert(read(fs, 1, 1, 0, out, sizeof(out), count) == Result::Unavailable && count == 0); + assert(snapshot(fs) == before && fs.directories == directories && fs.write_opens == 0); + ++scenarios; +} +static void failedUpdate(MemoryFS& fs) { + assert(save(fs, 1, "New title", NEW_BODY) != Result::Success); + fs.reset(); Index index = loaded(fs, 1, 1); bodyIs(fs, index.articles[0], OLD_BODY); ++scenarios; +} +int main() { + MemoryFS initial; Index empty = loaded(initial, 0, 0); (void)empty; + assert(save(initial, 1, "Old title", OLD_BODY) == Result::Success); + Index old = loaded(initial, 1, 1); bodyIs(initial, old.articles[0], OLD_BODY); + initial.reset(); + const auto old_index = initial.get(P); + const std::string old_path = bodyPath(old.articles[0]); + const std::string new_path = "/room_board_1.b"; + + { MemoryFS fs = clone(initial); + assert(save(fs, 1, "New title", NEW_BODY, 1) == Result::Success); + Index next = loaded(fs, 2, 1); assert(next.articles[0].version == 2 && next.articles[0].bank == 1); + bodyIs(fs, next.articles[0], NEW_BODY); + assert(save(fs, 1, "Again", "third", 2) == Result::Success); + next = loaded(fs, 3, 1); assert(next.articles[0].bank == 0); bodyIs(fs, next.articles[0], "third"); + assert(erase(fs, 1, 3) == Result::Success); loaded(fs, 4, 0); + assert(save(fs, 1, "Replacement", "", 0) == Result::Success); + next = loaded(fs, 5, 1); bodyIs(fs, next.articles[0], ""); ++scenarios; + } + { MemoryFS fs; + for (uint8_t id = 8; id != 0; --id) assert(save(fs, id, "Directions", std::string(2048, char('a' + id))) == Result::Success); + Index index = loaded(fs, 8, 8); + for (uint8_t i = 0; i < 8; ++i) { assert(index.articles[i].id == i + 1); bodyIs(fs, index.articles[i], std::string(2048, char('a' + i + 1))); } + assert(fs.files.size() == 9); // One metadata image and one streamed body per article. + assert(erase(fs, 4, index.articles[3].version) == Result::Success); + index = loaded(fs, 9, 7); assert(index.articles[3].id == 5); + assert(save(fs, 4, "Returned", "body") == Result::Success); + index = loaded(fs, 10, 8); assert(index.articles[3].id == 4 && index.articles[3].version == 10); ++scenarios; + } + { MemoryFS fs = clone(initial); + const std::string maximum_title(63, 't'); assert(save(fs, 2, maximum_title.c_str(), "body") == Result::Success); + assert(save(fs, 3, u8"Café 地図 📡", "body") == Result::Success); + fs.reset(); const auto before = snapshot(fs); + for (const std::string& title : {std::string(""), std::string(64, 't'), std::string("\xc0\xaf"), std::string("\xed\xa0\x80"), std::string("\xf4\x90\x80\x80"), std::string("\xe2\x82"), std::string("line\nfeed")}) { + assert(save(fs, 1, title.c_str(), "body") == Result::Invalid); + } + assert(save(fs, 0, "Title", "body") == Result::Invalid); + assert(save(fs, 9, "Title", "body") == Result::Invalid); + assert(save(fs, 1, nullptr, "body") == Result::Invalid); + assert(save(fs, 1, "Title", std::string(2049, 'x')) == Result::Invalid); + assert(fs.trace.empty() && snapshot(fs) == before); ++scenarios; + } + { MemoryFS fs = clone(initial); uint8_t out[130]; size_t count; + assert(read(fs, 1, 2, 0, out, 128, count) == Result::StaleVersion && count == 0); + assert(read(fs, 2, 1, 0, out, 128, count) == Result::NotFound && count == 0); + assert(read(fs, 1, 0, 0, out, 128, count) == Result::Invalid && count == 0); + assert(read(fs, 1, 1, OLD_BODY.size() + 1, out, 128, count) == Result::Invalid && count == 0); + assert(read(fs, 1, 1, 0, out, 129, count) == Result::Invalid && count == 0); + assert(read(fs, 1, 1, 0, out, 0, count) == Result::Invalid && count == 0); + assert(read(fs, 1, 1, 0, nullptr, 128, count) == Result::Invalid && count == 0); + const auto before = snapshot(fs); + assert(save(fs, 1, "Changed", NEW_BODY, 2) == Result::StaleVersion); + assert(save(fs, 2, "Changed", NEW_BODY, 2) == Result::StaleVersion); + assert(erase(fs, 1, 2) == Result::StaleVersion && erase(fs, 2) == Result::NotFound); + assert(snapshot(fs) == before); + assert(save(fs, 1, "Changed", NEW_BODY, 1) == Result::Success); + assert(read(fs, 1, 1, 0, out, 128, count) == Result::StaleVersion && count == 0); + assert(erase(fs, 1, 2) == Result::Success); + assert(read(fs, 1, 1, 0, out, 128, count) == Result::NotFound && count == 0); + assert(save(fs, 1, "Reused", "new") == Result::Success); + assert(read(fs, 1, 1, 0, out, 128, count) == Result::StaleVersion && count == 0); ++scenarios; + } + { MemoryFS fs = clone(initial); auto image = fs.get(P); + mesh::room_board_detail::put32(image.data() + 4, UINT32_MAX); checksum(image); fs.put(P, image); + loaded(fs, UINT32_MAX, 1); const auto before = snapshot(fs); fs.reset(); + assert(save(fs, 1, "Changed", NEW_BODY) == Result::Unavailable); + assert(erase(fs, 1) == Result::Unavailable && fs.write_opens == 0 && snapshot(fs) == before); ++scenarios; + } + { Index index; assert(!mesh::loadRoomBoard(static_cast(nullptr), index)); + uint8_t out[8]; size_t count; + assert(mesh::readRoomBoardArticle(static_cast(nullptr), 1, 1, 0, out, 8, count) == Result::Unavailable); + assert(mesh::deleteRoomBoardArticle(static_cast(nullptr), 1) == Result::Unavailable); ++scenarios; + } + + // Every single-byte corruption in the real index/body must fail closed and + // preserve the primary plus its backup/temp evidence without any writes. + for (const std::string& path : {std::string(P), old_path}) { + const auto original = initial.get(path.c_str()); + for (size_t byte = 0; byte < original.size(); ++byte) { + MemoryFS fs = clone(initial); auto corrupted = original; corrupted[byte] ^= 1; + fs.put(path.c_str(), corrupted); fs.put(B, old_index); fs.put(T, old_index); preserveUnavailable(fs); + } + for (const auto& invalid : {std::vector{}, std::vector{1}, std::vector(original.begin(), original.end() - 1)}) { + MemoryFS fs = clone(initial); fs.put(path.c_str(), invalid); fs.put(B, old_index); fs.put(T, old_index); preserveUnavailable(fs); + } + auto trailing = original; trailing.push_back(0); + MemoryFS fs = clone(initial); fs.put(path.c_str(), trailing); fs.put(B, old_index); preserveUnavailable(fs); + auto future = original; future[3] = 2; checksum(future); + fs = clone(initial); fs.put(path.c_str(), future); fs.put(B, old_index); preserveUnavailable(fs); + fs = clone(initial); fs.directories.insert(path); preserveUnavailable(fs); + for (const std::string operation : {"open:r:", "read:", "size:"}) { + fs = clone(initial); fs.faults.insert(operation + path); preserveUnavailable(fs); + } + } + for (const auto& mutation : {std::make_pair(size_t(9), uint8_t(1)), std::make_pair(size_t(8), uint8_t(9)), + std::make_pair(size_t(16), uint8_t(0)), std::make_pair(size_t(17), uint8_t(2)), + std::make_pair(size_t(19), uint8_t(9)), std::make_pair(size_t(20), uint8_t(0)), + std::make_pair(size_t(24), uint8_t(0)), std::make_pair(size_t(70), uint8_t('x'))}) { + MemoryFS fs = clone(initial); auto image = old_index; image[mutation.first] = mutation.second; checksum(image); + fs.put(P, image); preserveUnavailable(fs); + } + { MemoryFS fs = clone(initial); auto image = fs.get(old_path.c_str()); image[12] = 1; checksum(image); + fs.put(old_path.c_str(), image); preserveUnavailable(fs); } + { MemoryFS fs = clone(initial); fs.put(B, old_index); fs.files.erase(P); + Index index = loaded(fs, 1, 1); bodyIs(fs, index.articles[0], OLD_BODY); assert(!fs.files.count(B)); ++scenarios; } + { MemoryFS fs; fs.put(T, old_index); fs.put(old_path.c_str(), initial.get(old_path.c_str())); + loaded(fs, 0, 0); assert(!fs.files.count(P) && !fs.files.count(T)); ++scenarios; } + { MemoryFS fs = clone(initial); fs.files.erase(P); auto future = old_index; future[3] = 2; checksum(future); + fs.put(B, future); fs.put(T, old_index); preserveUnavailable(fs); } + + for (const std::string& fault : {std::string("open:w:") + new_path, std::string("write:") + new_path, + std::string("flush:") + new_path, std::string("close:w:") + new_path, + std::string("open:r:") + new_path, std::string("read:") + new_path, std::string("size:") + new_path, + std::string("directory:") + new_path, std::string("open:w:") + T, std::string("write:") + T, + std::string("flush:") + T, std::string("close:w:") + T, std::string("open:r:") + T, + std::string("read:") + T, std::string("size:") + T, + std::string("rename:") + P + ":" + B, std::string("rename:") + T + ":" + P}) { + MemoryFS fs = clone(initial); fs.faults.insert(fault); failedUpdate(fs); + } + { MemoryFS fs = clone(initial); fs.short_write = true; failedUpdate(fs); } + { MemoryFS fs = clone(initial); fs.short_read = true; failedUpdate(fs); } + { MemoryFS fs = clone(initial); assert(save(fs, 1, "Changed", NEW_BODY, 0, true) == Result::WriteFailure); + fs.reset(); loaded(fs, 1, 1); ++scenarios; } + { MemoryFS fs = clone(initial); fs.replacement_on_close = initial.get(old_path.c_str()); failedUpdate(fs); } + // A full filesystem can truncate any streamed body or metadata boundary. + for (size_t budget : {size_t(0), size_t(15), size_t(16), size_t(100), size_t(2067), size_t(2068), size_t(2100), size_t(2159)}) { + MemoryFS fs = clone(initial); fs.write_budget = budget; failedUpdate(fs); + } + { MemoryFS fs = clone(initial); fs.put(new_path.c_str(), std::vector{1}); + fs.faults.insert("remove:" + new_path); failedUpdate(fs); } + for (const char* artifact : {T, B}) { + MemoryFS fs = clone(initial); fs.put(artifact, old_index); fs.faults.insert(std::string("remove:") + artifact); + loaded(fs, 1, 1); fs.write_opens = 0; assert(save(fs, 1, "Changed", NEW_BODY) == Result::Unavailable && fs.write_opens == 0); + fs.reset(); loaded(fs, 1, 1); ++scenarios; + } + { MemoryFS fs = clone(initial); fs.faults.insert(std::string("remove:") + B); + assert(save(fs, 1, "Changed", NEW_BODY) == Result::Success); + loaded(fs, 2, 1); assert(fs.files.count(B)); + assert(save(fs, 1, "Changed again", "x") == Result::Unavailable); + fs.reset(); loaded(fs, 2, 1); ++scenarios; } + { MemoryFS fs = clone(initial); fs.faults.insert(std::string("rename:") + T + ":" + P); + fs.faults.insert(std::string("rename:") + B + ":" + P); fs.faults.insert(std::string("remove:") + T); + assert(save(fs, 1, "Changed", NEW_BODY) == Result::WriteFailure && !fs.files.count(P) && fs.files.count(B)); + fs.reset(); Index index = loaded(fs, 1, 1); bodyIs(fs, index.articles[0], OLD_BODY); ++scenarios; } +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) || defined(ESP32_PLATFORM) + { MemoryFS fs = clone(initial); fs.faults.insert(std::string("stat:") + P); preserveUnavailable(fs); } + { MemoryFS fs = clone(initial); fs.files.erase(P); fs.put(B, old_index); + fs.faults.insert(std::string("stat:") + B); preserveUnavailable(fs); } +#endif + + // Cut before and after every filesystem operation in publish, update, and + // delete. Reboot may expose only the preceding image or the committed image. + for (unsigned mode = 0; mode < 3; ++mode) { + MemoryFS baseline = mode == 0 ? MemoryFS{} : clone(initial); + assert((mode == 2 ? erase(baseline, 1) : save(baseline, 1, "New title", NEW_BODY)) == Result::Success); + const auto committed_image = baseline.get(P); const size_t boundaries = baseline.boundary; + for (size_t cut = 1; cut <= boundaries; ++cut) { + MemoryFS fs = mode == 0 ? MemoryFS{} : clone(initial); fs.cut_at = cut; + try { if (mode == 2) erase(fs, 1); else save(fs, 1, "New title", NEW_BODY); } catch (const PowerCut&) {} + const bool committed = fs.files.count(P) && fs.get(P) == committed_image; + fs.reset(); + Index index = loaded(fs, committed ? (mode == 0 ? 1 : 2) : (mode == 0 ? 0 : 1), + committed && mode == 2 ? 0 : (mode == 0 && !committed ? 0 : 1)); + if (index.count) bodyIs(fs, index.articles[0], committed ? NEW_BODY : OLD_BODY); + assert(!fs.files.count(T) && !fs.files.count(B)); ++scenarios; + } + } + printf("PASS: %u room board persistence, version/paging, fault and power-cut scenarios\n", scenarios); +} +''' + + +class RoomBoardStoreTest(unittest.TestCase): + def test_real_store_with_all_backend_modes_and_faults(self): + compiler = os.environ.get("CXX") or shutil.which("g++") or shutil.which("clang++") + if compiler is None: + self.skipTest("a host C++ compiler is required") + filesystem = FILESYSTEM_HARNESS.split("#define FILE_O_WRITE 1")[0] + filesystem = filesystem.replace(" void _lockFS() {}\n void _unlockFS() {}\n MemoryFS* _getFS() { return this; }\n", "", 1) + filesystem = filesystem.replace(" void point(const std::string& event) {", """ int lock_depth = 0; + void _lockFS() { assert(lock_depth++ == 0); } + void _unlockFS() { assert(--lock_depth == 0); } + MemoryFS* _getFS() { return this; } + size_t write_budget = std::numeric_limits::max(); + void point(const std::string& event) {""") + filesystem = filesystem.replace(" bytes->insert(bytes->end(), data, data + length);", """ length = std::min(length, fs->write_budget); + fs->write_budget -= length; + bytes->insert(bytes->end(), data, data + length);""") + filesystem = filesystem.replace("void reset() { trace.clear();", "void reset() { lock_depth = 0; write_budget = std::numeric_limits::max(); trace.clear();") + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="meshcore-room-board-") as temporary: + work = Path(temporary) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + source = work / "board.cpp" + for platform in (None, "NRF52_PLATFORM", "STM32_PLATFORM", "RP2040_PLATFORM", "ESP32_PLATFORM"): + with self.subTest(platform=platform): + source.write_text(filesystem + (LFS_MOCK if platform in ("NRF52_PLATFORM", "STM32_PLATFORM") else "") + TESTS) + binary = work / (platform or "generic") + defines = ["-D" + platform + "=1"] if platform else [] + built = subprocess.run([compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", *sanitizers, + *defines, "-I" + str(work), "-I" + str(ROOT / "src"), str(source), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=60) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("room board persistence, version/paging, fault and power-cut scenarios", checked.stdout) + print(checked.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_history_integration.py b/test/test_room_history_integration.py new file mode 100644 index 000000000..123830de2 --- /dev/null +++ b/test/test_room_history_integration.py @@ -0,0 +1,59 @@ +#!/usr/bin/env python3 +"""Execute actual room history admission, commands and boot loading with the real store.""" +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_replay_reset_integration import extract_braced +from test_room_history_store import STAT_MOCK + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/room_history_integration/test.cpp" + + +class RoomHistoryIntegrationTests(unittest.TestCase): + def test_actual_post_admission_persistence_commands_and_boot(self): + compiler = shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler, "a host C++ compiler is required") + source = (ROOT / "examples/simple_room_server/MyMesh.cpp").read_text() + header = (ROOT / "examples/simple_room_server/MyMesh.h").read_text() + acl = (ROOT / "src/helpers/ClientACL.h").read_text() + text = (ROOT / "src/helpers/TxtDataHelpers.cpp").read_text() + state = "\n".join(line for line in acl.splitlines() if line.startswith("#define PERM_ACL_")) + state += "\n" + extract_braced(header, "struct PostInfo") + ";\n" + definitions = "\n".join(extract_braced(source, signature) for signature in ( + "bool MyMesh::addPost(", "bool MyMesh::addSystemPost(", "bool MyMesh::storePost(", + "bool MyMesh::snapshotRoomHistory(", "void MyMesh::loadRoomHistory(", + "bool MyMesh::handleRoomHistoryCommand(")) + definitions += "\n" + extract_braced(text, "void StrHelper::strncpy(") + receive = extract_braced(source, "void MyMesh::onPeerDataRecv(") + gate = extract_braced(receive, "if (flags == TXT_TYPE_PLAIN)") + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", + "-fno-pie", "-no-pie"] if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="room-history-integration-") as directory: + work = Path(directory) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + (work / "state.inc").write_text(state) + (work / "production.inc").write_text(definitions) + (work / "post_gate.inc").write_text(gate) + for platform in (None, "NRF52_PLATFORM", "STM32_PLATFORM", "RP2040_PLATFORM", "ESP32_PLATFORM"): + with self.subTest(platform=platform): + binary = work / (platform or "generic") + defines = ["-D" + platform + "=1"] if platform else [] + built = subprocess.run([compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", + "-Wno-unused-parameter", *sanitizers, *defines, f"-I{work}", f"-I{ROOT / 'src'}", + f"-I{ROOT / 'test/mocks'}", str(FIXTURE), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + tested = subprocess.run([str(binary)], capture_output=True, text=True, timeout=30) + self.assertEqual(tested.returncode, 0, tested.stdout + tested.stderr) + self.assertIn("production room history integration scenarios passed", tested.stdout) + print((platform or "generic") + ": " + tested.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_history_store.py b/test/test_room_history_store.py new file mode 100644 index 000000000..b73035940 --- /dev/null +++ b/test/test_room_history_store.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +"""Compile the actual bounded journal with fault injection on five FS APIs.""" + +from pathlib import Path +import os +import shutil +import struct +import subprocess +import sys +import tempfile +import unittest +import zlib + +ROOT = Path(__file__).resolve().parents[1] +FIXTURES = ROOT / "test/fixtures/room_history_store" + +STAT_MOCK = r''' +#pragma once +struct stat {}; +inline int stat(const char* absolute, struct stat*) { + if (!metadata_filesystem || strncmp(absolute, "/spiffs", 7) != 0) { errno = EIO; return -1; } + const char* path = absolute + 7; + const std::string op = std::string("stat:") + path; + const bool okay = metadata_filesystem->begin(op); + const bool present = metadata_filesystem->files.count(path) || metadata_filesystem->directories.count(path); + metadata_filesystem->end(op); + if (!okay) { errno = EIO; return -1; } + if (!present) { errno = ENOENT; return -1; } + return 0; +} +''' + + +def crc_image(payload): + return payload + struct.pack(" " + name + " = {" + + ",".join(str(byte) for byte in image) + "};" + for name, image in vectors.items()) + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", + "-fno-pie", "-no-pie"] if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="meshcore-room-history-") as temporary: + work = Path(temporary) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + (work / "vectors.inc").write_text(rendered) + for platform in (None, "NRF52_PLATFORM", "STM32_PLATFORM", "RP2040_PLATFORM", "ESP32_PLATFORM"): + with self.subTest(platform=platform): + binary = work / (platform or "generic") + defines = ["-D" + platform + "=1"] if platform else [] + built = subprocess.run([compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", + *sanitizers, *defines, "-I" + str(work), "-I" + str(ROOT / "src"), + "-I" + str(FIXTURES), str(FIXTURES / "test.cpp"), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=30) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("bounded room history fault and recovery scenarios", checked.stdout) + print(checked.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_login_integration.py b/test/test_room_login_integration.py index 1737df724..2afb5cadc 100644 --- a/test/test_room_login_integration.py +++ b/test/test_room_login_integration.py @@ -19,6 +19,12 @@ HARNESS = r''' #include #include #include +#include "filesystem.h" +#include +#include +#include +#include +#include #include #include #include @@ -80,6 +86,8 @@ struct FakeACL : ClientACL { bool known = false, allocate_ok = true; unsigned lookups = 0, authorizations = 0, allocations = 0; uint8_t authorized_role = 255; + int getNumClients() const { return known ? 1 : 0; } + ClientInfo* getClientByIdx(int i) { assert(i == 0 && known); return &client; } ClientInfo* getClient(const uint8_t* key, size_t length) { ++lookups; assert(length == PUB_KEY_SIZE); @@ -117,8 +125,28 @@ struct FakeRng { memset(out, 0xB5, length); } }; +static uint32_t ticks = 0; +static uint32_t millis() { return ticks; } struct MyMesh { FakeACL acl; + MemoryFS policy_fs; + mesh::RoomAccessPolicy room_access; + MyMesh() { metadata_filesystem = &policy_fs; assert(room_access.load(&policy_fs)); } + int matching_peer_indexes[1] = {0}; + mesh::RemoteCliReplyCache remote_cli_reply_cache; + mesh::LogicalMessageCache recent_room_polls; + void serviceRoomQuotas(); + static bool saveFilter(ClientInfo*); + unsigned path_acks = 0; + bool processAck(const uint8_t*) { ++path_acks; return true; } + uint8_t getUnsyncedCount(ClientInfo*) { return 2; } + int getExtraAckTransmitCount() { return 0; } + unsigned requests = 0; + uint8_t last_request_type = 0; + int handleRequest(ClientInfo*, uint32_t, uint8_t* payload, size_t, size_t) { ++requests; last_request_type = payload[0]; return 0; } + void handleCommand(uint32_t, char*, char* reply, int, uint8_t) { strcpy(reply, "OK"); } + mesh::Packet* createAck(uint32_t ack) { ++post_acks; response.payload_len = 4; memcpy(response.payload, &ack, 4); return &response; } + mesh::Packet* createMultiAck(uint32_t ack, uint8_t) { return createAck(ack); } struct Prefs { char password[32] = "admin"; char guest_password[32] = "guest"; @@ -145,7 +173,8 @@ struct MyMesh { mesh::Packet* createDatagram(uint8_t type, const mesh::Identity& sender, const uint8_t* secret, const uint8_t* data, size_t len) { ++creations; - assert(type == PAYLOAD_TYPE_RESPONSE && len == 13); + assert(type == PAYLOAD_TYPE_RESPONSE || type == PAYLOAD_TYPE_TXT_MSG); + assert(len <= sizeof(captured_reply)); memcpy(response_key, sender.pub_key, PUB_KEY_SIZE); memcpy(response_secret, secret, PUB_KEY_SIZE); memcpy(captured_reply, data, len); @@ -161,29 +190,48 @@ struct MyMesh { } void sendDirect(mesh::Packet* packet, const uint8_t* path, uint8_t path_len, unsigned long delay) { - assert(packet == &response && delay == SERVER_RESPONSE_DELAY); + assert(packet == &response && (delay == SERVER_RESPONSE_DELAY || delay == TXT_ACK_DELAY)); assert(mesh::Packet::isValidPathLen(path_len)); ++direct_sends; direct_path_len = path_len; memcpy(direct_path, path, (path_len & 63) * ((path_len >> 6) + 1)); } void sendFloodReply(mesh::Packet* packet, unsigned long delay, uint8_t hash_size) { - assert(packet == &response && delay == SERVER_RESPONSE_DELAY); + assert(packet == &response && (delay == SERVER_RESPONSE_DELAY || delay == TXT_ACK_DELAY)); ++flood_sends; reply_hash_size = hash_size; } - void addPost(ClientInfo*, const char*) { ++posts; } - void post(ClientInfo* client, const char* text, uint32_t sender_timestamp) { - const uint8_t flags = TXT_TYPE_PLAIN; - const size_t text_len = strlen(text); - bool send_ack = false; - @POST_GATE@ - if (send_ack) ++post_acks; + bool store_ok = true; + bool addPost(ClientInfo*, const char*) { if (!store_ok) return false; ++posts; return true; } + void post(ClientInfo*, const char* text, uint32_t sender_timestamp) { + std::vector data(5 + strlen(text) + 1); + memcpy(data.data(), &sender_timestamp, 4); data[4] = TXT_TYPE_PLAIN << 2; + memcpy(data.data() + 5, text, strlen(text)); + mesh::Packet packet; packet.header = ROUTE_TYPE_DIRECT; + onPeerDataRecv(&packet, PAYLOAD_TYPE_TXT_MSG, 0, acl.client.shared_secret, data.data(), data.size() - 1); } + void poll(uint32_t timestamp, uint32_t since = 0, bool flood = false) { + uint8_t data[10] = {}; memcpy(data, ×tamp, 4); data[4] = REQ_TYPE_KEEP_ALIVE; + memcpy(data + 5, &since, 4); + mesh::Packet packet; packet.header = flood ? ROUTE_TYPE_FLOOD : ROUTE_TYPE_DIRECT; + onPeerDataRecv(&packet, PAYLOAD_TYPE_REQ, 0, acl.client.shared_secret, data, 9); + } + void request(uint32_t timestamp, uint8_t subtype, uint8_t argument = 0) { + uint8_t data[8] = {}; memcpy(data, ×tamp, 4); data[4] = subtype; + data[5] = argument; + mesh::Packet packet; packet.header = ROUTE_TYPE_DIRECT; + onPeerDataRecv(&packet, PAYLOAD_TYPE_REQ, 0, acl.client.shared_secret, data, sizeof(data)); + } + void onPeerDataRecv(mesh::Packet*, uint8_t, int, const uint8_t*, uint8_t*, size_t); + bool onPeerPathRecv(mesh::Packet*, int, const uint8_t*, uint8_t*, uint8_t, uint8_t, uint8_t*, uint8_t); void onAnonDataRecv(mesh::Packet*, const uint8_t*, const mesh::Identity&, uint8_t*, size_t); }; @LOGIN_HANDLER@ +@PEER_HANDLER@ +@PATH_HANDLER@ +@QUOTA_SERVICE@ +@SAVE_FILTER@ static mesh::Identity identity(uint8_t n = 1) { mesh::Identity sender{}; @@ -210,6 +258,8 @@ static void seedKnown(MyMesh& mesh, uint8_t permissions = PERM_ACL_ADMIN) { client.extra.room.topic_seen_revision = 11; client.extra.room.pending_topic_revision = 12; client.extra.room.topic_failures = 2; + client.extra.room.post_quota_used = 9; + client.extra.room.poll_quota_used = 8; mesh.acl.known = true; } static mesh::Packet incoming(bool flood = false) { @@ -235,7 +285,7 @@ static void login(MyMesh& mesh, const char* password = "guest", uint32_t timesta memset(secret, 0x77, sizeof(secret)); auto packet = incoming(flood); mesh.onAnonDataRecv(&packet, secret, identity(), data.data(), data.size() - 1); - assert(data.back() == 0); + if (mesh.room_access.allowsIdentity(identity().pub_key)) assert(data.back() == 0); } static void checkReply(const MyMesh& mesh, uint8_t permissions) { assert(mesh.creations == 1 && mesh.captured_reply_length == 13); @@ -325,8 +375,10 @@ static void preservedRoles() { MyMesh mesh; seedKnown(mesh, permissions); mesh._prefs.allow_read_only = reader; + mesh.acl.client.permissions_are_explicit = reader; login(mesh, password); assert(mesh.acl.client.permissions == permissions); + assert(mesh.acl.client.permissions_are_explicit == reader); assert(mesh.acl.authorized_role == (permissions & PERM_ACL_ROLE_MASK)); assert(mesh.acl.allocations == 0 && mesh.acl.client.last_timestamp == 100); checkReply(mesh, permissions); @@ -345,9 +397,11 @@ static void preservedRoles() { // A valid explicit Admin credential replaces delegated labels 4/5, too. MyMesh mesh; seedKnown(mesh, permissions); + mesh.acl.client.permissions_are_explicit = true; login(mesh, "admin"); const uint8_t expected = (permissions & ~PERM_ACL_ROLE_MASK) | PERM_ACL_ADMIN; assert(mesh.acl.client.permissions == expected); + assert(mesh.acl.client.permissions_are_explicit); checkReply(mesh, expected); } assert(preserved == 2048); @@ -438,6 +492,7 @@ static void sessionRefresh() { assert(client.extra.room.pending_ack == 0 && client.extra.room.push_failures == 0); assert(client.extra.room.topic_seen_revision == 0 && client.extra.room.pending_topic_revision == 0); assert(client.extra.room.topic_failures == 0); + assert(client.extra.room.post_quota_used == 9 && client.extra.room.poll_quota_used == 8); for (uint8_t value : client.shared_secret) assert(value == 0x77); assert(mesh.next_push == mesh.millis_now + PUSH_NOTIFY_DELAY_MILLIS); assert(mesh.dirty_contacts_expiry == mesh.millis_now + LAZY_CONTACTS_WRITE_DELAY); @@ -514,6 +569,119 @@ static void readOnlyPosting() { puts("actual room post gate blocks logged-in readers/managers and accepts RW/Admin once passed"); } + +static void moderationCases() { + for (bool known : {false, true}) for (const char* password : {"", "guest", "admin"}) { + MyMesh mesh; + if (known) seedKnown(mesh, PERM_ACL_ADMIN); + mesh._prefs.allow_read_only = true; + assert(mesh.room_access.addBan(&mesh.policy_fs, identity().pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + const ClientInfo before = mesh.acl.client; + login(mesh, password); + checkRejectedSession(mesh, before); + assert(mesh.acl.authorizations == 0 && mesh.acl.allocations == 0 && mesh.acl.lookups == 0); + assert(mesh.acl.storage.reads == 0 && mesh.acl.storage.writes == 0); + if (known) { + mesh.post(&mesh.acl.client, "banned", 1000); + mesh.poll(1000, 900); + assert(mesh.posts == 0 && mesh.post_acks == 0); + assert(memcmp(&mesh.acl.client, &before, sizeof(before)) == 0); + auto packet = incoming(); uint8_t path[] = {0x11, 0x22}, ack[4] = {}; + assert(!mesh.onPeerPathRecv(&packet, 0, mesh.acl.client.shared_secret, + path, 2, PAYLOAD_TYPE_ACK, ack, sizeof(ack))); + assert(mesh.path_acks == 0 && memcmp(&mesh.acl.client, &before, sizeof(before)) == 0); + } + assert(mesh.room_access.removeBan(&mesh.policy_fs, identity().pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + login(mesh, password); + assert(mesh.acl.known && mesh.creations == 1); + } + // A 31-byte collision is a different identity; bans always use the full key. + MyMesh collision; auto other = identity(); other.pub_key[31] ^= 1; + assert(collision.room_access.addBan(&collision.policy_fs, other.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + login(collision, "admin"); assert(collision.acl.known && collision.creations == 1); + // A bad policy image must not silently admit an Admin or public reader. + MyMesh unavailable; + unavailable.policy_fs.put(mesh::ROOM_ACCESS_PRIMARY_PATH, {1,2,3}); + assert(!unavailable.room_access.load(&unavailable.policy_fs)); + unavailable._prefs.allow_read_only = true; + login(unavailable, "admin"); + assert(!unavailable.acl.known && unavailable.acl.lookups == 0 && unavailable.creations == 0); + puts("actual room gates deny full-key bans on login, posts, polls, and paths passed"); +} + +static void quotaCases() { + MyMesh mesh; ticks = 100; seedKnown(mesh, PERM_ACL_ADMIN); + assert(mesh.room_access.setRates(&mesh.policy_fs, 1, 1)); + mesh.post(&mesh.acl.client, "one", 1000); + assert(mesh.posts == 1 && mesh.post_acks == 1 && mesh.acl.client.extra.room.post_quota_used == 1); + assert(mesh.acl.client.extra.room.poll_quota_used == 0); + login(mesh, "guest", 100); + assert(mesh.acl.client.extra.room.post_quota_used == 1); // Re-login never bypasses a posting quota. + mesh.post(&mesh.acl.client, "two", 1001); + assert(mesh.posts == 1 && mesh.post_acks == 1 && mesh.acl.client.extra.room.last_post_timestamp == 1000); + mesh.post(&mesh.acl.client, "one", 1000); + assert(mesh.posts == 1 && mesh.post_acks == 2 && mesh.acl.client.extra.room.post_quota_used == 1); + mesh.poll(1001, 50); + assert(mesh.post_acks == 3 && mesh.acl.client.extra.room.poll_quota_used == 1); + assert(mesh.acl.client.last_timestamp == 1001 && mesh.acl.client.extra.room.sync_since == 50); + mesh.acl.client.extra.room.pending_ack = 0x12345678; + mesh.acl.client.extra.room.pending_topic_revision = 55; + const auto poll_retry_before = mesh.acl.client; + mesh.poll(1001, 50); // Even equal-time retries cannot cancel a newer outstanding delivery. + assert(mesh.post_acks == 4 && mesh.acl.client.extra.room.poll_quota_used == 1); + assert(memcmp(&poll_retry_before, &mesh.acl.client, sizeof(poll_retry_before)) == 0); + mesh.poll(1002, 60); + assert(mesh.post_acks == 4 && mesh.acl.client.last_timestamp == 1001 && mesh.acl.client.extra.room.sync_since == 50); + login(mesh, "guest", 1002, 80); + assert(mesh.acl.client.extra.room.post_quota_used == 1 && mesh.acl.client.extra.room.poll_quota_used == 1); + mesh.poll(1001, 50); // Old exact retry ACK cannot rewind a refreshed/newer cursor. + assert(mesh.post_acks == 5 && mesh.acl.client.extra.room.sync_since == 80); + mesh.poll(1001, 99); // Changed content at an old timestamp isn't a cached retry. + assert(mesh.post_acks == 5 && mesh.acl.client.extra.room.sync_since == 80); + ticks = 60100; + mesh.post(&mesh.acl.client, "two", 1001); + assert(mesh.posts == 2 && mesh.post_acks == 6 && mesh.acl.client.extra.room.last_post_timestamp == 1001); + assert(mesh.acl.client.extra.room.post_quota_used == 1 && mesh.acl.client.extra.room.poll_quota_used == 0); + mesh.poll(1002, 60); // Earlier quota rejection must not have entered the retry cache. + assert(mesh.post_acks == 7 && mesh.acl.client.extra.room.poll_quota_used == 1); + assert(mesh.acl.client.extra.room.sync_since == 60); + ticks = 120100; mesh.store_ok = false; + const auto post_floor = mesh.acl.client.extra.room.last_post_timestamp; + const auto last_activity = mesh.acl.client.last_activity; + mesh.post(&mesh.acl.client, "storage-fails", 1003); + assert(mesh.posts == 2 && mesh.post_acks == 7 && mesh.acl.client.extra.room.post_quota_used == 0); + assert(mesh.acl.client.extra.room.last_post_timestamp == post_floor && mesh.acl.client.last_activity == last_activity); + mesh.store_ok = true; mesh.post(&mesh.acl.client, "storage-fails", 1003); + assert(mesh.posts == 3 && mesh.post_acks == 8 && mesh.acl.client.extra.room.post_quota_used == 1); + const auto before = mesh.acl.client; + mesh.poll(1004, 1000, true); // Flood polls do not consume allowance or refresh replay/session state. + assert(mesh.post_acks == 8 && memcmp(&before, &mesh.acl.client, sizeof(before)) == 0); + puts("actual room quotas survive relogin, exempt exact retries, and reject without replay or ACK passed"); +} + +static void allRequestQuotaCases() { + for (uint8_t subtype : {uint8_t(REQ_TYPE_GET_STATUS), uint8_t(REQ_TYPE_GET_TELEMETRY_DATA), + uint8_t(REQ_TYPE_GET_ACCESS_LIST), uint8_t(ROOM_BOARD_REQUEST_SUBTYPE)}) { + MyMesh mesh; ticks = 100; seedKnown(mesh, PERM_ACL_ADMIN); + assert(mesh.room_access.setRates(&mesh.policy_fs, 0, 1)); + mesh.request(1000, subtype, 8); + assert(mesh.requests == 1 && mesh.last_request_type == subtype && mesh.acl.client.extra.room.poll_quota_used == 1); + assert(mesh.acl.client.last_timestamp == 1000); + mesh.request(1000, subtype, 8); // Exact accepted request is re-readable without a second charge. + assert(mesh.requests == 2 && mesh.acl.client.extra.room.poll_quota_used == 1); + const auto before_reject = mesh.acl.client; + mesh.request(1001, subtype, 8); // New timestamp consumes a new allowance and is denied. + mesh.request(1000, subtype, 9); // Changed payload cannot masquerade as an exact retry. + assert(mesh.requests == 2 && memcmp(&before_reject, &mesh.acl.client, sizeof(before_reject)) == 0); + ticks = 60100; mesh.request(1001, subtype, 8); + assert(mesh.requests == 3 && mesh.acl.client.last_timestamp == 1001 && mesh.acl.client.extra.room.poll_quota_used == 1); + mesh.request(1000, subtype, 8); // Older accepted read may repeat without lowering the session replay floor. + assert(mesh.requests == 4 && mesh.acl.client.last_timestamp == 1001 && mesh.acl.client.extra.room.poll_quota_used == 1); + mesh.request(1000, subtype, 9); + assert(mesh.requests == 4 && mesh.acl.client.last_timestamp == 1001); + } + puts("actual request quota covers status, telemetry, ACL, board, and exact payload retries passed"); +} int main(int argc, char** argv) { assert(argc == 2); if (!strcmp(argv[1], "authorization")) authorizationCases(); @@ -521,6 +689,9 @@ int main(int argc, char** argv) { else if (!strcmp(argv[1], "storage")) storageFailures(); else if (!strcmp(argv[1], "refresh")) sessionRefresh(); else if (!strcmp(argv[1], "post")) readOnlyPosting(); + else if (!strcmp(argv[1], "moderation")) moderationCases(); + else if (!strcmp(argv[1], "quotas")) quotaCases(); + else if (!strcmp(argv[1], "requests")) allRequestQuotaCases(); else assert(false); } ''' @@ -542,7 +713,8 @@ class RoomLoginIntegrationTests(unittest.TestCase): constants = "\n".join(line for line in acl_header.splitlines() if line.startswith(("#define PERM_ACL_", "#define OUT_PATH_"))) for name in ("PUSH_NOTIFY_DELAY_MILLIS", "FIRMWARE_VER_LEVEL", "RESP_SERVER_LOGIN_OK", - "LAZY_CONTACTS_WRITE_DELAY", "ROOM_MESSAGE_CACHE_SIZE", "TXT_TYPE_PLAIN"): + "LAZY_CONTACTS_WRITE_DELAY", "ROOM_MESSAGE_CACHE_SIZE", "TXT_TYPE_PLAIN", + "TXT_TYPE_CLI_DATA", "TXT_TYPE_CLI_COMMAND", "TXT_ACK_DELAY", "REPLY_DELAY_MILLIS", "REQ_TYPE_KEEP_ALIVE", "REQ_TYPE_GET_STATUS", "REQ_TYPE_GET_TELEMETRY_DATA", "REQ_TYPE_GET_ACCESS_LIST"): match = re.search(r"^\s*#define\s+" + name + r"\s+.*$", room, re.MULTILINE) if match is None: for path in (ROOT / "examples/simple_room_server/MyMesh.h", ROOT / "src/Mesh.h", @@ -554,6 +726,11 @@ class RoomLoginIntegrationTests(unittest.TestCase): if match is None: raise AssertionError("production constant not found: " + name) constants += "\n" + match.group() + board = (ROOT / "src/helpers/RoomBoardProtocol.h").read_text() + board_subtype = re.search(r"static constexpr uint8_t ROOM_BOARD_REQUEST_SUBTYPE = ([^;]+);", board) + if board_subtype is None: + raise AssertionError("production board request subtype not found") + constants += "\n#define ROOM_BOARD_REQUEST_SUBTYPE " + board_subtype[1] delay = re.search(r"^\s*#define\s+SERVER_RESPONSE_DELAY\s+.*$", room_header, re.MULTILINE) if delay is None: raise AssertionError("production SERVER_RESPONSE_DELAY not found") @@ -564,8 +741,10 @@ class RoomLoginIntegrationTests(unittest.TestCase): "@PACKET_PATH_CHECK@": extract_braced(packet, "bool Packet::isValidPathLen("), "@CLIENT_INFO@": extract_braced(acl_header, "struct ClientInfo {"), "@REPLAY_HANDLER@": extract_braced(acl_source, "bool ClientACL::authorizeLoginTimestamp("), - "@POST_GATE@": extract_braced(extract_braced(room, "void MyMesh::onPeerDataRecv("), - "if (flags == TXT_TYPE_PLAIN)"), + "@PEER_HANDLER@": extract_braced(room, "void MyMesh::onPeerDataRecv("), + "@PATH_HANDLER@": extract_braced(room, "bool MyMesh::onPeerPathRecv("), + "@QUOTA_SERVICE@": extract_braced(room, "void MyMesh::serviceRoomQuotas("), + "@SAVE_FILTER@": extract_braced(room, "bool MyMesh::saveFilter("), "@LOGIN_HANDLER@": extract_braced(room, "void MyMesh::onAnonDataRecv("), } generated = HARNESS @@ -578,7 +757,7 @@ class RoomLoginIntegrationTests(unittest.TestCase): source.write_text(generated) cls.binary = work / "room-login" command = [compiler, "-std=c++17", "-O1", "-Wall", "-Wextra", "-Werror", - "-Wno-unused-parameter", "-I" + str(ROOT / "src"), + "-Wno-unused-parameter", "-I" + str(ROOT / "test/fixtures/room_history_store"), "-I" + str(ROOT / "src"), str(source), "-o", str(cls.binary)] if sys.platform.startswith("linux"): command[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", @@ -604,6 +783,15 @@ class RoomLoginIntegrationTests(unittest.TestCase): def test_successful_relogin_refreshes_session_and_routes_without_resetting_post_floor(self): self.run_case("refresh", "real-handler successful reconnect refreshes cursor, activity, secret, topic state, and route passed") + def test_all_room_request_types_consume_poll_budget_and_preserve_exact_retries(self): + self.run_case("requests", "actual request quota covers status, telemetry, ACL, board, and exact payload retries passed") + + def test_full_identity_bans_apply_to_new_and_existing_sessions(self): + self.run_case("moderation", "actual room gates deny full-key bans on login, posts, polls, and paths passed") + + def test_actual_post_and_poll_quotas_retries_and_relogin(self): + self.run_case("quotas", "actual room quotas survive relogin, exempt exact retries, and reject without replay or ACK passed") + def test_read_only_login_never_grants_post_permission(self): self.run_case("post", "actual room post gate blocks logged-in readers/managers and accepts RW/Admin once passed") diff --git a/test/test_room_post_permissions.py b/test/test_room_post_permissions.py index 734f5fd3b..16353eeee 100644 --- a/test/test_room_post_permissions.py +++ b/test/test_room_post_permissions.py @@ -16,6 +16,8 @@ HARNESS = r''' #include #include #include +#include "filesystem.h" +#include #include #define TXT_TYPE_PLAIN 0 #define ROOM_MESSAGE_CACHE_SIZE 8 @@ -35,13 +37,22 @@ struct Utils { struct ClientInfo { struct { uint8_t pub_key[PUB_KEY_SIZE] = {1}; } id; uint8_t permissions = 0; - struct { struct { uint32_t last_post_timestamp = 0; } room; } extra; + struct { struct { uint32_t last_post_timestamp = 0; uint16_t post_quota_used = 0, poll_quota_used = 0; } room; } extra; }; struct MyMesh { + MemoryFS fs; + mesh::RoomAccessPolicy room_access; + uint32_t ticks = 0; + ClientInfo* active = nullptr; + MyMesh() { metadata_filesystem = &fs; assert(room_access.load(&fs)); } + void serviceRoomQuotas() { room_access.serviceQuotaWindow(ticks, [this] { active->extra.room.post_quota_used = active->extra.room.poll_quota_used = 0; }); } mesh::LogicalMessageCache recent_room_posts; unsigned posts = 0, acks = 0; - void addPost(ClientInfo*, const char*) { ++posts; } + bool store_ok = true; + bool addPost(ClientInfo*, const char*) { if (!store_ok) return false; ++posts; return true; } void receive(ClientInfo* client, const char* text, uint32_t sender_timestamp) { + active = client; + @IDENTITY_GATE@ const uint8_t flags = TXT_TYPE_PLAIN; const size_t text_len = strlen(text); bool send_ack = false; @@ -77,6 +88,31 @@ int main() { assert(mesh.posts == 2 && mesh.acks == 4); } } + { + MyMesh mesh; ClientInfo client; client.permissions = PERM_ACL_READ_WRITE; + assert(mesh.room_access.setRates(&mesh.fs, 1, 2)); + mesh.receive(&client, "quota-one", 100); + assert(mesh.posts == 1 && mesh.acks == 1 && client.extra.room.post_quota_used == 1); + mesh.receive(&client, "quota-one", 100); + assert(mesh.posts == 1 && mesh.acks == 2 && client.extra.room.post_quota_used == 1); + mesh.receive(&client, "quota-two", 101); + assert(mesh.posts == 1 && mesh.acks == 2 && client.extra.room.last_post_timestamp == 100); + mesh.ticks = 60000; mesh.receive(&client, "quota-two", 101); + assert(mesh.posts == 2 && mesh.acks == 3 && client.extra.room.last_post_timestamp == 101); + assert(client.extra.room.post_quota_used == 1); // Rejected post was not remembered. + mesh.ticks = 120000; mesh.store_ok = false; mesh.receive(&client, "failed-store", 102); + assert(mesh.posts == 2 && mesh.acks == 3 && client.extra.room.post_quota_used == 0); + assert(client.extra.room.last_post_timestamp == 101); + mesh.store_ok = true; mesh.receive(&client, "failed-store", 102); + assert(mesh.posts == 3 && mesh.acks == 4 && client.extra.room.post_quota_used == 1); + assert(mesh.room_access.addBan(&mesh.fs, client.id.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + mesh.receive(&client, "failed-store", 102); // Ban blocks an already cached exact retry. + mesh.receive(&client, "banned-new", 103); + assert(mesh.posts == 3 && mesh.acks == 4 && client.extra.room.last_post_timestamp == 102); + assert(mesh.room_access.removeBan(&mesh.fs, client.id.pub_key) == mesh::RoomAccessPolicy::BanResult::Saved); + mesh.ticks = 180000; mesh.receive(&client, "banned-new", 103); + assert(mesh.posts == 4 && mesh.acks == 5); + } puts("256 room permission and replay scenarios passed"); } ''' @@ -92,13 +128,14 @@ class RoomPostPermissionTests(unittest.TestCase): header = (ROOT / "src/helpers/ClientACL.h").read_text() roles = "\n".join(line for line in header.splitlines() if line.startswith("#define PERM_ACL_")) - generated = HARNESS.replace("@POST_GATE@", gate).replace("@ROLES@", roles) + identity_gate = next(line.strip() for line in receive.splitlines() if "if (!room_access.allowsIdentity(" in line) + generated = HARNESS.replace("@POST_GATE@", gate).replace("@ROLES@", roles).replace("@IDENTITY_GATE@", identity_gate) with tempfile.TemporaryDirectory(prefix="room-post-permissions-") as directory: work = Path(directory) (work / "test.cpp").write_text(generated) binary = work / "room-posts" cmd = [compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", "-Wno-unused-parameter", - "-I" + str(ROOT / "test/mocks"), "-I" + str(ROOT / "src"), + "-I" + str(ROOT / "test/fixtures/room_history_store"), "-I" + str(ROOT / "test/mocks"), "-I" + str(ROOT / "src"), str(work / "test.cpp"), "-o", str(binary)] if sys.platform.startswith("linux"): cmd[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", diff --git a/test/test_room_receive_admission.py b/test/test_room_receive_admission.py new file mode 100644 index 000000000..30f2c1053 --- /dev/null +++ b/test/test_room_receive_admission.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Execute room reception, inbox admission and notifications under sanitizers.""" +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_replay_reset_integration import extract_braced + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE = ROOT / "test/fixtures/room_receive_admission/test.cpp" +SANITIZERS = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", + "-fno-pie", "-no-pie"] if sys.platform.startswith("linux") else []) + + +class RoomReceiveAdmissionTests(unittest.TestCase): + def test_actual_receive_queue_retry_and_notifications(self): + compiler = shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler, "a host C++17 compiler is required") + base = (ROOT / "src/helpers/BaseChatMesh.cpp").read_text() + source = (ROOT / "examples/companion_radio/MyMesh.cpp").read_text() + header = (ROOT / "examples/companion_radio/MyMesh.h").read_text() + receive = extract_braced(base, "void BaseChatMesh::onPeerDataRecv(") + queue = extract_braced(source, "bool MyMesh::queueMessage(") + common = "\n".join(extract_braced(source, signature) for signature in ( + "bool MyMesh::Frame::isChannelMsg(", "int MyMesh::getOfflineQueueCapacity(", + "MyMesh::Frame& MyMesh::offlineQueueFrameAt(", "bool MyMesh::addToOfflineQueue(", + "void MyMesh::popOfflineQueue(", "bool MyMesh::onSignedMessageRecv(", + "void MyMesh::onMessageRecv(")) + constants = "\n".join(re.findall( + r"^#define (?:RESP_CODE_(?:CONTACT_MSG_RECV(?:_V3)?|CHANNEL_MSG_RECV(?:_V3)?|CHANNEL_DATA_RECV)|PUSH_CODE_MSG_WAITING)\s+.+$", + source, re.M)) + constants += "\n" + "\n".join(line for line in ( + ROOT / "src/helpers/TxtDataHelpers.h").read_text().splitlines() + if line.startswith("#define TXT_TYPE_")) + rejected = extract_braced(receive, "if (!onSignedMessageRecv(") + bypass = "(void)previous_sync_since; (void)previous_lastmod;\n" + \ + "onSignedMessageRecv(from, packet, sender_timestamp, &data[5], (const char *) &data[9]);" + variants = ( + ("production", receive, queue, True), + ("ack-without-admission", receive.replace(rejected, bypass), queue, False), + ("notify-without-admission", receive, queue.replace( + "if (txt_type == TXT_TYPE_SIGNED_PLAIN && !queued) return false;", ""), False), + ) + with tempfile.TemporaryDirectory(prefix="room-receive-admission-") as directory: + work = Path(directory) + (work / "frame.inc").write_text(extract_braced(header, "struct Frame") + ";\n", encoding="ascii") + (work / "constants.inc").write_text(constants, encoding="ascii") + for name, actual_receive, actual_queue, passes in variants: + with self.subTest(variant=name): + (work / "production.inc").write_text( + actual_receive + "\n" + actual_queue + "\n" + common, encoding="ascii") + binary = work / name + built = subprocess.run([compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", + "-Wno-unused-parameter", *SANITIZERS, f"-I{work}", f"-I{ROOT / 'src'}", + str(FIXTURE), "-o", str(binary)], capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + tested = subprocess.run([str(binary)], capture_output=True, text=True, timeout=20) + if passes: + self.assertEqual(tested.returncode, 0, tested.stdout + tested.stderr) + self.assertIn("PASS: production room receive admission", tested.stdout) + else: + self.assertNotEqual(tested.returncode, 0, "negative control escaped the delivery checks") + self.assertIn("Assertion", tested.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_topic_delivery.py b/test/test_room_topic_delivery.py index d3c22aa5b..b9c64e732 100644 --- a/test/test_room_topic_delivery.py +++ b/test/test_room_topic_delivery.py @@ -36,7 +36,7 @@ class RoomTopicDeliveryTests(unittest.TestCase): (work / "production.inc").write_text(definitions) binary = work / "delivery" command = [compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", - "-Wno-unused-parameter", "-I" + str(work), + "-Wno-unused-parameter", "-I" + str(work), "-I" + str(ROOT / "src"), str(FIXTURE), "-o", str(binary)] if sys.platform.startswith("linux"): command[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", diff --git a/test/test_room_topic_store.py b/test/test_room_topic_store.py index ceaa508cf..715e452fc 100644 --- a/test/test_room_topic_store.py +++ b/test/test_room_topic_store.py @@ -49,6 +49,9 @@ public: }; class MemoryFS { public: + void _lockFS() {} + void _unlockFS() {} + MemoryFS* _getFS() { return this; } std::map>> files; std::set faults, directories; std::vector trace; @@ -136,6 +139,17 @@ void File::close() { valid = false; fs->end(op); } #define FILE_O_WRITE 1 +#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) +struct lfs_info {}; +static constexpr int LFS_ERR_NOENT = -2; +static int lfs_stat(MemoryFS* fs, const char* path, lfs_info*) { + const std::string op = std::string("stat:") + path; + const bool okay = fs->begin(op); + const bool present = fs->files.count(path) || fs->directories.count(path); + fs->end(op); + return !okay ? -5 : present ? 0 : LFS_ERR_NOENT; +} +#endif #include static const char* P = mesh::ROOM_TOPIC_PRIMARY_PATH; @@ -255,7 +269,7 @@ int main() { MemoryFS directory; directory.put(path, OLD); directory.directories.insert(path); assert(!load(directory, text) && !save(directory, "New topic")); assert(directory.get(path) == OLD); ++scenarios; } -#if defined(ESP32_PLATFORM) +#if defined(ESP32_PLATFORM) || defined(NRF52_PLATFORM) || defined(STM32_PLATFORM) for (bool present : {false, true}) { MemoryFS fs; if (present) fs.put(P, OLD); fs.faults.insert(std::string("stat:") + P); char text[152]; diff --git a/test/test_room_web_http.py b/test/test_room_web_http.py new file mode 100644 index 000000000..a35e5694f --- /dev/null +++ b/test/test_room_web_http.py @@ -0,0 +1,507 @@ +#!/usr/bin/env python3 +"""Execute actual room HTTP/loop handlers and mailbox under adversarial retries.""" +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_replay_reset_integration import extract_braced + +ROOT = Path(__file__).resolve().parents[1] + +HARNESS = r''' +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +using String = std::string; +#define PROGMEM +@HTML@ +static uint32_t ticks = 100; +static uint32_t millis() { return ticks; } +using SemaphoreHandle_t = std::mutex*; +static thread_local unsigned lock_depth = 0; +static constexpr unsigned portMAX_DELAY = 0xffffffffu; +static void xSemaphoreTake(SemaphoreHandle_t h, unsigned) { h->lock(); ++lock_depth; } +static void xSemaphoreGive(SemaphoreHandle_t h) { assert(lock_depth == 1); --lock_depth; h->unlock(); } +@LOCK@ +static std::mutex s_wc_route_mux; +#define portENTER_CRITICAL(h) (h)->lock() +#define portEXIT_CRITICAL(h) (h)->unlock() +namespace mesh { +struct Debug { void printf(const char*, ...) {} }; +static Debug& usbDebugPort() { static Debug value; return value; } +struct Utils { + static void sha256(uint8_t* out, size_t count, const uint8_t* data, size_t length) { + // Only callback ordering/ownership is under test; production still uses SHA256. + uint32_t hash = 2166136261u; + for (size_t i = 0; i < length; ++i) hash = (hash ^ data[i]) * 16777619u; + for (size_t i = 0; i < count; ++i) out[i] = uint8_t(hash >> ((i & 3) * 8)); + } +}; +} +struct Header { String text; const String& value() const { return text; } }; +struct AsyncWebServerResponse { + int status; + String type, body; + std::map headers; + AsyncWebServerResponse(int code, String content_type, String text) : status(code), type(content_type), body(text) {} + virtual ~AsyncWebServerResponse() = default; + void addHeader(const char* key, const char* value) { headers[key] = value; } +}; +static bool fail_page_allocation = false; +struct WebConfigPacedProgmemResponse : AsyncWebServerResponse { + WebConfigPacedProgmemResponse(const char* type, const uint8_t* bytes, size_t length) + : AsyncWebServerResponse(200, type, String(reinterpret_cast(bytes), length)) {} + static void* operator new(size_t n, const std::nothrow_t&) noexcept { + if (fail_page_allocation) return nullptr; + try { return ::operator new(n); } catch (...) { return nullptr; } + } + static void operator delete(void* p) { ::operator delete(p); } + static void operator delete(void* p, const std::nothrow_t&) { ::operator delete(p); } +}; +struct AsyncWebServerRequest { + std::map headers; + void* _tempObject = nullptr; + size_t length = 0; + String content_type = "application/json"; + int status = 0; + String reply, response_type; + std::map response_headers; + bool fail_response = false; + std::function on_send; + ~AsyncWebServerRequest() { free(_tempObject); } + bool hasHeader(const char* key) const { return headers.count(key) != 0; } + Header* getHeader(const char* key) { return &headers.at(key); } + const String& contentType() const { return content_type; } + size_t contentLength() const { return length; } + AsyncWebServerResponse* beginResponse(int code, const char* type, const String& text) { + if (fail_response) return nullptr; + return new AsyncWebServerResponse(code, type, text); + } + void send(int code, const char* type = "", const char* text = "") { + status = code; response_type = type; reply = text; + if (on_send) on_send(); + } + void send(AsyncWebServerResponse* response) { + assert(response); + status = response->status; response_type = response->type; + reply = response->body; response_headers = response->headers; + delete response; + if (on_send) on_send(); + } +}; +static void wcLogReq(AsyncWebServerRequest*) {} +static constexpr int HTTP_GET = 1, HTTP_POST = 2; +struct AsyncWebServer { + using Handler = std::function; + using Collector = void(*)(AsyncWebServerRequest*, uint8_t*, size_t, size_t, size_t); + struct Route { String path; int method; Handler handler; Collector collector; }; + std::vector routes; + Handler missing; + bool ended = false; + void on(const char* path, int method, Handler handler, void* = nullptr, Collector body = nullptr) { + routes.push_back({path, method, handler, body}); + } + void onNotFound(Handler handler) { missing = handler; } + void end() { ended = true; } + void invoke(const char* path, int method, AsyncWebServerRequest& request) { + for (const auto& route : routes) { + // Model AsyncWebServer's ordered prefix matching: /result must precede /room. + if (route.method == method && String(path).compare(0, route.path.size(), route.path) == 0) { + route.handler(&request); return; + } + } + assert(missing); missing(&request); + } +}; +struct DNS { unsigned stops = 0; void stop() { ++stops; } }; +struct Callbacks { + bool supported = true, authenticated = true; + bool roomRequestAuthenticated() const { return authenticated; } + unsigned calls = 0, stops = 0; + String last_body; + uint8_t last_token[32] = {}; + uint32_t last_sequence = 0; + size_t last_capacity = 0; + std::function in_callback; + bool supportsRoomService() const { return supported; } + void processRoomRequest(const uint8_t* token, uint32_t sequence, char* body, char* result, size_t capacity) { + assert(lock_depth == 0); // Mutating callback runs only on the loop, outside the HTTP mutex. + ++calls; last_body = body; memcpy(last_token, token, 32); + last_sequence = sequence; last_capacity = capacity; + strcpy(result, "{\"owner_secret\":\"result-one\"}"); + if (in_callback) in_callback(body, result); + } + void onWebConfigStopped() { ++stops; } +}; +struct WebConfigServer { + static constexpr size_t MAX_BODY = @MAX_BODY@; + static constexpr uint32_t STOP_WARN_MS = WebConfigBatch::kStopWarnMs; + enum Mode { MODE_OFF, MODE_SETUP, MODE_LAN }; + Mode _mode = MODE_LAN; + bool _stopping = false, _stop_warned = false, _board_cmds_probed = true; + uint32_t _handler_refs = 0, _stop_warn_at = 0, _last_activity = 0; + std::mutex mutex; + SemaphoreHandle_t _mux = &mutex; + mesh::RoomWebMailbox* _room_mailbox = nullptr; + Callbacks* _cb; + AsyncWebServer host; + AsyncWebServer* _server = &host; + DNS* _dns = nullptr; + static WebConfigServer* _active; + typedef void (WebConfigServer::*RequestHandler)(AsyncWebServerRequest*); + unsigned finalizations = 0; + explicit WebConfigServer(Callbacks& callbacks, bool allocate = true) : _cb(&callbacks) { + if (allocate) _room_mailbox = new mesh::RoomWebMailbox; + registerRoutes(); _active = this; + } + ~WebConfigServer() { if (_active == this) _active = nullptr; delete _room_mailbox; } + void finalizeTeardown() { // Dependency stub: production tick decides when reclaim is safe. + assert(_handler_refs == 0); ++finalizations; + delete _room_mailbox; _room_mailbox = nullptr; _server = nullptr; + _stopping = false; _cb->onWebConfigStopped(); + } + void probeBoardCommands() { _board_cmds_probed = true; } + void serviceTerminal(uint32_t) {} + void serviceRoom(uint32_t); + void tick(uint32_t); + void requestStop(); + void detachRoutes(); + uint32_t handlerRefCount() const; + void registerRoutes(); + static void dispatchRequest(AsyncWebServerRequest*, RequestHandler); + static void collectBody(AsyncWebServerRequest*, uint8_t*, size_t, size_t, size_t); + void handleRoomPage(AsyncWebServerRequest*); + void handleRoomPost(AsyncWebServerRequest*); + void handleRoomResult(AsyncWebServerRequest*); + @UNRELATED_HANDLERS@ +}; +WebConfigServer* WebConfigServer::_active = nullptr; +@PRODUCTION@ +static String token(unsigned value = 1) { + static const char digits[] = "0123456789abcdef"; + String text(64, '0'); text[62] = digits[(value >> 4) & 15]; text[63] = digits[value & 15]; return text; +} +static void identity(AsyncWebServerRequest& request, unsigned owner = 1, const char* sequence = "1") { + request.headers["X-Room-Token"].text = token(owner); + request.headers["X-Room-Seq"].text = sequence; + request.headers["Host"].text = "192.168.4.1"; + request.headers["Origin"].text = "http://192.168.4.1"; +} +static void body(AsyncWebServerRequest& request, const String& text) { + request.length = text.size(); + const size_t first = text.size() / 2; + WebConfigServer::collectBody(&request, reinterpret_cast(const_cast(text.data())), first, 0, text.size()); + WebConfigServer::collectBody(&request, reinterpret_cast(const_cast(text.data() + first)), text.size() - first, first, text.size()); +} +static void submit(WebConfigServer& server, AsyncWebServerRequest& request) { server.host.invoke("/api/room", HTTP_POST, request); } +static void result(WebConfigServer& server, AsyncWebServerRequest& request) { server.host.invoke("/api/room/result", HTTP_POST, request); } + +static void validation() { + for (unsigned fault = 0; fault < 19; ++fault) { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest request; + identity(request); body(request, "{\"op\":\"post\"}"); + switch (fault) { + case 0: request.headers.erase("X-Room-Token"); break; + case 1: request.headers.erase("X-Room-Seq"); break; + case 2: request.headers["X-Room-Token"].text = String(64, '0'); break; + case 3: request.headers["X-Room-Token"].text = token().substr(1); break; + case 4: request.headers["X-Room-Token"].text[0] = 'g'; break; + case 5: request.headers["X-Room-Seq"].text = "0"; break; + case 6: request.headers["X-Room-Seq"].text = "-1"; break; + case 7: request.headers["X-Room-Seq"].text = "4294967296"; break; + case 8: request.headers["X-Room-Seq"].text = "1junk"; break; + case 9: request.headers["X-Room-Seq"].text = "\n1"; break; + case 10: request.headers["Origin"].text = "http://attacker.invalid"; break; + case 11: request.headers.erase("Host"); break; + case 12: request.headers["Origin"].text = "https://192.168.4.1"; break; + case 13: request.content_type = "text/plain"; break; + case 14: request.content_type = "application/json; charset=utf-8"; break; + case 15: free(request._tempObject); request._tempObject = nullptr; break; + case 16: request.length = 0; break; + case 17: request.length = WebConfigServer::MAX_BODY + 1; break; + case 18: static_cast(request._tempObject)[2] = 0; break; + } + submit(server, request); + if (request.status != (fault == 18 ? 409 : 400)) fprintf(stderr, "validation fault=%u status=%d\n", fault, request.status); + assert(request.status == (fault == 18 ? 409 : 400)); + assert(cb.calls == 0 && server._room_mailbox->state() == mesh::RoomWebMailbox::State::Idle); + server.tick(ticks); assert(cb.calls == 0); + } + for (unsigned fault = 0; fault < 5; ++fault) { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest request; identity(request); + if (fault == 0) request.headers.erase("X-Room-Token"); + if (fault == 1) request.headers["X-Room-Seq"].text = "0"; + if (fault == 2) request.headers["Origin"].text = "null"; + if (fault == 3) request.headers["X-Room-Seq"].text = "1.0"; + if (fault == 4) request.headers["X-Room-Seq"].text = "+-1"; + result(server, request); assert(request.status == 400 && cb.calls == 0); + } + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest allowed; + identity(allowed); allowed.headers.erase("Origin"); body(allowed, "{}"); submit(server, allowed); + assert(allowed.status == 202 && cb.calls == 0); server.tick(ticks); assert(cb.calls == 1); + for (const char* sequence : {"+1", " \t1\t ", "0001"}) { + Callbacks canonical_cb; WebConfigServer canonical(canonical_cb); AsyncWebServerRequest request; + identity(request, 10, sequence); request.headers["X-Room-Token"].text[63] = 'A'; body(request, "{}"); + submit(canonical, request); assert(request.status == 202); canonical.tick(ticks); + assert(canonical_cb.calls == 1 && canonical_cb.last_sequence == 1 && canonical_cb.last_token[31] == 10); + } + puts("actual HTTP identity, origin, media type, and body validation passed"); +} +static void deferredAndOwnership() { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest post; + identity(post); body(post, "{\"op\":\"post\",\"text\":\"secret\"}"); submit(server, post); + assert(post.status == 202 && cb.calls == 0 && server._room_mailbox->state() == mesh::RoomWebMailbox::State::Pending); + AsyncWebServerRequest wrong, wrong_seq, pending; + identity(wrong, 2); result(server, wrong); assert(wrong.status == 404 && wrong.reply.find("secret") == String::npos); + identity(wrong_seq, 1, "2"); result(server, wrong_seq); assert(wrong_seq.status == 404); + identity(pending); result(server, pending); assert(pending.status == 202 && pending.reply.find("secret") == String::npos); + cb.in_callback = [&](char* input, char* output) { + assert(server._room_mailbox->state() == mesh::RoomWebMailbox::State::Running); + const String before = input; + const auto async_actor = [&] { + AsyncWebServerRequest competing, duplicate, changed, running, stranger; + identity(competing, 2); body(competing, "{\"op\":\"other\"}"); submit(server, competing); assert(competing.status == 429); + identity(duplicate); body(duplicate, before); submit(server, duplicate); assert(duplicate.status == 202); + identity(changed); body(changed, before + " "); submit(server, changed); assert(changed.status == 409); + identity(running); result(server, running); assert(running.status == 202); + identity(stranger, 2); result(server, stranger); assert(stranger.status == 404); + }; + std::thread first(async_actor), second(async_actor); + first.join(); second.join(); // Real concurrent HTTP handlers share the actual extracted lock. + assert(String(input) == before && String(output) == "{\"owner_secret\":\"result-one\"}"); + }; + server.tick(ticks); + assert(cb.calls == 1 && cb.last_sequence == 1 && cb.last_token[31] == 1); + assert(cb.last_capacity == mesh::RoomWebMailbox::OUTPUT_CAPACITY && cb.last_body == "{\"op\":\"post\",\"text\":\"secret\"}"); + assert(server._room_mailbox->state() == mesh::RoomWebMailbox::State::Done); + for (size_t i = 0; i < mesh::RoomWebMailbox::INPUT_CAPACITY; ++i) assert(server._room_mailbox->input()[i] == 0); + AsyncWebServerRequest competitor; identity(competitor, 2); body(competitor, "{}"); submit(server, competitor); assert(competitor.status == 429); + AsyncWebServerRequest done; identity(done); result(server, done); assert(done.status == 200); + assert(done.reply == "{\"owner_secret\":\"result-one\"}" && done.response_headers["Cache-Control"] == "no-store"); + AsyncWebServerRequest retry; identity(retry); body(retry, cb.last_body); submit(server, retry); assert(retry.status == 202); + server.tick(ticks + 1); assert(cb.calls == 1); + AsyncWebServerRequest mutated; identity(mutated); body(mutated, "{\"changed\":true}"); submit(server, mutated); assert(mutated.status == 409); + cb.in_callback = {}; body(competitor, "{}"); submit(server, competitor); assert(competitor.status == 202); server.tick(ticks + 2); assert(cb.calls == 2); + AsyncWebServerRequest old; identity(old); body(old, cb.last_body); submit(server, old); assert(old.status == 409); + AsyncWebServerRequest no_leak; identity(no_leak); result(server, no_leak); assert(no_leak.status == 404); + puts("actual deferred callback, running input, owner isolation, and duplicate sequence checks passed"); +} +static void boundsAndFloors() { + static_assert(sizeof(mesh::RoomWebMailbox) <= 6656, "bounded room heap"); + static_assert(WebConfigServer::MAX_BODY + 1 == mesh::RoomWebMailbox::INPUT_CAPACITY, "body cap agrees with mailbox"); + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest full; + identity(full, 1, "4294967295"); body(full, String(WebConfigServer::MAX_BODY, 'x')); + assert(static_cast(full._tempObject)[WebConfigServer::MAX_BODY] == 0); + submit(server, full); assert(full.status == 202 && cb.calls == 0 && full._tempObject == nullptr); + server.tick(ticks); assert(cb.last_body.size() == WebConfigServer::MAX_BODY); + AsyncWebServerRequest full_result; identity(full_result, 1, "4294967295"); result(server, full_result); assert(full_result.status == 200); + AsyncWebServerRequest wrapped; identity(wrapped, 1, "1"); body(wrapped, "{}"); submit(server, wrapped); assert(wrapped.status == 409); + for (unsigned owner = 2; owner <= 8; ++owner) { + AsyncWebServerRequest post, reply; identity(post, owner); body(post, "{}"); submit(server, post); assert(post.status == 202); + server.tick(ticks); identity(reply, owner); result(server, reply); assert(reply.status == 200); + } + AsyncWebServerRequest ninth; identity(ninth, 9); body(ninth, "{}"); submit(server, ninth); assert(ninth.status == 429 && cb.calls == 8); + ticks += mesh::RoomWebMailbox::OWNER_TTL_MS; body(ninth, "{}"); submit(server, ninth); assert(ninth.status == 202); server.tick(ticks); assert(cb.calls == 9); + Callbacks ttl_cb; WebConfigServer ttl(ttl_cb); AsyncWebServerRequest first; identity(first); body(first, "{}"); submit(ttl, first); ttl.tick(ticks); + AsyncWebServerRequest second; identity(second, 2); body(second, "{}"); ticks += mesh::RoomWebMailbox::RESULT_TTL_MS - 1; submit(ttl, second); assert(second.status == 429); + ++ticks; body(second, "{}"); submit(ttl, second); assert(second.status == 202); ttl.tick(ticks); assert(ttl_cb.calls == 2); + { + Callbacks output_cb; WebConfigServer output(output_cb); + output_cb.in_callback = [](char*, char* bytes) { memset(bytes, 'x', mesh::RoomWebMailbox::OUTPUT_CAPACITY); }; + AsyncWebServerRequest post, response; identity(post); body(post, "{}"); submit(output, post); output.tick(ticks); + identity(response); result(output, response); assert(response.status == 200); + assert(response.reply.size() == mesh::RoomWebMailbox::OUTPUT_CAPACITY - 1); + } + { + ticks = UINT32_MAX - 100; + Callbacks wrap_cb; WebConfigServer wrap(wrap_cb); + AsyncWebServerRequest first, second; identity(first); body(first, "{}"); submit(wrap, first); wrap.tick(ticks); + identity(second, 2); body(second, "{}"); ticks += mesh::RoomWebMailbox::RESULT_TTL_MS - 1; + submit(wrap, second); assert(second.status == 429); + ++ticks; body(second, "{}"); submit(wrap, second); assert(second.status == 202); wrap.tick(ticks); + assert(wrap_cb.calls == 2); + } + puts("actual full-size body, owner capacity, sequence overflow, and result expiry checks passed"); +} +static void pagesAndAvailability() { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest page; + server.host.invoke("/room", HTTP_GET, page); assert(page.status == 200); + assert(page.reply.find("MeshCore Room") != String::npos && page.response_type == "text/html; charset=utf-8"); + assert(page.response_headers["Cache-Control"] == "no-store"); + assert(page.response_headers["X-Content-Type-Options"] == "nosniff" && page.response_headers["Referrer-Policy"] == "no-referrer"); + assert(cb.calls == 0); + fail_page_allocation = true; server.host.invoke("/room", HTTP_GET, page); assert(page.status == 503); fail_page_allocation = false; + cb.supported = false; server.host.invoke("/room", HTTP_GET, page); assert(page.status == 404); cb.supported = true; + server._mode = WebConfigServer::MODE_OFF; server.host.invoke("/room", HTTP_GET, page); assert(page.status == 404); + server._mode = WebConfigServer::MODE_LAN; + Callbacks lazy_cb; WebConfigServer lazy(lazy_cb, false); AsyncWebServerRequest before; identity(before); body(before, "{}"); submit(lazy, before); assert(before.status == 503); + lazy.tick(ticks); assert(lazy._room_mailbox && lazy_cb.calls == 0); submit(lazy, before); assert(before.status == 202); lazy.tick(ticks); assert(lazy_cb.calls == 1); + puts("actual page headers, bounded allocation failure, and service availability checks passed"); +} +static void lifecycle() { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest pending; identity(pending); body(pending, "{}"); + pending.on_send = [&] { + assert(server.handlerRefCount() == 1); + server.requestStop(); server.tick(ticks); + assert(server._stopping && server._room_mailbox && server.finalizations == 0); + assert(cb.calls == 0); + }; + submit(server, pending); assert(pending.status == 202 && server.host.ended); + assert(server.handlerRefCount() == 0 && server._stopping && cb.calls == 0); + AsyncWebServerRequest unavailable; identity(unavailable); result(server, unavailable); assert(unavailable.status == 503); + server.tick(ticks); assert(!server._room_mailbox && !server._stopping && cb.stops == 1 && server.finalizations == 1); + assert(cb.calls == 0); // Pending work is canceled, never executed after stop. + Callbacks off_cb; WebConfigServer off(off_cb); AsyncWebServerRequest post; identity(post); body(post, "{}"); submit(off, post); + off._mode = WebConfigServer::MODE_OFF; off.tick(ticks); assert(off_cb.calls == 0); + body(post, "{}"); off.handleRoomPost(&post); assert(post.status == 503); + AsyncWebServerRequest poll; identity(poll); off.handleRoomResult(&poll); assert(poll.status == 503); + puts("actual stop reference retention, route detach, pending cancellation, and OFF gating passed"); +} +static void responseFailure() { + Callbacks cb; WebConfigServer server(cb); AsyncWebServerRequest post; identity(post); body(post, "{}"); submit(server, post); server.tick(ticks); + AsyncWebServerRequest result_failure; identity(result_failure); result_failure.fail_response = true; + result(server, result_failure); assert(result_failure.status == 503 && cb.calls == 1); + AsyncWebServerRequest competing; identity(competing, 2); body(competing, "{}"); + submit(server, competing); assert(competing.status == 429); // OOM is not delivery and cannot release the result. + AsyncWebServerRequest retry; identity(retry); result(server, retry); assert(retry.status == 200 && cb.calls == 1); + puts("actual room result allocation failure remains retryable without reexecution passed"); +} +static void authenticationFloors() { + Callbacks cb; WebConfigServer server(cb); cb.authenticated = false; + for (unsigned owner = 1; owner <= 12; ++owner) { + AsyncWebServerRequest post, response; + identity(post, owner); body(post, "{\"password\":\"wrong\"}"); submit(server, post); assert(post.status == 202); + server.tick(ticks); identity(response, owner); result(server, response); assert(response.status == 200); + } + assert(cb.calls == 12); // Failed authentication cannot reserve all eight identities for twenty minutes. + cb.authenticated = true; + AsyncWebServerRequest good, first_result; identity(good, 99); body(good, "{}"); submit(server, good); assert(good.status == 202); + server.tick(ticks); identity(first_result, 99); result(server, first_result); assert(first_result.status == 200); + cb.authenticated = false; + AsyncWebServerRequest failed, failed_result; identity(failed, 99, "2"); body(failed, "{\"password\":\"wrong\"}"); submit(server, failed); assert(failed.status == 202); + server.tick(ticks); identity(failed_result, 99, "2"); result(server, failed_result); assert(failed_result.status == 200); + cb.authenticated = true; + AsyncWebServerRequest different, different_result; identity(different, 100); body(different, "{}"); submit(server, different); server.tick(ticks); + identity(different_result, 100); result(server, different_result); assert(different_result.status == 200); + AsyncWebServerRequest replay; identity(replay, 99); body(replay, "{}"); submit(server, replay); assert(replay.status == 409); + assert(cb.calls == 15); + puts("actual unauthenticated identities release capacity while authenticated replay floors remain passed"); +} +int main(int argc, char** argv) { + assert(argc == 2); + if (!strcmp(argv[1], "validation")) validation(); + else if (!strcmp(argv[1], "ownership")) deferredAndOwnership(); + else if (!strcmp(argv[1], "bounds")) boundsAndFloors(); + else if (!strcmp(argv[1], "pages")) pagesAndAvailability(); + else if (!strcmp(argv[1], "lifecycle")) lifecycle(); + else if (!strcmp(argv[1], "allocation")) responseFailure(); + else if (!strcmp(argv[1], "authentication")) authenticationFloors(); + else assert(false); +} +''' + + +class RoomWebHttpTests(unittest.TestCase): + @classmethod + def setUpClass(cls): + compiler = shutil.which("g++") or shutil.which("clang++") + if compiler is None: + raise AssertionError("a host C++ compiler is required") + source = (ROOT / "src/helpers/esp32/WebConfigServer.cpp").read_text() + header = (ROOT / "src/helpers/esp32/WebConfigServer.h").read_text() + html = (ROOT / "src/helpers/esp32/RoomWebHtml.h").read_text() + html_constant = re.search(r'static const char ROOM_WEB_HTML\[\] PROGMEM = R"roomhtml\([\s\S]*?\)roomhtml";', html) + if html_constant is None: + raise AssertionError("production room HTML constant not found") + methods = [extract_braced(source, signature) for signature in ( + "void WebConfigServer::serviceRoom(", "static bool roomRequestIdentity(", + "void WebConfigServer::handleRoomPage(", "void WebConfigServer::handleRoomPost(", + "void WebConfigServer::handleRoomResult(", "void WebConfigServer::collectBody(", + "void WebConfigServer::dispatchRequest(", "void WebConfigServer::detachRoutes(", + "uint32_t WebConfigServer::handlerRefCount(", "void WebConfigServer::requestStop(", + "void WebConfigServer::registerRoutes(")] + tick = extract_braced(source, "void WebConfigServer::tick(") + # Execute the exact stop/OFF guards and loop service calls; SDK Wi-Fi, + # display and later batch handling are outside this HTTP/mailbox fixture. + endpoint = tick.index(" serviceRoom(now);") + len(" serviceRoom(now);") + methods.append(tick[:endpoint] + "\n}") + routes = methods[-2] + unrelated = sorted(set(re.findall(r"&WebConfigServer::(handle\w+)", routes)) + - {"handleRoomPage", "handleRoomPost", "handleRoomResult"}) + stubs = "\n".join("void " + name + "(AsyncWebServerRequest* req) { req->send(418); }" for name in unrelated) + capacity = re.search(r"static const size_t MAX_BODY = ([0-9]+);", header) + if capacity is None: + raise AssertionError("production MAX_BODY not found") + generated = HARNESS.replace("@HTML@", html_constant.group()).replace("@LOCK@", extract_braced(source, "struct WCLock") + ";") + generated = generated.replace("@MAX_BODY@", capacity[1]).replace("@UNRELATED_HANDLERS@", stubs).replace("@PRODUCTION@", "\n".join(methods)) + cls.directory = tempfile.TemporaryDirectory(prefix="room-web-http-") + cls.addClassCleanup(cls.directory.cleanup) + work = Path(cls.directory.name) + (work / "test.cpp").write_text(generated) + cls.binary = work / "http" + command = [compiler, "-std=c++17", "-O1", "-Wall", "-Wextra", "-Werror", "-Wno-unused-parameter", + "-pthread", "-I" + str(ROOT / "src"), str(work / "test.cpp"), "-o", str(cls.binary)] + if sys.platform.startswith("linux"): + command[1:1] = ["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + compiled = subprocess.run(command, capture_output=True, text=True, timeout=60) + if compiled.returncode != 0: + raise AssertionError(compiled.stdout + compiled.stderr) + + def run_case(self, name, expected): + checked = subprocess.run([str(self.binary), name], capture_output=True, text=True, timeout=15) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn(expected, checked.stdout) + + def test_request_validation_never_runs_mutating_callback(self): + self.run_case("validation", "actual HTTP identity, origin, media type, and body validation passed") + + def test_loop_only_mutation_running_ownership_and_duplicate_retries(self): + self.run_case("ownership", "actual deferred callback, running input, owner isolation, and duplicate sequence checks passed") + + def test_full_body_bounds_owner_capacity_and_expiry(self): + self.run_case("bounds", "actual full-size body, owner capacity, sequence overflow, and result expiry checks passed") + + def test_page_headers_allocation_and_lazy_service_start(self): + self.run_case("pages", "actual page headers, bounded allocation failure, and service availability checks passed") + + def test_stop_lifecycle_retains_requests_and_never_executes_off(self): + self.run_case("lifecycle", "actual stop reference retention, route detach, pending cancellation, and OFF gating passed") + + def test_result_allocation_failure_does_not_reexecute(self): + self.run_case("allocation", "actual room result allocation failure remains retryable without reexecution passed") + + def test_failed_authentication_cannot_hold_owner_capacity_or_erase_good_floors(self): + self.run_case("authentication", "actual unauthenticated identities release capacity while authenticated replay floors remain passed") + + def test_actual_route_registration_and_loop_order_are_not_shadowed(self): + source = (ROOT / "src/helpers/esp32/WebConfigServer.cpp").read_text() + routes = extract_braced(source, "void WebConfigServer::registerRoutes(") + self.assertLess(routes.index('_server->on("/api/room/result"'), routes.index('_server->on("/api/room"')) + self.assertIn('dispatchRequest(r, &WebConfigServer::handleRoomResult)', routes) + self.assertIn('dispatchRequest(r, &WebConfigServer::handleRoomPost)', routes) + tick = extract_braced(source, "void WebConfigServer::tick(") + self.assertLess(tick.index("if (_stopping)"), tick.index("serviceRoom(now)")) + self.assertLess(tick.index("if (_mode == MODE_OFF)"), tick.index("serviceRoom(now)")) + teardown = extract_braced(source, "void WebConfigServer::finalizeTeardown(") + self.assertIn("delete _room_mailbox;", teardown) + self.assertIn("_room_mailbox = nullptr;", teardown) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_web_mailbox.py b/test/test_room_web_mailbox.py new file mode 100644 index 000000000..9ba3460f9 --- /dev/null +++ b/test/test_room_web_mailbox.py @@ -0,0 +1,218 @@ +#!/usr/bin/env python3 +"""Execute production browser mailbox ownership and retry handshakes under sanitizers.""" +from pathlib import Path +import shutil +import subprocess +import sys +import tempfile +import unittest + +ROOT = Path(__file__).resolve().parents[1] +HARNESS = r''' +#include +#include +#include +#include +#include +#include +using Mailbox = mesh::RoomWebMailbox; +using Result = Mailbox::Result; +using State = Mailbox::State; +static unsigned cases = 0; +struct Client { + uint8_t token[32] = {}, digest[32] = {}; + explicit Client(uint8_t value) { token[0] = value; digest[0] = uint8_t(value + 1); } +}; +static Result submit(Mailbox& m, Client& c, uint32_t sequence, uint32_t now, + const char* body = "write-once") { + return m.submit(c.token, sequence, c.digest, body, strlen(body), now); +} +static void execute(Mailbox& m, uint32_t now, const char* output = "success") { + assert(m.begin()); assert(!m.begin()); strcpy(m.output(), output); m.finish(now); + assert(m.state() == State::Done); +} +static const char* read(Mailbox& m, Client& c, uint32_t sequence) { + bool pending = true; const char* result = m.read(c.token, sequence, pending); + assert(!pending); + if (result) m.delivered(c.token, sequence); // A successfully constructed HTTP response owns a copy. + return result; +} +int main() { + { + const std::string lower = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"; + std::string upper = lower; std::transform(upper.begin(), upper.end(), upper.begin(), + [](unsigned char c) { return c >= 'a' && c <= 'f' ? char(c - 'a' + 'A') : char(c); }); + uint8_t decoded[32], equivalent[32]; + assert(Mailbox::decodeToken(lower.c_str(), decoded)); + assert(Mailbox::decodeToken(upper.c_str(), equivalent)); assert(memcmp(decoded, equivalent, 32) == 0); + for (unsigned position = 0; position < 64; ++position) { + std::string invalid = lower; invalid[position] = position % 2 ? ':' : 'g'; + memset(decoded, 0xa5, sizeof(decoded)); assert(!Mailbox::decodeToken(invalid.c_str(), decoded)); + for (uint8_t byte : decoded) { assert(byte == 0xa5); } ++cases; + } + for (const char* invalid : {static_cast(nullptr), "", "a", "0000000000000000000000000000000000000000000000000000000000000000"}) { + memset(decoded, 0xa5, sizeof(decoded)); assert(!Mailbox::decodeToken(invalid, decoded)); + for (uint8_t byte : decoded) { assert(byte == 0xa5); } ++cases; + } + assert(!Mailbox::decodeToken((lower + "0").c_str(), decoded)); ++cases; + } + for (const uint32_t start : {0U, UINT32_MAX - 15U}) { + Mailbox m; Client owner(1), stranger(2); bool pending = true; + assert(m.state() == State::Idle && !m.begin()); m.finish(start); + assert(m.read(owner.token, 1, pending) == nullptr && !pending); + assert(submit(m, owner, 1, start) == Result::Accepted && m.state() == State::Pending); + const std::string input(m.input()); strcpy(m.output(), "running-output"); + m.finish(start); assert(m.state() == State::Pending && std::string(m.input()) == input); + assert(m.read(owner.token, 1, pending) == nullptr && pending); + assert(m.read(stranger.token, 1, pending) == nullptr && !pending); + assert(m.read(owner.token, 2, pending) == nullptr && !pending); + for (const uint32_t elapsed : {0U, 1U, Mailbox::OWNER_TTL_MS, UINT32_C(0x80000000)}) { + assert(submit(m, stranger, 1, start + elapsed) == Result::Busy); + assert(submit(m, owner, 2, start + elapsed) == Result::Busy); + assert(submit(m, owner, 1, start + elapsed) == Result::Accepted); + Client changed = owner; changed.digest[31] ^= 1; + assert(submit(m, changed, 1, start + elapsed, "different-body") == Result::Mismatch); + assert(std::string(m.input()) == input && std::string(m.output()) == "running-output"); ++cases; + } + assert(m.begin()); char* original_input = m.input(); char* original_output = m.output(); + for (const uint32_t elapsed : {1U, Mailbox::OWNER_TTL_MS + 1U, UINT32_C(0x90000000)}) { + assert(submit(m, stranger, 99, start + elapsed, "replace") == Result::Busy); + assert(submit(m, owner, 2, start + elapsed, "replace") == Result::Busy); + assert(submit(m, owner, 1, start + elapsed) == Result::Accepted && !m.begin()); + assert(m.input() == original_input && m.output() == original_output); + assert(std::string(m.input()) == input && memcmp(m.token(), owner.token, 32) == 0); ++cases; + } + memset(m.output(), 'R', Mailbox::OUTPUT_CAPACITY); m.finish(start + 10); + for (size_t i = 0; i < Mailbox::INPUT_CAPACITY; ++i) assert(m.input()[i] == 0); + assert(m.output()[Mailbox::OUTPUT_CAPACITY - 1] == 0 && strlen(m.output()) == Mailbox::OUTPUT_CAPACITY - 1); + assert(submit(m, owner, 1, start + 11) == Result::Accepted && !m.begin()); + Client changed = owner; changed.digest[0] ^= 1; + assert(submit(m, changed, 1, start + 12, "new body") == Result::Mismatch); + assert(submit(m, stranger, 1, start + 10 + Mailbox::RESULT_TTL_MS - 1) == Result::Busy); + assert(m.read(stranger.token, 1, pending) == nullptr && !pending); + assert(submit(m, stranger, 1, start + 10 + Mailbox::RESULT_TTL_MS - 1) == Result::Busy); + const char* result = read(m, owner, 1); assert(result == m.output()); + assert(read(m, owner, 1) == result && !m.begin()); + assert(submit(m, stranger, 1, start + 13) == Result::Accepted); + execute(m, start + 14); assert(std::string(read(m, stranger, 1)) == "success"); + assert(submit(m, owner, 1, start + 15) == Result::Completed); + assert(submit(m, owner, 2, start + 16) == Result::Accepted); + execute(m, start + 17, "second"); assert(std::string(read(m, owner, 2)) == "second"); + assert(submit(m, owner, 1, start + 18) == Result::Completed); + assert(submit(m, stranger, 1, start + 18) == Result::Completed); ++cases; + } + { + Mailbox m; Client c(1); std::string maximum(Mailbox::INPUT_CAPACITY - 1, 'P'); + const std::string too_large(Mailbox::INPUT_CAPACITY, 'X'); const char embedded[] = {'a', 0, 'b'}; + for (unsigned state = 0; state < 4; ++state) { + const std::string prior(m.input()); const State prior_state = m.state(); + assert(m.submit(c.token, 1, c.digest, nullptr, 10, state) == Result::Invalid); + assert(m.submit(c.token, 0, c.digest, "valid", 5, state) == Result::Invalid); + assert(m.submit(c.token, 1, c.digest, "", 0, state) == Result::Invalid); + assert(m.submit(c.token, 1, c.digest, too_large.data(), too_large.size(), state) == Result::Invalid); + assert(m.submit(c.token, 1, c.digest, embedded, sizeof(embedded), state) == Result::Invalid); + assert(m.state() == prior_state && std::string(m.input()) == prior); + if (state == 0) { + assert(m.submit(c.token, 1, c.digest, maximum.data(), maximum.size(), state) == Result::Accepted); + assert(strlen(m.input()) == 4096 && m.input()[4096] == 0); + } else if (state == 1) assert(m.begin()); + else if (state == 2) m.finish(state); + ++cases; + } + for (size_t i = 0; i < Mailbox::INPUT_CAPACITY; ++i) assert(m.input()[i] == 0); + } + for (const uint32_t start : {100U, UINT32_MAX - 100U}) { + Mailbox m; Client a(1), b(2); + assert(submit(m, a, 10, start) == Result::Accepted); execute(m, start + 5); + assert(submit(m, b, 1, start + 5 + Mailbox::RESULT_TTL_MS - 1) == Result::Busy); + assert(submit(m, b, 1, start + 5 + Mailbox::RESULT_TTL_MS) == Result::Accepted); + assert(read(m, a, 10) == nullptr); execute(m, start + 5 + Mailbox::RESULT_TTL_MS); assert(read(m, b, 1)); + assert(submit(m, a, 9, start + Mailbox::OWNER_TTL_MS - 1) == Result::Completed); + assert(submit(m, a, 9, start + Mailbox::OWNER_TTL_MS) == Result::Accepted); + execute(m, start + Mailbox::OWNER_TTL_MS); assert(read(m, a, 9)); ++cases; + } + { + Mailbox m; + for (uint8_t i = 1; i <= 8; ++i) { + Client owner(i); assert(submit(m, owner, 7, i) == Result::Accepted); + execute(m, i); assert(read(m, owner, 7)); ++cases; + } + Client ninth(9), first(1); + assert(submit(m, ninth, 1, 10) == Result::Busy); + assert(submit(m, first, 6, 10) == Result::Completed); + assert(submit(m, first, 8, 10) == Result::Accepted); execute(m, 10); assert(read(m, first, 8)); + assert(submit(m, ninth, 1, 2 + Mailbox::OWNER_TTL_MS - 1) == Result::Busy); + assert(submit(m, ninth, 1, 2 + Mailbox::OWNER_TTL_MS) == Result::Accepted); ++cases; + } + { + Mailbox m; Client owner(1), other(2); bool pending; + assert(submit(m, owner, 1, 100) == Result::Accepted); execute(m, 100); + assert(std::string(m.read(owner.token, 1, pending)) == "success" && !pending); + // Copy/allocation failure must leave the cached result protected for retry. + assert(submit(m, other, 1, 101) == Result::Busy); + m.delivered(other.token, 1); m.delivered(owner.token, 2); + assert(submit(m, other, 1, 102) == Result::Busy); + assert(std::string(m.read(owner.token, 1, pending)) == "success" && !pending); + m.delivered(owner.token, 1); + assert(submit(m, other, 1, 103) == Result::Accepted); ++cases; + } + { + Mailbox m; + for (uint8_t i = 1; i <= 100; ++i) { + Client unauthenticated(i); + assert(submit(m, unauthenticated, 1, i) == Result::Accepted); + assert(m.begin()); strcpy(m.output(), "bad password"); m.finish(i, false); + assert(std::string(read(m, unauthenticated, 1)) == "bad password"); + assert(!m.begin()); ++cases; + } + // Bad-password requests never exhaust the eight durable in-boot floors. + for (uint8_t i = 101; i <= 108; ++i) { + Client authenticated(i); assert(submit(m, authenticated, 7, i) == Result::Accepted); + execute(m, i); assert(read(m, authenticated, 7)); ++cases; + } + Client ninth(109); assert(submit(m, ninth, 1, 109) == Result::Busy); ++cases; + } + { + Mailbox m; Client owner(1), other(2); + assert(submit(m, owner, 7, 100) == Result::Accepted); execute(m, 100); assert(read(m, owner, 7)); + assert(submit(m, owner, 8, 101) == Result::Accepted); assert(m.begin()); m.finish(101, false); + assert(read(m, owner, 8)); + assert(submit(m, other, 1, 102) == Result::Accepted); execute(m, 102); assert(read(m, other, 1)); + assert(submit(m, owner, 7, 103) == Result::Completed); + assert(submit(m, owner, 8, 103) == Result::Completed); ++cases; + } + { + Mailbox m; Client c(1), other(2); + assert(submit(m, c, UINT32_MAX, 10) == Result::Accepted); execute(m, 10); assert(read(m, c, UINT32_MAX)); + assert(submit(m, c, 0, 11) == Result::Invalid); + assert(submit(m, c, 1, 11) == Result::Completed); + assert(submit(m, other, 1, 11) == Result::Accepted); execute(m, 11); assert(read(m, other, 1)); + assert(submit(m, c, UINT32_MAX, 12) == Result::Completed); ++cases; + } + printf("%u production room web mailbox handshake scenarios passed\n", cases); +} +''' + + +class RoomWebMailboxTests(unittest.TestCase): + def test_actual_bounded_mailbox_ownership_retry_and_expiry(self): + compiler = shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler, "a host C++ compiler is required") + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", + "-fno-pie", "-no-pie"] if sys.platform.startswith("linux") else []) + with tempfile.TemporaryDirectory(prefix="room-web-mailbox-") as directory: + work = Path(directory) + (work / "test.cpp").write_text(HARNESS) + binary = work / "mailbox" + built = subprocess.run([compiler, "-std=c++11", "-Wall", "-Wextra", "-Werror", + *sanitizers, f"-I{ROOT / 'src'}", str(work / "test.cpp"), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + tested = subprocess.run([str(binary)], capture_output=True, text=True, timeout=20) + self.assertEqual(tested.returncode, 0, tested.stdout + tested.stderr) + self.assertIn("production room web mailbox handshake scenarios passed", tested.stdout) + print(tested.stdout.strip()) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_web_service.py b/test/test_room_web_service.py new file mode 100644 index 000000000..ceefc51b3 --- /dev/null +++ b/test/test_room_web_service.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +"""Execute production room-browser callbacks with real JSON, stores and gates.""" + +import json +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tempfile +import unittest + +from test_replay_reset_integration import extract_braced +from test_room_history_store import STAT_MOCK + +ROOT = Path(__file__).resolve().parents[1] +FIXTURES = ROOT / "test/fixtures/room_web_service" + + +class RoomWebServiceTest(unittest.TestCase): + def test_actual_callbacks_with_real_arduinojson_and_stores(self): + compiler = shutil.which("g++") or shutil.which("clang++") + self.assertIsNotNone(compiler, "a host C++ compiler is required") + json_header = next((ROOT / ".pio/libdeps").glob("*/ArduinoJson/src/ArduinoJson.h"), None) + self.assertIsNotNone(json_header, "actual ArduinoJson headers from the checkout are required") + version = (json_header.parent / "ArduinoJson/version.hpp").read_text() + self.assertRegex(version, r'ARDUINOJSON_VERSION "7\.4\.\d+"') + source = (ROOT / "examples/simple_room_server/MyMesh.cpp").read_text() + header = (ROOT / "examples/simple_room_server/MyMesh.h").read_text() + production = "\n".join(extract_braced(source, signature) for signature in ( + "bool MyMesh::handleRoomWebCommand(", "static const char* roomWebString(", + "void MyMesh::processRoomRequest(", "void MyMesh::serviceRoomQuotas(", + "bool MyMesh::storePost(", "bool MyMesh::snapshotRoomHistory(")) + roles = "\n".join(line for line in (ROOT / "src/helpers/ClientACL.h").read_text().splitlines() + if line.startswith("#define PERM_ACL_")) + sanitizers = (["-fsanitize=address,undefined", "-fno-sanitize-recover=all", "-fno-pie", "-no-pie"] + if sys.platform.startswith("linux") else []) + # 64-bit host slots are twice ESP32's size. This reproduces the real + # ESP32 default 1024-byte pool, while testing its unchanged 8192-byte + # allocator budget and the actual ArduinoJson parser/temporary strings. + with tempfile.TemporaryDirectory(prefix="meshcore-room-web-service-") as temporary: + work = Path(temporary) + (work / "sys").mkdir() + (work / "sys/stat.h").write_text(STAT_MOCK) + (work / "filesystem.h").write_text((ROOT / "test/fixtures/room_history_store/filesystem.h").read_text()) + (work / "production.inc").write_text(production) + (work / "roles.inc").write_text(roles) + (work / "post.inc").write_text(extract_braced(header, "struct PostInfo {") + ";") + web_user = extract_braced(header, "struct RoomWebUser {") + (work / "web_user.inc").write_text(web_user + " room_web_users[8] = {};") + binary = work / "service" + built = subprocess.run([compiler, "-std=c++17", "-Wall", "-Wextra", "-Werror", + "-Wno-misleading-indentation", *sanitizers, "-DESP32_PLATFORM=1", + "-DARDUINOJSON_POOL_CAPACITY=64", + "-I" + str(work), "-I" + str(ROOT / "src"), "-I" + str(json_header.parent), + str(FIXTURES / "service.cpp"), "-o", str(binary)], + capture_output=True, text=True, timeout=60) + self.assertEqual(built.returncode, 0, built.stdout + built.stderr) + checked = subprocess.run([str(binary)], capture_output=True, text=True, timeout=60) + self.assertEqual(checked.returncode, 0, checked.stdout + checked.stderr) + self.assertIn("actual room web service", checked.stdout) + for kind in ("STATUS", "INDEX", "READ"): + match = re.search(r"^BOUNDARY_" + kind + r" (.+)$", checked.stdout, re.M) + self.assertIsNotNone(match) + actual = json.loads(match.group(1)) + expected = json.loads((FIXTURES / "boundary.json").read_text())[kind.lower()] + self.assertEqual(actual, expected) + print(checked.stdout.splitlines()[-1]) + + +if __name__ == "__main__": + unittest.main() diff --git a/test/test_room_web_ui.py b/test/test_room_web_ui.py new file mode 100644 index 000000000..84afbb3d6 --- /dev/null +++ b/test/test_room_web_ui.py @@ -0,0 +1,292 @@ +#!/usr/bin/env python3 +"""Execute the actual offline room page in Chromium with bounded HTTP mocks.""" + +import html +import json +import os +from pathlib import Path +import re +import subprocess +import tempfile +import unittest + +from test_webconfig_ui_runtime import BROWSER, run_browser + +ROOT = Path(__file__).resolve().parents[1] +HEADER = ROOT / "src/helpers/esp32/RoomWebHtml.h" +BOUNDARY = ROOT / "test/fixtures/room_web_service/boundary.json" + +COMMON = r''' +const boundary=__BOUNDARY__,calls=[]; +localStorage.setItem('mc-room-token','01'.repeat(32));localStorage.setItem('mc-room-sequence','0'); +window.attack=()=>document.body.dataset.testAttacked='yes'; +window.addEventListener('error',e=>document.body.dataset.testJsError=e.message); +window.addEventListener('unhandledrejection',e=>document.body.dataset.testJsError=String(e.reason)); +const response=(value,status=200)=>Promise.resolve({ok:status>=200&&status<300,status:status,json:()=>Promise.resolve(value)}); +const until=async predicate=>{for(let i=0;i<500;i++){if(predicate())return;await new Promise(r=>setTimeout(r,10))}throw Error('test condition timed out')}; +const submit=id=>document.getElementById(id).dispatchEvent(new Event('submit',{bubbles:true,cancelable:true})); +const record=(url,options)=>{let p=options.body?JSON.parse(options.body):null;calls.push({url:url,body:options.body||'',seq:options.headers['X-Room-Seq'],token:options.headers['X-Room-Token'],op:p?p.op:'result'});return p}; +const mark=(key,value)=>document.body.setAttribute('data-test-'+key,String(value)); +''' + + +class RoomWebUiTest(unittest.TestCase): + def run_page(self, scenario, virtual_time=7000): + if not BROWSER: + if os.environ.get("MESHCORE_REQUIRE_ROOM_BROWSER") == "1": + self.fail("Chromium is required for room browser validation") + self.skipTest("Chromium-family browser is unavailable") + source = HEADER.read_text().split('R"roomhtml(', 1)[1].split(')roomhtml";', 1)[0] + fixture = json.loads(BOUNDARY.read_text()) + prelude = "" + self.assertIn("