diff --git a/docs/locales/zh-Hans/geocaching-lxmf/24-实现授权与增补约束.md b/docs/locales/zh-Hans/geocaching-lxmf/24-实现授权与增补约束.md new file mode 100644 index 00000000..6ef58564 --- /dev/null +++ b/docs/locales/zh-Hans/geocaching-lxmf/24-实现授权与增补约束.md @@ -0,0 +1,144 @@ +# 实现授权与新增约束 + +2026-09-16用户明确授权：新建分支，按当前设计实现Geocaching；不要求用户切换当前通信协议；使用现有geocaching.c图标；保持现有TopBar使用习惯。 + +已创建功能分支：`codex/geocaching`。本页是新授权记录，更新此前“所有实现均等待授权”的阶段性约束；它不表示已完成或已验证功能。 + +## 必须保持的边界 + +- Geocaching在线操作不得自动修改用户选中的Meshtastic/MeshCore/Reticulum配置，也不得要求先去设置切换协议。 +- Reticulum仍是Geocaching网络协议；当前聊天协议与Geocaching传输可用性是不同状态。独立服务传输可使用现有可用IP接入，具体装配须核查共享身份/路径/接口owner。 +- 同一射频不能在两个协议间暗中来回重置。无可用独立网络路径时显示连接/配置状态并保持离线GPX功能，不偷切协议以制造“成功”。 +- 入口复用`modules/ui_shared/src/ui/assets/geocaching.c`，不重新生成替代图标。 +- 复用既有TopBar创建、页面profile、返回/焦点和状态投影。07静态图中的26px自绘顶栏不是实施模板，不覆盖目标现有chrome。 +- 所有符号修改前按AGENTS执行GitNexus impact；未索引的新符号明确记为新定义，不把UNKNOWN当低风险。 + +## GPX 复用约束（用户补充） + +项目已有 GPX 读写实现。允许改进现有实现，但不得为 Geocaching 另建一套并行的通用 GPX 读写逻辑。轨迹、Team 与 Geocaching 应共用 GPS 层的 GPX 能力；抽取公共实现时，原有调用方必须迁移到公共入口，不能保留两份相同职责的实现。 + +- 通用 XML 转义、坐标格式化、GPX 文档结构与解析属于共享 GPS 层。实际文件访问沿用既有平台存储接口。 +- Geocaching 层仅负责宝藏业务字段、Groundspeak 字段及 Trail Mate 签名扩展与公共 GPX 数据之间的映射；协议签名校验仍属于 Geocaching 协议职责。 +- 现有轨迹读取能力不能直接等同于完整航点读取能力。缺少 `wpt` 或扩展字段支持时，扩展同一个公共读取入口，并保留原有轨迹读取行为。 +- 验收必须同时覆盖原有轨迹/Team 路径与 Geocaching 路径。公共工具的单元测试通过，不代表设备 SD 卡读写、现有功能兼容或端到端下载已完成。 + +本约束优先于此前文档中可能被理解为“新增独立 GPX 编解码器”的目录建议；目录拆分只允许表达业务映射与公共格式能力的职责差异，不允许复制实现。 + +## 初始核查事实（工作区） + +工作区基于main，仅既有AGENTS.md改动及未跟踪构建目录；这些不属于本功能，保持原样。现有protocol_runtime_factory仍按MeshProtocol选择runtime；新功能需要独立服务生命周期设计，不直接修改聊天选择器。 + +首次GitNexus query提示FTS索引缺失导致关键词检索降级，未报告索引过期；后续对实际符号使用context/impact和源码核对。新GeocacheId查询显示尚不存在，不能据此声称已有调用链安全。 + +## 实施顺序 + +### 内存与共享 SPI 约束（用户再次明确） + +Geocaching 必须遵循工程已有的克制策略。把大数组从栈搬到堆或 PSRAM，并不等于满足内存约束；协议/文件允许的最大尺寸也不是为每个组件预留常驻缓冲的理由。 + +- 复用现有 memory admission/floor、共享 SPI 协调器与 `StorageOperationBudget`。逐项核算常驻、任务激活和峰值内存；功能未使用时不得预创建完整后台 Reticulum 实例。 +- SD runtime 的既有传输切片为 512 字节。上层维护 step 同样要有界，不能在一次 step 内循环读写完整大文件，再以“底层已经分片”为由认为已经让出了调度。记录读取返回 InProgress 时必须回到 owner。 +- 不以整页摘要重复编码、整事务大数组、全状态双份复制来简化设备实现。查询缓存与持久提交需要逐项/流式处理，仍要保留完整事务提交与恢复语义，不能用拆坏原子性的方式降低内存。 +- 文件大小上限属于输入验证边界，不构成一次性读入、一次性写完或预分配同等大小缓冲的许可。写入、刷新、重读验证应分成可恢复步骤。 + +此前已撤掉未接入的批量查询提交工作区和 16 KiB 事务缓冲扩张，移除开机预创建后台 Reticulum；记录读取已增加小型游标和每次调用最多 512 字节的读取预算。随后已删除 9,000 字节整事务缓冲和 8,500 字节 Outgoing 编码工作区；日志写入已改为分步执行，签发/发布任务的固定载荷数组也已删除。剩余的整帧解码、GPX 暂存文件的同步写入、传输层载荷副本与 LogicalState 双缓冲仍须重构和计量，不能据此宣称设备资源设计已经通过。 + +后续收缩：日志对象已删除 64 项 mutation 常驻数组，改为复用单项解析的常量辅助内存校验；发送调度器已删除 8 KiB 请求副本，在尝试事务提交后重新取得已提交请求视图。传输接口限定只在调用期间借用输入，异步传输保留自己的编码载荷。日志对象在加入分步游标和文件句柄后，编译检查上限为 768 字节；调度器上限仍为 192 字节。这些局部检查不替代剩余缓冲及整体峰值审计。 + +#### 资源验收条件 + +本节是上述约束的验收补充，不改变公开发布、协议签名或 GPX 兼容语义。设备内存工作集、文件格式容量和 SD 操作延迟必须分别核算，不能相互替代。 + +| 检查对象 | 必须提供的证据 | 不满足时的处理 | +| --- | --- | --- | +| 功能未启用 | 与未创建 Geocaching 运行时的基线对比，列出新增常驻对象、任务栈、队列与传输实例 | 去掉提前分配和重复实例，保留必要的小型入口状态 | +| 活动任务峰值 | 列出缓冲所有者、尺寸、生命周期和同时存活关系；包含协议库、GPX、状态恢复和传输层持有的副本 | 优先消除完整对象副本；按既有 memory admission/floor 拒绝或延后启动，不能无限排队 | +| 存储 step | 明确每步的字节数、文件/目录操作次数、让出点与取消边界；记录读取当前每步至多 512 字节 | 返回 owner 等待后续调度；禁止在调用方循环到完成而绕过预算 | +| 共享 SPI 占用 | 在目标板上记录单次操作和总线租约耗时，并在地图显示及活动通信同时运行时测量 | 调整工作粒度、调度时机或写入方案，不能仅移动到后台线程 | +| 文件与目录增长 | 验证大量缓存、较大输入、日志增长和重启恢复时的工作集及单步操作量 | 使用有界游标和可恢复维护，禁止整目录加载、全量状态复制或每次操作重写全部缓存 | +| 持久提交 | 在写入、刷新、验证及安装边界验证取消、拔卡和重启行为 | 维持待完成状态，只有满足既有持久性条件才能报告完成 | + +512 字节是数据处理粒度，不是延迟保证。SD 卡内部擦写、FAT 元数据更新，以及 `open`、`flush`、`close`、重命名等调用仍可能阻塞；将它们拆成独立调度步骤只能避免连续叠加，不能使单次不可抢占调用变成可抢占。需沿用现有 SD runtime 和共享 SPI 协调机制，核查锁的实际持有范围，并实测最大耗时及分布。未经设备测量，不承诺固定毫秒上限，也不以桌面模拟文件测试替代设备验收。 + +GPX 仍使用既有公共读写能力，按流处理宝藏文件；业务层不拼接整个 XML 字符串，不为了更新一个宝藏重写所有宝藏。列表只保留可见窗口和必要的少量预取，不能把所有已下载标点、全部描述和所有任务载荷常驻内存。对签名校验等确需连续输入的操作，应先确认现有密码库要求，再设计一个受预算控制的工作区及明确生命周期，不能为多个组件各预留一份最大载荷。 + +分步任务借用的数据必须在该任务使用期间有效；现有“仅同步调用期间借用”的接口不能直接延伸为跨 step 指针。队列满或内存预算不足时必须提供明确的繁忙/稍后重试结果，不能通过复制整包、扩容队列或静默丢失持久任务来规避资源约束。 + +`SdRequestStore::transaction_` 的 9,000 字节数组已删除。`TransactionEncoding` 只持有不超过 32 字节的借用元数据，使用 16 字节临时编码头，按调用方给出的窗口输出；连续输出接口复用同一实现。日志写入利用既有 512 字节缓冲生成片段，回读直接与编码视图比较，CRC 按片段累计。新任务直接生成两个 mutation 视图，不再“编码整个事务再解码”；发布结果校验直接借用请求内原始 SignedCache 字节。删除 Outgoing 编码副本后，请求存储对象的编译检查进一步收紧为小于 2 KiB；该数字不包含外部 LogicalState arena，不能当成整体运行时预算。 + +同步日志写入/回读循环已删除。`SdGeocachingJournal::begin` 只建立操作；每次 `step` 至多传输 512 字节，CRC、目录检查、卷检查、打开、写头、写载荷、刷新、关闭、回读和最终卷验证分别推进。生产代码不提供循环到完成的兼容包装。取消发生在步骤边界；创建过文件的操作必须经过恢复才能继续。 + +`SdRequestStore` 先验证并冻结候选状态，持有当前命令最多三个 mutation 的小型描述符和键，值借用冻结候选状态；调用方的临时请求、键和值可以在返回后失效。只有 `stepCommit` 返回 Verified 才切换可见状态和推进序号。进行中的其他写命令返回 Busy，签名前的版本预留与发送调度均等待持久提交，不能把 InProgress 当成失败或成功。Outgoing 使用原有编码器直接写入候选状态槽位，不再先生成 8,500 字节工作区副本；发布结果验签临时借用尚未进入 prepare 的空闲 arena，之后由候选状态重建覆盖。生成失败、空间不足或验证失败都不切换可见状态。此处仍沿用待改造的 LogicalState 双 arena 与整状态复制，既有维护 owner 的启动装配也仍未完成。 + +#### 页面配色与常驻内存纠正 + +用户在实机指出页面配色不协调、常驻内存增加不可接受。核查当前 `tlora_pager_lr1121` 固件符号表发现页面全局 Snapshot 独占 5,288 字节；该设计已删除。Snapshot 现在只包含元数据和状态文字，Source 按指定 generation 逐条提供当前可见行；页面最多创建四行控件，通过翻页访问全部结果，不以截断结果代替按需加载。 + +页面状态仅在 enter/exit 之间存在，编译检查不超过 640 字节；全局只保留 source 和 page 两个指针，不保留全局列表、TopBar 对象或页面定时器状态。退出时删除定时器、控件、焦点组及页面状态，并恢复之前的焦点组。配色直接复用 `two_pane_styles`，清除默认主题带来的蓝色按钮；TopBar 继续调用现有组件及返回、电量展示机制。 + +新增原生 LVGL 页面验证以实际页面、TopBar、公共样式和 profile 渲染 480×222、320×240 两种尺寸，并覆盖可见窗口、翻页、返回和重复进入退出；电量数据与内容源在测试中模拟。固件符号表的静态 RAM 变化与实机运行时堆内存是不同指标，不能以其中一个替代另一个。 + +### 存储执行线程的复用边界 + +源码核查确认：`platform/esp/common/include/platform/esp/common/storage/storage_maintenance_owner.h` 已提供稳定工作线程、事件队列、步骤预算、重试和边界取消；`storage_contracts.h` 的 `ISemanticStorageAdapter` 是业务存储接入点。Arduino 的 `storage_runtime.cpp` 当前适配器串行组织聊天和联系人维护，不能把它的 hydration 完成标记直接当成 Geocaching 恢复完成。 + +Geocaching 后续应通过该维护接口接入既有 owner 机制，不再实现一套工作线程和重试状态机。Hydrate 必须完成卷识别、检查点/日志恢复、任务引用及 GPX 安装恢复后才报告业务就绪；Persist 执行请求或结果事务；每个 step 遵守 `max_work_items` 并在边界检查取消。网络回调只转交有界工作，页面只读内存快照。不能从网络回调重入持锁路由器，也不能把“排入内存队列”返回为持久接受。 + +当前 `SdRequestStore`、日志读写与卷初始化组件尚未实现上述维护适配器或启动装配；它们的测试不构成设备后台恢复、请求调度或端到端功能已完成的证据。 + +### 独立传输源码核查补充 + +已确认`ReticulumInterfaceSet::applyConfig`按网络接口列表决定是否调用集成LoRa的applyConfig；单靠MeshConfig的WifiGatewayOnly不能当成不触碰射频的保证。`syncSharedLoRaRxGate`还会调用全局AppTasks::setRadioReceiveSuppressed，因此后台IP服务若沿用当前行为会抑制活动Meshtastic/MeshCore接收。 + +后续实现必须引入明确的射频所有权模式：活动Reticulum沿用原有控制；后台Geocaching IP服务不配置集成LoRa、不接收原始LoRa包、不改变全局RX gate。网络配置刷新也必须保持此约束，不能只在首次初始化时禁用。该隔离尚未完成，不能因为router已能后台安装/轮询就宣称无需切换协议已经实现。 + +先落实共享资源/独立传输与UI集成边界，再实现协议对象/GPX与请求核心、设备适配、页面/地图以及相应验证；保持设计的公开发现与GPX兼容目标，不以一个占位页替代完整功能。 + +## 2026-09-16 进度交接 + +后续用户已明确选择“继续开发”，下列交接清单是该时点的状态，不再表示暂停。恢复后继续执行相同的内存、共享 SPI、GPX 复用与 TopBar 约束。 + +用户因额度紧张要求报告已完成与未完成事项。本记录保留当时的核实结果，后续继续时须再次检查工作区，不能将其视为整个功能已交付。 + +- 分支为 `codex/geocaching`，本轮实现尚未提交、未刷入设备。 +- `tlora_pager_lr1121` 最近一次构建成功，退出码 0；日志位于本地 `.codex-build/geocaching-lr1121-build.log`。静态 RAM 从整改前同型号固件的 104,568 字节降为 99,200 字节，减少 5,368 字节。页面全局符号仅剩两个指针，共 8 字节；这不代表已经验证实机运行时堆内存。 +- 协议/存储最近一次全量本地验证为 71 项通过；之后增加的分步日志取消、未完成状态不可见、发送前持久化、停止任务后不发送等边界用例，在 `geocaching_sd_journal` 目标中通过。 +- 原生 LVGL 页面目标位于 `modules/ui_shared/tests/geocaching`，480×222 和 320×240 两项测试通过；覆盖可见行、翻页、返回及 20 次进入退出无 LVGL 内存增长。渲染图片位于本地 `.codex-build/geocaching-ui`；测试内容源和电量输入是模拟数据。 + +仍未完成的主线： + +1. 收缩 Outgoing 工作区、LogicalState 双 arena、整帧读取及签发/发送缓冲，并把 GPX 暂存写入改为有界步骤。 +2. 接入既有存储维护 owner，完成恢复、请求调度和网络回调的装配；落实后台 Reticulum 按需创建、身份/配置变化及生命周期管理。 +3. 连通目录发现、查询、发布和下载，完成多目录协调、错误重试、业务结果持久接受；不能把传输成功当成宝藏发布成功。 +4. 扩展同一套公共 GPX 读取能力，完成航点及扩展字段验证、正式安装、断电恢复和重新导入。 +5. 绑定页面 Source，完成创建/编辑/详情/下载交互，以及地图图层、详情联动和导航。当前页面不代表可用的端到端业务。 +6. 实现文档约定的 GitHub Pages 世界地图与 Reticulum 网关；目前仅有设计。 +7. 在设备上验证真实网络互通、SD 拔卡/断电、地图与通信并行时延及长期内存占用，再执行最终提交检查。 + +### 恢复开发后的签发/发布工作集收缩 + +- `AuthorIssue` 删除原始记录副本、签名输出数组和密码运算工作区数组，对象编译检查不超过 128 字节。任务借用已经固定的不可变草稿记录；owner 在内存准入后提供 `record.size + 70` 字节的操作缓冲。预留版本时用作哈希工作区，签名时先保存待签名消息，再原地写为 SignedCache。原记录必须与该缓冲不重叠；缓冲必须保留到结果被消费或任务终止。此处没有新增常驻最大缓冲，也不允许 owner 为每个并行任务重新预留最大记录容量。 +- `PublishAttempt` 删除 8,192 字节请求数组、4,166 字节验签工作区和 512 字节响应数组，对象编译检查不超过 192 字节。一次 begin 使用 `signed_record.size + 26` 字节的临时工作区，先验签、再编码请求；submit 必须在返回前完成对输入的消费，调用后可释放该临时工作区。已确认响应的全部字段通过规范解码与已保存的请求/记录元数据匹配，只需额外保留发布结果枚举即可区分重复确认和不同结果。 +- 使用既有真实 Ed25519 向量验证签名工作区与输出复用时的字节一致性；补充容量不足、输入/输出重叠、临时发布缓冲被覆盖后接收响应、非规范响应及不同结果拒绝测试。更新后本地 71 项测试通过。 +- `PublishAttemptPort` 已改为 Rejected/Pending/Committed 分步契约，通过 `SdPublishPort` 接入设备存储维护 owner；请求提交、确认保存、停止重试分别等待耐久提交。排队不代表发布成功，只有匹配当前版本的目录确认落盘后才显示接纳。 + +### 草稿发布的身份与重试约束 + +- 草稿 ID 由设备 CSPRNG 生成并随首次保存耐久提交；首次发布使用这 16 字节作为该对象的 creation_nonce。重开草稿或重试不重新生成 nonce。 +- 确认页显示完整作者公共身份。提交时重新检查作者和 edit_generation，先将作者绑定到 Draft，再预留版本、签名和排队；绑定后普通编辑不能替换或清除作者。 +- 首次签发构造 v1，created_at 与 updated_at 使用设备 UTC，签发记录同时保留这一 UTC。重试按最新签发版本的原时间、前驱和字段重建并核对 revision_hash；内容未变复用该版本。内容改变且前一版本已有目录确认时，生成 vN+1，引用 vN 的哈希并保留 v1 创建时间。确认页展示版本变化，提交及签发前再次核对目标版本。当前设备入口要求有效设备时钟和已发现目录，离线仍可保存草稿。 +- 版本预留与 Draft.base_hash、edit_generation 在同一事务中提交。完整 SignedCache 尚未耐久保存在原发布请求时，普通保存不得覆盖这一草稿快照；留存后可继续编辑。这样预留后掉电仍能重建原记录，不靠再次生成 nonce 或时间补造历史。前驱不匹配、创建时间改变、版本跳号和回退均拒绝。 +- 当前已连接首次发布、前一版本已确认后的后继发布、未确认任务恢复及同版本原请求复用。恢复重新验证保存的 SignedCache，并沿用本机身份、目的地、task_id 和 request_id；已有匹配的耐久确认直接恢复确认状态。已停止任务不会自动复活。前一版本未确认时的显式强制后继、多入口任务、完整版本历史页及真实目录/实机重启联调仍属于未完成项，不能将该入口视作完整发布生命周期已经验收。 + +### 本机发布互通验证 + +`tools/geocaching/interop_peer.py` 的 `publish-probe` 模式通过官方 Python RNS/LXMF 实际发送新签名标点，检查发布确认、相同 request_id 的确认重放、后续 query 可见性和 get 返回原签名记录；随后发布绑定 v1 哈希的 v2，检查查询前进到 v2、v1 仍可按哈希取回，且更新后旧请求仍重放原确认。目录模式只使用有界内存（8 个测试对象、32 个历史版本、32 条发布请求结果），默认监听 loopback；它不是正式公共目录，不证明目录掉电耐久性。 + +两个独立终端在安装 `interop-requirements.txt` 的虚拟环境中运行： + +```text +python tools/geocaching/interop_peer.py directory --state .codex-build/geocaching-publication-server --port 44243 --seconds 60 +python tools/geocaching/interop_peer.py publish-probe --state .codex-build/geocaching-publication-probe --port 44243 +``` + +探针保存 `received-{0,1,2,3}.lxmf`、来源公共身份和最终版本的 `sent-signed.bin`，另存 `received-history.lxmf/.pub` 与 `sent-signed-v1.bin` 用于历史下载校验。原生目标 `geocaching_verify_peer_capture` 接收三个参数：报文文件、来源 `.pub` 文件、预期 SignedCache 文件；使用工程的 C++ 代码核对 LXMF 签名、来源 destination、发布确认、查询摘要和下载记录的作者签名及精确版本。v1/v2 闭环及 C++ 报文校验已经在本机通过，仍不能替代 Pager 无线链路、SD 和运行内存实测。私有测试身份仅留在忽略目录，不提交到仓库。 diff --git a/docs/locales/zh-Hans/geocaching-lxmf/README.md b/docs/locales/zh-Hans/geocaching-lxmf/README.md index 7f26d521..ba9e4eb6 100644 --- a/docs/locales/zh-Hans/geocaching-lxmf/README.md +++ b/docs/locales/zh-Hans/geocaching-lxmf/README.md @@ -1,5 +1,7 @@ # 基于 LXMF 的藏宝点发布与下载设计 +> **2026-09-16更新：用户已授权在新分支实现。** 最新实现约束见[24](./24-实现授权与增补约束.md)：无需切换当前通信协议、使用geocaching.c、保持现有TopBar。下文较早“未授权”状态为设计阶段历史，不再阻止本次明确授权的实现。 + 状态：**本轮文档补全已交付，待用户评审，未授权开发** 文档版本：`0.2.0-draft` 拟议线协议版本：`1` diff --git a/modules/core_chat/include/chat/infra/lxmf/lxmf_wire.h b/modules/core_chat/include/chat/infra/lxmf/lxmf_wire.h index f0826ceb..b17e80b7 100644 --- a/modules/core_chat/include/chat/infra/lxmf/lxmf_wire.h +++ b/modules/core_chat/include/chat/infra/lxmf/lxmf_wire.h @@ -18,6 +18,8 @@ namespace chat::lxmf constexpr uint32_t kFieldTelemetry = 0x02; constexpr uint32_t kFieldTelemetryStream = 0x03; constexpr uint32_t kFieldCommands = 0x09; +constexpr uint32_t kFieldCustomType = 0xFB; +constexpr uint32_t kFieldCustomData = 0xFC; struct DecodedField { @@ -37,6 +39,13 @@ struct ByteSpanList size_t size = 0; }; +enum class CustomDataResult : uint8_t +{ + NotCustom, + Invalid, + Valid, +}; + using BinItemCallback = bool (*)(const uint8_t* data, size_t len, void* context); @@ -202,6 +211,11 @@ bool encodeTextPayload(double timestamp, uint8_t* out_payload, size_t* inout_len); +// Standard custom fields: type is MessagePack str; data is MessagePack bin. +bool encodeCustomDataPayload(double timestamp, const char* title, const char* content, + const char* custom_type, ByteSpan custom_data, + uint8_t* out_payload, size_t* inout_len); + bool encodeAppDataPayload(uint32_t portnum, uint32_t packet_id, uint32_t request_id, @@ -390,6 +404,9 @@ bool encodeSidebandTelemetryLocationPayload( uint8_t* out_payload, size_t* inout_len); const DecodedField* findField(const DecodedTextPayload& payload, uint32_t key); +// Output borrows payload.fields storage; reject duplicate and wrong-type fields. +CustomDataResult extractCustomData(const DecodedTextPayload& payload, + ByteSpan* out_type, ByteSpan* out_data); bool decodeSidebandTelemetryLocation(const DecodedTextPayload& payload, SidebandTelemetryLocation* out_location); bool decodeSidebandTelemetryRequest(const DecodedTextPayload& payload, diff --git a/modules/core_chat/include/chat/infra/mesh_adapter_router_core.h b/modules/core_chat/include/chat/infra/mesh_adapter_router_core.h index cb1e723f..48f3c6c3 100644 --- a/modules/core_chat/include/chat/infra/mesh_adapter_router_core.h +++ b/modules/core_chat/include/chat/infra/mesh_adapter_router_core.h @@ -11,6 +11,15 @@ class MeshAdapterRouterCore : public IMeshAdapter { public: bool installBackend(MeshProtocol protocol, std::unique_ptr backend); + // Installs a background service without changing the selected chat protocol. + // Refuses to replace any existing backend; its lifetime may have borrowers. + bool installServiceBackend(MeshProtocol protocol, std::unique_ptr backend); + std::unique_ptr takeServiceBackend(MeshProtocol protocol, const IMeshAdapter* expected = nullptr); + std::unique_ptr takeInactiveBackend(MeshProtocol protocol, const IMeshAdapter* expected = nullptr); + // Advances an inactive service only. The caller must configure its carrier + // so it cannot contend with the active backend for a shared radio. + bool processServiceQueue(MeshProtocol protocol); + bool isServiceBackend(MeshProtocol protocol) const; void setActiveProtocol(MeshProtocol protocol); bool hasBackend() const; MeshProtocol backendProtocol() const; @@ -74,6 +83,8 @@ class MeshAdapterRouterCore : public IMeshAdapter std::unique_ptr meshcore_backend_; std::unique_ptr reticulum_backend_; MeshProtocol active_protocol_ = MeshProtocol::Meshtastic; + // Only explicitly installed service backends may run while inactive. + uint8_t service_backend_mask_ = 0; }; } // namespace chat diff --git a/modules/core_chat/include/chat/ports/i_geocaching_transport.h b/modules/core_chat/include/chat/ports/i_geocaching_transport.h new file mode 100644 index 00000000..2faab24d --- /dev/null +++ b/modules/core_chat/include/chat/ports/i_geocaching_transport.h @@ -0,0 +1,52 @@ +#pragma once +#include "chat/domain/chat_types.h" +#include "chat/infra/lxmf/lxmf_wire.h" +#include +#include + +namespace chat +{ +namespace lxmf +{ +// These views borrow transport-owned memory for the callback duration only. +struct CustomDeliveryView +{ + ByteSpan source; + ByteSpan destination; + ByteSpan message_hash; + ByteSpan data; +}; + +// True means durable acceptance; volatile queueing alone must return false. +using CustomDeliveryHandler = bool (*)(const CustomDeliveryView&, void*); + +struct GeocachingAnnouncementView +{ + ByteSpan discovery_destination; + ByteSpan delivery_destination; + ByteSpan public_key; + ByteSpan app_data; +}; +using GeocachingAnnouncementHandler = void (*)(const GeocachingAnnouncementView&, void*); +} +// Optional application transport capability; unrelated adapters need not +// implement it. The owning router controls lifetime and serializes access. +class IGeocachingTransport +{ + public: + virtual ~IGeocachingTransport() = default; + virtual bool getGeocachingAuthorKey(uint8_t out[64]) + { if (out) std::memset(out, 0, 64); return false; } + virtual bool signGeocachingRecord(lxmf::ByteSpan, uint8_t*, size_t, uint8_t*, size_t, size_t& written) + { written = 0; return false; } + // Input bytes are borrowed for this call only; deferred transport work must + // own its encoded payload and must not retain the caller's pointer. + virtual MeshSendResult sendGeocachingData(const uint8_t destination_hash[16], + lxmf::ByteSpan data, bool response = false, + std::array* accepted_lxmf_hash = nullptr) = 0; + virtual void setGeocachingAnnouncementHandler( + void (*handler)(const lxmf::GeocachingAnnouncementView&, void*), void* context) = 0; + virtual void setGeocachingDeliveryHandler( + bool (*handler)(const lxmf::CustomDeliveryView&, void*), void* context) = 0; +}; +} // namespace chat diff --git a/modules/core_chat/include/chat/ports/i_mesh_adapter.h b/modules/core_chat/include/chat/ports/i_mesh_adapter.h index ae0991c2..a6f392aa 100644 --- a/modules/core_chat/include/chat/ports/i_mesh_adapter.h +++ b/modules/core_chat/include/chat/ports/i_mesh_adapter.h @@ -11,6 +11,7 @@ namespace chat { class IIncomingDeliveryCommitPort; +class IGeocachingTransport; namespace meshcore { @@ -63,6 +64,7 @@ struct ReticulumLocalIdentityInfo class IMeshAdapter { public: + virtual IGeocachingTransport* geocachingTransport() { return nullptr; } virtual ~IMeshAdapter() = default; /** diff --git a/modules/core_chat/src/infra/lxmf/lxmf_wire.cpp b/modules/core_chat/src/infra/lxmf/lxmf_wire.cpp index e4dfeca4..880767b7 100644 --- a/modules/core_chat/src/infra/lxmf/lxmf_wire.cpp +++ b/modules/core_chat/src/infra/lxmf/lxmf_wire.cpp @@ -903,6 +903,36 @@ bool encodeTextPayload(double timestamp, return true; } +bool encodeCustomDataPayload(double timestamp, const char* title, const char* content, + const char* custom_type, ByteSpan custom_data, + uint8_t* out_payload, size_t* inout_len) +{ + if (!inout_len) return false; + const size_t capacity = *inout_len; + *inout_len = 0; + if (!out_payload || !custom_type || !custom_type[0] || + (!custom_data.data && custom_data.size) || custom_data.size > 65535) return false; + const auto* title_bytes = reinterpret_cast(title ? title : ""); + const auto* content_bytes = reinterpret_cast(content ? content : ""); + const auto* type_bytes = reinterpret_cast(custom_type); + const size_t title_len = title ? strlen(title) : 0; + const size_t content_len = content ? strlen(content) : 0; + const size_t type_len = strlen(custom_type); + if (title_len > 65535 || content_len > 65535 || type_len > 65535) return false; + size_t used = 0; + if (!appendArrayHeader(4, out_payload, capacity, used) || + !appendFloat64(timestamp, out_payload, capacity, used) || + !appendBin(title_bytes, title_len, out_payload, capacity, used) || + !appendBin(content_bytes, content_len, out_payload, capacity, used) || + !appendMapHeader(2, out_payload, capacity, used) || + !appendUint(0xfb, out_payload, capacity, used) || + !appendString(type_bytes, type_len, out_payload, capacity, used) || + !appendUint(0xfc, out_payload, capacity, used) || + !appendBin(custom_data.data, custom_data.size, out_payload, capacity, used)) return false; + *inout_len = used; + return true; +} + bool encodeSidebandTelemetryLocationPayload( double message_timestamp, const SidebandTelemetryLocation& location, @@ -2472,8 +2502,15 @@ bool unpackTextPayload(const uint8_t* data, size_t len, DecodedTextPayload* out_ if (numeric_key && (field_key == kFieldTelemetry || field_key == kFieldTelemetryStream || - field_key == kFieldCommands)) + field_key == kFieldCommands || + field_key == kFieldCustomType || + field_key == kFieldCustomData)) { + if ((field_key == kFieldCustomType && cursor.pos - value_start > 258) || + (field_key == kFieldCustomData && cursor.pos - value_start > 8195)) + { + return false; + } DecodedField field{}; field.key = field_key; field.encoded_value.assign(data + value_start, data + cursor.pos); @@ -2509,6 +2546,43 @@ bool unpackTextPayload(const uint8_t* data, size_t len, DecodedTextPayload* out_ return true; } +CustomDataResult extractCustomData(const DecodedTextPayload& payload, + ByteSpan* out_type, ByteSpan* out_data) +{ + if (!out_type || !out_data) return CustomDataResult::Invalid; + *out_type = {}; + *out_data = {}; + const DecodedField* type = nullptr; + const DecodedField* data = nullptr; + for (const auto& field : payload.fields) + { + if (field.key == kFieldCustomType) + { + if (type) return CustomDataResult::Invalid; + type = &field; + } + if (field.key == kFieldCustomData) + { + if (data) return CustomDataResult::Invalid; + data = &field; + } + } + if (!type && !data) return CustomDataResult::NotCustom; + if (!type || !data || type->encoded_value.empty() || data->encoded_value.empty()) return CustomDataResult::Invalid; + const uint8_t type_tag = type->encoded_value[0]; + const uint8_t data_tag = data->encoded_value[0]; + if (!((type_tag & 0xe0) == 0xa0 || type_tag == 0xd9 || type_tag == 0xda) || + !(data_tag == 0xc4 || data_tag == 0xc5)) return CustomDataResult::Invalid; + Cursor tc{type->encoded_value.data(), type->encoded_value.size(), 0}; + Cursor dc{data->encoded_value.data(), data->encoded_value.size(), 0}; + ByteSpan tv, dv; + if (!readBinarySpan(tc, &tv.data, &tv.size) || tc.pos != tc.len || tv.size == 0 || tv.size > 96 || + !readBinarySpan(dc, &dv.data, &dv.size) || dc.pos != dc.len || dv.size > 8192) return CustomDataResult::Invalid; + *out_type = tv; + *out_data = dv; + return CustomDataResult::Valid; +} + const DecodedField* findField(const DecodedTextPayload& payload, uint32_t key) { const auto found = std::find_if(payload.fields.begin(), diff --git a/modules/core_chat/src/infra/mesh_adapter_router_core.cpp b/modules/core_chat/src/infra/mesh_adapter_router_core.cpp index 92c583ab..d70e4512 100644 --- a/modules/core_chat/src/infra/mesh_adapter_router_core.cpp +++ b/modules/core_chat/src/infra/mesh_adapter_router_core.cpp @@ -5,6 +5,22 @@ namespace chat namespace { +uint8_t serviceBit(MeshProtocol protocol) +{ + switch (protocol) + { + case MeshProtocol::Meshtastic: + return 1; + case MeshProtocol::MeshCore: + return 2; + case MeshProtocol::RNode: + case MeshProtocol::Reticulum: + return 4; + default: + return 0; + } +} + std::unique_ptr& backendSlot(MeshProtocol protocol, std::unique_ptr& meshtastic_backend, std::unique_ptr& meshcore_backend, @@ -51,6 +67,7 @@ bool MeshAdapterRouterCore::installBackend(MeshProtocol protocol, std::unique_pt } backendSlot(protocol, meshtastic_backend_, meshcore_backend_, reticulum_backend_) = std::move(backend); + service_backend_mask_ &= static_cast(~serviceBit(protocol)); active_protocol_ = protocol; return true; } @@ -60,11 +77,45 @@ void MeshAdapterRouterCore::setActiveProtocol(MeshProtocol protocol) active_protocol_ = protocol; } +bool MeshAdapterRouterCore::installServiceBackend(MeshProtocol protocol, std::unique_ptr backend) +{ + if (!backend || (protocol != MeshProtocol::Meshtastic && protocol != MeshProtocol::MeshCore && + protocol != MeshProtocol::Reticulum && protocol != MeshProtocol::RNode)) + { + return false; + } + auto& slot = backendSlot(protocol, meshtastic_backend_, meshcore_backend_, reticulum_backend_); + if (slot) + { + return false; + } + slot = std::move(backend); + service_backend_mask_ |= serviceBit(protocol); + return true; +} + bool MeshAdapterRouterCore::hasBackend() const { return activeBackend() != nullptr; } +std::unique_ptr MeshAdapterRouterCore::takeServiceBackend(MeshProtocol protocol, const IMeshAdapter* expected) +{ + const auto bit = serviceBit(protocol); + if (!bit || !(service_backend_mask_ & bit)) return {}; + return takeInactiveBackend(protocol, expected); +} + +std::unique_ptr MeshAdapterRouterCore::takeInactiveBackend(MeshProtocol protocol, const IMeshAdapter* expected) +{ + const auto bit = serviceBit(protocol); + if (!bit || serviceBit(active_protocol_) == bit) return {}; + auto& slot = backendSlot(protocol, meshtastic_backend_, meshcore_backend_, reticulum_backend_); + if (expected && slot.get() != expected) return {}; + service_backend_mask_ &= static_cast(~bit); + return std::move(slot); +} + MeshProtocol MeshAdapterRouterCore::backendProtocol() const { return active_protocol_; @@ -353,6 +404,32 @@ IMeshAdapter* MeshAdapterRouterCore::activeBackend() return backendForProtocol(active_protocol_); } +bool MeshAdapterRouterCore::processServiceQueue(MeshProtocol protocol) +{ + if ((service_backend_mask_ & serviceBit(protocol)) == 0) + { + return false; + } + if (protocol != MeshProtocol::Meshtastic && protocol != MeshProtocol::MeshCore && + protocol != MeshProtocol::Reticulum && protocol != MeshProtocol::RNode) + { + return false; + } + IMeshAdapter* service = backendForProtocol(protocol); + if (!service || service == activeBackend()) + { + return false; + } + // A not-yet-ready service still needs polling to establish its IP path. + service->processSendQueue(); + return true; +} + +bool MeshAdapterRouterCore::isServiceBackend(MeshProtocol protocol) const +{ + return (service_backend_mask_ & serviceBit(protocol)) != 0 && backendForProtocol(protocol) != nullptr; +} + const IMeshAdapter* MeshAdapterRouterCore::activeBackend() const { return backendForProtocol(active_protocol_); diff --git a/modules/core_geocaching/include/geocaching/domain/record.h b/modules/core_geocaching/include/geocaching/domain/record.h new file mode 100644 index 00000000..6c74c78c --- /dev/null +++ b/modules/core_geocaching/include/geocaching/domain/record.h @@ -0,0 +1,95 @@ +#pragma once + +#include +#include +#include +#include + +namespace geocaching +{ + +inline constexpr std::size_t kMaxRecordBytes = 4096; +inline constexpr std::size_t kMaxApplicationBytes = 8192; +inline constexpr std::size_t kMaxGpxBytes = 65536; +inline constexpr std::size_t kMaxNameBytes = 96; +inline constexpr std::size_t kMaxDescriptionBytes = 2048; +inline constexpr std::size_t kMaxHintBytes = 512; + +struct GeocacheId +{ + std::array bytes{}; +}; + +struct RevisionHash +{ + std::array bytes{}; +}; + +struct RequestId +{ + std::array bytes{}; +}; + +struct Destination +{ + std::array bytes{}; +}; + +enum class CacheState : std::uint8_t +{ + Active = 0, + Disabled = 1, + Archived = 2, +}; + +enum class ContainerSize : std::uint8_t +{ + Unspecified = 0, + Micro = 1, + Small = 2, + Regular = 3, + Large = 4, + Other = 5, +}; + +enum class Operation : std::uint8_t +{ + Capabilities = 0, + Publish = 1, + Query = 2, + Get = 3, + Sync = 4, +}; + +// Borrowed bytes. The caller owns the immutable backing payload and must keep +// it alive throughout validation and use of any views derived from it. +struct ByteView +{ + const std::uint8_t* data = nullptr; + std::size_t size = 0; +}; + +// Parsing is not verification. This view must never be advertised as an +// authenticated object until signature and version checks have completed. +// Text is borrowed from the original record, not copied into task-stack arrays. +struct RecordView +{ + ByteView encoded; + ByteView author_public_key; + ByteView creation_nonce; + ByteView previous_hash; + std::uint32_t revision = 0; + CacheState state = CacheState::Active; + std::int32_t latitude_e7 = 0; + std::int32_t longitude_e7 = 0; + std::string_view name; + std::string_view description; + std::string_view hint; + std::uint8_t difficulty_x2 = 0; + std::uint8_t terrain_x2 = 0; + ContainerSize container_size = ContainerSize::Unspecified; + std::uint64_t created_at = 0; + std::uint64_t updated_at = 0; +}; + +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/domain/version_policy.h b/modules/core_geocaching/include/geocaching/domain/version_policy.h new file mode 100644 index 00000000..34395eae --- /dev/null +++ b/modules/core_geocaching/include/geocaching/domain/version_policy.h @@ -0,0 +1,40 @@ +#pragma once +#include "geocaching/protocol/verify_record.h" +#include + +namespace geocaching +{ +enum class VersionRelation : std::uint8_t +{ + DifferentCache, + Identical, + Older, + NewerLinked, + NewerHistoryIncomplete, + Conflict, +}; + +// Call only for independently authenticated records. A repository must compare +// against relevant retained history as well, not just its current head. +inline VersionRelation compareGeocacheVersions(const protocol::VerifiedRecordView& known, + const protocol::VerifiedRecordView& incoming) +{ + if (known.id.bytes != incoming.id.bytes) return VersionRelation::DifferentCache; + if (known.hash.bytes == incoming.hash.bytes) return VersionRelation::Identical; + const auto& a = known.record; + const auto& b = incoming.record; + if (a.revision == b.revision || a.created_at != b.created_at) return VersionRelation::Conflict; + const bool newer = b.revision > a.revision; + const auto& lower = newer ? known : incoming; + const auto& higher = newer ? incoming : known; + if (lower.record.state == CacheState::Archived) return VersionRelation::Conflict; + if (static_cast(lower.record.revision) + 1 == higher.record.revision) + { + if (higher.record.previous_hash.size != 32 || !higher.record.previous_hash.data || + std::memcmp(higher.record.previous_hash.data, lower.hash.bytes.data(), 32) != 0) + return VersionRelation::Conflict; + return newer ? VersionRelation::NewerLinked : VersionRelation::Older; + } + return newer ? VersionRelation::NewerHistoryIncomplete : VersionRelation::Older; +} +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/gpx/write_record.h b/modules/core_geocaching/include/geocaching/gpx/write_record.h new file mode 100644 index 00000000..b6c0ee17 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/gpx/write_record.h @@ -0,0 +1,75 @@ +#pragma once +#include "gps/gpx/text_writer.h" +#include "geocaching/protocol/verify_record.h" +#include + +namespace geocaching::gpx +{ +using gps::gpx::OutputSink; +using gps::gpx::GpxTextWriter; +inline bool writeGeocacheGpx(const protocol::VerifiedRecordView& verified, + protocol::RecordCrypto& crypto, OutputSink& sink) +{ + const auto& r = verified.record; + if (!r.encoded.data || !verified.signature.data || verified.signature.size != 64 || + r.author_public_key.size != 64 || static_cast(r.state) > 2 || + static_cast(r.container_size) > 5) return false; + std::array author_hash{}; + if (!crypto.sha256(r.author_public_key, author_hash.data())) return false; + char timestamp[32]{}; + const auto seconds = r.updated_at ? r.updated_at : r.created_at; + if (seconds) + { + const std::time_t time = static_cast(seconds); + if (static_cast(time) != seconds) return false; + const auto* utc = std::gmtime(&time); + if (!utc || !std::strftime(timestamp, sizeof(timestamp), "%Y-%m-%dT%H:%M:%SZ", utc)) return false; + } + GpxTextWriter w(sink); + auto element = [&w](const char* tag, std::string_view value) + { + return w.raw("<") && w.raw(tag) && w.raw(">") && w.escaped(value) && + w.raw("\n"); + }; + auto hex = [&w](const std::uint8_t* bytes, std::size_t size) + { + constexpr char alphabet[] = "0123456789abcdef"; + for (std::size_t i = 0; i < size; ++i) + { + char pair[] = {alphabet[bytes[i] >> 4], alphabet[bytes[i] & 15]}; + if (!w.raw({pair, 2})) return false; + } + return true; + }; + char revision[16], difficulty[8], terrain[8]; + std::snprintf(revision, sizeof(revision), "%lu", static_cast(r.revision)); + std::snprintf(difficulty, sizeof(difficulty), "%u.%u", r.difficulty_x2 / 2, (r.difficulty_x2 % 2) * 5); + std::snprintf(terrain, sizeof(terrain), "%u.%u", r.terrain_x2 / 2, (r.terrain_x2 % 2) * 5); + const char* states[] = {"active", "disabled", "archived"}; + const char* containers[] = {"not-specified", "micro", "small", "regular", "large", "other"}; + const char* gs_containers[] = {"Not chosen", "Micro", "Small", "Regular", "Large", "Other"}; + const auto state = static_cast(r.state); + const auto container = static_cast(r.container_size); + if (!w.raw("\n\n\n")) return false; + if (seconds && !element("time", timestamp)) return false; + if (!element("name", r.name) || !element("cmt", r.hint) || !w.raw("") || !w.escaped(r.description) || + !w.raw("\n\n--- Reticulum Geocaching ---\nCache-ID: ") || !hex(verified.id.bytes.data(), 32) || + !w.raw("\nRevision: ") || !w.raw(revision) || !w.raw("\nState: ") || !w.raw(states[state]) || + !w.raw("\nDifficulty: ") || !w.raw(difficulty) || !w.raw("\nTerrain: ") || !w.raw(terrain) || + !w.raw("\nContainer: ") || !w.raw(containers[container]) || !w.raw("\nHint: ") || !w.escaped(r.hint) || !w.raw("\n") || + !element("src", "Reticulum Geocaching") || !element("sym", "Geocache") || !element("type", "Geocache|Traditional Cache") || + !w.raw("\n") || + !element("groundspeak:name", r.name) || !w.raw("Reticulum author ") || + !hex(author_hash.data(), 32) || !w.raw("\n") || + !element("groundspeak:type", "Traditional Cache") || !element("groundspeak:container", gs_containers[container]) || + !element("groundspeak:difficulty", difficulty) || !element("groundspeak:terrain", terrain) || + !w.raw("Public cache shared over Reticulum.\n") || + !w.escaped(r.description) || !w.raw("\n") || !element("groundspeak:encoded_hints", r.hint) || + !w.raw("") || !w.base64({r.encoded.data, r.encoded.size}) || + !w.raw("") || !w.base64({verified.signature.data, verified.signature.size}) || + !w.raw("\n")) return false; + return w.good(); +} +} // namespace geocaching::gpx diff --git a/modules/core_geocaching/include/geocaching/protocol/capabilities.h b/modules/core_geocaching/include/geocaching/protocol/capabilities.h new file mode 100644 index 00000000..a9b9084a --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/capabilities.h @@ -0,0 +1,64 @@ +#pragma once +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ +struct DirectoryCapabilities +{ + std::uint8_t max_query_items = 0; + std::uint32_t cursor_ttl_seconds = 0; + std::string_view name; +}; + +// This entry point accepts public directory capabilities only. The transport +// owner must first authenticate the source against the pending request target. +inline bool decodeDirectoryCapabilities(ByteView response, const RequestId& expected, + DirectoryCapabilities& out) +{ + out = {}; + if (!response.data || response.size > 512) return false; + CmpReader reader(response); + std::size_t count = 0; + std::uint64_t value = 0; + ByteView request; + if (!reader.array(count, 6) || count != 6 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 0 || + !reader.binary(request, 16) || request.size != 16 || + std::memcmp(request.data, expected.bytes.data(), 16) != 0 || + !reader.unsignedInteger(value) || value != 200 || + !reader.array(count, 12) || count != 12) return false; + for (unsigned list = 0; list < 2; ++list) + { + if (!reader.array(count, 8) || count == 0) return false; + bool supports_one = false; + std::uint64_t previous = 0; + for (std::size_t i = 0; i < count; ++i) + { + if (!reader.unsignedInteger(value) || value == 0 || (i && value <= previous)) return false; + supports_one = supports_one || value == 1; + previous = value; + } + if (!supports_one) return false; + } + if (!reader.array(count, 5) || count != 5) return false; + for (std::uint64_t i = 0; i < 5; ++i) + if (!reader.unsignedInteger(value) || value != i) return false; + if (!reader.unsignedInteger(value) || value != kMaxApplicationBytes || + !reader.unsignedInteger(value) || value != kMaxRecordBytes) return false; + DirectoryCapabilities candidate; + if (!reader.unsignedInteger(value) || value < 1 || value > 64) return false; + candidate.max_query_items = static_cast(value); + if (!reader.unsignedInteger(value) || value < 60 || value > 604800) return false; + candidate.cursor_ttl_seconds = static_cast(value); + if (!reader.unsignedInteger(value) || value != 2592000 || + !reader.unsignedInteger(value) || value != 2 || + !reader.text(candidate.name, 96) || !validRecordText(candidate.name, false, false) || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 2592000 || !reader.finished()) return false; + out = candidate; + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/cmp_reader.h b/modules/core_geocaching/include/geocaching/protocol/cmp_reader.h new file mode 100644 index 00000000..4fd0c45b --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/cmp_reader.h @@ -0,0 +1,163 @@ +#pragma once + +#include "geocaching/domain/record.h" +#include + +namespace geocaching::protocol +{ + +// Allocation-free cursor for the deterministic MessagePack subset. A failed +// operation poisons the cursor: callers cannot accidentally accept its suffix. +class CmpReader +{ + public: + explicit CmpReader(ByteView bytes) + : bytes_(bytes), valid_(bytes.data != nullptr || bytes.size == 0) {} + + bool good() const { return valid_; } + // Only use a position after checking the preceding decode succeeded. + std::size_t position() const { return offset_; } + bool finished() const { return valid_ && offset_ == bytes_.size; } + + bool nil() + { + std::uint8_t tag = 0; + return take(tag) && (tag == 0xc0 || fail()); + } + + bool unsignedInteger(std::uint64_t& out) + { + std::uint8_t tag = 0; + if (!take(tag)) return false; + if (tag <= 0x7f) + { + out = tag; + return true; + } + if (tag < 0xcc || tag > 0xcf) return fail(); + const unsigned width = 1U << (tag - 0xcc); + std::uint64_t value = 0; + if (!number(width, value)) return false; + const std::uint64_t minimum = width == 1 ? 128ULL : (1ULL << ((width / 2) * 8)); + if (value < minimum) return fail(); + out = value; + return true; + } + + bool signedInteger(std::int64_t& out) + { + if (!valid_ || offset_ == bytes_.size) return fail(); + const auto tag = bytes_.data[offset_]; + if (tag <= 0x7f || (tag >= 0xcc && tag <= 0xcf)) + { + std::uint64_t value = 0; + if (!unsignedInteger(value) || value > static_cast(INT64_MAX)) return fail(); + out = static_cast(value); + return true; + } + ++offset_; + if (tag >= 0xe0) + { + out = static_cast(tag) - 256; + return true; + } + if (tag < 0xd0 || tag > 0xd3) return fail(); + const unsigned width = 1U << (tag - 0xd0); + std::uint64_t raw = 0; + if (!number(width, raw)) return false; + const unsigned bits = width * 8; + const std::uint64_t mask = width == 8 ? UINT64_MAX : (1ULL << bits) - 1; + if ((raw & (1ULL << (bits - 1))) == 0) return fail(); + const auto value = -1 - static_cast((~raw) & mask); + const std::int64_t maximum = width == 1 ? -33 : -(1LL << ((width / 2) * 8 - 1)) - 1; + if (value > maximum) return fail(); + out = value; + return true; + } + + bool array(std::size_t& count, std::size_t limit) + { + std::uint8_t tag = 0; + if (!take(tag)) return false; + std::uint64_t n = 0; + if (tag >= 0x90 && tag <= 0x9f) n = tag - 0x90; + else if (tag == 0xdc || tag == 0xdd) + { + if (!number(tag == 0xdc ? 2 : 4, n)) return false; + if (n < (tag == 0xdc ? 16ULL : 65536ULL)) return fail(); + } + else return fail(); + if (n > limit) return fail(); + count = static_cast(n); + return true; + } + + bool binary(ByteView& out, std::size_t limit) + { + std::uint8_t tag = 0; + if (!take(tag) || tag < 0xc4 || tag > 0xc6) return fail(); + const unsigned width = 1U << (tag - 0xc4); + std::uint64_t n = 0; + if (!number(width, n)) return false; + if (width > 1 && n < (1ULL << ((width / 2) * 8))) return fail(); + return slice(n, limit, out); + } + + // Encoding and bounds only; record-level Unicode policy is checked by the + // record decoder before exposing a successfully parsed RecordView. + bool text(std::string_view& out, std::size_t limit) + { + std::uint8_t tag = 0; + if (!take(tag)) return false; + std::uint64_t n = 0; + if (tag >= 0xa0 && tag <= 0xbf) n = tag - 0xa0; + else if (tag >= 0xd9 && tag <= 0xdb) + { + const unsigned width = 1U << (tag - 0xd9); + if (!number(width, n)) return false; + const std::uint64_t minimum = width == 1 ? 32ULL : (1ULL << ((width / 2) * 8)); + if (n < minimum) return fail(); + } + else return fail(); + ByteView bytes; + if (!slice(n, limit, bytes)) return false; + out = std::string_view(reinterpret_cast(bytes.data), bytes.size); + return true; + } + + private: + bool fail() + { + valid_ = false; + return false; + } + bool take(std::uint8_t& out) + { + if (!valid_ || offset_ == bytes_.size) return fail(); + out = bytes_.data[offset_++]; + return true; + } + bool number(unsigned width, std::uint64_t& out) + { + out = 0; + for (unsigned i = 0; i < width; ++i) + { + std::uint8_t value = 0; + if (!take(value)) return false; + out = (out << 8) | value; + } + return true; + } + bool slice(std::uint64_t n, std::size_t limit, ByteView& out) + { + if (!valid_ || n > limit || n > bytes_.size - offset_) return fail(); + out = {bytes_.data + offset_, static_cast(n)}; + offset_ += out.size; + return true; + } + ByteView bytes_; + std::size_t offset_ = 0; + bool valid_ = false; +}; + +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/cmp_writer.h b/modules/core_geocaching/include/geocaching/protocol/cmp_writer.h new file mode 100644 index 00000000..5282589c --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/cmp_writer.h @@ -0,0 +1,122 @@ +#pragma once +#include "geocaching/domain/record.h" +#include + +namespace geocaching::protocol +{ +class CmpWriter +{ + public: + CmpWriter(std::uint8_t* data, std::size_t capacity) + : data_(data), capacity_(capacity), good_(data != nullptr) {} + bool good() const { return good_; } + std::size_t size() const { return good_ ? size_ : 0; } + bool nil() { return byte(0xc0); } + bool text(std::string_view value) + { + if ((!value.data() && value.size()) || value.size() > UINT32_MAX) return fail(); + if (value.size() < 32) + { + if (!byte(static_cast(0xa0 + value.size()))) return false; + } + else + { + const unsigned width = value.size() <= UINT8_MAX ? 1 : value.size() <= UINT16_MAX ? 2 + : 4; + if (!byte(width == 1 ? 0xd9 : width == 2 ? 0xda + : 0xdb) || + !number(value.size(), width)) return false; + } + if (!good_ || value.size() > capacity_ - size_) return fail(); + if (value.size()) std::memcpy(data_ + size_, value.data(), value.size()); + size_ += value.size(); + return true; + } + bool unsignedInteger(std::uint64_t value) + { + if (value < 128) return byte(static_cast(value)); + const unsigned width = value <= UINT8_MAX ? 1 : value <= UINT16_MAX ? 2 + : value <= UINT32_MAX ? 4 + : 8; + return byte(width == 1 ? 0xcc : width == 2 ? 0xcd + : width == 4 ? 0xce + : 0xcf) && + number(value, width); + } + bool signedInteger(std::int64_t value) + { + if (value >= 0) return unsignedInteger(static_cast(value)); + if (value >= -32) return byte(static_cast(value)); + const unsigned width = value >= INT8_MIN ? 1 : value >= INT16_MIN ? 2 + : value >= INT32_MIN ? 4 + : 8; + return byte(width == 1 ? 0xd0 : width == 2 ? 0xd1 + : width == 4 ? 0xd2 + : 0xd3) && + number(static_cast(value), width); + } + bool array(std::uint32_t count) + { + if (count < 16) return byte(static_cast(0x90 + count)); + return byte(count <= UINT16_MAX ? 0xdc : 0xdd) && number(count, count <= UINT16_MAX ? 2 : 4); + } + // Emit only the canonical length prefix for a borrowed binary span. + bool binaryHeader(std::size_t length) + { + if (length > UINT32_MAX) return fail(); + const unsigned width = length <= UINT8_MAX ? 1 : length <= UINT16_MAX ? 2 + : 4; + return byte(width == 1 ? 0xc4 : width == 2 ? 0xc5 + : 0xc6) && + number(length, width); + } + bool binary(ByteView value) + { + if ((!value.data && value.size) || !binaryHeader(value.size)) return fail(); + if (!good_ || value.size > capacity_ - size_) return fail(); + if (value.size) std::memcpy(data_ + size_, value.data, value.size); + size_ += value.size; + return true; + } + + private: + bool fail() + { + good_ = false; + return false; + } + bool byte(std::uint8_t value) + { + if (!good_ || size_ == capacity_) return fail(); + data_[size_++] = value; + return true; + } + bool number(std::uint64_t value, unsigned width) + { + for (unsigned i = width; i > 0; --i) + if (!byte(static_cast(value >> ((i - 1) * 8)))) return false; + return true; + } + std::uint8_t* data_; + std::size_t capacity_; + std::size_t size_ = 0; + bool good_; +}; + +inline bool encodeGetRequest(const RequestId& request, const GeocacheId& id, + const RevisionHash* wanted, const RevisionHash* known, + std::uint16_t budget, std::uint8_t* output, + std::size_t capacity, std::size_t& written) +{ + written = 0; + if (budget < 512 || budget > kMaxApplicationBytes || (wanted && known)) return false; + CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.unsignedInteger(1) || !writer.unsignedInteger(0) || + !writer.unsignedInteger(3) || !writer.binary({request.bytes.data(), request.bytes.size()}) || + !writer.unsignedInteger(budget) || !writer.array(3) || !writer.binary({id.bytes.data(), id.bytes.size()}) || + !(wanted ? writer.binary({wanted->bytes.data(), wanted->bytes.size()}) : writer.nil()) || + !(known ? writer.binary({known->bytes.data(), known->bytes.size()}) : writer.nil())) return false; + written = writer.size(); + return writer.good(); +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/directory_reply.h b/modules/core_geocaching/include/geocaching/protocol/directory_reply.h new file mode 100644 index 00000000..5d930787 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/directory_reply.h @@ -0,0 +1,37 @@ +#pragma once +#include "geocaching/protocol/capabilities.h" +#include "geocaching/protocol/query_request.h" +#include "geocaching/protocol/query_response.h" + +namespace geocaching::protocol +{ +inline bool matchesCapabilitiesRequest(ByteView encoded, const RequestId& expected, size_t response_size) +{ + CmpReader request(encoded); + size_t fields = 0; + uint64_t version = 0, type = 0, operation = 0, budget = 0; + ByteView id; + return request.array(fields, 6) && fields == 6 && request.unsignedInteger(version) && version == 1 && + request.unsignedInteger(type) && type == 0 && request.unsignedInteger(operation) && operation == 0 && + request.binary(id, 16) && id.size == 16 && !std::memcmp(id.data, expected.bytes.data(), 16) && + request.unsignedInteger(budget) && budget >= 512 && budget <= kMaxApplicationBytes && budget >= response_size && + request.array(fields, 0) && fields == 0 && request.finished(); +} + +// Shape, request identity, page budget and every summary's filter membership. +// Transport authentication and durable acceptance remain owner responsibilities. +inline bool matchesQueryReply(ByteView request, ByteView response, const RequestId& id) +{ + QueryRequestView wanted; + QueryPageView page; + if (!decodeQueryRequest(request, id, wanted) || + !decodeQueryPage(response, id, wanted.budget, wanted.page_limit, page)) return false; + CmpReader rows(page.encoded_items); + SummaryView item; + for (size_t i = 0; i < page.count; ++i) + if (!decodeSummary(rows, item) || !(wanted.state_mask & (1u << static_cast(item.state))) || + item.latitude_e7 < wanted.region.south_e7 || item.latitude_e7 > wanted.region.north_e7 || + item.longitude_e7 < wanted.region.west_e7 || item.longitude_e7 > wanted.region.east_e7) return false; + return rows.finished(); +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/discovery.h b/modules/core_geocaching/include/geocaching/protocol/discovery.h new file mode 100644 index 00000000..9296a0a5 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/discovery.h @@ -0,0 +1,42 @@ +#pragma once +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ +inline constexpr char kApplicationType[] = "trailmate.geocache"; +inline constexpr char kDirectoryAspect[] = "trailmate.geocache.directory"; + +struct DirectoryAnnouncement +{ + Destination delivery; + std::array epoch{}; + std::uint64_t sequence = 0; + std::string_view name; +}; + +// Only call after native announce signature/aspect verification. expected_delivery +// must be derived from that authenticated identity, never from this app_data. +inline bool decodeDirectoryAnnouncement(ByteView app_data, const Destination& expected_delivery, + DirectoryAnnouncement& out) +{ + out = {}; + if (!app_data.data || app_data.size > 96) return false; + CmpReader reader(app_data); + std::size_t count = 0; + std::uint64_t version = 0; + ByteView delivery, epoch; + DirectoryAnnouncement candidate; + if (!reader.array(count, 5) || count != 5 || + !reader.unsignedInteger(version) || version != 1 || + !reader.binary(delivery, 16) || delivery.size != 16 || + std::memcmp(delivery.data, expected_delivery.bytes.data(), 16) != 0 || + !reader.binary(epoch, 16) || epoch.size != 16 || + !reader.unsignedInteger(candidate.sequence) || !reader.text(candidate.name, 40) || + !validRecordText(candidate.name, false, false) || !reader.finished()) return false; + std::memcpy(candidate.delivery.bytes.data(), delivery.data, 16); + std::memcpy(candidate.epoch.data(), epoch.data, 16); + out = candidate; + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/get_request.h b/modules/core_geocaching/include/geocaching/protocol/get_request.h new file mode 100644 index 00000000..f568b600 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/get_request.h @@ -0,0 +1,35 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" + +namespace geocaching::protocol +{ +struct GetRequestView +{ + ByteView cache_id, wanted_hash, known_hash; + uint16_t budget = 0; +}; +inline bool decodeGetRequest(ByteView bytes, const RequestId& expected, GetRequestView& out) +{ + out = {}; + CmpReader reader(bytes); + size_t fields = 0; + uint64_t value = 0; + ByteView id; + GetRequestView request; + if (!reader.array(fields, 6) || fields != 6 || !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 0 || !reader.unsignedInteger(value) || value != 3 || + !reader.binary(id, 16) || id.size != 16 || std::memcmp(id.data, expected.bytes.data(), 16) || + !reader.unsignedInteger(value) || value < 512 || value > 8192) return false; + request.budget = static_cast(value); + if (!reader.array(fields, 3) || fields != 3 || !reader.binary(request.cache_id, 32) || request.cache_id.size != 32) return false; + auto optional = reader; + if (optional.nil()) reader = optional; + else if (!reader.binary(request.wanted_hash, 32) || request.wanted_hash.size != 32) return false; + optional = reader; + if (optional.nil()) reader = optional; + else if (!reader.binary(request.known_hash, 32) || request.known_hash.size != 32) return false; + if (!reader.finished() || (request.wanted_hash.size && request.known_hash.size)) return false; + out = request; + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/get_response.h b/modules/core_geocaching/include/geocaching/protocol/get_response.h new file mode 100644 index 00000000..9cb5fbdc --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/get_response.h @@ -0,0 +1,49 @@ +#pragma once +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ +struct GetResponseView +{ + ByteView signed_cache; + bool is_current = false; + bool has_conflict = false; +}; + +// Accepts status 200 only. Does not authenticate source or author: the caller +// must match the transport identity, then verifyGeocache against requested IDs. +inline bool decodeGetResponse(ByteView response, const RequestId& expected, + std::size_t response_budget, GetResponseView& out) +{ + out = {}; + if (!response.data || response_budget < 512 || response_budget > kMaxApplicationBytes || + response.size > response_budget) return false; + CmpReader reader(response); + std::size_t count = 0; + std::uint64_t value = 0; + ByteView request, encoded, signature; + if (!reader.array(count, 6) || count != 6 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 3 || + !reader.binary(request, 16) || request.size != 16 || + std::memcmp(request.data, expected.bytes.data(), 16) != 0 || + !reader.unsignedInteger(value) || value != 200 || + !reader.array(count, 3) || count != 3 || + !reader.array(count, 2) || count != 2 || + !reader.binary(encoded, kMaxRecordBytes) || encoded.size == 0 || + !reader.binary(signature, 64) || signature.size != 64) return false; + GetResponseView candidate; + // Canonical array(2) has a one-byte header; bin <=4096 has a two/three-byte + // header. Recover the exact original slice without re-encoding signed data. + const auto* start = encoded.data - (encoded.size <= 255 ? 3 : 4); + candidate.signed_cache = {start, static_cast(signature.data + signature.size - start)}; + if (!reader.unsignedInteger(value) || value > 1) return false; + candidate.is_current = value != 0; + if (!reader.unsignedInteger(value) || value > 1 || !reader.finished()) return false; + candidate.has_conflict = value != 0; + out = candidate; + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/publish_request.h b/modules/core_geocaching/include/geocaching/protocol/publish_request.h new file mode 100644 index 00000000..6e3165c8 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/publish_request.h @@ -0,0 +1,52 @@ +#pragma once +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/protocol/record_decoder.h" + +namespace geocaching::protocol +{ +struct PublishRequestView +{ + ByteView signed_cache; + uint16_t budget = 0; +}; +inline bool decodePublishRequest(ByteView bytes, const RequestId& expected, PublishRequestView& out) +{ + out = {}; + if (!bytes.data || bytes.size > kMaxApplicationBytes) return false; + CmpReader reader(bytes); + size_t fields = 0; + uint64_t value = 0; + ByteView id, record, signature; + if (!reader.array(fields, 6) || fields != 6 || !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 0 || !reader.unsignedInteger(value) || value != 1 || + !reader.binary(id, 16) || id.size != 16 || std::memcmp(id.data, expected.bytes.data(), 16) || + !reader.unsignedInteger(value) || value < 512 || value > kMaxApplicationBytes) return false; + const auto budget = static_cast(value); + if (!reader.array(fields, 1) || fields != 1) return false; + const auto offset = reader.position(); + if (!reader.array(fields, 2) || fields != 2 || !reader.binary(record, kMaxRecordBytes) || !record.size || + !reader.binary(signature, 64) || signature.size != 64 || !reader.finished()) return false; + out.signed_cache = {bytes.data + offset, bytes.size - offset}; + out.budget = budget; + return true; +} +// record/signature must already be authenticated by the caller. This function +// validates the wire shape but does not substitute for author verification. +inline bool encodePublishRequest(const RequestId& request, ByteView record, + ByteView signature, std::uint16_t budget, + std::uint8_t* output, std::size_t capacity, + std::size_t& written) +{ + written = 0; + RecordView parsed; + if (budget < 512 || budget > kMaxApplicationBytes || !signature.data || + signature.size != 64 || !decodeGeocacheRecord(record, parsed)) return false; + CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.unsignedInteger(1) || !writer.unsignedInteger(0) || + !writer.unsignedInteger(1) || !writer.binary({request.bytes.data(), 16}) || + !writer.unsignedInteger(budget) || !writer.array(1) || !writer.array(2) || + !writer.binary(record) || !writer.binary(signature)) return false; + written = writer.size(); + return writer.good(); +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/publish_response.h b/modules/core_geocaching/include/geocaching/protocol/publish_response.h new file mode 100644 index 00000000..acc52bc4 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/publish_response.h @@ -0,0 +1,47 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" +#include + +namespace geocaching::protocol +{ +enum class PublishDisposition : std::uint8_t +{ + Stored = 0, + AlreadyPresent = 1, +}; + +inline bool decodePublishResponse(ByteView response, const RequestId& request, + const GeocacheId& id, const RevisionHash& hash, + std::uint32_t revision, CacheState state, + PublishDisposition& disposition) +{ + disposition = PublishDisposition::Stored; + if (!response.data || response.size > 512 || revision == 0 || + static_cast(state) > 2) return false; + CmpReader reader(response); + std::size_t count = 0; + std::uint64_t value = 0, result = 0; + ByteView bytes; + if (!reader.array(count, 6) || count != 6 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 1 || + !reader.binary(bytes, 16) || bytes.size != 16 || + std::memcmp(bytes.data, request.bytes.data(), 16) != 0 || + !reader.unsignedInteger(value) || value != 200 || + !reader.array(count, 7) || count != 7 || + !reader.binary(bytes, 32) || bytes.size != 32 || + std::memcmp(bytes.data, id.bytes.data(), 32) != 0 || + !reader.unsignedInteger(value) || value != revision || + !reader.binary(bytes, 32) || bytes.size != 32 || + std::memcmp(bytes.data, hash.bytes.data(), 32) != 0 || + !reader.unsignedInteger(result) || result > 1 || + !reader.unsignedInteger(value) || value != revision || + !reader.binary(bytes, 32) || bytes.size != 32 || + std::memcmp(bytes.data, hash.bytes.data(), 32) != 0 || + !reader.unsignedInteger(value) || value != static_cast(state) || + !reader.finished()) return false; + disposition = static_cast(result); + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/query_request.h b/modules/core_geocaching/include/geocaching/protocol/query_request.h new file mode 100644 index 00000000..6cbdec81 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/query_request.h @@ -0,0 +1,101 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" +#include "geocaching/protocol/cmp_writer.h" + +namespace geocaching::protocol +{ +struct QueryRegion +{ + std::int32_t south_e7 = 0; + std::int32_t west_e7 = 0; + std::int32_t north_e7 = 0; + std::int32_t east_e7 = 0; +}; + +struct QueryRequestView +{ + QueryRegion region; + ByteView author_hash, cursor; + uint16_t budget = 0; + uint8_t state_mask = 0, page_limit = 0; +}; + +inline bool decodeQueryRequest(ByteView bytes, const RequestId& expected, QueryRequestView& out) +{ + out = {}; + CmpReader reader(bytes); + size_t fields = 0; + uint64_t value = 0; + int64_t coordinate = 0; + ByteView id; + QueryRequestView request; + if (!reader.array(fields, 6) || fields != 6 || !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 0 || !reader.unsignedInteger(value) || value != 2 || + !reader.binary(id, 16) || id.size != 16 || std::memcmp(id.data, expected.bytes.data(), 16) || + !reader.unsignedInteger(value) || value < 512 || value > 8192) return false; + request.budget = static_cast(value); + if (!reader.array(fields, 5) || fields != 5 || !reader.array(fields, 4) || fields != 4) return false; + int32_t* coordinates[] = {&request.region.south_e7, &request.region.west_e7, &request.region.north_e7, &request.region.east_e7}; + for (auto* component : coordinates) + { + if (!reader.signedInteger(coordinate) || coordinate < INT32_MIN || coordinate > INT32_MAX) return false; + *component = static_cast(coordinate); + } + const auto& region = request.region; + if (region.south_e7 < -900000000 || region.north_e7 > 900000000 || region.south_e7 > region.north_e7 || + region.west_e7 < -1800000000 || region.east_e7 > 1800000000 || region.west_e7 > region.east_e7 || + !reader.unsignedInteger(value) || value < 1 || value > 7) return false; + request.state_mask = static_cast(value); + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(request.author_hash, 32) || request.author_hash.size != 32) return false; + if (!reader.unsignedInteger(value) || value == 0 || value > 64) return false; + request.page_limit = static_cast(value); + nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(request.cursor, 64) || !request.cursor.size) return false; + if (!reader.finished()) return false; + out = request; + return true; +} + +inline bool encodeCapabilitiesRequest(const RequestId& request, std::uint8_t* output, + std::size_t capacity, std::size_t& written) +{ + written = 0; + CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.unsignedInteger(1) || !writer.unsignedInteger(0) || + !writer.unsignedInteger(0) || !writer.binary({request.bytes.data(), 16}) || + !writer.unsignedInteger(8192) || !writer.array(0)) return false; + written = writer.size(); + return writer.good(); +} + +inline bool encodeQueryRequest(const RequestId& request, const QueryRegion& region, + std::uint8_t state_mask, ByteView author_hash, + std::uint8_t page_limit, ByteView cursor, + std::uint16_t budget, std::uint8_t* output, + std::size_t capacity, std::size_t& written) +{ + written = 0; + if (region.south_e7 < -900000000 || region.north_e7 > 900000000 || + region.south_e7 > region.north_e7 || region.west_e7 < -1800000000 || + region.east_e7 > 1800000000 || region.west_e7 > region.east_e7 || + state_mask == 0 || state_mask > 7 || page_limit == 0 || page_limit > 64 || + budget < 512 || budget > kMaxApplicationBytes || + (author_hash.size != 0 && (author_hash.size != 32 || !author_hash.data)) || + cursor.size > 64 || (cursor.size && !cursor.data)) return false; + CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.unsignedInteger(1) || !writer.unsignedInteger(0) || + !writer.unsignedInteger(2) || !writer.binary({request.bytes.data(), 16}) || + !writer.unsignedInteger(budget) || !writer.array(5) || !writer.array(4) || + !writer.signedInteger(region.south_e7) || !writer.signedInteger(region.west_e7) || + !writer.signedInteger(region.north_e7) || !writer.signedInteger(region.east_e7) || + !writer.unsignedInteger(state_mask) || + !(author_hash.size ? writer.binary(author_hash) : writer.nil()) || + !writer.unsignedInteger(page_limit) || + !(cursor.size ? writer.binary(cursor) : writer.nil())) return false; + written = writer.size(); + return writer.good(); +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/query_response.h b/modules/core_geocaching/include/geocaching/protocol/query_response.h new file mode 100644 index 00000000..8a8ebd2d --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/query_response.h @@ -0,0 +1,114 @@ +#pragma once +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ +struct SummaryView +{ + GeocacheId id; + RevisionHash hash; + std::uint32_t revision = 0; + CacheState state = CacheState::Active; + std::int32_t latitude_e7 = 0; + std::int32_t longitude_e7 = 0; + std::string_view name; + std::uint8_t difficulty_x2 = 0; + std::uint8_t terrain_x2 = 0; + ContainerSize container_size = ContainerSize::Unspecified; + std::uint16_t signed_bytes = 0; +}; + +struct QueryPageView +{ + ByteView snapshot_id; + ByteView next_cursor; + std::uint32_t remaining_ttl = 0; + std::size_t count = 0; + ByteView encoded_items; +}; + +// Shared summary parser for wire pages and the local summary cache. +inline bool decodeSummary(CmpReader& r, SummaryView& out) +{ + out = {}; + SummaryView item; + size_t count = 0; + uint64_t value = 0; + int64_t coordinate = 0; + ByteView bytes; + if (!r.array(count, 11) || count != 11 || !r.binary(bytes, 32) || bytes.size != 32) return false; + std::memcpy(item.id.bytes.data(), bytes.data, 32); + if (!r.unsignedInteger(value) || value == 0 || value > UINT32_MAX) return false; + item.revision = static_cast(value); + if (!r.binary(bytes, 32) || bytes.size != 32) return false; + std::memcpy(item.hash.bytes.data(), bytes.data, 32); + if (!r.unsignedInteger(value) || value > 2) return false; + item.state = static_cast(value); + if (!r.signedInteger(coordinate) || coordinate < -900000000 || coordinate > 900000000) return false; + item.latitude_e7 = static_cast(coordinate); + if (!r.signedInteger(coordinate) || coordinate < -1800000000 || coordinate >= 1800000000) return false; + item.longitude_e7 = static_cast(coordinate); + if (!r.text(item.name, kMaxNameBytes) || !validRecordText(item.name, false, true) || + !r.unsignedInteger(value) || value < 2 || value > 10) return false; + item.difficulty_x2 = static_cast(value); + if (!r.unsignedInteger(value) || value < 2 || value > 10) return false; + item.terrain_x2 = static_cast(value); + if (!r.unsignedInteger(value) || value > 5) return false; + item.container_size = static_cast(value); + if (!r.unsignedInteger(value) || value == 0 || value > 4166) return false; + item.signed_bytes = static_cast(value); + out = item; + return true; +} + +// Validate the complete response with one transient summary. Returned spans +// borrow response bytes; no summary array or payload copy is retained. +inline bool decodeQueryPage(ByteView response, const RequestId& expected, + std::size_t budget, std::size_t capacity, QueryPageView& out) +{ + out = {}; + if (!response.data || budget < 512 || budget > kMaxApplicationBytes || response.size > budget) return false; + CmpReader r(response); + std::uint64_t value = 0; + std::size_t count = 0; + ByteView bytes; + QueryPageView page; + if (!r.array(count, 6) || count != 6 || !r.unsignedInteger(value) || value != 1 || + !r.unsignedInteger(value) || value != 1 || !r.unsignedInteger(value) || value != 2 || + !r.binary(bytes, 16) || bytes.size != 16 || std::memcmp(bytes.data, expected.bytes.data(), 16) != 0 || + !r.unsignedInteger(value) || value != 200 || !r.array(count, 4) || count != 4 || + !r.binary(page.snapshot_id, 16) || page.snapshot_id.size != 16 || !r.array(page.count, 64) || + page.count > capacity) return false; + const auto start = r.position(); + GeocacheId previous; + SummaryView item; + for (std::size_t i = 0; i < page.count; ++i) + { + if (!decodeSummary(r, item) || (i && !(previous.bytes < item.id.bytes))) return false; + previous = item.id; + } + page.encoded_items = {response.data + start, r.position() - start}; + auto probe = r; + if (probe.nil()) r = probe; + else if (!r.binary(page.next_cursor, 64) || page.next_cursor.size == 0 || page.count == 0) return false; + if (!r.unsignedInteger(value) || value > 604800 || !r.finished()) return false; + page.remaining_ttl = static_cast(value); + out = page; + return true; +} +// Contiguous summary callers share the same validation and single-item parser. +inline bool decodeQueryResponse(ByteView response, const RequestId& expected, + std::size_t budget, SummaryView* items, + std::size_t capacity, QueryPageView& out) +{ + out = {}; + QueryPageView page; + if (!decodeQueryPage(response, expected, budget, capacity, page) || (page.count && !items)) return false; + CmpReader reader(page.encoded_items); + for (size_t i = 0; i < page.count; ++i) + if (!decodeSummary(reader, items[i])) return false; + out = page; + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/record_decoder.h b/modules/core_geocaching/include/geocaching/protocol/record_decoder.h new file mode 100644 index 00000000..81dd021e --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/record_decoder.h @@ -0,0 +1,97 @@ +#pragma once + +#include "geocaching/protocol/cmp_reader.h" + +namespace geocaching::protocol +{ + +inline bool validRecordText(std::string_view text, bool multiline, bool require_content) +{ + bool content = false; + std::size_t i = 0; + while (i < text.size()) + { + const auto lead = static_cast(text[i++]); + std::uint32_t cp = lead; + unsigned continuation = 0; + std::uint32_t minimum = 0; + if (lead >= 0xc2 && lead <= 0xdf) + { + cp = lead & 0x1f; + continuation = 1; + minimum = 0x80; + } + else if (lead >= 0xe0 && lead <= 0xef) + { + cp = lead & 0x0f; + continuation = 2; + minimum = 0x800; + } + else if (lead >= 0xf0 && lead <= 0xf4) + { + cp = lead & 7; + continuation = 3; + minimum = 0x10000; + } + else if (lead >= 0x80) return false; + if (continuation > text.size() - i) return false; + for (unsigned n = 0; n < continuation; ++n) + { + const auto byte = static_cast(text[i++]); + if ((byte & 0xc0) != 0x80) return false; + cp = (cp << 6) | (byte & 0x3f); + } + if (cp < minimum || cp > 0x10ffff || (cp >= 0xd800 && cp <= 0xdfff) || cp == 0xfffe || cp == 0xffff) return false; + if ((cp < 0x20 && !(multiline && (cp == 9 || cp == 10))) || (cp >= 0x7f && cp <= 0x9f)) return false; + const bool whitespace = cp == 0x20 || cp == 9 || cp == 10 || cp == 0xa0 || + cp == 0x1680 || (cp >= 0x2000 && cp <= 0x200a) || cp == 0x2028 || + cp == 0x2029 || cp == 0x202f || cp == 0x205f || cp == 0x3000; + content = content || !whitespace; + } + return !require_content || content; +} + +// Validates encoding and field semantics only. The returned views borrow input. +// Public-key/signature verification and cross-version checks are separate. +inline bool decodeGeocacheRecord(ByteView input, RecordView& out) +{ + out = {}; + if (!input.data || input.size == 0 || input.size > kMaxRecordBytes) return false; + CmpReader reader(input); + RecordView candidate; + std::size_t fields = 0; + std::uint64_t value = 0; + std::int64_t coordinate = 0; + if (!reader.array(fields, 16) || fields != 16 || !reader.unsignedInteger(value) || value != 1) return false; + if (!reader.binary(candidate.author_public_key, 64) || candidate.author_public_key.size != 64 || + !reader.binary(candidate.creation_nonce, 16) || candidate.creation_nonce.size != 16) return false; + if (!reader.unsignedInteger(value) || value == 0 || value > UINT32_MAX) return false; + candidate.revision = static_cast(value); + if (candidate.revision == 1) + { + if (!reader.nil()) return false; + } + else if (!reader.binary(candidate.previous_hash, 32) || candidate.previous_hash.size != 32) return false; + if (!reader.unsignedInteger(value) || value > 2) return false; + candidate.state = static_cast(value); + if (!reader.signedInteger(coordinate) || coordinate < -900000000 || coordinate > 900000000) return false; + candidate.latitude_e7 = static_cast(coordinate); + if (!reader.signedInteger(coordinate) || coordinate < -1800000000 || coordinate >= 1800000000) return false; + candidate.longitude_e7 = static_cast(coordinate); + if (!reader.text(candidate.name, kMaxNameBytes) || !validRecordText(candidate.name, false, true) || + !reader.text(candidate.description, kMaxDescriptionBytes) || !validRecordText(candidate.description, true, false) || + !reader.text(candidate.hint, kMaxHintBytes) || !validRecordText(candidate.hint, true, false)) return false; + if (!reader.unsignedInteger(value) || value < 2 || value > 10) return false; + candidate.difficulty_x2 = static_cast(value); + if (!reader.unsignedInteger(value) || value < 2 || value > 10) return false; + candidate.terrain_x2 = static_cast(value); + if (!reader.unsignedInteger(value) || value > 5) return false; + candidate.container_size = static_cast(value); + if (!reader.unsignedInteger(candidate.created_at) || candidate.created_at > 253402300799ULL || + !reader.unsignedInteger(candidate.updated_at) || candidate.updated_at > 253402300799ULL || !reader.finished()) return false; + candidate.encoded = input; + out = candidate; + return true; +} + +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/record_encoder.h b/modules/core_geocaching/include/geocaching/protocol/record_encoder.h new file mode 100644 index 00000000..065c06b0 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/record_encoder.h @@ -0,0 +1,28 @@ +#pragma once +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/protocol/record_decoder.h" + +namespace geocaching::protocol +{ +// Input views must not overlap output. This produces unsigned CacheRecord +// bytes only; signing and publishing are separate owner-controlled operations. +inline bool encodeGeocacheRecord(const RecordView& record, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + if ((record.revision == 1 && record.previous_hash.size != 0) || record.name.size() > kMaxNameBytes || + record.description.size() > kMaxDescriptionBytes || record.hint.size() > kMaxHintBytes) return false; + CmpWriter writer(output, capacity < kMaxRecordBytes ? capacity : kMaxRecordBytes); + if (!writer.array(16) || !writer.unsignedInteger(1) || !writer.binary(record.author_public_key) || + !writer.binary(record.creation_nonce) || !writer.unsignedInteger(record.revision) || + !(record.revision == 1 ? writer.nil() : writer.binary(record.previous_hash)) || + !writer.unsignedInteger(static_cast(record.state)) || !writer.signedInteger(record.latitude_e7) || + !writer.signedInteger(record.longitude_e7) || !writer.text(record.name) || !writer.text(record.description) || + !writer.text(record.hint) || !writer.unsignedInteger(record.difficulty_x2) || !writer.unsignedInteger(record.terrain_x2) || + !writer.unsignedInteger(static_cast(record.container_size)) || !writer.unsignedInteger(record.created_at) || + !writer.unsignedInteger(record.updated_at)) return false; + RecordView checked; + if (!decodeGeocacheRecord({output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/sign_record.h b/modules/core_geocaching/include/geocaching/protocol/sign_record.h new file mode 100644 index 00000000..29e4c12a --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/sign_record.h @@ -0,0 +1,35 @@ +#pragma once +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ +// Signer matches the existing LxmfIdentity interface. Identity owns private +// keys. Encoded input must be disjoint from output and workspace. Output may +// reuse workspace: signing consumes the domain-prefixed message before the +// writer overwrites it with SignedCache. All spans are caller-owned. +// The caller reserves/persists the author revision before publishing this +// result; this function alone does not enforce issued-revision uniqueness. +template +bool signGeocacheRecord(ByteView encoded, Signer& identity, + uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written) +{ + written = 0; + RecordView record; + if (!identity.isReady() || !decodeGeocacheRecord(encoded, record)) return false; + constexpr char domain[] = "trailmate.geocache/sign/v1"; + if (!workspace || workspace_capacity < sizeof(domain) + encoded.size || !output) return false; + std::array public_key{}, signature{}; + identity.combinedPublicKey(public_key.data()); + if (std::memcmp(public_key.data(), record.author_public_key.data, public_key.size())) return false; + std::memcpy(workspace, domain, sizeof(domain)); + std::memcpy(workspace + sizeof(domain), encoded.data, encoded.size); + if (!identity.sign(workspace, sizeof(domain) + encoded.size, signature.data())) return false; + CmpWriter writer(output, output_capacity < 4166 ? output_capacity : 4166); + if (!writer.array(2) || !writer.binary(encoded) || !writer.binary({signature.data(), signature.size()})) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/protocol/verify_record.h b/modules/core_geocaching/include/geocaching/protocol/verify_record.h new file mode 100644 index 00000000..2f19c599 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/protocol/verify_record.h @@ -0,0 +1,107 @@ +#pragma once + +#include "geocaching/protocol/record_decoder.h" +#include + +namespace geocaching::protocol +{ + +enum class VerificationResult : std::uint8_t +{ + Valid, + InvalidRecord, + InvalidSignature, + CryptoUnavailable, + WorkspaceTooSmall, + IdentityMismatch, +}; + +// Implemented with the platform's existing SHA-256 and Ed25519 facilities. +// False from sha256 means unavailable; Ed25519 distinguishes this from invalid. +class RecordCrypto +{ + public: + virtual ~RecordCrypto() = default; + virtual bool sha256(ByteView input, std::uint8_t output[32]) = 0; + virtual VerificationResult verifyEd25519(ByteView public_key, ByteView signature, + ByteView message) = 0; +}; + +struct VerifiedRecordView +{ + RecordView record; + GeocacheId id; + RevisionHash hash; + ByteView signature; +}; + +// Computes identifiers for a structurally valid record before author-version +// reservation. Valid here does not mean the record has an authenticated signature. +inline VerificationResult deriveGeocacheHashes(ByteView encoded, RecordCrypto& crypto, + uint8_t* workspace, size_t capacity, + GeocacheId& id, RevisionHash& hash) +{ + id = {}; hash = {}; + RecordView record; + if (!decodeGeocacheRecord(encoded, record)) return VerificationResult::InvalidRecord; + constexpr char id_domain[] = "trailmate.geocache/id/v1"; + constexpr char revision_domain[] = "trailmate.geocache/revision/v1"; + const auto required = sizeof(revision_domain) + encoded.size; + if (!workspace || capacity < required || capacity < sizeof(id_domain) + 80) return VerificationResult::WorkspaceTooSmall; + GeocacheId candidate_id; RevisionHash candidate_hash; + std::memcpy(workspace, id_domain, sizeof(id_domain)); + std::memcpy(workspace + sizeof(id_domain), record.author_public_key.data, 64); + std::memcpy(workspace + sizeof(id_domain) + 64, record.creation_nonce.data, 16); + if (!crypto.sha256({workspace, sizeof(id_domain) + 80}, candidate_id.bytes.data())) return VerificationResult::CryptoUnavailable; + std::memcpy(workspace, revision_domain, sizeof(revision_domain)); + std::memcpy(workspace + sizeof(revision_domain), encoded.data, encoded.size); + if (!crypto.sha256({workspace, required}, candidate_hash.bytes.data())) return VerificationResult::CryptoUnavailable; + id = candidate_id; hash = candidate_hash; + return VerificationResult::Valid; +} + +// Workspace and input must not overlap. Workspace contains public data only. +// The caller owns both; result views keep borrowing input after return. +inline VerificationResult verifyGeocache(ByteView signed_cache, RecordCrypto& crypto, + std::uint8_t* workspace, std::size_t capacity, + VerifiedRecordView& out, + const GeocacheId* expected_id = nullptr, + const RevisionHash* expected_hash = nullptr) +{ + out = {}; + if (!signed_cache.data || signed_cache.size > 4166) return VerificationResult::InvalidRecord; + CmpReader reader(signed_cache); + std::size_t count = 0; + ByteView encoded, signature; + VerifiedRecordView candidate; + if (!reader.array(count, 2) || count != 2 || + !reader.binary(encoded, kMaxRecordBytes) || + !reader.binary(signature, 64) || signature.size != 64 || !reader.finished() || + !decodeGeocacheRecord(encoded, candidate.record)) return VerificationResult::InvalidRecord; + + // sizeof includes exactly the required domain-separating NUL byte. + constexpr char sign_domain[] = "trailmate.geocache/sign/v1"; + constexpr char id_domain[] = "trailmate.geocache/id/v1"; + constexpr char revision_domain[] = "trailmate.geocache/revision/v1"; + const std::size_t required = sizeof(revision_domain) + encoded.size; + if (!workspace || capacity < required || capacity < sizeof(id_domain) + 80) + return VerificationResult::WorkspaceTooSmall; + + std::memcpy(workspace, sign_domain, sizeof(sign_domain)); + std::memcpy(workspace + sizeof(sign_domain), encoded.data, encoded.size); + const auto verified = crypto.verifyEd25519( + {candidate.record.author_public_key.data + 32, 32}, signature, + {workspace, sizeof(sign_domain) + encoded.size}); + if (verified != VerificationResult::Valid) return verified; + + const auto derived = deriveGeocacheHashes(encoded, crypto, workspace, capacity, candidate.id, candidate.hash); + if (derived != VerificationResult::Valid) return derived; + if ((expected_id && expected_id->bytes != candidate.id.bytes) || + (expected_hash && expected_hash->bytes != candidate.hash.bytes)) + return VerificationResult::IdentityMismatch; + candidate.signature = signature; + out = candidate; + return VerificationResult::Valid; +} + +} // namespace geocaching::protocol diff --git a/modules/core_geocaching/include/geocaching/storage/attempt_references.h b/modules/core_geocaching/include/geocaching/storage/attempt_references.h new file mode 100644 index 00000000..f0efcbbe --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/attempt_references.h @@ -0,0 +1,20 @@ +#pragma once +#include "geocaching/storage/tx_attempt.h" +#include "geocaching/storage/logical_state.h" +#include "geocaching/storage/outgoing_record.h" + +namespace geocaching::storage +{ +inline bool validateAttemptReferences(const LogicalState::View& state) +{ + size_t cursor = 0; MutationView entry; + while (state.next(cursor, entry)) + { + if (entry.table != 13) continue; + TxAttemptView attempt; ByteView outgoing_bytes; OutgoingView outgoing; + if (!decodeTxAttempt(entry.key, entry.value, attempt) || !state.find(5, attempt.request_key, outgoing_bytes) || + !decodeOutgoing(attempt.request_key, outgoing_bytes, outgoing)) return false; + } + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/attempt_timeout.h b/modules/core_geocaching/include/geocaching/storage/attempt_timeout.h new file mode 100644 index 00000000..59f40f86 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/attempt_timeout.h @@ -0,0 +1,25 @@ +#pragma once +#include "geocaching/storage/tx_attempt.h" + +namespace geocaching::storage +{ +// Expiry means outcome unknown and eligible for policy-controlled retry, not +// proof that the remote side did not receive it. Never reuse the old LXMF hash. +inline bool attemptTimeoutReached(const TxAttemptView& attempt, const StoredTime& now, + uint64_t recovery_started_ms, uint64_t timeout_ms) +{ + if (!timeout_ms || attempt.has_finished) return false; + if (attempt.submitted.boot_id == now.boot_id) + return now.monotonic_ms >= attempt.submitted.monotonic_ms && + now.monotonic_ms - attempt.submitted.monotonic_ms >= timeout_ms; + if (attempt.submitted.utc_trusted && attempt.submitted.has_utc && now.utc_trusted && now.has_utc && + now.utc_seconds >= attempt.submitted.utc_seconds) + { + const auto seconds = timeout_ms / 1000 + (timeout_ms % 1000 != 0); + return now.utc_seconds - attempt.submitted.utc_seconds >= seconds; + } + // No comparable previous-boot clock: grant a full grace period from this + // recovery lifecycle rather than subtracting unrelated uptime counters. + return now.monotonic_ms >= recovery_started_ms && now.monotonic_ms - recovery_started_ms >= timeout_ms; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/author_history.h b/modules/core_geocaching/include/geocaching/storage/author_history.h new file mode 100644 index 00000000..04b4340f --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/author_history.h @@ -0,0 +1,29 @@ +#pragma once +#include "geocaching/storage/author_issued.h" +#include "geocaching/storage/logical_state.h" + +namespace geocaching::storage +{ +// AuthorIssued is append-only within one volume lifecycle. Retrying the same +// issuance keeps its original record, including timestamp; changing contents or +// deleting a reservation would permit a second signed version at the same key. +inline bool validateAuthorHistory(const LogicalState::View& before, const LogicalState::View& candidate) +{ + size_t cursor = 0; MutationView entry; + while (candidate.next(cursor, entry)) + { + if (entry.table != 3) continue; + AuthorIssuedView decoded; + if (!decodeAuthorIssued(entry.key, entry.value, decoded)) return false; + } + cursor = 0; + while (before.next(cursor, entry)) + { + if (entry.table != 3) continue; + ByteView retained; + if (!candidate.find(3, entry.key, retained) || retained.size != entry.value.size || + std::memcmp(retained.data, entry.value.data, retained.size)) return false; + } + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/author_issued.h b/modules/core_geocaching/include/geocaching/storage/author_issued.h new file mode 100644 index 00000000..13f56a22 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/author_issued.h @@ -0,0 +1,38 @@ +#pragma once +#include "geocaching/storage/stored_time.h" + +namespace geocaching::storage +{ +struct AuthorIssuedView +{ + ByteView cache_id; + uint32_t revision = 0; + ByteView revision_hash; + ByteView author_public_key; + StoredTime issued_at; +}; +inline bool decodeAuthorIssued(ByteView key, ByteView value, AuthorIssuedView& out) +{ + out = {}; + if (!key.data || key.size != 36 || !value.data || value.size > 32768) return false; + AuthorIssuedView candidate; + candidate.cache_id = {key.data, 32}; + for (unsigned i = 0; i < 4; ++i) candidate.revision = (candidate.revision << 8) | key.data[32 + i]; + if (!candidate.revision) return false; + protocol::CmpReader reader(value); size_t fields = 0; + if (!reader.array(fields, 3) || fields != 3 || !reader.binary(candidate.revision_hash, 32) || candidate.revision_hash.size != 32 || + !reader.binary(candidate.author_public_key, 64) || candidate.author_public_key.size != 64 || + !decodeStoredTime(reader, candidate.issued_at) || !reader.finished()) return false; + out = candidate; return true; +} +inline bool encodeAuthorIssued(const RevisionHash& hash, ByteView public_key, const StoredTime& issued_at, + uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + if (!public_key.data || public_key.size != 64) return false; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(3) || !writer.binary({hash.bytes.data(), 32}) || !writer.binary(public_key) || + !encodeStoredTime(writer, issued_at)) return false; + written = writer.size(); return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/cache_head.h b/modules/core_geocaching/include/geocaching/storage/cache_head.h new file mode 100644 index 00000000..d9fc3adb --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/cache_head.h @@ -0,0 +1,49 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" +#include "geocaching/protocol/cmp_writer.h" + +namespace geocaching::storage +{ +struct CacheHeadView +{ + ByteView current_hash; + uint8_t conflict_state = 0; + uint64_t install_generation = 0; + uint32_t highest_seen_revision = 0; +}; + +// Table 2 structural validation. A present current_hash must also resolve to +// a verified ObjectRef for this cache before the candidate state is committed. +inline bool decodeCacheHead(ByteView key, ByteView value, CacheHeadView& out) +{ + out = {}; + if (!key.data || key.size != 32 || !value.data || value.size > 32768) return false; + protocol::CmpReader reader(value); + CacheHeadView candidate; + size_t count = 0; + uint64_t conflict = 0, highest = 0; + if (!reader.array(count, 4) || count != 4) return false; + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.current_hash, 32) || candidate.current_hash.size != 32) return false; + if (!reader.unsignedInteger(conflict) || conflict > 2 || !reader.unsignedInteger(candidate.install_generation) || + candidate.install_generation == 0 || !reader.unsignedInteger(highest) || highest > UINT32_MAX || + !reader.finished() || (candidate.current_hash.size && highest == 0)) return false; + candidate.conflict_state = static_cast(conflict); + candidate.highest_seen_revision = static_cast(highest); + out = candidate; + return true; +} +inline bool encodeCacheHead(ByteView key, const CacheHeadView& head, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(4) || !(head.current_hash.size ? writer.binary(head.current_hash) : writer.nil()) || + !writer.unsignedInteger(head.conflict_state) || !writer.unsignedInteger(head.install_generation) || + !writer.unsignedInteger(head.highest_seen_revision)) return false; + CacheHeadView checked; + if (!decodeCacheHead(key, {output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/checkpoint.h b/modules/core_geocaching/include/geocaching/storage/checkpoint.h new file mode 100644 index 00000000..68bfff5a --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/checkpoint.h @@ -0,0 +1,114 @@ +#pragma once +#include "geocaching/storage/transaction.h" + +namespace geocaching::storage +{ +inline bool checkpointKeyBefore(const MutationView& left, const MutationView& right) +{ + if (left.table != right.table) return left.table < right.table; + const auto common = left.key.size < right.key.size ? left.key.size : right.key.size; + const int comparison = common ? std::memcmp(left.key.data, right.key.data, common) : 0; + return comparison < 0 || (comparison == 0 && left.key.size < right.key.size); +} + +inline bool readCheckpointEntry(protocol::CmpReader& reader, MutationView& entry) +{ + entry = {}; + size_t fields = 0; + uint64_t table = 0; + if (!reader.array(fields, 3) || fields != 3 || !reader.unsignedInteger(table) || table < 1 || table > 13 || + !reader.binary(entry.key, 96) || !entry.key.size || !reader.binary(entry.value, 32768)) return false; + entry.table = static_cast(table); + return true; +} + +// Borrows a validated page, with no entry array. Open checks the complete page +// and key ordering before exposing any row; caller keeps its bytes immutable. +class CheckpointPageCursor +{ + public: + bool open(ByteView payload, uint64_t expected_index, const MutationView* previous = nullptr) + { + *this = {}; + if (!payload.data || payload.size > 65536 || + (previous && (!previous->key.data || !previous->key.size || previous->key.size > 96))) return false; + protocol::CmpReader reader(payload); + size_t fields = 0; + uint64_t index = 0; + if (!reader.array(fields, 2) || fields != 2 || !reader.unsignedInteger(index) || index != expected_index || + !reader.array(count_, 65536)) return false; + reader_ = reader; + MutationView last = previous ? *previous : MutationView{}; + for (size_t i = 0; i < count_; ++i) + { + MutationView entry; + if (!readCheckpointEntry(reader, entry) || ((i || previous) && !checkpointKeyBefore(last, entry))) return false; + last = entry; + } + if (!reader.finished()) return false; + remaining_ = count_; + valid_ = true; + return true; + } + size_t count() const { return valid_ ? count_ : 0; } + bool complete() const { return valid_ && !remaining_; } + bool next(MutationView& entry) + { + entry = {}; + if (!valid_ || !remaining_) return false; + if (!readCheckpointEntry(reader_, entry)) + { + valid_ = false; + return false; + } + --remaining_; + return true; + } + + private: + protocol::CmpReader reader_{{}}; + size_t count_ = 0, remaining_ = 0; + bool valid_ = false; +}; + +// Caller first validates the GCR1 page wrapper. Previous is the last entry of +// the preceding page, copied by the caller before reusing its input buffer. +inline bool decodeCheckpointPage(ByteView payload, uint64_t expected_index, + MutationView* entries, size_t capacity, size_t& count, + const MutationView* previous = nullptr) +{ + count = 0; + CheckpointPageCursor cursor; + if (!entries || !cursor.open(payload, expected_index, previous) || cursor.count() > capacity) return false; + while (count < cursor.count()) + if (!cursor.next(entries[count++])) + { + count = 0; + return false; + } + return cursor.complete(); +} + +struct CheckpointTailView +{ + uint64_t page_count = 0; + uint64_t entry_count = 0; + ByteView digest; +}; + +// Digest covers complete kind-3 GCR1 frames. Merely decoding this tail does not +// authenticate a checkpoint: the reader must compare counts and computed SHA256. +inline bool decodeCheckpointTail(ByteView payload, CheckpointTailView& out) +{ + out = {}; + if (!payload.data || payload.size > 65536) return false; + protocol::CmpReader reader(payload); + CheckpointTailView candidate; + size_t fields = 0; + if (!reader.array(fields, 3) || fields != 3 || !reader.unsignedInteger(candidate.page_count) || + !reader.unsignedInteger(candidate.entry_count) || !reader.binary(candidate.digest, 32) || + candidate.digest.size != 32 || !reader.finished()) return false; + out = candidate; + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/checkpoint_index_cursor.h b/modules/core_geocaching/include/geocaching/storage/checkpoint_index_cursor.h new file mode 100644 index 00000000..7b65b197 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/checkpoint_index_cursor.h @@ -0,0 +1,47 @@ +#pragma once +#include "geocaching/storage/checkpoint.h" +#include "geocaching/storage/transaction_index_cursor.h" + +namespace geocaching::storage +{ +// The owner must first verify checkpoint counts/digest and pin its slot against +// replacement. This cursor validates one page and derives locations, not a root. +class CheckpointIndexCursor +{ + public: + bool open(ByteView frame, uint64_t checkpoint_sequence, char slot, uint32_t offset) + { + *this = {}; + RecordFrameView decoded; + if ((slot != 'a' && slot != 'b') || !checkpoint_sequence || frame.size > UINT32_MAX - offset || + !decodeRecordFrame(frame, decoded) || decoded.kind != RecordKind::CheckpointPage || decoded.sequence != checkpoint_sequence) return false; + protocol::CmpReader header(decoded.payload); + size_t fields = 0; + uint64_t index = 0; + if (!header.array(fields, 2) || fields != 2 || !header.unsignedInteger(index) || !page_.open(decoded.payload, index)) return false; + frame_ = frame; + base_ = {checkpoint_sequence, checkpoint_sequence, offset, 0, 0, + slot == 'a' ? IndexedValueSource::CheckpointA : IndexedValueSource::CheckpointB}; + return true; + } + bool complete() const { return page_.complete(); } + bool next(IndexedMutation& out) + { + out = {}; + MutationView row; + if (!page_.next(row)) return false; + out.table = row.table; + out.key = row.key; + out.location = base_; + out.location.value_offset += base_.frame_offset + static_cast(row.value.data - frame_.data); + out.location.value_size = static_cast(row.value.size); + return true; + } + + private: + CheckpointPageCursor page_; + ByteView frame_; + JournalValueLocation base_; +}; +static_assert(sizeof(CheckpointIndexCursor) <= 128, "Checkpoint indexing must not retain a page array"); +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/checkpoint_selection.h b/modules/core_geocaching/include/geocaching/storage/checkpoint_selection.h new file mode 100644 index 00000000..8fcb06ef --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/checkpoint_selection.h @@ -0,0 +1,34 @@ +#pragma once +#include +#include + +namespace geocaching::storage +{ +enum class CheckpointCandidateState : uint8_t { Missing, Unavailable, Invalid, Verified }; +struct CheckpointCandidate +{ + // Verified may be set only after full file framing, tail and EOF checks. + CheckpointCandidateState state = CheckpointCandidateState::Unavailable; + uint64_t sequence = 0; + std::array digest{}; +}; +enum class CheckpointChoice : uint8_t { NoCheckpoint, SlotA, SlotB, RetryLater, Corrupt }; + +inline CheckpointChoice selectCheckpoint(const CheckpointCandidate& a, const CheckpointCandidate& b) +{ + using State = CheckpointCandidateState; + // An unreadable slot could contain a newer committed checkpoint. It is not + // equivalent to a missing or fully inspected invalid candidate. + if (a.state == State::Unavailable || b.state == State::Unavailable) return CheckpointChoice::RetryLater; + if (a.state == State::Verified && b.state == State::Verified) + { + if (a.sequence == b.sequence) + return a.digest == b.digest ? CheckpointChoice::SlotA : CheckpointChoice::Corrupt; + return a.sequence > b.sequence ? CheckpointChoice::SlotA : CheckpointChoice::SlotB; + } + if (a.state == State::Verified) return CheckpointChoice::SlotA; + if (b.state == State::Verified) return CheckpointChoice::SlotB; + if (a.state == State::Missing && b.state == State::Missing) return CheckpointChoice::NoCheckpoint; + return CheckpointChoice::Corrupt; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/checkpoint_verifier.h b/modules/core_geocaching/include/geocaching/storage/checkpoint_verifier.h new file mode 100644 index 00000000..f3c61dac --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/checkpoint_verifier.h @@ -0,0 +1,92 @@ +#pragma once +#include "geocaching/storage/checkpoint.h" +#include "geocaching/storage/record_frame.h" + +namespace geocaching::storage +{ +// Digest implements update(data,size) and finalize(output,size), matching the +// existing platform Sha256Digest. Use a fresh digest for each candidate file. +// Page entries are tentative until finish() succeeds at actual file EOF. +template +class CheckpointVerifier +{ + public: + explicit CheckpointVerifier(Digest& digest) : digest_(digest) {} + bool accept(ByteView encoded_frame, MutationView* entries, size_t capacity, size_t& count) + { + count = 0; + CheckpointPageCursor cursor; + if (!accept(encoded_frame, cursor)) return false; + if ((cursor.complete() || cursor.count()) && (!entries || cursor.count() > capacity)) return fail(); + while (count < cursor.count()) + if (!cursor.next(entries[count++])) + { + count = 0; + return fail(); + } + return true; + } + bool accept(ByteView encoded_frame, CheckpointPageCursor& cursor) + { + cursor = {}; + if (failed_ || tail_seen_ || complete_) return fail(); + RecordFrameView frame; + if (!decodeRecordFrame(encoded_frame, frame)) return fail(); + if (!has_sequence_) + { + sequence_ = frame.sequence; + has_sequence_ = true; + } + if (frame.sequence != sequence_) return fail(); + if (frame.kind == RecordKind::CheckpointPage) + { + MutationView prior{last_table_, {last_key_.data(), last_key_size_}, {}, false}; + if (pages_ == UINT64_MAX || !cursor.open(frame.payload, pages_, last_key_size_ ? &prior : nullptr) || + cursor.count() > UINT64_MAX - entry_count_) return fail(); + auto scan = cursor; + MutationView last; + while (scan.next(last)) + { + last_table_ = last.table; + last_key_size_ = last.key.size; + std::memcpy(last_key_.data(), last.key.data, last.key.size); + } + if (!scan.complete()) return fail(); + digest_.update(encoded_frame.data, encoded_frame.size); + ++pages_; + entry_count_ += cursor.count(); + return true; + } + if (frame.kind != RecordKind::CheckpointTail) return fail(); + CheckpointTailView tail; + if (!decodeCheckpointTail(frame.payload, tail) || tail.page_count != pages_ || tail.entry_count != entry_count_ || + !digest_.finalize(digest_bytes_.data(), digest_bytes_.size()) || + std::memcmp(tail.digest.data, digest_bytes_.data(), digest_bytes_.size())) return fail(); + tail_seen_ = true; + return true; + } + bool finish() + { + if (failed_ || !tail_seen_) return fail(); + complete_ = true; + return true; + } + bool verified() const { return complete_ && !failed_; } + uint64_t sequence() const { return verified() ? sequence_ : 0; } + const std::array& digest() const { return digest_bytes_; } + + private: + bool fail() + { + failed_ = true; + return false; + } + Digest& digest_; + uint64_t sequence_ = 0, pages_ = 0, entry_count_ = 0; + std::array last_key_{}; + std::array digest_bytes_{}; + size_t last_key_size_ = 0; + uint8_t last_table_ = 0; + bool has_sequence_ = false, tail_seen_ = false, complete_ = false, failed_ = false; +}; +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/draft_publication.h b/modules/core_geocaching/include/geocaching/storage/draft_publication.h new file mode 100644 index 00000000..defea081 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/draft_publication.h @@ -0,0 +1,77 @@ +#pragma once +#include "geocaching/protocol/publish_request.h" +#include "geocaching/protocol/publish_response.h" +#include "geocaching/storage/draft_record.h" +#include "geocaching/storage/task_record.h" +#include "geocaching/storage/transaction.h" + +namespace geocaching::storage +{ +struct DraftPublication +{ + uint32_t latest_revision = 0, confirmed_revision = 0; + bool pending = false, stopped = false, local_changes = false; + bool base_retained = false; +}; + +// Read-only projection of the verified local request ledger, not a substitute +// for signature verification at submission/recovery. Owns no payload or list. +template +DraftPublication draftPublication(const View& view, ByteView draft_id, const DraftView& draft) +{ + DraftPublication result; + if (!draft_id.data || draft_id.size != 16 || draft.author.size != 64) return result; + size_t cursor = 0; + MutationView row; + while (view.next(cursor, row)) + { + if (row.table != 5) continue; + OutgoingView outgoing; + TaskView task; + ByteView value; + if (!decodeOutgoing(row.key, row.value, outgoing) || !view.find(10, outgoing.task_id, value) || + !decodeTask(outgoing.task_id, value, task) || task.kind != 1 || + !requestBelongsToTask(outgoing.task_id, task, row.key, outgoing) || task.cache_id.size != 32 || task.revision_hash.size != 32) continue; + RequestId request; + std::memcpy(request.bytes.data(), row.key.data + 32, 16); + protocol::PublishRequestView publish; + if (!protocol::decodePublishRequest(outgoing.request, request, publish)) continue; + protocol::CmpReader reader(publish.signed_cache); + size_t fields = 0; + ByteView encoded, signature; + RecordView record; + if (!reader.array(fields, 2) || fields != 2 || !reader.binary(encoded, kMaxRecordBytes) || + !reader.binary(signature, 64) || !reader.finished() || !protocol::decodeGeocacheRecord(encoded, record) || + std::memcmp(record.author_public_key.data, draft.author.data, 64) || + std::memcmp(record.creation_nonce.data, draft_id.data, 16)) continue; + if (draft.base_hash.size == 32 && !std::memcmp(draft.base_hash.data, task.revision_hash.data, 32)) result.base_retained = true; + if (record.revision > result.latest_revision) + { + result.latest_revision = record.revision; + result.pending = result.stopped = false; + result.local_changes = !draft.has_coordinates || record.state != static_cast(draft.state) || + record.latitude_e7 != draft.latitude_e7 || record.longitude_e7 != draft.longitude_e7 || + record.name != draft.name || record.description != draft.description || record.hint != draft.hint || + record.difficulty_x2 != draft.difficulty_x2 || record.terrain_x2 != draft.terrain_x2 || + record.container_size != static_cast(draft.container_size); + } + if (outgoing.state == 4) + { + GeocacheId id; + RevisionHash hash; + std::memcpy(id.bytes.data(), task.cache_id.data, 32); + std::memcpy(hash.bytes.data(), task.revision_hash.data, 32); + protocol::PublishDisposition disposition; + if (protocol::decodePublishResponse(outgoing.terminal_data, request, id, hash, record.revision, record.state, disposition) && + record.revision > result.confirmed_revision) result.confirmed_revision = record.revision; + } + else if (record.revision == result.latest_revision) + { + const bool active = outgoing.continue_intent && task.continue_intent && task.state != 5; + result.pending = result.pending || active; + result.stopped = result.stopped || !active; + } + } + return result; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/draft_record.h b/modules/core_geocaching/include/geocaching/storage/draft_record.h new file mode 100644 index 00000000..a7579fbf --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/draft_record.h @@ -0,0 +1,104 @@ +#pragma once +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/protocol/record_decoder.h" + +namespace geocaching::storage +{ +// Borrowed editor projection: neither text nor protocol-sized arrays are owned. +struct DraftView +{ + ByteView author, base_hash; + uint64_t generation = 1; + int32_t latitude_e7 = 0, longitude_e7 = 0; + std::string_view name, description, hint; + uint8_t state = 0, difficulty_x2 = 2, terrain_x2 = 2, container_size = 0; + bool has_coordinates = false; +}; + +enum class DraftUpdateCheck : uint8_t +{ + Allowed, + NeedsRetainedPublication, + InvalidGeneration, + Conflict +}; +// Both views must have passed decodeDraft. This policy is independent of the +// storage backend; signed publication retention is resolved separately by I/O. +inline DraftUpdateCheck checkDraftUpdate(const DraftView* previous, const DraftView& next, uint64_t expected_generation) +{ + if (expected_generation == UINT64_MAX || next.generation != expected_generation + 1) + return DraftUpdateCheck::InvalidGeneration; + if (!previous) return expected_generation ? DraftUpdateCheck::Conflict : DraftUpdateCheck::Allowed; + if (previous->generation != expected_generation || + (previous->author.size && (next.author.size != previous->author.size || + std::memcmp(previous->author.data, next.author.data, previous->author.size))) || + next.base_hash.size != previous->base_hash.size || + (previous->base_hash.size && std::memcmp(next.base_hash.data, previous->base_hash.data, previous->base_hash.size))) + return DraftUpdateCheck::Conflict; + return previous->base_hash.size ? DraftUpdateCheck::NeedsRetainedPublication : DraftUpdateCheck::Allowed; +} + +inline bool decodeDraft(ByteView key, ByteView value, DraftView& out) +{ + out = {}; + if (!key.data || key.size != 16 || !value.data || value.size > 32768) return false; + protocol::CmpReader reader(value); + DraftView draft; + size_t fields = 0; + uint64_t number = 0; + int64_t coordinate = 0; + if (!reader.array(fields, 4) || fields != 4) return false; + auto optional = reader; + if (optional.nil()) reader = optional; + else if (!reader.binary(draft.author, 64) || draft.author.size != 64) return false; + optional = reader; + if (optional.nil()) reader = optional; + else if (!reader.binary(draft.base_hash, 32) || draft.base_hash.size != 32) return false; + if (!reader.array(fields, 9) || fields != 9 || !reader.unsignedInteger(number) || number > 2) return false; + draft.state = static_cast(number); + optional = reader; + if (optional.nil()) + { + reader = optional; + if (!reader.nil()) return false; + } + else + { + if (!reader.signedInteger(coordinate) || coordinate < -900000000 || coordinate > 900000000) return false; + draft.latitude_e7 = static_cast(coordinate); + if (!reader.signedInteger(coordinate) || coordinate < -1800000000 || coordinate >= 1800000000) return false; + draft.longitude_e7 = static_cast(coordinate); + draft.has_coordinates = true; + } + if (!reader.text(draft.name, kMaxNameBytes) || !protocol::validRecordText(draft.name, false, false) || + !reader.text(draft.description, kMaxDescriptionBytes) || !protocol::validRecordText(draft.description, true, false) || + !reader.text(draft.hint, kMaxHintBytes) || !protocol::validRecordText(draft.hint, true, false) || + !reader.unsignedInteger(number) || number < 2 || number > 10) return false; + draft.difficulty_x2 = static_cast(number); + if (!reader.unsignedInteger(number) || number < 2 || number > 10) return false; + draft.terrain_x2 = static_cast(number); + if (!reader.unsignedInteger(number) || number > 5) return false; + draft.container_size = static_cast(number); + if (!reader.unsignedInteger(draft.generation) || !draft.generation || !reader.finished()) return false; + out = draft; + return true; +} + +inline bool encodeDraft(ByteView key, const DraftView& draft, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(4) || !(draft.author.size ? writer.binary(draft.author) : writer.nil()) || + !(draft.base_hash.size ? writer.binary(draft.base_hash) : writer.nil()) || + !writer.array(9) || !writer.unsignedInteger(draft.state) || + !(draft.has_coordinates ? writer.signedInteger(draft.latitude_e7) && writer.signedInteger(draft.longitude_e7) + : writer.nil() && writer.nil()) || + !writer.text(draft.name) || !writer.text(draft.description) || !writer.text(draft.hint) || + !writer.unsignedInteger(draft.difficulty_x2) || !writer.unsignedInteger(draft.terrain_x2) || + !writer.unsignedInteger(draft.container_size) || !writer.unsignedInteger(draft.generation)) return false; + DraftView checked; + if (!decodeDraft(key, {output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/index_entry.h b/modules/core_geocaching/include/geocaching/storage/index_entry.h new file mode 100644 index 00000000..e78305ec --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/index_entry.h @@ -0,0 +1,72 @@ +#pragma once +#include "geocaching/storage/transaction_index_cursor.h" +#include "geocaching/storage/volume_format.h" + +namespace geocaching::storage +{ +// Disposable index metadata, not an extension of the authoritative store schema. +// One entry fits one SD slice. A transaction watermark is published separately, +// after all its entries are durable; readers ignore entries beyond that watermark. +constexpr size_t kIndexEntrySize = 152; +using IndexEntryBytes = std::array; + +inline bool validIndexEntry(const IndexedMutation& entry) +{ + const auto& value = entry.location; + const bool checkpoint = value.source != IndexedValueSource::Journal; + const uint32_t segment_limit = checkpoint ? UINT32_MAX : 1024U * 1024U; + if (static_cast(value.source) > 2 || + (checkpoint && (entry.erase || value.segment_first_sequence != value.record_sequence))) return false; + if (entry.table < 1 || entry.table > 13 || !entry.key.data || !entry.key.size || entry.key.size > 96 || + !value.segment_first_sequence || value.record_sequence < value.segment_first_sequence || + value.frame_offset > segment_limit - 24 || value.value_size > 32768) return false; + if (entry.erase) return !value.value_offset && !value.value_size; + return value.value_offset >= value.frame_offset + 24 && + uint64_t(value.value_offset) + value.value_size <= segment_limit; +} + +inline bool encodeIndexEntry(const VolumeInstance& volume, const IndexedMutation& entry, IndexEntryBytes& out) +{ + if (!validIndexEntry(entry)) return false; + const auto input = reinterpret_cast(entry.key.data), output = reinterpret_cast(out.data()); + if (input <= output ? output - input < entry.key.size : input - output < out.size()) return false; + out.fill(0); + std::memcpy(out.data(), "GCI1", 4); + std::memcpy(out.data() + 4, volume.data(), volume.size()); + out[20] = entry.table; + out[21] = static_cast(entry.key.size); + out[22] = entry.erase ? 1 : 0; + out[23] = static_cast(entry.location.source); + const auto put = [&](size_t offset, uint64_t value, unsigned bytes) + { for (unsigned i = 0; i < bytes; ++i) out[offset + i] = static_cast(value >> ((bytes - i - 1) * 8)); }; + put(24, entry.location.segment_first_sequence, 8); + put(32, entry.location.record_sequence, 8); + put(40, entry.location.frame_offset, 4); + put(44, entry.location.value_offset, 4); + put(48, entry.location.value_size, 4); + std::memcpy(out.data() + 52, entry.key.data, entry.key.size); + put(148, ::sys::crc32(out.data(), 148), 4); + return true; +} + +inline bool decodeIndexEntry(ByteView bytes, const VolumeInstance& volume, IndexedMutation& out) +{ + out = {}; + if (!bytes.data || bytes.size != kIndexEntrySize || std::memcmp(bytes.data, "GCI1", 4) || + std::memcmp(bytes.data + 4, volume.data(), volume.size()) || bytes.data[22] > 1 || bytes.data[23] > 2) return false; + const auto get = [&](size_t offset, unsigned count) + { uint64_t value = 0; for (unsigned i = 0; i < count; ++i) value = (value << 8) | bytes.data[offset + i]; return value; }; + if (get(148, 4) != ::sys::crc32(bytes.data, 148)) return false; + IndexedMutation entry; + entry.table = bytes.data[20]; + entry.key = {bytes.data + 52, bytes.data[21]}; + entry.erase = bytes.data[22] != 0; + entry.location = {get(24, 8), get(32, 8), static_cast(get(40, 4)), static_cast(get(44, 4)), static_cast(get(48, 4))}; + entry.location.source = static_cast(bytes.data[23]); + if (!validIndexEntry(entry)) return false; + for (size_t i = 52 + entry.key.size; i < 148; ++i) + if (bytes.data[i]) return false; + out = entry; + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/index_root.h b/modules/core_geocaching/include/geocaching/storage/index_root.h new file mode 100644 index 00000000..37b29e0c --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/index_root.h @@ -0,0 +1,85 @@ +#pragma once +#include "geocaching/storage/index_entry.h" + +namespace geocaching::storage +{ +constexpr size_t kIndexShardBitmapSize = 13 * 32; +using IndexRootBytes = std::array; +// Bitmap is borrowed from the caller's metadata lease, never copied into a view. +struct IndexRootView +{ + uint64_t epoch = 0, sequence = 0, revision = 0; + char slot = 'a'; + ByteView shards; +}; +inline bool validIndexRoot(const IndexRootView& root) +{ + if (!root.epoch || !root.revision || (root.slot != 'a' && root.slot != 'b') || + !root.shards.data || root.shards.size != kIndexShardBitmapSize) return false; + if (!root.sequence) + for (size_t i = 0; i < root.shards.size; ++i) + if (root.shards.data[i]) return false; + return true; +} +inline bool indexHasShard(const IndexRootView& root, uint8_t table, uint8_t bucket) +{ + return table >= 1 && table <= 13 && root.shards.data && root.shards.size == kIndexShardBitmapSize && + (root.shards.data[(table - 1) * 32 + bucket / 8] & (1u << (bucket % 8))); +} +inline bool encodeIndexRoot(const VolumeInstance& volume, const IndexRootView& root, IndexRootBytes& out) +{ + if (!validIndexRoot(root)) return false; + // Permit a new header to reuse the same bitmap lease without a second copy. + std::memmove(out.data() + 48, root.shards.data, kIndexShardBitmapSize); + std::memset(out.data(), 0, 48); + std::memcpy(out.data(), "GCM1", 4); + std::memcpy(out.data() + 4, volume.data(), volume.size()); + const auto put = [&](size_t offset, uint64_t value, unsigned count) + { for (unsigned i = 0; i < count; ++i) out[offset + i] = static_cast(value >> ((count - i - 1) * 8)); }; + put(20, root.epoch, 8); + put(28, root.sequence, 8); + put(36, root.revision, 8); + out[44] = static_cast(root.slot); + put(464, ::sys::crc32(out.data(), 464), 4); + return true; +} +inline bool decodeIndexRoot(ByteView bytes, const VolumeInstance& volume, IndexRootView& out) +{ + out = {}; + if (!bytes.data || bytes.size != 468 || std::memcmp(bytes.data, "GCM1", 4) || + std::memcmp(bytes.data + 4, volume.data(), volume.size()) || bytes.data[45] || bytes.data[46] || bytes.data[47]) return false; + const auto get = [&](size_t offset, unsigned count) + { uint64_t value = 0; for (unsigned i = 0; i < count; ++i) value = (value << 8) | bytes.data[offset + i]; return value; }; + if (get(464, 4) != ::sys::crc32(bytes.data, 464)) return false; + IndexRootView root{get(20, 8), get(28, 8), get(36, 8), static_cast(bytes.data[44]), {bytes.data + 48, kIndexShardBitmapSize}}; + if (!validIndexRoot(root)) return false; + out = root; + return true; +} +// Both copies must be validated. Missing/corrupt metadata requires rebuilding +// the derived index, not treating an unknown set of keys as absent. +inline bool selectIndexRoot(const IndexRootView& first, const IndexRootView& second, IndexRootView& out) +{ + out = {}; + if (!validIndexRoot(first) || !validIndexRoot(second)) return false; + if (first.revision == second.revision) + { + if (first.epoch != second.epoch || first.sequence != second.sequence || first.slot != second.slot || + std::memcmp(first.shards.data, second.shards.data, kIndexShardBitmapSize)) return false; + out = first; + return true; + } + const auto& newer = first.revision > second.revision ? first : second; + const auto& older = first.revision > second.revision ? second : first; + if (older.revision == UINT64_MAX || newer.revision != older.revision + 1 || newer.sequence < older.sequence) return false; + if (newer.epoch == older.epoch) + { + if (newer.slot != older.slot || older.sequence == UINT64_MAX || newer.sequence != older.sequence + 1) return false; + for (size_t i = 0; i < kIndexShardBitmapSize; ++i) + if ((newer.shards.data[i] & older.shards.data[i]) != older.shards.data[i]) return false; + } + else if (newer.slot == older.slot) return false; + out = newer; + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/index_shard_head.h b/modules/core_geocaching/include/geocaching/storage/index_shard_head.h new file mode 100644 index 00000000..2ce9f189 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/index_shard_head.h @@ -0,0 +1,64 @@ +#pragma once +#include "geocaching/storage/index_entry.h" + +namespace geocaching::storage +{ +struct IndexShardHead +{ + uint64_t epoch = 0, sequence = 0, length = 0; + uint8_t table = 0, bucket = 0; +}; +using IndexShardHeadBytes = std::array; + +inline bool validIndexShardHead(const IndexShardHead& head) +{ + return head.epoch && head.table >= 1 && head.table <= 13 && head.length % kIndexEntrySize == 0 && + ((head.sequence == 0) == (head.length == 0)); +} +inline bool encodeIndexShardHead(const VolumeInstance& volume, const IndexShardHead& head, IndexShardHeadBytes& out) +{ + if (!validIndexShardHead(head)) return false; + out.fill(0); + std::memcpy(out.data(), "GCS1", 4); + std::memcpy(out.data() + 4, volume.data(), volume.size()); + const auto put = [&](size_t offset, uint64_t value, unsigned count) + { for (unsigned i = 0; i < count; ++i) out[offset + i] = static_cast(value >> ((count - i - 1) * 8)); }; + put(20, head.epoch, 8); + put(28, head.sequence, 8); + put(36, head.length, 8); + out[44] = head.table; + out[45] = head.bucket; + put(48, ::sys::crc32(out.data(), 48), 4); + return true; +} +inline bool decodeIndexShardHead(ByteView bytes, const VolumeInstance& volume, uint64_t epoch, + uint8_t table, uint8_t bucket, IndexShardHead& out) +{ + out = {}; + if (!bytes.data || bytes.size != 52 || std::memcmp(bytes.data, "GCS1", 4) || + std::memcmp(bytes.data + 4, volume.data(), volume.size()) || bytes.data[46] || bytes.data[47]) return false; + const auto get = [&](size_t offset, unsigned count) + { uint64_t value = 0; for (unsigned i = 0; i < count; ++i) value = (value << 8) | bytes.data[offset + i]; return value; }; + if (get(48, 4) != ::sys::crc32(bytes.data, 48)) return false; + IndexShardHead head{get(20, 8), get(28, 8), get(36, 8), bytes.data[44], bytes.data[45]}; + if (!validIndexShardHead(head) || head.epoch != epoch || head.table != table || head.bucket != bucket) return false; + out = head; + return true; +} +// BOTH copies must already have decoded successfully. An invalid/missing copy +// might have held a committed update; silently falling back could return stale data. +// Initialize both copies to the same empty head before publishing a new shard. +inline bool selectIndexShardHead(const IndexShardHead& first, const IndexShardHead& second, + uint64_t visible_sequence, IndexShardHead& out) +{ + out = {}; + if (!validIndexShardHead(first) || !validIndexShardHead(second) || first.epoch != second.epoch || + first.table != second.table || first.bucket != second.bucket || + (first.sequence == second.sequence && first.length != second.length) || + (first.sequence < second.sequence && first.length >= second.length) || + (second.sequence < first.sequence && second.length >= first.length)) return false; + if (first.sequence > visible_sequence && second.sequence > visible_sequence) return false; + out = first.sequence <= visible_sequence && (second.sequence > visible_sequence || first.sequence >= second.sequence) ? first : second; + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/install_record.h b/modules/core_geocaching/include/geocaching/storage/install_record.h new file mode 100644 index 00000000..753be0af --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/install_record.h @@ -0,0 +1,83 @@ +#pragma once +#include "geocaching/storage/cache_head.h" +#include + +namespace geocaching::storage +{ +enum class InstallPhase : uint8_t +{ + Prepared, + Installed, + ConflictExternal, + RolledBack +}; +struct InstallRecordView +{ + ByteView cache_id; + ByteView revision_hash; + ByteView new_file_hash; + ByteView old_file_hash; + uint64_t generation = 0; + InstallPhase phase = InstallPhase::Prepared; +}; +inline bool decodeInstallRecord(ByteView key, ByteView value, InstallRecordView& out) +{ + out = {}; + if (!key.data || key.size != 16 || !value.data || value.size > 32768) return false; + protocol::CmpReader reader(value); + InstallRecordView candidate; + size_t fields = 0; + uint64_t phase = 0; + if (!reader.array(fields, 6) || fields != 6 || !reader.binary(candidate.cache_id, 32) || candidate.cache_id.size != 32 || + !reader.binary(candidate.revision_hash, 32) || candidate.revision_hash.size != 32 || + !reader.binary(candidate.new_file_hash, 32) || candidate.new_file_hash.size != 32) return false; + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.old_file_hash, 32) || candidate.old_file_hash.size != 32) return false; + if (!reader.unsignedInteger(candidate.generation) || candidate.generation == 0 || + !reader.unsignedInteger(phase) || phase > 3 || !reader.finished()) return false; + candidate.phase = static_cast(phase); + out = candidate; + return true; +} + +inline bool encodeInstallRecord(ByteView key, const InstallRecordView& install, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.binary(install.cache_id) || !writer.binary(install.revision_hash) || + !writer.binary(install.new_file_hash) || !(install.old_file_hash.size ? writer.binary(install.old_file_hash) : writer.nil()) || + !writer.unsignedInteger(install.generation) || !writer.unsignedInteger(static_cast(install.phase))) return false; + InstallRecordView checked; + if (!decodeInstallRecord(key, {output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} + +enum class InstallRecoveryAction : uint8_t +{ + VerifyFiles, + Superseded, + ExternalConflict, + FinishedRollback, + Inconsistent +}; + +// Classifies metadata only. VerifyFiles is not permission to rename: file hashes, +// signature mappings, and generation must still be checked at each mutation. +inline InstallRecoveryAction classifyInstallRecovery(ByteView head_key, const CacheHeadView& head, + const InstallRecordView& install) +{ + if (!head_key.data || head_key.size != 32 || !install.cache_id.data || install.cache_id.size != 32 || + std::memcmp(head_key.data, install.cache_id.data, 32) || !head.install_generation || !install.generation || + install.generation > head.install_generation) return InstallRecoveryAction::Inconsistent; + if (install.generation < head.install_generation) return InstallRecoveryAction::Superseded; + if (install.phase == InstallPhase::ConflictExternal) return InstallRecoveryAction::ExternalConflict; + if (install.phase == InstallPhase::RolledBack) return InstallRecoveryAction::FinishedRollback; + if (install.phase == InstallPhase::Installed && + (!head.current_hash.data || head.current_hash.size != 32 || !install.revision_hash.data || install.revision_hash.size != 32 || + std::memcmp(head.current_hash.data, install.revision_hash.data, 32))) return InstallRecoveryAction::Inconsistent; + if (install.phase != InstallPhase::Prepared && install.phase != InstallPhase::Installed) return InstallRecoveryAction::Inconsistent; + return InstallRecoveryAction::VerifyFiles; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/installable_record.h b/modules/core_geocaching/include/geocaching/storage/installable_record.h new file mode 100644 index 00000000..81da9039 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/installable_record.h @@ -0,0 +1,27 @@ +#pragma once +#include "geocaching/domain/version_policy.h" +#include "geocaching/storage/cache_head.h" +#include "geocaching/storage/object_ref.h" + +namespace geocaching::storage +{ +// Incoming was independently authenticated. A current object is required only +// when the cache head already points at a retained verified version. +inline bool installableRecord(const CacheHeadView& head, const ObjectRefView* object, + const protocol::VerifiedRecordView& incoming) +{ + if (incoming.record.revision < head.highest_seen_revision || head.conflict_state == 2) return false; + if (!head.current_hash.size) return true; + if (head.current_hash.size != 32 || !object || object->cache_id.size != 32 || + std::memcmp(object->cache_id.data, incoming.id.bytes.data(), 32)) return false; + protocol::VerifiedRecordView known; + known.id = incoming.id; + std::memcpy(known.hash.bytes.data(), head.current_hash.data, 32); + known.record.revision = object->revision; + known.record.state = object->state; + known.record.created_at = object->created_at; + known.record.previous_hash = object->previous_hash; + const auto relation = compareGeocacheVersions(known, incoming); + return relation == VersionRelation::Identical || relation == VersionRelation::NewerLinked; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/logical_state.h b/modules/core_geocaching/include/geocaching/storage/logical_state.h new file mode 100644 index 00000000..5766d0fa --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/logical_state.h @@ -0,0 +1,265 @@ +#pragma once +#include "geocaching/storage/checkpoint.h" +#include "geocaching/storage/transaction.h" + +namespace geocaching::storage +{ +// Internal RAM layout only, never a disk format. Caller provides two disjoint +// arenas, outside task stacks, and serializes access. Views expire on commit. +class LogicalState +{ + public: + class View + { + public: + size_t size() const { return count_; } + bool next(size_t& cursor, MutationView& entry) const + { + entry = {}; + if (cursor >= used_ || used_ - cursor < 4) return false; + const auto* p = data_ + cursor; + const size_t key_size = p[1], value_size = (size_t(p[2]) << 8) | p[3]; + if (key_size + value_size > used_ - cursor - 4) return false; + entry = {p[0], {p + 4, key_size}, {p + 4 + key_size, value_size}, false}; + cursor += 4 + key_size + value_size; + return true; + } + bool find(uint8_t table, ByteView key, ByteView& value) const + { + value = {}; + if (!key.data || !key.size) return false; + size_t cursor = 0; + MutationView entry; + while (next(cursor, entry)) + if (entry.table == table && entry.key.size == key.size && !std::memcmp(entry.key.data, key.data, key.size)) + { + value = entry.value; + return true; + } + return false; + } + + private: + friend class LogicalState; + const uint8_t* data_ = nullptr; + size_t used_ = 0, count_ = 0; + }; + + LogicalState(uint8_t* first, uint8_t* second, size_t capacity) + : active_(first), spare_(second), capacity_(capacity) {} + View view() const + { + View out; + out.data_ = active_; + out.used_ = used_; + out.count_ = count_; + return out; + } + + bool beginSnapshot() + { + if (snapshot_active_ || prepared_ || !validArenas()) return false; + snapshot_active_ = snapshot_valid_ = true; + snapshot_used_ = snapshot_count_ = 0; + snapshot_last_ = {}; + return true; + } + bool appendSnapshot(const MutationView* entries, size_t count) + { + if (!snapshot_active_ || !snapshot_valid_ || (!entries && count)) return false; + for (size_t i = 0; i < count; ++i) + { + const auto& entry = entries[i]; + if (entry.erase || entry.table < 1 || entry.table > 13 || !entry.key.data || !entry.key.size || entry.key.size > 96 || + entry.value.size > 32768 || (entry.value.size && !entry.value.data) || + (snapshot_count_ && !checkpointKeyBefore(snapshot_last_, entry))) + return snapshot_valid_ = false; + const auto start = snapshot_used_; + if (!appendEntry(entry, snapshot_used_, snapshot_count_)) return snapshot_valid_ = false; + snapshot_last_ = {entry.table, {spare_ + start + 4, entry.key.size}, {}, false}; + } + return true; + } + void discardSnapshot() { snapshot_active_ = snapshot_valid_ = false; } + template + bool commitSnapshot(Validate validate) + { + if (!snapshot_active_ || !snapshot_valid_) return false; + View candidate; + candidate.data_ = spare_; + candidate.used_ = snapshot_used_; + candidate.count_ = snapshot_count_; + if (!validate(candidate)) return false; + auto* previous = active_; + active_ = spare_; + spare_ = previous; + used_ = snapshot_used_; + count_ = snapshot_count_; + discardSnapshot(); + return true; + } + + // validate sees the entire candidate state, allowing table schemas and + // cross-references to be checked together. On failure live state is intact. + // Mutations and their bytes must not borrow the spare arena being written. + template + bool prepare(const MutationView* mutations, size_t count, Validate validate) + { + return prepareGenerated( + mutations, count, count, + [](uint8_t*, size_t, size_t&) + { return false; }, + validate); + } + + // Exactly one mutation may encode directly into its candidate value slot. + // The callback is synchronous and may only borrow live-state/input bytes. + // No generated bytes become visible unless validation and commit succeed. + template + bool prepareGenerated(const MutationView* mutations, size_t count, size_t generated_index, + Generate generate, Validate validate) + { + if (snapshot_active_ || prepared_ || !validArenas() || !mutations || !count || count > 64) return false; + if (generated_index > count || (generated_index < count && + (mutations[generated_index].erase || mutations[generated_index].value.size))) return false; + for (size_t i = 0; i < count; ++i) + { + const auto& m = mutations[i]; + if (m.table < 1 || m.table > 13 || !m.key.data || !m.key.size || m.key.size > 96 || + m.value.size > 32768 || (m.value.size && !m.value.data) || (m.erase && m.value.size)) return false; + for (size_t j = 0; j < i; ++j) + if (sameKey(mutations[j], m)) return false; + } + size_t candidate_used = 0, candidate_count = 0; + auto append = [&](const MutationView& entry) + { + if (generated_index < count && sameKey(entry, mutations[generated_index])) + return appendGeneratedEntry(entry, candidate_used, candidate_count, generate); + return appendEntry(entry, candidate_used, candidate_count); + }; + const auto live = view(); + size_t cursor = 0; + MutationView entry; + while (live.next(cursor, entry)) + { + const MutationView* replacement = &entry; + for (size_t i = 0; i < count; ++i) + if (sameKey(entry, mutations[i])) + { + replacement = &mutations[i]; + break; + } + if (!append(*replacement)) return false; + } + for (size_t i = 0; i < count; ++i) + { + ByteView prior; + if (!live.find(mutations[i].table, mutations[i].key, prior) && !append(mutations[i])) return false; + } + View candidate; + candidate.data_ = spare_; + candidate.used_ = candidate_used; + candidate.count_ = candidate_count; + if (!validate(candidate)) return false; + prepared_used_ = candidate_used; + prepared_count_ = candidate_count; + prepared_ = true; + return true; + } + + // Transient scratch only before preparing a candidate. Never retain the + // pointer across this call, perform I/O, or reenter this state owner. + template + bool withScratch(Work work) + { + if (snapshot_active_ || prepared_ || !validArenas()) return false; + work(spare_, capacity_); + return true; + } + + View preparedView() const + { + View out; + if (prepared_) + { + out.data_ = spare_; + out.used_ = prepared_used_; + out.count_ = prepared_count_; + } + return out; + } + bool commitPrepared() + { + if (!prepared_) return false; + auto* previous = active_; + active_ = spare_; + spare_ = previous; + used_ = prepared_used_; + count_ = prepared_count_; + discardPrepared(); + return true; + } + void discardPrepared() + { + prepared_ = false; + prepared_used_ = prepared_count_ = 0; + } + + // In-memory callers share the same staging implementation. Device storage + // calls prepare, then commits only after incremental durable verification. + template + bool apply(const MutationView* mutations, size_t count, Validate validate) + { + return prepare(mutations, count, validate) && commitPrepared(); + } + + private: + bool validArenas() const + { + if (!active_ || !spare_ || !capacity_) return false; + const auto a = reinterpret_cast(active_), b = reinterpret_cast(spare_); + return (a <= b ? b - a : a - b) >= capacity_; + } + template + bool appendGeneratedEntry(const MutationView& entry, size_t& used, size_t& count, Generate generate) + { + if (entry.erase) return true; + const size_t prefix = 4 + entry.key.size; + if (prefix > capacity_ - used) return false; + auto* p = spare_ + used; + const size_t available = capacity_ - used - prefix; + const size_t limit = available < 32768 ? available : 32768; + size_t size = 0; + if (!generate(p + prefix, limit, size) || size > limit) return false; + p[0] = entry.table; + p[1] = static_cast(entry.key.size); + p[2] = static_cast(size >> 8); + p[3] = static_cast(size); + std::memcpy(p + 4, entry.key.data, entry.key.size); + used += prefix + size; + ++count; + return true; + } + bool appendEntry(const MutationView& entry, size_t& used, size_t& count) + { + return appendGeneratedEntry(entry, used, count, [&](uint8_t* output, size_t capacity, size_t& written) + { + if (entry.value.size > capacity) return false; + if (entry.value.size) std::memcpy(output, entry.value.data, entry.value.size); + written = entry.value.size; + return true; }); + } + static bool sameKey(const MutationView& a, const MutationView& b) + { + return a.table == b.table && a.key.size == b.key.size && !std::memcmp(a.key.data, b.key.data, a.key.size); + } + uint8_t* active_; + uint8_t* spare_; + size_t capacity_, used_ = 0, count_ = 0; + size_t snapshot_used_ = 0, snapshot_count_ = 0; + MutationView snapshot_last_; + bool snapshot_active_ = false, snapshot_valid_ = false; + size_t prepared_used_ = 0, prepared_count_ = 0; + bool prepared_ = false; +}; +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/object_ref.h b/modules/core_geocaching/include/geocaching/storage/object_ref.h new file mode 100644 index 00000000..753bc5ef --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/object_ref.h @@ -0,0 +1,57 @@ +#pragma once +#include "geocaching/storage/stored_time.h" + +namespace geocaching::storage +{ +struct ObjectRefView +{ + ByteView cache_id, previous_hash; + uint32_t revision = 0; + CacheState state = CacheState::Active; + uint64_t created_at = 0; + StoredTime retained_until; + bool has_deadline = false; +}; +inline bool decodeObjectRef(ByteView key, ByteView value, ObjectRefView& out) +{ + out = {}; + if (!key.data || key.size != 32 || !value.data) return false; + protocol::CmpReader reader(value); + size_t count = 0; + uint64_t revision = 0, state = 0, verified = 0; + ObjectRefView object; + if (!reader.array(count, 7) || count != 7 || !reader.binary(object.cache_id, 32) || object.cache_id.size != 32 || + !reader.unsignedInteger(revision) || !revision || revision > UINT32_MAX) return false; + auto optional = reader; + if (optional.nil()) reader = optional; + else if (!reader.binary(object.previous_hash, 32) || object.previous_hash.size != 32) return false; + if ((revision == 1) != (object.previous_hash.size == 0) || !reader.unsignedInteger(state) || state > 2 || + !reader.unsignedInteger(object.created_at) || object.created_at > 253402300799ULL || + !reader.unsignedInteger(verified) || verified != 1) return false; + optional = reader; + if (optional.nil()) reader = optional; + else + { + if (!decodeStoredTime(reader, object.retained_until)) return false; + object.has_deadline = true; + } + if (!reader.finished()) return false; + object.revision = static_cast(revision); + object.state = static_cast(state); + out = object; + return true; +} +inline bool encodeObjectRef(ByteView key, const ObjectRefView& object, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(7) || !writer.binary(object.cache_id) || !writer.unsignedInteger(object.revision) || + !(object.previous_hash.size ? writer.binary(object.previous_hash) : writer.nil()) || + !writer.unsignedInteger(static_cast(object.state)) || !writer.unsignedInteger(object.created_at) || + !writer.unsignedInteger(1) || !(object.has_deadline ? encodeStoredTime(writer, object.retained_until) : writer.nil())) return false; + ObjectRefView checked; + if (!decodeObjectRef(key, {output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/outgoing_record.h b/modules/core_geocaching/include/geocaching/storage/outgoing_record.h new file mode 100644 index 00000000..4abc2644 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/outgoing_record.h @@ -0,0 +1,65 @@ +#pragma once +#include "geocaching/storage/stored_time.h" + +namespace geocaching::storage +{ +struct OutgoingView +{ + ByteView request; + uint8_t state = 0; + bool continue_intent = false; + ByteView task_id; + uint64_t install_generation = 0; + StoredTime created; + ByteView terminal_data; +}; + +inline bool decodeOutgoing(ByteView key, ByteView value, OutgoingView& out) +{ + out = {}; + if (!key.data || key.size != 48 || !value.data || value.size > 32768) return false; + OutgoingView candidate; + protocol::CmpReader reader(value); + size_t fields = 0; + uint64_t state = 0, intent = 0; + if (!reader.array(fields, 7) || fields != 7 || !reader.binary(candidate.request, 8192) || + candidate.request.size == 0 || !reader.unsignedInteger(state) || state > 5 || + !reader.unsignedInteger(intent) || intent > 1 || !reader.binary(candidate.task_id, 16) || candidate.task_id.size != 16) return false; + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.unsignedInteger(candidate.install_generation) || candidate.install_generation == 0) return false; + if (!decodeStoredTime(reader, candidate.created)) return false; + nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.terminal_data, 8192) || candidate.terminal_data.size == 0) return false; + if (!reader.finished() || (state == 4 && candidate.terminal_data.size == 0) || + (state < 4 && candidate.terminal_data.size != 0)) return false; + protocol::CmpReader request(candidate.request); + uint64_t version = 0, type = 0, operation = 0, budget = 0; + ByteView request_id; + if (!request.array(fields, 6) || fields != 6 || !request.unsignedInteger(version) || version != 1 || + !request.unsignedInteger(type) || type != 0 || !request.unsignedInteger(operation) || operation > 4 || + !request.binary(request_id, 16) || request_id.size != 16 || std::memcmp(request_id.data, key.data + 32, 16) || + !request.unsignedInteger(budget) || budget < 512 || budget > 8192 || request.finished()) return false; + candidate.state = static_cast(state); + candidate.continue_intent = intent != 0; + out = candidate; + return true; +} +// Updates reuse original request/time views. Input views must not overlap output. +inline bool encodeOutgoing(ByteView key, const OutgoingView& value, + uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + if (!value.request.data || value.request.size > 8192 || value.terminal_data.size > 8192) return false; + protocol::CmpWriter writer(output, capacity < 32768 ? capacity : 32768); + if (!writer.array(7) || !writer.binary(value.request) || !writer.unsignedInteger(value.state) || + !writer.unsignedInteger(value.continue_intent ? 1 : 0) || !writer.binary(value.task_id) || + !(value.install_generation ? writer.unsignedInteger(value.install_generation) : writer.nil()) || + !encodeStoredTime(writer, value.created) || + !(value.terminal_data.size ? writer.binary(value.terminal_data) : writer.nil())) return false; + OutgoingView checked; + if (!decodeOutgoing(key, {output, writer.size()}, checked)) return false; + written = writer.size(); return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/pending_request.h b/modules/core_geocaching/include/geocaching/storage/pending_request.h new file mode 100644 index 00000000..5b98f51e --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/pending_request.h @@ -0,0 +1,65 @@ +#pragma once +#include "geocaching/storage/logical_state.h" +#include "geocaching/storage/task_record.h" +#include "geocaching/storage/cache_head.h" + +namespace geocaching::storage +{ +enum class PendingRequestResult : uint8_t { Ready, None, Corrupt }; +struct PendingRequestView +{ + std::array key{}; + Destination destination; + RequestId request_id; + ByteView request; + ByteView task_id; +}; + +inline PendingRequestResult inspectPendingRequest(const LogicalState::View& state, const Destination& local, + ByteView key, ByteView value, PendingRequestView& out) +{ + out = {}; + OutgoingView outgoing; + if (!decodeOutgoing(key, value, outgoing)) return PendingRequestResult::Corrupt; + if (!outgoing.continue_intent || (outgoing.state != 0 && outgoing.state != 3) || + std::memcmp(key.data, local.bytes.data(), 16)) return PendingRequestResult::None; + ByteView task_bytes; TaskView task; + if (!state.find(10, outgoing.task_id, task_bytes) || !decodeTask(outgoing.task_id, task_bytes, task) || + !requestBelongsToTask(outgoing.task_id, task, key, outgoing)) return PendingRequestResult::Corrupt; + if (!task.continue_intent || task.state > 2) return PendingRequestResult::None; + if ((task.kind == 2 || task.kind == 4) && outgoing.install_generation) + { + ByteView head_bytes; CacheHeadView head; + if (!state.find(2, task.cache_id, head_bytes) || !decodeCacheHead(task.cache_id, head_bytes, head)) return PendingRequestResult::Corrupt; + if (head.install_generation != outgoing.install_generation) return PendingRequestResult::None; + } + std::memcpy(out.key.data(), key.data, 48); + std::memcpy(out.destination.bytes.data(), key.data + 16, 16); + std::memcpy(out.request_id.bytes.data(), key.data + 32, 16); + out.request = outgoing.request; out.task_id = outgoing.task_id; + return PendingRequestResult::Ready; +} + +// Result borrows the state: copy request bytes before committing any state +// changes or admitting transport work. after_key permits bounded round-robin +// traversal; when exhausted the owner may start from an empty cursor. +inline PendingRequestResult nextPendingRequest(const LogicalState::View& state, const Destination& local, + ByteView after_key, PendingRequestView& out) +{ + out = {}; + if (after_key.size && (!after_key.data || after_key.size != 48)) return PendingRequestResult::Corrupt; + bool found = false; + size_t cursor = 0; MutationView entry; + while (state.next(cursor, entry)) + { + if (entry.table != 5) continue; + PendingRequestView candidate; + const auto result = inspectPendingRequest(state, local, entry.key, entry.value, candidate); + if (result == PendingRequestResult::Corrupt) { out = {}; return result; } + if (result != PendingRequestResult::Ready || (after_key.size && std::memcmp(entry.key.data, after_key.data, 48) <= 0)) continue; + if (found && std::memcmp(entry.key.data, out.key.data(), 48) >= 0) continue; + out = candidate; found = true; + } + return found ? PendingRequestResult::Ready : PendingRequestResult::None; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/queued_request.h b/modules/core_geocaching/include/geocaching/storage/queued_request.h new file mode 100644 index 00000000..047721d9 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/queued_request.h @@ -0,0 +1,115 @@ +#pragma once +#include "geocaching/storage/stored_time.h" +#include "geocaching/storage/task_record.h" +#include "geocaching/storage/transaction.h" + +namespace geocaching::storage +{ +// Contiguous codec callers supply a bounded value buffer. The device store +// instead encodes into candidate storage and does not allocate this workspace. +struct QueuedRequestWorkspace +{ + QueuedRequestWorkspace(uint8_t* data, size_t capacity) : outgoing(data), outgoing_capacity(capacity) {} + uint8_t* outgoing; + size_t outgoing_capacity; + std::array task{}; +}; + +struct RequestTaskTarget +{ + ByteView cache_id; + ByteView revision_hash; + uint64_t install_generation = 0; +}; + +// Builds a new one-request task and Outgoing atomically. The caller has already +// validated the operation-specific request body. Later retries/pages update the +// existing task; they must not call this constructor to overwrite task history. +inline bool describeNewRequestTask( + const Destination& local, const Destination& remote, + const RequestId& request_id, const std::array& task_id, + uint8_t task_kind, ByteView request, const StoredTime& time, + std::array& key, OutgoingView& outgoing, TaskView& task, + const RequestTaskTarget& target = {}) +{ + outgoing = {}; + task = {}; + if (task_kind < 1 || task_kind > 4 || !request.data || request.size > kMaxApplicationBytes || + (time.utc_trusted && !time.has_utc) || (time.has_utc && time.utc_seconds > 253402300799ULL)) return false; + if (task_kind == 3) + { + if (target.cache_id.size || target.revision_hash.size || target.install_generation) return false; + } + else if (!target.cache_id.data || target.cache_id.size != 32 || !target.revision_hash.data || + target.revision_hash.size != 32 || (task_kind == 2 && target.install_generation == 0)) return false; + protocol::CmpReader reader(request); + size_t fields = 0; + uint64_t version = 0, type = 0, operation = 0, budget = 0; + ByteView encoded_id; + if (!reader.array(fields, 6) || fields != 6 || !reader.unsignedInteger(version) || version != 1 || + !reader.unsignedInteger(type) || type != 0 || !reader.unsignedInteger(operation) || + operation > 4 || !reader.binary(encoded_id, 16) || encoded_id.size != 16 || + std::memcmp(encoded_id.data, request_id.bytes.data(), 16) || !reader.unsignedInteger(budget) || + budget < 512 || budget > 8192 || reader.finished()) return false; + const bool compatible = operation == 0 || + (task_kind == 1 && operation == 1) || + (task_kind == 2 && operation == 3) || + (task_kind == 3 && operation == 2) || + (task_kind == 4 && (operation == 3 || operation == 4)); + if (!compatible) return false; + std::memcpy(key.data(), local.bytes.data(), 16); + std::memcpy(key.data() + 16, remote.bytes.data(), 16); + std::memcpy(key.data() + 32, request_id.bytes.data(), 16); + outgoing.request = request; + outgoing.continue_intent = true; + outgoing.task_id = {task_id.data(), task_id.size()}; + outgoing.install_generation = target.install_generation; + outgoing.created = time; + task.kind = task_kind; + task.cache_id = target.cache_id; + task.revision_hash = target.revision_hash; + task.continue_intent = true; + task.request_count = 1; + task.requests[0] = {key.data(), key.size()}; + return true; +} + +// Compatibility codec path, sharing the same record construction and encoders. +inline bool prepareNewRequestTask( + const Destination& local, const Destination& remote, + const RequestId& request_id, const std::array& task_id, + uint8_t task_kind, ByteView request, const StoredTime& time, + QueuedRequestWorkspace& scratch, + std::array& key, MutationView (&mutations)[2], + const RequestTaskTarget& target = {}) +{ + mutations[0] = {}; + mutations[1] = {}; + OutgoingView outgoing; + TaskView task; + if (!describeNewRequestTask(local, remote, request_id, task_id, task_kind, request, time, key, outgoing, task, target)) + return false; + size_t outgoing_size = 0, task_size = 0; + if (!encodeOutgoing({key.data(), key.size()}, outgoing, scratch.outgoing, scratch.outgoing_capacity, outgoing_size) || + !encodeTask(outgoing.task_id, task, scratch.task.data(), scratch.task.size(), task_size)) return false; + mutations[0] = {5, {key.data(), key.size()}, {scratch.outgoing, outgoing_size}, false}; + mutations[1] = {10, outgoing.task_id, {scratch.task.data(), task_size}, false}; + return true; +} +// Both entry points share one task construction and one transaction encoder. +inline bool encodeNewRequestTask(uint64_t previous_sequence, + const Destination& local, const Destination& remote, + const RequestId& request_id, const std::array& task_id, + uint8_t task_kind, ByteView request, const StoredTime& time, + QueuedRequestWorkspace& scratch, + uint8_t* output, size_t capacity, size_t& written, + const RequestTaskTarget& target = {}) +{ + written = 0; + std::array key{}; + MutationView mutations[2]; + return prepareNewRequestTask(local, remote, request_id, task_id, task_kind, request, time, + scratch, key, mutations, target) && + encodeTransaction(previous_sequence, mutations, 2, output, capacity, written); +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/record_frame.h b/modules/core_geocaching/include/geocaching/storage/record_frame.h new file mode 100644 index 00000000..4592b3db --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/record_frame.h @@ -0,0 +1,73 @@ +#pragma once +#include "geocaching/domain/record.h" +#include "sys/crc32.h" +#include +#include + +namespace geocaching::storage +{ +enum class RecordKind : uint8_t +{ + Transaction = 2, + CheckpointPage = 3, + CheckpointTail = 4 +}; +using RecordHeader = std::array; +struct RecordFrameView +{ + RecordKind kind = RecordKind::Transaction; + uint64_t sequence = 0; + ByteView payload; +}; + +// CRC is filled only after the caller has consumed all payload spans. +inline bool makeRecordPrefix(RecordKind kind, uint64_t sequence, size_t payload_size, RecordHeader& out) +{ + out = {}; + const auto tag = static_cast(kind); + if (tag < 2 || tag > 4 || payload_size == 0 || payload_size > 65536) return false; + out[0] = 'G'; + out[1] = 'C'; + out[2] = 'R'; + out[3] = '1'; + out[4] = tag; + out[7] = 24; + for (unsigned i = 0; i < 4; ++i) out[8 + i] = static_cast(payload_size >> ((3 - i) * 8)); + for (unsigned i = 0; i < 8; ++i) out[12 + i] = static_cast(sequence >> ((7 - i) * 8)); + return true; +} + +inline void finishRecordHeader(RecordHeader& out, uint32_t crc) +{ + for (unsigned i = 0; i < 4; ++i) out[20 + i] = static_cast(crc >> ((3 - i) * 8)); +} + +inline bool makeRecordHeader(RecordKind kind, uint64_t sequence, ByteView payload, RecordHeader& out) +{ + out = {}; + if (!payload.data || !makeRecordPrefix(kind, sequence, payload.size, out)) return false; + finishRecordHeader(out, sys::crc32(payload.data, payload.size, sys::crc32(out.data(), 20))); + return true; +} + +// One complete frame only. No allocation based on disk-controlled lengths. +// CRC is corruption detection; payload schema and sequence-chain validation +// must still succeed before a recovered transaction can be applied. +inline bool decodeRecordFrame(ByteView bytes, RecordFrameView& out) +{ + out = {}; + if (!bytes.data || bytes.size < 24 || bytes.size > 65560) return false; + const auto* h = bytes.data; + if (std::memcmp(h, "GCR1", 4) || h[4] < 2 || h[4] > 4 || h[5] || h[6] || h[7] != 24) return false; + uint32_t length = 0, stored_crc = 0; + uint64_t sequence = 0; + for (unsigned i = 0; i < 4; ++i) length = (length << 8) | h[8 + i]; + if (length == 0 || length > 65536 || bytes.size - 24 != length) return false; + for (unsigned i = 0; i < 8; ++i) sequence = (sequence << 8) | h[12 + i]; + for (unsigned i = 0; i < 4; ++i) stored_crc = (stored_crc << 8) | h[20 + i]; + const ByteView payload{h + 24, length}; + if (sys::crc32(payload.data, payload.size, sys::crc32(h, 20)) != stored_crc) return false; + out = {static_cast(h[4]), sequence, payload}; + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/record_shape.h b/modules/core_geocaching/include/geocaching/storage/record_shape.h new file mode 100644 index 00000000..dd3b378d --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/record_shape.h @@ -0,0 +1,136 @@ +#pragma once +#include "geocaching/storage/author_issued.h" +#include "geocaching/storage/cache_head.h" +#include "geocaching/storage/draft_record.h" +#include "geocaching/storage/install_record.h" +#include "geocaching/storage/object_ref.h" +#include "geocaching/storage/outgoing_record.h" +#include "geocaching/storage/summary_cache.h" +#include "geocaching/storage/task_record.h" +#include "geocaching/storage/transaction.h" +#include "geocaching/storage/tx_attempt.h" + +namespace geocaching::storage +{ +// Encoding checks only. Signatures, transport origin, immutable-author history +// and cross-row references must still be checked by the business transaction. +inline bool validStoredRowShape(const MutationView& row) +{ + constexpr uint8_t key_sizes[] = {0, 32, 32, 36, 16, 48, 48, 32, 16, 48, 16, 32, 16, 64}; + if (row.table < 1 || row.table > 13 || !row.key.data || row.key.size != key_sizes[row.table]) return false; + if (row.erase) return !row.value.size; + if (!row.value.data || row.value.size > 32768) return false; + switch (row.table) + { + case 1: + { + ObjectRefView value; + return decodeObjectRef(row.key, row.value, value); + } + case 2: + { + CacheHeadView value; + return decodeCacheHead(row.key, row.value, value); + } + case 3: + { + AuthorIssuedView value; + return decodeAuthorIssued(row.key, row.value, value); + } + case 4: + { + DraftView value; + return decodeDraft(row.key, row.value, value); + } + case 5: + { + OutgoingView value; + return decodeOutgoing(row.key, row.value, value); + } + case 9: + { + SummaryCacheView value; + return decodeSummaryCache(row.key, row.value, value); + } + case 10: + { + TaskView value; + return decodeTask(row.key, row.value, value); + } + case 12: + { + InstallRecordView value; + return decodeInstallRecord(row.key, row.value, value); + } + case 13: + { + TxAttemptView value; + return decodeTxAttempt(row.key, row.value, value); + } + default: + break; + } + protocol::CmpReader reader(row.value); + size_t count = 0; + uint64_t number = 0; + ByteView bytes; + StoredTime time; + const auto binary = [&](size_t size) + { return reader.binary(bytes, size) && bytes.size == size; }; + const auto optional_time = [&](bool deadline) + { + auto probe = reader; + if (probe.nil()) + { + reader = probe; + return true; + } + return decodeStoredTime(reader, time) && (!deadline || (time.has_utc && time.utc_trusted)); + }; + if (row.table == 6) + { + return reader.array(count, 5) && count == 5 && reader.unsignedInteger(number) && number == 1 && binary(32) && + reader.binary(bytes, 8192) && bytes.size && decodeStoredTime(reader, time) && optional_time(true) && reader.finished(); + } + if (row.table == 7) + { + if (!reader.array(count, 4) || count != 4 || !reader.unsignedInteger(number) || number > 2) return false; + auto probe = reader; + if (probe.nil()) reader = probe; + else if (!binary(32)) return false; + std::string_view note; + return reader.text(note, 2048) && protocol::validRecordText(note, true, false) && decodeStoredTime(reader, time) && reader.finished(); + } + if (row.table == 8) + { + return reader.array(count, 8) && count == 8 && binary(16) && binary(64) && binary(16) && + reader.unsignedInteger(number) && reader.unsignedInteger(number) && number <= 5 && + decodeStoredTime(reader, time) && optional_time(false) && optional_time(true) && reader.finished(); + } + if (!reader.array(count, 2) || count != 2) return false; + for (unsigned field = 0; field < 2; ++field) + { + if (!reader.array(count, 2) || (field == 0 && count == 1)) return false; + ByteView previous; + for (size_t i = 0; i < count; ++i) + { + if (!binary(32) || (i && !std::memcmp(previous.data, bytes.data, 32))) return false; + previous = bytes; + } + } + return reader.finished(); +} +inline bool validTransactionRowShapes(ByteView payload, uint64_t previous_sequence) +{ + if (!validateTransaction(payload, previous_sequence)) return false; + protocol::CmpReader reader(payload); + size_t count = 0; + if (!readTransactionHeader(reader, previous_sequence, count)) return false; + for (size_t i = 0; i < count; ++i) + { + MutationView row; + if (!readTransactionMutation(reader, row) || !validStoredRowShape(row)) return false; + } + return reader.finished(); +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/stored_time.h b/modules/core_geocaching/include/geocaching/storage/stored_time.h new file mode 100644 index 00000000..a62d426c --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/stored_time.h @@ -0,0 +1,39 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" +#include "geocaching/protocol/cmp_writer.h" + +namespace geocaching::storage +{ +struct StoredTime +{ + std::array boot_id{}; + uint64_t monotonic_ms = 0; + uint64_t utc_seconds = 0; + bool has_utc = false; + bool utc_trusted = false; +}; +inline bool encodeStoredTime(protocol::CmpWriter& writer, const StoredTime& time) +{ + if ((time.utc_trusted && !time.has_utc) || (time.has_utc && time.utc_seconds > 253402300799ULL)) return false; + return writer.array(4) && writer.binary({time.boot_id.data(), 16}) && writer.unsignedInteger(time.monotonic_ms) && + (time.has_utc ? writer.unsignedInteger(time.utc_seconds) : writer.nil()) && writer.unsignedInteger(time.utc_trusted ? 1 : 0); +} +inline bool decodeStoredTime(protocol::CmpReader& reader, StoredTime& out) +{ + out = {}; + StoredTime candidate; ByteView boot; size_t fields = 0; uint64_t trusted = 0; + if (!reader.array(fields, 4) || fields != 4 || !reader.binary(boot, 16) || boot.size != 16 || + !reader.unsignedInteger(candidate.monotonic_ms)) return false; + std::memcpy(candidate.boot_id.data(), boot.data, 16); + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else + { + if (!reader.unsignedInteger(candidate.utc_seconds) || candidate.utc_seconds > 253402300799ULL) return false; + candidate.has_utc = true; + } + if (!reader.unsignedInteger(trusted) || trusted > 1 || (trusted && !candidate.has_utc)) return false; + candidate.utc_trusted = trusted != 0; + out = candidate; return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/summary_cache.h b/modules/core_geocaching/include/geocaching/storage/summary_cache.h new file mode 100644 index 00000000..58dcd59a --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/summary_cache.h @@ -0,0 +1,38 @@ +#pragma once +#include "geocaching/protocol/query_response.h" +#include "geocaching/storage/stored_time.h" + +namespace geocaching::storage +{ +struct SummaryCacheView +{ + protocol::SummaryView summary; + ByteView snapshot_id; + StoredTime received; +}; +inline bool decodeSummaryCache(ByteView key, ByteView value, SummaryCacheView& out) +{ + out = {}; + if (!key.data || key.size != 48 || !value.data || value.size > 32768) return false; + protocol::CmpReader reader(value); size_t fields = 0; SummaryCacheView candidate; + if (!reader.array(fields, 3) || fields != 3 || !protocol::decodeSummary(reader, candidate.summary) || + std::memcmp(key.data + 16, candidate.summary.id.bytes.data(), 32) || !reader.binary(candidate.snapshot_id, 16) || + candidate.snapshot_id.size != 16 || !decodeStoredTime(reader, candidate.received) || !reader.finished()) return false; + out = candidate; return true; +} +inline bool encodeSummaryCache(ByteView key, const protocol::SummaryView& item, ByteView snapshot_id, const StoredTime& received, + uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(3) || !writer.array(11) || !writer.binary({item.id.bytes.data(), 32}) || !writer.unsignedInteger(item.revision) || + !writer.binary({item.hash.bytes.data(), 32}) || !writer.unsignedInteger(static_cast(item.state)) || + !writer.signedInteger(item.latitude_e7) || !writer.signedInteger(item.longitude_e7) || !writer.text(item.name) || + !writer.unsignedInteger(item.difficulty_x2) || !writer.unsignedInteger(item.terrain_x2) || + !writer.unsignedInteger(static_cast(item.container_size)) || !writer.unsignedInteger(item.signed_bytes) || + !writer.binary(snapshot_id) || !encodeStoredTime(writer, received)) return false; + SummaryCacheView checked; + if (!decodeSummaryCache(key, {output, writer.size()}, checked)) return false; + written = writer.size(); return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/task_record.h b/modules/core_geocaching/include/geocaching/storage/task_record.h new file mode 100644 index 00000000..45fb3667 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/task_record.h @@ -0,0 +1,78 @@ +#pragma once +#include "geocaching/storage/outgoing_record.h" + +namespace geocaching::storage +{ +struct TaskView +{ + uint8_t kind = 0; + ByteView cache_id; + ByteView revision_hash; + uint8_t state = 0; + bool continue_intent = false; + std::array requests{}; + size_t request_count = 0; +}; + +inline bool decodeTask(ByteView key, ByteView value, TaskView& out) +{ + out = {}; + if (!key.data || key.size != 16 || !value.data || value.size > 32768) return false; + TaskView candidate; + protocol::CmpReader reader(value); + size_t count = 0; + uint64_t kind = 0, state = 0, intent = 0; + if (!reader.array(count, 6) || count != 6 || !reader.unsignedInteger(kind) || kind < 1 || kind > 4) return false; + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.cache_id, 32) || candidate.cache_id.size != 32) return false; + nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.revision_hash, 32) || candidate.revision_hash.size != 32) return false; + if (!reader.unsignedInteger(state) || state > 5 || !reader.unsignedInteger(intent) || intent > 1 || + !reader.array(candidate.request_count, 3)) return false; + for (size_t i = 0; i < candidate.request_count; ++i) + { + auto& request = candidate.requests[i]; + if (!reader.binary(request, 48) || request.size != 48) return false; + for (size_t j = 0; j < i; ++j) + if (std::memcmp(candidate.requests[j].data, request.data, 48) == 0) return false; + } + if (!reader.finished()) return false; + candidate.kind = static_cast(kind); + candidate.state = static_cast(state); + candidate.continue_intent = intent != 0; + out = candidate; + return true; +} + +inline bool encodeTask(ByteView key, const TaskView& task, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + if (task.request_count > task.requests.size()) return false; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(6) || !writer.unsignedInteger(task.kind) || + !(task.cache_id.size ? writer.binary(task.cache_id) : writer.nil()) || + !(task.revision_hash.size ? writer.binary(task.revision_hash) : writer.nil()) || + !writer.unsignedInteger(task.state) || !writer.unsignedInteger(task.continue_intent ? 1 : 0) || + !writer.array(static_cast(task.request_count))) return false; + for (size_t i = 0; i < task.request_count; ++i) if (!writer.binary(task.requests[i])) return false; + TaskView checked; + if (!decodeTask(key, {output, writer.size()}, checked)) return false; + written = writer.size(); return true; +} + +// Both records must first pass their decoders. Recovery checks every task child +// and every Outgoing parent using the transaction's resulting logical tables. +inline bool requestBelongsToTask(ByteView task_key, const TaskView& task, + ByteView request_key, const OutgoingView& outgoing) +{ + if (!task_key.data || task_key.size != 16 || !request_key.data || request_key.size != 48 || + !outgoing.task_id.data || outgoing.task_id.size != 16 || task.request_count > task.requests.size() || + std::memcmp(task_key.data, outgoing.task_id.data, 16)) return false; + for (size_t i = 0; i < task.request_count; ++i) + if (task.requests[i].data && task.requests[i].size == 48 && + std::memcmp(task.requests[i].data, request_key.data, 48) == 0) return true; + return false; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/task_references.h b/modules/core_geocaching/include/geocaching/storage/task_references.h new file mode 100644 index 00000000..c563b79e --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/task_references.h @@ -0,0 +1,104 @@ +#pragma once +#include "geocaching/storage/logical_state.h" +#include "geocaching/storage/task_record.h" + +namespace geocaching::storage +{ +// Owns only reference keys. The source row may be released before the owner +// asynchronously loads each referenced row into the same read workspace. +class TaskReferenceCheck +{ + public: + bool begin(const MutationView& entry) + { + valid_ = false; + position_ = count_ = 0; + table_ = entry.table; + if (entry.erase) return false; + if (table_ == 5) + { + OutgoingView outgoing; + if (!decodeOutgoing(entry.key, entry.value, outgoing)) return false; + std::memcpy(task_.data(), outgoing.task_id.data, task_.size()); + std::memcpy(requests_[0].data(), entry.key.data, requests_[0].size()); + count_ = 1; + } + else if (table_ == 10) + { + TaskView task; + if (!decodeTask(entry.key, entry.value, task)) return false; + std::memcpy(task_.data(), entry.key.data, task_.size()); + count_ = task.request_count; + for (size_t i = 0; i < count_; ++i) + std::memcpy(requests_[i].data(), task.requests[i].data, requests_[i].size()); + } + valid_ = true; + return true; + } + + bool next(uint8_t& table, ByteView& key) const + { + if (!valid_ || position_ == count_) return false; + table = table_ == 5 ? 10 : 5; + key = table_ == 5 ? ByteView{task_.data(), task_.size()} + : ByteView{requests_[position_].data(), requests_[position_].size()}; + return true; + } + + // A missing row or I/O error must abort validation; neither is acceptance. + bool accept(ByteView value) + { + if (!valid_ || position_ == count_) return false; + bool matches = false; + if (table_ == 5) + { + TaskView task; + if (decodeTask({task_.data(), task_.size()}, value, task)) + for (size_t i = 0; i < task.request_count; ++i) + matches |= std::memcmp(task.requests[i].data, requests_[0].data(), 48) == 0; + } + else + { + OutgoingView outgoing; + matches = decodeOutgoing({requests_[position_].data(), 48}, value, outgoing) && + std::memcmp(outgoing.task_id.data, task_.data(), 16) == 0; + } + if (!matches) return valid_ = false; + ++position_; + return true; + } + + bool complete() const { return valid_ && position_ == count_; } + + private: + std::array task_{}; + std::array, 3> requests_{}; + size_t position_ = 0, count_ = 0; + uint8_t table_ = 0; + bool valid_ = false; +}; +static_assert(sizeof(TaskReferenceCheck) <= 192, "Reference validation must not retain row payloads"); + +// Checks the resulting logical state, not mutations individually: creating or +// deleting a task and its requests together is valid. Other tables require +// their own schema/reference checks before the owner commits the candidate. +inline bool validateTaskReferences(const LogicalState::View& state) +{ + size_t cursor = 0; + MutationView entry; + while (state.next(cursor, entry)) + { + TaskReferenceCheck check; + if (!check.begin(entry)) return false; + uint8_t table = 0; + ByteView key; + while (check.next(table, key)) + { + ByteView value; + if (!state.find(table, key, value) || !check.accept(value)) return false; + } + if (!check.complete()) return false; + } + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/transaction.h b/modules/core_geocaching/include/geocaching/storage/transaction.h new file mode 100644 index 00000000..3579428b --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/transaction.h @@ -0,0 +1,241 @@ +#pragma once +#include "geocaching/protocol/cmp_reader.h" +#include "geocaching/protocol/cmp_writer.h" + +namespace geocaching::storage +{ +struct MutationView +{ + uint8_t table = 0; + ByteView key; + ByteView value; + bool erase = false; +}; + +struct TransactionView +{ + uint64_t previous_sequence = 0; + const MutationView* mutations = nullptr; + size_t count = 0; +}; + +inline bool readTransactionHeader(protocol::CmpReader& reader, uint64_t expected_previous, size_t& count) +{ + size_t fields = 0; + uint64_t schema = 0, previous = 0; + count = 0; + return reader.array(fields, 3) && fields == 3 && reader.unsignedInteger(schema) && schema == 1 && + reader.unsignedInteger(previous) && previous == expected_previous && reader.array(count, 64) && count != 0; +} + +inline bool readTransactionMutation(protocol::CmpReader& reader, MutationView& out) +{ + out = {}; + MutationView mutation; + size_t fields = 0; + uint64_t table = 0; + if (!reader.array(fields, 3) || fields != 3 || !reader.unsignedInteger(table) || + table < 1 || table > 13 || !reader.binary(mutation.key, 96) || mutation.key.size == 0) return false; + mutation.table = static_cast(table); + auto nullable = reader; + if (nullable.nil()) + { + reader = nullable; + mutation.erase = true; + } + else if (!reader.binary(mutation.value, 32768)) return false; + out = mutation; + return true; +} + +// Constant auxiliary storage. Revisit only in-memory views for duplicate keys; +// binary values are skipped without copying and no filesystem reads occur. +inline bool validateTransaction(ByteView payload, uint64_t expected_previous) +{ + if (!payload.data || payload.size > 65536) return false; + protocol::CmpReader reader(payload); + size_t count = 0; + if (!readTransactionHeader(reader, expected_previous, count)) return false; + const auto first = reader; + for (size_t i = 0; i < count; ++i) + { + MutationView current; + if (!readTransactionMutation(reader, current)) return false; + auto prefix = first; + for (size_t j = 0; j < i; ++j) + { + MutationView prior; + if (!readTransactionMutation(prefix, prior)) return false; + if (prior.table == current.table && prior.key.size == current.key.size && + !std::memcmp(prior.key.data, current.key.data, current.key.size)) return false; + } + } + return reader.finished(); +} + +// The caller validates GCR1 framing/CRC before decoding and validates table +// values/references before applying any mutation. Output stays empty on failure; +// scratch is unspecified and must never be applied independently of success. +inline bool decodeTransaction(ByteView payload, uint64_t expected_previous, + MutationView* scratch, size_t capacity, TransactionView& out) +{ + out = {}; + if (!payload.data || payload.size > 65536 || !scratch) return false; + protocol::CmpReader reader(payload); + size_t count = 0; + if (!readTransactionHeader(reader, expected_previous, count) || count > capacity) return false; + for (size_t i = 0; i < count; ++i) + { + MutationView mutation; + if (!readTransactionMutation(reader, mutation)) return false; + for (size_t j = 0; j < i; ++j) + { + const auto& prior = scratch[j]; + if (prior.table == mutation.table && prior.key.size == mutation.key.size && + std::memcmp(prior.key.data, mutation.key.data, mutation.key.size) == 0) return false; + } + scratch[i] = mutation; + } + if (!reader.finished()) return false; + out = {expected_previous, scratch, count}; + return true; +} + +// Borrows descriptors and their bytes. They must remain immutable and alive +// until the caller finishes consuming the encoding, including across I/O steps. +// No file operations or full-payload scratch storage are owned here. +class TransactionEncoding +{ + public: + bool open(uint64_t previous_sequence, const MutationView* mutations, size_t count) + { + *this = {}; + if (!mutations || count == 0 || count > 64) return false; + for (size_t i = 0; i < count; ++i) + { + const auto& m = mutations[i]; + if (m.table < 1 || m.table > 13 || !m.key.data || !m.key.size || m.key.size > 96 || + m.value.size > 32768 || (m.value.size && !m.value.data) || (m.erase && m.value.size)) return false; + for (size_t j = 0; j < i; ++j) + { + const auto& prior = mutations[j]; + if (prior.table == m.table && prior.key.size == m.key.size && + !std::memcmp(prior.key.data, m.key.data, m.key.size)) return false; + } + } + previous_ = previous_sequence; + mutations_ = mutations; + count_ = count; + size_t total = 0; + if (!visit([&](ByteView part) + { + if (part.size > 65536 - total) return false; + total += part.size; + return true; })) + { + *this = {}; + return false; + } + size_ = total; + return true; + } + + size_t size() const { return size_; } + bool inputOverlaps(ByteView storage) const + { + const auto overlaps = [&](ByteView input) + { + if (!input.size || !storage.size) return false; + const auto a = reinterpret_cast(input.data), b = reinterpret_cast(storage.data); + return a <= b ? b - a < input.size : a - b < storage.size; + }; + if (overlaps({reinterpret_cast(mutations_), count_ * sizeof(MutationView)})) return true; + for (size_t i = 0; i < count_; ++i) + if (overlaps(mutations_[i].key) || overlaps(mutations_[i].value)) return true; + return false; + } + + // The sink consumes each span synchronously; header spans are temporary. + // It must not retain span pointers. Returning false stops immediately. + template + bool visit(Sink&& sink) const + { + if (!mutations_) return false; + uint8_t token[16]; + protocol::CmpWriter prefix(token, sizeof(token)); + if (!prefix.array(3) || !prefix.unsignedInteger(1) || + !prefix.unsignedInteger(previous_) || !prefix.array(static_cast(count_)) || + !sink(ByteView{token, prefix.size()})) return false; + for (size_t i = 0; i < count_; ++i) + { + const auto& m = mutations_[i]; + protocol::CmpWriter key(token, sizeof(token)); + if (!key.array(3) || !key.unsignedInteger(m.table) || !key.binaryHeader(m.key.size) || + !sink(ByteView{token, key.size()}) || !sink(m.key)) return false; + protocol::CmpWriter value(token, sizeof(token)); + if (!(m.erase ? value.nil() : value.binaryHeader(m.value.size)) || + !sink(ByteView{token, value.size()})) return false; + if (!m.erase && m.value.size && !sink(m.value)) return false; + } + return true; + } + + // A bounded copy for the storage owner's next slice. Traversal only skips + // borrowed spans in memory; it does not read their contents or perform I/O. + bool readSlice(size_t offset, uint8_t* output, size_t capacity, size_t& written) const + { + written = 0; + if (!mutations_ || offset > size_ || (!output && capacity)) return false; + if (!capacity || offset == size_) return true; + size_t skip = offset; + visit([&](ByteView part) + { + if (skip >= part.size) { skip -= part.size; return true; } + const size_t remaining = part.size - skip; + const size_t take = remaining < capacity - written ? remaining : capacity - written; + std::memcpy(output + written, part.data + skip, take); + written += take; + skip = 0; + return written < capacity; }); + const size_t available = size_ - offset; + return written == (available < capacity ? available : capacity); + } + + // Verify a readback slice without allocating a second I/O buffer. + bool matchesSlice(size_t offset, ByteView bytes) const + { + if (!mutations_ || offset > size_ || bytes.size > size_ - offset || (!bytes.data && bytes.size)) return false; + if (!bytes.size) return true; + size_t skip = offset, compared = 0; + bool equal = true; + visit([&](ByteView part) + { + if (skip >= part.size) { skip -= part.size; return true; } + const size_t available = part.size - skip; + const size_t take = available < bytes.size - compared ? available : bytes.size - compared; + equal = !std::memcmp(part.data + skip, bytes.data + compared, take); + compared += take; + skip = 0; + return equal && compared < bytes.size; }); + return equal && compared == bytes.size; + } + + private: + const MutationView* mutations_ = nullptr; + uint64_t previous_ = 0; + size_t count_ = 0; + size_t size_ = 0; +}; +static_assert(sizeof(TransactionEncoding) <= 32, "Transaction encoding must only retain borrowed metadata"); + +// Compatibility entry point: the same encoder supplies contiguous callers. +// Table-specific validation and durable commit remain the store owner's job. +inline bool encodeTransaction(uint64_t previous_sequence, const MutationView* mutations, + size_t count, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + TransactionEncoding encoding; + if (!output || !encoding.open(previous_sequence, mutations, count) || encoding.size() > capacity) return false; + return encoding.readSlice(0, output, capacity, written); +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/transaction_index_cursor.h b/modules/core_geocaching/include/geocaching/storage/transaction_index_cursor.h new file mode 100644 index 00000000..d91b0915 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/transaction_index_cursor.h @@ -0,0 +1,85 @@ +#pragma once +#include "geocaching/storage/record_frame.h" +#include "geocaching/storage/transaction.h" + +namespace geocaching::storage +{ +enum class IndexedValueSource : uint8_t +{ + Journal = 0, + CheckpointA = 1, + CheckpointB = 2 +}; +// Derived locations only; the GCR1 journal remains authoritative. An index +// consumer must bind its index to the volume and recheck the referenced frame, +// key/table and value span. Schema/reference validation must finish before +// derived entries become visible; a cursor does not commit index mutations. +struct JournalValueLocation +{ + uint64_t segment_first_sequence = 0, record_sequence = 0; + uint32_t frame_offset = 0, value_offset = 0, value_size = 0; + IndexedValueSource source = IndexedValueSource::Journal; +}; +struct IndexedMutation +{ + uint8_t table = 0; + ByteView key; + JournalValueLocation location; + bool erase = false; +}; + +// Borrows one caller-owned complete frame, never copies its values. Keys remain +// borrowed until the caller consumes each descriptor. This is a locator, not an +// SD reader: incremental frame I/O and index persistence belong to the owner. +class TransactionIndexCursor +{ + public: + bool open(ByteView frame, uint64_t expected_previous, uint64_t segment_first_sequence, uint32_t frame_offset) + { + *this = {}; + RecordFrameView decoded; + if (expected_previous == UINT64_MAX || !segment_first_sequence || frame.size > UINT32_MAX - frame_offset || + !decodeRecordFrame(frame, decoded) || decoded.kind != RecordKind::Transaction || + decoded.sequence != expected_previous + 1 || segment_first_sequence > decoded.sequence || + !validateTransaction(decoded.payload, expected_previous)) return false; + reader_ = protocol::CmpReader(decoded.payload); + if (!readTransactionHeader(reader_, expected_previous, remaining_)) return false; + frame_ = frame; + base_ = {segment_first_sequence, decoded.sequence, frame_offset, 0, 0}; + opened_ = true; + return true; + } + bool complete() const { return opened_ && !failed_ && !remaining_; } + bool next(IndexedMutation& out) + { + out = {}; + if (!remaining_) return false; + MutationView mutation; + if (!readTransactionMutation(reader_, mutation)) + { + remaining_ = 0; + failed_ = true; + return false; + } + --remaining_; + out.table = mutation.table; + out.key = mutation.key; + out.erase = mutation.erase; + out.location = base_; + if (!mutation.erase) + { + out.location.value_offset = base_.frame_offset + static_cast(mutation.value.data - frame_.data); + out.location.value_size = static_cast(mutation.value.size); + } + return true; + } + + private: + protocol::CmpReader reader_{{}}; + ByteView frame_; + JournalValueLocation base_; + size_t remaining_ = 0; + bool opened_ = false, failed_ = false; +}; +static_assert(sizeof(TransactionIndexCursor) <= 96, "Index cursors must not retain a payload or key array"); +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/tx_attempt.h b/modules/core_geocaching/include/geocaching/storage/tx_attempt.h new file mode 100644 index 00000000..19accfe4 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/tx_attempt.h @@ -0,0 +1,92 @@ +#pragma once +#include "geocaching/storage/stored_time.h" + +namespace geocaching::storage +{ +enum class TxAttemptState : uint8_t +{ + Accepted, + InFlight, + Delivered, + Failed, + CancelledBeforeSend +}; +struct TxAttemptView +{ + ByteView request_key; + ByteView attempt_id; + ByteView lxmf_hash; + TxAttemptState state = TxAttemptState::Accepted; + StoredTime submitted; + StoredTime finished; + bool has_finished = false; +}; +enum class AttemptTransition : uint8_t +{ + Rejected, + Unchanged, + Changed +}; +// Call only with a decoded attempt. Hash bytes remain borrowed until encoding. +inline AttemptTransition recordAttemptTransportHash(TxAttemptView& attempt, ByteView hash) +{ + if (!hash.data || hash.size != 32 || (attempt.lxmf_hash.size && (attempt.lxmf_hash.size != hash.size || std::memcmp(attempt.lxmf_hash.data, hash.data, hash.size)))) return AttemptTransition::Rejected; + if (attempt.state == TxAttemptState::InFlight && attempt.lxmf_hash.size == 32) return AttemptTransition::Unchanged; + if (attempt.state != TxAttemptState::Accepted) return AttemptTransition::Rejected; + attempt.lxmf_hash = hash; + attempt.state = TxAttemptState::InFlight; + return AttemptTransition::Changed; +} +inline AttemptTransition finishAttemptTransport(TxAttemptView& attempt, TxAttemptState terminal, const StoredTime& finished) +{ + if (terminal != TxAttemptState::Delivered && terminal != TxAttemptState::Failed && terminal != TxAttemptState::CancelledBeforeSend) + return AttemptTransition::Rejected; + if (attempt.has_finished) return attempt.state == terminal ? AttemptTransition::Unchanged : AttemptTransition::Rejected; + if ((terminal == TxAttemptState::Delivered && attempt.lxmf_hash.size != 32) || + (terminal == TxAttemptState::CancelledBeforeSend && (attempt.state != TxAttemptState::Accepted || attempt.lxmf_hash.size))) + return AttemptTransition::Rejected; + attempt.state = terminal; + attempt.finished = finished; + attempt.has_finished = true; + return AttemptTransition::Changed; +} +inline bool decodeTxAttempt(ByteView key, ByteView value, TxAttemptView& out) +{ + out = {}; + if (!key.data || key.size != 64 || !value.data || value.size > 32768) return false; + protocol::CmpReader reader(value); + size_t fields = 0; + uint64_t state = 0; + TxAttemptView candidate; + candidate.request_key = {key.data, 48}; + candidate.attempt_id = {key.data + 48, 16}; + if (!reader.array(fields, 4) || fields != 4) return false; + auto nullable = reader; + if (nullable.nil()) reader = nullable; + else if (!reader.binary(candidate.lxmf_hash, 32) || candidate.lxmf_hash.size != 32) return false; + if (!reader.unsignedInteger(state) || state > 4 || !decodeStoredTime(reader, candidate.submitted)) return false; + nullable = reader; + if (nullable.nil()) reader = nullable; + else + { + if (!decodeStoredTime(reader, candidate.finished)) return false; + candidate.has_finished = true; + } + if (!reader.finished() || (state >= 2) != candidate.has_finished) return false; + candidate.state = static_cast(state); + out = candidate; + return true; +} +inline bool encodeTxAttempt(ByteView key, const TxAttemptView& value, uint8_t* output, size_t capacity, size_t& written) +{ + written = 0; + protocol::CmpWriter writer(output, capacity); + if (!writer.array(4) || !(value.lxmf_hash.size ? writer.binary(value.lxmf_hash) : writer.nil()) || + !writer.unsignedInteger(static_cast(value.state)) || !encodeStoredTime(writer, value.submitted) || + !(value.has_finished ? encodeStoredTime(writer, value.finished) : writer.nil())) return false; + TxAttemptView checked; + if (!decodeTxAttempt(key, {output, writer.size()}, checked)) return false; + written = writer.size(); + return true; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/storage/volume_format.h b/modules/core_geocaching/include/geocaching/storage/volume_format.h new file mode 100644 index 00000000..c3ae7ffe --- /dev/null +++ b/modules/core_geocaching/include/geocaching/storage/volume_format.h @@ -0,0 +1,39 @@ +#pragma once +#include "geocaching/domain/record.h" +#include "sys/crc32.h" +#include +#include + +namespace geocaching::storage +{ +using VolumeInstance = std::array; +using VolumeHeader = std::array; +enum class VolumeFormatResult : uint8_t { Supported, Unsupported, Corrupt }; + +inline VolumeHeader encodeVolumeHeader(const VolumeInstance& instance) +{ + VolumeHeader header{}; + std::memcpy(header.data(), "TMGC", 4); + header[5] = 1; + header[7] = 28; + std::memcpy(header.data() + 8, instance.data(), instance.size()); + const uint32_t crc = sys::crc32(header.data(), 24); + for (unsigned i = 0; i < 4; ++i) header[24 + i] = static_cast(crc >> ((3 - i) * 8)); + return header; +} + +inline VolumeFormatResult decodeVolumeHeader(ByteView bytes, VolumeInstance& instance) +{ + instance = {}; + if (!bytes.data || bytes.size < 8 || std::memcmp(bytes.data, "TMGC", 4)) return VolumeFormatResult::Corrupt; + // Unknown versions remain read-only; their length and checksum scheme may + // differ, so do not interpret them using schema 1 or attempt repair. + if (bytes.data[4] != 0 || bytes.data[5] != 1) return VolumeFormatResult::Unsupported; + if (bytes.size != 28 || bytes.data[6] != 0 || bytes.data[7] != 28) return VolumeFormatResult::Corrupt; + uint32_t expected = 0; + for (unsigned i = 0; i < 4; ++i) expected = (expected << 8) | bytes.data[24 + i]; + if (expected != sys::crc32(bytes.data, 24)) return VolumeFormatResult::Corrupt; + std::memcpy(instance.data(), bytes.data + 8, instance.size()); + return VolumeFormatResult::Supported; +} +} // namespace geocaching::storage diff --git a/modules/core_geocaching/include/geocaching/usecase/author_issue.h b/modules/core_geocaching/include/geocaching/usecase/author_issue.h new file mode 100644 index 00000000..fa9c13c0 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/author_issue.h @@ -0,0 +1,113 @@ +#pragma once +#include "geocaching/protocol/record_decoder.h" +#include +#include + +namespace geocaching +{ +enum class AuthorReservationResult : uint8_t +{ + Pending, + Reserved, + Failed +}; +class AuthorIssuePort +{ + public: + virtual ~AuthorIssuePort() = default; + virtual bool getAuthorKey(uint8_t out[64]) = 0; + virtual AuthorReservationResult reserve(ByteView encoded, uint8_t* workspace, size_t capacity) = 0; + virtual void cancelReservation() {} + // The output may alias workspace; encoded must remain disjoint. Consume + // scratch synchronously and do not retain it after this call. + virtual bool sign(ByteView encoded, uint8_t* workspace, size_t capacity, + uint8_t* output, size_t output_capacity, size_t& written) = 0; +}; +enum class AuthorIssuePhase : uint8_t +{ + Idle, + CheckAuthor, + Reserve, + Sign, + Signed, + Failed, + Cancelled +}; + +// Small serialized job; no owned payload copies or allocation. The owner pins +// the immutable draft record until a terminal phase, and leases one buffer of +// record.size + 70 bytes after memory admission. That buffer is scratch during +// reservation/signing, then SignedCache output; keep it until the result is +// consumed. Signed is not published or network accepted. +class AuthorIssue +{ + public: + explicit AuthorIssue(AuthorIssuePort& port) : port_(port) {} + AuthorIssue(const AuthorIssue&) = delete; + AuthorIssue& operator=(const AuthorIssue&) = delete; + bool begin(ByteView record, uint8_t* workspace, size_t capacity) + { + if (phase_ != AuthorIssuePhase::Idle || !record.data || record.size > kMaxRecordBytes || + !workspace || capacity < record.size + 70) return false; + // A borrowed draft cannot share bytes with the mutable operation lease. + const auto input = reinterpret_cast(record.data); + const auto scratch = reinterpret_cast(workspace); + if (input <= scratch ? scratch - input < record.size : input - scratch < capacity) return false; + RecordView decoded; + if (!protocol::decodeGeocacheRecord(record, decoded)) return false; + encoded_ = record; + author_ = decoded.author_public_key; + workspace_ = workspace; + capacity_ = capacity; + phase_ = AuthorIssuePhase::CheckAuthor; + return true; + } + AuthorIssuePhase phase() const { return phase_; } + ByteView signedRecord() const { return phase_ == AuthorIssuePhase::Signed ? ByteView{workspace_, signed_size_} : ByteView{}; } + void cancel() + { + if (phase_ == AuthorIssuePhase::Reserve) port_.cancelReservation(); + if (phase_ != AuthorIssuePhase::Signed && phase_ != AuthorIssuePhase::Failed) phase_ = AuthorIssuePhase::Cancelled; + } + void advance() + { + const ByteView record = encoded_; + if (phase_ == AuthorIssuePhase::CheckAuthor) + { + std::array current{}; + phase_ = port_.getAuthorKey(current.data()) && !std::memcmp(current.data(), author_.data, current.size()) ? AuthorIssuePhase::Reserve : AuthorIssuePhase::Failed; + } + else if (phase_ == AuthorIssuePhase::Reserve) + { + const auto result = port_.reserve(record, workspace_, capacity_); + if (result == AuthorReservationResult::Reserved) phase_ = AuthorIssuePhase::Sign; + else if (result == AuthorReservationResult::Failed) phase_ = AuthorIssuePhase::Failed; + } + else if (phase_ == AuthorIssuePhase::Sign) + { + signed_size_ = 0; + if (!port_.sign(record, workspace_, capacity_, workspace_, capacity_, signed_size_) || + signed_size_ > capacity_) + { + phase_ = AuthorIssuePhase::Failed; + return; + } + protocol::CmpReader reader({workspace_, signed_size_}); + size_t count = 0; + ByteView preserved, signature; + const bool valid = reader.array(count, 2) && count == 2 && reader.binary(preserved, kMaxRecordBytes) && + preserved.size == encoded_.size && !std::memcmp(preserved.data, encoded_.data, encoded_.size) && + reader.binary(signature, 64) && signature.size == 64 && reader.finished(); + phase_ = valid ? AuthorIssuePhase::Signed : AuthorIssuePhase::Failed; + } + } + + private: + AuthorIssuePort& port_; + ByteView encoded_, author_; + uint8_t* workspace_ = nullptr; + size_t capacity_ = 0, signed_size_ = 0; + AuthorIssuePhase phase_ = AuthorIssuePhase::Idle; +}; +static_assert(sizeof(AuthorIssue) <= 128, "Author signing jobs retain metadata, not record copies"); +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/directory_pool.h b/modules/core_geocaching/include/geocaching/usecase/directory_pool.h new file mode 100644 index 00000000..98b1a3d5 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/directory_pool.h @@ -0,0 +1,84 @@ +#pragma once +#include "geocaching/protocol/discovery.h" +#include + +namespace geocaching +{ +struct DirectoryEntry +{ + Destination discovery; + Destination delivery; + std::array public_key{}; + std::array epoch{}; + std::array name{}; + std::uint64_t sequence = 0; + std::uint64_t last_seen = 0; + std::uint64_t retry_after = 0; + bool capabilities_verified = false; + std::uint8_t max_query_items = 0; + bool in_flight = false; +}; + +class DirectoryPool +{ + public: + static constexpr std::size_t kCapacity = 32; + DirectoryPool(DirectoryEntry* entries, size_t capacity) + : entries_(entries), capacity_(entries ? (capacity < kCapacity ? capacity : kCapacity) : 0) {} + // Caller has authenticated the announce and derived both destination hashes. + bool observe(const Destination& discovery, const protocol::DirectoryAnnouncement& announce, + ByteView public_key, std::uint64_t now) + { + if (!public_key.data || public_key.size != 64 || announce.name.size() > 40) return false; + DirectoryEntry* entry = nullptr; + for (std::size_t i = 0; i < count_; ++i) + if (entries_[i].discovery.bytes == discovery.bytes) + { + entry = &entries_[i]; + break; + } + if (entry) + { + if (entry->delivery.bytes != announce.delivery.bytes || + std::memcmp(entry->public_key.data(), public_key.data, 64) != 0) return false; + } + else + { + if (count_ == capacity_) return false; + entry = &entries_[count_++]; + entry->discovery = discovery; + entry->delivery = announce.delivery; + std::memcpy(entry->public_key.data(), public_key.data, 64); + } + if (entry->epoch != announce.epoch) + { + entry->epoch = announce.epoch; + entry->sequence = announce.sequence; + } + else if (announce.sequence > entry->sequence) entry->sequence = announce.sequence; + entry->last_seen = now; + entry->name.fill(0); + if (!announce.name.empty()) std::memcpy(entry->name.data(), announce.name.data(), announce.name.size()); + return true; + } + + DirectoryEntry* next(std::uint64_t now, bool require_capabilities) + { + for (std::size_t checked = 0; checked < count_; ++checked) + { + const auto index = cursor_++ % count_; + auto& entry = entries_[index]; + if (!entry.in_flight && now >= entry.retry_after && + entry.capabilities_verified == require_capabilities) return &entry; + } + return nullptr; + } + std::size_t size() const { return count_; } + + private: + DirectoryEntry* entries_; + size_t capacity_; + std::size_t count_ = 0; + std::size_t cursor_ = 0; +}; +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/download_client.h b/modules/core_geocaching/include/geocaching/usecase/download_client.h new file mode 100644 index 00000000..a43f16ef --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/download_client.h @@ -0,0 +1,145 @@ +#pragma once +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/protocol/get_response.h" +#include "geocaching/protocol/query_response.h" +#include "geocaching/protocol/verify_record.h" + +namespace geocaching +{ +enum class DownloadOperationResult : uint8_t +{ + Rejected, + Pending, + Complete +}; +class DownloadPort +{ + public: + virtual ~DownloadPort() = default; + virtual DownloadOperationResult submit(const Destination&, const RequestId&, ByteView request) = 0; + // Atomically install verified GPX and the response result, checking the + // caller's installation generation. False must never be shown as downloaded. + // Pending takes responsibility for stable input ownership until completion. + // Views are borrowed only during the call; stage into owner-held storage. + virtual DownloadOperationResult commit(const Destination&, const RequestId&, std::uint64_t generation, + ByteView response, const protocol::VerifiedRecordView&) = 0; + virtual DownloadOperationResult poll() = 0; + virtual DownloadOperationResult cancel(const Destination&, const RequestId&, uint64_t generation) = 0; +}; +enum class DownloadPhase : std::uint8_t +{ + Idle, + Waiting, + Stored, + Cancelled, + Submitting, + Installing, + Cancelling, + Failed +}; + +class DownloadClient +{ + public: + DownloadClient(DownloadPort& port, protocol::RecordCrypto& crypto) : port_(port), crypto_(crypto) {} + // Owner has restored the matching durable request and prepared its port. + // Preserve all preview checks, without submitting a second task. + bool resume(const Destination& source, const RequestId& request, + const protocol::SummaryView& summary, std::uint64_t generation) + { + if (phase_ != DownloadPhase::Idle || generation == 0 || summary.name.size() > kMaxNameBytes || + !protocol::validRecordText(summary.name, false, true)) return false; + expected_ = summary; + std::memcpy(name_.data(), summary.name.data(), summary.name.size()); + expected_.name = {name_.data(), summary.name.size()}; + source_ = source; + request_ = request; + generation_ = generation; + phase_ = DownloadPhase::Waiting; + return true; + } + bool begin(const Destination& source, const RequestId& request, + const protocol::SummaryView& summary, std::uint64_t generation) + { + if (phase_ != DownloadPhase::Idle || generation == 0 || summary.name.size() > kMaxNameBytes || + !protocol::validRecordText(summary.name, false, true)) return false; + std::size_t size = 0; + if (!protocol::encodeGetRequest(request, summary.id, &summary.hash, nullptr, 8192, + request_bytes_.data(), request_bytes_.size(), size)) return false; + expected_ = summary; + name_.fill(0); + std::memcpy(name_.data(), summary.name.data(), summary.name.size()); + expected_.name = {name_.data(), summary.name.size()}; + source_ = source; + request_ = request; + generation_ = generation; + const auto result = port_.submit(source_, request_, {request_bytes_.data(), size}); + if (result == DownloadOperationResult::Rejected) return false; + phase_ = result == DownloadOperationResult::Pending ? DownloadPhase::Submitting : DownloadPhase::Waiting; + return true; + } + // Authenticated transport event only; local destination checked by owner. + bool accept(const Destination& source, ByteView response, uint8_t* scratch, size_t capacity) + { + if (phase_ != DownloadPhase::Waiting || source.bytes != source_.bytes) return false; + if (!scratch) return false; + const auto input = reinterpret_cast(response.data), output = reinterpret_cast(scratch); + if (input <= output ? output - input < response.size : input - output < capacity) return false; + protocol::GetResponseView parsed; + protocol::VerifiedRecordView verified; + if (!protocol::decodeGetResponse(response, request_, 8192, parsed) || parsed.has_conflict || + protocol::verifyGeocache(parsed.signed_cache, crypto_, scratch, capacity, verified, + &expected_.id, &expected_.hash) != protocol::VerificationResult::Valid) return false; + const auto& r = verified.record; + if (r.revision != expected_.revision || r.state != expected_.state || + r.latitude_e7 != expected_.latitude_e7 || r.longitude_e7 != expected_.longitude_e7 || + r.name != expected_.name || r.difficulty_x2 != expected_.difficulty_x2 || + r.terrain_x2 != expected_.terrain_x2 || r.container_size != expected_.container_size || + parsed.signed_cache.size != expected_.signed_bytes) return false; + const auto result = port_.commit(source_, request_, generation_, response, verified); + if (result == DownloadOperationResult::Rejected) return false; + phase_ = result == DownloadOperationResult::Pending ? DownloadPhase::Installing : DownloadPhase::Stored; + return result == DownloadOperationResult::Complete; + } + void advance() + { + if (phase_ != DownloadPhase::Submitting && phase_ != DownloadPhase::Installing && phase_ != DownloadPhase::Cancelling) return; + const auto result = port_.poll(); + if (result == DownloadOperationResult::Pending) return; + if (result == DownloadOperationResult::Rejected) + { + phase_ = DownloadPhase::Failed; + return; + } + phase_ = phase_ == DownloadPhase::Submitting ? DownloadPhase::Waiting : phase_ == DownloadPhase::Installing ? DownloadPhase::Stored + : DownloadPhase::Cancelled; + } + bool cancel() + { + if (phase_ == DownloadPhase::Stored) return false; + if (phase_ == DownloadPhase::Cancelled || phase_ == DownloadPhase::Cancelling) return true; + if (phase_ == DownloadPhase::Idle) + { + phase_ = DownloadPhase::Cancelled; + return true; + } + const auto result = port_.cancel(source_, request_, generation_); + if (result == DownloadOperationResult::Rejected) return false; + phase_ = result == DownloadOperationResult::Pending ? DownloadPhase::Cancelling : DownloadPhase::Cancelled; + return true; + } + DownloadPhase phase() const { return phase_; } + + private: + DownloadPort& port_; + protocol::RecordCrypto& crypto_; + Destination source_; + RequestId request_; + protocol::SummaryView expected_; + std::array name_{}; + std::array request_bytes_{}; + std::uint64_t generation_ = 0; + DownloadPhase phase_ = DownloadPhase::Idle; +}; +static_assert(sizeof(DownloadClient) <= 512, "Download clients must not own a full-record crypto workspace"); +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/gpx_install.h b/modules/core_geocaching/include/geocaching/usecase/gpx_install.h new file mode 100644 index 00000000..b2fb3ac5 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/gpx_install.h @@ -0,0 +1,81 @@ +#pragma once +#include "geocaching/domain/record.h" + +namespace geocaching +{ +enum class InstallStep : std::uint8_t +{ + CheckIntent, + ValidateStagedFile, + PrepareJournal, + ReplaceFile, + ValidateInstalledFile, + CommitJournal, + Complete, + Failed, + Cancelled, +}; + +enum class InstallEffectResult : std::uint8_t +{ + Complete, + Pending, + Failed, + StaleIntent, +}; + +struct InstallIdentity +{ + GeocacheId id; + RevisionHash hash; + std::uint64_t generation = 0; +}; + +class GpxInstallPort +{ + public: + virtual ~GpxInstallPort() = default; + // Called repeatedly with the same step while Pending. Implementations must + // coalesce rather than start duplicate work. Check generation under the + // store owner for every mutation, not just CheckIntent. Failed replacement + // must retain the prepared journal/backup for recovery, never delete it. + virtual InstallEffectResult execute(InstallStep step, const InstallIdentity& identity) = 0; +}; + +// One bounded maintenance step per invocation. Staged GPX generation is done +// before starting; the store verifies it against the expected authenticated hash. +class GpxInstall +{ + public: + explicit GpxInstall(InstallIdentity identity) : identity_(identity) {} + InstallStep step() const { return step_; } + bool installed() const { return step_ == InstallStep::Complete; } + void advance(GpxInstallPort& port) + { + if (step_ >= InstallStep::Complete) return; + if (identity_.generation == 0) + { + step_ = InstallStep::Failed; + return; + } + switch (port.execute(step_, identity_)) + { + case InstallEffectResult::Pending: + return; + case InstallEffectResult::Failed: + step_ = InstallStep::Failed; + return; + case InstallEffectResult::StaleIntent: + step_ = InstallStep::Cancelled; + return; + case InstallEffectResult::Complete: + step_ = static_cast(static_cast(step_) + 1); + return; + } + } + + private: + InstallIdentity identity_; + InstallStep step_ = InstallStep::CheckIntent; +}; +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/publish_attempt.h b/modules/core_geocaching/include/geocaching/usecase/publish_attempt.h new file mode 100644 index 00000000..7ad07bad --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/publish_attempt.h @@ -0,0 +1,167 @@ +#pragma once +#include "geocaching/protocol/publish_request.h" +#include "geocaching/protocol/publish_response.h" +#include "geocaching/protocol/verify_record.h" + +namespace geocaching +{ +enum class PublishPersistence : uint8_t +{ + Rejected, + Pending, + Committed +}; +class PublishAttemptPort +{ + public: + virtual ~PublishAttemptPort() = default; + // Persist exact bytes and task linkage before admitting transport work. + // Input is borrowed for this call only; Pending owns a stable copy. + virtual PublishPersistence submit(const Destination&, const RequestId&, ByteView request) = 0; + virtual PublishPersistence commitResult(const Destination&, const RequestId&, ByteView response) = 0; + virtual PublishPersistence cancel(const Destination&, const RequestId&) = 0; + virtual PublishPersistence poll() = 0; +}; +enum class PublishAttemptPhase : uint8_t +{ + Idle, + Waiting, + Confirmed, + Cancelled, + Submitting, + Committing, + Cancelling, + Failed +}; + +// One destination of a public publication task; not the multi-directory quorum +// itself. A caller-owned transient lease is reused for verification and request +// encoding, and may be released when begin returns. Transport/source checks +// happen before accept(), and request/body policies remain owner responsibilities. +class PublishAttempt +{ + public: + PublishAttempt(PublishAttemptPort& port, protocol::RecordCrypto& crypto) : port_(port), crypto_(crypto) {} + // Owner supplies an already durable request/result and attaches its port to + // that same task. No submit, new request ID, or persistence occurs here. + bool resume(const Destination& destination, const RequestId& request, ByteView encoded_request, + uint8_t* workspace, size_t capacity, const GeocacheId& expected_id, const RevisionHash& expected_hash, + ByteView durable_response = {}) + { + if (phase_ != PublishAttemptPhase::Idle || !workspace) return false; + const auto overlaps = [&](ByteView bytes) + { + if (!bytes.size) return false; + if (!bytes.data) return true; + const auto input = reinterpret_cast(bytes.data), output = reinterpret_cast(workspace); + return input <= output ? output - input < bytes.size : input - output < capacity; + }; + if (overlaps(encoded_request) || overlaps(durable_response)) return false; + protocol::PublishRequestView parsed; + protocol::VerifiedRecordView verified; + if (!protocol::decodePublishRequest(encoded_request, request, parsed) || + protocol::verifyGeocache(parsed.signed_cache, crypto_, workspace, capacity, verified, &expected_id, &expected_hash) != + protocol::VerificationResult::Valid) return false; + protocol::PublishDisposition disposition; + if (durable_response.size && !protocol::decodePublishResponse(durable_response, request, verified.id, verified.hash, + verified.record.revision, verified.record.state, disposition)) return false; + destination_ = destination; + request_ = request; + id_ = verified.id; + hash_ = verified.hash; + revision_ = verified.record.revision; + state_ = verified.record.state; + if (durable_response.size) disposition_ = disposition; + phase_ = durable_response.size ? PublishAttemptPhase::Confirmed : PublishAttemptPhase::Waiting; + return true; + } + bool begin(const Destination& destination, const RequestId& request, ByteView signed_record, + uint8_t* workspace, size_t capacity) + { + // At a 512-byte response budget, the request envelope adds 26 bytes + // to the original SignedCache. This also covers verification scratch. + if (phase_ != PublishAttemptPhase::Idle || !signed_record.data || signed_record.size > 4166 || + !workspace || capacity < signed_record.size + 26) return false; + const auto input = reinterpret_cast(signed_record.data); + const auto scratch = reinterpret_cast(workspace); + if (input <= scratch ? scratch - input < signed_record.size : input - scratch < capacity) return false; + protocol::VerifiedRecordView verified; + if (protocol::verifyGeocache(signed_record, crypto_, workspace, capacity, verified) != protocol::VerificationResult::Valid) + return false; + size_t size = 0; + if (!protocol::encodePublishRequest(request, verified.record.encoded, verified.signature, 512, + workspace, capacity, size)) return false; + // Capture borrowed input metadata before a durable submit can swap + // the storage arena that supplied the signed record. + destination_ = destination; + request_ = request; + id_ = verified.id; + hash_ = verified.hash; + revision_ = verified.record.revision; + state_ = verified.record.state; + const auto result = port_.submit(destination, request, {workspace, size}); + if (result == PublishPersistence::Rejected) return false; + phase_ = result == PublishPersistence::Pending ? PublishAttemptPhase::Submitting : PublishAttemptPhase::Waiting; + return true; + } + bool accept(const Destination& source, ByteView response) + { + if (source.bytes != destination_.bytes || !response.data) return false; + if (phase_ != PublishAttemptPhase::Waiting && phase_ != PublishAttemptPhase::Confirmed) return false; + protocol::PublishDisposition disposition; + if (!protocol::decodePublishResponse(response, request_, id_, hash_, revision_, state_, disposition)) return false; + // CMP1 is canonical and every other response field was compared above. + // Retaining the disposition is sufficient to reject a different result + // without retaining another 512-byte encoded response. + if (phase_ == PublishAttemptPhase::Confirmed) return disposition == disposition_; + const auto result = port_.commitResult(source, request_, response); + if (result == PublishPersistence::Rejected) return false; + disposition_ = disposition; + phase_ = result == PublishPersistence::Pending ? PublishAttemptPhase::Committing : PublishAttemptPhase::Confirmed; + return result == PublishPersistence::Committed; + } + void advance() + { + if (phase_ != PublishAttemptPhase::Submitting && phase_ != PublishAttemptPhase::Committing && + phase_ != PublishAttemptPhase::Cancelling) return; + const auto result = port_.poll(); + if (result == PublishPersistence::Pending) return; + if (result == PublishPersistence::Rejected) + { + phase_ = PublishAttemptPhase::Failed; + return; + } + phase_ = phase_ == PublishAttemptPhase::Submitting ? PublishAttemptPhase::Waiting : phase_ == PublishAttemptPhase::Committing ? PublishAttemptPhase::Confirmed + : PublishAttemptPhase::Cancelled; + } + bool cancel() + { + // Once an acknowledgement is being committed, preserve its result. + if (phase_ == PublishAttemptPhase::Confirmed || phase_ == PublishAttemptPhase::Committing) return false; + if (phase_ == PublishAttemptPhase::Cancelled || phase_ == PublishAttemptPhase::Cancelling) return true; + if (phase_ == PublishAttemptPhase::Idle) + { + phase_ = PublishAttemptPhase::Cancelled; + return true; + } + const auto result = port_.cancel(destination_, request_); + if (result == PublishPersistence::Rejected) return false; + phase_ = result == PublishPersistence::Pending ? PublishAttemptPhase::Cancelling : PublishAttemptPhase::Cancelled; + return true; + } + PublishAttemptPhase phase() const { return phase_; } + + private: + PublishAttemptPort& port_; + protocol::RecordCrypto& crypto_; + Destination destination_; + RequestId request_; + GeocacheId id_; + RevisionHash hash_; + uint32_t revision_ = 0; + CacheState state_ = CacheState::Active; + protocol::PublishDisposition disposition_ = protocol::PublishDisposition::Stored; + PublishAttemptPhase phase_ = PublishAttemptPhase::Idle; +}; +static_assert(sizeof(PublishAttempt) <= 192, "Publication attempts must not retain payload-sized buffers"); +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/query_client.h b/modules/core_geocaching/include/geocaching/usecase/query_client.h new file mode 100644 index 00000000..dae8939e --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/query_client.h @@ -0,0 +1,274 @@ +#pragma once +#include "geocaching/protocol/capabilities.h" +#include "geocaching/protocol/query_request.h" +#include "geocaching/protocol/query_response.h" +#include "geocaching/usecase/directory_pool.h" + +namespace geocaching +{ +enum class QueryPersistence : uint8_t +{ + Rejected, + Pending, + Committed +}; + +class QueryClientPort +{ + public: + virtual ~QueryClientPort() = default; + virtual bool newRequestId(RequestId& out) = 0; + // Persist exact request and destination before queueing transport. Bytes + // borrow the client buffer, kept immutable until Pending persistence ends. + // The port may queue that lease before acquiring a shared I/O workspace. + // Only Committed permits transport. pollPersistence advances one bounded step. + virtual QueryPersistence submit(const DirectoryEntry& directory, const RequestId& id, ByteView request) = 0; + virtual QueryPersistence commitCapabilities(const Destination& source, const RequestId& id, + ByteView response) = 0; + // Persist the exact response before reporting Committed. Summary views can + // be rebuilt from that response; all spans are borrowed during this call. + virtual QueryPersistence commitPage(const Destination& source, const RequestId& id, ByteView response, + const protocol::QueryPageView& page) = 0; + virtual QueryPersistence pollPersistence() = 0; + virtual QueryPersistence cancel(const Destination& source, const RequestId& id) = 0; +}; + +enum class QueryClientPhase : std::uint8_t +{ + Idle, + FindingDirectory, + CheckingCapabilities, + Querying, + PageReady, + PersistingRequest, + PersistingCapabilities, + PersistingPage, + Cancelling, + Failed +}; +enum class QueryFailure : uint8_t +{ + None, + Timeout, + Storage, + Cancelled +}; + +// Long-lived owner: allocate off task stack. One outstanding request at a time. +class QueryClient +{ + public: + explicit QueryClient(QueryClientPort& port) : port_(port), directories_(directory_entries_.data(), directory_entries_.size()) {} + static constexpr uint64_t kReplyTimeoutMs = 120000; + QueryClient(const QueryClient&) = delete; + QueryClient& operator=(const QueryClient&) = delete; + bool observe(const Destination& discovery, const Destination& derived_delivery, + ByteView public_key, ByteView app_data, std::uint64_t now) + { + protocol::DirectoryAnnouncement announcement; + return protocol::decodeDirectoryAnnouncement(app_data, derived_delivery, announcement) && + directories_.observe(discovery, announcement, public_key, now); + } + bool query(const protocol::QueryRegion& region) + { + if (phase_ != QueryClientPhase::Idle && phase_ != QueryClientPhase::PageReady && phase_ != QueryClientPhase::Failed) return false; + std::size_t size = 0; + if (!protocol::encodeQueryRequest(request_, region, 3, {}, 20, {}, 2048, + request_bytes_.data(), request_bytes_.size(), size)) return false; + region_ = region; + cursor_size_ = 0; + has_snapshot_ = false; + has_last_id_ = false; + failure_ = QueryFailure::None; + timer_armed_ = false; + phase_ = QueryClientPhase::FindingDirectory; + return true; + } + bool tick(std::uint64_t now) + { + if (phase_ == QueryClientPhase::PersistingRequest || phase_ == QueryClientPhase::PersistingCapabilities || + phase_ == QueryClientPhase::PersistingPage || phase_ == QueryClientPhase::Cancelling) + { + const auto result = port_.pollPersistence(); + if (result == QueryPersistence::Pending) return false; + if (result == QueryPersistence::Rejected) + { + if (selected_) selected_->in_flight = false; + phase_ = QueryClientPhase::Failed; + failure_ = QueryFailure::Storage; + return false; + } + timer_armed_ = false; + if (phase_ == QueryClientPhase::Cancelling) + { + selected_->in_flight = false; + phase_ = QueryClientPhase::Failed; + } + else if (phase_ == QueryClientPhase::PersistingRequest) phase_ = after_persist_; + else if (phase_ == QueryClientPhase::PersistingCapabilities) finishCapabilities(); + else + { + selected_->in_flight = false; + phase_ = QueryClientPhase::PageReady; + } + return true; + } + if (phase_ == QueryClientPhase::FindingDirectory || phase_ == QueryClientPhase::CheckingCapabilities || + phase_ == QueryClientPhase::Querying) + { + if (!timer_armed_ || now < waiting_since_) + { + waiting_since_ = now; + timer_armed_ = true; + } + if (now - waiting_since_ >= kReplyTimeoutMs) + { + if (selected_) selected_->retry_after = now > UINT64_MAX - 5000 ? UINT64_MAX : now + 5000; + return cancel(QueryFailure::Timeout); + } + } + if (phase_ != QueryClientPhase::FindingDirectory) return false; + selected_ = directories_.next(now, true); + const bool ready = selected_ != nullptr; + if (!selected_) selected_ = directories_.next(now, false); + if (!selected_ || !port_.newRequestId(request_)) return false; + std::size_t size = 0; + page_limit_ = ready ? (selected_->max_query_items < 20 ? selected_->max_query_items : 20) : 20; + const bool encoded = ready + ? protocol::encodeQueryRequest(request_, region_, 3, {}, page_limit_, {}, 2048, request_bytes_.data(), request_bytes_.size(), size) + : protocol::encodeCapabilitiesRequest(request_, request_bytes_.data(), request_bytes_.size(), size); + if (!encoded) return false; + return submit(size, ready ? QueryClientPhase::Querying : QueryClientPhase::CheckingCapabilities); + } + // Caller must authenticate the LXMF source and match local destination. + bool loadMore() + { + if (phase_ != QueryClientPhase::PageReady || !selected_ || cursor_size_ == 0) return false; + if (!port_.newRequestId(request_)) return false; + std::size_t size = 0; + if (!protocol::encodeQueryRequest(request_, region_, 3, {}, page_limit_, {cursor_.data(), cursor_size_}, + 2048, request_bytes_.data(), request_bytes_.size(), size)) return false; + return submit(size, QueryClientPhase::Querying); + } + bool hasMore() const { return phase_ == QueryClientPhase::PageReady && cursor_size_ != 0; } + + bool accept(const Destination& source, ByteView response) + { + if (!selected_ || source.bytes != selected_->delivery.bytes) return false; + if (phase_ == QueryClientPhase::CheckingCapabilities) + { + protocol::DirectoryCapabilities capabilities; + if (!protocol::decodeDirectoryCapabilities(response, request_, capabilities)) return false; + const auto result = port_.commitCapabilities(source, request_, response); + if (result == QueryPersistence::Rejected) return false; + pending_max_items_ = capabilities.max_query_items; + if (result == QueryPersistence::Pending) + { + phase_ = QueryClientPhase::PersistingCapabilities; + return false; + } + finishCapabilities(); + return true; + } + if (phase_ != QueryClientPhase::Querying) return false; + protocol::QueryPageView page; + if (!protocol::decodeQueryPage(response, request_, 2048, page_limit_, page)) return false; + if (has_snapshot_ && std::memcmp(snapshot_.data(), page.snapshot_id.data, 16) != 0) return false; + if (cursor_size_ && page.next_cursor.size == cursor_size_ && + std::memcmp(cursor_.data(), page.next_cursor.data, cursor_size_) == 0) return false; + protocol::CmpReader rows(page.encoded_items); + protocol::SummaryView point; + for (std::size_t i = 0; i < page.count; ++i) + { + if (!protocol::decodeSummary(rows, point) || (i == 0 && has_last_id_ && !(last_id_.bytes < point.id.bytes))) return false; + if (point.state == CacheState::Archived || point.latitude_e7 < region_.south_e7 || + point.latitude_e7 > region_.north_e7 || point.longitude_e7 < region_.west_e7 || + point.longitude_e7 > region_.east_e7) return false; + } + const auto result = port_.commitPage(source, request_, response, page); + if (result == QueryPersistence::Rejected) return false; + std::memcpy(snapshot_.data(), page.snapshot_id.data, 16); + has_snapshot_ = true; + if (page.count) + { + last_id_ = point.id; + has_last_id_ = true; + } + cursor_size_ = page.next_cursor.size; + if (cursor_size_) std::memcpy(cursor_.data(), page.next_cursor.data, cursor_size_); + if (result == QueryPersistence::Pending) + { + phase_ = QueryClientPhase::PersistingPage; + return false; + } + selected_->in_flight = false; + phase_ = QueryClientPhase::PageReady; + return true; + } + QueryClientPhase phase() const { return phase_; } + QueryFailure failure() const { return failure_; } + bool persistencePending() const + { + return phase_ == QueryClientPhase::PersistingRequest || phase_ == QueryClientPhase::PersistingCapabilities || + phase_ == QueryClientPhase::PersistingPage || phase_ == QueryClientPhase::Cancelling; + } + // Retry after the current commit finishes. The runtime never sends during + // cancellation; Failed is exposed only after the stop is durably committed. + bool cancel(QueryFailure reason = QueryFailure::Cancelled) + { + if (persistencePending()) return false; + if (!selected_ || !selected_->in_flight) + { + failure_ = reason; + phase_ = QueryClientPhase::Failed; + return true; + } + const auto result = port_.cancel(selected_->delivery, request_); + if (result == QueryPersistence::Rejected) return false; + failure_ = reason; + phase_ = result == QueryPersistence::Pending ? QueryClientPhase::Cancelling : QueryClientPhase::Failed; + if (result == QueryPersistence::Committed) selected_->in_flight = false; + return true; + } + + private: + bool submit(size_t size, QueryClientPhase next) + { + const auto result = port_.submit(*selected_, request_, {request_bytes_.data(), size}); + if (result == QueryPersistence::Rejected) return false; + selected_->in_flight = true; + after_persist_ = next; + phase_ = result == QueryPersistence::Pending ? QueryClientPhase::PersistingRequest : next; + timer_armed_ = false; + return true; + } + void finishCapabilities() + { + selected_->capabilities_verified = true; + selected_->max_query_items = pending_max_items_; + selected_->in_flight = false; + phase_ = QueryClientPhase::FindingDirectory; + timer_armed_ = false; + } + QueryClientPhase after_persist_ = QueryClientPhase::Idle; + uint8_t pending_max_items_ = 0; + QueryClientPort& port_; + std::array directory_entries_{}; + DirectoryPool directories_; + DirectoryEntry* selected_ = nullptr; + protocol::QueryRegion region_; + RequestId request_; + std::array cursor_{}; + std::array snapshot_{}; + GeocacheId last_id_; + std::size_t cursor_size_ = 0; + std::uint8_t page_limit_ = 20; + bool has_snapshot_ = false; + bool has_last_id_ = false; + std::array request_bytes_{}; + QueryClientPhase phase_ = QueryClientPhase::Idle; + QueryFailure failure_ = QueryFailure::None; + uint64_t waiting_since_ = 0; + bool timer_armed_ = false; +}; +} // namespace geocaching diff --git a/modules/core_geocaching/include/geocaching/usecase/request_tracker.h b/modules/core_geocaching/include/geocaching/usecase/request_tracker.h new file mode 100644 index 00000000..ac0e05a5 --- /dev/null +++ b/modules/core_geocaching/include/geocaching/usecase/request_tracker.h @@ -0,0 +1,102 @@ +#pragma once +#include "geocaching/domain/record.h" + +namespace geocaching +{ +enum class RequestPhase : std::uint8_t +{ + Idle, + Queued, + InFlight, + AwaitingResult, + Unconfirmed, + ResultRecorded, +}; + +enum class ResultAdmission : std::uint8_t +{ + Accepted, + Duplicate, + WrongRequest, + ConflictingResult, +}; + +// One logical request. Ownership/persistence are external: call recordResult +// only after the authenticated, fully validated result has durably committed. +class RequestTracker +{ + public: + bool begin(const Destination& local, const Destination& remote, + const RequestId& id, Operation operation, std::uint16_t budget) + { + if (phase_ != RequestPhase::Idle || budget < 512 || budget > kMaxApplicationBytes || + static_cast(operation) > 4) return false; + local_ = local; + remote_ = remote; + id_ = id; + operation_ = operation; + budget_ = budget; + continue_ = true; + phase_ = RequestPhase::Queued; + return true; + } + + bool startAttempt() + { + if (!continue_ || (phase_ != RequestPhase::Queued && phase_ != RequestPhase::Unconfirmed)) return false; + phase_ = RequestPhase::InFlight; + return true; + } + + void delivered() + { + if (phase_ == RequestPhase::InFlight) phase_ = RequestPhase::AwaitingResult; + } + + void unconfirmed() + { + if (phase_ == RequestPhase::InFlight || phase_ == RequestPhase::AwaitingResult) + phase_ = RequestPhase::Unconfirmed; + } + + void stopFurtherAttempts() { continue_ = false; } + + ResultAdmission classifyResult(const Destination& source, const Destination& recipient, + const RequestId& id, Operation operation, + std::size_t payload_size, + const std::array& result_digest) const + { + if (phase_ == RequestPhase::Idle || source.bytes != remote_.bytes || + recipient.bytes != local_.bytes || id.bytes != id_.bytes || + operation != operation_ || payload_size == 0 || payload_size > budget_) + return ResultAdmission::WrongRequest; + if (phase_ == RequestPhase::ResultRecorded) + return result_digest == result_digest_ ? ResultAdmission::Duplicate : ResultAdmission::ConflictingResult; + return ResultAdmission::Accepted; + } + + bool recordResult(const Destination& source, const Destination& recipient, + const RequestId& id, Operation operation, std::size_t payload_size, + const std::array& result_digest) + { + if (classifyResult(source, recipient, id, operation, payload_size, result_digest) != ResultAdmission::Accepted) + return false; + result_digest_ = result_digest; + phase_ = RequestPhase::ResultRecorded; + return true; + } + + RequestPhase phase() const { return phase_; } + bool continueRequested() const { return continue_; } + + private: + Destination local_; + Destination remote_; + RequestId id_; + Operation operation_ = Operation::Capabilities; + std::uint16_t budget_ = 0; + RequestPhase phase_ = RequestPhase::Idle; + bool continue_ = false; + std::array result_digest_{}; +}; +} // namespace geocaching diff --git a/modules/core_geocaching/library.json b/modules/core_geocaching/library.json new file mode 100644 index 00000000..6b387003 --- /dev/null +++ b/modules/core_geocaching/library.json @@ -0,0 +1,9 @@ +{ + "name": "core_geocaching", + "version": "0.1.0", + "dependencies": [{"name": "core_gps"}, {"name": "core_sys"}], + "build": { + "includeDir": "include", + "srcDir": "src" + } +} diff --git a/modules/core_geocaching/tests/CMakeLists.txt b/modules/core_geocaching/tests/CMakeLists.txt new file mode 100644 index 00000000..35a71f17 --- /dev/null +++ b/modules/core_geocaching/tests/CMakeLists.txt @@ -0,0 +1,343 @@ +cmake_minimum_required(VERSION 3.20) +project(GeocachingCoreTests LANGUAGES C CXX) +include_directories(../../core_gps/include) +add_library(geocaching_lxmf_wire_compile OBJECT ../../core_chat/src/infra/lxmf/lxmf_wire.cpp) +target_include_directories(geocaching_lxmf_wire_compile PRIVATE ../../core_chat/include) +target_compile_features(geocaching_lxmf_wire_compile PRIVATE cxx_std_17) +enable_testing() +add_executable(geocaching_record_shape_test test_record_shape.cpp) +target_include_directories(geocaching_record_shape_test PRIVATE ../include ../../core_sys/include) +target_compile_features(geocaching_record_shape_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record_shape COMMAND geocaching_record_shape_test) +add_executable(geocaching_index_cursor_test test_transaction_index_cursor.cpp) +target_include_directories(geocaching_index_cursor_test PRIVATE ../include ../../core_sys/include) +target_compile_features(geocaching_index_cursor_test PRIVATE cxx_std_17) +add_test(NAME geocaching_index_cursor COMMAND geocaching_index_cursor_test) +add_executable(geocaching_query_timeout_test test_query_timeout.cpp) +target_include_directories(geocaching_query_timeout_test PRIVATE ../include) +target_compile_features(geocaching_query_timeout_test PRIVATE cxx_std_17) +add_test(NAME geocaching_query_timeout COMMAND geocaching_query_timeout_test) +add_executable(geocaching_generated_state_test test_generated_state.cpp) +target_include_directories(geocaching_generated_state_test PRIVATE ../include) +target_compile_features(geocaching_generated_state_test PRIVATE cxx_std_17) +add_test(NAME geocaching_generated_state COMMAND geocaching_generated_state_test) +add_executable(geocaching_transaction_stream_test test_transaction_stream.cpp) +target_include_directories(geocaching_transaction_stream_test PRIVATE ../include) +target_compile_features(geocaching_transaction_stream_test PRIVATE cxx_std_17) +add_test(NAME geocaching_transaction_stream COMMAND geocaching_transaction_stream_test) +add_executable(geocaching_shared_track_test test_shared_track.cpp) +target_compile_features(geocaching_shared_track_test PRIVATE cxx_std_17) +add_test(NAME geocaching_shared_track COMMAND geocaching_shared_track_test) +add_executable(geocaching_track_options_test test_track_options.cpp) +target_compile_features(geocaching_track_options_test PRIVATE cxx_std_17) +add_test(NAME geocaching_track_options COMMAND geocaching_track_options_test) +add_executable(geocaching_track_motion_test test_track_motion.cpp) +target_compile_features(geocaching_track_motion_test PRIVATE cxx_std_17) +add_test(NAME geocaching_track_motion COMMAND geocaching_track_motion_test ${CMAKE_CURRENT_BINARY_DIR}/track-motion.gpx) +add_executable(geocaching_cmp_test test_cmp.cpp) +target_include_directories(geocaching_cmp_test PRIVATE ../include) +target_compile_features(geocaching_cmp_test PRIVATE cxx_std_17) +add_test(NAME geocaching_cmp COMMAND geocaching_cmp_test) +add_executable(geocaching_record_test test_record.cpp) +target_include_directories(geocaching_record_test PRIVATE ../include) +target_compile_features(geocaching_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record COMMAND geocaching_record_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/record-v1.bin) +add_executable(geocaching_verify_test test_verify.cpp) +target_include_directories(geocaching_verify_test PRIVATE ../include) +target_compile_features(geocaching_verify_test PRIVATE cxx_std_17) +add_test(NAME geocaching_verify COMMAND geocaching_verify_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +set(ED25519_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../../core_chat/src/infra/meshcore/crypto/ed25519) +add_executable(geocaching_verify_peer_capture verify_peer_capture.cpp $ + ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_verify_peer_capture PRIVATE ../include) +target_link_libraries(geocaching_verify_peer_capture PRIVATE geocaching_native_hash) +target_compile_features(geocaching_verify_peer_capture PRIVATE cxx_std_17) +add_executable(geocaching_publish_attempt_test test_publish_attempt.cpp + ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_publish_attempt_test PRIVATE ../include ../../core_chat/include) +target_link_libraries(geocaching_publish_attempt_test PRIVATE geocaching_native_hash) +target_compile_features(geocaching_publish_attempt_test PRIVATE cxx_std_17) +add_test(NAME geocaching_publish_attempt COMMAND geocaching_publish_attempt_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/publish-response-v1.bin) +add_executable(geocaching_record_signing_test test_record_signing.cpp + ${ED25519_DIR}/sign.c ${ED25519_DIR}/keypair.c ${ED25519_DIR}/fe.c + ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_record_signing_test PRIVATE ../include ../../core_chat/include) +target_compile_features(geocaching_record_signing_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record_signing COMMAND geocaching_record_signing_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +add_executable(geocaching_ed25519_test test_ed25519.cpp + ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c + ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_ed25519_test PRIVATE ../include ../../core_chat/include) +target_compile_features(geocaching_ed25519_test PRIVATE cxx_std_17) +add_test(NAME geocaching_ed25519 COMMAND geocaching_ed25519_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +add_executable(geocaching_writer_test test_writer.cpp) +target_include_directories(geocaching_writer_test PRIVATE ../include) +target_compile_features(geocaching_writer_test PRIVATE cxx_std_17) +add_test(NAME geocaching_writer COMMAND geocaching_writer_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/get-v1.bin) +add_executable(geocaching_query_test test_query.cpp) +target_include_directories(geocaching_query_test PRIVATE ../include) +target_compile_features(geocaching_query_test PRIVATE cxx_std_17) +add_test(NAME geocaching_query COMMAND geocaching_query_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/capabilities-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-v1.bin) +add_executable(geocaching_capabilities_test test_capabilities.cpp) +target_include_directories(geocaching_capabilities_test PRIVATE ../include) +target_compile_features(geocaching_capabilities_test PRIVATE cxx_std_17) +add_test(NAME geocaching_capabilities COMMAND geocaching_capabilities_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/capabilities-response-v1.bin) +add_executable(geocaching_get_response_test test_get_response.cpp) +target_include_directories(geocaching_get_response_test PRIVATE ../include) +target_compile_features(geocaching_get_response_test PRIVATE cxx_std_17) +add_test(NAME geocaching_get_response COMMAND geocaching_get_response_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/get-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +add_executable(geocaching_query_response_test test_query_response.cpp) +target_include_directories(geocaching_query_response_test PRIVATE ../include) +target_compile_features(geocaching_query_response_test PRIVATE cxx_std_17) +add_test(NAME geocaching_query_response COMMAND geocaching_query_response_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-response-v1.bin) +add_executable(geocaching_discovery_test test_discovery.cpp) +target_include_directories(geocaching_discovery_test PRIVATE ../include) +target_compile_features(geocaching_discovery_test PRIVATE cxx_std_17) +add_test(NAME geocaching_discovery COMMAND geocaching_discovery_test) +add_executable(geocaching_publish_test test_publish.cpp) +target_include_directories(geocaching_publish_test PRIVATE ../include) +target_compile_features(geocaching_publish_test PRIVATE cxx_std_17) +add_test(NAME geocaching_publish COMMAND geocaching_publish_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/publish-v1.bin) +add_executable(geocaching_publish_response_test test_publish_response.cpp) +target_include_directories(geocaching_publish_response_test PRIVATE ../include) +target_compile_features(geocaching_publish_response_test PRIVATE cxx_std_17) +add_test(NAME geocaching_publish_response COMMAND geocaching_publish_response_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/publish-response-v1.bin) +add_executable(geocaching_request_tracker_test test_request_tracker.cpp) +target_include_directories(geocaching_request_tracker_test PRIVATE ../include) +target_compile_features(geocaching_request_tracker_test PRIVATE cxx_std_17) +add_test(NAME geocaching_request_tracker COMMAND geocaching_request_tracker_test) +add_executable(geocaching_gpx_text_test test_gpx_text.cpp) +target_include_directories(geocaching_gpx_text_test PRIVATE ../include) +target_compile_features(geocaching_gpx_text_test PRIVATE cxx_std_17) +add_test(NAME geocaching_gpx_text COMMAND geocaching_gpx_text_test) +add_executable(geocaching_buffered_test test_buffered.cpp) +target_include_directories(geocaching_buffered_test PRIVATE ../include) +target_compile_features(geocaching_buffered_test PRIVATE cxx_std_17) +add_test(NAME geocaching_buffered COMMAND geocaching_buffered_test) +add_executable(geocaching_gpx_record_test test_gpx_record.cpp) +target_include_directories(geocaching_gpx_record_test PRIVATE ../include) +target_compile_features(geocaching_gpx_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_gpx_record COMMAND geocaching_gpx_record_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/gpx-metadata.bin ${CMAKE_CURRENT_BINARY_DIR}/generated.gpx) +add_executable(geocaching_version_test test_version.cpp) +target_include_directories(geocaching_version_test PRIVATE ../include) +target_compile_features(geocaching_version_test PRIVATE cxx_std_17) +add_test(NAME geocaching_version COMMAND geocaching_version_test) +add_executable(geocaching_install_test test_install.cpp) +target_include_directories(geocaching_install_test PRIVATE ../include) +target_compile_features(geocaching_install_test PRIVATE cxx_std_17) +add_test(NAME geocaching_install COMMAND geocaching_install_test) +add_executable(geocaching_directory_pool_test test_directory_pool.cpp) +target_include_directories(geocaching_directory_pool_test PRIVATE ../include) +target_compile_features(geocaching_directory_pool_test PRIVATE cxx_std_17) +add_test(NAME geocaching_directory_pool COMMAND geocaching_directory_pool_test) +add_executable(geocaching_query_client_test test_query_client.cpp) +target_include_directories(geocaching_query_client_test PRIVATE ../include) +target_compile_features(geocaching_query_client_test PRIVATE cxx_std_17) +add_test(NAME geocaching_query_client COMMAND geocaching_query_client_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/capabilities-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-response-v1.bin) +add_executable(geocaching_paging_test test_paging.cpp) +target_include_directories(geocaching_paging_test PRIVATE ../include) +target_compile_features(geocaching_paging_test PRIVATE cxx_std_17) +add_test(NAME geocaching_paging COMMAND geocaching_paging_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/capabilities-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-page1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-page2.bin) +add_executable(geocaching_router_regression + ../../core_chat/tests/test_mesh_adapter_router_core.cpp + ../../core_chat/src/infra/mesh_adapter_router_core.cpp) +target_include_directories(geocaching_router_regression PRIVATE ../../core_chat/include ../../core_sys/include) +target_compile_features(geocaching_router_regression PRIVATE cxx_std_17) +add_test(NAME geocaching_router_regression COMMAND geocaching_router_regression) +add_executable(geocaching_service_backend_test test_service_backend.cpp ../../core_chat/src/infra/mesh_adapter_router_core.cpp) +target_include_directories(geocaching_service_backend_test PRIVATE ../../core_chat/include ../../core_sys/include) +target_compile_features(geocaching_service_backend_test PRIVATE cxx_std_17) +add_test(NAME geocaching_service_backend COMMAND geocaching_service_backend_test) +add_executable(geocaching_service_poll_test test_service_poll.cpp ../../core_chat/src/infra/mesh_adapter_router_core.cpp) +target_include_directories(geocaching_service_poll_test PRIVATE ../../core_chat/include ../../core_sys/include) +target_compile_features(geocaching_service_poll_test PRIVATE cxx_std_17) +add_test(NAME geocaching_service_poll COMMAND geocaching_service_poll_test) +add_executable(geocaching_service_ownership_test test_service_ownership.cpp ../../core_chat/src/infra/mesh_adapter_router_core.cpp) +target_include_directories(geocaching_service_ownership_test PRIVATE ../../core_chat/include ../../core_sys/include) +target_compile_features(geocaching_service_ownership_test PRIVATE cxx_std_17) +add_test(NAME geocaching_service_ownership COMMAND geocaching_service_ownership_test) +add_executable(gpx_attributes_test test_gpx_attributes.cpp) +add_executable(geocaching_transaction_validation_test test_transaction_validation.cpp) +target_include_directories(geocaching_transaction_validation_test PRIVATE ../include) +target_compile_features(geocaching_transaction_validation_test PRIVATE cxx_std_17) +add_test(NAME geocaching_transaction_validation COMMAND geocaching_transaction_validation_test) +add_executable(geocaching_record_read_budget_test test_record_read_budget.cpp) +target_include_directories(geocaching_record_read_budget_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_record_read_budget_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record_read_budget COMMAND geocaching_record_read_budget_test) +add_executable(geocaching_attempt_timeout_test test_attempt_timeout.cpp) +target_include_directories(geocaching_attempt_timeout_test PRIVATE ../include) +target_compile_features(geocaching_attempt_timeout_test PRIVATE cxx_std_17) +add_test(NAME geocaching_attempt_timeout COMMAND geocaching_attempt_timeout_test) +add_executable(geocaching_tx_attempt_test test_tx_attempt.cpp) +target_include_directories(geocaching_tx_attempt_test PRIVATE ../include) +target_compile_features(geocaching_tx_attempt_test PRIVATE cxx_std_17) +add_test(NAME geocaching_tx_attempt COMMAND geocaching_tx_attempt_test) +add_executable(geocaching_pending_request_test test_pending_request.cpp) +target_include_directories(geocaching_pending_request_test PRIVATE ../include) +target_compile_features(geocaching_pending_request_test PRIVATE cxx_std_17) +add_test(NAME geocaching_pending_request COMMAND geocaching_pending_request_test) +add_executable(geocaching_author_issue_test test_author_issue.cpp) +target_include_directories(geocaching_author_issue_test PRIVATE ../include) +target_compile_features(geocaching_author_issue_test PRIVATE cxx_std_17) +add_test(NAME geocaching_author_issue COMMAND geocaching_author_issue_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +add_executable(geocaching_author_history_test test_author_history.cpp) +target_include_directories(geocaching_author_history_test PRIVATE ../include) +target_compile_features(geocaching_author_history_test PRIVATE cxx_std_17) +add_test(NAME geocaching_author_history COMMAND geocaching_author_history_test) +add_executable(geocaching_author_issued_test test_author_issued.cpp) +target_include_directories(geocaching_author_issued_test PRIVATE ../include) +target_compile_features(geocaching_author_issued_test PRIVATE cxx_std_17) +add_test(NAME geocaching_author_issued COMMAND geocaching_author_issued_test) +add_executable(geocaching_record_encoder_test test_record_encoder.cpp) +target_include_directories(geocaching_record_encoder_test PRIVATE ../include) +target_compile_features(geocaching_record_encoder_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record_encoder COMMAND geocaching_record_encoder_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +add_executable(geocaching_gpx_hash_test test_gpx_hash.cpp) +target_include_directories(geocaching_gpx_hash_test PRIVATE ../include ../../../platform/esp/arduino_common/include) +target_link_libraries(geocaching_gpx_hash_test PRIVATE geocaching_native_hash) +target_compile_features(geocaching_gpx_hash_test PRIVATE cxx_std_17) +add_test(NAME geocaching_gpx_hash COMMAND geocaching_gpx_hash_test) +add_executable(geocaching_install_record_test test_install_record.cpp) +target_include_directories(geocaching_install_record_test PRIVATE ../include) +target_compile_features(geocaching_install_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_install_record COMMAND geocaching_install_record_test) +add_executable(geocaching_cache_head_test test_cache_head.cpp) +target_include_directories(geocaching_cache_head_test PRIVATE ../include) +target_compile_features(geocaching_cache_head_test PRIVATE cxx_std_17) +add_test(NAME geocaching_cache_head COMMAND geocaching_cache_head_test) +add_executable(geocaching_snapshot_state_test test_snapshot_state.cpp) +target_include_directories(geocaching_snapshot_state_test PRIVATE ../include) +target_compile_features(geocaching_snapshot_state_test PRIVATE cxx_std_17) +add_test(NAME geocaching_snapshot_state COMMAND geocaching_snapshot_state_test) +add_executable(geocaching_task_references_test test_task_references.cpp) +target_include_directories(geocaching_task_references_test PRIVATE ../include) +target_compile_features(geocaching_task_references_test PRIVATE cxx_std_17) +add_test(NAME geocaching_task_references COMMAND geocaching_task_references_test) +add_executable(geocaching_logical_state_test test_logical_state.cpp) +target_include_directories(geocaching_logical_state_test PRIVATE ../include) +target_compile_features(geocaching_logical_state_test PRIVATE cxx_std_17) +add_test(NAME geocaching_logical_state COMMAND geocaching_logical_state_test) +add_executable(geocaching_checkpoint_selection_test test_checkpoint_selection.cpp) +target_include_directories(geocaching_checkpoint_selection_test PRIVATE ../include) +target_compile_features(geocaching_checkpoint_selection_test PRIVATE cxx_std_17) +add_test(NAME geocaching_checkpoint_selection COMMAND geocaching_checkpoint_selection_test) +add_executable(geocaching_sd_checkpoint_test test_sd_checkpoint.cpp) +target_include_directories(geocaching_sd_checkpoint_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_link_libraries(geocaching_sd_checkpoint_test PRIVATE geocaching_native_hash) +target_compile_features(geocaching_sd_checkpoint_test PRIVATE cxx_std_17) +add_test(NAME geocaching_sd_checkpoint COMMAND geocaching_sd_checkpoint_test) +add_executable(geocaching_checkpoint_verifier_test test_checkpoint_verifier.cpp) +target_include_directories(geocaching_checkpoint_verifier_test PRIVATE ../include ../../core_sys/include) +target_link_libraries(geocaching_checkpoint_verifier_test PRIVATE geocaching_native_hash) +target_compile_features(geocaching_checkpoint_verifier_test PRIVATE cxx_std_17) +add_test(NAME geocaching_checkpoint_verifier COMMAND geocaching_checkpoint_verifier_test) +add_executable(geocaching_checkpoint_test test_checkpoint.cpp) +target_include_directories(geocaching_checkpoint_test PRIVATE ../include) +target_compile_features(geocaching_checkpoint_test PRIVATE cxx_std_17) +add_test(NAME geocaching_checkpoint COMMAND geocaching_checkpoint_test) +add_executable(geocaching_journal_segment_test test_journal_segment.cpp) +target_include_directories(geocaching_journal_segment_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_journal_segment_test PRIVATE cxx_std_17) +add_test(NAME geocaching_journal_segment COMMAND geocaching_journal_segment_test) +add_executable(geocaching_journal_inventory_test test_journal_inventory.cpp) +target_include_directories(geocaching_journal_inventory_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_journal_inventory_test PRIVATE cxx_std_17) +add_test(NAME geocaching_journal_inventory COMMAND geocaching_journal_inventory_test) +add_executable(geocaching_journal_replay_test test_journal_replay.cpp) +target_sources(geocaching_journal_replay_test PRIVATE ../../../platform/esp/arduino_common/src/geocaching/request_dispatcher.cpp) +target_include_directories(geocaching_journal_replay_test BEFORE PRIVATE fakes) +target_include_directories(geocaching_journal_replay_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_journal_replay_test PRIVATE cxx_std_17) +target_link_libraries(geocaching_journal_replay_test PRIVATE geocaching_native_hash) +target_sources(geocaching_journal_replay_test PRIVATE ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +add_test(NAME geocaching_journal_replay COMMAND geocaching_journal_replay_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/get-response-v1.bin) +add_executable(geocaching_sd_volume_test test_sd_volume.cpp) +target_include_directories(geocaching_sd_volume_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_volume_test PRIVATE cxx_std_17) +add_test(NAME geocaching_sd_volume COMMAND geocaching_sd_volume_test) +add_library(geocaching_sd_request_store_compile OBJECT compile_sd_request_store.cpp) +target_include_directories(geocaching_sd_request_store_compile PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_request_store_compile PRIVATE cxx_std_17) +add_executable(geocaching_task_record_test test_task_record.cpp) +target_include_directories(geocaching_task_record_test PRIVATE ../include) +target_compile_features(geocaching_task_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_task_record COMMAND geocaching_task_record_test) +add_executable(geocaching_outgoing_record_test test_outgoing_record.cpp) +target_include_directories(geocaching_outgoing_record_test PRIVATE ../include) +target_compile_features(geocaching_outgoing_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_outgoing_record COMMAND geocaching_outgoing_record_test) +add_executable(geocaching_download_task_record_test test_download_task_record.cpp) +target_include_directories(geocaching_download_task_record_test PRIVATE ../include) +target_compile_features(geocaching_download_task_record_test PRIVATE cxx_std_17) +add_test(NAME geocaching_download_task_record COMMAND geocaching_download_task_record_test) +add_executable(geocaching_queued_request_test test_queued_request.cpp) +target_include_directories(geocaching_queued_request_test PRIVATE ../include) +target_compile_features(geocaching_queued_request_test PRIVATE cxx_std_17) +add_test(NAME geocaching_queued_request COMMAND geocaching_queued_request_test) +add_executable(geocaching_volume_format_test test_volume_format.cpp) +target_include_directories(geocaching_volume_format_test PRIVATE ../include ../../core_sys/include) +target_compile_features(geocaching_volume_format_test PRIVATE cxx_std_17) +add_test(NAME geocaching_volume_format COMMAND geocaching_volume_format_test) +add_library(geocaching_sd_journal_compile OBJECT compile_sd_journal.cpp) +add_executable(geocaching_journal_read_test test_journal_read.cpp) +target_include_directories(geocaching_journal_read_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_journal_read_test PRIVATE cxx_std_17) +add_test(NAME geocaching_journal_read COMMAND geocaching_journal_read_test) +add_executable(geocaching_sd_journal_test test_sd_journal.cpp) +target_sources(geocaching_sd_journal_test PRIVATE ../../../platform/esp/arduino_common/src/geocaching/request_dispatcher.cpp) +target_include_directories(geocaching_sd_journal_test BEFORE PRIVATE fakes) +target_include_directories(geocaching_sd_journal_test PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_journal_test PRIVATE cxx_std_17) +target_link_libraries(geocaching_sd_journal_test PRIVATE geocaching_native_hash) +target_sources(geocaching_sd_journal_test PRIVATE ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +add_test(NAME geocaching_sd_journal COMMAND geocaching_sd_journal_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/publish-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/capabilities-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/get-response-v1.bin) +target_include_directories(geocaching_sd_journal_test PRIVATE ../../ui_presentation/include) +target_include_directories(geocaching_sd_journal_compile PRIVATE ../include ../../core_sys/include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_journal_compile PRIVATE cxx_std_17) +add_executable(geocaching_record_frame_test test_record_frame.cpp) +target_include_directories(geocaching_record_frame_test PRIVATE ../include ../../core_sys/include) +target_compile_features(geocaching_record_frame_test PRIVATE cxx_std_17) +add_test(NAME geocaching_record_frame COMMAND geocaching_record_frame_test) +add_executable(geocaching_crc_test test_crc.cpp) +target_include_directories(geocaching_crc_test PRIVATE ../../core_sys/include) +target_compile_features(geocaching_crc_test PRIVATE cxx_std_17) +add_test(NAME geocaching_crc COMMAND geocaching_crc_test) +add_executable(geocaching_transaction_test test_transaction.cpp) +target_include_directories(geocaching_transaction_test PRIVATE ../include) +target_compile_features(geocaching_transaction_test PRIVATE cxx_std_17) +add_test(NAME geocaching_transaction COMMAND geocaching_transaction_test) +add_executable(geocaching_transaction_decode_test test_transaction_decode.cpp) +target_include_directories(geocaching_transaction_decode_test PRIVATE ../include) +target_compile_features(geocaching_transaction_decode_test PRIVATE cxx_std_17) +add_test(NAME geocaching_transaction_decode COMMAND geocaching_transaction_decode_test) +add_library(geocaching_sd_stage_compile OBJECT compile_sd_gpx_stage.cpp) +add_executable(geocaching_sd_stage_test test_sd_stage.cpp) +target_include_directories(geocaching_sd_stage_test PRIVATE ../include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_stage_test PRIVATE cxx_std_17) +add_test(NAME geocaching_sd_stage COMMAND geocaching_sd_stage_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin) +target_include_directories(geocaching_sd_stage_compile PRIVATE ../include ../../../platform/esp/arduino_common/include) +target_compile_features(geocaching_sd_stage_compile PRIVATE cxx_std_17) +target_compile_features(gpx_attributes_test PRIVATE cxx_std_17) +add_test(NAME gpx_attributes COMMAND gpx_attributes_test) +add_library(geocaching_native_hash STATIC ../../core_chat/src/infra/reticulum/reticulum_wire.cpp) +target_include_directories(geocaching_native_hash PUBLIC ../../core_chat/include) +target_compile_definitions(geocaching_native_hash PRIVATE TRAIL_MATE_RETICULUM_HASH_ONLY=1) +target_compile_features(geocaching_native_hash PUBLIC cxx_std_17) +add_executable(geocaching_lxmf_custom_test test_lxmf_custom.cpp $) +target_include_directories(geocaching_lxmf_custom_test PRIVATE ../include) +target_link_libraries(geocaching_lxmf_custom_test PRIVATE geocaching_native_hash) +add_test(NAME geocaching_lxmf_custom COMMAND geocaching_lxmf_custom_test) +add_executable(geocaching_custom_extract_test test_custom_extract.cpp $) +target_link_libraries(geocaching_custom_extract_test PRIVATE geocaching_native_hash) +add_test(NAME geocaching_custom_extract COMMAND geocaching_custom_extract_test) +add_executable(geocaching_crypto_pipeline_test test_crypto_pipeline.cpp + ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_crypto_pipeline_test PRIVATE ../include) +target_link_libraries(geocaching_crypto_pipeline_test PRIVATE geocaching_native_hash) +add_test(NAME geocaching_crypto_pipeline COMMAND geocaching_crypto_pipeline_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/signed-cache-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/identity-hashes.bin) +add_executable(geocaching_download_test test_download.cpp + ${ED25519_DIR}/verify.c ${ED25519_DIR}/fe.c ${ED25519_DIR}/ge.c ${ED25519_DIR}/sc.c ${ED25519_DIR}/sha512.c) +target_include_directories(geocaching_download_test PRIVATE ../include) +target_link_libraries(geocaching_download_test PRIVATE geocaching_native_hash) +add_test(NAME geocaching_download COMMAND geocaching_download_test ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/query-response-v1.bin ${CMAKE_CURRENT_SOURCE_DIR}/fixtures/get-response-v1.bin) diff --git a/modules/core_geocaching/tests/compile_sd_gpx_stage.cpp b/modules/core_geocaching/tests/compile_sd_gpx_stage.cpp new file mode 100644 index 00000000..fe686d7e --- /dev/null +++ b/modules/core_geocaching/tests/compile_sd_gpx_stage.cpp @@ -0,0 +1,4 @@ +#include "platform/esp/arduino_common/geocaching/sd_gpx_stage.h" + +static_assert(sizeof(platform::esp::arduino_common::geocaching::SdGpxStage) < 2048, + "GPX staging job must remain a bounded worker-owned allocation"); diff --git a/modules/core_geocaching/tests/compile_sd_journal.cpp b/modules/core_geocaching/tests/compile_sd_journal.cpp new file mode 100644 index 00000000..dd9a6557 --- /dev/null +++ b/modules/core_geocaching/tests/compile_sd_journal.cpp @@ -0,0 +1,4 @@ +#include "platform/esp/arduino_common/geocaching/sd_journal.h" + +static_assert(sizeof(platform::esp::arduino_common::geocaching::SdGeocachingJournal) <= 768, + "Journal owner must have bounded storage"); diff --git a/modules/core_geocaching/tests/compile_sd_request_store.cpp b/modules/core_geocaching/tests/compile_sd_request_store.cpp new file mode 100644 index 00000000..e9a2409a --- /dev/null +++ b/modules/core_geocaching/tests/compile_sd_request_store.cpp @@ -0,0 +1,3 @@ +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" +static_assert(sizeof(platform::esp::arduino_common::geocaching::SdRequestStore) < 2048, + "Request store must not regain transaction or Outgoing payload buffers"); diff --git a/modules/core_geocaching/tests/fakes/esp_random.h b/modules/core_geocaching/tests/fakes/esp_random.h new file mode 100644 index 00000000..93d9a27d --- /dev/null +++ b/modules/core_geocaching/tests/fakes/esp_random.h @@ -0,0 +1,11 @@ +#pragma once +#include +#include +// Deterministic attempt IDs only for native dispatcher tests. +inline void esp_fill_random(void* output, std::size_t size) +{ + static uint8_t generation = 0; + auto* bytes = static_cast(output); + ++generation; + for (std::size_t i = 0; i < size; ++i) bytes[i] = static_cast(generation + i); +} diff --git a/modules/core_geocaching/tests/fakes/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h b/modules/core_geocaching/tests/fakes/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h new file mode 100644 index 00000000..063fd755 --- /dev/null +++ b/modules/core_geocaching/tests/fakes/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h @@ -0,0 +1,36 @@ +#pragma once +// Host-only transport seam for compiling the real dispatcher state machine. +// Firmware always includes the production router; this header is test-local. +#include "chat/domain/chat_types.h" +#include "chat/infra/lxmf/lxmf_wire.h" +#include +#include +#include + +namespace chat +{ +class MeshAdapterRouter +{ + public: + bool ready = true, send_ok = true; + unsigned sends = 0; + std::array local{}; + std::vector sent_bytes; + bool getGeocachingDispatchDestination(uint8_t out[16]) + { + if (!ready) return false; + std::memcpy(out, local.data(), 16); + return true; + } + MeshSendResult sendGeocachingData(const uint8_t[16], lxmf::ByteSpan bytes, bool, + std::array* hash, const uint8_t expected[16]) + { + ++sends; + if (!ready || std::memcmp(expected, local.data(), 16)) return MeshSendResult::fail(MeshOperationFailure::NotReady); + if (!send_ok) return MeshSendResult::fail(MeshOperationFailure::RadioTxFailed); + sent_bytes.assign(bytes.data, bytes.data + bytes.size); + hash->fill(0x42); + return MeshSendResult::success(1); + } +}; +} // namespace chat diff --git a/modules/core_geocaching/tests/test_attempt_timeout.cpp b/modules/core_geocaching/tests/test_attempt_timeout.cpp new file mode 100644 index 00000000..799649b5 --- /dev/null +++ b/modules/core_geocaching/tests/test_attempt_timeout.cpp @@ -0,0 +1,23 @@ +#include "geocaching/storage/attempt_timeout.h" +int main() +{ + using namespace geocaching::storage; + TxAttemptView attempt; StoredTime now; + attempt.submitted.monotonic_ms = 100; + now.monotonic_ms = 1099; + if (attemptTimeoutReached(attempt, now, 0, 1000)) return 1; + now.monotonic_ms = 1100; + if (!attemptTimeoutReached(attempt, now, 0, 1000)) return 2; + now.boot_id[0] = 1; now.monotonic_ms = 5000; + if (attemptTimeoutReached(attempt, now, 4500, 1000)) return 3; + now.monotonic_ms = 5500; + if (!attemptTimeoutReached(attempt, now, 4500, 1000)) return 4; + attempt.submitted.has_utc = attempt.submitted.utc_trusted = now.has_utc = now.utc_trusted = true; + attempt.submitted.utc_seconds = 100; now.utc_seconds = 101; + if (attemptTimeoutReached(attempt, now, 0, 1001)) return 5; + now.utc_seconds = 102; + if (!attemptTimeoutReached(attempt, now, 0, 1001)) return 6; + attempt.has_finished = true; + if (attemptTimeoutReached(attempt, now, 0, 1000)) return 7; + return 0; +} diff --git a/modules/core_geocaching/tests/test_author_history.cpp b/modules/core_geocaching/tests/test_author_history.cpp new file mode 100644 index 00000000..ea8dffd5 --- /dev/null +++ b/modules/core_geocaching/tests/test_author_history.cpp @@ -0,0 +1,26 @@ +#include "geocaching/storage/author_history.h" +int main() +{ + using namespace geocaching; + uint8_t a[512]{}, b[512]{}, key[36]{}, public_key[64]{}, value[192]{}; + key[35] = 1; + storage::LogicalState state(a, b, sizeof(a)); + RevisionHash hash; size_t length = 0; + if (!storage::encodeAuthorIssued(hash, {public_key, 64}, {}, value, sizeof(value), length)) return 1; + storage::MutationView mutation{3, {key, 36}, {value, length}, false}; + auto apply = [&]() + { + const auto before = state.view(); + return state.apply(&mutation, 1, [&](const auto& candidate) { return storage::validateAuthorHistory(before, candidate); }); + }; + if (!apply() || !apply() || state.view().size() != 1) return 2; + hash.bytes[0] = 1; + if (!storage::encodeAuthorIssued(hash, {public_key, 64}, {}, value, sizeof(value), length)) return 3; + mutation.value = {value, length}; + if (apply() || state.view().size() != 1) return 4; + mutation.erase = true; mutation.value = {}; + if (apply() || state.view().size() != 1) return 5; + mutation.erase = false; mutation.value = {value, length}; key[35] = 2; + if (!apply() || state.view().size() != 2) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_author_issue.cpp b/modules/core_geocaching/tests/test_author_issue.cpp new file mode 100644 index 00000000..34aed054 --- /dev/null +++ b/modules/core_geocaching/tests/test_author_issue.cpp @@ -0,0 +1,107 @@ +#include "geocaching/protocol/cmp_writer.h" +#include "geocaching/usecase/author_issue.h" +#include +#include +#include +#include +struct Port : geocaching::AuthorIssuePort +{ + std::array author{}; + bool reserved = false, reject = false; + unsigned signs = 0; + unsigned pending_steps = 0, cancellations = 0; + bool getAuthorKey(uint8_t out[64]) override + { + std::memcpy(out, author.data(), 64); + return true; + } + geocaching::AuthorReservationResult reserve(geocaching::ByteView, uint8_t*, size_t) override + { + if (pending_steps) + { + --pending_steps; + return geocaching::AuthorReservationResult::Pending; + } + reserved = !reject; + return reserved ? geocaching::AuthorReservationResult::Reserved : geocaching::AuthorReservationResult::Failed; + } + void cancelReservation() override { ++cancellations; } + bool sign(geocaching::ByteView record, uint8_t*, size_t, uint8_t* out, size_t capacity, size_t& written) override + { + ++signs; + if (!reserved) return false; + std::array signature{}; + geocaching::protocol::CmpWriter writer(out, capacity); + if (!writer.array(2) || !writer.binary(record) || !writer.binary({signature.data(), 64})) return false; + written = writer.size(); + return true; + } +}; +int main(int argc, char** argv) +{ + using namespace geocaching; + if (argc != 2) return 1; + std::ifstream file(argv[1], std::ios::binary); + std::vector bytes((std::istreambuf_iterator(file)), {}); + protocol::CmpReader reader({bytes.data(), bytes.size()}); + size_t count = 0; + ByteView record; + if (!reader.array(count, 2) || !reader.binary(record, 4096)) return 2; + RecordView decoded; + if (!protocol::decodeGeocacheRecord(record, decoded)) return 3; + std::vector workspace(record.size + 70); + Port port; + std::memcpy(port.author.data(), decoded.author_public_key.data, 64); + auto issue = std::make_unique(port); + if (!issue->begin(record, workspace.data(), workspace.size())) return 4; + issue->advance(); + issue->advance(); + if (!port.reserved || port.signs || issue->phase() != AuthorIssuePhase::Sign) return 5; + issue->advance(); + if (issue->phase() != AuthorIssuePhase::Signed || !issue->signedRecord().size || port.signs != 1) return 6; + port.signs = 0; + port.reject = true; + auto rejected = std::make_unique(port); + if (!rejected->begin(record, workspace.data(), workspace.size())) return 7; + rejected->advance(); + rejected->advance(); + rejected->advance(); + if (rejected->phase() != AuthorIssuePhase::Failed || port.signs) return 8; + port.reject = false; + auto cancelled = std::make_unique(port); + if (!cancelled->begin(record, workspace.data(), workspace.size())) return 9; + cancelled->advance(); + cancelled->advance(); + cancelled->cancel(); + cancelled->advance(); + if (cancelled->phase() != AuthorIssuePhase::Cancelled || port.signs || cancelled->signedRecord().size) return 10; + port.reserved = false; + port.pending_steps = 2; + auto delayed = std::make_unique(port); + if (!delayed->begin(record, workspace.data(), workspace.size())) return 11; + delayed->advance(); + delayed->advance(); + delayed->advance(); + if (delayed->phase() != AuthorIssuePhase::Reserve || port.signs || port.reserved) return 12; + delayed->advance(); + if (delayed->phase() != AuthorIssuePhase::Sign || port.signs || !port.reserved) return 13; + delayed->advance(); + if (delayed->phase() != AuthorIssuePhase::Signed || port.signs != 1) return 14; + port.pending_steps = 2; + port.signs = 0; + auto pending_cancel = std::make_unique(port); + if (!pending_cancel->begin(record, workspace.data(), workspace.size())) return 15; + pending_cancel->advance(); + pending_cancel->advance(); + pending_cancel->cancel(); + pending_cancel->advance(); + if (pending_cancel->phase() != AuthorIssuePhase::Cancelled || port.cancellations != 1 || port.signs) return 16; + AuthorIssue too_small(port), overlapping(port); + if (too_small.begin(record, workspace.data(), workspace.size() - 1) || + too_small.phase() != AuthorIssuePhase::Idle) return 17; + std::vector aliased(record.size + 70); + std::memcpy(aliased.data(), record.data, record.size); + if (overlapping.begin({aliased.data(), record.size}, aliased.data(), aliased.size()) || + overlapping.phase() != AuthorIssuePhase::Idle) return 18; + return 0; +} diff --git a/modules/core_geocaching/tests/test_author_issued.cpp b/modules/core_geocaching/tests/test_author_issued.cpp new file mode 100644 index 00000000..076440c1 --- /dev/null +++ b/modules/core_geocaching/tests/test_author_issued.cpp @@ -0,0 +1,19 @@ +#include "geocaching/storage/author_issued.h" +int main() +{ + using namespace geocaching; + uint8_t key[36]{}, public_key[64]{}, bytes[192]{}; + key[35] = 1; public_key[0] = 7; + RevisionHash hash; hash.bytes[0] = 9; + storage::StoredTime time; time.monotonic_ms = 123; + size_t size = 0; + if (!storage::encodeAuthorIssued(hash, {public_key, 64}, time, bytes, sizeof(bytes), size)) return 1; + storage::AuthorIssuedView out; + if (!storage::decodeAuthorIssued({key, 36}, {bytes, size}, out) || out.revision != 1 || + out.author_public_key.data[0] != 7 || out.revision_hash.data[0] != 9 || out.issued_at.monotonic_ms != 123) return 2; + key[35] = 0; + if (storage::decodeAuthorIssued({key, 36}, {bytes, size}, out)) return 3; + time.utc_trusted = true; + if (storage::encodeAuthorIssued(hash, {public_key, 64}, time, bytes, sizeof(bytes), size) || size) return 4; + return 0; +} diff --git a/modules/core_geocaching/tests/test_buffered.cpp b/modules/core_geocaching/tests/test_buffered.cpp new file mode 100644 index 00000000..7796a3a1 --- /dev/null +++ b/modules/core_geocaching/tests/test_buffered.cpp @@ -0,0 +1,34 @@ +#include "gps/gpx/buffered_sink.h" +#include +#include +#include +struct Sink : gps::gpx::OutputSink +{ + std::string data; + unsigned writes = 0; + bool fail = false; + bool write(std::string_view bytes) override + { + ++writes; + if (fail) return false; + data.append(bytes); + return true; + } +}; +int main() +{ + Sink sink; + auto buffered = std::make_unique(sink); + for (unsigned i = 0; i < 1100; ++i) assert(buffered->write("abc")); + assert(sink.writes == 3 && sink.data.size() == 3072); + assert(buffered->finish()); + assert(sink.data.size() == 3300 && sink.writes == 4); + for (std::size_t i = 0; i < sink.data.size(); ++i) assert(sink.data[i] == "abc"[i % 3]); + assert(buffered->finish() && sink.writes == 4 && !buffered->write("x")); + Sink bad; + auto failing = std::make_unique(bad); + assert(failing->write("x")); + bad.fail = true; + assert(!failing->finish()); + assert(!failing->write("y") && !failing->finish()); +} diff --git a/modules/core_geocaching/tests/test_cache_head.cpp b/modules/core_geocaching/tests/test_cache_head.cpp new file mode 100644 index 00000000..b33ae67f --- /dev/null +++ b/modules/core_geocaching/tests/test_cache_head.cpp @@ -0,0 +1,21 @@ +#include "geocaching/storage/cache_head.h" +#include "geocaching/protocol/cmp_writer.h" +int main() +{ + using namespace geocaching; + uint8_t key[32]{}, hash[32]{}, value[128]{}; + protocol::CmpWriter writer(value, sizeof(value)); + if (!writer.array(4) || !writer.binary({hash, 32}) || !writer.unsignedInteger(0) || + !writer.unsignedInteger(1) || !writer.unsignedInteger(3)) return 1; + storage::CacheHeadView head; + if (!storage::decodeCacheHead({key, 32}, {value, writer.size()}, head) || head.install_generation != 1 || head.highest_seen_revision != 3) return 2; + for (size_t n = 0; n < writer.size(); ++n) + if (storage::decodeCacheHead({key, 32}, {value, n}, head) || head.current_hash.data) return 3; + uint8_t empty[] = {0x94, 0xc0, 0, 1, 0}; + if (!storage::decodeCacheHead({key, 32}, {empty, sizeof(empty)}, head) || head.current_hash.size) return 4; + empty[3] = 0; + if (storage::decodeCacheHead({key, 32}, {empty, sizeof(empty)}, head)) return 5; + empty[3] = 1; empty[2] = 3; + if (storage::decodeCacheHead({key, 32}, {empty, sizeof(empty)}, head)) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_capabilities.cpp b/modules/core_geocaching/tests/test_capabilities.cpp new file mode 100644 index 00000000..0242485c --- /dev/null +++ b/modules/core_geocaching/tests/test_capabilities.cpp @@ -0,0 +1,31 @@ +#include "geocaching/protocol/capabilities.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 2); + std::ifstream file(argv[1], std::ios::binary); + assert(file.good()); + std::vector bytes((std::istreambuf_iterator(file)), {}); + geocaching::RequestId request; + request.bytes.fill(1); + geocaching::protocol::DirectoryCapabilities result; + using geocaching::protocol::decodeDirectoryCapabilities; + assert(decodeDirectoryCapabilities({bytes.data(), bytes.size()}, request, result)); + assert(result.max_query_items == 64 && result.cursor_ttl_seconds == 604800); + assert(result.name == "Public test directory"); + for (std::size_t n = 0; n < bytes.size(); ++n) + { + assert(!decodeDirectoryCapabilities({bytes.data(), n}, request, result)); + assert(result.name.empty() && result.max_query_items == 0); + } + request.bytes[0] = 2; + assert(!decodeDirectoryCapabilities({bytes.data(), bytes.size()}, request, result)); + request.bytes.fill(1); + // Public role is the byte preceding the uint32 sync TTL. + bytes[bytes.size() - 6] = 0; + assert(!decodeDirectoryCapabilities({bytes.data(), bytes.size()}, request, result)); +} diff --git a/modules/core_geocaching/tests/test_checkpoint.cpp b/modules/core_geocaching/tests/test_checkpoint.cpp new file mode 100644 index 00000000..361b8ea8 --- /dev/null +++ b/modules/core_geocaching/tests/test_checkpoint.cpp @@ -0,0 +1,22 @@ +#include "geocaching/storage/checkpoint.h" +int main() +{ + using namespace geocaching::storage; + uint8_t page[] = {0x92, 0, 0x92, 0x93, 5, 0xc4, 1, 1, 0xc4, 0, 0x93, 5, 0xc4, 1, 2, 0xc4, 0}; + MutationView entries[2]; size_t count = 0; + if (!decodeCheckpointPage({page, sizeof(page)}, 0, entries, 2, count) || count != 2) return 1; + page[14] = 1; + if (decodeCheckpointPage({page, sizeof(page)}, 0, entries, 2, count) || count) return 2; + page[14] = 2; + const uint8_t prior_key = 3; + MutationView prior{5, {&prior_key, 1}, {}, false}; + if (decodeCheckpointPage({page, sizeof(page)}, 0, entries, 2, count, &prior)) return 3; + if (decodeCheckpointPage({page, sizeof(page)}, 1, entries, 2, count)) return 4; + for (size_t n = 0; n < sizeof(page); ++n) + if (decodeCheckpointPage({page, n}, 0, entries, 2, count)) return 5; + uint8_t tail[37] = {0x93, 1, 2, 0xc4, 32}; + CheckpointTailView decoded; + if (!decodeCheckpointTail({tail, sizeof(tail)}, decoded) || decoded.page_count != 1 || decoded.entry_count != 2 || decoded.digest.size != 32) return 6; + if (decodeCheckpointTail({tail, sizeof(tail) - 1}, decoded) || decoded.digest.data) return 7; + return 0; +} diff --git a/modules/core_geocaching/tests/test_checkpoint_selection.cpp b/modules/core_geocaching/tests/test_checkpoint_selection.cpp new file mode 100644 index 00000000..1ac35a53 --- /dev/null +++ b/modules/core_geocaching/tests/test_checkpoint_selection.cpp @@ -0,0 +1,22 @@ +#include "geocaching/storage/checkpoint_selection.h" +int main() +{ + using namespace geocaching::storage; + CheckpointCandidate a, b; + if (selectCheckpoint(a, b) != CheckpointChoice::RetryLater) return 1; + a.state = b.state = CheckpointCandidateState::Missing; + if (selectCheckpoint(a, b) != CheckpointChoice::NoCheckpoint) return 2; + a.state = CheckpointCandidateState::Invalid; + if (selectCheckpoint(a, b) != CheckpointChoice::Corrupt) return 3; + b.state = CheckpointCandidateState::Verified; b.sequence = 4; + if (selectCheckpoint(a, b) != CheckpointChoice::SlotB) return 4; + a.state = CheckpointCandidateState::Verified; a.sequence = 5; + if (selectCheckpoint(a, b) != CheckpointChoice::SlotA) return 5; + a.sequence = 4; + if (selectCheckpoint(a, b) != CheckpointChoice::SlotA) return 6; + a.digest[0] = 1; + if (selectCheckpoint(a, b) != CheckpointChoice::Corrupt) return 7; + a.state = CheckpointCandidateState::Unavailable; + if (selectCheckpoint(a, b) != CheckpointChoice::RetryLater) return 8; + return 0; +} diff --git a/modules/core_geocaching/tests/test_checkpoint_verifier.cpp b/modules/core_geocaching/tests/test_checkpoint_verifier.cpp new file mode 100644 index 00000000..86fe2eb4 --- /dev/null +++ b/modules/core_geocaching/tests/test_checkpoint_verifier.cpp @@ -0,0 +1,37 @@ +#include "geocaching/storage/checkpoint_verifier.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include +struct Digest +{ + std::vector bytes; + void update(const uint8_t* data, size_t size) { bytes.insert(bytes.end(), data, data + size); } + bool finalize(uint8_t* output, size_t) { chat::reticulum::fullHash(bytes.data(), bytes.size(), output); return true; } +}; +std::vector wrap(geocaching::storage::RecordKind kind, geocaching::ByteView payload) +{ + geocaching::storage::RecordHeader header; + if (!geocaching::storage::makeRecordHeader(kind, 7, payload, header)) return {}; + std::vector result(header.begin(), header.end()); + result.insert(result.end(), payload.data, payload.data + payload.size); return result; +} +int main() +{ + using namespace geocaching::storage; + const uint8_t page_data[] = {0x92, 0, 0x91, 0x93, 5, 0xc4, 1, 1, 0xc4, 0}; + auto page = wrap(RecordKind::CheckpointPage, {page_data, sizeof(page_data)}); + uint8_t tail_data[37] = {0x93, 1, 1, 0xc4, 32}; + chat::reticulum::fullHash(page.data(), page.size(), tail_data + 5); + auto tail = wrap(RecordKind::CheckpointTail, {tail_data, sizeof(tail_data)}); + Digest digest; CheckpointVerifier verifier(digest); MutationView entry; size_t count = 0; + if (!verifier.accept({page.data(), page.size()}, &entry, 1, count) || count != 1 || verifier.verified() || + !verifier.accept({tail.data(), tail.size()}, &entry, 1, count) || verifier.verified() || + !verifier.finish() || verifier.sequence() != 7) return 1; + if (verifier.accept({page.data(), page.size()}, &entry, 1, count) || verifier.verified()) return 2; + Digest missing_digest; CheckpointVerifier missing(missing_digest); + if (!missing.accept({page.data(), page.size()}, &entry, 1, count) || missing.finish()) return 3; + tail_data[5] ^= 1; + tail = wrap(RecordKind::CheckpointTail, {tail_data, sizeof(tail_data)}); + Digest bad_digest; CheckpointVerifier bad(bad_digest); + if (!bad.accept({page.data(), page.size()}, &entry, 1, count) || bad.accept({tail.data(), tail.size()}, &entry, 1, count)) return 4; + return 0; +} diff --git a/modules/core_geocaching/tests/test_cmp.cpp b/modules/core_geocaching/tests/test_cmp.cpp new file mode 100644 index 00000000..43ecc6ed --- /dev/null +++ b/modules/core_geocaching/tests/test_cmp.cpp @@ -0,0 +1,33 @@ +#include "geocaching/protocol/cmp_reader.h" +#include + +int main() +{ + using geocaching::protocol::CmpReader; + const std::uint8_t valid[] = {0xcc, 0x80, 0xcd, 1, 0, 0xd0, 0xdf, + 0xc4, 2, 0xaa, 0xbb, 0xc0}; + CmpReader reader({valid, sizeof(valid)}); + std::uint64_t u = 0; + std::int64_t s = 0; + geocaching::ByteView bytes; + assert(reader.unsignedInteger(u) && u == 128); + assert(reader.unsignedInteger(u) && u == 256); + assert(reader.signedInteger(s) && s == -33); + assert(reader.binary(bytes, 2) && bytes.size == 2 && bytes.data[1] == 0xbb); + assert(reader.nil() && reader.finished()); + const std::uint8_t redundant[] = {0xcc, 0x7f, 1}; + CmpReader bad({redundant, sizeof(redundant)}); + assert(!bad.unsignedInteger(u)); + assert(!bad.unsignedInteger(u) && !bad.finished()); + const std::uint8_t truncated[] = {0xc5, 1}; + CmpReader short_input({truncated, sizeof(truncated)}); + assert(!short_input.binary(bytes, 4096)); + const std::uint8_t positive_signed[] = {0xd0, 1}; + CmpReader wrong({positive_signed, sizeof(positive_signed)}); + assert(!wrong.signedInteger(s)); + const std::uint8_t minimum[] = {0xd3, 0x80, 0, 0, 0, 0, 0, 0, 0}; + CmpReader min_reader({minimum, sizeof(minimum)}); + assert(min_reader.signedInteger(s) && s == INT64_MIN && min_reader.finished()); + CmpReader null_reader({nullptr, 1}); + assert(!null_reader.unsignedInteger(u)); +} diff --git a/modules/core_geocaching/tests/test_crc.cpp b/modules/core_geocaching/tests/test_crc.cpp new file mode 100644 index 00000000..de93cc67 --- /dev/null +++ b/modules/core_geocaching/tests/test_crc.cpp @@ -0,0 +1,10 @@ +#include "sys/crc32.h" + +int main() +{ + const char vector[] = "123456789"; + if (sys::crc32(vector, 9) != 0xcbf43926U || sys::crc32(nullptr, 0) != 0) return 1; + for (unsigned split = 0; split <= 9; ++split) + if (sys::crc32(vector + split, 9 - split, sys::crc32(vector, split)) != 0xcbf43926U) return 2; + return 0; +} diff --git a/modules/core_geocaching/tests/test_crypto_pipeline.cpp b/modules/core_geocaching/tests/test_crypto_pipeline.cpp new file mode 100644 index 00000000..6134d6de --- /dev/null +++ b/modules/core_geocaching/tests/test_crypto_pipeline.cpp @@ -0,0 +1,38 @@ +#include "geocaching/protocol/verify_record.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include +#include +#include +#include + +struct NativeCrypto : geocaching::protocol::RecordCrypto +{ + bool sha256(geocaching::ByteView in, std::uint8_t out[32]) override + { chat::reticulum::fullHash(in.data, in.size, out); return true; } + geocaching::protocol::VerificationResult verifyEd25519(geocaching::ByteView key, geocaching::ByteView sig, geocaching::ByteView msg) override + { + using R = geocaching::protocol::VerificationResult; + return ed25519_verify(sig.data, msg.data, msg.size, key.data) ? R::Valid : R::InvalidSignature; + } +}; +int main(int argc, char** argv) +{ + assert(argc == 3); + std::ifstream f(argv[1], std::ios::binary), metadata(argv[2], std::ios::binary); + assert(f.good() && metadata.good()); + std::vector data((std::istreambuf_iterator(f)), {}), workspace(4200); + geocaching::GeocacheId id; + geocaching::RevisionHash hash; + metadata.read(reinterpret_cast(id.bytes.data()), 32); + metadata.read(reinterpret_cast(hash.bytes.data()), 32); + assert(metadata.good()); + NativeCrypto crypto; + geocaching::protocol::VerifiedRecordView out; + using namespace geocaching::protocol; + assert(verifyGeocache({data.data(), data.size()}, crypto, workspace.data(), workspace.size(), out, &id, &hash) == VerificationResult::Valid); + hash.bytes[0] ^= 1; + assert(verifyGeocache({data.data(), data.size()}, crypto, workspace.data(), workspace.size(), out, &id, &hash) == VerificationResult::IdentityMismatch); + data.back() ^= 1; + assert(verifyGeocache({data.data(), data.size()}, crypto, workspace.data(), workspace.size(), out) == VerificationResult::InvalidSignature); +} diff --git a/modules/core_geocaching/tests/test_custom_extract.cpp b/modules/core_geocaching/tests/test_custom_extract.cpp new file mode 100644 index 00000000..4afdbdc7 --- /dev/null +++ b/modules/core_geocaching/tests/test_custom_extract.cpp @@ -0,0 +1,21 @@ +#include "chat/infra/lxmf/lxmf_wire.h" +#include + +int main() +{ + using namespace chat::lxmf; + DecodedTextPayload payload; + ByteSpan type, data; + assert(extractCustomData(payload, &type, &data) == CustomDataResult::NotCustom); + payload.fields.push_back({kFieldCustomType, {0xa1, 'x'}}); + assert(extractCustomData(payload, &type, &data) == CustomDataResult::Invalid); + payload.fields.push_back({kFieldCustomData, {0xc4, 1, 42}}); + assert(extractCustomData(payload, &type, &data) == CustomDataResult::Valid); + assert(type.size == 1 && data.size == 1 && data.data[0] == 42); + payload.fields.push_back(payload.fields[0]); + assert(extractCustomData(payload, &type, &data) == CustomDataResult::Invalid); + assert(!type.data && !data.data); + payload.fields.pop_back(); + payload.fields[1].encoded_value = {0xa1, 'x'}; + assert(extractCustomData(payload, &type, &data) == CustomDataResult::Invalid); +} diff --git a/modules/core_geocaching/tests/test_directory_pool.cpp b/modules/core_geocaching/tests/test_directory_pool.cpp new file mode 100644 index 00000000..fa31b7ca --- /dev/null +++ b/modules/core_geocaching/tests/test_directory_pool.cpp @@ -0,0 +1,37 @@ +#include "geocaching/usecase/directory_pool.h" +#include +#include + +int main() +{ + std::array entries{}; + auto pool = std::make_unique(entries.data(), entries.size()); + std::array key{}; + geocaching::Destination discovery; + geocaching::protocol::DirectoryAnnouncement announce; + announce.name = "Public"; + announce.sequence = 5; + assert(pool->next(0, false) == nullptr); + for (unsigned i = 0; i < 32; ++i) + { + discovery.bytes[0] = static_cast(i); + announce.delivery.bytes[0] = static_cast(i); + assert(pool->observe(discovery, announce, {key.data(), key.size()}, 1)); + } + assert(pool->size() == 32); + for (unsigned i = 0; i < 32; ++i) + { + auto* entry = pool->next(1, false); + assert(entry && entry->discovery.bytes[0] == i); + entry->capabilities_verified = true; + } + assert(pool->next(1, false) == nullptr); + auto* ready = pool->next(1, true); + assert(ready && ready->sequence == 5); + ready->in_flight = true; + assert(pool->next(1, true) != ready); + discovery.bytes[0] = 32; + announce.delivery.bytes[0] = 32; + assert(!pool->observe(discovery, announce, {key.data(), key.size()}, 2)); + assert(pool->size() == 32); +} diff --git a/modules/core_geocaching/tests/test_discovery.cpp b/modules/core_geocaching/tests/test_discovery.cpp new file mode 100644 index 00000000..1c9e2b11 --- /dev/null +++ b/modules/core_geocaching/tests/test_discovery.cpp @@ -0,0 +1,31 @@ +#include "geocaching/protocol/discovery.h" +#include +#include + +int main() +{ + // [1, bin16(0x11...), bin16(0x22...), uint64_max, "Node"] + std::vector data{0x95, 1, 0xc4, 16}; + data.insert(data.end(), 16, 0x11); + data.insert(data.end(), {0xc4, 16}); + data.insert(data.end(), 16, 0x22); + data.push_back(0xcf); + data.insert(data.end(), 8, 0xff); + data.insert(data.end(), {0xa4, 'N', 'o', 'd', 'e'}); + geocaching::Destination expected; + expected.bytes.fill(0x11); + geocaching::protocol::DirectoryAnnouncement out; + using geocaching::protocol::decodeDirectoryAnnouncement; + assert(decodeDirectoryAnnouncement({data.data(), data.size()}, expected, out)); + assert(out.sequence == UINT64_MAX && out.epoch[0] == 0x22 && out.name == "Node"); + for (std::size_t n = 0; n < data.size(); ++n) + { + assert(!decodeDirectoryAnnouncement({data.data(), n}, expected, out)); + assert(out.name.empty() && out.sequence == 0); + } + expected.bytes[0] = 0x33; + assert(!decodeDirectoryAnnouncement({data.data(), data.size()}, expected, out)); + expected.bytes.fill(0x11); + data.push_back(0); + assert(!decodeDirectoryAnnouncement({data.data(), data.size()}, expected, out)); +} diff --git a/modules/core_geocaching/tests/test_download.cpp b/modules/core_geocaching/tests/test_download.cpp new file mode 100644 index 00000000..da35c934 --- /dev/null +++ b/modules/core_geocaching/tests/test_download.cpp @@ -0,0 +1,93 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include "geocaching/usecase/download_client.h" +#include +#include +#include +#include +#include +struct Crypto : geocaching::protocol::RecordCrypto +{ + bool sha256(geocaching::ByteView in, std::uint8_t out[32]) override + { + chat::reticulum::fullHash(in.data, in.size, out); + return true; + } + geocaching::protocol::VerificationResult verifyEd25519(geocaching::ByteView k, geocaching::ByteView s, geocaching::ByteView m) override + { + using R = geocaching::protocol::VerificationResult; + return ed25519_verify(s.data, m.data, m.size, k.data) ? R::Valid : R::InvalidSignature; + } +}; +struct Port : geocaching::DownloadPort +{ + using Result = geocaching::DownloadOperationResult; + unsigned commits = 0; + Result submission = Result::Complete, installation = Result::Rejected, polled = Result::Pending; + Result submit(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView r) override + { + return r.size == 95 ? submission : Result::Rejected; + } + Result commit(const geocaching::Destination&, const geocaching::RequestId&, uint64_t generation, geocaching::ByteView, + const geocaching::protocol::VerifiedRecordView& value) override + { + if (generation != 1 || value.record.name != "Test") return Result::Rejected; + if (installation != Result::Rejected) ++commits; + return installation; + } + Result poll() override { return polled; } + Result cancel(const geocaching::Destination&, const geocaching::RequestId&, uint64_t) override { return Result::Pending; } +}; +int main(int argc, char** argv) +{ + using namespace geocaching; + if (argc != 3) return 1; + std::vector data[2]; + for (int i = 0; i < 2; ++i) + { + std::ifstream file(argv[i + 1], std::ios::binary); + if (!file.good()) return 2; + data[i].assign(std::istreambuf_iterator(file), {}); + } + protocol::SummaryView summary; + protocol::QueryPageView page; + RequestId id; + id.bytes.fill(3); + if (!protocol::decodeQueryResponse({data[0].data(), data[0].size()}, id, 2048, &summary, 1, page)) return 3; + id.bytes.fill(4); + Destination source; + Crypto crypto; + Port port; + std::vector scratch(summary.signed_bytes + 64); + auto client = std::make_unique(port, crypto); + const ByteView response{data[1].data(), data[1].size()}; + if (!client->begin(source, id, summary, 1)) return 4; + if (client->accept(source, response, scratch.data(), scratch.size()) || client->phase() != DownloadPhase::Waiting || port.commits) return 5; + port.installation = DownloadOperationResult::Pending; + if (client->accept(source, response, data[1].data(), data[1].size()) || port.commits) return 6; + if (client->accept(source, response, scratch.data(), 1) || port.commits) return 7; + if (client->accept(source, response, scratch.data(), scratch.size()) || client->phase() != DownloadPhase::Installing || port.commits != 1) return 8; + std::fill(scratch.begin(), scratch.end(), 0xa5); + client->advance(); + if (client->phase() != DownloadPhase::Installing) return 9; + port.polled = DownloadOperationResult::Complete; + client->advance(); + if (client->phase() != DownloadPhase::Stored || client->cancel()) return 10; + + Port delayed; + delayed.submission = DownloadOperationResult::Pending; + DownloadClient pending(delayed, crypto); + if (!pending.begin(source, id, summary, 1) || pending.phase() != DownloadPhase::Submitting || + pending.accept(source, response, scratch.data(), scratch.size())) return 11; + delayed.polled = DownloadOperationResult::Complete; + pending.advance(); + if (pending.phase() != DownloadPhase::Waiting || !pending.cancel() || pending.phase() != DownloadPhase::Cancelling) return 12; + if (pending.accept(source, response, scratch.data(), scratch.size())) return 13; + delayed.polled = DownloadOperationResult::Pending; + pending.advance(); + if (pending.phase() != DownloadPhase::Cancelling) return 14; + delayed.polled = DownloadOperationResult::Complete; + pending.advance(); + if (pending.phase() != DownloadPhase::Cancelled) return 15; + return 0; +} diff --git a/modules/core_geocaching/tests/test_download_task_record.cpp b/modules/core_geocaching/tests/test_download_task_record.cpp new file mode 100644 index 00000000..4f16a279 --- /dev/null +++ b/modules/core_geocaching/tests/test_download_task_record.cpp @@ -0,0 +1,28 @@ +#include "geocaching/storage/queued_request.h" +#include +int main() +{ + using namespace geocaching; + RequestId request_id; Destination local, remote; GeocacheId cache; RevisionHash hash; + std::array task{}; + uint8_t request[256]{}; size_t request_size = 0; + if (!protocol::encodeGetRequest(request_id, cache, &hash, nullptr, 8192, request, sizeof(request), request_size)) return 1; + std::array value_buffer{}; + auto workspace = std::make_unique(value_buffer.data(), value_buffer.size()); + std::array output{}; size_t size = 0; + storage::RequestTaskTarget target{{cache.bytes.data(), 32}, {hash.bytes.data(), 32}, 1}; + if (!storage::encodeNewRequestTask(0, local, remote, request_id, task, 2, {request, request_size}, {}, + *workspace, output.data(), output.size(), size, target)) return 2; + storage::MutationView mutations[2]; storage::TransactionView transaction; + if (!storage::decodeTransaction({output.data(), size}, 0, mutations, 2, transaction)) return 3; + protocol::CmpReader record(mutations[1].value); size_t fields = 0; uint64_t kind = 0; ByteView stored_id, stored_hash; + if (!record.array(fields, 6) || fields != 6 || !record.unsignedInteger(kind) || kind != 2 || + !record.binary(stored_id, 32) || stored_id.size != 32 || !record.binary(stored_hash, 32) || stored_hash.size != 32) return 4; + target.install_generation = 0; + if (storage::encodeNewRequestTask(0, local, remote, request_id, task, 2, {request, request_size}, {}, + *workspace, output.data(), output.size(), size, target)) return 5; + target.install_generation = 1; + if (storage::encodeNewRequestTask(0, local, remote, request_id, task, 1, {request, request_size}, {}, + *workspace, output.data(), output.size(), size, target)) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_ed25519.cpp b/modules/core_geocaching/tests/test_ed25519.cpp new file mode 100644 index 00000000..46b381e1 --- /dev/null +++ b/modules/core_geocaching/tests/test_ed25519.cpp @@ -0,0 +1,31 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "geocaching/protocol/cmp_reader.h" +#include +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 2); + std::ifstream file(argv[1], std::ios::binary); + assert(file.good()); + std::vector bytes((std::istreambuf_iterator(file)), {}); + geocaching::protocol::CmpReader reader({bytes.data(), bytes.size()}); + std::size_t count = 0; + geocaching::ByteView record, signature; + assert(reader.array(count, 2) && count == 2); + assert(reader.binary(record, 4096) && reader.binary(signature, 64) && reader.finished()); + geocaching::protocol::CmpReader fields(record); + std::uint64_t schema = 0; + geocaching::ByteView key; + assert(fields.array(count, 16) && fields.unsignedInteger(schema) && fields.binary(key, 64)); + constexpr char domain[] = "trailmate.geocache/sign/v1"; + std::vector message(sizeof(domain) + record.size); + std::memcpy(message.data(), domain, sizeof(domain)); + std::memcpy(message.data() + sizeof(domain), record.data, record.size); + assert(ed25519_verify(signature.data, message.data(), message.size(), key.data + 32) == 1); + message.back() ^= 1; + assert(ed25519_verify(signature.data, message.data(), message.size(), key.data + 32) == 0); +} diff --git a/modules/core_geocaching/tests/test_generated_state.cpp b/modules/core_geocaching/tests/test_generated_state.cpp new file mode 100644 index 00000000..13fefe52 --- /dev/null +++ b/modules/core_geocaching/tests/test_generated_state.cpp @@ -0,0 +1,68 @@ +#include "geocaching/storage/logical_state.h" +#include + +int main() +{ + using namespace geocaching; + using namespace geocaching::storage; + std::array first{}, second{}; + LogicalState state(first.data(), second.data(), first.size()); + const uint8_t key = 1, other_key = 2, old_bytes[] = {7, 8, 9}; + const MutationView original{5, {&key, 1}, {old_bytes, 3}, false}; + const auto valid = [](const auto&) + { return true; }; + if (!state.apply(&original, 1, valid)) return 1; + const MutationView updates[] = {{5, {&key, 1}, {}, false}, {10, {&other_key, 1}, {old_bytes, 3}, false}}; + unsigned generated = 0; + const auto encode = [&](uint8_t* out, size_t capacity, size_t& written) + { + ++generated; + if (capacity < 6) return false; + std::memcpy(out, "abcdef", 6); + written = 6; + return true; + }; + ByteView value; + const auto unchanged = [&]() + { + return state.view().size() == 1 && state.view().find(5, {&key, 1}, value) && + value.size == 3 && !std::memcmp(value.data, old_bytes, 3); + }; + if (!state.prepareGenerated(updates, 2, 0, encode, valid) || generated != 1 || !unchanged()) return 2; + if (!state.preparedView().find(5, {&key, 1}, value) || value.size != 6 || std::memcmp(value.data, "abcdef", 6)) return 3; + if (state.withScratch([](uint8_t*, size_t) {}) || state.beginSnapshot() || state.prepare(updates, 2, valid)) return 4; + state.discardPrepared(); + if (!unchanged()) return 5; + if (state.prepareGenerated( + updates, 2, 0, [](uint8_t* out, size_t, size_t& n) + { + out[0] = 0xff; n = 1; return false; }, + valid) || + !unchanged()) return 6; + if (state.prepareGenerated( + updates, 2, 0, [](uint8_t*, size_t capacity, size_t& n) + { + n = capacity + 1; return true; }, + valid) || + !unchanged()) return 7; + if (state.prepareGenerated(updates, 2, 0, encode, [](const auto&) + { return false; }) || + !unchanged()) return 8; + // Exhaust remaining capacity after a generated value, never publish a prefix. + if (state.prepareGenerated( + updates, 2, 0, [](uint8_t* out, size_t capacity, size_t& n) + { + std::memset(out, 1, capacity); n = capacity; return true; }, + valid) || + !unchanged()) return 9; + if (!state.withScratch([](uint8_t* out, size_t n) + { std::memset(out, 0xa5, n); }) || + !unchanged()) return 10; + if (!state.prepareGenerated(updates, 2, 0, encode, valid) || !state.commitPrepared() || state.view().size() != 2 || + !state.view().find(5, {&key, 1}, value) || value.size != 6 || std::memcmp(value.data, "abcdef", 6)) return 11; + // Existing keys and a newly inserted generated key both use the same path. + const uint8_t new_key = 3; + const MutationView insert{5, {&new_key, 1}, {}, false}; + if (!state.prepareGenerated(&insert, 1, 0, encode, valid) || !state.commitPrepared() || state.view().size() != 3) return 12; + return 0; +} diff --git a/modules/core_geocaching/tests/test_get_response.cpp b/modules/core_geocaching/tests/test_get_response.cpp new file mode 100644 index 00000000..1a011522 --- /dev/null +++ b/modules/core_geocaching/tests/test_get_response.cpp @@ -0,0 +1,32 @@ +#include "geocaching/protocol/get_response.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 3); + std::ifstream response_file(argv[1], std::ios::binary), signed_file(argv[2], std::ios::binary); + assert(response_file.good() && signed_file.good()); + std::vector response((std::istreambuf_iterator(response_file)), {}); + std::vector signed_cache((std::istreambuf_iterator(signed_file)), {}); + geocaching::RequestId id; + id.bytes.fill(4); + geocaching::protocol::GetResponseView out; + using geocaching::protocol::decodeGetResponse; + assert(decodeGetResponse({response.data(), response.size()}, id, 8192, out)); + assert(out.is_current && !out.has_conflict); + assert(out.signed_cache.size == signed_cache.size()); + assert(std::memcmp(out.signed_cache.data, signed_cache.data(), signed_cache.size()) == 0); + for (std::size_t n = 0; n < response.size(); ++n) + { + assert(!decodeGetResponse({response.data(), n}, id, 8192, out)); + assert(out.signed_cache.data == nullptr); + } + response.back() = 2; + assert(!decodeGetResponse({response.data(), response.size()}, id, 8192, out)); + response.back() = 0; + id.bytes[0] = 5; + assert(!decodeGetResponse({response.data(), response.size()}, id, 8192, out)); +} diff --git a/modules/core_geocaching/tests/test_gpx_attributes.cpp b/modules/core_geocaching/tests/test_gpx_attributes.cpp new file mode 100644 index 00000000..17fe9432 --- /dev/null +++ b/modules/core_geocaching/tests/test_gpx_attributes.cpp @@ -0,0 +1,23 @@ +#include "gps/gpx/attribute_reader.h" +#include + +int main() +{ + double value = 42; + const auto reject = [&value](const char* text) + { + value = 42; + return !gps::gpx::readDoubleAttribute(text, "lat", value) && value == 42; + }; + if (!gps::gpx::readDoubleAttribute("", "lat", value) || value != 12.5 || + !gps::gpx::readDoubleAttribute("", "lat", value) || value != -12.25 || + !reject("") || !reject("") || + !reject("") || !reject("") || + !reject("") || !reject("") || !reject("")) + { + std::cerr << "GPX attribute regression\n"; + return 1; + } + return 0; +} diff --git a/modules/core_geocaching/tests/test_gpx_hash.cpp b/modules/core_geocaching/tests/test_gpx_hash.cpp new file mode 100644 index 00000000..15af0f3a --- /dev/null +++ b/modules/core_geocaching/tests/test_gpx_hash.cpp @@ -0,0 +1,42 @@ +#include "platform/esp/arduino_common/geocaching/sd_gpx_hash.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include +#include +std::vector bytes(1025, 7); +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeFile::Impl { public: bool open = false; size_t offset = 0; }; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char*, const char*) { impl_->open = true; impl_->offset = 0; return true; } +void SdRuntimeFile::close() { impl_->open = false; } +bool SdRuntimeFile::is_open() const { return impl_->open; } +uint64_t SdRuntimeFile::size() const { return bytes.size(); } +int SdRuntimeFile::read(void* out, size_t count) +{ + const auto n = std::min(count, bytes.size() - impl_->offset); + std::memcpy(out, bytes.data() + impl_->offset, n); impl_->offset += n; return static_cast(n); +} +} +struct Digest +{ + std::vector data; + void update(const uint8_t* bytes, size_t size) { data.insert(data.end(), bytes, bytes + size); } + bool finalize(uint8_t* out, size_t) { chat::reticulum::fullHash(data.data(), data.size(), out); return true; } +}; +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + Digest digest; SdGpxHash reader(digest); + std::array actual, expected; + chat::reticulum::fullHash(bytes.data(), bytes.size(), expected.data()); + if (!reader.open("local.gpx") || reader.result(actual)) return 1; + for (unsigned i = 0; i < 3; ++i) if (reader.step() != GpxHashStep::Reading) return 2; + if (reader.step() != GpxHashStep::Complete || !reader.result(actual) || actual != expected) return 3; + Digest growing_digest; SdGpxHash growing(growing_digest); + if (!growing.open("local.gpx")) return 4; + for (unsigned i = 0; i < 3; ++i) if (growing.step() != GpxHashStep::Reading) return 5; + bytes.push_back(9); + if (growing.step() != GpxHashStep::IoError || growing.result(actual)) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_gpx_record.cpp b/modules/core_geocaching/tests/test_gpx_record.cpp new file mode 100644 index 00000000..662eaf43 --- /dev/null +++ b/modules/core_geocaching/tests/test_gpx_record.cpp @@ -0,0 +1,42 @@ +#include "geocaching/gpx/write_record.h" +#include +#include +#include +#include + +struct FileSink : geocaching::gpx::OutputSink +{ + std::ofstream file; + explicit FileSink(const char* path) : file(path, std::ios::binary) {} + bool write(std::string_view bytes) override { file.write(bytes.data(), bytes.size()); return file.good(); } +}; +struct FixtureCrypto : geocaching::protocol::RecordCrypto +{ + std::array author_hash{}; + bool sha256(geocaching::ByteView input, std::uint8_t output[32]) override + { + if (input.size != 64) return false; + std::memcpy(output, author_hash.data(), 32); return true; + } + geocaching::protocol::VerificationResult verifyEd25519(geocaching::ByteView, geocaching::ByteView, geocaching::ByteView) override + { return geocaching::protocol::VerificationResult::CryptoUnavailable; } +}; +int main(int argc, char** argv) +{ + assert(argc == 4); + std::ifstream f(argv[1], std::ios::binary), metadata(argv[2], std::ios::binary); + assert(f.good() && metadata.good()); + std::vector bytes((std::istreambuf_iterator(f)), {}); + geocaching::protocol::CmpReader reader({bytes.data(), bytes.size()}); + std::size_t count = 0; + geocaching::ByteView record; + geocaching::protocol::VerifiedRecordView verified; + assert(reader.array(count, 2) && reader.binary(record, 4096) && reader.binary(verified.signature, 64)); + assert(geocaching::protocol::decodeGeocacheRecord(record, verified.record)); + metadata.read(reinterpret_cast(verified.id.bytes.data()), 32); + FixtureCrypto crypto; + metadata.read(reinterpret_cast(crypto.author_hash.data()), 32); + assert(metadata.good()); + FileSink sink(argv[3]); + assert(geocaching::gpx::writeGeocacheGpx(verified, crypto, sink)); +} diff --git a/modules/core_geocaching/tests/test_gpx_text.cpp b/modules/core_geocaching/tests/test_gpx_text.cpp new file mode 100644 index 00000000..e2a0f46a --- /dev/null +++ b/modules/core_geocaching/tests/test_gpx_text.cpp @@ -0,0 +1,42 @@ +#include "gps/gpx/text_writer.h" +#include +#include + +struct Sink : gps::gpx::OutputSink +{ + std::string text; + bool fail = false; + bool write(std::string_view bytes) override + { + if (fail) return false; + text.append(bytes); + return true; + } +}; +int main() +{ + using gps::gpx::GpxTextWriter; + Sink sink; + GpxTextWriter writer(sink); + assert(writer.escaped("<&>\"'\n")); + assert(sink.text == "<&>"'\n"); + sink.text.clear(); + assert(writer.coordinate(-1)); + assert(sink.text == "-0.0000001"); + sink.text.clear(); + assert(writer.coordinate(1201500000)); + assert(sink.text == "120.1500000"); + const std::uint8_t data[] = {'f', 'o', 'o'}; + for (std::size_t n = 1; n <= 3; ++n) + { + sink.text.clear(); + assert(writer.base64({data, n})); + assert(sink.text == (n == 1 ? "Zg==" : n == 2 ? "Zm8=" + : "Zm9v")); + } + GpxTextWriter limited(sink, 3); + assert(!limited.raw("four") && !limited.good()); + assert(!limited.raw("x")); + sink.fail = true; + assert(!writer.raw("x") && !writer.good()); +} diff --git a/modules/core_geocaching/tests/test_install.cpp b/modules/core_geocaching/tests/test_install.cpp new file mode 100644 index 00000000..473dbf53 --- /dev/null +++ b/modules/core_geocaching/tests/test_install.cpp @@ -0,0 +1,52 @@ +#include "geocaching/usecase/gpx_install.h" +#include +#include + +struct Port : geocaching::GpxInstallPort +{ + std::vector calls; + geocaching::InstallStep stop_at = geocaching::InstallStep::Complete; + geocaching::InstallEffectResult stop_result = geocaching::InstallEffectResult::Failed; + geocaching::InstallEffectResult execute(geocaching::InstallStep step, const geocaching::InstallIdentity&) override + { + calls.push_back(step); + return step == stop_at ? stop_result : geocaching::InstallEffectResult::Complete; + } +}; +int main() +{ + using namespace geocaching; + InstallIdentity identity; + identity.generation = 1; + GpxInstall install(identity); + Port port; + for (unsigned i = 0; i < 6; ++i) + { + assert(!install.installed()); + install.advance(port); + } + assert(install.installed()); + install.advance(port); + assert(port.calls.size() == 6); + for (unsigned i = 0; i < 6; ++i) + { + Port failure; + failure.stop_at = static_cast(i); + GpxInstall rejected(identity); + for (unsigned j = 0; j < 8; ++j) rejected.advance(failure); + assert(!rejected.installed() && rejected.step() == InstallStep::Failed); + assert(failure.calls.size() == i + 1); + } + Port cancelled; + cancelled.stop_at = InstallStep::ReplaceFile; + cancelled.stop_result = InstallEffectResult::StaleIntent; + GpxInstall stale(identity); + for (unsigned i = 0; i < 8; ++i) stale.advance(cancelled); + assert(stale.step() == InstallStep::Cancelled && !stale.installed()); + Port pending; + pending.stop_at = InstallStep::CheckIntent; + pending.stop_result = InstallEffectResult::Pending; + GpxInstall waiting(identity); + waiting.advance(pending); + assert(waiting.step() == InstallStep::CheckIntent); +} diff --git a/modules/core_geocaching/tests/test_install_record.cpp b/modules/core_geocaching/tests/test_install_record.cpp new file mode 100644 index 00000000..ffcb7314 --- /dev/null +++ b/modules/core_geocaching/tests/test_install_record.cpp @@ -0,0 +1,29 @@ +#include "geocaching/storage/install_record.h" +#include "geocaching/protocol/cmp_writer.h" +int main() +{ + using namespace geocaching; + uint8_t key[16]{}, cache[32]{}, revision[32]{}, file_hash[32]{}, bytes[256]{}; + revision[0] = 1; file_hash[0] = 2; + protocol::CmpWriter writer(bytes, sizeof(bytes)); + if (!writer.array(6) || !writer.binary({cache, 32}) || !writer.binary({revision, 32}) || !writer.binary({file_hash, 32}) || + !writer.nil() || !writer.unsignedInteger(1) || !writer.unsignedInteger(0)) return 1; + storage::InstallRecordView record; + if (!storage::decodeInstallRecord({key, 16}, {bytes, writer.size()}, record) || record.new_file_hash.data[0] != 2) return 2; + storage::CacheHeadView head; head.install_generation = 1; + using Action = storage::InstallRecoveryAction; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::VerifyFiles) return 3; + head.install_generation = 2; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::Superseded) return 4; + head.install_generation = 1; record.generation = 2; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::Inconsistent) return 5; + record.generation = 1; record.phase = storage::InstallPhase::ConflictExternal; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::ExternalConflict) return 6; + record.phase = storage::InstallPhase::Installed; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::Inconsistent) return 7; + head.current_hash = {revision, 32}; + if (storage::classifyInstallRecovery({cache, 32}, head, record) != Action::VerifyFiles) return 8; + for (size_t n = 0; n < writer.size(); ++n) + if (storage::decodeInstallRecord({key, 16}, {bytes, n}, record) || record.cache_id.data) return 9; + return 0; +} diff --git a/modules/core_geocaching/tests/test_journal_inventory.cpp b/modules/core_geocaching/tests/test_journal_inventory.cpp new file mode 100644 index 00000000..0c31b784 --- /dev/null +++ b/modules/core_geocaching/tests/test_journal_inventory.cpp @@ -0,0 +1,67 @@ +#include "platform/esp/arduino_common/geocaching/sd_journal_inventory.h" +#include +#include +#include + +std::vector names; +bool busy_once = false; +bool fail_after_first = false; +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeDir::Impl { public: bool open = false; size_t index = 0; }; +SdRuntimeDir::SdRuntimeDir() : impl_(new Impl) {} +SdRuntimeDir::~SdRuntimeDir() { delete impl_; } +bool SdRuntimeDir::open(const char*) { impl_->open = true; impl_->index = 0; return true; } +void SdRuntimeDir::close() { impl_->open = false; } +bool SdRuntimeDir::is_open() const { return impl_->open; } +SdDirReadStatus SdRuntimeDir::read_next_status(char* name, size_t capacity, bool* is_dir) +{ + if (busy_once) { busy_once = false; return SdDirReadStatus::Busy; } + if (fail_after_first && impl_->index == 1) + { + fail_after_first = false; + ++impl_->index; // A failed read may leave the device cursor uncertain. + return SdDirReadStatus::IoError; + } + if (impl_->index == names.size()) return SdDirReadStatus::End; + std::snprintf(name, capacity, "%s", names[impl_->index++].c_str()); *is_dir = false; + return SdDirReadStatus::Entry; +} +SdFileReadResult sd_read_file(const char*, uint8_t* buffer, size_t capacity) +{ + const auto header = ::geocaching::storage::encodeVolumeHeader({}); + SdFileReadResult result; result.status = SdFileReadStatus::Ready; + result.file_size = header.size(); result.bytes_read = std::min(capacity, header.size()); + std::memcpy(buffer, header.data(), result.bytes_read); return result; +} +} +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + names = {"0000000000000002.gcj", "0000000000000001.gcj"}; + SdJournalInventory valid({}, 0); + if (valid.step() != InventoryStep::Scanning || valid.range().complete || + valid.step() != InventoryStep::Scanning || valid.step() != InventoryStep::Complete || valid.range().last_start != 2) return 1; + names = {"0000000000000002.gcj"}; + SdJournalInventory gap({}, 0); + if (gap.step() != InventoryStep::Scanning || gap.step() != InventoryStep::Complete || gap.range().first_start != 1) return 2; + SdJournalInventory checkpoint({}, 1); + if (checkpoint.step() != InventoryStep::Scanning || checkpoint.step() != InventoryStep::Complete || checkpoint.range().first_start != 2) return 3; + names = {"000000000000000A.gcj"}; + SdJournalInventory invalid({}, 0); + if (invalid.step() != InventoryStep::Corrupt) return 4; + names = {"0000000000000001.gcj", "0000000000000002.gcj"}; + SdJournalInventory interrupted({}, 0); + busy_once = true; + if (interrupted.step() != InventoryStep::RetryLater || interrupted.range().complete || + interrupted.step() != InventoryStep::Scanning) return 5; + fail_after_first = true; + if (interrupted.step() != InventoryStep::RetryLater || interrupted.range().complete) return 6; + if (interrupted.step() != InventoryStep::Scanning || interrupted.step() != InventoryStep::Scanning || + interrupted.step() != InventoryStep::Complete || interrupted.range().last_start != 2) return 7; + names = {"0000000000000001.gcj", "0000000000000008.gcj"}; + SdJournalInventory spanning({}, 4); + if (spanning.step() != InventoryStep::Scanning || spanning.step() != InventoryStep::Scanning || + spanning.step() != InventoryStep::Complete || spanning.range().first_start != 1 || spanning.range().last_start != 8) return 8; + return 0; +} diff --git a/modules/core_geocaching/tests/test_journal_read.cpp b/modules/core_geocaching/tests/test_journal_read.cpp new file mode 100644 index 00000000..93b10749 --- /dev/null +++ b/modules/core_geocaching/tests/test_journal_read.cpp @@ -0,0 +1,35 @@ +#include "platform/esp/arduino_common/geocaching/sd_journal.h" +#include + +std::vector disk; +namespace platform::esp::arduino_common::storage +{ +SdFileReadResult sd_read_file(const char*, uint8_t* buffer, size_t capacity) +{ + SdFileReadResult result; + result.status = disk.empty() ? SdFileReadStatus::Missing : SdFileReadStatus::Ready; + result.file_size = disk.size(); + result.bytes_read = std::min(capacity, disk.size()); + if (result.bytes_read) std::memcpy(buffer, disk.data(), result.bytes_read); + return result; +} +} +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + const uint8_t payload[] = {0x93, 1, 0, 0x91, 0x93, 5, 0xc4, 1, 0xab, 0xc0}; + ::geocaching::storage::RecordHeader header; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 1, {payload, sizeof(payload)}, header)) return 1; + disk.assign(header.begin(), header.end()); disk.insert(disk.end(), payload, payload + sizeof(payload)); + std::array buffer{}; + ::geocaching::storage::MutationView scratch; + ::geocaching::storage::TransactionView out; + if (readJournalTransaction(1, 0, buffer.data(), buffer.size(), &scratch, 1, out) != JournalReadResult::Parsed || out.count != 1) return 2; + if (readJournalTransaction(2, 1, buffer.data(), buffer.size(), &scratch, 1, out) != JournalReadResult::Corrupt || out.count) return 3; + if (readJournalTransaction(1, 0, buffer.data(), 24, &scratch, 1, out) != JournalReadResult::Corrupt) return 4; + disk.back() ^= 1; + if (readJournalTransaction(1, 0, buffer.data(), buffer.size(), &scratch, 1, out) != JournalReadResult::Corrupt) return 5; + disk.clear(); + if (readJournalTransaction(1, 0, buffer.data(), buffer.size(), &scratch, 1, out) != JournalReadResult::Missing) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_journal_replay.cpp b/modules/core_geocaching/tests/test_journal_replay.cpp new file mode 100644 index 00000000..ced19b40 --- /dev/null +++ b/modules/core_geocaching/tests/test_journal_replay.cpp @@ -0,0 +1,2065 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include "geocaching/protocol/query_request.h" +#include "geocaching/protocol/record_encoder.h" +#include "geocaching/storage/draft_record.h" +#include "geocaching/storage/queued_request.h" +#include "platform/esp/arduino_common/geocaching/indexed_dispatch_store.h" +#include "platform/esp/arduino_common/geocaching/indexed_query_store_port.h" +#include "platform/esp/arduino_common/geocaching/request_dispatcher.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_index_import.h" +#include "platform/esp/arduino_common/geocaching/sd_index_append.h" +#include "platform/esp/arduino_common/geocaching/sd_index_get.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_writer.h" +#include "platform/esp/arduino_common/geocaching/sd_index_initialize.h" +#include "platform/esp/arduino_common/geocaching/sd_index_lookup.h" +#include "platform/esp/arduino_common/geocaching/sd_index_references.h" +#include "platform/esp/arduino_common/geocaching/sd_index_replay.h" +#include "platform/esp/arduino_common/geocaching/sd_index_root_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_root_writer.h" +#include "platform/esp/arduino_common/geocaching/sd_index_scan.h" +#include "platform/esp/arduino_common/geocaching/sd_index_transaction.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_attempt_update.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_begin_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_directory_reply.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_download_context.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_download_reply.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_draft_save.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_new_task.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_pending_request.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_recovery.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_stop_task.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_replay.h" +#include +#include +#include +#include +#include +#include +#include + +std::map> files; +std::set index_directories; +std::map read_bytes; +bool fail_read_once = false; +size_t step_bytes = 0; +bool exceeded_budget = false; +bool flush_ok = true; +size_t write_limit = SIZE_MAX; +unsigned flush_calls = 0; +namespace platform::esp::arduino_common::storage +{ +bool sd_card_ready() { return true; } +bool sd_external_block_owner_active() { return false; } +bool sd_is_directory(const char* path) +{ + if (std::string(path).find("/trailmate/geocaching/.state/index") == 0) + return index_directories.count(path) != 0; + return true; +} +bool sd_mkdir(const char* path) +{ + const std::string directory(path); + const auto parent = directory.substr(0, directory.find_last_of('/')); + if (!sd_is_directory(parent.c_str()) || files.count(directory)) return false; + index_directories.insert(directory); + return true; +} +bool sd_exists(const char* path) { return files.count(path) != 0; } +class SdRuntimeDir::Impl +{ + public: + bool open = false; + std::vector names; + size_t position = 0; +}; +SdRuntimeDir::SdRuntimeDir() : impl_(new Impl) {} +SdRuntimeDir::~SdRuntimeDir() { delete impl_; } +bool SdRuntimeDir::open(const char* path) +{ + impl_->names.clear(); + impl_->position = 0; + const std::string prefix = std::string(path) + '/'; + for (const auto& file : files) + if (file.first.compare(0, prefix.size(), prefix) == 0 && file.first.find('/', prefix.size()) == std::string::npos) + impl_->names.push_back(file.first.substr(prefix.size())); + return impl_->open = true; +} +void SdRuntimeDir::close() { impl_->open = false; } +bool SdRuntimeDir::is_open() const { return impl_->open; } +SdDirReadStatus SdRuntimeDir::read_next_status(char* name, size_t capacity, bool* is_dir) +{ + if (!impl_->open) return SdDirReadStatus::IoError; + if (impl_->position == impl_->names.size()) return SdDirReadStatus::End; + const auto& next = impl_->names[impl_->position++]; + if (next.size() >= capacity) return SdDirReadStatus::IoError; + std::memcpy(name, next.c_str(), next.size() + 1); + if (is_dir) *is_dir = false; + return SdDirReadStatus::Entry; +} +class SdRuntimeFile::Impl +{ + public: + std::string path; + size_t offset = 0; + bool open = false; + bool append = false; + bool writable = false; +}; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char* path, const char* mode) +{ + impl_->path = path; + impl_->offset = 0; + impl_->append = mode[0] == 'a'; + impl_->writable = impl_->append || mode[0] == 'w'; + if (mode[0] == 'w') files[path].clear(); + if (impl_->append) files.try_emplace(path); + return impl_->open = files.count(path) != 0; +} +void SdRuntimeFile::close() { impl_->open = false; } +bool SdRuntimeFile::is_open() const { return impl_->open; } +size_t SdRuntimeFile::write(const void* bytes, size_t count) +{ + if (!impl_->open || !impl_->writable) return 0; + auto& data = files.at(impl_->path); + const auto written = std::min(count, write_limit); + const auto* input = static_cast(bytes); + data.insert(data.end(), input, input + written); + impl_->offset = data.size(); + step_bytes += written; + return written; +} +bool SdRuntimeFile::flush() +{ + ++flush_calls; + return flush_ok; +} +uint64_t SdRuntimeFile::size() const { return files.at(impl_->path).size(); } +bool SdRuntimeFile::seek(uint64_t offset) +{ + if (!impl_->open || offset > files.at(impl_->path).size()) return false; + impl_->offset = static_cast(offset); + return true; +} +int SdRuntimeFile::read(void* output, size_t count) +{ + if (fail_read_once) + { + fail_read_once = false; + return -1; + } + const auto& data = files.at(impl_->path); + const auto n = std::min(count, data.size() - impl_->offset); + step_bytes += n; + ::read_bytes[impl_->path] += n; + std::memcpy(output, data.data() + impl_->offset, n); + impl_->offset += n; + return static_cast(n); +} +SdFileReadResult sd_read_file(const char* path, uint8_t* buffer, size_t capacity) +{ + SdFileReadResult result; + auto entry = files.find(path); + if (entry == files.end()) + { + result.status = SdFileReadStatus::Missing; + return result; + } + result.file_size = entry->second.size(); + result.bytes_read = std::min(capacity, entry->second.size()); + step_bytes += result.bytes_read; + std::memcpy(buffer, entry->second.data(), result.bytes_read); + result.status = SdFileReadStatus::Ready; + return result; +} +} // namespace platform::esp::arduino_common::storage + +using namespace platform::esp::arduino_common::geocaching; +// Test driver only; production owners yield between every operation. +ReplayStep nextReady(SdJournalReplay& replay, ::geocaching::storage::TransactionView& transaction) +{ + for (unsigned step = 0; step < 512; ++step) + { + step_bytes = 0; + const auto result = replay.next(transaction); + if (step_bytes > 512) + { + exceeded_budget = true; + return ReplayStep::Corrupt; + } + if (result != ReplayStep::Advancing) return result; + } + return ReplayStep::Corrupt; +} +template +ReplayStep applyReady(SdJournalReplay& replay, ::geocaching::storage::LogicalState& state, Validate validate) +{ + for (unsigned step = 0; step < 512; ++step) + { + step_bytes = 0; + const auto result = replay.applyNext(state, validate); + if (step_bytes > 512) + { + exceeded_budget = true; + return ReplayStep::Corrupt; + } + if (result != ReplayStep::Advancing) return result; + } + return ReplayStep::Corrupt; +} +int checkIndexTransactions() +{ + using namespace ::geocaching; + using namespace ::geocaching::storage; + files.clear(); + VolumeInstance volume{}; + const auto format = encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + std::array keys[2]{}; + keys[0][0] = 1; + keys[1][0] = 2; + const auto bucket = static_cast(::sys::crc32(keys[0].data(), 16)); + bool collision = false; + for (unsigned i = 0; i < 256; ++i) + { + keys[1][15] = static_cast(i); + if (static_cast(::sys::crc32(keys[1].data(), 16)) == bucket) + { + collision = true; + break; + } + } + if (!collision) return 80; + const auto frameFor = [&](uint64_t sequence) + { + uint8_t values[2][64], payload[256]; + size_t sizes[2]{}, size = 0; + MutationView changes[2]; + for (unsigned i = 0; i < 2; ++i) + { + DraftView draft; + draft.generation = sequence; + draft.name = sequence == 1 ? (i ? "Beta" : "Alpha") : (i ? "Beta updated" : "Alpha updated"); + if (!encodeDraft({keys[i].data(), 16}, draft, values[i], sizeof(values[i]), sizes[i])) return std::vector{}; + changes[i] = {4, {keys[i].data(), 16}, {values[i], sizes[i]}, false}; + } + RecordHeader header; + if (!encodeTransaction(sequence - 1, changes, 2, payload, sizeof(payload), size) || + !makeRecordHeader(RecordKind::Transaction, sequence, {payload, size}, header)) return std::vector{}; + std::vector frame(header.begin(), header.end()); + frame.insert(frame.end(), payload, payload + size); + return frame; + }; + auto frame = frameFor(1); + if (frame.empty()) return 81; + files["/trailmate/geocaching/.state/journal/0000000000000001.gcj"] = frame; + std::array bitmap{}; + IndexRootBytes base, candidate; + IndexRootView initial{23, 0, 1, 'a', {bitmap.data(), bitmap.size()}}, parent; + if (!encodeIndexRoot(volume, initial, base) || !decodeIndexRoot({base.data(), base.size()}, volume, parent)) return 82; + for (unsigned copy = 0; copy < 2; ++copy) + { + SdIndexRootWriter writer(volume); + if (!writer.begin(copy, base)) return 83; + auto status = IndexRootWriteStep::Working; + for (unsigned i = 0; i < 64 && status == IndexRootWriteStep::Working; ++i) status = writer.step(); + if (status != IndexRootWriteStep::Verified) return 84; + } + const auto readDraftName = [&](const IndexRootView& root, unsigned key, std::string_view expected) + { + uint8_t buffer[512]; + SdIndexGet reader(volume); + if (!reader.begin(root, 4, {keys[key].data(), 16}, buffer, sizeof(buffer))) return false; + auto rs = IndexGetStep::Working; + for (unsigned i = 0; i < 128 && rs == IndexGetStep::Working; ++i) + { + if (reader.value().data) return false; + step_bytes = 0; + rs = reader.step(); + if (step_bytes > 512) return false; + } + DraftView draft; + return rs == IndexGetStep::Ready && decodeDraft({keys[key].data(), 16}, reader.value(), draft) && draft.name == expected; + }; + auto transaction = std::make_unique(volume); + if (!transaction->begin(parent, 0, {frame.data(), frame.size()}, 1, 0, candidate)) return 85; + auto result = IndexTransactionStep::Working; + IndexRootView committed; + for (unsigned i = 0; i < 1024 && result == IndexTransactionStep::Working; ++i) + { + if (transaction->committed(committed)) return 86; + step_bytes = 0; + result = transaction->step(); + if (step_bytes > 512) return 87; + } + if (result != IndexTransactionStep::Verified || transaction->completedEntries() != 2 || !transaction->committed(committed) || + committed.sequence != 1 || !readDraftName(committed, 0, "Alpha") || !readDraftName(committed, 1, "Beta")) return 88; + uint8_t tiny[24]; + SdIndexGet missing(volume), limited(volume), overlap(volume); + if (!missing.begin(committed, 7, {keys[0].data(), 16}, tiny, sizeof(tiny)) || missing.step() != IndexGetStep::NotFound || missing.value().data || + overlap.begin(committed, 4, {keys[0].data(), 16}, candidate.data(), candidate.size()) || + !limited.begin(committed, 4, {keys[0].data(), 16}, tiny, sizeof(tiny))) return 95; + auto limited_result = IndexGetStep::Working; + for (unsigned step = 0; step < 128 && limited_result == IndexGetStep::Working; ++step) limited_result = limited.step(); + if (limited_result != IndexGetStep::WorkspaceTooSmall || limited.value().data) return 96; + auto next_frame = frameFor(2); + files["/trailmate/geocaching/.state/journal/0000000000000002.gcj"] = next_frame; + transaction = std::make_unique(volume); + if (!transaction->begin(committed, 1, {next_frame.data(), next_frame.size()}, 2, 0, base)) return 89; + result = IndexTransactionStep::Working; + std::map> after_first; + for (unsigned i = 0; i < 1024 && result == IndexTransactionStep::Working; ++i) + { + step_bytes = 0; + result = transaction->step(); + if (step_bytes > 512) return 90; + if (transaction->completedEntries() == 1 && after_first.empty()) + { + after_first = files; + write_limit = 7; + } + } + write_limit = SIZE_MAX; + if (result != IndexTransactionStep::IoError || after_first.empty() || + !readDraftName(committed, 0, "Alpha") || !readDraftName(committed, 1, "Beta")) return 91; + // Restart from a crash after the first complete shard entry, before the + // injected short write. Replaying the transaction must not duplicate it. + transaction.reset(); + files = after_first; + transaction = std::make_unique(volume); + if (!transaction->begin(committed, 1, {next_frame.data(), next_frame.size()}, 2, 0, base)) return 92; + result = IndexTransactionStep::Working; + for (unsigned i = 0; i < 1024 && result == IndexTransactionStep::Working; ++i) result = transaction->step(); + IndexRootView updated; + if (result != IndexTransactionStep::Verified || !transaction->committed(updated) || updated.sequence != 2 || + !readDraftName(updated, 0, "Alpha updated") || !readDraftName(updated, 1, "Beta updated")) return 93; + char path[80]; + if (!indexShardPath('a', 4, {keys[0].data(), 16}, path, sizeof(path)) || files[path].size() != 4 * kIndexEntrySize) return 94; + uint8_t erase_payload[64]; + size_t erase_size = 0; + MutationView erase{4, {keys[0].data(), 16}, {}, true}; + RecordHeader erase_header; + if (!encodeTransaction(2, &erase, 1, erase_payload, sizeof(erase_payload), erase_size) || + !makeRecordHeader(RecordKind::Transaction, 3, {erase_payload, erase_size}, erase_header)) return 97; + std::vector erase_frame(erase_header.begin(), erase_header.end()); + erase_frame.insert(erase_frame.end(), erase_payload, erase_payload + erase_size); + files["/trailmate/geocaching/.state/journal/0000000000000003.gcj"] = erase_frame; + transaction = std::make_unique(volume); + if (!transaction->begin(updated, 0, {erase_frame.data(), erase_frame.size()}, 3, 0, candidate)) return 98; + result = IndexTransactionStep::Working; + for (unsigned i = 0; i < 1024 && result == IndexTransactionStep::Working; ++i) result = transaction->step(); + IndexRootView deleted; + if (result != IndexTransactionStep::Verified || !transaction->committed(deleted) || + !readDraftName(updated, 0, "Alpha updated") || !readDraftName(deleted, 1, "Beta updated")) return 99; + uint8_t read_buffer[512]; + SdIndexGet removed(volume); + if (!removed.begin(deleted, 4, {keys[0].data(), 16}, read_buffer, sizeof(read_buffer))) return 100; + auto removed_result = IndexGetStep::Working; + for (unsigned i = 0; i < 128 && removed_result == IndexGetStep::Working; ++i) removed_result = removed.step(); + if (removed_result != IndexGetStep::NotFound || removed.value().data) return 101; + SdIndexScan live_rows(volume); + if (!live_rows.begin(deleted, 4, read_buffer, sizeof(read_buffer))) return 144; + auto scan_result = IndexScanStep::Working; + unsigned live_count = 0; + for (unsigned step = 0; step < 1024; ++step) + { + step_bytes = 0; + scan_result = live_rows.step(); + if (step_bytes > 512) return 145; + if (scan_result == IndexScanStep::Item) + { + MutationView row; + DraftView draft; + if (!live_rows.item(row) || !decodeDraft(row.key, row.value, draft) || draft.name != "Beta updated" || !live_rows.advance()) return 146; + ++live_count; + } + else if (scan_result != IndexScanStep::Working) break; + } + if (scan_result != IndexScanStep::End || live_count != 1) return 147; + auto& damaged = files["/trailmate/geocaching/.state/journal/0000000000000002.gcj"]; + damaged.back() ^= 1; + SdIndexGet corrupt(volume); + if (!corrupt.begin(deleted, 4, {keys[1].data(), 16}, read_buffer, sizeof(read_buffer))) return 102; + auto corrupt_result = IndexGetStep::Working; + for (unsigned i = 0; i < 128 && corrupt_result == IndexGetStep::Working; ++i) corrupt_result = corrupt.step(); + if (corrupt_result != IndexGetStep::Invalid || corrupt.value().data) return 103; + transaction.reset(); + files.clear(); + return 0; +} + +int checkIndexedCommitCapacity() +{ + using namespace ::geocaching; + using namespace ::geocaching::storage; + files.clear(); + read_bytes.clear(); + VolumeInstance volume{}; + const auto format = encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + index_directories.clear(); + IndexRootBytes roots[2]; + IndexRootView current; + for (const auto* checkpoint : {"/trailmate/geocaching/.state/checkpoint/a.gcs", "/trailmate/geocaching/.state/checkpoint/b.gcs"}) + { + files[checkpoint] = {1, 2, 3}; + const auto before = files; + SdIndexInitialize blocked(volume); + if (!blocked.begin(roots[0])) return 233; + auto status = IndexRootWriteStep::Working; + for (unsigned i = 0; i < 16 && status == IndexRootWriteStep::Working; ++i) status = blocked.step(); + if (status != IndexRootWriteStep::Invalid || files != before || !index_directories.empty()) return 234; + files.erase(checkpoint); + } + SdIndexInitialize initialize(volume); + if (!initialize.begin(roots[0])) return 104; + auto initialized = IndexRootWriteStep::Working; + for (unsigned step = 0; step < 128 && initialized == IndexRootWriteStep::Working; ++step) + { + step_bytes = 0; + initialized = initialize.step(); + if (step_bytes > 512) return 169; + } + if (initialized != IndexRootWriteStep::Verified || + files["/trailmate/geocaching/.state/index/root.h0"] != std::vector(roots[0].begin(), roots[0].end()) || + files["/trailmate/geocaching/.state/index/root.h1"] != files["/trailmate/geocaching/.state/index/root.h0"]) return 170; + roots[1] = roots[0]; + SdIndexInitialize duplicate(volume); + if (!duplicate.begin(roots[1])) return 171; + initialized = IndexRootWriteStep::Working; + for (unsigned step = 0; step < 128 && initialized == IndexRootWriteStep::Working; ++step) initialized = duplicate.step(); + if (initialized != IndexRootWriteStep::Invalid) return 172; + if (!decodeIndexRoot({roots[0].data(), roots[0].size()}, volume, current)) return 105; + uint8_t frame[512], encoded[256]; + unsigned copy = 0; + size_t logical_bytes = 0; + const std::string description(96, 'x'); + for (unsigned row = 1; row <= 40; ++row) + { + std::array key{}; + key[0] = static_cast(row); + const auto name = std::string("Draft ") + std::to_string(row); + DraftView draft; + draft.name = name; + draft.description = description; + size_t size = 0; + if (!encodeDraft({key.data(), key.size()}, draft, encoded, sizeof(encoded), size)) return 106; + logical_bytes += key.size() + size + 4; + auto commit = std::make_unique(volume); + if (!commit->begin(current, copy, {key.data(), key.size()}, {encoded, size}, 0, frame, sizeof(frame), roots[1 - copy])) return 107; + auto result = IndexedCommitStep::Working; + bool released = false; + for (unsigned step = 0; step < 512 && result == IndexedCommitStep::Working; ++step) + { + step_bytes = 0; + result = commit->step(); + if (step_bytes > 512) return 108; + if (commit->inputConsumed() && !released) + { + std::memset(encoded, 0xa5, sizeof(encoded)); + key.fill(0xa5); + released = true; + } + } + if (result != IndexedCommitStep::Verified || !commit->committed(current) || current.sequence != row || !released) return 109; + char journal_path[96]; + std::snprintf(journal_path, sizeof(journal_path), "/trailmate/geocaching/.state/journal/%016llx.gcj", static_cast(row)); + if (read_bytes[journal_path] != files[journal_path].size()) return 118; + copy = 1 - copy; + } + if (logical_bytes <= 4096) return 110; + { + std::array key{}; + key[0] = 1; + DraftView next; + next.name = "Stale editor overwrite"; + size_t size = 0; + if (!encodeDraft({key.data(), key.size()}, next, encoded, sizeof(encoded), size)) return 193; + const auto before = files; + auto stale = std::make_unique(volume); + if (!stale->begin(current, copy, {key.data(), key.size()}, {encoded, size}, 0, frame, sizeof(frame), roots[1 - copy])) return 194; + auto result = IndexedCommitStep::Working; + for (unsigned step = 0; step < 512 && result == IndexedCommitStep::Working; ++step) result = stale->step(); + if (result != IndexedCommitStep::Invalid || files != before) return 195; + } + for (unsigned row : {1U, 20U, 40U}) + { + std::array key{}; + key[0] = static_cast(row); + SdIndexGet get(volume); + if (!get.begin(current, 4, {key.data(), key.size()}, frame, sizeof(frame))) return 111; + auto result = IndexGetStep::Working; + for (unsigned step = 0; step < 256 && result == IndexGetStep::Working; ++step) result = get.step(); + DraftView draft; + if (result != IndexGetStep::Ready || !decodeDraft({key.data(), key.size()}, get.value(), draft) || + draft.name != std::string("Draft ") + std::to_string(row) || draft.description != description) return 112; + } + std::array extra_key{}; + extra_key[0] = 41; + DraftView extra; + extra.name = "Extra"; + extra.description = description; + size_t extra_size = 0; + if (!encodeDraft({extra_key.data(), extra_key.size()}, extra, encoded, sizeof(encoded), extra_size)) return 113; + MutationView extra_mutation{4, {extra_key.data(), extra_key.size()}, {encoded, extra_size}, false}; + const auto before = files; + const uint8_t malformed_value = 0xc0; + auto malformed = extra_mutation; + malformed.value = {&malformed_value, 1}; + SdIndexedCommit bad_row(volume); + if (bad_row.begin(current, copy, &malformed, 1, frame, sizeof(frame), roots[1 - copy]) || files != before) return 140; + SdIndexedCommit too_small(volume); + if (too_small.begin(current, copy, &extra_mutation, 1, frame, 24, roots[1 - copy]) || files != before) return 114; + IndexRootView stale; + if (!decodeIndexRoot({roots[1 - copy].data(), roots[1 - copy].size()}, volume, stale)) return 115; + SdIndexedCommit conflict(volume); + if (!conflict.begin(stale, 1 - copy, &extra_mutation, 1, frame, sizeof(frame), roots[copy])) return 116; + auto conflict_result = IndexedCommitStep::Working; + for (unsigned step = 0; step < 128 && conflict_result == IndexedCommitStep::Working; ++step) conflict_result = conflict.step(); + if (conflict_result != IndexedCommitStep::RecoveryRequired || files != before) return 117; + if (!encodeDraft({extra_key.data(), extra_key.size()}, extra, frame, sizeof(frame), extra_size)) return 119; + extra_mutation.value = {frame, extra_size}; + SdIndexedCommit aliased(volume); + if (!aliased.begin(current, copy, &extra_mutation, 1, frame, sizeof(frame), roots[1 - copy])) return 120; + auto aliased_result = IndexedCommitStep::Working; + bool released_alias = false; + for (unsigned step = 0; step < 512 && aliased_result == IndexedCommitStep::Working; ++step) + { + aliased_result = aliased.step(); + if (aliased.inputConsumed() && !released_alias) + { + std::memset(frame, 0xa5, sizeof(frame)); + released_alias = true; + } + } + const std::string alias_path = "/trailmate/geocaching/.state/journal/0000000000000029.gcj"; + if (aliased_result != IndexedCommitStep::Verified || !released_alias || read_bytes[alias_path] != 2 * files[alias_path].size()) return 121; + if (!aliased.committed(current)) return 122; + copy = 1 - copy; + extra_key[0] = 42; + if (!encodeDraft({extra_key.data(), extra_key.size()}, extra, encoded, sizeof(encoded), extra_size)) return 123; + extra_mutation.value = {encoded, extra_size}; + const auto old_root0 = files["/trailmate/geocaching/.state/index/root.h0"]; + const auto old_root1 = files["/trailmate/geocaching/.state/index/root.h1"]; + SdIndexedCommit interrupted(volume); + if (!interrupted.begin(current, copy, &extra_mutation, 1, frame, sizeof(frame), roots[1 - copy])) return 124; + auto interrupted_result = IndexedCommitStep::Working; + for (unsigned step = 0; step < 512 && interrupted_result == IndexedCommitStep::Working; ++step) + { + interrupted_result = interrupted.step(); + if (interrupted.inputConsumed()) flush_ok = false; + } + flush_ok = true; + IndexRootView unpublished; + if (interrupted_result != IndexedCommitStep::RecoveryRequired || interrupted.committed(unpublished) || + files["/trailmate/geocaching/.state/index/root.h0"] != old_root0 || files["/trailmate/geocaching/.state/index/root.h1"] != old_root1) return 125; + // Discard transient input and reconstruct the already durable transaction. + std::memset(frame, 0, sizeof(frame)); + SdIndexedCommit rejected(volume); + if (!rejected.resume(current, copy, frame, sizeof(frame), roots[1 - copy])) return 126; + auto rejected_state = IndexedCommitStep::Working; + for (unsigned step = 0; step < 64 && rejected_state == IndexedCommitStep::Working; ++step) rejected_state = rejected.step(); + if (rejected_state != IndexedCommitStep::NeedsValidation || rejected.step() != IndexedCommitStep::NeedsValidation || + !rejected.validateRecovered(false) || rejected.step() != IndexedCommitStep::Invalid || rejected.committed(unpublished) || + files["/trailmate/geocaching/.state/index/root.h0"] != old_root0 || files["/trailmate/geocaching/.state/index/root.h1"] != old_root1) return 127; + for (unsigned after_validation = 0; after_validation < 2; ++after_validation) + { + SdIndexedCommit changed(volume); + if (!changed.resume(current, copy, frame, sizeof(frame), roots[1 - copy])) return 134; + auto status = IndexedCommitStep::Working; + for (unsigned step = 0; step < 64 && status == IndexedCommitStep::Working; ++step) status = changed.step(); + RecordFrameView original; + MutationView mutation; + TransactionView transaction; + const auto saved_frame = changed.recoveredFrame(); + if (status != IndexedCommitStep::NeedsValidation || !decodeRecordFrame(saved_frame, original) || + !decodeTransaction(original.payload, current.sequence, &mutation, 1, transaction)) return 135; + if (after_validation && !changed.validateRecovered(true)) return 136; + frame[mutation.key.data - frame] ^= 1; + RecordHeader replacement; + if (!makeRecordHeader(RecordKind::Transaction, original.sequence, original.payload, replacement)) return 137; + std::memcpy(frame, replacement.data(), replacement.size()); + if (!after_validation && !changed.validateRecovered(true)) return 138; + if (changed.step() != IndexedCommitStep::Invalid || changed.committed(unpublished) || + files["/trailmate/geocaching/.state/index/root.h0"] != old_root0 || files["/trailmate/geocaching/.state/index/root.h1"] != old_root1) return 139; + } + SdIndexedCommit recovery(volume); + if (!recovery.resume(current, copy, frame, sizeof(frame), roots[1 - copy])) return 128; + auto restored = IndexedCommitStep::Working; + for (unsigned step = 0; step < 64 && restored == IndexedCommitStep::Working; ++step) + { + step_bytes = 0; + restored = recovery.step(); + if (step_bytes > 512) return 132; + } + RecordFrameView recovered; + TransactionView restored_transaction; + MutationView restored_mutation; + DraftView validated_draft; + if (restored != IndexedCommitStep::NeedsValidation || !decodeRecordFrame(recovery.recoveredFrame(), recovered) || + !decodeTransaction(recovered.payload, current.sequence, &restored_mutation, 1, restored_transaction) || restored_mutation.table != 4 || + !decodeDraft(restored_mutation.key, restored_mutation.value, validated_draft) || validated_draft.name != "Extra" || + !recovery.validateRecovered(true)) return 133; + restored = IndexedCommitStep::Working; + for (unsigned step = 0; step < 512 && restored == IndexedCommitStep::Working; ++step) restored = recovery.step(); + if (restored != IndexedCommitStep::Verified || !recovery.committed(current) || current.sequence != 42) return 129; + SdIndexGet recovered_value(volume); + if (!recovered_value.begin(current, 4, {extra_key.data(), extra_key.size()}, frame, sizeof(frame))) return 130; + auto fetched = IndexGetStep::Working; + for (unsigned step = 0; step < 128 && fetched == IndexGetStep::Working; ++step) fetched = recovered_value.step(); + DraftView recovered_draft; + if (fetched != IndexGetStep::Ready || !decodeDraft({extra_key.data(), extra_key.size()}, recovered_value.value(), recovered_draft) || recovered_draft.name != "Extra") return 131; + copy = 1 - copy; + const auto current_root0 = files["/trailmate/geocaching/.state/index/root.h0"]; + const auto current_root1 = files["/trailmate/geocaching/.state/index/root.h1"]; + uint8_t invalid_payload[128]; + size_t invalid_size = 0; + extra_key[0] = 43; + malformed.key = {extra_key.data(), extra_key.size()}; + RecordHeader invalid_header; + if (!encodeTransaction(current.sequence, &malformed, 1, invalid_payload, sizeof(invalid_payload), invalid_size) || + !makeRecordHeader(RecordKind::Transaction, 43, {invalid_payload, invalid_size}, invalid_header)) return 141; + auto& invalid_file = files["/trailmate/geocaching/.state/journal/000000000000002b.gcj"]; + invalid_file.assign(invalid_header.begin(), invalid_header.end()); + invalid_file.insert(invalid_file.end(), invalid_payload, invalid_payload + invalid_size); + SdIndexedCommit bad_recovery(volume); + if (!bad_recovery.resume(current, copy, frame, sizeof(frame), roots[1 - copy])) return 142; + auto bad_result = IndexedCommitStep::Working; + for (unsigned step = 0; step < 64 && bad_result == IndexedCommitStep::Working; ++step) bad_result = bad_recovery.step(); + if (bad_result != IndexedCommitStep::RecoveryRequired || bad_recovery.recoveredFrame().data || + files["/trailmate/geocaching/.state/index/root.h0"] != current_root0 || files["/trailmate/geocaching/.state/index/root.h1"] != current_root1) return 143; + auto scan = std::make_unique(volume); + if (!scan->begin(current, 4, frame, sizeof(frame))) return 148; + std::array seen{}; + unsigned count = 0; + auto scanned = IndexScanStep::Working; + for (unsigned step = 0; step < 8192; ++step) + { + step_bytes = 0; + scanned = scan->step(); + if (step_bytes > 512) return 149; + if (scanned == IndexScanStep::Item) + { + MutationView row; + DraftView draft; + if (!scan->item(row) || row.key.size != 16 || !row.key.data[0] || row.key.data[0] > 42 || seen[row.key.data[0]] || + !decodeDraft(row.key, row.value, draft) || scan->step() != IndexScanStep::Item) return 150; + seen[row.key.data[0]] = true; + ++count; + if (!scan->advance()) return 151; + } + else if (scanned != IndexScanStep::Working) break; + } + if (scanned != IndexScanStep::End || count != 42) return 152; + if (!index_directories.count("/trailmate/geocaching/.state/index") || + !index_directories.count("/trailmate/geocaching/.state/index/a") || + !index_directories.count("/trailmate/geocaching/.state/index/a/04")) return 168; + scan.reset(); + files.erase("/trailmate/geocaching/.state/journal/000000000000002b.gcj"); + uint8_t request[64], values[512], transaction_bytes[512]; + size_t request_size = 0, transaction_size = 0; + ::geocaching::RequestId request_id; + if (!::geocaching::protocol::encodeCapabilitiesRequest(request_id, request, sizeof(request), request_size)) return 153; + QueuedRequestWorkspace queued(values, sizeof(values)); + if (!encodeNewRequestTask(current.sequence, {}, {}, request_id, {}, 3, {request, request_size}, {}, queued, + transaction_bytes, sizeof(transaction_bytes), transaction_size)) return 154; + MutationView linked[2]; + TransactionView linked_transaction; + if (!decodeTransaction({transaction_bytes, transaction_size}, current.sequence, linked, 2, linked_transaction)) return 155; + for (unsigned round = 0; round < 2; ++round) + { + MutationView remove_task{10, linked[1].key, {}, true}; + // Validate the candidate before writing any journal bytes: creation + // resolves both rows from the overlay; parent-only deletion is rejected. + auto preflight = std::make_unique(volume); + if (!preflight->begin(current, frame, sizeof(frame), round ? &remove_task : linked, round ? 1 : 2)) return 162; + auto checked = IndexScanStep::Working; + const auto unchanged_files = files; + for (unsigned step = 0; step < 8192 && checked == IndexScanStep::Working; ++step) + { + step_bytes = 0; + checked = preflight->step(); + if (step_bytes > 512) return 163; + } + if (checked != (round ? IndexScanStep::Invalid : IndexScanStep::End) || files != unchanged_files) return 164; + if (round) + { + MutationView remove_both[] = {remove_task, {5, linked[0].key, {}, true}}; + preflight = std::make_unique(volume); + if (!preflight->begin(current, frame, sizeof(frame), remove_both, 2)) return 165; + checked = IndexScanStep::Working; + for (unsigned step = 0; step < 8192 && checked == IndexScanStep::Working; ++step) checked = preflight->step(); + if (checked != IndexScanStep::End || files != unchanged_files) return 166; + } + SdIndexedCommit commit(volume); + if (!commit.begin(current, copy, round ? &remove_task : linked, round ? 1 : 2, frame, sizeof(frame), roots[1 - copy], false, round ? 0 : 2)) return 156; + auto committed = IndexedCommitStep::Working; + for (unsigned step = 0; step < 8192 && committed == IndexedCommitStep::Working; ++step) + { + step_bytes = 0; + committed = commit.step(); + if (step_bytes > 512) return 157; + } + if (round) + { + if (committed != IndexedCommitStep::Invalid || files != unchanged_files) return 167; + continue; + } + if (committed != IndexedCommitStep::Verified || !commit.committed(current)) return 158; + copy = 1 - copy; + const auto existing_task = files; + SdIndexedCommit duplicate_task(volume); + if (!duplicate_task.begin(current, copy, linked, 2, frame, sizeof(frame), roots[1 - copy], false, 2)) return 196; + auto duplicate_status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && duplicate_status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + duplicate_status = duplicate_task.step(); + if (step_bytes > 512) return 197; + } + if (duplicate_status != IndexedCommitStep::Invalid || files != existing_task) return 198; + auto validation = std::make_unique(volume); + if (!validation->begin(current, frame, sizeof(frame))) return 159; + auto validated = IndexScanStep::Working; + for (unsigned step = 0; step < 8192 && validated == IndexScanStep::Working; ++step) + { + step_bytes = 0; + validated = validation->step(); + if (step_bytes > 512) return 160; + } + if (validated != (round ? IndexScanStep::Invalid : IndexScanStep::End)) return 161; + } + std::array issuance_key{}; + issuance_key.back() = 1; + std::array author{}; + uint8_t issuance[160]; + size_t issuance_size = 0; + RevisionHash issued_hash; + if (!encodeAuthorIssued(issued_hash, {author.data(), author.size()}, {}, issuance, sizeof(issuance), issuance_size)) return 173; + for (unsigned round = 0; round < 4; ++round) + { + // Creation and byte-identical retry succeed; rewriting or deleting an + // issued revision must fail before touching journal or index files. + if (round == 2) issuance[3] ^= 1; + MutationView mutation{3, {issuance_key.data(), issuance_key.size()}, round == 3 ? ByteView{} : ByteView{issuance, issuance_size}, round == 3}; + const auto before = files; + SdIndexedCommit validation(volume); + if (!validation.begin(current, copy, &mutation, 1, frame, sizeof(frame), roots[1 - copy], true)) return 186; + auto validation_result = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && validation_result == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + validation_result = validation.step(); + if (step_bytes > 512) return 187; + } + IndexRootView unpublished; + if (validation_result != (round < 2 ? IndexedCommitStep::Validated : IndexedCommitStep::Invalid) || + files != before || validation.committed(unpublished)) return 188; + SdIndexedCommit commit(volume); + if (!commit.begin(current, copy, &mutation, 1, frame, sizeof(frame), roots[1 - copy])) return 174; + auto result = IndexedCommitStep::Working; + for (unsigned step = 0; step < 8192 && result == IndexedCommitStep::Working; ++step) + { + step_bytes = 0; + result = commit.step(); + if (step_bytes > 512) return 175; + } + if (round < 2) + { + if (result != IndexedCommitStep::Verified || !commit.committed(current)) return 176; + copy = 1 - copy; + } + else if (result != IndexedCommitStep::Invalid || files != before) return 177; + } + // Simulate loss of power after a valid journal is durable but before its + // index publication. Recovery must await business acceptance and then + // advance the root only after the entire index transaction is verified. + std::array resumed_key{}; + resumed_key[0] = 90; + DraftView resumed_draft; + resumed_draft.generation = 1; + resumed_draft.name = "Recovered draft"; + size_t resumed_size = 0; + if (!encodeDraft({resumed_key.data(), resumed_key.size()}, resumed_draft, encoded, sizeof(encoded), resumed_size)) return 178; + MutationView resume_row{4, {resumed_key.data(), resumed_key.size()}, {encoded, resumed_size}, false}; + SdGeocachingJournal journal(volume); + if (journal.begin(current.sequence, &resume_row, 1) != JournalWriteResult::InProgress) return 179; + auto journal_status = JournalWriteResult::InProgress; + for (unsigned i = 0; i < 256 && journal_status == JournalWriteResult::InProgress; ++i) journal_status = journal.step(); + if (journal_status != JournalWriteResult::Verified) return 180; + const auto resumed_sequence = current.sequence + 1; + MutationView pending_rows[3]; + uint8_t validation_frame[512]; + const auto before_replay = files; + for (unsigned accepted = 0; accepted < 2; ++accepted) + { + auto replay = std::make_unique(volume, current, copy, roots[0], roots[1], + JournalSegmentRange{true, resumed_sequence, resumed_sequence}, + frame, sizeof(frame), pending_rows, 3); + auto status = IndexReplayStep::Working; + bool asked = false; + for (unsigned i = 0; i < 8192; ++i) + { + step_bytes = 0; + status = replay->step(); + if (step_bytes > 512) return 181; + if (status == IndexReplayStep::NeedsValidation) + { + TransactionView pending; + if (asked || files != before_replay || !replay->pending(pending) || pending.count != 1 || + !replay->accept(accepted != 0, validation_frame, sizeof(validation_frame))) return 182; + asked = true; + } + else if (status != IndexReplayStep::Working) break; + } + if (!asked || status != (accepted ? IndexReplayStep::Complete : IndexReplayStep::Invalid)) return 183; + if (accepted && (!replay->selected(current, copy) || current.sequence != resumed_sequence)) return 184; + if (!accepted && files != before_replay) return 185; + } + // Recovery cannot bypass immutable issuance checks, even when its caller + // accepts the transaction's operation-specific business rules. + MutationView rewrite{3, {issuance_key.data(), issuance_key.size()}, {issuance, issuance_size}, false}; + SdGeocachingJournal bad_journal(volume); + if (bad_journal.begin(current.sequence, &rewrite, 1) != JournalWriteResult::InProgress) return 189; + journal_status = JournalWriteResult::InProgress; + for (unsigned i = 0; i < 256 && journal_status == JournalWriteResult::InProgress; ++i) journal_status = bad_journal.step(); + if (journal_status != JournalWriteResult::Verified) return 190; + const auto before_bad_replay = files; + auto bad_replay = std::make_unique(volume, current, copy, roots[0], roots[1], + JournalSegmentRange{true, current.sequence + 1, current.sequence + 1}, + frame, sizeof(frame), pending_rows, 3); + auto bad_status = IndexReplayStep::Working; + for (unsigned i = 0; i < 8192; ++i) + { + bad_status = bad_replay->step(); + if (bad_status == IndexReplayStep::NeedsValidation) + { + if (!bad_replay->accept(true, validation_frame, sizeof(validation_frame))) return 191; + } + else if (bad_status != IndexReplayStep::Working) break; + } + if (bad_status != IndexReplayStep::Invalid || files != before_bad_replay) return 192; + files.clear(); + return 0; +} + +int checkIndexedDraftPublication() +{ + using namespace ::geocaching; + using namespace ::geocaching::storage; + files.clear(); + index_directories.clear(); + VolumeInstance volume{}; + const auto format = encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + IndexRootBytes roots[2]; + SdIndexInitialize initialize(volume); + if (!initialize.begin(roots[0])) return 201; + auto initialized = IndexRootWriteStep::Working; + for (unsigned i = 0; i < 128 && initialized == IndexRootWriteStep::Working; ++i) initialized = initialize.step(); + if (initialized != IndexRootWriteStep::Verified) return 202; + roots[1] = roots[0]; + IndexRootView current; + if (!decodeIndexRoot({roots[0].data(), roots[0].size()}, volume, current)) return 203; + unsigned copy = 0; + uint8_t frame[1024], encoded[256], record_bytes[256], request_bytes[512], outgoing_bytes[768]; + std::array draft_id{}, task_id{}; + draft_id.fill(3); + task_id.fill(4); + std::array author{}, signature{}; + std::array hash{}, cache_id{}; + hash.fill(5); + DraftView draft; + draft.author = {author.data(), author.size()}; + draft.base_hash = {hash.data(), hash.size()}; + draft.name = "Frozen draft"; + size_t size = 0; + if (!encodeDraft({draft_id.data(), draft_id.size()}, draft, encoded, sizeof(encoded), size)) return 204; + auto commit = [&](const MutationView* rows, size_t count) + { + auto operation = std::make_unique(volume); + if (!operation->begin(current, copy, rows, count, frame, sizeof(frame), roots[1 - copy])) return false; + auto result = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && result == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + result = operation->step(); + if (step_bytes > 512) return false; + } + if (result != IndexedCommitStep::Verified || !operation->committed(current)) return false; + copy = 1 - copy; + return true; + }; + MutationView frozen{4, {draft_id.data(), draft_id.size()}, {encoded, size}, false}; + if (!commit(&frozen, 1)) return 205; + auto save_draft = [&](uint64_t expected) + { + if (!encodeDraft({draft_id.data(), draft_id.size()}, draft, encoded, sizeof(encoded), size)) return IndexedCommitStep::Invalid; + auto operation = std::make_unique(volume); + if (!operation->begin(current, copy, {draft_id.data(), draft_id.size()}, {encoded, size}, expected, + frame, sizeof(frame), roots[1 - copy])) return IndexedCommitStep::Invalid; + auto result = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && result == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + result = operation->step(); + if (step_bytes > 512) return IndexedCommitStep::IoError; + } + if (result == IndexedCommitStep::Verified) + { + if (!operation->committed(current)) return IndexedCommitStep::Invalid; + copy = 1 - copy; + } + return result; + }; + draft.generation = 2; + draft.name = "Next edit"; + const auto frozen_files = files; + if (save_draft(1) != IndexedCommitStep::Invalid || files != frozen_files) return 206; + RecordView record; + record.author_public_key = {author.data(), author.size()}; + record.creation_nonce = {draft_id.data(), draft_id.size()}; + record.revision = 1; + record.name = "Frozen draft"; + record.difficulty_x2 = record.terrain_x2 = 2; + size_t record_size = 0, request_size = 0; + RequestId request; + if (!protocol::encodeGeocacheRecord(record, record_bytes, sizeof(record_bytes), record_size) || + !protocol::encodePublishRequest(request, {record_bytes, record_size}, {signature.data(), signature.size()}, + 512, request_bytes, sizeof(request_bytes), request_size)) return 207; + QueuedRequestWorkspace workspace(outgoing_bytes, sizeof(outgoing_bytes)); + auto publish = std::make_unique(volume); + if (!publish->begin(current, copy, {}, {}, request, task_id, 1, {request_bytes, request_size}, {}, + {{cache_id.data(), cache_id.size()}, {hash.data(), hash.size()}, 0}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 208; + auto publish_status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && publish_status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + publish_status = publish->step(); + if (step_bytes > 512) return 212; + } + if (publish_status != IndexedCommitStep::Verified || !publish->committed(current)) return 213; + copy = 1 - copy; + if (save_draft(1) != IndexedCommitStep::Verified) return 209; + const auto saved_files = files; + draft.generation = 3; + author[0] ^= 1; + if (save_draft(2) != IndexedCommitStep::Invalid || files != saved_files) return 210; + author[0] ^= 1; + hash[0] ^= 1; + if (save_draft(2) != IndexedCommitStep::Invalid || files != saved_files) return 211; + hash[0] ^= 1; + GeocacheId download_id; + RevisionHash download_hash; + download_id.bytes = cache_id; + download_hash.bytes = hash; + for (unsigned attempt = 0; attempt < 3; ++attempt) + { + request.bytes.fill(static_cast(8 + attempt)); + task_id.fill(static_cast(12 + attempt)); + if (!protocol::encodeGetRequest(request, download_id, &download_hash, nullptr, 8192, + request_bytes, sizeof(request_bytes), request_size)) return 214; + const uint64_t generation = attempt == 2 ? 2 : 1; + auto download = std::make_unique(volume); + if (!download->begin(current, copy, {}, {}, request, task_id, 2, {request_bytes, request_size}, {}, + {{cache_id.data(), cache_id.size()}, {hash.data(), hash.size()}, generation}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 215; + const auto before_download = files; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = download->step(); + if (step_bytes > 512) return 216; + } + if (attempt == 1) + { + if (status != IndexedCommitStep::Invalid || files != before_download) return 217; + continue; + } + if (status != IndexedCommitStep::Verified || !download->committed(current)) return 218; + copy = 1 - copy; + SdIndexGet head(volume); + if (!head.begin(current, 2, {cache_id.data(), cache_id.size()}, frame, sizeof(frame))) return 219; + auto read = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && read == IndexGetStep::Working; ++i) read = head.step(); + CacheHeadView decoded; + if (read != IndexGetStep::Ready || !decodeCacheHead({cache_id.data(), cache_id.size()}, head.value(), decoded) || + decoded.install_generation != generation) return 220; + } + std::array stopped_request{}; + std::copy(request.bytes.begin(), request.bytes.end(), stopped_request.begin() + 32); + for (unsigned scenario = 0; scenario < 3; ++scenario) + { + auto context_key = stopped_request; + if (scenario == 1) std::fill(context_key.begin() + 32, context_key.end(), 8); + auto context = std::make_unique(volume); + if (!context->begin(current, {context_key.data(), context_key.size()}, scenario == 0 ? 2 : scenario == 1 ? 1 + : 99, + frame, sizeof(frame))) return 133; + auto status = IndexGetStep::Working; + for (unsigned i = 0; i < 4096 && status == IndexGetStep::Working; ++i) + { + step_bytes = 0; + status = context->step(); + if (step_bytes > 512) return 134; + } + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + if (!scenario) + { + if (status != IndexGetStep::Ready || !context->intentActive() || !context->view(outgoing, task, head) || + head.install_generation != 2 || outgoing.request.size != request_size || + std::memcmp(outgoing.request.data, request_bytes, request_size)) return 135; + } + else if (status != IndexGetStep::Invalid || context->view(outgoing, task, head) || context->intentActive()) return 136; + } + for (unsigned repeat = 0; repeat < 2; ++repeat) + { + const auto before_stop = files; + const auto sequence = current.sequence; + auto stop = std::make_unique(volume); + const ByteView stop_key = repeat ? ByteView{task_id.data(), task_id.size()} : ByteView{stopped_request.data(), stopped_request.size()}; + if (!stop->begin(current, copy, stop_key, !repeat, frame, sizeof(frame), roots[1 - copy])) return 78; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = stop->step(); + if (step_bytes > 512) return 79; + } + if (status != IndexedCommitStep::Verified || !stop->committed(current)) return 80; + if (repeat) + { + if (current.sequence != sequence || files != before_stop) return 81; + } + else + { + if (current.sequence != sequence + 1) return 82; + copy = 1 - copy; + } + } + SdIndexGet stopped_task(volume); + if (!stopped_task.begin(current, 10, {task_id.data(), task_id.size()}, frame, sizeof(frame))) return 83; + auto stopped_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && stopped_status == IndexGetStep::Working; ++i) stopped_status = stopped_task.step(); + TaskView stopped; + if (stopped_status != IndexGetStep::Ready || !decodeTask({task_id.data(), task_id.size()}, stopped_task.value(), stopped) || + stopped.state != 5 || stopped.continue_intent) return 84; + auto stopped_context = std::make_unique(volume); + if (!stopped_context->begin(current, {stopped_request.data(), stopped_request.size()}, 2, frame, sizeof(frame))) return 137; + auto context_status = IndexGetStep::Working; + for (unsigned i = 0; i < 4096 && context_status == IndexGetStep::Working; ++i) context_status = stopped_context->step(); + if (context_status != IndexGetStep::Ready || stopped_context->intentActive()) return 138; + stopped_context.reset(); + std::array attempt_id{}; + attempt_id.back() = 1; + for (uint8_t blocked_request : {uint8_t(8), uint8_t(10)}) + { + std::array blocked_key{}; + std::fill(blocked_key.begin() + 32, blocked_key.end(), blocked_request); + const auto before = files; + auto send = std::make_unique(volume); + if (!send->begin(current, copy, {}, {blocked_key.data(), blocked_key.size()}, attempt_id, {}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 97; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) status = send->step(); + if (status != IndexedCommitStep::Invalid || files != before) return 98; + } + request.bytes.fill(30); + task_id.fill(31); + if (!protocol::encodeQueryRequest(request, {-900000000, -1800000000, 900000000, 1800000000}, + 7, {}, 4, {}, 512, request_bytes, sizeof(request_bytes), request_size)) return 221; + auto query = std::make_unique(volume); + if (!query->begin(current, copy, {}, {}, request, task_id, 3, {request_bytes, request_size}, {}, {}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 222; + auto query_status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && query_status == IndexedCommitStep::Working; ++i) query_status = query->step(); + if (query_status != IndexedCommitStep::Verified || !query->committed(current)) return 223; + copy = 1 - copy; + std::array reply_key{}; + std::copy(request.bytes.begin(), request.bytes.end(), reply_key.begin() + 32); + std::array attempt_key{}; + std::copy(reply_key.begin(), reply_key.end(), attempt_key.begin()); + std::copy(attempt_id.begin(), attempt_id.end(), attempt_key.begin() + 48); + std::array publication_key{}; + for (unsigned selection = 0; selection < 4; ++selection) + { + Destination local; + if (selection == 3) local.bytes.fill(99); + const ByteView after = selection == 0 ? ByteView{} : selection == 2 ? ByteView{reply_key.data(), reply_key.size()} + : ByteView{publication_key.data(), publication_key.size()}; + auto pending = std::make_unique(volume); + if (!pending->begin(current, local, after, frame, sizeof(frame))) return 103; + auto status = IndexedPendingStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedPendingStep::Working; ++i) + { + step_bytes = 0; + status = pending->step(); + if (step_bytes > 512) return 104; + } + PendingRequestView selected_request; + if (selection < 2) + { + if (status != IndexedPendingStep::Ready || !pending->selected(selected_request) || + selected_request.key != (selection ? reply_key : publication_key)) return 105; + if (selection && (selected_request.request.size != request_size || + std::memcmp(selected_request.request.data, request_bytes, request_size))) return 106; + } + else if (status != IndexedPendingStep::None || pending->selected(selected_request)) return 107; + } + for (unsigned repeat = 0; repeat < 2; ++repeat) + { + const auto before = files; + auto send = std::make_unique(volume); + if (!send->begin(current, copy, {}, {reply_key.data(), reply_key.size()}, attempt_id, {}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 99; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = send->step(); + if (step_bytes > 512) return 100; + } + if (repeat) + { + if (status != IndexedCommitStep::Invalid || files != before) return 101; + } + else + { + if (status != IndexedCommitStep::Verified || !send->committed(current)) return 102; + copy = 1 - copy; + } + } + auto unresolved = std::make_unique(volume); + if (!unresolved->begin(current, {}, {publication_key.data(), publication_key.size()}, frame, sizeof(frame))) return 108; + auto unresolved_status = IndexedPendingStep::Working; + for (unsigned i = 0; i < 8192 && unresolved_status == IndexedPendingStep::Working; ++i) unresolved_status = unresolved->step(); + if (unresolved_status != IndexedPendingStep::None) return 109; + unresolved.reset(); + uint8_t response[128]; + protocol::CmpWriter writer(response, sizeof(response)); + if (!writer.array(6) || !writer.unsignedInteger(1) || !writer.unsignedInteger(1) || !writer.unsignedInteger(2) || + !writer.binary({request.bytes.data(), request.bytes.size()}) || !writer.unsignedInteger(200) || + !writer.array(4) || !writer.binary({draft_id.data(), draft_id.size()}) || !writer.array(0) || + !writer.nil() || !writer.unsignedInteger(60)) return 224; + for (unsigned attempt = 0; attempt < 3; ++attempt) + { + if (attempt == 2) response[writer.size() - 1] = 61; + const auto before_reply = files; + const auto sequence = current.sequence; + auto reply = std::make_unique(volume); + if (!reply->begin(current, copy, {reply_key.data(), reply_key.size()}, 2, {response, writer.size()}, + workspace, frame, sizeof(frame), roots[1 - copy])) return 225; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = reply->step(); + if (step_bytes > 512) return 226; + } + if (attempt == 2) + { + if (status != IndexedCommitStep::Invalid || files != before_reply) return 227; + } + else + { + if (status != IndexedCommitStep::Verified || !reply->committed(current)) return 228; + if (attempt == 0) + { + if (current.sequence != sequence + 1) return 229; + copy = 1 - copy; + } + else if (current.sequence != sequence || files != before_reply) return 230; + } + } + SdIndexGet task_read(volume); + if (!task_read.begin(current, 10, {task_id.data(), task_id.size()}, frame, sizeof(frame))) return 231; + auto task_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && task_status == IndexGetStep::Working; ++i) task_status = task_read.step(); + TaskView completed_task; + if (task_status != IndexGetStep::Ready || !decodeTask({task_id.data(), task_id.size()}, task_read.value(), completed_task) || + completed_task.state != 3) return 232; + std::array transport_hash{}; + transport_hash.fill(0x44); + for (unsigned event = 0; event < 7; ++event) + { + if (event == 2) transport_hash[0] ^= 1; + const auto before_event = files; + const auto sequence = current.sequence; + const auto terminal = event == 3 ? TxAttemptState::CancelledBeforeSend : event == 6 ? TxAttemptState::Failed + : TxAttemptState::Delivered; + auto update = std::make_unique(volume); + if (!update->begin(current, copy, {attempt_key.data(), attempt_key.size()}, + event < 3 ? ByteView{transport_hash.data(), transport_hash.size()} : ByteView{}, terminal, {}, + outgoing_bytes, sizeof(outgoing_bytes), frame, sizeof(frame), roots[1 - copy])) return 89; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = update->step(); + if (step_bytes > 512) return 90; + } + const bool rejected = event == 2 || event == 3 || event == 6; + if (rejected) + { + if (status != IndexedCommitStep::Invalid || files != before_event) return 91; + } + else + { + if (status != IndexedCommitStep::Verified || !update->committed(current)) return 92; + if (event == 0 || event == 4) + { + if (current.sequence != sequence + 1) return 93; + copy = 1 - copy; + } + else if (current.sequence != sequence || files != before_event) return 94; + } + } + SdIndexGet retained_response(volume); + if (!retained_response.begin(current, 5, {reply_key.data(), reply_key.size()}, frame, sizeof(frame))) return 95; + auto retained_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && retained_status == IndexGetStep::Working; ++i) retained_status = retained_response.step(); + OutgoingView retained; + if (retained_status != IndexGetStep::Ready || !decodeOutgoing({reply_key.data(), reply_key.size()}, retained_response.value(), retained) || + retained.state != 4 || retained.terminal_data.size != writer.size() || retained.terminal_data.data[writer.size() - 1] != 60) return 96; + const auto before_completed_stop = files; + auto completed_stop = std::make_unique(volume); + if (!completed_stop->begin(current, copy, {task_id.data(), task_id.size()}, false, frame, sizeof(frame), roots[1 - copy])) return 85; + auto completed_status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 512 && completed_status == IndexedCommitStep::Working; ++i) completed_status = completed_stop->step(); + if (completed_status != IndexedCommitStep::Invalid || files != before_completed_stop) return 86; + // Exercise the production dispatcher against the disk-indexed adapter, + // including an asynchronous empty expiry scan and timeout-driven retry. + SdIndexGet expected_publication(volume); + if (!expected_publication.begin(current, 5, {publication_key.data(), publication_key.size()}, frame, sizeof(frame))) return 110; + auto expected_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && expected_status == IndexGetStep::Working; ++i) expected_status = expected_publication.step(); + OutgoingView publication; + if (expected_status != IndexGetStep::Ready || !decodeOutgoing({publication_key.data(), publication_key.size()}, expected_publication.value(), publication)) return 111; + const std::vector expected_request(publication.request.data, publication.request.data + publication.request.size); + const auto sequence_before_dispatch = current.sequence; + IndexWorkspaceOwner workspace_owner; + auto dispatch_store = std::make_unique(volume, current, copy, roots[0], roots[1], workspace_owner, workspace, frame, sizeof(frame)); + chat::MeshAdapterRouter router; + RequestDispatcher dispatcher(router, *dispatch_store, 100, 1000); + DispatchResult dispatched; + for (unsigned i = 0; i < 32768; ++i) + { + step_bytes = 0; + dispatched = dispatcher.dispatchOne({}); + if (step_bytes > 512 || dispatch_store->needsRecovery()) return 112; + if (dispatched.status == DispatchStatus::Submitted) break; + } + if (dispatched.status != DispatchStatus::Submitted || router.sends != 1 || router.sent_bytes != expected_request || + current.sequence != sequence_before_dispatch + 2) return 113; + StoredTime timeout; + timeout.monotonic_ms = 2000; + bool expired = false; + for (unsigned i = 0; i < 32768; ++i) + { + step_bytes = 0; + dispatcher.dispatchOne(timeout); + if (step_bytes > 512 || dispatch_store->needsRecovery() || router.sends != 1) return 114; + if (dispatch_store->expirationChanged() && !dispatch_store->busy()) + { + expired = true; + break; + } + } + if (!expired || current.sequence != sequence_before_dispatch + 3) return 115; + timeout.monotonic_ms = 2200; + for (unsigned i = 0; i < 32768; ++i) + { + step_bytes = 0; + dispatched = dispatcher.dispatchOne(timeout); + if (step_bytes > 512 || dispatch_store->needsRecovery()) return 116; + if (dispatched.status == DispatchStatus::Submitted) break; + } + if (dispatched.status != DispatchStatus::Submitted || router.sends != 2 || router.sent_bytes != expected_request || + current.sequence != sequence_before_dispatch + 5) return 117; + struct ReceiptCrypto : protocol::RecordCrypto + { + bool sha256(ByteView input, uint8_t output[32]) override + { + chat::reticulum::fullHash(input.data, input.size, output); + return true; + } + protocol::VerificationResult verifyEd25519(ByteView, ByteView, ByteView) override { return protocol::VerificationResult::CryptoUnavailable; } + } receipt_crypto; + uint8_t page_cache[2048]; + uint8_t random_counter = 60; + IndexedQueryStorePort port( + volume, current, copy, roots[0], roots[1], {}, workspace_owner, workspace, frame, sizeof(frame), page_cache, sizeof(page_cache), receipt_crypto, + [](void* context, uint8_t out[16]) + { const auto next = ++*static_cast(context); std::memset(out, next, 16); return true; }, + [](void*) + { return StoredTime{}; }, + &random_counter); + auto persist = [&](QueryPersistence result) + { + for (unsigned i = 0; i < 8192 && result == QueryPersistence::Pending; ++i) + { + step_bytes = 0; + result = port.pollPersistence(); + if (workspace_owner.heldBy(&port)) + { + const auto before_wait = step_bytes; + const auto waiting = dispatcher.dispatchOne(timeout); + if (waiting.status != DispatchStatus::Deferred || dispatch_store->needsRecovery() || step_bytes != before_wait) + return QueryPersistence::Rejected; + } + if (step_bytes > 512) return QueryPersistence::Rejected; + } + return result; + }; + RequestId first_page_id; + std::vector first_page_response; + for (unsigned page_number = 0; page_number < 2; ++page_number) + { + if (!port.newRequestId(request) || !protocol::encodeQueryRequest(request, {-900000000, -1800000000, 900000000, 1800000000}, + 7, {}, 4, {}, 512, request_bytes, sizeof(request_bytes), request_size)) return 118; + DirectoryEntry directory; + const int other_owner = 0; + if (!workspace_owner.acquire(&other_owner)) return 131; + const std::vector prior_frame(frame, frame + sizeof(frame)), prior_encoding(outgoing_bytes, outgoing_bytes + sizeof(outgoing_bytes)); + const auto submitted = port.submit(directory, request, {request_bytes, request_size}); + step_bytes = 0; + if (submitted != QueryPersistence::Pending || port.pollPersistence() != QueryPersistence::Pending || step_bytes || + std::memcmp(frame, prior_frame.data(), sizeof(frame)) || std::memcmp(outgoing_bytes, prior_encoding.data(), sizeof(outgoing_bytes))) return 132; + workspace_owner.release(&other_owner); + if (persist(submitted) != QueryPersistence::Committed || workspace_owner.holder()) return 119; + protocol::CmpWriter page_writer(response, sizeof(response)); + if (!page_writer.array(6) || !page_writer.unsignedInteger(1) || !page_writer.unsignedInteger(1) || !page_writer.unsignedInteger(2) || + !page_writer.binary({request.bytes.data(), request.bytes.size()}) || !page_writer.unsignedInteger(200) || + !page_writer.array(4) || !page_writer.binary({draft_id.data(), draft_id.size()}) || !page_writer.array(0) || + !page_writer.nil() || !page_writer.unsignedInteger(60)) return 120; + protocol::QueryPageView page; + if (!protocol::decodeQueryPage({response, page_writer.size()}, request, 512, 4, page)) return 121; + const auto pending = port.commitPage({}, request, {response, page_writer.size()}, page); + step_bytes = 0; + if (pending != QueryPersistence::Pending || port.page(page) || port.accepted({}, request, {response, page_writer.size()}) || step_bytes) return 122; + if (persist(pending) != QueryPersistence::Committed) return 123; + step_bytes = 0; + if (!port.page(page) || page.count || !port.accepted({}, request, {response, page_writer.size()}) || step_bytes) return 124; + for (unsigned i = 0; i < 8192 && port.maintenancePending(); ++i) + { + step_bytes = 0; + port.maintenanceStep(); + if (step_bytes > 512) return 125; + } + if (port.maintenancePending()) return 126; + if (!page_number) + { + first_page_id = request; + first_page_response.assign(response, response + page_writer.size()); + } + } + step_bytes = 0; + if (port.accepted({}, first_page_id, {first_page_response.data(), first_page_response.size()}) || step_bytes || !port.maintenancePending()) return 127; + for (unsigned i = 0; i < 8192 && port.maintenancePending(); ++i) port.maintenanceStep(); + step_bytes = 0; + if (!port.accepted({}, first_page_id, {first_page_response.data(), first_page_response.size()}) || step_bytes) return 128; + first_page_response.back() = 61; + if (port.accepted({}, first_page_id, {first_page_response.data(), first_page_response.size()})) return 129; + for (unsigned i = 0; i < 8192 && port.maintenancePending(); ++i) port.maintenanceStep(); + if (port.accepted({}, first_page_id, {first_page_response.data(), first_page_response.size()})) return 130; + files.clear(); + index_directories.clear(); + return 0; +} + +int checkCheckpointIndexedRead() +{ + using namespace ::geocaching; + using namespace ::geocaching::storage; + files.clear(); + VolumeInstance volume{}; + const auto format = encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + std::array key{}; + uint8_t value[128], payload[256], frame[512]; + DraftView draft; + draft.name = "Checkpoint cache"; + size_t value_size = 0; + if (!encodeDraft({key.data(), key.size()}, draft, value, sizeof(value), value_size)) return 235; + protocol::CmpWriter writer(payload, sizeof(payload)); + if (!writer.array(2) || !writer.unsignedInteger(0) || !writer.array(1) || !writer.array(3) || + !writer.unsignedInteger(4) || !writer.binary({key.data(), key.size()}) || !writer.binary({value, value_size})) return 236; + RecordHeader header; + if (!makeRecordHeader(RecordKind::CheckpointPage, 12, {payload, writer.size()}, header)) return 237; + std::vector encoded(header.begin(), header.end()); + encoded.insert(encoded.end(), payload, payload + writer.size()); + for (char slot : {'a', 'b'}) + { + CheckpointIndexCursor cursor; + IndexedMutation hint; + if (!cursor.open({encoded.data(), encoded.size()}, 12, slot, 17) || !cursor.next(hint) || !cursor.complete()) return 238; + IndexEntryBytes index; + IndexedMutation decoded; + if (!encodeIndexEntry(volume, hint, index) || !decodeIndexEntry({index.data(), index.size()}, volume, decoded) || + decoded.location.source != (slot == 'a' ? IndexedValueSource::CheckpointA : IndexedValueSource::CheckpointB)) return 239; + const std::string path = std::string("/trailmate/geocaching/.state/checkpoint/") + slot + ".gcs"; + auto& file = files[path]; + file.assign(17, 0); + file.insert(file.end(), encoded.begin(), encoded.end()); + for (unsigned changed = 0; changed < 2; ++changed) + { + if (changed) + { + RecordHeader replacement; + if (!makeRecordHeader(RecordKind::CheckpointPage, 13, {payload, writer.size()}, replacement)) return 240; + std::copy(replacement.begin(), replacement.end(), file.begin() + 17); + } + SdIndexedValueReader read(volume); + if (!read.begin(decoded, frame, sizeof(frame))) return 241; + auto status = IndexedReadStep::Working; + for (unsigned i = 0; i < 128 && status == IndexedReadStep::Working; ++i) + { + step_bytes = 0; + status = read.step(); + if (step_bytes > 512) return 242; + } + if (status != (changed ? IndexedReadStep::Invalid : IndexedReadStep::Ready)) return 243; + if (!changed && (read.value().size != value_size || std::memcmp(read.value().data, value, value_size))) return 244; + if (changed && read.value().data) return 245; + } + } + files.clear(); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + CheckpointCandidate selected; + selected.state = CheckpointCandidateState::Verified; + selected.sequence = 12; + chat::reticulum::fullHash(encoded.data(), encoded.size(), selected.digest.data()); + uint8_t tail[64]; + protocol::CmpWriter tail_writer(tail, sizeof(tail)); + if (!tail_writer.array(3) || !tail_writer.unsignedInteger(1) || !tail_writer.unsignedInteger(1) || + !tail_writer.binary({selected.digest.data(), selected.digest.size()}) || + !makeRecordHeader(RecordKind::CheckpointTail, 12, {tail, tail_writer.size()}, header)) return 246; + auto& checkpoint = files["/trailmate/geocaching/.state/checkpoint/a.gcs"]; + checkpoint = encoded; + checkpoint.insert(checkpoint.end(), header.begin(), header.end()); + checkpoint.insert(checkpoint.end(), tail, tail + tail_writer.size()); + const auto source_files = files; + struct TestDigest + { + std::vector bytes; + void update(const uint8_t* data, size_t size) { bytes.insert(bytes.end(), data, data + size); } + bool finalize(uint8_t* out, size_t) + { + chat::reticulum::fullHash(bytes.data(), bytes.size(), out); + return true; + } + }; + for (unsigned correct = 0; correct < 2; ++correct) + { + files = source_files; + index_directories.clear(); + auto expected = selected; + if (!correct) expected.digest[0] ^= 1; + TestDigest digest; + IndexRootBytes root_bytes; + auto import = std::make_unique>(volume, digest); + if (!import->begin('a', expected, frame, sizeof(frame), root_bytes)) return 247; + auto status = IndexRootWriteStep::Working; + for (unsigned i = 0; i < 4096 && status == IndexRootWriteStep::Working; ++i) + { + step_bytes = 0; + status = import->step(); + if (step_bytes > 512) return 248; + } + IndexRootView root; + if (!correct) + { + if (status != IndexRootWriteStep::Invalid || import->selected(root) || + files.count("/trailmate/geocaching/.state/index/root.h0") || files.count("/trailmate/geocaching/.state/index/root.h1")) return 249; + continue; + } + if (status != IndexRootWriteStep::Verified || !import->selected(root) || root.sequence != 12 || + files["/trailmate/geocaching/.state/index/root.h0"] != files["/trailmate/geocaching/.state/index/root.h1"] || + files["/trailmate/geocaching/.state/checkpoint/a.gcs"] != source_files.at("/trailmate/geocaching/.state/checkpoint/a.gcs")) return 250; + SdIndexGet read(volume); + if (!read.begin(root, 4, {key.data(), key.size()}, frame, sizeof(frame))) return 251; + auto read_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && read_status == IndexGetStep::Working; ++i) read_status = read.step(); + if (read_status != IndexGetStep::Ready || read.value().size != value_size || std::memcmp(read.value().data, value, value_size)) return 252; + } + files = source_files; + index_directories.clear(); + draft.generation = 2; + draft.name = "After checkpoint"; + if (!encodeDraft({key.data(), key.size()}, draft, value, sizeof(value), value_size)) return 70; + MutationView updated{4, {key.data(), key.size()}, {value, value_size}, false}; + SdGeocachingJournal suffix(volume); + if (suffix.begin(12, &updated, 1) != JournalWriteResult::InProgress) return 71; + auto suffix_status = JournalWriteResult::InProgress; + for (unsigned i = 0; i < 256 && suffix_status == JournalWriteResult::InProgress; ++i) suffix_status = suffix.step(); + if (suffix_status != JournalWriteResult::Verified) return 72; + uint8_t validation_frame[512]; + MutationView mutations[3]; + for (unsigned scenario = 0; scenario < 3; ++scenario) + { + if (scenario == 2) + { + files.clear(); + index_directories.clear(); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + } + const auto before = files; + IndexRootBytes roots[2]; + auto recovery = std::make_unique>(volume, roots[0], roots[1], frame, sizeof(frame), + validation_frame, sizeof(validation_frame), mutations, 3); + auto status = IndexedRecoveryStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedRecoveryStep::Working; ++i) + { + step_bytes = 0; + status = recovery->step(); + if (step_bytes > 512) return 73; + } + IndexRootView root; + unsigned copy = 0; + if (status != IndexedRecoveryStep::Restored || !recovery->selected(root, copy) || root.sequence != (scenario == 2 ? 0 : 13)) return 74; + if (scenario == 1 && files != before) return 75; + if (scenario != 2) + { + SdIndexGet read(volume); + if (!read.begin(root, 4, {key.data(), key.size()}, frame, sizeof(frame))) return 76; + auto read_status = IndexGetStep::Working; + for (unsigned i = 0; i < 512 && read_status == IndexGetStep::Working; ++i) read_status = read.step(); + if (read_status != IndexGetStep::Ready || read.value().size != value_size || std::memcmp(read.value().data, value, value_size)) return 77; + } + } + files.clear(); + index_directories.clear(); + return 0; +} + +int checkIndexedDownloadReceipt(const char* path) +{ + using namespace ::geocaching; + using namespace ::geocaching::storage; + struct Crypto : protocol::RecordCrypto + { + bool sha256(ByteView bytes, uint8_t out[32]) override + { + chat::reticulum::fullHash(bytes.data, bytes.size, out); + return true; + } + protocol::VerificationResult verifyEd25519(ByteView key, ByteView signature, ByteView message) override + { + return ed25519_verify(signature.data, message.data, message.size, key.data) ? protocol::VerificationResult::Valid : protocol::VerificationResult::InvalidSignature; + } + } crypto; + std::ifstream input(path, std::ios::binary); + std::vector response((std::istreambuf_iterator(input)), {}); + protocol::CmpReader envelope({response.data(), response.size()}); + size_t fields = 0; + uint64_t number = 0; + ByteView id_bytes; + if (!envelope.array(fields, 6) || !envelope.unsignedInteger(number) || !envelope.unsignedInteger(number) || + !envelope.unsignedInteger(number) || !envelope.binary(id_bytes, 16) || id_bytes.size != 16) return 139; + RequestId id; + std::memcpy(id.bytes.data(), id_bytes.data, 16); + protocol::GetResponseView reply; + uint8_t verification[512], frame[1024], request[256], outgoing[1024]; + protocol::VerifiedRecordView record; + if (!protocol::decodeGetResponse({response.data(), response.size()}, id, 8192, reply) || + protocol::verifyGeocache(reply.signed_cache, crypto, verification, sizeof(verification), record) != protocol::VerificationResult::Valid) return 140; + protocol::CmpReader signed_record(reply.signed_cache); + ByteView encoded, signature; + if (!signed_record.array(fields, 2) || !signed_record.binary(encoded, 4096) || !signed_record.binary(signature, 64)) return 141; + const auto signature_offset = static_cast(signature.data - response.data()); + files.clear(); + index_directories.clear(); + VolumeInstance volume{}; + const auto format = encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + IndexRootBytes roots[2]; + SdIndexInitialize initialize(volume); + if (!initialize.begin(roots[0])) return 142; + auto initialized = IndexRootWriteStep::Working; + for (unsigned i = 0; i < 128 && initialized == IndexRootWriteStep::Working; ++i) initialized = initialize.step(); + if (initialized != IndexRootWriteStep::Verified) return 143; + roots[1] = roots[0]; + IndexRootView root; + if (!decodeIndexRoot({roots[0].data(), roots[0].size()}, volume, root)) return 144; + unsigned copy = 0; + size_t request_size = 0; + if (!protocol::encodeGetRequest(id, record.id, &record.hash, nullptr, 8192, request, sizeof(request), request_size)) return 145; + QueuedRequestWorkspace workspace(outgoing, sizeof(outgoing)); + std::array task{}; + auto create = std::make_unique(volume); + if (!create->begin(root, copy, {}, {}, id, task, 2, {request, request_size}, {}, + {{record.id.bytes.data(), 32}, {record.hash.bytes.data(), 32}, 1}, workspace, + frame, sizeof(frame), roots[1 - copy])) return 146; + auto created = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && created == IndexedCommitStep::Working; ++i) created = create->step(); + if (created != IndexedCommitStep::Verified || !create->committed(root)) return 147; + copy = 1 - copy; + std::array key{}; + std::copy(id.bytes.begin(), id.bytes.end(), key.begin() + 32); + for (unsigned scenario = 0; scenario < 4; ++scenario) + { + auto received = response; + if (scenario == 1) received[signature_offset] ^= 1; + const auto before = files; + const auto sequence = root.sequence; + auto receipt = std::make_unique(volume, crypto); + if (!receipt->begin(root, copy, {key.data(), key.size()}, scenario == 0 ? 2 : 1, {received.data(), received.size()}, + workspace, frame, sizeof(frame), verification, sizeof(verification), roots[1 - copy])) return 148; + auto status = IndexedCommitStep::Working; + for (unsigned i = 0; i < 8192 && status == IndexedCommitStep::Working; ++i) + { + step_bytes = 0; + status = receipt->step(); + if (step_bytes > 512) return 149; + } + if (scenario < 2) + { + if (status != IndexedCommitStep::Invalid || files != before) return 150; + } + else + { + if (status != IndexedCommitStep::Verified || !receipt->committed(root)) return 151; + if (scenario == 2) + { + if (root.sequence != sequence + 1) return 152; + copy = 1 - copy; + } + else if (root.sequence != sequence || files != before) return 153; + } + } + SdIndexedDownloadContext context(volume); + if (!context.begin(root, {key.data(), key.size()}, 1, frame, sizeof(frame))) return 154; + auto loaded = IndexGetStep::Working; + for (unsigned i = 0; i < 4096 && loaded == IndexGetStep::Working; ++i) loaded = context.step(); + OutgoingView saved; + TaskView running; + CacheHeadView head; + if (loaded != IndexGetStep::Ready || !context.view(saved, running, head) || saved.state != 4 || running.state != 1 || + head.current_hash.size || !context.intentActive() || saved.terminal_data.size != response.size() || + std::memcmp(saved.terminal_data.data, response.data(), response.size())) return 155; + files.clear(); + index_directories.clear(); + return 0; +} + +int main(int argc, char** argv) +{ + if (argc != 2) return 1; + if (const int result = checkIndexedDownloadReceipt(argv[1])) return result; + if (const int result = checkCheckpointIndexedRead()) return result; + if (const int result = checkIndexedDraftPublication()) return result; + if (const int result = checkIndexedCommitCapacity()) return result; + if (const int result = checkIndexTransactions()) return result; + using namespace platform::esp::arduino_common::geocaching; + ::geocaching::storage::VolumeInstance volume{}; + auto format = ::geocaching::storage::encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + const uint8_t payload[] = {0x93, 1, 0, 0x91, 0x93, 5, 0xc4, 1, 0xab, 0xc0}; + ::geocaching::storage::RecordHeader header; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 1, {payload, sizeof(payload)}, header)) return 1; + auto& file = files["/trailmate/geocaching/.state/journal/0000000000000001.gcj"]; + file.assign(header.begin(), header.end()); + file.insert(file.end(), payload, payload + sizeof(payload)); + uint8_t bytes[256]; + ::geocaching::storage::MutationView mutation; + ::geocaching::storage::TransactionView transaction; + SdJournalReplay replay(volume, 0, {true, 1, 1}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(replay, transaction) != ReplayStep::Transaction || replay.appliedSequence() != 0 || + nextReady(replay, transaction) != ReplayStep::Transaction || replay.acknowledgeApplied(2) || + !replay.acknowledgeApplied(1) || replay.acknowledgeApplied(1) || nextReady(replay, transaction) != ReplayStep::JournalComplete) return 2; + SdJournalReplay gap(volume, 1, {true, 2, 2}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(gap, transaction) != ReplayStep::Corrupt || gap.appliedSequence() != 1) return 3; + uint8_t next_payload[sizeof(payload)]; + std::memcpy(next_payload, payload, sizeof(payload)); + next_payload[2] = 1; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 2, {next_payload, sizeof(next_payload)}, header)) return 5; + file.insert(file.end(), header.begin(), header.end()); + file.insert(file.end(), next_payload, next_payload + sizeof(next_payload)); + SdJournalReplay spanning(volume, 1, {true, 1, 1}, bytes, sizeof(bytes), &mutation, 1); + ::geocaching::storage::TransactionIndexCursor index; + ::geocaching::storage::IndexedMutation located; + if (spanning.pendingIndex(index) || nextReady(spanning, transaction) != ReplayStep::Transaction || + !spanning.pendingIndex(index) || !index.next(located) || located.location.segment_first_sequence != 1 || + located.location.record_sequence != 2 || located.location.frame_offset != 24 + sizeof(payload) || + !spanning.acknowledgeApplied(2) || nextReady(spanning, transaction) != ReplayStep::JournalComplete) return 6; + if (spanning.pendingIndex(index) || index.next(located)) return 17; + SdJournalReplay retry(volume, 0, {true, 1, 1}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(retry, transaction) != ReplayStep::Transaction || !retry.acknowledgeApplied(1)) return 8; + fail_read_once = true; + if (nextReady(retry, transaction) != ReplayStep::RetryLater || retry.appliedSequence() != 1 || + nextReady(retry, transaction) != ReplayStep::Transaction || + !retry.acknowledgeApplied(2) || nextReady(retry, transaction) != ReplayStep::JournalComplete) return 9; + file.pop_back(); + SdJournalReplay truncated(volume, 0, {true, 1, 1}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(truncated, transaction) != ReplayStep::Transaction || !truncated.acknowledgeApplied(1) || + nextReady(truncated, transaction) != ReplayStep::TailTruncated || truncated.appliedSequence() != 1) return 7; + SdJournalReplay broken_middle(volume, 0, {true, 1, 3}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(broken_middle, transaction) != ReplayStep::Transaction || !broken_middle.acknowledgeApplied(1) || + nextReady(broken_middle, transaction) != ReplayStep::Corrupt) return 10; + file.resize(24 + sizeof(payload)); + auto& second_file = files["/trailmate/geocaching/.state/journal/0000000000000002.gcj"]; + second_file.assign(header.begin(), header.end()); + second_file.insert(second_file.end(), next_payload, next_payload + sizeof(next_payload)); + SdJournalReplay cross_segment(volume, 0, {true, 1, 2}, bytes, sizeof(bytes), &mutation, 1); + if (nextReady(cross_segment, transaction) != ReplayStep::Transaction || !cross_segment.acknowledgeApplied(1) || + nextReady(cross_segment, transaction) != ReplayStep::Transaction || + !cross_segment.acknowledgeApplied(2) || nextReady(cross_segment, transaction) != ReplayStep::JournalComplete) return 11; + uint8_t key = 1, value = 7, state_payload[64]{}; + size_t state_payload_size = 0; + ::geocaching::storage::MutationView update{5, {&key, 1}, {&value, 1}, false}; + if (!::geocaching::storage::encodeTransaction(0, &update, 1, state_payload, sizeof(state_payload), state_payload_size) || + !::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 1, {state_payload, state_payload_size}, header)) return 12; + file.assign(header.begin(), header.end()); + file.insert(file.end(), state_payload, state_payload + state_payload_size); + uint8_t first[128]{}, second[128]{}; + ::geocaching::storage::LogicalState state(first, second, sizeof(first)); + SdJournalReplay application(volume, 0, {true, 1, 1}, bytes, sizeof(bytes), &mutation, 1); + if (applyReady(application, state, [](const auto&) + { return false; }) != ReplayStep::ApplicationRejected || + application.appliedSequence() != 0 || state.view().size() != 0) return 13; + if (applyReady(application, state, [](const auto&) + { return true; }) != ReplayStep::Applied || + application.appliedSequence() != 1 || state.view().size() != 1) return 14; + ::geocaching::ByteView stored; + if (!state.view().find(5, {&key, 1}, stored) || stored.size != 1 || stored.data[0] != 7 || + applyReady(application, state, [](const auto&) + { return true; }) != ReplayStep::JournalComplete) return 15; + std::vector large_value(1300, 0x5a), large_payload(1400), large_frame(1500); + update.value = {large_value.data(), large_value.size()}; + if (!::geocaching::storage::encodeTransaction(0, &update, 1, large_payload.data(), large_payload.size(), state_payload_size) || + !::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 1, {large_payload.data(), state_payload_size}, header)) return 18; + file.assign(header.begin(), header.end()); + file.insert(file.end(), large_payload.begin(), large_payload.begin() + state_payload_size); + SdJournalReplay indexed(volume, 0, {true, 1, 1}, large_frame.data(), large_frame.size(), &mutation, 1); + bool ready = false, full_slice = false; + for (unsigned step = 0; step < 128 && !ready; ++step) + { + step_bytes = 0; + const auto result = indexed.next(transaction); + if (step_bytes > 512) return 19; + full_slice = full_slice || step_bytes == 512; + if (result == ReplayStep::Advancing) + { + if (indexed.pendingIndex(index)) return 20; + } + else if (result == ReplayStep::Transaction) ready = true; + else return 21; + } + if (!ready || !full_slice || !indexed.pendingIndex(index) || !index.next(located) || located.erase || + located.location.value_size != large_value.size() || located.location.frame_offset || + located.location.value_offset + located.location.value_size > file.size() || + std::memcmp(file.data() + located.location.value_offset, large_value.data(), large_value.size())) return 22; + const auto saved_location = located; + if (!indexed.acknowledgeApplied(1) || indexed.pendingIndex(index) || index.next(located)) return 23; + for (unsigned scenario = 0; scenario < 5; ++scenario) + { + auto hint = saved_location; + uint8_t wrong_key = 0xee; + uint8_t small[64]; + if (scenario == 1) ++hint.location.value_offset; + if (scenario == 2) hint.key = {&wrong_key, 1}; + SdIndexedValueReader reader(volume); + if (!reader.begin(hint, scenario == 3 ? small : large_frame.data(), scenario == 3 ? sizeof(small) : large_frame.size())) return 24; + auto result = IndexedReadStep::Working; + for (unsigned step = 0; step < 128 && result == IndexedReadStep::Working; ++step) + { + if (reader.value().data) return 25; + if (scenario == 4 && step == 8) + { + auto changed = volume; + changed[0] = 1; + const auto changed_header = ::geocaching::storage::encodeVolumeHeader(changed); + files["/trailmate/geocaching/.state/format.bin"] = {changed_header.begin(), changed_header.end()}; + } + step_bytes = 0; + result = reader.step(); + if (step_bytes > 512) return 26; + } + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + if (scenario == 0) + { + if (result != IndexedReadStep::Ready || reader.value().size != large_value.size() || + std::memcmp(reader.value().data, large_value.data(), large_value.size())) return 27; + } + else if (reader.value().data || result != (scenario == 3 ? IndexedReadStep::WorkspaceTooSmall : scenario == 4 ? IndexedReadStep::VolumeChanged + : IndexedReadStep::Invalid)) return 28; + } + auto deletion = saved_location; + deletion.erase = true; + deletion.location = {2, 2, 0, 0, 0}; + ::geocaching::storage::MutationView erased{5, {&key, 1}, {}, true}; + if (!::geocaching::storage::encodeTransaction(1, &erased, 1, state_payload, sizeof(state_payload), state_payload_size) || + !::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 2, {state_payload, state_payload_size}, header)) return 29; + second_file.assign(header.begin(), header.end()); + second_file.insert(second_file.end(), state_payload, state_payload + state_payload_size); + char shard_path[80]; + if (!indexShardPath('a', 5, {&key, 1}, shard_path, sizeof(shard_path))) return 30; + auto& shard = files[shard_path]; + ::geocaching::storage::IndexEntryBytes index_bytes; + if (!::geocaching::storage::encodeIndexEntry(volume, saved_location, index_bytes)) return 31; + shard.assign(index_bytes.begin(), index_bytes.end()); + if (!::geocaching::storage::encodeIndexEntry(volume, deletion, index_bytes)) return 32; + shard.insert(shard.end(), index_bytes.begin(), index_bytes.end()); + uint8_t collision[2] = {key, 0}; + bool collided = false; + for (unsigned i = 0; i < 256; ++i) + { + collision[1] = static_cast(i); + if ((::sys::crc32(collision, 2) & 0xff) == (::sys::crc32(&key, 1) & 0xff)) + { + collided = true; + break; + } + } + auto other = deletion; + other.key = {collision, 2}; + other.location = {3, 3, 0, 0, 0}; + if (!collided || !::geocaching::storage::encodeIndexEntry(volume, other, index_bytes)) return 33; + shard.insert(shard.end(), index_bytes.begin(), index_bytes.end()); + for (uint64_t visible = 0; visible <= 3; ++visible) + { + SdIndexLookup lookup(volume, 'a', visible, visible * ::geocaching::storage::kIndexEntrySize); + if (!lookup.begin(5, {&key, 1})) return 34; + auto result = IndexLookupStep::Working; + for (unsigned step = 0; step < 128 && result == IndexLookupStep::Working; ++step) + { + step_bytes = 0; + result = lookup.step(); + if (step_bytes > 512) return 35; + } + ::geocaching::storage::IndexedMutation hint; + if (!visible) + { + if (result != IndexLookupStep::NotFound || lookup.result(hint)) return 36; + continue; + } + if (result != IndexLookupStep::Found || !lookup.result(hint) || hint.location.record_sequence != (visible == 1 ? 1 : 2) || hint.erase != (visible >= 2)) return 37; + SdIndexedValueReader reader(volume); + if (!reader.begin(hint, large_frame.data(), large_frame.size())) return 38; + auto read = IndexedReadStep::Working; + for (unsigned step = 0; step < 128 && read == IndexedReadStep::Working; ++step) read = reader.step(); + if (read != (visible == 1 ? IndexedReadStep::Ready : IndexedReadStep::Erased)) return 39; + } + shard.pop_back(); + SdIndexLookup old_prefix(volume, 'a', 1, ::geocaching::storage::kIndexEntrySize); + if (!old_prefix.begin(5, {&key, 1})) return 51; + auto prefix_result = IndexLookupStep::Working; + for (unsigned step = 0; step < 64 && prefix_result == IndexLookupStep::Working; ++step) prefix_result = old_prefix.step(); + if (prefix_result != IndexLookupStep::Found) return 52; + SdIndexLookup torn(volume, 'a', 3, 3 * ::geocaching::storage::kIndexEntrySize); + if (!torn.begin(5, {&key, 1})) return 40; + auto invalid = IndexLookupStep::Working; + for (unsigned step = 0; step < 64 && invalid == IndexLookupStep::Working; ++step) invalid = torn.step(); + if (invalid != IndexLookupStep::Invalid) return 41; + const auto append_entry = [&](const ::geocaching::storage::IndexedMutation& entry) + { + SdIndexAppend writer(volume, 'b'); + if (!writer.begin(entry)) return IndexAppendStep::Invalid; + auto result = IndexAppendStep::Working; + for (unsigned step = 0; step < 128 && result == IndexAppendStep::Working; ++step) + { + step_bytes = 0; + result = writer.step(); + if (step_bytes > 512) + { + exceeded_budget = true; + return IndexAppendStep::Invalid; + } + } + return result; + }; + char written_path[80]; + if (!indexShardPath('b', saved_location.table, saved_location.key, written_path, sizeof(written_path)) || + append_entry(saved_location) != IndexAppendStep::Verified || files[written_path].size() != ::geocaching::storage::kIndexEntrySize) return 42; + const auto flushed = flush_calls; + if (append_entry(saved_location) != IndexAppendStep::Verified || files[written_path].size() != ::geocaching::storage::kIndexEntrySize || flush_calls != flushed + 1) return 43; + if (append_entry(deletion) != IndexAppendStep::Verified || files[written_path].size() != 2 * ::geocaching::storage::kIndexEntrySize) return 44; + auto same_transaction = other; + same_transaction.location = {2, 2, 0, 0, 0}; + if (append_entry(same_transaction) != IndexAppendStep::Verified || files[written_path].size() != 3 * ::geocaching::storage::kIndexEntrySize) return 48; + const auto durable_prefix = files[written_path]; + if (append_entry(deletion) != IndexAppendStep::Verified || files[written_path] != durable_prefix) return 49; + auto conflicting = deletion; + conflicting.location.frame_offset = 1; + if (append_entry(conflicting) != IndexAppendStep::Invalid || files[written_path] != durable_prefix) return 50; + if (append_entry(saved_location) != IndexAppendStep::Invalid || files[written_path] != durable_prefix) return 45; + auto next = deletion; + next.location.segment_first_sequence = next.location.record_sequence = 3; + write_limit = 7; + const auto partial = append_entry(next); + write_limit = SIZE_MAX; + if (partial != IndexAppendStep::IoError || files[written_path].size() != durable_prefix.size() + 7 || + append_entry(next) != IndexAppendStep::Invalid) return 46; + auto separate = saved_location; + separate.table = 4; + flush_ok = false; + const auto failed_flush = append_entry(separate); + flush_ok = true; + if (failed_flush != IndexAppendStep::IoError || append_entry(separate) != IndexAppendStep::Verified) return 47; + char head_paths[2][80]; + const auto bucket = static_cast(::sys::crc32(&key, 1)); + for (unsigned copy = 0; copy < 2; ++copy) + { + ::geocaching::storage::IndexShardHead head{17, copy + 1, (copy + 1) * ::geocaching::storage::kIndexEntrySize, 5, bucket}; + ::geocaching::storage::IndexShardHeadBytes encoded; + if (!indexShardHeadPath('a', 5, {&key, 1}, copy, head_paths[copy], sizeof(head_paths[copy])) || + !::geocaching::storage::encodeIndexShardHead(volume, head, encoded)) return 53; + files[head_paths[copy]] = {encoded.begin(), encoded.end()}; + } + for (unsigned visible = 1; visible <= 2; ++visible) + { + SdIndexHeadReader heads(volume, 'a', 17, visible); + if (!heads.begin(5, {&key, 1})) return 54; + auto result = IndexHeadReadStep::Working; + ::geocaching::storage::IndexShardHead selected; + for (unsigned step = 0; step < 8 && result == IndexHeadReadStep::Working; ++step) + { + if (heads.selected(selected)) return 55; + step_bytes = 0; + result = heads.step(); + if (step_bytes > 512) return 56; + } + if (result != IndexHeadReadStep::Ready || !heads.selected(selected) || selected.sequence != visible || + selected.length != visible * ::geocaching::storage::kIndexEntrySize || heads.selectedCopy() != int(visible - 1)) return 57; + } + const auto valid_head = files[head_paths[0]]; + for (unsigned fault = 0; fault < 3; ++fault) + { + if (fault == 0) files[head_paths[0]][0] ^= 1; + if (fault == 1) files.erase(head_paths[0]); + SdIndexHeadReader heads(volume, 'a', fault == 2 ? 18 : 17, 2); + if (!heads.begin(5, {&key, 1})) return 58; + auto result = IndexHeadReadStep::Working; + for (unsigned step = 0; step < 8 && result == IndexHeadReadStep::Working; ++step) result = heads.step(); + ::geocaching::storage::IndexShardHead selected; + if (result != IndexHeadReadStep::Invalid || heads.selected(selected) || heads.selectedCopy() != -1) return 59; + files[head_paths[0]] = valid_head; + } + const auto preserved = files[head_paths[1]]; + const auto write_head = [&]() + { + SdIndexHeadWriter writer(volume, 'a'); + ::geocaching::storage::IndexShardHead head{17, 3, 3 * ::geocaching::storage::kIndexEntrySize, 5, bucket}; + if (!writer.begin({&key, 1}, 0, head)) return IndexHeadWriteStep::Invalid; + auto result = IndexHeadWriteStep::Working; + for (unsigned step = 0; step < 16 && result == IndexHeadWriteStep::Working; ++step) + { + step_bytes = 0; + result = writer.step(); + if (step_bytes > 512) + { + exceeded_budget = true; + return IndexHeadWriteStep::Invalid; + } + } + return result; + }; + if (write_head() != IndexHeadWriteStep::Verified || files[head_paths[1]] != preserved) return 60; + for (unsigned visible = 2; visible <= 3; ++visible) + { + SdIndexHeadReader reader(volume, 'a', 17, visible); + if (!reader.begin(5, {&key, 1})) return 61; + auto result = IndexHeadReadStep::Working; + for (unsigned step = 0; step < 8 && result == IndexHeadReadStep::Working; ++step) result = reader.step(); + ::geocaching::storage::IndexShardHead selected; + if (result != IndexHeadReadStep::Ready || !reader.selected(selected) || selected.sequence != visible) return 62; + } + write_limit = 7; + const auto short_head = write_head(); + write_limit = SIZE_MAX; + if (short_head != IndexHeadWriteStep::IoError || files[head_paths[0]].size() != 7 || files[head_paths[1]] != preserved) return 63; + flush_ok = false; + const auto unflushed_head = write_head(); + flush_ok = true; + if (unflushed_head != IndexHeadWriteStep::IoError || files[head_paths[1]] != preserved || write_head() != IndexHeadWriteStep::Verified) return 64; + std::array bitmap{}; + ::geocaching::storage::IndexRootBytes roots[2], loaded[2]; + ::geocaching::storage::IndexRootView initial{17, 0, 1, 'a', {bitmap.data(), bitmap.size()}}; + if (!::geocaching::storage::encodeIndexRoot(volume, initial, roots[0])) return 65; + bitmap[(5 - 1) * 32 + bucket / 8] |= 1u << (bucket % 8); + auto committed = initial; + committed.sequence = 1; + committed.revision = 2; + if (!::geocaching::storage::encodeIndexRoot(volume, committed, roots[1])) return 66; + const char* root_paths[] = {"/trailmate/geocaching/.state/index/root.h0", "/trailmate/geocaching/.state/index/root.h1"}; + for (unsigned copy = 0; copy < 2; ++copy) files[root_paths[copy]] = {roots[copy].begin(), roots[copy].end()}; + for (unsigned fault = 0; fault < 3; ++fault) + { + if (fault == 1) files[root_paths[1]][0] ^= 1; + if (fault == 2) files.erase(root_paths[1]); + SdIndexRootReader reader(volume); + if (reader.begin(loaded[0], loaded[0]) || !reader.begin(loaded[0], loaded[1])) return 67; + auto result = IndexRootReadStep::Working; + ::geocaching::storage::IndexRootView selected; + for (unsigned step = 0; step < 8 && result == IndexRootReadStep::Working; ++step) + { + if (reader.selected(selected)) return 68; + step_bytes = 0; + result = reader.step(); + if (step_bytes > 512) return 69; + } + if (!fault) + { + if (result != IndexRootReadStep::Ready || !reader.selected(selected) || selected.sequence != 1 || + reader.selectedCopy() != 1 || !::geocaching::storage::indexHasShard(selected, 5, bucket)) return 70; + } + else if (result != IndexRootReadStep::Invalid || reader.selected(selected)) return 71; + files[root_paths[1]] = {roots[1].begin(), roots[1].end()}; + } + const auto publish_root = [&](unsigned copy, const ::geocaching::storage::IndexRootBytes& encoded) + { + SdIndexRootWriter writer(volume); + if (!writer.begin(copy, encoded)) return IndexRootWriteStep::Invalid; + auto result = IndexRootWriteStep::Working; + for (unsigned step = 0; step < 32 && result == IndexRootWriteStep::Working; ++step) + { + step_bytes = 0; + result = writer.step(); + if (step_bytes > 512) + { + exceeded_budget = true; + return IndexRootWriteStep::Invalid; + } + } + return result; + }; + auto next_root = committed; + next_root.sequence = 2; + next_root.revision = 3; + ::geocaching::storage::IndexRootBytes next_bytes; + if (!::geocaching::storage::encodeIndexRoot(volume, next_root, next_bytes) || publish_root(0, next_bytes) != IndexRootWriteStep::Verified || + files[root_paths[1]] != std::vector(roots[1].begin(), roots[1].end())) return 72; + SdIndexRootReader latest_root(volume); + if (!latest_root.begin(loaded[0], loaded[1])) return 73; + auto root_result = IndexRootReadStep::Working; + for (unsigned step = 0; step < 8 && root_result == IndexRootReadStep::Working; ++step) root_result = latest_root.step(); + ::geocaching::storage::IndexRootView latest; + if (root_result != IndexRootReadStep::Ready || !latest_root.selected(latest) || latest.sequence != 2 || latest_root.selectedCopy() != 0) return 74; + const auto committed_root = files[root_paths[0]]; + next_root.sequence = 3; + next_root.revision = 4; + if (!::geocaching::storage::encodeIndexRoot(volume, next_root, next_bytes)) return 75; + write_limit = 7; + const auto short_root = publish_root(1, next_bytes); + write_limit = SIZE_MAX; + if (short_root != IndexRootWriteStep::IoError || files[root_paths[0]] != committed_root || files[root_paths[1]].size() != 7) return 76; + flush_ok = false; + const auto unflushed_root = publish_root(1, next_bytes); + flush_ok = true; + if (unflushed_root != IndexRootWriteStep::IoError || files[root_paths[0]] != committed_root || publish_root(1, next_bytes) != IndexRootWriteStep::Verified) return 77; + volume[0] = 1; + format = ::geocaching::storage::encodeVolumeHeader(volume); + files["/trailmate/geocaching/.state/format.bin"] = {format.begin(), format.end()}; + if (nextReady(replay, transaction) != ReplayStep::VolumeChanged) return 4; + if (exceeded_budget) return 16; + return 0; +} diff --git a/modules/core_geocaching/tests/test_journal_segment.cpp b/modules/core_geocaching/tests/test_journal_segment.cpp new file mode 100644 index 00000000..2e541b28 --- /dev/null +++ b/modules/core_geocaching/tests/test_journal_segment.cpp @@ -0,0 +1,38 @@ +#include "platform/esp/arduino_common/geocaching/sd_journal_segment.h" +#include +std::vector data; +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeFile::Impl { public: size_t offset = 0; bool open = false; }; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char*, const char*) { impl_->open = true; impl_->offset = 0; return true; } +void SdRuntimeFile::close() { impl_->open = false; } +bool SdRuntimeFile::is_open() const { return impl_->open; } +uint64_t SdRuntimeFile::size() const { return data.size(); } +int SdRuntimeFile::read(void* buffer, size_t count) +{ + const auto n = std::min(count, data.size() - impl_->offset); + std::memcpy(buffer, data.data() + impl_->offset, n); impl_->offset += n; return static_cast(n); +} +} +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + const uint8_t payload[] = {0x93, 1, 0, 0x90}; + for (uint64_t sequence = 1; sequence <= 2; ++sequence) + { + ::geocaching::storage::RecordHeader header; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, sequence, {payload, sizeof(payload)}, header)) return 1; + data.insert(data.end(), header.begin(), header.end()); data.insert(data.end(), payload, payload + sizeof(payload)); + } + SdJournalSegment segment; uint8_t buffer[128]; ::geocaching::storage::RecordFrameView frame; + if (!segment.open(1) || segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::Record || frame.sequence != 1 || + segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::Record || frame.sequence != 2 || + segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::End) return 2; + data.pop_back(); + if (!segment.open(1) || segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::Record || + segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::Truncated || frame.payload.data || + segment.next(buffer, sizeof(buffer), frame) != SegmentReadResult::Truncated) return 3; + return 0; +} diff --git a/modules/core_geocaching/tests/test_logical_state.cpp b/modules/core_geocaching/tests/test_logical_state.cpp new file mode 100644 index 00000000..96029274 --- /dev/null +++ b/modules/core_geocaching/tests/test_logical_state.cpp @@ -0,0 +1,20 @@ +#include "geocaching/storage/logical_state.h" +int main() +{ + using namespace geocaching::storage; + uint8_t first[32]{}, second[32]{}; + LogicalState state(first, second, sizeof(first)); + uint8_t key = 1, other_key = 2, value = 9; + MutationView initial{5, {&key, 1}, {&value, 1}, false}; + auto accept = [](const LogicalState::View&) { return true; }; + if (!state.apply(&initial, 1, accept)) return 1; + MutationView changes[] = {{5, {&key, 1}, {}, true}, {5, {&other_key, 1}, {&value, 1}, false}}; + if (state.apply(changes, 2, [](const LogicalState::View&) { return false; })) return 2; + geocaching::ByteView found; + if (!state.view().find(5, {&key, 1}, found) || found.size != 1 || found.data[0] != 9 || state.view().size() != 1) return 3; + if (!state.apply(changes, 2, accept) || state.view().find(5, {&key, 1}, found) || + !state.view().find(5, {&other_key, 1}, found)) return 4; + uint8_t large[32]{}; MutationView overflow{5, {&key, 1}, {large, sizeof(large)}, false}; + if (state.apply(&overflow, 1, accept) || state.view().size() != 1 || !state.view().find(5, {&other_key, 1}, found)) return 5; + return 0; +} diff --git a/modules/core_geocaching/tests/test_lxmf_custom.cpp b/modules/core_geocaching/tests/test_lxmf_custom.cpp new file mode 100644 index 00000000..db17e92b --- /dev/null +++ b/modules/core_geocaching/tests/test_lxmf_custom.cpp @@ -0,0 +1,40 @@ +#include "chat/infra/lxmf/lxmf_wire.h" +#include "geocaching/protocol/cmp_reader.h" +#include +#include +#include +#include +#ifdef _MSC_VER +#include +#endif + +int main() +{ +#ifdef _MSC_VER + _set_error_mode(_OUT_TO_STDERR); + _CrtSetReportMode(_CRT_ASSERT, _CRTDBG_MODE_FILE); + _CrtSetReportFile(_CRT_ASSERT, _CRTDBG_FILE_STDERR); +#endif + std::vector data(8192), output(8400); + for (std::size_t i = 0; i < data.size(); ++i) data[i] = static_cast(i); + std::size_t size = output.size(); + assert(chat::lxmf::encodeCustomDataPayload(1234, "title", "content", "trailmate.geocache", + {data.data(), data.size()}, output.data(), &size)); + chat::lxmf::DecodedTextPayload decoded; + assert(chat::lxmf::unpackTextPayload(output.data(), size, &decoded)); + assert(decoded.timestamp == 1234 && decoded.title == "title" && decoded.content == "content"); + const auto* type = chat::lxmf::findField(decoded, 0xfb); + const auto* payload = chat::lxmf::findField(decoded, 0xfc); + assert(type && payload && decoded.fields.size() == 2); + geocaching::protocol::CmpReader type_reader({type->encoded_value.data(), type->encoded_value.size()}); + std::string_view name; + assert(type_reader.text(name, 64) && name == "trailmate.geocache" && type_reader.finished()); + geocaching::protocol::CmpReader data_reader({payload->encoded_value.data(), payload->encoded_value.size()}); + geocaching::ByteView view; + assert(data_reader.binary(view, 8192) && view.size == data.size() && data_reader.finished()); + assert(std::memcmp(view.data, data.data(), data.size()) == 0); + std::size_t short_size = size - 1; + assert(!chat::lxmf::encodeCustomDataPayload(1234, "title", "content", "trailmate.geocache", + {data.data(), data.size()}, output.data(), &short_size)); + assert(short_size == 0); +} diff --git a/modules/core_geocaching/tests/test_outgoing_record.cpp b/modules/core_geocaching/tests/test_outgoing_record.cpp new file mode 100644 index 00000000..4cf5aabb --- /dev/null +++ b/modules/core_geocaching/tests/test_outgoing_record.cpp @@ -0,0 +1,28 @@ +#include "geocaching/storage/outgoing_record.h" +#include "geocaching/storage/queued_request.h" +#include "geocaching/protocol/query_request.h" +#include +int main() +{ + using namespace geocaching; + RequestId id; id.bytes[0] = 1; + uint8_t request[64]; size_t request_size = 0; + if (!protocol::encodeCapabilitiesRequest(id, request, sizeof(request), request_size)) return 1; + std::array value_buffer{}; + auto scratch = std::make_unique(value_buffer.data(), value_buffer.size()); + std::array bytes; size_t size = 0; + storage::StoredTime time; time.monotonic_ms = 123; time.boot_id[0] = 7; + if (!storage::encodeNewRequestTask(0, {}, {}, id, {}, 3, {request, request_size}, time, + *scratch, bytes.data(), bytes.size(), size)) return 2; + storage::MutationView mutations[2]; storage::TransactionView transaction; + if (!storage::decodeTransaction({bytes.data(), size}, 0, mutations, 2, transaction)) return 3; + storage::OutgoingView out; + if (!storage::decodeOutgoing(mutations[0].key, mutations[0].value, out) || out.state != 0 || + !out.continue_intent || out.created.monotonic_ms != 123 || out.created.boot_id[0] != 7 || + out.created.has_utc || out.install_generation || out.request.size != request_size) return 4; + for (size_t n = 0; n < mutations[0].value.size; ++n) + if (storage::decodeOutgoing(mutations[0].key, {mutations[0].value.data, n}, out) || out.request.data) return 5; + std::array wrong_key{}; + if (storage::decodeOutgoing({wrong_key.data(), wrong_key.size()}, mutations[0].value, out)) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_paging.cpp b/modules/core_geocaching/tests/test_paging.cpp new file mode 100644 index 00000000..759dd454 --- /dev/null +++ b/modules/core_geocaching/tests/test_paging.cpp @@ -0,0 +1,75 @@ +#include "geocaching/usecase/query_client.h" +#include +#include +#include +#include +#include + +struct PagingPort : geocaching::QueryClientPort +{ + unsigned sends = 0, commits = 0; + bool fail_commit = false; + std::vector sent; + bool newRequestId(geocaching::RequestId& id) override + { + id.bytes.fill(sends == 0 ? 1 : sends == 1 ? 3 + : 4); + return true; + } + geocaching::QueryPersistence submit(const geocaching::DirectoryEntry&, const geocaching::RequestId&, geocaching::ByteView bytes) override + { + sent.assign(bytes.data, bytes.data + bytes.size); + ++sends; + return geocaching::QueryPersistence::Committed; + } + geocaching::QueryPersistence pollPersistence() override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence cancel(const geocaching::Destination&, const geocaching::RequestId&) override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence commitCapabilities(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView) override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence commitPage(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView, + const geocaching::protocol::QueryPageView&) override + { + if (fail_commit) return geocaching::QueryPersistence::Rejected; + ++commits; + return geocaching::QueryPersistence::Committed; + } +}; +int main(int argc, char** argv) +{ + assert(argc == 4); + std::vector fixture[3]; + for (unsigned i = 0; i < 3; ++i) + { + std::ifstream f(argv[i + 1], std::ios::binary); + assert(f.good()); + fixture[i].assign(std::istreambuf_iterator(f), {}); + } + PagingPort port; + auto client = std::make_unique(port); + geocaching::Destination address; + std::array key{}; + std::vector app{0x95, 1, 0xc4, 16}; + app.insert(app.end(), 16, 0); + app.insert(app.end(), {0xc4, 16}); + app.insert(app.end(), 16, 0); + app.insert(app.end(), {0, 0xa0}); + assert(client->observe(address, address, {key.data(), 64}, {app.data(), app.size()}, 0)); + assert(client->query({300000000, 1200000000, 310000000, 1210000000}) && client->tick(0)); + assert(client->accept(address, {fixture[0].data(), fixture[0].size()}) && client->tick(1)); + assert(client->accept(address, {fixture[1].data(), fixture[1].size()}) && client->hasMore()); + std::fill(fixture[1].begin(), fixture[1].end(), 0); + assert(client->loadMore()); + assert(port.sent.size() >= 18); + assert(port.sent[port.sent.size() - 18] == 0xc4 && port.sent[port.sent.size() - 17] == 16); + for (std::size_t i = port.sent.size() - 16; i < port.sent.size(); ++i) assert(port.sent[i] == 0x72); + auto wrong = fixture[2]; + // Snapshot starts after response header, body array header and bin16 header. + wrong[27] ^= 1; + assert(!client->accept(address, {wrong.data(), wrong.size()})); + assert(port.commits == 1); + port.fail_commit = true; + assert(!client->accept(address, {fixture[2].data(), fixture[2].size()})); + assert(client->phase() == geocaching::QueryClientPhase::Querying); + port.fail_commit = false; + assert(client->accept(address, {fixture[2].data(), fixture[2].size()})); + assert(port.commits == 2 && !client->hasMore() && !client->loadMore()); +} diff --git a/modules/core_geocaching/tests/test_pending_request.cpp b/modules/core_geocaching/tests/test_pending_request.cpp new file mode 100644 index 00000000..a595f873 --- /dev/null +++ b/modules/core_geocaching/tests/test_pending_request.cpp @@ -0,0 +1,35 @@ +#include "geocaching/storage/pending_request.h" +#include "geocaching/storage/queued_request.h" +#include "geocaching/storage/task_references.h" +#include "geocaching/protocol/query_request.h" +#include +int main() +{ + using namespace geocaching; + Destination local, remote; local.bytes[0] = 1; remote.bytes[0] = 2; + RequestId id; id.bytes[0] = 3; + uint8_t request[64]; size_t request_size = 0; + if (!protocol::encodeCapabilitiesRequest(id, request, sizeof(request), request_size)) return 1; + std::array value_buffer{}; + auto workspace = std::make_unique(value_buffer.data(), value_buffer.size()); + uint8_t encoded[512]{}; size_t size = 0; + if (!storage::encodeNewRequestTask(0, local, remote, id, {}, 3, {request, request_size}, {}, + *workspace, encoded, sizeof(encoded), size)) return 2; + storage::MutationView mutations[2]; storage::TransactionView transaction; + if (!storage::decodeTransaction({encoded, size}, 0, mutations, 2, transaction)) return 3; + uint8_t a[512]{}, b[512]{}; storage::LogicalState state(a, b, sizeof(a)); + if (!state.apply(mutations, 2, storage::validateTaskReferences)) return 4; + storage::PendingRequestView pending; + if (storage::nextPendingRequest(state.view(), local, {}, pending) != storage::PendingRequestResult::Ready || + pending.destination.bytes != remote.bytes || pending.request_id.bytes != id.bytes || pending.request.size != request_size) return 5; + if (storage::nextPendingRequest(state.view(), remote, {}, pending) != storage::PendingRequestResult::None) return 6; + uint8_t task_key[16]{}; ByteView task_bytes; storage::TaskView task; + if (!state.view().find(10, {task_key, 16}, task_bytes) || !storage::decodeTask({task_key, 16}, task_bytes, task)) return 7; + task.state = 5; task.continue_intent = false; + uint8_t stopped[256]{}; + if (!storage::encodeTask({task_key, 16}, task, stopped, sizeof(stopped), size)) return 8; + storage::MutationView stop{10, {task_key, 16}, {stopped, size}, false}; + if (!state.apply(&stop, 1, storage::validateTaskReferences) || + storage::nextPendingRequest(state.view(), local, {}, pending) != storage::PendingRequestResult::None) return 9; + return 0; +} diff --git a/modules/core_geocaching/tests/test_publish.cpp b/modules/core_geocaching/tests/test_publish.cpp new file mode 100644 index 00000000..80ceddeb --- /dev/null +++ b/modules/core_geocaching/tests/test_publish.cpp @@ -0,0 +1,33 @@ +#include "geocaching/protocol/publish_request.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 3); + std::ifstream signed_file(argv[1], std::ios::binary), request_file(argv[2], std::ios::binary); + assert(signed_file.good() && request_file.good()); + std::vector signed_bytes((std::istreambuf_iterator(signed_file)), {}); + std::vector expected((std::istreambuf_iterator(request_file)), {}); + geocaching::protocol::CmpReader reader({signed_bytes.data(), signed_bytes.size()}); + std::size_t count = 0; + geocaching::ByteView record, signature; + assert(reader.array(count, 2) && reader.binary(record, 4096) && reader.binary(signature, 64)); + geocaching::RequestId id; + id.bytes.fill(2); + std::vector output(512); + std::size_t size = 0; + using geocaching::protocol::encodePublishRequest; + assert(encodePublishRequest(id, record, signature, 512, output.data(), output.size(), size)); + assert(size == expected.size() && std::memcmp(output.data(), expected.data(), size) == 0); + for (std::size_t n = 0; n < expected.size(); ++n) + { + assert(!encodePublishRequest(id, record, signature, 512, output.data(), n, size)); + assert(size == 0); + } + assert(!encodePublishRequest(id, record, {signature.data, 63}, 512, output.data(), output.size(), size)); + assert(!encodePublishRequest(id, record, signature, 511, output.data(), output.size(), size)); + assert(!encodePublishRequest(id, {record.data, record.size - 1}, signature, 512, output.data(), output.size(), size)); +} diff --git a/modules/core_geocaching/tests/test_publish_attempt.cpp b/modules/core_geocaching/tests/test_publish_attempt.cpp new file mode 100644 index 00000000..30c565c9 --- /dev/null +++ b/modules/core_geocaching/tests/test_publish_attempt.cpp @@ -0,0 +1,138 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include "geocaching/usecase/publish_attempt.h" +#include +#include +#include +#include +struct Crypto : geocaching::protocol::RecordCrypto +{ + bool sha256(geocaching::ByteView in, uint8_t out[32]) override + { + chat::reticulum::fullHash(in.data, in.size, out); + return true; + } + geocaching::protocol::VerificationResult verifyEd25519(geocaching::ByteView key, geocaching::ByteView sig, geocaching::ByteView message) override + { + return ed25519_verify(sig.data, message.data, message.size, key.data) ? geocaching::protocol::VerificationResult::Valid : geocaching::protocol::VerificationResult::InvalidSignature; + } +}; +struct Port : geocaching::PublishAttemptPort +{ + using Result = geocaching::PublishPersistence; + Result submission = Result::Committed, saving = Result::Committed, polled = Result::Pending; + bool commit_ok = false; + unsigned commits = 0; + Result submit(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView) override { return submission; } + Result commitResult(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView) override + { + ++commits; + return commit_ok ? saving : Result::Rejected; + } + Result cancel(const geocaching::Destination&, const geocaching::RequestId&) override { return Result::Committed; } + Result poll() override { return polled; } +}; +int main(int argc, char** argv) +{ + using namespace geocaching; + if (argc != 3) return 1; + std::ifstream a(argv[1], std::ios::binary), b(argv[2], std::ios::binary); + std::vector record((std::istreambuf_iterator(a)), {}), response((std::istreambuf_iterator(b)), {}); + std::vector workspace(record.size() + 26); + Crypto crypto; + Port port; + RequestId request; + request.bytes.fill(2); + Destination source; + auto attempt = std::make_unique(port, crypto); + if (!attempt->begin(source, request, {record.data(), record.size()}, workspace.data(), workspace.size())) return 2; + std::fill(workspace.begin(), workspace.end(), 0xa5); + if (attempt->accept(source, {response.data(), response.size()}) || attempt->phase() != PublishAttemptPhase::Waiting) return 3; + port.commit_ok = true; + if (!attempt->accept(source, {response.data(), response.size()}) || attempt->phase() != PublishAttemptPhase::Confirmed) return 4; + const auto commits = port.commits; + if (!attempt->accept(source, {response.data(), response.size()}) || port.commits != commits) return 5; + source.bytes[0] = 1; + if (attempt->accept(source, {response.data(), response.size()})) return 6; + auto cancelled = std::make_unique(port, crypto); + if (!cancelled->begin(source, request, {record.data(), record.size()}, workspace.data(), workspace.size()) || !cancelled->cancel() || + cancelled->accept(source, {response.data(), response.size()})) return 7; + PublishAttempt rejected(port, crypto); + if (rejected.begin(source, request, {record.data(), record.size()}, workspace.data(), workspace.size() - 1)) return 8; + std::vector aliased(record.size() + 26); + std::memcpy(aliased.data(), record.data(), record.size()); + if (rejected.begin(source, request, {aliased.data(), record.size()}, aliased.data(), aliased.size())) return 9; + source.bytes[0] = 0; + auto trailing = response; + trailing.push_back(0); + if (attempt->accept(source, {trailing.data(), trailing.size()})) return 10; + protocol::CmpReader reader({response.data(), response.size()}); + size_t fields = 0; + uint64_t value = 0; + ByteView bytes; + if (!reader.array(fields, 6) || !reader.unsignedInteger(value) || !reader.unsignedInteger(value) || + !reader.unsignedInteger(value) || !reader.binary(bytes, 16) || !reader.unsignedInteger(value) || + !reader.array(fields, 7) || !reader.binary(bytes, 32) || !reader.unsignedInteger(value) || + !reader.binary(bytes, 32)) return 11; + const auto disposition_offset = reader.position(); + auto changed = response; + changed[disposition_offset] ^= 1; + if (attempt->accept(source, {changed.data(), changed.size()}) || port.commits != commits) return 12; + auto noncanonical = response; + noncanonical.insert(noncanonical.begin() + disposition_offset, 0xcc); + if (attempt->accept(source, {noncanonical.data(), noncanonical.size()})) return 13; + Port delayed; + delayed.submission = PublishPersistence::Pending; + delayed.saving = PublishPersistence::Pending; + delayed.commit_ok = true; + PublishAttempt asynchronous(delayed, crypto); + if (!asynchronous.begin(source, request, {record.data(), record.size()}, workspace.data(), workspace.size()) || + asynchronous.phase() != PublishAttemptPhase::Submitting || asynchronous.accept(source, {response.data(), response.size()})) return 14; + asynchronous.advance(); + if (asynchronous.phase() != PublishAttemptPhase::Submitting) return 15; + delayed.polled = PublishPersistence::Committed; + asynchronous.advance(); + if (asynchronous.phase() != PublishAttemptPhase::Waiting || asynchronous.accept(source, {response.data(), response.size()}) || + asynchronous.phase() != PublishAttemptPhase::Committing || asynchronous.cancel()) return 16; + delayed.polled = PublishPersistence::Pending; + asynchronous.advance(); + if (asynchronous.phase() != PublishAttemptPhase::Committing) return 17; + delayed.polled = PublishPersistence::Rejected; + asynchronous.advance(); + if (asynchronous.phase() != PublishAttemptPhase::Failed || asynchronous.accept(source, {response.data(), response.size()})) return 18; + PublishAttempt durable(delayed, crypto); + if (!durable.begin(source, request, {record.data(), record.size()}, workspace.data(), workspace.size())) return 19; + delayed.polled = PublishPersistence::Committed; + durable.advance(); + if (durable.accept(source, {response.data(), response.size()})) return 20; + durable.advance(); + if (durable.phase() != PublishAttemptPhase::Confirmed) return 21; + protocol::VerifiedRecordView verified; + if (protocol::verifyGeocache({record.data(), record.size()}, crypto, workspace.data(), workspace.size(), verified) != + protocol::VerificationResult::Valid) return 22; + std::vector original_request(record.size() + 26); + size_t request_size = 0; + if (!protocol::encodePublishRequest(request, verified.record.encoded, verified.signature, 512, + original_request.data(), original_request.size(), request_size)) return 23; + original_request.resize(request_size); + Port restored_port; + restored_port.submission = PublishPersistence::Rejected; + restored_port.commit_ok = true; + PublishAttempt restored(restored_port, crypto); + if (!restored.resume(source, request, {original_request.data(), original_request.size()}, workspace.data(), workspace.size(), verified.id, verified.hash) || + restored.phase() != PublishAttemptPhase::Waiting || restored_port.commits) return 24; + if (!restored.accept(source, {response.data(), response.size()}) || restored_port.commits != 1) return 25; + PublishAttempt confirmed(restored_port, crypto); + if (!confirmed.resume(source, request, {original_request.data(), original_request.size()}, workspace.data(), workspace.size(), verified.id, verified.hash, + {response.data(), response.size()}) || + confirmed.phase() != PublishAttemptPhase::Confirmed || + !confirmed.accept(source, {response.data(), response.size()}) || restored_port.commits != 1) return 26; + PublishAttempt invalid(restored_port, crypto); + const auto untouched = original_request; + if (invalid.resume(source, request, {original_request.data(), original_request.size()}, original_request.data(), original_request.size(), verified.id, verified.hash) || + original_request != untouched) return 27; + original_request.back() ^= 1; + if (invalid.resume(source, request, {original_request.data(), original_request.size()}, workspace.data(), workspace.size(), verified.id, verified.hash) || + invalid.phase() != PublishAttemptPhase::Idle) return 28; + return 0; +} diff --git a/modules/core_geocaching/tests/test_publish_response.cpp b/modules/core_geocaching/tests/test_publish_response.cpp new file mode 100644 index 00000000..02b259be --- /dev/null +++ b/modules/core_geocaching/tests/test_publish_response.cpp @@ -0,0 +1,35 @@ +#include "geocaching/protocol/publish_response.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 2); + std::ifstream file(argv[1], std::ios::binary); + assert(file.good()); + std::vector bytes((std::istreambuf_iterator(file)), {}); + assert(bytes.size() == 131); + geocaching::RequestId request; + request.bytes.fill(2); + geocaching::GeocacheId id; + geocaching::RevisionHash hash; + // Fixed external fixture layout: cache ID begins at 27, hash at 62. + std::memcpy(id.bytes.data(), bytes.data() + 27, 32); + std::memcpy(hash.bytes.data(), bytes.data() + 62, 32); + using namespace geocaching::protocol; + PublishDisposition disposition; + assert(decodePublishResponse({bytes.data(), bytes.size()}, request, id, hash, 1, + geocaching::CacheState::Active, disposition)); + assert(disposition == PublishDisposition::Stored); + assert(!decodePublishResponse({bytes.data(), bytes.size()}, request, id, hash, 2, + geocaching::CacheState::Active, disposition)); + hash.bytes[0] ^= 1; + assert(!decodePublishResponse({bytes.data(), bytes.size()}, request, id, hash, 1, + geocaching::CacheState::Active, disposition)); + hash.bytes[0] ^= 1; + for (std::size_t n = 0; n < bytes.size(); ++n) + assert(!decodePublishResponse({bytes.data(), n}, request, id, hash, 1, + geocaching::CacheState::Active, disposition)); +} diff --git a/modules/core_geocaching/tests/test_query.cpp b/modules/core_geocaching/tests/test_query.cpp new file mode 100644 index 00000000..0d662a1f --- /dev/null +++ b/modules/core_geocaching/tests/test_query.cpp @@ -0,0 +1,36 @@ +#include "geocaching/protocol/query_request.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 3); + using namespace geocaching::protocol; + std::ifstream cap_file(argv[1], std::ios::binary), query_file(argv[2], std::ios::binary); + assert(cap_file.good() && query_file.good()); + std::vector cap((std::istreambuf_iterator(cap_file)), {}); + std::vector query((std::istreambuf_iterator(query_file)), {}); + std::vector output(256); + geocaching::RequestId id; + id.bytes.fill(1); + std::size_t size = 0; + assert(encodeCapabilitiesRequest(id, output.data(), output.size(), size)); + assert(size == cap.size() && std::memcmp(output.data(), cap.data(), size) == 0); + id.bytes.fill(3); + QueryRegion region{300000000, 1200000000, 310000000, 1210000000}; + assert(encodeQueryRequest(id, region, 3, {}, 20, {}, 2048, output.data(), output.size(), size)); + assert(size == query.size() && std::memcmp(output.data(), query.data(), size) == 0); + for (std::size_t n = 0; n < query.size(); ++n) + { + assert(!encodeQueryRequest(id, region, 3, {}, 20, {}, 2048, output.data(), n, size)); + assert(size == 0); + } + region.south_e7 = region.north_e7 + 1; + assert(!encodeQueryRequest(id, region, 3, {}, 20, {}, 2048, output.data(), output.size(), size)); + region = {-900000000, -1800000000, 900000000, 1800000000}; + assert(encodeQueryRequest(id, region, 7, {}, 64, {}, 8192, output.data(), output.size(), size)); + assert(!encodeQueryRequest(id, region, 0, {}, 20, {}, 2048, output.data(), output.size(), size)); + assert(!encodeQueryRequest(id, region, 3, {nullptr, 32}, 20, {}, 2048, output.data(), output.size(), size)); +} diff --git a/modules/core_geocaching/tests/test_query_client.cpp b/modules/core_geocaching/tests/test_query_client.cpp new file mode 100644 index 00000000..d9fdd612 --- /dev/null +++ b/modules/core_geocaching/tests/test_query_client.cpp @@ -0,0 +1,57 @@ +#include "geocaching/usecase/query_client.h" +#include +#include +#include +#include +#include +struct Port : geocaching::QueryClientPort +{ + unsigned requests = 0, pages = 0; + bool newRequestId(geocaching::RequestId& id) override + { + id.bytes.fill(requests == 0 ? 1 : 3); + return true; + } + geocaching::QueryPersistence submit(const geocaching::DirectoryEntry&, const geocaching::RequestId&, geocaching::ByteView) override + { + ++requests; + return geocaching::QueryPersistence::Committed; + } + geocaching::QueryPersistence pollPersistence() override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence cancel(const geocaching::Destination&, const geocaching::RequestId&) override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence commitCapabilities(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView) override { return geocaching::QueryPersistence::Committed; } + geocaching::QueryPersistence commitPage(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView, + const geocaching::protocol::QueryPageView& page) override + { + geocaching::protocol::CmpReader rows(page.encoded_items); + geocaching::protocol::SummaryView item; + assert(page.count == 1 && geocaching::protocol::decodeSummary(rows, item) && item.name == "Test"); + ++pages; + return geocaching::QueryPersistence::Committed; + } +}; +int main(int argc, char** argv) +{ + assert(argc == 3); + Port port; + auto client = std::make_unique(port); + geocaching::Destination discovery, delivery; + std::array key{}; + std::vector app{0x95, 1, 0xc4, 16}; + app.insert(app.end(), 16, 0); + app.insert(app.end(), {0xc4, 16}); + app.insert(app.end(), 16, 0); + app.insert(app.end(), {0, 0xa1, 'x'}); + assert(client->observe(discovery, delivery, {key.data(), 64}, {app.data(), app.size()}, 0)); + assert(client->query({300000000, 1200000000, 310000000, 1210000000})); + assert(client->tick(0) && port.requests == 1); + for (int i = 1; i < 3; ++i) + { + std::ifstream f(argv[i], std::ios::binary); + assert(f.good()); + std::vector data((std::istreambuf_iterator(f)), {}); + assert(client->accept(delivery, {data.data(), data.size()})); + if (i == 1) assert(client->tick(1)); + } + assert(port.requests == 2 && port.pages == 1 && client->phase() == geocaching::QueryClientPhase::PageReady); +} diff --git a/modules/core_geocaching/tests/test_query_response.cpp b/modules/core_geocaching/tests/test_query_response.cpp new file mode 100644 index 00000000..ac27eee7 --- /dev/null +++ b/modules/core_geocaching/tests/test_query_response.cpp @@ -0,0 +1,27 @@ +#include "geocaching/protocol/query_response.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 2); + std::ifstream f(argv[1], std::ios::binary); + assert(f.good()); + std::vector bytes((std::istreambuf_iterator(f)), {}); + geocaching::RequestId id; + id.bytes.fill(3); + std::vector items(1); + geocaching::protocol::QueryPageView page; + using geocaching::protocol::decodeQueryResponse; + assert(decodeQueryResponse({bytes.data(), bytes.size()}, id, 2048, items.data(), 1, page)); + assert(page.count == 1 && page.next_cursor.size == 0 && page.remaining_ttl == 604800); + assert(items[0].name == "Test" && items[0].latitude_e7 == 302500000 && items[0].signed_bytes == 190); + assert(!decodeQueryResponse({bytes.data(), bytes.size()}, id, 2048, nullptr, 0, page)); + assert(page.count == 0); + for (std::size_t n = 0; n < bytes.size(); ++n) + assert(!decodeQueryResponse({bytes.data(), n}, id, 2048, items.data(), 1, page)); + bytes.push_back(0); + assert(!decodeQueryResponse({bytes.data(), bytes.size()}, id, 2048, items.data(), 1, page)); +} diff --git a/modules/core_geocaching/tests/test_query_timeout.cpp b/modules/core_geocaching/tests/test_query_timeout.cpp new file mode 100644 index 00000000..865c9c41 --- /dev/null +++ b/modules/core_geocaching/tests/test_query_timeout.cpp @@ -0,0 +1,64 @@ +#include "geocaching/usecase/query_client.h" +#include + +struct Port final : geocaching::QueryClientPort +{ + using Result = geocaching::QueryPersistence; + unsigned requests = 0, stops = 0; + Result submitted = Result::Committed, polled = Result::Pending; + bool newRequestId(geocaching::RequestId& id) override + { + id.bytes.fill(static_cast(requests + 1)); + return true; + } + Result submit(const geocaching::DirectoryEntry&, const geocaching::RequestId&, geocaching::ByteView) override + { + ++requests; + return submitted; + } + Result commitCapabilities(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView) override { return Result::Committed; } + Result commitPage(const geocaching::Destination&, const geocaching::RequestId&, geocaching::ByteView, + const geocaching::protocol::QueryPageView&) override { return Result::Committed; } + Result pollPersistence() override { return polled; } + Result cancel(const geocaching::Destination&, const geocaching::RequestId&) override + { + ++stops; + return Result::Pending; + } +}; + +int main() +{ + using namespace geocaching; + Port port; + QueryClient client(port); + const protocol::QueryRegion world{-900000000, -1800000000, 900000000, 1800000000}; + const auto timeout = QueryClient::kReplyTimeoutMs; + if (!client.query(world) || client.tick(0) || client.tick(timeout - 1) || !client.tick(timeout) || + client.phase() != QueryClientPhase::Failed || client.failure() != QueryFailure::Timeout || port.stops) return 1; + std::array key{}; + std::vector announce{0x95, 1, 0xc4, 16}; + announce.insert(announce.end(), 16, 0); + announce.insert(announce.end(), {0xc4, 16}); + announce.insert(announce.end(), 16, 0); + announce.insert(announce.end(), {0, 0xa1, 'x'}); + if (!client.observe({}, {}, {key.data(), key.size()}, {announce.data(), announce.size()}, 0) || + !client.query(world) || !client.tick(0) || client.phase() != QueryClientPhase::CheckingCapabilities) return 2; + client.tick(0); + if (client.tick(timeout - 1) || !client.tick(timeout) || client.phase() != QueryClientPhase::Cancelling || + client.query(world) || port.stops != 1) return 3; + if (client.tick(timeout + 1) || !client.persistencePending() || client.accept({}, {key.data(), key.size()})) return 4; + port.polled = QueryPersistence::Committed; + if (!client.tick(timeout + 2) || client.persistencePending() || client.phase() != QueryClientPhase::Failed || + client.failure() != QueryFailure::Timeout) return 5; + if (!client.query(world) || client.tick(timeout + 3) || !client.tick(timeout + 5000)) return 6; + if (!client.cancel()) return 7; + port.polled = QueryPersistence::Rejected; + if (client.tick(timeout + 5001) || client.phase() != QueryClientPhase::Failed || client.failure() != QueryFailure::Storage) return 8; + port.submitted = QueryPersistence::Pending; + if (!client.query(world) || !client.tick(timeout + 6000) || !client.persistencePending() || client.cancel()) return 9; + port.polled = QueryPersistence::Committed; + if (!client.tick(timeout + 6001) || client.persistencePending() || !client.cancel() || !client.tick(timeout + 6002) || + client.failure() != QueryFailure::Cancelled) return 10; + return 0; +} diff --git a/modules/core_geocaching/tests/test_queued_request.cpp b/modules/core_geocaching/tests/test_queued_request.cpp new file mode 100644 index 00000000..10d91463 --- /dev/null +++ b/modules/core_geocaching/tests/test_queued_request.cpp @@ -0,0 +1,29 @@ +#include "geocaching/storage/queued_request.h" +#include "geocaching/protocol/query_request.h" +#include + +int main() +{ + using namespace geocaching; + RequestId request_id; request_id.bytes[0] = 1; + Destination local, remote; local.bytes[0] = 2; remote.bytes[0] = 3; + std::array task_id{}; task_id[0] = 4; + uint8_t request[128]{}; size_t request_size = 0; + if (!protocol::encodeCapabilitiesRequest(request_id, request, sizeof(request), request_size)) return 1; + std::array value_buffer{}; + auto scratch = std::make_unique(value_buffer.data(), value_buffer.size()); + std::array output{}; size_t size = 0; + storage::StoredTime time; time.boot_id[0] = 5; + if (!storage::encodeNewRequestTask(7, local, remote, request_id, task_id, 3, + {request, request_size}, time, *scratch, output.data(), output.size(), size)) return 2; + storage::MutationView mutations[2]; storage::TransactionView transaction; + if (!storage::decodeTransaction({output.data(), size}, 7, mutations, 2, transaction) || transaction.count != 2 || + mutations[0].table != 5 || mutations[0].key.size != 48 || mutations[1].table != 10 || mutations[1].key.size != 16) return 3; + protocol::CmpReader outgoing(mutations[0].value); size_t count = 0; ByteView stored_request; + if (!outgoing.array(count, 7) || count != 7 || !outgoing.binary(stored_request, 8192) || + stored_request.size != request_size || std::memcmp(stored_request.data, request, request_size)) return 4; + request_id.bytes[0] = 9; + if (storage::encodeNewRequestTask(7, local, remote, request_id, task_id, 3, + {request, request_size}, time, *scratch, output.data(), output.size(), size) || size) return 5; + return 0; +} diff --git a/modules/core_geocaching/tests/test_record.cpp b/modules/core_geocaching/tests/test_record.cpp new file mode 100644 index 00000000..fe0840b4 --- /dev/null +++ b/modules/core_geocaching/tests/test_record.cpp @@ -0,0 +1,34 @@ +#include "geocaching/protocol/record_decoder.h" +#include +#include +#include +#include + +int main(int argc, char** argv) +{ + assert(argc == 2); + std::ifstream stream(argv[1], std::ios::binary); + assert(stream.good()); + std::vector bytes((std::istreambuf_iterator(stream)), {}); + geocaching::RecordView record; + using namespace geocaching::protocol; + assert(decodeGeocacheRecord({bytes.data(), bytes.size()}, record)); + assert(record.name == "Test" && record.description == "Demo" && record.hint == "Root"); + assert(record.latitude_e7 == 302500000 && record.longitude_e7 == 1201500000); + assert(record.revision == 1 && record.previous_hash.size == 0); + assert(record.encoded.data == bytes.data()); + for (std::size_t n = 0; n < bytes.size(); ++n) + { + assert(!decodeGeocacheRecord({bytes.data(), n}, record)); + assert(record.encoded.data == nullptr); + } + bytes.push_back(0); + assert(!decodeGeocacheRecord({bytes.data(), bytes.size()}, record)); + assert(!validRecordText("\xc0\xaf", false, true)); + assert(!validRecordText("\xed\xa0\x80", false, true)); + assert(!validRecordText("\xef\xbf\xbe", false, true)); + assert(!validRecordText("\xe3\x80\x80", false, true)); + assert(!validRecordText("a\nb", false, true)); + assert(validRecordText("a\nb\t", true, true)); + assert(validRecordText("\xe4\xb8\xad\xe6\x96\x87", false, true)); +} diff --git a/modules/core_geocaching/tests/test_record_encoder.cpp b/modules/core_geocaching/tests/test_record_encoder.cpp new file mode 100644 index 00000000..c57d0af1 --- /dev/null +++ b/modules/core_geocaching/tests/test_record_encoder.cpp @@ -0,0 +1,23 @@ +#include "geocaching/protocol/record_encoder.h" +#include +#include +#include +int main(int argc, char** argv) +{ + using namespace geocaching; + if (argc != 2) return 1; + std::ifstream file(argv[1], std::ios::binary); + std::vector data((std::istreambuf_iterator(file)), {}); + protocol::CmpReader reader({data.data(), data.size()}); size_t fields = 0; ByteView encoded; + if (!reader.array(fields, 2) || !reader.binary(encoded, 4096)) return 2; + RecordView record; + if (!protocol::decodeGeocacheRecord(encoded, record)) return 3; + std::array output{}; size_t written = 0; + if (!protocol::encodeGeocacheRecord(record, output.data(), output.size(), written) || written != encoded.size || + std::memcmp(output.data(), encoded.data, written)) return 4; + record.name = " "; + if (protocol::encodeGeocacheRecord(record, output.data(), output.size(), written) || written) return 5; + record.name = "A"; record.longitude_e7 = 1800000000; + if (protocol::encodeGeocacheRecord(record, output.data(), output.size(), written) || written) return 6; + return 0; +} diff --git a/modules/core_geocaching/tests/test_record_frame.cpp b/modules/core_geocaching/tests/test_record_frame.cpp new file mode 100644 index 00000000..65d24aa7 --- /dev/null +++ b/modules/core_geocaching/tests/test_record_frame.cpp @@ -0,0 +1,30 @@ +#include "geocaching/storage/record_frame.h" +#include + +int main() +{ + using namespace geocaching::storage; + const uint8_t payload[] = {0x93, 1, 0, 0x90}; + RecordHeader header; + if (!makeRecordHeader(RecordKind::Transaction, 0x0102030405060708ULL, {payload, sizeof(payload)}, header)) return 1; + if (header[12] != 1 || header[19] != 8 || header[11] != 4 || header[7] != 24) return 2; + std::vector frame(header.begin(), header.end()); + frame.insert(frame.end(), payload, payload + sizeof(payload)); + RecordFrameView decoded; + if (!decodeRecordFrame({frame.data(), frame.size()}, decoded) || decoded.sequence != 0x0102030405060708ULL || + decoded.payload.size != sizeof(payload)) return 3; + for (size_t i = 0; i < frame.size(); ++i) + { + frame[i] ^= 1; + if (decodeRecordFrame({frame.data(), frame.size()}, decoded) || decoded.payload.data) return 4; + frame[i] ^= 1; + } + for (size_t size = 0; size < frame.size(); ++size) + if (decodeRecordFrame({frame.data(), size}, decoded)) return 5; + frame.push_back(0); + if (decodeRecordFrame({frame.data(), frame.size()}, decoded)) return 6; + std::vector large(65537, 0); + if (makeRecordHeader(RecordKind::Transaction, 1, {large.data(), large.size()}, header)) return 7; + if (!makeRecordHeader(RecordKind::Transaction, 1, {large.data(), 65536}, header)) return 8; + return 0; +} diff --git a/modules/core_geocaching/tests/test_record_read_budget.cpp b/modules/core_geocaching/tests/test_record_read_budget.cpp new file mode 100644 index 00000000..d7a9e3c1 --- /dev/null +++ b/modules/core_geocaching/tests/test_record_read_budget.cpp @@ -0,0 +1,55 @@ +#include "platform/esp/arduino_common/geocaching/sd_journal_segment.h" +#include + +std::vector data; +size_t bytes_read = 0, calls = 0, short_limit = SIZE_MAX; +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeFile::Impl { public: bool open = false; size_t offset = 0; }; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char*, const char*) { impl_->open = true; impl_->offset = 0; return true; } +void SdRuntimeFile::close() { impl_->open = false; } +bool SdRuntimeFile::is_open() const { return impl_->open; } +uint64_t SdRuntimeFile::size() const { return data.size(); } +int SdRuntimeFile::read(void* out, size_t count) +{ + ++calls; + const auto n = std::min(std::min(count, short_limit), data.size() - impl_->offset); + std::memcpy(out, data.data() + impl_->offset, n); impl_->offset += n; bytes_read += n; + return static_cast(n); +} +} +bool exercise(size_t size, size_t read_limit) +{ + using namespace platform::esp::arduino_common::geocaching; + std::vector payload(size, 7), output(size + 24); + ::geocaching::storage::RecordHeader header; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::Transaction, 1, {payload.data(), payload.size()}, header)) return false; + data.assign(header.begin(), header.end()); data.insert(data.end(), payload.begin(), payload.end()); + bytes_read = calls = 0; short_limit = read_limit; + SdJournalSegment segment; + if (!segment.open(1)) return false; + bool yielded = false; + ::geocaching::storage::RecordFrameView frame; + for (size_t iteration = 0; iteration < data.size() + 1; ++iteration) + { + const auto prior_bytes = bytes_read, prior_calls = calls; + const auto result = segment.next(output.data(), output.size(), frame); + if (bytes_read - prior_bytes > 512 || calls - prior_calls > 2) return false; + if (result == SegmentReadResult::InProgress) + { + if (frame.payload.data || frame.payload.size) return false; + yielded = true; continue; + } + return result == SegmentReadResult::Record && yielded && frame.payload.size == payload.size() && + !std::memcmp(frame.payload.data, payload.data(), payload.size()) && bytes_read == data.size(); + } + return false; +} +int main() +{ + if (!exercise(65536, SIZE_MAX)) return 1; + if (!exercise(2048, 13)) return 2; + return 0; +} diff --git a/modules/core_geocaching/tests/test_record_shape.cpp b/modules/core_geocaching/tests/test_record_shape.cpp new file mode 100644 index 00000000..357344b4 --- /dev/null +++ b/modules/core_geocaching/tests/test_record_shape.cpp @@ -0,0 +1,55 @@ +#include "geocaching/storage/record_shape.h" + +int main() +{ + using namespace geocaching; + using namespace geocaching::storage; + uint8_t key[64]{}, hash[32]{}, other[32]{}, bytes[512]{}; + other[0] = 1; + DraftView draft; + size_t size = 0; + if (!encodeDraft({key, 16}, draft, bytes, sizeof(bytes), size) || !validStoredRowShape({4, {key, 16}, {bytes, size}, false}) || + validStoredRowShape({4, {key, 15}, {bytes, size}, false}) || validStoredRowShape({14, {key, 16}, {bytes, size}, false})) return 1; + bytes[size] = 0; + if (validStoredRowShape({4, {key, 16}, {bytes, size + 1}, false}) || validStoredRowShape({4, {key, 16}, {bytes, 1}, true}) || + !validStoredRowShape({4, {key, 16}, {}, true})) return 2; + StoredTime time; + { + protocol::CmpWriter writer(bytes, sizeof(bytes)); + if (!writer.array(4) || !writer.unsignedInteger(1) || !writer.nil() || !writer.text("Found\nlocal note") || !encodeStoredTime(writer, time) || + !validStoredRowShape({7, {key, 32}, {bytes, writer.size()}, false})) return 3; + bytes[1] = 3; + if (validStoredRowShape({7, {key, 32}, {bytes, writer.size()}, false})) return 4; + } + for (unsigned hashes = 0; hashes <= 2; ++hashes) + { + protocol::CmpWriter writer(bytes, sizeof(bytes)); + if (!writer.array(2) || !writer.array(hashes)) return 5; + if (hashes && !writer.binary({hash, 32})) return 6; + if (hashes == 2 && !writer.binary({other, 32})) return 7; + if (!writer.array(0) || validStoredRowShape({11, {key, 32}, {bytes, writer.size()}, false}) != (hashes != 1)) return 8; + if (hashes == 2) + { + bytes[38] = 0; // Make the second hash identical to the first. + if (validStoredRowShape({11, {key, 32}, {bytes, writer.size()}, false})) return 9; + } + } + { + protocol::CmpWriter writer(bytes, sizeof(bytes)); + if (!writer.array(8) || !writer.binary({key, 16}) || !writer.binary({key, 64}) || !writer.binary({key, 16}) || + !writer.unsignedInteger(1) || !writer.unsignedInteger(2) || !encodeStoredTime(writer, time) || !writer.nil() || !writer.nil() || + !validStoredRowShape({8, {key, 16}, {bytes, writer.size()}, false})) return 10; + } + for (bool trusted : {false, true}) + { + protocol::CmpWriter writer(bytes, sizeof(bytes)); + StoredTime deadline; + deadline.has_utc = true; + deadline.utc_trusted = trusted; + deadline.utc_seconds = 100; + if (!writer.array(5) || !writer.unsignedInteger(1) || !writer.binary({hash, 32}) || !writer.binary({key, 1}) || + !encodeStoredTime(writer, time) || !encodeStoredTime(writer, deadline) || + validStoredRowShape({6, {key, 48}, {bytes, writer.size()}, false}) != trusted) return 11; + } + return 0; +} diff --git a/modules/core_geocaching/tests/test_record_signing.cpp b/modules/core_geocaching/tests/test_record_signing.cpp new file mode 100644 index 00000000..f534256b --- /dev/null +++ b/modules/core_geocaching/tests/test_record_signing.cpp @@ -0,0 +1,52 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "geocaching/protocol/sign_record.h" +#include +#include +#include +struct TestIdentity +{ + std::array public_key{}, private_key{}; + unsigned calls = 0; + bool ready = true; + bool isReady() const { return ready; } + void combinedPublicKey(uint8_t* out) const { std::memcpy(out, public_key.data(), public_key.size()); } + bool sign(const uint8_t* data, size_t size, uint8_t* signature) + { + ++calls; + ed25519_sign(signature, data, size, public_key.data() + 32, private_key.data()); + return true; + } +}; +int main(int argc, char** argv) +{ + using namespace geocaching; + if (argc != 2) return 1; + std::ifstream file(argv[1], std::ios::binary); + std::vector expected((std::istreambuf_iterator(file)), {}); + protocol::CmpReader reader({expected.data(), expected.size()}); + size_t count = 0; + ByteView encoded; + if (!reader.array(count, 2) || !reader.binary(encoded, 4096)) return 2; + RecordView record; + if (!protocol::decodeGeocacheRecord(encoded, record)) return 3; + TestIdentity identity; + std::memcpy(identity.public_key.data(), record.author_public_key.data, 64); + // Public seed from the checked-in interoperability fixture, never device keys. + uint8_t seed[32]; + for (unsigned i = 0; i < 32; ++i) seed[i] = static_cast(i); + ed25519_create_keypair(identity.public_key.data() + 32, identity.private_key.data(), seed); + std::array workspace{}, output{}; + size_t size = 0; + if (!protocol::signGeocacheRecord(encoded, identity, workspace.data(), workspace.size(), output.data(), output.size(), size) || + size != expected.size() || std::memcmp(output.data(), expected.data(), size) || identity.calls != 1) return 4; + identity.public_key[0] ^= 1; + if (protocol::signGeocacheRecord(encoded, identity, workspace.data(), workspace.size(), output.data(), output.size(), size) || size || identity.calls != 1) return 5; + identity.public_key[0] ^= 1; + identity.ready = false; + if (protocol::signGeocacheRecord(encoded, identity, workspace.data(), workspace.size(), output.data(), output.size(), size) || identity.calls != 1) return 6; + identity.ready = true; + std::vector shared(encoded.size + 70); + if (!protocol::signGeocacheRecord(encoded, identity, shared.data(), shared.size(), shared.data(), shared.size(), size) || + size != expected.size() || std::memcmp(shared.data(), expected.data(), size) || identity.calls != 2) return 7; + return 0; +} diff --git a/modules/core_geocaching/tests/test_request_tracker.cpp b/modules/core_geocaching/tests/test_request_tracker.cpp new file mode 100644 index 00000000..030543f5 --- /dev/null +++ b/modules/core_geocaching/tests/test_request_tracker.cpp @@ -0,0 +1,37 @@ +#include "geocaching/usecase/request_tracker.h" +#include + +int main() +{ + using namespace geocaching; + Destination local, remote, stranger; + local.bytes.fill(1); + remote.bytes.fill(2); + stranger.bytes.fill(3); + RequestId id; + id.bytes.fill(4); + std::array digest{}; + RequestTracker tracker; + assert(tracker.begin(local, remote, id, Operation::Get, 8192)); + assert(!tracker.begin(local, remote, id, Operation::Get, 8192)); + assert(tracker.startAttempt()); + assert(!tracker.startAttempt()); + tracker.delivered(); + assert(tracker.phase() == RequestPhase::AwaitingResult); + assert(tracker.classifyResult(stranger, local, id, Operation::Get, 100, digest) == ResultAdmission::WrongRequest); + tracker.unconfirmed(); + assert(tracker.startAttempt()); + tracker.stopFurtherAttempts(); + tracker.unconfirmed(); + assert(!tracker.startAttempt()); + // Stopping retries does not discard a legitimate late committed result. + assert(tracker.recordResult(remote, local, id, Operation::Get, 100, digest)); + tracker.delivered(); + tracker.unconfirmed(); + assert(tracker.phase() == RequestPhase::ResultRecorded); + assert(tracker.classifyResult(remote, local, id, Operation::Get, 100, digest) == ResultAdmission::Duplicate); + digest[0] = 1; + assert(tracker.classifyResult(remote, local, id, Operation::Get, 100, digest) == ResultAdmission::ConflictingResult); + assert(!tracker.recordResult(remote, local, id, Operation::Get, 100, digest)); + assert(!tracker.continueRequested()); +} diff --git a/modules/core_geocaching/tests/test_sd_checkpoint.cpp b/modules/core_geocaching/tests/test_sd_checkpoint.cpp new file mode 100644 index 00000000..5f37e123 --- /dev/null +++ b/modules/core_geocaching/tests/test_sd_checkpoint.cpp @@ -0,0 +1,285 @@ +#include "chat/infra/reticulum/reticulum_wire.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_loader.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h" +#include "platform/esp/arduino_common/geocaching/sd_state_recovery.h" +#include + +std::vector file_bytes; +std::vector slot_b_bytes; +bool separate_b = false, b_busy = false; +bool slots_missing = false; +size_t step_data_bytes = 0; +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeDir::Impl +{ + public: + bool open = false; +}; +SdRuntimeDir::SdRuntimeDir() : impl_(new Impl) {} +SdRuntimeDir::~SdRuntimeDir() { delete impl_; } +bool SdRuntimeDir::open(const char*) +{ + impl_->open = true; + return true; +} +void SdRuntimeDir::close() { impl_->open = false; } +bool SdRuntimeDir::is_open() const { return impl_->open; } +SdDirReadStatus SdRuntimeDir::read_next_status(char*, size_t, bool*) { return SdDirReadStatus::End; } +SdFileReadResult sd_read_file(const char* path, uint8_t* buffer, size_t capacity) +{ + step_data_bytes += capacity; + SdFileReadResult result; + if (std::strstr(path, "format.bin")) + { + const auto header = ::geocaching::storage::encodeVolumeHeader({}); + result.file_size = header.size(); + result.bytes_read = std::min(capacity, header.size()); + std::memcpy(buffer, header.data(), result.bytes_read); + } + else + { + if (slots_missing) + { + result.status = SdFileReadStatus::Missing; + return result; + } + const bool slot_b = std::strstr(path, "b.gcs") != nullptr; + if (slot_b && b_busy) + { + result.status = SdFileReadStatus::Busy; + return result; + } + const auto& bytes = separate_b && slot_b ? slot_b_bytes : file_bytes; + result.file_size = bytes.size(); + result.bytes_read = std::min(capacity, bytes.size()); + std::memcpy(buffer, bytes.data(), result.bytes_read); + } + result.status = SdFileReadStatus::Ready; + return result; +} +class SdRuntimeFile::Impl +{ + public: + bool open = false; + size_t offset = 0; + const std::vector* bytes = &file_bytes; +}; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char* path, const char*) +{ + impl_->bytes = separate_b && std::strstr(path, "b.gcs") ? &slot_b_bytes : &file_bytes; + impl_->open = true; + impl_->offset = 0; + return true; +} +void SdRuntimeFile::close() { impl_->open = false; } +bool SdRuntimeFile::is_open() const { return impl_->open; } +uint64_t SdRuntimeFile::size() const { return impl_->bytes->size(); } +int SdRuntimeFile::read(void* output, size_t count) +{ + step_data_bytes += count; + const auto n = std::min(count, impl_->bytes->size() - impl_->offset); + std::memcpy(output, impl_->bytes->data() + impl_->offset, n); + impl_->offset += n; + return static_cast(n); +} +} // namespace platform::esp::arduino_common::storage +struct Digest +{ + std::vector bytes; + void update(const uint8_t* data, size_t size) { bytes.insert(bytes.end(), data, data + size); } + bool finalize(uint8_t* out, size_t) + { + chat::reticulum::fullHash(bytes.data(), bytes.size(), out); + return true; + } +}; +std::vector checkpointBytes(uint64_t sequence, uint8_t key) +{ + uint8_t page[] = {0x92, 0, 0x91, 0x93, 5, 0xc4, 1, key, 0xc4, 0}; + ::geocaching::storage::RecordHeader header; + ::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointPage, sequence, {page, sizeof(page)}, header); + std::vector result(header.begin(), header.end()); + result.insert(result.end(), page, page + sizeof(page)); + uint8_t tail[37] = {0x93, 1, 1, 0xc4, 32}; + chat::reticulum::fullHash(result.data(), result.size(), tail + 5); + ::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointTail, sequence, {tail, sizeof(tail)}, header); + result.insert(result.end(), header.begin(), header.end()); + result.insert(result.end(), tail, tail + sizeof(tail)); + return result; +} +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + const uint8_t page[] = {0x92, 0, 0x91, 0x93, 5, 0xc4, 1, 1, 0xc4, 0}; + ::geocaching::storage::RecordHeader header; + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointPage, 7, {page, sizeof(page)}, header)) return 1; + file_bytes.assign(header.begin(), header.end()); + file_bytes.insert(file_bytes.end(), page, page + sizeof(page)); + const auto page_size = file_bytes.size(); + uint8_t tail[37] = {0x93, 1, 1, 0xc4, 32}; + chat::reticulum::fullHash(file_bytes.data(), file_bytes.size(), tail + 5); + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointTail, 7, {tail, sizeof(tail)}, header)) return 2; + file_bytes.insert(file_bytes.end(), header.begin(), header.end()); + file_bytes.insert(file_bytes.end(), tail, tail + sizeof(tail)); + const auto complete_file = file_bytes; + uint8_t buffer[256]; + ::geocaching::storage::MutationView entry; + size_t count = 0; + Digest cursor_digest; + SdCheckpointReader cursor_reader(cursor_digest); + ::geocaching::storage::CheckpointPageCursor page_cursor; + ::geocaching::storage::CheckpointIndexCursor index_cursor; + ::geocaching::storage::IndexedMutation indexed; + if (!cursor_reader.open('b') || cursor_reader.stepCursor(buffer, sizeof(buffer), page_cursor) != CheckpointReadStep::Reading || + page_cursor.count() != 1 || !page_cursor.next(entry) || !page_cursor.complete() || cursor_reader.sequence() != 0 || + !cursor_reader.pendingIndex(index_cursor) || !index_cursor.next(indexed) || !index_cursor.complete() || + indexed.table != entry.table || indexed.location.source != ::geocaching::storage::IndexedValueSource::CheckpointB || + indexed.location.record_sequence != 7 || indexed.location.frame_offset != 0 || indexed.location.value_size != entry.value.size) return 21; + if (cursor_reader.stepCursor(buffer, sizeof(buffer), page_cursor) != CheckpointReadStep::Reading || + cursor_reader.pendingIndex(index_cursor) || cursor_reader.sequence() != 0 || + cursor_reader.stepCursor(buffer, sizeof(buffer), page_cursor) != CheckpointReadStep::Verified || + cursor_reader.sequence() != 7 || std::memcmp(cursor_reader.digest().data(), tail + 5, 32)) return 22; + Digest valid_digest; + SdCheckpointReader valid(valid_digest); + if (!valid.open('a') || valid.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Reading || count != 1 || + valid.sequence() != 0 || valid.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Reading || + valid.sequence() != 0 || valid.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Verified || valid.sequence() != 7) return 3; + file_bytes.push_back(0); + Digest extra_digest; + SdCheckpointReader extra(extra_digest); + if (!extra.open('b') || extra.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Reading || + extra.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Reading || + extra.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Invalid || extra.sequence() != 0) return 4; + file_bytes.resize(page_size); + Digest missing_digest; + SdCheckpointReader missing(missing_digest); + if (!missing.open('a') || missing.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Reading || + missing.step(buffer, sizeof(buffer), &entry, 1, count) != CheckpointReadStep::Invalid) return 5; + file_bytes = complete_file; + SdCheckpointSelection selection({}); + auto selected = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 64 && selected == CheckpointSelectionStep::Reading; ++i) + selected = selection.step(buffer, sizeof(buffer), &entry, 1); + if (selected != CheckpointSelectionStep::Selected || selection.choice() != ::geocaching::storage::CheckpointChoice::SlotA || + selection.candidate(false).sequence != 7 || selection.candidate(true).sequence != 7) return 6; + slots_missing = true; + SdCheckpointSelection empty({}); + auto empty_result = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 16 && empty_result == CheckpointSelectionStep::Reading; ++i) empty_result = empty.step(buffer, sizeof(buffer), &entry, 1); + if (empty_result != CheckpointSelectionStep::NoCheckpoint) return 7; + slots_missing = false; + separate_b = true; + auto check_selection = [&](CheckpointSelectionStep expected, ::geocaching::storage::CheckpointChoice choice) + { + SdCheckpointSelection candidate({}); + auto state = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 64 && state == CheckpointSelectionStep::Reading; ++i) + state = candidate.step(buffer, sizeof(buffer), &entry, 1); + return state == expected && (state != CheckpointSelectionStep::Selected || candidate.choice() == choice); + }; + using Choice = ::geocaching::storage::CheckpointChoice; + slot_b_bytes = checkpointBytes(8, 1); + if (!check_selection(CheckpointSelectionStep::Selected, Choice::SlotB)) return 8; + slot_b_bytes.pop_back(); + if (!check_selection(CheckpointSelectionStep::Selected, Choice::SlotA)) return 9; + slot_b_bytes = checkpointBytes(7, 2); + if (!check_selection(CheckpointSelectionStep::Corrupt, Choice::Corrupt)) return 10; + b_busy = true; + if (!check_selection(CheckpointSelectionStep::RetryLater, Choice::RetryLater)) return 11; + b_busy = false; + uint8_t live[128]{}, spare[128]{}, old_key = 9; + ::geocaching::storage::LogicalState state(live, spare, sizeof(live)); + ::geocaching::storage::MutationView old{1, {&old_key, 1}, {}, false}; + auto allow = [](const auto&) + { return true; }; + if (!state.apply(&old, 1, allow)) return 12; + Digest load_digest; + SdCheckpointLoader loader(load_digest, state, {}, 'a', selection.candidate(false)); + ::geocaching::ByteView value; + auto load_result = CheckpointLoadStep::Loading; + for (unsigned i = 0; i < 64 && load_result == CheckpointLoadStep::Loading; ++i) + { + if (!state.view().find(1, {&old_key, 1}, value)) return 13; + load_result = loader.step(buffer, sizeof(buffer), &entry, 1, allow); + } + if (load_result != CheckpointLoadStep::Applied || state.view().find(1, {&old_key, 1}, value) || state.view().size() != 1) return 13; + if (!state.apply(&old, 1, allow)) return 14; + auto wrong_candidate = selection.candidate(false); + wrong_candidate.digest[0] ^= 1; + Digest changed_digest; + SdCheckpointLoader changed(changed_digest, state, {}, 'a', wrong_candidate); + load_result = CheckpointLoadStep::Loading; + for (unsigned i = 0; i < 64 && load_result == CheckpointLoadStep::Loading; ++i) load_result = changed.step(buffer, sizeof(buffer), &entry, 1, allow); + if (load_result != CheckpointLoadStep::Invalid || !state.view().find(1, {&old_key, 1}, value) || state.view().size() != 2) return 15; + separate_b = false; + uint8_t recovered_a[256]{}, recovered_b[256]{}; + ::geocaching::storage::LogicalState recovered(recovered_a, recovered_b, sizeof(recovered_a)); + SdStateRecovery recovery({}, recovered); + auto recovery_result = StateRecoveryStep::Working; + for (unsigned i = 0; i < 128 && recovery_result == StateRecoveryStep::Working; ++i) + recovery_result = recovery.step(buffer, sizeof(buffer), &entry, 1, allow); + if (recovery_result != StateRecoveryStep::JournalRestored || recovery.replayedSequence() != 7 || recovered.view().size() != 1) return 16; + slots_missing = true; + SdStateRecovery fresh({}, recovered); + recovery_result = StateRecoveryStep::Working; + for (unsigned i = 0; i < 10 && recovery_result == StateRecoveryStep::Working; ++i) + recovery_result = fresh.step(buffer, sizeof(buffer), &entry, 1, allow); + if (recovery_result != StateRecoveryStep::JournalRestored || fresh.replayedSequence() != 0 || recovered.view().size() != 0) return 17; + slots_missing = false; + Digest small_digest; + SdCheckpointReader small(small_digest); + if (!small.open('a') || small.step(buffer, 24, &entry, 1, count) != CheckpointReadStep::WorkspaceTooSmall) return 18; + Digest entries_digest; + SdCheckpointReader no_entries(entries_digest); + if (!no_entries.open('a') || no_entries.step(buffer, sizeof(buffer), &entry, 0, count) != CheckpointReadStep::WorkspaceTooSmall) return 19; + SdCheckpointSelection constrained({}); + auto constrained_result = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 32 && constrained_result == CheckpointSelectionStep::Reading; ++i) constrained_result = constrained.step(buffer, sizeof(buffer), &entry, 0); + if (constrained_result != CheckpointSelectionStep::RetryLater || constrained.choice() != Choice::RetryLater) return 20; + SdCheckpointSelection streaming({}); + auto streaming_result = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 64 && streaming_result == CheckpointSelectionStep::Reading; ++i) streaming_result = streaming.stepCursor(buffer, sizeof(buffer)); + if (streaming_result != CheckpointSelectionStep::Selected || streaming.candidate(false).sequence != 7) return 23; + // A page larger than one SPI slice catches accidental accumulation of + // volume-header, probe and page reads in the same maintenance step. + std::vector large_value(700, 42), large_payload(1024); + ::geocaching::protocol::CmpWriter large_writer(large_payload.data(), large_payload.size()); + const uint8_t large_key = 1; + if (!large_writer.array(2) || !large_writer.unsignedInteger(0) || !large_writer.array(1) || !large_writer.array(3) || + !large_writer.unsignedInteger(5) || !large_writer.binary({&large_key, 1}) || + !large_writer.binary({large_value.data(), large_value.size()}) || + !::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointPage, 7, + {large_payload.data(), large_writer.size()}, header)) return 24; + file_bytes.assign(header.begin(), header.end()); + file_bytes.insert(file_bytes.end(), large_payload.data(), large_payload.data() + large_writer.size()); + chat::reticulum::fullHash(file_bytes.data(), file_bytes.size(), tail + 5); + if (!::geocaching::storage::makeRecordHeader(::geocaching::storage::RecordKind::CheckpointTail, 7, {tail, sizeof(tail)}, header)) return 25; + file_bytes.insert(file_bytes.end(), header.begin(), header.end()); + file_bytes.insert(file_bytes.end(), tail, tail + sizeof(tail)); + SdCheckpointSelection budgeted({}); + auto budgeted_result = CheckpointSelectionStep::Reading; + for (unsigned i = 0; i < 128 && budgeted_result == CheckpointSelectionStep::Reading; ++i) + { + step_data_bytes = 0; + budgeted_result = budgeted.stepCursor(large_payload.data(), large_payload.size()); + if (step_data_bytes > 512) return 26; + } + if (budgeted_result != CheckpointSelectionStep::Selected) return 27; + std::vector large_live(1024), large_spare(1024); + ::geocaching::storage::LogicalState large_state(large_live.data(), large_spare.data(), large_live.size()); + Digest large_digest; + SdCheckpointLoader large_loader(large_digest, large_state, {}, 'a', budgeted.candidate(false)); + load_result = CheckpointLoadStep::Loading; + for (unsigned i = 0; i < 128 && load_result == CheckpointLoadStep::Loading; ++i) + { + step_data_bytes = 0; + load_result = large_loader.step(large_payload.data(), large_payload.size(), &entry, 1, allow); + if (step_data_bytes > 512) return 28; + } + if (load_result != CheckpointLoadStep::Applied || large_state.view().size() != 1) return 29; + return 0; +} diff --git a/modules/core_geocaching/tests/test_sd_journal.cpp b/modules/core_geocaching/tests/test_sd_journal.cpp new file mode 100644 index 00000000..45d18a7b --- /dev/null +++ b/modules/core_geocaching/tests/test_sd_journal.cpp @@ -0,0 +1,1324 @@ +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "chat/infra/reticulum/reticulum_wire.h" +#include "geocaching/protocol/publish_request.h" +#include "geocaching/protocol/query_request.h" +#include "geocaching/protocol/record_encoder.h" +#include "geocaching/storage/draft_publication.h" +#include "geocaching/storage/task_record.h" +#include "platform/esp/arduino_common/geocaching/query_browse_source.h" +#include "platform/esp/arduino_common/geocaching/query_store_port.h" +#include "platform/esp/arduino_common/geocaching/request_dispatcher.h" +#include "platform/esp/arduino_common/geocaching/saved_cache_catalog.h" +#include "platform/esp/arduino_common/geocaching/sd_author_issue_port.h" +#include "platform/esp/arduino_common/geocaching/sd_download_port.h" +#include "platform/esp/arduino_common/geocaching/sd_gpx_stage.h" +#include "platform/esp/arduino_common/geocaching/sd_journal.h" +#include "platform/esp/arduino_common/geocaching/sd_publish_port.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" +#include +#include +#include +#include +#include +#include + +namespace fixture +{ +std::map files; +bool flush_ok = true, ready = true, busy = false; +size_t write_limit = SIZE_MAX, corrupt_at = SIZE_MAX; +unsigned writes = 0, step_data_calls = 0, step_io_calls = 0; +size_t step_data_bytes = 0; +bool supported_volume = true; +::geocaching::storage::VolumeInstance volume{}; +} // namespace fixture +namespace platform::esp::arduino_common::storage +{ +SdFileReadResult sd_read_file(const char*, uint8_t* buffer, size_t capacity) +{ + ++fixture::step_io_calls; + ++fixture::step_data_calls; + fixture::step_data_bytes += 28; + auto header = ::geocaching::storage::encodeVolumeHeader(fixture::volume); + if (!fixture::supported_volume) header[5] = 2; + SdFileReadResult result; + result.status = SdFileReadStatus::Ready; + result.file_size = header.size(); + result.bytes_read = std::min(capacity, header.size()); + std::memcpy(buffer, header.data(), result.bytes_read); + return result; +} +class SdRuntimeFile::Impl +{ + public: + std::string path; + size_t position = 0; + bool opened = false; +}; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char* path, const char* mode) +{ + ++fixture::step_io_calls; + impl_->path = path; + impl_->position = 0; + if (*mode == 'w') fixture::files[path].clear(); + impl_->opened = fixture::files.count(path) != 0; + return impl_->opened; +} +void SdRuntimeFile::close() +{ + ++fixture::step_io_calls; + impl_->opened = false; +} +bool SdRuntimeFile::is_open() const { return impl_->opened; } +size_t SdRuntimeFile::write(const void* bytes, size_t count) +{ + ++fixture::writes; + ++fixture::step_io_calls; + ++fixture::step_data_calls; + fixture::step_data_bytes += count; + const auto n = std::min(count, fixture::write_limit); + fixture::files[impl_->path].append(static_cast(bytes), n); + return n; +} +bool SdRuntimeFile::flush() +{ + ++fixture::step_io_calls; + return fixture::flush_ok; +} +uint64_t SdRuntimeFile::size() const +{ + ++fixture::step_io_calls; + return fixture::files[impl_->path].size(); +} +int SdRuntimeFile::read(void* bytes, size_t count) +{ + ++fixture::step_io_calls; + ++fixture::step_data_calls; + fixture::step_data_bytes += count; + const auto& data = fixture::files[impl_->path]; + const auto n = std::min(count, data.size() - impl_->position); + std::memcpy(bytes, data.data() + impl_->position, n); + if (fixture::corrupt_at >= impl_->position && fixture::corrupt_at - impl_->position < n) + static_cast(bytes)[fixture::corrupt_at - impl_->position] ^= 1; + impl_->position += n; + return static_cast(n); +} +bool sd_card_ready() { return fixture::ready; } +bool sd_external_block_owner_active() { return fixture::busy; } +bool sd_is_directory(const char*) +{ + ++fixture::step_io_calls; + return true; +} +bool sd_exists(const char* path) +{ + ++fixture::step_io_calls; + return fixture::files.count(path) != 0; +} +bool sd_rename(const char* from, const char* to) +{ + ++fixture::step_io_calls; + if (!fixture::files.count(from) || fixture::files.count(to)) return false; + fixture::files[to] = std::move(fixture::files[from]); + fixture::files.erase(from); + return true; +} +} // namespace platform::esp::arduino_common::storage +struct NativeRecordCrypto final : ::geocaching::protocol::RecordCrypto +{ + bool sha256(::geocaching::ByteView bytes, uint8_t out[32]) override + { + ::chat::reticulum::fullHash(bytes.data, bytes.size, out); + return true; + } + ::geocaching::protocol::VerificationResult verifyEd25519(::geocaching::ByteView key, ::geocaching::ByteView signature, ::geocaching::ByteView message) override + { + return ed25519_verify(signature.data, message.data, message.size, key.data) ? ::geocaching::protocol::VerificationResult::Valid : ::geocaching::protocol::VerificationResult::InvalidSignature; + } +}; +// Test driver only: production callers return to their owner after each step. +using namespace platform::esp::arduino_common::geocaching; +bool stepBudgetOk() { return fixture::step_io_calls <= 1 && fixture::step_data_calls <= 1 && fixture::step_data_bytes <= 512; } +JournalWriteResult drainJournal(SdGeocachingJournal& journal, JournalWriteResult result) +{ + for (unsigned i = 0; result == JournalWriteResult::InProgress && i < 1024; ++i) + { + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + result = journal.step(); + if (!stepBudgetOk()) return JournalWriteResult::Invalid; + } + return result; +} +JournalWriteResult drainStore(SdRequestStore& store, JournalWriteResult result) +{ + const auto sequence = store.committedSequence(); + for (unsigned i = 0; result == JournalWriteResult::InProgress && i < 1024; ++i) + { + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + result = store.stepCommit(); + if (!stepBudgetOk() || (result == JournalWriteResult::InProgress && store.committedSequence() != sequence)) + return JournalWriteResult::Invalid; + } + return result; +} +int checkIncrementalBoundaries() +{ + // Host fixture deliberately exceeds one I/O slice. + std::array bytes{}; + const uint8_t key = 1; + ::geocaching::storage::MutationView mutation{5, {&key, 1}, {bytes.data(), bytes.size()}, false}; + unsigned completed_steps = 0; + { + fixture::files.clear(); + auto journal = std::make_unique(fixture::volume); + fixture::step_io_calls = 0; + auto result = journal->begin(0, &mutation, 1); + if (result != JournalWriteResult::InProgress || fixture::step_io_calls) return 90; + if (journal->begin(1, &mutation, 1) != JournalWriteResult::Busy) return 91; + while (result == JournalWriteResult::InProgress && completed_steps < 1024) + { + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + result = journal->step(); + ++completed_steps; + if (!stepBudgetOk()) return 92; + } + if (result != JournalWriteResult::Verified || completed_steps < 30) return 93; + } + // Cancel at every pre-completion boundary, including open, flush and readback. + for (unsigned boundary = 0; boundary < completed_steps; ++boundary) + { + fixture::files.clear(); + auto journal = std::make_unique(fixture::volume); + auto result = journal->begin(0, &mutation, 1); + for (unsigned i = 0; i < boundary; ++i) result = journal->step(); + if (result != JournalWriteResult::InProgress) return 94; + fixture::step_io_calls = 0; + if (journal->cancel() != JournalWriteResult::Cancelled || fixture::step_io_calls > 1 || + journal->mayHaveWritten() != !fixture::files.empty()) return 95; + fixture::step_io_calls = 0; + if (journal->step() != JournalWriteResult::Cancelled || fixture::step_io_calls) return 96; + } + fixture::files.clear(); + uint8_t first[1024]{}, second[1024]{}; + ::geocaching::storage::LogicalState state(first, second, sizeof(first)); + auto store = std::make_unique(fixture::volume, 0, state); + const auto start = [&]() + { + uint8_t request[128]{}; + ::geocaching::RequestId id; + std::array task{}; + size_t size = 0; + if (!::geocaching::protocol::encodeCapabilitiesRequest(id, request, sizeof(request), size)) + return JournalWriteResult::Invalid; + const auto result = store->persistNewTask({}, {}, id, task, 3, {request, size}, {}); + // All caller-owned bytes expire before the first storage step. + std::memset(request, 0xa5, sizeof(request)); + task.fill(0xa5); + return result; + }; + if (start() != JournalWriteResult::InProgress || state.view().size() || store->committedSequence() || + !fixture::files.empty() || state.beginSnapshot()) return 97; + if (start() != JournalWriteResult::Busy || state.view().size()) return 98; + if (store->cancelCommit() != JournalWriteResult::Cancelled || store->needsRecovery() || + store->commitPending() || state.view().size()) return 99; + if (start() != JournalWriteResult::InProgress) return 100; + unsigned steps = 0; + auto result = JournalWriteResult::InProgress; + while (result == JournalWriteResult::InProgress && steps++ < 1024) + { + result = store->stepCommit(); + if (result == JournalWriteResult::InProgress && (state.view().size() || store->committedSequence())) return 101; + } + if (result != JournalWriteResult::Verified || state.view().size() != 2 || store->committedSequence() != 1) return 102; + fixture::files.clear(); + return 0; +} + +int checkIncrementalDispatch() +{ + class DelayedStore : public SdRequestStore + { + public: + using SdRequestStore::SdRequestStore; + unsigned selection_wait = 0, send_wait = 0, expiration_calls = 0; + DispatchReadResult readPending(const ::geocaching::Destination& local, ::geocaching::ByteView after, + ::geocaching::storage::PendingRequestView& out) override + { + if (selection_wait) + { + --selection_wait; + out = {}; + return DispatchReadResult::Pending; + } + return SdRequestStore::readPending(local, after, out); + } + DispatchReadResult readForSend(::geocaching::ByteView key, DispatchSendView& out) override + { + if (send_wait) + { + --send_wait; + out = {}; + return DispatchReadResult::Pending; + } + return SdRequestStore::readForSend(key, out); + } + JournalWriteResult expireOneAttempt(const ::geocaching::storage::StoredTime& now, uint64_t started, uint64_t timeout, bool& expired) override + { + ++expiration_calls; + return SdRequestStore::expireOneAttempt(now, started, timeout, expired); + } + }; + for (unsigned scenario = 0; scenario < 8; ++scenario) + { + const auto mode = scenario % 4; + fixture::files.clear(); + fixture::flush_ok = true; + uint8_t first[2048]{}, second[2048]{}, request[128]{}; + ::geocaching::storage::LogicalState state(first, second, sizeof(first)); + auto store = std::make_unique(fixture::volume, 0, state); + store->selection_wait = store->send_wait = scenario >= 4 ? 3 : 0; + ::geocaching::RequestId id; + size_t size = 0; + if (!::geocaching::protocol::encodeCapabilitiesRequest(id, request, sizeof(request), size) || + drainStore(*store, store->persistNewTask({}, {}, id, {}, 3, {request, size}, {})) != JournalWriteResult::Verified) return 110; + chat::MeshAdapterRouter router; + router.send_ok = mode != 1; + auto dispatcher = std::make_unique(router, *store, 100, 1000); + if (mode == 2) fixture::flush_ok = false; + bool stopped = false, done = false; + for (unsigned step = 0; step < 300; ++step) + { + // Stop after attempt reservation commits but before sending. The + // real dispatcher must reacquire state and honor the task intent. + if (mode == 3 && !stopped && store->committedSequence() == 2 && !store->commitPending()) + { + if (drainStore(*store, store->stopTask({})) != JournalWriteResult::Verified) return 111; + stopped = true; + } + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + const auto old_sends = router.sends; + const auto result = dispatcher->dispatchOne({}); + if (store->selection_wait && (router.sends || store->expiration_calls != 1)) return 240; + if (store->send_wait && router.sends) return 241; + if (!stepBudgetOk()) return 112; + if (router.sends != old_sends && store->committedSequence() != 2) return 113; + if (mode == 0 && result.status == DispatchStatus::Submitted) + { + if (router.sends != 1 || store->committedSequence() != 3 || + router.sent_bytes.size() != size || std::memcmp(router.sent_bytes.data(), request, size)) return 114; + done = true; + break; + } + if (mode == 1 && store->committedSequence() == 3 && !store->commitPending()) + { + if (router.sends != 1 || result.status != DispatchStatus::Deferred) return 115; + dispatcher->dispatchOne({}); + if (router.sends != 1) return 116; + done = true; + break; + } + if (mode == 2 && result.status == DispatchStatus::StorageBlocked) + { + if (router.sends || !store->needsRecovery() || store->committedSequence() != 1) return 117; + done = true; + break; + } + if (mode == 3 && store->committedSequence() == 4 && !store->commitPending()) + { + if (router.sends || !stopped) return 118; + done = true; + break; + } + } + if (!done) return 119; + } + fixture::files.clear(); + fixture::flush_ok = true; + return 0; +} + +int checkBrowseFlow(const char* capabilities_path, const char* query_path) +{ + using namespace ::geocaching; + fixture::files.clear(); + std::ifstream caps_file(capabilities_path, std::ios::binary), query_file(query_path, std::ios::binary); + std::vector caps((std::istreambuf_iterator(caps_file)), {}); + std::vector response((std::istreambuf_iterator(query_file)), {}); + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + auto store = std::make_unique(fixture::volume, 0, state); + unsigned sequence = 0; + QueryStorePort port( + *store, state, {}, [](void* context, uint8_t out[16]) + { + std::memset(out, ++*static_cast(context), 16); return true; }, + [](void*) + { return storage::StoredTime{}; }, + &sequence); + auto client = std::make_unique(port); + QueryBrowseSource source(*client, port, {300000000, 1200000000, 310000000, 1210000000}); + const auto advance = [&]() + { + for (unsigned i = 0; i < 1024; ++i) + { + const auto phase = client->phase(); + if (phase != QueryClientPhase::PersistingRequest && phase != QueryClientPhase::PersistingCapabilities && + phase != QueryClientPhase::PersistingPage) return phase != QueryClientPhase::Failed; + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + client->tick(0); + if (!stepBudgetOk()) return false; + } + return false; + }; + std::array key{}; + std::vector announce{0x95, 1, 0xc4, 16}; + announce.insert(announce.end(), 16, 0); + announce.insert(announce.end(), {0xc4, 16}); + announce.insert(announce.end(), 16, 0); + announce.insert(announce.end(), {0, 0xa1, 'x'}); + if (!client->observe({}, {}, {key.data(), key.size()}, {announce.data(), announce.size()}, 0)) return 120; + source.refresh(::ui::geocaching::Section::Discover); + if (!client->tick(0) || client->phase() != QueryClientPhase::PersistingRequest || !advance() || + client->phase() != QueryClientPhase::CheckingCapabilities) return 121; + if (client->accept({}, {caps.data(), caps.size()}) || client->phase() != QueryClientPhase::PersistingCapabilities || + !advance() || !client->tick(1) || !advance() || client->phase() != QueryClientPhase::Querying) return 122; + ::ui::geocaching::Snapshot snapshot; + source.snapshot(::ui::geocaching::Section::Discover, snapshot); + if (snapshot.count || client->accept({}, {response.data(), response.size()}) || + client->phase() != QueryClientPhase::PersistingPage) return 123; + source.snapshot(::ui::geocaching::Section::Discover, snapshot); + RequestId id; + id.bytes.fill(3); + if (snapshot.count || port.accepted({}, id, {response.data(), response.size()})) return 124; + const auto original = response; + std::fill(response.begin(), response.end(), 0xa5); + if (!advance() || client->phase() != QueryClientPhase::PageReady) return 125; + source.snapshot(::ui::geocaching::Section::Discover, snapshot); + ::ui::geocaching::Item item; + if (snapshot.count != 1 || !source.item(::ui::geocaching::Section::Discover, 0, snapshot.generation, item) || + std::strcmp(item.name.data(), "Test") || !std::strstr(item.detail.data(), "Directory preview") || + !port.accepted({}, id, {original.data(), original.size()})) return 126; + Destination wrong; + wrong.bytes[0] = 1; + if (port.accepted(wrong, id, {original.data(), original.size()})) return 127; + source.refresh(::ui::geocaching::Section::Discover); + if (source.item(::ui::geocaching::Section::Discover, 0, snapshot.generation, item)) return 128; + if (!client->tick(10) || !advance() || client->phase() != QueryClientPhase::Querying) return 129; + client->tick(10); + if (!client->tick(10 + QueryClient::kReplyTimeoutMs) || client->phase() != QueryClientPhase::Cancelling) return 130; + id.bytes.fill(5); + auto late = original; + protocol::CmpReader late_reader({late.data(), late.size()}); + size_t late_fields = 0; + uint64_t late_value = 0; + ByteView late_id; + if (!late_reader.array(late_fields, 6) || !late_reader.unsignedInteger(late_value) || + !late_reader.unsignedInteger(late_value) || !late_reader.unsignedInteger(late_value) || + !late_reader.binary(late_id, 16)) return 135; + std::memcpy(late.data() + (late_id.data - late.data()), id.bytes.data(), 16); + protocol::QueryPageView late_page; + if (!protocol::decodeQueryPage({late.data(), late.size()}, id, 2048, 20, late_page)) return 136; + if (port.accepted({}, id, {late.data(), late.size()})) return 131; + for (unsigned i = 0; client->persistencePending() && i < 1024; ++i) client->tick(10 + QueryClient::kReplyTimeoutMs); + if (client->phase() != QueryClientPhase::Failed || client->failure() != QueryFailure::Timeout || + !port.accepted({}, id, {late.data(), late.size()})) return 132; + storage::PendingRequestView pending; + if (storage::nextPendingRequest(state.view(), {}, {}, pending) != storage::PendingRequestResult::None) return 133; + source.snapshot(::ui::geocaching::Section::Discover, snapshot); + if (!snapshot.can_refresh || !std::strstr(snapshot.status.data(), "did not reply")) return 134; + fixture::files.clear(); + return 0; +} + +int checkDownloadReceipt(const char* query_path, const char* response_path) +{ + using namespace ::geocaching; + fixture::files.clear(); + std::ifstream query_file(query_path, std::ios::binary), reply_file(response_path, std::ios::binary); + std::vector query((std::istreambuf_iterator(query_file)), {}), reply((std::istreambuf_iterator(reply_file)), {}); + RequestId id; + id.bytes.fill(3); + protocol::SummaryView summary; + protocol::QueryPageView page; + if (!protocol::decodeQueryResponse({query.data(), query.size()}, id, 2048, &summary, 1, page)) return 140; + id.bytes.fill(4); + uint8_t request[128]{}; + size_t request_size = 0; + if (!protocol::encodeGetRequest(id, summary.id, &summary.hash, nullptr, 8192, request, sizeof(request), request_size)) return 141; + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + auto store = std::make_unique(fixture::volume, 0, state); + storage::RequestTaskTarget target{{summary.id.bytes.data(), 32}, {summary.hash.bytes.data(), 32}, 1}; + std::array task_id{}; + if (store->nextDownloadGeneration(summary.id) != 1 || + store->persistNewTask({}, {}, id, task_id, 2, {request, request_size}, {}, target) != JournalWriteResult::InProgress || + store->nextDownloadGeneration(summary.id) != 1) return 142; + if (drainStore(*store, JournalWriteResult::InProgress) != JournalWriteResult::Verified || + store->nextDownloadGeneration(summary.id) != 2) return 143; + NativeRecordCrypto crypto; + if (store->recordDownloadResponse({}, {}, id, 2, {reply.data(), reply.size()}, crypto) != JournalWriteResult::StateRejected) return 144; + protocol::GetResponseView parsed; + if (!protocol::decodeGetResponse({reply.data(), reply.size()}, id, 8192, parsed)) return 145; + protocol::CmpReader signed_record(parsed.signed_cache); + size_t count = 0; + ByteView record, signature; + if (!signed_record.array(count, 2) || !signed_record.binary(record, 4096) || !signed_record.binary(signature, 64)) return 146; + auto corrupt = reply; + corrupt[signature.data - reply.data()] ^= 1; + if (store->recordDownloadResponse({}, {}, id, 1, {corrupt.data(), corrupt.size()}, crypto) != JournalWriteResult::Invalid || + store->committedSequence() != 1) return 147; + if (drainStore(*store, store->recordDownloadResponse({}, {}, id, 1, {reply.data(), reply.size()}, crypto)) != JournalWriteResult::Verified) return 148; + ByteView bytes; + storage::TaskView task; + storage::CacheHeadView head; + if (!state.view().find(10, {task_id.data(), task_id.size()}, bytes) || !storage::decodeTask({task_id.data(), task_id.size()}, bytes, task) || + task.state != 1 || !state.view().find(2, target.cache_id, bytes) || !storage::decodeCacheHead(target.cache_id, bytes, head) || + head.current_hash.size) return 149; // Received is not Installed. + const auto committed = store->committedSequence(); + // Use the durably recorded response as the source for actual GPX staging. + uint8_t request_key[48]{}; + std::memcpy(request_key + 32, id.bytes.data(), 16); + storage::OutgoingView outgoing; + if (!state.view().find(5, {request_key, 48}, bytes) || !storage::decodeOutgoing({request_key, 48}, bytes, outgoing) || + !protocol::decodeGetResponse(outgoing.terminal_data, id, 8192, parsed)) return 155; + protocol::VerifiedRecordView verified; + auto verification = protocol::VerificationResult::WorkspaceTooSmall; + state.withScratch([&](uint8_t* scratch, size_t capacity) + { verification = protocol::verifyGeocache(parsed.signed_cache, crypto, scratch, capacity, verified, &summary.id, &summary.hash); }); + if (verification != protocol::VerificationResult::Valid) return 156; + std::array transaction{}; + transaction[0] = 7; + auto gpx = std::make_unique(); + auto written = gpx->begin(transaction, verified, crypto); + for (unsigned i = 0; written == StageResult::InProgress && i < 512; ++i) + { + fixture::step_io_calls = fixture::step_data_calls = 0; + fixture::step_data_bytes = 0; + written = gpx->step(); + if (!stepBudgetOk()) return 157; + } + if (written != StageResult::Written || fixture::files.at(gpx->path()).find("Test") == std::string::npos || + fixture::files.at(gpx->path()).find("recordDownloadResponse({}, {}, id, 1, {reply.data(), reply.size()}, crypto) != JournalWriteResult::Verified || + store->committedSequence() != committed) return 150; + std::array file_hash{}; + const auto& staged = fixture::files.at(gpx->path()); + crypto.sha256({reinterpret_cast(staged.data()), staged.size()}, file_hash.data()); + if (store->finishDownloadInstall({request_key, 48}, transaction, 1, file_hash) != JournalWriteResult::StateRejected) return 159; + if (drainStore(*store, store->prepareDownloadInstall({request_key, 48}, transaction, 1, file_hash, {}, crypto)) != JournalWriteResult::Verified) return 160; + const auto prepared_sequence = store->committedSequence(); + if (store->prepareDownloadInstall({request_key, 48}, transaction, 1, file_hash, {}, crypto) != JournalWriteResult::Verified || + store->committedSequence() != prepared_sequence) return 161; + auto wrong_hash = file_hash; + wrong_hash[0] ^= 1; + if (store->finishDownloadInstall({request_key, 48}, transaction, 1, wrong_hash) != JournalWriteResult::StateRejected || + store->finishDownloadInstall({request_key, 48}, transaction, 2, file_hash) != JournalWriteResult::StateRejected) return 162; + std::string target_path = "/trailmate/geocaching/caches/"; + constexpr char alphabet[] = "0123456789abcdef"; + for (auto byte : summary.id.bytes) + { + target_path += alphabet[byte >> 4]; + target_path += alphabet[byte & 15]; + } + target_path += ".gpx"; + if (!platform::esp::arduino_common::storage::sd_rename(gpx->path(), target_path.c_str())) return 163; + const auto& installed = fixture::files.at(target_path); + crypto.sha256({reinterpret_cast(installed.data()), installed.size()}, file_hash.data()); + if (store->finishDownloadInstall({request_key, 48}, transaction, 1, file_hash) != JournalWriteResult::InProgress) return 164; + if (!state.view().find(10, {task_id.data(), task_id.size()}, bytes) || !storage::decodeTask({task_id.data(), task_id.size()}, bytes, task) || + task.state != 1) return 165; + if (drainStore(*store, JournalWriteResult::InProgress) != JournalWriteResult::Verified) return 166; + storage::InstallRecordView install; + if (!state.view().find(10, {task_id.data(), task_id.size()}, bytes) || !storage::decodeTask({task_id.data(), task_id.size()}, bytes, task) || + task.state != 3 || !state.view().find(2, target.cache_id, bytes) || !storage::decodeCacheHead(target.cache_id, bytes, head) || + head.current_hash.size != 32 || std::memcmp(head.current_hash.data, summary.hash.bytes.data(), 32) || + !state.view().find(12, {transaction.data(), transaction.size()}, bytes) || + !storage::decodeInstallRecord({transaction.data(), transaction.size()}, bytes, install) || install.phase != storage::InstallPhase::Installed) return 167; + const auto finished_sequence = store->committedSequence(); + if (store->finishDownloadInstall({request_key, 48}, transaction, 1, file_hash) != JournalWriteResult::Verified || + store->committedSequence() != finished_sequence) return 168; + id.bytes.fill(5); + task_id[0] = 1; + if (!protocol::encodeGetRequest(id, summary.id, &summary.hash, nullptr, 8192, request, sizeof(request), request_size)) return 151; + if (store->persistNewTask({}, {}, id, task_id, 2, {request, request_size}, {}, target) != JournalWriteResult::StateRejected) return 152; + target.install_generation = 2; + if (drainStore(*store, store->persistNewTask({}, {}, id, task_id, 2, {request, request_size}, {}, target)) != JournalWriteResult::Verified) return 153; + id.bytes.fill(4); + if (store->recordDownloadResponse({}, {}, id, 1, {reply.data(), reply.size()}, crypto) != JournalWriteResult::StateRejected) return 154; + fixture::files.clear(); + return 0; +} + +struct FileDigest +{ + std::vector bytes; + void update(const uint8_t* data, size_t size) { bytes.insert(bytes.end(), data, data + size); } + bool finalize(uint8_t* out, size_t) + { + chat::reticulum::fullHash(bytes.data(), bytes.size(), out); + return true; + } +}; +int checkDownloadController(const char* query_path, const char* response_path) +{ + using namespace ::geocaching; + std::ifstream q(query_path, std::ios::binary), r(response_path, std::ios::binary); + std::vector query((std::istreambuf_iterator(q)), {}), response((std::istreambuf_iterator(r)), {}); + RequestId id; + id.bytes.fill(3); + protocol::SummaryView summary; + protocol::QueryPageView page; + if (!protocol::decodeQueryResponse({query.data(), query.size()}, id, 2048, &summary, 1, page)) return 170; + id.bytes.fill(4); + std::string target = "/trailmate/geocaching/caches/"; + constexpr char hex[] = "0123456789abcdef"; + for (auto byte : summary.id.bytes) + { + target += hex[byte >> 4]; + target += hex[byte & 15]; + } + target += ".gpx"; + for (unsigned scenario = 0; scenario < 5; ++scenario) + { + fixture::files.clear(); + if (scenario == 2) fixture::files[target] = "user-owned file"; + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + auto store = std::make_unique(fixture::volume, 0, state); + NativeRecordCrypto crypto; + std::array task; + task.fill(0x42); + auto checkRecovery = [&](const std::array& task, const RequestId& id, uint64_t generation) -> int + { + // Power cut at every quiescent storage boundary. Rebuild all + // logical state from journal bytes, not from the live arena. + const auto disk = fixture::files; + std::array recovered_a{}, recovered_b{}; + storage::LogicalState recovered(recovered_a.data(), recovered_b.data(), recovered_a.size()); + uint64_t sequence = 0; + for (const auto& file : disk) + { + if (file.first.rfind("/trailmate/geocaching/.state/journal/", 0) != 0) continue; + storage::RecordFrameView frame; + storage::TransactionView transaction; + storage::MutationView mutations[3]; + if (!storage::decodeRecordFrame({reinterpret_cast(file.second.data()), file.second.size()}, frame) || + frame.sequence != sequence + 1 || !storage::decodeTransaction(frame.payload, sequence, mutations, 3, transaction) || + !recovered.apply(transaction.mutations, transaction.count, [](const auto& candidate) + { return storage::validateTaskReferences(candidate) && storage::validateAttemptReferences(candidate); })) return 190; + sequence = frame.sequence; + } + ByteView task_bytes; + storage::TaskView recovered_task; + if (!recovered.view().find(10, {task.data(), task.size()}, task_bytes) || + !storage::decodeTask({task.data(), task.size()}, task_bytes, recovered_task)) return 191; + if (recovered_task.state != 3) + { + SdRequestStore recovered_store(fixture::volume, sequence, recovered); + SdDownloadPort recovered_port(recovered_store, recovered, crypto, {}, {summary.id, summary.hash, generation}, task, {}); + auto result = recovered_port.resume({}, id); + for (unsigned attempt = 0; result == DownloadOperationResult::Pending && attempt < 512; ++attempt) + { + fixture::step_data_bytes = 0; + result = recovered_port.poll(); + if (fixture::step_data_bytes > 512) return 192; + } + if (result != DownloadOperationResult::Complete || !fixture::files.count(target) || + fixture::files.at(target).find("Test") == std::string::npos) return 193; + } + fixture::files = disk; + return 0; + }; + SdDownloadPort port(*store, state, crypto, {}, {summary.id, summary.hash, 1}, task, {}); + DownloadClient client(port, crypto); + if (!client.begin({}, id, summary, 1)) return 171; + for (unsigned i = 0; client.phase() == DownloadPhase::Submitting && i < 256; ++i) client.advance(); + if (client.phase() != DownloadPhase::Waiting) return 172; + if (scenario == 4) + { + const auto disk = fixture::files; + const auto sequence = store->committedSequence(); + SdDownloadPort resumed_port(*store, state, crypto, {}, {summary.id, summary.hash, 1}, task, {}); + DownloadClient resumed(resumed_port, crypto); + if (resumed_port.resumeWaiting({}, id) != DownloadOperationResult::Complete || + !resumed.resume({}, id, summary, 1) || resumed.phase() != DownloadPhase::Waiting || + store->committedSequence() != sequence || store->commitPending() || fixture::files != disk) return 194; + Destination wrong_source; + wrong_source.bytes.fill(0x99); + std::vector scratch(summary.signed_bytes + 64); + if (resumed.accept(wrong_source, {response.data(), response.size()}, scratch.data(), scratch.size()) || + resumed.phase() != DownloadPhase::Waiting || store->commitPending()) return 195; + if (resumed.accept({}, {response.data(), response.size()}, scratch.data(), scratch.size()) || + resumed.phase() != DownloadPhase::Installing) return 196; + for (unsigned i = 0; resumed.phase() == DownloadPhase::Installing && i < 512; ++i) resumed.advance(); + if (resumed.phase() != DownloadPhase::Stored || !fixture::files.count(target) || + fixture::files.at(target).find("Test") == std::string::npos) return 197; + continue; + } + if (scenario == 1) + { + if (!client.cancel()) return 173; + for (unsigned i = 0; client.phase() == DownloadPhase::Cancelling && i < 256; ++i) client.advance(); + if (client.phase() != DownloadPhase::Cancelled || fixture::files.count(target)) return 174; + continue; + } + bool accepted = false; + state.withScratch([&](uint8_t* bytes, size_t capacity) + { accepted = client.accept({}, {response.data(), response.size()}, bytes, capacity); }); + if (accepted || client.phase() != DownloadPhase::Installing) return 175; + for (unsigned i = 0; client.phase() == DownloadPhase::Installing && i < 512; ++i) + { + fixture::step_data_bytes = 0; + client.advance(); + if (fixture::step_data_bytes > 512) return 176; + if (scenario == 0 && client.phase() == DownloadPhase::Installing && !store->commitPending()) + { + if (const int result = checkRecovery(task, id, 1)) return result; + } + } + if (scenario == 2) + { + if (client.phase() != DownloadPhase::Failed || fixture::files.at(target) != "user-owned file") return 177; + } + else if (client.phase() != DownloadPhase::Stored || fixture::files.at(target).find("Test") == std::string::npos) return 178; + if (scenario == 0) + { + SavedCacheCatalog saved(state, crypto); + ::ui::geocaching::Snapshot snapshot; + saved.snapshot(snapshot); + if (snapshot.count) return 184; + for (unsigned i = 0; saved.pending() && i < 256; ++i) saved.advance(); + saved.snapshot(snapshot); + ::ui::geocaching::Item item; + fixture::step_io_calls = 0; + if (snapshot.count != 1 || !saved.item(0, snapshot.generation, item) || !item.downloaded || + std::strcmp(item.name.data(), "Test") || fixture::step_io_calls) return 185; + const auto valid_file = fixture::files[target]; + fixture::files[target][0] ^= 1; + saved.reset(); + for (unsigned i = 0; saved.pending() && i < 256; ++i) saved.advance(); + saved.snapshot(snapshot); + if (snapshot.count || !std::strstr(snapshot.status.data(), "changed")) return 186; + fixture::files[target] = valid_file; + std::array restored_a{}, restored_b{}; + storage::LogicalState restored(restored_a.data(), restored_b.data(), restored_a.size()); + uint64_t replayed = 0; + for (const auto& file : fixture::files) + { + if (file.first.rfind("/trailmate/geocaching/.state/journal/", 0) != 0) continue; + storage::RecordFrameView frame; + storage::TransactionView transaction; + storage::MutationView mutations[3]; + if (!storage::decodeRecordFrame({reinterpret_cast(file.second.data()), file.second.size()}, frame) || + frame.sequence != replayed + 1 || !storage::decodeTransaction(frame.payload, replayed, mutations, 3, transaction) || + !restored.apply(transaction.mutations, transaction.count, [](const auto& candidate) + { return storage::validateTaskReferences(candidate) && storage::validateAttemptReferences(candidate); })) return 188; + replayed = frame.sequence; + } + if (replayed != store->committedSequence()) return 189; + SavedCacheCatalog reopened(restored, crypto); + for (unsigned i = 0; reopened.pending() && i < 256; ++i) reopened.advance(); + reopened.snapshot(snapshot); + if (snapshot.count != 1) return 187; + } + if (scenario == 0 || scenario == 3) + { + if (scenario == 3) fixture::files[target] = "external edit"; + RequestId next_id; + next_id.bytes.fill(5); + auto next_response = response; + protocol::CmpReader envelope({next_response.data(), next_response.size()}); + size_t fields = 0; + uint64_t value = 0; + ByteView encoded_id; + if (!envelope.array(fields, 6) || !envelope.unsignedInteger(value) || !envelope.unsignedInteger(value) || + !envelope.unsignedInteger(value) || !envelope.binary(encoded_id, 16)) return 179; + std::memcpy(next_response.data() + (encoded_id.data - next_response.data()), next_id.bytes.data(), 16); + std::array next_task; + next_task.fill(0x43); + SdDownloadPort update_port(*store, state, crypto, {}, {summary.id, summary.hash, 2}, next_task, {}); + DownloadClient update(update_port, crypto); + if (!update.begin({}, next_id, summary, 2)) return 180; + for (unsigned i = 0; update.phase() == DownloadPhase::Submitting && i < 256; ++i) update.advance(); + std::vector scratch(summary.signed_bytes + 64); + if (update.accept({}, {next_response.data(), next_response.size()}, scratch.data(), scratch.size()) || + update.phase() != DownloadPhase::Installing) return 181; + for (unsigned i = 0; update.phase() == DownloadPhase::Installing && i < 512; ++i) + { + update.advance(); + if (scenario == 0 && update.phase() == DownloadPhase::Installing && !store->commitPending()) + if (const int result = checkRecovery(next_task, next_id, 2)) return result; + } + if (scenario == 3) + { + if (update.phase() != DownloadPhase::Failed || fixture::files.at(target) != "external edit") return 182; + } + else + { + std::string history = "/trailmate/geocaching/.state/history/"; + for (auto byte : summary.hash.bytes) + { + history += hex[byte >> 4]; + history += hex[byte & 15]; + } + history += ".gpx"; + if (update.phase() != DownloadPhase::Stored || update_port.historyPending() || !fixture::files.count(history)) return 183; + } + } + } + fixture::files.clear(); + return 0; +} + +int checkDraftPersistence() +{ + using namespace ::geocaching; + fixture::files.clear(); + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + SdRequestStore store(fixture::volume, 0, state); + std::array id{}; + const ByteView key{id.data(), id.size()}; + storage::DraftView draft; + uint8_t encoded[256]; + size_t size = 0; + ByteView stored; + if (!storage::encodeDraft(key, draft, encoded, sizeof(encoded), size) || + store.saveDraft(key, {encoded, size}, 0) != JournalWriteResult::InProgress || + state.view().find(4, key, stored)) return 198; + std::memset(encoded, 0xa5, sizeof(encoded)); // candidate owns submitted bytes + if (drainStore(store, JournalWriteResult::InProgress) != JournalWriteResult::Verified || + !state.view().find(4, key, stored) || !storage::decodeDraft(key, stored, draft) || + draft.generation != 1 || draft.has_coordinates || !draft.name.empty()) return 199; + draft.name = "New cache"; + draft.has_coordinates = true; + draft.latitude_e7 = 305000000; + draft.longitude_e7 = 1205000000; + draft.generation = 2; + if (!storage::encodeDraft(key, draft, encoded, sizeof(encoded), size) || + drainStore(store, store.saveDraft(key, {encoded, size}, 1)) != JournalWriteResult::Verified) return 200; + if (store.saveDraft(key, {encoded, size}, 1) != JournalWriteResult::StateRejected || store.commitPending()) return 201; + std::array author{}, other_author{}; + other_author[0] = 1; + if (drainStore(store, store.bindDraftAuthor(key, 2, {author.data(), author.size()}, encoded, sizeof(encoded))) != JournalWriteResult::Verified || + !state.view().find(4, key, stored) || !storage::decodeDraft(key, stored, draft) || draft.generation != 3 || draft.author.size != 64) return 219; + const auto sequence = store.committedSequence(); + if (store.bindDraftAuthor(key, 3, {author.data(), author.size()}, encoded, sizeof(encoded)) != JournalWriteResult::Verified || + store.committedSequence() != sequence || store.commitPending() || + store.bindDraftAuthor(key, 3, {other_author.data(), other_author.size()}, encoded, sizeof(encoded)) != JournalWriteResult::StateRejected || + store.bindDraftAuthor(key, 2, {author.data(), author.size()}, encoded, sizeof(encoded)) != JournalWriteResult::StateRejected) return 220; + draft.author = {other_author.data(), other_author.size()}; + draft.generation = 4; + if (!storage::encodeDraft(key, draft, encoded, sizeof(encoded), size) || + store.saveDraft(key, {encoded, size}, 3) != JournalWriteResult::StateRejected) return 221; + draft.author = {author.data(), author.size()}; + draft.name = "Must not replace saved text"; + draft.generation = 4; + if (!storage::encodeDraft(key, draft, encoded, sizeof(encoded), size)) return 202; + fixture::flush_ok = false; + const auto failed = drainStore(store, store.saveDraft(key, {encoded, size}, 3)); + fixture::flush_ok = true; + if (failed == JournalWriteResult::Verified || !state.view().find(4, key, stored) || + !storage::decodeDraft(key, stored, draft) || draft.generation != 3 || draft.name != "New cache") return 203; + fixture::files.clear(); + return 0; +} + +int checkAuthorPort(const char* path) +{ + using namespace ::geocaching; + std::ifstream file(path, std::ios::binary); + const std::vector signed_bytes((std::istreambuf_iterator(file)), {}); + protocol::CmpReader reader({signed_bytes.data(), signed_bytes.size()}); + ByteView encoded, signature; + size_t count = 0; + RecordView record; + if (!reader.array(count, 2) || count != 2 || !reader.binary(encoded, 4096) || + !reader.binary(signature, 64) || !protocol::decodeGeocacheRecord(encoded, record)) return 204; + struct Transport + { + ByteView author, signed_record; + SdRequestStore* store; + unsigned signatures = 0; + bool getGeocachingAuthorKey(uint8_t out[64]) + { + std::memcpy(out, author.data, 64); + return true; + } + bool signGeocachingRecord(ByteView, uint8_t*, size_t, uint8_t* out, size_t capacity, size_t& written) + { + ++signatures; + if (store->committedSequence() != 1 || store->commitPending() || capacity < signed_record.size) return false; + std::memcpy(out, signed_record.data, signed_record.size); + written = signed_record.size; + return true; + } + }; + for (unsigned scenario = 0; scenario < 3; ++scenario) + { + fixture::files.clear(); + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + SdRequestStore store(fixture::volume, 0, state); + NativeRecordCrypto crypto; + Transport transport{record.author_public_key, {signed_bytes.data(), signed_bytes.size()}, &store}; + SdAuthorIssuePort port(transport, store, crypto, {}); + AuthorIssue issue(port); + std::vector scratch(encoded.size + 70); + if (!issue.begin(encoded, scratch.data(), scratch.size())) return 205; + issue.advance(); + issue.advance(); + if (issue.phase() != AuthorIssuePhase::Reserve || transport.signatures || !store.commitPending()) return 206; + if (scenario == 2) + { + issue.cancel(); + if (issue.phase() != AuthorIssuePhase::Cancelled || transport.signatures || store.commitPending()) return 207; + continue; + } + fixture::flush_ok = scenario == 0; + for (unsigned step = 0; step < 512 && issue.phase() != AuthorIssuePhase::Signed && issue.phase() != AuthorIssuePhase::Failed; ++step) + { + fixture::step_data_bytes = 0; + issue.advance(); + if (fixture::step_data_bytes > 512) return 208; + } + fixture::flush_ok = true; + if (scenario == 0 ? issue.phase() != AuthorIssuePhase::Signed || transport.signatures != 1 + : issue.phase() != AuthorIssuePhase::Failed || transport.signatures != 0) return 209; + } + fixture::files.clear(); + return 0; +} + +int checkPublishPort(const char* record_path, const char* response_path) +{ + using namespace ::geocaching; + std::ifstream a(record_path, std::ios::binary), b(response_path, std::ios::binary); + std::vector record((std::istreambuf_iterator(a)), {}), response((std::istreambuf_iterator(b)), {}); + std::vector workspace(record.size() + 26); + NativeRecordCrypto crypto; + protocol::VerifiedRecordView verified; + if (protocol::verifyGeocache({record.data(), record.size()}, crypto, workspace.data(), workspace.size(), verified) != + protocol::VerificationResult::Valid) return 210; + storage::DraftView draft; + draft.author = verified.record.author_public_key; + draft.has_coordinates = true; + draft.latitude_e7 = verified.record.latitude_e7; + draft.longitude_e7 = verified.record.longitude_e7; + draft.name = verified.record.name; + draft.description = verified.record.description; + draft.hint = verified.record.hint; + draft.state = static_cast(verified.record.state); + draft.difficulty_x2 = verified.record.difficulty_x2; + draft.terrain_x2 = verified.record.terrain_x2; + draft.container_size = static_cast(verified.record.container_size); + for (unsigned scenario = 0; scenario < 3; ++scenario) + { + fixture::files.clear(); + std::array first{}, second{}; + storage::LogicalState state(first.data(), second.data(), first.size()); + SdRequestStore store(fixture::volume, 0, state); + if (scenario == 2) + { + workspace.resize(record.size() + 256); + const auto key = verified.record.creation_nonce; + size_t size = 0; + if (!storage::encodeDraft(key, draft, workspace.data(), workspace.size(), size) || + drainStore(store, store.saveDraft(key, {workspace.data(), size}, 0)) != JournalWriteResult::Verified) return 229; + storage::StoredTime issued; + issued.has_utc = true; + issued.utc_seconds = verified.record.updated_at; + const auto reserved = store.reserveDraftUnsignedRecord(key, 1, verified.record.encoded, crypto, workspace.data(), workspace.size(), issued); + ByteView value; + storage::DraftView frozen; + if (reserved != JournalWriteResult::InProgress || !state.view().find(4, key, value) || + !storage::decodeDraft(key, value, frozen) || frozen.base_hash.size || frozen.generation != 1) return 230; + if (drainStore(store, reserved) != JournalWriteResult::Verified || !state.view().find(4, key, value) || + !storage::decodeDraft(key, value, frozen) || frozen.base_hash.size != 32 || frozen.generation != 2) return 231; + frozen.name = "Edited too early"; + frozen.generation = 3; + if (!storage::encodeDraft(key, frozen, workspace.data(), workspace.size(), size) || + store.saveDraft(key, {workspace.data(), size}, 2) != JournalWriteResult::StateRejected) return 232; + std::array task{}; + RequestId request; + request.bytes.fill(2); + SdPublishPort port(store, crypto, {}, verified.id, verified.hash, task, issued); + PublishAttempt attempt(port, crypto); + if (!attempt.begin({}, request, {record.data(), record.size()}, workspace.data(), workspace.size())) return 233; + for (unsigned i = 0; i < 512 && attempt.phase() == PublishAttemptPhase::Submitting; ++i) attempt.advance(); + if (attempt.phase() != PublishAttemptPhase::Waiting || !state.view().find(4, key, value) || + !storage::decodeDraft(key, value, frozen) || !storage::draftPublication(state.view(), key, frozen).base_retained) return 234; + frozen.name = "Editable after signed request retention"; + frozen.generation = 3; + if (!storage::encodeDraft(key, frozen, workspace.data(), workspace.size(), size) || + drainStore(store, store.saveDraft(key, {workspace.data(), size}, 2)) != JournalWriteResult::Verified) return 235; + RecordView successor = verified.record; + successor.revision = 2; + successor.name = "Editable after signed request retention"; + ++successor.updated_at; + issued.utc_seconds = successor.updated_at; + std::vector next_record(record.size() + 128); + std::array wrong_parent{}; + successor.previous_hash = {wrong_parent.data(), wrong_parent.size()}; + if (!protocol::encodeGeocacheRecord(successor, next_record.data(), next_record.size(), size) || + store.reserveDraftUnsignedRecord(key, 3, {next_record.data(), size}, crypto, workspace.data(), workspace.size(), issued) != JournalWriteResult::StateRejected) return 236; + successor.previous_hash = {verified.hash.bytes.data(), verified.hash.bytes.size()}; + ++successor.created_at; + if (!protocol::encodeGeocacheRecord(successor, next_record.data(), next_record.size(), size) || + store.reserveDraftUnsignedRecord(key, 3, {next_record.data(), size}, crypto, workspace.data(), workspace.size(), issued) != JournalWriteResult::StateRejected) return 237; + --successor.created_at; + if (!protocol::encodeGeocacheRecord(successor, next_record.data(), next_record.size(), size) || + drainStore(store, store.reserveDraftUnsignedRecord(key, 3, {next_record.data(), size}, crypto, workspace.data(), workspace.size(), issued)) != JournalWriteResult::Verified || + !state.view().find(4, key, value) || !storage::decodeDraft(key, value, frozen) || frozen.generation != 4 || + storage::draftPublication(state.view(), key, frozen).base_retained) return 238; + const auto sequence = store.committedSequence(); + if (store.reserveDraftUnsignedRecord(key, 4, {next_record.data(), size}, crypto, workspace.data(), workspace.size(), issued) != JournalWriteResult::Verified || + store.committedSequence() != sequence) return 239; + continue; + } + if (drainStore(store, store.reserveUnsignedRecord(verified.record.encoded, crypto, workspace.data(), workspace.size(), {})) != + JournalWriteResult::Verified) return 211; + std::array task{}; + RequestId request; + request.bytes.fill(2); + SdPublishPort port(store, crypto, {}, verified.id, verified.hash, task, {}); + PublishAttempt attempt(port, crypto); + if (!attempt.begin({}, request, {record.data(), record.size()}, workspace.data(), workspace.size())) return 212; + std::fill(workspace.begin(), workspace.end(), 0xa5); + if (scenario && !attempt.cancel()) return 213; + for (unsigned i = 0; i < 512 && (attempt.phase() == PublishAttemptPhase::Submitting || attempt.phase() == PublishAttemptPhase::Cancelling); ++i) + { + fixture::step_data_bytes = 0; + attempt.advance(); + if (fixture::step_data_bytes > 512) return 214; + } + if (scenario) + { + ByteView value; + storage::TaskView stopped; + if (attempt.phase() != PublishAttemptPhase::Cancelled || !state.view().find(10, {task.data(), task.size()}, value) || + !storage::decodeTask({task.data(), task.size()}, value, stopped) || stopped.state != 5 || stopped.continue_intent) return 215; + const auto history = storage::draftPublication(state.view(), verified.record.creation_nonce, draft); + if (history.latest_revision != 1 || history.confirmed_revision || !history.stopped || history.pending) return 225; + continue; + } + if (attempt.phase() != PublishAttemptPhase::Waiting) return 216; + const auto pending_history = storage::draftPublication(state.view(), verified.record.creation_nonce, draft); + if (pending_history.latest_revision != 1 || pending_history.confirmed_revision || !pending_history.pending || pending_history.local_changes) return 226; + chat::MeshAdapterRouter router; + router.send_ok = true; + RequestDispatcher dispatcher(router, store, 100, 1000); + bool sent = false; + for (unsigned i = 0; i < 512 && !sent; ++i) + sent = dispatcher.dispatchOne({}).status == DispatchStatus::Submitted; + if (!sent || router.sends != 1) return 217; + std::array request_key{}; + std::memcpy(request_key.data() + 32, request.bytes.data(), 16); + ByteView outgoing_bytes; + storage::OutgoingView outgoing; + if (!state.view().find(5, {request_key.data(), request_key.size()}, outgoing_bytes) || + !storage::decodeOutgoing({request_key.data(), request_key.size()}, outgoing_bytes, outgoing)) return 222; + const auto sequence = store.committedSequence(); + SdPublishPort restored_port(store, crypto, {}, verified.id, verified.hash, task, {}); + PublishAttempt restored(restored_port, crypto); + if (!restored_port.attachRestoredRequest({}, request) || + !restored.resume({}, request, outgoing.request, workspace.data(), workspace.size(), verified.id, verified.hash) || + store.committedSequence() != sequence || store.commitPending() || router.sends != 1) return 223; + if (restored.accept({}, {response.data(), response.size()}) || restored.phase() != PublishAttemptPhase::Committing) return 224; + for (unsigned i = 0; i < 512 && restored.phase() == PublishAttemptPhase::Committing; ++i) restored.advance(); + ByteView value; + storage::TaskView completed; + if (restored.phase() != PublishAttemptPhase::Confirmed || !state.view().find(10, {task.data(), task.size()}, value) || + !storage::decodeTask({task.data(), task.size()}, value, completed) || completed.state != 3) return 218; + const auto published_history = storage::draftPublication(state.view(), verified.record.creation_nonce, draft); + if (published_history.latest_revision != 1 || published_history.confirmed_revision != 1 || published_history.local_changes) return 227; + auto changed_draft = draft; + changed_draft.name = "Edited after publication"; + const auto edited_history = storage::draftPublication(state.view(), verified.record.creation_nonce, changed_draft); + if (!edited_history.local_changes || edited_history.confirmed_revision != 1) return 228; + } + fixture::files.clear(); + return 0; +} + +int main(int argc, char** argv) +{ + if (argc != 6) return 22; + if (const int result = checkPublishPort(argv[1], argv[2])) return result; + if (const int result = checkAuthorPort(argv[1])) return result; + if (const int result = checkDraftPersistence()) return result; + if (const int result = checkDownloadController(argv[4], argv[5])) return result; + if (const int result = checkDownloadReceipt(argv[4], argv[5])) return result; + if (const int result = checkBrowseFlow(argv[3], argv[4])) return result; + if (const int result = checkIncrementalBoundaries()) return result; + if (const int result = checkIncrementalDispatch()) return result; + using namespace platform::esp::arduino_common::geocaching; + const uint8_t key = 1; + ::geocaching::storage::MutationView mutation{5, {&key, 1}, {}, true}; + uint8_t payload[64]{}; + size_t size = 0; + if (!::geocaching::storage::encodeTransaction(0, &mutation, 1, payload, sizeof(payload), size)) return 1; + auto journal = std::make_unique(fixture::volume); + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::Verified) return 2; + const auto saved = fixture::files; + fixture::files.clear(); + if (drainJournal(*journal, journal->begin(0, &mutation, 1)) != JournalWriteResult::Verified || fixture::files != saved) return 80; + for (size_t offset : {size_t(0), size_t(24), size_t(24 + size - 1)}) + { + fixture::files.clear(); + fixture::corrupt_at = offset; + if (drainJournal(*journal, journal->begin(0, &mutation, 1)) != JournalWriteResult::IoError) return 81; + } + fixture::corrupt_at = SIZE_MAX; + fixture::files = saved; + const auto writes = fixture::writes; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::Exists || + fixture::writes != writes || fixture::files != saved) return 3; + fixture::files.clear(); + fixture::write_limit = 3; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::IoError || fixture::files.empty()) return 4; + fixture::files.clear(); + fixture::write_limit = SIZE_MAX; + fixture::flush_ok = false; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::IoError) return 5; + fixture::flush_ok = true; + for (size_t offset : {size_t(0), size_t(24)}) + { + fixture::files.clear(); + fixture::corrupt_at = offset; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::IoError) return 6; + } + fixture::files.clear(); + fixture::corrupt_at = SIZE_MAX; + fixture::busy = true; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::Unavailable || !fixture::files.empty()) return 7; + fixture::busy = false; + if (drainJournal(*journal, journal->begin(1, {payload, size})) != JournalWriteResult::Invalid || !fixture::files.empty()) return 8; + fixture::supported_volume = false; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::UnsupportedVolume || !fixture::files.empty()) return 9; + fixture::supported_volume = true; + fixture::volume[0] = 1; + if (drainJournal(*journal, journal->begin(0, {payload, size})) != JournalWriteResult::VolumeChanged || !fixture::files.empty()) return 10; + uint8_t state_a[1024]{}, state_b[1024]{}; + ::geocaching::storage::LogicalState live(state_a, state_b, sizeof(state_a)); + auto store = std::make_unique(fixture::volume, 0, live); + ::geocaching::RequestId request_id; + uint8_t request[128]{}; + size_t request_size = 0; + if (!::geocaching::protocol::encodeCapabilitiesRequest(request_id, request, sizeof(request), request_size)) return 11; + if (drainStore(*store, store->persistNewTask({}, {}, request_id, {}, 3, {request, request_size}, {})) != JournalWriteResult::Verified || + store->committedSequence() != 1 || store->needsRecovery() || live.view().size() != 2) return 12; + const auto writes_after_commit = fixture::writes; + if (drainStore(*store, store->persistNewTask({}, {}, request_id, {}, 3, {request, request_size}, {})) != JournalWriteResult::StateRejected || + fixture::writes != writes_after_commit || store->needsRecovery()) return 17; + const auto& committed = fixture::files.at("/trailmate/geocaching/.state/journal/0000000000000001.gcj"); + ::geocaching::storage::RecordFrameView frame; + ::geocaching::storage::TransactionView transaction; + ::geocaching::storage::MutationView entries[2]; + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::storage::TaskView task; + if (!::geocaching::storage::decodeRecordFrame({reinterpret_cast(committed.data()), committed.size()}, frame) || + frame.sequence != 1 || !::geocaching::storage::decodeTransaction(frame.payload, 0, entries, 2, transaction) || + !::geocaching::storage::decodeOutgoing(entries[0].key, entries[0].value, outgoing) || + !::geocaching::storage::decodeTask(entries[1].key, entries[1].value, task) || + !::geocaching::storage::requestBelongsToTask(entries[1].key, task, entries[0].key, outgoing)) return 13; + fixture::flush_ok = false; + std::array next_task{}; + next_task[0] = 1; + request_id.bytes[0] = 1; + if (!::geocaching::protocol::encodeCapabilitiesRequest(request_id, request, sizeof(request), request_size)) return 14; + if (drainStore(*store, store->persistNewTask({}, {}, request_id, next_task, 3, {request, request_size}, {})) != JournalWriteResult::IoError || + store->committedSequence() != 1 || !store->needsRecovery() || live.view().size() != 2) return 15; + fixture::flush_ok = true; + const auto writes_after_failure = fixture::writes; + if (drainStore(*store, store->persistNewTask({}, {}, request_id, {}, 3, {request, request_size}, {})) != JournalWriteResult::Unavailable || + fixture::writes != writes_after_failure || store->committedSequence() != 1) return 16; + fixture::files.clear(); + uint8_t author_a[512]{}, author_b[512]{}, author_key[64]{}; + ::geocaching::storage::LogicalState author_state(author_a, author_b, sizeof(author_a)); + auto author_store = std::make_unique(fixture::volume, 0, author_state); + ::geocaching::GeocacheId cache_id; + ::geocaching::RevisionHash hash; + if (drainStore(*author_store, author_store->reserveAuthorVersion(cache_id, 1, hash, {author_key, 64}, {})) != JournalWriteResult::Verified || + author_store->committedSequence() != 1 || author_state.view().size() != 1) return 18; + const auto reservation_writes = fixture::writes; + if (drainStore(*author_store, author_store->reserveAuthorVersion(cache_id, 1, hash, {author_key, 64}, {})) != JournalWriteResult::Verified || + fixture::writes != reservation_writes || author_store->committedSequence() != 1) return 19; + hash.bytes[0] = 1; + if (drainStore(*author_store, author_store->reserveAuthorVersion(cache_id, 1, hash, {author_key, 64}, {})) != JournalWriteResult::StateRejected || + fixture::writes != reservation_writes || author_state.view().size() != 1) return 20; + if (drainStore(*author_store, author_store->reserveAuthorVersion(cache_id, 2, hash, {author_key, 64}, {})) != JournalWriteResult::Verified || + author_store->committedSequence() != 2 || author_state.view().size() != 2) return 21; + std::ifstream vector_file(argv[1], std::ios::binary); + std::vector vector_bytes((std::istreambuf_iterator(vector_file)), {}); + ::geocaching::protocol::CmpReader vector_reader({vector_bytes.data(), vector_bytes.size()}); + size_t fields = 0; + ::geocaching::ByteView encoded; + if (!vector_reader.array(fields, 2) || !vector_reader.binary(encoded, 4096)) return 23; + fixture::files.clear(); + uint8_t unsigned_a[512]{}, unsigned_b[512]{}; + ::geocaching::storage::LogicalState unsigned_state(unsigned_a, unsigned_b, sizeof(unsigned_a)); + auto unsigned_store = std::make_unique(fixture::volume, 0, unsigned_state); + NativeRecordCrypto crypto; + std::array crypto_workspace{}; + if (drainStore(*unsigned_store, unsigned_store->reserveUnsignedRecord(encoded, crypto, crypto_workspace.data(), crypto_workspace.size(), {})) != JournalWriteResult::Verified) return 24; + ::geocaching::GeocacheId derived_id; + ::geocaching::RevisionHash derived_hash; + if (::geocaching::protocol::deriveGeocacheHashes(encoded, crypto, crypto_workspace.data(), crypto_workspace.size(), derived_id, derived_hash) != + ::geocaching::protocol::VerificationResult::Valid) return 25; + uint8_t issued_key[36]{}; + std::memcpy(issued_key, derived_id.bytes.data(), 32); + issued_key[35] = 1; + ::geocaching::ByteView issued_value; + ::geocaching::storage::AuthorIssuedView issued; + if (!unsigned_state.view().find(3, {issued_key, sizeof(issued_key)}, issued_value) || + !::geocaching::storage::decodeAuthorIssued({issued_key, sizeof(issued_key)}, issued_value, issued) || + std::memcmp(issued.revision_hash.data, derived_hash.bytes.data(), 32)) return 26; + ::geocaching::RecordView changed; + if (!::geocaching::protocol::decodeGeocacheRecord(encoded, changed)) return 27; + changed.name = "Changed"; + std::array changed_bytes{}; + size_t changed_size = 0; + if (!::geocaching::protocol::encodeGeocacheRecord(changed, changed_bytes.data(), changed_bytes.size(), changed_size)) return 28; + const auto before_conflict = fixture::writes; + if (drainStore(*unsigned_store, unsigned_store->reserveUnsignedRecord({changed_bytes.data(), changed_size}, crypto, crypto_workspace.data(), crypto_workspace.size(), {})) != + JournalWriteResult::StateRejected || + unsigned_store->committedSequence() != 1 || fixture::writes != before_conflict) return 29; + ::geocaching::ByteView record_signature; + if (!vector_reader.binary(record_signature, 64)) return 30; + std::ifstream response_file(argv[2], std::ios::binary); + std::vector response((std::istreambuf_iterator(response_file)), {}); + fixture::files.clear(); + uint8_t publish_a[2048]{}, publish_b[2048]{}, publish_request[512]{}; + ::geocaching::storage::LogicalState publish_state(publish_a, publish_b, sizeof(publish_a)); + auto publish_store = std::make_unique(fixture::volume, 0, publish_state); + ::geocaching::RequestId publish_id; + publish_id.bytes.fill(2); + size_t publish_size = 0; + if (!::geocaching::protocol::encodePublishRequest(publish_id, encoded, record_signature, 512, publish_request, sizeof(publish_request), publish_size)) return 31; + ::geocaching::storage::RequestTaskTarget publish_target{{derived_id.bytes.data(), 32}, {derived_hash.bytes.data(), 32}, 0}; + if (drainStore(*publish_store, publish_store->persistNewTask({}, {}, publish_id, {}, 1, {publish_request, publish_size}, {}, publish_target)) != JournalWriteResult::Verified) return 32; + auto wrong_response = response; + wrong_response.back() ^= 1; + if (drainStore(*publish_store, publish_store->commitPublishResult({}, {}, publish_id, {wrong_response.data(), wrong_response.size()}, crypto)) != JournalWriteResult::Invalid || + publish_store->committedSequence() != 1) return 33; + if (drainStore(*publish_store, publish_store->commitPublishResult({}, {}, publish_id, {response.data(), response.size()}, crypto)) != JournalWriteResult::Verified || + publish_store->committedSequence() != 2) return 34; + uint8_t task_key[16]{}; + ::geocaching::ByteView task_bytes; + ::geocaching::storage::TaskView completed_task; + if (!publish_state.view().find(10, {task_key, 16}, task_bytes) || !::geocaching::storage::decodeTask({task_key, 16}, task_bytes, completed_task) || + completed_task.state != 3) return 35; + const auto committed_writes = fixture::writes; + if (drainStore(*publish_store, publish_store->commitPublishResult({}, {}, publish_id, {response.data(), response.size()}, crypto)) != JournalWriteResult::Verified || + publish_store->committedSequence() != 2 || fixture::writes != committed_writes) return 36; + fixture::files.clear(); + uint8_t stopped_a[2048]{}, stopped_b[2048]{}; + ::geocaching::storage::LogicalState stopped_state(stopped_a, stopped_b, sizeof(stopped_a)); + auto stopped_store = std::make_unique(fixture::volume, 0, stopped_state); + if (drainStore(*stopped_store, stopped_store->persistNewTask({}, {}, publish_id, {}, 1, {publish_request, publish_size}, {}, publish_target)) != JournalWriteResult::Verified || + drainStore(*stopped_store, stopped_store->stopTask({})) != JournalWriteResult::Verified || stopped_store->committedSequence() != 2) return 37; + if (!stopped_state.view().find(10, {task_key, 16}, task_bytes) || !::geocaching::storage::decodeTask({task_key, 16}, task_bytes, completed_task) || + completed_task.state != 5 || completed_task.continue_intent) return 38; + const auto stop_writes = fixture::writes; + if (drainStore(*stopped_store, stopped_store->stopTask({})) != JournalWriteResult::Verified || fixture::writes != stop_writes || + drainStore(*stopped_store, stopped_store->commitPublishResult({}, {}, publish_id, {response.data(), response.size()}, crypto)) != JournalWriteResult::StateRejected || + stopped_store->committedSequence() != 2 || fixture::writes != stop_writes) return 39; + fixture::files.clear(); + uint8_t attempt_a[2048]{}, attempt_b[2048]{}, request_key[48]{}; + std::memcpy(request_key + 32, publish_id.bytes.data(), 16); + ::geocaching::storage::LogicalState attempt_state(attempt_a, attempt_b, sizeof(attempt_a)); + auto attempt_store = std::make_unique(fixture::volume, 0, attempt_state); + if (drainStore(*attempt_store, attempt_store->persistNewTask({}, {}, publish_id, {}, 1, {publish_request, publish_size}, {}, publish_target)) != JournalWriteResult::Verified) return 40; + std::array attempt_id{}; + attempt_id[0] = 1; + if (drainStore(*attempt_store, attempt_store->beginAttempt({}, {request_key, 48}, attempt_id, {})) != JournalWriteResult::Verified || + attempt_store->committedSequence() != 2 || attempt_state.view().size() != 3) return 41; + ::geocaching::storage::PendingRequestView pending; + if (::geocaching::storage::nextPendingRequest(attempt_state.view(), {}, {}, pending) != ::geocaching::storage::PendingRequestResult::None) return 42; + const auto attempt_writes = fixture::writes; + if (drainStore(*attempt_store, attempt_store->beginAttempt({}, {request_key, 48}, attempt_id, {})) != JournalWriteResult::StateRejected || fixture::writes != attempt_writes) return 43; + uint8_t attempt_key[64]{}; + std::memcpy(attempt_key, request_key, 48); + std::memcpy(attempt_key + 48, attempt_id.data(), 16); + ::geocaching::ByteView attempt_value; + ::geocaching::storage::TxAttemptView attempt; + if (!attempt_state.view().find(13, {attempt_key, 64}, attempt_value) || + !::geocaching::storage::decodeTxAttempt({attempt_key, 64}, attempt_value, attempt) || + attempt.state != ::geocaching::storage::TxAttemptState::Accepted) return 44; + std::array lxmf_hash{}; + lxmf_hash[0] = 9; + if (drainStore(*attempt_store, attempt_store->recordAttemptHash({attempt_key, 64}, lxmf_hash)) != JournalWriteResult::Verified || + attempt_store->committedSequence() != 3) return 45; + if (!attempt_state.view().find(13, {attempt_key, 64}, attempt_value) || + !::geocaching::storage::decodeTxAttempt({attempt_key, 64}, attempt_value, attempt) || + attempt.state != ::geocaching::storage::TxAttemptState::InFlight || attempt.lxmf_hash.size != 32 || attempt.lxmf_hash.data[0] != 9) return 46; + const auto hash_writes = fixture::writes; + if (drainStore(*attempt_store, attempt_store->recordAttemptHash({attempt_key, 64}, lxmf_hash)) != JournalWriteResult::Verified || fixture::writes != hash_writes) return 47; + lxmf_hash[0] = 10; + if (drainStore(*attempt_store, attempt_store->recordAttemptHash({attempt_key, 64}, lxmf_hash)) != JournalWriteResult::StateRejected || + fixture::writes != hash_writes || attempt_store->committedSequence() != 3) return 48; + using AttemptState = ::geocaching::storage::TxAttemptState; + if (drainStore(*attempt_store, attempt_store->finishAttempt({attempt_key, 64}, AttemptState::Delivered, {})) != JournalWriteResult::Verified || + attempt_store->committedSequence() != 4) return 49; + ::geocaching::ByteView outgoing_value; + ::geocaching::storage::OutgoingView delivered_request; + if (!attempt_state.view().find(5, {request_key, 48}, outgoing_value) || + !::geocaching::storage::decodeOutgoing({request_key, 48}, outgoing_value, delivered_request) || delivered_request.state != 2) return 50; + const auto finish_writes = fixture::writes; + if (drainStore(*attempt_store, attempt_store->finishAttempt({attempt_key, 64}, AttemptState::Delivered, {})) != JournalWriteResult::Verified || + drainStore(*attempt_store, attempt_store->finishAttempt({attempt_key, 64}, AttemptState::Failed, {})) != JournalWriteResult::StateRejected || + fixture::writes != finish_writes) return 51; + fixture::files.clear(); + uint8_t failed_a[2048]{}, failed_b[2048]{}; + ::geocaching::storage::LogicalState failed_state(failed_a, failed_b, sizeof(failed_a)); + auto failed_store = std::make_unique(fixture::volume, 0, failed_state); + if (drainStore(*failed_store, failed_store->persistNewTask({}, {}, publish_id, {}, 1, {publish_request, publish_size}, {}, publish_target)) != JournalWriteResult::Verified || + drainStore(*failed_store, failed_store->beginAttempt({}, {request_key, 48}, attempt_id, {})) != JournalWriteResult::Verified || + drainStore(*failed_store, failed_store->finishAttempt({attempt_key, 64}, AttemptState::Failed, {})) != JournalWriteResult::Verified) return 52; + if (::geocaching::storage::nextPendingRequest(failed_state.view(), {}, {}, pending) != ::geocaching::storage::PendingRequestResult::Ready) return 53; + if (drainStore(*failed_store, failed_store->stopTask({})) != JournalWriteResult::Verified || + ::geocaching::storage::nextPendingRequest(failed_state.view(), {}, {}, pending) != ::geocaching::storage::PendingRequestResult::None) return 54; + fixture::files.clear(); + uint8_t timeout_a[2048]{}, timeout_b[2048]{}; + ::geocaching::storage::LogicalState timeout_state(timeout_a, timeout_b, sizeof(timeout_a)); + auto timeout_store = std::make_unique(fixture::volume, 0, timeout_state); + ::geocaching::storage::StoredTime submitted, later; + submitted.monotonic_ms = 100; + later.monotonic_ms = 1099; + if (drainStore(*timeout_store, timeout_store->persistNewTask({}, {}, publish_id, {}, 1, {publish_request, publish_size}, {}, publish_target)) != JournalWriteResult::Verified || + drainStore(*timeout_store, timeout_store->beginAttempt({}, {request_key, 48}, attempt_id, submitted)) != JournalWriteResult::Verified) return 55; + bool expired = false; + if (drainStore(*timeout_store, timeout_store->expireOneAttempt(later, 0, 1000, expired)) != JournalWriteResult::Verified || expired || timeout_store->committedSequence() != 2) return 56; + later.monotonic_ms = 1100; + if (drainStore(*timeout_store, timeout_store->expireOneAttempt(later, 0, 1000, expired)) != JournalWriteResult::Verified || expired || timeout_store->committedSequence() != 3 || + ::geocaching::storage::nextPendingRequest(timeout_state.view(), {}, {}, pending) != ::geocaching::storage::PendingRequestResult::Ready) return 57; + if (drainStore(*timeout_store, timeout_store->expireOneAttempt(later, 0, 1000, expired)) != JournalWriteResult::Verified || expired || timeout_store->committedSequence() != 3) return 58; + fixture::files.clear(); + uint8_t caps_a[1024]{}, caps_b[1024]{}, caps_request[128]{}; + ::geocaching::storage::LogicalState caps_state(caps_a, caps_b, sizeof(caps_a)); + auto caps_store = std::make_unique(fixture::volume, 0, caps_state); + ::geocaching::RequestId caps_id; + caps_id.bytes.fill(1); + size_t caps_size = 0; + if (!::geocaching::protocol::encodeCapabilitiesRequest(caps_id, caps_request, sizeof(caps_request), caps_size) || + drainStore(*caps_store, caps_store->persistNewTask({}, {}, caps_id, {}, 3, {caps_request, caps_size}, {})) != JournalWriteResult::Verified) return 59; + std::ifstream caps_file(argv[3], std::ios::binary); + std::vector caps_response((std::istreambuf_iterator(caps_file)), {}); + ::geocaching::Destination wrong_remote; + wrong_remote.bytes[0] = 9; + if (drainStore(*caps_store, caps_store->commitDirectoryCapabilities({}, wrong_remote, caps_id, {caps_response.data(), caps_response.size()})) != JournalWriteResult::StateRejected) return 60; + if (drainStore(*caps_store, caps_store->commitDirectoryCapabilities({}, {}, caps_id, {caps_response.data(), caps_response.size()})) != JournalWriteResult::Verified || + caps_store->committedSequence() != 2) return 61; + const auto caps_writes = fixture::writes; + if (drainStore(*caps_store, caps_store->commitDirectoryCapabilities({}, {}, caps_id, {caps_response.data(), caps_response.size()})) != JournalWriteResult::Verified || + fixture::writes != caps_writes) return 62; + return 0; +} diff --git a/modules/core_geocaching/tests/test_sd_stage.cpp b/modules/core_geocaching/tests/test_sd_stage.cpp new file mode 100644 index 00000000..42a7dbef --- /dev/null +++ b/modules/core_geocaching/tests/test_sd_stage.cpp @@ -0,0 +1,174 @@ +#include "platform/esp/arduino_common/geocaching/sd_gpx_stage.h" +#include +#include +#include +#include +#include +#include + +namespace fixture +{ +std::map files; +bool ready = true, external = false, flush_ok = true; +bool corrupt_read = false; +size_t write_limit = SIZE_MAX; +unsigned opens = 0, calls = 0; +size_t transferred = 0; +} // namespace fixture +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeFile::Impl +{ + public: + std::string path; + bool open = false; + size_t offset = 0; +}; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char* path, const char* mode) +{ + ++fixture::opens; + ++fixture::calls; + impl_->path = path; + impl_->open = true; + impl_->offset = 0; + if (*mode == 'w') fixture::files[path].clear(); + return fixture::files.count(path) != 0; +} +void SdRuntimeFile::close() +{ + ++fixture::calls; + impl_->open = false; +} +bool SdRuntimeFile::is_open() const { return impl_->open; } +std::size_t SdRuntimeFile::write(const void* data, std::size_t size) +{ + ++fixture::calls; + fixture::transferred += size; + const auto n = std::min(size, fixture::write_limit); + fixture::files[impl_->path].append(static_cast(data), n); + return n; +} +bool SdRuntimeFile::flush() +{ + ++fixture::calls; + return fixture::flush_ok; +} +int SdRuntimeFile::read(void* data, size_t size) +{ + ++fixture::calls; + fixture::transferred += size; + const auto& bytes = fixture::files[impl_->path]; + const auto count = std::min(size, bytes.size() - impl_->offset); + std::memcpy(data, bytes.data() + impl_->offset, count); + impl_->offset += count; + if (fixture::corrupt_read && count) static_cast(data)[0] ^= 1; + return static_cast(count); +} +uint64_t SdRuntimeFile::size() const +{ + ++fixture::calls; + return fixture::files[impl_->path].size(); +} +bool sd_card_ready() { return fixture::ready; } +bool sd_external_block_owner_active() { return fixture::external; } +bool sd_is_directory(const char*) +{ + ++fixture::calls; + return true; +} +bool sd_exists(const char* path) +{ + ++fixture::calls; + return fixture::files.count(path) != 0; +} +} // namespace platform::esp::arduino_common::storage +struct WriterCrypto final : geocaching::protocol::RecordCrypto +{ + bool sha256(geocaching::ByteView, uint8_t out[32]) override + { + std::memset(out, 0, 32); + return true; + } + geocaching::protocol::VerificationResult verifyEd25519(geocaching::ByteView, geocaching::ByteView, geocaching::ByteView) override + { + return geocaching::protocol::VerificationResult::CryptoUnavailable; + } +}; +using namespace platform::esp::arduino_common::geocaching; +// Only the test driver loops; production code returns to the owner per step. +StageResult finish(SdGpxStage& stage, StageResult result) +{ + for (unsigned i = 0; result == StageResult::InProgress && i < 256; ++i) + { + fixture::calls = 0; + fixture::transferred = 0; + result = stage.step(); + if (fixture::calls > 1 || fixture::transferred > 512) return StageResult::InvalidTransaction; + } + return result; +} +struct StringSink final : gps::gpx::OutputSink +{ + std::string value; + bool write(std::string_view bytes) override + { + value.append(bytes.data(), bytes.size()); + return true; + } +}; +int main(int argc, char** argv) +{ + using namespace platform::esp::arduino_common::geocaching; + if (argc != 2) return 1; + std::ifstream input(argv[1], std::ios::binary); + std::vector data((std::istreambuf_iterator(input)), {}); + geocaching::protocol::CmpReader reader({data.data(), data.size()}); + geocaching::protocol::VerifiedRecordView record; + geocaching::ByteView encoded; + size_t count = 0; + if (!reader.array(count, 2) || !reader.binary(encoded, 4096) || !reader.binary(record.signature, 64) || + !geocaching::protocol::decodeGeocacheRecord(encoded, record.record)) return 2; + // Storage fault tests only; signature verification is covered by crypto_pipeline. + WriterCrypto crypto; + std::array transaction{}; + transaction[0] = 1; + auto stage = std::make_unique(); + if (finish(*stage, stage->begin(transaction, record, crypto)) != StageResult::Written) return 3; + const auto original = fixture::files.at(stage->path()); + StringSink expected; + if (!geocaching::gpx::writeGeocacheGpx(record, crypto, expected) || original != expected.value) return 10; + if (original.find("") == std::string::npos) return 4; + auto collision = std::make_unique(); + const auto opens = fixture::opens; + if (finish(*collision, collision->begin(transaction, record, crypto)) != StageResult::AlreadyExists || fixture::opens != opens || + fixture::files.at(stage->path()) != original) return 5; + transaction[0] = 2; + fixture::write_limit = 7; + auto short_write = std::make_unique(); + if (finish(*short_write, short_write->begin(transaction, record, crypto)) != StageResult::IoError) return 6; + fixture::write_limit = SIZE_MAX; + fixture::flush_ok = false; + transaction[0] = 3; + auto flush_failure = std::make_unique(); + if (finish(*flush_failure, flush_failure->begin(transaction, record, crypto)) != StageResult::IoError) return 7; + fixture::flush_ok = true; + fixture::external = true; + transaction[0] = 4; + auto busy = std::make_unique(); + if (finish(*busy, busy->begin(transaction, record, crypto)) != StageResult::Unavailable) return 8; + if (fixture::files.at(stage->path()) != original) return 9; + fixture::external = false; + transaction[0] = 5; + auto cancelled = std::make_unique(); + fixture::calls = 0; + if (cancelled->begin(transaction, record, crypto) != StageResult::InProgress || fixture::calls) return 11; + if (cancelled->cancel() != StageResult::Cancelled || cancelled->step() != StageResult::Cancelled || + fixture::files.count(cancelled->path())) return 12; + transaction[0] = 6; + fixture::corrupt_read = true; + auto corrupt = std::make_unique(); + if (finish(*corrupt, corrupt->begin(transaction, record, crypto)) != StageResult::IoError) return 13; + return 0; +} diff --git a/modules/core_geocaching/tests/test_sd_volume.cpp b/modules/core_geocaching/tests/test_sd_volume.cpp new file mode 100644 index 00000000..6a0a734a --- /dev/null +++ b/modules/core_geocaching/tests/test_sd_volume.cpp @@ -0,0 +1,49 @@ +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include +#include +#include +#include + +std::map files; +std::set directories; +unsigned writes = 0; +namespace platform::esp::arduino_common::storage +{ +class SdRuntimeFile::Impl { public: std::string path; }; +SdRuntimeFile::SdRuntimeFile() : impl_(new Impl) {} +SdRuntimeFile::~SdRuntimeFile() { delete impl_; } +bool SdRuntimeFile::open(const char* path, const char*) { impl_->path = path; files[path].clear(); return true; } +void SdRuntimeFile::close() {} +size_t SdRuntimeFile::write(const void* data, size_t count) +{ + ++writes; files[impl_->path].append(static_cast(data), count); return count; +} +bool SdRuntimeFile::flush() { return true; } +bool sd_card_ready() { return true; } +bool sd_external_block_owner_active() { return false; } +bool sd_exists(const char* path) { return directories.count(path) || files.count(path); } +bool sd_is_directory(const char* path) { return directories.count(path) != 0; } +bool sd_mkdir(const char* path) { directories.insert(path); return true; } +SdFileReadResult sd_read_file(const char* path, uint8_t* buffer, size_t capacity) +{ + SdFileReadResult result; + auto found = files.find(path); + if (found == files.end()) { result.status = SdFileReadStatus::Missing; return result; } + result.file_size = found->second.size(); result.bytes_read = std::min(capacity, found->second.size()); + std::memcpy(buffer, found->second.data(), result.bytes_read); + result.status = SdFileReadStatus::Ready; return result; +} +} +int main() +{ + using namespace platform::esp::arduino_common::geocaching; + ::geocaching::storage::VolumeInstance candidate{}, actual; + candidate[0] = 1; + if (createNewSdVolume(candidate, actual) != SdVolumeResult::Ready || actual != candidate || writes != 1) return 1; + const auto saved = files; + candidate[0] = 2; + if (createNewSdVolume(candidate, actual) != SdVolumeResult::Ready || actual[0] != 1 || writes != 1 || files != saved) return 2; + files.clear(); + if (createNewSdVolume(candidate, actual) != SdVolumeResult::Corrupt || actual != ::geocaching::storage::VolumeInstance{} || writes != 1) return 3; + return 0; +} diff --git a/modules/core_geocaching/tests/test_service_backend.cpp b/modules/core_geocaching/tests/test_service_backend.cpp new file mode 100644 index 00000000..1525fa79 --- /dev/null +++ b/modules/core_geocaching/tests/test_service_backend.cpp @@ -0,0 +1,66 @@ +#include "chat/infra/mesh_adapter_router_core.h" +#include + +class Backend final : public chat::IMeshAdapter +{ + public: + unsigned sends = 0; + bool sendText(chat::ChannelId, const std::string&, chat::MessageId*, chat::NodeId) override + { + ++sends; + return true; + } + bool pollIncomingText(chat::MeshIncomingText*) override { return false; } + bool sendAppData(chat::ChannelId, std::uint32_t, const std::uint8_t*, std::size_t, + chat::NodeId, bool, chat::MessageId, bool) override { return false; } + bool pollIncomingData(chat::MeshIncomingData*) override { return false; } + void applyConfig(const chat::MeshConfig&) override {} + bool isReady() const override { return true; } + bool pollIncomingRawPacket(std::uint8_t*, std::size_t&, std::size_t) override { return false; } +}; + +int main() +{ + chat::MeshAdapterRouterCore router; + auto active = std::make_unique(); + auto* active_ptr = active.get(); + assert(router.installBackend(chat::MeshProtocol::MeshCore, std::move(active))); + auto service = std::make_unique(); + auto* service_ptr = service.get(); + assert(router.installServiceBackend(chat::MeshProtocol::Reticulum, std::move(service))); + assert(router.backendProtocol() == chat::MeshProtocol::MeshCore); + assert(!router.takeServiceBackend(chat::MeshProtocol::Reticulum, active_ptr)); + assert(router.backendForProtocol(chat::MeshProtocol::Reticulum) == service_ptr); + router.setActiveProtocol(chat::MeshProtocol::RNode); + assert(!router.takeServiceBackend(chat::MeshProtocol::Reticulum, service_ptr)); + router.setActiveProtocol(chat::MeshProtocol::MeshCore); + auto detached = router.takeServiceBackend(chat::MeshProtocol::Reticulum, service_ptr); + assert(detached.get() == service_ptr && !router.isServiceBackend(chat::MeshProtocol::Reticulum)); + assert(router.installServiceBackend(chat::MeshProtocol::Reticulum, std::move(detached))); + assert(router.backendForProtocol(chat::MeshProtocol::Reticulum) == service_ptr); + assert(router.sendText(chat::ChannelId::PRIMARY, "chat", nullptr, 0)); + assert(active_ptr->sends == 1 && service_ptr->sends == 0); + assert(!router.installServiceBackend(chat::MeshProtocol::Reticulum, std::make_unique())); + assert(router.backendForProtocol(chat::MeshProtocol::Reticulum) == service_ptr); + assert(!router.installServiceBackend(chat::MeshProtocol::Meshtastic, nullptr)); + assert(router.backendProtocol() == chat::MeshProtocol::MeshCore); + // A former active chat backend is not a service, but can be retired after + // the user switches away. Neither spelling of the active shared slot may go. + chat::MeshAdapterRouterCore switched; + auto old_reticulum = std::make_unique(); + auto* old_ptr = old_reticulum.get(); + assert(switched.installBackend(chat::MeshProtocol::Reticulum, std::move(old_reticulum))); + assert(!switched.takeInactiveBackend(chat::MeshProtocol::Reticulum, old_ptr)); + assert(!switched.takeInactiveBackend(chat::MeshProtocol::RNode, old_ptr)); + auto current = std::make_unique(); + auto* current_ptr = current.get(); + assert(switched.installBackend(chat::MeshProtocol::MeshCore, std::move(current))); + assert(!switched.takeServiceBackend(chat::MeshProtocol::Reticulum)); + assert(!switched.takeInactiveBackend(chat::MeshProtocol::Reticulum, current_ptr)); + auto retired = switched.takeInactiveBackend(chat::MeshProtocol::Reticulum, old_ptr); + assert(retired.get() == old_ptr && switched.backendProtocol() == chat::MeshProtocol::MeshCore); + assert(switched.sendText(chat::ChannelId::PRIMARY, "chat after retirement", nullptr, 0)); + assert(current_ptr->sends == 1); + assert(switched.installServiceBackend(chat::MeshProtocol::Reticulum, std::make_unique())); + assert(switched.backendProtocol() == chat::MeshProtocol::MeshCore); +} diff --git a/modules/core_geocaching/tests/test_service_ownership.cpp b/modules/core_geocaching/tests/test_service_ownership.cpp new file mode 100644 index 00000000..87f584e3 --- /dev/null +++ b/modules/core_geocaching/tests/test_service_ownership.cpp @@ -0,0 +1,40 @@ +#include "chat/infra/mesh_adapter_router_core.h" +#include + +class PollBackend final : public chat::IMeshAdapter +{ + public: + unsigned polls = 0; + bool sendText(chat::ChannelId, const std::string&, chat::MessageId*, chat::NodeId) override { return false; } + bool pollIncomingText(chat::MeshIncomingText*) override { return false; } + bool sendAppData(chat::ChannelId, std::uint32_t, const std::uint8_t*, std::size_t, + chat::NodeId, bool, chat::MessageId, bool) override { return false; } + bool pollIncomingData(chat::MeshIncomingData*) override { return false; } + void applyConfig(const chat::MeshConfig&) override {} + bool isReady() const override { return false; } + bool pollIncomingRawPacket(std::uint8_t*, std::size_t&, std::size_t) override { return false; } + void processSendQueue() override { ++polls; } +}; + +int main() +{ + chat::MeshAdapterRouterCore router; + auto radio = std::make_unique(); + auto* previous = radio.get(); + assert(router.installBackend(chat::MeshProtocol::Reticulum, std::move(radio))); + assert(router.installBackend(chat::MeshProtocol::MeshCore, std::make_unique())); + assert(!router.processServiceQueue(chat::MeshProtocol::Reticulum)); + assert(!router.processServiceQueue(chat::MeshProtocol::RNode)); + assert(previous->polls == 0); + + chat::MeshAdapterRouterCore replaced; + assert(replaced.installServiceBackend(chat::MeshProtocol::RNode, std::make_unique())); + assert(replaced.processServiceQueue(chat::MeshProtocol::Reticulum)); + auto replacement = std::make_unique(); + auto* replacement_ptr = replacement.get(); + assert(replaced.installBackend(chat::MeshProtocol::Reticulum, std::move(replacement))); + replaced.setActiveProtocol(chat::MeshProtocol::Meshtastic); + assert(!replaced.processServiceQueue(chat::MeshProtocol::RNode)); + assert(replacement_ptr->polls == 0); + return 0; +} diff --git a/modules/core_geocaching/tests/test_service_poll.cpp b/modules/core_geocaching/tests/test_service_poll.cpp new file mode 100644 index 00000000..36111448 --- /dev/null +++ b/modules/core_geocaching/tests/test_service_poll.cpp @@ -0,0 +1,38 @@ +#include "chat/infra/mesh_adapter_router_core.h" +#include + +class PollBackend final : public chat::IMeshAdapter +{ + public: + unsigned polls = 0; + bool sendText(chat::ChannelId, const std::string&, chat::MessageId*, chat::NodeId) override { return false; } + bool pollIncomingText(chat::MeshIncomingText*) override { return false; } + bool sendAppData(chat::ChannelId, std::uint32_t, const std::uint8_t*, std::size_t, + chat::NodeId, bool, chat::MessageId, bool) override { return false; } + bool pollIncomingData(chat::MeshIncomingData*) override { return false; } + void applyConfig(const chat::MeshConfig&) override {} + bool isReady() const override { return false; } + bool pollIncomingRawPacket(std::uint8_t*, std::size_t&, std::size_t) override { return false; } + void processSendQueue() override { ++polls; } +}; +int main() +{ + chat::MeshAdapterRouterCore router; + auto active = std::make_unique(); + auto* a = active.get(); + assert(router.installBackend(chat::MeshProtocol::Meshtastic, std::move(active))); + auto service = std::make_unique(); + auto* s = service.get(); + assert(router.installServiceBackend(chat::MeshProtocol::Reticulum, std::move(service))); + assert(router.processServiceQueue(chat::MeshProtocol::Reticulum)); + assert(s->polls == 1 && a->polls == 0); + assert(!router.processServiceQueue(chat::MeshProtocol::Meshtastic)); + router.processSendQueue(); + assert(a->polls == 1 && s->polls == 1); + router.setActiveProtocol(chat::MeshProtocol::RNode); + assert(!router.processServiceQueue(chat::MeshProtocol::Reticulum)); + assert(!router.processServiceQueue(chat::MeshProtocol::RNode)); + assert(!router.processServiceQueue(static_cast(255))); + router.processSendQueue(); + assert(s->polls == 2); +} diff --git a/modules/core_geocaching/tests/test_shared_track.cpp b/modules/core_geocaching/tests/test_shared_track.cpp new file mode 100644 index 00000000..24552c8c --- /dev/null +++ b/modules/core_geocaching/tests/test_shared_track.cpp @@ -0,0 +1,12 @@ +#include "gps/gpx/track_writer.h" +#include "gps/gpx/ostream_sink.h" +#include +#include +int main() +{ + std::ostringstream stream;gps::gpx::OstreamSink sink(stream); + assert(gps::gpx::writeTrackPoint(sink,30.25,120.15,"2026-09-16T00:00:00Z",8)); + assert(stream.str()=="\n 0.0\n \n \n 0.00\n 0.0\n 0.0\n 8\n \n\n"); + stream.str("");assert(gps::gpx::writeTrackPoint(sink,30.25,120.15,"",0,7)); + assert(stream.str().find("30.2500000")!=std::string::npos && stream.str().find("\n"); +} +} // namespace gps::gpx diff --git a/modules/core_gps/include/gps/gpx/window_sink.h b/modules/core_gps/include/gps/gpx/window_sink.h new file mode 100644 index 00000000..7f5359a2 --- /dev/null +++ b/modules/core_gps/include/gps/gpx/window_sink.h @@ -0,0 +1,45 @@ +#pragma once +#include "gps/gpx/text_writer.h" +#include + +namespace gps::gpx +{ +// A bounded window over the existing GPX serializer, with no file operations. +// A zero-capacity window counts output. The serializer remains the sole source +// of XML formatting/escaping; callers retain only one transfer-sized slice. +class WindowSink final : public OutputSink +{ + public: + WindowSink(size_t offset, uint8_t* output, size_t capacity) + : offset_(offset), capacity_(capacity), output_(output) {} + WindowSink(size_t offset, std::string_view expected) + : offset_(offset), capacity_(expected.size()), output_(nullptr), expected_(expected.data()) {} + bool write(std::string_view bytes) override + { + if ((!bytes.data() && bytes.size()) || bytes.size() > SIZE_MAX - total_ || (!output_ && !expected_ && capacity_)) return false; + const size_t start = total_; + total_ += bytes.size(); + if (total_ <= offset_ || written_ == capacity_) return true; + const size_t skip = offset_ > start ? offset_ - start : 0; + const size_t available = bytes.size() - skip; + const size_t count = available < capacity_ - written_ ? available : capacity_ - written_; + if (count) + { + if (expected_) + { + if (std::memcmp(expected_ + written_, bytes.data() + skip, count)) return false; + } + else std::memcpy(output_ + written_, bytes.data() + skip, count); + } + written_ += count; + return true; + } + size_t written() const { return written_; } + size_t total() const { return total_; } + + private: + size_t offset_, capacity_, written_ = 0, total_ = 0; + uint8_t* output_; + const char* expected_ = nullptr; +}; +} // namespace gps::gpx diff --git a/modules/core_sys/include/sys/crc32.h b/modules/core_sys/include/sys/crc32.h new file mode 100644 index 00000000..13d9bdc4 --- /dev/null +++ b/modules/core_sys/include/sys/crc32.h @@ -0,0 +1,23 @@ +#pragma once +#include +#include + +namespace sys +{ +// CRC-32/ISO-HDLC. Pass a previous returned CRC to continue over another span. +// Extracted from the existing chat storage codec; default preserves its format. +inline uint32_t crc32(const void* data, std::size_t len, uint32_t previous = 0) +{ + const auto* bytes = static_cast(data); + uint32_t crc = ~previous; + for (std::size_t index = 0; index < len; ++index) + { + crc ^= bytes[index]; + for (uint8_t bit = 0; bit < 8; ++bit) + { + crc = (crc & 1U) != 0U ? (crc >> 1U) ^ 0xEDB88320U : crc >> 1U; + } + } + return ~crc; +} +} // namespace sys diff --git a/modules/ui_presentation/include/ui_presentation/geocaching/geocaching_source.h b/modules/ui_presentation/include/ui_presentation/geocaching/geocaching_source.h new file mode 100644 index 00000000..7138ccaf --- /dev/null +++ b/modules/ui_presentation/include/ui_presentation/geocaching/geocaching_source.h @@ -0,0 +1,91 @@ +#pragma once +#include +#include +#include +#include + +namespace ui::geocaching +{ +enum class Section : std::uint8_t +{ + Discover, + Downloaded, + Published +}; +struct Item +{ + // Draft rows use their 16-byte local draft ID in the leading bytes of id; + // they are never cache IDs and cannot be downloaded or navigated to. + bool is_draft = false; + uint64_t edit_generation = 0; + uint32_t publication_revision = 0; + bool publication_confirmed = false; + std::array id{}; + std::array revision_hash{}; + std::array name{}; + std::array detail{}; + int32_t latitude_e7 = 0, longitude_e7 = 0; + bool can_download = false; + bool downloaded = false; +}; +struct Snapshot +{ + std::uint64_t generation = 0; + // Metadata only; count covers all results, rows are read individually. + std::size_t count = 0; + std::array status{}; + bool can_refresh = false; + bool has_more = false; + bool can_create = false; +}; +struct DraftInput +{ + std::array id{}; + uint64_t generation = 0; + std::string_view name, description, hint; + int32_t latitude_e7 = 0, longitude_e7 = 0; + uint8_t state = 0, difficulty_x2 = 2, terrain_x2 = 2, container_size = 0; + bool has_coordinates = false; +}; +enum class DraftSaveStatus : uint8_t +{ + Failed, + Pending, + Saved +}; +enum class DraftReadStatus : uint8_t +{ + Failed, + Pending, + Ready +}; +class Source +{ + public: + virtual ~Source() = default; + virtual void activate(bool) {} + // UI-thread call: copy a coherent, bounded in-memory projection, without + // network or SD I/O. Increment generation for every visible change within + // a section, including status and action availability. Strings are NUL + // terminated. No list-sized copy or I/O is allowed here. + virtual void snapshot(Section section, Snapshot& out) = 0; + // Read a single row from this generation without I/O. False means stale. + virtual bool item(Section section, std::size_t index, std::uint64_t generation, Item& out) = 0; + virtual void refresh(Section section) = 0; + virtual void open(const Item& item, std::uint64_t generation) = 0; + virtual bool loadMore() { return false; } + virtual bool download(const Item&, std::uint64_t) { return false; } + // UI-thread call: enqueue/poll without I/O. Pending never calls or retains + // the sink/context. Ready calls it synchronously with borrowed text; the + // sink consumes it immediately and must not reenter Source. + virtual DraftReadStatus readDraft(const std::array&, void (*)(const DraftInput&, void*), void*) { return DraftReadStatus::Failed; } + // Nonblocking UI notification: drop interest in this pending read. The + // storage owner releases its lease on its next step; no UI callback follows. + virtual void cancelDraftRead(const std::array&) {} + // Assigns a stable ID to a new draft; Saved is queried separately after I/O. + virtual bool saveDraft(DraftInput&) { return false; } + virtual DraftSaveStatus draftSaveStatus(const std::array&, uint64_t) { return DraftSaveStatus::Failed; } + virtual bool publicationAuthor(const std::array&, uint64_t, std::array&, uint32_t* = nullptr, uint32_t* = nullptr) { return false; } + virtual bool publishDraft(const std::array&, uint64_t, const std::array&, uint32_t) { return false; } +}; +} // namespace ui::geocaching diff --git a/modules/ui_presentation/include/ui_presentation/map/map_overlay_snapshot.h b/modules/ui_presentation/include/ui_presentation/map/map_overlay_snapshot.h index afa8ae4a..b598fffb 100644 --- a/modules/ui_presentation/include/ui_presentation/map/map_overlay_snapshot.h +++ b/modules/ui_presentation/include/ui_presentation/map/map_overlay_snapshot.h @@ -20,6 +20,7 @@ enum class MapOverlayKind : uint8_t MeasurementPoint, SelectedTarget, Warning, + Geocache, }; enum class MapOverlayStyle : uint8_t diff --git a/modules/ui_shared/include/ui/screens/geocaching/geocaching_page_shell.h b/modules/ui_shared/include/ui/screens/geocaching/geocaching_page_shell.h new file mode 100644 index 00000000..c9d18f97 --- /dev/null +++ b/modules/ui_shared/include/ui/screens/geocaching/geocaching_page_shell.h @@ -0,0 +1,11 @@ +#pragma once +#include "lvgl.h" +#include "ui/page/page_host.h" +#include "ui_presentation/geocaching/geocaching_source.h" + +namespace geocaching::ui::shell +{ +void bind(::ui::geocaching::Source* source); +void enter(void* user_data, lv_obj_t* parent); +void exit(void* user_data, lv_obj_t* parent); +} // namespace geocaching::ui::shell diff --git a/modules/ui_shared/include/ui/screens/gps/gps_page_runtime.h b/modules/ui_shared/include/ui/screens/gps/gps_page_runtime.h index b6666baa..d1a7a7fd 100644 --- a/modules/ui_shared/include/ui/screens/gps/gps_page_runtime.h +++ b/modules/ui_shared/include/ui/screens/gps/gps_page_runtime.h @@ -1,6 +1,7 @@ #pragma once #include "ui/screens/gps/gps_page_shell.h" +#include "ui_presentation/map/map_overlay_snapshot.h" #include @@ -8,6 +9,16 @@ namespace gps::ui::runtime { bool is_available(); +struct MapTarget +{ + int32_t latitude_e7 = 0, longitude_e7 = 0; + char name[97]{}; + void* overlay_context = nullptr; + void (*append_overlays)(void*, ::ui::map::MapOverlaySnapshot&) = nullptr; + void (*select_overlay)(void*, uint32_t) = nullptr; +}; +// Target is borrowed until exit; caller owns its lifetime. +bool enter_target(const shell::Host* host, lv_obj_t* parent, const MapTarget& target); void enter(const shell::Host* host, lv_obj_t* parent, shell::Projection projection = shell::Projection::Map); diff --git a/modules/ui_shared/include/ui/widgets/map/map_viewport.h b/modules/ui_shared/include/ui/widgets/map/map_viewport.h index fa31bb29..23d2a684 100644 --- a/modules/ui_shared/include/ui/widgets/map/map_viewport.h +++ b/modules/ui_shared/include/ui/widgets/map/map_viewport.h @@ -84,6 +84,7 @@ enum class GesturePhase : uint8_t DragUpdate, DragEnd, Cancel, + Tapped, }; struct GestureEvent diff --git a/modules/ui_shared/src/ui/app_catalog_builder.cpp b/modules/ui_shared/src/ui/app_catalog_builder.cpp index 2a4cdeab..584d0a65 100644 --- a/modules/ui_shared/src/ui/app_catalog_builder.cpp +++ b/modules/ui_shared/src/ui/app_catalog_builder.cpp @@ -18,8 +18,10 @@ #if defined(ARDUINO_T_LORA_PAGER) || defined(ARDUINO_T_DECK) || defined(ARDUINO_WIO_TRACKER_L2) || \ defined(TRAIL_MATE_ESP_BOARD_T_DISPLAY_P4) #define TRAIL_MATE_ENABLE_CALCULATOR_APP 1 +#define TRAIL_MATE_ENABLE_GEOCACHING_APP 1 #else #define TRAIL_MATE_ENABLE_CALCULATOR_APP 0 +#define TRAIL_MATE_ENABLE_GEOCACHING_APP 0 #endif #if TRAIL_MATE_USE_MONO_SCREEN_240X320 #include "ui/mono/screens/screen_240x320/screen_app.h" @@ -46,6 +48,9 @@ #if TRAIL_MATE_ENABLE_CALCULATOR_APP #include "ui/screens/calculator/calculator_page_shell.h" #endif +#if TRAIL_MATE_ENABLE_GEOCACHING_APP +#include "ui/screens/geocaching/geocaching_page_shell.h" +#endif #include "ui/screens/contacts/contacts_page_shell.h" #include "ui/screens/energy_sweep/energy_sweep_page_shell.h" #include "ui/screens/extensions/extensions_page_shell.h" @@ -84,6 +89,7 @@ extern "C" { extern const lv_image_dsc_t Chat; extern const lv_image_dsc_t calc; + extern const lv_image_dsc_t geocaching; extern const lv_image_dsc_t gps_icon; extern const lv_image_dsc_t Satellite; extern const lv_image_dsc_t contact; @@ -222,6 +228,12 @@ ui::CallbackAppScreen s_tracker_app("tracker", "Tracker", CATALOG_ICON(tracker_i tracker::ui::shell::enter, tracker::ui::shell::exit, &s_menu_host); +#if TRAIL_MATE_ENABLE_GEOCACHING_APP +ui::CallbackAppScreen s_geocaching_app("geocaching", "Geocaching", CATALOG_ICON(geocaching), + ::geocaching::ui::shell::enter, + ::geocaching::ui::shell::exit, + &s_menu_host); +#endif #if TRAIL_MATE_ENABLE_CALCULATOR_APP ui::CallbackAppScreen s_calculator_app("calculator", "Calculator", CATALOG_ICON(calc), calculator::ui::shell::enter, @@ -476,6 +488,9 @@ AppCatalog build(const FeatureFlags& flags) add(&s_team_app); #endif } +#if TRAIL_MATE_ENABLE_GEOCACHING_APP + add(&s_geocaching_app); +#endif #if TRAIL_MATE_ENABLE_CALCULATOR_APP add(&s_calculator_app); #endif diff --git a/modules/ui_shared/src/ui/screens/geocaching/geocaching_page_shell.cpp b/modules/ui_shared/src/ui/screens/geocaching/geocaching_page_shell.cpp new file mode 100644 index 00000000..cc3c375b --- /dev/null +++ b/modules/ui_shared/src/ui/screens/geocaching/geocaching_page_shell.cpp @@ -0,0 +1,859 @@ +#include "ui/screens/geocaching/geocaching_page_shell.h" +#include "ui/app_runtime.h" +#include "ui/components/two_pane_styles.h" +#include "ui/page/page_profile.h" +#include "ui/screens/gps/gps_page_runtime.h" +#include "ui/widgets/top_bar.h" +#include +#include +#include +#include + +namespace geocaching::ui::shell +{ +namespace +{ +namespace styles = ::ui::components::two_pane_styles; +using Section = ::ui::geocaching::Section; +constexpr size_t kVisibleRows = 4; +struct Editor +{ + ::ui::geocaching::DraftInput input; + std::array fields{}; + lv_obj_t* container = nullptr; + lv_obj_t* publication_notice = nullptr; + std::array publication_author{}; + uint32_t previous_revision = 0, publication_revision = 0; + bool public_confirmation = false; + bool dirty = false, saving = false, confirming = false, loading = false; +}; +struct PageState +{ + Editor* editor = nullptr; + ::ui::geocaching::Snapshot snapshot; + ::ui::widgets::TopBar topbar; + const ::ui::page::Host* host = nullptr; + lv_obj_t *root = nullptr, *list = nullptr, *status = nullptr; + lv_obj_t *refresh = nullptr, *previous = nullptr, *next = nullptr, *range = nullptr; + std::array tabs{}; + std::array rows{}; + std::array, kVisibleRows> row_ids{}; + lv_group_t *group = nullptr, *previous_group = nullptr; + lv_timer_t* timer = nullptr; + ::ui::geocaching::Source* rendered_source = nullptr; + Section section = Section::Discover, rendered_section = Section::Discover; + size_t offset = 0, rendered_offset = 0, row_count = 0, window = kVisibleRows; + size_t detail_index = 0; + uint64_t detail_generation = 0; + std::array detail_id{}, detail_hash{}; + bool valid = false, details = false; +}; +static_assert(sizeof(PageState) <= 640, "Page state must not contain an entire result list"); +::ui::geocaching::Source* source = nullptr; +PageState* page = nullptr; +struct MapVisit +{ + ::gps::ui::runtime::MapTarget target; + std::array selected_id{}; + uint64_t overlay_generation = 0; + ::ui::page::Host host; + lv_obj_t* parent = nullptr; +}; +MapVisit* map_visit = nullptr; + +void refreshView(); +void closeDetails(); +void showDetails(const ::ui::geocaching::Item& item, size_t index); +void openEditor(const ::ui::geocaching::Item* item); +void editorBack(); +void saveEditor(); +void closeEditor(); +void previewPublication(); + +void returnFromMap(void*) +{ + if (!map_visit) return; + ::gps::ui::runtime::exit(map_visit->parent); + delete map_visit; + map_visit = nullptr; + if (!page) return; + lv_obj_remove_flag(page->root, LV_OBJ_FLAG_HIDDEN); + set_default_group(page->group); + if (page->timer) lv_timer_resume(page->timer); + refreshView(); + if (page->group) lv_group_focus_obj(page->topbar.back_btn); +} + +bool openMap(const ::ui::geocaching::Item& item) +{ + if (!page || map_visit || !item.downloaded) return false; + map_visit = new (std::nothrow) MapVisit; + if (!map_visit) return false; + map_visit->target.latitude_e7 = item.latitude_e7; + map_visit->target.longitude_e7 = item.longitude_e7; + map_visit->selected_id = item.id; + map_visit->target.overlay_context = map_visit; + map_visit->target.append_overlays = [](void* context, ::ui::map::MapOverlaySnapshot& overlays) + { + if (!source) return; + auto& visit = *static_cast(context); + ::ui::geocaching::Snapshot saved; + source->snapshot(Section::Downloaded, saved); + visit.overlay_generation = saved.generation; + ::ui::geocaching::Item cached; + for (size_t index = 0; index < saved.count; ++index) + { + if (overlays.item_count == ::ui::map::MapOverlaySnapshot::kMaxItems) + { + overlays.truncated = true; + break; + } + if (!source->item(Section::Downloaded, index, saved.generation, cached)) break; + if (!cached.downloaded || cached.id == visit.selected_id) continue; + auto& marker = overlays.items[overlays.item_count++]; + marker = ::ui::map::MapOverlayItem{}; + marker.kind = ::ui::map::MapOverlayKind::Geocache; + marker.stable_id = static_cast(index + 1); + marker.point.valid = true; + marker.point.lat = cached.latitude_e7 / 10000000.0; + marker.point.lon = cached.longitude_e7 / 10000000.0; + ::ui::copyText(marker.label, cached.name.data()); + } + }; + map_visit->target.select_overlay = [](void* context, uint32_t id) + { + if (!page || !source) return; + const auto& visit = *static_cast(context); + if (id) + { + ::ui::geocaching::Item selected; + if (!source->item(Section::Downloaded, id - 1, visit.overlay_generation, selected) || !selected.downloaded) return; + page->section = Section::Downloaded; + page->snapshot.generation = visit.overlay_generation; + showDetails(selected, id - 1); + } + lv_async_call_cancel(returnFromMap, nullptr); + lv_async_call(returnFromMap, nullptr); + }; + std::snprintf(map_visit->target.name, sizeof(map_visit->target.name), "%s", item.name.data()); + map_visit->parent = lv_obj_get_parent(page->root); + map_visit->host.request_exit = [](void*) + { lv_async_call_cancel(returnFromMap, nullptr); lv_async_call(returnFromMap, nullptr); }; + lv_obj_add_flag(page->root, LV_OBJ_FLAG_HIDDEN); + if (page->timer) lv_timer_pause(page->timer); + if (::gps::ui::runtime::enter_target(&map_visit->host, map_visit->parent, map_visit->target)) return true; + delete map_visit; + map_visit = nullptr; + lv_obj_remove_flag(page->root, LV_OBJ_FLAG_HIDDEN); + if (page->timer) lv_timer_resume(page->timer); + set_default_group(page->group); + return false; +} + +void keyEvent(lv_event_t* event) +{ + if (!page) return; + const auto key = lv_event_get_key(event); + if (key == LV_KEY_ESC) + { + lv_event_stop_processing(event); + if (page->editor) editorBack(); + else if (page->details) closeDetails(); + else if (page->host) ::ui::page::request_exit(page->host); + } + else if (page->editor && page->group && lv_group_get_editing(page->group)) return; + else if (page->details && (key == LV_KEY_DOWN || key == LV_KEY_UP)) + lv_obj_scroll_by(page->list, 0, key == LV_KEY_DOWN ? -28 : 28, LV_ANIM_OFF); + else if (page->group && (key == LV_KEY_DOWN || key == LV_KEY_RIGHT)) lv_group_focus_next(page->group); + else if (page->group && (key == LV_KEY_UP || key == LV_KEY_LEFT)) lv_group_focus_prev(page->group); +} +void addFocusable(lv_obj_t* object) +{ + if (page->group) lv_group_add_obj(page->group, object); + lv_obj_add_event_cb(object, keyEvent, LV_EVENT_KEY, nullptr); +} +void setEnabled(lv_obj_t* object, bool enabled) +{ + if (page->group) lv_group_remove_obj(object); + if (enabled) + { + lv_obj_remove_state(object, LV_STATE_DISABLED); + if (page->group) lv_group_add_obj(page->group, object); + } + else lv_obj_add_state(object, LV_STATE_DISABLED); +} +lv_obj_t* button(lv_obj_t* parent, const char* text, lv_coord_t height) +{ + auto* object = lv_button_create(parent); + lv_obj_remove_style_all(object); + styles::apply_btn_filter(object); + lv_obj_set_style_pad_hor(object, 6, 0); + lv_obj_set_style_pad_ver(object, 2, 0); + lv_obj_set_style_opa(object, LV_OPA_50, LV_STATE_DISABLED); + lv_obj_set_height(object, height); + auto* label = lv_label_create(object); + lv_label_set_text(label, text); + lv_obj_center(label); + addFocusable(object); + return object; +} +void refreshView() +{ + if (!page) return; + if (page->editor) + { + auto& editor = *page->editor; + if (editor.loading) + { + ::ui::geocaching::Item item; + std::copy(editor.input.id.begin(), editor.input.id.end(), item.id.begin()); + openEditor(&item); + return; + } + if (!editor.confirming && !editor.public_confirmation && !editor.saving && editor.input.generation) + { + lv_obj_remove_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + lv_label_set_text(lv_obj_get_child(page->previous, 0), "Publish"); + if (lv_obj_has_state(page->previous, LV_STATE_DISABLED) != editor.dirty) setEnabled(page->previous, !editor.dirty); + } + if (editor.saving && source) + { + const auto status = source->draftSaveStatus(editor.input.id, editor.input.generation); + if (status == ::ui::geocaching::DraftSaveStatus::Saved) + { + closeEditor(); + return; + } + if (status == ::ui::geocaching::DraftSaveStatus::Failed) + { + editor.saving = false; + for (auto* field : editor.fields) lv_obj_remove_state(field, LV_STATE_DISABLED); + lv_obj_remove_state(editor.container, LV_STATE_DISABLED); + lv_label_set_text(page->status, "Save failed; edits retained"); + setEnabled(page->next, true); + } + } + return; + } + if (page->details) + { + if (!source) return; + ::ui::geocaching::Snapshot current; + source->snapshot(page->section, current); + if (current.generation == page->detail_generation) return; + ::ui::geocaching::Item item; + const bool same = source->item(page->section, page->detail_index, current.generation, item) && + item.id == page->detail_id && item.revision_hash == page->detail_hash; + lv_label_set_text(page->status, current.status.data()); + if (same) lv_label_set_text(lv_obj_get_child(page->list, 1), item.detail.data()); + lv_label_set_text(lv_obj_get_child(page->next, 0), same && item.downloaded ? "Map" : "Download"); + setEnabled(page->next, same && (item.can_download || item.downloaded)); + page->detail_generation = current.generation; + return; + } + auto& p = *page; + const auto old_generation = p.snapshot.generation; + auto* focused = p.group ? lv_group_get_focused(p.group) : nullptr; + std::array focused_id{}; + bool focused_row = false; + for (size_t i = 0; i < p.row_count; ++i) + if (focused == p.rows[i]) + { + focused_id = p.row_ids[i]; + focused_row = true; + break; + } + p.snapshot = {}; + if (source) source->snapshot(p.section, p.snapshot); + else std::snprintf(p.snapshot.status.data(), p.snapshot.status.size(), "Geocaching service unavailable"); + if (p.offset >= p.snapshot.count) p.offset = p.snapshot.count ? ((p.snapshot.count - 1) / p.window) * p.window : 0; + if (p.valid && p.rendered_source == source && p.rendered_section == p.section && + p.rendered_offset == p.offset && old_generation == p.snapshot.generation) return; + p.valid = true; + p.rendered_source = source; + p.rendered_section = p.section; + p.rendered_offset = p.offset; + for (size_t i = 0; i < p.tabs.size(); ++i) + { + if (i == static_cast(p.section)) lv_obj_add_state(p.tabs[i], LV_STATE_CHECKED); + else lv_obj_remove_state(p.tabs[i], LV_STATE_CHECKED); + } + lv_label_set_text(p.status, p.snapshot.status.data()); + lv_obj_clean(p.list); + p.rows.fill(nullptr); + p.row_count = 0; + ::ui::geocaching::Item item; + while (p.row_count < p.window && p.offset + p.row_count < p.snapshot.count) + { + const size_t i = p.row_count; + if (!source || !source->item(p.section, p.offset + i, p.snapshot.generation, item)) + { + p.valid = false; + break; + } + auto* row = button(p.list, item.name.data(), ::ui::page_profile::current().list_item_height); + lv_obj_set_width(row, LV_PCT(100)); + auto* label = lv_obj_get_child(row, 0); + lv_obj_set_width(label, LV_PCT(100)); + lv_label_set_long_mode(label, LV_LABEL_LONG_DOT); + if (item.is_draft && item.publication_revision) + lv_label_set_text_fmt(label, "v%lu %s | %s", static_cast(item.publication_revision), + item.publication_confirmed ? "accepted" : "unconfirmed", item.name.data()); + p.rows[i] = row; + p.row_ids[i] = item.id; + ++p.row_count; + lv_obj_add_event_cb( + row, [](lv_event_t* event) + { + if (!page || !source) return; + const auto index = reinterpret_cast(lv_event_get_user_data(event)); + ::ui::geocaching::Item selected; + if (index < page->row_count && + source->item(page->section, page->offset + index, page->snapshot.generation, selected) && + selected.id == page->row_ids[index]) + { const auto row = page->offset + index; source->open(selected, page->snapshot.generation); if (page) showDetails(selected, row); } + else { page->valid = false; refreshView(); } }, + LV_EVENT_CLICKED, reinterpret_cast(i)); + } + if (!p.row_count) + { + auto* empty = lv_label_create(p.list); + lv_label_set_text(empty, source ? "No caches in this view" : "No cached items"); + styles::apply_label_muted(empty); + lv_obj_set_width(empty, LV_PCT(100)); + lv_obj_set_style_text_align(empty, LV_TEXT_ALIGN_CENTER, 0); + lv_obj_set_style_pad_top(empty, 12, 0); + } + lv_label_set_text_fmt(p.range, "%lu-%lu / %lu", + static_cast(p.row_count ? p.offset + 1 : 0), + static_cast(p.offset + p.row_count), static_cast(p.snapshot.count)); + setEnabled(p.refresh, p.section == Section::Published ? p.snapshot.can_create : p.snapshot.can_refresh); + lv_label_set_text(lv_obj_get_child(p.refresh, 0), p.section == Section::Published ? "New" : "Refresh"); + setEnabled(p.previous, p.offset != 0); + setEnabled(p.next, (p.offset + p.row_count < p.snapshot.count || p.snapshot.has_more) && p.row_count != 0 && p.valid); + if (p.group) + { + if (focused_row) + { + auto* restore = p.tabs[static_cast(p.section)]; + for (size_t i = 0; i < p.row_count; ++i) + if (p.row_ids[i] == focused_id) restore = p.rows[i]; + lv_group_focus_obj(restore); + } + else if (focused && !lv_obj_has_state(focused, LV_STATE_DISABLED)) lv_group_focus_obj(focused); + else lv_group_focus_obj(p.tabs[static_cast(p.section)]); + } +} +void closeDetails() +{ + if (!page) return; + page->details = false; + page->valid = false; + lv_obj_remove_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + lv_obj_set_width(page->next, ::ui::page_profile::current().control_button_height); + lv_label_set_text(lv_obj_get_child(page->next, 0), LV_SYMBOL_RIGHT); + lv_obj_remove_flag(page->list, LV_OBJ_FLAG_SCROLLABLE); + lv_obj_scroll_to_y(page->list, 0, LV_ANIM_OFF); + refreshView(); +} +void showDetails(const ::ui::geocaching::Item& item, size_t index) +{ + if (item.is_draft) + { + openEditor(&item); + return; + } + auto& p = *page; + p.details = true; + p.detail_index = index; + p.detail_generation = p.snapshot.generation; + p.detail_id = item.id; + p.detail_hash = item.revision_hash; + lv_obj_clean(p.list); + p.rows.fill(nullptr); + p.row_count = 0; + lv_obj_add_flag(p.list, LV_OBJ_FLAG_SCROLLABLE); + auto* title = lv_label_create(p.list); + lv_obj_set_width(title, LV_PCT(100)); + lv_label_set_text(title, item.name.data()); + styles::apply_label_primary(title); + auto* body = lv_label_create(p.list); + lv_obj_set_width(body, LV_PCT(100)); + lv_label_set_text(body, item.detail.data()); + styles::apply_label_muted(body); + lv_label_set_text(p.range, "Preview"); + lv_label_set_text(lv_obj_get_child(p.refresh, 0), "Back"); + setEnabled(p.refresh, true); + setEnabled(p.previous, false); + lv_obj_add_flag(p.previous, LV_OBJ_FLAG_HIDDEN); + lv_obj_set_width(p.next, ::ui::page_profile::current().control_button_min_width); + lv_label_set_text(lv_obj_get_child(p.next, 0), item.downloaded ? "Map" : "Download"); + setEnabled(p.next, item.can_download || item.downloaded); + if (p.group) lv_group_focus_obj(p.topbar.back_btn); +} +bool parseDecimal(const char* text, int32_t& out) +{ + if (!text || !*text) return false; + const bool negative = *text == '-'; + if (negative) ++text; + if (*text < '0' || *text > '9') return false; + int64_t whole = 0, fraction = 0, scale = 1000000; + while (*text >= '0' && *text <= '9') + { + whole = whole * 10 + (*text++ - '0'); + if (whole > 180) return false; + } + if (*text == '.') + { + ++text; + if (*text < '0' || *text > '9') return false; + while (*text >= '0' && *text <= '9') + { + if (!scale) return false; + fraction += (*text++ - '0') * scale; + scale /= 10; + } + } + if (*text) return false; + const auto value = whole * 10000000 + fraction; + if (value > 1800000000) return false; + out = static_cast(negative ? -value : value); + return true; +} + +void closeEditor() +{ + if (!page || !page->editor) return; + if (page->editor->loading && source) source->cancelDraftRead(page->editor->input.id); + delete page->editor; + page->editor = nullptr; + for (auto* tab : page->tabs) setEnabled(tab, true); + lv_obj_set_width(page->previous, ::ui::page_profile::current().control_button_height); + lv_label_set_text(lv_obj_get_child(page->previous, 0), LV_SYMBOL_LEFT); + closeDetails(); +} + +void editorBack() +{ + if (!page || !page->editor) return; + auto& editor = *page->editor; + if (editor.public_confirmation) + { + editor.public_confirmation = false; + if (editor.publication_notice) lv_obj_delete(editor.publication_notice); + editor.publication_notice = nullptr; + for (auto* field : editor.fields) lv_obj_remove_state(field, LV_STATE_DISABLED); + lv_obj_remove_state(editor.container, LV_STATE_DISABLED); + lv_label_set_text(lv_obj_get_child(page->refresh, 0), "Back"); + lv_label_set_text(lv_obj_get_child(page->next, 0), "Save"); + lv_label_set_text(page->status, "Editing local draft"); + refreshView(); + return; + } + if (editor.saving) + { + lv_label_set_text(page->status, "Saving; please wait"); + return; + } + if (!editor.dirty) + { + closeEditor(); + return; + } + editor.confirming = !editor.confirming; + lv_label_set_text(lv_obj_get_child(page->previous, 0), "Discard"); + lv_label_set_text(page->status, editor.confirming ? "Unsaved edits: save, discard or keep editing" : "Editing local draft"); + lv_label_set_text(lv_obj_get_child(page->refresh, 0), editor.confirming ? "Keep" : "Back"); + if (editor.confirming) lv_obj_remove_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + else lv_obj_add_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + setEnabled(page->previous, editor.confirming); +} + +void previewPublication() +{ + if (!page || !page->editor || !source) return; + auto& editor = *page->editor; + if (editor.dirty || editor.saving || !editor.input.generation || + !source->publicationAuthor(editor.input.id, editor.input.generation, editor.publication_author, &editor.previous_revision, &editor.publication_revision)) + { + lv_label_set_text(page->status, "Save location/name; check author, clock and directory"); + return; + } + editor.public_confirmation = true; + char author[129]; + constexpr char hex[] = "0123456789abcdef"; + for (size_t i = 0; i < editor.publication_author.size(); ++i) + { + author[i * 2] = hex[editor.publication_author[i] >> 4]; + author[i * 2 + 1] = hex[editor.publication_author[i] & 15]; + } + author[128] = 0; + editor.publication_notice = lv_label_create(page->list); + lv_obj_set_width(editor.publication_notice, LV_PCT(100)); + lv_label_set_long_mode(editor.publication_notice, LV_LABEL_LONG_WRAP); + if (editor.previous_revision) + lv_label_set_text_fmt(editor.publication_notice, "Publish this cache publicly on Reticulum?\nv%lu -> v%lu\nAuthor public identity:\n%s", + static_cast(editor.previous_revision), static_cast(editor.publication_revision), author); + else + lv_label_set_text_fmt(editor.publication_notice, "Publish this cache publicly on Reticulum?\nFirst publication: v%lu\nAuthor public identity:\n%s", + static_cast(editor.publication_revision), author); + styles::apply_label_primary(editor.publication_notice); + lv_obj_move_to_index(editor.publication_notice, 0); + for (auto* field : editor.fields) lv_obj_add_state(field, LV_STATE_DISABLED); + lv_obj_add_state(editor.container, LV_STATE_DISABLED); + lv_obj_scroll_to_y(page->list, 0, LV_ANIM_OFF); + lv_label_set_text(page->status, "Review author and saved fields before publishing"); + lv_label_set_text(lv_obj_get_child(page->refresh, 0), "Cancel"); + lv_label_set_text(lv_obj_get_child(page->next, 0), "Publish"); + setEnabled(page->previous, false); + lv_obj_add_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + if (page->group) lv_group_focus_obj(page->refresh); +} + +void saveEditor() +{ + if (!page || !page->editor || !source || page->editor->saving || page->editor->loading) return; + auto& editor = *page->editor; + if (editor.public_confirmation) + { + if (source->publishDraft(editor.input.id, editor.input.generation, editor.publication_author, editor.publication_revision)) closeEditor(); + else lv_label_set_text(page->status, "Publication not started; review identity and draft again"); + return; + } + auto& input = editor.input; + const char* lat = lv_textarea_get_text(editor.fields[1]); + const char* lon = lv_textarea_get_text(editor.fields[2]); + input.has_coordinates = *lat || *lon; + int32_t difficulty = 0, terrain = 0; + if ((input.has_coordinates && (!parseDecimal(lat, input.latitude_e7) || !parseDecimal(lon, input.longitude_e7) || + input.latitude_e7 < -900000000 || input.latitude_e7 > 900000000 || input.longitude_e7 >= 1800000000)) || + !parseDecimal(lv_textarea_get_text(editor.fields[5]), difficulty) || difficulty < 10000000 || difficulty > 50000000 || difficulty % 5000000 || + !parseDecimal(lv_textarea_get_text(editor.fields[6]), terrain) || terrain < 10000000 || terrain > 50000000 || terrain % 5000000) + { + lv_label_set_text(page->status, "Check coordinates and 1-5 ratings (step 0.5)"); + return; + } + input.difficulty_x2 = static_cast(difficulty / 5000000); + input.terrain_x2 = static_cast(terrain / 5000000); + input.container_size = static_cast(lv_dropdown_get_selected(editor.container)); + input.name = lv_textarea_get_text(editor.fields[0]); + input.description = lv_textarea_get_text(editor.fields[3]); + input.hint = lv_textarea_get_text(editor.fields[4]); + if (!source->saveDraft(input)) + { + lv_label_set_text(page->status, "Cannot save now; edits retained"); + return; + } + editor.saving = true; + for (auto* field : editor.fields) lv_obj_add_state(field, LV_STATE_DISABLED); + lv_obj_add_state(editor.container, LV_STATE_DISABLED); + setEnabled(page->next, false); + setEnabled(page->previous, false); + lv_label_set_text(page->status, "Saving local draft..."); +} + +void openEditor(const ::ui::geocaching::Item* item) +{ + if (!page || !source || (page->editor && !page->editor->loading)) return; + if (!page->editor) page->editor = new (std::nothrow) Editor; + if (!page->editor) return; + auto& editor = *page->editor; + if (!editor.loading) + { + page->details = false; + page->row_count = 0; + page->rows.fill(nullptr); + lv_obj_clean(page->list); + lv_obj_add_flag(page->list, LV_OBJ_FLAG_SCROLLABLE); + const char* names[] = {"Name", "Latitude (blank if unset)", "Longitude", "Description", "Hint", "Difficulty (1-5)", "Terrain (1-5)"}; + const uint32_t limits[] = {96, 12, 13, 2048, 512, 3, 3}; + for (size_t i = 0; i < editor.fields.size(); ++i) + { + auto* label = lv_label_create(page->list); + lv_label_set_text(label, names[i]); + styles::apply_label_primary(label); + auto* field = editor.fields[i] = lv_textarea_create(page->list); + lv_obj_set_width(field, LV_PCT(100)); + lv_textarea_set_one_line(field, i != 3 && i != 4); + lv_obj_set_height(field, i == 3 || i == 4 ? 64 : 32); + lv_textarea_set_max_length(field, limits[i]); + lv_textarea_set_text(field, i == 5 || i == 6 ? "1.0" : ""); + lv_obj_set_style_bg_color(field, lv_color_hex(styles::kSidePanelBg), 0); + lv_obj_set_style_text_color(field, lv_color_hex(styles::kTextPrimary), 0); + lv_obj_set_style_border_color(field, lv_color_hex(styles::kBorder), 0); + lv_obj_set_style_border_color(field, lv_color_hex(styles::kAccent), LV_STATE_FOCUSED); + lv_obj_set_style_outline_color(field, lv_color_hex(styles::kAccent), LV_STATE_FOCUSED); + addFocusable(field); + lv_obj_add_event_cb( + field, [](lv_event_t*) + { if (page && page->editor) page->editor->dirty = true; }, + LV_EVENT_VALUE_CHANGED, nullptr); + } + auto* label = lv_label_create(page->list); + lv_label_set_text(label, "Container"); + editor.container = lv_dropdown_create(page->list); + lv_dropdown_set_options(editor.container, "Unspecified\nMicro\nSmall\nRegular\nLarge\nOther"); + lv_obj_set_style_bg_color(editor.container, lv_color_hex(styles::kMainPanelBg), 0); + lv_obj_set_style_text_color(editor.container, lv_color_hex(styles::kTextPrimary), 0); + lv_obj_set_style_border_color(editor.container, lv_color_hex(styles::kAccent), LV_STATE_FOCUSED); + lv_obj_set_style_outline_color(editor.container, lv_color_hex(styles::kAccent), LV_STATE_FOCUSED); + lv_obj_add_event_cb( + editor.container, [](lv_event_t* event) + { + auto* list = lv_dropdown_get_list(lv_event_get_target_obj(event)); + if (!list) return; + lv_obj_set_style_bg_color(list, lv_color_hex(styles::kMainPanelBg), 0); + lv_obj_set_style_text_color(list, lv_color_hex(styles::kTextPrimary), 0); + lv_obj_set_style_bg_color(list, lv_color_hex(styles::kAccent), LV_PART_SELECTED | LV_STATE_CHECKED); }, + LV_EVENT_CLICKED, nullptr); + addFocusable(editor.container); + lv_obj_add_event_cb( + editor.container, [](lv_event_t*) + { if (page && page->editor) page->editor->dirty = true; }, + LV_EVENT_VALUE_CHANGED, nullptr); + } + if (item) + { + std::array id; + std::copy_n(item->id.data(), id.size(), id.data()); + const auto result = source->readDraft( + id, [](const ::ui::geocaching::DraftInput& input, void* context) + { + auto& e = *static_cast(context); + e.input = input; + const std::string_view texts[] = {input.name, input.description, input.hint}; + const size_t slots[] = {0, 3, 4}; + for (size_t i = 0; i < 3; ++i) + { + auto remaining = texts[i]; + while (!remaining.empty()) + { + size_t count = std::min(64, remaining.size()); + if (count < remaining.size()) + while (count && (static_cast(remaining[count]) & 0xc0) == 0x80) --count; + char chunk[65]; + std::memcpy(chunk, remaining.data(), count); chunk[count] = 0; + lv_textarea_add_text(e.fields[slots[i]], chunk); + remaining.remove_prefix(count); + } + } + char text[24]; + if (input.has_coordinates) + { + std::snprintf(text, sizeof(text), "%.7f", input.latitude_e7 / 10000000.0); lv_textarea_set_text(e.fields[1], text); + std::snprintf(text, sizeof(text), "%.7f", input.longitude_e7 / 10000000.0); lv_textarea_set_text(e.fields[2], text); + } + std::snprintf(text, sizeof(text), "%.1f", input.difficulty_x2 / 2.0); lv_textarea_set_text(e.fields[5], text); + std::snprintf(text, sizeof(text), "%.1f", input.terrain_x2 / 2.0); lv_textarea_set_text(e.fields[6], text); + lv_dropdown_set_selected(e.container, input.container_size); + e.input.name = e.input.description = e.input.hint = {}; }, + &editor); + if (result == ::ui::geocaching::DraftReadStatus::Pending) + { + editor.loading = true; + editor.input.id = id; + editor.dirty = false; + for (auto* field : editor.fields) setEnabled(field, false); + setEnabled(editor.container, false); + for (auto* tab : page->tabs) setEnabled(tab, false); + setEnabled(page->previous, false); + setEnabled(page->next, false); + setEnabled(page->refresh, true); + lv_label_set_text(lv_obj_get_child(page->refresh, 0), "Back"); + lv_label_set_text(page->status, "Loading local draft..."); + if (page->group) lv_group_focus_obj(page->refresh); + return; + } + if (result == ::ui::geocaching::DraftReadStatus::Failed) + { + closeEditor(); + lv_label_set_text(page->status, "Cannot read local draft"); + return; + } + } + editor.loading = false; + for (auto* field : editor.fields) setEnabled(field, true); + setEnabled(editor.container, true); + editor.dirty = !item; + for (auto* tab : page->tabs) setEnabled(tab, false); + lv_label_set_text(page->status, item && item->detail[0] ? item->detail.data() : "Editing local draft"); + lv_label_set_text(page->range, "Draft"); + lv_label_set_text(lv_obj_get_child(page->refresh, 0), "Back"); + setEnabled(page->refresh, true); + lv_label_set_text(lv_obj_get_child(page->previous, 0), "Discard"); + lv_obj_set_width(page->previous, ::ui::page_profile::current().control_button_min_width); + setEnabled(page->previous, false); + lv_obj_add_flag(page->previous, LV_OBJ_FLAG_HIDDEN); + lv_label_set_text(lv_obj_get_child(page->next, 0), "Save"); + lv_obj_set_width(page->next, ::ui::page_profile::current().control_button_min_width); + setEnabled(page->next, true); + if (page->group) lv_group_focus_obj(editor.fields[0]); +} +} // namespace + +void bind(::ui::geocaching::Source* value) +{ + if (value != source && page && page->editor && page->editor->loading) closeEditor(); + source = value; + if (page) page->valid = false; +} +void enter(void* user_data, lv_obj_t* parent) +{ + if (page || !parent) return; + page = new (std::nothrow) PageState; + if (!page) + { + if (user_data) ::ui::page::request_exit(static_cast(user_data)); + return; + } + auto& p = *page; + const auto& profile = ::ui::page_profile::current(); + if (source) source->activate(true); + p.previous_group = lv_group_get_default(); + p.group = lv_group_create(); + set_default_group(nullptr); + p.host = static_cast(user_data); + p.root = lv_obj_create(parent); + lv_obj_remove_style_all(p.root); + lv_obj_set_size(p.root, LV_PCT(100), LV_PCT(100)); + lv_obj_set_flex_flow(p.root, LV_FLEX_FLOW_COLUMN); + lv_obj_set_style_bg_opa(p.root, LV_OPA_COVER, 0); + lv_obj_set_style_bg_color(p.root, lv_color_hex(styles::kSidePanelBg), 0); + lv_obj_set_style_text_color(p.root, lv_color_hex(styles::kTextPrimary), 0); + lv_obj_set_style_text_font(p.root, ::ui::page_profile::resolve_body_font(), 0); + lv_obj_set_style_pad_row(p.root, profile.top_content_gap, 0); + lv_obj_remove_flag(p.root, LV_OBJ_FLAG_SCROLLABLE); + ::ui::widgets::top_bar_init(p.topbar, p.root); + ::ui::widgets::top_bar_set_title(p.topbar, "Geocaching"); + addFocusable(p.topbar.back_btn); + ::ui::widgets::top_bar_set_back_callback( + p.topbar, [](void*) + { + if (page && page->editor) editorBack(); + else if (page && page->details) closeDetails(); + else if (page && page->host) ::ui::page::request_exit(page->host); }, + nullptr); + auto* tabs = lv_obj_create(p.root); + lv_obj_remove_style_all(tabs); + lv_obj_set_size(tabs, LV_PCT(100), profile.filter_button_height); + lv_obj_set_style_pad_hor(tabs, 4, 0); + lv_obj_set_style_pad_column(tabs, 4, 0); + lv_obj_set_flex_flow(tabs, LV_FLEX_FLOW_ROW); + lv_obj_remove_flag(tabs, LV_OBJ_FLAG_SCROLLABLE); + const char* names[] = {"Discover", "Downloaded", "My caches"}; + for (uintptr_t i = 0; i < 3; ++i) + { + p.tabs[i] = button(tabs, names[i], profile.filter_button_height); + lv_obj_set_width(p.tabs[i], 0); + lv_obj_set_flex_grow(p.tabs[i], 1); + lv_obj_add_event_cb( + p.tabs[i], [](lv_event_t* event) + { + if (!page) return; + page->section = static_cast
(reinterpret_cast(lv_event_get_user_data(event))); + page->offset = 0; closeDetails(); }, + LV_EVENT_CLICKED, reinterpret_cast(i)); + } + p.status = lv_label_create(p.root); + lv_obj_set_width(p.status, LV_PCT(100)); + lv_label_set_long_mode(p.status, LV_LABEL_LONG_DOT); + lv_obj_set_style_pad_hor(p.status, 6, 0); + styles::apply_label_muted(p.status); + p.list = lv_obj_create(p.root); + lv_obj_remove_style_all(p.list); + styles::apply_container_main(p.list); + lv_obj_set_width(p.list, LV_PCT(100)); + lv_obj_set_height(p.list, 0); + lv_obj_set_flex_grow(p.list, 1); + lv_obj_set_flex_flow(p.list, LV_FLEX_FLOW_COLUMN); + lv_obj_set_style_pad_row(p.list, 3, 0); + lv_obj_remove_flag(p.list, LV_OBJ_FLAG_SCROLLABLE); + auto* footer = lv_obj_create(p.root); + lv_obj_remove_style_all(footer); + lv_obj_set_size(footer, LV_PCT(100), profile.control_button_height); + lv_obj_set_style_pad_hor(footer, 4, 0); + lv_obj_set_style_pad_column(footer, 4, 0); + lv_obj_set_flex_flow(footer, LV_FLEX_FLOW_ROW); + lv_obj_set_flex_align(footer, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER); + lv_obj_remove_flag(footer, LV_OBJ_FLAG_SCROLLABLE); + p.refresh = button(footer, "Refresh", profile.control_button_height); + lv_obj_set_width(p.refresh, profile.control_button_min_width); + lv_obj_add_event_cb( + p.refresh, [](lv_event_t*) + { + if (page && page->editor) editorBack(); + else if (page && page->details) closeDetails(); + else if (page && page->section == Section::Published && page->snapshot.can_create) openEditor(nullptr); + else if (page && source) { source->refresh(page->section); refreshView(); } }, + LV_EVENT_CLICKED, nullptr); + p.range = lv_label_create(footer); + lv_obj_set_flex_grow(p.range, 1); + lv_obj_set_style_text_align(p.range, LV_TEXT_ALIGN_CENTER, 0); + styles::apply_label_muted(p.range); + p.previous = button(footer, LV_SYMBOL_LEFT, profile.control_button_height); + p.next = button(footer, LV_SYMBOL_RIGHT, profile.control_button_height); + lv_obj_set_width(p.previous, profile.control_button_height); + lv_obj_set_width(p.next, profile.control_button_height); + lv_obj_add_event_cb( + p.previous, [](lv_event_t*) + { + if (page && page->editor) { if (page->editor->confirming && !page->editor->saving) closeEditor(); else previewPublication(); } + else if (page) { page->offset = page->offset >= page->window ? page->offset - page->window : 0; refreshView(); } }, + LV_EVENT_CLICKED, nullptr); + lv_obj_add_event_cb( + p.next, [](lv_event_t*) + { + if (!page) return; + if (page->editor) { saveEditor(); return; } + if (page->details) + { + ::ui::geocaching::Item item; + if (source && source->item(page->section, page->detail_index, page->detail_generation, item) && + item.id == page->detail_id && item.revision_hash == page->detail_hash && + (item.downloaded ? openMap(item) : item.can_download && source->download(item, page->detail_generation))) + { if (!map_visit) refreshView(); } + else lv_label_set_text(page->status, "Download could not start"); + return; + } + if (page->offset + page->row_count < page->snapshot.count) page->offset += page->window; + else if (page->snapshot.has_more && source && source->loadMore()) { page->offset = 0; page->valid = false; } + refreshView(); }, + LV_EVENT_CLICKED, nullptr); + lv_label_set_text(p.status, " "); + lv_obj_update_layout(p.root); + const auto available = std::max(0, lv_obj_get_content_height(p.list)); + p.window = std::max(1, std::min(kVisibleRows, (available + 3) / (profile.list_item_height + 3))); + refreshView(); + if (p.group) + { + set_default_group(p.group); + lv_group_set_editing(p.group, false); + lv_group_focus_obj(p.tabs[static_cast(p.section)]); + } + else set_default_group(p.previous_group); + p.timer = lv_timer_create([](lv_timer_t*) + { refreshView(); }, + 500, nullptr); +} +void exit(void*, lv_obj_t*) +{ + if (!page) return; + if (page->editor && page->editor->loading && source) source->cancelDraftRead(page->editor->input.id); + delete page->editor; + page->editor = nullptr; + if (map_visit) + { + lv_async_call_cancel(returnFromMap, nullptr); + ::gps::ui::runtime::exit(map_visit->parent); + delete map_visit; + map_visit = nullptr; + } + if (page->timer) lv_timer_delete(page->timer); + if (page->group && lv_group_get_default() == page->group) set_default_group(page->previous_group); + if (page->root) lv_obj_delete(page->root); + if (page->group) lv_group_delete(page->group); + if (source) source->activate(false); + delete page; + page = nullptr; +} +} // namespace geocaching::ui::shell diff --git a/modules/ui_shared/src/ui/screens/gps/gps_page_runtime.cpp b/modules/ui_shared/src/ui/screens/gps/gps_page_runtime.cpp index 476be1db..257364b1 100644 --- a/modules/ui_shared/src/ui/screens/gps/gps_page_runtime.cpp +++ b/modules/ui_shared/src/ui/screens/gps/gps_page_runtime.cpp @@ -6,6 +6,7 @@ using Projection = gps::ui::shell::Projection; #include "app/app_config.h" #include "app/app_facade_access.h" #include "gps/domain/gps_diagnostics.h" +#include "gps/gpx/attribute_reader.h" #include "platform/ui/device_runtime.h" #include "platform/ui/gps_runtime.h" #include "platform/ui/route_storage.h" @@ -35,6 +36,7 @@ using Projection = gps::ui::shell::Projection; #include "ui_presentation/map/map_overlay_snapshot.h" #include "ui_presentation/map/map_workspace_model.h" +#include "ui/menu/dashboard/dashboard_style.h" #include #include #include @@ -166,6 +168,7 @@ int s_map_pan_y = 0; bool s_map_view_initialized = false; bool s_map_info_visible = true; ::ui::map::MapOverlaySnapshot* s_overlay_snapshot = nullptr; +const ::gps::ui::runtime::MapTarget* s_map_target = nullptr; Projection s_projection = Projection::Map; bool s_gps_power_lease_active = false; lv_obj_t* s_gps_status_label = nullptr; @@ -381,7 +384,7 @@ bool has_valid_viewport_center(const ::ui::map::MapViewport& viewport) { return std::isfinite(viewport.center_lat) && std::isfinite(viewport.center_lon) && - (viewport.center_lat != 0.0 || viewport.center_lon != 0.0); + (s_map_target || viewport.center_lat != 0.0 || viewport.center_lon != 0.0); } void sync_workspace_layers_from_renderer() @@ -1683,6 +1686,29 @@ void sync_map_notice_overlay() lv_obj_move_foreground(s_map_notice_panel); return; } + if (s_map_target) + { + char notice[64]{}; + double lat = 0.0, lon = 0.0; + if (current_fix_lat_lon(lat, lon)) + { + const double target_lat = s_map_target->latitude_e7 / 10000000.0; + const double target_lon = s_map_target->longitude_e7 / 10000000.0; + const double meters = ::ui::menu::dashboard::haversine_m(lat, lon, target_lat, target_lon); + const float bearing = ::ui::menu::dashboard::bearing_between(lat, lon, target_lat, target_lon); + char distance[20]{}; + ::ui::menu::dashboard::format_distance(meters, distance, sizeof(distance)); + if (meters < 1.0) std::snprintf(notice, sizeof(notice), "Cache: <1 m (straight line)"); + else std::snprintf(notice, sizeof(notice), "Cache: %s %s %.0f deg N", distance, + ::ui::menu::dashboard::compass_rose(bearing), static_cast(bearing)); + } + else std::snprintf(notice, sizeof(notice), "Cache: waiting for GPS fix"); + set_compact_label(s_map_notice_label, notice); + lv_obj_set_style_bg_color(s_map_notice_panel, lv_color_hex(0x25170D), 0); + lv_obj_clear_flag(s_map_notice_panel, LV_OBJ_FLAG_HIDDEN); + lv_obj_move_foreground(s_map_notice_panel); + return; + } lv_obj_set_style_bg_color(s_map_notice_panel, lv_color_hex(0x25170D), 0); lv_obj_add_flag(s_map_notice_panel, LV_OBJ_FLAG_HIDDEN); } @@ -1948,19 +1974,7 @@ bool parse_double_token(const std::string& token, double& out) bool parse_attr_double(const std::string& line, const char* key, double& out) { - const std::string token = std::string(key) + "=\""; - const std::size_t start = line.find(token); - if (start == std::string::npos) - { - return false; - } - const std::size_t value_start = start + token.size(); - const std::size_t value_end = line.find('"', value_start); - if (value_end == std::string::npos || value_end <= value_start) - { - return false; - } - return parse_double_token(line.substr(value_start, value_end - value_start), out); + return key && ::gps::gpx::readDoubleAttribute(line, key, out); } bool ascii_equal_ignore_case(const std::string& value, const char* expected) @@ -3103,10 +3117,43 @@ bool load_map_track_file_impl(const char* path, bool show_fail_toast) return true; } +bool select_cache_at(lv_point_t point) +{ + if (!s_map_target || !s_map_target->select_overlay || !s_overlay_snapshot) return false; + const ::ui::map::MapOverlayItem* closest = nullptr; + int64_t best = 24 * 24 + 1; + for (size_t index = 0; index < s_overlay_snapshot->item_count; ++index) + { + const auto& item = s_overlay_snapshot->items[index]; + if (item.kind != ::ui::map::MapOverlayKind::Geocache || !item.visible || !item.point.valid) continue; + lv_point_t projected; + if (!::ui::widgets::map::project_point(s_map_runtime, {true, item.point.lat, item.point.lon}, projected)) continue; + const int64_t dx = int64_t(point.x) - projected.x, dy = int64_t(point.y) - projected.y; + const auto distance = dx * dx + dy * dy; + if (distance < best) + { + best = distance; + closest = &item; + } + } + if (!closest) return false; + s_map_target->select_overlay(s_map_target->overlay_context, closest->stable_id); + return true; +} + void map_gesture_callback(const ::ui::widgets::map::GestureEvent& event, void*) { switch (event.phase) { + case ::ui::widgets::map::GesturePhase::Tapped: + { + const auto* root = ::ui::widgets::map::widgets(s_map_runtime).root; + if (!root) break; + lv_area_t bounds; + lv_obj_get_coords(root, &bounds); + (void)select_cache_at({event.point.x - bounds.x1, event.point.y - bounds.y1}); + break; + } case ::ui::widgets::map::GesturePhase::Pressed: s_map_drag_start_pan_x = s_map_pan_x; s_map_drag_start_pan_y = s_map_pan_y; @@ -3179,8 +3226,22 @@ void refresh_view() sync_workspace_layers_from_renderer(); auto snapshot = map_workspace_model().snapshot(); (void)map_overlay_source().buildMapOverlaySnapshot(*s_overlay_snapshot); + if (s_map_target && s_overlay_snapshot->item_count < ::ui::map::MapOverlaySnapshot::kMaxItems) + { + auto& target = s_overlay_snapshot->items[s_overlay_snapshot->item_count++]; + target = ::ui::map::MapOverlayItem{}; + target.kind = ::ui::map::MapOverlayKind::Geocache; + target.style = ::ui::map::MapOverlayStyle::Warning; + target.point.valid = true; + target.point.lat = s_map_target->latitude_e7 / 10000000.0; + target.point.lon = s_map_target->longitude_e7 / 10000000.0; + target.selected = target.visible = true; + ::ui::copyText(target.label, s_map_target->name); + } append_route_image_overlay(*s_overlay_snapshot); append_track_overlay(*s_overlay_snapshot); + if (s_map_target && s_map_target->append_overlays) + s_map_target->append_overlays(s_map_target->overlay_context, *s_overlay_snapshot); if (!s_map_info_visible) { keep_only_current_position_overlay(*s_overlay_snapshot); @@ -3915,6 +3976,14 @@ void on_map_control_clicked(lv_event_t* e) bool handle_map_key(uint32_t key, lv_event_t* e) { + if (s_map_target && (key == 'g' || key == 'G')) + { + auto* root = ::ui::widgets::map::widgets(s_map_runtime).root; + if (root && !select_cache_at({lv_obj_get_width(root) / 2, lv_obj_get_height(root) / 2})) + set_map_notice("No cache near map center", 1200); + consume_key_event(e); + return true; + } if (::ui::widgets::route_image_strip::handle_key(s_route_image_strip, key)) { consume_key_event(e); @@ -4491,9 +4560,28 @@ void enter(const shell::Host* host, lv_obj_t* parent, shell::Projection projecti } } +bool enter_target(const shell::Host* host, lv_obj_t* parent, const MapTarget& target) +{ + if (!parent || s_root || target.latitude_e7 < -900000000 || target.latitude_e7 > 900000000 || + target.longitude_e7 < -1800000000 || target.longitude_e7 >= 1800000000 || !is_available()) return false; + enter(host, parent, shell::Projection::Map); + if (!s_root) return false; + s_map_target = ⌖ + auto viewport = map_workspace_model().viewport(); + viewport.center_lat = target.latitude_e7 / 10000000.0; + viewport.center_lon = target.longitude_e7 / 10000000.0; + viewport.zoom = current_map_zoom(); + (void)map_workspace_model().setViewport(viewport); + s_map_pan_x = s_map_pan_y = 0; + set_map_notice("Tap cache or G near map center", 3500); + refresh_view(); + return true; +} + void exit(lv_obj_t* parent) { (void)parent; + s_map_target = nullptr; if (s_timer) { diff --git a/modules/ui_shared/src/ui/widgets/map/map_viewport.cpp b/modules/ui_shared/src/ui/widgets/map/map_viewport.cpp index 281ce0ae..b196e9a4 100644 --- a/modules/ui_shared/src/ui/widgets/map/map_viewport.cpp +++ b/modules/ui_shared/src/ui/widgets/map/map_viewport.cpp @@ -362,6 +362,10 @@ void gesture_surface_event_cb(lv_event_t* e) point); } + else if (code == LV_EVENT_RELEASED) + { + emit_gesture_event(*impl, GesturePhase::Tapped, point); + } reset_gesture_state(*impl); break; diff --git a/modules/ui_shared/tests/geocaching/CMakeLists.txt b/modules/ui_shared/tests/geocaching/CMakeLists.txt new file mode 100644 index 00000000..0a439d80 --- /dev/null +++ b/modules/ui_shared/tests/geocaching/CMakeLists.txt @@ -0,0 +1,25 @@ +cmake_minimum_required(VERSION 3.20) +project(GeocachingPageTests LANGUAGES C CXX) +enable_testing() +get_filename_component(REPO_ROOT "${CMAKE_CURRENT_LIST_DIR}/../../../.." ABSOLUTE) +set(LVGL_SOURCE_DIR "${REPO_ROOT}/.pio/libdeps/tlora_pager_lr1121/lvgl" CACHE PATH "Firmware LVGL source") +file(GLOB_RECURSE LVGL_SOURCES CONFIGURE_DEPENDS "${LVGL_SOURCE_DIR}/src/*.c") +add_library(page_test_lvgl STATIC ${LVGL_SOURCES}) +target_include_directories(page_test_lvgl PUBLIC "${LVGL_SOURCE_DIR}") +target_compile_definitions(page_test_lvgl PUBLIC LV_CONF_SKIP=1 LV_USE_STDLIB_MALLOC=0 + LV_MEM_SIZE=1048576 LV_USE_STDLIB_STRING=1 LV_USE_STDLIB_SPRINTF=1 + LV_USE_SNAPSHOT=1 LV_FONT_MONTSERRAT_12=1 LV_FONT_MONTSERRAT_16=1) +add_executable(geocaching_page_test test_page.cpp + "${REPO_ROOT}/modules/ui_shared/src/ui/screens/geocaching/geocaching_page_shell.cpp" + "${REPO_ROOT}/modules/ui_shared/src/ui/components/two_pane_styles.cpp" + "${REPO_ROOT}/modules/ui_shared/src/ui/widgets/top_bar.cpp" + "${REPO_ROOT}/modules/ui_shared/src/ui/page/page_profile.cpp") +target_include_directories(geocaching_page_test PRIVATE stubs + "${REPO_ROOT}/modules/ui_shared/include" "${REPO_ROOT}/modules/ui_presentation/include") +target_compile_features(geocaching_page_test PRIVATE cxx_std_17) +target_link_libraries(geocaching_page_test PRIVATE page_test_lvgl) +if(MSVC) + target_compile_options(geocaching_page_test PRIVATE /utf-8) +endif() +add_test(NAME geocaching_page_480 COMMAND geocaching_page_test 480 222 "${CMAKE_CURRENT_BINARY_DIR}/pager") +add_test(NAME geocaching_page_320 COMMAND geocaching_page_test 320 240 "${CMAKE_CURRENT_BINARY_DIR}/tdeck") diff --git a/modules/ui_shared/tests/geocaching/stubs/ui/app_runtime.h b/modules/ui_shared/tests/geocaching/stubs/ui/app_runtime.h new file mode 100644 index 00000000..029f7a61 --- /dev/null +++ b/modules/ui_shared/tests/geocaching/stubs/ui/app_runtime.h @@ -0,0 +1,3 @@ +#pragma once +#include "lvgl.h" +inline void set_default_group(lv_group_t* group) { lv_group_set_default(group); } diff --git a/modules/ui_shared/tests/geocaching/stubs/ui/assets/fonts/font_utils.h b/modules/ui_shared/tests/geocaching/stubs/ui/assets/fonts/font_utils.h new file mode 100644 index 00000000..30907515 --- /dev/null +++ b/modules/ui_shared/tests/geocaching/stubs/ui/assets/fonts/font_utils.h @@ -0,0 +1,9 @@ +#pragma once +#include "lvgl.h" +namespace ui::fonts +{ +inline void apply_content_font(lv_obj_t* object, const char*, const lv_font_t* font) +{ lv_obj_set_style_text_font(object, font, 0); } +inline void apply_font(lv_obj_t* object, const lv_font_t* font) +{ lv_obj_set_style_text_font(object, font, 0); } +} diff --git a/modules/ui_shared/tests/geocaching/test_page.cpp b/modules/ui_shared/tests/geocaching/test_page.cpp new file mode 100644 index 00000000..4a4788dd --- /dev/null +++ b/modules/ui_shared/tests/geocaching/test_page.cpp @@ -0,0 +1,338 @@ +#include "ui/components/two_pane_styles.h" +#include "ui/page/page_profile.h" +#include "ui/screens/geocaching/geocaching_page_shell.h" +#include "ui/screens/gps/gps_page_runtime.h" +#include "ui/widgets/top_bar_power_presenter.h" +#include +#include +#include +#include +#include +#include + +namespace gps::ui::runtime +{ +const shell::Host* test_map_host = nullptr; +MapTarget test_map_target; +bool enter_target(const shell::Host* host, lv_obj_t*, const MapTarget& target) +{ + test_map_host = host; + test_map_target = target; + return true; +} +void exit(lv_obj_t*) { test_map_host = nullptr; } +} // namespace gps::ui::runtime + +namespace ui::widgets::top_bar_power +{ +void bind(TopBar& bar) { lv_label_set_text(bar.right_label, LV_SYMBOL_BATTERY_FULL " 100%"); } +void unbind(TopBar&) {} +} // namespace ui::widgets::top_bar_power +struct TestSource : ui::geocaching::Source +{ + ui::geocaching::DraftInput draft; + std::string draft_name, draft_description, draft_hint; + bool has_draft = false, fail_save = false; + unsigned pending_reads = 0; + unsigned cancelled_reads = 0; + bool fail_read = false; + unsigned publications = 0; + uint8_t current_author = 7; + unsigned reads = 0, opens = 0, refreshes = 0, downloads = 0; + size_t saved_index = SIZE_MAX; + size_t last_open = 0; + uint64_t generation = 1; + void snapshot(ui::geocaching::Section section, ui::geocaching::Snapshot& out) override + { + out = {}; + out.generation = generation; + out.count = 20; + out.can_refresh = true; + if (section == ui::geocaching::Section::Published) + { + out.count = has_draft ? 1 : 0; + out.can_create = true; + } + std::snprintf(out.status.data(), out.status.size(), "20 shared caches"); + } + bool item(ui::geocaching::Section section, size_t index, uint64_t expected, ui::geocaching::Item& out) override + { + ++reads; + if (index >= 20 || expected != generation) return false; + out = {}; + if (section == ui::geocaching::Section::Published) + { + if (!has_draft || index) return false; + out.is_draft = true; + out.edit_generation = draft.generation; + std::memcpy(out.id.data(), draft.id.data(), 16); + std::snprintf(out.name.data(), out.name.size(), "%s", draft_name.c_str()); + return true; + } + out.id[0] = static_cast(index); + out.downloaded = saved_index == index; + out.can_download = !out.downloaded; + std::snprintf(out.name.data(), out.name.size(), "Cache %02u - woodland trail", unsigned(index + 1)); + std::snprintf(out.detail.data(), out.detail.size(), "30.5000000, 120.5000000\nDifficulty 2.0 / Terrain 2.5\nDirectory preview - not yet downloaded"); + return true; + } + void refresh(ui::geocaching::Section) override + { + ++refreshes; + ++generation; + } + void open(const ui::geocaching::Item& item, uint64_t expected) override + { + if (expected == generation) + { + ++opens; + last_open = item.id[0]; + } + } + bool download(const ui::geocaching::Item& item, uint64_t expected) override + { + if (expected != generation) return false; + ++downloads; + saved_index = item.id[0]; + ++generation; + return true; + } + bool saveDraft(ui::geocaching::DraftInput& input) override + { + if (fail_save) return true; + input.id.fill(0x42); + draft = input; + ++draft.generation; + draft_name = input.name; + draft_description = input.description; + draft_hint = input.hint; + draft.name = draft_name; + draft.description = draft_description; + draft.hint = draft_hint; + has_draft = true; + ++generation; + return true; + } + ui::geocaching::DraftSaveStatus draftSaveStatus(const std::array&, uint64_t) override + { + return fail_save ? ui::geocaching::DraftSaveStatus::Failed : ui::geocaching::DraftSaveStatus::Saved; + } + ui::geocaching::DraftReadStatus readDraft(const std::array& id, void (*sink)(const ui::geocaching::DraftInput&, void*), void* context) override + { + if (pending_reads) + { + --pending_reads; + return ui::geocaching::DraftReadStatus::Pending; + } + if (fail_read) return ui::geocaching::DraftReadStatus::Failed; + if (!has_draft || id != draft.id) return ui::geocaching::DraftReadStatus::Failed; + sink(draft, context); + return ui::geocaching::DraftReadStatus::Ready; + } + void cancelDraftRead(const std::array& id) override + { + if (id == draft.id) ++cancelled_reads; + } + bool publicationAuthor(const std::array& id, uint64_t version, std::array& author, uint32_t* from, uint32_t* to) override + { + if (!has_draft || id != draft.id || version != draft.generation) return false; + author.fill(current_author); + if (from) *from = 0; + if (to) *to = 1; + return true; + } + bool publishDraft(const std::array& id, uint64_t version, const std::array& expected, uint32_t revision) override + { + std::array current; + if (!publicationAuthor(id, version, current, nullptr, nullptr) || current != expected || revision != 1) return false; + ++publications; + return true; + } +}; +bool save(const std::string& path, lv_obj_t* screen) +{ + lv_obj_update_layout(screen); + auto* image = lv_snapshot_take(screen, LV_COLOR_FORMAT_RGB888); + if (!image) return false; + std::ofstream out(path, std::ios::binary); + out << "P6\n" + << image->header.w << " " << image->header.h << "\n255\n"; + for (unsigned y = 0; y < image->header.h; ++y) + for (unsigned x = 0; x < image->header.w; ++x) + { + const auto* pixel = image->data + y * image->header.stride + x * 3; + const char rgb[] = {char(pixel[2]), char(pixel[1]), char(pixel[0])}; + out.write(rgb, 3); + } + lv_draw_buf_destroy(image); + return out.good(); +} +int main(int argc, char** argv) +{ + if (argc != 4) return 1; + const int width = std::atoi(argv[1]), height = std::atoi(argv[2]); + lv_init(); + auto* display = lv_display_create(width, height); + std::vector pixels(width * 32 * 4); + lv_display_set_color_format(display, LV_COLOR_FORMAT_XRGB8888); + lv_display_set_buffers(display, pixels.data(), nullptr, pixels.size(), LV_DISPLAY_RENDER_MODE_PARTIAL); + lv_display_set_flush_cb(display, [](lv_display_t* display, const lv_area_t*, uint8_t*) + { lv_display_flush_ready(display); }); + const auto profile = width == 480 ? ui::page_profile::make_pager_profile() : ui::page_profile::make_tdeck_profile(); + ui::page_profile::set_active_profile(&profile); + auto* screen = lv_screen_active(); + auto* old_group = lv_group_create(); + lv_group_set_default(old_group); + unsigned exits = 0; + ui::page::Host host{&exits, [](void* context) + { ++*static_cast(context); }}; + geocaching::ui::shell::enter(&host, screen); + auto* root = lv_obj_get_child(screen, 0); + if (!root || !save(std::string(argv[3]) + "-empty.ppm", screen)) return 2; + auto* tabs = lv_obj_get_child(root, 1); + for (unsigned i = 0; i < 3; ++i) + { + auto* tab = lv_obj_get_child(tabs, i); + if (lv_color_to_u32(lv_obj_get_style_bg_color(tab, LV_PART_MAIN)) == lv_color_to_u32(lv_palette_main(LV_PALETTE_BLUE))) return 3; + } + TestSource source; + geocaching::ui::shell::bind(&source); + lv_tick_inc(501); + lv_timer_handler(); + auto* list = lv_obj_get_child(root, 3); + const auto visible = lv_obj_get_child_count(list); + if (visible < 2 || visible > 4 || source.reads != visible) return 4; + if (!save(std::string(argv[3]) + "-list.ppm", screen)) return 5; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + if (source.opens != 1 || source.last_open != 0) return 6; + if (!save(std::string(argv[3]) + "-detail.ppm", screen)) return 11; + auto* details_footer = lv_obj_get_child(root, 4); + lv_obj_send_event(lv_obj_get_child(details_footer, 3), LV_EVENT_CLICKED, nullptr); + if (source.downloads != 1 || source.saved_index != 0 || + lv_obj_has_state(lv_obj_get_child(details_footer, 3), LV_STATE_DISABLED)) return 14; + lv_obj_send_event(lv_obj_get_child(details_footer, 3), LV_EVENT_CLICKED, nullptr); + if (!gps::ui::runtime::test_map_host || !lv_obj_has_flag(root, LV_OBJ_FLAG_HIDDEN) || + std::strcmp(gps::ui::runtime::test_map_target.name, "Cache 01 - woodland trail")) return 15; + source.saved_index = 1; + ++source.generation; + ui::map::MapOverlaySnapshot overlays; + const auto& map_target = gps::ui::runtime::test_map_target; + if (!map_target.append_overlays) return 17; + map_target.append_overlays(map_target.overlay_context, overlays); + if (overlays.item_count != 1 || overlays.items[0].kind != ui::map::MapOverlayKind::Geocache) return 18; + overlays.item_count = ui::map::MapOverlaySnapshot::kMaxItems; + map_target.append_overlays(map_target.overlay_context, overlays); + if (!overlays.truncated || overlays.item_count != ui::map::MapOverlaySnapshot::kMaxItems) return 19; + if (!map_target.select_overlay || overlays.items[0].stable_id != 2) return 20; + map_target.select_overlay(map_target.overlay_context, overlays.items[0].stable_id); + ui::page::request_exit(gps::ui::runtime::test_map_host); + ui::page::request_exit(gps::ui::runtime::test_map_host); + lv_tick_inc(10); + lv_timer_handler(); + if (gps::ui::runtime::test_map_host || lv_obj_has_flag(root, LV_OBJ_FLAG_HIDDEN) || exits) return 16; + if (std::strcmp(lv_label_get_text(lv_obj_get_child(list, 0)), "Cache 02 - woodland trail")) return 21; + lv_obj_send_event(lv_obj_get_child(lv_obj_get_child(root, 0), 0), LV_EVENT_CLICKED, nullptr); + if (exits || lv_obj_get_child_count(list) != visible) return 12; + auto* footer = lv_obj_get_child(root, 4); + lv_obj_send_event(lv_obj_get_child(footer, 3), LV_EVENT_CLICKED, nullptr); + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + if (source.opens != 2 || source.last_open != visible) return 7; + lv_obj_send_event(lv_obj_get_child(lv_obj_get_child(root, 0), 0), LV_EVENT_CLICKED, nullptr); + if (exits) return 13; + lv_obj_send_event(lv_obj_get_child(lv_obj_get_child(root, 0), 0), LV_EVENT_CLICKED, nullptr); + if (exits != 1) return 8; + geocaching::ui::shell::exit(nullptr, screen); + if (lv_obj_get_child_count(screen) || lv_group_get_default() != old_group) return 9; + geocaching::ui::shell::enter(&host, screen); + root = lv_obj_get_child(screen, 0); + tabs = lv_obj_get_child(root, 1); + lv_obj_send_event(lv_obj_get_child(tabs, 2), LV_EVENT_CLICKED, nullptr); + footer = lv_obj_get_child(root, 4); + lv_obj_send_event(lv_obj_get_child(footer, 0), LV_EVENT_CLICKED, nullptr); + list = lv_obj_get_child(root, 3); + auto* name_field = lv_obj_get_child(list, 1); + lv_textarea_set_text(name_field, "林间宝藏"); + lv_textarea_set_text(lv_obj_get_child(list, 3), "30.5"); + lv_textarea_set_text(lv_obj_get_child(list, 5), "120.5"); + const std::string description = std::string(63, 'a') + "宝藏"; + lv_textarea_set_text(lv_obj_get_child(list, 7), description.c_str()); + if (!save(std::string(argv[3]) + "-editor.ppm", screen)) return 27; + source.fail_save = true; + lv_obj_send_event(lv_obj_get_child(footer, 3), LV_EVENT_CLICKED, nullptr); + lv_tick_inc(600); + lv_timer_handler(); + if (lv_obj_has_state(name_field, LV_STATE_DISABLED) || std::strcmp(lv_textarea_get_text(name_field), "林间宝藏")) return 22; + source.fail_save = false; + lv_obj_send_event(lv_obj_get_child(footer, 3), LV_EVENT_CLICKED, nullptr); + lv_tick_inc(600); + lv_timer_handler(); + if (!source.has_draft || source.draft.latitude_e7 != 305000000 || source.draft.longitude_e7 != 1205000000) return 23; + source.pending_reads = 1; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + if (!lv_obj_has_state(lv_obj_get_child(list, 1), LV_STATE_DISABLED) || + !lv_obj_has_state(lv_obj_get_child(footer, 3), LV_STATE_DISABLED)) return 33; + lv_tick_inc(600); + lv_timer_handler(); + if (lv_obj_has_state(lv_obj_get_child(list, 1), LV_STATE_DISABLED)) return 34; + if (std::strcmp(lv_textarea_get_text(lv_obj_get_child(list, 1)), "林间宝藏") || + description != lv_textarea_get_text(lv_obj_get_child(list, 7))) return 24; + lv_textarea_set_text(lv_obj_get_child(list, 1), "Discard this"); + lv_obj_send_event(lv_obj_get_child(footer, 0), LV_EVENT_CLICKED, nullptr); + if (lv_obj_has_flag(lv_obj_get_child(footer, 2), LV_OBJ_FLAG_HIDDEN)) return 25; + lv_obj_send_event(lv_obj_get_child(footer, 2), LV_EVENT_CLICKED, nullptr); + if (source.draft_name != "林间宝藏") return 26; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + lv_tick_inc(600); + lv_timer_handler(); + lv_obj_send_event(lv_obj_get_child(footer, 2), LV_EVENT_CLICKED, nullptr); + if (!std::strstr(lv_label_get_text(lv_obj_get_child(list, 0)), "publicly") || source.publications) return 28; + if (!save(std::string(argv[3]) + "-publish.ppm", screen)) return 29; + lv_obj_send_event(lv_obj_get_child(footer, 0), LV_EVENT_CLICKED, nullptr); + if (source.publications || lv_obj_has_state(lv_obj_get_child(list, 1), LV_STATE_DISABLED)) return 30; + lv_obj_send_event(lv_obj_get_child(footer, 2), LV_EVENT_CLICKED, nullptr); + ++source.current_author; + lv_obj_send_event(lv_obj_get_child(footer, 3), LV_EVENT_CLICKED, nullptr); + if (source.publications || !std::strstr(lv_label_get_text(lv_obj_get_child(root, 2)), "not started")) return 31; + lv_obj_send_event(lv_obj_get_child(footer, 0), LV_EVENT_CLICKED, nullptr); + lv_obj_send_event(lv_obj_get_child(footer, 2), LV_EVENT_CLICKED, nullptr); + lv_obj_send_event(lv_obj_get_child(footer, 3), LV_EVENT_CLICKED, nullptr); + if (source.publications != 1 || lv_obj_get_child_count(list) != 1) return 32; + source.pending_reads = 2; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + lv_obj_send_event(lv_obj_get_child(footer, 0), LV_EVENT_CLICKED, nullptr); + lv_tick_inc(600); + lv_timer_handler(); + if (lv_obj_get_child_count(list) != 1 || source.pending_reads != 1 || source.cancelled_reads != 1) return 35; + source.pending_reads = 0; + source.fail_read = true; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + if (lv_obj_get_child_count(list) != 1 || !std::strstr(lv_label_get_text(lv_obj_get_child(root, 2)), "Cannot read")) return 36; + source.fail_read = false; + source.pending_reads = 2; + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + geocaching::ui::shell::bind(nullptr); + if (source.cancelled_reads != 2 || lv_obj_get_child_count(list) != 1) return 37; + geocaching::ui::shell::bind(&source); + source.pending_reads = 2; + lv_tick_inc(600); + lv_timer_handler(); + lv_obj_send_event(lv_obj_get_child(list, 0), LV_EVENT_CLICKED, nullptr); + geocaching::ui::shell::exit(nullptr, screen); + if (source.cancelled_reads != 3) return 38; + // Warm LVGL style caches before checking repeated page lifetime cleanup. + geocaching::ui::shell::enter(&host, screen); + geocaching::ui::shell::exit(nullptr, screen); + lv_mem_monitor_t baseline{}, after{}; + lv_mem_monitor(&baseline); + for (unsigned i = 0; i < 20; ++i) + { + geocaching::ui::shell::enter(&host, screen); + geocaching::ui::shell::exit(nullptr, screen); + } + lv_mem_monitor(&after); + if (after.free_size != baseline.free_size || lv_obj_get_child_count(screen)) return 10; + geocaching::ui::shell::bind(nullptr); + lv_group_delete(old_group); + std::printf("%dx%d: visible=%u, no LVGL growth after 20 enter/exit cycles\n", width, height, unsigned(visible)); + return 0; +} diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h index e6bd874c..7369566d 100644 --- a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h @@ -9,6 +9,7 @@ #include "chat/infra/lxmf/lxmf_wire.h" #include "chat/infra/mesh_incoming_queue.h" #include "chat/ports/i_mesh_adapter.h" +#include "chat/ports/i_geocaching_transport.h" #include "chat/ports/i_mesh_peer_directory.h" #include "platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter_scratch.h" #include "platform/esp/arduino_common/chat/infra/lxmf/lxmf_announce_ingestor.h" @@ -45,13 +46,30 @@ class LxmfAdapter : public IMeshAdapter, private runtime::IPeerProjectionSink { public: explicit LxmfAdapter(LoraBoard& board, - IMeshPeerDirectory* peer_directory = nullptr); + IMeshPeerDirectory* peer_directory = nullptr, + bool owns_integrated_radio = true); static void* operator new(std::size_t size); static void operator delete(void* ptr) noexcept; static void operator delete(void* ptr, std::size_t size) noexcept; MeshCapabilities getCapabilities() const override; + bool getGeocachingAuthorKey(uint8_t out[64]) const; + bool signGeocachingRecord(ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written); + void setGeocachingAnnouncementHandler(GeocachingAnnouncementHandler handler, void* context) + { + geocaching_announcement_handler_ = handler; + geocaching_announcement_context_ = context; + } + void setGeocachingDeliveryHandler(CustomDeliveryHandler handler, void* context) + { + geocaching_handler_ = handler; + geocaching_handler_context_ = context; + } + MeshSendResult sendCustomDataToDestination(const uint8_t destination_hash[16], + const char* custom_type, ByteSpan data, + bool response, std::array* accepted_lxmf_hash = nullptr); bool sendText(ChannelId channel, const std::string& text, MessageId* out_msg_id, NodeId peer = 0) override; MeshSendResult sendTextDetailed(ChannelId channel, const std::string& text, @@ -105,6 +123,10 @@ class LxmfAdapter : public IMeshAdapter, private runtime::IPeerProjectionSink void processSendQueue() override; private: + CustomDeliveryHandler geocaching_handler_ = nullptr; + GeocachingAnnouncementHandler geocaching_announcement_handler_ = nullptr; + void* geocaching_announcement_context_ = nullptr; + void* geocaching_handler_context_ = nullptr; using PeerInfo = runtime::PeerInfo; using PathEntry = runtime::PathEntry; using PacketFilterEntry = runtime::PacketFilterEntry; @@ -177,6 +199,7 @@ class LxmfAdapter : public IMeshAdapter, private runtime::IPeerProjectionSink runtime::PropagationClient propagation_client_; runtime::LxstTelephonyClient lxst_telephony_client_; runtime::PeerDirectoryService peer_directory_service_; + const bool geocaching_only_; runtime::LxmfDeliveryNotifier delivery_notifier_; std::string user_long_name_; std::string user_short_name_; @@ -336,7 +359,8 @@ class LxmfAdapter : public IMeshAdapter, private runtime::IPeerProjectionSink bool update_favorite, bool favorite) const; PeerInfo* rememberPeerIdentity(const uint8_t combined_pub[reticulum::kCombinedPublicKeySize], - const char* display_name = nullptr); + const char* display_name = nullptr, + bool publish_contact = true); void pumpPendingPeerUpdates(); void publishPeerUpdate(const PeerInfo& peer) override; void loadPersistedPeers(); diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h index 7ea30fb1..39af951a 100644 --- a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/mesh_adapter_router.h @@ -7,6 +7,8 @@ #include "chat/domain/chat_types.h" #include "chat/infra/mesh_adapter_router_core.h" +#include "chat/infra/lxmf/lxmf_wire.h" +#include #include "chat/ports/i_incoming_delivery_commit_port.h" #include "chat/ports/i_mesh_adapter.h" #include "freertos/FreeRTOS.h" @@ -14,6 +16,11 @@ namespace chat { +namespace lxmf +{ +struct GeocachingAnnouncementView; +struct CustomDeliveryView; +} class MeshAdapterRouter : public IMeshAdapter, public IIncomingDeliveryCommitPort @@ -23,6 +30,28 @@ class MeshAdapterRouter : public IMeshAdapter, ~MeshAdapterRouter() override; bool installBackend(MeshProtocol protocol, std::unique_ptr backend) override; + bool installServiceBackend(MeshProtocol protocol, std::unique_ptr backend); + std::unique_ptr takeServiceBackend(MeshProtocol protocol, const IMeshAdapter* expected = nullptr); + // Retire a cached former chat instance, never an active/shared service. + std::unique_ptr takeInactiveReticulumCache(); + bool processServiceQueue(MeshProtocol protocol); + // Worker-thread call. Resolves the current instance while holding the + // router lock; callers must not retain a backend pointer across switches. + MeshSendResult sendGeocachingData(const uint8_t destination_hash[16], + lxmf::ByteSpan data, bool response = false, + std::array* accepted_lxmf_hash = nullptr, + const uint8_t expected_source[16] = nullptr); + bool getGeocachingDispatchDestination(uint8_t out[16]); + bool getGeocachingAuthorKey(uint8_t out[64]); + // The storage owner must reserve the author revision before publication. + bool signGeocachingRecord(lxmf::ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written); + // Callbacks run under the router lock: enqueue/copy bounded work, never + // reenter the router. Delivery true means durable acceptance, not queued + // volatile work. Unbind before destroying the context object. + bool bindGeocachingHandlers(void (*announcement)(const lxmf::GeocachingAnnouncementView&, void*), + bool (*delivery)(const lxmf::CustomDeliveryView&, void*), + void* context); bool hasBackend() const override; MeshProtocol backendProtocol() const override; IMeshAdapter* backendForProtocol(MeshProtocol protocol) override; @@ -99,6 +128,12 @@ class MeshAdapterRouter : public IMeshAdapter, mutable SemaphoreHandle_t mutex_ = nullptr; chat::MeshAdapterRouterCore core_; + IGeocachingTransport* geocachingTransportLocked(); + bool geocachingDestinationLocked(uint8_t out[16]); + void applyGeocachingHandlers(MeshProtocol protocol, IMeshAdapter* backend); + void (*geocaching_announcement_)(const lxmf::GeocachingAnnouncementView&, void*) = nullptr; + bool (*geocaching_delivery_)(const lxmf::CustomDeliveryView&, void*) = nullptr; + void* geocaching_context_ = nullptr; }; } // namespace chat diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h index 630f8d1b..efb7a15e 100644 --- a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h @@ -12,7 +12,9 @@ #include "chat/ports/i_incoming_delivery_commit_port.h" #include "chat/ports/i_mesh_adapter.h" +#include "chat/ports/i_geocaching_transport.h" #include "chat/ports/i_mesh_peer_directory.h" +#include "chat/infra/lxmf/lxmf_wire.h" #include @@ -21,6 +23,8 @@ class LoraBoard; namespace chat::lxmf { class LxmfAdapter; +struct CustomDeliveryView; +struct GeocachingAnnouncementView; } namespace chat::reticulum @@ -28,18 +32,37 @@ namespace chat::reticulum namespace lxmf = ::chat::lxmf; +enum class ReticulumUsage : uint8_t +{ + ActiveChat, + BackgroundIpService, +}; + class ReticulumAdapter final : public IMeshAdapter, + public IGeocachingTransport, public IIncomingDeliveryCommitPort { public: explicit ReticulumAdapter(LoraBoard& board, - IMeshPeerDirectory* peer_directory = nullptr); + IMeshPeerDirectory* peer_directory = nullptr, + ReticulumUsage usage = ReticulumUsage::ActiveChat); ~ReticulumAdapter() override; ReticulumAdapter(const ReticulumAdapter&) = delete; ReticulumAdapter& operator=(const ReticulumAdapter&) = delete; MeshCapabilities getCapabilities() const override; + IGeocachingTransport* geocachingTransport() override { return this; } + bool getGeocachingAuthorKey(uint8_t out[64]) override; + bool signGeocachingRecord(lxmf::ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written) override; + void setGeocachingAnnouncementHandler(void (*handler)(const lxmf::GeocachingAnnouncementView&, void*), + void* context); + MeshSendResult sendGeocachingData(const uint8_t destination_hash[16], + lxmf::ByteSpan data, bool response = false, + std::array* accepted_lxmf_hash = nullptr) override; + void setGeocachingDeliveryHandler(bool (*handler)(const lxmf::CustomDeliveryView&, void*), + void* context); bool sendText(ChannelId channel, const std::string& text, MessageId* out_msg_id, NodeId peer = 0) override; MeshSendResult sendTextDetailed(ChannelId channel, const std::string& text, @@ -81,6 +104,7 @@ class ReticulumAdapter final : public IMeshAdapter, private: std::unique_ptr service_; + const ReticulumUsage usage_; }; } // namespace chat::reticulum diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_interfaces.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_interfaces.h index a18a3bd9..f948cd7c 100644 --- a/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_interfaces.h +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/chat/infra/reticulum/reticulum_interfaces.h @@ -258,7 +258,7 @@ class AutoReticulumInterface class ReticulumInterfaceSet { public: - explicit ReticulumInterfaceSet(LoraBoard& board); + explicit ReticulumInterfaceSet(LoraBoard& board, bool owns_integrated_radio = true); void applyConfig(const MeshConfig& config, const reticulum::ReticulumNetworkConfig& network_config); @@ -290,6 +290,7 @@ class ReticulumInterfaceSet private: LoRaReticulumInterface lora_; + const bool owns_integrated_radio_; AutoReticulumInterface auto_; std::array diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/author_issue_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/author_issue_port.h new file mode 100644 index 00000000..96eefe0d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/author_issue_port.h @@ -0,0 +1,9 @@ +#pragma once +#include "platform/esp/arduino_common/chat/infra/mesh_adapter_router.h" +#include "platform/esp/arduino_common/geocaching/sd_author_issue_port.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Device and host verification share one reservation/signing implementation. +using DeviceAuthorIssuePort = SdAuthorIssuePort; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/browse_runtime.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/browse_runtime.h new file mode 100644 index 00000000..16c13251 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/browse_runtime.h @@ -0,0 +1,20 @@ +#pragma once +#include +#include +namespace chat +{ +class MeshAdapterRouter; +} +class LoraBoard; +namespace platform::esp::arduino_common::geocaching::browse_runtime +{ +void configure(chat::MeshAdapterRouter& router, LoraBoard& board); +bool workPending(); +// Called only after explicit publication confirmation. Copies exact SignedCache +// bytes; verification and persistence run on the storage maintenance owner. +bool queuePublication(const uint8_t* signed_cache, size_t size); +// Local draft transaction; expected_generation is zero for a new draft. +bool queueDraftSave(const uint8_t draft_id[16], const uint8_t* encoded, size_t size, uint64_t expected_generation); +// Exactly one operation slice, called by the existing storage maintenance owner. +void step(); +} // namespace platform::esp::arduino_common::geocaching::browse_runtime diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/index_workspace_owner.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/index_workspace_owner.h new file mode 100644 index 00000000..f656cb7d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/index_workspace_owner.h @@ -0,0 +1,26 @@ +#pragma once + +namespace platform::esp::arduino_common::geocaching +{ +// The session mutex serializes access. This lease spans asynchronous slices; +// the maintenance scheduler must advance the current holder before waiters. +class IndexWorkspaceOwner +{ + public: + bool acquire(const void* owner) + { + if (!owner || (owner_ && owner_ != owner)) return false; + owner_ = owner; + return true; + } + void release(const void* owner) + { + if (owner_ == owner) owner_ = nullptr; + } + bool heldBy(const void* owner) const { return owner && owner_ == owner; } + const void* holder() const { return owner_; } + + private: + const void* owner_ = nullptr; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_dispatch_store.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_dispatch_store.h new file mode 100644 index 00000000..dc466660 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_dispatch_store.h @@ -0,0 +1,280 @@ +#pragma once +#include "geocaching/storage/attempt_timeout.h" +#include "platform/esp/arduino_common/geocaching/index_workspace_owner.h" +#include "platform/esp/arduino_common/geocaching/request_dispatch_store.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_attempt_update.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_begin_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_pending_request.h" + +namespace platform::esp::arduino_common::geocaching +{ +// The session owns roots and workspaces. Other operations may publish a root +// only while !busy(). Completed mutation roots update the same session view. +class IndexedDispatchStore final : public RequestDispatchStore +{ + public: + IndexedDispatchStore(const ::geocaching::storage::VolumeInstance& volume, + ::geocaching::storage::IndexRootView& root, unsigned& copy, + ::geocaching::storage::IndexRootBytes& first, ::geocaching::storage::IndexRootBytes& second, + IndexWorkspaceOwner& owner, ::geocaching::storage::QueuedRequestWorkspace& workspace, uint8_t* frame, size_t capacity) + : volume_(volume), root_(root), copy_(copy), roots_{&first, &second}, owner_(owner), workspace_(workspace), frame_(frame), capacity_(capacity) + { + blocked_ = copy > 1 || !::geocaching::storage::validIndexRoot(root) || root.shards.data != roots_[copy]->data() + 48; + } + bool needsRecovery() const override { return blocked_; } + ~IndexedDispatchStore() override { clear(); } + bool busy() const { return phase_ != Phase::None; } + bool commitPending() const override { return phase_ == Phase::Begin || phase_ == Phase::Update || phase_ == Phase::Expire; } + bool expirationChanged() const override { return expired_; } + DispatchReadResult readPending(const ::geocaching::Destination& local, ::geocaching::ByteView after, + ::geocaching::storage::PendingRequestView& out) override + { + out = {}; + if (blocked_) return DispatchReadResult::Corrupt; + if (phase_ == Phase::None) + { + if (!owner_.acquire(this)) return DispatchReadResult::Pending; + revision_ = root_.revision; + if (!io_.emplace(volume_).begin(root_, local, after, frame_, capacity_)) return readError(); + phase_ = Phase::Select; + return DispatchReadResult::Pending; + } + if (phase_ != Phase::Select || revision_ != root_.revision) return readError(); + auto& select = std::get(io_); + const auto status = select.step(); + if (status == IndexedPendingStep::Working) return DispatchReadResult::Pending; + if (status == IndexedPendingStep::Ready && select.selected(out)) + { + clear(); + return DispatchReadResult::Ready; + } + if (status == IndexedPendingStep::None) + { + clear(); + return DispatchReadResult::None; + } + return readError(); + } + DispatchReadResult readForSend(::geocaching::ByteView key, DispatchSendView& out) override + { + using namespace ::geocaching::storage; + out = {}; + if (blocked_ || !key.data || key.size != 48) return DispatchReadResult::Corrupt; + if (phase_ == Phase::None) + { + if (!owner_.acquire(this)) return DispatchReadResult::Pending; + revision_ = root_.revision; + std::memcpy(key_.data(), key.data, 48); + if (!io_.emplace(volume_).begin(root_, 5, {key_.data(), 48}, frame_, capacity_)) return readError(); + phase_ = Phase::SendRequest; + return DispatchReadResult::Pending; + } + if (revision_ != root_.revision || std::memcmp(key_.data(), key.data, 48) || + (phase_ != Phase::SendRequest && phase_ != Phase::SendTask && phase_ != Phase::SendReload)) return readError(); + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return DispatchReadResult::Pending; + if (status != IndexGetStep::Ready) return readError(); + if (phase_ == Phase::SendTask) + { + TaskView task; + if (!decodeTask({task_id_.data(), task_id_.size()}, read.value(), task)) return readError(); + bool linked = false; + for (size_t i = 0; i < task.request_count; ++i) linked |= !std::memcmp(task.requests[i].data, key_.data(), 48); + if (!linked) return readError(); + stopped_ = stopped_ || !task.continue_intent || task.state == 5; + if (stopped_) + { + out.stopped = true; + clear(); + return DispatchReadResult::Ready; + } + if (!io_.emplace(volume_).begin(root_, 5, {key_.data(), 48}, frame_, capacity_)) return readError(); + phase_ = Phase::SendReload; + return DispatchReadResult::Pending; + } + OutgoingView outgoing; + if (!decodeOutgoing({key_.data(), 48}, read.value(), outgoing) || outgoing.state != 1) return readError(); + if (phase_ == Phase::SendReload) + { + out.request = outgoing.request; + out.stopped = stopped_; + clear(); + return DispatchReadResult::Ready; + } + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + stopped_ = !outgoing.continue_intent; + if (!io_.emplace(volume_).begin(root_, 10, {task_id_.data(), task_id_.size()}, frame_, capacity_)) return readError(); + phase_ = Phase::SendTask; + return DispatchReadResult::Pending; + } + JournalWriteResult beginAttempt(const ::geocaching::Destination& local, ::geocaching::ByteView key, + const std::array& id, const ::geocaching::storage::StoredTime& time) override + { + if (blocked_) return JournalWriteResult::Unavailable; + if (busy()) return JournalWriteResult::Busy; + if (!owner_.acquire(this)) return JournalWriteResult::Busy; + revision_ = root_.revision; + if (!io_.emplace(volume_).begin(root_, copy_, local, key, id, time, workspace_, frame_, capacity_, *roots_[1 - copy_])) + { + clear(); + return JournalWriteResult::Invalid; + } + phase_ = Phase::Begin; + return JournalWriteResult::InProgress; + } + JournalWriteResult recordAttemptHash(::geocaching::ByteView key, const std::array& hash) override + { + return update(key, {hash.data(), hash.size()}, ::geocaching::storage::TxAttemptState::Failed, {}); + } + JournalWriteResult finishAttempt(::geocaching::ByteView key, ::geocaching::storage::TxAttemptState terminal, + const ::geocaching::storage::StoredTime& time) override + { + return update(key, {}, terminal, time); + } + JournalWriteResult expireOneAttempt(const ::geocaching::storage::StoredTime& now, uint64_t started, uint64_t timeout, bool& expired) override + { + expired = false; + if (blocked_) return JournalWriteResult::Unavailable; + if (busy()) return JournalWriteResult::Busy; + expired_ = false; + if (!timeout) return JournalWriteResult::Verified; + if (!owner_.acquire(this)) return JournalWriteResult::Busy; + revision_ = root_.revision; + now_ = now; + started_ = started; + timeout_ = timeout; + if (!io_.emplace(volume_).begin(root_, 13, frame_, capacity_)) + { + clear(); + return JournalWriteResult::Invalid; + } + phase_ = Phase::Expire; + return JournalWriteResult::InProgress; + } + JournalWriteResult stepCommit() override + { + using namespace ::geocaching::storage; + if (blocked_) return JournalWriteResult::Unavailable; + if (!commitPending()) return JournalWriteResult::Invalid; + if (!owner_.heldBy(this) || root_.revision != revision_) + { + blocked_ = true; + clear(); + return JournalWriteResult::StateRejected; + } + if (phase_ == Phase::Expire) + { + auto& scan = std::get(io_); + const auto status = scan.step(); + if (status == IndexScanStep::Working) return JournalWriteResult::InProgress; + if (status == IndexScanStep::End) + { + clear(); + return JournalWriteResult::Verified; + } + MutationView row; + TxAttemptView attempt; + if (status != IndexScanStep::Item || !scan.item(row) || !decodeTxAttempt(row.key, row.value, attempt)) + { + blocked_ = true; + clear(); + return JournalWriteResult::StateRejected; + } + if (!attemptTimeoutReached(attempt, now_, started_, timeout_)) + { + if (!scan.advance()) + { + blocked_ = true; + clear(); + return JournalWriteResult::StateRejected; + } + return JournalWriteResult::InProgress; + } + std::memcpy(key_.data(), row.key.data, key_.size()); + clear(); + const auto begun = update({key_.data(), key_.size()}, {}, TxAttemptState::Failed, now_); + expired_ = begun == JournalWriteResult::InProgress; + return begun; + } + const auto status = phase_ == Phase::Begin ? std::get(io_).step() : std::get(io_).step(); + if (status == IndexedCommitStep::Working) return JournalWriteResult::InProgress; + if (status == IndexedCommitStep::Verified) + { + IndexRootView committed; + const bool ready = phase_ == Phase::Begin ? std::get(io_).committed(committed) : std::get(io_).committed(committed); + if (!ready) + { + blocked_ = true; + clear(); + return JournalWriteResult::StateRejected; + } + if (committed.revision != root_.revision) copy_ = 1 - copy_; + root_ = committed; + clear(); + return JournalWriteResult::Verified; + } + blocked_ = status == IndexedCommitStep::IoError || status == IndexedCommitStep::VolumeChanged || status == IndexedCommitStep::RecoveryRequired; + clear(); + return status == IndexedCommitStep::VolumeChanged ? JournalWriteResult::VolumeChanged : status == IndexedCommitStep::IoError ? JournalWriteResult::IoError + : JournalWriteResult::StateRejected; + } + + private: + enum class Phase : uint8_t + { + None, + Select, + SendRequest, + SendTask, + SendReload, + Begin, + Update, + Expire + }; + JournalWriteResult update(::geocaching::ByteView key, ::geocaching::ByteView hash, ::geocaching::storage::TxAttemptState terminal, + const ::geocaching::storage::StoredTime& time) + { + if (blocked_) return JournalWriteResult::Unavailable; + if (busy()) return JournalWriteResult::Busy; + if (!owner_.acquire(this)) return JournalWriteResult::Busy; + revision_ = root_.revision; + if (!io_.emplace(volume_).begin(root_, copy_, key, hash, terminal, time, + workspace_.outgoing, workspace_.outgoing_capacity, frame_, capacity_, *roots_[1 - copy_])) + { + clear(); + return JournalWriteResult::Invalid; + } + phase_ = Phase::Update; + return JournalWriteResult::InProgress; + } + void clear() + { + io_.emplace(); + phase_ = Phase::None; + owner_.release(this); + } + DispatchReadResult readError() + { + blocked_ = true; + clear(); + return DispatchReadResult::Corrupt; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView& root_; + unsigned& copy_; + ::geocaching::storage::IndexRootBytes* roots_[2]; + IndexWorkspaceOwner& owner_; + ::geocaching::storage::QueuedRequestWorkspace& workspace_; + uint8_t* frame_; + size_t capacity_; + ::geocaching::storage::StoredTime now_; + std::array key_{}; + std::array task_id_{}; + uint64_t revision_ = 0, started_ = 0, timeout_ = 0; + bool blocked_ = false, stopped_ = false, expired_ = false; + std::variant io_; + Phase phase_ = Phase::None; +}; +static_assert(sizeof(IndexedDispatchStore) <= 2048, "Dispatch store overlays bounded operations, not a logical ledger"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_query_store_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_query_store_port.h new file mode 100644 index 00000000..2039a864 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/indexed_query_store_port.h @@ -0,0 +1,339 @@ +#pragma once +#include "geocaching/protocol/verify_record.h" +#include "platform/esp/arduino_common/geocaching/index_workspace_owner.h" +#include "platform/esp/arduino_common/geocaching/query_browse_port.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_directory_reply.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_new_task.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_stop_task.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Session-owned buffers: one current response, transient encoding/read leases. +// Operations are allocated only while active. The owner serializes this port +// with dispatcher/storage work sharing the same roots and scratch buffers. +class IndexedQueryStorePort final : public QueryBrowsePort +{ + public: + using RandomId = bool (*)(void*, uint8_t[16]); + using Now = ::geocaching::storage::StoredTime (*)(void*); + IndexedQueryStorePort(const ::geocaching::storage::VolumeInstance& volume, ::geocaching::storage::IndexRootView& root, + unsigned& copy, ::geocaching::storage::IndexRootBytes& first, ::geocaching::storage::IndexRootBytes& second, + const ::geocaching::Destination& local, IndexWorkspaceOwner& owner, ::geocaching::storage::QueuedRequestWorkspace& workspace, + uint8_t* frame, size_t capacity, uint8_t* response, size_t response_capacity, + ::geocaching::protocol::RecordCrypto& crypto, RandomId random, Now now, void* context) + : volume_(volume), root_(root), copy_(copy), roots_{&first, &second}, local_(local), owner_(owner), workspace_(workspace), frame_(frame), + capacity_(capacity), response_(response), response_capacity_(response_capacity), crypto_(crypto), random_(random), now_(now), context_(context) + { + valid_ = copy < 2 && ::geocaching::storage::validIndexRoot(root) && root.shards.data == roots_[copy]->data() + 48 && + frame && capacity >= 24 && response && response_capacity >= 512 && workspace.outgoing; + const ::geocaching::ByteView leases[] = {{first.data(), first.size()}, {second.data(), second.size()}, {frame, capacity}, {response, response_capacity}, {workspace.outgoing, workspace.outgoing_capacity}, {workspace.task.data(), workspace.task.size()}}; + for (size_t i = 0; i < 6; ++i) + for (size_t j = 0; j < i; ++j) + { + const auto a = reinterpret_cast(leases[i].data), b = reinterpret_cast(leases[j].data); + if (leases[i].size && leases[j].size && (a <= b ? b - a < leases[i].size : a - b < leases[j].size)) valid_ = false; + } + } + ~IndexedQueryStorePort() override + { + io_.reset(); + owner_.release(this); + } + bool newRequestId(::geocaching::RequestId& out) override { return random_ && random_(context_, out.bytes.data()); } + ::geocaching::QueryPersistence submit(const ::geocaching::DirectoryEntry& directory, const ::geocaching::RequestId& id, + ::geocaching::ByteView request) override + { + if (!available() || !random_ || !now_ || !allocate()) return Result::Rejected; + std::array task; + if (!random_(context_, task.data())) return reject(); + auto& queued = io_->emplace(); + queued.destination = directory.delivery; + queued.id = id; + queued.task = task; + queued.request = request; + queued.time = now_(context_); + pending_ = Pending::Request; + return Result::Pending; + } + ::geocaching::QueryPersistence commitCapabilities(const ::geocaching::Destination& source, const ::geocaching::RequestId& id, + ::geocaching::ByteView response) override + { + ::geocaching::protocol::DirectoryCapabilities capabilities; + if (!::geocaching::protocol::decodeDirectoryCapabilities(response, id, capabilities)) return Result::Rejected; + return beginReply(source, id, response, Pending::Capabilities); + } + ::geocaching::QueryPersistence commitPage(const ::geocaching::Destination& source, const ::geocaching::RequestId& id, + ::geocaching::ByteView response, const ::geocaching::protocol::QueryPageView&) override + { + ::geocaching::protocol::QueryPageView checked; + if (!::geocaching::protocol::decodeQueryPage(response, id, 2048, 20, checked)) return Result::Rejected; + return beginReply(source, id, response, Pending::Page); + } + ::geocaching::QueryPersistence cancel(const ::geocaching::Destination& source, const ::geocaching::RequestId& id) override + { + if (!available() || !allocate()) return Result::Rejected; + makeKey(source, id, pending_key_); + pending_ = Pending::Cancel; + return Result::Pending; + } + ::geocaching::QueryPersistence pollPersistence() override + { + if (pending_ == Pending::None) return Result::Rejected; + if (!io_) return reject(); + if (std::holds_alternative(*io_) || std::holds_alternative(*io_)) + { + if (!owner_.acquire(this)) return Result::Pending; + revision_ = root_.revision; + bool begun = false; + if (pending_ == Pending::Request) + { + const auto queued = std::get(*io_); + begun = io_->emplace(volume_).begin(root_, copy_, local_, queued.destination, queued.id, queued.task, + 3, queued.request, queued.time, {}, workspace_, frame_, capacity_, *roots_[1 - copy_]); + } + else if (pending_ == Pending::Cancel) + begun = io_->emplace(volume_).begin(root_, copy_, {pending_key_.data(), pending_key_.size()}, true, + frame_, capacity_, *roots_[1 - copy_]); + else + begun = io_->emplace(volume_).begin(root_, copy_, {response_key_.data(), response_key_.size()}, + pending_ == Pending::Page ? 2 : 0, {response_, response_size_}, workspace_, frame_, capacity_, *roots_[1 - copy_]); + return begun ? Result::Pending : reject(); + } + if (!owner_.heldBy(this) || root_.revision != revision_) return reject(); + IndexedCommitStep status; + ::geocaching::storage::IndexRootView committed; + bool selected = false; + if (pending_ == Pending::Request) + { + auto& operation = std::get(*io_); + status = operation.step(); + if (status == IndexedCommitStep::Verified) selected = operation.committed(committed); + } + else if (pending_ == Pending::Cancel) + { + auto& operation = std::get(*io_); + status = operation.step(); + if (status == IndexedCommitStep::Verified) selected = operation.committed(committed); + } + else + { + auto& operation = std::get(*io_); + status = operation.step(); + if (status == IndexedCommitStep::Verified) selected = operation.committed(committed); + } + if (status == IndexedCommitStep::Working) return Result::Pending; + if (status != IndexedCommitStep::Verified || !selected) return reject(); + if (committed.revision != root_.revision) copy_ = 1 - copy_; + root_ = committed; + if (pending_ == Pending::Capabilities || pending_ == Pending::Page) + { + response_committed_ = true; + page_visible_ = pending_ == Pending::Page; + if (page_visible_) ++generation_; + } + if (pending_ == Pending::Cancel) + { + stopped_key_ = pending_key_; + has_stop_ = true; + stopped_revision_ = root_.revision; + } + pending_ = Pending::None; + io_.reset(); + owner_.release(this); + return Result::Committed; + } + uint64_t generation() const override { return generation_; } + bool pageSource(::geocaching::Destination& out) const override + { + if (!page_visible_) return false; + std::memcpy(out.bytes.data(), response_key_.data() + 16, 16); + return true; + } + bool page(::geocaching::protocol::QueryPageView& out) const override + { + out = {}; + if (!page_visible_ || !response_committed_) return false; + ::geocaching::RequestId id; + std::memcpy(id.bytes.data(), response_key_.data() + 32, 16); + return ::geocaching::protocol::decodeQueryPage({response_, response_size_}, id, 2048, 20, out); + } + bool summary(size_t index, ::geocaching::protocol::SummaryView& out) const override + { + out = {}; + ::geocaching::protocol::QueryPageView current; + if (!page(current) || index >= current.count) return false; + ::geocaching::protocol::CmpReader rows(current.encoded_items); + for (size_t i = 0; i <= index; ++i) + if (!::geocaching::protocol::decodeSummary(rows, out)) return false; + return true; + } + bool accepted(const ::geocaching::Destination& source, const ::geocaching::RequestId& id, ::geocaching::ByteView response) const override + { + if (!valid_ || !response.data || response.size > ::geocaching::kMaxApplicationBytes) return false; + std::array key; + makeKey(source, id, key); + if (has_stop_ && stopped_revision_ == root_.revision && key == stopped_key_) return true; + if (response_committed_ && key == response_key_ && response.size == response_size_ && !std::memcmp(response.data, response_, response.size)) return true; + std::array hash; + if (!crypto_.sha256(response, hash.data())) return false; + if (proof_ == Proof::Accepted && proof_revision_ == root_.revision && proof_key_ == key && proof_hash_ == hash && proof_size_ == response.size) return true; + if (proof_ == Proof::Queued || proof_ == Proof::Outgoing || proof_ == Proof::Task) return false; + proof_key_ = key; + proof_hash_ = hash; + proof_size_ = response.size; + proof_ = Proof::Queued; + return false; + } + bool maintenancePending() const override { return proof_ == Proof::Queued || proof_ == Proof::Outgoing || proof_ == Proof::Task; } + void maintenanceStep() override + { + using namespace ::geocaching::storage; + if (pending_ != Pending::None || !maintenancePending()) return; + if (proof_ == Proof::Queued) + { + if (!owner_.acquire(this)) return; + if (!allocate() || !io_->emplace(volume_).begin(root_, 5, {proof_key_.data(), proof_key_.size()}, frame_, capacity_)) + { + finishProof(false); + return; + } + revision_ = root_.revision; + proof_ = Proof::Outgoing; + return; + } + if (!io_ || !owner_.heldBy(this) || revision_ != root_.revision) + { + io_.reset(); + owner_.release(this); + proof_ = Proof::Queued; + return; + } + auto& read = std::get(*io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return; + if (status != IndexGetStep::Ready) + { + finishProof(false); + return; + } + if (proof_ == Proof::Task) + { + TaskView task; + finishProof(decodeTask({proof_task_.data(), proof_task_.size()}, read.value(), task) && task.state == 5 && !task.continue_intent); + return; + } + OutgoingView outgoing; + if (!decodeOutgoing({proof_key_.data(), proof_key_.size()}, read.value(), outgoing)) + { + finishProof(false); + return; + } + if (outgoing.state == 4) + { + std::array hash; + finishProof(outgoing.terminal_data.size == proof_size_ && crypto_.sha256(outgoing.terminal_data, hash.data()) && hash == proof_hash_); + return; + } + std::memcpy(proof_task_.data(), outgoing.task_id.data, proof_task_.size()); + if (!io_->emplace(volume_).begin(root_, 10, {proof_task_.data(), proof_task_.size()}, frame_, capacity_)) + { + finishProof(false); + return; + } + proof_ = Proof::Task; + } + + private: + using Result = ::geocaching::QueryPersistence; + enum class Pending : uint8_t + { + None, + Request, + Capabilities, + Page, + Cancel + }; + enum class Proof : uint8_t + { + None, + Queued, + Outgoing, + Task, + Accepted, + Rejected + }; + struct QueuedRequest + { + ::geocaching::Destination destination; + ::geocaching::RequestId id; + std::array task{}; + ::geocaching::ByteView request; + ::geocaching::storage::StoredTime time; + }; + using Operation = std::variant; + bool available() const { return valid_ && pending_ == Pending::None && proof_ != Proof::Outgoing && proof_ != Proof::Task && copy_ < 2; } + bool allocate() + { + io_.reset(new (std::nothrow) Operation); + return io_ != nullptr; + } + Result reject() + { + pending_ = Pending::None; + io_.reset(); + owner_.release(this); + return Result::Rejected; + } + void finishProof(bool accepted) + { + io_.reset(); + owner_.release(this); + proof_revision_ = root_.revision; + proof_ = accepted ? Proof::Accepted : Proof::Rejected; + } + void makeKey(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id, std::array& key) const + { + std::memcpy(key.data(), local_.bytes.data(), 16); + std::memcpy(key.data() + 16, remote.bytes.data(), 16); + std::memcpy(key.data() + 32, id.bytes.data(), 16); + } + Result beginReply(const ::geocaching::Destination& source, const ::geocaching::RequestId& id, ::geocaching::ByteView response, + Pending pending) + { + if (!available() || !response_ || response.size > response_capacity_ || !allocate()) return Result::Rejected; + if (page_visible_) ++generation_; + page_visible_ = response_committed_ = false; + makeKey(source, id, response_key_); + std::memmove(response_, response.data, response.size); + response_size_ = response.size; + pending_ = pending; + return Result::Pending; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView& root_; + unsigned& copy_; + ::geocaching::storage::IndexRootBytes* roots_[2]; + ::geocaching::Destination local_; + IndexWorkspaceOwner& owner_; + ::geocaching::storage::QueuedRequestWorkspace& workspace_; + uint8_t *frame_, *response_; + size_t capacity_, response_capacity_, response_size_ = 0; + ::geocaching::protocol::RecordCrypto& crypto_; + RandomId random_; + Now now_; + void* context_; + std::unique_ptr io_; + std::array response_key_{}, pending_key_{}, stopped_key_{}; + mutable std::array proof_key_{}; + mutable std::array proof_hash_{}; + std::array proof_task_{}; + mutable size_t proof_size_ = 0; + uint64_t generation_ = 0, revision_ = 0; + uint64_t stopped_revision_ = 0, proof_revision_ = 0; + bool valid_ = false; + bool response_committed_ = false, page_visible_ = false, has_stop_ = false; + Pending pending_ = Pending::None; + mutable Proof proof_ = Proof::None; +}; +static_assert(sizeof(IndexedQueryStorePort) <= 512, "Idle query adapter owns neither operation workspaces nor page payloads"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_port.h new file mode 100644 index 00000000..5bbf6b6e --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_port.h @@ -0,0 +1,22 @@ +#pragma once +#include "geocaching/usecase/query_client.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Serialized browse-session boundary. Read-only page methods are UI-safe and +// must never perform SD I/O. Views borrow only the currently published page; +// asynchronous persistence belongs to QueryClientPort. +class QueryBrowsePort : public ::geocaching::QueryClientPort +{ + public: + virtual uint64_t generation() const = 0; + virtual bool pageSource(::geocaching::Destination&) const = 0; + virtual bool page(::geocaching::protocol::QueryPageView&) const = 0; + virtual bool summary(size_t, ::geocaching::protocol::SummaryView&) const = 0; + // Called from authenticated delivery, without I/O. Only durable matching + // responses or durable stopped tasks may be acknowledged here. + virtual bool accepted(const ::geocaching::Destination&, const ::geocaching::RequestId&, ::geocaching::ByteView) const = 0; + virtual bool maintenancePending() const { return false; } + virtual void maintenanceStep() {} +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_source.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_source.h new file mode 100644 index 00000000..8b58d8bc --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_browse_source.h @@ -0,0 +1,120 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/query_browse_port.h" +#include "ui_presentation/geocaching/geocaching_source.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +// UI projection of the same browse session driven by the maintenance owner. +// The runtime supplies its lock: UI reads never access SD or the router. +class QueryBrowseSource final : public ::ui::geocaching::Source +{ + public: + using Lock = bool (*)(void*); + using Unlock = void (*)(void*); + QueryBrowseSource(::geocaching::QueryClient& client, QueryBrowsePort& store, + const ::geocaching::protocol::QueryRegion& region, + Lock lock = nullptr, Unlock unlock = nullptr, void* context = nullptr) + : client_(client), store_(store), region_(region), lock_(lock), unlock_(unlock), context_(context) {} + void snapshot(::ui::geocaching::Section section, ::ui::geocaching::Snapshot& out) override + { + out = {}; + Guard guard(*this); + if (!guard.locked) + { + std::snprintf(out.status.data(), out.status.size(), "Updating..."); + return; + } + out.generation = generation(); + if (section != ::ui::geocaching::Section::Discover) + { + std::snprintf(out.status.data(), out.status.size(), "No cached items"); + return; + } + ::geocaching::protocol::QueryPageView page; + if (store_.page(page)) out.count = page.count; + const auto phase = client_.phase(); + out.can_refresh = phase == ::geocaching::QueryClientPhase::Idle || phase == ::geocaching::QueryClientPhase::PageReady || + phase == ::geocaching::QueryClientPhase::Failed; + out.has_more = client_.hasMore(); + std::snprintf(out.status.data(), out.status.size(), "%s", status(phase)); + } + bool item(::ui::geocaching::Section section, size_t index, uint64_t expected, ::ui::geocaching::Item& out) override + { + out = {}; + Guard guard(*this); + if (!guard.locked || section != ::ui::geocaching::Section::Discover || expected != generation()) return false; + ::geocaching::protocol::SummaryView row; + if (!store_.summary(index, row)) return false; + out.id = row.id.bytes; + out.revision_hash = row.hash.bytes; + out.latitude_e7 = row.latitude_e7; + out.longitude_e7 = row.longitude_e7; + std::memcpy(out.name.data(), row.name.data(), row.name.size()); + const auto lat = row.latitude_e7 < 0 ? -int64_t(row.latitude_e7) : int64_t(row.latitude_e7); + const auto lon = row.longitude_e7 < 0 ? -int64_t(row.longitude_e7) : int64_t(row.longitude_e7); + std::snprintf(out.detail.data(), out.detail.size(), + "%s%ld.%07ld, %s%ld.%07ld\nDifficulty %u.%u / Terrain %u.%u\nDirectory preview - not yet downloaded", + row.latitude_e7 < 0 ? "-" : "", long(lat / 10000000), long(lat % 10000000), + row.longitude_e7 < 0 ? "-" : "", long(lon / 10000000), long(lon % 10000000), + unsigned(row.difficulty_x2 / 2), unsigned((row.difficulty_x2 % 2) * 5), + unsigned(row.terrain_x2 / 2), unsigned((row.terrain_x2 % 2) * 5)); + return true; + } + void refresh(::ui::geocaching::Section section) override + { + Guard guard(*this); + if (guard.locked && section == ::ui::geocaching::Section::Discover) client_.query(region_); + } + bool loadMore() override + { + Guard guard(*this); + return guard.locked && client_.loadMore(); + } + void open(const ::ui::geocaching::Item&, uint64_t) override {} + + private: + struct Guard + { + explicit Guard(QueryBrowseSource& source) : source(source), locked(!source.lock_ || source.lock_(source.context_)) {} + ~Guard() + { + if (locked && source.unlock_) source.unlock_(source.context_); + } + QueryBrowseSource& source; + bool locked; + }; + uint64_t generation() const { return (store_.generation() << 8) | static_cast(client_.phase()) | (static_cast(client_.failure()) << 4); } + const char* status(::geocaching::QueryClientPhase phase) const + { + using Phase = ::geocaching::QueryClientPhase; + switch (phase) + { + case Phase::Idle: + return "Refresh to discover shared caches"; + case Phase::FindingDirectory: + return "Finding a public directory..."; + case Phase::CheckingCapabilities: + return "Checking directory capabilities..."; + case Phase::Querying: + return "Querying shared caches..."; + case Phase::PageReady: + return "Shared caches - directory preview"; + case Phase::Failed: + if (client_.failure() == ::geocaching::QueryFailure::Timeout) return "Directory did not reply - Refresh to retry"; + if (client_.failure() == ::geocaching::QueryFailure::Cancelled) return "Query cancelled"; + return "Storage failed - recovery required"; + case Phase::Cancelling: + return "Stopping query..."; + default: + return "Saving query progress..."; + } + } + ::geocaching::QueryClient& client_; + QueryBrowsePort& store_; + ::geocaching::protocol::QueryRegion region_; + Lock lock_; + Unlock unlock_; + void* context_; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_store_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_store_port.h new file mode 100644 index 00000000..e39c4b64 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/query_store_port.h @@ -0,0 +1,164 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/query_browse_port.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" + +namespace platform::esp::arduino_common::geocaching +{ +// The browse owner serializes this port, the dispatcher and UI projections. +// Only exact committed responses are visible; all returned views borrow state. +class QueryStorePort final : public QueryBrowsePort +{ + public: + using RandomId = bool (*)(void*, uint8_t out[16]); + using Now = ::geocaching::storage::StoredTime (*)(void*); + QueryStorePort(SdRequestStore& store, ::geocaching::storage::LogicalState& state, + const ::geocaching::Destination& local, RandomId random, Now now, void* context) + : store_(store), state_(state), local_(local), random_(random), now_(now), context_(context) {} + + bool newRequestId(::geocaching::RequestId& out) override + { + return random_ && random_(context_, out.bytes.data()); + } + + ::geocaching::QueryPersistence submit(const ::geocaching::DirectoryEntry& directory, + const ::geocaching::RequestId& id, ::geocaching::ByteView request) override + { + if (pending_ != Pending::None || !random_ || !now_) return Result::Rejected; + std::array task; + if (!random_(context_, task.data())) return Result::Rejected; + const auto result = store_.persistNewTask(local_, directory.delivery, id, task, 3, request, now_(context_)); + return started(result, Pending::Request, directory.delivery, id); + } + ::geocaching::QueryPersistence commitCapabilities(const ::geocaching::Destination& source, + const ::geocaching::RequestId& id, ::geocaching::ByteView response) override + { + if (pending_ != Pending::None) return Result::Rejected; + return started(store_.commitDirectoryCapabilities(local_, source, id, response), Pending::Capabilities, source, id); + } + ::geocaching::QueryPersistence commitPage(const ::geocaching::Destination& source, + const ::geocaching::RequestId& id, ::geocaching::ByteView response, + const ::geocaching::protocol::QueryPageView&) override + { + if (pending_ != Pending::None) return Result::Rejected; + return started(store_.commitQueryResult(local_, source, id, response), Pending::Page, source, id); + } + ::geocaching::QueryPersistence pollPersistence() override + { + if (pending_ == Pending::None) return Result::Rejected; + const auto result = store_.stepCommit(); + if (result == JournalWriteResult::InProgress) return Result::Pending; + if (result == JournalWriteResult::Verified && pending_ == Pending::Page) publishPage(); + pending_ = Pending::None; + return result == JournalWriteResult::Verified ? Result::Committed : Result::Rejected; + } + uint64_t generation() const override { return generation_; } + bool pageSource(::geocaching::Destination& out) const override + { + if (!generation_) return false; + std::memcpy(out.bytes.data(), page_key_.data() + 16, 16); + return true; + } + ::geocaching::QueryPersistence cancel(const ::geocaching::Destination& source, const ::geocaching::RequestId& id) override + { + if (pending_ != Pending::None) return Result::Rejected; + std::array key; + makeKey(source, id, key); + ::geocaching::ByteView value; + ::geocaching::storage::OutgoingView outgoing; + if (!state_.view().find(5, {key.data(), key.size()}, value) || + !::geocaching::storage::decodeOutgoing({key.data(), key.size()}, value, outgoing)) return Result::Rejected; + std::array task; + std::memcpy(task.data(), outgoing.task_id.data, task.size()); + return started(store_.stopTask(task), Pending::Cancellation, source, id); + } + bool page(::geocaching::protocol::QueryPageView& out) const override + { + out = {}; + if (!generation_) return false; + ::geocaching::ByteView bytes; + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::RequestId id; + std::memcpy(id.bytes.data(), page_key_.data() + 32, 16); + return state_.view().find(5, {page_key_.data(), page_key_.size()}, bytes) && + ::geocaching::storage::decodeOutgoing({page_key_.data(), page_key_.size()}, bytes, outgoing) && outgoing.state == 4 && + ::geocaching::protocol::decodeQueryPage(outgoing.terminal_data, id, 2048, 20, out); + } + bool summary(size_t index, ::geocaching::protocol::SummaryView& out) const override + { + out = {}; + ::geocaching::protocol::QueryPageView current; + if (!page(current) || index >= current.count) return false; + ::geocaching::protocol::CmpReader rows(current.encoded_items); + for (size_t i = 0; i <= index; ++i) + if (!::geocaching::protocol::decodeSummary(rows, out)) return false; + return true; + } + // The transport may retry while the first callback was still persisting. + // A duplicate is acknowledged only after exact durable response matching. + bool accepted(const ::geocaching::Destination& source, const ::geocaching::RequestId& id, + ::geocaching::ByteView response) const override + { + std::array key; + makeKey(source, id, key); + ::geocaching::ByteView bytes; + ::geocaching::storage::OutgoingView outgoing; + if (!response.data || !state_.view().find(5, {key.data(), key.size()}, bytes) || + !::geocaching::storage::decodeOutgoing({key.data(), key.size()}, bytes, outgoing)) return false; + if (outgoing.state == 4) + return outgoing.terminal_data.size == response.size && !std::memcmp(outgoing.terminal_data.data, response.data, response.size); + ::geocaching::storage::TaskView task; + // A durable stop discards late replies without reviving the query. + return state_.view().find(10, outgoing.task_id, bytes) && + ::geocaching::storage::decodeTask(outgoing.task_id, bytes, task) && task.state == 5 && !task.continue_intent; + } + + private: + using Result = ::geocaching::QueryPersistence; + enum class Pending : uint8_t + { + None, + Request, + Capabilities, + Page, + Cancellation + }; + void makeKey(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id, + std::array& key) const + { + std::memcpy(key.data(), local_.bytes.data(), 16); + std::memcpy(key.data() + 16, remote.bytes.data(), 16); + std::memcpy(key.data() + 32, id.bytes.data(), 16); + } + Result started(JournalWriteResult result, Pending operation, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& id) + { + if (result != JournalWriteResult::Verified && result != JournalWriteResult::InProgress) return Result::Rejected; + makeKey(remote, id, pending_key_); + if (result == JournalWriteResult::Verified) + { + if (operation == Pending::Page) publishPage(); + return Result::Committed; + } + pending_ = operation; + return Result::Pending; + } + void publishPage() + { + if (!generation_ || page_key_ != pending_key_) + { + page_key_ = pending_key_; + ++generation_; + } + } + SdRequestStore& store_; + ::geocaching::storage::LogicalState& state_; + ::geocaching::Destination local_; + RandomId random_; + Now now_; + void* context_; + std::array pending_key_{}, page_key_{}; + uint64_t generation_ = 0; + Pending pending_ = Pending::None; +}; +static_assert(sizeof(QueryStorePort) <= 192, "Query adapter must not keep a duplicate page payload"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatch_store.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatch_store.h new file mode 100644 index 00000000..fc356fac --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatch_store.h @@ -0,0 +1,43 @@ +#pragma once +#include "geocaching/storage/pending_request.h" +#include "geocaching/storage/tx_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_journal.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class DispatchReadResult : uint8_t +{ + Ready, + None, + Pending, + Corrupt +}; +struct DispatchSendView +{ + ::geocaching::ByteView request; + bool stopped = false; +}; +// Serialized storage owner. Reads may yield; returned bytes remain borrowed +// until the next storage operation. Persistence progresses through stepCommit; +// implementations must respect the serialized owner's per-step I/O budget. +class RequestDispatchStore +{ + public: + virtual ~RequestDispatchStore() = default; + virtual bool needsRecovery() const = 0; + virtual bool commitPending() const = 0; + virtual JournalWriteResult stepCommit() = 0; + virtual DispatchReadResult readPending(const ::geocaching::Destination&, ::geocaching::ByteView, + ::geocaching::storage::PendingRequestView&) = 0; + virtual DispatchReadResult readForSend(::geocaching::ByteView, DispatchSendView&) = 0; + virtual JournalWriteResult beginAttempt(const ::geocaching::Destination&, ::geocaching::ByteView, + const std::array&, const ::geocaching::storage::StoredTime&) = 0; + virtual JournalWriteResult recordAttemptHash(::geocaching::ByteView, const std::array&) = 0; + virtual JournalWriteResult finishAttempt(::geocaching::ByteView, ::geocaching::storage::TxAttemptState, + const ::geocaching::storage::StoredTime&) = 0; + virtual JournalWriteResult expireOneAttempt(const ::geocaching::storage::StoredTime&, uint64_t, uint64_t, bool&) = 0; + // Legacy synchronous scanning yields only when it found an expiry. An + // asynchronous implementation reports whether its completed scan changed it. + virtual bool expirationChanged() const { return true; } +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatcher.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatcher.h new file mode 100644 index 00000000..9986a97a --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/request_dispatcher.h @@ -0,0 +1,56 @@ +#pragma once +#include "platform/esp/arduino_common/chat/infra/mesh_adapter_router.h" +#include "platform/esp/arduino_common/geocaching/request_dispatch_store.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class DispatchStatus : uint8_t +{ + Idle, + Submitted, + Deferred, + StorageBlocked, + Corrupt +}; +struct DispatchResult +{ + DispatchStatus status = DispatchStatus::Idle; + chat::MeshOperationFailure failure = chat::MeshOperationFailure::None; +}; + +// Serialized worker only, off task stack. The owner supplies its configured +// retry delay and a current time observation. Submitted is not Delivered. +class RequestDispatcher +{ + public: + RequestDispatcher(chat::MeshAdapterRouter& router, RequestDispatchStore& store, + uint32_t retry_delay_ms, uint64_t attempt_timeout_ms); + DispatchResult dispatchOne(const ::geocaching::storage::StoredTime& now); + + private: + enum class Phase : uint8_t + { + Select, + SelectRequest, + ExpireCommit, + BeginCommit, + Send, + SuccessCommit, + FailureCommit + }; + Phase phase_ = Phase::Select; + std::array attempt_id_{}; + chat::MeshOperationFailure send_failure_ = chat::MeshOperationFailure::None; + chat::MeshAdapterRouter& router_; + RequestDispatchStore& store_; + uint32_t retry_delay_ms_; + uint64_t attempt_timeout_ms_; + uint64_t recovery_started_ms_ = 0; + bool clock_initialized_ = false; + uint64_t not_before_ = 0; + std::array boot_{}; + std::array cursor_{}; + bool has_cursor_ = false; +}; +static_assert(sizeof(RequestDispatcher) <= 192, "Dispatcher retains metadata only, not a request-sized buffer"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/saved_cache_catalog.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/saved_cache_catalog.h new file mode 100644 index 00000000..f4f1a188 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/saved_cache_catalog.h @@ -0,0 +1,226 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_gpx_hash.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" +#include "ui_presentation/geocaching/geocaching_source.h" +#include +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +// A projection, not another copy of the cache database. The owner serializes +// this with state mutations and resets it after installs or media handoff. +template +class SavedCacheCatalog +{ + public: + SavedCacheCatalog(::geocaching::storage::LogicalState& state, ::geocaching::protocol::RecordCrypto& crypto) + : state_(state), crypto_(crypto) {} + void reset() + { + reader_.reset(); + digest_.reset(); + cursor_ = 0; + valid_ = invalid_ = 0; + checking_ = true; + ++generation_; + } + bool pending() const { return checking_; } + bool contains(const std::array& id, const std::array& hash) const + { + for (size_t i = 0; i < 64; ++i) + { + if (!(valid_ & (uint64_t(1) << i))) continue; + Candidate entry; + if (candidate(i, entry) == Lookup::Ready && entry.id.bytes == id && entry.hash.bytes == hash) return true; + } + return false; + } + void advance() + { + if (!checking_) return; + if (reader_) + { + if (reader_->step() == GpxHashStep::Reading) return; + std::array hash; + finish(reader_->result(hash) && hash == expected_hash_); + return; + } + if (cursor_ == 64) + { + checking_ = false; + ++generation_; + return; + } + Candidate entry; + const auto result = candidate(cursor_, entry); + if (result == Lookup::End) + { + checking_ = false; + ++generation_; + return; + } + if (result == Lookup::Pending) + { + ++cursor_; + return; + } + if (result != Lookup::Ready) + { + finish(false); + return; + } + auto verified = ::geocaching::protocol::VerificationResult::WorkspaceTooSmall; + ::geocaching::protocol::VerifiedRecordView record; + state_.withScratch([&](uint8_t* bytes, size_t capacity) + { verified = ::geocaching::protocol::verifyGeocache(entry.signed_cache, crypto_, bytes, capacity, record, &entry.id, &entry.hash); }); + if (verified != ::geocaching::protocol::VerificationResult::Valid) + { + finish(false); + return; + } + expected_hash_ = entry.file_hash; + char path[112]{}; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/caches/"); + size_t offset = std::strlen(path); + constexpr char hex[] = "0123456789abcdef"; + for (auto byte : entry.id.bytes) + { + path[offset++] = hex[byte >> 4]; + path[offset++] = hex[byte & 15]; + } + std::memcpy(path + offset, ".gpx", 5); + digest_.reset(new (std::nothrow) Digest); + if (digest_) reader_.reset(new (std::nothrow) SdGpxHash(*digest_)); + if (!reader_ || !reader_->open(path)) finish(false); + } + void snapshot(::ui::geocaching::Snapshot& out) const + { + out = {}; + out.generation = generation_; + out.can_refresh = !checking_; + for (uint64_t bits = valid_; bits; bits &= bits - 1) ++out.count; + std::snprintf(out.status.data(), out.status.size(), "%s", checking_ ? "Checking saved GPX files..." : invalid_ ? "Some GPX files changed or are unavailable" + : out.count ? "Saved GPX - available offline" + : "No downloaded caches"); + } + bool item(size_t visible_index, uint64_t generation, ::ui::geocaching::Item& out) const + { + out = {}; + if (generation != generation_) return false; + for (size_t i = 0; i < 64; ++i) + { + if (!(valid_ & (uint64_t(1) << i))) continue; + if (visible_index--) continue; + Candidate entry; + if (candidate(i, entry) != Lookup::Ready) return false; + out.id = entry.id.bytes; + out.revision_hash = entry.hash.bytes; + out.latitude_e7 = entry.record.latitude_e7; + out.longitude_e7 = entry.record.longitude_e7; + out.downloaded = true; + std::memcpy(out.name.data(), entry.record.name.data(), entry.record.name.size()); + const auto lat = entry.record.latitude_e7 < 0 ? -int64_t(entry.record.latitude_e7) : int64_t(entry.record.latitude_e7); + const auto lon = entry.record.longitude_e7 < 0 ? -int64_t(entry.record.longitude_e7) : int64_t(entry.record.longitude_e7); + std::snprintf(out.detail.data(), out.detail.size(), "%s%ld.%07ld, %s%ld.%07ld\nSaved GPX on SD card\nRevision %lu", + entry.record.latitude_e7 < 0 ? "-" : "", long(lat / 10000000), long(lat % 10000000), + entry.record.longitude_e7 < 0 ? "-" : "", long(lon / 10000000), long(lon % 10000000), static_cast(entry.record.revision)); + return true; + } + return false; + } + + private: + enum class Lookup : uint8_t + { + End, + Pending, + Invalid, + Ready + }; + struct Candidate + { + ::geocaching::GeocacheId id; + ::geocaching::RevisionHash hash; + ::geocaching::ByteView signed_cache; + ::geocaching::RecordView record; + std::array file_hash{}; + }; + Lookup candidate(size_t ordinal, Candidate& out) const + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + out = {}; + const auto view = state_.view(); + size_t cursor = 0; + MutationView head_row; + while (view.next(cursor, head_row)) + { + if (head_row.table != 2) continue; + if (ordinal--) continue; + CacheHeadView head; + if (!decodeCacheHead(head_row.key, head_row.value, head)) return Lookup::Invalid; + if (!head.current_hash.size) return Lookup::Pending; + std::memcpy(out.id.bytes.data(), head_row.key.data, 32); + std::memcpy(out.hash.bytes.data(), head.current_hash.data, 32); + size_t scan = 0; + MutationView row; + uint64_t latest = 0; + while (view.next(scan, row)) + { + if (row.table != 12) continue; + InstallRecordView install; + if (!decodeInstallRecord(row.key, row.value, install)) return Lookup::Invalid; + if (install.phase == InstallPhase::Installed && install.generation <= head.install_generation && install.generation > latest && + !std::memcmp(install.cache_id.data, out.id.bytes.data(), 32) && !std::memcmp(install.revision_hash.data, out.hash.bytes.data(), 32)) + { + latest = install.generation; + std::memcpy(out.file_hash.data(), install.new_file_hash.data, 32); + } + } + if (!latest) return Lookup::Invalid; + scan = 0; + while (view.next(scan, row)) + { + if (row.table != 5) continue; + OutgoingView outgoing; + ByteView value; + TaskView task; + if (!decodeOutgoing(row.key, row.value, outgoing) || outgoing.state != 4 || + !view.find(10, outgoing.task_id, value) || !decodeTask(outgoing.task_id, value, task) || task.kind != 2 || task.state != 3 || + task.cache_id.size != 32 || task.revision_hash.size != 32 || + std::memcmp(task.cache_id.data, out.id.bytes.data(), 32) || std::memcmp(task.revision_hash.data, out.hash.bytes.data(), 32)) continue; + RequestId id; + std::memcpy(id.bytes.data(), row.key.data + 32, 16); + protocol::GetResponseView response; + if (!protocol::decodeGetResponse(outgoing.terminal_data, id, 8192, response) || response.has_conflict) return Lookup::Invalid; + protocol::CmpReader reader(response.signed_cache); + size_t fields = 0; + ByteView encoded, signature; + if (!reader.array(fields, 2) || !reader.binary(encoded, kMaxRecordBytes) || !reader.binary(signature, 64) || + !protocol::decodeGeocacheRecord(encoded, out.record)) return Lookup::Invalid; + out.signed_cache = response.signed_cache; + return Lookup::Ready; + } + return Lookup::Invalid; + } + return Lookup::End; + } + void finish(bool valid) + { + (valid ? valid_ : invalid_) |= uint64_t(1) << cursor_; + ++cursor_; + ++generation_; + reader_.reset(); + digest_.reset(); + } + ::geocaching::storage::LogicalState& state_; + ::geocaching::protocol::RecordCrypto& crypto_; + std::unique_ptr digest_; + std::unique_ptr> reader_; + std::array expected_hash_{}; + uint64_t valid_ = 0, invalid_ = 0, generation_ = 1; + size_t cursor_ = 0; + bool checking_ = true; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_author_issue_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_author_issue_port.h new file mode 100644 index 00000000..812001ea --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_author_issue_port.h @@ -0,0 +1,63 @@ +#pragma once +#include "geocaching/usecase/author_issue.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Owned by the serialized storage worker. Transport is the existing router; +// templating permits the same persistence boundary to be tested without radio. +template +class SdAuthorIssuePort final : public ::geocaching::AuthorIssuePort +{ + public: + SdAuthorIssuePort(Transport& transport, SdRequestStore& store, + ::geocaching::protocol::RecordCrypto& crypto, + const ::geocaching::storage::StoredTime& issued, ::geocaching::ByteView draft = {}, uint64_t generation = 0) + : store_(store), transport_(transport), crypto_(crypto), issued_(issued), draft_(draft), generation_(generation) {} + + bool getAuthorKey(uint8_t out[64]) override { return transport_.getGeocachingAuthorKey(out); } + + ::geocaching::AuthorReservationResult reserve(::geocaching::ByteView record, uint8_t* workspace, size_t capacity) override + { + using Result = ::geocaching::AuthorReservationResult; + if (failed_) return Result::Failed; + if (reserved_) return Result::Reserved; + const auto result = committing_ ? store_.stepCommit() : draft_.size ? store_.reserveDraftUnsignedRecord(draft_, generation_, record, crypto_, workspace, capacity, issued_) + : store_.reserveUnsignedRecord(record, crypto_, workspace, capacity, issued_); + if (result == JournalWriteResult::Busy) return Result::Pending; + if (result == JournalWriteResult::InProgress) + { + committing_ = true; + return Result::Pending; + } + committing_ = false; + reserved_ = result == JournalWriteResult::Verified; + failed_ = !reserved_; + return reserved_ ? Result::Reserved : Result::Failed; + } + + void cancelReservation() override + { + if (committing_) store_.cancelCommit(); + committing_ = false; + failed_ = true; + } + + bool sign(::geocaching::ByteView record, uint8_t* workspace, size_t capacity, + uint8_t* output, size_t output_capacity, size_t& written) override + { + written = 0; + if (!reserved_ || failed_ || committing_ || store_.needsRecovery()) return false; + return transport_.signGeocachingRecord({record.data, record.size}, workspace, capacity, output, output_capacity, written); + } + + private: + SdRequestStore& store_; + Transport& transport_; + ::geocaching::protocol::RecordCrypto& crypto_; + ::geocaching::storage::StoredTime issued_; + ::geocaching::ByteView draft_; + uint64_t generation_ = 0; + bool committing_ = false, reserved_ = false, failed_ = false; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_index_import.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_index_import.h new file mode 100644 index 00000000..22774ee5 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_index_import.h @@ -0,0 +1,225 @@ +#pragma once +#include "geocaching/storage/checkpoint_selection.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_append.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_writer.h" +#include "platform/esp/arduino_common/geocaching/sd_index_references.h" +#include "platform/esp/arduino_common/geocaching/sd_index_root_writer.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +// First index creation only. The owner first selects a verified checkpoint, +// pins that checkpoint slot, and later replays its journal suffix before use. +// A partial index directory is recovery evidence; this operation never erases it. +template +class SdCheckpointIndexImport +{ + public: + SdCheckpointIndexImport(const ::geocaching::storage::VolumeInstance& volume, Digest& digest) : volume_(volume), reader_(digest) {} + bool begin(char slot, const ::geocaching::storage::CheckpointCandidate& selected, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& root) + { + using namespace ::geocaching::storage; + if (result_ != IndexRootWriteStep::Idle || (slot != 'a' && slot != 'b') || + selected.state != CheckpointCandidateState::Verified || !selected.sequence || !frame || capacity < 24) return false; + const auto a = reinterpret_cast(frame), b = reinterpret_cast(root.data()); + if (a <= b ? b - a < capacity : a - b < root.size()) return false; + selected_ = selected; + slot_ = slot; + frame_ = frame; + capacity_ = capacity; + root_ = &root; + root.fill(0); + result_ = IndexRootWriteStep::Working; + return true; + } + bool selected(::geocaching::storage::IndexRootView& root) const + { + root = {}; + return result_ == IndexRootWriteStep::Verified && + ::geocaching::storage::decodeIndexRoot({root_->data(), root_->size()}, volume_, root); + } + IndexRootWriteStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexRootWriteStep::Working) return result_; + switch (phase_) + { + case Phase::Volume: + { + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexRootWriteStep::IoError); + if (current != volume_) return fail(IndexRootWriteStep::VolumeChanged); + phase_ = Phase::Exists; + return result_; + } + case Phase::Exists: + if (storage::sd_exists("/trailmate/geocaching/.state/index")) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::Directory; + return result_; + case Phase::Directory: + if (storage::sd_is_directory("/trailmate/geocaching/.state/index")) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::Create; + return result_; + case Phase::Create: + if (!storage::sd_mkdir("/trailmate/geocaching/.state/index")) return fail(IndexRootWriteStep::IoError); + phase_ = Phase::Slot; + return result_; + case Phase::Slot: + if (!storage::sd_mkdir("/trailmate/geocaching/.state/index/a")) return fail(IndexRootWriteStep::IoError); + phase_ = Phase::Open; + return result_; + case Phase::Open: + if (!reader_.open(slot_)) return fail(IndexRootWriteStep::IoError); + phase_ = Phase::Read; + return result_; + case Phase::Read: + { + CheckpointPageCursor page; + const auto status = reader_.stepCursor(frame_, capacity_, page); + if (status == CheckpointReadStep::Reading) + { + if (reader_.pendingIndex(cursor_)) phase_ = Phase::Entry; + return result_; + } + if (status != CheckpointReadStep::Verified) return fail(status == CheckpointReadStep::IoError ? IndexRootWriteStep::IoError : IndexRootWriteStep::Invalid); + if (reader_.sequence() != selected_.sequence || reader_.digest() != selected_.digest) return fail(IndexRootWriteStep::Invalid); + IndexRootView root{1, selected_.sequence, 1, 'a', {root_->data() + 48, kIndexShardBitmapSize}}; + if (!encodeIndexRoot(volume_, root, *root_)) return fail(IndexRootWriteStep::Invalid); + references_.reset(new (std::nothrow) SdIndexReferences(volume_)); + if (!references_ || !references_->begin(root, frame_, capacity_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::References; + return result_; + } + case Phase::Entry: + if (!cursor_.next(entry_)) + { + if (!cursor_.complete()) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::Read; + return result_; + } + if (entry_.location.record_sequence != selected_.sequence || + !validStoredRowShape({entry_.table, entry_.key, {frame_ + entry_.location.value_offset - entry_.location.frame_offset, entry_.location.value_size}, false})) + return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::Table; + return result_; + case Phase::Table: + case Phase::CreateTable: + { + char path[64]; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index/a/%02x", static_cast(entry_.table)); + if (phase_ == Phase::Table && !storage::sd_is_directory(path)) + { + phase_ = Phase::CreateTable; + return result_; + } + if (phase_ == Phase::CreateTable && !storage::sd_mkdir(path)) return fail(IndexRootWriteStep::IoError); + if (!io_.template emplace(volume_, 'a').begin(entry_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::Append; + return result_; + } + case Phase::Append: + { + auto& append = std::get(io_); + const auto status = append.step(); + if (status == IndexAppendStep::Working) return result_; + if (status != IndexAppendStep::Verified) return mapError(status); + head_ = {1, selected_.sequence, append.writtenLength(), entry_.table, static_cast(::sys::crc32(entry_.key.data, entry_.key.size))}; + if (!io_.template emplace(volume_, 'a').begin(entry_.key, 0, head_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::HeadFirst; + return result_; + } + case Phase::HeadFirst: + case Phase::HeadSecond: + { + const auto status = std::get(io_).step(); + if (status == IndexHeadWriteStep::Working) return result_; + if (status != IndexHeadWriteStep::Verified) return mapError(status); + if (phase_ == Phase::HeadFirst) + { + if (!io_.template emplace(volume_, 'a').begin(entry_.key, 1, head_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::HeadSecond; + } + else + { + (*root_)[48 + (entry_.table - 1) * 32 + head_.bucket / 8] |= static_cast(1u << (head_.bucket % 8)); + io_.template emplace(); + phase_ = Phase::Entry; + } + return result_; + } + case Phase::References: + { + const auto status = references_->step(); + if (status == IndexScanStep::Working) return result_; + if (status != IndexScanStep::End) return mapError(status); + references_.reset(); + if (!io_.template emplace(volume_).begin(0, *root_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::RootFirst; + return result_; + } + case Phase::RootFirst: + case Phase::RootSecond: + { + const auto status = std::get(io_).step(); + if (status == IndexRootWriteStep::Working) return result_; + if (status != IndexRootWriteStep::Verified) return fail(status); + if (phase_ == Phase::RootSecond) return fail(IndexRootWriteStep::Verified); + if (!io_.template emplace(volume_).begin(1, *root_)) return fail(IndexRootWriteStep::Invalid); + phase_ = Phase::RootSecond; + return result_; + } + } + return fail(IndexRootWriteStep::Invalid); + } + + private: + enum class Phase : uint8_t + { + Volume, + Exists, + Directory, + Create, + Slot, + Open, + Read, + Entry, + Table, + CreateTable, + Append, + HeadFirst, + HeadSecond, + References, + RootFirst, + RootSecond + }; + template + IndexRootWriteStep mapError(Status status) + { + return fail(status == Status::VolumeChanged ? IndexRootWriteStep::VolumeChanged : status == Status::IoError ? IndexRootWriteStep::IoError + : IndexRootWriteStep::Invalid); + } + IndexRootWriteStep fail(IndexRootWriteStep status) + { + references_.reset(); + io_.template emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::CheckpointCandidate selected_; + SdCheckpointReader reader_; + ::geocaching::storage::CheckpointIndexCursor cursor_; + ::geocaching::storage::IndexedMutation entry_; + ::geocaching::storage::IndexShardHead head_; + std::variant io_; + std::unique_ptr references_; + ::geocaching::storage::IndexRootBytes* root_ = nullptr; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + char slot_ = 0; + Phase phase_ = Phase::Volume; + IndexRootWriteStep result_ = IndexRootWriteStep::Idle; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_loader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_loader.h new file mode 100644 index 00000000..0b15c8cf --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_loader.h @@ -0,0 +1,99 @@ +#pragma once +#include "geocaching/storage/checkpoint_selection.h" +#include "geocaching/storage/logical_state.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class CheckpointLoadStep : uint8_t +{ + Loading, + Applied, + Invalid, + IoError, + StateRejected, + VolumeChanged +}; + +// State and fresh digest outlive this one-shot loader. No tentative page is +// published. The selected candidate must be from a completed selection pass. +template +class SdCheckpointLoader +{ + public: + SdCheckpointLoader(Digest& digest, ::geocaching::storage::LogicalState& state, + const ::geocaching::storage::VolumeInstance& volume, char slot, + const ::geocaching::storage::CheckpointCandidate& expected) + : state_(state), volume_(volume), slot_(slot), expected_(expected), reader_(digest) {} + SdCheckpointLoader(const SdCheckpointLoader&) = delete; + SdCheckpointLoader& operator=(const SdCheckpointLoader&) = delete; + ~SdCheckpointLoader() + { + if (staging_) state_.discardSnapshot(); + } + + template + CheckpointLoadStep step(uint8_t* bytes, size_t capacity, + ::geocaching::storage::MutationView* entries, size_t entry_capacity, Validate validate) + { + if (result_ != CheckpointLoadStep::Loading) return result_; + if (expected_.state != ::geocaching::storage::CheckpointCandidateState::Verified || (slot_ != 'a' && slot_ != 'b')) + return fail(CheckpointLoadStep::Invalid); + if (!volume_checked_) + { + ::geocaching::storage::VolumeInstance current; + const auto volume = inspectSdVolume(current); + if (volume != SdVolumeResult::Ready) return fail(volume == SdVolumeResult::Corrupt || volume == SdVolumeResult::Unsupported + ? CheckpointLoadStep::Invalid + : CheckpointLoadStep::IoError); + if (current != volume_) return fail(CheckpointLoadStep::VolumeChanged); + volume_checked_ = true; + return result_; + } + volume_checked_ = false; + if (!staging_) + { + if (!reader_.open(slot_)) return fail(CheckpointLoadStep::IoError); + if (!state_.beginSnapshot()) return fail(CheckpointLoadStep::StateRejected); + staging_ = true; + return result_; + } + size_t count = 0; + switch (reader_.step(bytes, capacity, entries, entry_capacity, count)) + { + case CheckpointReadStep::Reading: + if (count && !state_.appendSnapshot(entries, count)) return fail(CheckpointLoadStep::StateRejected); + return result_; + case CheckpointReadStep::Invalid: + return fail(CheckpointLoadStep::Invalid); + case CheckpointReadStep::WorkspaceTooSmall: + return fail(CheckpointLoadStep::StateRejected); + case CheckpointReadStep::IoError: + return fail(CheckpointLoadStep::IoError); + case CheckpointReadStep::Verified: + break; + } + if (reader_.sequence() != expected_.sequence || reader_.digest() != expected_.digest) return fail(CheckpointLoadStep::Invalid); + if (!state_.commitSnapshot(validate)) return fail(CheckpointLoadStep::StateRejected); + staging_ = false; + return result_ = CheckpointLoadStep::Applied; + } + + private: + CheckpointLoadStep fail(CheckpointLoadStep result) + { + if (staging_) state_.discardSnapshot(); + staging_ = false; + return result_ = result; + } + ::geocaching::storage::LogicalState& state_; + const ::geocaching::storage::VolumeInstance volume_; + const char slot_; + const ::geocaching::storage::CheckpointCandidate expected_; + SdCheckpointReader reader_; + bool staging_ = false; + bool volume_checked_ = false; + CheckpointLoadStep result_ = CheckpointLoadStep::Loading; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h new file mode 100644 index 00000000..f89743d4 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h @@ -0,0 +1,97 @@ +#pragma once +#include "geocaching/storage/checkpoint_index_cursor.h" +#include "geocaching/storage/checkpoint_verifier.h" +#include "platform/esp/arduino_common/geocaching/sd_record_reader.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class CheckpointReadStep : uint8_t +{ + Reading, + Verified, + Invalid, + IoError, + WorkspaceTooSmall +}; + +// One-shot candidate reader. Keep tentative entries isolated from live state +// until Verified; restarting requires a fresh digest and reader instance. +template +class SdCheckpointReader +{ + public: + explicit SdCheckpointReader(Digest& digest) : verifier_(digest) {} + bool open(char slot) + { + if (started_ || (slot != 'a' && slot != 'b')) return false; + started_ = true; + slot_ = slot; + if (!file_.open(slot == 'a' ? "/trailmate/geocaching/.state/checkpoint/a.gcs" : "/trailmate/geocaching/.state/checkpoint/b.gcs", "r")) return false; + length_ = file_.size(); + return true; + } + CheckpointReadStep step(uint8_t* bytes, size_t capacity, + ::geocaching::storage::MutationView* entries, size_t entry_capacity, size_t& count) + { + count = 0; + ::geocaching::storage::CheckpointPageCursor cursor; + const auto result = stepCursor(bytes, capacity, cursor); + if (result != CheckpointReadStep::Reading) return result; + if ((cursor.complete() || cursor.count()) && (!entries || cursor.count() > entry_capacity)) + return state_ = CheckpointReadStep::WorkspaceTooSmall; + while (count < cursor.count()) + if (!cursor.next(entries[count++])) + { + count = 0; + return state_ = CheckpointReadStep::Invalid; + } + return state_; + } + // Cursor and derived index locations borrow bytes until the next step. + // They remain tentative until the complete checkpoint reaches Verified. + CheckpointReadStep stepCursor(uint8_t* bytes, size_t capacity, ::geocaching::storage::CheckpointPageCursor& cursor) + { + cursor = {}; + pending_frame_ = {}; + if (state_ != CheckpointReadStep::Reading) return state_; + ::geocaching::storage::RecordFrameView frame; + const auto read = readSdRecord(file_, length_, offset_, bytes, capacity, cursor_, frame); + if (read == SegmentReadResult::InProgress) return state_; + if (read == SegmentReadResult::End) + { + file_.close(); + return state_ = verifier_.finish() ? CheckpointReadStep::Verified : CheckpointReadStep::Invalid; + } + if (read == SegmentReadResult::IoError) return state_ = CheckpointReadStep::IoError; + if (read == SegmentReadResult::WorkspaceTooSmall) return state_ = CheckpointReadStep::WorkspaceTooSmall; + if (read != SegmentReadResult::Record || + !verifier_.accept({bytes, frame.payload.size + 24}, cursor)) + return state_ = CheckpointReadStep::Invalid; + if (frame.kind == ::geocaching::storage::RecordKind::CheckpointPage) + { + pending_frame_ = {bytes, frame.payload.size + 24}; + pending_sequence_ = frame.sequence; + } + return state_; + } + bool pendingIndex(::geocaching::storage::CheckpointIndexCursor& cursor) const + { + cursor = {}; + if (state_ != CheckpointReadStep::Reading || !pending_frame_.data || offset_ < pending_frame_.size || offset_ > UINT32_MAX) return false; + return cursor.open(pending_frame_, pending_sequence_, slot_, static_cast(offset_ - pending_frame_.size)); + } + uint64_t sequence() const { return verifier_.sequence(); } + const std::array& digest() const { return verifier_.digest(); } + + private: + storage::SdRuntimeFile file_; + SdRecordReadCursor cursor_; + ::geocaching::storage::CheckpointVerifier verifier_; + uint64_t length_ = 0, offset_ = 0; + ::geocaching::ByteView pending_frame_; + uint64_t pending_sequence_ = 0; + char slot_ = 0; + bool started_ = false; + CheckpointReadStep state_ = CheckpointReadStep::Reading; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h new file mode 100644 index 00000000..31775513 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h @@ -0,0 +1,130 @@ +#pragma once +#include "geocaching/storage/checkpoint_selection.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class CheckpointSelectionStep : uint8_t +{ + Reading, + Selected, + NoCheckpoint, + RetryLater, + Corrupt, + VolumeChanged +}; + +// One pass over both slots, one framed record per step. RetryLater requires a +// fresh pass (fresh digest contexts). Caller buffers are scratch only: selected +// contents are loaded in a subsequent verified pass before becoming live state. +template +class SdCheckpointSelection +{ + public: + explicit SdCheckpointSelection(const ::geocaching::storage::VolumeInstance& volume) + : volume_(volume), reader_a_(digest_a_), reader_b_(digest_b_) {} + ::geocaching::storage::CheckpointChoice choice() const + { + return state_ == CheckpointSelectionStep::Selected || state_ == CheckpointSelectionStep::NoCheckpoint + ? ::geocaching::storage::selectCheckpoint(candidates_[0], candidates_[1]) + : ::geocaching::storage::CheckpointChoice::RetryLater; + } + const ::geocaching::storage::CheckpointCandidate& candidate(bool slot_b) const { return candidates_[slot_b ? 1 : 0]; } + + CheckpointSelectionStep stepCursor(uint8_t* bytes, size_t capacity) + { + return step(bytes, capacity, nullptr, 0, true); + } + + CheckpointSelectionStep step(uint8_t* bytes, size_t capacity, + ::geocaching::storage::MutationView* entries, size_t entry_capacity, bool streaming = false) + { + if (state_ != CheckpointSelectionStep::Reading) return state_; + if (!volume_checked_) + { + ::geocaching::storage::VolumeInstance current; + const auto volume = inspectSdVolume(current); + if (volume == SdVolumeResult::Missing || volume == SdVolumeResult::Unavailable || volume == SdVolumeResult::IoError) + return state_ = CheckpointSelectionStep::RetryLater; + if (volume != SdVolumeResult::Ready) return state_ = CheckpointSelectionStep::Corrupt; + if (current != volume_) return state_ = CheckpointSelectionStep::VolumeChanged; + volume_checked_ = true; + return state_; + } + volume_checked_ = false; + auto& reader = slot_ ? reader_b_ : reader_a_; + auto& candidate = candidates_[slot_]; + using CandidateState = ::geocaching::storage::CheckpointCandidateState; + if (!opened_) + { + if (probed_) + { + if (!reader.open(slot_ ? 'b' : 'a')) return state_ = CheckpointSelectionStep::RetryLater; + opened_ = true; + return state_; + } + uint8_t probe[24]; + const auto result = storage::sd_read_file(slot_ ? "/trailmate/geocaching/.state/checkpoint/b.gcs" : "/trailmate/geocaching/.state/checkpoint/a.gcs", probe, sizeof(probe)); + if (result.status == storage::SdFileReadStatus::Missing) + { + candidate.state = CandidateState::Missing; + return advance(); + } + if (result.status != storage::SdFileReadStatus::Ready && result.status != storage::SdFileReadStatus::Invalid) + return state_ = CheckpointSelectionStep::RetryLater; + probed_ = true; + return state_; + } + size_t count = 0; + ::geocaching::storage::CheckpointPageCursor cursor; + switch (streaming ? reader.stepCursor(bytes, capacity, cursor) : reader.step(bytes, capacity, entries, entry_capacity, count)) + { + case CheckpointReadStep::Reading: + return state_; + case CheckpointReadStep::WorkspaceTooSmall: + case CheckpointReadStep::IoError: + return state_ = CheckpointSelectionStep::RetryLater; + case CheckpointReadStep::Invalid: + candidate.state = CandidateState::Invalid; + break; + case CheckpointReadStep::Verified: + candidate.state = CandidateState::Verified; + candidate.sequence = reader.sequence(); + candidate.digest = reader.digest(); + break; + } + return advance(); + } + + private: + CheckpointSelectionStep advance() + { + opened_ = false; + probed_ = false; + if (++slot_ < 2) return state_; + using Choice = ::geocaching::storage::CheckpointChoice; + switch (::geocaching::storage::selectCheckpoint(candidates_[0], candidates_[1])) + { + case Choice::SlotA: + case Choice::SlotB: + return state_ = CheckpointSelectionStep::Selected; + case Choice::NoCheckpoint: + return state_ = CheckpointSelectionStep::NoCheckpoint; + case Choice::RetryLater: + return state_ = CheckpointSelectionStep::RetryLater; + case Choice::Corrupt: + return state_ = CheckpointSelectionStep::Corrupt; + } + return state_ = CheckpointSelectionStep::Corrupt; + } + const ::geocaching::storage::VolumeInstance volume_; + Digest digest_a_, digest_b_; + SdCheckpointReader reader_a_, reader_b_; + std::array<::geocaching::storage::CheckpointCandidate, 2> candidates_{}; + unsigned slot_ = 0; + bool opened_ = false; + bool volume_checked_ = false, probed_ = false; + CheckpointSelectionStep state_ = CheckpointSelectionStep::Reading; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_download_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_download_port.h new file mode 100644 index 00000000..cb1b8337 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_download_port.h @@ -0,0 +1,497 @@ +#pragma once +#include "geocaching/usecase/download_client.h" +#include "geocaching/usecase/gpx_install.h" +#include "platform/esp/arduino_common/geocaching/sd_gpx_hash.h" +#include "platform/esp/arduino_common/geocaching/sd_gpx_stage.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +// One storage-owner job. While staging, the owner must not admit another state +// mutation: the GPX serializer borrows the committed Get response's arena. +template +class SdDownloadPort final : public ::geocaching::DownloadPort, private ::geocaching::GpxInstallPort +{ + public: + using Result = ::geocaching::DownloadOperationResult; + SdDownloadPort(SdRequestStore& store, ::geocaching::storage::LogicalState& state, + ::geocaching::protocol::RecordCrypto& crypto, const ::geocaching::Destination& local, + const ::geocaching::InstallIdentity& identity, const std::array& task, + const ::geocaching::storage::StoredTime& created) + : store_(store), state_(state), crypto_(crypto), local_(local), identity_(identity), task_(task), created_(created) {} + Result submit(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id, ::geocaching::ByteView request) override + { + if (phase_ != Phase::Idle) return Result::Rejected; + remote_ = remote; + request_ = id; + std::memcpy(key_.data(), local_.bytes.data(), 16); + std::memcpy(key_.data() + 16, remote.bytes.data(), 16); + std::memcpy(key_.data() + 32, id.bytes.data(), 16); + const ::geocaching::storage::RequestTaskTarget target{{identity_.id.bytes.data(), 32}, {identity_.hash.bytes.data(), 32}, identity_.generation}; + const auto result = store_.persistNewTask(local_, remote, id, task_, 2, request, created_, target); + if (result != JournalWriteResult::InProgress) return Result::Rejected; + phase_ = Phase::Submitting; + return Result::Pending; + } + Result commit(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id, uint64_t generation, + ::geocaching::ByteView response, const ::geocaching::protocol::VerifiedRecordView&) override + { + if (phase_ != Phase::Waiting || remote.bytes != remote_.bytes || id.bytes != request_.bytes || generation != identity_.generation) return Result::Rejected; + const auto result = store_.recordDownloadResponse(local_, remote, id, generation, response, crypto_); + if (result != JournalWriteResult::InProgress && result != JournalWriteResult::Verified) return Result::Rejected; + phase_ = result == JournalWriteResult::Verified ? Phase::StartStage : Phase::Receiving; + return Result::Pending; + } + Result resumeWaiting(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id) + { + if (phase_ != Phase::Idle) return Result::Rejected; + remote_ = remote; + request_ = id; + std::memcpy(key_.data(), local_.bytes.data(), 16); + std::memcpy(key_.data() + 16, remote.bytes.data(), 16); + std::memcpy(key_.data() + 32, id.bytes.data(), 16); + ::geocaching::ByteView bytes; + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::protocol::GetRequestView request; + if (!store_.downloadIntentActive(key(), identity_.generation) || !state_.view().find(5, key(), bytes) || + !::geocaching::storage::decodeOutgoing(key(), bytes, outgoing) || outgoing.state >= 4 || + !::geocaching::protocol::decodeGetRequest(outgoing.request, id, request) || request.wanted_hash.size != 32 || + std::memcmp(request.cache_id.data, identity_.id.bytes.data(), 32) || + std::memcmp(request.wanted_hash.data, identity_.hash.bytes.data(), 32) || + std::memcmp(outgoing.task_id.data, task_.data(), 16)) return fail(); + phase_ = Phase::Waiting; + return Result::Complete; + } + // Reconstruct an installation whose exact Get response is already durable. + // This never resubmits the network request or trusts an unverified file. + Result resume(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id) + { + if (phase_ != Phase::Idle) return Result::Rejected; + remote_ = remote; + request_ = id; + std::memcpy(key_.data(), local_.bytes.data(), 16); + std::memcpy(key_.data() + 16, remote.bytes.data(), 16); + std::memcpy(key_.data() + 32, id.bytes.data(), 16); + ::geocaching::protocol::VerifiedRecordView record; + if (!readRecord(record)) return fail(); + path(target_, "/trailmate/geocaching/caches/", identity_.id.bytes.data(), 32, ".gpx"); + path(staged_, "/trailmate/geocaching/.state/staging/", task_.data(), 16, ".gpx"); + path(backup_, "/trailmate/geocaching/.state/staging/", task_.data(), 16, ".old.gpx"); + ::geocaching::ByteView value; + if (!state_.view().find(12, {task_.data(), task_.size()}, value)) + { + phase_ = Phase::RecoverUnprepared; + return Result::Pending; + } + ::geocaching::storage::InstallRecordView install; + if (!::geocaching::storage::decodeInstallRecord({task_.data(), task_.size()}, value, install) || + install.phase != ::geocaching::storage::InstallPhase::Prepared || install.generation != identity_.generation || + std::memcmp(install.cache_id.data, identity_.id.bytes.data(), 32) || + std::memcmp(install.revision_hash.data, identity_.hash.bytes.data(), 32)) return fail(); + std::memcpy(new_hash_.data(), install.new_file_hash.data, 32); + old_present_ = install.old_file_hash.size != 0; + if (old_present_) + { + std::memcpy(old_hash_.data(), install.old_file_hash.data, 32); + std::array recorded; + if (!store_.installedFileProof(key(), identity_.generation, recorded, old_revision_) || recorded != old_hash_) return fail(); + } + phase_ = Phase::RecoverTarget; + return Result::Pending; + } + + Result poll() override + { + if (phase_ == Phase::Complete) return Result::Complete; + if (phase_ == Phase::Failed) return Result::Rejected; + if (!storage::sd_card_ready() || storage::sd_external_block_owner_active()) return Result::Pending; + if (phase_ == Phase::Submitting || phase_ == Phase::Receiving || phase_ == Phase::Stopping) + { + const auto result = store_.stepCommit(); + if (result == JournalWriteResult::InProgress) return Result::Pending; + if (result != JournalWriteResult::Verified) return fail(); + if (phase_ == Phase::Stopping) + { + phase_ = Phase::Complete; + return Result::Complete; + } + const bool submitted = phase_ == Phase::Submitting; + phase_ = submitted ? Phase::Waiting : Phase::StartStage; + if (cancel_requested_) return stop(); + return submitted ? Result::Complete : Result::Pending; + } + if (cancel_requested_) return stop(); + if (phase_ >= Phase::RecoverUnprepared) return recover(); + if (phase_ == Phase::StartStage) + { + ::geocaching::protocol::VerifiedRecordView record; + if (!readRecord(record)) return fail(); + stage_.reset(new (std::nothrow) SdGpxStage); + if (!stage_ || stage_->begin( + task_, record, crypto_, [](void* context, const uint8_t* data, size_t size) + { static_cast(context)->update(data, size); }, + &stage_digest_) != StageResult::InProgress) return fail(); + std::snprintf(staged_.data(), staged_.size(), "%s", stage_->path()); + path(target_, "/trailmate/geocaching/caches/", identity_.id.bytes.data(), 32, ".gpx"); + path(backup_, "/trailmate/geocaching/.state/staging/", task_.data(), 16, ".old.gpx"); + phase_ = Phase::Staging; + return Result::Pending; + } + if (phase_ == Phase::Staging) + { + if (!store_.downloadIntentActive(key(), identity_.generation)) return fail(); + const auto result = stage_->step(); + if (result == StageResult::InProgress) return Result::Pending; + if (result != StageResult::Written || !stage_digest_.finalize(new_hash_.data(), new_hash_.size())) return fail(); + stage_.reset(); + install_.reset(new (std::nothrow)::geocaching::GpxInstall(identity_)); + if (!install_) return fail(); + phase_ = Phase::Installing; + return Result::Pending; + } + if (phase_ == Phase::Installing) + { + install_->advance(*this); + if (install_->step() == ::geocaching::InstallStep::Failed || install_->step() == ::geocaching::InstallStep::Cancelled) return fail(); + if (install_->installed()) + { + phase_ = backup_moved_ ? Phase::RetainHistory : Phase::Complete; + return phase_ == Phase::Complete ? Result::Complete : Result::Pending; + } + return Result::Pending; + } + if (phase_ == Phase::RetainHistory) + { + std::array history; + path(history, "/trailmate/geocaching/.state/history/", old_revision_.bytes.data(), 32, ".gpx"); + if (!history_checked_) + { + if (storage::sd_exists(history.data())) + { + phase_ = Phase::Complete; + return Result::Complete; + } + history_checked_ = true; + return Result::Pending; + } + // The new target is already committed. On retention failure keep + // the backup and its Installed marker as recovery evidence. + if (storage::sd_rename(backup_.data(), history.data())) backup_moved_ = false; + phase_ = Phase::Complete; + return Result::Complete; + } + return Result::Rejected; + } + Result cancel(const ::geocaching::Destination& remote, const ::geocaching::RequestId& id, uint64_t generation) override + { + if (remote.bytes != remote_.bytes || id.bytes != request_.bytes || generation != identity_.generation || + phase_ == Phase::Complete || target_moved_ || backup_moved_) return Result::Rejected; + cancel_requested_ = true; + return Result::Pending; + } + const char* targetPath() const { return target_.data(); } + bool historyPending() const { return phase_ == Phase::Complete && backup_moved_; } + + private: + using Effect = ::geocaching::InstallEffectResult; + using Step = ::geocaching::InstallStep; + enum class Phase : uint8_t + { + Idle, + Submitting, + Waiting, + Receiving, + StartStage, + Staging, + Installing, + RetainHistory, + Stopping, + Complete, + Failed, + RecoverUnprepared, + RecoverOrphanName, + RecoverOrphanMove, + RecoverTarget, + RecoverTargetOpen, + RecoverTargetHash, + RecoverBackup, + RecoverBackupOpen, + RecoverBackupHash, + RecoverStage, + RecoverStageOpen, + RecoverStageHash + }; + ::geocaching::ByteView key() const { return {key_.data(), key_.size()}; } + Result fail() + { + phase_ = Phase::Failed; + return Result::Rejected; + } + Result stop() + { + if (store_.commitPending()) + { + const auto result = store_.stepCommit(); + if (result == JournalWriteResult::InProgress) return Result::Pending; + if (result != JournalWriteResult::Verified) return fail(); + } + if (stage_) + { + stage_->cancel(); + stage_.reset(); + return Result::Pending; + } + const auto stopped = store_.stopTask(task_); + if (stopped == JournalWriteResult::InProgress) + { + phase_ = Phase::Stopping; + return Result::Pending; + } + if (stopped != JournalWriteResult::Verified) return fail(); + phase_ = Phase::Complete; + return Result::Complete; + } + template + static void path(std::array& output, const char* prefix, const uint8_t* id, size_t size, const char* suffix) + { + constexpr char hex[] = "0123456789abcdef"; + size_t offset = std::strlen(prefix); + std::memcpy(output.data(), prefix, offset); + for (size_t i = 0; i < size; ++i) + { + output[offset++] = hex[id[i] >> 4]; + output[offset++] = hex[id[i] & 15]; + } + std::snprintf(output.data() + offset, output.size() - offset, "%s", suffix); + } + bool readRecord(::geocaching::protocol::VerifiedRecordView& record) + { + ::geocaching::ByteView bytes; + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::protocol::GetResponseView response; + if (!store_.downloadIntentActive(key(), identity_.generation) || !state_.view().find(5, key(), bytes) || + !::geocaching::storage::decodeOutgoing(key(), bytes, outgoing) || + !::geocaching::protocol::decodeGetResponse(outgoing.terminal_data, request_, 8192, response)) return false; + auto verified = ::geocaching::protocol::VerificationResult::WorkspaceTooSmall; + state_.withScratch([&](uint8_t* scratch, size_t capacity) + { verified = ::geocaching::protocol::verifyGeocache(response.signed_cache, crypto_, scratch, capacity, record, &identity_.id, &identity_.hash); }); + if (verified != ::geocaching::protocol::VerificationResult::Valid) return false; + + return true; + } + Result continueInstall() + { + recovered_files_ = true; + hasher_.reset(); + hash_digest_.reset(); + install_.reset(new (std::nothrow)::geocaching::GpxInstall(identity_)); + if (!install_) return fail(); + phase_ = Phase::Installing; + return Result::Pending; + } + Result recover() + { + if (!store_.downloadIntentActive(key(), identity_.generation)) return fail(); + std::array orphan{}; + if (phase_ == Phase::RecoverUnprepared) + { + phase_ = storage::sd_exists(staged_.data()) ? Phase::RecoverOrphanName : Phase::StartStage; + return Result::Pending; + } + if (phase_ == Phase::RecoverOrphanName || phase_ == Phase::RecoverOrphanMove) + { + const int length = std::snprintf(orphan.data(), orphan.size(), "%s.partial.%lu", staged_.data(), static_cast(orphan_number_)); + if (length < 0 || static_cast(length) >= orphan.size()) return fail(); + if (phase_ == Phase::RecoverOrphanName) + { + if (storage::sd_exists(orphan.data())) + { + if (orphan_number_ == UINT32_MAX) return fail(); + ++orphan_number_; + } + else phase_ = Phase::RecoverOrphanMove; + return Result::Pending; + } + if (!storage::sd_rename(staged_.data(), orphan.data())) return fail(); + phase_ = Phase::StartStage; + return Result::Pending; + } + if (phase_ == Phase::RecoverTarget) + { + phase_ = storage::sd_exists(target_.data()) ? Phase::RecoverTargetOpen : old_present_ ? Phase::RecoverBackup + : Phase::RecoverStage; + return Result::Pending; + } + if (phase_ == Phase::RecoverBackup) + { + if (!storage::sd_exists(backup_.data())) + { + // Identical-version downloads have identical old/new hashes. + // Without a backup, the verified target is still the old file; + // validate staging and perform the normal replacement sequence. + if (!target_moved_ || new_hash_ != old_hash_) return fail(); + target_moved_ = false; + phase_ = Phase::RecoverStage; + return Result::Pending; + } + phase_ = Phase::RecoverBackupOpen; + return Result::Pending; + } + if (phase_ == Phase::RecoverStage) + { + if (!storage::sd_exists(staged_.data())) return fail(); + phase_ = Phase::RecoverStageOpen; + return Result::Pending; + } + if (phase_ == Phase::RecoverTargetOpen || phase_ == Phase::RecoverBackupOpen || phase_ == Phase::RecoverStageOpen) + { + const bool opened = startHash(phase_ == Phase::RecoverTargetOpen ? target_.data() : phase_ == Phase::RecoverBackupOpen ? backup_.data() + : staged_.data()); + if (!opened) return fail(); + phase_ = phase_ == Phase::RecoverTargetOpen ? Phase::RecoverTargetHash : phase_ == Phase::RecoverBackupOpen ? Phase::RecoverBackupHash + : Phase::RecoverStageHash; + return Result::Pending; + } + if (hasher_->step() == GpxHashStep::Reading) return Result::Pending; + std::array actual; + if (!hasher_->result(actual)) return fail(); + if (phase_ == Phase::RecoverTargetHash) + { + if (actual == new_hash_) + { + target_moved_ = true; + if (old_present_) + { + phase_ = Phase::RecoverBackup; + return Result::Pending; + } + return continueInstall(); + } + if (!old_present_ || actual != old_hash_) return fail(); + phase_ = Phase::RecoverStage; + return Result::Pending; + } + if (phase_ == Phase::RecoverBackupHash) + { + if (actual != old_hash_) return fail(); + backup_moved_ = true; + if (target_moved_) return continueInstall(); + phase_ = Phase::RecoverStage; + return Result::Pending; + } + if (actual != new_hash_) return fail(); + return continueInstall(); + } + bool startHash(const char* filename) + { + hasher_.reset(); + hash_digest_.reset(new (std::nothrow) Digest); + if (!hash_digest_) return false; + hasher_.reset(new (std::nothrow) SdGpxHash(*hash_digest_)); + return hasher_ && hasher_->open(filename); + } + Effect execute(Step step, const ::geocaching::InstallIdentity&) override + { + if (!store_.downloadIntentActive(key(), identity_.generation)) return Effect::StaleIntent; + if (step != last_step_) + { + last_step_ = step; + substep_ = 0; + } + switch (step) + { + case Step::CheckIntent: + return Effect::Complete; + case Step::ValidateStagedFile: + if (recovered_files_) return Effect::Complete; + if (substep_ == 0) + { + old_present_ = storage::sd_exists(target_.data()); + if (!old_present_) return Effect::Complete; + if (!store_.installedFileProof(key(), identity_.generation, old_hash_, old_revision_)) return Effect::Failed; + ++substep_; + return Effect::Pending; + } + if (substep_ == 1) + { + if (!startHash(target_.data())) return Effect::Failed; + ++substep_; + return Effect::Pending; + } + if (hasher_->step() == GpxHashStep::Reading) return Effect::Pending; + { + std::array hash; + return hasher_->result(hash) && hash == old_hash_ ? Effect::Complete : Effect::Failed; + } + case Step::PrepareJournal: + { + const auto result = substep_++ == 0 ? store_.prepareDownloadInstall(key(), task_, identity_.generation, new_hash_, + old_present_ ? ::geocaching::ByteView{old_hash_.data(), old_hash_.size()} : ::geocaching::ByteView{}, crypto_) + : store_.stepCommit(); + return result == JournalWriteResult::InProgress ? Effect::Pending : result == JournalWriteResult::Verified ? Effect::Complete + : Effect::Failed; + } + case Step::ReplaceFile: + if (target_moved_) return Effect::Complete; + if (substep_ == 0) + { + if (old_present_ && !backup_moved_ && storage::sd_exists(backup_.data())) return Effect::Failed; + if ((!old_present_ || backup_moved_) && storage::sd_exists(target_.data())) return Effect::Failed; + ++substep_; + return Effect::Pending; + } + if (substep_ == 1 && old_present_ && !backup_moved_) + { + if (!storage::sd_rename(target_.data(), backup_.data())) return Effect::Failed; + backup_moved_ = true; + ++substep_; + return Effect::Pending; + } + if (!storage::sd_rename(staged_.data(), target_.data())) return Effect::Failed; + target_moved_ = true; + return Effect::Complete; + case Step::ValidateInstalledFile: + if (substep_++ == 0) return startHash(target_.data()) ? Effect::Pending : Effect::Failed; + if (hasher_->step() == GpxHashStep::Reading) return Effect::Pending; + { + std::array hash; + return hasher_->result(hash) && hash == new_hash_ ? Effect::Complete : Effect::Failed; + } + case Step::CommitJournal: + { + const auto result = substep_++ == 0 ? store_.finishDownloadInstall(key(), task_, identity_.generation, new_hash_) : store_.stepCommit(); + return result == JournalWriteResult::InProgress ? Effect::Pending : result == JournalWriteResult::Verified ? Effect::Complete + : Effect::Failed; + } + default: + return Effect::Failed; + } + } + SdRequestStore& store_; + ::geocaching::storage::LogicalState& state_; + ::geocaching::protocol::RecordCrypto& crypto_; + ::geocaching::Destination local_, remote_; + ::geocaching::RequestId request_; + ::geocaching::InstallIdentity identity_; + ::geocaching::RevisionHash old_revision_; + std::array task_; + ::geocaching::storage::StoredTime created_; + std::array key_{}; + std::array new_hash_{}, old_hash_{}; + std::array target_{}; + std::array staged_{}, backup_{}; + Digest stage_digest_; + std::unique_ptr hash_digest_; + std::unique_ptr> hasher_; + std::unique_ptr stage_; + std::unique_ptr<::geocaching::GpxInstall> install_; + Phase phase_ = Phase::Idle; + Step last_step_ = Step::Complete; + unsigned substep_ = 0; + uint32_t orphan_number_ = 0; + bool recovered_files_ = false; + bool old_present_ = false, target_moved_ = false, backup_moved_ = false, cancel_requested_ = false, history_checked_ = false; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_hash.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_hash.h new file mode 100644 index 00000000..08cf5495 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_hash.h @@ -0,0 +1,66 @@ +#pragma once +#include "geocaching/domain/record.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class GpxHashStep : uint8_t { Reading, Complete, InvalidSize, IoError }; + +// One-shot worker operation with a fresh platform Sha256Digest. Path must be +// locally derived by the store, never taken from a remote name or request. +// Computes exact file bytes only; it does not parse GPX or verify signatures. +template +class SdGpxHash +{ + public: + explicit SdGpxHash(Digest& digest) : digest_(digest) {} + bool open(const char* path) + { + if (started_ || !path) return false; + started_ = true; + if (!file_.open(path, "r")) { state_ = GpxHashStep::IoError; return false; } + length_ = file_.size(); + if (!length_ || length_ > ::geocaching::kMaxGpxBytes) + { file_.close(); state_ = GpxHashStep::InvalidSize; return false; } + return true; + } + GpxHashStep step() + { + if (state_ != GpxHashStep::Reading) return state_; + if (!file_.is_open()) return state_ = GpxHashStep::IoError; + if (offset_ == length_) + { + // A changed length invalidates this observation. The owning install + // operation must still recheck volume/generation before mutation. + const bool unchanged = file_.size() == length_; + file_.close(); + if (!unchanged || !digest_.finalize(hash_.data(), hash_.size())) return state_ = GpxHashStep::IoError; + return state_ = GpxHashStep::Complete; + } + const auto requested = static_cast(std::min(buffer_.size(), length_ - offset_)); + const int count = file_.read(buffer_.data(), requested); + if (count <= 0 || static_cast(count) > requested) + { file_.close(); return state_ = GpxHashStep::IoError; } + digest_.update(buffer_.data(), static_cast(count)); + offset_ += static_cast(count); + return state_; + } + bool result(std::array& out) const + { + out = {}; + if (state_ != GpxHashStep::Complete) return false; + out = hash_; return true; + } + + private: + Digest& digest_; + storage::SdRuntimeFile file_; + std::array buffer_{}; + std::array hash_{}; + uint64_t length_ = 0, offset_ = 0; + bool started_ = false; + GpxHashStep state_ = GpxHashStep::Reading; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_stage.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_stage.h new file mode 100644 index 00000000..53ec1107 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_gpx_stage.h @@ -0,0 +1,197 @@ +#pragma once +#include "geocaching/gpx/write_record.h" +#include "gps/gpx/window_sink.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class StageResult : uint8_t +{ + Written, + Unavailable, + AlreadyExists, + InvalidTransaction, + IoError, + InvalidRecord, + InProgress, + Cancelled +}; + +// The owner pins the verified response and crypto provider until terminal state. +// Written means staging was flushed and read back against the existing GPX +// serializer, not that the target file was installed. The optional observer +// receives verified readback chunks; its result is usable only after Written. +class SdGpxStage +{ + public: + using VerifiedBytes = void (*)(void*, const uint8_t*, size_t); + const char* path() const { return path_.data(); } + uint64_t bytes() const { return bytes_; } + StageResult begin(const std::array& transaction_id, + const ::geocaching::protocol::VerifiedRecordView& record, + ::geocaching::protocol::RecordCrypto& crypto, + VerifiedBytes verified_bytes = nullptr, void* context = nullptr) + { + if (started_) return StageResult::InvalidTransaction; + started_ = true; + bool nonzero = false; + char hex[33]{}; + constexpr char alphabet[] = "0123456789abcdef"; + for (size_t i = 0; i < transaction_id.size(); ++i) + { + nonzero = nonzero || transaction_id[i] != 0; + hex[i * 2] = alphabet[transaction_id[i] >> 4]; + hex[i * 2 + 1] = alphabet[transaction_id[i] & 15]; + } + if (!nonzero) return result_ = StageResult::InvalidTransaction; + ::gps::gpx::WindowSink count(0, nullptr, 0); + if (!::geocaching::gpx::writeGeocacheGpx(record, crypto, count) || + !count.total() || count.total() > ::geocaching::kMaxGpxBytes) return result_ = StageResult::InvalidRecord; + record_ = record; + crypto_ = &crypto; + bytes_ = count.total(); + verified_bytes_ = verified_bytes; + context_ = context; + std::snprintf(path_.data(), path_.size(), "/trailmate/geocaching/.state/staging/%s.gpx", hex); + return result_ = StageResult::InProgress; + } + StageResult step() + { + if (result_ != StageResult::InProgress) return result_; + if (phase_ == Phase::CloseFailure) + { + file_.close(); + opened_ = false; + return result_ = failure_; + } + if (!storage::sd_card_ready() || storage::sd_external_block_owner_active()) return fail(StageResult::Unavailable); + switch (phase_) + { + case Phase::Directory: + if (!storage::sd_is_directory("/trailmate/geocaching/.state/staging")) return fail(StageResult::Unavailable); + phase_ = Phase::Exists; + break; + case Phase::Exists: + if (storage::sd_exists(path_.data())) return fail(StageResult::AlreadyExists); + phase_ = Phase::Open; + break; + case Phase::Open: + if (!file_.open(path_.data(), "w")) return fail(StageResult::IoError); + opened_ = true; + phase_ = Phase::Write; + break; + case Phase::Write: + { + const size_t remaining = static_cast(bytes_ - offset_); + const size_t count = remaining < buffer_.size() ? remaining : buffer_.size(); + ::gps::gpx::WindowSink window(offset_, buffer_.data(), count); + if (!::geocaching::gpx::writeGeocacheGpx(record_, *crypto_, window) || + window.total() != bytes_ || window.written() != count) return fail(StageResult::InvalidRecord); + if (file_.write(buffer_.data(), count) != count) return fail(StageResult::IoError); + offset_ += count; + if (offset_ == bytes_) phase_ = Phase::Flush; + break; + } + case Phase::Flush: + if (!file_.flush()) return fail(StageResult::IoError); + phase_ = Phase::Size; + break; + case Phase::Size: + if (file_.size() != bytes_) return fail(StageResult::IoError); + phase_ = Phase::Close; + break; + case Phase::Close: + file_.close(); + opened_ = false; + phase_ = Phase::OpenRead; + break; + case Phase::OpenRead: + if (!file_.open(path_.data(), "r")) return fail(StageResult::IoError); + opened_ = true; + offset_ = 0; + phase_ = Phase::ReadSize; + break; + case Phase::ReadSize: + if (file_.size() != bytes_) return fail(StageResult::IoError); + phase_ = Phase::Read; + break; + case Phase::Read: + { + const size_t remaining = static_cast(bytes_ - offset_); + const size_t count = remaining < buffer_.size() ? remaining : buffer_.size(); + if (file_.read(buffer_.data(), count) != static_cast(count)) return fail(StageResult::IoError); + ::gps::gpx::WindowSink compare(offset_, {reinterpret_cast(buffer_.data()), count}); + if (!::geocaching::gpx::writeGeocacheGpx(record_, *crypto_, compare) || + compare.total() != bytes_ || compare.written() != count) return fail(StageResult::IoError); + if (verified_bytes_) verified_bytes_(context_, buffer_.data(), count); + offset_ += count; + if (offset_ == bytes_) phase_ = Phase::FinalSize; + break; + } + case Phase::FinalSize: + if (file_.size() != bytes_) return fail(StageResult::IoError); + phase_ = Phase::CloseRead; + break; + case Phase::CloseRead: + file_.close(); + opened_ = false; + return result_ = StageResult::Written; + case Phase::CloseFailure: + break; + } + return result_; + } + StageResult cancel() + { + if (result_ != StageResult::InProgress) return result_; + if (opened_) + { + file_.close(); + opened_ = false; + } + return result_ = StageResult::Cancelled; + } + + private: + StageResult fail(StageResult result) + { + if (opened_) + { + failure_ = result; + phase_ = Phase::CloseFailure; + return result_; + } + return result_ = result; + } + enum class Phase : uint8_t + { + Directory, + Exists, + Open, + Write, + Flush, + Size, + Close, + OpenRead, + ReadSize, + Read, + FinalSize, + CloseRead, + CloseFailure + }; + storage::SdRuntimeFile file_; + ::geocaching::protocol::VerifiedRecordView record_; + ::geocaching::protocol::RecordCrypto* crypto_ = nullptr; + VerifiedBytes verified_bytes_ = nullptr; + void* context_ = nullptr; + std::array buffer_{}; + std::array path_{}; + uint64_t bytes_ = 0; + size_t offset_ = 0; + Phase phase_ = Phase::Directory; + StageResult result_ = StageResult::InvalidTransaction, failure_ = StageResult::IoError; + bool started_ = false, opened_ = false; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_append.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_append.h new file mode 100644 index 00000000..0d9be6de --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_append.h @@ -0,0 +1,166 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_index_path.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexAppendStep : uint8_t +{ + Idle, + Working, + Verified, + Invalid, + IoError, + VolumeChanged +}; + +// Owner prepares the slot/table directories and supplies a validated journal +// locator. Verified applies to ONE entry only: never publish the transaction +// watermark until every entry has reached Verified. Partial tails require rebuild. +class SdIndexAppend +{ + public: + SdIndexAppend(const ::geocaching::storage::VolumeInstance& volume, char slot) : volume_(volume), slot_(slot) {} + uint64_t writtenLength() const { return result_ == IndexAppendStep::Verified ? length_ + (duplicate_ ? 0 : ::geocaching::storage::kIndexEntrySize) : 0; } + bool begin(const ::geocaching::storage::IndexedMutation& entry) + { + if (result_ != IndexAppendStep::Idle || (slot_ != 'a' && slot_ != 'b') || !::geocaching::storage::encodeIndexEntry(volume_, entry, bytes_)) return false; + result_ = IndexAppendStep::Working; + return true; + } + IndexAppendStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexAppendStep::Working) return result_; + if (phase_ == Phase::Volume || phase_ == Phase::VerifyVolume) + { + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexAppendStep::IoError); + if (current != volume_) return fail(IndexAppendStep::VolumeChanged); + if (phase_ == Phase::VerifyVolume) return result_ = IndexAppendStep::Verified; + phase_ = Phase::OpenWrite; + return result_; + } + if (phase_ == Phase::OpenWrite || phase_ == Phase::OpenRead) + { + char path[80]; + if (!indexShardPath(slot_, bytes_[20], {bytes_.data() + 52, bytes_[21]}, path, sizeof(path)) || + !file_.open(path, phase_ == Phase::OpenWrite ? "a+" : "r")) return fail(IndexAppendStep::IoError); + phase_ = phase_ == Phase::OpenWrite ? Phase::Size : Phase::SeekVerify; + return result_; + } + if (phase_ == Phase::Size) + { + length_ = file_.size(); + if (length_ % kIndexEntrySize || length_ > UINT64_MAX - kIndexEntrySize) return fail(IndexAppendStep::Invalid); + verify_offset_ = length_; + tail_offset_ = length_ ? length_ - kIndexEntrySize : 0; + phase_ = length_ ? Phase::SeekTail : Phase::Write; + return result_; + } + if (phase_ == Phase::SeekTail || phase_ == Phase::SeekVerify) + { + const bool tail = phase_ == Phase::SeekTail; + if (!file_.seek(tail ? tail_offset_ : verify_offset_)) return fail(IndexAppendStep::IoError); + read_ = 0; + phase_ = tail ? Phase::ReadTail : Phase::ReadVerify; + return result_; + } + if (phase_ == Phase::ReadTail || phase_ == Phase::ReadVerify) + { + const int count = file_.read(verify_.data() + read_, verify_.size() - read_); + if (count < 0) return fail(IndexAppendStep::IoError); + if (!count || static_cast(count) > verify_.size() - read_) return fail(IndexAppendStep::Invalid); + read_ += static_cast(count); + if (read_ != verify_.size()) return result_; + if (phase_ == Phase::ReadVerify) + { + if (verify_ != bytes_) return fail(IndexAppendStep::Invalid); + phase_ = Phase::CheckSize; + return result_; + } + IndexedMutation previous, next; + if (!decodeIndexEntry({verify_.data(), verify_.size()}, volume_, previous) || + !decodeIndexEntry({bytes_.data(), bytes_.size()}, volume_, next) || previous.table != next.table || + (::sys::crc32(previous.key.data, previous.key.size) & 0xff) != (::sys::crc32(next.key.data, next.key.size) & 0xff) || + previous.location.record_sequence > next.location.record_sequence) return fail(IndexAppendStep::Invalid); + if (previous.location.record_sequence == next.location.record_sequence && previous.key.size == next.key.size && + !std::memcmp(previous.key.data, next.key.data, next.key.size)) + { + if (verify_ != bytes_) return fail(IndexAppendStep::Invalid); + duplicate_ = true; + verify_offset_ = tail_offset_; + phase_ = Phase::Flush; + } + else if (previous.location.record_sequence == next.location.record_sequence && tail_offset_) + { + tail_offset_ -= kIndexEntrySize; + phase_ = Phase::SeekTail; + } + else phase_ = Phase::Write; + return result_; + } + if (phase_ == Phase::Write) + { + if (file_.write(bytes_.data(), bytes_.size()) != bytes_.size()) return fail(IndexAppendStep::IoError); + phase_ = Phase::Flush; + return result_; + } + if (phase_ == Phase::Flush) + { + if (!file_.flush()) return fail(IndexAppendStep::IoError); + phase_ = Phase::CloseWrite; + return result_; + } + if (phase_ == Phase::CloseWrite) + { + file_.close(); + phase_ = Phase::OpenRead; + return result_; + } + if (phase_ == Phase::CheckSize) + { + if (file_.size() != length_ + (duplicate_ ? 0 : kIndexEntrySize)) return fail(IndexAppendStep::Invalid); + phase_ = Phase::CloseRead; + return result_; + } + file_.close(); + phase_ = Phase::VerifyVolume; + return result_; + } + + private: + enum class Phase : uint8_t + { + Volume, + OpenWrite, + Size, + SeekTail, + ReadTail, + Write, + Flush, + CloseWrite, + OpenRead, + SeekVerify, + ReadVerify, + CheckSize, + CloseRead, + VerifyVolume + }; + IndexAppendStep fail(IndexAppendStep result) + { + file_.close(); + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexEntryBytes bytes_{}, verify_{}; + storage::SdRuntimeFile file_; + uint64_t length_ = 0, verify_offset_ = 0, tail_offset_ = 0; + uint16_t read_ = 0; + char slot_; + bool duplicate_ = false; + Phase phase_ = Phase::Volume; + IndexAppendStep result_ = IndexAppendStep::Idle; +}; +static_assert(sizeof(SdIndexAppend) <= 384, "Index append retains metadata only"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_get.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_get.h new file mode 100644 index 00000000..91b826bd --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_get.h @@ -0,0 +1,130 @@ +#pragma once +#include "geocaching/storage/index_root.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_lookup.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexGetStep : uint8_t +{ + Idle, + Working, + Ready, + NotFound, + Invalid, + IoError, + VolumeChanged, + WorkspaceTooSmall +}; + +// The owner pins a validated root for the read and supplies a disjoint frame +// lease. Operation workspaces overlap; neither values nor the full index live +// in this object. Keep it in owner storage, not an ESP task-stack local. +class SdIndexGet +{ + public: + explicit SdIndexGet(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, uint8_t table, ::geocaching::ByteView key, + uint8_t* frame, size_t capacity) + { + if (result_ != IndexGetStep::Idle || !::geocaching::storage::validIndexRoot(root) || table < 1 || table > 13 || + !key.data || !key.size || key.size > key_.size() || !frame || capacity < 24) return false; + const auto input = reinterpret_cast(root.shards.data), output = reinterpret_cast(frame); + if (input <= output ? output - input < root.shards.size : input - output < capacity) return false; + root_ = root; + table_ = table; + key_size_ = key.size; + std::memcpy(key_.data(), key.data, key_size_); + frame_ = frame; + capacity_ = capacity; + const auto bucket = static_cast(::sys::crc32(key_.data(), key_size_)); + if (::geocaching::storage::indexHasShard(root_, table_, bucket)) + { + if (!operation_.emplace(volume_, root_.slot, root_.epoch, root_.sequence).begin(table_, {key_.data(), key_size_})) return false; + phase_ = Phase::Head; + } + result_ = IndexGetStep::Working; + return true; + } + ::geocaching::ByteView value() const { return result_ == IndexGetStep::Ready ? value_ : ::geocaching::ByteView{}; } + IndexGetStep step() + { + if (result_ != IndexGetStep::Working) return result_; + if (phase_ == Phase::Absent) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexGetStep::IoError); + return fail(current == volume_ ? IndexGetStep::NotFound : IndexGetStep::VolumeChanged); + } + if (phase_ == Phase::Head) + { + auto& reader = std::get(operation_); + const auto status = reader.step(); + if (status == IndexHeadReadStep::Working) return result_; + ::geocaching::storage::IndexShardHead head; + if (status != IndexHeadReadStep::Ready || !reader.selected(head)) return error(status); + if (!operation_.emplace(volume_, root_.slot, head.sequence, head.length).begin(table_, {key_.data(), key_size_})) return fail(IndexGetStep::Invalid); + phase_ = Phase::Lookup; + return result_; + } + if (phase_ == Phase::Lookup) + { + auto& lookup = std::get(operation_); + const auto status = lookup.step(); + if (status == IndexLookupStep::Working) return result_; + if (status == IndexLookupStep::NotFound) return fail(IndexGetStep::NotFound); + ::geocaching::storage::IndexedMutation hint; + if (status != IndexLookupStep::Found || !lookup.result(hint)) return error(status); + // The lookup workspace is about to be destroyed; pin the key in + // this owner before constructing the next operation in its place. + hint.key = {key_.data(), key_size_}; + if (!operation_.emplace(volume_).begin(hint, frame_, capacity_)) return fail(IndexGetStep::Invalid); + phase_ = Phase::Value; + return result_; + } + auto& reader = std::get(operation_); + const auto status = reader.step(); + if (status == IndexedReadStep::Working) return result_; + if (status == IndexedReadStep::WorkspaceTooSmall) return fail(IndexGetStep::WorkspaceTooSmall); + if (status == IndexedReadStep::Erased) return fail(IndexGetStep::NotFound); + if (status != IndexedReadStep::Ready) return error(status); + value_ = reader.value(); + result_ = IndexGetStep::Ready; + return result_; + } + + private: + enum class Phase : uint8_t + { + Absent, + Head, + Lookup, + Value + }; + template + IndexGetStep error(Status status) + { + return fail(status == Status::VolumeChanged ? IndexGetStep::VolumeChanged : status == Status::IoError ? IndexGetStep::IoError + : IndexGetStep::Invalid); + } + IndexGetStep fail(IndexGetStep result) + { + operation_.emplace(); + value_ = {}; + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + std::array key_{}; + size_t key_size_ = 0, capacity_ = 0; + uint8_t* frame_ = nullptr; + ::geocaching::ByteView value_; + std::variant operation_; + uint8_t table_ = 0; + Phase phase_ = Phase::Absent; + IndexGetStep result_ = IndexGetStep::Idle; +}; +static_assert(sizeof(SdIndexGet) <= 640, "Indexed gets must overlay their metadata workspaces"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_reader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_reader.h new file mode 100644 index 00000000..438b0960 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_reader.h @@ -0,0 +1,86 @@ +#pragma once +#include "geocaching/storage/index_shard_head.h" +#include "platform/esp/arduino_common/geocaching/sd_index_path.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexHeadReadStep : uint8_t +{ + Idle, + Working, + Ready, + Invalid, + IoError, + VolumeChanged +}; + +class SdIndexHeadReader +{ + public: + SdIndexHeadReader(const ::geocaching::storage::VolumeInstance& volume, char slot, uint64_t epoch, uint64_t visible_sequence) + : volume_(volume), epoch_(epoch), visible_(visible_sequence), slot_(slot) {} + bool begin(uint8_t table, ::geocaching::ByteView key) + { + return key.data && key.size && key.size <= 96 && beginBucket(table, static_cast(::sys::crc32(key.data, key.size))); + } + bool beginBucket(uint8_t table, uint8_t bucket) + { + if (result_ != IndexHeadReadStep::Idle || !epoch_ || !indexShardHeadPathForBucket(slot_, table, bucket, 0, path_.data(), path_.size())) return false; + table_ = table; + bucket_ = bucket; + result_ = IndexHeadReadStep::Working; + return true; + } + bool selected(::geocaching::storage::IndexShardHead& out) const + { + out = {}; + if (result_ != IndexHeadReadStep::Ready) return false; + out = selected_; + return true; + } + int selectedCopy() const + { + if (result_ != IndexHeadReadStep::Ready) return -1; + return selected_.sequence == heads_[0].sequence ? 0 : 1; + } + IndexHeadReadStep step() + { + if (result_ != IndexHeadReadStep::Working) return result_; + if (phase_ == 0 || phase_ == 3) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return result_ = IndexHeadReadStep::IoError; + if (current != volume_) return result_ = IndexHeadReadStep::VolumeChanged; + if (phase_ == 3) return result_ = IndexHeadReadStep::Ready; + phase_ = 1; + return result_; + } + const unsigned copy = phase_ - 1; + path_[std::strlen(path_.data()) - 1] = static_cast('0' + copy); + const auto read = storage::sd_read_file(path_.data(), bytes_.data(), bytes_.size()); + if (read.status == storage::SdFileReadStatus::Missing || read.status == storage::SdFileReadStatus::Invalid) + return result_ = IndexHeadReadStep::Invalid; + if (read.status != storage::SdFileReadStatus::Ready) return result_ = IndexHeadReadStep::IoError; + if (read.file_size != bytes_.size() || read.bytes_read != bytes_.size() || + !::geocaching::storage::decodeIndexShardHead({bytes_.data(), bytes_.size()}, volume_, epoch_, table_, bucket_, heads_[copy])) + return result_ = IndexHeadReadStep::Invalid; + if (copy == 1 && !::geocaching::storage::selectIndexShardHead(heads_[0], heads_[1], visible_, selected_)) + return result_ = IndexHeadReadStep::Invalid; + ++phase_; + return result_; + } + + private: + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexShardHeadBytes bytes_{}; + std::array<::geocaching::storage::IndexShardHead, 2> heads_{}; + ::geocaching::storage::IndexShardHead selected_; + std::array path_{}; + uint64_t epoch_, visible_; + char slot_; + uint8_t table_ = 0, bucket_ = 0, phase_ = 0; + IndexHeadReadStep result_ = IndexHeadReadStep::Idle; +}; +static_assert(sizeof(SdIndexHeadReader) <= 288, "Shard head readers retain metadata only"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_writer.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_writer.h new file mode 100644 index 00000000..461cb8d2 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_head_writer.h @@ -0,0 +1,102 @@ +#pragma once +#include "geocaching/storage/index_shard_head.h" +#include "platform/esp/arduino_common/geocaching/sd_index_path.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexHeadWriteStep : uint8_t +{ + Idle, + Working, + Verified, + Invalid, + IoError, + VolumeChanged +}; + +// Owner chooses the inactive copy after reading BOTH heads, or initializes +// both empty copies in an unpublished generation. This does not publish a +// global watermark and must never overwrite the only committed boundary. +class SdIndexHeadWriter +{ + public: + SdIndexHeadWriter(const ::geocaching::storage::VolumeInstance& volume, char slot) : volume_(volume), slot_(slot) {} + bool begin(::geocaching::ByteView key, unsigned copy, const ::geocaching::storage::IndexShardHead& head) + { + if (result_ != IndexHeadWriteStep::Idle || !key.data || !key.size || key.size > 96 || + head.bucket != static_cast(::sys::crc32(key.data, key.size)) || + !indexShardHeadPath(slot_, head.table, key, copy, path_.data(), path_.size()) || + !::geocaching::storage::encodeIndexShardHead(volume_, head, bytes_)) return false; + result_ = IndexHeadWriteStep::Working; + return true; + } + IndexHeadWriteStep step() + { + if (result_ != IndexHeadWriteStep::Working) return result_; + if (phase_ == Phase::Volume || phase_ == Phase::VerifyVolume) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexHeadWriteStep::IoError); + if (current != volume_) return fail(IndexHeadWriteStep::VolumeChanged); + if (phase_ == Phase::VerifyVolume) return result_ = IndexHeadWriteStep::Verified; + phase_ = Phase::Open; + return result_; + } + if (phase_ == Phase::Open) + { + if (!file_.open(path_.data(), "w")) return fail(IndexHeadWriteStep::IoError); + phase_ = Phase::Write; + return result_; + } + if (phase_ == Phase::Write) + { + if (file_.write(bytes_.data(), bytes_.size()) != bytes_.size()) return fail(IndexHeadWriteStep::IoError); + phase_ = Phase::Flush; + return result_; + } + if (phase_ == Phase::Flush) + { + if (!file_.flush()) return fail(IndexHeadWriteStep::IoError); + phase_ = Phase::Close; + return result_; + } + if (phase_ == Phase::Close) + { + file_.close(); + phase_ = Phase::Readback; + return result_; + } + const auto read = storage::sd_read_file(path_.data(), verify_.data(), verify_.size()); + if (read.status != storage::SdFileReadStatus::Ready) return fail(IndexHeadWriteStep::IoError); + if (read.file_size != bytes_.size() || read.bytes_read != bytes_.size() || verify_ != bytes_) return fail(IndexHeadWriteStep::Invalid); + phase_ = Phase::VerifyVolume; + return result_; + } + + private: + enum class Phase : uint8_t + { + Volume, + Open, + Write, + Flush, + Close, + Readback, + VerifyVolume + }; + IndexHeadWriteStep fail(IndexHeadWriteStep result) + { + file_.close(); + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexShardHeadBytes bytes_{}, verify_{}; + std::array path_{}; + storage::SdRuntimeFile file_; + char slot_; + Phase phase_ = Phase::Volume; + IndexHeadWriteStep result_ = IndexHeadWriteStep::Idle; +}; +static_assert(sizeof(SdIndexHeadWriter) <= 224, "Boundary writes retain metadata only"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_initialize.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_initialize.h new file mode 100644 index 00000000..77e6c972 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_initialize.h @@ -0,0 +1,101 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_index_root_writer.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +// Creates an empty derived index only when no index directory exists. It does +// not declare the authoritative journal empty: the owner must replay it before +// admitting requests. Interrupted initialization requires explicit recovery. +class SdIndexInitialize +{ + public: + explicit SdIndexInitialize(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(::geocaching::storage::IndexRootBytes& bytes) + { + if (result_ != IndexRootWriteStep::Idle) return false; + bytes.fill(0); + const ::geocaching::storage::IndexRootView empty{1, 0, 1, 'a', {bytes.data() + 48, ::geocaching::storage::kIndexShardBitmapSize}}; + if (!::geocaching::storage::encodeIndexRoot(volume_, empty, bytes)) return false; + bytes_ = &bytes; + result_ = IndexRootWriteStep::Working; + return true; + } + IndexRootWriteStep step() + { + if (result_ != IndexRootWriteStep::Working) return result_; + if (phase_ == 0) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return finish(IndexRootWriteStep::IoError); + if (current != volume_) return finish(IndexRootWriteStep::VolumeChanged); + ++phase_; + return result_; + } + if (phase_ == 1) + { + if (storage::sd_exists("/trailmate/geocaching/.state/index")) return finish(IndexRootWriteStep::Invalid); + ++phase_; + return result_; + } + if (phase_ == 2) + { + if (storage::sd_is_directory("/trailmate/geocaching/.state/index")) return finish(IndexRootWriteStep::Invalid); + ++phase_; + return result_; + } + if (phase_ == 3) + { + uint8_t header[24]; + const auto probe = storage::sd_read_file("/trailmate/geocaching/.state/checkpoint/a.gcs", header, sizeof(header)); + if (probe.status != storage::SdFileReadStatus::Missing) + return finish(probe.status == storage::SdFileReadStatus::Ready || probe.status == storage::SdFileReadStatus::Invalid + ? IndexRootWriteStep::Invalid + : IndexRootWriteStep::IoError); + ++phase_; + return result_; + } + if (phase_ == 4) + { + uint8_t header[24]; + const auto probe = storage::sd_read_file("/trailmate/geocaching/.state/checkpoint/b.gcs", header, sizeof(header)); + if (probe.status != storage::SdFileReadStatus::Missing) + return finish(probe.status == storage::SdFileReadStatus::Ready || probe.status == storage::SdFileReadStatus::Invalid + ? IndexRootWriteStep::Invalid + : IndexRootWriteStep::IoError); + ++phase_; + return result_; + } + if (phase_ == 5) + { + if (!storage::sd_mkdir("/trailmate/geocaching/.state/index")) return finish(IndexRootWriteStep::IoError); + if (!writer_.emplace(volume_).begin(0, *bytes_)) return finish(IndexRootWriteStep::Invalid); + ++phase_; + return result_; + } + const auto status = writer_->step(); + if (status == IndexRootWriteStep::Working) return result_; + if (status != IndexRootWriteStep::Verified) return finish(status); + if (phase_ == 6) + { + if (!writer_.emplace(volume_).begin(1, *bytes_)) return finish(IndexRootWriteStep::Invalid); + ++phase_; + return result_; + } + return finish(IndexRootWriteStep::Verified); + } + + private: + IndexRootWriteStep finish(IndexRootWriteStep status) + { + writer_.reset(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootBytes* bytes_ = nullptr; + std::optional writer_; + uint8_t phase_ = 0; + IndexRootWriteStep result_ = IndexRootWriteStep::Idle; +}; +static_assert(sizeof(SdIndexInitialize) <= 192, "Index initialization borrows root storage"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_lookup.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_lookup.h new file mode 100644 index 00000000..856904db --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_lookup.h @@ -0,0 +1,159 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_index_path.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexLookupStep : uint8_t +{ + Idle, + Working, + Found, + NotFound, + Invalid, + IoError, + VolumeChanged +}; + +// Owner supplies the sequence/length of the selected, validated IndexShardHead, +// not the global journal watermark. Missing +// expected shards are invalid, not proof of absence. Returned locations remain +// hints and must pass SdIndexedValueReader before a value is trusted. +class SdIndexLookup +{ + public: + SdIndexLookup(const ::geocaching::storage::VolumeInstance& volume, char slot, uint64_t committed_sequence, uint64_t committed_length) + : volume_(volume), visible_(committed_sequence), position_(committed_length), length_(committed_length), slot_(slot) {} + bool begin(uint8_t table, ::geocaching::ByteView key) + { + if (result_ != IndexLookupStep::Idle || (slot_ != 'a' && slot_ != 'b') || table < 1 || table > 13 || + !key.data || !key.size || key.size > key_.size() || length_ % ::geocaching::storage::kIndexEntrySize || + ((visible_ == 0) != (length_ == 0))) return false; + table_ = table; + key_size_ = key.size; + std::memcpy(key_.data(), key.data, key_size_); + result_ = IndexLookupStep::Working; + return true; + } + bool result(::geocaching::storage::IndexedMutation& out) const + { + out = {}; + if (result_ != IndexLookupStep::Found) return false; + out = {table_, {key_.data(), key_size_}, location_, erased_}; + return true; + } + uint64_t entryOffset() const { return result_ == IndexLookupStep::Found ? position_ : UINT64_MAX; } + IndexLookupStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexLookupStep::Working) return result_; + if (phase_ == Phase::Volume || phase_ == Phase::VerifyVolume) + { + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexLookupStep::IoError); + if (current != volume_) return fail(IndexLookupStep::VolumeChanged); + if (phase_ == Phase::VerifyVolume) return result_ = completion_; + phase_ = length_ ? Phase::Probe : Phase::VerifyVolume; + return result_; + } + if (phase_ == Phase::Probe || phase_ == Phase::Open) + { + char path[80]; + if (!indexShardPath(slot_, table_, {key_.data(), key_size_}, path, sizeof(path))) return fail(IndexLookupStep::Invalid); + if (phase_ == Phase::Probe) + { + const auto probe = storage::sd_read_file(path, bytes_.data(), 1); + if (probe.status == storage::SdFileReadStatus::Missing) return fail(IndexLookupStep::Invalid); + if (probe.status != storage::SdFileReadStatus::Ready && probe.status != storage::SdFileReadStatus::Invalid) return fail(IndexLookupStep::IoError); + if (probe.file_size < length_) return fail(IndexLookupStep::Invalid); + phase_ = Phase::Open; + return result_; + } + if (!file_.open(path, "r")) return fail(IndexLookupStep::IoError); + phase_ = Phase::Seek; + return result_; + } + if (phase_ == Phase::Seek) + { + if (!position_) + { + completion_ = IndexLookupStep::NotFound; + phase_ = Phase::CheckSize; + return result_; + } + position_ -= kIndexEntrySize; + if (!file_.seek(position_)) return fail(IndexLookupStep::IoError); + read_ = 0; + phase_ = Phase::Read; + return result_; + } + if (phase_ == Phase::Read) + { + const int count = file_.read(bytes_.data() + read_, bytes_.size() - read_); + if (count < 0) return fail(IndexLookupStep::IoError); + if (!count || static_cast(count) > bytes_.size() - read_) return fail(IndexLookupStep::Invalid); + read_ += static_cast(count); + if (read_ != bytes_.size()) return result_; + IndexedMutation entry; + if (!decodeIndexEntry({bytes_.data(), bytes_.size()}, volume_, entry) || entry.location.record_sequence > newer_sequence_ || + (first_entry_ && entry.location.record_sequence != visible_) || + entry.table != table_ || (::sys::crc32(entry.key.data, entry.key.size) & 0xff) != (::sys::crc32(key_.data(), key_size_) & 0xff)) + return fail(IndexLookupStep::Invalid); + newer_sequence_ = entry.location.record_sequence; + first_entry_ = false; + if (entry.location.record_sequence <= visible_ && entry.table == table_ && entry.key.size == key_size_ && + !std::memcmp(entry.key.data, key_.data(), key_size_)) + { + location_ = entry.location; + erased_ = entry.erase; + completion_ = IndexLookupStep::Found; + phase_ = Phase::CheckSize; + } + else phase_ = Phase::Seek; + return result_; + } + if (phase_ == Phase::CheckSize) + { + if (file_.size() < length_) return fail(IndexLookupStep::Invalid); + phase_ = Phase::Close; + return result_; + } + file_.close(); + phase_ = Phase::VerifyVolume; + return result_; + } + + private: + enum class Phase : uint8_t + { + Volume, + Probe, + Open, + Seek, + Read, + CheckSize, + Close, + VerifyVolume + }; + IndexLookupStep fail(IndexLookupStep value) + { + file_.close(); + return result_ = value; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexEntryBytes bytes_{}; + std::array key_{}; + ::geocaching::storage::JournalValueLocation location_; + uint64_t visible_, position_ = 0, length_ = 0, newer_sequence_ = UINT64_MAX; + storage::SdRuntimeFile file_; + size_t key_size_ = 0; + uint16_t read_ = 0; + char slot_; + uint8_t table_ = 0; + bool erased_ = false; + bool first_entry_ = true; + Phase phase_ = Phase::Volume; + IndexLookupStep result_ = IndexLookupStep::Idle, completion_ = IndexLookupStep::NotFound; +}; +static_assert(sizeof(SdIndexLookup) <= 384, "Index lookup must not load a whole shard"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_path.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_path.h new file mode 100644 index 00000000..9b0535eb --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_path.h @@ -0,0 +1,31 @@ +#pragma once +#include "geocaching/storage/index_entry.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +inline bool indexShardPathForBucket(char slot, uint8_t table, uint8_t bucket, char* out, size_t capacity) +{ + if ((slot != 'a' && slot != 'b') || table < 1 || table > 13 || !out) return false; + const int size = std::snprintf(out, capacity, "/trailmate/geocaching/.state/index/%c/%02x/%02x.gci", slot, + static_cast(table), static_cast(bucket)); + return size > 0 && static_cast(size) < capacity; +} +inline bool indexShardPath(char slot, uint8_t table, ::geocaching::ByteView key, char* out, size_t capacity) +{ + return key.data && key.size && key.size <= 96 && + indexShardPathForBucket(slot, table, static_cast(::sys::crc32(key.data, key.size)), out, capacity); +} +inline bool indexShardHeadPathForBucket(char slot, uint8_t table, uint8_t bucket, unsigned copy, char* out, size_t capacity) +{ + if (copy > 1 || !indexShardPathForBucket(slot, table, bucket, out, capacity)) return false; + const auto length = std::strlen(out); + const int suffix = std::snprintf(out + length, capacity - length, ".h%u", copy); + return suffix > 0 && static_cast(suffix) < capacity - length; +} +inline bool indexShardHeadPath(char slot, uint8_t table, ::geocaching::ByteView key, unsigned copy, char* out, size_t capacity) +{ + return key.data && key.size && key.size <= 96 && indexShardHeadPathForBucket(slot, table, static_cast(::sys::crc32(key.data, key.size)), copy, out, capacity); +} +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_references.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_references.h new file mode 100644 index 00000000..5accc5db --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_references.h @@ -0,0 +1,176 @@ +#pragma once +#include "geocaching/storage/task_references.h" +#include "geocaching/storage/tx_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_index_get.h" +#include "platform/esp/arduino_common/geocaching/sd_index_scan.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +// Pins one immutable root and one frame lease. The owner must not publish a +// new root during validation. Allocate in owner storage, never on an ESP stack. +class SdIndexReferences +{ + public: + explicit SdIndexReferences(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + + // Pending mutations are borrowed and immutable until completion. Their + // storage must be disjoint from the reusable read frame. + bool begin(const ::geocaching::storage::IndexRootView& root, uint8_t* frame, size_t capacity, + const ::geocaching::storage::MutationView* mutations = nullptr, size_t count = 0) + { + if (result_ != IndexScanStep::Idle || count > 64 || (count && !mutations)) return false; + auto overlaps = [&](const void* data, size_t size) + { + if (!size) return false; + const auto a = reinterpret_cast(data), b = reinterpret_cast(frame); + return a <= b ? b - a < size : a - b < capacity; + }; + if (overlaps(mutations, count * sizeof(*mutations))) return false; + for (size_t i = 0; i < count; ++i) + { + const auto& row = mutations[i]; + if (!::geocaching::storage::validStoredRowShape(row) || overlaps(row.key.data, row.key.size) || + overlaps(row.value.data, row.value.size)) return false; + for (size_t j = 0; j < i; ++j) + if (mutations[j].table == row.table && mutations[j].key.size == row.key.size && + !std::memcmp(mutations[j].key.data, row.key.data, row.key.size)) return false; + } + if (!scan_.emplace(volume_).begin(root, 5, frame, capacity)) return false; + mutations_ = mutations; + mutation_count_ = count; + root_ = root; + frame_ = frame; + capacity_ = capacity; + result_ = IndexScanStep::Working; + return true; + } + + // End means every live task/outgoing/attempt reference was checked. + IndexScanStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexScanStep::Working) return result_; + if (get_ || overlay_ready_) + { + const auto status = overlay_ready_ ? IndexGetStep::Ready : get_->step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) + return finish(status == IndexGetStep::IoError ? IndexScanStep::IoError : status == IndexGetStep::VolumeChanged ? IndexScanStep::VolumeChanged + : status == IndexGetStep::WorkspaceTooSmall ? IndexScanStep::WorkspaceTooSmall + : IndexScanStep::Invalid); + const auto value = overlay_ready_ ? overlay_value_ : get_->value(); + overlay_ready_ = false; + if (table_ == 13) + { + OutgoingView outgoing; + if (!decodeOutgoing({attempt_key_.data(), attempt_key_.size()}, value, outgoing)) + return finish(IndexScanStep::Invalid); + get_.reset(); + return result_; + } + if (!references_.accept(value)) return finish(IndexScanStep::Invalid); + get_.reset(); + return startReference(); + } + + MutationView row; + if (scanned_) + { + while (mutation_position_ < mutation_count_) + { + row = mutations_[mutation_position_++]; + if (!row.erase && (row.table == 5 || row.table == 10 || row.table == 13)) + { + table_ = row.table; + return checkRow(row); + } + } + return finish(IndexScanStep::End); + } + const auto status = scan_->step(); + if (status == IndexScanStep::Working) return result_; + if (status == IndexScanStep::End) + { + if (table_ == 13) + { + scanned_ = true; + scan_.reset(); + return result_; + } + table_ = table_ == 5 ? 10 : 13; + if (!scan_.emplace(volume_).begin(root_, table_, frame_, capacity_)) return finish(IndexScanStep::Invalid); + return result_; + } + if (status != IndexScanStep::Item) return finish(status); + if (!scan_->item(row)) return finish(IndexScanStep::Invalid); + const bool replaced = replacement(row.table, row.key) != nullptr; + if (!replaced && checkRow(row) != IndexScanStep::Working) return result_; + if (!scan_->advance()) return finish(IndexScanStep::Invalid); + return result_; + } + + private: + const ::geocaching::storage::MutationView* replacement(uint8_t table, ::geocaching::ByteView key) const + { + for (size_t i = 0; i < mutation_count_; ++i) + if (mutations_[i].table == table && mutations_[i].key.size == key.size && + !std::memcmp(mutations_[i].key.data, key.data, key.size)) return &mutations_[i]; + return nullptr; + } + IndexScanStep loadReference(uint8_t table, ::geocaching::ByteView key) + { + if (const auto* row = replacement(table, key)) + { + if (row->erase) return finish(IndexScanStep::Invalid); + overlay_value_ = row->value; + overlay_ready_ = true; + } + else if (!get_.emplace(volume_).begin(root_, table, key, frame_, capacity_)) return finish(IndexScanStep::Invalid); + return result_; + } + IndexScanStep checkRow(const ::geocaching::storage::MutationView& row) + { + using namespace ::geocaching::storage; + if (table_ == 13) + { + TxAttemptView attempt; + if (!decodeTxAttempt(row.key, row.value, attempt)) return finish(IndexScanStep::Invalid); + std::memcpy(attempt_key_.data(), attempt.request_key.data, attempt_key_.size()); + return loadReference(5, {attempt_key_.data(), attempt_key_.size()}); + } + if (!references_.begin(row)) return finish(IndexScanStep::Invalid); + return startReference(); + } + + IndexScanStep startReference() + { + uint8_t table = 0; + ::geocaching::ByteView key; + if (!references_.next(table, key)) + return references_.complete() ? result_ : finish(IndexScanStep::Invalid); + return loadReference(table, key); + } + IndexScanStep finish(IndexScanStep status) + { + get_.reset(); + scan_.reset(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + std::optional scan_; + std::optional get_; + ::geocaching::storage::TaskReferenceCheck references_; + std::array attempt_key_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + const ::geocaching::storage::MutationView* mutations_ = nullptr; + size_t mutation_count_ = 0, mutation_position_ = 0; + ::geocaching::ByteView overlay_value_; + bool overlay_ready_ = false, scanned_ = false; + uint8_t table_ = 5; + IndexScanStep result_ = IndexScanStep::Idle; +}; +static_assert(sizeof(SdIndexReferences) <= 1792, "Reference recovery must use bounded metadata and an external frame"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_replay.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_replay.h new file mode 100644 index 00000000..74e9b354 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_replay.h @@ -0,0 +1,149 @@ +#pragma once +#include "geocaching/storage/record_shape.h" +#include "platform/esp/arduino_common/geocaching/sd_index_transaction.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_replay.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexReplayStep : uint8_t +{ + Working, + NeedsValidation, + Complete, + RetryLater, + Invalid, + VolumeChanged, + RecoveryRequired +}; + +// The root pair has been selected/validated and range comes from a completed +// journal inventory. All buffers are caller leases; no logical table is loaded. +// Business validation must inspect the resulting state before accept(true). +class SdIndexReplay +{ + public: + SdIndexReplay(const ::geocaching::storage::VolumeInstance& volume, + const ::geocaching::storage::IndexRootView& root, unsigned copy, + ::geocaching::storage::IndexRootBytes& first, ::geocaching::storage::IndexRootBytes& second, + JournalSegmentRange range, uint8_t* frame, size_t capacity, + ::geocaching::storage::MutationView* mutations, size_t mutation_capacity) + : volume_(volume), root_(root), copy_(copy), roots_{&first, &second}, + replay_(volume, root.sequence, range, frame, capacity, mutations, mutation_capacity) + { + if (copy > 1 || &first == &second || !::geocaching::storage::validIndexRoot(root) || + root.shards.data != roots_[copy]->data() + 48) result_ = IndexReplayStep::Invalid; + const auto overlaps = [](const void* a, size_t as, const void* b, size_t bs) + { + const auto x = reinterpret_cast(a), y = reinterpret_cast(b); + return as && bs && (x <= y ? y - x < as : x - y < bs); + }; + if (!frame || !mutations || mutation_capacity > 64 || + overlaps(frame, capacity, first.data(), first.size()) || overlaps(frame, capacity, second.data(), second.size()) || + overlaps(mutations, mutation_capacity * sizeof(*mutations), frame, capacity) || + overlaps(mutations, mutation_capacity * sizeof(*mutations), first.data(), first.size()) || + overlaps(mutations, mutation_capacity * sizeof(*mutations), second.data(), second.size())) result_ = IndexReplayStep::Invalid; + } + bool pending(::geocaching::storage::TransactionView& transaction) const + { + transaction = {}; + if (result_ != IndexReplayStep::NeedsValidation) return false; + transaction = pending_; + return true; + } + ::geocaching::storage::IndexRootView base() const { return root_; } + bool selected(::geocaching::storage::IndexRootView& root, unsigned& copy) const + { + root = {}; + if (result_ != IndexReplayStep::Complete) return false; + root = root_; + copy = copy_; + return true; + } + // The caller supplies a separate temporary read lease because replay's + // transaction bytes remain pinned while references are checked on disk. + bool accept(bool valid, uint8_t* validation_frame = nullptr, size_t validation_capacity = 0) + { + if (result_ != IndexReplayStep::NeedsValidation) return false; + if (!valid) + { + result_ = IndexReplayStep::Invalid; + return true; + } + const auto pending_frame = replay_.pendingFrame(); + const auto a = reinterpret_cast(pending_frame.data), b = reinterpret_cast(validation_frame); + if (!validation_frame || !validation_capacity || + (a <= b ? b - a < pending_frame.size : a - b < validation_capacity)) return false; + validation_.reset(new (std::nothrow) SdIndexedCommit(volume_)); + if (!validation_ || !validation_->begin(root_, copy_, pending_.mutations, pending_.count, + validation_frame, validation_capacity, *roots_[1 - copy_], true)) + { + validation_.reset(); + result_ = IndexReplayStep::Invalid; + } + else result_ = IndexReplayStep::Working; + return true; + } + IndexReplayStep step() + { + if (result_ != IndexReplayStep::Working && result_ != IndexReplayStep::RetryLater) return result_; + result_ = IndexReplayStep::Working; + if (validation_) + { + const auto status = validation_->step(); + if (status == IndexedCommitStep::Working) return result_; + validation_.reset(); + if (status != IndexedCommitStep::Validated) + return result_ = status == IndexedCommitStep::VolumeChanged ? IndexReplayStep::VolumeChanged : IndexReplayStep::Invalid; + const auto frame = replay_.pendingFrame(); + ::geocaching::storage::TransactionIndexCursor cursor; + ::geocaching::storage::IndexedMutation first; + if (frame.size < 24 || std::memcmp(frame.data + 20, pending_crc_.data(), pending_crc_.size()) || + !replay_.pendingIndex(cursor) || !cursor.next(first) || + !index_.emplace(volume_).begin(root_, copy_, frame, first.location.segment_first_sequence, + first.location.frame_offset, *roots_[1 - copy_])) return result_ = IndexReplayStep::Invalid; + return result_; + } + if (index_) + { + const auto status = index_->step(); + if (status == IndexTransactionStep::Working) return result_; + if (status == IndexTransactionStep::VolumeChanged) return result_ = IndexReplayStep::VolumeChanged; + ::geocaching::storage::IndexRootView committed; + if (status != IndexTransactionStep::Verified || !index_->committed(committed) || + !replay_.acknowledgeApplied(committed.sequence)) return result_ = IndexReplayStep::RecoveryRequired; + root_ = committed; + copy_ = 1 - copy_; + index_.reset(); + pending_ = {}; + return result_; + } + const auto status = replay_.next(pending_); + if (status == ReplayStep::Advancing) return result_; + if (status == ReplayStep::RetryLater) return result_ = IndexReplayStep::RetryLater; + if (status == ReplayStep::VolumeChanged) return result_ = IndexReplayStep::VolumeChanged; + if (status == ReplayStep::JournalComplete) return result_ = IndexReplayStep::Complete; + if (status != ReplayStep::Transaction) return result_ = IndexReplayStep::RecoveryRequired; + for (size_t i = 0; i < pending_.count; ++i) + if (!::geocaching::storage::validStoredRowShape(pending_.mutations[i])) return result_ = IndexReplayStep::Invalid; + const auto frame = replay_.pendingFrame(); + if (frame.size < 24) return result_ = IndexReplayStep::Invalid; + std::memcpy(pending_crc_.data(), frame.data + 20, pending_crc_.size()); + return result_ = IndexReplayStep::NeedsValidation; + } + + private: + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + unsigned copy_; + ::geocaching::storage::IndexRootBytes* roots_[2]; + SdJournalReplay replay_; + std::optional index_; + std::unique_ptr validation_; + std::array pending_crc_{}; + ::geocaching::storage::TransactionView pending_; + IndexReplayStep result_ = IndexReplayStep::Working; +}; +static_assert(sizeof(SdIndexReplay) <= 1280, "Indexed replay must not retain a logical ledger"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_reader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_reader.h new file mode 100644 index 00000000..a06b869d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_reader.h @@ -0,0 +1,80 @@ +#pragma once +#include "geocaching/storage/index_root.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexRootReadStep : uint8_t +{ + Idle, + Working, + Ready, + Invalid, + IoError, + VolumeChanged +}; + +// Caller leases two disjoint metadata buffers until the selected borrowed view +// is consumed. No full-index or value allocation is performed by this reader. +class SdIndexRootReader +{ + public: + explicit SdIndexRootReader(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(::geocaching::storage::IndexRootBytes& first, ::geocaching::storage::IndexRootBytes& second) + { + if (result_ != IndexRootReadStep::Idle || &first == &second) return false; + buffers_[0] = &first; + buffers_[1] = &second; + result_ = IndexRootReadStep::Working; + return true; + } + bool selected(::geocaching::storage::IndexRootView& out) const + { + out = {}; + if (result_ != IndexRootReadStep::Ready) return false; + out = selected_; + return true; + } + int selectedCopy() const { return result_ == IndexRootReadStep::Ready ? selected_copy_ : -1; } + IndexRootReadStep step() + { + if (result_ != IndexRootReadStep::Working) return result_; + if (phase_ == 0 || phase_ == 3) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return result_ = IndexRootReadStep::IoError; + if (current != volume_) return result_ = IndexRootReadStep::VolumeChanged; + if (phase_ == 3) return result_ = IndexRootReadStep::Ready; + phase_ = 1; + return result_; + } + const unsigned copy = phase_ - 1; + auto& buffer = *buffers_[copy]; + char path[64]; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index/root.h%u", copy); + const auto read = storage::sd_read_file(path, buffer.data(), buffer.size()); + if (read.status == storage::SdFileReadStatus::Missing || read.status == storage::SdFileReadStatus::Invalid) + return result_ = IndexRootReadStep::Invalid; + if (read.status != storage::SdFileReadStatus::Ready) return result_ = IndexRootReadStep::IoError; + if (read.file_size != buffer.size() || read.bytes_read != buffer.size() || + !::geocaching::storage::decodeIndexRoot({buffer.data(), buffer.size()}, volume_, roots_[copy])) + return result_ = IndexRootReadStep::Invalid; + if (copy == 1) + { + if (!::geocaching::storage::selectIndexRoot(roots_[0], roots_[1], selected_)) return result_ = IndexRootReadStep::Invalid; + selected_copy_ = selected_.revision == roots_[0].revision ? 0 : 1; + } + ++phase_; + return result_; + } + + private: + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootBytes* buffers_[2]{}; + ::geocaching::storage::IndexRootView roots_[2], selected_; + uint8_t phase_ = 0, selected_copy_ = 0; + IndexRootReadStep result_ = IndexRootReadStep::Idle; +}; +static_assert(sizeof(SdIndexRootReader) <= 192, "Root readers borrow their metadata buffers"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_writer.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_writer.h new file mode 100644 index 00000000..a185e06a --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_root_writer.h @@ -0,0 +1,125 @@ +#pragma once +#include "geocaching/storage/index_root.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexRootWriteStep : uint8_t +{ + Idle, + Working, + Verified, + Invalid, + IoError, + VolumeChanged +}; + +// Final publication primitive. Owner validates the root transition, completes +// all shard writes, and chooses the non-current copy before calling begin(). +// Encoded metadata is borrowed and immutable through the terminal result. +class SdIndexRootWriter +{ + public: + explicit SdIndexRootWriter(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(unsigned copy, const ::geocaching::storage::IndexRootBytes& bytes) + { + ::geocaching::storage::IndexRootView checked; + if (result_ != IndexRootWriteStep::Idle || copy > 1 || + !::geocaching::storage::decodeIndexRoot({bytes.data(), bytes.size()}, volume_, checked)) return false; + copy_ = static_cast(copy); + bytes_ = &bytes; + result_ = IndexRootWriteStep::Working; + return true; + } + IndexRootWriteStep step() + { + if (result_ != IndexRootWriteStep::Working) return result_; + if (phase_ == Phase::Volume || phase_ == Phase::VerifyVolume) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexRootWriteStep::IoError); + if (current != volume_) return fail(IndexRootWriteStep::VolumeChanged); + if (phase_ == Phase::VerifyVolume) return result_ = IndexRootWriteStep::Verified; + phase_ = Phase::OpenWrite; + return result_; + } + if (phase_ == Phase::OpenWrite || phase_ == Phase::OpenRead) + { + char path[64]; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index/root.h%u", static_cast(copy_)); + if (!file_.open(path, phase_ == Phase::OpenWrite ? "w" : "r")) return fail(IndexRootWriteStep::IoError); + phase_ = phase_ == Phase::OpenWrite ? Phase::Write : Phase::Size; + return result_; + } + if (phase_ == Phase::Write) + { + if (file_.write(bytes_->data(), bytes_->size()) != bytes_->size()) return fail(IndexRootWriteStep::IoError); + phase_ = Phase::Flush; + return result_; + } + if (phase_ == Phase::Flush) + { + if (!file_.flush()) return fail(IndexRootWriteStep::IoError); + phase_ = Phase::CloseWrite; + return result_; + } + if (phase_ == Phase::CloseWrite) + { + file_.close(); + phase_ = Phase::OpenRead; + return result_; + } + if (phase_ == Phase::Size || phase_ == Phase::FinalSize) + { + if (file_.size() != bytes_->size()) return fail(IndexRootWriteStep::Invalid); + phase_ = phase_ == Phase::Size ? Phase::Readback : Phase::CloseRead; + return result_; + } + if (phase_ == Phase::Readback) + { + const auto count = std::min(verify_.size(), bytes_->size() - verified_); + const int read = file_.read(verify_.data(), count); + if (read < 0) return fail(IndexRootWriteStep::IoError); + if (!read || static_cast(read) > count || std::memcmp(verify_.data(), bytes_->data() + verified_, read)) return fail(IndexRootWriteStep::Invalid); + verified_ += static_cast(read); + if (verified_ == bytes_->size()) phase_ = Phase::FinalSize; + return result_; + } + file_.close(); + phase_ = Phase::VerifyVolume; + return result_; + } + + private: + enum class Phase : uint8_t + { + Volume, + OpenWrite, + Write, + Flush, + CloseWrite, + OpenRead, + Size, + Readback, + FinalSize, + CloseRead, + VerifyVolume + }; + IndexRootWriteStep fail(IndexRootWriteStep result) + { + file_.close(); + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + const ::geocaching::storage::IndexRootBytes* bytes_ = nullptr; + std::array verify_{}; + storage::SdRuntimeFile file_; + size_t verified_ = 0; + uint8_t copy_ = 0; + Phase phase_ = Phase::Volume; + IndexRootWriteStep result_ = IndexRootWriteStep::Idle; +}; +static_assert(sizeof(SdIndexRootWriter) <= 128, "Root publishing borrows metadata and verifies small slices"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_scan.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_scan.h new file mode 100644 index 00000000..3efbf034 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_scan.h @@ -0,0 +1,211 @@ +#pragma once +#include "geocaching/storage/index_root.h" +#include "geocaching/storage/record_shape.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_lookup.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexScanStep : uint8_t +{ + Idle, + Working, + Item, + End, + Invalid, + IoError, + VolumeChanged, + WorkspaceTooSmall +}; + +// Snapshot and frame leases are pinned through the scan. An Item borrows both +// the key and value until advance(). Owner storage only, never an ESP stack local. +class SdIndexScan +{ + public: + explicit SdIndexScan(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, uint8_t table, uint8_t* frame, size_t capacity) + { + if (result_ != IndexScanStep::Idle || !::geocaching::storage::validIndexRoot(root) || table < 1 || table > 13 || !frame || capacity < 24) return false; + const auto a = reinterpret_cast(root.shards.data), b = reinterpret_cast(frame); + if (a <= b ? b - a < root.shards.size : a - b < capacity) return false; + root_ = root; + table_ = table; + frame_ = frame; + capacity_ = capacity; + result_ = IndexScanStep::Working; + return true; + } + bool item(::geocaching::storage::MutationView& out) const + { + out = {}; + if (result_ != IndexScanStep::Item) return false; + out = {table_, entry_.key, value_, false}; + return true; + } + bool advance() + { + if (result_ != IndexScanStep::Item) return false; + value_ = {}; + result_ = IndexScanStep::Working; + return true; + } + IndexScanStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexScanStep::Working) return result_; + if (phase_ == Phase::Bucket) + { + while (bucket_ < 256 && !indexHasShard(root_, table_, static_cast(bucket_))) ++bucket_; + if (bucket_ == 256) + { + phase_ = Phase::Finish; + return result_; + } + if (!operation_.emplace(volume_, root_.slot, root_.epoch, root_.sequence).beginBucket(table_, static_cast(bucket_))) return fail(IndexScanStep::Invalid); + phase_ = Phase::Head; + return result_; + } + if (phase_ == Phase::Head) + { + auto& reader = std::get(operation_); + const auto status = reader.step(); + if (status == IndexHeadReadStep::Working) return result_; + if (status != IndexHeadReadStep::Ready || !reader.selected(head_) || !head_.length) return error(status); + operation_.emplace(); + phase_ = Phase::Open; + return result_; + } + if (phase_ == Phase::Open) + { + char path[80]; + if (!indexShardPathForBucket(root_.slot, table_, static_cast(bucket_), path, sizeof(path)) || !file_.open(path, "r")) return fail(IndexScanStep::IoError); + phase_ = Phase::Size; + return result_; + } + if (phase_ == Phase::Size) + { + if (file_.size() < head_.length) return fail(IndexScanStep::Invalid); + position_ = head_.length; + newer_ = UINT64_MAX; + phase_ = Phase::Seek; + return result_; + } + if (phase_ == Phase::Seek) + { + if (!position_) + { + phase_ = Phase::EndShard; + return result_; + } + position_ -= kIndexEntrySize; + if (!file_.seek(position_)) return fail(IndexScanStep::IoError); + read_ = 0; + phase_ = Phase::Read; + return result_; + } + if (phase_ == Phase::Read) + { + const int count = file_.read(bytes_.data() + read_, bytes_.size() - read_); + if (count < 0) return fail(IndexScanStep::IoError); + if (!count || static_cast(count) > bytes_.size() - read_) return fail(IndexScanStep::Invalid); + read_ += static_cast(count); + if (read_ != bytes_.size()) return result_; + if (!decodeIndexEntry({bytes_.data(), bytes_.size()}, volume_, entry_) || entry_.table != table_ || + static_cast(::sys::crc32(entry_.key.data, entry_.key.size)) != bucket_ || entry_.location.record_sequence > newer_ || + (position_ == head_.length - kIndexEntrySize && entry_.location.record_sequence != head_.sequence)) return fail(IndexScanStep::Invalid); + newer_ = entry_.location.record_sequence; + if (!operation_.emplace(volume_, root_.slot, head_.sequence, head_.length).begin(table_, entry_.key)) return fail(IndexScanStep::Invalid); + phase_ = Phase::Latest; + return result_; + } + if (phase_ == Phase::Latest) + { + auto& lookup = std::get(operation_); + const auto status = lookup.step(); + if (status == IndexLookupStep::Working) return result_; + if (status != IndexLookupStep::Found) return error(status); + if (lookup.entryOffset() != position_) + { + operation_.emplace(); + phase_ = Phase::Seek; + return result_; + } + if (!operation_.emplace(volume_).begin(entry_, frame_, capacity_)) return fail(IndexScanStep::Invalid); + phase_ = Phase::Value; + return result_; + } + if (phase_ == Phase::Value) + { + auto& reader = std::get(operation_); + const auto status = reader.step(); + if (status == IndexedReadStep::Working) return result_; + if (status == IndexedReadStep::WorkspaceTooSmall) return fail(IndexScanStep::WorkspaceTooSmall); + if (status != IndexedReadStep::Ready && status != IndexedReadStep::Erased) return error(status); + value_ = reader.value(); + operation_.emplace(); + phase_ = Phase::Seek; + if (status == IndexedReadStep::Erased) return result_; + if (!validStoredRowShape({table_, entry_.key, value_, false})) return fail(IndexScanStep::Invalid); + return result_ = IndexScanStep::Item; + } + if (phase_ == Phase::EndShard) + { + if (file_.size() < head_.length) return fail(IndexScanStep::Invalid); + file_.close(); + ++bucket_; + phase_ = Phase::Bucket; + return result_; + } + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexScanStep::IoError); + return fail(current == volume_ ? IndexScanStep::End : IndexScanStep::VolumeChanged); + } + + private: + enum class Phase : uint8_t + { + Bucket, + Head, + Open, + Size, + Seek, + Read, + Latest, + Value, + EndShard, + Finish + }; + template + IndexScanStep error(Status status) + { + return fail(status == Status::VolumeChanged ? IndexScanStep::VolumeChanged : status == Status::IoError ? IndexScanStep::IoError + : IndexScanStep::Invalid); + } + IndexScanStep fail(IndexScanStep status) + { + file_.close(); + operation_.emplace(); + value_ = {}; + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::storage::IndexShardHead head_; + ::geocaching::storage::IndexEntryBytes bytes_{}; + ::geocaching::storage::IndexedMutation entry_; + ::geocaching::ByteView value_; + storage::SdRuntimeFile file_; + std::variant operation_; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + uint64_t position_ = 0, newer_ = 0; + uint16_t bucket_ = 0, read_ = 0; + uint8_t table_ = 0; + Phase phase_ = Phase::Bucket; + IndexScanStep result_ = IndexScanStep::Idle; +}; +static_assert(sizeof(SdIndexScan) <= 832, "Table scans retain one current row, not a table"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_transaction.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_transaction.h new file mode 100644 index 00000000..c906d429 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_index_transaction.h @@ -0,0 +1,237 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_index_append.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_index_head_writer.h" +#include "platform/esp/arduino_common/geocaching/sd_index_root_writer.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexTransactionStep : uint8_t +{ + Idle, + Working, + Verified, + Invalid, + IoError, + VolumeChanged +}; + +// Storage owner prevalidates table values/references and selects a valid +// generation. Parent bitmap and journal frame stay immutable; candidate root +// is an exclusive output lease. No application view changes until Verified. +// Keep this coordinator in owner storage, not on an ESP task stack. After an +// uncertain root write, reload the root pair before retrying with a parent view. +class SdIndexTransaction +{ + public: + explicit SdIndexTransaction(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + uint8_t completedEntries() const { return completed_entries_; } + bool begin(const ::geocaching::storage::IndexRootView& parent, unsigned parent_copy, + ::geocaching::ByteView frame, uint64_t segment_first, uint32_t frame_offset, + ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching::storage; + if (result_ != IndexTransactionStep::Idle || parent_copy > 1 || !validIndexRoot(parent) || + parent.sequence == UINT64_MAX || parent.revision == UINT64_MAX) return false; + const auto overlaps = [&](::geocaching::ByteView input) + { + const auto a = reinterpret_cast(input.data), b = reinterpret_cast(candidate.data()); + return a <= b ? b - a < input.size : a - b < candidate.size(); + }; + if (overlaps(frame) || overlaps(parent.shards) || !cursor_.open(frame, parent.sequence, segment_first, frame_offset)) return false; + parent_ = parent; + frame_ = frame; + candidate_ = &candidate; + target_copy_ = static_cast(1 - parent_copy); + segment_first_ = segment_first; + frame_offset_ = frame_offset; + std::memcpy(frame_crc_.data(), frame.data + 20, frame_crc_.size()); + parent_crc_ = ::sys::crc32(parent.shards.data, parent.shards.size); + result_ = IndexTransactionStep::Working; + return true; + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + return result_ == IndexTransactionStep::Verified && + ::geocaching::storage::decodeIndexRoot({candidate_->data(), candidate_->size()}, volume_, out); + } + IndexTransactionStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexTransactionStep::Working) return result_; + if (phase_ == Phase::DirectoryVolume) + { + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexTransactionStep::IoError); + if (!(current == volume_)) return fail(IndexTransactionStep::VolumeChanged); + directory_depth_ = 0; + phase_ = Phase::DirectoryCheck; + return result_; + } + if (phase_ == Phase::DirectoryCheck || phase_ == Phase::DirectoryCreate) + { + char path[64]; + if (directory_depth_ == 0) std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index"); + else if (directory_depth_ == 1) std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index/%c", parent_.slot); + else std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/index/%c/%02x", parent_.slot, static_cast(entry_.table)); + if (phase_ == Phase::DirectoryCheck && !storage::sd_is_directory(path)) + { + phase_ = Phase::DirectoryCreate; + return result_; + } + if (phase_ == Phase::DirectoryCreate && !storage::sd_mkdir(path)) return fail(IndexTransactionStep::IoError); + if (++directory_depth_ < 3) phase_ = Phase::DirectoryCheck; + else + { + if (!startHead(0, baseline_)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::InitializeFirst; + } + return result_; + } + if (phase_ == Phase::Next) + { + if (!cursor_.next(entry_)) + { + TransactionIndexCursor verify; + if (!cursor_.complete() || std::memcmp(frame_.data + 20, frame_crc_.data(), frame_crc_.size()) || + ::sys::crc32(parent_.shards.data, parent_.shards.size) != parent_crc_ || + !verify.open(frame_, parent_.sequence, segment_first_, frame_offset_)) return fail(IndexTransactionStep::Invalid); + std::memcpy(candidate_->data() + 48, parent_.shards.data, kIndexShardBitmapSize); + IndexedMutation changed; + while (verify.next(changed)) + { + const auto bucket = ::sys::crc32(changed.key.data, changed.key.size) & 0xff; + (*candidate_)[48 + (changed.table - 1) * 32 + bucket / 8] |= static_cast(1u << (bucket % 8)); + } + auto next = parent_; + ++next.sequence; + ++next.revision; + next.shards = {candidate_->data() + 48, kIndexShardBitmapSize}; + IndexRootView selected; + if (!verify.complete() || !selectIndexRoot(parent_, next, selected) || !encodeIndexRoot(volume_, next, *candidate_)) return fail(IndexTransactionStep::Invalid); + auto& writer = operation_.emplace(volume_); + if (!writer.begin(target_copy_, *candidate_)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::Root; + return result_; + } + const auto bucket = static_cast(::sys::crc32(entry_.key.data, entry_.key.size)); + baseline_ = {parent_.epoch, 0, 0, entry_.table, bucket}; + if (indexHasShard(parent_, entry_.table, bucket)) + { + auto& reader = operation_.emplace(volume_, parent_.slot, parent_.epoch, parent_.sequence); + if (!reader.begin(entry_.table, entry_.key)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::ReadHead; + } + else + { + phase_ = Phase::DirectoryVolume; + } + return result_; + } + if (phase_ == Phase::ReadHead) + { + auto& reader = std::get(operation_); + const auto status = reader.step(); + if (status == IndexHeadReadStep::Working) return result_; + if (status != IndexHeadReadStep::Ready || !reader.selected(baseline_)) return error(status); + head_copy_ = static_cast(1 - reader.selectedCopy()); + return startAppend(); + } + if (phase_ == Phase::InitializeFirst || phase_ == Phase::InitializeSecond || phase_ == Phase::Head) + { + const auto status = std::get(operation_).step(); + if (status == IndexHeadWriteStep::Working) return result_; + if (status != IndexHeadWriteStep::Verified) return error(status); + if (phase_ == Phase::InitializeFirst) + { + if (!startHead(1, baseline_)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::InitializeSecond; + return result_; + } + if (phase_ == Phase::InitializeSecond) + { + head_copy_ = 1; + return startAppend(); + } + ++completed_entries_; + operation_.emplace(); + phase_ = Phase::Next; + return result_; + } + if (phase_ == Phase::Append) + { + auto& writer = std::get(operation_); + const auto status = writer.step(); + if (status == IndexAppendStep::Working) return result_; + if (status != IndexAppendStep::Verified) return error(status); + const auto length = writer.writtenLength(); + if (length <= baseline_.length) return fail(IndexTransactionStep::Invalid); + auto next = baseline_; + next.sequence = parent_.sequence + 1; + next.length = length; + if (!startHead(head_copy_, next)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::Head; + return result_; + } + const auto status = std::get(operation_).step(); + if (status == IndexRootWriteStep::Working) return result_; + if (status != IndexRootWriteStep::Verified) return error(status); + return result_ = IndexTransactionStep::Verified; + } + + private: + enum class Phase : uint8_t + { + Next, + DirectoryVolume, + DirectoryCheck, + DirectoryCreate, + ReadHead, + InitializeFirst, + InitializeSecond, + Append, + Head, + Root + }; + bool startHead(unsigned copy, const ::geocaching::storage::IndexShardHead& head) + { + return operation_.emplace(volume_, parent_.slot).begin(entry_.key, copy, head); + } + IndexTransactionStep startAppend() + { + if (!operation_.emplace(volume_, parent_.slot).begin(entry_)) return fail(IndexTransactionStep::Invalid); + phase_ = Phase::Append; + return result_; + } + template + IndexTransactionStep error(Status status) + { + return fail(status == Status::VolumeChanged ? IndexTransactionStep::VolumeChanged : status == Status::IoError ? IndexTransactionStep::IoError + : IndexTransactionStep::Invalid); + } + IndexTransactionStep fail(IndexTransactionStep result) + { + operation_.emplace(); + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView parent_; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + ::geocaching::ByteView frame_; + ::geocaching::storage::TransactionIndexCursor cursor_; + ::geocaching::storage::IndexedMutation entry_; + ::geocaching::storage::IndexShardHead baseline_; + std::variant operation_; + std::array frame_crc_{}; + uint64_t segment_first_ = 0; + uint32_t frame_offset_ = 0, parent_crc_ = 0; + uint8_t target_copy_ = 0, head_copy_ = 0; + uint8_t completed_entries_ = 0; + uint8_t directory_depth_ = 0; + Phase phase_ = Phase::Next; + IndexTransactionStep result_ = IndexTransactionStep::Idle; +}; +static_assert(sizeof(SdIndexTransaction) <= 768, "Transaction coordination must overlay its operation workspaces"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_attempt_update.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_attempt_update.h new file mode 100644 index 00000000..3b9ed98d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_attempt_update.h @@ -0,0 +1,134 @@ +#pragma once +#include "geocaching/storage/tx_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +class SdIndexedAttemptUpdate +{ + public: + explicit SdIndexedAttemptUpdate(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + // Nonempty hash selects transport-hash recording; empty hash selects a + // terminal event. Only terminal events need caller-owned outgoing encoding. + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, ::geocaching::ByteView key, + ::geocaching::ByteView hash, ::geocaching::storage::TxAttemptState terminal, + const ::geocaching::storage::StoredTime& finished, uint8_t* outgoing, size_t outgoing_capacity, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Idle || !key.data || key.size != key_.size() || copy > 1 || + (hash.size && (!hash.data || hash.size != 32)) || + (!hash.size && (!outgoing || !outgoing_capacity || + (terminal != TxAttemptState::Delivered && terminal != TxAttemptState::Failed && terminal != TxAttemptState::CancelledBeforeSend)))) return false; + const auto overlaps = [](::geocaching::ByteView a, ::geocaching::ByteView b) + { + const auto x = reinterpret_cast(a.data), y = reinterpret_cast(b.data); + return a.size && b.size && (x <= y ? y - x < a.size : x - y < b.size); + }; + const ::geocaching::ByteView output{candidate.data(), candidate.size()}, scratch{frame, capacity}, encoding{outgoing, outgoing_capacity}; + if (overlaps(output, scratch) || overlaps(output, root.shards) || overlaps(encoding, output) || + overlaps(encoding, scratch) || overlaps(encoding, root.shards)) return false; + std::memcpy(key_.data(), key.data, key_.size()); + hash_update_ = hash.size != 0; + if (hash_update_) std::memcpy(hash_.data(), hash.data, hash_.size()); + terminal_ = terminal; + finished_ = finished; + root_ = root; + copy_ = copy; + outgoing_ = outgoing; + outgoing_capacity_ = outgoing_capacity; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + if (!io_.emplace(volume_).begin(root_, 13, {key_.data(), key_.size()}, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + if (result_ != IndexedCommitStep::Verified) return false; + if (duplicate_) + { + out = root_; + return true; + } + return std::get(io_).committed(out); + } + IndexedCommitStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::Commit) return result_ = std::get(io_).step(); + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + if (phase_ == Phase::Attempt) + { + TxAttemptView attempt; + const ::geocaching::ByteView key{key_.data(), key_.size()}; + if (!decodeTxAttempt(key, read.value(), attempt)) return fail(IndexedCommitStep::Invalid); + const auto transition = hash_update_ ? recordAttemptTransportHash(attempt, {hash_.data(), hash_.size()}) : finishAttemptTransport(attempt, terminal_, finished_); + if (transition == AttemptTransition::Rejected) return fail(IndexedCommitStep::Invalid); + duplicate_ = transition == AttemptTransition::Unchanged; + if (duplicate_ && hash_update_) return fail(IndexedCommitStep::Verified); + size_t size = 0; + if (!encodeTxAttempt(key, attempt, encoded_.data(), encoded_.size(), size)) return fail(IndexedCommitStep::Invalid); + mutations_[0] = {13, key, {encoded_.data(), size}, false}; + if (hash_update_) return commit(1); + if (!io_.emplace(volume_).begin(root_, 5, {key_.data(), 48}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Outgoing; + return result_; + } + OutgoingView outgoing; + const ::geocaching::ByteView key{key_.data(), 48}; + if (!decodeOutgoing(key, read.value(), outgoing)) return fail(IndexedCommitStep::Invalid); + if (duplicate_) return fail(IndexedCommitStep::Verified); + if (outgoing.state < 4) outgoing.state = terminal_ == TxAttemptState::Delivered ? 2 : 3; + size_t size = 0; + if (!encodeOutgoing(key, outgoing, outgoing_, outgoing_capacity_, size)) return fail(IndexedCommitStep::Invalid); + mutations_[1] = {5, key, {outgoing_, size}, false}; + return commit(2); + } + + private: + enum class Phase : uint8_t + { + Attempt, + Outgoing, + Commit + }; + IndexedCommitStep commit(size_t count) + { + if (!io_.emplace(volume_).begin(root_, copy_, mutations_.data(), count, frame_, capacity_, *candidate_)) + return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Commit; + return result_; + } + IndexedCommitStep fail(IndexedCommitStep status) + { + io_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::storage::StoredTime finished_; + ::geocaching::storage::TxAttemptState terminal_ = ::geocaching::storage::TxAttemptState::Failed; + std::array<::geocaching::storage::MutationView, 2> mutations_{}; + std::array key_{}; + std::array hash_{}; + std::array encoded_{}; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + uint8_t *frame_ = nullptr, *outgoing_ = nullptr; + size_t capacity_ = 0, outgoing_capacity_ = 0; + unsigned copy_ = 0; + bool hash_update_ = false, duplicate_ = false; + std::variant io_; + Phase phase_ = Phase::Attempt; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedAttemptUpdate) <= 1664, "Attempt updates own no request or response payload"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_begin_attempt.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_begin_attempt.h new file mode 100644 index 00000000..2816a1ed --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_begin_attempt.h @@ -0,0 +1,148 @@ +#pragma once +#include "geocaching/storage/cache_head.h" +#include "geocaching/storage/queued_request.h" +#include "geocaching/storage/tx_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Transport may run only after Verified and after reloading the committed +// request. Caller owns the encoding/read leases; no payload array lives here. +class SdIndexedBeginAttempt +{ + public: + explicit SdIndexedBeginAttempt(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, + const ::geocaching::Destination& local, ::geocaching::ByteView request_key, + const std::array& attempt_id, const ::geocaching::storage::StoredTime& submitted, + ::geocaching::storage::QueuedRequestWorkspace& workspace, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Idle || copy > 1 || !request_key.data || request_key.size != 48 || + std::memcmp(request_key.data, local.bytes.data(), 16) || !workspace.outgoing) return false; + const ByteView leases[] = {{workspace.outgoing, workspace.outgoing_capacity}, {workspace.task.data(), workspace.task.size()}, {frame, capacity}, {candidate.data(), candidate.size()}, root.shards}; + for (size_t i = 0; i < 5; ++i) + for (size_t j = 0; j < i; ++j) + { + const auto a = reinterpret_cast(leases[i].data), b = reinterpret_cast(leases[j].data); + if (leases[i].size && leases[j].size && (a <= b ? b - a < leases[i].size : a - b < leases[j].size)) return false; + } + std::memcpy(key_.data(), request_key.data, 48); + std::memcpy(key_.data() + 48, attempt_id.data(), 16); + TxAttemptView attempt; + attempt.submitted = submitted; + size_t size = 0; + if (!encodeTxAttempt({key_.data(), key_.size()}, attempt, attempt_.data(), attempt_.size(), size)) return false; + mutations_[0] = {13, {key_.data(), key_.size()}, {attempt_.data(), size}, false}; + root_ = root; + copy_ = copy; + workspace_ = &workspace; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + if (!io_.emplace(volume_).begin(root_, 5, {key_.data(), 48}, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool committed(::geocaching::storage::IndexRootView& root) const + { + root = {}; + return result_ == IndexedCommitStep::Verified && std::get(io_).committed(root); + } + IndexedCommitStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::Commit) return result_ = std::get(io_).step(); + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + const ::geocaching::ByteView request_key{key_.data(), 48}; + if (phase_ == Phase::Outgoing) + { + OutgoingView outgoing; + if (!decodeOutgoing(request_key, read.value(), outgoing) || !outgoing.continue_intent || + (outgoing.state != 0 && outgoing.state != 3)) return fail(IndexedCommitStep::Invalid); + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + generation_ = outgoing.install_generation; + outgoing.state = 1; + size_t size = 0; + if (!encodeOutgoing(request_key, outgoing, workspace_->outgoing, workspace_->outgoing_capacity, size)) return fail(IndexedCommitStep::Invalid); + mutations_[1] = {5, request_key, {workspace_->outgoing, size}, false}; + if (!io_.emplace(volume_).begin(root_, 10, {task_id_.data(), task_id_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Task; + return result_; + } + if (phase_ == Phase::Task) + { + TaskView task; + OutgoingView outgoing; + const ::geocaching::ByteView task_key{task_id_.data(), task_id_.size()}; + if (!decodeTask(task_key, read.value(), task) || !task.continue_intent || task.state > 2 || + !decodeOutgoing(request_key, mutations_[1].value, outgoing) || + !requestBelongsToTask(task_key, task, request_key, outgoing)) return fail(IndexedCommitStep::Invalid); + const bool check_generation = (task.kind == 2 || task.kind == 4) && generation_; + if (check_generation) + { + if (task.cache_id.size != cache_id_.size()) return fail(IndexedCommitStep::Invalid); + std::memcpy(cache_id_.data(), task.cache_id.data, cache_id_.size()); + } + if (task.state == 0) task.state = 1; + size_t size = 0; + if (!encodeTask(task_key, task, workspace_->task.data(), workspace_->task.size(), size)) return fail(IndexedCommitStep::Invalid); + mutations_[2] = {10, task_key, {workspace_->task.data(), size}, false}; + if (!check_generation) return commit(); + if (!io_.emplace(volume_).begin(root_, 2, {cache_id_.data(), cache_id_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Head; + return result_; + } + CacheHeadView head; + if (!decodeCacheHead({cache_id_.data(), cache_id_.size()}, read.value(), head) || head.install_generation != generation_) + return fail(IndexedCommitStep::Invalid); + return commit(); + } + + private: + enum class Phase : uint8_t + { + Outgoing, + Task, + Head, + Commit + }; + IndexedCommitStep commit() + { + if (!io_.emplace(volume_).begin(root_, copy_, mutations_.data(), mutations_.size(), frame_, capacity_, *candidate_, false, 1)) + return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Commit; + return result_; + } + IndexedCommitStep fail(IndexedCommitStep status) + { + io_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::storage::QueuedRequestWorkspace* workspace_ = nullptr; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + std::array<::geocaching::storage::MutationView, 3> mutations_{}; + std::array key_{}; + std::array task_id_{}; + std::array cache_id_{}; + std::array attempt_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + uint64_t generation_ = 0; + unsigned copy_ = 0; + std::variant io_; + Phase phase_ = Phase::Outgoing; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedBeginAttempt) <= 1664, "Send intent must not own request payloads"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_commit.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_commit.h new file mode 100644 index 00000000..9919d0ac --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_commit.h @@ -0,0 +1,357 @@ +#pragma once +#include "geocaching/storage/record_shape.h" +#include "platform/esp/arduino_common/geocaching/sd_index_references.h" +#include "platform/esp/arduino_common/geocaching/sd_index_transaction.h" +#include "platform/esp/arduino_common/geocaching/sd_journal.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_segment.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexedCommitStep : uint8_t +{ + Idle, + Working, + Verified, + Invalid, + IoError, + VolumeChanged, + RecoveryRequired, + NeedsValidation, + Validated +}; + +// Caller prevalidates operation-specific business rules. +// Existing author issuance records cannot be changed or removed. +// Task/request/attempt references are checked here before journal creation. +// Mutation bytes stay immutable until inputConsumed(). The frame/output-root +// leases stay alive through completion. Store this coordinator off task stacks. +class SdIndexedCommit +{ + public: + explicit SdIndexedCommit(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& parent, unsigned parent_copy, + const ::geocaching::storage::MutationView* mutations, size_t count, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& candidate, + bool validation_only = false, size_t required_absent = 0) + { + using namespace ::geocaching::storage; + TransactionEncoding encoding; + if (result_ != IndexedCommitStep::Idle || !validIndexRoot(parent) || parent_copy > 1 || + parent.sequence == UINT64_MAX || parent.revision == UINT64_MAX || !frame || capacity < 24 || + required_absent > count || !encoding.open(parent.sequence, mutations, count) || encoding.size() > capacity - 24) return false; + for (size_t i = 0; i < count; ++i) + if (!validStoredRowShape(mutations[i])) return false; + if (!prepare(parent, parent_copy, frame, capacity, candidate)) return false; + validation_only_ = validation_only; + absent_count_ = required_absent; + expected_size_ = encoding.size() + 24; + bool references = false, authors = false; + for (size_t i = 0; i < count; ++i) + { + references |= mutations[i].table == 5 || mutations[i].table == 10 || mutations[i].table == 13; + authors |= mutations[i].table == 3; + } + if ((references || authors || required_absent) && encoding.inputOverlaps({frame, capacity})) return false; + mutations_ = mutations; + mutation_count_ = count; + if (references) + { + references_.reset(new (std::nothrow) SdIndexReferences(volume_)); + if (!references_ || !references_->begin(parent, frame, capacity, mutations, count)) + { + fail(IndexedCommitStep::Invalid); + return false; + } + mutations_ = mutations; + mutation_count_ = count; + phase_ = Phase::References; + result_ = IndexedCommitStep::Working; + return true; + } + if (authors || required_absent) + { + phase_ = Phase::Absent; + result_ = IndexedCommitStep::Working; + return true; + } + if (validation_only_) + { + phase_ = Phase::ValidationVolume; + result_ = IndexedCommitStep::Working; + return true; + } + auto& journal = operation_.emplace(volume_); + const auto started = journal.begin(parent.sequence, mutations, count); + if (started != JournalWriteResult::InProgress) + { + fail(IndexedCommitStep::Invalid); + return false; + } + captured_ = journal.captureReadback(frame, capacity); + result_ = IndexedCommitStep::Working; + return true; + } + bool resume(const ::geocaching::storage::IndexRootView& parent, unsigned parent_copy, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& candidate) + { + if (!prepare(parent, parent_copy, frame, capacity, candidate)) return false; + recovering_ = consumed_ = true; + operation_.emplace(); + phase_ = Phase::CheckVolume; + result_ = IndexedCommitStep::Working; + return true; + } + ::geocaching::ByteView recoveredFrame() const + { + return result_ == IndexedCommitStep::NeedsValidation ? ::geocaching::ByteView{frame_, expected_size_} : ::geocaching::ByteView{}; + } + bool validateRecovered(bool valid) + { + if (result_ != IndexedCommitStep::NeedsValidation) return false; + if (!valid || !recoveredUnchanged()) + { + fail(IndexedCommitStep::Invalid); + return true; + } + phase_ = Phase::StartIndex; + result_ = IndexedCommitStep::Working; + return true; + } + bool inputConsumed() const { return consumed_; } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + if (result_ != IndexedCommitStep::Verified) return false; + out = committed_; + return true; + } + IndexedCommitStep step() + { + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::ValidationVolume) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexedCommitStep::IoError); + if (current != volume_) return fail(IndexedCommitStep::VolumeChanged); + return fail(IndexedCommitStep::Validated); + } + if (phase_ == Phase::References) + { + const auto status = references_->step(); + if (status == IndexScanStep::Working) return result_; + if (status != IndexScanStep::End) + return fail(status == IndexScanStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexScanStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + references_.reset(); + phase_ = Phase::Absent; + return result_; + } + if (phase_ == Phase::Absent) + { + if (absent_position_ == absent_count_) + { + phase_ = Phase::Authors; + return result_; + } + if (!absent_pending_) + { + const auto& row = mutations_[absent_position_]; + if (!operation_.emplace(volume_).begin(parent_, row.table, row.key, frame_, capacity_)) + return fail(IndexedCommitStep::Invalid); + absent_pending_ = true; + return result_; + } + const auto status = std::get(operation_).step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::NotFound) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + operation_.emplace(); + absent_pending_ = false; + ++absent_position_; + return result_; + } + if (phase_ == Phase::Authors) + { + if (author_pending_) + { + auto& read = std::get(operation_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status == IndexGetStep::Ready) + { + const auto prior = read.value(); + const auto& row = mutations_[author_position_]; + if (row.erase || prior.size != row.value.size || std::memcmp(prior.data, row.value.data, prior.size)) + return fail(IndexedCommitStep::Invalid); + } + else if (status != IndexGetStep::NotFound) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + operation_.emplace(); + author_pending_ = false; + ++author_position_; + return result_; + } + while (author_position_ < mutation_count_ && mutations_[author_position_].table != 3) ++author_position_; + if (author_position_ < mutation_count_) + { + if (!operation_.emplace(volume_).begin(parent_, 3, mutations_[author_position_].key, frame_, capacity_)) + return fail(IndexedCommitStep::Invalid); + author_pending_ = true; + return result_; + } + if (validation_only_) + { + phase_ = Phase::ValidationVolume; + return result_; + } + auto& journal = operation_.emplace(volume_); + if (journal.begin(parent_.sequence, mutations_, mutation_count_) != JournalWriteResult::InProgress) + return fail(IndexedCommitStep::Invalid); + captured_ = journal.captureReadback(frame_, capacity_); + phase_ = Phase::Journal; + return result_; + } + if (phase_ == Phase::CheckVolume || phase_ == Phase::CheckRecoveredVolume) + { + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return fail(IndexedCommitStep::RecoveryRequired); + if (current != volume_) return fail(IndexedCommitStep::VolumeChanged); + if (phase_ == Phase::CheckRecoveredVolume) return result_ = IndexedCommitStep::NeedsValidation; + phase_ = Phase::Open; + return result_; + } + if (phase_ == Phase::Journal) + { + auto& journal = std::get(operation_); + const auto status = journal.step(); + if (status == JournalWriteResult::InProgress) return result_; + if (status != JournalWriteResult::Verified) + { + if (status == JournalWriteResult::VolumeChanged) return fail(IndexedCommitStep::VolumeChanged); + return fail(journal.mayHaveWritten() || status == JournalWriteResult::Exists ? IndexedCommitStep::RecoveryRequired : IndexedCommitStep::IoError); + } + consumed_ = true; + if (captured_) + { + if (!operation_.emplace(volume_).begin(parent_, parent_copy_, {frame_, expected_size_}, parent_.sequence + 1, 0, *candidate_)) + return fail(IndexedCommitStep::RecoveryRequired); + phase_ = Phase::Index; + return result_; + } + operation_.emplace(); + phase_ = Phase::Open; + return result_; + } + if (phase_ == Phase::Open) + { + if (!std::get(operation_).open(parent_.sequence + 1)) return fail(IndexedCommitStep::RecoveryRequired); + phase_ = Phase::Read; + return result_; + } + if (phase_ == Phase::Read) + { + ::geocaching::storage::RecordFrameView frame; + const auto status = std::get(operation_).next(frame_, capacity_, frame); + if (status == SegmentReadResult::InProgress) return result_; + if (status != SegmentReadResult::Record || frame.sequence != parent_.sequence + 1 || + (expected_size_ && frame.payload.size + 24 != expected_size_)) + return fail(IndexedCommitStep::RecoveryRequired); + expected_size_ = frame.payload.size + 24; + if (!::geocaching::storage::validTransactionRowShapes(frame.payload, parent_.sequence)) return fail(IndexedCommitStep::RecoveryRequired); + if (recovering_) std::memcpy(recovery_crc_.data(), frame_ + 20, recovery_crc_.size()); + phase_ = recovering_ ? Phase::CheckRecoveredVolume : Phase::StartIndex; + return result_; + } + if (phase_ == Phase::StartIndex) + { + if (recovering_ && !recoveredUnchanged()) return fail(IndexedCommitStep::Invalid); + if (!operation_.emplace(volume_).begin(parent_, parent_copy_, {frame_, expected_size_}, parent_.sequence + 1, 0, *candidate_)) + return fail(IndexedCommitStep::RecoveryRequired); + phase_ = Phase::Index; + return result_; + } + auto& index = std::get(operation_); + const auto status = index.step(); + if (status == IndexTransactionStep::Working) return result_; + if (status == IndexTransactionStep::VolumeChanged) return fail(IndexedCommitStep::VolumeChanged); + if (status != IndexTransactionStep::Verified || !index.committed(committed_)) return fail(IndexedCommitStep::RecoveryRequired); + operation_.emplace(); + result_ = IndexedCommitStep::Verified; + return result_; + } + + private: + bool recoveredUnchanged() const + { + ::geocaching::storage::RecordFrameView decoded; + return expected_size_ >= 24 && !std::memcmp(frame_ + 20, recovery_crc_.data(), recovery_crc_.size()) && + ::geocaching::storage::decodeRecordFrame({frame_, expected_size_}, decoded) && decoded.sequence == parent_.sequence + 1 && + ::geocaching::storage::validTransactionRowShapes(decoded.payload, parent_.sequence); + } + bool prepare(const ::geocaching::storage::IndexRootView& parent, unsigned parent_copy, uint8_t* frame, size_t capacity, + ::geocaching::storage::IndexRootBytes& candidate) + { + if (result_ != IndexedCommitStep::Idle || !::geocaching::storage::validIndexRoot(parent) || parent_copy > 1 || + parent.sequence == UINT64_MAX || parent.revision == UINT64_MAX || !frame || capacity < 24) return false; + const auto overlap = [](::geocaching::ByteView a, ::geocaching::ByteView b) + { + const auto x = reinterpret_cast(a.data), y = reinterpret_cast(b.data); + return x <= y ? y - x < a.size : x - y < b.size; + }; + if (overlap({frame, capacity}, {candidate.data(), candidate.size()}) || overlap(parent.shards, {frame, capacity}) || + overlap(parent.shards, {candidate.data(), candidate.size()})) return false; + parent_ = parent; + parent_copy_ = static_cast(parent_copy); + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + return true; + } + enum class Phase : uint8_t + { + References, + Absent, + Authors, + ValidationVolume, + Journal, + CheckVolume, + Open, + Read, + CheckRecoveredVolume, + StartIndex, + Index + }; + IndexedCommitStep fail(IndexedCommitStep result) + { + references_.reset(); + operation_.emplace(); + consumed_ = true; + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView parent_, committed_; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0, expected_size_ = 0; + std::unique_ptr references_; + const ::geocaching::storage::MutationView* mutations_ = nullptr; + size_t mutation_count_ = 0; + std::variant operation_; + size_t author_position_ = 0; + bool author_pending_ = false; + size_t absent_count_ = 0, absent_position_ = 0; + bool absent_pending_ = false; + uint8_t parent_copy_ = 0; + bool consumed_ = false; + bool captured_ = false; + bool recovering_ = false; + bool validation_only_ = false; + std::array recovery_crc_{}; + Phase phase_ = Phase::Journal; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedCommit) <= 1024, "Commit coordination overlays its I/O workspaces"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_directory_reply.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_directory_reply.h new file mode 100644 index 00000000..044c815d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_directory_reply.h @@ -0,0 +1,181 @@ +#pragma once +#include "geocaching/protocol/directory_reply.h" +#include "geocaching/storage/queued_request.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Authenticated endpoints form key. Response and encoding workspace are caller +// leases; no response-sized object is stored in this coordinator. +class SdIndexedDirectoryReply +{ + public: + explicit SdIndexedDirectoryReply(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, ::geocaching::ByteView key, + uint8_t operation, ::geocaching::ByteView response, + ::geocaching::storage::QueuedRequestWorkspace& workspace, uint8_t* frame, size_t capacity, + ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching; + if (result_ != IndexedCommitStep::Idle || key.size != key_.size() || !key.data || copy > 1 || + (operation != 0 && operation != 2) || !response.data || response.size > kMaxApplicationBytes) return false; + const auto overlaps = [](ByteView a, ByteView b) + { + const auto x = reinterpret_cast(a.data), y = reinterpret_cast(b.data); + return a.size && b.size && (x <= y ? y - x < a.size : x - y < b.size); + }; + const ByteView buffers[] = {{workspace.outgoing, workspace.outgoing_capacity}, {workspace.task.data(), workspace.task.size()}, {frame, capacity}, {candidate.data(), candidate.size()}, root.shards}; + for (size_t i = 0; i < 5; ++i) + { + if (overlaps(response, buffers[i])) return false; + for (size_t j = 0; j < i; ++j) + if (overlaps(buffers[i], buffers[j])) return false; + } + std::memcpy(key_.data(), key.data, key_.size()); + root_ = root; + copy_ = copy; + response_ = response; + operation_code_ = operation; + workspace_ = &workspace; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + if (!io_.emplace(volume_).begin(root_, 5, {key_.data(), key_.size()}, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool inputConsumed() const + { + return result_ != IndexedCommitStep::Working || (phase_ == Phase::Commit && std::get(io_).inputConsumed()); + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + if (result_ != IndexedCommitStep::Verified) return false; + if (duplicate_) + { + out = root_; + return true; + } + return std::get(io_).committed(out); + } + IndexedCommitStep step() + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::Commit) return result_ = std::get(io_).step(); + if (phase_ == Phase::NextChild) return nextChild(); + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + if (phase_ == Phase::Outgoing) + { + OutgoingView outgoing; + const ByteView key{key_.data(), key_.size()}; + if (!decodeOutgoing(key, read.value(), outgoing)) return fail(IndexedCommitStep::Invalid); + RequestId id; + std::memcpy(id.bytes.data(), key_.data() + 32, 16); + protocol::DirectoryCapabilities capabilities; + const bool valid = operation_code_ == 0 + ? protocol::decodeDirectoryCapabilities(response_, id, capabilities) && protocol::matchesCapabilitiesRequest(outgoing.request, id, response_.size) + : protocol::matchesQueryReply(outgoing.request, response_, id); + if (!valid) return fail(IndexedCommitStep::Invalid); + duplicate_ = outgoing.state == 4; + if (duplicate_ && (outgoing.terminal_data.size != response_.size || + std::memcmp(outgoing.terminal_data.data, response_.data, response_.size))) return fail(IndexedCommitStep::Invalid); + if (!duplicate_ && !outgoing.continue_intent) return fail(IndexedCommitStep::Invalid); + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + outgoing.state = 4; + outgoing.terminal_data = response_; + if (!encodeOutgoing(key, outgoing, workspace_->outgoing, workspace_->outgoing_capacity, outgoing_size_)) return fail(IndexedCommitStep::Invalid); + if (!io_.emplace(volume_).begin(root_, 10, {task_id_.data(), task_id_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Task; + return result_; + } + if (phase_ == Phase::Task) + { + TaskView task; + OutgoingView outgoing; + const ByteView task_key{task_id_.data(), task_id_.size()}, key{key_.data(), key_.size()}; + if (!decodeTask(task_key, read.value(), task) || task.kind != 3 || + !decodeOutgoing(key, {workspace_->outgoing, outgoing_size_}, outgoing) || + !requestBelongsToTask(task_key, task, key, outgoing)) return fail(IndexedCommitStep::Invalid); + if (duplicate_) return fail(IndexedCommitStep::Verified); + if (!task.continue_intent || task.state == 5 || read.value().size > workspace_->task.size()) return fail(IndexedCommitStep::Invalid); + task_size_ = read.value().size; + std::memcpy(workspace_->task.data(), read.value().data, task_size_); + phase_ = Phase::NextChild; + return result_; + } + OutgoingView child; + if (!decodeOutgoing({child_key_.data(), child_key_.size()}, read.value(), child)) return fail(IndexedCommitStep::Invalid); + all_confirmed_ = all_confirmed_ && child.state == 4; + phase_ = Phase::NextChild; + return result_; + } + + private: + enum class Phase : uint8_t + { + Outgoing, + Task, + NextChild, + Child, + Commit + }; + IndexedCommitStep nextChild() + { + using namespace ::geocaching::storage; + TaskView task; + const ::geocaching::ByteView task_key{task_id_.data(), task_id_.size()}; + if (!decodeTask(task_key, {workspace_->task.data(), task_size_}, task)) return fail(IndexedCommitStep::Invalid); + while (child_ < task.request_count) + { + const auto key = task.requests[child_++]; + if (!std::memcmp(key.data, key_.data(), key_.size())) continue; + std::memcpy(child_key_.data(), key.data, child_key_.size()); + if (!io_.emplace(volume_).begin(root_, 5, {child_key_.data(), child_key_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Child; + return result_; + } + task.state = all_confirmed_ ? 3 : 2; + size_t size = 0; + // Encode into the now-free read frame, then retain only the small task + // encoding in its caller workspace before journal readback reuses frame. + if (!encodeTask(task_key, task, frame_, capacity_, size) || size > workspace_->task.size()) return fail(IndexedCommitStep::Invalid); + std::memcpy(workspace_->task.data(), frame_, size); + mutations_[0] = {5, {key_.data(), key_.size()}, {workspace_->outgoing, outgoing_size_}, false}; + mutations_[1] = {10, task_key, {workspace_->task.data(), size}, false}; + if (!io_.emplace(volume_).begin(root_, copy_, mutations_.data(), mutations_.size(), frame_, capacity_, *candidate_)) + return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Commit; + return result_; + } + IndexedCommitStep fail(IndexedCommitStep status) + { + io_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::ByteView response_; + ::geocaching::storage::QueuedRequestWorkspace* workspace_ = nullptr; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + std::array<::geocaching::storage::MutationView, 2> mutations_{}; + std::array key_{}, child_key_{}; + std::array task_id_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0, outgoing_size_ = 0, task_size_ = 0, child_ = 0; + unsigned copy_ = 0; + uint8_t operation_code_ = 0; + bool duplicate_ = false, all_confirmed_ = true; + std::variant io_; + Phase phase_ = Phase::Outgoing; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedDirectoryReply) <= 1536, "Reply coordination owns metadata, not response bytes"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_context.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_context.h new file mode 100644 index 00000000..10fb9786 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_context.h @@ -0,0 +1,122 @@ +#pragma once +#include "geocaching/storage/cache_head.h" +#include "geocaching/storage/task_record.h" +#include "platform/esp/arduino_common/geocaching/sd_index_get.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +// One pinned root/read-frame lease. Task/head encodings are small metadata; +// the request/response payload remains in the caller frame and is never copied. +class SdIndexedDownloadContext +{ + public: + explicit SdIndexedDownloadContext(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, ::geocaching::ByteView key, uint64_t generation, + uint8_t* frame, size_t capacity) + { + if (result_ != IndexGetStep::Idle || !key.data || key.size != key_.size() || !generation) return false; + std::memcpy(key_.data(), key.data, key_.size()); + root_ = root; + generation_ = generation; + frame_ = frame; + capacity_ = capacity; + if (!get_.emplace(volume_).begin(root_, 5, {key_.data(), key_.size()}, frame_, capacity_)) return false; + result_ = IndexGetStep::Working; + return true; + } + bool view(::geocaching::storage::OutgoingView& outgoing, ::geocaching::storage::TaskView& task, + ::geocaching::storage::CacheHeadView& head) const + { + using namespace ::geocaching::storage; + outgoing = {}; + task = {}; + head = {}; + return result_ == IndexGetStep::Ready && + decodeOutgoing({key_.data(), key_.size()}, get_->value(), outgoing) && + decodeTask({task_id_.data(), task_id_.size()}, {task_.data(), task_size_}, task) && + decodeCacheHead(task.cache_id, {head_.data(), head_size_}, head); + } + bool intentActive() const + { + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::storage::TaskView task; + ::geocaching::storage::CacheHeadView head; + return view(outgoing, task, head) && outgoing.continue_intent && task.continue_intent && task.state < 3; + } + IndexGetStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexGetStep::Working) return result_; + const auto status = get_->step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) return fail(status == IndexGetStep::NotFound ? IndexGetStep::Invalid : status); + if (phase_ == Phase::Outgoing || phase_ == Phase::Reload) + { + OutgoingView outgoing; + if (!decodeOutgoing({key_.data(), key_.size()}, get_->value(), outgoing) || outgoing.install_generation != generation_) + return fail(IndexGetStep::Invalid); + if (phase_ == Phase::Reload) + { + if (std::memcmp(outgoing.task_id.data, task_id_.data(), task_id_.size())) return fail(IndexGetStep::Invalid); + return result_ = IndexGetStep::Ready; + } + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + if (!get_.emplace(volume_).begin(root_, 10, {task_id_.data(), task_id_.size()}, frame_, capacity_)) return fail(IndexGetStep::Invalid); + phase_ = Phase::Task; + return result_; + } + TaskView task; + if (phase_ == Phase::Task) + { + if (!decodeTask({task_id_.data(), task_id_.size()}, get_->value(), task) || task.kind != 2 || + task.cache_id.size != 32 || task.revision_hash.size != 32 || get_->value().size > task_.size()) return fail(IndexGetStep::Invalid); + bool linked = false; + for (size_t i = 0; i < task.request_count; ++i) linked |= !std::memcmp(task.requests[i].data, key_.data(), key_.size()); + if (!linked) return fail(IndexGetStep::Invalid); + task_size_ = get_->value().size; + std::memcpy(task_.data(), get_->value().data, task_size_); + if (!decodeTask({task_id_.data(), task_id_.size()}, {task_.data(), task_size_}, task) || + !get_.emplace(volume_).begin(root_, 2, task.cache_id, frame_, capacity_)) return fail(IndexGetStep::Invalid); + phase_ = Phase::Head; + return result_; + } + CacheHeadView head; + if (!decodeTask({task_id_.data(), task_id_.size()}, {task_.data(), task_size_}, task) || + !decodeCacheHead(task.cache_id, get_->value(), head) || head.install_generation != generation_ || get_->value().size > head_.size()) + return fail(IndexGetStep::Invalid); + head_size_ = get_->value().size; + std::memcpy(head_.data(), get_->value().data, head_size_); + if (!get_.emplace(volume_).begin(root_, 5, {key_.data(), key_.size()}, frame_, capacity_)) return fail(IndexGetStep::Invalid); + phase_ = Phase::Reload; + return result_; + } + + private: + enum class Phase : uint8_t + { + Outgoing, + Task, + Head, + Reload + }; + IndexGetStep fail(IndexGetStep status) + { + get_.reset(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + std::array key_{}; + std::array task_id_{}; + std::array task_{}; + std::array head_{}; + std::optional get_; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0, task_size_ = 0, head_size_ = 0; + uint64_t generation_ = 0; + Phase phase_ = Phase::Outgoing; + IndexGetStep result_ = IndexGetStep::Idle; +}; +static_assert(sizeof(SdIndexedDownloadContext) <= 1152, "Download context retains metadata, not SignedCache bytes"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_reply.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_reply.h new file mode 100644 index 00000000..2291ed43 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_download_reply.h @@ -0,0 +1,169 @@ +#pragma once +#include "geocaching/protocol/get_request.h" +#include "geocaching/protocol/get_response.h" +#include "geocaching/storage/installable_record.h" +#include "geocaching/storage/queued_request.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_download_context.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Caller pins exact authenticated-source response bytes through inputConsumed. +// Signature verification, lookup and commit use separate explicit buffer leases. +class SdIndexedDownloadReply +{ + public: + SdIndexedDownloadReply(const ::geocaching::storage::VolumeInstance& volume, ::geocaching::protocol::RecordCrypto& crypto) + : volume_(volume), crypto_(crypto) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, ::geocaching::ByteView key, + uint64_t generation, ::geocaching::ByteView response, ::geocaching::storage::QueuedRequestWorkspace& workspace, + uint8_t* frame, size_t capacity, uint8_t* verify, size_t verify_capacity, ::geocaching::storage::IndexRootBytes& candidate) + { + using ::geocaching::ByteView; + if (result_ != IndexedCommitStep::Idle || copy > 1 || !key.data || key.size != key_.size() || !response.data || + response.size > ::geocaching::kMaxApplicationBytes || !verify || !verify_capacity) return false; + const ByteView leases[] = {response, {workspace.outgoing, workspace.outgoing_capacity}, {workspace.task.data(), workspace.task.size()}, {frame, capacity}, {verify, verify_capacity}, {candidate.data(), candidate.size()}, root.shards}; + for (size_t i = 0; i < 7; ++i) + for (size_t j = 0; j < i; ++j) + { + const auto a = reinterpret_cast(leases[i].data), b = reinterpret_cast(leases[j].data); + if (leases[i].size && leases[j].size && (a <= b ? b - a < leases[i].size : a - b < leases[j].size)) return false; + } + std::memcpy(key_.data(), key.data, key_.size()); + root_ = root; + copy_ = copy; + response_ = response; + workspace_ = &workspace; + frame_ = frame; + capacity_ = capacity; + verify_ = verify; + verify_capacity_ = verify_capacity; + candidate_ = &candidate; + if (!io_.emplace(volume_).begin(root_, {key_.data(), key_.size()}, generation, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool inputConsumed() const + { + return result_ != IndexedCommitStep::Working || (phase_ == Phase::Commit && std::get(io_).inputConsumed()); + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + if (result_ != IndexedCommitStep::Verified) return false; + if (duplicate_) + { + out = root_; + return true; + } + return std::get(io_).committed(out); + } + IndexedCommitStep step() + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::Commit) return result_ = std::get(io_).step(); + if (phase_ == Phase::Context) + { + auto& context = std::get(io_); + const auto status = context.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) return readError(status); + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + if (!context.view(outgoing, task, head)) return fail(IndexedCommitStep::Invalid); + RequestId id; + std::memcpy(id.bytes.data(), key_.data() + 32, 16); + protocol::GetRequestView requested; + protocol::GetResponseView reply; + if (!protocol::decodeGetRequest(outgoing.request, id, requested) || + !protocol::decodeGetResponse(response_, id, requested.budget, reply) || reply.has_conflict) return fail(IndexedCommitStep::Invalid); + GeocacheId cache_id; + RevisionHash hash; + std::memcpy(cache_id.bytes.data(), task.cache_id.data, 32); + std::memcpy(hash.bytes.data(), task.revision_hash.data, 32); + if (protocol::verifyGeocache(reply.signed_cache, crypto_, verify_, verify_capacity_, verified_, &cache_id, &hash) != protocol::VerificationResult::Valid || + verified_.record.revision < head.highest_seen_revision || head.conflict_state == 2) return fail(IndexedCommitStep::Invalid); + duplicate_ = outgoing.state == 4; + if (duplicate_ && (outgoing.terminal_data.size != response_.size || std::memcmp(outgoing.terminal_data.data, response_.data, response_.size))) + return fail(IndexedCommitStep::Invalid); + if (!duplicate_) + { + if (!context.intentActive()) return fail(IndexedCommitStep::Invalid); + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + outgoing.state = 4; + outgoing.terminal_data = response_; + task.state = 1; + size_t outgoing_size = 0, task_size = 0; + if (!encodeOutgoing({key_.data(), key_.size()}, outgoing, workspace_->outgoing, workspace_->outgoing_capacity, outgoing_size) || + !encodeTask({task_id_.data(), task_id_.size()}, task, workspace_->task.data(), workspace_->task.size(), task_size)) return fail(IndexedCommitStep::Invalid); + mutations_[0] = {5, {key_.data(), key_.size()}, {workspace_->outgoing, outgoing_size}, false}; + mutations_[1] = {10, {task_id_.data(), task_id_.size()}, {workspace_->task.data(), task_size}, false}; + } + if (!head.current_hash.size) return installableRecord(head, nullptr, verified_) ? commit() : fail(IndexedCommitStep::Invalid); + if (!encodeCacheHead(task.cache_id, head, head_.data(), head_.size(), head_size_)) return fail(IndexedCommitStep::Invalid); + std::memcpy(current_hash_.data(), head.current_hash.data, current_hash_.size()); + if (!io_.emplace(volume_).begin(root_, 1, {current_hash_.data(), current_hash_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Object; + return result_; + } + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) return readError(status); + ObjectRefView object; + CacheHeadView head; + if (!decodeObjectRef({current_hash_.data(), current_hash_.size()}, read.value(), object) || + !decodeCacheHead({verified_.id.bytes.data(), 32}, {head_.data(), head_size_}, head) || !installableRecord(head, &object, verified_)) + return fail(IndexedCommitStep::Invalid); + return commit(); + } + + private: + enum class Phase : uint8_t + { + Context, + Object, + Commit + }; + IndexedCommitStep commit() + { + if (duplicate_) return fail(IndexedCommitStep::Verified); + if (!io_.emplace(volume_).begin(root_, copy_, mutations_.data(), mutations_.size(), frame_, capacity_, *candidate_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Commit; + return result_; + } + IndexedCommitStep readError(IndexGetStep status) + { + return fail(status == IndexGetStep::IoError ? IndexedCommitStep::IoError : status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged + : IndexedCommitStep::Invalid); + } + IndexedCommitStep fail(IndexedCommitStep status) + { + io_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::protocol::RecordCrypto& crypto_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::protocol::VerifiedRecordView verified_; + ::geocaching::ByteView response_; + ::geocaching::storage::QueuedRequestWorkspace* workspace_ = nullptr; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + std::array<::geocaching::storage::MutationView, 2> mutations_{}; + std::array key_{}; + std::array task_id_{}; + std::array current_hash_{}; + std::array head_{}; + uint8_t *frame_ = nullptr, *verify_ = nullptr; + size_t capacity_ = 0, verify_capacity_ = 0, head_size_ = 0; + unsigned copy_ = 0; + bool duplicate_ = false; + std::variant io_; + Phase phase_ = Phase::Context; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedDownloadReply) <= 2048, "Download receipt owns metadata, not response-sized buffers"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_draft_save.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_draft_save.h new file mode 100644 index 00000000..a8d719dc --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_draft_save.h @@ -0,0 +1,171 @@ +#pragma once +#include "geocaching/storage/draft_publication.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +// One serialized owner pins the root, input encoding and frame until terminal. +// No draft list or text copy is owned. Keep this operation off ESP task stacks. +class SdIndexedDraftSave +{ + public: + explicit SdIndexedDraftSave(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, ::geocaching::ByteView key, + ::geocaching::ByteView encoded, uint64_t expected_generation, uint8_t* frame, size_t capacity, + ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Idle || copy > 1 || !decodeDraft(key, encoded, next_) || + expected_generation == UINT64_MAX || next_.generation != expected_generation + 1) return false; + const auto overlaps = [](::geocaching::ByteView a, ::geocaching::ByteView b) + { + const auto x = reinterpret_cast(a.data), y = reinterpret_cast(b.data); + return a.size && b.size && (x <= y ? y - x < a.size : x - y < b.size); + }; + if (overlaps(encoded, {frame, capacity}) || overlaps(encoded, {candidate.data(), candidate.size()}) || + overlaps({frame, capacity}, {candidate.data(), candidate.size()}) || overlaps(root.shards, {candidate.data(), candidate.size()})) return false; + std::memcpy(key_.data(), key.data, key_.size()); + mutation_ = {4, {key_.data(), key_.size()}, encoded, false}; + root_ = root; + copy_ = copy; + expected_ = expected_generation; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + if (!operation_.emplace(volume_).begin(root_, 4, mutation_.key, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool inputConsumed() const + { + return result_ != IndexedCommitStep::Working || + (phase_ == Phase::Commit && std::get(operation_).inputConsumed()); + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + return result_ == IndexedCommitStep::Verified && std::get(operation_).committed(out); + } + IndexedCommitStep step() + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (phase_ == Phase::Previous) + { + auto& read = std::get(operation_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready && status != IndexGetStep::NotFound) return readFailure(status); + DraftView previous; + if (status == IndexGetStep::Ready && !decodeDraft(mutation_.key, read.value(), previous)) return fail(IndexedCommitStep::Invalid); + const auto check = checkDraftUpdate(status == IndexGetStep::Ready ? &previous : nullptr, next_, expected_); + if (check == DraftUpdateCheck::Allowed) return startCommit(); + if (check != DraftUpdateCheck::NeedsRetainedPublication) return fail(IndexedCommitStep::Invalid); + scan_.reset(new (std::nothrow) SdIndexScan(volume_)); + if (!scan_ || !scan_->begin(root_, 5, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + operation_.emplace(); + phase_ = Phase::Requests; + return result_; + } + if (phase_ == Phase::Requests) + { + const auto status = scan_->step(); + if (status == IndexScanStep::Working) return result_; + if (status != IndexScanStep::Item) + return fail(status == IndexScanStep::IoError ? IndexedCommitStep::IoError : status == IndexScanStep::VolumeChanged ? IndexedCommitStep::VolumeChanged + : IndexedCommitStep::Invalid); + MutationView row; + OutgoingView outgoing; + if (!scan_->item(row) || !decodeOutgoing(row.key, row.value, outgoing)) return fail(IndexedCommitStep::Invalid); + RequestId request; + std::memcpy(request.bytes.data(), row.key.data + 32, 16); + protocol::PublishRequestView publish; + bool matches = false; + if (next_.author.size == 64 && protocol::decodePublishRequest(outgoing.request, request, publish)) + { + protocol::CmpReader reader(publish.signed_cache); + size_t fields = 0; + ByteView encoded, signature; + RecordView record; + matches = reader.array(fields, 2) && fields == 2 && reader.binary(encoded, kMaxRecordBytes) && + reader.binary(signature, 64) && reader.finished() && protocol::decodeGeocacheRecord(encoded, record) && + !std::memcmp(record.author_public_key.data, next_.author.data, 64) && + !std::memcmp(record.creation_nonce.data, key_.data(), 16); + } + if (matches) + { + std::memcpy(request_key_.data(), row.key.data, request_key_.size()); + std::memcpy(task_key_.data(), outgoing.task_id.data, task_key_.size()); + if (!operation_.emplace(volume_).begin(root_, 10, outgoing.task_id, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Task; + } + if (!scan_->advance()) return fail(IndexedCommitStep::Invalid); + return result_; + } + if (phase_ == Phase::Task) + { + auto& read = std::get(operation_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) return readFailure(status); + TaskView task; + if (!decodeTask({task_key_.data(), task_key_.size()}, read.value(), task)) return fail(IndexedCommitStep::Invalid); + bool linked = false; + for (size_t i = 0; i < task.request_count; ++i) + linked |= !std::memcmp(task.requests[i].data, request_key_.data(), request_key_.size()); + if (linked && task.kind == 1 && task.cache_id.size == 32 && task.revision_hash.size == 32 && + !std::memcmp(task.revision_hash.data, next_.base_hash.data, 32)) return startCommit(); + operation_.emplace(); + phase_ = Phase::Requests; + return result_; + } + return result_ = std::get(operation_).step(); + } + + private: + enum class Phase : uint8_t + { + Previous, + Requests, + Task, + Commit + }; + IndexedCommitStep startCommit() + { + scan_.reset(); + if (!operation_.emplace(volume_).begin(root_, copy_, &mutation_, 1, frame_, capacity_, *candidate_)) + return fail(IndexedCommitStep::Invalid); + phase_ = Phase::Commit; + return result_; + } + IndexedCommitStep readFailure(IndexGetStep status) + { + return fail(status == IndexGetStep::IoError ? IndexedCommitStep::IoError : status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged + : IndexedCommitStep::Invalid); + } + IndexedCommitStep fail(IndexedCommitStep status) + { + scan_.reset(); + operation_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::storage::DraftView next_; + ::geocaching::storage::MutationView mutation_; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + std::array key_{}; + std::array task_key_{}; + std::array request_key_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + uint64_t expected_ = 0; + unsigned copy_ = 0; + std::variant operation_; + std::unique_ptr scan_; + Phase phase_ = Phase::Previous; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedDraftSave) <= 1536, "Draft save owns metadata, not draft text"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_new_task.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_new_task.h new file mode 100644 index 00000000..2e05717d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_new_task.h @@ -0,0 +1,130 @@ +#pragma once +#include "geocaching/protocol/get_request.h" +#include "geocaching/storage/object_ref.h" +#include "geocaching/storage/queued_request.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +// The caller leases encoding workspace, frame and roots until inputConsumed / +// completion. The same task/request encoders serve both storage implementations. +class SdIndexedNewTask +{ + public: + explicit SdIndexedNewTask(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, + const ::geocaching::Destination& local, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& request_id, const std::array& task_id, uint8_t kind, + ::geocaching::ByteView request, const ::geocaching::storage::StoredTime& time, + const ::geocaching::storage::RequestTaskTarget& target, + ::geocaching::storage::QueuedRequestWorkspace& workspace, + uint8_t* frame, size_t capacity, ::geocaching::storage::IndexRootBytes& candidate) + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Idle || copy > 1 || !validIndexRoot(root)) return false; + const auto overlaps = [](ByteView a, ByteView b) + { + const auto x = reinterpret_cast(a.data), y = reinterpret_cast(b.data); + return a.size && b.size && (x <= y ? y - x < a.size : x - y < b.size); + }; + const ByteView outgoing{workspace.outgoing, workspace.outgoing_capacity}, task{workspace.task.data(), workspace.task.size()}; + const ByteView scratch{frame, capacity}, output{candidate.data(), candidate.size()}; + if (overlaps(request, outgoing) || overlaps(request, task) || overlaps(outgoing, task) || + overlaps(outgoing, scratch) || overlaps(task, scratch) || overlaps(outgoing, output) || overlaps(task, output) || + overlaps(scratch, output) || overlaps(root.shards, output) || overlaps(root.shards, outgoing) || overlaps(root.shards, task) || + overlaps(target.cache_id, outgoing) || overlaps(target.cache_id, task) || + overlaps(target.revision_hash, outgoing) || overlaps(target.revision_hash, task)) return false; + task_id_ = task_id; + // prepareNewRequestTask requires a two-element array reference. + MutationView prepared[2]; + if (!prepareNewRequestTask(local, remote, request_id, task_id_, kind, request, time, workspace, key_, prepared, target)) return false; + mutations_[0] = prepared[0]; + mutations_[1] = prepared[1]; + root_ = root; + copy_ = copy; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + count_ = 2; + result_ = IndexedCommitStep::Working; + if (kind != 2) return startCommit() == IndexedCommitStep::Working; + protocol::GetRequestView get; + if (!protocol::decodeGetRequest(request, request_id, get) || get.wanted_hash.size != 32 || + std::memcmp(get.cache_id.data, target.cache_id.data, 32) || std::memcmp(get.wanted_hash.data, target.revision_hash.data, 32)) + { + fail(IndexedCommitStep::Invalid); + return false; + } + std::memcpy(cache_id_.data(), target.cache_id.data, cache_id_.size()); + generation_ = target.install_generation; + if (!operation_.emplace(volume_).begin(root_, 2, {cache_id_.data(), cache_id_.size()}, frame_, capacity_)) + { + fail(IndexedCommitStep::Invalid); + return false; + } + return true; + } + bool inputConsumed() const + { + return result_ != IndexedCommitStep::Working || (committing_ && std::get(operation_).inputConsumed()); + } + bool committed(::geocaching::storage::IndexRootView& out) const + { + out = {}; + return result_ == IndexedCommitStep::Verified && std::get(operation_).committed(out); + } + IndexedCommitStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (committing_) return result_ = std::get(operation_).step(); + auto& read = std::get(operation_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready && status != IndexGetStep::NotFound) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + CacheHeadView head; + const ::geocaching::ByteView key{cache_id_.data(), cache_id_.size()}; + if (status == IndexGetStep::Ready && !decodeCacheHead(key, read.value(), head)) return fail(IndexedCommitStep::Invalid); + if (head.install_generation == UINT64_MAX || generation_ != head.install_generation + 1) return fail(IndexedCommitStep::Invalid); + head.install_generation = generation_; + size_t size = 0; + if (!encodeCacheHead(key, head, head_.data(), head_.size(), size)) return fail(IndexedCommitStep::Invalid); + mutations_[2] = {2, key, {head_.data(), size}, false}; + count_ = 3; + return startCommit(); + } + + private: + IndexedCommitStep startCommit() + { + if (!operation_.emplace(volume_).begin(root_, copy_, mutations_.data(), count_, frame_, capacity_, *candidate_, false, 2)) + return fail(IndexedCommitStep::Invalid); + committing_ = true; + return result_; + } + IndexedCommitStep fail(IndexedCommitStep status) + { + operation_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + std::array<::geocaching::storage::MutationView, 3> mutations_{}; + std::array key_{}; + std::array task_id_{}; + std::array cache_id_{}; + std::array head_{}; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0, count_ = 0; + uint64_t generation_ = 0; + unsigned copy_ = 0; + bool committing_ = false; + std::variant operation_; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedNewTask) <= 1536, "Task creation owns metadata only"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_pending_request.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_pending_request.h new file mode 100644 index 00000000..f288617d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_pending_request.h @@ -0,0 +1,163 @@ +#pragma once +#include "geocaching/storage/pending_request.h" +#include "platform/esp/arduino_common/geocaching/sd_index_get.h" +#include "platform/esp/arduino_common/geocaching/sd_index_scan.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexedPendingStep : uint8_t +{ + Idle, + Working, + Ready, + None, + Invalid, + IoError, + VolumeChanged, + WorkspaceTooSmall +}; + +// Snapshot root remains pinned. A selected request borrows the caller frame +// until this operation is released; the dispatcher reloads it after committing +// a send attempt. One key is retained, not an array of pending requests. +class SdIndexedPendingRequest +{ + public: + explicit SdIndexedPendingRequest(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume), scan_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, const ::geocaching::Destination& local, + ::geocaching::ByteView after, uint8_t* frame, size_t capacity) + { + if (result_ != IndexedPendingStep::Idle || (after.size && (!after.data || after.size != 48))) return false; + if (!scan_.begin(root, 5, frame, capacity)) return false; + root_ = root; + local_ = local; + if (after.size) std::memcpy(after_.data(), after.data, after_.size()); + has_after_ = after.size != 0; + frame_ = frame; + capacity_ = capacity; + result_ = IndexedPendingStep::Working; + return true; + } + bool selected(::geocaching::storage::PendingRequestView& out) const + { + out = {}; + if (result_ != IndexedPendingStep::Ready) return false; + ::geocaching::storage::OutgoingView row; + if (!::geocaching::storage::decodeOutgoing({best_.data(), best_.size()}, get_->value(), row)) return false; + out.key = best_; + std::memcpy(out.destination.bytes.data(), best_.data() + 16, 16); + std::memcpy(out.request_id.bytes.data(), best_.data() + 32, 16); + out.request = row.request; + out.task_id = row.task_id; + return true; + } + IndexedPendingStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedPendingStep::Working) return result_; + if (phase_ == Phase::Scan) + { + const auto status = scan_.step(); + if (status == IndexScanStep::Working) return result_; + if (status == IndexScanStep::End) + { + if (!found_) return finish(IndexedPendingStep::None); + if (!get_.emplace(volume_).begin(root_, 5, {best_.data(), best_.size()}, frame_, capacity_)) return finish(IndexedPendingStep::Invalid); + phase_ = Phase::Selected; + return result_; + } + if (status != IndexScanStep::Item) return error(status); + MutationView row; + OutgoingView outgoing; + if (!scan_.item(row) || !decodeOutgoing(row.key, row.value, outgoing)) return finish(IndexedPendingStep::Invalid); + const bool candidate = outgoing.continue_intent && (outgoing.state == 0 || outgoing.state == 3) && + !std::memcmp(row.key.data, local_.bytes.data(), 16) && + (!has_after_ || std::memcmp(row.key.data, after_.data(), 48) > 0) && + (!found_ || std::memcmp(row.key.data, best_.data(), 48) < 0); + if (candidate) + { + std::memcpy(key_.data(), row.key.data, key_.size()); + std::memcpy(task_.data(), outgoing.task_id.data, task_.size()); + generation_ = outgoing.install_generation; + if (!get_.emplace(volume_).begin(root_, 10, {task_.data(), task_.size()}, frame_, capacity_)) return finish(IndexedPendingStep::Invalid); + phase_ = Phase::Task; + } + if (!scan_.advance()) return finish(IndexedPendingStep::Invalid); + return result_; + } + const auto status = get_->step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) return error(status); + if (phase_ == Phase::Selected) return result_ = IndexedPendingStep::Ready; + if (phase_ == Phase::Task) + { + TaskView task; + if (!decodeTask({task_.data(), task_.size()}, get_->value(), task)) return finish(IndexedPendingStep::Invalid); + bool linked = false; + for (size_t i = 0; i < task.request_count; ++i) linked |= !std::memcmp(task.requests[i].data, key_.data(), key_.size()); + if (!linked) return finish(IndexedPendingStep::Invalid); + if (!task.continue_intent || task.state > 2) return continueScan(false); + if ((task.kind == 2 || task.kind == 4) && generation_) + { + if (task.cache_id.size != cache_.size()) return finish(IndexedPendingStep::Invalid); + std::memcpy(cache_.data(), task.cache_id.data, cache_.size()); + if (!get_.emplace(volume_).begin(root_, 2, {cache_.data(), cache_.size()}, frame_, capacity_)) return finish(IndexedPendingStep::Invalid); + phase_ = Phase::Head; + return result_; + } + return continueScan(true); + } + CacheHeadView head; + if (!decodeCacheHead({cache_.data(), cache_.size()}, get_->value(), head)) return finish(IndexedPendingStep::Invalid); + return continueScan(head.install_generation == generation_); + } + + private: + enum class Phase : uint8_t + { + Scan, + Task, + Head, + Selected + }; + IndexedPendingStep continueScan(bool eligible) + { + if (eligible) + { + best_ = key_; + found_ = true; + } + get_.reset(); + phase_ = Phase::Scan; + return result_; + } + template + IndexedPendingStep error(Status status) + { + return finish(status == Status::VolumeChanged ? IndexedPendingStep::VolumeChanged : status == Status::IoError ? IndexedPendingStep::IoError + : status == Status::WorkspaceTooSmall ? IndexedPendingStep::WorkspaceTooSmall + : IndexedPendingStep::Invalid); + } + IndexedPendingStep finish(IndexedPendingStep status) + { + get_.reset(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::Destination local_; + SdIndexScan scan_; + std::optional get_; + std::array after_{}, key_{}, best_{}; + std::array task_{}; + std::array cache_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + uint64_t generation_ = 0; + bool has_after_ = false, found_ = false; + Phase phase_ = Phase::Scan; + IndexedPendingStep result_ = IndexedPendingStep::Idle; +}; +static_assert(sizeof(SdIndexedPendingRequest) <= 1792, "Pending selection retains keys, not request payloads"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_recovery.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_recovery.h new file mode 100644 index 00000000..83c7ac56 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_recovery.h @@ -0,0 +1,217 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_index_import.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h" +#include "platform/esp/arduino_common/geocaching/sd_index_initialize.h" +#include "platform/esp/arduino_common/geocaching/sd_index_replay.h" +#include "platform/esp/arduino_common/geocaching/sd_index_root_reader.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexedRecoveryStep : uint8_t +{ + Working, + Restored, + IoError, + VolumeChanged, + RecoveryRequired +}; + +// One storage owner, fresh instance per recovery. Buffers are disjoint caller +// leases; no logical table arena is used. Restored precedes GPX-install recovery +// and application readiness. Partial index generations are never overwritten. +template +class SdIndexedRecovery +{ + public: + SdIndexedRecovery(const ::geocaching::storage::VolumeInstance& volume, + ::geocaching::storage::IndexRootBytes& first, ::geocaching::storage::IndexRootBytes& second, + uint8_t* frame, size_t capacity, uint8_t* validation_frame, size_t validation_capacity, + ::geocaching::storage::MutationView* mutations, size_t mutation_capacity) + : volume_(volume), roots_{&first, &second}, frame_(frame), capacity_(capacity), validation_frame_(validation_frame), + validation_capacity_(validation_capacity), mutations_(mutations), mutation_capacity_(mutation_capacity) + { + using ::geocaching::ByteView; + if (!frame || !validation_frame || !mutations || !mutation_capacity || mutation_capacity > 64 || capacity < 24 || validation_capacity < 24) + { + result_ = IndexedRecoveryStep::RecoveryRequired; + return; + } + const ByteView leases[] = {{first.data(), first.size()}, {second.data(), second.size()}, {frame, capacity}, {validation_frame, validation_capacity}, {reinterpret_cast(mutations), mutation_capacity * sizeof(*mutations)}}; + for (size_t i = 0; i < 5; ++i) + for (size_t j = 0; j < i; ++j) + { + const auto a = reinterpret_cast(leases[i].data), b = reinterpret_cast(leases[j].data); + if (a <= b ? b - a < leases[i].size : a - b < leases[j].size) result_ = IndexedRecoveryStep::RecoveryRequired; + } + } + bool selected(::geocaching::storage::IndexRootView& root, unsigned& copy) const + { + root = {}; + if (result_ != IndexedRecoveryStep::Restored) return false; + root = root_; + copy = copy_; + return true; + } + IndexedRecoveryStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedRecoveryStep::Working) return result_; + switch (phase_) + { + case Phase::Volume: + { + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) return finish(IndexedRecoveryStep::IoError); + if (current != volume_) return finish(IndexedRecoveryStep::VolumeChanged); + phase_ = Phase::ProbeFirst; + return result_; + } + case Phase::ProbeFirst: + case Phase::ProbeSecond: + { + uint8_t probe[24]; + const auto read = storage::sd_read_file(phase_ == Phase::ProbeFirst ? "/trailmate/geocaching/.state/index/root.h0" : "/trailmate/geocaching/.state/index/root.h1", probe, sizeof(probe)); + if (read.status != storage::SdFileReadStatus::Missing && read.status != storage::SdFileReadStatus::Ready && + read.status != storage::SdFileReadStatus::Invalid) return finish(IndexedRecoveryStep::IoError); + root_present_ |= read.status != storage::SdFileReadStatus::Missing; + if (phase_ == Phase::ProbeFirst) + { + phase_ = Phase::ProbeSecond; + return result_; + } + if (root_present_) + { + if (!io_.template emplace(volume_).begin(*roots_[0], *roots_[1])) return finish(IndexedRecoveryStep::RecoveryRequired); + phase_ = Phase::Roots; + } + else + { + io_.template emplace>(volume_); + phase_ = Phase::Checkpoint; + } + return result_; + } + case Phase::Roots: + { + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexRootReadStep::Working) return result_; + if (status != IndexRootReadStep::Ready || !read.selected(root_)) return error(status); + copy_ = static_cast(read.selectedCopy()); + return inventory(); + } + case Phase::Checkpoint: + { + auto& selection = std::get>(io_); + const auto status = selection.stepCursor(frame_, capacity_); + if (status == CheckpointSelectionStep::Reading) return result_; + if (status == CheckpointSelectionStep::NoCheckpoint) + { + if (!io_.template emplace(volume_).begin(*roots_[0])) return finish(IndexedRecoveryStep::RecoveryRequired); + phase_ = Phase::Initialize; + return result_; + } + if (status == CheckpointSelectionStep::RetryLater) return finish(IndexedRecoveryStep::IoError); + if (status == CheckpointSelectionStep::VolumeChanged) return finish(IndexedRecoveryStep::VolumeChanged); + if (status != CheckpointSelectionStep::Selected) return finish(IndexedRecoveryStep::RecoveryRequired); + const bool second = selection.choice() == CheckpointChoice::SlotB; + const auto checkpoint = selection.candidate(second); + if (!io_.template emplace>(volume_, import_digest_).begin(second ? 'b' : 'a', checkpoint, frame_, capacity_, *roots_[0])) return finish(IndexedRecoveryStep::RecoveryRequired); + phase_ = Phase::Import; + return result_; + } + case Phase::Initialize: + case Phase::Import: + { + const auto status = phase_ == Phase::Initialize ? std::get(io_).step() : std::get>(io_).step(); + if (status == IndexRootWriteStep::Working) return result_; + if (status != IndexRootWriteStep::Verified) return error(status); + *roots_[1] = *roots_[0]; + if (!decodeIndexRoot({roots_[0]->data(), roots_[0]->size()}, volume_, root_)) return finish(IndexedRecoveryStep::RecoveryRequired); + copy_ = 0; + return inventory(); + } + case Phase::Inventory: + { + auto& scan = std::get(io_); + const auto status = scan.step(); + if (status == InventoryStep::Scanning) return result_; + if (status == InventoryStep::RetryLater) return finish(IndexedRecoveryStep::IoError); + if (status == InventoryStep::VolumeChanged) return finish(IndexedRecoveryStep::VolumeChanged); + if (status != InventoryStep::Complete) return finish(IndexedRecoveryStep::RecoveryRequired); + const auto range = scan.range(); + io_.template emplace(volume_, root_, copy_, *roots_[0], *roots_[1], range, + frame_, capacity_, mutations_, mutation_capacity_); + phase_ = Phase::Replay; + return result_; + } + case Phase::Replay: + { + auto& replay = std::get(io_); + const auto status = replay.step(); + if (status == IndexReplayStep::Working) return result_; + if (status == IndexReplayStep::NeedsValidation) + { + // The shared validator enforces stored row shapes, task/attempt + // references and immutable author history before root publication. + if (!replay.accept(true, validation_frame_, validation_capacity_)) return finish(IndexedRecoveryStep::RecoveryRequired); + return result_; + } + if (status == IndexReplayStep::RetryLater) return finish(IndexedRecoveryStep::IoError); + if (status == IndexReplayStep::VolumeChanged) return finish(IndexedRecoveryStep::VolumeChanged); + if (status != IndexReplayStep::Complete || !replay.selected(root_, copy_)) return finish(IndexedRecoveryStep::RecoveryRequired); + return finish(IndexedRecoveryStep::Restored); + } + } + return finish(IndexedRecoveryStep::RecoveryRequired); + } + + private: + enum class Phase : uint8_t + { + Volume, + ProbeFirst, + ProbeSecond, + Roots, + Checkpoint, + Initialize, + Import, + Inventory, + Replay + }; + IndexedRecoveryStep inventory() + { + io_.template emplace(volume_, root_.sequence); + phase_ = Phase::Inventory; + return result_; + } + template + IndexedRecoveryStep error(Status status) + { + return finish(status == Status::VolumeChanged ? IndexedRecoveryStep::VolumeChanged : status == Status::IoError ? IndexedRecoveryStep::IoError + : IndexedRecoveryStep::RecoveryRequired); + } + IndexedRecoveryStep finish(IndexedRecoveryStep status) + { + io_.template emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootBytes* roots_[2]; + ::geocaching::storage::IndexRootView root_; + uint8_t* frame_; + size_t capacity_; + uint8_t* validation_frame_; + size_t validation_capacity_; + ::geocaching::storage::MutationView* mutations_; + size_t mutation_capacity_; + Digest import_digest_; + std::variant, SdIndexInitialize, + SdCheckpointIndexImport, SdJournalInventory, SdIndexReplay> + io_; + unsigned copy_ = 0; + bool root_present_ = false; + Phase phase_ = Phase::Volume; + IndexedRecoveryStep result_ = IndexedRecoveryStep::Working; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_stop_task.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_stop_task.h new file mode 100644 index 00000000..49fdf856 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_stop_task.h @@ -0,0 +1,110 @@ +#pragma once +#include "geocaching/storage/task_record.h" +#include "platform/esp/arduino_common/geocaching/sd_indexed_commit.h" + +namespace platform::esp::arduino_common::geocaching +{ +// A task stop governs all child requests. Only small task encoding is retained; +// request payloads remain in the caller's reusable read frame. +class SdIndexedStopTask +{ + public: + explicit SdIndexedStopTask(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexRootView& root, unsigned copy, + ::geocaching::ByteView key, bool by_request, uint8_t* frame, size_t capacity, + ::geocaching::storage::IndexRootBytes& candidate) + { + if (result_ != IndexedCommitStep::Idle || copy > 1 || !key.data || key.size != (by_request ? 48 : 16)) return false; + const auto a = reinterpret_cast(frame), b = reinterpret_cast(candidate.data()); + if (a <= b ? b - a < capacity : a - b < candidate.size()) return false; + root_ = root; + copy_ = copy; + frame_ = frame; + capacity_ = capacity; + candidate_ = &candidate; + request_ = by_request; + std::memcpy(key_.data(), key.data, key.size); + if (!io_.emplace(volume_).begin(root_, by_request ? 5 : 10, {key_.data(), key.size}, frame_, capacity_)) return false; + result_ = IndexedCommitStep::Working; + return true; + } + bool committed(::geocaching::storage::IndexRootView& root) const + { + root = {}; + if (result_ != IndexedCommitStep::Verified) return false; + if (duplicate_) + { + root = root_; + return true; + } + return std::get(io_).committed(root); + } + IndexedCommitStep step() + { + using namespace ::geocaching::storage; + if (result_ != IndexedCommitStep::Working) return result_; + if (committing_) return result_ = std::get(io_).step(); + auto& read = std::get(io_); + const auto status = read.step(); + if (status == IndexGetStep::Working) return result_; + if (status != IndexGetStep::Ready) + return fail(status == IndexGetStep::VolumeChanged ? IndexedCommitStep::VolumeChanged : status == IndexGetStep::IoError ? IndexedCommitStep::IoError + : IndexedCommitStep::Invalid); + if (request_) + { + OutgoingView outgoing; + if (!decodeOutgoing({key_.data(), 48}, read.value(), outgoing)) return fail(IndexedCommitStep::Invalid); + std::memcpy(task_id_.data(), outgoing.task_id.data, task_id_.size()); + if (!io_.emplace(volume_).begin(root_, 10, {task_id_.data(), task_id_.size()}, frame_, capacity_)) return fail(IndexedCommitStep::Invalid); + request_ = false; + from_request_ = true; + return result_; + } + if (!from_request_) std::memcpy(task_id_.data(), key_.data(), task_id_.size()); + const ::geocaching::ByteView task_key{task_id_.data(), task_id_.size()}; + TaskView task; + if (!decodeTask(task_key, read.value(), task)) return fail(IndexedCommitStep::Invalid); + if (from_request_) + { + bool linked = false; + for (size_t i = 0; i < task.request_count; ++i) linked |= !std::memcmp(task.requests[i].data, key_.data(), 48); + if (!linked) return fail(IndexedCommitStep::Invalid); + } + if (task.state == 5 && !task.continue_intent) + { + duplicate_ = true; + return fail(IndexedCommitStep::Verified); + } + if (task.state == 3) return fail(IndexedCommitStep::Invalid); + task.state = 5; + task.continue_intent = false; + size_t size = 0; + if (!encodeTask(task_key, task, encoded_.data(), encoded_.size(), size)) return fail(IndexedCommitStep::Invalid); + mutation_ = {10, task_key, {encoded_.data(), size}, false}; + if (!io_.emplace(volume_).begin(root_, copy_, &mutation_, 1, frame_, capacity_, *candidate_)) return fail(IndexedCommitStep::Invalid); + committing_ = true; + return result_; + } + + private: + IndexedCommitStep fail(IndexedCommitStep status) + { + io_.emplace(); + return result_ = status; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::IndexRootView root_; + ::geocaching::storage::IndexRootBytes* candidate_ = nullptr; + ::geocaching::storage::MutationView mutation_; + std::array key_{}; + std::array task_id_{}; + std::array encoded_{}; + uint8_t* frame_ = nullptr; + size_t capacity_ = 0; + unsigned copy_ = 0; + bool request_ = false, from_request_ = false, committing_ = false, duplicate_ = false; + std::variant io_; + IndexedCommitStep result_ = IndexedCommitStep::Idle; +}; +static_assert(sizeof(SdIndexedStopTask) <= 1536, "Stopping a task must not own request payloads"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h new file mode 100644 index 00000000..56b5b983 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_indexed_value_reader.h @@ -0,0 +1,172 @@ +#pragma once +#include "geocaching/storage/checkpoint_index_cursor.h" +#include "geocaching/storage/index_entry.h" +#include "platform/esp/arduino_common/geocaching/sd_record_reader.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class IndexedReadStep : uint8_t +{ + Idle, + Working, + Ready, + Erased, + Invalid, + IoError, + VolumeChanged, + WorkspaceTooSmall +}; + +// One immutable value lease. Caller owns the frame buffer through consumption; +// each step performs at most 512 data bytes of I/O. The index is only a hint. +class SdIndexedValueReader +{ + public: + explicit SdIndexedValueReader(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + bool begin(const ::geocaching::storage::IndexedMutation& entry, uint8_t* frame, size_t capacity) + { + if (result_ != IndexedReadStep::Idle || !::geocaching::storage::validIndexEntry(entry) || !entry.key.data || !entry.key.size || entry.key.size > key_.size() || + entry.table < 1 || entry.table > 13 || !entry.location.record_sequence || !entry.location.segment_first_sequence || + entry.location.segment_first_sequence > entry.location.record_sequence || !frame || capacity < 24) return false; + table_ = entry.table; + key_size_ = entry.key.size; + erase_ = entry.erase; + location_ = entry.location; + std::memcpy(key_.data(), entry.key.data, key_size_); + bytes_ = frame; + capacity_ = capacity; + result_ = IndexedReadStep::Working; + return true; + } + ::geocaching::ByteView value() const { return result_ == IndexedReadStep::Ready ? value_ : ::geocaching::ByteView{}; } + IndexedReadStep step() + { + if (result_ != IndexedReadStep::Working) return result_; + if (phase_ == Phase::Volume || phase_ == Phase::VerifyVolume) + { + ::geocaching::storage::VolumeInstance current; + const auto status = inspectSdVolume(current); + if (status != SdVolumeResult::Ready) return fail(IndexedReadStep::IoError); + if (current != volume_) return fail(IndexedReadStep::VolumeChanged); + if (phase_ == Phase::VerifyVolume) return result_ = erase_ ? IndexedReadStep::Erased : IndexedReadStep::Ready; + phase_ = Phase::Open; + return result_; + } + if (phase_ == Phase::Open) + { + char path[80]; + if (location_.source == ::geocaching::storage::IndexedValueSource::Journal) + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/journal/%016llx.gcj", + static_cast(location_.segment_first_sequence)); + else std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/checkpoint/%c.gcs", + location_.source == ::geocaching::storage::IndexedValueSource::CheckpointA ? 'a' : 'b'); + if (!file_.open(path, "r")) return fail(IndexedReadStep::IoError); + phase_ = Phase::Size; + return result_; + } + if (phase_ == Phase::Size) + { + length_ = file_.size(); + const uint64_t limit = location_.source == ::geocaching::storage::IndexedValueSource::Journal ? 1024U * 1024U : UINT32_MAX; + if (!length_ || length_ > limit || location_.frame_offset >= length_) return fail(IndexedReadStep::Invalid); + phase_ = Phase::Seek; + return result_; + } + if (phase_ == Phase::Seek) + { + if (!file_.seek(location_.frame_offset)) return fail(IndexedReadStep::IoError); + offset_ = location_.frame_offset; + phase_ = Phase::Read; + return result_; + } + if (phase_ == Phase::Read) + { + ::geocaching::storage::RecordFrameView frame; + const auto read = readSdRecord(file_, length_, offset_, bytes_, capacity_, read_cursor_, frame); + if (read == SegmentReadResult::InProgress) return result_; + if (read == SegmentReadResult::WorkspaceTooSmall) return fail(IndexedReadStep::WorkspaceTooSmall); + if (read == SegmentReadResult::IoError) return fail(IndexedReadStep::IoError); + if (read != SegmentReadResult::Record) return fail(IndexedReadStep::Invalid); + frame_size_ = frame.payload.size + 24; + phase_ = Phase::Validate; + return result_; + } + if (phase_ == Phase::Validate) + { + auto locate = [&](auto& cursor) + { + ::geocaching::storage::IndexedMutation entry; + while (cursor.next(entry)) + { + if (entry.table != table_ || entry.key.size != key_size_ || std::memcmp(entry.key.data, key_.data(), key_size_)) continue; + if (entry.erase != erase_ || entry.location.value_offset != location_.value_offset || entry.location.value_size != location_.value_size) return false; + if (!erase_) value_ = {bytes_ + (entry.location.value_offset - location_.frame_offset), entry.location.value_size}; + return true; + } + return false; + }; + bool found = false; + if (location_.source == ::geocaching::storage::IndexedValueSource::Journal) + { + ::geocaching::storage::TransactionIndexCursor cursor; + found = cursor.open({bytes_, frame_size_}, location_.record_sequence - 1, location_.segment_first_sequence, location_.frame_offset) && locate(cursor); + } + else + { + ::geocaching::storage::CheckpointIndexCursor cursor; + found = cursor.open({bytes_, frame_size_}, location_.record_sequence, + location_.source == ::geocaching::storage::IndexedValueSource::CheckpointA ? 'a' : 'b', location_.frame_offset) && + locate(cursor); + } + if (!found) return fail(IndexedReadStep::Invalid); + phase_ = Phase::CheckSize; + return result_; + } + if (phase_ == Phase::CheckSize) + { + if (file_.size() != length_) return fail(IndexedReadStep::Invalid); + phase_ = Phase::Close; + return result_; + } + file_.close(); + phase_ = Phase::VerifyVolume; + return result_; + } + + private: + enum class Phase : uint8_t + { + Volume, + Open, + Size, + Seek, + Read, + Validate, + CheckSize, + Close, + VerifyVolume + }; + IndexedReadStep fail(IndexedReadStep result) + { + file_.close(); + value_ = {}; + return result_ = result; + } + ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::JournalValueLocation location_; + std::array key_{}; + size_t key_size_ = 0, capacity_ = 0, frame_size_ = 0; + uint8_t table_ = 0; + bool erase_ = false; + uint8_t* bytes_ = nullptr; + uint64_t length_ = 0, offset_ = 0; + ::geocaching::ByteView value_; + storage::SdRuntimeFile file_; + SdRecordReadCursor read_cursor_; + Phase phase_ = Phase::Volume; + IndexedReadStep result_ = IndexedReadStep::Idle; +}; +static_assert(sizeof(SdIndexedValueReader) <= 320, "Indexed reads must not own a frame buffer"); +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal.h new file mode 100644 index 00000000..d6ed43da --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal.h @@ -0,0 +1,337 @@ +#pragma once +#include "geocaching/storage/record_frame.h" +#include "geocaching/storage/transaction.h" +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class JournalWriteResult : uint8_t +{ + Verified, + Unavailable, + Exists, + Invalid, + IoError, + UnsupportedVolume, + CorruptVolume, + VolumeChanged, + StateRejected, + InProgress, + Busy, + Cancelled +}; + +enum class JournalReadResult : uint8_t +{ + Parsed, + Missing, + Unavailable, + Corrupt, + IoError +}; + +// Caller owns payload storage and mutation scratch, both retained until the +// transaction is applied. Parsed validates framing/chain only; the store must +// still validate all table values and references atomically before applying. +inline JournalReadResult readJournalTransaction(uint64_t sequence, uint64_t previous_sequence, + uint8_t* buffer, size_t capacity, + ::geocaching::storage::MutationView* mutations, + size_t mutation_capacity, + ::geocaching::storage::TransactionView& out) +{ + out = {}; + if (!buffer || capacity < 24 || !mutations || previous_sequence == UINT64_MAX || + sequence != previous_sequence + 1) return JournalReadResult::Corrupt; + char path[80]{}; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/journal/%016llx.gcj", + static_cast(sequence)); + const auto read = storage::sd_read_file(path, buffer, std::min(capacity, size_t(65560))); + switch (read.status) + { + case storage::SdFileReadStatus::Missing: + return JournalReadResult::Missing; + case storage::SdFileReadStatus::Busy: + case storage::SdFileReadStatus::Unavailable: + return JournalReadResult::Unavailable; + case storage::SdFileReadStatus::Invalid: + return JournalReadResult::Corrupt; + case storage::SdFileReadStatus::IoError: + return JournalReadResult::IoError; + case storage::SdFileReadStatus::Ready: + break; + } + ::geocaching::storage::RecordFrameView frame; + if (read.file_size != read.bytes_read || read.bytes_read > capacity || + !::geocaching::storage::decodeRecordFrame({buffer, read.bytes_read}, frame) || + frame.kind != ::geocaching::storage::RecordKind::Transaction || frame.sequence != sequence || + !::geocaching::storage::decodeTransaction(frame.payload, previous_sequence, mutations, mutation_capacity, out)) + return JournalReadResult::Corrupt; + return JournalReadResult::Parsed; +} + +// One serialized owner drives begin/step. Input bytes and mutation descriptors +// remain immutable until a terminal result. No method loops over file contents. +// Each data step transfers at most 512 bytes; metadata calls have separate steps. +class SdGeocachingJournal +{ + public: + explicit SdGeocachingJournal(const ::geocaching::storage::VolumeInstance& volume) : volume_(volume) {} + SdGeocachingJournal(const SdGeocachingJournal&) = delete; + SdGeocachingJournal& operator=(const SdGeocachingJournal&) = delete; + + JournalWriteResult begin(uint64_t previous, ::geocaching::ByteView payload) + { + if (result_ == JournalWriteResult::InProgress) return JournalWriteResult::Busy; + capture_ = nullptr; + if (!::geocaching::storage::validateTransaction(payload, previous)) return JournalWriteResult::Invalid; + raw_ = payload; + streamed_ = false; + return beginPrepared(previous, payload.size); + } + + JournalWriteResult begin(uint64_t previous, const ::geocaching::storage::MutationView* mutations, size_t count) + { + if (result_ == JournalWriteResult::InProgress) return JournalWriteResult::Busy; + capture_ = nullptr; + if (!encoding_.open(previous, mutations, count)) return JournalWriteResult::Invalid; + raw_ = {}; + streamed_ = true; + return beginPrepared(previous, encoding_.size()); + } + + JournalWriteResult result() const { return result_; } + bool mayHaveWritten() const { return may_have_written_; } + // Optional caller lease populated during the existing readback pass. Its + // contents are usable only after Verified, never after a partial/failing run. + bool captureReadback(uint8_t* bytes, size_t capacity) + { + if (result_ != JournalWriteResult::InProgress || phase_ != Phase::Checksum || offset_ || !bytes || + capacity < header_.size() + payload_size_) return false; + if (streamed_) + { + if (encoding_.inputOverlaps({bytes, header_.size() + payload_size_})) return false; + } + else + { + const auto a = reinterpret_cast(raw_.data), b = reinterpret_cast(bytes); + if (a <= b ? b - a < raw_.size : a - b < header_.size() + payload_size_) return false; + } + capture_ = bytes; + return true; + } + + // Cancellation is a single boundary action; closing may itself block in the + // filesystem. A created or partially written file always requires recovery. + JournalWriteResult cancel() + { + if (result_ != JournalWriteResult::InProgress) return result_; + if (opened_) + { + file_.close(); + opened_ = false; + } + return result_ = JournalWriteResult::Cancelled; + } + + JournalWriteResult step() + { + using namespace ::geocaching::storage; + if (result_ != JournalWriteResult::InProgress) return result_; + if (phase_ == Phase::CloseFailure) + { + file_.close(); + opened_ = false; + return result_ = failure_; + } + if (phase_ != Phase::Checksum && + (!storage::sd_card_ready() || storage::sd_external_block_owner_active())) + return fail(JournalWriteResult::Unavailable); + const size_t length = std::min(buffer_.size(), payload_size_ - offset_); + switch (phase_) + { + case Phase::Checksum: + if (!fill(offset_, length)) return fail(JournalWriteResult::Invalid); + crc_ = sys::crc32(buffer_.data(), length, crc_); + offset_ += length; + if (offset_ == payload_size_) + { + finishRecordHeader(header_, crc_); + offset_ = 0; + phase_ = Phase::Directory; + } + break; + case Phase::Directory: + if (!storage::sd_is_directory("/trailmate/geocaching/.state/journal")) + return fail(JournalWriteResult::Unavailable); + phase_ = Phase::VolumeBefore; + break; + case Phase::VolumeBefore: + case Phase::VolumeAfter: + { + VolumeInstance volume; + const auto inspected = inspectSdVolume(volume); + if (inspected != SdVolumeResult::Ready) return fail(volumeFailure(inspected)); + if (volume != volume_) return fail(JournalWriteResult::VolumeChanged); + if (phase_ == Phase::VolumeAfter) return result_ = JournalWriteResult::Verified; + phase_ = Phase::Exists; + break; + } + case Phase::Exists: + if (storage::sd_exists(path_.data())) return fail(JournalWriteResult::Exists); + phase_ = Phase::OpenWrite; + break; + case Phase::OpenWrite: + may_have_written_ = true; + if (!file_.open(path_.data(), "w")) return fail(JournalWriteResult::IoError); + opened_ = true; + phase_ = Phase::WriteHeader; + break; + case Phase::WriteHeader: + if (file_.write(header_.data(), header_.size()) != header_.size()) return fail(JournalWriteResult::IoError); + phase_ = Phase::WritePayload; + break; + case Phase::WritePayload: + if (!fill(offset_, length)) return fail(JournalWriteResult::Invalid); + if (file_.write(buffer_.data(), length) != length) return fail(JournalWriteResult::IoError); + offset_ += length; + if (offset_ == payload_size_) phase_ = Phase::Flush; + break; + case Phase::Flush: + if (!file_.flush()) return fail(JournalWriteResult::IoError); + phase_ = Phase::CloseWrite; + break; + case Phase::CloseWrite: + file_.close(); + opened_ = false; + phase_ = Phase::OpenRead; + break; + case Phase::OpenRead: + if (!file_.open(path_.data(), "r")) return fail(JournalWriteResult::IoError); + opened_ = true; + phase_ = Phase::ReadSize; + break; + case Phase::ReadSize: + if (file_.size() != header_.size() + payload_size_) return fail(JournalWriteResult::IoError); + phase_ = Phase::ReadHeader; + break; + case Phase::ReadHeader: + if (file_.read(buffer_.data(), header_.size()) != static_cast(header_.size()) || + std::memcmp(buffer_.data(), header_.data(), header_.size())) return fail(JournalWriteResult::IoError); + if (capture_) std::memcpy(capture_, buffer_.data(), header_.size()); + offset_ = 0; + phase_ = Phase::ReadPayload; + break; + case Phase::ReadPayload: + if (file_.read(buffer_.data(), length) != static_cast(length) || + !matches(offset_, length)) return fail(JournalWriteResult::IoError); + if (capture_) std::memcpy(capture_ + header_.size() + offset_, buffer_.data(), length); + offset_ += length; + if (offset_ == payload_size_) phase_ = Phase::CloseRead; + break; + case Phase::CloseRead: + file_.close(); + opened_ = false; + phase_ = Phase::VolumeAfter; + break; + case Phase::CloseFailure: + break; + } + return result_; + } + + private: + enum class Phase : uint8_t + { + Checksum, + Directory, + VolumeBefore, + Exists, + OpenWrite, + WriteHeader, + WritePayload, + Flush, + CloseWrite, + OpenRead, + ReadSize, + ReadHeader, + ReadPayload, + CloseRead, + VolumeAfter, + CloseFailure + }; + + JournalWriteResult beginPrepared(uint64_t previous, size_t size) + { + if (previous == UINT64_MAX || + !::geocaching::storage::makeRecordPrefix(::geocaching::storage::RecordKind::Transaction, + previous + 1, size, header_)) + return JournalWriteResult::Invalid; + payload_size_ = size; + offset_ = 0; + may_have_written_ = false; + crc_ = sys::crc32(header_.data(), 20); + std::snprintf(path_.data(), path_.size(), "/trailmate/geocaching/.state/journal/%016llx.gcj", + static_cast(previous + 1)); + phase_ = Phase::Checksum; + return result_ = JournalWriteResult::InProgress; + } + + bool fill(size_t offset, size_t length) + { + if (!streamed_) + { + std::memcpy(buffer_.data(), raw_.data + offset, length); + return true; + } + size_t written = 0; + return encoding_.readSlice(offset, buffer_.data(), length, written) && written == length; + } + bool matches(size_t offset, size_t length) const + { + return streamed_ ? encoding_.matchesSlice(offset, {buffer_.data(), length}) + : !std::memcmp(buffer_.data(), raw_.data + offset, length); + } + JournalWriteResult fail(JournalWriteResult result) + { + if (opened_) + { + failure_ = result; + phase_ = Phase::CloseFailure; + return result_; + } + return result_ = result; + } + static JournalWriteResult volumeFailure(SdVolumeResult result) + { + switch (result) + { + case SdVolumeResult::Missing: + case SdVolumeResult::Unavailable: + return JournalWriteResult::Unavailable; + case SdVolumeResult::Unsupported: + return JournalWriteResult::UnsupportedVolume; + case SdVolumeResult::Corrupt: + return JournalWriteResult::CorruptVolume; + default: + return JournalWriteResult::IoError; + } + } + + const ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::TransactionEncoding encoding_; + ::geocaching::ByteView raw_; + uint8_t* capture_ = nullptr; + storage::SdRuntimeFile file_; + ::geocaching::storage::RecordHeader header_{}; + std::array buffer_{}; + std::array path_{}; + size_t offset_ = 0, payload_size_ = 0; + uint32_t crc_ = 0; + Phase phase_ = Phase::Checksum; + JournalWriteResult result_ = JournalWriteResult::Invalid, failure_ = JournalWriteResult::Invalid; + bool streamed_ = false, opened_ = false, may_have_written_ = false; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_inventory.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_inventory.h new file mode 100644 index 00000000..5cdabb06 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_inventory.h @@ -0,0 +1,74 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_volume.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class InventoryStep : uint8_t { Scanning, Complete, RetryLater, Corrupt, VolumeChanged }; + +struct JournalSegmentRange +{ + bool complete = false; + uint64_t first_start = 0; + uint64_t last_start = 0; +}; + +// Filenames identify segment starts, not individual transactions. Replay must +// check sequence continuity using the records inside each segment. +class SdJournalInventory +{ + public: + SdJournalInventory(const ::geocaching::storage::VolumeInstance& volume, uint64_t checkpoint) + : volume_(volume), checkpoint_(checkpoint) {} + JournalSegmentRange range() const + { + if (state_ != InventoryStep::Complete) return {}; + return {true, last_ ? (first_ ? first_ : checkpoint_ + 1) : 0, last_}; + } + + InventoryStep step() + { + if (state_ == InventoryStep::Complete || state_ == InventoryStep::Corrupt || state_ == InventoryStep::VolumeChanged) return state_; + ::geocaching::storage::VolumeInstance current; + const auto volume = inspectSdVolume(current); + if (volume == SdVolumeResult::Missing || volume == SdVolumeResult::Unavailable || volume == SdVolumeResult::IoError) + return restart(); + if (volume != SdVolumeResult::Ready) return finish(InventoryStep::Corrupt); + if (current != volume_) return finish(InventoryStep::VolumeChanged); + if (!directory_.is_open() && !directory_.open("/trailmate/geocaching/.state/journal")) return restart(); + char name[128]{}; bool is_dir = false; + const auto result = directory_.read_next_status(name, sizeof(name), &is_dir); + if (result == storage::SdDirReadStatus::Busy) return InventoryStep::RetryLater; + if (result == storage::SdDirReadStatus::IoError || result == storage::SdDirReadStatus::Unavailable) return restart(); + if (result == storage::SdDirReadStatus::End) + return finish(InventoryStep::Complete); + if (result != storage::SdDirReadStatus::Entry || is_dir || std::strlen(name) != 20 || std::strcmp(name + 16, ".gcj")) + return finish(InventoryStep::Corrupt); + uint64_t sequence = 0; + for (size_t i = 0; i < 16; ++i) + { + const char c = name[i]; + if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f'))) return finish(InventoryStep::Corrupt); + sequence = (sequence << 4) | static_cast(c <= '9' ? c - '0' : c - 'a' + 10); + } + if (sequence == 0) return finish(InventoryStep::Corrupt); + if (sequence > last_) last_ = sequence; + const auto replay_sequence = checkpoint_ == UINT64_MAX ? UINT64_MAX : checkpoint_ + 1; + if (sequence <= replay_sequence && sequence > first_) first_ = sequence; + return state_ = InventoryStep::Scanning; + } + + private: + InventoryStep finish(InventoryStep state) { directory_.close(); return state_ = state; } + InventoryStep restart() + { + directory_.close(); first_ = 0; last_ = 0; + return state_ = InventoryStep::RetryLater; + } + const ::geocaching::storage::VolumeInstance volume_; + const uint64_t checkpoint_; + uint64_t first_ = 0, last_ = 0; + storage::SdRuntimeDir directory_; + InventoryStep state_ = InventoryStep::Scanning; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_replay.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_replay.h new file mode 100644 index 00000000..4323e117 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_replay.h @@ -0,0 +1,204 @@ +#pragma once +#include "geocaching/storage/logical_state.h" +#include "geocaching/storage/transaction_index_cursor.h" +#include "platform/esp/arduino_common/geocaching/sd_journal.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_inventory.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_segment.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class ReplayStep : uint8_t +{ + Transaction, + Advancing, + JournalComplete, + RetryLater, + Corrupt, + VolumeChanged, + TailTruncated, + Applied, + ApplicationRejected +}; + +// Range comes from a completed segment inventory. Caller supplies worker buffers. +// JournalComplete does not imply GPX-install recovery or business hydration. +class SdJournalReplay +{ + public: + SdJournalReplay(const ::geocaching::storage::VolumeInstance& volume, + uint64_t checkpoint_sequence, JournalSegmentRange range, + uint8_t* bytes, size_t capacity, + ::geocaching::storage::MutationView* mutations, size_t mutation_capacity) + : volume_(volume), applied_(checkpoint_sequence), range_(range), segment_start_(range.first_start), bytes_(bytes), + capacity_(capacity), mutations_(mutations), mutation_capacity_(mutation_capacity), + corrupt_(!range.complete || range.first_start > range.last_start || + (!range.first_start && range.last_start) || !bytes || !mutations || capacity < 24) {} + + uint64_t appliedSequence() const { return applied_; } + ::geocaching::ByteView pendingFrame() const + { + return !corrupt_ && pending_.count ? ::geocaching::ByteView{bytes_, pending_frame_size_} : ::geocaching::ByteView{}; + } + // Owner may stage derived index entries while this transaction is pending. + // They must not become visible until its value/reference validation passes. + // The frame lease ends at acknowledgeApplied()/the next record read. + bool pendingIndex(::geocaching::storage::TransactionIndexCursor& cursor) const + { + cursor = {}; + return !corrupt_ && pending_.count && pending_frame_size_ && + cursor.open({bytes_, pending_frame_size_}, applied_, segment_start_, pending_frame_offset_); + } + + // Serialized owner convenience: validate/apply the complete candidate + // before acknowledging its sequence. Rejected transactions remain pending. + template + ReplayStep applyNext(::geocaching::storage::LogicalState& state, Validate validate) + { + ::geocaching::storage::TransactionView transaction; + const auto result = next(transaction); + if (result != ReplayStep::Transaction) return result; + if (!state.apply(transaction.mutations, transaction.count, validate)) return ReplayStep::ApplicationRejected; + // next() proved exactly one sequential transaction, so acknowledgment + // cannot fail under this object's single-owner contract. + if (!acknowledgeApplied(applied_ + 1)) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + return ReplayStep::Applied; + } + + ReplayStep next(::geocaching::storage::TransactionView& out) + { + out = {}; + if (corrupt_) return ReplayStep::Corrupt; + if (!volume_checked_) + { + ::geocaching::storage::VolumeInstance current; + const auto volume = inspectSdVolume(current); + if (volume == SdVolumeResult::Missing || volume == SdVolumeResult::Unavailable || volume == SdVolumeResult::IoError) + return ReplayStep::RetryLater; + if (volume != SdVolumeResult::Ready) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + if (current != volume_) return ReplayStep::VolumeChanged; + volume_checked_ = true; + return ReplayStep::Advancing; + } + volume_checked_ = false; + if (pending_.count) + { + out = pending_; + return ReplayStep::Transaction; + } + if (!range_.last_start) return ReplayStep::JournalComplete; + if (!opened_) + { + if (probed_) + { + probed_ = false; + if (!segment_.open(segment_start_)) return ReplayStep::RetryLater; + opened_ = true; + return ReplayStep::Advancing; + } + char path[80]{}; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/journal/%016llx.gcj", + static_cast(segment_start_)); + uint8_t probe[24]; + const auto status = storage::sd_read_file(path, probe, sizeof(probe)); + if (status.status == storage::SdFileReadStatus::Missing) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + if (status.status != storage::SdFileReadStatus::Ready && status.status != storage::SdFileReadStatus::Invalid) + return ReplayStep::RetryLater; + if (!status.file_size || status.file_size > 1024U * 1024U) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + probed_ = true; + return ReplayStep::Advancing; + } + ::geocaching::storage::RecordFrameView frame; + const auto result = segment_.next(bytes_, capacity_, frame); + if (result == SegmentReadResult::InProgress) return ReplayStep::Advancing; + if (result == SegmentReadResult::WorkspaceTooSmall) return ReplayStep::ApplicationRejected; + if (result == SegmentReadResult::IoError) + { + opened_ = false; + return ReplayStep::RetryLater; + } + if (result == SegmentReadResult::Truncated && segment_start_ == range_.last_start) return ReplayStep::TailTruncated; + if (result == SegmentReadResult::End) + { + if (segment_start_ == range_.last_start) return ReplayStep::JournalComplete; + if (applied_ == UINT64_MAX || applied_ + 1 > range_.last_start) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + segment_start_ = applied_ + 1; + opened_ = false; + return ReplayStep::Advancing; + } + if (result != SegmentReadResult::Record || frame.sequence == 0 || + !::geocaching::storage::decodeTransaction(frame.payload, frame.sequence - 1, mutations_, mutation_capacity_, pending_)) + { + corrupt_ = true; + return ReplayStep::Corrupt; + } + if (frame.sequence <= applied_) + { + pending_ = {}; + return ReplayStep::Advancing; + } + if (applied_ == UINT64_MAX || frame.sequence != applied_ + 1) + { + pending_ = {}; + corrupt_ = true; + return ReplayStep::Corrupt; + } + pending_frame_size_ = static_cast(24 + frame.payload.size); + if (segment_.position() < pending_frame_size_ || segment_.position() > UINT32_MAX) + { + pending_ = {}; + corrupt_ = true; + return ReplayStep::Corrupt; + } + pending_frame_offset_ = static_cast(segment_.position() - pending_frame_size_); + out = pending_; + return ReplayStep::Transaction; + } + + // Call only after validating values/references and atomically applying the + // entire pending transaction. A failed application leaves the cursor put. + bool acknowledgeApplied(uint64_t sequence) + { + if (corrupt_ || !pending_.count || applied_ == UINT64_MAX || sequence != applied_ + 1) return false; + applied_ = sequence; + pending_ = {}; + pending_frame_size_ = 0; + return true; + } + + private: + const ::geocaching::storage::VolumeInstance volume_; + uint64_t applied_; + JournalSegmentRange range_; + uint64_t segment_start_; + SdJournalSegment segment_; + bool opened_ = false; + bool volume_checked_ = false, probed_ = false; + uint32_t pending_frame_size_ = 0, pending_frame_offset_ = 0; + uint8_t* bytes_; + size_t capacity_; + ::geocaching::storage::MutationView* mutations_; + size_t mutation_capacity_; + bool corrupt_; + ::geocaching::storage::TransactionView pending_; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_segment.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_segment.h new file mode 100644 index 00000000..d52d844d --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_journal_segment.h @@ -0,0 +1,60 @@ +#pragma once +#include "geocaching/storage/record_frame.h" +#include "platform/esp/arduino_common/geocaching/sd_record_reader.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include +#include + +namespace platform::esp::arduino_common::geocaching +{ +// One GCR1 record per step, using caller-owned storage. A failed or partial +// record is terminal for this open segment; never search ahead for another magic. +class SdJournalSegment +{ + public: + uint64_t position() const { return offset_; } + bool open(uint64_t first_sequence) + { + file_.close(); + failure_ = SegmentReadResult::Record; + offset_ = 0; + cursor_ = {}; + expected_ = first_sequence; + if (!first_sequence) return false; + char path[80]{}; + std::snprintf(path, sizeof(path), "/trailmate/geocaching/.state/journal/%016llx.gcj", + static_cast(first_sequence)); + if (!file_.open(path, "r")) return false; + length_ = file_.size(); + if (length_ == 0 || length_ > 1024U * 1024U) + { + file_.close(); + return false; + } + return true; + } + + SegmentReadResult next(uint8_t* buffer, size_t capacity, ::geocaching::storage::RecordFrameView& out) + { + out = {}; + if (failure_ != SegmentReadResult::Record) return failure_; + ::geocaching::storage::RecordFrameView frame; + const auto result = readSdRecord(file_, length_, offset_, buffer, capacity, cursor_, frame); + if (result != SegmentReadResult::Record) + return result == SegmentReadResult::End || result == SegmentReadResult::InProgress ? result : fail(result); + if (frame.kind != ::geocaching::storage::RecordKind::Transaction || frame.sequence != expected_) + return fail(SegmentReadResult::Corrupt); + if (expected_ == UINT64_MAX && offset_ != length_) return fail(SegmentReadResult::Corrupt); + if (expected_ != UINT64_MAX) ++expected_; + out = frame; + return SegmentReadResult::Record; + } + + private: + SegmentReadResult fail(SegmentReadResult result) { return failure_ = result; } + storage::SdRuntimeFile file_; + SdRecordReadCursor cursor_; + uint64_t offset_ = 0, length_ = 0, expected_ = 0; + SegmentReadResult failure_ = SegmentReadResult::Record; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_publish_port.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_publish_port.h new file mode 100644 index 00000000..1d8e52da --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_publish_port.h @@ -0,0 +1,109 @@ +#pragma once +#include "geocaching/usecase/publish_attempt.h" +#include "platform/esp/arduino_common/geocaching/sd_request_store.h" + +namespace platform::esp::arduino_common::geocaching +{ +class SdPublishPort final : public ::geocaching::PublishAttemptPort +{ + public: + using Result = ::geocaching::PublishPersistence; + SdPublishPort(SdRequestStore& store, ::geocaching::protocol::RecordCrypto& crypto, + const ::geocaching::Destination& local, const ::geocaching::GeocacheId& cache, + const ::geocaching::RevisionHash& hash, const std::array& task, + const ::geocaching::storage::StoredTime& created) + : store_(store), crypto_(crypto), local_(local), cache_(cache), hash_(hash), task_(task), created_(created) {} + + Result submit(const ::geocaching::Destination& remote, const ::geocaching::RequestId& request, ::geocaching::ByteView bytes) override + { + if (phase_ != Phase::Idle) return Result::Rejected; + const ::geocaching::storage::RequestTaskTarget target{{cache_.bytes.data(), 32}, {hash_.bytes.data(), 32}, 0}; + const auto result = store_.persistNewTask(local_, remote, request, task_, 1, bytes, created_, target); + if (result != JournalWriteResult::InProgress && result != JournalWriteResult::Verified) return Result::Rejected; + remote_ = remote; + request_ = request; + phase_ = result == JournalWriteResult::InProgress ? Phase::Submitting : Phase::Waiting; + return convert(result); + } + // Called after owner validates the persisted Task/Outgoing and original ID. + bool attachRestoredRequest(const ::geocaching::Destination& remote, const ::geocaching::RequestId& request) + { + if (phase_ != Phase::Idle) return false; + remote_ = remote; + request_ = request; + phase_ = Phase::Waiting; + return true; + } + Result commitResult(const ::geocaching::Destination& remote, const ::geocaching::RequestId& request, ::geocaching::ByteView bytes) override + { + if (phase_ != Phase::Waiting || !matches(remote, request)) return Result::Rejected; + const auto result = store_.commitPublishResult(local_, remote, request, bytes, crypto_); + if (result == JournalWriteResult::InProgress) phase_ = Phase::Committing; + else if (result == JournalWriteResult::Verified) phase_ = Phase::Complete; + return convert(result); + } + Result cancel(const ::geocaching::Destination& remote, const ::geocaching::RequestId& request) override + { + if (!matches(remote, request) || (phase_ != Phase::Submitting && phase_ != Phase::Waiting)) return Result::Rejected; + if (phase_ == Phase::Submitting) + { + stop_after_submit_ = true; + return Result::Pending; + } + return stop(); + } + Result poll() override + { + if (phase_ != Phase::Submitting && phase_ != Phase::Committing && phase_ != Phase::Stopping) return Result::Rejected; + const auto result = store_.stepCommit(); + if (result == JournalWriteResult::InProgress) return Result::Pending; + if (result != JournalWriteResult::Verified) + { + phase_ = Phase::Failed; + return Result::Rejected; + } + const bool submitted = phase_ == Phase::Submitting; + phase_ = submitted ? Phase::Waiting : Phase::Complete; + if (submitted && stop_after_submit_) return stop(); + return Result::Committed; + } + + private: + enum class Phase : uint8_t + { + Idle, + Submitting, + Waiting, + Committing, + Stopping, + Complete, + Failed + }; + static Result convert(JournalWriteResult value) + { + return value == JournalWriteResult::InProgress ? Result::Pending : value == JournalWriteResult::Verified ? Result::Committed + : Result::Rejected; + } + bool matches(const ::geocaching::Destination& remote, const ::geocaching::RequestId& request) const + { + return remote.bytes == remote_.bytes && request.bytes == request_.bytes; + } + Result stop() + { + const auto result = store_.stopTask(task_); + phase_ = result == JournalWriteResult::InProgress ? Phase::Stopping : result == JournalWriteResult::Verified ? Phase::Complete + : Phase::Failed; + return convert(result); + } + SdRequestStore& store_; + ::geocaching::protocol::RecordCrypto& crypto_; + ::geocaching::Destination local_, remote_; + ::geocaching::RequestId request_; + ::geocaching::GeocacheId cache_; + ::geocaching::RevisionHash hash_; + std::array task_; + ::geocaching::storage::StoredTime created_; + Phase phase_ = Phase::Idle; + bool stop_after_submit_ = false; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_record_reader.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_record_reader.h new file mode 100644 index 00000000..cfd73b6a --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_record_reader.h @@ -0,0 +1,73 @@ +#pragma once +#include "geocaching/storage/record_frame.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class SegmentReadResult : uint8_t { Record, End, Truncated, Corrupt, IoError, WorkspaceTooSmall, InProgress }; + +struct SdRecordReadCursor +{ + uint8_t* buffer = nullptr; + uint64_t sequence = 0; + uint32_t payload_size = 0, payload_read = 0, crc = 0, expected_crc = 0; + uint8_t header_read = 0; + ::geocaching::storage::RecordKind kind = ::geocaching::storage::RecordKind::Transaction; +}; +static_assert(sizeof(SdRecordReadCursor) <= 64, "Record cursor is metadata, not a payload buffer"); + +// At most 512 file bytes per invocation, including the header. InProgress MUST +// return to the maintenance owner; callers must not spin until Record. The +// caller keeps the same output buffer across steps. Full-frame storage remains +// an existing decoder constraint, not an allocation performed by this cursor. +inline SegmentReadResult readSdRecord(storage::SdRuntimeFile& file, uint64_t length, uint64_t& offset, + uint8_t* buffer, size_t capacity, SdRecordReadCursor& cursor, + ::geocaching::storage::RecordFrameView& out) +{ + out = {}; + if (!file.is_open()) return SegmentReadResult::IoError; + if (offset > length) return SegmentReadResult::Corrupt; + if (!cursor.buffer && offset == length) return SegmentReadResult::End; + if (!buffer || capacity < 24) return SegmentReadResult::WorkspaceTooSmall; + if (cursor.buffer && cursor.buffer != buffer) return SegmentReadResult::Corrupt; + cursor.buffer = buffer; + size_t budget = 512; + if (cursor.header_read < 24) + { + const size_t needed = 24 - cursor.header_read; + if (needed > length - offset) return SegmentReadResult::Truncated; + const int n = file.read(buffer + cursor.header_read, needed); + if (n == 0) return SegmentReadResult::Truncated; + if (n < 0 || static_cast(n) > needed) return SegmentReadResult::IoError; + cursor.header_read += static_cast(n); offset += static_cast(n); budget -= static_cast(n); + if (cursor.header_read != 24) return SegmentReadResult::InProgress; + if (std::memcmp(buffer, "GCR1", 4) || buffer[4] < 2 || buffer[4] > 4 || buffer[5] || buffer[6] || buffer[7] != 24) + return SegmentReadResult::Corrupt; + for (unsigned i = 0; i < 4; ++i) + { + cursor.payload_size = (cursor.payload_size << 8) | buffer[8 + i]; + cursor.expected_crc = (cursor.expected_crc << 8) | buffer[20 + i]; + } + for (unsigned i = 0; i < 8; ++i) cursor.sequence = (cursor.sequence << 8) | buffer[12 + i]; + if (!cursor.payload_size || cursor.payload_size > 65536) return SegmentReadResult::Corrupt; + cursor.kind = static_cast<::geocaching::storage::RecordKind>(buffer[4]); + cursor.crc = ::sys::crc32(buffer, 20); + } + if (cursor.payload_size > capacity - 24) return SegmentReadResult::WorkspaceTooSmall; + const auto remaining = cursor.payload_size - cursor.payload_read; + if (remaining > length - offset) return SegmentReadResult::Truncated; + const size_t requested = std::min(budget, remaining); + auto* destination = buffer + 24 + cursor.payload_read; + const int n = file.read(destination, requested); + if (n == 0) return SegmentReadResult::Truncated; + if (n < 0 || static_cast(n) > requested) return SegmentReadResult::IoError; + cursor.crc = ::sys::crc32(destination, static_cast(n), cursor.crc); + cursor.payload_read += static_cast(n); offset += static_cast(n); + if (cursor.payload_read != cursor.payload_size) return SegmentReadResult::InProgress; + if (cursor.crc != cursor.expected_crc) return SegmentReadResult::Corrupt; + out = {cursor.kind, cursor.sequence, {buffer + 24, cursor.payload_size}}; + cursor = {}; + return SegmentReadResult::Record; +} +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_request_store.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_request_store.h new file mode 100644 index 00000000..891aa5d2 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_request_store.h @@ -0,0 +1,956 @@ +#pragma once +#include "geocaching/domain/version_policy.h" +#include "geocaching/protocol/capabilities.h" +#include "geocaching/protocol/directory_reply.h" +#include "geocaching/protocol/get_request.h" +#include "geocaching/protocol/get_response.h" +#include "geocaching/protocol/publish_request.h" +#include "geocaching/protocol/publish_response.h" +#include "geocaching/protocol/query_request.h" +#include "geocaching/protocol/query_response.h" +#include "geocaching/protocol/verify_record.h" +#include "geocaching/storage/attempt_references.h" +#include "geocaching/storage/attempt_timeout.h" +#include "geocaching/storage/author_history.h" +#include "geocaching/storage/draft_publication.h" +#include "geocaching/storage/draft_record.h" +#include "geocaching/storage/install_record.h" +#include "geocaching/storage/installable_record.h" +#include "geocaching/storage/object_ref.h" +#include "geocaching/storage/pending_request.h" +#include "geocaching/storage/queued_request.h" +#include "geocaching/storage/task_references.h" +#include "platform/esp/arduino_common/geocaching/request_dispatch_store.h" +#include "platform/esp/arduino_common/geocaching/sd_journal.h" + +namespace platform::esp::arduino_common::geocaching +{ +// Construct only after volume/checkpoint/journal recovery has established the +// committed sequence. One serialized storage worker owns this object; its +// buffers must be allocated off ESP task stacks. No network calls occur here. +class SdRequestStore : public RequestDispatchStore +{ + public: + SdRequestStore(const ::geocaching::storage::VolumeInstance& volume, uint64_t recovered_sequence, + ::geocaching::storage::LogicalState& state) + : journal_(volume), state_(state), volume_(volume), sequence_(recovered_sequence) {} + uint64_t committedSequence() const { return sequence_; } + bool needsRecovery() const { return needs_recovery_; } + + bool commitPending() const { return pending_; } + + DispatchReadResult readPending(const ::geocaching::Destination& local, ::geocaching::ByteView after, + ::geocaching::storage::PendingRequestView& out) override + { + using namespace ::geocaching::storage; + const auto result = nextPendingRequest(state_.view(), local, after, out); + return result == PendingRequestResult::Ready ? DispatchReadResult::Ready : result == PendingRequestResult::None ? DispatchReadResult::None + : DispatchReadResult::Corrupt; + } + DispatchReadResult readForSend(::geocaching::ByteView key, DispatchSendView& out) override + { + using namespace ::geocaching::storage; + out = {}; + ::geocaching::ByteView stored, task_bytes; + OutgoingView outgoing; + TaskView task; + const auto view = state_.view(); + if (!view.find(5, key, stored) || !decodeOutgoing(key, stored, outgoing) || outgoing.state != 1 || + !view.find(10, outgoing.task_id, task_bytes) || !decodeTask(outgoing.task_id, task_bytes, task) || + !requestBelongsToTask(outgoing.task_id, task, key, outgoing)) return DispatchReadResult::Corrupt; + out.request = outgoing.request; + out.stopped = !outgoing.continue_intent || !task.continue_intent || task.state == 5; + return DispatchReadResult::Ready; + } + + JournalWriteResult saveDraft(::geocaching::ByteView key, ::geocaching::ByteView encoded, uint64_t expected_generation) + { + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + using namespace ::geocaching::storage; + DraftView next, previous; + ::geocaching::ByteView existing; + if (!decodeDraft(key, encoded, next)) return JournalWriteResult::Invalid; + const bool found = state_.view().find(4, key, existing); + if (found && !decodeDraft(key, existing, previous)) return JournalWriteResult::StateRejected; + const auto check = checkDraftUpdate(found ? &previous : nullptr, next, expected_generation); + if (check == DraftUpdateCheck::InvalidGeneration) return JournalWriteResult::Invalid; + if (check == DraftUpdateCheck::Conflict || + (check == DraftUpdateCheck::NeedsRetainedPublication && !draftPublication(state_.view(), key, previous).base_retained)) + return JournalWriteResult::StateRejected; + const MutationView mutation{4, key, encoded, false}; + return commitMutations(&mutation, 1); + } + + // Confirmation binds the selected public identity before signing starts. + // The caller supplies a transient encoding lease; no payload is retained here. + JournalWriteResult bindDraftAuthor(::geocaching::ByteView key, uint64_t expected_generation, + ::geocaching::ByteView author, uint8_t* workspace, size_t capacity) + { + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + using namespace ::geocaching::storage; + ::geocaching::ByteView value; + DraftView draft; + if (!author.data || author.size != 64 || !state_.view().find(4, key, value) || + !decodeDraft(key, value, draft) || draft.generation != expected_generation) return JournalWriteResult::StateRejected; + if (draft.author.size) + return std::memcmp(draft.author.data, author.data, 64) ? JournalWriteResult::StateRejected : JournalWriteResult::Verified; + if (draft.generation == UINT64_MAX) return JournalWriteResult::StateRejected; + draft.author = author; + ++draft.generation; + size_t size = 0; + if (!encodeDraft(key, draft, workspace, capacity, size)) return JournalWriteResult::Invalid; + return saveDraft(key, {workspace, size}, expected_generation); + } + + JournalWriteResult stepCommit() + { + if (!pending_) return JournalWriteResult::Invalid; + const auto result = journal_.step(); + if (result == JournalWriteResult::InProgress) return result; + pending_ = false; + if (result == JournalWriteResult::Verified) + { + if (!state_.commitPrepared()) + { + needs_recovery_ = true; + return JournalWriteResult::StateRejected; + } + ++sequence_; + } + else + { + state_.discardPrepared(); + // Even an existing next-sequence file must be reconciled before + // another command is admitted. Never overwrite a partial commit. + needs_recovery_ = true; + } + return result; + } + + JournalWriteResult cancelCommit() + { + if (!pending_) return JournalWriteResult::Invalid; + const auto result = journal_.cancel(); + pending_ = false; + state_.discardPrepared(); + needs_recovery_ = journal_.mayHaveWritten(); + return result; + } + + // Canonical record bytes stay immutable across reservation and signing. + // The caller checks the selected local author before starting this operation. + JournalWriteResult reserveUnsignedRecord(::geocaching::ByteView encoded, + ::geocaching::protocol::RecordCrypto& crypto, + uint8_t* workspace, size_t capacity, + const ::geocaching::storage::StoredTime& time) + { + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + ::geocaching::GeocacheId id; + ::geocaching::RevisionHash hash; + const auto result = ::geocaching::protocol::deriveGeocacheHashes(encoded, crypto, workspace, capacity, id, hash); + if (result == ::geocaching::protocol::VerificationResult::CryptoUnavailable) return JournalWriteResult::Unavailable; + if (result == ::geocaching::protocol::VerificationResult::WorkspaceTooSmall) return JournalWriteResult::StateRejected; + if (result != ::geocaching::protocol::VerificationResult::Valid) return JournalWriteResult::Invalid; + ::geocaching::RecordView record; + if (!::geocaching::protocol::decodeGeocacheRecord(encoded, record)) return JournalWriteResult::Invalid; + return reserveAuthorVersion(id, record.revision, hash, record.author_public_key, time); + } + + JournalWriteResult reserveDraftUnsignedRecord(::geocaching::ByteView draft_key, uint64_t generation, + ::geocaching::ByteView encoded, ::geocaching::protocol::RecordCrypto& crypto, + uint8_t* workspace, size_t capacity, const ::geocaching::storage::StoredTime& time) + { + using namespace ::geocaching; + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + VolumeInstance volume; + if (inspectSdVolume(volume) != SdVolumeResult::Ready || volume != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + ByteView value; + DraftView draft; + RecordView record; + if (!state_.view().find(4, draft_key, value) || !decodeDraft(draft_key, value, draft) || draft.generation != generation || + !protocol::decodeGeocacheRecord(encoded, record) || draft.author.size != 64 || !draft.has_coordinates || + std::memcmp(record.author_public_key.data, draft.author.data, 64) || std::memcmp(record.creation_nonce.data, draft_key.data, 16) || + record.state != static_cast(draft.state) || record.latitude_e7 != draft.latitude_e7 || record.longitude_e7 != draft.longitude_e7 || + record.name != draft.name || record.description != draft.description || record.hint != draft.hint || + record.difficulty_x2 != draft.difficulty_x2 || record.terrain_x2 != draft.terrain_x2 || record.container_size != static_cast(draft.container_size)) + return JournalWriteResult::StateRejected; + GeocacheId id; + RevisionHash hash; + if (protocol::deriveGeocacheHashes(encoded, crypto, workspace, capacity, id, hash) != protocol::VerificationResult::Valid) + return JournalWriteResult::Invalid; + if (!time.has_utc || time.utc_seconds != record.updated_at) return JournalWriteResult::Invalid; + if (record.revision == 1 && record.created_at != record.updated_at) return JournalWriteResult::StateRejected; + size_t cursor = 0; + MutationView row; + uint32_t highest = 0; + while (state_.view().next(cursor, row)) + { + if (row.table != 3 || row.key.size != 36 || std::memcmp(row.key.data, id.bytes.data(), 32)) continue; + AuthorIssuedView issued; + if (!decodeAuthorIssued(row.key, row.value, issued)) return JournalWriteResult::StateRejected; + if (issued.revision > highest) highest = issued.revision; + } + if (record.revision != highest && (highest == UINT32_MAX || record.revision != highest + 1)) return JournalWriteResult::StateRejected; + if (record.revision > 1) + { + std::array predecessor; + std::memcpy(predecessor.data(), id.bytes.data(), 32); + for (unsigned i = 0; i < 4; ++i) predecessor[32 + i] = static_cast((record.revision - 1) >> ((3 - i) * 8)); + ByteView parent; + AuthorIssuedView issued; + if (!state_.view().find(3, {predecessor.data(), predecessor.size()}, parent) || + !decodeAuthorIssued({predecessor.data(), predecessor.size()}, parent, issued) || + std::memcmp(issued.revision_hash.data, record.previous_hash.data, 32) || !issued.issued_at.has_utc || + record.updated_at < issued.issued_at.utc_seconds) return JournalWriteResult::StateRejected; + predecessor[32] = predecessor[33] = predecessor[34] = 0; + predecessor[35] = 1; + if (!state_.view().find(3, {predecessor.data(), predecessor.size()}, parent) || + !decodeAuthorIssued({predecessor.data(), predecessor.size()}, parent, issued) || !issued.issued_at.has_utc || + record.created_at != issued.issued_at.utc_seconds) return JournalWriteResult::StateRejected; + } + std::array key; + std::memcpy(key.data(), id.bytes.data(), 32); + for (unsigned i = 0; i < 4; ++i) key[32 + i] = static_cast(record.revision >> ((3 - i) * 8)); + const bool existed = state_.view().find(3, {key.data(), key.size()}, value); + if (existed) + { + AuthorIssuedView issued; + if (!decodeAuthorIssued({key.data(), key.size()}, value, issued) || std::memcmp(issued.revision_hash.data, hash.bytes.data(), 32) || + std::memcmp(issued.author_public_key.data, draft.author.data, 64)) return JournalWriteResult::StateRejected; + if (draft.base_hash.size == 32 && !std::memcmp(draft.base_hash.data, hash.bytes.data(), 32)) return JournalWriteResult::Verified; + } + if (draft.generation == UINT64_MAX) return JournalWriteResult::StateRejected; + // Freeze exact draft fields and their issuance identity in one commit. + draft.base_hash = {hash.bytes.data(), hash.bytes.size()}; + ++draft.generation; + size_t draft_size = 0, issued_size = 0; + if (!encodeDraft(draft_key, draft, workspace, capacity, draft_size)) return JournalWriteResult::Invalid; + if (!existed && !encodeAuthorIssued(hash, record.author_public_key, time, workspace_.data(), workspace_.size(), issued_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = { + {4, draft_key, {workspace, draft_size}, false}, + {3, {key.data(), key.size()}, {workspace_.data(), issued_size}, false}}; + return commitMutations(mutations, existed ? 1 : 2); + } + + JournalWriteResult persistNewTask(const ::geocaching::Destination& local, + const ::geocaching::Destination& remote, + const ::geocaching::RequestId& request_id, + const std::array& task_id, uint8_t task_kind, + ::geocaching::ByteView request, + const ::geocaching::storage::StoredTime& time, + const ::geocaching::storage::RequestTaskTarget& target = {}) + { + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + using namespace ::geocaching::storage; + std::array key{}; + OutgoingView outgoing; + TaskView task; + if (sequence_ == UINT64_MAX || + !describeNewRequestTask(local, remote, request_id, task_id, task_kind, request, time, key, outgoing, task, target)) + return JournalWriteResult::Invalid; + const auto before = state_.view(); + ::geocaching::ByteView existing; + if (before.find(5, {key.data(), key.size()}, existing) || before.find(10, outgoing.task_id, existing)) + return JournalWriteResult::StateRejected; + size_t task_size = 0; + if (!encodeTask(outgoing.task_id, task, workspace_.data(), workspace_.size(), task_size)) return JournalWriteResult::Invalid; + if (task_kind == 2) + { + ::geocaching::protocol::GetRequestView get; + if (!::geocaching::protocol::decodeGetRequest(request, request_id, get) || get.wanted_hash.size != 32 || + std::memcmp(get.cache_id.data, target.cache_id.data, 32) || std::memcmp(get.wanted_hash.data, target.revision_hash.data, 32)) + return JournalWriteResult::Invalid; + CacheHeadView head; + if (before.find(2, target.cache_id, existing) && !decodeCacheHead(target.cache_id, existing, head)) return JournalWriteResult::StateRejected; + if (head.install_generation == UINT64_MAX || target.install_generation != head.install_generation + 1) return JournalWriteResult::StateRejected; + head.install_generation = target.install_generation; + uint8_t head_value[64]; + size_t head_size = 0; + if (!encodeCacheHead(target.cache_id, head, head_value, sizeof(head_value), head_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = { + {5, {key.data(), key.size()}, {}, false}, + {10, outgoing.task_id, {workspace_.data(), task_size}, false}, + {2, target.cache_id, {head_value, head_size}, false}}; + return commitMutations(mutations, 3, &outgoing, 0); + } + const MutationView mutations[] = { + {5, {key.data(), key.size()}, {}, false}, + {10, outgoing.task_id, {workspace_.data(), task_size}, false}}; + return commitMutations(mutations, 2, &outgoing, 0); + } + + uint64_t nextDownloadGeneration(const ::geocaching::GeocacheId& id) const + { + ::geocaching::ByteView bytes; + ::geocaching::storage::CacheHeadView head; + const ::geocaching::ByteView key{id.bytes.data(), id.bytes.size()}; + if (state_.view().find(2, key, bytes) && !::geocaching::storage::decodeCacheHead(key, bytes, head)) return 0; + return head.install_generation == UINT64_MAX ? 0 : head.install_generation + 1; + } + + bool downloadIntentActive(::geocaching::ByteView key, uint64_t generation) const + { + ::geocaching::storage::OutgoingView outgoing; + ::geocaching::storage::TaskView task; + ::geocaching::storage::CacheHeadView head; + return !needs_recovery_ && loadDownload(key, generation, outgoing, task, head) && + outgoing.continue_intent && task.continue_intent && task.state < 3; + } + + bool installedFileProof(::geocaching::ByteView request_key, uint64_t generation, + std::array& file_hash, ::geocaching::RevisionHash& revision) const + { + using namespace ::geocaching::storage; + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + if (!loadDownload(request_key, generation, outgoing, task, head) || head.current_hash.size != 32) return false; + size_t cursor = 0; + uint64_t latest = 0; + MutationView row; + const auto view = state_.view(); + while (view.next(cursor, row)) + { + if (row.table != 12) continue; + InstallRecordView installed; + if (!decodeInstallRecord(row.key, row.value, installed)) return false; + if (installed.phase == InstallPhase::Installed && installed.generation <= generation && installed.generation > latest && + !std::memcmp(installed.cache_id.data, task.cache_id.data, 32) && + !std::memcmp(installed.revision_hash.data, head.current_hash.data, 32)) + { + std::memcpy(file_hash.data(), installed.new_file_hash.data, 32); + latest = installed.generation; + } + } + if (!latest) return false; + std::memcpy(revision.bytes.data(), head.current_hash.data, 32); + return true; + } + + // Durable receipt is distinct from local installation: the task remains + // Running until GPX replacement and the final install transaction succeed. + JournalWriteResult recordDownloadResponse(const ::geocaching::Destination& local, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& id, uint64_t generation, ::geocaching::ByteView response, + ::geocaching::protocol::RecordCrypto& crypto) + { + using namespace ::geocaching::storage; + using namespace ::geocaching::protocol; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + uint8_t key[48]; + std::memcpy(key, local.bytes.data(), 16); + std::memcpy(key + 16, remote.bytes.data(), 16); + std::memcpy(key + 32, id.bytes.data(), 16); + ::geocaching::ByteView bytes; + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + const auto before = state_.view(); + if (!before.find(5, {key, 48}, bytes) || !decodeOutgoing({key, 48}, bytes, outgoing) || + !before.find(10, outgoing.task_id, bytes) || !decodeTask(outgoing.task_id, bytes, task) || + task.kind != 2 || task.cache_id.size != 32 || task.revision_hash.size != 32 || + !requestBelongsToTask(outgoing.task_id, task, {key, 48}, outgoing) || + !before.find(2, task.cache_id, bytes) || !decodeCacheHead(task.cache_id, bytes, head) || + !generation || outgoing.install_generation != generation || head.install_generation != generation) + return JournalWriteResult::StateRejected; + GetRequestView requested; + GetResponseView reply; + if (!decodeGetRequest(outgoing.request, id, requested) || + !decodeGetResponse(response, id, requested.budget, reply) || reply.has_conflict) return JournalWriteResult::Invalid; + ::geocaching::GeocacheId cache_id; + ::geocaching::RevisionHash hash; + std::memcpy(cache_id.bytes.data(), task.cache_id.data, 32); + std::memcpy(hash.bytes.data(), task.revision_hash.data, 32); + VerifiedRecordView verified; + auto result = VerificationResult::WorkspaceTooSmall; + if (!state_.withScratch([&](uint8_t* scratch, size_t capacity) + { result = verifyGeocache(reply.signed_cache, crypto, scratch, capacity, verified, &cache_id, &hash); })) return JournalWriteResult::StateRejected; + if (result == VerificationResult::CryptoUnavailable) return JournalWriteResult::Unavailable; + if (result != VerificationResult::Valid) return JournalWriteResult::Invalid; + if (!installableVersion(head, verified)) return JournalWriteResult::StateRejected; + if (outgoing.state == 4) + return outgoing.terminal_data.size == response.size && !std::memcmp(outgoing.terminal_data.data, response.data, response.size) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + if (!outgoing.continue_intent || !task.continue_intent || task.state >= 3) return JournalWriteResult::StateRejected; + outgoing.state = 4; + outgoing.terminal_data = response; + task.state = 1; + size_t task_size = 0; + if (!encodeTask(outgoing.task_id, task, workspace_.data(), workspace_.size(), task_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = {{5, {key, 48}, {}, false}, {10, outgoing.task_id, {workspace_.data(), task_size}, false}}; + return commitMutations(mutations, 2, &outgoing, 0); + } + + // Call only after the staging job has flushed and verified the exact GPX. + JournalWriteResult prepareDownloadInstall(::geocaching::ByteView request_key, + const std::array& transaction, uint64_t generation, + const std::array& new_file_hash, ::geocaching::ByteView old_file_hash, + ::geocaching::protocol::RecordCrypto& crypto) + { + using namespace ::geocaching::storage; + using namespace ::geocaching::protocol; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (old_file_hash.size && (!old_file_hash.data || old_file_hash.size != 32)) return JournalWriteResult::Invalid; + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + if (!loadDownload(request_key, generation, outgoing, task, head) || outgoing.state != 4 || + !outgoing.continue_intent || !task.continue_intent || task.state >= 3) return JournalWriteResult::StateRejected; + if (!head.current_hash.size && old_file_hash.size) return JournalWriteResult::StateRejected; + ::geocaching::RequestId id; + std::memcpy(id.bytes.data(), request_key.data + 32, 16); + GetResponseView response; + if (!decodeGetResponse(outgoing.terminal_data, id, 8192, response) || response.has_conflict) return JournalWriteResult::Invalid; + ::geocaching::GeocacheId cache; + ::geocaching::RevisionHash hash; + std::memcpy(cache.bytes.data(), task.cache_id.data, 32); + std::memcpy(hash.bytes.data(), task.revision_hash.data, 32); + VerifiedRecordView verified; + auto checked = VerificationResult::WorkspaceTooSmall; + if (!state_.withScratch([&](uint8_t* bytes, size_t capacity) + { checked = verifyGeocache(response.signed_cache, crypto, bytes, capacity, verified, &cache, &hash); })) return JournalWriteResult::StateRejected; + if (checked == VerificationResult::CryptoUnavailable) return JournalWriteResult::Unavailable; + if (checked != VerificationResult::Valid) return JournalWriteResult::Invalid; + const auto& record = verified.record; + if (!installableVersion(head, verified)) return JournalWriteResult::StateRejected; + ::geocaching::ByteView prior; + const ::geocaching::ByteView transaction_key{transaction.data(), transaction.size()}; + if (state_.view().find(12, transaction_key, prior)) + { + InstallRecordView existing; + if (!decodeInstallRecord(transaction_key, prior, existing)) return JournalWriteResult::StateRejected; + const bool same = existing.phase == InstallPhase::Prepared && existing.generation == generation && + !std::memcmp(existing.cache_id.data, cache.bytes.data(), 32) && + !std::memcmp(existing.revision_hash.data, hash.bytes.data(), 32) && + !std::memcmp(existing.new_file_hash.data, new_file_hash.data(), 32) && + existing.old_file_hash.size == old_file_hash.size && + (!old_file_hash.size || !std::memcmp(existing.old_file_hash.data, old_file_hash.data, old_file_hash.size)); + return same ? JournalWriteResult::Verified : JournalWriteResult::StateRejected; + } + ObjectRefView object; + object.cache_id = task.cache_id; + object.previous_hash = record.previous_hash; + object.revision = record.revision; + object.state = record.state; + object.created_at = record.created_at; + if (state_.view().find(1, task.revision_hash, prior)) + { + ObjectRefView existing; + if (!decodeObjectRef(task.revision_hash, prior, existing) || existing.revision != object.revision || existing.state != object.state || + existing.created_at != object.created_at || std::memcmp(existing.cache_id.data, object.cache_id.data, 32) || + existing.previous_hash.size != object.previous_hash.size || (object.previous_hash.size && std::memcmp(existing.previous_hash.data, object.previous_hash.data, object.previous_hash.size))) return JournalWriteResult::StateRejected; + object = existing; + } + InstallRecordView install{task.cache_id, task.revision_hash, {new_file_hash.data(), new_file_hash.size()}, old_file_hash, generation, InstallPhase::Prepared}; + uint8_t object_bytes[160], install_bytes[160]; + size_t object_size = 0, install_size = 0; + if (!encodeObjectRef(task.revision_hash, object, object_bytes, sizeof(object_bytes), object_size) || + !encodeInstallRecord(transaction_key, install, install_bytes, sizeof(install_bytes), install_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = {{1, task.revision_hash, {object_bytes, object_size}, false}, + {12, transaction_key, {install_bytes, install_size}, false}}; + return commitMutations(mutations, 2); + } + + // The install owner supplies the hash observed by reopening the target GPX. + // Current version, install marker and task completion become visible together. + JournalWriteResult finishDownloadInstall(::geocaching::ByteView request_key, + const std::array& transaction, uint64_t generation, + const std::array& observed_file_hash) + { + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + OutgoingView outgoing; + TaskView task; + CacheHeadView head; + if (!loadDownload(request_key, generation, outgoing, task, head) || outgoing.state != 4) return JournalWriteResult::StateRejected; + ::geocaching::ByteView value; + const ::geocaching::ByteView key{transaction.data(), transaction.size()}; + InstallRecordView install; + ObjectRefView object; + if (!state_.view().find(12, key, value) || !decodeInstallRecord(key, value, install) || install.generation != generation || + std::memcmp(install.cache_id.data, task.cache_id.data, 32) || std::memcmp(install.revision_hash.data, task.revision_hash.data, 32) || + std::memcmp(install.new_file_hash.data, observed_file_hash.data(), 32) || + !state_.view().find(1, task.revision_hash, value) || !decodeObjectRef(task.revision_hash, value, object) || + std::memcmp(object.cache_id.data, task.cache_id.data, 32)) return JournalWriteResult::StateRejected; + if (install.phase == InstallPhase::Installed) + return task.state == 3 && head.current_hash.size == 32 && !std::memcmp(head.current_hash.data, task.revision_hash.data, 32) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + if (install.phase != InstallPhase::Prepared || !outgoing.continue_intent || !task.continue_intent || task.state >= 3 || + object.revision < head.highest_seen_revision || head.conflict_state == 2) return JournalWriteResult::StateRejected; + head.current_hash = task.revision_hash; + head.highest_seen_revision = object.revision; + install.phase = InstallPhase::Installed; + task.state = 3; + uint8_t head_bytes[64], install_bytes[160]; + size_t head_size = 0, install_size = 0, task_size = 0; + if (!encodeCacheHead(task.cache_id, head, head_bytes, sizeof(head_bytes), head_size) || + !encodeInstallRecord(key, install, install_bytes, sizeof(install_bytes), install_size) || + !encodeTask(outgoing.task_id, task, workspace_.data(), workspace_.size(), task_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = {{2, task.cache_id, {head_bytes, head_size}, false}, + {12, key, {install_bytes, install_size}, false}, + {10, outgoing.task_id, {workspace_.data(), task_size}, false}}; + return commitMutations(mutations, 3); + } + + JournalWriteResult reserveAuthorVersion(const ::geocaching::GeocacheId& id, uint32_t revision, + const ::geocaching::RevisionHash& hash, ::geocaching::ByteView public_key, + const ::geocaching::storage::StoredTime& time) + { + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (!revision || !public_key.data || public_key.size != 64) return JournalWriteResult::Invalid; + ::geocaching::storage::VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + uint8_t key[36]{}; + std::memcpy(key, id.bytes.data(), 32); + for (unsigned i = 0; i < 4; ++i) key[32 + i] = static_cast(revision >> ((3 - i) * 8)); + ::geocaching::ByteView existing; + if (state_.view().find(3, {key, sizeof(key)}, existing)) + { + ::geocaching::storage::AuthorIssuedView issued; + if (!::geocaching::storage::decodeAuthorIssued({key, sizeof(key)}, existing, issued)) return JournalWriteResult::StateRejected; + return !std::memcmp(issued.revision_hash.data, hash.bytes.data(), 32) && + !std::memcmp(issued.author_public_key.data, public_key.data, 64) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + } + uint8_t value[192]{}; + size_t value_size = 0; + if (sequence_ == UINT64_MAX || !::geocaching::storage::encodeAuthorIssued(hash, public_key, time, value, sizeof(value), value_size)) + return JournalWriteResult::Invalid; + const ::geocaching::storage::MutationView mutation{3, {key, sizeof(key)}, {value, value_size}, false}; + return commitMutations(&mutation, 1); + } + + // Authenticated source/local destination are supplied by the delivery owner. + // Revalidates the stored signed publication and commits result + task together. + JournalWriteResult commitPublishResult(const ::geocaching::Destination& local, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& id, ::geocaching::ByteView response, + ::geocaching::protocol::RecordCrypto& crypto) + { + using namespace ::geocaching::storage; + using namespace ::geocaching::protocol; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (!response.data || response.size > 512) return JournalWriteResult::Invalid; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + uint8_t key[48]; + std::memcpy(key, local.bytes.data(), 16); + std::memcpy(key + 16, remote.bytes.data(), 16); + std::memcpy(key + 32, id.bytes.data(), 16); + const auto before = state_.view(); + ::geocaching::ByteView value, task_value; + OutgoingView outgoing; + TaskView task; + if (!before.find(5, {key, 48}, value) || !decodeOutgoing({key, 48}, value, outgoing) || + !before.find(10, outgoing.task_id, task_value) || !decodeTask(outgoing.task_id, task_value, task) || task.kind != 1 || + !requestBelongsToTask(outgoing.task_id, task, {key, 48}, outgoing)) return JournalWriteResult::StateRejected; + if (outgoing.state == 4) + return outgoing.terminal_data.size == response.size && !std::memcmp(outgoing.terminal_data.data, response.data, response.size) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + if (!outgoing.continue_intent || !task.continue_intent || task.state == 5) return JournalWriteResult::StateRejected; + PublishRequestView request; + if (!decodePublishRequest(outgoing.request, id, request)) return JournalWriteResult::Invalid; + VerifiedRecordView verified; + // Borrow the original canonical SignedCache subspan. Verification uses + // the existing workspace, which can be reused after verification ends. + auto verified_result = VerificationResult::WorkspaceTooSmall; + if (!state_.withScratch([&](uint8_t* scratch, size_t capacity) + { verified_result = verifyGeocache( + request.signed_cache, crypto, + scratch, capacity, verified); })) return JournalWriteResult::StateRejected; + if (verified_result == VerificationResult::CryptoUnavailable) return JournalWriteResult::Unavailable; + if (verified_result != VerificationResult::Valid || task.cache_id.size != 32 || task.revision_hash.size != 32 || + std::memcmp(task.cache_id.data, verified.id.bytes.data(), 32) || std::memcmp(task.revision_hash.data, verified.hash.bytes.data(), 32)) + return JournalWriteResult::StateRejected; + PublishDisposition disposition; + if (!decodePublishResponse(response, id, verified.id, verified.hash, verified.record.revision, verified.record.state, disposition)) + return JournalWriteResult::Invalid; + return commitRequestResult(key, outgoing, task, response); + } + + JournalWriteResult commitDirectoryCapabilities(const ::geocaching::Destination& local, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& id, ::geocaching::ByteView response) + { + using namespace ::geocaching::storage; + using namespace ::geocaching::protocol; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + DirectoryCapabilities capabilities; + if (!decodeDirectoryCapabilities(response, id, capabilities)) return JournalWriteResult::Invalid; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + uint8_t key[48]; + std::memcpy(key, local.bytes.data(), 16); + std::memcpy(key + 16, remote.bytes.data(), 16); + std::memcpy(key + 32, id.bytes.data(), 16); + ::geocaching::ByteView value, task_value; + OutgoingView outgoing; + TaskView task; + const auto before = state_.view(); + if (!before.find(5, {key, 48}, value) || !decodeOutgoing({key, 48}, value, outgoing) || + !before.find(10, outgoing.task_id, task_value) || !decodeTask(outgoing.task_id, task_value, task) || task.kind != 3 || + !requestBelongsToTask(outgoing.task_id, task, {key, 48}, outgoing)) return JournalWriteResult::StateRejected; + if (!matchesCapabilitiesRequest(outgoing.request, id, response.size)) return JournalWriteResult::Invalid; + if (outgoing.state == 4) + return outgoing.terminal_data.size == response.size && !std::memcmp(outgoing.terminal_data.data, response.data, response.size) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + if (!outgoing.continue_intent || !task.continue_intent || task.state == 5) return JournalWriteResult::StateRejected; + return commitRequestResult(key, outgoing, task, response); + } + + // Store the complete authenticated page in Outgoing before exposing it. + // SummaryCache is a rebuildable index; it is not the acceptance authority. + JournalWriteResult commitQueryResult(const ::geocaching::Destination& local, const ::geocaching::Destination& remote, + const ::geocaching::RequestId& id, ::geocaching::ByteView response) + { + using namespace ::geocaching::storage; + using namespace ::geocaching::protocol; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + uint8_t key[48]; + std::memcpy(key, local.bytes.data(), 16); + std::memcpy(key + 16, remote.bytes.data(), 16); + std::memcpy(key + 32, id.bytes.data(), 16); + const auto before = state_.view(); + ::geocaching::ByteView value, task_value; + OutgoingView outgoing; + TaskView task; + if (!before.find(5, {key, 48}, value) || !decodeOutgoing({key, 48}, value, outgoing) || + !before.find(10, outgoing.task_id, task_value) || !decodeTask(outgoing.task_id, task_value, task) || + task.kind != 3 || !requestBelongsToTask(outgoing.task_id, task, {key, 48}, outgoing)) return JournalWriteResult::StateRejected; + if (!matchesQueryReply(outgoing.request, response, id)) return JournalWriteResult::Invalid; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + if (outgoing.state == 4) + return outgoing.terminal_data.size == response.size && !std::memcmp(outgoing.terminal_data.data, response.data, response.size) + ? JournalWriteResult::Verified + : JournalWriteResult::StateRejected; + if (!outgoing.continue_intent || !task.continue_intent || task.state == 5) return JournalWriteResult::StateRejected; + return commitRequestResult(key, outgoing, task, response); + } + + // Task-level stop is authoritative for every child request. Dispatch must + // require both task and Outgoing continuation intent, including after reboot. + JournalWriteResult stopTask(const std::array& task_id) + { + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + const ::geocaching::ByteView key{task_id.data(), task_id.size()}; + ::geocaching::ByteView value; + TaskView task; + if (!state_.view().find(10, key, value) || !decodeTask(key, value, task)) return JournalWriteResult::StateRejected; + if (task.state == 5 && !task.continue_intent) return JournalWriteResult::Verified; + if (task.state == 3 || sequence_ == UINT64_MAX) return JournalWriteResult::StateRejected; + task.state = 5; + task.continue_intent = false; + size_t value_size = 0; + if (!encodeTask(key, task, workspace_.data(), workspace_.size(), value_size)) return JournalWriteResult::Invalid; + const MutationView mutation{10, key, {workspace_.data(), value_size}, false}; + return commitMutations(&mutation, 1); + } + + // Caller reacquires the committed request view and invokes transport only + // after Verified. An unresolved attempt requires explicit recovery/retry. + JournalWriteResult beginAttempt(const ::geocaching::Destination& local, ::geocaching::ByteView request_key, + const std::array& attempt_id, const ::geocaching::storage::StoredTime& time) + { + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (!request_key.data || request_key.size != 48 || sequence_ == UINT64_MAX) return JournalWriteResult::Invalid; + const auto before = state_.view(); + ::geocaching::ByteView value, task_value, existing; + PendingRequestView pending; + OutgoingView outgoing; + TaskView task; + if (!before.find(5, request_key, value) || inspectPendingRequest(before, local, request_key, value, pending) != PendingRequestResult::Ready || + !decodeOutgoing(request_key, value, outgoing) || !before.find(10, outgoing.task_id, task_value) || + !decodeTask(outgoing.task_id, task_value, task)) return JournalWriteResult::StateRejected; + uint8_t key[64]; + std::memcpy(key, request_key.data, 48); + std::memcpy(key + 48, attempt_id.data(), 16); + if (before.find(13, {key, 64}, existing)) return JournalWriteResult::StateRejected; + TxAttemptView attempt; + attempt.submitted = time; + uint8_t attempt_value[192]; + size_t attempt_size = 0, task_size = 0; + outgoing.state = 1; + if (task.state == 0) task.state = 1; + if (!encodeTxAttempt({key, 64}, attempt, attempt_value, sizeof(attempt_value), attempt_size) || + !encodeTask(outgoing.task_id, task, workspace_.data(), workspace_.size(), task_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = { + {5, request_key, {}, false}, + {10, outgoing.task_id, {workspace_.data(), task_size}, false}, + {13, {key, 64}, {attempt_value, attempt_size}, false}}; + return commitMutations(mutations, 3, &outgoing, 0); + } + + JournalWriteResult recordAttemptHash(::geocaching::ByteView attempt_key, const std::array& hash) + { + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (!attempt_key.data || attempt_key.size != 64) return JournalWriteResult::Invalid; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + ::geocaching::ByteView stored; + TxAttemptView attempt; + if (!state_.view().find(13, attempt_key, stored) || !decodeTxAttempt(attempt_key, stored, attempt)) return JournalWriteResult::StateRejected; + const auto transition = recordAttemptTransportHash(attempt, {hash.data(), hash.size()}); + if (transition == AttemptTransition::Unchanged) return JournalWriteResult::Verified; + if (transition == AttemptTransition::Rejected || sequence_ == UINT64_MAX) return JournalWriteResult::StateRejected; + uint8_t value[192]{}; + size_t value_size = 0; + if (!encodeTxAttempt(attempt_key, attempt, value, sizeof(value), value_size)) return JournalWriteResult::Invalid; + const MutationView mutation{13, attempt_key, {value, value_size}, false}; + return commitMutations(&mutation, 1); + } + + // The delivery owner maps an authenticated event to this exact attempt. + // Delivered means transport delivery, not acceptance of the public record. + JournalWriteResult finishAttempt(::geocaching::ByteView attempt_key, ::geocaching::storage::TxAttemptState terminal, + const ::geocaching::storage::StoredTime& finished) + { + using namespace ::geocaching::storage; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + if (!attempt_key.data || attempt_key.size != 64 || + (terminal != TxAttemptState::Delivered && terminal != TxAttemptState::Failed && terminal != TxAttemptState::CancelledBeforeSend)) + return JournalWriteResult::Invalid; + VolumeInstance current; + if (inspectSdVolume(current) != SdVolumeResult::Ready) + { + needs_recovery_ = true; + return JournalWriteResult::Unavailable; + } + if (current != volume_) + { + needs_recovery_ = true; + return JournalWriteResult::VolumeChanged; + } + const auto before = state_.view(); + ::geocaching::ByteView value, outgoing_value; + TxAttemptView attempt; + OutgoingView outgoing; + if (!before.find(13, attempt_key, value) || !decodeTxAttempt(attempt_key, value, attempt) || + !before.find(5, attempt.request_key, outgoing_value) || !decodeOutgoing(attempt.request_key, outgoing_value, outgoing)) + return JournalWriteResult::StateRejected; + const auto transition = finishAttemptTransport(attempt, terminal, finished); + if (transition == AttemptTransition::Unchanged) return JournalWriteResult::Verified; + if (transition == AttemptTransition::Rejected || sequence_ == UINT64_MAX) return JournalWriteResult::StateRejected; + // Do not downgrade a business result that arrived before its transport + // event. Task/Outgoing continuation flags remain unchanged by delivery. + if (outgoing.state < 4) outgoing.state = terminal == TxAttemptState::Delivered ? 2 : 3; + uint8_t attempt_value[192]{}; + size_t attempt_size = 0; + if (!encodeTxAttempt(attempt_key, attempt, attempt_value, sizeof(attempt_value), attempt_size)) + return JournalWriteResult::Invalid; + const MutationView mutations[] = { + {13, attempt_key, {attempt_value, attempt_size}, false}, + {5, attempt.request_key, {}, false}}; + return commitMutations(mutations, 2, &outgoing, 1); + } + + JournalWriteResult expireOneAttempt(const ::geocaching::storage::StoredTime& now, uint64_t recovery_started_ms, + uint64_t timeout_ms, bool& expired) + { + using namespace ::geocaching::storage; + expired = false; + if (needs_recovery_) return JournalWriteResult::Unavailable; + if (pending_) return JournalWriteResult::Busy; + size_t cursor = 0; + MutationView entry; + const auto before = state_.view(); + while (before.next(cursor, entry)) + { + if (entry.table != 13) continue; + TxAttemptView attempt; + if (!decodeTxAttempt(entry.key, entry.value, attempt)) return JournalWriteResult::StateRejected; + if (!attemptTimeoutReached(attempt, now, recovery_started_ms, timeout_ms)) continue; + const auto result = finishAttempt(entry.key, TxAttemptState::Failed, now); + expired = result == JournalWriteResult::Verified; + return result; + } + return JournalWriteResult::Verified; + } + + private: + bool installableVersion(const ::geocaching::storage::CacheHeadView& head, + const ::geocaching::protocol::VerifiedRecordView& incoming) const + { + if (incoming.record.revision < head.highest_seen_revision || head.conflict_state == 2) return false; + if (!head.current_hash.size) return ::geocaching::storage::installableRecord(head, nullptr, incoming); + ::geocaching::ByteView bytes; + ::geocaching::storage::ObjectRefView object; + if (!state_.view().find(1, head.current_hash, bytes) || !::geocaching::storage::decodeObjectRef(head.current_hash, bytes, object) || + std::memcmp(object.cache_id.data, incoming.id.bytes.data(), 32)) return false; + return ::geocaching::storage::installableRecord(head, &object, incoming); + } + + bool loadDownload(::geocaching::ByteView key, uint64_t generation, + ::geocaching::storage::OutgoingView& outgoing, ::geocaching::storage::TaskView& task, + ::geocaching::storage::CacheHeadView& head) const + { + using namespace ::geocaching::storage; + ::geocaching::ByteView value; + const auto view = state_.view(); + return generation && view.find(5, key, value) && decodeOutgoing(key, value, outgoing) && + view.find(10, outgoing.task_id, value) && decodeTask(outgoing.task_id, value, task) && task.kind == 2 && + task.cache_id.size == 32 && task.revision_hash.size == 32 && requestBelongsToTask(outgoing.task_id, task, key, outgoing) && + view.find(2, task.cache_id, value) && decodeCacheHead(task.cache_id, value, head) && + outgoing.install_generation == generation && head.install_generation == generation; + } + + JournalWriteResult commitRequestResult(const uint8_t key[48], ::geocaching::storage::OutgoingView outgoing, + ::geocaching::storage::TaskView task, ::geocaching::ByteView response) + { + using namespace ::geocaching::storage; + const auto before = state_.view(); + outgoing.state = 4; + outgoing.terminal_data = response; + bool all_confirmed = true; + for (size_t i = 0; i < task.request_count; ++i) + { + if (!std::memcmp(task.requests[i].data, key, 48)) continue; + ::geocaching::ByteView other_value; + OutgoingView other; + if (!before.find(5, task.requests[i], other_value) || !decodeOutgoing(task.requests[i], other_value, other)) return JournalWriteResult::StateRejected; + all_confirmed = all_confirmed && other.state == 4; + } + task.state = all_confirmed ? 3 : 2; + size_t task_size = 0; + if (!encodeTask(outgoing.task_id, task, workspace_.data(), workspace_.size(), task_size)) return JournalWriteResult::Invalid; + const MutationView mutations[] = { + {5, {key, 48}, {}, false}, + {10, outgoing.task_id, {workspace_.data(), task_size}, false}}; + return commitMutations(mutations, 2, &outgoing, 0); + } + + JournalWriteResult commitMutations(const ::geocaching::storage::MutationView* mutations, size_t count, + const ::geocaching::storage::OutgoingView* outgoing = nullptr, + size_t outgoing_index = 0) + { + using namespace ::geocaching::storage; + if (pending_) return JournalWriteResult::Busy; + TransactionEncoding encoding; + if (sequence_ == UINT64_MAX || count > pending_mutations_.size() || + !encoding.open(sequence_, mutations, count)) return JournalWriteResult::Invalid; + if (outgoing && (outgoing_index >= count || mutations[outgoing_index].table != 5 || + mutations[outgoing_index].erase)) return JournalWriteResult::Invalid; + const auto before = state_.view(); + if (!state_.prepareGenerated( + mutations, count, outgoing ? outgoing_index : count, + [&](uint8_t* output, size_t capacity, size_t& written) + { + return outgoing && encodeOutgoing(mutations[outgoing_index].key, *outgoing, output, capacity, written); + }, + [&](const auto& candidate) + { return validateTaskReferences(candidate) && validateAuthorHistory(before, candidate) && + validateAttemptReferences(candidate); })) return JournalWriteResult::StateRejected; + const auto candidate = state_.preparedView(); + for (size_t i = 0; i < count; ++i) + { + const auto& source = mutations[i]; + std::memcpy(pending_keys_[i].data(), source.key.data, source.key.size); + pending_mutations_[i] = {source.table, {pending_keys_[i].data(), source.key.size}, {}, source.erase}; + if (!source.erase && !candidate.find(source.table, pending_mutations_[i].key, pending_mutations_[i].value)) + { + state_.discardPrepared(); + return JournalWriteResult::StateRejected; + } + } + const auto result = journal_.begin(sequence_, pending_mutations_.data(), count); + pending_ = result == JournalWriteResult::InProgress; + if (!pending_) state_.discardPrepared(); + return result; + } + + private: + // Current commands atomically touch at most three records. Values borrow + // the frozen candidate arena; only small keys/descriptors survive steps. + std::array<::geocaching::storage::MutationView, 3> pending_mutations_{}; + std::array, 3> pending_keys_{}; + bool pending_ = false; + SdGeocachingJournal journal_; + ::geocaching::storage::LogicalState& state_; + const ::geocaching::storage::VolumeInstance volume_; + uint64_t sequence_ = 0; + bool needs_recovery_ = false; + std::array workspace_{}; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_state_recovery.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_state_recovery.h new file mode 100644 index 00000000..5699aece --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_state_recovery.h @@ -0,0 +1,113 @@ +#pragma once +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_selection.h" +#include "platform/esp/arduino_common/geocaching/sd_checkpoint_loader.h" +#include "platform/esp/arduino_common/geocaching/sd_journal_replay.h" +#include + +namespace platform::esp::arduino_common::geocaching +{ +enum class StateRecoveryStep : uint8_t { Working, JournalRestored, RetryRequired, Corrupt, VolumeChanged, StateRejected, TailNeedsRepair }; + +// The supplied state is an isolated recovery state, never the UI's live state. +// A fresh object is required after RetryRequired. One step advances one existing +// component; the storage owner supplies operation budgeting and cancellation. +// JournalRestored is followed by GPX/install validation before business readiness. +template +class SdStateRecovery +{ + public: + SdStateRecovery(const ::geocaching::storage::VolumeInstance& volume, + ::geocaching::storage::LogicalState& recovery_state) + : volume_(volume), state_(recovery_state), selection_(volume) {} + uint64_t replayedSequence() const { return sequence_; } + + template + StateRecoveryStep step(uint8_t* bytes, size_t capacity, + ::geocaching::storage::MutationView* entries, size_t entry_capacity, Validate validate) + { + if (result_ != StateRecoveryStep::Working) return result_; + if (phase_ == Phase::Select) + { + switch (selection_.step(bytes, capacity, entries, entry_capacity)) + { + case CheckpointSelectionStep::Reading: return result_; + case CheckpointSelectionStep::RetryLater: return result_ = StateRecoveryStep::RetryRequired; + case CheckpointSelectionStep::Corrupt: return result_ = StateRecoveryStep::Corrupt; + case CheckpointSelectionStep::VolumeChanged: return result_ = StateRecoveryStep::VolumeChanged; + case CheckpointSelectionStep::NoCheckpoint: + if (!state_.beginSnapshot() || !state_.commitSnapshot(validate)) + { state_.discardSnapshot(); return result_ = StateRecoveryStep::StateRejected; } + inventory_.emplace(volume_, 0); + phase_ = Phase::Inventory; + return result_; + case CheckpointSelectionStep::Selected: break; + } + const bool slot_b = selection_.choice() == ::geocaching::storage::CheckpointChoice::SlotB; + const auto& selected = selection_.candidate(slot_b); + sequence_ = selected.sequence; + loader_.emplace(load_digest_, state_, volume_, slot_b ? 'b' : 'a', selected); + phase_ = Phase::Load; + return result_; + } + if (phase_ == Phase::Load) + { + switch (loader_->step(bytes, capacity, entries, entry_capacity, validate)) + { + case CheckpointLoadStep::Loading: return result_; + case CheckpointLoadStep::IoError: return result_ = StateRecoveryStep::RetryRequired; + case CheckpointLoadStep::Invalid: return result_ = StateRecoveryStep::Corrupt; + case CheckpointLoadStep::StateRejected: return result_ = StateRecoveryStep::StateRejected; + case CheckpointLoadStep::VolumeChanged: return result_ = StateRecoveryStep::VolumeChanged; + case CheckpointLoadStep::Applied: break; + } + loader_.reset(); + inventory_.emplace(volume_, sequence_); + phase_ = Phase::Inventory; + return result_; + } + if (phase_ == Phase::Inventory) + { + switch (inventory_->step()) + { + case InventoryStep::Scanning: + case InventoryStep::RetryLater: return result_; + case InventoryStep::Corrupt: return result_ = StateRecoveryStep::Corrupt; + case InventoryStep::VolumeChanged: return result_ = StateRecoveryStep::VolumeChanged; + case InventoryStep::Complete: break; + } + replay_.emplace(volume_, sequence_, inventory_->range(), bytes, capacity, entries, entry_capacity); + inventory_.reset(); + phase_ = Phase::Replay; + return result_; + } + switch (replay_->applyNext(state_, validate)) + { + case ReplayStep::Applied: + sequence_ = replay_->appliedSequence(); + return result_; + case ReplayStep::Advancing: return result_; + case ReplayStep::JournalComplete: return result_ = StateRecoveryStep::JournalRestored; + case ReplayStep::RetryLater: return result_ = StateRecoveryStep::RetryRequired; + case ReplayStep::VolumeChanged: return result_ = StateRecoveryStep::VolumeChanged; + case ReplayStep::ApplicationRejected: return result_ = StateRecoveryStep::StateRejected; + case ReplayStep::TailTruncated: return result_ = StateRecoveryStep::TailNeedsRepair; + case ReplayStep::Transaction: + case ReplayStep::Corrupt: return result_ = StateRecoveryStep::Corrupt; + } + return result_ = StateRecoveryStep::Corrupt; + } + + private: + enum class Phase : uint8_t { Select, Load, Inventory, Replay }; + const ::geocaching::storage::VolumeInstance volume_; + ::geocaching::storage::LogicalState& state_; + SdCheckpointSelection selection_; + Digest load_digest_; + std::optional> loader_; + std::optional inventory_; + std::optional replay_; + uint64_t sequence_ = 0; + Phase phase_ = Phase::Select; + StateRecoveryStep result_ = StateRecoveryStep::Working; +}; +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_volume.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_volume.h new file mode 100644 index 00000000..b3e67d65 --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/geocaching/sd_volume.h @@ -0,0 +1,71 @@ +#pragma once +#include "geocaching/storage/volume_format.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" + +namespace platform::esp::arduino_common::geocaching +{ +enum class SdVolumeResult : uint8_t { Ready, Missing, Unavailable, Unsupported, Corrupt, IoError }; + +inline SdVolumeResult inspectSdVolume(::geocaching::storage::VolumeInstance& instance) +{ + instance = {}; + ::geocaching::storage::VolumeHeader header; + const auto result = storage::sd_read_file("/trailmate/geocaching/.state/format.bin", header.data(), header.size()); + switch (result.status) + { + case storage::SdFileReadStatus::Missing: return SdVolumeResult::Missing; + case storage::SdFileReadStatus::Busy: + case storage::SdFileReadStatus::Unavailable: return SdVolumeResult::Unavailable; + case storage::SdFileReadStatus::IoError: return SdVolumeResult::IoError; + case storage::SdFileReadStatus::Invalid: return SdVolumeResult::Corrupt; + case storage::SdFileReadStatus::Ready: break; + } + if (result.bytes_read != result.file_size || result.bytes_read > header.size()) return SdVolumeResult::Corrupt; + switch (::geocaching::storage::decodeVolumeHeader({header.data(), result.bytes_read}, instance)) + { + case ::geocaching::storage::VolumeFormatResult::Supported: return SdVolumeResult::Ready; + case ::geocaching::storage::VolumeFormatResult::Unsupported: return SdVolumeResult::Unsupported; + case ::geocaching::storage::VolumeFormatResult::Corrupt: return SdVolumeResult::Corrupt; + } + return SdVolumeResult::Corrupt; +} +// Storage-worker operation, before any request is admitted. The caller supplies +// a freshly generated CSPRNG volume ID. An existing .state directory without a +// valid header is recovery evidence, including interrupted initialization; it +// must never be silently replaced by another empty task ledger. +inline SdVolumeResult createNewSdVolume(const ::geocaching::storage::VolumeInstance& instance, + ::geocaching::storage::VolumeInstance& confirmed_instance) +{ + confirmed_instance = {}; + if (!storage::sd_card_ready() || storage::sd_external_block_owner_active()) return SdVolumeResult::Unavailable; + ::geocaching::storage::VolumeInstance existing; + const auto inspection = inspectSdVolume(existing); + if (inspection != SdVolumeResult::Missing) + { + if (inspection == SdVolumeResult::Ready) confirmed_instance = existing; + return inspection; + } + if (storage::sd_exists("/trailmate/geocaching/.state")) return SdVolumeResult::Corrupt; + const char* directories[] = { + "/trailmate", "/trailmate/geocaching", "/trailmate/geocaching/caches", + "/trailmate/geocaching/imports", "/trailmate/geocaching/exports", + "/trailmate/geocaching/.state", "/trailmate/geocaching/.state/history", + "/trailmate/geocaching/.state/journal", "/trailmate/geocaching/.state/checkpoint", + "/trailmate/geocaching/.state/staging"}; + for (const auto* directory : directories) + { + if (!storage::sd_is_directory(directory) && !storage::sd_mkdir(directory)) return SdVolumeResult::IoError; + } + const auto header = ::geocaching::storage::encodeVolumeHeader(instance); + storage::SdRuntimeFile file; + if (!file.open("/trailmate/geocaching/.state/format.bin", "w")) return SdVolumeResult::IoError; + const bool written = file.write(header.data(), header.size()) == header.size() && file.flush(); + file.close(); + if (!written) return SdVolumeResult::IoError; + const auto verified = inspectSdVolume(existing); + if (verified != SdVolumeResult::Ready) return verified; + if (existing != instance) return SdVolumeResult::Corrupt; + confirmed_instance = existing; + return SdVolumeResult::Ready; +} +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/gps/sd_gpx_output.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/gps/sd_gpx_output.h new file mode 100644 index 00000000..d241c7ee --- /dev/null +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/gps/sd_gpx_output.h @@ -0,0 +1,42 @@ +#pragma once +#include "gps/gpx/text_writer.h" +#include "platform/esp/arduino_common/storage/sd_card_runtime.h" + +namespace platform::esp::arduino_common::gps +{ +// Caller owns the opened staging file and the storage-maintenance lifecycle. +// Wrap with BufferedOutputSink, finish it, then flushFile before validation. +class SdGpxOutput final : public ::gps::gpx::OutputSink +{ + public: + explicit SdGpxOutput(storage::SdRuntimeFile& file) : file_(file) {} + bool good() const { return good_; } + bool write(std::string_view bytes) override + { + if (!good_ || !file_.is_open()) + { + good_ = false; + return false; + } + if (file_.write(bytes.data(), bytes.size()) != bytes.size()) + { + good_ = false; + return false; + } + return true; + } + bool flushFile() + { + if (!good_ || !file_.is_open() || !file_.flush()) + { + good_ = false; + return false; + } + return true; + } + + private: + storage::SdRuntimeFile& file_; + bool good_ = true; +}; +} // namespace platform::esp::arduino_common::gps diff --git a/platform/esp/arduino_common/include/platform/esp/arduino_common/storage/sd_card_runtime.h b/platform/esp/arduino_common/include/platform/esp/arduino_common/storage/sd_card_runtime.h index f2886e04..60738839 100644 --- a/platform/esp/arduino_common/include/platform/esp/arduino_common/storage/sd_card_runtime.h +++ b/platform/esp/arduino_common/include/platform/esp/arduino_common/storage/sd_card_runtime.h @@ -148,6 +148,8 @@ class SdRuntimeFile Impl* impl_; }; +enum class SdDirReadStatus : uint8_t { Entry, End, Busy, Unavailable, Invalid, IoError }; + class SdRuntimeDir { public: @@ -161,6 +163,7 @@ class SdRuntimeDir void close(); bool is_open() const; bool read_next(char* name, std::size_t name_size, bool* is_dir); + SdDirReadStatus read_next_status(char* name, std::size_t name_size, bool* is_dir); private: class Impl; diff --git a/platform/esp/arduino_common/library.json b/platform/esp/arduino_common/library.json index b75eef0c..78fe4e3a 100644 --- a/platform/esp/arduino_common/library.json +++ b/platform/esp/arduino_common/library.json @@ -38,6 +38,7 @@ ] }, "dependencies": [ + { "name": "core_geocaching" }, { "name": "esp_common" }, diff --git a/platform/esp/arduino_common/src/app_context_platform_bindings.cpp b/platform/esp/arduino_common/src/app_context_platform_bindings.cpp index 22442401..fc6c1691 100644 --- a/platform/esp/arduino_common/src/app_context_platform_bindings.cpp +++ b/platform/esp/arduino_common/src/app_context_platform_bindings.cpp @@ -19,6 +19,7 @@ #include "platform/esp/arduino_common/gps/track_recorder.h" #include "platform/esp/arduino_common/storage/sd_card_runtime.h" #include "platform/esp/arduino_common/storage/storage_runtime.h" +#include "platform/esp/arduino_common/geocaching/browse_runtime.h" #include "platform/esp/arduino_common/team/crypto/team_crypto.h" #include "platform/esp/arduino_common/team/event/team_app_data_event_bus_bridge.h" #include "platform/esp/arduino_common/team/event/team_event_bus_sink.h" @@ -277,6 +278,7 @@ app::ChatServicesBundle create_chat_services(const app::AppConfig& config, } } + bundle.service = std::unique_ptr( new chat::ChatService(*bundle.model, *bundle.mesh_runtime, *bundle.store, config.mesh_protocol)); if (!bundle.service) @@ -295,6 +297,9 @@ app::ChatServicesBundle create_chat_services(const app::AppConfig& config, { bundle.start_deferred_storage = start_deferred_storage; } + if (lora_board) + ::platform::esp::arduino_common::geocaching::browse_runtime::configure( + *static_cast(bundle.mesh_runtime.get()), *lora_board); return bundle; } diff --git a/platform/esp/arduino_common/src/chat/infra/lxmf/lxmf_adapter.cpp b/platform/esp/arduino_common/src/chat/infra/lxmf/lxmf_adapter.cpp index c5ff58c7..12be0cdb 100644 --- a/platform/esp/arduino_common/src/chat/infra/lxmf/lxmf_adapter.cpp +++ b/platform/esp/arduino_common/src/chat/infra/lxmf/lxmf_adapter.cpp @@ -4,6 +4,7 @@ */ #include "platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h" +#include "geocaching/protocol/sign_record.h" #include "platform/esp/arduino_common/voice/vmp_pager_session.h" @@ -1069,9 +1070,11 @@ bool computeLinkIdFromLinkRequest(const uint8_t* raw_packet, size_t raw_len, } // namespace LxmfAdapter::LxmfAdapter(LoraBoard& board, - IMeshPeerDirectory* peer_directory) - : interfaces_(board), - peer_directory_service_(peer_directory) + IMeshPeerDirectory* peer_directory, + bool owns_integrated_radio) + : interfaces_(board, owns_integrated_radio), + peer_directory_service_(peer_directory), + geocaching_only_(!owns_integrated_radio) { uint8_t seed[sizeof(next_app_packet_id_)] = {}; fillRandomBytes(seed, sizeof(seed)); @@ -1144,7 +1147,7 @@ bool LxmfAdapter::dispatchLxmfPayload(PeerInfo& peer, bool track_user_message, OutboundLxmfDispatch* out_dispatch) { - if (!packed_payload || packed_payload_len == 0 || !out_dispatch) + if (!packed_payload || packed_payload_len == 0 || packed_payload_len > 8448 || !out_dispatch) { return false; } @@ -1155,7 +1158,7 @@ bool LxmfAdapter::dispatchLxmfPayload(PeerInfo& peer, (void)sendPathRequest(peer); } - runtime::RuntimeByteBuffer signed_part(kSignedPartMaxLen, 0); + runtime::RuntimeByteBuffer signed_part(packed_payload_len + 64, 0); size_t signed_part_len = signed_part.size(); if (!buildSignedPart(peer.destination_hash, identity_.destinationHash(), @@ -1170,7 +1173,7 @@ bool LxmfAdapter::dispatchLxmfPayload(PeerInfo& peer, } uint8_t signature[reticulum::kSignatureSize] = {}; - runtime::RuntimeByteBuffer lxmf_message(kMaxLxmfMessageLen, 0); + runtime::RuntimeByteBuffer lxmf_message(packed_payload_len + 96, 0); size_t lxmf_message_len = lxmf_message.size(); if (!identity_.sign(signed_part.data(), signed_part_len, signature) || !packMessage(peer.destination_hash, @@ -1189,7 +1192,7 @@ bool LxmfAdapter::dispatchLxmfPayload(PeerInfo& peer, LinkSession* active_link = findActiveLinkSessionByDestination(peer.destination_hash, LocalDestinationKind::Delivery); - const bool use_opportunistic = !active_link && peerHasUsableRatchet(peer); + const bool use_opportunistic = !active_link && packed_payload_len <= 256 && peerHasUsableRatchet(peer); const auto& propagation_config = rtnet::active().propagation; bool propagation_peer_available = false; if (propagation_config.enabled && @@ -2105,6 +2108,49 @@ bool LxmfAdapter::respondToSidebandTelemetryRequest( return sent; } +bool LxmfAdapter::getGeocachingAuthorKey(uint8_t out[64]) const +{ + if (!out) return false; + std::memset(out, 0, 64); + if (!identity_.isReady()) return false; + identity_.combinedPublicKey(out); + return true; +} + +bool LxmfAdapter::signGeocachingRecord(ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written) +{ + return ::geocaching::protocol::signGeocacheRecord({record.data, record.size}, identity_, + workspace, workspace_capacity, output, output_capacity, written); +} + +MeshSendResult LxmfAdapter::sendCustomDataToDestination(const uint8_t destination_hash[16], + const char* custom_type, ByteSpan data, + bool response, std::array* accepted_lxmf_hash) +{ + if (accepted_lxmf_hash) accepted_lxmf_hash->fill(0); + if (!destination_hash || !custom_type || !custom_type[0] || strlen(custom_type) > 96 || + !data.data || data.size == 0 || data.size > 8192) + return MeshSendResult::fail(MeshOperationFailure::InvalidInput); + if (!isReady()) return MeshSendResult::fail(MeshOperationFailure::NotReady); + PeerInfo* peer = findOrLoadPeerByDestinationHash(destination_hash); + if (!peer) return MeshSendResult::fail(MeshOperationFailure::PeerKeyMissing); + runtime::RuntimeByteBuffer payload(data.size + 256, 0); + size_t size = payload.size(); + if (!encodeCustomDataPayload(static_cast(currentTimestampSeconds()), + "Trail Mate Geocache v1", + response ? "Geocache response" : "Geocache request", + custom_type, data, payload.data(), &size)) + return MeshSendResult::fail(MeshOperationFailure::EncodeFailed); + OutboundLxmfDispatch dispatch{}; + const bool ok = dispatchLxmfPayload(*peer, payload.data(), size, false, &dispatch); + if (ok && accepted_lxmf_hash) std::memcpy(accepted_lxmf_hash->data(), dispatch.message_hash, accepted_lxmf_hash->size()); + MeshSendResult result = ok ? MeshSendResult::success(dispatch.message_id) + : MeshSendResult::fail(dispatch.failure, dispatch.message_id); + result.reticulum_identity = runtime::reticulumIdentityForPeer(*peer); + return result; +} + MeshSendResult LxmfAdapter::sendTextDetailed(ChannelId channel, const std::string& text, MessageId forced_msg_id, @@ -4029,6 +4075,30 @@ bool LxmfAdapter::handleAnnouncePacket(const uint8_t* raw_packet, size_t raw_len return false; } + if (geocaching_announcement_handler_ && !ingest.local_destination && ingest.announce.name_hash && + ingest.announce.public_key && ingest.announce.app_data && ingest.announce.app_data_len <= 96) + { + uint8_t name_hash[reticulum::kNameHashSize] = {}; + reticulum::computeNameHash("trailmate", "geocache.directory", name_hash); + if (hashesEqual(name_hash, ingest.announce.name_hash, sizeof(name_hash))) + { + // The verified service announcement authenticates the same identity + // used by lxmf.delivery. Retain its encryption key without adding a + // service announcement to the user's contact presentation. + rememberPeerIdentity(ingest.announce.public_key, nullptr, false); + uint8_t delivery_hash[reticulum::kTruncatedHashSize] = {}; + reticulum::computeNameHash("lxmf", "delivery", name_hash); + reticulum::computeDestinationHash(name_hash, ingest.identity_hash, delivery_hash); + const GeocachingAnnouncementView view{ + {packet.destination_hash, reticulum::kTruncatedHashSize}, + {delivery_hash, sizeof(delivery_hash)}, + {ingest.announce.public_key, reticulum::kCombinedPublicKeySize}, + {ingest.announce.app_data, ingest.announce.app_data_len}}; + // Borrowed only for this callback; consumers copy bounded metadata. + geocaching_announcement_handler_(view, geocaching_announcement_context_); + } + } + PathEntry& path = *ingest.path; link_manager_.forEachSession( [this, &packet, &path](LinkSession& session) @@ -9299,7 +9369,7 @@ void LxmfAdapter::cullLinkSessions() LxmfAdapter::PeerInfo* LxmfAdapter::rememberPeerIdentity( const uint8_t combined_pub[reticulum::kCombinedPublicKeySize], - const char* display_name) + const char* display_name, bool publish_contact) { if (!combined_pub) { @@ -9325,6 +9395,10 @@ LxmfAdapter::PeerInfo* LxmfAdapter::rememberPeerIdentity( { copyCString(peer.display_name, sizeof(peer.display_name), display_name); } + if (!publish_contact) + { + return &peer; + } const bool allow_persistence = screen_runtime::is_sleeping() && !screen_runtime::is_saver_active(); if (allow_persistence) @@ -9602,6 +9676,11 @@ bool LxmfAdapter::acceptVerifiedEnvelopeForDestination( source_hash); } + if (geocaching_only_ && delivery.kind != runtime::LxmfDeliveryKind::Text) + { + release_packet_for_retry(); + return false; + } if (delivery.kind == runtime::LxmfDeliveryKind::AppData) { if (delivery.app_data.incoming.portnum == @@ -9664,6 +9743,40 @@ bool LxmfAdapter::acceptVerifiedEnvelopeForDestination( return false; } + ByteSpan custom_type, custom_data; + const auto custom_result = extractCustomData(delivery.text.payload, &custom_type, &custom_data); + if (custom_result == CustomDataResult::Invalid) + { + release_packet_for_retry(); + return false; + } + constexpr char geocaching_type[] = "trailmate.geocache"; + if (custom_result == CustomDataResult::Valid && custom_type.size == sizeof(geocaching_type) - 1 && + std::memcmp(custom_type.data, geocaching_type, custom_type.size) == 0) + { + if (delivery_context.source_unverified || destination_is_group || !geocaching_handler_) + { + release_packet_for_retry(); + return false; + } + const CustomDeliveryView view{ + {envelope.source_hash, sizeof(envelope.source_hash)}, + {expected_destination_hash, reticulum::kTruncatedHashSize}, + {delivery_context.message_hash, sizeof(delivery_context.message_hash)}, + custom_data}; + if (!geocaching_handler_(view, geocaching_handler_context_)) + { + release_packet_for_retry(); + return false; + } + return true; + } + + if (geocaching_only_) + { + release_packet_for_retry(); + return false; + } if (!delivery_context.source_unverified) { SidebandTelemetryLocation location{}; diff --git a/platform/esp/arduino_common/src/chat/infra/mesh_adapter_router.cpp b/platform/esp/arduino_common/src/chat/infra/mesh_adapter_router.cpp index 22093ceb..ab3e0139 100644 --- a/platform/esp/arduino_common/src/chat/infra/mesh_adapter_router.cpp +++ b/platform/esp/arduino_common/src/chat/infra/mesh_adapter_router.cpp @@ -8,6 +8,7 @@ #include "platform/esp/arduino_common/chat/infra/meshcore/meshcore_adapter.h" #include "platform/esp/arduino_common/chat/infra/meshtastic/mt_adapter.h" #include "platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h" +#include "platform/ui/reticulum_call_runtime.h" namespace chat { @@ -47,7 +48,9 @@ MeshAdapterRouter::~MeshAdapterRouter() bool MeshAdapterRouter::installBackend(MeshProtocol protocol, std::unique_ptr backend) { LockGuard lock(mutex_); - return lock.locked() && core_.installBackend(protocol, std::move(backend)); + if (!lock.locked()) return false; + applyGeocachingHandlers(protocol, backend.get()); + return core_.installBackend(protocol, std::move(backend)); } bool MeshAdapterRouter::hasBackend() const @@ -56,6 +59,126 @@ bool MeshAdapterRouter::hasBackend() const return lock.locked() && core_.hasBackend(); } +bool MeshAdapterRouter::installServiceBackend(MeshProtocol protocol, std::unique_ptr backend) +{ + LockGuard lock(mutex_); + if (!lock.locked()) return false; + applyGeocachingHandlers(protocol, backend.get()); + return core_.installServiceBackend(protocol, std::move(backend)); +} + +void MeshAdapterRouter::applyGeocachingHandlers(MeshProtocol protocol, IMeshAdapter* backend) +{ + if (!backend || (protocol != MeshProtocol::Reticulum && protocol != MeshProtocol::RNode)) return; + auto* service = backend->geocachingTransport(); + if (!service) return; + service->setGeocachingAnnouncementHandler(geocaching_announcement_, geocaching_context_); + service->setGeocachingDeliveryHandler(geocaching_delivery_, geocaching_context_); +} + +bool MeshAdapterRouter::bindGeocachingHandlers( + void (*announcement)(const lxmf::GeocachingAnnouncementView&, void*), + bool (*delivery)(const lxmf::CustomDeliveryView&, void*), void* context) +{ + LockGuard lock(mutex_); + if (!lock.locked()) return false; + geocaching_announcement_ = announcement; + geocaching_delivery_ = delivery; + geocaching_context_ = context; + applyGeocachingHandlers(MeshProtocol::Reticulum, core_.backendForProtocol(MeshProtocol::Reticulum)); + return true; +} + +bool MeshAdapterRouter::processServiceQueue(MeshProtocol protocol) +{ + LockGuard lock(mutex_, 0); + return lock.locked() && core_.processServiceQueue(protocol); +} + +std::unique_ptr MeshAdapterRouter::takeServiceBackend(MeshProtocol protocol, const IMeshAdapter* expected) +{ + LockGuard lock(mutex_, 0); + return lock.locked() ? core_.takeServiceBackend(protocol, expected) : nullptr; +} + +std::unique_ptr MeshAdapterRouter::takeInactiveReticulumCache() +{ + LockGuard lock(mutex_, 0); + if (!lock.locked() || core_.isServiceBackend(MeshProtocol::Reticulum)) return {}; + return core_.takeInactiveBackend(MeshProtocol::Reticulum); +} + +MeshSendResult MeshAdapterRouter::sendGeocachingData(const uint8_t destination_hash[16], + lxmf::ByteSpan data, bool response, std::array* accepted_lxmf_hash, + const uint8_t expected_source[16]) +{ + if (accepted_lxmf_hash) accepted_lxmf_hash->fill(0); + LockGuard lock(mutex_, 0); + if (!lock.locked()) return MeshSendResult::fail(MeshOperationFailure::Busy); + auto* service = geocachingTransportLocked(); + if (!service) return MeshSendResult::fail(MeshOperationFailure::NotReady); + if (expected_source) + { + uint8_t current[16]{}; + if (!geocachingDestinationLocked(current) || std::memcmp(current, expected_source, 16)) + return MeshSendResult::fail(MeshOperationFailure::LocalIdentityMissing); + } + return service->sendGeocachingData(destination_hash, data, response, accepted_lxmf_hash); +} + +bool MeshAdapterRouter::geocachingDestinationLocked(uint8_t out[16]) +{ + auto* service = geocachingTransportLocked(); + uint8_t public_key[64]{}, identity[16]{}, name[reticulum::kNameHashSize]{}; + if (!service || !service->getGeocachingAuthorKey(public_key)) return false; + reticulum::computeIdentityHash(public_key, identity); + reticulum::computeNameHash("lxmf", "delivery", name); + reticulum::computeDestinationHash(name, identity, out); + return true; +} + +bool MeshAdapterRouter::getGeocachingDispatchDestination(uint8_t out[16]) +{ + if (!out) return false; + std::memset(out, 0, 16); + LockGuard lock(mutex_, 0); + if (!lock.locked() || !geocachingTransportLocked()) return false; + auto* backend = core_.backendForProtocol(MeshProtocol::Reticulum); + return backend && backend->isReady() && geocachingDestinationLocked(out); +} + +IGeocachingTransport* MeshAdapterRouter::geocachingTransportLocked() +{ + const auto active = core_.backendProtocol(); + if (active != MeshProtocol::Reticulum && active != MeshProtocol::RNode && + !core_.isServiceBackend(MeshProtocol::Reticulum)) + { + return nullptr; + } + auto* backend = core_.backendForProtocol(MeshProtocol::Reticulum); + return backend ? backend->geocachingTransport() : nullptr; +} + +bool MeshAdapterRouter::getGeocachingAuthorKey(uint8_t out[64]) +{ + if (!out) return false; + std::memset(out, 0, 64); + LockGuard lock(mutex_, 0); + if (!lock.locked()) return false; + auto* service = geocachingTransportLocked(); + return service && service->getGeocachingAuthorKey(out); +} + +bool MeshAdapterRouter::signGeocachingRecord(lxmf::ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written) +{ + written = 0; + LockGuard lock(mutex_, 0); + if (!lock.locked()) return false; + auto* service = geocachingTransportLocked(); + return service && service->signGeocachingRecord(record, workspace, workspace_capacity, output, output_capacity, written); +} + MeshProtocol MeshAdapterRouter::backendProtocol() const { LockGuard lock(mutex_); @@ -394,6 +517,12 @@ void MeshAdapterRouter::processSendQueue() if (lock.locked()) { core_.processSendQueue(); + if (!::platform::ui::reticulum_call::realtime_mode_active()) + { + // Only an explicitly installed service is eligible. The core also + // skips the active Reticulum/RNode slot to prevent double polling. + core_.processServiceQueue(MeshProtocol::Reticulum); + } } } diff --git a/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_adapter.cpp b/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_adapter.cpp index b64645aa..d11e73e4 100644 --- a/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_adapter.cpp +++ b/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_adapter.cpp @@ -99,9 +99,15 @@ bool cancelIncomingResource( } // namespace ReticulumAdapter::ReticulumAdapter(LoraBoard& board, - IMeshPeerDirectory* peer_directory) - : service_(new lxmf::LxmfAdapter(board, peer_directory)) + IMeshPeerDirectory* peer_directory, + ReticulumUsage usage) + : service_(new lxmf::LxmfAdapter(board, peer_directory, usage == ReticulumUsage::ActiveChat)), + usage_(usage) { + if (usage_ != ReticulumUsage::ActiveChat) + { + return; + } rtpage::bind_request_start_handler(startNomadPageRequest, service_.get()); rtpage::bind_request_cancel_handler(cancelNomadPageRequest, service_.get()); ::platform::ui::reticulum_receive::bind_cancel_handler( @@ -110,6 +116,10 @@ ReticulumAdapter::ReticulumAdapter(LoraBoard& board, ReticulumAdapter::~ReticulumAdapter() { + if (usage_ != ReticulumUsage::ActiveChat) + { + return; + } rtpage::bind_request_start_handler(nullptr, nullptr); rtpage::bind_request_cancel_handler(nullptr, nullptr); ::platform::ui::reticulum_receive::bind_cancel_handler(nullptr, nullptr); @@ -120,6 +130,35 @@ MeshCapabilities ReticulumAdapter::getCapabilities() const return service_->getCapabilities(); } +void ReticulumAdapter::setGeocachingAnnouncementHandler( + void (*handler)(const lxmf::GeocachingAnnouncementView&, void*), void* context) +{ + service_->setGeocachingAnnouncementHandler(handler, context); +} + +bool ReticulumAdapter::getGeocachingAuthorKey(uint8_t out[64]) +{ + return service_->getGeocachingAuthorKey(out); +} + +bool ReticulumAdapter::signGeocachingRecord(lxmf::ByteSpan record, uint8_t* workspace, size_t workspace_capacity, + uint8_t* output, size_t output_capacity, size_t& written) +{ + return service_->signGeocachingRecord(record, workspace, workspace_capacity, output, output_capacity, written); +} + +MeshSendResult ReticulumAdapter::sendGeocachingData(const uint8_t destination_hash[16], + lxmf::ByteSpan data, bool response, std::array* accepted_lxmf_hash) +{ + return service_->sendCustomDataToDestination(destination_hash, "trailmate.geocache", data, response, accepted_lxmf_hash); +} + +void ReticulumAdapter::setGeocachingDeliveryHandler( + bool (*handler)(const lxmf::CustomDeliveryView&, void*), void* context) +{ + service_->setGeocachingDeliveryHandler(handler, context); +} + bool ReticulumAdapter::sendText(ChannelId channel, const std::string& text, MessageId* out_msg_id, NodeId peer) { diff --git a/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_interfaces.cpp b/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_interfaces.cpp index 22a7d505..dd54933d 100644 --- a/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_interfaces.cpp +++ b/platform/esp/arduino_common/src/chat/infra/reticulum/reticulum_interfaces.cpp @@ -1600,8 +1600,8 @@ AutoReticulumInterface::Peer* AutoReticulumInterface::findPeer( return nullptr; } -ReticulumInterfaceSet::ReticulumInterfaceSet(LoraBoard& board) - : lora_(board) +ReticulumInterfaceSet::ReticulumInterfaceSet(LoraBoard& board, bool owns_integrated_radio) + : lora_(board), owns_integrated_radio_(owns_integrated_radio) { } @@ -1612,6 +1612,16 @@ void ReticulumInterfaceSet::applyConfig( config_ = config; network_config_ = network_config; + // Service-local effective policy only: never change persisted user settings. + // Reapply on every config refresh so a radio interface in the shared network + // configuration cannot seize the active chat protocol's hardware. + if (!owns_integrated_radio_) + { + config_.reticulum_lora_enabled = false; + config_.reticulum_wifi_gateway_enabled = true; + config_.reticulum_interface_policy = ReticulumInterfacePolicy::WifiGatewayOnly; + } + bool lora_configured = false; const reticulum::NetworkInterfaceConfig* auto_config = nullptr; tcp_count_ = 0; @@ -1633,7 +1643,7 @@ void ReticulumInterfaceSet::applyConfig( switch (interface_config.type) { case reticulum::NetworkInterfaceType::IntegratedLoRa: - lora_configured = true; + lora_configured = owns_integrated_radio_; break; case reticulum::NetworkInterfaceType::Auto: if (!auto_config) @@ -1987,6 +1997,10 @@ float ReticulumInterfaceSet::lastRxSnr() const bool ReticulumInterfaceSet::loraAllowed() const { + if (!owns_integrated_radio_) + { + return false; + } if (::platform::ui::reticulum_call::resource_preempt_active()) { return false; @@ -2060,6 +2074,11 @@ bool ReticulumInterfaceSet::hasConfiguredIpInterface() const void ReticulumInterfaceSet::syncSharedLoRaRxGate() { + // Background IP services must not suppress Meshtastic/MeshCore reception. + if (!owns_integrated_radio_) + { + return; + } const bool suppress = !loraSelectedForRuntime(); #if defined(ARDUINO) if (shared_lora_rx_suppressed_ == suppress && diff --git a/platform/esp/arduino_common/src/chat/infra/store/protocol_chat_codec.cpp b/platform/esp/arduino_common/src/chat/infra/store/protocol_chat_codec.cpp index 901526cf..13ae6e3d 100644 --- a/platform/esp/arduino_common/src/chat/infra/store/protocol_chat_codec.cpp +++ b/platform/esp/arduino_common/src/chat/infra/store/protocol_chat_codec.cpp @@ -1,4 +1,5 @@ #include "platform/esp/arduino_common/chat/infra/store/protocol_chat_codec.h" +#include "sys/crc32.h" #include #include @@ -371,18 +372,7 @@ bool decodeReadPrefix(MeshProtocol protocol, uint32_t crc32(const void* data, std::size_t len) { - const auto* bytes = static_cast(data); - uint32_t crc = 0xFFFFFFFFU; - for (std::size_t index = 0; index < len; ++index) - { - crc ^= bytes[index]; - for (uint8_t bit = 0; bit < 8; ++bit) - { - crc = (crc & 1U) != 0U ? (crc >> 1U) ^ 0xEDB88320U - : crc >> 1U; - } - } - return ~crc; + return ::sys::crc32(data, len); } MeshProtocol canonicalProtocol(MeshProtocol protocol) diff --git a/platform/esp/arduino_common/src/geocaching/browse_runtime.cpp b/platform/esp/arduino_common/src/geocaching/browse_runtime.cpp new file mode 100644 index 00000000..bb01aaec --- /dev/null +++ b/platform/esp/arduino_common/src/geocaching/browse_runtime.cpp @@ -0,0 +1,1602 @@ +#include "platform/esp/arduino_common/geocaching/browse_runtime.h" +#include "app/app_context.h" +#include "geocaching/protocol/record_encoder.h" +#include "geocaching/storage/draft_publication.h" +#include "geocaching/storage/record_shape.h" +#include "platform/esp/arduino_common/chat/infra/lxmf/lxmf_adapter.h" +#include "platform/esp/arduino_common/chat/infra/reticulum/reticulum_adapter.h" +#include "platform/esp/arduino_common/geocaching/author_issue_port.h" +#include "platform/esp/arduino_common/geocaching/query_browse_source.h" +#include "platform/esp/arduino_common/geocaching/query_store_port.h" +#include "platform/esp/arduino_common/geocaching/request_dispatcher.h" +#include "platform/esp/arduino_common/geocaching/saved_cache_catalog.h" +#include "platform/esp/arduino_common/geocaching/sd_download_port.h" +#include "platform/esp/arduino_common/geocaching/sd_publish_port.h" +#include "platform/esp/arduino_common/geocaching/sd_state_recovery.h" +#include "platform/esp/common/geocaching_crypto.h" +#include "platform/esp/common/memory_budget.h" +#include "platform/esp/common/meshcore_runtime_compat.h" +#include "ui/screens/geocaching/geocaching_page_shell.h" +#include +#include +#include +#include +#include +#include +#include + +namespace platform::esp::arduino_common::geocaching::browse_runtime +{ +namespace +{ +namespace gc = ::geocaching; +namespace mem = ::platform::esp::common::memory; +using Digest = ::platform::esp::common::meshcore_runtime::Sha256Digest; +constexpr size_t kStateCapacity = 4096, kFrameCapacity = 4096; +constexpr gc::protocol::QueryRegion kWorld{-900000000, -1800000000, 900000000, 1800000000}; +enum class Phase : uint8_t +{ + Inspect, + CheckNew, + Directories, + OpenFormat, + WriteFormat, + FlushFormat, + CloseFormat, + VerifyFormat, + Recover, + ResumeDownloads, + Connect, + Ready, + Failed +}; +struct Announcement +{ + gc::Destination discovery, delivery; + std::array key{}; + std::array data{}; + size_t size = 0; +}; +struct Session +{ + struct Publication + { + enum class DraftStage : uint8_t + { + None, + Bind, + Binding, + Encode, + Sign + }; + DraftStage draft_stage = DraftStage::None; + std::array draft_id{}; + std::array author{}; + uint64_t draft_generation = 0; + uint32_t expected_revision = 0; + gc::storage::StoredTime issued; + uint8_t* unsigned_bytes = nullptr; + size_t unsigned_size = 0; + std::unique_ptr author_port; + std::unique_ptr issue; + const char* error = nullptr; + uint8_t* bytes = nullptr; + size_t size = 0; + gc::Destination remote; + std::unique_ptr port; + std::unique_ptr attempt; + uint64_t started = 0; + uint32_t wait_ms = gc::QueryClient::kReplyTimeoutMs; + ~Publication() + { + heap_caps_free(bytes); + heap_caps_free(unsigned_bytes); + } + }; + std::unique_ptr publication; + struct DraftSave + { + std::array id{}; + uint8_t* bytes = nullptr; + size_t size = 0; + uint64_t expected = 0; + bool started = false, done = false, saved = false; + ~DraftSave() { heap_caps_free(bytes); } + }; + std::unique_ptr draft_save; + struct DraftRead + { + std::array id{}; + uint64_t generation = 0; + ::ui::geocaching::DraftReadStatus status = ::ui::geocaching::DraftReadStatus::Pending; + }; + std::unique_ptr draft_read; + Phase phase = Phase::Inspect; + const char* status = "Opening geocaching storage..."; + const char* notice = nullptr; + gc::storage::VolumeInstance volume{}; + storage::SdRuntimeFile format; + size_t directory = 0; + uint64_t connect_since = 0; + bool mkdir_pending = false; + uint8_t *first = nullptr, *second = nullptr, *frame = nullptr, *response = nullptr; + size_t response_size = 0; + gc::Destination response_source; + gc::Destination local; + gc::RequestId response_id; + uint8_t response_operation = 0; + std::unique_ptr announcement; + std::array mutations{}; + std::unique_ptr state; + std::unique_ptr> recovery; + std::unique_ptr store; + std::unique_ptr port; + std::unique_ptr client; + std::unique_ptr dispatcher; + std::unique_ptr source; + std::unique_ptr> saved; + ::platform::esp::common::EspGeocachingCrypto crypto; + std::unique_ptr> download_port; + std::unique_ptr download; + uint64_t download_started = 0; + uint32_t download_wait_ms = gc::QueryClient::kReplyTimeoutMs; + size_t download_scratch = 0; + chat::IMeshAdapter* created_backend = nullptr; + ~Session() + { + if (store && store->commitPending()) store->cancelCommit(); + source.reset(); + saved.reset(); + download.reset(); + download_port.reset(); + publication.reset(); + dispatcher.reset(); + client.reset(); + port.reset(); + store.reset(); + recovery.reset(); + state.reset(); + heap_caps_free(response); + heap_caps_free(frame); + heap_caps_free(first); + heap_caps_free(second); + } +}; +chat::MeshAdapterRouter* router = nullptr; +LoraBoard* board = nullptr; +SemaphoreHandle_t mutex = nullptr; +std::unique_ptr session; +std::array boot{}; +std::atomic wanted{false}, active{false}, restart{false}; +std::atomic cancel_draft_read{false}; +std::atomic next_step{0}; +uint64_t epoch = 0; +bool downloadActive() +{ + if (!session || !session->download) return false; + const auto phase = session->download->phase(); + return phase == gc::DownloadPhase::Submitting || phase == gc::DownloadPhase::Waiting || + phase == gc::DownloadPhase::Installing || phase == gc::DownloadPhase::Cancelling; +} +bool publicationActive() +{ + if (!session || !session->publication) return false; + const auto& job = *session->publication; + if (job.draft_stage != Session::Publication::DraftStage::None) return true; + if (job.bytes) return true; + if (!job.attempt) return false; + const auto phase = job.attempt->phase(); + return phase == gc::PublishAttemptPhase::Submitting || phase == gc::PublishAttemptPhase::Waiting || + phase == gc::PublishAttemptPhase::Committing || phase == gc::PublishAttemptPhase::Cancelling; +} +bool draftSaveActive() { return session && session->draft_save && !session->draft_save->done; } +struct Guard +{ + bool locked = mutex && xSemaphoreTake(mutex, 0) == pdTRUE; + ~Guard() + { + if (locked) xSemaphoreGive(mutex); + } +}; +gc::storage::StoredTime now(void*) +{ + gc::storage::StoredTime time; + time.boot_id = boot; + time.monotonic_ms = static_cast(esp_timer_get_time()) / 1000; + return time; +} +bool randomId(void*, uint8_t out[16]) +{ + esp_fill_random(out, 16); + return true; +} +void fail(const char* reason) +{ + session->publication.reset(); + session->download.reset(); + session->download_port.reset(); + session->phase = Phase::Failed; + session->status = reason; + ++epoch; +} + +bool resumeWaitingDownload(Session& s) +{ + const auto view = s.state->view(); + size_t cursor = 0; + gc::storage::MutationView row; + while (view.next(cursor, row)) + { + if (row.table != 5 || row.key.size != 48 || std::memcmp(row.key.data, s.local.bytes.data(), 16)) continue; + gc::storage::OutgoingView outgoing; + gc::storage::TaskView task; + gc::ByteView value; + if (!gc::storage::decodeOutgoing(row.key, row.value, outgoing) || outgoing.state >= 4 || + !outgoing.continue_intent || !view.find(10, outgoing.task_id, value) || + !gc::storage::decodeTask(outgoing.task_id, value, task) || task.kind != 2 || + !task.continue_intent || task.state == 3 || task.state == 5 || + !s.store->downloadIntentActive(row.key, outgoing.install_generation)) continue; + gc::protocol::SummaryView summary; + size_t scan = 0; + gc::storage::MutationView query_row; + bool found = false; + while (!found && view.next(scan, query_row)) + { + if (query_row.table != 5 || query_row.key.size != 48 || std::memcmp(query_row.key.data, row.key.data, 32)) continue; + gc::storage::OutgoingView query; + gc::protocol::QueryPageView page; + gc::RequestId query_id; + std::memcpy(query_id.bytes.data(), query_row.key.data + 32, 16); + if (!gc::storage::decodeOutgoing(query_row.key, query_row.value, query) || query.state != 4 || + !gc::protocol::decodeQueryPage(query.terminal_data, query_id, 8192, 64, page)) continue; + gc::protocol::CmpReader items(page.encoded_items); + for (size_t i = 0; i < page.count; ++i) + { + if (!gc::protocol::decodeSummary(items, summary)) break; + if (task.cache_id.size == 32 && task.revision_hash.size == 32 && + !std::memcmp(summary.id.bytes.data(), task.cache_id.data, 32) && + !std::memcmp(summary.hash.bytes.data(), task.revision_hash.data, 32)) + { + found = true; + break; + } + } + } + if (!found) + { + fail("Download preview missing - recovery needs attention"); + return true; + } + gc::Destination remote; + gc::RequestId request; + std::array task_id; + std::memcpy(remote.bytes.data(), row.key.data + 16, 16); + std::memcpy(request.bytes.data(), row.key.data + 32, 16); + std::memcpy(task_id.data(), outgoing.task_id.data, 16); + s.download_port.reset(new (std::nothrow) SdDownloadPort(*s.store, *s.state, s.crypto, s.local, + {summary.id, summary.hash, outgoing.install_generation}, task_id, outgoing.created)); + if (s.download_port) s.download.reset(new (std::nothrow) gc::DownloadClient(*s.download_port, s.crypto)); + if (!s.download || s.download_port->resumeWaiting(remote, request) != gc::DownloadOperationResult::Complete || + !s.download->resume(remote, request, summary, outgoing.install_generation)) + { + fail("Cannot resume download request"); + return true; + } + s.download_started = now(nullptr).monotonic_ms; + // Allow the dispatcher to expire an attempt from the previous boot, + // retry it, and then give the response its normal timeout window. + s.download_wait_ms = 2 * gc::QueryClient::kReplyTimeoutMs + 5000; + s.download_scratch = summary.signed_bytes + 64; + ++epoch; + return true; + } + return false; +} + +void announcementReceived(const chat::lxmf::GeocachingAnnouncementView& message, void*) +{ + Guard guard; + if (!guard.locked || !session || !wanted.load() || message.discovery_destination.size != 16 || + message.delivery_destination.size != 16 || message.public_key.size != 64 || message.app_data.size > 128) return; + if (!session->announcement) session->announcement.reset(new (std::nothrow) Announcement); + if (!session->announcement) return; + auto& out = *session->announcement; + std::memcpy(out.discovery.bytes.data(), message.discovery_destination.data, 16); + std::memcpy(out.delivery.bytes.data(), message.delivery_destination.data, 16); + std::memcpy(out.key.data(), message.public_key.data, 64); + std::memcpy(out.data.data(), message.app_data.data, message.app_data.size); + out.size = message.app_data.size; + next_step.store(0); +} +bool responseReceived(const chat::lxmf::CustomDeliveryView& message, void*) +{ + Guard guard; + if (!guard.locked || !session || !session->port || (!wanted.load() && !downloadActive() && !publicationActive()) || message.source.size != 16 || + message.destination.size != 16 || !message.data.data || message.data.size > 8192) return false; + if (std::memcmp(message.destination.data, session->local.bytes.data(), 16)) return false; + gc::Destination source; + std::memcpy(source.bytes.data(), message.source.data, 16); + gc::protocol::CmpReader reader({message.data.data, message.data.size}); + size_t fields = 0; + uint64_t value = 0; + gc::ByteView id; + if (!reader.array(fields, 6) || fields != 6 || !reader.unsignedInteger(value) || value != 1 || + !reader.unsignedInteger(value) || value != 1 || !reader.unsignedInteger(value) || value > 3 || + !reader.binary(id, 16) || id.size != 16) return false; + gc::RequestId request; + std::memcpy(request.bytes.data(), id.data, 16); + if (session->port->accepted(source, request, {message.data.data, message.data.size})) return true; + if (value == 1 && (!session->publication || !session->publication->attempt || + session->publication->attempt->phase() != gc::PublishAttemptPhase::Waiting || message.data.size > 512)) return false; + if (value == 3 && (!session->download || session->download->phase() != gc::DownloadPhase::Waiting || + message.data.size > session->download_scratch)) return false; + if (value != 3 && message.data.size > 2048) return false; + if (session->response) return false; + auto* bytes = static_cast(mem::allocatePreferred("geocaching.rx", message.data.size, false)); + if (!bytes) return false; + std::memcpy(bytes, message.data.data, message.data.size); + session->response = bytes; + session->response_size = message.data.size; + session->response_source = source; + session->response_id = request; + session->response_operation = static_cast(value); + next_step.store(0); + // The sender can retry; only the exact committed response is acknowledged. + return false; +} + +bool recoveredStateValid(const gc::storage::LogicalState::View& view) +{ + size_t cursor = 0; + gc::storage::MutationView row; + while (view.next(cursor, row)) + if (!gc::storage::validStoredRowShape(row)) return false; + return session && session->state && gc::storage::validateAuthorHistory(session->state->view(), view) && + gc::storage::validateTaskReferences(view) && gc::storage::validateAttemptReferences(view); +} +void startRecovery() +{ + auto& s = *session; + if (!mem::admit("geocaching", 4096, 0, kStateCapacity * 2 + kFrameCapacity, 40 * 1024, 0)) + { + fail("Insufficient memory for browsing"); + return; + } + s.first = static_cast(mem::allocatePreferred("geocaching.state", kStateCapacity, false)); + s.second = static_cast(mem::allocatePreferred("geocaching.candidate", kStateCapacity, false)); + s.frame = static_cast(mem::allocatePreferred("geocaching.recovery", kFrameCapacity, false)); + if (!s.first || !s.second || !s.frame) + { + fail("Insufficient storage workspace"); + return; + } + s.state.reset(new (std::nothrow) gc::storage::LogicalState(s.first, s.second, kStateCapacity)); + if (!s.state) + { + fail("Insufficient memory"); + return; + } + s.recovery.reset(new (std::nothrow) SdStateRecovery(s.volume, *s.state)); + if (!s.recovery) + { + fail("Insufficient memory"); + return; + } + s.phase = Phase::Recover; + s.status = "Restoring geocaching tasks..."; + ++epoch; +} + +enum class PublicationRestore : uint8_t +{ + None, + Restored, + Invalid +}; +PublicationRestore restorePublication(Session& s, const gc::GeocacheId* cache = nullptr, + const gc::RevisionHash* hash = nullptr, const gc::Destination* remote = nullptr) +{ + const auto view = s.state->view(); + size_t cursor = 0; + gc::storage::MutationView row, selected; + bool found = false; + while (view.next(cursor, row)) + { + if (row.table != 5 || row.key.size != 48 || std::memcmp(row.key.data, s.local.bytes.data(), 16) || + (remote && std::memcmp(row.key.data + 16, remote->bytes.data(), 16))) continue; + gc::storage::OutgoingView outgoing; + gc::storage::TaskView task; + gc::ByteView value; + if (!gc::storage::decodeOutgoing(row.key, row.value, outgoing) || !view.find(10, outgoing.task_id, value) || + !gc::storage::decodeTask(outgoing.task_id, value, task) || task.kind != 1) continue; + if (!gc::storage::requestBelongsToTask(outgoing.task_id, task, row.key, outgoing) || task.cache_id.size != 32 || task.revision_hash.size != 32) + return PublicationRestore::Invalid; + if ((cache && std::memcmp(task.cache_id.data, cache->bytes.data(), 32)) || + (hash && std::memcmp(task.revision_hash.data, hash->bytes.data(), 32))) continue; + if (outgoing.state > 4 || (outgoing.state == 4 && !cache) || + (outgoing.state < 4 && (!outgoing.continue_intent || !task.continue_intent || task.state == 3 || task.state == 5))) continue; + if (!found || outgoing.state == 4) + { + selected = row; + found = true; + } + if (outgoing.state == 4) break; + } + if (!found) return PublicationRestore::None; + gc::storage::OutgoingView outgoing; + gc::storage::TaskView task; + gc::ByteView value; + if (!gc::storage::decodeOutgoing(selected.key, selected.value, outgoing) || !view.find(10, outgoing.task_id, value) || + !gc::storage::decodeTask(outgoing.task_id, value, task)) return PublicationRestore::Invalid; + gc::GeocacheId id; + gc::RevisionHash revision; + gc::RequestId request; + std::array task_id; + std::memcpy(id.bytes.data(), task.cache_id.data, 32); + std::memcpy(revision.bytes.data(), task.revision_hash.data, 32); + std::memcpy(request.bytes.data(), selected.key.data + 32, 16); + std::memcpy(task_id.data(), outgoing.task_id.data, 16); + auto job = std::unique_ptr(new (std::nothrow) Session::Publication); + if (!job) return PublicationRestore::Invalid; + std::memcpy(job->remote.bytes.data(), selected.key.data + 16, 16); + job->port.reset(new (std::nothrow) SdPublishPort(*s.store, s.crypto, s.local, id, revision, task_id, outgoing.created)); + if (job->port) job->attempt.reset(new (std::nothrow) gc::PublishAttempt(*job->port, s.crypto)); + if (!job->attempt || !job->port->attachRestoredRequest(job->remote, request)) return PublicationRestore::Invalid; + bool restored = false; + s.state->withScratch([&](uint8_t* scratch, size_t capacity) + { restored = job->attempt->resume(job->remote, request, outgoing.request, scratch, capacity, id, revision, + outgoing.state == 4 ? outgoing.terminal_data : gc::ByteView{}); }); + if (!restored) return PublicationRestore::Invalid; + job->started = now(nullptr).monotonic_ms; + job->wait_ms = 2 * gc::QueryClient::kReplyTimeoutMs + 5000; + s.publication = std::move(job); + s.draft_save.reset(); + ++epoch; + return PublicationRestore::Restored; +} + +void advanceDraftPublication(Session& s) +{ + auto& job = *s.publication; + using Stage = Session::Publication::DraftStage; + const auto stop = [&](const char* reason) + { + job.error = reason; + job.draft_stage = Stage::None; + job.issue.reset(); + job.author_port.reset(); + heap_caps_free(job.bytes); + job.bytes = nullptr; + heap_caps_free(job.unsigned_bytes); + job.unsigned_bytes = nullptr; + ++epoch; + }; + if (job.draft_stage == Stage::Binding) + { + const auto result = s.store->stepCommit(); + if (result == JournalWriteResult::InProgress) return; + if (result != JournalWriteResult::Verified) + { + stop("Author binding could not be saved"); + return; + } + job.draft_stage = Stage::Encode; + return; + } + if (job.draft_stage == Stage::Sign) + { + job.issue->advance(); + if (job.issue->phase() == gc::AuthorIssuePhase::Failed) + { + stop("Version reservation or signing failed"); + return; + } + if (job.issue->phase() != gc::AuthorIssuePhase::Signed) return; + job.size = job.issue->signedRecord().size; + job.issue.reset(); + job.author_port.reset(); + heap_caps_free(job.unsigned_bytes); + job.unsigned_bytes = nullptr; + job.draft_stage = Stage::None; + ++epoch; + return; + } + gc::ByteView value; + gc::storage::DraftView draft; + const gc::ByteView key{job.draft_id.data(), job.draft_id.size()}; + if (!s.state->view().find(4, key, value) || !gc::storage::decodeDraft(key, value, draft) || draft.generation != job.draft_generation) + { + stop("Draft changed; review before publishing"); + return; + } + if (job.draft_stage == Stage::Bind) + { + const bool unbound = !draft.author.size; + auto* scratch = static_cast(mem::allocatePreferred("geocaching.author", value.size + 80, false)); + if (!scratch) + { + stop("Insufficient author workspace"); + return; + } + const auto result = s.store->bindDraftAuthor(key, job.draft_generation, {job.author.data(), job.author.size()}, scratch, value.size + 80); + heap_caps_free(scratch); + if (result != JournalWriteResult::InProgress && result != JournalWriteResult::Verified) + { + stop("Author binding rejected"); + return; + } + if (unbound) ++job.draft_generation; + job.draft_stage = result == JournalWriteResult::InProgress ? Stage::Binding : Stage::Encode; + return; + } + if (draft.author.size != 64 || std::memcmp(draft.author.data, job.author.data(), 64) || !draft.has_coordinates) + { + stop("Draft is not ready for first publication"); + return; + } + gc::RecordView record; + record.author_public_key = {job.author.data(), job.author.size()}; + // draft_id is already a durable CSPRNG nonce. Reuse it for this cache's + // creation nonce so reopening or retrying never creates a different ID. + record.creation_nonce = key; + record.revision = 1; + record.state = static_cast(draft.state); + record.latitude_e7 = draft.latitude_e7; + record.longitude_e7 = draft.longitude_e7; + record.name = draft.name; + record.description = draft.description; + record.hint = draft.hint; + record.difficulty_x2 = draft.difficulty_x2; + record.terrain_x2 = draft.terrain_x2; + record.container_size = static_cast(draft.container_size); + record.created_at = record.updated_at = job.issued.utc_seconds; + const size_t capacity = draft.name.size() + draft.description.size() + draft.hint.size() + 200; + job.unsigned_bytes = static_cast(mem::allocatePreferred("geocaching.record", capacity, false)); + job.size = capacity + 70; + job.bytes = static_cast(mem::allocatePreferred("geocaching.sign", 2 * job.size + 26, false)); + gc::GeocacheId cache; + gc::RevisionHash hash; + if (!job.unsigned_bytes || !job.bytes || !gc::protocol::encodeGeocacheRecord(record, job.unsigned_bytes, capacity, job.unsigned_size) || + gc::protocol::deriveGeocacheHashes({job.unsigned_bytes, job.unsigned_size}, s.crypto, job.bytes, job.size, cache, hash) != gc::protocol::VerificationResult::Valid) + { + stop("Cannot prepare signed record"); + return; + } + std::array issued_key{}; + std::memcpy(issued_key.data(), cache.bytes.data(), 32); + issued_key[35] = 1; + gc::ByteView previous; + if (s.state->view().find(3, {issued_key.data(), issued_key.size()}, previous)) + { + gc::storage::AuthorIssuedView issued; + if (!gc::storage::decodeAuthorIssued({issued_key.data(), issued_key.size()}, previous, issued) || !issued.issued_at.has_utc) + { + stop("Previous issuance requires recovery"); + return; + } + record.created_at = issued.issued_at.utc_seconds; + auto latest = issued; + size_t cursor = 0; + gc::storage::MutationView row; + const auto view = s.state->view(); + while (view.next(cursor, row)) + { + if (row.table != 3 || row.key.size != 36 || std::memcmp(row.key.data, cache.bytes.data(), 32)) continue; + gc::storage::AuthorIssuedView candidate; + if (!gc::storage::decodeAuthorIssued(row.key, row.value, candidate) || !candidate.issued_at.has_utc || + std::memcmp(candidate.author_public_key.data, job.author.data(), 64)) + { + stop("Author history requires recovery"); + return; + } + if (candidate.revision > latest.revision) latest = candidate; + } + record.revision = latest.revision; + record.updated_at = latest.issued_at.utc_seconds; + if (record.revision > 1) + { + for (unsigned i = 0; i < 4; ++i) issued_key[32 + i] = static_cast((record.revision - 1) >> ((3 - i) * 8)); + if (!view.find(3, {issued_key.data(), issued_key.size()}, previous) || + !gc::storage::decodeAuthorIssued({issued_key.data(), issued_key.size()}, previous, issued)) + { + stop("Predecessor history missing"); + return; + } + record.previous_hash = issued.revision_hash; + } + if (!gc::protocol::encodeGeocacheRecord(record, job.unsigned_bytes, capacity, job.unsigned_size) || + gc::protocol::deriveGeocacheHashes({job.unsigned_bytes, job.unsigned_size}, s.crypto, job.bytes, job.size, cache, hash) != gc::protocol::VerificationResult::Valid) + { + stop("Cannot reconstruct latest version"); + return; + } + if (!std::memcmp(hash.bytes.data(), latest.revision_hash.data, 32)) job.issued = latest.issued_at; + else + { + const auto published = gc::storage::draftPublication(view, key, draft); + if (published.confirmed_revision != latest.revision) + { + stop("Previous version unconfirmed; resolve it first"); + return; + } + if (latest.revision == UINT32_MAX || job.issued.utc_seconds < latest.issued_at.utc_seconds) + { + stop("Version limit or clock regression"); + return; + } + record.revision = latest.revision + 1; + record.previous_hash = latest.revision_hash; + record.updated_at = job.issued.utc_seconds; + if (!gc::protocol::encodeGeocacheRecord(record, job.unsigned_bytes, capacity, job.unsigned_size)) + { + stop("Cannot encode successor version"); + return; + } + } + } + if (job.expected_revision && record.revision != job.expected_revision) + { + stop("Version changed; review publication again"); + return; + } + job.author_port.reset(new (std::nothrow) DeviceAuthorIssuePort(*router, *s.store, s.crypto, job.issued, key, job.draft_generation)); + if (job.author_port) job.issue.reset(new (std::nothrow) gc::AuthorIssue(*job.author_port)); + if (!job.issue || !job.issue->begin({job.unsigned_bytes, job.unsigned_size}, job.bytes, job.size)) + { + stop("Cannot start record signing"); + return; + } + job.draft_stage = Stage::Sign; +} + +bool publicationDraftReady(const std::array& id, uint64_t generation, std::array& author, + uint32_t* from = nullptr, uint32_t* to = nullptr) +{ + gc::ByteView value; + gc::storage::DraftView draft; + gc::Destination remote; + const auto utc = std::time(nullptr); + const bool ready = session && session->phase == Phase::Ready && session->port && session->store && + !session->store->needsRecovery() && !session->store->commitPending() && !session->client->persistencePending() && + !downloadActive() && !publicationActive() && !draftSaveActive() && utc >= 946684800 && + static_cast(utc) <= 253402300799ULL && session->port->pageSource(remote) && + session->state->view().find(4, {id.data(), id.size()}, value) && gc::storage::decodeDraft({id.data(), id.size()}, value, draft) && + draft.generation == generation && draft.has_coordinates && + gc::protocol::validRecordText(draft.name, false, true) && router->getGeocachingAuthorKey(author.data()) && + (!draft.author.size || !std::memcmp(draft.author.data, author.data(), author.size())); + if (!ready) return false; + const auto history = gc::storage::draftPublication(session->state->view(), {id.data(), id.size()}, draft); + if (history.local_changes && (history.latest_revision == UINT32_MAX || history.confirmed_revision != history.latest_revision)) return false; + if (from) *from = history.latest_revision; + if (to) *to = history.latest_revision ? history.latest_revision + (history.local_changes ? 1 : 0) : 1; + return true; +} + +class Facade final : public ::ui::geocaching::Source +{ + public: + void activate(bool open) override + { + wanted.store(open); + next_step.store(0); + } + void snapshot(::ui::geocaching::Section section, ::ui::geocaching::Snapshot& out) override + { + out = {}; + Guard guard; + if (!guard.locked) + { + std::snprintf(out.status.data(), out.status.size(), "Updating..."); + return; + } + if (section == ::ui::geocaching::Section::Published && session && session->store && !session->store->needsRecovery()) + { + size_t cursor = 0; + gc::storage::MutationView row; + const auto view = session->state->view(); + while (view.next(cursor, row)) + if (row.table == 4) ++out.count; + out.generation = session->store->committedSequence(); + out.can_create = !session->store->commitPending() && !downloadActive() && !publicationActive() && !draftSaveActive() && + storage::sd_card_ready() && !storage::sd_external_block_owner_active(); + std::snprintf(out.status.data(), out.status.size(), "%s", out.count ? "Drafts and saved publication status" : "No local drafts"); + } + else if (section == ::ui::geocaching::Section::Downloaded && session && session->saved && !session->store->needsRecovery()) + session->saved->snapshot(out); + else if (session && session->source && session->phase == Phase::Ready) session->source->snapshot(section, out); + else + { + std::snprintf(out.status.data(), out.status.size(), "%s", session ? session->status : "Starting Geocaching..."); + out.can_refresh = session && session->phase == Phase::Failed; + } + if (session && session->notice) std::snprintf(out.status.data(), out.status.size(), "%s", session->notice); + if (downloadActive()) + { + out.can_refresh = false; + out.has_more = false; + const auto phase = session->download->phase(); + std::snprintf(out.status.data(), out.status.size(), "%s", phase == gc::DownloadPhase::Waiting ? "Downloading cache..." : phase == gc::DownloadPhase::Cancelling ? "Cancelling download..." + : "Saving and verifying GPX..."); + } + else if (session && session->download && session->download->phase() == gc::DownloadPhase::Failed) + std::snprintf(out.status.data(), out.status.size(), "Download could not be saved"); + else if (session && session->download_port && session->download_port->historyPending()) + std::snprintf(out.status.data(), out.status.size(), "Saved; history retention pending"); + if (session && session->publication && (publicationActive() || section == ::ui::geocaching::Section::Published)) + { + const auto& job = *session->publication; + const auto phase = job.attempt ? job.attempt->phase() : gc::PublishAttemptPhase::Failed; + const char* status = job.bytes ? "Preparing publication..." : phase == gc::PublishAttemptPhase::Confirmed ? "Directory accepted publication" + : phase == gc::PublishAttemptPhase::Waiting ? "Waiting for directory confirmation..." + : phase == gc::PublishAttemptPhase::Submitting || phase == gc::PublishAttemptPhase::Committing ? "Saving publication state..." + : phase == gc::PublishAttemptPhase::Cancelling ? "Stopping publication retries..." + : phase == gc::PublishAttemptPhase::Cancelled ? "Stopped; publication result unconfirmed" + : "Publication could not complete"; + if (job.error) status = job.error; + else if (job.draft_stage != Session::Publication::DraftStage::None) status = "Preparing publication..."; + std::snprintf(out.status.data(), out.status.size(), "%s", status); + if (publicationActive()) + { + out.can_refresh = false; + out.has_more = false; + } + } + if (session && session->draft_save && (draftSaveActive() || section == ::ui::geocaching::Section::Published)) + { + const auto& job = *session->draft_save; + std::snprintf(out.status.data(), out.status.size(), "%s", !job.done ? "Saving local draft..." : job.saved ? "Draft saved locally" + : "Draft save failed; changes not saved"); + if (!job.done) + { + out.can_refresh = false; + out.has_more = false; + } + } + out.generation ^= epoch << 32; + } + bool item(::ui::geocaching::Section section, size_t index, uint64_t generation, ::ui::geocaching::Item& out) override + { + Guard guard; + if (guard.locked && section == ::ui::geocaching::Section::Published && session && session->store && !session->store->needsRecovery()) + { + out = {}; + if ((generation ^ (epoch << 32)) != session->store->committedSequence()) return false; + size_t cursor = 0; + gc::storage::MutationView row; + const auto view = session->state->view(); + while (view.next(cursor, row)) + { + if (row.table != 4 || index--) continue; + gc::storage::DraftView draft; + if (!gc::storage::decodeDraft(row.key, row.value, draft)) return false; + out.is_draft = true; + out.edit_generation = draft.generation; + std::memcpy(out.id.data(), row.key.data, 16); + if (draft.name.empty()) std::snprintf(out.name.data(), out.name.size(), "Untitled draft"); + else std::memcpy(out.name.data(), draft.name.data(), draft.name.size()); + out.latitude_e7 = draft.latitude_e7; + out.longitude_e7 = draft.longitude_e7; + std::snprintf(out.detail.data(), out.detail.size(), "Local draft - not published\n%s\n%s", draft.has_coordinates ? "Location set" : "Location not set", draft.author.size ? "Author selected" : "Author not selected"); + const auto publication = gc::storage::draftPublication(view, row.key, draft); + out.publication_revision = publication.latest_revision; + out.publication_confirmed = publication.latest_revision && publication.confirmed_revision == publication.latest_revision; + if (publication.latest_revision) + { + std::snprintf(out.detail.data(), out.detail.size(), publication.confirmed_revision ? "v%lu %s%s\nLast directory-confirmed version: %lu" : "v%lu %s%s\nNo directory confirmation recorded", + static_cast(publication.latest_revision), out.publication_confirmed ? "accepted by directory" : publication.pending ? "awaiting confirmation" + : "stopped; result unconfirmed", + publication.local_changes ? "; local edits" : "", + static_cast(publication.confirmed_revision)); + } + return true; + } + return false; + } + if (guard.locked && section == ::ui::geocaching::Section::Downloaded && session && session->saved && !session->store->needsRecovery()) + return session->saved->item(index, generation ^ (epoch << 32), out); + if (!guard.locked || !session || !session->source || !session->source->item(section, index, generation ^ (epoch << 32), out)) return false; + out.downloaded = session->saved && session->saved->contains(out.id, out.revision_hash); + out.can_download = !out.downloaded && !downloadActive() && !publicationActive() && !draftSaveActive() && !session->store->commitPending() && + (!session->saved || !session->saved->pending()) && + !session->store->needsRecovery() && storage::sd_card_ready() && + (session->client->phase() == gc::QueryClientPhase::PageReady || session->client->phase() == gc::QueryClientPhase::Failed); + if (out.downloaded) + { + if (auto* suffix = std::strstr(out.detail.data(), "Directory preview - not yet downloaded")) + std::snprintf(suffix, out.detail.size() - static_cast(suffix - out.detail.data()), "Saved GPX on SD card"); + } + return true; + } + void refresh(::ui::geocaching::Section section) override + { + Guard guard; + if (!guard.locked || downloadActive() || publicationActive() || draftSaveActive()) return; + if (section == ::ui::geocaching::Section::Downloaded && session && session->saved && !session->store->needsRecovery()) session->saved->reset(); + else if (session && session->phase == Phase::Failed) restart.store(true); + else if (session && session->source) session->source->refresh(section); + next_step.store(0); + } + bool loadMore() override + { + Guard guard; + const bool begun = guard.locked && !downloadActive() && !publicationActive() && !draftSaveActive() && session && session->source && session->source->loadMore(); + if (begun) next_step.store(0); + return begun; + } + void open(const ::ui::geocaching::Item&, uint64_t) override {} + bool publicationAuthor(const std::array& id, uint64_t generation, std::array& author, uint32_t* from, uint32_t* to) override + { + Guard guard; + return guard.locked && publicationDraftReady(id, generation, author, from, to); + } + bool publishDraft(const std::array& id, uint64_t generation, const std::array& expected_author, uint32_t expected_revision) override + { + Guard guard; + std::array author; + uint32_t revision = 0; + if (!guard.locked || !publicationDraftReady(id, generation, author, nullptr, &revision) || author != expected_author || revision != expected_revision) return false; + auto job = std::unique_ptr(new (std::nothrow) Session::Publication); + if (!job || !session->port->pageSource(job->remote)) return false; + job->draft_id = id; + job->draft_generation = generation; + job->author = author; + job->expected_revision = expected_revision; + job->issued = now(nullptr); + job->issued.has_utc = true; + job->issued.utc_seconds = static_cast(std::time(nullptr)); + job->draft_stage = Session::Publication::DraftStage::Bind; + session->draft_save.reset(); + session->publication = std::move(job); + next_step.store(0); + ++epoch; + return true; + } + ::ui::geocaching::DraftReadStatus readDraft(const std::array& id, void (*sink)(const ::ui::geocaching::DraftInput&, void*), void* context) override + { + Guard guard; + using Status = ::ui::geocaching::DraftReadStatus; + if (!guard.locked) return Status::Pending; + if (!sink || !session || !session->store || session->store->needsRecovery()) return Status::Failed; + if (cancel_draft_read.exchange(false)) session->draft_read.reset(); + if (!session->draft_read || session->draft_read->id != id) + { + session->draft_read.reset(new (std::nothrow) Session::DraftRead); + if (!session->draft_read) return Status::Failed; + session->draft_read->id = id; + next_step.store(0); + return Status::Pending; + } + if (session->draft_read->status == Status::Pending) return Status::Pending; + if (session->draft_read->status == Status::Failed) + { + session->draft_read.reset(); + return Status::Failed; + } + gc::ByteView bytes; + gc::storage::DraftView draft; + if (!session->state->view().find(4, {id.data(), id.size()}, bytes) || + !gc::storage::decodeDraft({id.data(), id.size()}, bytes, draft) || draft.generation != session->draft_read->generation) + { + session->draft_read.reset(); + return Status::Failed; + } + ::ui::geocaching::DraftInput out; + out.id = id; + out.generation = draft.generation; + out.name = draft.name; + out.description = draft.description; + out.hint = draft.hint; + out.latitude_e7 = draft.latitude_e7; + out.longitude_e7 = draft.longitude_e7; + out.has_coordinates = draft.has_coordinates; + out.state = draft.state; + out.difficulty_x2 = draft.difficulty_x2; + out.terrain_x2 = draft.terrain_x2; + out.container_size = draft.container_size; + sink(out, context); + session->draft_read.reset(); + return Status::Ready; + } + void cancelDraftRead(const std::array& id) override + { + Guard guard; + if (guard.locked) + { + if (session && session->draft_read && session->draft_read->id == id) session->draft_read.reset(); + } + else cancel_draft_read.store(true); + next_step.store(0); + } + bool saveDraft(::ui::geocaching::DraftInput& input) override + { + Guard guard; + if (!guard.locked || !session || !session->store || session->store->needsRecovery() || session->store->commitPending() || + downloadActive() || publicationActive() || draftSaveActive() || session->phase == Phase::ResumeDownloads || + (session->client && session->client->persistencePending()) || input.generation == UINT64_MAX || + input.name.size() > 96 || input.description.size() > 2048 || input.hint.size() > 512) return false; + gc::storage::DraftView draft; + if (input.generation) + { + gc::ByteView stored; + if (!session->state->view().find(4, {input.id.data(), input.id.size()}, stored) || + !gc::storage::decodeDraft({input.id.data(), input.id.size()}, stored, draft) || draft.generation != input.generation) return false; + } + else if (input.id == std::array{}) esp_fill_random(input.id.data(), input.id.size()); + draft.generation = input.generation + 1; + draft.name = input.name; + draft.description = input.description; + draft.hint = input.hint; + draft.latitude_e7 = input.latitude_e7; + draft.longitude_e7 = input.longitude_e7; + draft.has_coordinates = input.has_coordinates; + draft.state = input.state; + draft.difficulty_x2 = input.difficulty_x2; + draft.terrain_x2 = input.terrain_x2; + draft.container_size = input.container_size; + auto job = std::unique_ptr(new (std::nothrow) Session::DraftSave); + if (!job) return false; + const auto capacity = input.name.size() + input.description.size() + input.hint.size() + 160; + job->bytes = static_cast(mem::allocatePreferred("geocaching.draft", capacity, false)); + if (!job->bytes || !gc::storage::encodeDraft({input.id.data(), input.id.size()}, draft, job->bytes, capacity, job->size)) return false; + job->id = input.id; + job->expected = input.generation; + session->draft_save = std::move(job); + next_step.store(0); + ++epoch; + return true; + } + ::ui::geocaching::DraftSaveStatus draftSaveStatus(const std::array& id, uint64_t generation) override + { + using Status = ::ui::geocaching::DraftSaveStatus; + Guard guard; + if (!guard.locked) return Status::Pending; + if (!session || !session->draft_save || session->draft_save->id != id || session->draft_save->expected != generation) return Status::Failed; + const auto& job = *session->draft_save; + return !job.done ? Status::Pending : job.saved ? Status::Saved + : Status::Failed; + } + bool download(const ::ui::geocaching::Item& item, uint64_t generation) override + { + Guard guard; + if (!guard.locked || !session || !session->source || !session->port || downloadActive() || publicationActive() || draftSaveActive() || session->store->commitPending()) return false; + if (session->client->phase() != gc::QueryClientPhase::PageReady && session->client->phase() != gc::QueryClientPhase::Failed) return false; + ::ui::geocaching::Snapshot snapshot; + session->source->snapshot(::ui::geocaching::Section::Discover, snapshot); + if ((generation ^ (epoch << 32)) != snapshot.generation) return false; + gc::protocol::SummaryView summary; + bool found = false; + for (size_t i = 0; i < snapshot.count; ++i) + if (session->port->summary(i, summary) && summary.id.bytes == item.id && summary.hash.bytes == item.revision_hash) + { + found = true; + break; + } + gc::Destination remote; + if (!found || !session->port->pageSource(remote)) return false; + const auto install_generation = session->store->nextDownloadGeneration(summary.id); + if (!install_generation) return false; + std::array task; + gc::RequestId request; + randomId(nullptr, task.data()); + randomId(nullptr, request.bytes.data()); + session->download.reset(); + session->download_port.reset(); + session->download_port.reset(new (std::nothrow) SdDownloadPort(*session->store, *session->state, session->crypto, + session->local, {summary.id, summary.hash, install_generation}, task, now(nullptr))); + if (!session->download_port) return false; + session->download.reset(new (std::nothrow) gc::DownloadClient(*session->download_port, session->crypto)); + if (!session->download || !session->download->begin(remote, request, summary, install_generation)) + { + session->download.reset(); + session->download_port.reset(); + return false; + } + session->download_started = now(nullptr).monotonic_ms; + session->download_wait_ms = gc::QueryClient::kReplyTimeoutMs; + session->download_scratch = summary.signed_bytes + 64; + if (session->saved) session->saved->reset(); + ++epoch; + next_step.store(0); + return true; + } +} facade; + +bool closeSession() +{ + if (!session) return true; + if (session->client && session->store && !session->store->needsRecovery()) + { + if (session->client->persistencePending()) + { + session->client->tick(now(nullptr).monotonic_ms); + return false; + } + if (session->store->commitPending()) + { + session->dispatcher->dispatchOne(now(nullptr)); + return false; + } + const auto phase = session->client->phase(); + if (phase == gc::QueryClientPhase::FindingDirectory || phase == gc::QueryClientPhase::CheckingCapabilities || + phase == gc::QueryClientPhase::Querying) + { + if (!session->client->cancel() || session->client->persistencePending()) return false; + } + } + if (!router->bindGeocachingHandlers(nullptr, nullptr, nullptr)) return false; + if (session->created_backend && router->backendForProtocol(chat::MeshProtocol::Reticulum) == session->created_backend && + router->backendProtocol() != chat::MeshProtocol::Reticulum && router->backendProtocol() != chat::MeshProtocol::RNode) + { + auto removed = router->takeServiceBackend(chat::MeshProtocol::Reticulum, session->created_backend); + if (!removed) return false; + } + session.reset(); + active.store(false); + ++epoch; + return true; +} +} // namespace + +void configure(chat::MeshAdapterRouter& value, LoraBoard& radio) +{ + router = &value; + board = &radio; + if (!mutex) mutex = xSemaphoreCreateMutex(); + esp_fill_random(boot.data(), boot.size()); + ::geocaching::ui::shell::bind(&facade); +} +bool queueDraftSave(const uint8_t id[16], const uint8_t* bytes, size_t size, uint64_t expected) +{ + Guard guard; + gc::storage::DraftView draft; + if (!guard.locked || !id || !session || !session->store || session->store->commitPending() || session->store->needsRecovery() || + downloadActive() || publicationActive() || draftSaveActive() || session->phase == Phase::ResumeDownloads || + (session->client && session->client->persistencePending()) || + !gc::storage::decodeDraft({id, 16}, {bytes, size}, draft) || expected == UINT64_MAX || draft.generation != expected + 1) return false; + auto job = std::unique_ptr(new (std::nothrow) Session::DraftSave); + if (!job) return false; + job->bytes = static_cast(mem::allocatePreferred("geocaching.draft", size, false)); + if (!job->bytes) return false; + std::memcpy(job->id.data(), id, 16); + std::memcpy(job->bytes, bytes, size); + job->size = size; + job->expected = expected; + session->draft_save = std::move(job); + next_step.store(0); + ++epoch; + return true; +} +bool queuePublication(const uint8_t* bytes, size_t size) +{ + Guard guard; + if (!guard.locked || !bytes || !size || size > 4166 || !session || session->phase != Phase::Ready || + !session->port || !session->store || session->store->commitPending() || session->store->needsRecovery() || + session->client->persistencePending() || downloadActive() || publicationActive() || draftSaveActive()) return false; + gc::Destination remote; + if (!session->port->pageSource(remote)) return false; + auto job = std::unique_ptr(new (std::nothrow) Session::Publication); + if (!job) return false; + job->bytes = static_cast(mem::allocatePreferred("geocaching.publish", 2 * size + 26, false)); + if (!job->bytes) return false; + std::memcpy(job->bytes, bytes, size); + job->size = size; + job->remote = remote; + session->draft_save.reset(); + session->publication = std::move(job); + next_step.store(0); + ++epoch; + return true; +} +bool workPending() +{ + if (!router || !board || !mutex) return false; + if (!wanted.load()) return active.load(); + return static_cast(millis() - next_step.load()) >= 0; +} +void step() +{ + Guard guard; + if (!guard.locked || !router || !board) return; + next_step.store(millis() + 5); + // Finish the small first-use volume header before closing; abandoning an + // otherwise healthy initialization would leave an unrecoverable empty ledger. + const bool finishing_initialization = session && session->phase >= Phase::Directories && + session->phase <= Phase::VerifyFormat && storage::sd_card_ready() && !storage::sd_external_block_owner_active(); + const bool recovering_download = session && session->phase == Phase::ResumeDownloads; + if ((!wanted.load() && !finishing_initialization && !downloadActive() && !publicationActive() && !draftSaveActive() && !recovering_download) || restart.load()) + { + if (closeSession()) restart.store(false); + return; + } + if (!session) + { + session.reset(new (std::nothrow) Session); + if (!session) + { + next_step.store(millis() + 2000); + return; + } + active.store(true); + ++epoch; + } + auto& s = *session; + if (cancel_draft_read.exchange(false)) s.draft_read.reset(); + if (!storage::sd_card_ready() || storage::sd_external_block_owner_active()) + { + if (!s.notice) + { + s.notice = "SD card unavailable"; + if (s.saved) s.saved->reset(); + ++epoch; + } + next_step.store(millis() + 1000); + if (s.draft_read) s.draft_read->status = ::ui::geocaching::DraftReadStatus::Failed; + return; + } + if (s.notice) + { + s.notice = nullptr; + ++epoch; + } + if (s.draft_read && s.draft_read->status == ::ui::geocaching::DraftReadStatus::Pending) + { + auto& job = *s.draft_read; + gc::ByteView bytes; + gc::storage::DraftView draft; + const bool valid = s.store && !s.store->needsRecovery() && s.state && + s.state->view().find(4, {job.id.data(), job.id.size()}, bytes) && + gc::storage::decodeDraft({job.id.data(), job.id.size()}, bytes, draft); + job.status = valid ? ::ui::geocaching::DraftReadStatus::Ready : ::ui::geocaching::DraftReadStatus::Failed; + if (valid) job.generation = draft.generation; + return; + } + if (draftSaveActive()) + { + auto& job = *s.draft_save; + const auto result = job.started ? s.store->stepCommit() : s.store->saveDraft({job.id.data(), job.id.size()}, {job.bytes, job.size}, job.expected); + job.started = true; + heap_caps_free(job.bytes); + job.bytes = nullptr; + if (result != JournalWriteResult::InProgress) + { + job.done = true; + job.saved = result == JournalWriteResult::Verified; + ++epoch; + } + return; + } + if (s.saved && s.saved->pending() && s.phase != Phase::ResumeDownloads && !downloadActive() && !publicationActive() && !s.store->commitPending() && !s.store->needsRecovery()) + { + s.saved->advance(); + return; + } + if (s.phase == Phase::Failed) + { + next_step.store(millis() + 2000); + return; + } + switch (s.phase) + { + case Phase::Inspect: + { + const auto result = inspectSdVolume(s.volume); + if (result == SdVolumeResult::Ready) startRecovery(); + else if (result == SdVolumeResult::Missing) s.phase = Phase::CheckNew; + else fail("Geocaching storage requires recovery"); + return; + } + case Phase::CheckNew: + if (storage::sd_exists("/trailmate/geocaching/.state")) + { + fail("Interrupted storage initialization"); + return; + } + esp_fill_random(s.volume.data(), s.volume.size()); + s.phase = Phase::Directories; + return; + case Phase::Directories: + { + static constexpr const char* paths[] = {"/trailmate", "/trailmate/geocaching", "/trailmate/geocaching/caches", + "/trailmate/geocaching/imports", "/trailmate/geocaching/exports", "/trailmate/geocaching/.state", + "/trailmate/geocaching/.state/history", "/trailmate/geocaching/.state/journal", + "/trailmate/geocaching/.state/checkpoint", "/trailmate/geocaching/.state/staging"}; + if (s.directory == sizeof(paths) / sizeof(paths[0])) + { + s.phase = Phase::OpenFormat; + return; + } + if (s.mkdir_pending) + { + if (!storage::sd_mkdir(paths[s.directory])) + { + fail("Cannot create geocaching storage"); + return; + } + s.mkdir_pending = false; + ++s.directory; + } + else if (storage::sd_is_directory(paths[s.directory])) ++s.directory; + else s.mkdir_pending = true; + return; + } + case Phase::OpenFormat: + if (!s.format.open("/trailmate/geocaching/.state/format.bin", "w")) + { + fail("Cannot initialize storage"); + return; + } + s.phase = Phase::WriteFormat; + return; + case Phase::WriteFormat: + { + const auto header = gc::storage::encodeVolumeHeader(s.volume); + if (s.format.write(header.data(), header.size()) != header.size()) + { + fail("Storage write failed"); + return; + } + s.phase = Phase::FlushFormat; + return; + } + case Phase::FlushFormat: + if (!s.format.flush()) + { + fail("Storage flush failed"); + return; + } + s.phase = Phase::CloseFormat; + return; + case Phase::CloseFormat: + s.format.close(); + s.phase = Phase::VerifyFormat; + return; + case Phase::VerifyFormat: + { + gc::storage::VolumeInstance found; + if (inspectSdVolume(found) != SdVolumeResult::Ready || found != s.volume) + { + fail("Storage verification failed"); + return; + } + if (!wanted.load()) + { + s.phase = Phase::Inspect; + return; + } + startRecovery(); + return; + } + case Phase::Recover: + { + const auto result = s.recovery->step(s.frame, kFrameCapacity, s.mutations.data(), s.mutations.size(), recoveredStateValid); + if (result == StateRecoveryStep::Working) return; + if (result != StateRecoveryStep::JournalRestored) + { + fail("Cached state needs recovery or more workspace"); + return; + } + s.store.reset(new (std::nothrow) SdRequestStore(s.volume, s.recovery->replayedSequence(), *s.state)); + s.recovery.reset(); + heap_caps_free(s.frame); + s.frame = nullptr; + if (!s.store) + { + fail("Insufficient memory"); + return; + } + s.saved.reset(new (std::nothrow) SavedCacheCatalog(*s.state, s.crypto)); + if (!s.saved) + { + fail("Insufficient catalogue memory"); + return; + } + s.phase = Phase::ResumeDownloads; + s.status = "Recovering downloaded GPX files..."; + ++epoch; + return; + } + case Phase::ResumeDownloads: + { + if (s.download_port) + { + const auto result = s.download_port->poll(); + if (result == gc::DownloadOperationResult::Pending) return; + if (result != gc::DownloadOperationResult::Complete) + { + fail("Downloaded GPX recovery needs attention"); + return; + } + s.download_port.reset(); + s.saved->reset(); + ++epoch; + return; + } + const auto view = s.state->view(); + size_t cursor = 0; + gc::storage::MutationView row; + while (view.next(cursor, row)) + { + if (row.table != 5) continue; + gc::storage::OutgoingView outgoing; + gc::storage::TaskView task; + gc::ByteView value; + if (!gc::storage::decodeOutgoing(row.key, row.value, outgoing) || outgoing.state != 4 || + !outgoing.continue_intent || !view.find(10, outgoing.task_id, value) || + !gc::storage::decodeTask(outgoing.task_id, value, task) || task.kind != 2 || + task.state == 3 || task.state == 5 || !task.continue_intent || + task.cache_id.size != 32 || task.revision_hash.size != 32 || + !gc::storage::requestBelongsToTask(outgoing.task_id, task, row.key, outgoing) || + !s.store->downloadIntentActive(row.key, outgoing.install_generation)) continue; + gc::Destination local, remote; + gc::RequestId request; + gc::InstallIdentity identity; + std::array task_id; + std::memcpy(local.bytes.data(), row.key.data, 16); + std::memcpy(remote.bytes.data(), row.key.data + 16, 16); + std::memcpy(request.bytes.data(), row.key.data + 32, 16); + std::memcpy(task_id.data(), outgoing.task_id.data, 16); + std::memcpy(identity.id.bytes.data(), task.cache_id.data, 32); + std::memcpy(identity.hash.bytes.data(), task.revision_hash.data, 32); + identity.generation = outgoing.install_generation; + s.download_port.reset(new (std::nothrow) SdDownloadPort(*s.store, *s.state, s.crypto, + local, identity, task_id, outgoing.created)); + if (!s.download_port || s.download_port->resume(remote, request) != gc::DownloadOperationResult::Pending) + fail("Cannot resume downloaded GPX"); + return; + } + s.phase = Phase::Connect; + s.connect_since = now(nullptr).monotonic_ms; + s.status = "Connecting to Reticulum..."; + ++epoch; + return; + } + case Phase::Connect: + { + // A previous chat selection can leave an inactive Reticulum instance. + // Release it outside the router lock before allocating the IP service. + if (auto cached = router->takeInactiveReticulumCache()) return; + if (!router->bindGeocachingHandlers(announcementReceived, responseReceived, nullptr)) return; + if (!router->backendForProtocol(chat::MeshProtocol::Reticulum)) + { + // The existing LXMF allocator uses PSRAM, not the internal heap. + // Check its actual object size and contiguous capacity before creation. + if (!mem::admit("geocaching.transport", sizeof(chat::reticulum::ReticulumAdapter) + 4096, 0, + sizeof(chat::lxmf::LxmfAdapter), 40 * 1024, 0, 4096)) + { + fail("Insufficient transport memory"); + return; + } + auto backend = std::unique_ptr(new (std::nothrow) + chat::reticulum::ReticulumAdapter(*board, nullptr, chat::reticulum::ReticulumUsage::BackgroundIpService)); + if (!backend) + { + fail("Cannot start Reticulum"); + return; + } + backend->applyConfig(app::AppContext::getInstance().readConfig().reticulumConfig()); + auto* created = backend.get(); + if (!router->installServiceBackend(chat::MeshProtocol::Reticulum, std::move(backend))) return; + s.created_backend = created; + } + gc::Destination local; + if (!router->getGeocachingDispatchDestination(local.bytes.data())) + { + if (now(nullptr).monotonic_ms - s.connect_since >= gc::QueryClient::kReplyTimeoutMs) + { + fail("Reticulum IP unavailable - check connection settings"); + return; + } + if (std::strcmp(s.status, "Waiting for Reticulum IP connection")) + { + s.status = "Waiting for Reticulum IP connection"; + ++epoch; + } + next_step.store(millis() + 500); + return; + } + if (s.client) + { + if (local.bytes != s.local.bytes) + { + fail("Reticulum identity changed - reopen Geocaching"); + return; + } + s.phase = Phase::Ready; + ++epoch; + return; + } + s.port.reset(new (std::nothrow) QueryStorePort(*s.store, *s.state, local, randomId, now, nullptr)); + s.local = local; + if (!s.port) + { + fail("Insufficient memory"); + return; + } + s.client.reset(new (std::nothrow) gc::QueryClient(*s.port)); + s.dispatcher.reset(new (std::nothrow) RequestDispatcher(*router, *s.store, 5000, 120000)); + if (!s.client || !s.dispatcher) + { + fail("Insufficient memory"); + return; + } + s.source.reset(new (std::nothrow) QueryBrowseSource(*s.client, *s.port, kWorld)); + if (!s.source) + { + fail("Insufficient memory"); + return; + } + s.client->query(kWorld); + s.phase = Phase::Ready; + ++epoch; + return; + } + case Phase::Ready: + break; + case Phase::Failed: + return; + } + if (s.store->needsRecovery()) + { + fail("Storage interrupted - reopen to recover"); + return; + } + if (publicationActive()) + { + auto& job = *s.publication; + if (job.draft_stage != Session::Publication::DraftStage::None) + { + advanceDraftPublication(s); + return; + } + if (job.bytes) + { + auto* scratch = job.bytes + job.size; + gc::protocol::VerifiedRecordView record; + std::array author; + if (gc::protocol::verifyGeocache({job.bytes, job.size}, s.crypto, scratch, job.size + 26, record) == gc::protocol::VerificationResult::Valid && + router->getGeocachingAuthorKey(author.data()) && !std::memcmp(author.data(), record.record.author_public_key.data, 64)) + { + const auto restored = restorePublication(s, &record.id, &record.hash, &job.remote); + if (restored == PublicationRestore::Restored) return; + if (restored == PublicationRestore::Invalid) + { + job.error = "Saved publication could not be recovered"; + heap_caps_free(job.bytes); + job.bytes = nullptr; + ++epoch; + return; + } + std::array task; + gc::RequestId request; + esp_fill_random(task.data(), task.size()); + esp_fill_random(request.bytes.data(), request.bytes.size()); + job.port.reset(new (std::nothrow) SdPublishPort(*s.store, s.crypto, s.local, record.id, record.hash, task, now(nullptr))); + if (job.port) job.attempt.reset(new (std::nothrow) gc::PublishAttempt(*job.port, s.crypto)); + if (job.attempt && !job.attempt->begin(job.remote, request, {job.bytes, job.size}, scratch, job.size + 26)) job.attempt.reset(); + } + heap_caps_free(job.bytes); + job.bytes = nullptr; + job.started = now(nullptr).monotonic_ms; + ++epoch; + return; + } + const auto phase = job.attempt->phase(); + if (phase != gc::PublishAttemptPhase::Waiting) + { + job.attempt->advance(); + ++epoch; + return; + } + if (now(nullptr).monotonic_ms - job.started >= job.wait_ms && !s.store->commitPending()) + { + job.attempt->cancel(); + ++epoch; + return; + } + } + if (!s.publication && !downloadActive() && !s.store->commitPending() && !s.client->persistencePending()) + { + const auto restored = restorePublication(s); + if (restored == PublicationRestore::Invalid) + { + fail("Saved publication could not be recovered"); + return; + } + if (restored == PublicationRestore::Restored) return; + } + if (!s.download && !publicationActive() && !s.store->commitPending() && !s.client->persistencePending() && resumeWaitingDownload(s)) return; + if (s.download && (s.download->phase() == gc::DownloadPhase::Submitting || s.download->phase() == gc::DownloadPhase::Installing || + s.download->phase() == gc::DownloadPhase::Cancelling)) + { + const auto before = s.download->phase(); + s.download->advance(); + if (before != s.download->phase()) + { + if (s.saved) s.saved->reset(); + ++epoch; + } + return; + } + if (auto cached = router->takeInactiveReticulumCache()) + { + s.created_backend = nullptr; + s.phase = Phase::Connect; + s.connect_since = now(nullptr).monotonic_ms; + s.status = "Reconnecting Reticulum service..."; + ++epoch; + return; + } + const auto phase = s.client->phase(); + if (s.client->persistencePending()) + { + s.client->tick(now(nullptr).monotonic_ms); + return; + } + if (s.store->commitPending()) + { + s.dispatcher->dispatchOne(now(nullptr)); + return; + } + if (s.response) + { + if (s.response_operation == 1 && s.publication && s.publication->attempt) + { + s.publication->attempt->accept(s.response_source, {s.response, s.response_size}); + ++epoch; + } + else if (s.response_operation == 3 && s.download) + { + auto* scratch = static_cast(mem::allocatePreferred("geocaching.verify", s.download_scratch, false)); + if (scratch) + { + const auto before = s.download->phase(); + s.download->accept(s.response_source, {s.response, s.response_size}, scratch, s.download_scratch); + if (before != s.download->phase()) ++epoch; + heap_caps_free(scratch); + } + } + else s.client->accept(s.response_source, {s.response, s.response_size}); + heap_caps_free(s.response); + s.response = nullptr; + s.response_size = 0; + return; + } + if (s.download && s.download->phase() == gc::DownloadPhase::Waiting && + now(nullptr).monotonic_ms - s.download_started >= s.download_wait_ms) + { + s.download->cancel(); + ++epoch; + return; + } + if (s.announcement) + { + const auto& incoming = *s.announcement; + s.client->observe(incoming.discovery, incoming.delivery, {incoming.key.data(), incoming.key.size()}, + {incoming.data.data(), incoming.size}, now(nullptr).monotonic_ms); + s.announcement.reset(); + } + if (publicationActive() || (s.download && s.download->phase() == gc::DownloadPhase::Waiting)) + { + const auto sent = s.dispatcher->dispatchOne(now(nullptr)); + if (sent.status == DispatchStatus::StorageBlocked || sent.status == DispatchStatus::Corrupt) + fail("Download storage is blocked"); + return; + } + if (phase == gc::QueryClientPhase::FindingDirectory) + { + if (!s.client->tick(now(nullptr).monotonic_ms)) next_step.store(millis() + 500); + return; + } + if (s.client->tick(now(nullptr).monotonic_ms)) return; + if (s.client->persistencePending()) return; + if (s.client->phase() == gc::QueryClientPhase::Failed) + { + next_step.store(millis() + 2000); + return; + } + const auto sent = s.dispatcher->dispatchOne(now(nullptr)); + if (sent.status == DispatchStatus::StorageBlocked || sent.status == DispatchStatus::Corrupt) + fail("Query storage is blocked"); + else if (!s.store->commitPending()) next_step.store(millis() + 250); +} +} // namespace platform::esp::arduino_common::geocaching::browse_runtime diff --git a/platform/esp/arduino_common/src/geocaching/request_dispatcher.cpp b/platform/esp/arduino_common/src/geocaching/request_dispatcher.cpp new file mode 100644 index 00000000..02da12ee --- /dev/null +++ b/platform/esp/arduino_common/src/geocaching/request_dispatcher.cpp @@ -0,0 +1,140 @@ +#include "platform/esp/arduino_common/geocaching/request_dispatcher.h" +#include "esp_random.h" + +namespace platform::esp::arduino_common::geocaching +{ +RequestDispatcher::RequestDispatcher(chat::MeshAdapterRouter& router, RequestDispatchStore& store, + uint32_t retry_delay_ms, uint64_t attempt_timeout_ms) + : router_(router), store_(store), retry_delay_ms_(retry_delay_ms), attempt_timeout_ms_(attempt_timeout_ms) {} + +DispatchResult RequestDispatcher::dispatchOne(const ::geocaching::storage::StoredTime& now) +{ + using namespace ::geocaching::storage; + if (store_.needsRecovery()) return {DispatchStatus::StorageBlocked}; + const auto defer = [&]() + { + not_before_ = now.monotonic_ms > UINT64_MAX - retry_delay_ms_ ? UINT64_MAX : now.monotonic_ms + retry_delay_ms_; + }; + if (phase_ != Phase::Select && phase_ != Phase::SelectRequest && phase_ != Phase::Send) + { + const auto result = store_.stepCommit(); + if (result == JournalWriteResult::InProgress) return {DispatchStatus::Deferred}; + const auto completed = phase_; + phase_ = Phase::Select; + if (result != JournalWriteResult::Verified) return {DispatchStatus::StorageBlocked}; + if (completed == Phase::BeginCommit) + { + phase_ = Phase::Send; + return {DispatchStatus::Deferred}; + } + if (completed == Phase::SuccessCommit) return {DispatchStatus::Submitted}; + if (completed == Phase::ExpireCommit && !store_.expirationChanged()) + { + phase_ = Phase::SelectRequest; + return {DispatchStatus::Deferred}; + } + defer(); + return {DispatchStatus::Deferred, completed == Phase::FailureCommit ? send_failure_ : chat::MeshOperationFailure::None}; + } + // Another serialized operation may own the pending transaction. + if (store_.commitPending()) return {DispatchStatus::Deferred}; + if (phase_ == Phase::Send) + { + DispatchSendView outgoing; + const auto read = store_.readForSend({cursor_.data(), cursor_.size()}, outgoing); + if (read == DispatchReadResult::Pending) return {DispatchStatus::Deferred}; + if (read != DispatchReadResult::Ready) + { + phase_ = Phase::Select; + return {DispatchStatus::Corrupt}; + } + std::array attempt_key{}; + std::memcpy(attempt_key.data(), cursor_.data(), 48); + std::memcpy(attempt_key.data() + 48, attempt_id_.data(), 16); + JournalWriteResult saved; + if (outgoing.stopped) + { + send_failure_ = chat::MeshOperationFailure::None; + saved = store_.finishAttempt({attempt_key.data(), attempt_key.size()}, TxAttemptState::CancelledBeforeSend, now); + phase_ = Phase::FailureCommit; + } + else + { + std::array hash{}; + // Source/destination come from the committed request key. The + // router checks the current local identity while holding its lock. + const auto sent = router_.sendGeocachingData(cursor_.data() + 16, + {outgoing.request.data, outgoing.request.size}, false, &hash, cursor_.data()); + send_failure_ = sent.failure; + saved = sent.ok ? store_.recordAttemptHash({attempt_key.data(), attempt_key.size()}, hash) + : store_.finishAttempt({attempt_key.data(), attempt_key.size()}, TxAttemptState::Failed, now); + phase_ = sent.ok ? Phase::SuccessCommit : Phase::FailureCommit; + } + if (saved == JournalWriteResult::InProgress) return {DispatchStatus::Deferred}; + const bool success = phase_ == Phase::SuccessCommit; + phase_ = Phase::Select; + if (saved != JournalWriteResult::Verified) return {DispatchStatus::StorageBlocked}; + if (!success) defer(); + return {success ? DispatchStatus::Submitted : DispatchStatus::Deferred, send_failure_}; + } + if (!clock_initialized_ || boot_ != now.boot_id) + { + boot_ = now.boot_id; + not_before_ = 0; + has_cursor_ = false; + recovery_started_ms_ = now.monotonic_ms; + clock_initialized_ = true; + } + if (phase_ == Phase::Select) + { + bool expired = false; + const auto expiration = store_.expireOneAttempt(now, recovery_started_ms_, attempt_timeout_ms_, expired); + if (expiration == JournalWriteResult::Busy) return {DispatchStatus::Deferred}; + if (expiration == JournalWriteResult::InProgress) + { + phase_ = Phase::ExpireCommit; + return {DispatchStatus::Deferred}; + } + if (expiration != JournalWriteResult::Verified) return {DispatchStatus::StorageBlocked}; + if (expired) + { + defer(); + return {DispatchStatus::Deferred}; + } + if (now.monotonic_ms < not_before_) return {DispatchStatus::Deferred}; + phase_ = Phase::SelectRequest; + } + ::geocaching::Destination local; + if (now.monotonic_ms < not_before_) return {DispatchStatus::Deferred}; + if (!router_.getGeocachingDispatchDestination(local.bytes.data())) + return {DispatchStatus::Deferred, chat::MeshOperationFailure::NotReady}; + PendingRequestView pending; + auto selected = store_.readPending(local, has_cursor_ ? ::geocaching::ByteView{cursor_.data(), cursor_.size()} : ::geocaching::ByteView{}, pending); + if (selected == DispatchReadResult::Pending) return {DispatchStatus::Deferred}; + if (selected == DispatchReadResult::None && has_cursor_) + { + has_cursor_ = false; + return {DispatchStatus::Deferred}; + } + phase_ = Phase::Select; + if (selected == DispatchReadResult::Corrupt) return {DispatchStatus::Corrupt}; + if (selected == DispatchReadResult::None) return {DispatchStatus::Idle}; + cursor_ = pending.key; + has_cursor_ = true; + esp_fill_random(attempt_id_.data(), attempt_id_.size()); + const auto begun = store_.beginAttempt(local, {cursor_.data(), cursor_.size()}, attempt_id_, now); + if (begun == JournalWriteResult::InProgress) + { + phase_ = Phase::BeginCommit; + return {DispatchStatus::Deferred}; + } + if (begun == JournalWriteResult::Verified) + { + phase_ = Phase::Send; + return {DispatchStatus::Deferred}; + } + return {begun == JournalWriteResult::StateRejected || begun == JournalWriteResult::Busy + ? DispatchStatus::Deferred + : DispatchStatus::StorageBlocked}; +} +} // namespace platform::esp::arduino_common::geocaching diff --git a/platform/esp/arduino_common/src/gps/track_recorder.cpp b/platform/esp/arduino_common/src/gps/track_recorder.cpp index da095d40..4996a841 100644 --- a/platform/esp/arduino_common/src/gps/track_recorder.cpp +++ b/platform/esp/arduino_common/src/gps/track_recorder.cpp @@ -1,5 +1,7 @@ #include "platform/esp/arduino_common/gps/track_recorder.h" #include "platform/esp/arduino_common/storage/sd_card_runtime.h" +#include "platform/esp/arduino_common/gps/sd_gpx_output.h" +#include "gps/gpx/track_writer.h" #include #include @@ -107,16 +109,8 @@ void write_track_point(SdRuntimeFile& f, TrackFormat format, const TrackPoint& p else { const String time_str = track_iso_time(pt.timestamp); - f.printf("\n", pt.lat, pt.lon); - f.printf(" %.1f\n", 0.0); - f.printf(" \n", time_str.c_str()); - f.print(" \n"); - f.printf(" %.2f\n", 0.0); - f.printf(" %.1f\n", 0.0); - f.printf(" %.1f\n", 0.0); - f.printf(" %u\n", (unsigned)pt.satellites); - f.print(" \n"); - f.print("\n"); + ::platform::esp::arduino_common::gps::SdGpxOutput output(f); + (void)::gps::gpx::writeTrackPoint(output, pt.lat, pt.lon, time_str.c_str(), pt.satellites); } } diff --git a/platform/esp/arduino_common/src/storage/sd_card_runtime.cpp b/platform/esp/arduino_common/src/storage/sd_card_runtime.cpp index f0c095b9..fc331df3 100644 --- a/platform/esp/arduino_common/src/storage/sd_card_runtime.cpp +++ b/platform/esp/arduino_common/src/storage/sd_card_runtime.cpp @@ -1788,15 +1788,21 @@ bool SdRuntimeDir::is_open() const bool SdRuntimeDir::read_next(char* name, std::size_t name_size, bool* is_dir) { - if (!is_open() || name == nullptr || name_size == 0) + return read_next_status(name, name_size, is_dir) == SdDirReadStatus::Entry; +} + +SdDirReadStatus SdRuntimeDir::read_next_status(char* name, std::size_t name_size, bool* is_dir) +{ + if (name == nullptr || name_size == 0) { - return false; + return SdDirReadStatus::Invalid; } name[0] = '\0'; if (is_dir != nullptr) { *is_dir = false; } + if (!is_open()) return SdDirReadStatus::Unavailable; if (impl_->backend == SdCardBackend::SdFat) { @@ -1805,7 +1811,7 @@ bool SdRuntimeDir::read_next(char* name, std::size_t name_size, bool* is_dir) if (!guard.locked()) { sd_io_end("dir_read", impl_->path, start_ms, false, 0, -2); - return false; + return SdDirReadStatus::Busy; } if (impl_->entry_scratch) { @@ -1815,8 +1821,9 @@ bool SdRuntimeDir::read_next(char* name, std::size_t name_size, bool* is_dir) FsFile& entry = impl_->entry_scratch; if (!entry) { - sd_io_end("dir_read", impl_->path, start_ms, true, 0, 0); - return false; + const bool failed = impl_->sdfat_dir.getError() != 0; + sd_io_end("dir_read", impl_->path, start_ms, !failed, 0, failed ? -1 : 0); + return failed ? SdDirReadStatus::IoError : SdDirReadStatus::End; } entry.getName(name, name_size); if (is_dir != nullptr) @@ -1825,10 +1832,10 @@ bool SdRuntimeDir::read_next(char* name, std::size_t name_size, bool* is_dir) } entry.close(); sd_io_end("dir_read", impl_->path, start_ms, true, 0, name[0] != '\0' ? 1 : 0); - return name[0] != '\0'; + return name[0] != '\0' ? SdDirReadStatus::Entry : SdDirReadStatus::IoError; } - return false; + return SdDirReadStatus::Unavailable; } bool sd_read_raw(uint32_t lba, uint8_t* buffer) diff --git a/platform/esp/arduino_common/src/storage/storage_runtime.cpp b/platform/esp/arduino_common/src/storage/storage_runtime.cpp index dd2b9e85..1b1799fc 100644 --- a/platform/esp/arduino_common/src/storage/storage_runtime.cpp +++ b/platform/esp/arduino_common/src/storage/storage_runtime.cpp @@ -1,4 +1,5 @@ #include "platform/esp/arduino_common/storage/storage_runtime.h" +#include "platform/esp/arduino_common/geocaching/browse_runtime.h" #include "platform/esp/arduino_common/chat/infra/store/sd_protocol_peer_repository.h" #include "platform/esp/arduino_common/chat/infra/store/sd_store.h" @@ -71,6 +72,19 @@ class SdMaintenanceAdapter final : public Adapter Result begin(Operation operation, OperationGeneration generation) override { next_step_ = Step::None; + if (operation == Operation::Persist && geocaching::browse_runtime::workPending()) + { + const bool other_pending = (context_.chat_store && context_.chat_store->persistencePending()) || + (context_.peer_directory && context_.peer_directory->persistencePending()); + if (geocaching_turn_ || !other_pending) + { + geocaching_turn_ = false; + geocaching_steps_ = 8; + next_step_ = Step::Geocaching; + return Result::inProgressResult(operation, generation); + } + } + geocaching_turn_ = true; if (operation == Operation::Hydrate) { const Result chat_result = hydrateChat(generation); @@ -144,6 +158,21 @@ class SdMaintenanceAdapter final : public Adapter budget) override { const Step step = next_step_; + if (step == Step::Geocaching) + { + if (budget.max_work_items == 0) return Result::inProgressResult(operation, generation); + if (geocaching::browse_runtime::workPending()) geocaching::browse_runtime::step(); + // Recheck demand after every slice: messages or contacts may have + // become dirty after this Geocaching batch was admitted. Return to + // the shared owner instead of consuming the remaining batch first. + const bool other_pending = (context_.chat_store && context_.chat_store->persistencePending()) || + (context_.peer_directory && context_.peer_directory->persistencePending()); + if (geocaching_steps_) --geocaching_steps_; + if (!other_pending && geocaching_steps_ && geocaching::browse_runtime::workPending()) + return Result::inProgressResult(operation, generation); + next_step_ = Step::None; + return Result::completedResult(operation, generation); + } if (step == Step::Chat) { if (!context_.chat_store) @@ -248,6 +277,7 @@ class SdMaintenanceAdapter final : public Adapter HydratePeer, PersistPeer, CompactPeer, + Geocaching, }; Result hydrateChat(OperationGeneration generation) @@ -288,6 +318,8 @@ class SdMaintenanceAdapter final : public Adapter WorkerContext& context_; Step next_step_ = Step::None; + bool geocaching_turn_ = true; + uint8_t geocaching_steps_ = 0; }; SdMaintenanceAdapter s_adapter(s_context); @@ -407,6 +439,7 @@ void tick_deferred_storage() Demand demand{}; demand.persistence_pending = + geocaching::browse_runtime::workPending() || (s_context.chat_store && s_context.chat_store->persistencePending()) || (s_context.peer_directory && diff --git a/platform/esp/arduino_common/src/ui/screens/team/team_ui_store.cpp b/platform/esp/arduino_common/src/ui/screens/team/team_ui_store.cpp index e2494408..19ab73a6 100644 --- a/platform/esp/arduino_common/src/ui/screens/team/team_ui_store.cpp +++ b/platform/esp/arduino_common/src/ui/screens/team/team_ui_store.cpp @@ -1,3 +1,5 @@ +#include "gps/gpx/track_writer.h" +#include "platform/esp/arduino_common/gps/sd_gpx_output.h" /** * @file team_ui_store.cpp * @brief ESP/SD-backed Team UI snapshot store (kept outside ui_shared) @@ -1617,20 +1619,9 @@ bool team_ui_append_member_track(const TeamId& team_id, const double lat = static_cast(pt.lat_e7) / 1e7; const double lon = static_cast(pt.lon_e7) / 1e7; const uint32_t ts = track.start_ts + static_cast(track.interval_s) * static_cast(i); - f.printf("\n", lat, lon); - f.printf(" %.1f\n", 0.0); - if (ts >= kMinValidEpoch) - { - const std::string time_str = iso_time(static_cast(ts)); - f.printf(" \n", time_str.c_str()); - } - f.print(" \n"); - f.printf(" %.2f\n", 0.0); - f.printf(" %.1f\n", 0.0); - f.printf(" %.1f\n", 0.0); - f.printf(" %u\n", 0u); - f.print(" \n"); - f.print("\n"); + const std::string time_str = ts >= kMinValidEpoch ? iso_time(static_cast(ts)) : ""; + ::platform::esp::arduino_common::gps::SdGpxOutput output(f); + if (!::gps::gpx::writeTrackPoint(output, lat, lon, time_str, 0, 7)) return false; } f.flush(); f.close(); diff --git a/platform/esp/common/include/platform/esp/common/geocaching_crypto.h b/platform/esp/common/include/platform/esp/common/geocaching_crypto.h new file mode 100644 index 00000000..d4b6afc5 --- /dev/null +++ b/platform/esp/common/include/platform/esp/common/geocaching_crypto.h @@ -0,0 +1,41 @@ +#pragma once + +#include "chat/infra/meshcore/crypto/ed25519/ed_25519.h" +#include "geocaching/protocol/verify_record.h" +#include "platform/esp/common/mbedtls_sha256_compat.h" + +namespace platform::esp::common +{ + +// Uses existing platform crypto, independent of the selected chat backend. +// No identity persistence or radio configuration is touched by verification. +class EspGeocachingCrypto final : public geocaching::protocol::RecordCrypto +{ + public: + bool sha256(geocaching::ByteView input, std::uint8_t output[32]) override + { + if (!output || (!input.data && input.size != 0)) return false; + mbedtls_sha256_context context; + mbedtls_sha256_init(&context); + const bool ok = crypto::sha256_starts(&context, 0) == 0 && + crypto::sha256_update(&context, input.data, input.size) == 0 && + crypto::sha256_finish(&context, output) == 0; + mbedtls_sha256_free(&context); + return ok; + } + + geocaching::protocol::VerificationResult verifyEd25519( + geocaching::ByteView public_key, geocaching::ByteView signature, + geocaching::ByteView message) override + { + using Result = geocaching::protocol::VerificationResult; + if (!public_key.data || public_key.size != 32 || !signature.data || + signature.size != 64 || (!message.data && message.size != 0)) + return Result::InvalidSignature; + return ed25519_verify(signature.data, message.data, message.size, public_key.data) != 0 + ? Result::Valid + : Result::InvalidSignature; + } +}; + +} // namespace platform::esp::common diff --git a/platform/esp/idf_common/src/platform_ui_tracker_runtime.cpp b/platform/esp/idf_common/src/platform_ui_tracker_runtime.cpp index b5fe009a..29c7aecb 100644 --- a/platform/esp/idf_common/src/platform_ui_tracker_runtime.cpp +++ b/platform/esp/idf_common/src/platform_ui_tracker_runtime.cpp @@ -1,4 +1,6 @@ #include "platform/ui/tracker_runtime.h" +#include "gps/gpx/track_writer.h" +#include "platform/esp/arduino_common/gps/sd_gpx_output.h" #include "esp_timer.h" #include "platform/esp/arduino_common/storage/sd_card_runtime.h" @@ -266,18 +268,13 @@ void write_point(TrackerRuntimeState& runtime, { char iso[24] = {}; format_iso_time(seconds, iso, sizeof(iso)); - runtime.file.printf(" ", - gps.lat, - gps.lng); - if (gps.has_alt) - { - runtime.file.printf("%.2f", gps.alt_m); - } - if (iso[0] != '\0') - { - runtime.file.printf("", iso); - } - runtime.file.print("\n"); + ::gps::gpx::TrackPointOptions options; + options.has_elevation = gps.has_alt; + options.elevation = gps.alt_m; + options.elevation_precision = 2; + options.legacy_extensions = false; + ::platform::esp::arduino_common::gps::SdGpxOutput output(runtime.file); + (void)::gps::gpx::writeTrackPoint(output, gps.lat, gps.lng, iso, gps.satellites, 7, options); break; } } diff --git a/platform/linux/common/src/platform/ui/team_ui_store_runtime.cpp b/platform/linux/common/src/platform/ui/team_ui_store_runtime.cpp index 9022bbf6..8b26d509 100644 --- a/platform/linux/common/src/platform/ui/team_ui_store_runtime.cpp +++ b/platform/linux/common/src/platform/ui/team_ui_store_runtime.cpp @@ -1,4 +1,6 @@ #include "platform/ui/team_ui_store_runtime.h" +#include "gps/gpx/track_writer.h" +#include "gps/gpx/ostream_sink.h" #include "platform/linux/runtime_paths.h" #include @@ -452,21 +454,9 @@ bool team_ui_append_member_track(const TeamId& team_id, const double lon = static_cast(pt.lon_e7) / 1e7; const uint32_t ts = track.start_ts + static_cast(track.interval_s) * static_cast(i); - char line[128] = {0}; - std::snprintf(line, sizeof(line), "\n", lat, lon); - stream << line; - stream << " 0.0\n"; - if (ts >= kMinValidEpoch) - { - stream << " \n"; - } - stream << " \n"; - stream << " 0.00\n"; - stream << " 0.0\n"; - stream << " 0.0\n"; - stream << " 0\n"; - stream << " \n"; - stream << "\n"; + const std::string time_str = ts >= kMinValidEpoch ? iso_time(static_cast(ts)) : ""; + ::gps::gpx::OstreamSink output(stream); + if (!::gps::gpx::writeTrackPoint(output, lat, lon, time_str, 0, 7)) return false; } stream.flush(); return stream.good(); diff --git a/platform/linux/common/src/platform/ui/tracker_runtime.cpp b/platform/linux/common/src/platform/ui/tracker_runtime.cpp index cba896d7..3e2edd3d 100644 --- a/platform/linux/common/src/platform/ui/tracker_runtime.cpp +++ b/platform/linux/common/src/platform/ui/tracker_runtime.cpp @@ -1,4 +1,7 @@ #include "platform/ui/tracker_runtime.h" +#include "gps/gpx/track_writer.h" +#include "gps/gpx/ostream_sink.h" +#include #include #include @@ -116,31 +119,24 @@ void append_gpx_point(std::ofstream& stream, const platform::ui::gps::GpsState& state, uint32_t epoch_s) { - stream << ""; - if (state.has_alt) + char iso[32]{}; + const std::time_t seconds = static_cast(epoch_s); + if (epoch_s != 0) { - stream << "" - << std::fixed << std::setprecision(1) << state.alt_m - << ""; + const std::tm* utc = std::gmtime(&seconds); + if (utc) std::strftime(iso, sizeof(iso), "%Y-%m-%dT%H:%M:%SZ", utc); } - stream << ""; - if (state.has_speed) - { - stream << "" - << std::fixed << std::setprecision(2) << state.speed_mps - << ""; - } - if (state.has_course) - { - stream << "" - << std::fixed << std::setprecision(1) << state.course_deg - << ""; - } - stream << "\n"; + ::gps::gpx::TrackPointOptions options; + options.has_elevation = state.has_alt; + options.elevation = state.alt_m; + options.legacy_extensions = false; + options.has_speed = state.has_speed; + options.speed_mps = state.speed_mps; + options.has_course = state.has_course; + options.course_deg = state.course_deg; + ::gps::gpx::OstreamSink output(stream); + if (!::gps::gpx::writeTrackPoint(output, state.lat, state.lng, iso, state.satellites, 7, options)) + stream.setstate(std::ios::badbit); } void append_binary_point(std::ofstream& stream, diff --git a/tools/geocaching/interop-requirements.txt b/tools/geocaching/interop-requirements.txt new file mode 100644 index 00000000..2e90bbef --- /dev/null +++ b/tools/geocaching/interop-requirements.txt @@ -0,0 +1,3 @@ +lxmf==1.1.1 +rns==1.5.4 +msgpack==1.2.2 diff --git a/tools/geocaching/interop_peer.py b/tools/geocaching/interop_peer.py new file mode 100644 index 00000000..cbfb20e0 --- /dev/null +++ b/tools/geocaching/interop_peer.py @@ -0,0 +1,330 @@ +"""Local Reticulum/LXMF fixture peer for the device browse milestone. + +This is an interoperability test peer, not a production public directory. +It serves fixtures and a bounded, in-memory publication test. By default its +Reticulum interface is loopback-only; no existing RNS configuration is used. +API usage follows https://github.com/markqvist/LXMF/blob/master/docs/example_receiver.py +""" +import argparse +import copy +import json +import hashlib +import os +from pathlib import Path +import queue +import threading +import time + +import LXMF +import RNS +import msgpack + +APP_TYPE = "trailmate.geocache" +ASPECT = "trailmate.geocache.directory" +FIXTURES = Path(__file__).resolve().parents[2] / "modules/core_geocaching/tests/fixtures" + + +def packed(value): + return msgpack.packb(value, use_bin_type=True) + + +def verified_cache(signed): + if not isinstance(signed, list) or len(signed) != 2: + raise ValueError("Invalid SignedCache") + raw, signature = signed + if not isinstance(raw, bytes) or len(raw) > 4096 or not isinstance(signature, bytes) or len(signature) != 64: + raise ValueError("Invalid record/signature sizes") + record = msgpack.unpackb(raw, raw=False) + if not isinstance(record, list) or len(record) != 16 or packed(record) != raw or type(record[0]) is not int or record[0] != 1: + raise ValueError("Invalid canonical record") + if not isinstance(record[1], bytes) or len(record[1]) != 64 or not isinstance(record[2], bytes) or len(record[2]) != 16: + raise ValueError("Invalid author or nonce") + if type(record[3]) is not int or not 1 <= record[3] <= 0xFFFFFFFF: + raise ValueError("Invalid revision") + if (record[3] == 1 and record[4] is not None) or (record[3] > 1 and (not isinstance(record[4], bytes) or len(record[4]) != 32)): + raise ValueError("Invalid predecessor") + for index, low, high in ((5, 0, 2), (6, -900000000, 900000000), (7, -1800000000, 1799999999), + (11, 2, 10), (12, 2, 10), (13, 0, 5), (14, 0, 253402300799), (15, 0, 253402300799)): + if type(record[index]) is not int or not low <= record[index] <= high: + raise ValueError("Record field out of range") + if record[15] < record[14]: + raise ValueError("Record timestamps reversed") + for index, limit in ((8, 96), (9, 2048), (10, 512)): + value = record[index] + if not isinstance(value, str) or len(value.encode("utf-8")) > limit or (index == 8 and not value.strip()): + raise ValueError("Invalid record text") + if any((ord(c) < 32 and not (index != 8 and c in "\t\n")) or 127 <= ord(c) <= 159 or ord(c) in (65534, 65535) for c in value): + raise ValueError("Forbidden record control character") + identity = RNS.Identity(create_keys=False) + identity.load_public_key(record[1]) + if not identity.validate(signature, b"trailmate.geocache/sign/v1\0" + raw): + raise ValueError("Invalid author signature") + cache_id = hashlib.sha256(b"trailmate.geocache/id/v1\0" + record[1] + record[2]).digest() + revision = hashlib.sha256(b"trailmate.geocache/revision/v1\0" + raw).digest() + summary = [cache_id, record[3], revision, record[5], record[6], record[7], record[8], record[11], record[12], record[13], len(packed(signed))] + return cache_id, revision, record, summary + + +def payload(message): + if not message.signature_validated or message.fields.get(0xFB) != APP_TYPE: + raise ValueError("Unauthenticated or unrelated LXMF message") + raw = message.fields.get(0xFC) + if not isinstance(raw, bytes) or len(raw) > 8192: + raise ValueError("Invalid application payload") + result = msgpack.unpackb(raw, raw=False) + if packed(result) != raw or not isinstance(result, list) or len(result) != 6: + raise ValueError("Noncanonical envelope") + if result[0] != 1 or not isinstance(result[3], bytes) or len(result[3]) != 16: + raise ValueError("Invalid request identity") + return result + + +class Peer: + def __init__(self, args): + self.args = args + self.state = Path(args.state).resolve() + self.state.mkdir(parents=True, exist_ok=True) + config_dir = self.state / "rns" + config_dir.mkdir(exist_ok=True) + interface = ( + f"type = TCPServerInterface\nlisten_ip = {args.host}\nlisten_port = {args.port}\n" + if args.mode == "directory" else + f"type = TCPClientInterface\ntarget_host = {args.host}\ntarget_port = {args.port}\n" + ) + config = "[reticulum]\nshare_instance = No\nenable_transport = Yes\n[interfaces]\n[[Interop TCP]]\nenabled = Yes\n" + interface + config_path = config_dir / "config" + if config_path.exists() and config_path.read_text(encoding="utf-8") != config: + raise ValueError("Use a fresh --state directory for a different test configuration") + config_path.write_text(config, encoding="utf-8") + self.rns = RNS.Reticulum(configdir=str(config_dir), loglevel=RNS.LOG_WARNING) + identity_path = self.state / "identity" + self.identity = RNS.Identity.from_file(str(identity_path)) if identity_path.exists() else RNS.Identity() + if not identity_path.exists(): + self.identity.to_file(str(identity_path)) + self.router = LXMF.LXMRouter(storagepath=str(self.state / "lxmf"), enforce_stamps=False) + self.delivery = self.router.register_delivery_identity(self.identity, display_name="Trail Mate interop peer", stamp_cost=None) + self.router.register_delivery_callback(self.received) + self.responses = queue.Queue(maxsize=4) + self.found = threading.Event() + self.remote = None + self.epoch = os.urandom(16) + self.directory = None + self.published = {} + self.history = {} + self.publication_replies = {} + if args.mode == "directory": + self.directory = RNS.Destination(self.identity, RNS.Destination.IN, RNS.Destination.SINGLE, + "trailmate", "geocache", "directory") + else: + RNS.Transport.register_announce_handler(self) + self.router.announce(self.delivery.hash) + + aspect_filter = ASPECT + + def received_announce(self, destination_hash, announced_identity, app_data): + try: + announcement = msgpack.unpackb(app_data, raw=False) + delivery = RNS.Destination(announced_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, "lxmf", "delivery") + if len(announcement) != 5 or announcement[0] != 1 or announcement[1] != delivery.hash: + return + self.remote = delivery + self.found.set() + except (ValueError, TypeError, IndexError): + return + + def send(self, destination, application): + message = LXMF.LXMessage(destination, self.delivery, "", "Geocaching interop", + fields={0xFB: APP_TYPE, 0xFC: packed(application)}, + desired_method=LXMF.LXMessage.DIRECT) + self.router.handle_outbound(message) + + def received(self, message): + try: + envelope = payload(message) + if self.args.mode != "directory": + if self.remote and message.source_hash == self.remote.hash: + (self.state / f"received-{envelope[2]}.lxmf").write_bytes(message.packed) + (self.state / f"received-{envelope[2]}.pub").write_bytes(message.get_source().identity.get_public_key()) + self.responses.put_nowait(envelope) + return + if envelope[1] != 0 or envelope[2] not in (0, 1, 2, 3): + return + if envelope[2] == 1: + if len(envelope[5]) != 1: + raise ValueError("Invalid publication body") + key = (message.source_hash, envelope[3]) + fingerprint = hashlib.sha256(packed(envelope)).digest() + prior = self.publication_replies.get(key) + if prior: + if fingerprint != prior[0]: + raise ValueError("Request ID reused for different publication") + reply = prior[1] + else: + signed = envelope[5][0] + cache_id, revision, record, summary = verified_cache(signed) + existing = self.published.get(cache_id) + duplicate = existing is not None and existing[1] == revision + if not existing and record[3] != 1: + raise ValueError("First version missing") + if existing and not duplicate: + predecessor = msgpack.unpackb(existing[0][0], raw=False) + if record[3] != predecessor[3] + 1 or record[4] != existing[1] or record[14] != predecessor[14] or record[15] < predecessor[15]: + raise ValueError("Invalid successor chain") + reply = [1, 1, 1, envelope[3], 200, [cache_id, record[3], revision, int(duplicate), record[3], revision, record[5]]] + if len(packed(reply)) > envelope[4] or (not existing and len(self.published) >= 8) or len(self.publication_replies) >= 32 or (revision not in self.history and len(self.history) >= 32): + raise ValueError("Test directory capacity/budget reached") + self.published[cache_id] = (signed, revision, summary) + self.history[revision] = self.published[cache_id] + self.publication_replies[key] = (fingerprint, reply) + self.send(message.get_source(), reply) + print(json.dumps({"event": "publication", "replayed": prior is not None}), flush=True) + return + filename = {0: "capabilities-response-v1.bin", 2: "query-response-v1.bin", 3: "get-response-v1.bin"}[envelope[2]] + reply = msgpack.unpackb((FIXTURES / filename).read_bytes(), raw=False) + reply = copy.deepcopy(reply) + reply[3] = envelope[3] + if envelope[2] == 2: + region, mask, author, limit, cursor = envelope[5] + south, west, north, east = region + rows = sorted(reply[5][1] + [entry[2] for entry in self.published.values()], key=lambda row: row[0]) + reply[5][1] = [row for row in rows if south <= row[4] <= north and west <= row[5] <= east + and mask & (1 << row[3])][:min(limit, 20)] if author is None and cursor is None else [] + reply[5][2] = None + if len(packed(reply)) > envelope[4]: + raise ValueError("Requested response budget is too small") + if envelope[2] == 3: + row = msgpack.unpackb((FIXTURES / "query-response-v1.bin").read_bytes(), raw=False)[5][1][0] + cache_id, wanted, known = envelope[5] + head = self.published.get(cache_id) + published = head if wanted is None else self.history.get(wanted) + if head and published and published[2][0] == cache_id and known is None: + reply[5] = [published[0], int(published[1] == head[1]), 0] + elif cache_id != row[0] or wanted not in (None, row[2]) or known is not None: + raise ValueError("This test peer only serves the fixture revision") + if len(packed(reply)) > envelope[4]: + raise ValueError("Requested response budget is too small") + self.send(message.get_source(), reply) + print(json.dumps({"event": "reply", "operation": envelope[2], "request": envelope[3].hex()}), flush=True) + except (ValueError, TypeError, IndexError, queue.Full) as error: + print(json.dumps({"event": "rejected", "reason": str(error)}), flush=True) + + def run_directory(self): + print(json.dumps({"event": "ready", "directory": self.directory.hash.hex(), + "delivery": self.delivery.hash.hex(), "interface": f"{self.args.host}:{self.args.port}", + "data": "checked-in test fixture, not a real treasure"}), flush=True) + deadline = time.monotonic() + self.args.seconds + while time.monotonic() < deadline: + self.router.announce(self.delivery.hash) + self.directory.announce(app_data=packed([1, self.delivery.hash, self.epoch, 1, "Trail Mate fixture directory"])) + time.sleep(min(3, max(0, deadline - time.monotonic()))) + + def exchange(self, operation, body, request=None): + request = request or os.urandom(16) + self.send(self.remote, [1, 0, operation, request, 2048, body]) + deadline = time.monotonic() + 20 + while time.monotonic() < deadline: + reply = self.responses.get(timeout=max(0.01, deadline - time.monotonic())) + if reply[3] != request: + continue # A retried transport response from a preceding step. + if reply[:3] != [1, 1, operation] or reply[4] != 200: + raise ValueError("Unexpected response") + return reply + raise TimeoutError("Application response not received") + + def run_publication_probe(self): + if not self.found.wait(20): + raise TimeoutError("Directory announcement was not received") + utc = int(time.time()) + record = [1, self.identity.get_public_key(), os.urandom(16), 1, None, 0, + 305000000, 1205000000, "Interop published cache", "Local test only", "", 2, 2, 0, utc, utc] + unsigned = packed(record) + signed = [unsigned, self.identity.sign(b"trailmate.geocache/sign/v1\0" + unsigned)] + cache_id, revision, _, _ = verified_cache(signed) + (self.state / "sent-signed.bin").write_bytes(packed(signed)) + self.exchange(0, []) + request = os.urandom(16) + accepted = self.exchange(1, [signed], request) + if accepted[5] != [cache_id, 1, revision, 0, 1, revision, 0]: + raise ValueError("Directory did not confirm the exact submitted version") + repeated = self.exchange(1, [signed], request) + if repeated != accepted: + raise ValueError("Request retry did not replay the original result") + query = self.exchange(2, [[-900000000, -1800000000, 900000000, 1800000000], 7, None, 20, None]) + if not any(row[0] == cache_id and row[2] == revision and row[6] == record[8] for row in query[5][1]): + raise ValueError("Published cache absent from subsequent query") + downloaded = self.exchange(3, [cache_id, revision, None]) + if downloaded[5] != [signed, 1, 0] or verified_cache(downloaded[5][0])[:2] != (cache_id, revision): + raise ValueError("Downloaded version differs from the publication") + (self.state / "sent-signed-v1.bin").write_bytes(packed(signed)) + successor = copy.deepcopy(record) + successor[3], successor[4] = 2, revision + successor[8] = "Interop published cache v2" + successor[15] = max(int(time.time()), utc) + raw = packed(successor) + signed_v2 = [raw, self.identity.sign(b"trailmate.geocache/sign/v1\0" + raw)] + same_id, hash_v2, _, _ = verified_cache(signed_v2) + if same_id != cache_id: + raise ValueError("Successor changed cache identity") + request_v2 = os.urandom(16) + accepted_v2 = self.exchange(1, [signed_v2], request_v2) + if accepted_v2[5] != [cache_id, 2, hash_v2, 0, 2, hash_v2, 0]: + raise ValueError("Successor was not accepted") + if self.exchange(1, [signed], request) != accepted: + raise ValueError("Head update changed an earlier request result") + if self.exchange(1, [signed_v2], request_v2) != accepted_v2: + raise ValueError("Successor request retry changed its result") + query = self.exchange(2, [[-900000000, -1800000000, 900000000, 1800000000], 7, None, 20, None]) + if not any(row[0] == cache_id and row[1] == 2 and row[2] == hash_v2 for row in query[5][1]): + raise ValueError("Query did not advance to successor") + historic = self.exchange(3, [cache_id, revision, None]) + if historic[5] != [signed, 0, 0]: + raise ValueError("Predecessor no longer retrievable") + for suffix in ("lxmf", "pub"): + (self.state / f"received-history.{suffix}").write_bytes((self.state / f"received-3.{suffix}").read_bytes()) + latest = self.exchange(3, [cache_id, hash_v2, None]) + if latest[5] != [signed_v2, 1, 0]: + raise ValueError("Successor download differs") + (self.state / "sent-signed.bin").write_bytes(packed(signed_v2)) + print(json.dumps({"event": "publication_verified", "retry": "same result", "query": "contains new cache", + "download": "same signed record", "successor": 2, "history": "v1 retained", "cache_id": cache_id.hex()}), flush=True) + + def run_probe(self): + if not self.found.wait(20): + raise TimeoutError("Directory announcement was not received") + operations = [(0, []), (2, [[-900000000, -1800000000, 900000000, 1800000000], 3, None, 20, None])] + for operation, body in operations: + request = os.urandom(16) + self.send(self.remote, [1, 0, operation, request, 2048, body]) + reply = self.responses.get(timeout=20) + if reply[:3] != [1, 1, operation] or reply[3] != request or reply[4] != 200: + raise ValueError("Unexpected response") + if operation == 2 and (len(reply[5][1]) != 1 or reply[5][1][0][6] != "Test"): + raise ValueError("Fixture cache was not returned") + if operation == 2: + row = reply[5][1][0] + operations.append((3, [row[0], row[2], None])) + if operation == 3 and (len(reply[5][0]) != 2 or len(reply[5][0][1]) != 64): + raise ValueError("Signed fixture record was not returned") + print(json.dumps({"event": "verified", "operation": operation, "signature": "valid", + "rows": len(reply[5][1]) if operation == 2 else None}), flush=True) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("mode", choices=("directory", "probe", "publish-probe")) + parser.add_argument("--state", required=True, help="Private test state directory; do not commit it") + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=44242) + parser.add_argument("--seconds", type=int, default=60, help="Directory lifetime for a bounded test") + args = parser.parse_args() + peer = Peer(args) + if args.mode == "directory": + peer.run_directory() + elif args.mode == "publish-probe": + peer.run_publication_probe() + else: + peer.run_probe() + + +if __name__ == "__main__": + main()