From 300a52df735205c79921337a5a94db6ae01a61aa Mon Sep 17 00:00:00 2001 From: shum Date: Thu, 29 Oct 2026 14:50:22 +0000 Subject: [PATCH] store: relay a names links Save as one transaction --- apps/simplex-store-service/README.md | 32 +- .../src/StoreService/Names/Records.hs | 96 +++--- .../src/StoreService/Names/Registration.hs | 5 + .../src/StoreService/Names/Store.hs | 107 ++++-- .../StoreService/Store/Postgres/Migrations.hs | 54 ++- .../StoreService/Store/SQLite/Migrations.hs | 54 ++- .../src/StoreService/Web/Names/Records.hs | 52 ++- tests/Bots/StoreService/NamesRecordsTests.hs | 326 +++++++++++++++--- tests/Bots/StoreService/NamesSearchTests.hs | 6 +- tests/Bots/StoreService/NamesStoreTests.hs | 104 ++++-- tests/Bots/StoreService/WebTests.hs | 2 +- 11 files changed, 666 insertions(+), 172 deletions(-) diff --git a/apps/simplex-store-service/README.md b/apps/simplex-store-service/README.md index 6dd3054904..752ddfb5b9 100644 --- a/apps/simplex-store-service/README.md +++ b/apps/simplex-store-service/README.md @@ -910,31 +910,33 @@ ini and a restart, with the same build. Codes for investors come from the group ### Link edits -Each "Edit links" save is relayed by the service as `setTextWithSig` (or `clearRecordsWithSig`), a -transaction whose gas it pays (see [The gas key](#the-gas-key)). `[names] record_edits_per_year` -(default 3, from 1 to 100, read at startup) bounds them per name in any 365 days: +Each "Edit links" save is one request, relayed by the service as one transaction whose gas it pays +(see [The gas key](#the-gas-key)): one changed link as `setTextWithSig` (or `clearRecordsWithSig`), +both links as the resolver's `multicall` of two `setTextWithSig` calls, which lands both or neither. +`[names] record_edits_per_year` (default 3, from 1 to 100, read at startup) bounds them per name in +any 365 days: -- each accepted request counts once, from when it is stored to be sent: one changed link, or a clear - of both. The page sends one request per changed link, so saving both links takes two; +- each accepted request counts once, from when it is stored to be sent, whether it changes one link + or both, or clears them; - links set at registration ride on the register transaction and count nothing; - a request refused before any transaction, a dry run that reverts, a send the chain fails, and an edit that reverts on chain (or expires) count nothing, the last once it settles. A pending edit counts until then; -- an exact resend of an accepted edit is answered as before and counts nothing more. +- an exact resend of an accepted request is answered as before and counts nothing more. When a name has none left, `POST /api/name/records` answers `429 {"error": "edit_limit", "nextAt": }` before any chain read, and the search answer for the name carries `recordEdits: {"left": 0, "nextAt": …}`, from which the page shows when the owner can edit again. Lowering the setting keeps the edits already counted. The hourly cap of 10 -relayed edits per name still applies, and still counts reverted edits. +relayed requests per name still applies, and still counts reverted ones. ### The gas key `[chain] service_key` is the registrar key: an Ethereum account (EOA) that sends every names transaction, `commit` to the controller, `registerWithCredit` to the sponsor, and the owners' record -edits (`setTextWithSig`, `clearRecordsWithSig`) to the resolver. It pays only their gas, in ETH. The -names' rent is spent from its allowance on the sponsor, in USD, which the guardian sets. It holds no -names. +edits (`setTextWithSig`, `clearRecordsWithSig`, or a `multicall` of two) to the resolver. It pays +only their gas, in ETH. The names' rent is spent from its allowance on the sponsor, in USD, which the +guardian sets. It holds no names. The commands run from `scripts/store-service/` on the service host, with Foundry's `cast` ([getfoundry.sh](https://getfoundry.sh), checked with 1.5.1) and `jq`. They read the ini through this @@ -1066,15 +1068,19 @@ The gas below was measured on a local anvil deployment of the contracts (the nam | `setTextWithSig`, a link replacing one | 462,289 | 462,289 | 0.0462 ETH | | `setTextWithSig`, a link removed | 698,644 | 558,916 | 0.0699 ETH | | `clearRecordsWithSig` | 80,312 | 80,312 | 0.0080 ETH | +| `multicall`, two new links | 6,054,042 | 6,008,868 | 0.6054 ETH | - The service sets each transaction's gas limit to its estimate and its max fee to at most `max_fee_per_gas` (default 100 gwei). The node accepts a transaction only while the balance covers its limit × max fee, so the last column is what each one needs held. What it is charged is gas used × the block's base fee plus tip, usually far less. - An order is one commit and one register: 0.6850 ETH at the cap with two links, 0.0627 ETH with - none. The expected orders × 0.6850 ETH, plus 0.3026 ETH per expected link edit, pays for all of - them even at the cap. A name's link edits are at most `record_edits_per_year` (3) a year, see - [Link edits](#link-edits). The figures scale with `max_fee_per_gas`. + none. The expected orders × 0.6850 ETH, plus 0.6054 ETH per expected Save of both links (0.3026 ETH + for one), pays for all of them even at the cap. A name's link edits are at most + `record_edits_per_year` (3) a year, see [Link edits](#link-edits). The figures scale with + `max_fee_per_gas`. +- A Save of both links in one `multicall` uses 26,906 gas less than the same two edits sent as two + transactions with links of 4096 bytes, and 28,082 less with short links. - `gas_limit_cap` (default 8,000,000) must stay above 6,791,417: a transaction whose estimate is above it is not sent. diff --git a/apps/simplex-store-service/src/StoreService/Names/Records.hs b/apps/simplex-store-service/src/StoreService/Names/Records.hs index a5b774651a..2b35f60101 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Records.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Records.hs @@ -1,9 +1,10 @@ {-# LANGUAGE LambdaCase #-} +{-# LANGUAGE MultiWayIf #-} {-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TupleSections #-} --- | Record edits signed by a name's owner, relayed through the registrar key's sender and followed to their outcome. +-- | Record edits signed by a name's owner, each request relayed as one transaction through the registrar key's sender and followed to its outcome. module StoreService.Names.Records ( Records (rsPendingAfter, rsEditsPerYear), EditRequest (..), @@ -24,7 +25,10 @@ import Control.Monad (forM, forM_, forever, unless, void, when) import Control.Monad.Except (ExceptT (..), runExceptT) import Control.Monad.IO.Class (liftIO) import Data.ByteString (ByteString) +import qualified Data.ByteString as B import Data.Int (Int64) +import Data.List.NonEmpty (NonEmpty (..)) +import qualified Data.List.NonEmpty as L import Data.Maybe (listToMaybe) import Data.Set (Set) import qualified Data.Set as S @@ -35,7 +39,6 @@ import Data.Word (Word64) import Simplex.Messaging.Agent.Store.Common (DBStore, withTransaction) import qualified Simplex.Messaging.Agent.Store.DB as DB import Simplex.Messaging.Crypto (sha256Hash) -import Simplex.Messaging.Crypto.Secp256k1 (RecoverableSignature) import Simplex.Messaging.Eth.Address (Address, recoverAddress) import Simplex.Messaging.Eth.EIP712 (Eip712Domain, serializeSignature) import Simplex.Messaging.SimplexName (LabelHash, SimplexTLD) @@ -44,7 +47,7 @@ import StoreService.Config (ChainConfig (..)) import StoreService.Log (logWarn, safely) import StoreService.Names.Chain import StoreService.Names.Contracts (Contracts, NameStatus (..), RegisteredName (..), nameStatus, resolverNonce, textRecord) -import StoreService.Names.Registration (Node, channelRecordKey, clearRecordsDigest, clearRecordsWithSigCalldata, contactRecordKey, labelNode, resolverDomain, setTextDigest, setTextWithSigCalldata) +import StoreService.Names.Registration (Node, channelRecordKey, clearRecordsDigest, clearRecordsWithSigCalldata, contactRecordKey, labelNode, multicallCalldata, resolverDomain, setTextDigest, setTextWithSigCalldata) import StoreService.Names.Revert (RevertName (..), revertText) import StoreService.Names.Sender (SendError (..), Sender, submitTransaction) import StoreService.Names.Store @@ -69,12 +72,10 @@ data Records = Records rsChanges :: TVar Int } +-- | The changes of one Save, in nonce order, relayed together or not at all. data EditRequest = EditRequest { erLabelHash :: LabelHash, - erChange :: RecordChange, - erNonce :: Word64, - erDeadline :: Word64, - erSignature :: RecoverableSignature + erChanges :: NonEmpty SignedChange } deriving (Eq, Show) @@ -85,7 +86,7 @@ data EditAnswer EAReverted (Maybe Text) | -- | not settled within the wait, or the chain stopped answering after the send; the edit goes on EAPending - | -- | the signer, the deadline, the owner or the nonce + | -- | the signers, the deadline, the owner or the nonce EARefused | -- | the chain failed before anything was sent, and nothing is stored EAUnavailable @@ -147,20 +148,21 @@ usedUpUntil EditsLeft {elLeft, elNextAt} = if elLeft == 0 then elNextAt else Not -- | An identical repeat of a sent edit is answered from it, and the yearly limit is checked, before any chain read. -- The wait for an outcome is bounded from the request's arrival, so the checks and the send count toward it. relayEdit :: Records -> EditRequest -> IO EditAnswer -relayEdit rs@Records {rsPendingAfter, rsEditsPerYear, rsNow} req@EditRequest {erLabelHash, erSignature} = do +relayEdit rs@Records {rsPendingAfter, rsEditsPerYear, rsNow} req@EditRequest {erLabelHash, erChanges} = do timer <- registerDelay rsPendingAfter - recoverAddress erSignature digest >>= \case - Left _ -> pure EARefused - Right signer -> + sequence <$> mapM (\(digest, SignedChange {scSignature}) -> recoverAddress scSignature digest) signed >>= \case + Right (signer :| others) | all (== signer) others -> withStore rs (`getRecordEditByBody` bodyHash) >>= \case Just RecordEdit {reId} -> awaitEdit rs timer reId Nothing -> do now <- rsNow withStore rs (\db -> usedUpUntil <$> nameEditsLeft db rsEditsPerYear erLabelHash now) >>= maybe (newEdit rs timer req signer bodyHash) (pure . EALimited) + _ -> pure EARefused where - digest = editDigest rs req - bodyHash = sha256Hash $ digest <> serializeSignature erSignature + signed = L.map (\change -> (changeDigest rs erLabelHash change, change)) erChanges + -- one change hashes as edits stored before requests took two, so their repeats still match + bodyHash = sha256Hash $ B.concat [digest <> serializeSignature scSignature | (digest, SignedChange {scSignature}) <- L.toList signed] newEdit :: Records -> TVar Bool -> EditRequest -> Address -> ByteString -> IO EditAnswer newEdit rs timer req signer bodyHash = @@ -174,22 +176,22 @@ newEdit rs timer req signer bodyHash = -- a pending or mined edit holds the nonce: the identical twin is waited for, any other body refused Right Nothing -> withStore rs (`getRecordEditByBody` bodyHash) >>= maybe (pure EARefused) (awaitEdit rs timer . reId) --- | The deadline after the latest block's time, the signer the registry's owner on the default resolver, the nonce the owner's next. +-- | The earliest deadline after the latest block's time, the signer the registry's owner on the default resolver, the first nonce the owner's next. chainAccepts :: Records -> EditRequest -> Address -> IO (Either ChainError Bool) -chainAccepts Records {rsChain, rsContracts} EditRequest {erLabelHash, erNonce, erDeadline} signer = runExceptT $ do +chainAccepts Records {rsChain, rsContracts} EditRequest {erLabelHash, erChanges} signer = runExceptT $ do Block {blockTimestamp} <- ExceptT $ latestBlock rsChain - if erDeadline <= blockTimestamp + if earliestDeadline erChanges <= blockTimestamp then pure False else ExceptT (nameStatus rsContracts erLabelHash) >>= \case NSRegistered RegisteredName {rnOwner, rnResolver} - | rnOwner == signer && rnResolver == cResolver (ccConfig rsChain) -> (== erNonce) <$> ExceptT (resolverNonce rsContracts signer) + | rnOwner == signer && rnResolver == cResolver (ccConfig rsChain) -> (== firstNonce erChanges) <$> ExceptT (resolverNonce rsContracts signer) _ -> pure False -- | Left when the name's yearly limit or hourly cap is used up, checked again under the name's turn against a concurrent edit. --- Right Nothing when a pending or mined edit holds the nonce. +-- Right Nothing when a pending or mined edit holds the first nonce. insertEdit :: Records -> EditRequest -> Address -> ByteString -> IO (Either EditAnswer (Maybe Int64)) -insertEdit Records {rsStore, rsTurns, rsEditsPerYear, rsNow} EditRequest {erLabelHash, erChange, erNonce, erDeadline, erSignature} signer bodyHash = +insertEdit Records {rsStore, rsTurns, rsEditsPerYear, rsNow} EditRequest {erLabelHash, erChanges} signer bodyHash = withLabelTurn rsTurns erLabelHash $ do now <- rsNow withTransaction rsStore $ \db -> do @@ -202,7 +204,7 @@ insertEdit Records {rsStore, rsTurns, rsEditsPerYear, rsNow} EditRequest {erLabe | relayed >= editCap -> pure $ Left $ EACapped $ max 1 $ ceiling $ diffUTCTime (addUTCTime editCapWindow $ times !! (relayed - editCap)) now | otherwise -> Right <$> insertRecordEdit db edit now where - edit = NewRecordEdit {nreLabelHash = erLabelHash, nreOwner = signer, nreNonce = erNonce, nreChange = erChange, nreDeadline = erDeadline, nreSignature = erSignature, nreBodyHash = bodyHash} + edit = NewRecordEdit {nreLabelHash = erLabelHash, nreOwner = signer, nreChanges = erChanges, nreBodyHash = bodyHash} -- | An edit that was not sent is answered and deleted while still claimed, so the lane cannot send it after. sendNew :: Records -> TVar Bool -> Int64 -> IO EditAnswer @@ -278,7 +280,7 @@ deepBlock c tip@Block {blockNumber} -- | One step for a pending edit, from its transactions' statuses as the sender settled them. follow :: Records -> Block -> Block -> RecordEdit -> IO () -follow rs@Records {rsChain} Block {blockTimestamp} Block {blockTimestamp = deepTime} edit@RecordEdit {reId, reDeadline} = do +follow rs@Records {rsChain} Block {blockTimestamp} Block {blockTimestamp = deepTime} edit@RecordEdit {reId, reChanges} = do txs <- withStore rs (`editTransactions` reId) case txs of [] -> if ahead then sendUnsent else forget rs reId @@ -293,6 +295,7 @@ follow rs@Records {rsChain} Block {blockTimestamp} Block {blockTimestamp = deepT -- every transaction dropped: its registrar nonce went to another one | otherwise -> judged rs edit $ if ahead then resend else expire where + reDeadline = earliestDeadline reChanges ahead = reDeadline > blockTimestamp expire = judged rs edit $ settle rs reId $ ESReverted signatureExpired sendUnsent = @@ -314,23 +317,26 @@ judged rs edit@RecordEdit {reId} whenFree = Right NonceTaken -> settle rs reId $ ESReverted $ revertText RNInvalidNonce Right NonceFree -> whenFree --- | Its own transactions first: a status-1 receipt answers its hash. Without one, the owner's nonce and the record. +-- | Its own transactions first: a status-1 receipt answers its hash. Without one, the owner's nonce and the records. +-- A nonce past the first change but not the last means another body took one of its nonces. judge :: Records -> RecordEdit -> IO (Either ChainError Judgment) -judge rs@Records {rsChain, rsContracts} RecordEdit {reId, reLabelHash, reOwner, reNonce, reChange} = runExceptT $ do +judge rs@Records {rsChain, rsContracts} RecordEdit {reId, reLabelHash, reOwner, reChanges} = runExceptT $ do txs <- liftIO $ withStore rs (`editTransactions` reId) receipts <- forM txs $ \NameTransaction {ntHash} -> (ntHash,) <$> ExceptT (transactionReceipt rsChain ntHash) case [h | (h, Just Receipt {receiptSuccess = True}) <- receipts] of h : _ -> pure $ Applied $ Just h [] -> do nonce <- ExceptT $ resolverNonce rsContracts reOwner - if nonce <= reNonce - then pure NonceFree - else do - holds <- case reChange of - RCSetText key value -> (== value) <$> record (encodeUtf8 key) - RCClear -> (&&) <$> (unset <$> record contactRecordKey) <*> (unset <$> record channelRecordKey) - pure $ if holds then Applied Nothing else NonceTaken + if + | nonce <= firstNonce reChanges -> pure NonceFree + | nonce <= lastNonce reChanges -> pure NonceTaken + | otherwise -> do + holds <- and <$> mapM (holdsChange . scChange) reChanges + pure $ if holds then Applied Nothing else NonceTaken where + holdsChange = \case + RCSetText key value -> (== value) <$> record (encodeUtf8 key) + RCClear -> (&&) <$> (unset <$> record contactRecordKey) <*> (unset <$> record channelRecordKey) record = ExceptT . textRecord rsContracts reLabelHash unset = (== "") @@ -366,19 +372,31 @@ withClaim Records {rsClaimed} editId fallback action = if S.member editId held then pure False else True <$ writeTVar rsClaimed (S.insert editId held) release = atomically $ modifyTVar' rsClaimed $ S.delete editId -editDigest :: Records -> EditRequest -> ByteString -editDigest rs EditRequest {erLabelHash, erChange, erNonce, erDeadline} = case erChange of - RCSetText key value -> setTextDigest (domain rs) node key value erNonce erDeadline - RCClear -> clearRecordsDigest (domain rs) node erNonce erDeadline +changeDigest :: Records -> LabelHash -> SignedChange -> ByteString +changeDigest rs lh SignedChange {scChange, scNonce, scDeadline} = case scChange of + RCSetText key value -> setTextDigest (domain rs) node key value scNonce scDeadline + RCClear -> clearRecordsDigest (domain rs) node scNonce scDeadline where - node = nodeOf rs erLabelHash + node = nodeOf rs lh +-- | One change is sent as its own call, which keeps the revert's name; two go in one multicall. editCalldata :: Records -> RecordEdit -> ByteString -editCalldata rs RecordEdit {reLabelHash, reChange, reNonce, reDeadline, reSignature} = case reChange of - RCSetText key value -> setTextWithSigCalldata node key value reNonce reDeadline reSignature - RCClear -> clearRecordsWithSigCalldata node reNonce reDeadline reSignature +editCalldata rs RecordEdit {reLabelHash, reChanges} = case reChanges of + change :| [] -> changeCalldata change + _ -> multicallCalldata $ map changeCalldata $ L.toList reChanges where node = nodeOf rs reLabelHash + changeCalldata SignedChange {scChange, scNonce, scDeadline, scSignature} = case scChange of + RCSetText key value -> setTextWithSigCalldata node key value scNonce scDeadline scSignature + RCClear -> clearRecordsWithSigCalldata node scNonce scDeadline scSignature + +firstNonce, lastNonce :: NonEmpty SignedChange -> Word64 +firstNonce = scNonce . L.head +lastNonce = scNonce . L.last + +-- | Past it the edit cannot land whole, so it is SignatureExpired. +earliestDeadline :: NonEmpty SignedChange -> Word64 +earliestDeadline = minimum . L.map scDeadline nodeOf :: Records -> LabelHash -> Node nodeOf Records {rsTLD} = labelNode rsTLD diff --git a/apps/simplex-store-service/src/StoreService/Names/Registration.hs b/apps/simplex-store-service/src/StoreService/Names/Registration.hs index 313f90ee9c..8d986bf5c1 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Registration.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Registration.hs @@ -21,6 +21,7 @@ module StoreService.Names.Registration clearRecordsDigest, setTextWithSigCalldata, clearRecordsWithSigCalldata, + multicallCalldata, ) where @@ -143,6 +144,10 @@ clearRecordsWithSigCalldata (Node node) nonce deadline sig = selector "clearRecordsWithSig(bytes32,uint256,uint256,bytes)" <> abiTuple [Static node, Static $ uint256 nonce, Static $ uint256 deadline, Dynamic $ abiBytes $ serializeSignature sig] +-- | The resolver runs each call on itself in order, and reverts all of them when one fails. +multicallCalldata :: [ByteString] -> ByteString +multicallCalldata calls = selector "multicall(bytes[])" <> abiTuple [Dynamic $ abiArray $ map abiBytes calls] + bytes32 :: Node -> Eip712Value bytes32 (Node node) = VUint $ fromInteger $ os2ip node diff --git a/apps/simplex-store-service/src/StoreService/Names/Store.hs b/apps/simplex-store-service/src/StoreService/Names/Store.hs index dd3e5680bd..9709a32076 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Store.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Store.hs @@ -34,6 +34,7 @@ module StoreService.Names.Store NameTransaction (..), NewTransaction (..), RecordChange (..), + SignedChange (..), EditState (..), RecordEdit (..), NewRecordEdit (..), @@ -107,6 +108,8 @@ import Data.ByteArray.Encoding (Base (Base16), convertToBase) import Data.ByteString (ByteString) import qualified Data.ByteString as B import Data.Int (Int64) +import Data.List.NonEmpty (NonEmpty (..)) +import qualified Data.List.NonEmpty as L import Data.Maybe (isJust, listToMaybe) import Data.String (fromString) import Data.Text (Text) @@ -446,14 +449,21 @@ data RecordChange = RCSetText Text Text | RCClear data EditState = ESPending | ESMined (Maybe TxHash) | ESReverted (Maybe Text) deriving (Eq, Show) +-- | A change the owner signed for the resolver's nonce it names. +data SignedChange = SignedChange + { scChange :: RecordChange, + scNonce :: Word64, + scDeadline :: Word64, + scSignature :: RecoverableSignature + } + deriving (Eq, Show) + +-- | One request's changes, relayed in one transaction, their nonces consecutive in this order. data RecordEdit = RecordEdit { reId :: Int64, reLabelHash :: LabelHash, reOwner :: Address, - reNonce :: Word64, - reChange :: RecordChange, - reDeadline :: Word64, - reSignature :: RecoverableSignature, + reChanges :: NonEmpty SignedChange, reBodyHash :: ByteString, reState :: EditState, reCreatedAt :: UTCTime @@ -463,10 +473,7 @@ data RecordEdit = RecordEdit data NewRecordEdit = NewRecordEdit { nreLabelHash :: LabelHash, nreOwner :: Address, - nreNonce :: Word64, - nreChange :: RecordChange, - nreDeadline :: Word64, - nreSignature :: RecoverableSignature, + nreChanges :: NonEmpty SignedChange, nreBodyHash :: ByteString } @@ -1415,69 +1422,95 @@ minedEdit = "mined" revertedEdit :: Text revertedEdit = "reverted" --- | Nothing when a pending or mined edit already holds the owner's nonce. +-- | Nothing when a pending or mined edit already starts at the owner's first nonce. insertRecordEdit :: DB.Connection -> NewRecordEdit -> UTCTime -> IO (Maybe Int64) -insertRecordEdit db NewRecordEdit {nreLabelHash = LabelHash lh, nreOwner, nreNonce, nreChange, nreDeadline, nreSignature, nreBodyHash} now = do - let (key, value) = case nreChange of - RCSetText k v -> (Just k, Just v) - RCClear -> (Nothing, Nothing) +insertRecordEdit db NewRecordEdit {nreLabelHash = LabelHash lh, nreOwner, nreChanges, nreBodyHash} now = do inserted <- executeChanging db [sql| - INSERT INTO sx_badge_service_name_record_edits - (label_hash, owner, nonce, record_key, record_value, deadline, signature, body_hash, state, created_at) - VALUES (?,?,?,?,?,?,?,?,?,?) + INSERT INTO sx_badge_service_name_record_edits (label_hash, owner, nonce, body_hash, state, created_at) + VALUES (?,?,?,?,?,?) ON CONFLICT DO NOTHING |] - ( (Binary lh, Binary $ unAddress nreOwner, int64 nreNonce, key, value, int64 nreDeadline, Binary $ serializeSignature nreSignature, Binary nreBodyHash) - :. (pendingEdit, truncateToSecond now) - ) - if inserted > 0 then Just <$> insertedRowId db else pure Nothing + (Binary lh, Binary $ unAddress nreOwner, int64 $ scNonce $ L.head nreChanges, Binary nreBodyHash, pendingEdit, truncateToSecond now) + if inserted > 0 + then do + editId <- insertedRowId db + mapM_ (insertChange editId) nreChanges + pure $ Just editId + else pure Nothing + where + insertChange editId SignedChange {scChange, scNonce, scDeadline, scSignature} = + DB.execute + db + "INSERT INTO sx_badge_service_name_record_changes (name_record_edit_id, nonce, record_key, record_value, deadline, signature) VALUES (?,?,?,?,?,?)" + (editId, int64 scNonce, key, value, int64 scDeadline, Binary $ serializeSignature scSignature) + where + (key, value) = case scChange of + RCSetText k v -> (Just k, Just v) + RCClear -> (Nothing, Nothing) recordEditColumns :: Query recordEditColumns = [sql| - SELECT name_record_edit_id, label_hash, owner, nonce, record_key, record_value, deadline, signature, body_hash, state, revert, mined_tx, created_at + SELECT name_record_edit_id, label_hash, owner, body_hash, state, revert, mined_tx, created_at FROM sx_badge_service_name_record_edits |] -type RecordEditRow = - (Int64, Binary ByteString, Binary ByteString, Int64, Maybe Text, Maybe Text, Int64, Binary ByteString, Binary ByteString, Text) - :. (Maybe Text, Maybe (Binary ByteString), UTCTime) +type RecordEditRow = (Int64, Binary ByteString, Binary ByteString, Binary ByteString, Text, Maybe Text, Maybe (Binary ByteString), UTCTime) -toRecordEdit :: RecordEditRow -> Either Text RecordEdit -toRecordEdit ((reId, Binary lh, Binary owner, nonce, key_, value_, deadline, Binary sig, Binary reBodyHash, state) :. (revert, minedTx, reCreatedAt)) = do +type ChangeRow = (Int64, Maybe Text, Maybe Text, Int64, Binary ByteString) + +-- | An edit's row with its changes, in nonce order. +readRecordEdit :: DB.Connection -> RecordEditRow -> IO RecordEdit +readRecordEdit db row@(reId, _, _, _, _, _, _, _) = do + changes <- DB.query db "SELECT nonce, record_key, record_value, deadline, signature FROM sx_badge_service_name_record_changes WHERE name_record_edit_id = ? ORDER BY nonce" (Only reId) + either (E.throwIO . StoreDecodeError) pure $ toRecordEdit row changes + +toRecordEdit :: RecordEditRow -> [ChangeRow] -> Either Text RecordEdit +toRecordEdit (reId, Binary lh, Binary owner, Binary reBodyHash, state, revert, minedTx, reCreatedAt) changeRows = do reOwner <- decodeAddress owner - reSignature <- decodeSignature sig - reChange <- case (key_, value_) of - (Just key, Just value) -> Right $ RCSetText key value - (Nothing, Nothing) -> Right RCClear - _ -> Left $ "name_record_edits record: " <> tshow reId + reChanges <- note ("name_record_edits changes: " <> tshow reId) . L.nonEmpty =<< mapM toChange changeRows reState <- decodeState - pure RecordEdit {reId, reLabelHash = LabelHash lh, reOwner, reNonce = word64 nonce, reChange, reDeadline = word64 deadline, reSignature, reBodyHash, reState, reCreatedAt} + pure RecordEdit {reId, reLabelHash = LabelHash lh, reOwner, reChanges, reBodyHash, reState, reCreatedAt} where + toChange (nonce, key_, value_, deadline, Binary sig) = do + scSignature <- decodeSignature sig + scChange <- case (key_, value_) of + (Just key, Just value) -> Right $ RCSetText key value + (Nothing, Nothing) -> Right RCClear + _ -> Left $ "name_record_changes record: " <> tshow reId + pure SignedChange {scChange, scNonce = word64 nonce, scDeadline = word64 deadline, scSignature} decodeState | state == pendingEdit = Right ESPending | state == minedEdit = Right $ ESMined $ txHash <$> minedTx | state == revertedEdit = Right $ ESReverted revert | otherwise = Left $ "name_record_edits.state: " <> state +firstRecordEdit :: DB.Connection -> IO [RecordEditRow] -> IO (Maybe RecordEdit) +firstRecordEdit db rows = rows >>= mapM (readRecordEdit db) . listToMaybe + getRecordEdit :: DB.Connection -> Int64 -> IO (Maybe RecordEdit) getRecordEdit db editId = - firstDecoded toRecordEdit $ DB.query db (recordEditColumns <> " WHERE name_record_edit_id = ?") (Only editId) + firstRecordEdit db $ DB.query db (recordEditColumns <> " WHERE name_record_edit_id = ?") (Only editId) -- | The newest edit sent with this signed body. getRecordEditByBody :: DB.Connection -> ByteString -> IO (Maybe RecordEdit) getRecordEditByBody db bodyHash = - firstDecoded toRecordEdit $ + firstRecordEdit db $ DB.query db (recordEditColumns <> " WHERE body_hash = ? ORDER BY name_record_edit_id DESC LIMIT 1") (Only $ Binary bodyHash) --- | The pending or mined edit holding the owner's nonce. +-- | The pending or mined edit holding the owner's nonce among its changes. getRecordEditByNonce :: DB.Connection -> Address -> Word64 -> IO (Maybe RecordEdit) getRecordEditByNonce db owner nonce = - firstDecoded toRecordEdit $ - DB.query db (recordEditColumns <> " WHERE owner = ? AND nonce = ? AND state IN (?,?)") (Binary $ unAddress owner, int64 nonce, pendingEdit, minedEdit) + firstRecordEdit db $ + DB.query + db + ( recordEditColumns + <> " WHERE owner = ? AND state IN (?,?) AND name_record_edit_id IN (SELECT name_record_edit_id FROM sx_badge_service_name_record_changes WHERE nonce = ?)" + ) + (Binary $ unAddress owner, pendingEdit, minedEdit, int64 nonce) -- | Settles a pending edit; False when it was settled already. settleRecordEdit :: DB.Connection -> Int64 -> EditState -> IO Bool diff --git a/apps/simplex-store-service/src/StoreService/Store/Postgres/Migrations.hs b/apps/simplex-store-service/src/StoreService/Store/Postgres/Migrations.hs index 74014fc6a4..cc05857906 100644 --- a/apps/simplex-store-service/src/StoreService/Store/Postgres/Migrations.hs +++ b/apps/simplex-store-service/src/StoreService/Store/Postgres/Migrations.hs @@ -26,7 +26,8 @@ schemaMigrations = ("20261022_name_balances", m20261022_name_balances, Just down_m20261022_name_balances), ("20261022_name_late_wait", m20261022_name_late_wait, Just down_m20261022_name_late_wait), ("20261023_name_terms", m20261023_name_terms, Just down_m20261023_name_terms), - ("20261027_name_invoiced_units", m20261027_name_invoiced_units, Just down_m20261027_name_invoiced_units) + ("20261027_name_invoiced_units", m20261027_name_invoiced_units, Just down_m20261027_name_invoiced_units), + ("20261029_name_record_changes", m20261029_name_record_changes, Just down_m20261029_name_record_changes) ] -- | The client tables share this database, so the service tables are the same names behind a prefix. @@ -620,6 +621,57 @@ DROP INDEX @idx_payments_overpaid; ALTER TABLE @payments DROP COLUMN invoiced_units; |] +-- An edit is one request of the page, relayed as one transaction; its signed changes are its rows here, one per nonce. +m20261029_name_record_changes :: Text +m20261029_name_record_changes = + withPrefix + servicePrefix + [r| +CREATE TABLE @name_record_changes( + name_record_edit_id BIGINT NOT NULL REFERENCES @name_record_edits ON DELETE CASCADE, + nonce BIGINT NOT NULL, + record_key TEXT, + record_value TEXT, + deadline BIGINT NOT NULL, + signature BYTEA NOT NULL, + PRIMARY KEY (name_record_edit_id, nonce) +); + +INSERT INTO @name_record_changes (name_record_edit_id, nonce, record_key, record_value, deadline, signature) +SELECT name_record_edit_id, nonce, record_key, record_value, deadline, signature FROM @name_record_edits; + +ALTER TABLE @name_record_edits DROP COLUMN record_key; + +ALTER TABLE @name_record_edits DROP COLUMN record_value; + +ALTER TABLE @name_record_edits DROP COLUMN deadline; + +ALTER TABLE @name_record_edits DROP COLUMN signature; +|] + +-- An edit of two changes keeps only its first, the one its nonce names. +down_m20261029_name_record_changes :: Text +down_m20261029_name_record_changes = + withPrefix + servicePrefix + [r| +ALTER TABLE @name_record_edits ADD COLUMN record_key TEXT; + +ALTER TABLE @name_record_edits ADD COLUMN record_value TEXT; + +ALTER TABLE @name_record_edits ADD COLUMN deadline BIGINT NOT NULL DEFAULT 0; + +ALTER TABLE @name_record_edits ADD COLUMN signature BYTEA NOT NULL DEFAULT ''; + +UPDATE @name_record_edits SET + record_key = (SELECT c.record_key FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + record_value = (SELECT c.record_value FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + deadline = (SELECT c.deadline FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + signature = (SELECT c.signature FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce); + +DROP TABLE @name_record_changes; +|] + {- TODO [badges] deferred with the draft in M20260915_user_badges, service only. ALTER TABLE @payments ADD COLUMN receipt_hash BYTEA; diff --git a/apps/simplex-store-service/src/StoreService/Store/SQLite/Migrations.hs b/apps/simplex-store-service/src/StoreService/Store/SQLite/Migrations.hs index 922fec7431..ae29f59256 100644 --- a/apps/simplex-store-service/src/StoreService/Store/SQLite/Migrations.hs +++ b/apps/simplex-store-service/src/StoreService/Store/SQLite/Migrations.hs @@ -27,7 +27,8 @@ schemaMigrations = ("20261022_name_balances", m20261022_name_balances, Just down_m20261022_name_balances), ("20261022_name_late_wait", m20261022_name_late_wait, Just down_m20261022_name_late_wait), ("20261023_name_terms", m20261023_name_terms, Just down_m20261023_name_terms), - ("20261027_name_invoiced_units", m20261027_name_invoiced_units, Just down_m20261027_name_invoiced_units) + ("20261027_name_invoiced_units", m20261027_name_invoiced_units, Just down_m20261027_name_invoiced_units), + ("20261029_name_record_changes", m20261029_name_record_changes, Just down_m20261029_name_record_changes) ] -- | The client tables share this database, so the service tables are the same names behind a prefix. @@ -622,6 +623,57 @@ DROP INDEX @idx_payments_overpaid; ALTER TABLE @payments DROP COLUMN invoiced_units; |] +-- An edit is one request of the page, relayed as one transaction; its signed changes are its rows here, one per nonce. +m20261029_name_record_changes :: Query +m20261029_name_record_changes = + withPrefix + servicePrefix + [sql| +CREATE TABLE @name_record_changes( + name_record_edit_id INTEGER NOT NULL REFERENCES @name_record_edits ON DELETE CASCADE, + nonce INTEGER NOT NULL, + record_key TEXT, + record_value TEXT, + deadline INTEGER NOT NULL, + signature BLOB NOT NULL, + PRIMARY KEY (name_record_edit_id, nonce) +) STRICT; + +INSERT INTO @name_record_changes (name_record_edit_id, nonce, record_key, record_value, deadline, signature) +SELECT name_record_edit_id, nonce, record_key, record_value, deadline, signature FROM @name_record_edits; + +ALTER TABLE @name_record_edits DROP COLUMN record_key; + +ALTER TABLE @name_record_edits DROP COLUMN record_value; + +ALTER TABLE @name_record_edits DROP COLUMN deadline; + +ALTER TABLE @name_record_edits DROP COLUMN signature; +|] + +-- An edit of two changes keeps only its first, the one its nonce names. +down_m20261029_name_record_changes :: Query +down_m20261029_name_record_changes = + withPrefix + servicePrefix + [sql| +ALTER TABLE @name_record_edits ADD COLUMN record_key TEXT; + +ALTER TABLE @name_record_edits ADD COLUMN record_value TEXT; + +ALTER TABLE @name_record_edits ADD COLUMN deadline INTEGER NOT NULL DEFAULT 0; + +ALTER TABLE @name_record_edits ADD COLUMN signature BLOB NOT NULL DEFAULT x''; + +UPDATE @name_record_edits SET + record_key = (SELECT c.record_key FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + record_value = (SELECT c.record_value FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + deadline = (SELECT c.deadline FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce), + signature = (SELECT c.signature FROM @name_record_changes c WHERE c.name_record_edit_id = @name_record_edits.name_record_edit_id AND c.nonce = @name_record_edits.nonce); + +DROP TABLE @name_record_changes; +|] + {- TODO [badges] deferred with the draft in M20260915_user_badges, service only. ALTER TABLE @payments ADD COLUMN receipt_hash BLOB; diff --git a/apps/simplex-store-service/src/StoreService/Web/Names/Records.hs b/apps/simplex-store-service/src/StoreService/Web/Names/Records.hs index cf4762df9a..6010553fd6 100644 --- a/apps/simplex-store-service/src/StoreService/Web/Names/Records.hs +++ b/apps/simplex-store-service/src/StoreService/Web/Names/Records.hs @@ -1,17 +1,22 @@ {-# LANGUAGE LambdaCase #-} {-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE TupleSections #-} module StoreService.Web.Names.Records (EditInvalid (..), recordsHandler, editRequest, editsLeftJSON) where -import Control.Monad (unless) +import Control.Monad (forM_, unless) import Data.Aeson ((.:), (.:?), (.=)) import qualified Data.Aeson as J import qualified Data.Aeson.Types as JT +import Data.List (nub) +import qualified Data.List.NonEmpty as L +import Data.Maybe (isJust) import Data.Text (Text) import Data.Text.Encoding (decodeLatin1) import Data.Time.Clock (UTCTime) import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds) +import Data.Word (Word64) import Network.HTTP.Types (status200, status409, status429, status504) import Network.Wai (Request, Response, ResponseReceived) import Simplex.Messaging.SimplexName (LabelHash (..)) @@ -21,39 +26,52 @@ import StoreService.Names.Links (LinkKind (..), linkKindOf) import StoreService.Names.Records (EditAnswer (..), EditRequest (..), EditsLeft (..), Records, relayEdit) import StoreService.Names.Registration (channelRecordKey, contactRecordKey) import StoreService.Names.Signatures (parseSignature) -import StoreService.Names.Store (RecordChange (..)) -import StoreService.Web.Http (Respond, badRequest, invalidLink, jsonError, jsonResponse, maxBodyBytes, providerUnavailable, rateLimited, readBoundedBody) +import StoreService.Names.Store (RecordChange (..), SignedChange (..)) +import StoreService.Web.Http (Respond, badRequest, invalidLink, jsonError, jsonResponse, nameBodyBytes, providerUnavailable, rateLimited, readBoundedBody) data EditInvalid = EIBadRequest | EIInvalidLink deriving (Eq, Show) recordsHandler :: Records -> Request -> Respond -> IO ResponseReceived recordsHandler rs req respond = - readBoundedBody maxBodyBytes req >>= \case + readBoundedBody nameBodyBytes req >>= \case Just body | Just o <- J.decode body -> case editRequest o of Right edit -> relayEdit rs edit >>= respond . editResponse Left EIInvalidLink -> respond invalidLink Left EIBadRequest -> respond badRequest _ -> respond badRequest --- | The formats and the key, then the link, then the signature's format and low s, as the page's checks run. +-- | An edit as sent, before its key, link and signature are checked. +data SentEdit = SentEdit {seChange :: Maybe (Text, Text), seNonce :: Word64, seDeadline :: Word64, seSignature :: Text} + +-- | One link of each kind, so a Save of both links is the most one request carries. +maxEdits :: Int +maxEdits = 2 + +-- | Every edit's format and key, then every link, then every signature's format and low s, as the page's checks run. editRequest :: J.Object -> Either EditInvalid EditRequest editRequest o = do - (labelHash, change, erNonce, erDeadline, signature) <- maybe (Left EIBadRequest) Right $ JT.parseMaybe fields o + (labelHash, edits) <- maybe (Left EIBadRequest) Right $ JT.parseMaybe fields o erLabelHash <- maybe (Left EIBadRequest) (Right . LabelHash) $ bytes32FromHex labelHash - erChange <- case change of - Nothing -> Right RCClear - Just (key, value) -> do - kind <- maybe (Left EIBadRequest) Right $ lookup key recordKinds - unless (value == "" || linkKindOf value == Just kind) $ Left EIInvalidLink - Right $ RCSetText key value - erSignature <- either (const $ Left EIBadRequest) Right $ parseSignature signature - pure EditRequest {erLabelHash, erChange, erNonce, erDeadline, erSignature} + sent <- maybe (Left EIBadRequest) Right $ L.nonEmpty edits + unless (length edits <= maxEdits && keysOnce edits && clearAlone edits && consecutive edits) $ Left EIBadRequest + links <- mapM (\SentEdit {seChange} -> mapM linkOf seChange) sent + forM_ links $ mapM_ $ \(kind, value) -> unless (value == "" || linkKindOf value == Just kind) $ Left EIInvalidLink + erChanges <- mapM signedChange sent + pure EditRequest {erLabelHash, erChanges} where - fields v = do + fields v = (,) <$> v .: "labelHash" <*> (v .: "edits" >>= mapM (J.withObject "edit" sentEdit)) + sentEdit v = do clear <- v .:? "clear" - change <- if clear == Just True then pure Nothing else Just <$> ((,) <$> v .: "key" <*> v .: "value") - (,,,,) <$> v .: "labelHash" <*> pure change <*> v .: "nonce" <*> v .: "deadline" <*> v .: "signature" + seChange <- if clear == Just True then pure Nothing else Just <$> ((,) <$> v .: "key" <*> v .: "value") + SentEdit seChange <$> v .: "nonce" <*> v .: "deadline" <*> v .: "signature" + keysOnce edits = let keys = [k | SentEdit {seChange = Just (k, _)} <- edits] in length keys == length (nub keys) + clearAlone edits = length edits == 1 || all (isJust . seChange) edits + consecutive edits = and $ zipWith (\a b -> seNonce b == seNonce a + 1) edits (drop 1 edits) + linkOf (key, value) = maybe (Left EIBadRequest) (Right . (,value)) $ lookup key recordKinds + signedChange SentEdit {seChange, seNonce, seDeadline, seSignature} = do + scSignature <- either (const $ Left EIBadRequest) Right $ parseSignature seSignature + pure SignedChange {scChange = maybe RCClear (uncurry RCSetText) seChange, scNonce = seNonce, scDeadline = seDeadline, scSignature} recordKinds :: [(Text, LinkKind)] recordKinds = [(decodeLatin1 contactRecordKey, LKContact), (decodeLatin1 channelRecordKey, LKChannel)] diff --git a/tests/Bots/StoreService/NamesRecordsTests.hs b/tests/Bots/StoreService/NamesRecordsTests.hs index 3e1a2f5e1c..8f686ae6c6 100644 --- a/tests/Bots/StoreService/NamesRecordsTests.hs +++ b/tests/Bots/StoreService/NamesRecordsTests.hs @@ -21,6 +21,8 @@ import qualified Data.Aeson.KeyMap as KM import Data.ByteArray.Encoding (Base (Base16), convertFromBase) import Data.ByteString (ByteString) import qualified Data.ByteString as B +import Data.List.NonEmpty (NonEmpty (..)) +import qualified Data.List.NonEmpty as L import qualified Data.ByteString.Lazy as LB import Data.Text (Text) import qualified Data.Text as T @@ -45,10 +47,10 @@ import StoreService.Names.Abi import StoreService.Names.Chain import StoreService.Names.Contracts (newContracts) import StoreService.Names.Records -import StoreService.Names.Registration (Node, clearRecordsDigest, clearRecordsWithSigCalldata, labelNode, nodeBytes, resolverDomain, setTextDigest, setTextWithSigCalldata) +import StoreService.Names.Registration (Node, clearRecordsDigest, clearRecordsWithSigCalldata, labelNode, multicallCalldata, nodeBytes, resolverDomain, setTextDigest, setTextWithSigCalldata) import StoreService.Names.Sender (Sender, followTransactions, newSender) import StoreService.Names.Signatures (parseSignature) -import StoreService.Names.Store (EditState (..), NewRecordEdit (..), RecordChange (..), RecordEdit (..), getRecordEditByBody, insertRecordEdit) +import StoreService.Names.Store (EditState (..), NewRecordEdit (..), RecordChange (..), RecordEdit (..), SignedChange (..), getRecordEditByBody, insertRecordEdit) import StoreService.Names.Turns (newTurns) import StoreService.Web.Names.Records (EditInvalid (..), editRequest, recordsHandler) import System.Timeout (timeout) @@ -66,7 +68,9 @@ namesRecordsTests = describe "StoreService names: records" $ do g <- runIO C.newRandom describe "requests" $ do it "signs and encodes SetText and ClearRecords edits as viem does" $ testViemVectors g + it "encodes two edits as one multicall as viem does" $ testViemMulticall g it "reads a request's formats, then its link, then its signature" testRequestOrder + it "takes one or two edits of distinct keys with consecutive nonces, and a clear only alone" testRequestEdits it "bounds a link at 4096 bytes and takes only a link of the key's kind" testLinkBound it "refuses a high s, though recovery accepts it" testHighS describe "relay" $ do @@ -84,6 +88,17 @@ namesRecordsTests = describe "StoreService names: records" $ do it "answers edit_pending after the wait, and its repeat with the hash once mined" $ testPendingThenMined g it "answers edit_pending at once when the chain stops answering after the send" $ testChainStopsAfterSend g it "bounds the wait for edit_pending from the request's arrival, counting the checks and the send" $ testPendingFromArrival g + describe "two edits" $ do + it "relays two edits as one multicall transaction with one row, and answers its hash once mined" $ testTwoMined g + it "refuses two edits when the second is signed by another key, or its nonce is not the owner's next, before any row or gas" $ testTwoRefused g + it "answers an exact resend of two edits from the stored request, and checks either edit alone anew" $ testTwoResend g + it "counts two edits as one edit of the yearly limit, taking the last one left" $ testTwoYearLimit g + it "counts two edits as one request of the hourly cap" $ testTwoCap g + it "answers a mined revert of two edits record_reverted with no name, giving its edit back" $ testTwoReverted g + it "stores nothing when the dry run of two edits reverts, answering it with no name" $ testTwoDryRunReverted g + it "settles two edits SignatureExpired once the earliest deadline is passed at depth" $ testTwoExpired g + it "answers two edits applied with no hash only when the nonce passed both and both records hold their values" $ testTwoAppliedByAnother g + it "answers two edits record_reverted InvalidNonce when another body took only the first nonce" $ testTwoFirstNonceTaken g describe "outcomes" $ do it "answers an InvalidNonce revert of an edit another relayer applied as mined with no hash, and its repeat too" $ testAppliedByAnother g it "answers a dry run's InvalidNonce revert of an edit another relayer applied as applied with no hash, storing nothing" $ testDryRunAppliedByAnother g @@ -109,6 +124,7 @@ namesRecordsTests = describe "StoreService names: records" $ do it "answers the contract's statuses and bodies" $ testHttp g it "answers 503, 504, and 429 with the real wait in Retry-After" $ testHttpUnsettled g it "answers 429 edit_limit with nextAt in unix seconds and no Retry-After" $ testHttpEditLimit g + it "relays a body of two edits with links of 4096 bytes, above the 8192-byte cap of other endpoints" $ testHttpTwoLongLinks g data Setup = Setup { node :: FakeEthNode, @@ -168,10 +184,27 @@ setRecord Setup {node} key value = answerCall node fakeResolver (selector "text(bytes32,string)" <> abiTuple [Static $ nodeBytes aliceNode, Dynamic $ abiBytes $ encodeUtf8 key]) $ Returns $ abiTuple [Dynamic $ abiBytes $ encodeUtf8 value] +signChange :: Setup -> EphemeralWallet -> RecordChange -> Word64 -> Word64 -> IO SignedChange +signChange Setup {random} EphemeralWallet {ewKey} scChange scNonce scDeadline = do + scSignature <- signRecoverable random ewKey (digestOf scChange scNonce scDeadline) >>= either fail pure + pure SignedChange {scChange, scNonce, scDeadline, scSignature} + +-- | A request of one edit. signEdit :: Setup -> EphemeralWallet -> RecordChange -> Word64 -> Word64 -> IO EditRequest -signEdit Setup {random} EphemeralWallet {ewKey} change nonce erDeadline = do - sig <- signRecoverable random ewKey (digestOf change nonce erDeadline) >>= either fail pure - pure EditRequest {erLabelHash = alice, erChange = change, erNonce = nonce, erDeadline, erSignature = sig} +signEdit s w change nonce dl = editOf . (:| []) <$> signChange s w change nonce dl + +editOf :: NonEmpty SignedChange -> EditRequest +editOf erChanges = EditRequest {erLabelHash = alice, erChanges} + +-- | The owner's contact and channel edits, at this nonce and the next, in one request. +ownerTwo :: Setup -> Word64 -> Text -> Text -> IO EditRequest +ownerTwo s@Setup {owner} nonce contactValue channelValue = do + first <- signChange s owner (contact contactValue) nonce deadline + second <- signChange s owner (channel channelValue) (nonce + 1) deadline + pure $ editOf $ first :| [second] + +contact :: Text -> RecordChange +contact = RCSetText contactKey channel :: Text -> RecordChange channel = RCSetText channelKey @@ -208,16 +241,17 @@ editRows Setup {store} = withTransaction store $ \db -> do [Only n] <- DB.query_ db "SELECT count(*) FROM sx_badge_service_name_record_edits" pure n --- | As the relay keys a sent edit: the signed digest and the signature. +-- | As the relay keys a sent request: each signed digest and its signature, in order. bodyHashOf :: EditRequest -> ByteString -bodyHashOf EditRequest {erChange, erNonce, erDeadline, erSignature} = C.sha256Hash $ digestOf erChange erNonce erDeadline <> serializeSignature erSignature +bodyHashOf EditRequest {erChanges} = + C.sha256Hash $ B.concat [digestOf scChange scNonce scDeadline <> serializeSignature scSignature | SignedChange {scChange, scNonce, scDeadline, scSignature} <- L.toList erChanges] -- | Saves an edit as a request does before it joins the sender, as if the service stopped right after. saveUnsent :: Setup -> EditRequest -> IO () -saveUnsent Setup {store, clock, owner = EphemeralWallet {ewAddress}} req@EditRequest {erChange, erNonce, erDeadline, erSignature} = do +saveUnsent Setup {store, clock, owner = EphemeralWallet {ewAddress}} req@EditRequest {erChanges} = do now <- readTVarIO clock Just _ <- withTransaction store $ \db -> - insertRecordEdit db NewRecordEdit {nreLabelHash = alice, nreOwner = ewAddress, nreNonce = erNonce, nreChange = erChange, nreDeadline = erDeadline, nreSignature = erSignature, nreBodyHash = bodyHashOf req} now + insertRecordEdit db NewRecordEdit {nreLabelHash = alice, nreOwner = ewAddress, nreChanges = erChanges, nreBodyHash = bodyHashOf req} now pure () passTime :: Setup -> NominalDiffTime -> IO () @@ -228,38 +262,79 @@ passTime Setup {clock} t = atomically $ modifyTVar' clock (addUTCTime t) testViemVectors :: TVar ChaChaDRG -> IO () testViemVectors g = withSetup g $ \s@Setup {owner = EphemeralWallet {ewAddress}} -> do ewAddress `shouldBe` viemAddress - EditRequest {erSignature = setSig} <- ownerEdit s viemChannelLink + SignedChange {scSignature = setSig} <- signChange s (owner s) (channel viemChannelLink) 0 deadline setSig `shouldBe` viemSetSignature recoverAddress setSig (digestOf (channel viemChannelLink) 0 deadline) `shouldReturn` Right viemAddress setTextWithSigCalldata aliceNode channelKey viemChannelLink 0 deadline setSig `shouldBe` viemSetCall - EditRequest {erSignature = clearSig} <- signEdit s (owner s) RCClear 1 deadline + SignedChange {scSignature = clearSig} <- signChange s (owner s) RCClear 1 deadline clearSig `shouldBe` viemClearSignature clearRecordsWithSigCalldata aliceNode 1 deadline clearSig `shouldBe` viemClearCall +testViemMulticall :: TVar ChaChaDRG -> IO () +testViemMulticall g = withSetup g $ \s -> do + EditRequest {erChanges = first :| rest} <- ownerTwo s 0 viemContactLink viemChannelLink + [second] <- pure rest + (scSignature first, scSignature second) `shouldBe` (viemContactSignature, viemChannelSignature) + multicallCalldata + [ setTextWithSigCalldata aliceNode contactKey viemContactLink 0 deadline (scSignature first), + setTextWithSigCalldata aliceNode channelKey viemChannelLink 1 deadline (scSignature second) + ] + `shouldBe` viemMulticall + testRequestOrder :: IO () testRequestOrder = do - let valid = requestJson [("key", J.String channelKey), ("value", J.String viemChannelLink)] - fmap erChange (editRequest valid) `shouldBe` Right (channel viemChannelLink) - fmap erChange (editRequest $ requestJson [("clear", J.Bool True)]) `shouldBe` Right RCClear - fmap erChange (editRequest $ requestJson [("key", J.String contactKey), ("value", "")]) `shouldBe` Right (RCSetText contactKey "") + let valid = editFields [("key", J.String channelKey), ("value", J.String viemChannelLink)] + request = editRequest . requestOf . (: []) + changesOf (request valid) `shouldBe` Right [channel viemChannelLink] + changesOf (request $ editFields [("clear", J.Bool True)]) `shouldBe` Right [RCClear] + changesOf (request $ editFields [("key", J.String contactKey), ("value", "")]) `shouldBe` Right [RCSetText contactKey ""] -- formats first: a bad label hash or key is bad_request even with a bad link - editRequest (KM.insert "labelHash" (J.String $ "0x" <> T.toUpper (T.drop 2 aliceHashText)) $ withLink "https://x/a") `shouldBe` Left EIBadRequest - editRequest (requestJson [("key", "simplex.other"), ("value", J.String viemChannelLink)]) `shouldBe` Left EIBadRequest - editRequest (KM.insert "nonce" (J.Number (-1)) valid) `shouldBe` Left EIBadRequest - editRequest (KM.insert "deadline" (J.String "1767229200") valid) `shouldBe` Left EIBadRequest - editRequest (KM.delete "value" valid) `shouldBe` Left EIBadRequest + editRequest (KM.insert "labelHash" (J.String $ "0x" <> T.toUpper (T.drop 2 aliceHashText)) $ requestOf [withLink "https://x/a"]) `shouldBe` Left EIBadRequest + request (editFields [("key", "simplex.other"), ("value", J.String viemChannelLink)]) `shouldBe` Left EIBadRequest + request (KM.insert "nonce" (J.Number (-1)) valid) `shouldBe` Left EIBadRequest + request (KM.insert "deadline" (J.String "1767229200") valid) `shouldBe` Left EIBadRequest + request (KM.delete "value" valid) `shouldBe` Left EIBadRequest + -- the old body of one edit, with no list + editRequest (KM.insert "labelHash" (J.String aliceHashText) valid) `shouldBe` Left EIBadRequest -- the link before the signature - editRequest (KM.insert "signature" "0x00" $ withLink "https://x/a") `shouldBe` Left EIInvalidLink - editRequest (KM.insert "signature" "0x00" valid) `shouldBe` Left EIBadRequest + request (KM.insert "signature" "0x00" $ withLink "https://x/a") `shouldBe` Left EIInvalidLink + request (KM.insert "signature" "0x00" valid) `shouldBe` Left EIBadRequest where - withLink link = requestJson [("key", J.String channelKey), ("value", J.String link)] + withLink link = editFields [("key", J.String channelKey), ("value", J.String link)] + +testRequestEdits :: IO () +testRequestEdits = do + let edit key value nonce = editFields [("key", J.String key), ("value", J.String value), ("nonce", J.toJSON (nonce :: Int))] + clearAt nonce = editFields [("clear", J.Bool True), ("nonce", J.toJSON (nonce :: Int))] + request = editRequest . requestOf + contactAt = edit contactKey "https://smp8.simplex.im/a#Y8m3" + channelAt = edit channelKey viemChannelLink + changesOf (request [contactAt 4, channelAt 5]) `shouldBe` Right [contact "https://smp8.simplex.im/a#Y8m3", channel viemChannelLink] + fmap (map scNonce . L.toList . erChanges) (request [channelAt 4, contactAt 5]) `shouldBe` Right [4, 5] + forM_ + [ ("no edit", []), + ("three edits", [contactAt 4, channelAt 5, edit channelKey "" 6]), + ("one key twice", [channelAt 4, edit channelKey "" 5]), + ("a clear after a set", [contactAt 4, clearAt 5]), + ("a clear before a set", [clearAt 4, contactAt 5]), + ("a nonce skipped", [contactAt 4, channelAt 6]), + ("nonces reversed", [contactAt 5, channelAt 4]), + ("one nonce twice", [contactAt 4, channelAt 4]) + ] + $ \(what, edits) -> (what :: String, request edits) `shouldBe` (what, Left EIBadRequest) + -- every edit's format before any link, and every link before any signature + request [edit contactKey "https://x/a" 4, edit "simplex.other" viemChannelLink 5] `shouldBe` Left EIBadRequest + request [KM.insert "signature" "0x00" $ contactAt 4, edit channelKey "https://x/c" 5] `shouldBe` Left EIInvalidLink + +changesOf :: Either EditInvalid EditRequest -> Either EditInvalid [RecordChange] +changesOf = fmap (map scChange . L.toList . erChanges) testLinkBound :: IO () testLinkBound = do let prefix = "https://smp8.simplex.im/c#" linkOf n = prefix <> T.replicate (n - T.length prefix) "a" - edit key value = editRequest $ requestJson [("key", J.String key), ("value", J.String value)] - fmap erChange (edit channelKey $ linkOf 4096) `shouldBe` Right (channel $ linkOf 4096) + edit key value = editRequest $ requestOf [editFields [("key", J.String key), ("value", J.String value)]] + fmap (L.head . L.map scChange . erChanges) (edit channelKey $ linkOf 4096) `shouldBe` Right (channel $ linkOf 4096) edit channelKey (linkOf 4097) `shouldBe` Left EIInvalidLink edit contactKey viemChannelLink `shouldBe` Left EIInvalidLink edit channelKey "https://smp8.simplex.im/c#Y8m3;https://smp9.simplex.im/c#Z1" `shouldBe` Left EIInvalidLink @@ -270,7 +345,7 @@ testHighS = do highS = i2ospOf_ 32 $ curveOrder - os2ip rsS flipped = RecoverableSignature {rsR, rsS = highS, rsRecId = 1 - rsRecId} recoverAddress flipped (digestOf (channel viemChannelLink) 0 deadline) `shouldReturn` Right viemAddress - editRequest (KM.insert "signature" (J.String $ hex $ serializeSignature flipped) $ requestJson [("key", J.String channelKey), ("value", J.String viemChannelLink)]) + editRequest (requestOf [KM.insert "signature" (J.String $ hex $ serializeSignature flipped) $ editFields [("key", J.String channelKey), ("value", J.String viemChannelLink)]]) `shouldBe` Left EIBadRequest where curveOrder = ECC.ecc_n $ ECC.common_curve $ ECC.getCurveByName ECC.SEC_p256k1 @@ -443,6 +518,143 @@ testChainStopsAfterSend g = withSetup g $ \s@Setup {node, records} -> do h <- mineSent s 1 True relayEdit records req `shouldReturn` EAApplied (Just h) +-- two edits + +testTwoMined :: TVar ChaChaDRG -> IO () +testTwoMined g = withSetup g $ \s@Setup {node} -> do + req <- ownerTwo s 0 viemContactLink viemChannelLink + r <- relay s req + [raw] <- within $ awaitSent node 1 + fmap (\TxRequest {trTo, trData} -> (trTo, trData)) (decodeSignedTx raw) `shouldBe` Right (fakeResolver, viemMulticall) + h <- mineSent s 1 True + answerOf r `shouldReturn` EAApplied (Just h) + stateOf s req `shouldReturn` Just (ESMined $ Just h) + editRows s `shouldReturn` 1 + length <$> sentTransactions node `shouldReturn` 1 + +testTwoRefused :: TVar ChaChaDRG -> IO () +testTwoRefused g = withSetup g $ \s@Setup {node, records, owner} -> do + other <- newEphemeralWallet g + first <- signChange s owner (contact viemContactLink) 0 deadline + stranger <- signChange s other (channel viemChannelLink) 1 deadline + relayEdit records (editOf $ first :| [stranger]) `shouldReturn` EARefused + passed <- signChange s owner (channel viemChannelLink) 1 (genesis - 1) + relayEdit records (editOf $ first :| [passed]) `shouldReturn` EARefused + setNonce s 1 + (relayEdit records =<< ownerTwo s 0 viemContactLink viemChannelLink) `shouldReturn` EARefused + editRows s `shouldReturn` 0 + sentTransactions node `shouldReturn` [] + +testTwoResend :: TVar ChaChaDRG -> IO () +testTwoResend g = withSetup g $ \s@Setup {node, records} -> do + req@EditRequest {erChanges = first :| rest} <- ownerTwo s 0 viemContactLink viemChannelLink + r <- relay s req + h <- mineSent s 1 True + answerOf r `shouldReturn` EAApplied (Just h) + relayEdit records req `shouldReturn` EAApplied (Just h) + editsLeftNow s `shouldReturn` EditsLeft (editsPerYear - 1) (Just $ addUTCTime year t0) + -- either edit alone is another body, checked anew against the nonce both moved + setNonce s 2 + relayEdit records (editOf $ first :| []) `shouldReturn` EARefused + relayEdit records (editOf $ L.fromList rest) `shouldReturn` EARefused + editRows s `shouldReturn` 1 + length <$> sentTransactions node `shouldReturn` 1 + +testTwoYearLimit :: TVar ChaChaDRG -> IO () +testTwoYearLimit g = withSetup g $ \s@Setup {records} -> do + _ <- appliedEdits s 2 + setNonce s 2 + editsLeftNow s `shouldReturn` EditsLeft 1 (Just $ addUTCTime year t0) + r <- relay s =<< ownerTwo s 2 viemContactLink viemChannelLink + h <- mineSent s 3 True + answerOf r `shouldReturn` EAApplied (Just h) + setNonce s 4 + editsLeftNow s `shouldReturn` EditsLeft 0 (Just $ addUTCTime year t0) + (relayEdit records =<< ownerTwo s 4 "https://smp8.simplex.im/a#next" "https://smp9.simplex.im/c#next") `shouldReturn` EALimited (addUTCTime year t0) + editRows s `shouldReturn` 3 + +testTwoCap :: TVar ChaChaDRG -> IO () +testTwoCap g = withSetup g $ \s0 -> do + let s = roomy s0 + quick = (records s) {rsPendingAfter = shortWait} + twoAt n = ownerTwo s n viemContactLink ("https://smp9.simplex.im/c#n" <> T.pack (show n)) + forM_ [0, 2 .. 18] $ \n -> do + setNonce s n + (relayEdit quick =<< twoAt n) `shouldReturn` EAPending + setNonce s 20 + (relayEdit (records s) =<< twoAt 20) `shouldReturn` EACapped 3600 + editRows s `shouldReturn` 10 + +testTwoReverted :: TVar ChaChaDRG -> IO () +testTwoReverted g = withSetup g $ \s@Setup {records} -> do + req <- ownerTwo s 0 viemContactLink viemChannelLink + r <- relay s req + -- reverted before the deadlines with the nonce unused: multicall drops the inner error + _ <- mineSent s 1 False + answerOf r `shouldReturn` EAReverted Nothing + stateOf s req `shouldReturn` Just (ESReverted Nothing) + editsLeftNow s `shouldReturn` EditsLeft editsPerYear Nothing + relayEdit records req `shouldReturn` EAReverted Nothing + +testTwoDryRunReverted :: TVar ChaChaDRG -> IO () +testTwoDryRunReverted g = withSetup g $ \s@Setup {node, records} -> do + req <- ownerTwo s 0 viemContactLink viemChannelLink + revertDryRun node viemMulticall $ Just "" + relayEdit records req `shouldReturn` EAReverted Nothing + editRows s `shouldReturn` 0 + sentTransactions node `shouldReturn` [] + revertDryRun node viemMulticall Nothing + r <- relay s req + h <- mineSent s 1 True + answerOf r `shouldReturn` EAApplied (Just h) + +testTwoExpired :: TVar ChaChaDRG -> IO () +testTwoExpired g = withSetup g $ \s@Setup {node, records, owner} -> do + first <- signChange s owner (contact viemContactLink) 0 (deadline + 600) + second <- signChange s owner (channel viemChannelLink) 1 deadline + let req = editOf $ first :| [second] + relayEdit records {rsPendingAfter = shortWait} req `shouldReturn` EAPending + _ <- within $ awaitSent node 1 + -- the second edit's deadline, the earlier one, is passed at the latest block but not yet two confirmations deep + setBlock node 300 deadline + setBlock node 301 $ deadline + 1 + recordsRound records + stateOf s req `shouldReturn` Just ESPending + setBlock node 302 $ deadline + 2 + recordsRound records + stateOf s req `shouldReturn` Just (ESReverted $ Just "SignatureExpired") + editsLeftNow s `shouldReturn` EditsLeft editsPerYear Nothing + +testTwoAppliedByAnother :: TVar ChaChaDRG -> IO () +testTwoAppliedByAnother g = + forM_ [(viemChannelLink, EAApplied Nothing), ("https://smp9.simplex.im/c#other", EAReverted $ Just "InvalidNonce")] $ \(held, answer) -> withSetup g $ \s@Setup {node} -> do + req <- ownerTwo s 0 viemContactLink viemChannelLink + r <- relay s req + _ <- within $ awaitSent node 1 + -- another relayer's transaction used both nonces first + setNonce s 2 + setRecord s contactKey viemContactLink + setRecord s channelKey held + _ <- mineSent s 1 False + answerOf r `shouldReturn` answer + +testTwoFirstNonceTaken :: TVar ChaChaDRG -> IO () +testTwoFirstNonceTaken g = withSetup g $ \s@Setup {node} -> do + req <- ownerTwo s 0 viemContactLink viemChannelLink + r <- relay s req + _ <- within $ awaitSent node 1 + -- the records happen to hold both values, but only the first nonce moved, so the request cannot land + setNonce s 1 + setRecord s contactKey viemContactLink + setRecord s channelKey viemChannelLink + _ <- mineSent s 1 False + answerOf r `shouldReturn` EAReverted (Just "InvalidNonce") + stateOf s req `shouldReturn` Just (ESReverted $ Just "InvalidNonce") + +-- | The name's yearly edits left now, by the relay's clock. +editsLeftNow :: Setup -> IO EditsLeft +editsLeftNow Setup {store, clock} = readTVarIO clock >>= \now -> withTransaction store $ \db -> nameEditsLeft db editsPerYear alice now + -- outcomes testAppliedByAnother :: TVar ChaChaDRG -> IO () @@ -577,11 +789,15 @@ testBadRowSkipped :: TVar ChaChaDRG -> IO () testBadRowSkipped g = withSetup g $ \s@Setup {store, clock} -> do now <- readTVarIO clock -- a text record with a key and no value, which no edit can be - withTransaction store $ \db -> + withTransaction store $ \db -> do DB.execute db - "INSERT INTO sx_badge_service_name_record_edits (label_hash, owner, nonce, record_key, record_value, deadline, signature, body_hash, state, created_at) VALUES (?,?,5,'simplex.channel',NULL,?,?,?,'pending',?)" - (Binary $ unLabel alice, Binary $ unAddress viemAddress, deadline, Binary $ serializeSignature viemSetSignature, Binary $ B.replicate 32 0, now) + "INSERT INTO sx_badge_service_name_record_edits (label_hash, owner, nonce, body_hash, state, created_at) VALUES (?,?,5,?,'pending',?)" + (Binary $ unLabel alice, Binary $ unAddress viemAddress, Binary $ B.replicate 32 0, now) + DB.execute + db + "INSERT INTO sx_badge_service_name_record_changes (name_record_edit_id, nonce, record_key, record_value, deadline, signature) SELECT name_record_edit_id, 5, 'simplex.channel', NULL, ?, ? FROM sx_badge_service_name_record_edits WHERE nonce = 5" + (deadline, Binary $ serializeSignature viemSetSignature) req <- ownerEdit s viemChannelLink saveUnsent s req s' <- restart s @@ -753,7 +969,7 @@ t0Seconds = floor $ utcTimeToPOSIXSeconds t0 testHttp :: TVar ChaChaDRG -> IO () testHttp g = withSetup g $ \s@Setup {records} -> withRecordsApp records $ \post -> do - let editBody value = J.encode $ KM.insert "value" (J.String value) $ requestJson [("key", J.String channelKey)] + let editBody value = J.encode $ requestOf [editFields [("key", J.String channelKey), ("value", J.String value)]] post "[]" `shouldReturn` (400, Just $ errorJson "bad_request", Nothing) post (editBody "https://x/a") `shouldReturn` (400, Just $ errorJson "invalid_link", Nothing) answer <- async $ post $ editBody viemChannelLink @@ -787,6 +1003,15 @@ testHttpEditLimit g = withSetup g $ \s@Setup {records, owner} -> do (post . J.encode . editJson =<< signEdit s owner (channel viemChannelLink) 3 deadline) `shouldReturn` (429, Just $ J.object ["error" .= ("edit_limit" :: Text), "nextAt" .= (t0Seconds + yearSeconds)], Nothing) +testHttpTwoLongLinks :: TVar ChaChaDRG -> IO () +testHttpTwoLongLinks g = withSetup g $ \s@Setup {records} -> withRecordsApp records $ \post -> do + let longLink prefix = prefix <> T.replicate (4096 - T.length prefix) "a" + body <- J.encode . editJson <$> ownerTwo s 0 (longLink "https://smp8.simplex.im/a#") (longLink "https://smp8.simplex.im/c#") + LB.length body `shouldSatisfy` (> 8192) + answer <- async $ post body + TxHash h <- mineSent s 1 True + within (wait answer) `shouldReturn` (200, Just $ J.object ["tx" .= hex h], Nothing) + -- | Posts a body to the records handler, answering the status, the JSON body and Retry-After. withRecordsApp :: Records -> ((LB.ByteString -> IO (Int, Maybe J.Value, Maybe ByteString)) -> IO a) -> IO a withRecordsApp rs action = Warp.testWithApplication (pure $ recordsHandler rs) $ \port -> do @@ -802,17 +1027,23 @@ errorJson tag = J.object ["error" .= tag] -- fixtures editJson :: EditRequest -> J.Object -editJson EditRequest {erChange, erNonce, erDeadline, erSignature} = +editJson EditRequest {erChanges} = requestOf $ map changeJson $ L.toList erChanges + +changeJson :: SignedChange -> J.Object +changeJson SignedChange {scChange, scNonce, scDeadline, scSignature} = KM.fromList $ - [ ("labelHash", J.String aliceHashText), - ("nonce", J.toJSON erNonce), - ("deadline", J.toJSON erDeadline), - ("signature", J.String $ hex $ serializeSignature erSignature) + [ ("nonce", J.toJSON scNonce), + ("deadline", J.toJSON scDeadline), + ("signature", J.String $ hex $ serializeSignature scSignature) ] - <> case erChange of + <> case scChange of RCSetText key value -> [("key", J.String key), ("value", J.String value)] RCClear -> [("clear", J.Bool True)] +-- | The body of a request with these edits for alicebakery. +requestOf :: [J.Object] -> J.Object +requestOf edits = KM.fromList [("labelHash", J.String aliceHashText), ("edits", J.toJSON edits)] + -- | The service's default. editsPerYear :: Int editsPerYear = 3 @@ -822,11 +1053,11 @@ longWait = 4000000 shortWait = 200000 -- | The fields of viem's SetText edit but its key and value, with these added or replaced. -requestJson :: [(J.Key, J.Value)] -> J.Object -requestJson extra = KM.union (KM.fromList extra) $ KM.delete "key" $ KM.delete "value" $ editJson viemEdit +editFields :: [(J.Key, J.Value)] -> J.Object +editFields extra = KM.union (KM.fromList extra) $ KM.delete "key" $ KM.delete "value" $ changeJson viemChange -viemEdit :: EditRequest -viemEdit = EditRequest {erLabelHash = alice, erChange = channel viemChannelLink, erNonce = 0, erDeadline = deadline, erSignature = viemSetSignature} +viemChange :: SignedChange +viemChange = SignedChange {scChange = channel viemChannelLink, scNonce = 0, scDeadline = deadline, scSignature = viemSetSignature} digestOf :: RecordChange -> Word64 -> Word64 -> ByteString digestOf change nonce dl = case change of @@ -886,6 +1117,21 @@ viemSetSignature = either (error . show) id $ parseSignature "0xfce4c6865591609c viemClearSignature :: RecoverableSignature viemClearSignature = either (error . show) id $ parseSignature "0x0565f7e5a591c200fcc358c1d9e9293721d0b99f0285b589211203e311e8e6167ee822d664048870bdd8552263d25bbf45e7f4985fd3b8c3a19cfced44ef9d561c" +viemContactLink :: Text +viemContactLink = "https://smp8.simplex.im/a#Y8m3" + +-- | viem: SetText(alicebakery.simplex, simplex.contact, viemContactLink, 0, deadline) +viemContactSignature :: RecoverableSignature +viemContactSignature = either (error . show) id $ parseSignature "0x2154ef3fbd7244fe34a2c9b2850d44eda1181d1e234900e8419518cab458848f08b5033fd3cd37e7be2346d81266684b4ced6275e84d22829fd0aaa836fae8261c" + +-- | viem: SetText(alicebakery.simplex, simplex.channel, viemChannelLink, 1, deadline) +viemChannelSignature :: RecoverableSignature +viemChannelSignature = either (error . show) id $ parseSignature "0xaecc4fa43c1387e50594ceb43365fe9028a546c63f2ebc3d9d266f992d0b493808009060f7b6831a97b81453ccfb71413e608d48d8eddc7be3c086887b0b46481b" + +-- | viem: multicall([setTextWithSig of the contact edit, setTextWithSig of the channel edit]) +viemMulticall :: ByteString +viemMulticall = unhex "ac9650d8000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000024000000000000000000000000000000000000000000000000000000000000001c471a218a70f4104919a21c22f566d9c7947c7c7d8addbf546c973851a7e449276ea54ec1a00000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006955c7100000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000000f73696d706c65782e636f6e746163740000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001e68747470733a2f2f736d70382e73696d706c65782e696d2f612359386d33000000000000000000000000000000000000000000000000000000000000000000412154ef3fbd7244fe34a2c9b2850d44eda1181d1e234900e8419518cab458848f08b5033fd3cd37e7be2346d81266684b4ced6275e84d22829fd0aaa836fae8261c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001c471a218a70f4104919a21c22f566d9c7947c7c7d8addbf546c973851a7e449276ea54ec1a00000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000006955c7100000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000000f73696d706c65782e6368616e6e656c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001e68747470733a2f2f736d70382e73696d706c65782e696d2f632359386d3300000000000000000000000000000000000000000000000000000000000000000041aecc4fa43c1387e50594ceb43365fe9028a546c63f2ebc3d9d266f992d0b493808009060f7b6831a97b81453ccfb71413e608d48d8eddc7be3c086887b0b46481b0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" + viemSetCall, viemClearCall :: ByteString viemSetCall = unhex "71a218a70f4104919a21c22f566d9c7947c7c7d8addbf546c973851a7e449276ea54ec1a00000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006955c7100000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000000f73696d706c65782e6368616e6e656c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001e68747470733a2f2f736d70382e73696d706c65782e696d2f632359386d3300000000000000000000000000000000000000000000000000000000000000000041fce4c6865591609c07ef3d94f376187d8a20cd9d94b4affcb60b55e999038fbd4e45c42b5fafe66a8d4f756e99398602eacef777b4dce4afd0a767cddadf3ca91c00000000000000000000000000000000000000000000000000000000000000" viemClearCall = unhex "6463e6140f4104919a21c22f566d9c7947c7c7d8addbf546c973851a7e449276ea54ec1a0000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000006955c710000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000410565f7e5a591c200fcc358c1d9e9293721d0b99f0285b589211203e311e8e6167ee822d664048870bdd8552263d25bbf45e7f4985fd3b8c3a19cfced44ef9d561c00000000000000000000000000000000000000000000000000000000000000" diff --git a/tests/Bots/StoreService/NamesSearchTests.hs b/tests/Bots/StoreService/NamesSearchTests.hs index a2d71cb293..3dc1766d5e 100644 --- a/tests/Bots/StoreService/NamesSearchTests.hs +++ b/tests/Bots/StoreService/NamesSearchTests.hs @@ -27,7 +27,7 @@ import Bots.StoreService.NamesContractsTests year, ) import Bots.StoreService.NamesProfilesTests (bakeryLink, jpeg, newsLink) -import Bots.StoreService.NamesStoreTests (issue, newEdit, order, ownerA, placeCode, setChannel, t0, usable) +import Bots.StoreService.NamesStoreTests (atNonce, issue, newEdit, order, ownerA, placeCode, setChannel, t0, usable) import Bots.StoreService.WebTests (withServiceStore) import Control.Concurrent.STM import Control.Monad (forM_, replicateM_) @@ -60,7 +60,7 @@ import StoreService.Names.Abi (AbiValue (..), abiBytes, abiTuple, bytes32FromHex import StoreService.Names.Contracts (Contracts (..)) import StoreService.Names.Phases (RegistrationId (..)) import StoreService.Names.Profiles (ChatKind (..), HostResolver, LinkProfile (..), ProfileFetcher, ProfileLimits (..), defaultProfileLimits, monotonicSeconds, newProfiles) -import StoreService.Names.Store (EditState (..), NamePhase (..), NameRegistration (..), NewRecordEdit (..), VersionWrite (..), getRegistration, insertRecordEdit, settleRecordEdit, updateRegistration) +import StoreService.Names.Store (EditState (..), NamePhase (..), NameRegistration (..), VersionWrite (..), getRegistration, insertRecordEdit, settleRecordEdit, updateRegistration) import StoreService.Web.Http (newHttpEnv, notFound, previewLimit, searchLimit) import StoreService.Web.Names (Route (..), namesApp) import StoreService.Web.Names.Search (previewHandler, searchHandler) @@ -420,7 +420,7 @@ testRecordEdits g = forM_ [(3, 1), (2, 0)] $ \(perYear, left) -> withSearch g (\ registeredWith node fullLink "" let daysAgo :: Int -> UTCTime daysAgo d = addUTCTime (fromIntegral $ negate d * 86400) t0 - inserted n d = withTransaction store $ \db -> insertRecordEdit db (newEdit ownerA n setChannel) {nreNonce = fromIntegral n} (daysAgo d) + inserted n d = withTransaction store $ \db -> insertRecordEdit db (atNonce (fromIntegral n) $ newEdit ownerA n setChannel) (daysAgo d) Just _ <- inserted 1 366 Just minedId <- inserted 2 10 Just revertedId <- inserted 3 5 diff --git a/tests/Bots/StoreService/NamesStoreTests.hs b/tests/Bots/StoreService/NamesStoreTests.hs index cde242c7c5..96751099dc 100644 --- a/tests/Bots/StoreService/NamesStoreTests.hs +++ b/tests/Bots/StoreService/NamesStoreTests.hs @@ -3,7 +3,7 @@ {-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE OverloadedStrings #-} -module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, invoiceStatus, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where +module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, invoiceStatus, atNonce, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where import Bots.StoreService.ConfigTests (termsVersion) import Bots.StoreService.WebTests (columnsOf, digestFixture, runMigrations, withServiceStore) @@ -16,7 +16,8 @@ import Data.Text (Text) import qualified Data.Text as T import Data.Time.Calendar (fromGregorian) import Data.Time.Clock (UTCTime (..), addUTCTime) -import Data.Word (Word8) +import Data.List.NonEmpty (NonEmpty (..)) +import Data.Word (Word64, Word8) import Simplex.Chat.Badges.Types (BadgeCodePaymentStatus (..)) import Simplex.Chat.PaymentService.Types (CardProvider (..), CryptoCurrency (..), InvoiceId (..), ServicePaymentMethod (..)) import Simplex.Messaging.Agent.Store.Common (DBStore, withTransaction) @@ -24,6 +25,7 @@ import qualified Simplex.Messaging.Agent.Store.DB as DB import Simplex.Messaging.Agent.Store.Shared (Migration (..), MigrationsToRun (..), toDownMigration) import Simplex.Messaging.Crypto.Secp256k1 (RecoverableSignature) import Simplex.Messaging.Eth.Address (Address, unAddress) +import Simplex.Messaging.Eth.EIP712 (serializeSignature) import Simplex.Messaging.SimplexName (LabelHash (..), labelHash) import StoreService.Names.Chain (TxHash (..)) import StoreService.Names.Phases (RegistrationId (..)) @@ -50,6 +52,7 @@ namesStoreTests = describe "StoreService names: store" $ do it "makes taken orders not live and gives a taken paid order a balance of its price, none to a code order or a price of 0, and rolls back the balance tables and columns" testNameBalancesMigration it "gives the orders stored before it empty agreed terms, and rolls the column back keeping them" testNameTermsMigration it "adds the payments' invoiced units with the index of overpaid payments, and rolls both back keeping the payments" testInvoicedUnitsMigration + it "moves each stored edit's change to a row of its own, and rolls back keeping each edit's first change" testRecordChangesMigration describe "order hash" $ do it "is equal for the same order fields, and differs when any one field differs" testOrderHashFields it "keeps field boundaries, so moving a byte between two fields changes it" testOrderHashBoundaries @@ -89,6 +92,8 @@ namesStoreTests = describe "StoreService names: store" $ do it "holds one pending or mined edit per owner and nonce, freed by a revert" testEditNonceUnique it "settles a pending edit once, with its mined hash" testSettleEdit it "round-trips a clear edit" testClearEdit + it "stores an edit's two changes and reads them back in nonce order, found by either nonce" testTwoChangeEdit + it "deletes an unsent edit with its changes" testDeleteUnsentChanges it "gives the times of a name's relayed edits in the window" testRelayedEditTimes it "gives the times of a name's pending and mined edits in the window, leaving out reverted ones" testCountedEditTimes @@ -168,7 +173,7 @@ write :: DBStore -> NameRegistration -> IO VersionWrite write st r = withTransaction st $ \db -> updateRegistration db r (at 1) namesTables :: [Text] -namesTables = map ("sx_badge_service_name_" <>) ["prepaid_codes", "registrations", "invoices", "record_edits", "transactions", "balances", "late_credits"] +namesTables = map ("sx_badge_service_name_" <>) ["prepaid_codes", "registrations", "invoices", "record_edits", "record_changes", "transactions", "balances", "late_credits"] -- | Added to the badge tables' payments by the names migrations. balancePaymentColumns :: [Text] @@ -192,14 +197,15 @@ testNamesColumns = withServiceStore $ \st -> do has "sx_badge_service_name_balances" ["registration_id", "amount", "credited", "refunded", "reserved_by", "refunded_at", "created_at", "updated_at"] has "sx_badge_service_name_late_credits" ["invoice_id", "credit_no", "registration_id", "units", "invoice_rate", "current_rate", "rate_used", "cents", "created_at"] has "sx_badge_service_payments" balancePaymentColumns - has "sx_badge_service_name_record_edits" ["label_hash", "owner", "nonce", "record_key", "record_value", "deadline", "signature", "body_hash", "state", "revert", "mined_tx", "created_at"] + has "sx_badge_service_name_record_edits" ["label_hash", "owner", "nonce", "body_hash", "state", "revert", "mined_tx", "created_at"] + has "sx_badge_service_name_record_changes" ["name_record_edit_id", "nonce", "record_key", "record_value", "deadline", "signature"] has "sx_badge_service_name_transactions" ["registration_id", "name_record_edit_id", "kind", "raw", "tx_hash", "registrar_nonce", "sent_at", "mined_block", "status"] testNamesDownUp :: IO () testNamesDownUp = withServiceStore $ \st -> do let names = dropWhile ((/= "20261013_names") . name) storeServiceSchemaMigrations badgeColumns = mapM (fmap sort . columnsOf st) badgeTables - map name names `shouldBe` ["20261013_names", "20261022_drop_taken_codes", "20261022_name_balances", "20261022_name_late_wait", "20261023_name_terms", "20261027_name_invoiced_units"] + map name names `shouldBe` ["20261013_names", "20261022_drop_taken_codes", "20261022_name_balances", "20261022_name_late_wait", "20261023_name_terms", "20261027_name_invoiced_units", "20261029_name_record_changes"] badgeBefore <- badgeColumns code <- issue st 1 usable placeCode st "r1" ownerA (order "alicebakery" 1) code `shouldReturn` Right () @@ -322,6 +328,35 @@ testInvoicedUnitsMigration = withServiceStore $ \st -> do applied `shouldReturn` (False, False) payments `shouldReturn` stored +testRecordChangesMigration :: IO () +testRecordChangesMigration = withServiceStore $ \st -> do + let changesMigration = [m | m@Migration {name = "20261029_name_record_changes"} <- storeServiceSchemaMigrations] + editColumns = sort <$> columnsOf st "sx_badge_service_name_record_edits" + oldEdits :: IO [(Int64, Maybe Text, Maybe Text, Int64)] + oldEdits = withTransaction st $ \db -> DB.query_ db "SELECT nonce, record_key, record_value, deadline FROM sx_badge_service_name_record_edits ORDER BY nonce" + length changesMigration `shouldBe` 1 + runMigrations st $ MTRDown (mapMaybe toDownMigration changesMigration) + columnsOf st "sx_badge_service_name_record_changes" `shouldReturn` [] + oldColumns <- editColumns + withTransaction st $ \db -> + forM_ [(4 :: Int64, Just ("simplex.channel" :: Text), Just ("https://smp9.simplex.im/c#abc" :: Text), 1 :: Word8), (7, Nothing, Nothing, 2)] $ \(nonce, key, value, n) -> + DB.execute + db + "INSERT INTO sx_badge_service_name_record_edits (label_hash, owner, nonce, record_key, record_value, deadline, signature, body_hash, state, created_at) VALUES (?,?,?,?,?,1792832400,?,?,'pending',?)" + ((DB.Binary $ unLabelHash $ labelHash "alicebakery", DB.Binary $ unAddress ownerA, nonce, key, value) :. (DB.Binary $ serializeSignature signature, DB.Binary $ digestFixture n, t0)) + stored <- oldEdits + runMigrations st $ MTRUp changesMigration + editColumns `shouldReturn` sort (filter (`notElem` ["record_key", "record_value", "deadline", "signature"]) oldColumns) + Just setEdit <- withTransaction st $ \db -> getRecordEditByBody db (digestFixture 1) + reChanges setEdit `shouldBe` signedAt 4 setChannel :| [] + Just clearEdit <- withTransaction st $ \db -> getRecordEditByBody db (digestFixture 2) + reChanges clearEdit `shouldBe` signedAt 7 RCClear :| [] + runMigrations st $ MTRDown (mapMaybe toDownMigration changesMigration) + editColumns `shouldReturn` oldColumns + oldEdits `shouldReturn` stored + where + unLabelHash (LabelHash h) = h + -- | A registration row as the code before the balances wrote it, paid by the invoice or the code given. insertOldRegistration :: DB.Connection -> Text -> Word8 -> Text -> Int -> Maybe Int64 -> Maybe Text -> IO () insertOldRegistration db rid n phase cents codeId invId = do @@ -850,20 +885,26 @@ testRevoke = withServiceStore $ \st -> do held <- withTransaction st $ \db -> getPrepaidCodeById db (pcId reserved) (pcReservedBy =<< held, pcRevokedAt =<< held) `shouldBe` (Just (RegistrationId "r1"), Just (at 1)) +-- | An edit of one change at nonce 4. newEdit :: Address -> Word8 -> RecordChange -> NewRecordEdit newEdit owner n change = NewRecordEdit { nreLabelHash = labelHash "alicebakery", nreOwner = owner, - nreNonce = 4, - nreChange = change, - nreDeadline = 1792832400, - nreSignature = signature, + nreChanges = signedAt 4 change :| [], nreBodyHash = digestFixture n } -setChannel :: RecordChange +-- | The edit with its one change at this nonce. +atNonce :: Word64 -> NewRecordEdit -> NewRecordEdit +atNonce nonce edit@NewRecordEdit {nreChanges = change :| _} = edit {nreChanges = change {scNonce = nonce} :| []} + +signedAt :: Word64 -> RecordChange -> SignedChange +signedAt scNonce scChange = SignedChange {scChange, scNonce, scDeadline = 1792832400, scSignature = signature} + +setChannel, setContact :: RecordChange setChannel = RCSetText "simplex.channel" "https://smp9.simplex.im/c#abc" +setContact = RCSetText "simplex.contact" "https://smp8.simplex.im/a#abc" testTransactions :: IO () testTransactions = withServiceStore $ \st -> do @@ -929,10 +970,7 @@ testSettleEdit = withServiceStore $ \st -> do { reId = editId, reLabelHash = labelHash "alicebakery", reOwner = ownerA, - reNonce = 4, - reChange = setChannel, - reDeadline = 1792832400, - reSignature = signature, + reChanges = signedAt 4 setChannel :| [], reBodyHash = digestFixture 1, reState = ESMined (Just $ TxHash $ digestFixture 9), reCreatedAt = t0 @@ -941,22 +979,48 @@ testSettleEdit = withServiceStore $ \st -> do testClearEdit :: IO () testClearEdit = withServiceStore $ \st -> do Just editId <- withTransaction st $ \db -> insertRecordEdit db (newEdit ownerA 1 RCClear) t0 - fmap (\RecordEdit {reChange, reState} -> (reChange, reState)) <$> withTransaction st (\db -> getRecordEdit db editId) - `shouldReturn` Just (RCClear, ESPending) + fmap (\RecordEdit {reChanges, reState} -> (reChanges, reState)) <$> withTransaction st (\db -> getRecordEdit db editId) + `shouldReturn` Just (signedAt 4 RCClear :| [], ESPending) + +testTwoChangeEdit :: IO () +testTwoChangeEdit = withServiceStore $ \st -> do + let run action = withTransaction st action + changes = signedAt 4 setContact :| [(signedAt 5 setChannel) {scDeadline = 1792832460}] + Just editId <- run $ \db -> insertRecordEdit db (newEdit ownerA 1 setContact) {nreChanges = changes} t0 + fmap reChanges <$> run (\db -> getRecordEdit db editId) `shouldReturn` Just changes + fmap reId <$> run (\db -> getRecordEditByNonce db ownerA 4) `shouldReturn` Just editId + fmap reId <$> run (\db -> getRecordEditByNonce db ownerA 5) `shouldReturn` Just editId + fmap reChanges <$> run (\db -> getRecordEditByBody db (digestFixture 1)) `shouldReturn` Just changes + -- the first nonce is the edit's: another edit starting there waits for it to settle + run (\db -> insertRecordEdit db (newEdit ownerA 2 setChannel) t0) `shouldReturn` Nothing + changeRows st `shouldReturn` 2 + +testDeleteUnsentChanges :: IO () +testDeleteUnsentChanges = withServiceStore $ \st -> do + let changes = signedAt 4 setContact :| [signedAt 5 setChannel] + Just editId <- withTransaction st $ \db -> insertRecordEdit db (newEdit ownerA 1 setContact) {nreChanges = changes} t0 + withTransaction st $ \db -> deleteUnsentEdit db editId + withTransaction st (\db -> getRecordEdit db editId) `shouldReturn` Nothing + changeRows st `shouldReturn` 0 + +changeRows :: DBStore -> IO Int +changeRows st = withTransaction st $ \db -> do + [Only n] <- DB.query_ db "SELECT count(*) FROM sx_badge_service_name_record_changes" + pure n testRelayedEditTimes :: IO () testRelayedEditTimes = withServiceStore $ \st -> do forM_ (zip [1 ..] [0, 10, 20]) $ \(n, m) -> - withTransaction st $ \db -> void $ insertRecordEdit db (newEdit ownerA n setChannel) {nreNonce = fromIntegral n} (at m) - withTransaction st (\db -> void $ insertRecordEdit db (newEdit ownerA 9 setChannel) {nreLabelHash = labelHash "bobsbakery", nreNonce = 9} (at 15)) + withTransaction st $ \db -> void $ insertRecordEdit db (atNonce (fromIntegral n) $ newEdit ownerA n setChannel) (at m) + withTransaction st (\db -> void $ insertRecordEdit db (atNonce 9 $ newEdit ownerA 9 setChannel) {nreLabelHash = labelHash "bobsbakery"} (at 15)) withTransaction st (\db -> relayedEditTimes db (labelHash "alicebakery") (at 5)) `shouldReturn` [at 10, at 20] testCountedEditTimes :: IO () testCountedEditTimes = withServiceStore $ \st -> do [_, Just minedId, Just revertedId, _] <- forM (zip [1 ..] [0, 10, 20, 30]) $ \(n, m) -> - withTransaction st $ \db -> insertRecordEdit db (newEdit ownerA n setChannel) {nreNonce = fromIntegral n} (at m) + withTransaction st $ \db -> insertRecordEdit db (atNonce (fromIntegral n) $ newEdit ownerA n setChannel) (at m) withTransaction st $ \db -> do settleRecordEdit db minedId (ESMined Nothing) `shouldReturn` True settleRecordEdit db revertedId (ESReverted Nothing) `shouldReturn` True - withTransaction st (\db -> void $ insertRecordEdit db (newEdit ownerA 9 setChannel) {nreLabelHash = labelHash "bobsbakery", nreNonce = 9} (at 15)) + withTransaction st (\db -> void $ insertRecordEdit db (atNonce 9 $ newEdit ownerA 9 setChannel) {nreLabelHash = labelHash "bobsbakery"} (at 15)) withTransaction st (\db -> countedEditTimes db (labelHash "alicebakery") (at 5)) `shouldReturn` [at 10, at 30] diff --git a/tests/Bots/StoreService/WebTests.hs b/tests/Bots/StoreService/WebTests.hs index 6238734c4e..ff797e5d32 100644 --- a/tests/Bots/StoreService/WebTests.hs +++ b/tests/Bots/StoreService/WebTests.hs @@ -423,7 +423,7 @@ testMigrationOrder = do [(a, b) | (a, b) <- zip declared (drop 1 declared), a >= b] `shouldBe` [] let (badges, ours) = span (`elem` badgeMigrations) (map name storeServiceSchemaMigrations) badges `shouldBe` badgeMigrations - ours `shouldBe` ["20261013_names", "20261022_drop_taken_codes", "20261022_name_balances", "20261022_name_late_wait", "20261023_name_terms", "20261027_name_invoiced_units"] + ours `shouldBe` ["20261013_names", "20261022_drop_taken_codes", "20261022_name_balances", "20261022_name_late_wait", "20261023_name_terms", "20261027_name_invoiced_units", "20261029_name_record_changes"] where badgeMigrations = ["20260915_badge_service_schema", "20260918_badge_group_ops", "20260925_badge_store_receipts", "20261012_badge_subscriptions"]