refactor WalletSeed type, see mq wip

bots/api/TYPES.md

@@ -4665,8 +4665,8 @@ AccountNotHeld:
 CounterUnknown:
 - type: "counterUnknown"
 
-IndexTooLarge:
-- type: "indexTooLarge"
+AccountsExhausted:
+- type: "accountsExhausted"
 
 Derivation:
 - type: "derivation"

docs/rfcs/2026-09-10-wallet-keys.md

@@ -85,7 +85,7 @@ A command that acts on a profile's accounts names the profile and is rejected wh
 
 `bind` without `account=` binds the account at a counter on the master, which is a high-water mark and not a count of bound accounts, and returns the bound account's index, path and address. With `account=<n>` it binds that account, which is how an account found by a scan is attached to the profile it belongs to, and it is rejected for an account another profile holds. After an import the counter is unknown rather than zero, because the phrase does not encode how many accounts it has been used for, so binding the next account is rejected until a scan sets the counter, while binding a known account is still allowed.
 
-BIP-32 marks an index as hardened by setting its top bit, so an index at or above 2^31 already has that bit set and derives the same key as the index 2^31 below it: account 2^31 is account 0. That is a collision, not a loss of hardening, and it would put one key under two account indexes. Every index at or above 2^31 is rejected, including one read from the counter; the simplexmq function that builds the path rejects it too, and the columns have CHECK constraints for that bound, so later writes cannot exceed it. The counter's bound is one higher than an account's, because it contains the next index to bind, and 2^31 there means the counter has passed every index that can be hardened.
+BIP-32 marks an index as hardened by setting its top bit, so an index at or above 2^31 already has that bit set and derives the same key as the index 2^31 below it: account 2^31 is account 0. That is a collision, not a loss of hardening, and it would put one key under two account indexes. An account index is a simplexmq type that only holds values below 2^31, so the command parser rejects 2^31 and above as a bad command, the columns have CHECK constraints for that bound, and reading a row that violates it is an error. The counter's bound is one higher than an account's, because it contains the next index to bind, and 2^31 there means the counter has passed every index that can be hardened, which `bind` without an index reports.
 
 `address` reads the counter without changing it, so two calls return the same address, and it derives an address for an account the database has no row for, which a device that lost its database requires. One address per call is sufficient: a caller that scans the tree calls it in a loop.
 
@@ -102,7 +102,7 @@ data WalletError
   | WEAccountBound    -- bind, on an account another profile holds
   | WEAccountNotHeld  -- export account, on an account the profile does not hold
   | WECounterUnknown  -- the counter is not set yet, after an import
-  | WEIndexTooLarge   -- at or above 2^31
+  | WEAccountsExhausted -- bind without an index, when the counter has passed every index
   | WEDerivation {derivationError :: String} -- BIP-32 or BIP-39 derivation failed
 ```
 
@@ -124,6 +124,7 @@ What the scan finds is unbound, and the user attaches each account to a profile
 CREATE TABLE wallet_seeds (
   wallet_seed_id INTEGER PRIMARY KEY AUTOINCREMENT,
   entropy BLOB NOT NULL CHECK (length(entropy) = 32),
+  master BLOB NOT NULL CHECK (length(master) = 64),
   next_account_index INTEGER CHECK (next_account_index BETWEEN 0 AND 2147483648), -- null means not known yet
   single_seed INTEGER NOT NULL DEFAULT 1
 ) STRICT;
@@ -140,7 +141,7 @@ CREATE UNIQUE INDEX idx_wallet_accounts_wallet_seed_id_account_index ON wallet_a
 CREATE INDEX idx_wallet_accounts_user_id ON wallet_accounts(user_id);
 ```
 
-Only entropy that nothing can derive is stored: the master, always 32 bytes, since it is generated and imported as 24 words. An account key is never stored, because the master entropy and an account index derive it whenever it is required. So `wallet_accounts` contains what derivation cannot produce: which account indexes are recorded on the device and which profile each belongs to. A row with no `user_id` is an account no profile holds, which is the result of deleting a chat profile and what a scan writes.
+The master entropy, always 32 bytes since it is generated and imported as 24 words, is stored with the BIP-32 master key it derives, the 32-byte private key followed by the 32-byte chain code. Reading the row recomputes the master key from the entropy and rejects a row where the two do not match, so a wallet read from the database always derives accounts. An account key is never stored, because the master key and an account index derive it whenever it is required. So `wallet_accounts` contains what derivation cannot produce: which account indexes are recorded on the device and which profile each belongs to. A row with no `user_id` is an account no profile holds, which is the result of deleting a chat profile and what a scan writes.
 
 `users` is not changed: the mapping is stored in the account row, and the index on `user_id` is not unique, because a profile can hold any number of accounts. One seed per device is enforced by `single_seed` and the unique index on it, which a later change removes with a `DROP INDEX` and a `DROP COLUMN`; it is a named index rather than an inline `UNIQUE` because SQLite cannot drop an inline constraint without rebuilding the table. Deleting the master deletes its account rows, because an account index without its entropy derives nothing. The migration has no down migration, because a down migration would delete the master entropy, which may have no other copy. A down migration runs when an older app opens a newer database and the user confirms "Downgrade and open chat", and the backup made then is overwritten by the next upgrade. Without a down migration the older app reports that the database is newer than the app, and changes nothing.
 
@@ -183,6 +184,6 @@ A null `account_index` marks an account whose key was imported rather than deriv
 
 1. **Vectors.** The two addresses above are derived from `abandon ... about`, as is account 0's secret, pinned to the value another wallet shows for it. A 24 word phrase imported through the command derives a pinned address end to end, so a change of path fails here rather than in a release, and the account a command names is the account whose key is returned.
 2. **Isolation.** Ten accounts' addresses are all different, and the paths of accounts 0 and 7 have a hardened account component.
-3. **Rejections.** A second generate; a phrase that is not 24 valid words; `bind`, `delete` and `export master` on a device with no wallet; `bind` on a hidden profile, on an account another profile holds, and on an imported master whose counter is unknown; `export account` for an account the profile does not hold; index 2^31, on `address`, `bind` and `export account` alike; an index of 2^32 or more is rejected as a bad command.
+3. **Rejections.** A second generate; a phrase that is not 24 valid words; `bind`, `delete` and `export master` on a device with no wallet; `bind` on a hidden profile, on an account another profile holds, and on an imported master whose counter is unknown; `export account` for an account the profile does not hold; `bind` without an index once the counter has passed 2^31 - 1; an index of 2^31 or more is rejected as a bad command on `address`, `bind` and `export account` alike.
 4. **Binding and reads.** Several accounts are bound to one profile, each `bind` returns the account it bound, and that profile's accounts can be exported; binding an account by index moves the counter past it so the next one does not collide, and never moves it back; `bind account=<n>` binds an account after an import; an account left unbound by deleting its profile is bound to another profile; and two consecutive `address` calls return the same address without changing the counter, and derive an address for an account with no row.
 5. **Encoding and persistence.** An account secret whose first byte is zero is rendered with 64 hex digits; the wallet, its accounts, the counter and the phrase persist across a restart; and deleting the wallet deletes its accounts and resets the counter.

packages/simplex-chat-client/types/typescript/src/types.ts

@@ -5315,7 +5315,7 @@ export type WalletError =
   | WalletError.AccountBound
   | WalletError.AccountNotHeld
   | WalletError.CounterUnknown
-  | WalletError.IndexTooLarge
+  | WalletError.AccountsExhausted
   | WalletError.Derivation
 
 export namespace WalletError {
@@ -5327,7 +5327,7 @@ export namespace WalletError {
     | "accountBound"
     | "accountNotHeld"
     | "counterUnknown"
-    | "indexTooLarge"
+    | "accountsExhausted"
     | "derivation"
 
   interface Interface {
@@ -5362,8 +5362,8 @@ export namespace WalletError {
     type: "counterUnknown"
   }
 
-  export interface IndexTooLarge extends Interface {
-    type: "indexTooLarge"
+  export interface AccountsExhausted extends Interface {
+    type: "accountsExhausted"
   }
 
   export interface Derivation extends Interface {

packages/simplex-chat-python/src/simplex_chat/types/_types.py

@@ -3743,8 +3743,8 @@ class WalletError_accountNotHeld(TypedDict):
 class WalletError_counterUnknown(TypedDict):
     type: Literal["counterUnknown"]
 
-class WalletError_indexTooLarge(TypedDict):
-    type: Literal["indexTooLarge"]
+class WalletError_accountsExhausted(TypedDict):
+    type: Literal["accountsExhausted"]
 
 class WalletError_derivation(TypedDict):
     type: Literal["derivation"]
@@ -3758,11 +3758,11 @@ WalletError = (
     | WalletError_accountBound
     | WalletError_accountNotHeld
     | WalletError_counterUnknown
-    | WalletError_indexTooLarge
+    | WalletError_accountsExhausted
     | WalletError_derivation
 )
 
-WalletError_Tag = Literal["noMaster", "masterExists", "badMnemonic", "hiddenProfile", "accountBound", "accountNotHeld", "counterUnknown", "indexTooLarge", "derivation"]
+WalletError_Tag = Literal["noMaster", "masterExists", "badMnemonic", "hiddenProfile", "accountBound", "accountNotHeld", "counterUnknown", "accountsExhausted", "derivation"]
 
 class XFTPErrorType_BLOCK(TypedDict):
     type: Literal["BLOCK"]

src/Simplex/Chat/Library/Commands.hs

@@ -65,7 +65,7 @@ import Simplex.Chat.Badges.Types (BadgeAlert (..), BadgeAlertKind (..), BadgeIss
 import Simplex.Chat.Badges.Code (badgeCodeText, parseBadgeCode)
 import Simplex.Chat.Badges.Service (BadgeBalance (..), BadgeServiceCommand (..), BadgeServiceErrorCode (..), BadgeServiceRequest (..), BadgeServiceResponse (..), BadgeStatement (..), StatementDebitType (..), StatementEntry (..), StatementEntryType (..), currentBadgeServiceVersion)
 import Simplex.Chat.Names (SimplexDomainProof (..), SimplexDomainClaim (..), claimDomain, mkDomainClaim)
-import Simplex.Chat.Wallet (AccountIndex, AccountKey, WalletAddress, WalletError (..), accountSecret, deriveAccount, entropyFromMnemonic, newSeedEntropy, seedMnemonic)
+import Simplex.Chat.Wallet (AccountIndex, AccountKey, WalletAddress, WalletError (..), accountSecret, deriveAccount, entropyFromMnemonic, newSeedEntropy, newWalletMaster, seedMnemonic)
 import Simplex.Chat.Call
 import Simplex.Chat.Controller
 import Simplex.Chat.Delivery (DeliveryJobScope (..), DeliveryJobSpec (..), DeliveryWorkerScope (..))
@@ -96,7 +96,7 @@ import Simplex.Chat.Store.Messages
 import Simplex.Chat.Store.NoteFolders
 import Simplex.Chat.Store.Profiles
 import Simplex.Chat.Store.Shared
-import Simplex.Chat.Store.Wallets (WalletSeed (..), accountHeldBy, bindAccount, createWalletSeed, deleteWalletSeed, getUserAccounts, getWalletSeed, resolveAccount)
+import Simplex.Chat.Store.Wallets (Wallet (..), accountHeldBy, bindAccount, createWallet, deleteWallet, getUserAccounts, getWallet, resolveAccount)
 import Simplex.Chat.Types
 import Simplex.Chat.Types.Preferences
 import Simplex.Chat.Types.Shared
@@ -116,6 +116,7 @@ import Simplex.Messaging.Agent.Store.Interface (getCurrentMigrations)
 import Simplex.Messaging.Client (NetworkConfig (..), NetworkRequestMode (..), NetworkTimeout (..), SMPWebPortServers (..), SocksMode (SMAlways), pattern NRMInteractive, textToHostMode)
 import qualified Simplex.Messaging.Crypto as C
 import qualified Simplex.Messaging.Crypto.ShortLink as SL
+import Simplex.Messaging.Crypto.BIP44 (mkAccountIndex)
 import Simplex.Messaging.Crypto.File (CryptoFile (..), CryptoFileArgs (..))
 import qualified Simplex.Messaging.Crypto.File as CF
 import Simplex.Messaging.Crypto.Ratchet (E2ERatchetParamsUri (..), InitialKeys (..), PQEncryption (..), PQSupport (..), pattern IKPQOff, pattern IKPQOn, pattern PQSupportOff, pattern PQSupportOn)
@@ -1500,35 +1501,36 @@ processChatCommand cxt nm = \case
     connId <- withAgent $ \a -> sendServiceReplyAsync a "" (aUserId user) invId (LB.toStrict $ J.encode responseData)
     pure $ CRServiceReplyAccepted user (AgentConnId connId)
   APIGetWallet userId -> withUserId userId $ \user ->
-    CRWallet user <$> withFastStore' (\db -> getWalletSeed db >>= mapM (\WalletSeed {wsId} -> getUserAccounts db wsId userId))
+    CRWallet user <$> withFastStore (\db -> getWallet db >>= mapM (\Wallet {walletId} -> liftIO $ getUserAccounts db walletId userId))
   APICreateWallet mnemonic_ -> withUser $ \user -> do
-    seed_ <- withFastStore' getWalletSeed
-    when (isJust seed_) $ throwWalletError WEMasterExists
+    wallet_ <- withFastStore getWallet
+    when (isJust wallet_) $ throwWalletError WEMasterExists
     -- the counter starts at 0 for a generated seed and is unknown for an imported one
     (entropy, nextAccount) <- case mnemonic_ of
       Nothing -> (,Just 0) <$> (asks random >>= atomically . newSeedEntropy)
       Just phrase -> (,Nothing) <$> liftWallet (entropyFromMnemonic phrase)
-    created <- withFastStore' $ \db -> createWalletSeed db entropy nextAccount
+    master <- liftWallet $ newWalletMaster entropy
+    created <- withFastStore' $ \db -> createWallet db master nextAccount
     unless created $ throwWalletError WEMasterExists
     pure $ CRWallet user (Just [])
   APIBindWalletAccount userId accountIdx_ -> withUserId userId $ \user@User {viewPwdHash} -> do
     when (isJust viewPwdHash) $ throwWalletError WEHiddenProfile
-    (seed, n) <- withWalletStore $ \db -> bindAccount db userId accountIdx_
-    CRWalletAddress user . snd <$> seedAccount seed n
+    (wallet, n) <- withWalletStore $ \db -> bindAccount db userId accountIdx_
+    CRWalletAddress user . snd <$> walletAccount wallet n
   APIGetWalletAddress accountIdx_ -> withUser $ \user -> do
-    (seed, n) <- withWalletStore (`resolveAccount` accountIdx_)
-    CRWalletAddress user . snd <$> seedAccount seed n
+    (wallet, n) <- withWalletStore (`resolveAccount` accountIdx_)
+    CRWalletAddress user . snd <$> walletAccount wallet n
   APIExportWalletMnemonic -> withUser $ \user -> do
-    WalletSeed {wsEntropy} <- withFastStore' getWalletSeed >>= maybe (throwWalletError WENoMaster) pure
-    CRWalletMnemonic user <$> liftWallet (seedMnemonic wsEntropy)
+    Wallet {walletMaster} <- withFastStore getWallet >>= maybe (throwWalletError WENoMaster) pure
+    pure $ CRWalletMnemonic user (seedMnemonic walletMaster)
   APIExportWalletAccount userId n -> withUserId userId $ \user -> do
-    (seed@WalletSeed {wsId}, _) <- withWalletStore (`resolveAccount` Just n)
-    held <- withFastStore' $ \db -> accountHeldBy db wsId userId n
+    (wallet@Wallet {walletId}, _) <- withWalletStore (`resolveAccount` Just n)
+    held <- withFastStore' $ \db -> accountHeldBy db walletId userId n
     unless held $ throwWalletError WEAccountNotHeld
-    (k, a) <- seedAccount seed n
+    (k, a) <- walletAccount wallet n
     pure $ CRWalletAccountSecret user a (accountSecret k)
   APIDeleteWallet -> withUser_ $ do
-    deleted <- withFastStore' deleteWalletSeed
+    deleted <- withFastStore' deleteWallet
     unless deleted $ throwWalletError WENoMaster
     ok_
   APISendCallInvitation contactId callType -> withUser $ \user -> do
@@ -6031,13 +6033,13 @@ throwWalletError = throwChatError . CEWallet
 liftWallet :: Either WalletError a -> CM a
 liftWallet = liftEitherWith (ChatError . CEWallet)
 
-withWalletStore :: (DB.Connection -> IO (Either WalletError a)) -> CM a
-withWalletStore action = liftWallet =<< withFastStore' action
+withWalletStore :: (DB.Connection -> ExceptT StoreError IO (Either WalletError a)) -> CM a
+withWalletStore action = liftWallet =<< withFastStore action
 
-seedAccount :: WalletSeed -> AccountIndex -> CM (AccountKey, WalletAddress)
-seedAccount WalletSeed {wsEntropy} n = do
+walletAccount :: Wallet -> AccountIndex -> CM (AccountKey, WalletAddress)
+walletAccount Wallet {walletMaster} n = do
   g <- asks random
-  liftError' (ChatError . CEWallet) (deriveAccount g wsEntropy n)
+  liftError' (ChatError . CEWallet) (deriveAccount g walletMaster n)
 
 chatCommandP :: Parser ChatCommand
 chatCommandP =
@@ -6714,7 +6716,7 @@ chatCommandP =
     char_ = optional . A.char
     accountIndexP = do
       i <- A.decimal
-      if i <= toInteger (maxBound :: AccountIndex) then pure (fromInteger i) else fail "account index too large"
+      maybe (fail "account index too large") pure $ if i <= toInteger (maxBound :: Word32) then mkAccountIndex (fromInteger i) else Nothing
 
 displayNameP :: Parser Text
 displayNameP = safeDecodeUtf8 <$> displayNameP_

src/Simplex/Chat/Store/Postgres/Migrations/M20260924_wallet_seeds.hs

@@ -12,6 +12,7 @@ m20260924_wallet_seeds =
 CREATE TABLE wallet_seeds (
   wallet_seed_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
   entropy BYTEA NOT NULL CHECK (length(entropy) = 32),
+  master BYTEA NOT NULL CHECK (length(master) = 64),
   next_account_index BIGINT CHECK (next_account_index BETWEEN 0 AND 2147483648),
   single_seed SMALLINT NOT NULL DEFAULT 1
 );

src/Simplex/Chat/Store/Postgres/Migrations/chat_schema.sql

@@ -1754,9 +1754,11 @@ ALTER TABLE test_chat_schema.wallet_accounts ALTER COLUMN wallet_account_id ADD
 CREATE TABLE test_chat_schema.wallet_seeds (
     wallet_seed_id bigint NOT NULL,
     entropy bytea NOT NULL,
+    master bytea NOT NULL,
     next_account_index bigint,
     single_seed smallint DEFAULT 1 NOT NULL,
     CONSTRAINT wallet_seeds_entropy_check CHECK ((length(entropy) = 32)),
+    CONSTRAINT wallet_seeds_master_check CHECK ((length(master) = 64)),
     CONSTRAINT wallet_seeds_next_account_index_check CHECK (((next_account_index >= 0) AND (next_account_index <= '2147483648'::bigint)))
 );
 

src/Simplex/Chat/Store/SQLite/Migrations/M20260924_wallet_seeds.hs

@@ -11,6 +11,7 @@ m20260924_wallet_seeds =
 CREATE TABLE wallet_seeds (
   wallet_seed_id INTEGER PRIMARY KEY AUTOINCREMENT,
   entropy BLOB NOT NULL CHECK (length(entropy) = 32),
+  master BLOB NOT NULL CHECK (length(master) = 64),
   next_account_index INTEGER CHECK (next_account_index BETWEEN 0 AND 2147483648),
   single_seed INTEGER NOT NULL DEFAULT 1
 ) STRICT;

src/Simplex/Chat/Store/SQLite/Migrations/chat_query_plans.txt

@@ -2049,7 +2049,7 @@ Query:
 Plan:
 
 Query: 
-        INSERT INTO wallet_seeds (entropy, next_account_index) VALUES (?, ?)
+        INSERT INTO wallet_seeds (entropy, master, next_account_index) VALUES (?, ?, ?)
         ON CONFLICT (single_seed) DO NOTHING
         RETURNING wallet_seed_id
       
@@ -7820,7 +7820,7 @@ Query: SELECT via_contact_uri, via_contact_uri_hash FROM connections WHERE conne
 Plan:
 SEARCH connections USING INTEGER PRIMARY KEY (rowid=?)
 
-Query: SELECT wallet_seed_id, entropy, next_account_index FROM wallet_seeds ORDER BY wallet_seed_id LIMIT 1
+Query: SELECT wallet_seed_id, entropy, master, next_account_index FROM wallet_seeds ORDER BY wallet_seed_id LIMIT 1
 Plan:
 SCAN wallet_seeds
 

src/Simplex/Chat/Store/SQLite/Migrations/chat_schema.sql

@@ -984,6 +984,7 @@ CREATE TABLE badge_code_redemptions(
 CREATE TABLE wallet_seeds(
   wallet_seed_id INTEGER PRIMARY KEY AUTOINCREMENT,
   entropy BLOB NOT NULL CHECK(length(entropy) = 32),
+  master BLOB NOT NULL CHECK(length(master) = 64),
   next_account_index INTEGER CHECK(next_account_index BETWEEN 0 AND 2147483648),
   single_seed INTEGER NOT NULL DEFAULT 1
 ) STRICT;

src/Simplex/Chat/Store/Wallets.hs

@@ -3,13 +3,14 @@
 {-# LANGUAGE NamedFieldPuns #-}
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE QuasiQuotes #-}
+{-# LANGUAGE TupleSections #-}
 
 module Simplex.Chat.Store.Wallets
   ( SeedId,
-    WalletSeed (..),
-    getWalletSeed,
-    createWalletSeed,
-    deleteWalletSeed,
+    Wallet (..),
+    getWallet,
+    createWallet,
+    deleteWallet,
     resolveAccount,
     getUserAccounts,
     accountHeldBy,
@@ -17,17 +18,21 @@ module Simplex.Chat.Store.Wallets
   )
 where
 
-import Control.Applicative ((<|>))
-import Control.Monad (unless)
 import Control.Monad.Except
 import Control.Monad.IO.Class (liftIO)
 import qualified Data.ByteArray as BA
 import Data.ByteString (ByteString)
 import Data.Int (Int64)
-import Simplex.Chat.Wallet (AccountIndex, WalletError (..), checkAccountIndex)
+import Data.Word (Word32)
+import Simplex.Chat.Store.Shared (StoreError (..))
+import Simplex.Chat.Wallet (AccountIndex, WalletError (..))
 import Simplex.Messaging.Agent.Protocol (UserId)
 import Simplex.Messaging.Agent.Store.AgentStore (maybeFirstRow)
 import qualified Simplex.Messaging.Agent.Store.DB as DB
+import Simplex.Messaging.Crypto.BIP32 (WalletMaster, masterBytes, masterEntropy, parseWalletMaster)
+import Simplex.Messaging.Crypto.BIP39 (unEntropy)
+import Simplex.Messaging.Crypto.BIP44 (mkAccountIndex, unAccountIndex)
+import Simplex.Messaging.Util (liftEitherWith)
 
 #if defined(dbPostgres)
 import Database.PostgreSQL.Simple (Only (..))
@@ -39,42 +44,44 @@ import Database.SQLite.Simple.QQ (sql)
 
 type SeedId = Int64
 
-data WalletSeed = WalletSeed
-  { wsId :: SeedId,
-    wsEntropy :: BA.ScrubbedBytes,
-    wsNextAccount :: Maybe AccountIndex
+data Wallet = Wallet
+  { walletId :: SeedId,
+    walletMaster :: WalletMaster,
+    nextAccount :: Maybe Word32
   }
 
-toSeed :: (SeedId, ByteString, Maybe AccountIndex) -> WalletSeed
-toSeed (wsId, entropy, wsNextAccount) = WalletSeed {wsId, wsEntropy = BA.convert entropy, wsNextAccount}
+getWallet :: DB.Connection -> ExceptT StoreError IO (Maybe Wallet)
+getWallet db =
+  liftIO (maybeFirstRow id $ DB.query_ db "SELECT wallet_seed_id, entropy, master, next_account_index FROM wallet_seeds ORDER BY wallet_seed_id LIMIT 1")
+    >>= mapM toWallet
+  where
+    toWallet :: (SeedId, ByteString, ByteString, Maybe Word32) -> ExceptT StoreError IO Wallet
+    toWallet (walletId, entropy, master, nextAccount) =
+      liftEitherWith SEInternalError $ (\walletMaster -> Wallet {walletId, walletMaster, nextAccount}) <$> parseWalletMaster (BA.convert entropy) (BA.convert master)
 
-getWalletSeed :: DB.Connection -> IO (Maybe WalletSeed)
-getWalletSeed db =
-  maybeFirstRow toSeed $
-    DB.query_ db "SELECT wallet_seed_id, entropy, next_account_index FROM wallet_seeds ORDER BY wallet_seed_id LIMIT 1"
-
-createWalletSeed :: DB.Connection -> BA.ScrubbedBytes -> Maybe AccountIndex -> IO Bool
-createWalletSeed db entropy nextAccount =
+createWallet :: DB.Connection -> WalletMaster -> Maybe Word32 -> IO Bool
+createWallet db master nextAccount =
   rowReturned $
     DB.query
       db
       [sql|
-        INSERT INTO wallet_seeds (entropy, next_account_index) VALUES (?, ?)
+        INSERT INTO wallet_seeds (entropy, master, next_account_index) VALUES (?, ?, ?)
         ON CONFLICT (single_seed) DO NOTHING
         RETURNING wallet_seed_id
       |]
-      (DB.Binary (BA.convert entropy :: ByteString), nextAccount)
+      (DB.Binary (BA.convert (unEntropy $ masterEntropy master) :: ByteString), DB.Binary (BA.convert (masterBytes master) :: ByteString), nextAccount)
 
-deleteWalletSeed :: DB.Connection -> IO Bool
-deleteWalletSeed db =
+deleteWallet :: DB.Connection -> IO Bool
+deleteWallet db =
   rowReturned $ DB.query_ db "DELETE FROM wallet_seeds RETURNING wallet_seed_id"
 
-resolveAccount :: DB.Connection -> Maybe AccountIndex -> IO (Either WalletError (WalletSeed, AccountIndex))
-resolveAccount db accountIdx_ = runExceptT $ do
-  seed@WalletSeed {wsNextAccount} <- ExceptT $ maybe (Left WENoMaster) Right <$> getWalletSeed db
-  n <- liftEither $ maybe (Left WECounterUnknown) Right (accountIdx_ <|> wsNextAccount)
-  liftEither $ checkAccountIndex n
-  pure (seed, n)
+resolveAccount :: DB.Connection -> Maybe AccountIndex -> ExceptT StoreError IO (Either WalletError (Wallet, AccountIndex))
+resolveAccount db accountIdx_ =
+  getWallet db >>= \case
+    Nothing -> pure $ Left WENoMaster
+    Just w@Wallet {nextAccount} -> pure $ (w,) <$> maybe next Right accountIdx_
+      where
+        next = maybe (Left WECounterUnknown) (maybe (Left WEAccountsExhausted) Right . mkAccountIndex) nextAccount
 
 getUserAccounts :: DB.Connection -> SeedId -> UserId -> IO [AccountIndex]
 getUserAccounts db sId userId =
@@ -94,16 +101,16 @@ accountUser db sId n =
   maybeFirstRow fromOnly $
     DB.query db "SELECT user_id FROM wallet_accounts WHERE wallet_seed_id = ? AND account_index = ?" (sId, n)
 
-bindAccount :: DB.Connection -> UserId -> Maybe AccountIndex -> IO (Either WalletError (WalletSeed, AccountIndex))
-bindAccount db userId accountIdx_ = runExceptT $ do
-  r@(WalletSeed {wsId}, n) <- ExceptT $ resolveAccount db accountIdx_
-  held <- liftIO $ accountUser db wsId n >>= \case
-    Just (Just heldBy) -> pure $ heldBy == userId
-    Just Nothing -> setAccountUser db wsId userId n
-    Nothing -> True <$ insertAccount db wsId userId n
-  unless held $ throwError WEAccountBound
-  liftIO $ raiseNextAccount db wsId n
-  pure r
+bindAccount :: DB.Connection -> UserId -> Maybe AccountIndex -> ExceptT StoreError IO (Either WalletError (Wallet, AccountIndex))
+bindAccount db userId accountIdx_ = resolveAccount db accountIdx_ >>= either (pure . Left) (liftIO . bind)
+  where
+    bind r@(Wallet {walletId}, n) = do
+      held <-
+        accountUser db walletId n >>= \case
+          Just (Just heldBy) -> pure $ heldBy == userId
+          Just Nothing -> setAccountUser db walletId userId n
+          Nothing -> True <$ insertAccount db walletId userId n
+      if held then Right r <$ raiseNextAccount db walletId n else pure $ Left WEAccountBound
 
 setAccountUser :: DB.Connection -> SeedId -> UserId -> AccountIndex -> IO Bool
 setAccountUser db sId userId n =
@@ -128,7 +135,7 @@ raiseNextAccount db sId n =
       UPDATE wallet_seeds SET next_account_index = ?
       WHERE wallet_seed_id = ? AND next_account_index IS NOT NULL AND next_account_index <= ?
     |]
-    (n + 1, sId, n)
+    (unAccountIndex n + 1, sId, n)
 
 insertAccount :: DB.Connection -> SeedId -> UserId -> AccountIndex -> IO ()
 insertAccount db sId userId n =

src/Simplex/Chat/View.hs

@@ -69,6 +69,7 @@ import Simplex.Messaging.Agent.Protocol
 import Simplex.Messaging.Agent.Store.Entity
 import Simplex.Messaging.Client (SMPProxyFallback, SMPProxyMode (..), SocksMode (..))
 import qualified Simplex.Messaging.Crypto as C
+import Simplex.Messaging.Crypto.BIP44 (unAccountIndex)
 import Simplex.Messaging.Crypto.File (CryptoFile (..), CryptoFileArgs (..))
 import qualified Simplex.Messaging.Crypto.Ratchet as CR
 import Simplex.Messaging.Encoding
@@ -199,7 +200,7 @@ chatResponseToView hu cfg@ChatConfig {logLevel, showReactions, showFullLinks, te
   CRWallet u accounts_ -> ttyUser u $ case accounts_ of
     Nothing -> ["no wallet on this device"]
     Just [] -> ["wallet, no accounts for this profile"]
-    Just accounts -> [plain $ "accounts: " <> T.intercalate ", " (map tshow accounts)]
+    Just accounts -> [plain $ "accounts: " <> T.intercalate ", " (map (tshow . unAccountIndex) accounts)]
   CRWalletMnemonic u mnemonic -> ttyUser u [plain mnemonic]
   CRWalletAddress u a -> ttyUser u [walletAddressRow a]
   CRWalletAccountSecret u a secret -> ttyUser u [walletAddressRow a <> "  " <> plain secret]
@@ -1112,7 +1113,7 @@ viewChatCleared (AChatInfo _ chatInfo) = case chatInfo of
 
 walletAddressRow :: WalletAddress -> StyledString
 walletAddressRow WalletAddress {accountIndex, keyPath, address} =
-  plain $ tshow accountIndex <> "  " <> keyPath <> "  " <> address
+  plain $ tshow (unAccountIndex accountIndex) <> "  " <> keyPath <> "  " <> address
 
 viewContactsList :: [Contact] -> [StyledString]
 viewContactsList =
@@ -2820,7 +2821,7 @@ viewChatError isCmd logLevel testView = \case
             WEAccountBound -> "another profile holds this account"
             WEAccountNotHeld -> "this profile does not hold this account"
             WECounterUnknown -> "the next account is unknown after an import"
-            WEIndexTooLarge -> "account index must be below 2^31"
+            WEAccountsExhausted -> "every account index is used"
             WEDerivation e -> "derivation failed: " <> T.pack e
        in [plain $ "wallet: " <> reason]
     CENotResolvedLocally -> ["no matching chat found, name resolution is disabled"]

src/Simplex/Chat/Wallet.hs

@@ -8,10 +8,10 @@ module Simplex.Chat.Wallet
     WalletError (..),
     newSeedEntropy,
     entropyFromMnemonic,
+    newWalletMaster,
     seedMnemonic,
     deriveAccount,
     accountSecret,
-    checkAccountIndex,
   )
 where
 
@@ -20,21 +20,18 @@ import Control.Monad.Except
 import Control.Monad.IO.Class (liftIO)
 import Crypto.Random (ChaChaDRG)
 import qualified Data.Aeson.TH as JQ
-import Data.Bifunctor (bimap)
-import qualified Data.ByteArray as BA
+import Data.Bifunctor (first)
 import qualified Data.ByteArray.Encoding as BAE
 import Data.Text (Text)
 import Data.Text.Encoding (decodeLatin1)
-import Data.Word (Word32)
 import qualified Simplex.Messaging.Crypto.BIP32 as B32
 import qualified Simplex.Messaging.Crypto.BIP39 as B39
+import Simplex.Messaging.Crypto.BIP44 (AccountIndex, CoinType (..), bip44Path)
 import qualified Simplex.Messaging.Crypto.Secp256k1 as S
 import Simplex.Messaging.Encoding.String (strEncode)
-import Simplex.Messaging.Eth.Address (addressFromPrivateKey, ethereumPath)
+import Simplex.Messaging.Eth.Address (addressFromPrivateKey)
 import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, sumTypeJSON)
-import Simplex.Messaging.Util (liftEitherWith, liftError')
-
-type AccountIndex = Word32
+import Simplex.Messaging.Util (liftError')
 
 type AccountKey = S.Secp256k1PrivateKey
 
@@ -53,39 +50,34 @@ data WalletError
   | WEAccountBound
   | WEAccountNotHeld
   | WECounterUnknown
-  | WEIndexTooLarge
+  | WEAccountsExhausted
   | WEDerivation {derivationError :: String}
   deriving (Eq, Show)
 
-checkAccountIndex :: AccountIndex -> Either WalletError ()
-checkAccountIndex n = () <$ accountPath n
+masterStrength :: B39.EntropyStrength
+masterStrength = B39.ES256
 
-accountPath :: AccountIndex -> Either WalletError [Word32]
-accountPath n = maybe (Left WEIndexTooLarge) Right $ ethereumPath n 0
+newSeedEntropy :: TVar ChaChaDRG -> STM B39.WalletEntropy
+newSeedEntropy = B39.randomEntropy masterStrength
 
-masterStrength :: B39.MnemonicStrength
-masterStrength = B39.MS256
-
-newSeedEntropy :: TVar ChaChaDRG -> STM BA.ScrubbedBytes
-newSeedEntropy g = B39.mnemonicToEntropy <$> B39.randomMnemonic masterStrength g
-
-entropyFromMnemonic :: Text -> Either WalletError BA.ScrubbedBytes
-entropyFromMnemonic phrase = case B39.parseMnemonic phrase of
-  Right m | length (B39.mnemonicWords m) == B39.strengthWordCount masterStrength ->
-    Right $ B39.mnemonicToEntropy m
+entropyFromMnemonic :: Text -> Either WalletError B39.WalletEntropy
+entropyFromMnemonic phrase = case B39.parsePhrase phrase of
+  Right ent | B39.entropyWordCount ent == 24 -> Right ent
   _ -> Left WEBadMnemonic
 
-seedMnemonic :: BA.ScrubbedBytes -> Either WalletError Text
-seedMnemonic = bimap WEDerivation (decodeLatin1 . B39.mnemonicPhrase) . B39.entropyToMnemonic
+newWalletMaster :: B39.WalletEntropy -> Either WalletError B32.WalletMaster
+newWalletMaster ent = first WEDerivation $ B32.mkWalletMaster ent ""
 
-deriveAccount :: TVar ChaChaDRG -> BA.ScrubbedBytes -> AccountIndex -> IO (Either WalletError (AccountKey, WalletAddress))
-deriveAccount g entropy n = runExceptT $ do
-  path <- liftEither $ accountPath n
-  m <- liftEitherWith WEDerivation $ B39.entropyToMnemonic entropy
-  master <- liftError' WEDerivation $ B32.masterKey g (B39.mnemonicToSeed m "")
-  k <- B32.xkKey <$> liftError' WEDerivation (B32.derivePath g master path)
+seedMnemonic :: B32.WalletMaster -> Text
+seedMnemonic = decodeLatin1 . B39.entropyPhrase . B32.masterEntropy
+
+deriveAccount :: TVar ChaChaDRG -> B32.WalletMaster -> AccountIndex -> IO (Either WalletError (AccountKey, WalletAddress))
+deriveAccount g master n = runExceptT $ do
+  k <- B32.xkKey <$> liftError' WEDerivation (B32.derivePath g (B32.walletMasterKey master) path)
   a <- liftIO $ addressFromPrivateKey g k
   pure (k, WalletAddress {accountIndex = n, keyPath = decodeLatin1 $ B32.renderPath path, address = decodeLatin1 $ strEncode a})
+  where
+    path = bip44Path Ethereum n
 
 accountSecret :: AccountKey -> Text
 accountSecret k = "0x" <> decodeLatin1 (BAE.convertToBase BAE.Base16 $ S.unPrivateKey k)

tests/WalletTests.hs

@@ -5,18 +5,21 @@ module WalletTests where
 import ChatClient
 import ChatTests.DBUtils
 import ChatTests.Utils
-import Control.Monad (void)
 import qualified Data.ByteArray as BA
 import qualified Data.ByteArray.Encoding as BAE
 import qualified Data.ByteString.Char8 as B
 import Data.Char (toUpper)
 import Data.Either (isRight)
 import Data.List (nub)
+import Data.Maybe (fromJust)
 import Data.Text (Text)
 import qualified Data.Text as T
-import Simplex.Chat.Wallet (AccountIndex, AccountKey, WalletAddress (..), WalletError (..), accountSecret, deriveAccount, entropyFromMnemonic, seedMnemonic)
+import Data.Word (Word32)
+import Simplex.Chat.Wallet (AccountKey, WalletAddress (..), WalletError (..), accountSecret, deriveAccount, entropyFromMnemonic, seedMnemonic)
 import qualified Simplex.Messaging.Crypto as C
+import qualified Simplex.Messaging.Crypto.BIP32 as B32
 import qualified Simplex.Messaging.Crypto.BIP39 as B39
+import Simplex.Messaging.Crypto.BIP44 (mkAccountIndex)
 import qualified Simplex.Messaging.Crypto.Secp256k1 as S
 import Simplex.Messaging.Encoding.String (strEncode)
 import Simplex.Messaging.Eth.Address (addressFromPrivateKey)
@@ -29,18 +32,18 @@ testPhrase12 = T.unwords $ replicate 11 "abandon" <> ["about"]
 testPhrase24 :: Text
 testPhrase24 = T.unwords $ replicate 23 "abandon" <> ["art"]
 
-seedEntropy :: Text -> BA.ScrubbedBytes
-seedEntropy phrase = B39.mnemonicToEntropy . either error id $ B39.parseMnemonic phrase
+walletMaster :: Text -> B32.WalletMaster
+walletMaster phrase = either error id $ B32.mkWalletMaster (either error id $ B39.parsePhrase phrase) ""
 
-walletAccount :: Text -> AccountIndex -> IO (AccountKey, WalletAddress)
+walletAccount :: Text -> Word32 -> IO (AccountKey, WalletAddress)
 walletAccount phrase n = do
   g <- C.newRandom
-  either (error . show) id <$> deriveAccount g (seedEntropy phrase) n
+  either (error . show) id <$> deriveAccount g (walletMaster phrase) (fromJust $ mkAccountIndex n)
 
 addressFromSecret :: String -> IO String
 addressFromSecret secret = do
   g <- C.newRandom
-  k <- either error id <$> S.mkPrivateKey g (either error id $ BAE.convertFromBase BAE.Base16 (B.drop 2 $ B.pack secret))
+  let k = either error id $ S.mkPrivateKey (either error id $ BAE.convertFromBase BAE.Base16 (B.drop 2 $ B.pack secret))
   B.unpack . strEncode <$> addressFromPrivateKey g k
 
 exportRow :: HasCallStack => String -> (String, String, String, String)
@@ -64,19 +67,15 @@ walletDerivationTests = do
     addrs <- mapM (fmap (address . snd) . walletAccount testPhrase12) [0 .. 9]
     length (nub addrs) `shouldBe` 10
   Hspec.it "renders a secret whose first byte is zero with 64 hex digits" $ do
-    g <- C.newRandom
-    k <- either error id <$> S.mkPrivateKey g (BA.convert $ B.pack ('\0' : replicate 31 '\1'))
+    let k = either error id $ S.mkPrivateKey (BA.convert $ B.pack ('\0' : replicate 31 '\1'))
     let secret = T.unpack $ accountSecret k
     take 4 secret `shouldBe` "0x00"
     length secret `shouldBe` 66
   Hspec.it "renders the path an account is derived at" $ do
     (keyPath . snd <$> walletAccount testPhrase12 0) `shouldReturn` "m/44'/60'/0'/0/0"
     (keyPath . snd <$> walletAccount testPhrase12 7) `shouldReturn` "m/44'/60'/7'/0/0"
-  Hspec.it "rejects an account index at or above 2^31" $ do
-    g <- C.newRandom
-    (void <$> deriveAccount g (seedEntropy testPhrase12) 2147483648) `shouldReturn` Left WEIndexTooLarge
   Hspec.it "round-trips the phrase it was imported from" $
-    seedMnemonic (seedEntropy testPhrase24) `shouldBe` Right testPhrase24
+    seedMnemonic (walletMaster testPhrase24) `shouldBe` testPhrase24
   Hspec.it "accepts only 24 words with a valid checksum" $ do
     entropyFromMnemonic testPhrase24 `shouldSatisfy` isRight
     entropyFromMnemonic testPhrase12 `shouldBe` Left WEBadMnemonic
@@ -304,11 +303,11 @@ testWalletIndexTooLarge ps = withNewTestChat ps "alice" aliceProfile $ \alice ->
   alice ##> "/_wallet create new"
   alice <## "wallet, no accounts for this profile"
   alice ##> "/_wallet address account=2147483648"
-  alice <## "wallet: account index must be below 2^31"
+  alice <## "bad chat command: Failed reading: empty"
   alice ##> "/_wallet bind 1 account=2147483648"
-  alice <## "wallet: account index must be below 2^31"
+  alice <## "bad chat command: Failed reading: empty"
   alice ##> "/_wallet export account 1 2147483648"
-  alice <## "wallet: account index must be below 2^31"
+  alice <## "bad chat command: Failed reading: empty"
   alice ##> "/_wallet address account=4294967296"
   alice <## "bad chat command: Failed reading: empty"
   alice ##> "/_wallet bind 1 account=4294967296"
@@ -316,4 +315,4 @@ testWalletIndexTooLarge ps = withNewTestChat ps "alice" aliceProfile $ \alice ->
   alice ##> "/_wallet bind 1 account=2147483647"
   alice `accountBound` "2147483647"
   alice ##> "/_wallet bind 1"
-  alice <## "wallet: account index must be below 2^31"
+  alice <## "wallet: every account index is used"