Files
simplexmq/tests/Test.hs
T
sh 209f7826cb smp-server: support namespaces (#1784)
* smp-server: namespaces resolver scaffolding

* smp-server: Names resolver hardening + cleanup

* smp-server: fuse parallel dispatchers

* smp-server: JSON wire format for NameRecord + Names.hs restructure

* smp-server: redact RpcAuth in Show

* smp-server: JSON wire fixups + spec rewrite + small cleanups

* plan: prepend implementation-diverged banner

* move SimplexName into shared module

* smp-server: name + contract whitelist on RSLV

* smp-server: address audit findings (canonical JSON, INI guards, SSRF, TLD case, shutdown)

* smp-server: round 2 audit fixes (label case, response cap, ipv6 link-local)

* smp-server: round 3 audit fixes (SSRF coverage, drop noop closeManager, CSV order)

* smp-server: round 4 audit fixes (0X-hex host, expanded IPv6 forms, pingEndpoint timeout)

* smp-server: hardcode TldRegistries (drop registry_tld_* INI keys)

* smp-server: round 6 audit fixes (IPv6 SSRF, redirects, ASCII labels)

- Reject IPv6 aliases of 169.254.169.254 (IPv4-compatible / IPv4-mapped /
  6to4 / NAT64) via numeric range check on parsed IPv6.
- Disable HTTP redirects on the Eth RPC request.
- Restrict SimplexName labels to ASCII (Cyrillic/Greek/full-width otherwise
  hash to different on-chain records and diverge from UTS-46 registrars).
- pingEndpoint: only JsonRpcErr means "reachable"; transport/decode failures
  fail startup. boundedIniInt: readMaybe over partial read.
- Add 127.0.0.0/8 and 0.0.0.0 to isLoopback.
- Replace hand-rolled hex helpers with Data.ByteArray.Encoding; raise
  managerConnCount to match rpcMaxConcurrency; hex Show for NameOwner.
- Fuse parallel http/https when into unless+case; drop reverse/re-reverse
  in mkDomain TLDWeb; first AbiInvariantViolated; Nothing <$ decodeAddress;
  forM_ (eitherToMaybe ...); >>= chain in NameOwner FromJSON.
- Drop dead imports/exports/pragmas and two restating comments.
- Tests: factor unsafeOwner/unsafeLink, addr1/2/3, testNamesConfig; add
  non-ASCII label rejection coverage.

* namespace: bound parser input to 253 bytes (DoS defense)

The bare-name fallback and bareDomain parser would otherwise consume
arbitrarily many non-space bytes via takeWhile1 before any validation
or length check. A crafted multi-megabyte token would be decoded as
UTF-8 and re-parsed in full before being rejected.

Introduce `boundedNonSpace` (scan with 253-byte cap) at the two
takeWhile1 sites. Inputs longer than 253 bytes leave residue that
parseOnly's implicit endOfInput rejects, so the parser fails fast
without ever allocating the full input.

The bound is the DNS full-domain limit, chosen for being a familiar
ceiling generous enough to cover any realistic SimpleX name (longest
plausible @user.subdomain.simplex stays well under 100 bytes). No
per-label cap — SimpleX names don't go through DNS label resolution
and there's no semantic reason to constrain individual labels.

* namespace: switch to Python HTTP resolver + agent plumbing (#1796)

* namespace: relax resolver_endpoint validation (path prefix, http without auth)

validateUrl gains two operator-friendly relaxations and a regression test:

- Allow a path prefix (e.g. https://gw.example.com:443/snrc) for a resolver
  behind a reverse-proxy sub-path; /resolve/<name> and /health are appended
  (HttpResolver already strips one trailing slash, so root and sub-path
  behave identically). Query/fragment/userinfo stay rejected.

- Off-loopback, reject only http WITH resolver_auth (the Authorization header
  would travel in cleartext). http without auth is now allowed (no secret to
  leak; resolver data is public — also lets dev setups reach a host resolver
  via http://host.docker.internal). https is always allowed, with or without
  auth. Plain http has no response integrity; intended for trusted/local
  networks only.

Exports validateUrl and adds validateUrlSpec (11 cases) to SMPNamesTests.

* namespace: NameRecord links as arrays (multi-link, cap 5)

* namespace: distinct RSLV error responses

RSLV collapsed every non-hit (no resolver, malformed name, not found,
backing-store failure) to ERR AUTH, so a client iterating its configured
servers could not tell "this router has no resolver, try the next" from
"name not registered, stop", and a transient backend error read as an
authoritative miss.

Names capability is runtime config, orthogonal to the linear SMP version
(a future v21 router without [NAMES] must still advertise v21), so it is
signalled by a command-time error like allowSMPProxy, not by the version
range:

  no resolver configured -> ERR CMD PROHIBITED  (client skips, tries next)
  backing-store failure   -> ERR INTERNAL        (transient: retry/surface)
  not found / malformed   -> ERR AUTH            (authoritative "no such name")

Update the protocol spec error table and add agent tests for the
no-resolver (CMD PROHIBITED) and backend-failure (INTERNAL) paths.

* refactor(names): server role + one error type

Addresses epoberezkin's review (PR #1784). Name resolution becomes a
server role like proxy; the agent owns resolution + server selection;
one error type flows through the whole stack.

- ServerRoles gains `names`; UserServers gains `nameSrvs` (opt-in list);
  resolveSimplexName drops the explicit server arg and picks a
  names-capable server via getNextServer.
- RSLV carries SimplexNameDomain (was RslvRequest): no JSON on the wire,
  contract dropped, name validated at parse (invalid -> CMD SYNTAX).
- Version check moves from the encoder to Client.hs (no ERR to server).
- ErrorType.NAME {nameErr :: NameErrorType} (+ AgentErrorType.NAME),
  wire- and JSON-encoded; resolver errors surface with diagnostics.
  Success response renamed NAME -> RNAME to free the collision.
- NameOwner -> EthAddress (record selector); NameRecord derives FromJSON
  and gains field-ordered Encoding; per-field caps removed.
- Remove newEnvWithNames / runSMPServerBlockingWithNames test seams;
  stub resolver folded into ServerConfig.namesResolverCall_.

* test(server): update stats backup line count

NameResolverStatsData adds 6 lines to the server stats backup (the
"rslvStats:" header plus the reqs/succ/notFound/resolverErrs/disabled
fields), so testRestoreMessages' expected stats-backup line count is
95 -> 101.

* feat(names): public-namespace resolution via RSLV/RNAME

SNRC names resolver role: RSLV command -> HTTP resolver -> RNAME record.
Agent owns server selection (ServerRoles.names); NAME error family; async,
concurrency-bounded resolution; length-prefixed extensible wire; spec.

* remove comments

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* simplify

* move tests name

* simplify: text addresses, Tail JSON, drop admitRslv

* fix

* remove spaghetti

* reduce diff

* async again, refactor

* different threads limit for name resolutions

* remove comment

* FromField instance for SimplexNameInfo

* remove comments

* unStrJSON

* add sameConnShortLink

* remove scheme prefix

* remove unused import

* remove connecttarget tests

* remove comment

* comment

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-06-30 22:54:55 +01:00

204 lines
9.3 KiB
Haskell

{-# LANGUAGE CPP #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE TypeApplications #-}
import AgentTests (agentCoreTests, agentTests)
import CLITests
import Control.Concurrent (threadDelay)
import qualified Control.Exception as E
import Control.Logger.Simple
import CoreTests.BatchingTests
import CoreTests.CryptoFileTests
import CoreTests.CryptoTests
import CoreTests.EncodingTests
import CoreTests.MsgStoreTests
import CoreTests.RetryIntervalTests
import CoreTests.SOCKSSettings
import CoreTests.StoreLogTests
import CoreTests.TSessionSubs
import CoreTests.UtilTests
import CoreTests.VersionRangeTests
import FileDescriptionTests (fileDescriptionTests)
import GHC.IO.Exception (IOException (..))
import qualified GHC.IO.Exception as IOException
import RSLVTests (rslvTests)
import RemoteControl (remoteControlTests)
import SMPNamesTests (smpNamesTests)
import SMPProxyTests (smpProxyTests)
import ServerTests
import Simplex.Messaging.Server.Env.STM (AStoreType (..))
import Simplex.Messaging.Server.MsgStore.Types (SMSType (..), SQSType (..))
import Simplex.Messaging.Transport (TLS, Transport (..))
-- import Simplex.Messaging.Transport.WebSockets (WS)
import System.Directory (createDirectoryIfMissing, removeDirectoryRecursive)
import System.Environment (setEnv)
import Test.Hspec hiding (fit, it)
import Util
import XFTPAgent
import XFTPCLI (xftpCLIFileTests)
import Simplex.FileTransfer.Server.Env (AFStoreType (..))
import Simplex.FileTransfer.Server.Store (SFSType (..))
import XFTPServerTests (xftpServerTests)
import WebTests (webTests)
import XFTPWebTests (xftpWebTests)
#if defined(dbPostgres)
import Fixtures
import SMPAgentClient (testDB)
import Simplex.Messaging.Agent.Store.Postgres.Migrations.App
import Simplex.Messaging.Agent.Store.Postgres.Util (dropAllSchemasExceptSystem)
#else
import AgentTests.SchemaDump (schemaDumpTest)
#endif
#if defined(dbServerPostgres)
import CoreTests.XFTPStoreTests (xftpStoreTests, xftpMigrationTests)
import NtfServerTests (ntfServerTests)
import NtfClient (ntfTestServerDBConnectInfo, ntfTestStoreDBOpts)
import SMPClient (testServerDBConnectInfo, testStoreDBOpts)
import Simplex.Messaging.Notifications.Server.Store.Migrations (ntfServerMigrations)
import Simplex.Messaging.Server.QueueStore.Postgres.Migrations (serverMigrations)
import XFTPClient (testXFTPDBConnectInfo)
#endif
#if defined(dbPostgres) || defined(dbServerPostgres)
import PostgresSchemaDump (postgresSchemaDumpTest)
import SMPClient (postgressBracket)
#endif
logCfg :: LogConfig
logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}
main :: IO ()
main = do
setLogLevel testLogLevel
withGlobalLogging logCfg $ do
setEnv "APNS_KEY_ID" "H82WD9K9AQ"
setEnv "APNS_KEY_FILE" "./tests/fixtures/AuthKey_H82WD9K9AQ.p8"
hspec
#if defined(dbPostgres)
. aroundAll_ (postgressBracket testDBConnectInfo)
#endif
. before_ (createDirectoryIfMissing False "tests/tmp")
. after_ (eventuallyRemove "tests/tmp" 3)
$ do
describe "Core tests" $ do
describe "Batching tests" batchingTests
describe "Encoding tests" encodingTests
describe "Version range" versionRangeTests
describe "Encryption tests" cryptoTests
describe "Encrypted files tests" cryptoFileTests
describe "Message store tests" msgStoreTests
describe "Retry interval tests" retryIntervalTests
describe "SOCKS settings tests" socksSettingsTests
#if defined(dbServerPostgres)
around_ (postgressBracket testServerDBConnectInfo) $
describe "Store log tests" storeLogTests
#else
describe "Store log tests" storeLogTests
#endif
describe "TSessionSubs tests" tSessionSubsTests
describe "Util tests" utilTests
describe "Names resolver tests" smpNamesTests
describe "RSLV functional API tests" rslvTests
describe "Agent core tests" agentCoreTests
#if defined(dbServerPostgres)
around_ (postgressBracket testServerDBConnectInfo) $
describe "SMP server schema dump" $
postgresSchemaDumpTest
serverMigrations
[ "20250320_short_links" -- snd_secure moves to the bottom on down migration
] -- skipComparisonForDownMigrations
testStoreDBOpts
"src/Simplex/Messaging/Server/QueueStore/Postgres/server_schema.sql"
around_ (postgressBracket testServerDBConnectInfo) $ do
-- xdescribe "SMP server via TLS, postgres+jornal message store" $
-- before (pure (transport @TLS, ASType SQSPostgres SMSJournal)) serverTests
describe "SMP server via TLS, postgres-only message store" $
before (pure (transport @TLS, ASType SQSPostgres SMSPostgres)) serverTests
#endif
describe "SMP server via TLS, jornal message store" $ do
describe "SMP syntax" $ serverSyntaxTests (transport @TLS)
before (pure (transport @TLS, ASType SQSMemory SMSJournal)) serverTests
describe "SMP server via TLS, memory message store" $
before (pure (transport @TLS, ASType SQSMemory SMSMemory)) serverTests
-- xdescribe "SMP server via WebSockets" $ do
-- describe "SMP syntax" $ serverSyntaxTests (transport @WS)
-- before (pure (transport @WS, ASType SQSMemory SMSJournal)) serverTests
#if defined(dbServerPostgres)
around_ (postgressBracket ntfTestServerDBConnectInfo) $
describe "Ntf server schema dump" $
postgresSchemaDumpTest
ntfServerMigrations
[] -- skipComparisonForDownMigrations
ntfTestStoreDBOpts
"src/Simplex/Messaging/Notifications/Server/Store/ntf_server_schema.sql"
around_ (postgressBracket ntfTestServerDBConnectInfo) $ do
describe "Notifications server (SMP server: memory store)" $
ntfServerTests (transport @TLS, ASType SQSMemory SMSMemory)
around_ (postgressBracket testServerDBConnectInfo) $ do
-- xdescribe "Notifications server (SMP server: postgres+jornal store)" $
-- ntfServerTests (transport @TLS, ASType SQSPostgres SMSJournal)
describe "Notifications server (SMP server: postgres-only store)" $
ntfServerTests (transport @TLS, ASType SQSPostgres SMSPostgres)
around_ (postgressBracket testServerDBConnectInfo) $ do
-- xdescribe "SMP client agent, postgres+jornal message store" $ agentTests (transport @TLS, ASType SQSPostgres SMSJournal)
describe "SMP client agent, server postgres-only message store" $ agentTests (transport @TLS, ASType SQSPostgres SMSPostgres)
-- xdescribe "SMP proxy, postgres+jornal message store" $
-- before (pure $ ASType SQSPostgres SMSJournal) smpProxyTests
describe "SMP proxy, postgres-only message store" $
before (pure $ ASType SQSPostgres SMSPostgres) smpProxyTests
#endif
-- xdescribe "SMP client agent, server jornal message store" $ agentTests (transport @TLS, ASType SQSMemory SMSJournal)
describe "SMP client agent, server memory message store" $ agentTests (transport @TLS, ASType SQSMemory SMSMemory)
describe "SMP proxy, jornal message store" $
before (pure $ ASType SQSMemory SMSJournal) smpProxyTests
describe "XFTP" $ do
describe "XFTP server" $
before (pure $ AFSType SFSMemory) xftpServerTests
describe "XFTP file description" fileDescriptionTests
describe "XFTP CLI (memory)" $
before (pure $ AFSType SFSMemory) xftpCLIFileTests
describe "XFTP agent" $
before (pure $ AFSType SFSMemory) xftpAgentTests
#if defined(dbServerPostgres)
around_ (postgressBracket testXFTPDBConnectInfo) $ do
describe "XFTP Postgres store operations" xftpStoreTests
describe "XFTP migration round-trip" xftpMigrationTests
describe "XFTP server (PostgreSQL)" $
before (pure $ AFSType SFSPostgres) xftpServerTests
describe "XFTP agent (PostgreSQL)" $
before (pure $ AFSType SFSPostgres) xftpAgentTests
describe "XFTP CLI (PostgreSQL)" $
before (pure $ AFSType SFSPostgres) xftpCLIFileTests
#endif
#if defined(dbPostgres)
describe "XFTP Web Client" $ xftpWebTests (dropAllSchemasExceptSystem testDBConnectInfo)
#else
describe "XFTP Web Client" $ xftpWebTests (pure ())
#endif
describe "XRCP" remoteControlTests
describe "Web" webTests
describe "Server CLIs" cliTests
#if defined(dbPostgres)
around_ (postgressBracket testDBConnectInfo) $
describe "Agent PostgreSQL schema dump" $
postgresSchemaDumpTest
appMigrations
["20250322_short_links"] -- snd_secure and last_broker_ts columns swap order on down migration
(testDBOpts testDB)
"src/Simplex/Messaging/Agent/Store/Postgres/Migrations/agent_postgres_schema.sql"
#else
describe "Agent SQLite schema dump" schemaDumpTest
#endif
eventuallyRemove :: FilePath -> Int -> IO ()
eventuallyRemove path retries = case retries of
0 -> action
n ->
action `E.catch` \ioe@IOError {ioe_type, ioe_filename} -> case ioe_type of
IOException.UnsatisfiedConstraints | ioe_filename == Just path -> threadDelay 1000000 >> eventuallyRemove path (n - 1)
_ -> E.throwIO ioe
where
action = removeDirectoryRecursive path