From ea43df2349f6d3dedd60d5e4aed21fd99316cad9 Mon Sep 17 00:00:00 2001 From: brenzi Date: Wed, 16 Sep 2026 10:58:44 +0200 Subject: [PATCH] extend SMP protocol to support name availability queries by labelhash (#1863) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * implement resolving 2LD names by labelhash * tiny test addition * simplify language * report expiry and availability correctly * compare block instead of wall clock * simplify language * simplify language * map resolver 410 to NAME NOT_FOUND (a lapsed name is an answer, not a failure) * split error into a machine-readable code and a human message * resolver: reduce comments * resolver: reduce comments, remove REVIEW.html * extend SMP protocol to support name availability queries with accurate and meaningful replies * review fixes * more review fixes * docs shortening and other review fixes * add catch all for furture variants * adversarial review (against simplex-chat) fix * next iteration fixes * adapt house style * eth_call guard and cache constants * revert NAVL command and wrap it all into RSLV * doc fixes * Update src/Simplex/Messaging/Server/Names.hs Co-authored-by: Evgeny * Update src/Simplex/Messaging/Protocol.hs Co-authored-by: Evgeny * Update src/Simplex/Messaging/Protocol.hs Co-authored-by: Evgeny * Update src/Simplex/Messaging/Protocol.hs Co-authored-by: Evgeny * Update src/Simplex/Messaging/Protocol.hs Co-authored-by: Evgeny * protocol types refactoring * next iteration on types only * rentPrices map * claude answering ep review. to be continued... * separate labelhash and plaintext names cleanly * align implementation with latest type changes to test against client * fix adversarial review findings * trim diff * align resolver with contract changes for names v2 * fix reverse compatibility with .testing mainnet * fix more robustly * NameQuery simplification * revert drive-by refactoring * implement full type change and introduce resolver endpoint versioning * fix stale docs * derive NameRegistration JSON the same way on every build sumTypeJSON switches to the _owsf form on swift builds, but this JSON is the RNAME payload and the resolver's HTTP contract, so a swift client and a Linux relay would disagree on every field. taggedObjectJSON is what it already resolves to everywhere else. The label modifier keeps the reservedReason_ collision escape out of the API, as AgentWorkersDetails does: both arms now say reservedReason. * fix resolver boundary: status before body, cover /v2, drop dead field httpGet read the response body before checking the status, so an oversized error page surfaced as a transient "response too large" instead of the authoritative status. Reverting it to its previous shape restores that and removes the status test both callers had been re-deriving. registration() had no tests at all, though it is the endpoint SMP v22 consumes. RegistrationV2Tests covers the three answer shapes, the error paths and the exact key set of each, which is the wire contract. auctionUntil was always None with no consumer. The spec claimed a reason word travels unchanged; a resolver can only send a word it has, and SNRC's registry records a number. * fix review findings * resolver errors say what went wrong, not "no such name" /v2/resolve answers 200, 400 or 502, and an unregistered name is NRAvailable, so no status means "not registered". Mapping 400/404/410 to NOT_FOUND made a misconfigured relay deny every name, and hid a relay upgraded ahead of its resolver. All three now surface as RESOLVER. rslvNotFound would have gone dead, so it counts what its name says: an availability answer the encoder downgrades for a session below v22. The wire is unchanged. A hashed query the registrar cannot name is refused with 502 rather than answered with a record named "unknown", which the client rejects anyway. NRRUnknown is capped to 32 printable characters again, as the spec says. A registered name that is also reserved no longer offers a date it will never free up on. rentPrices is registrationPrices throughout, and yearPriceUSD is USDCents rather than a bare Int64. * fix regressions found reviewing the last two commits resolveNameMsg read the version off thParams', which on the PFWD path is the proxy's session, not the client's. It takes the version as an argument now, so each call site passes its own — the forwarded one uses fwdVersion. reservedReasonOf matched the reason words before capping, so "internal review" became NRRUnknown "internal", which encodes back as NRRInternal. Capping precedes the match, so what is kept encodes to what it decoded. Four agent tests still pinned NAME NOT_FOUND from a 404 stub, and two spec statements still described the old mapping. A registrar that does not record labels cannot answer a hashed query, which the resolver README now says. * docs sweep * simplify encoding * drop resolver caching (#1866) * rename * remove trailing_ filter * fix resolver v2 for subnames (#1867) * fix resolver v2 for subnames * simplify doc * resolver should report response truth freshness (#1868) * first shot at reporting freshness * fix review findings * renaming --------- Co-authored-by: sh Co-authored-by: Evgeny --- .github/workflows/build.yml | 27 + .gitignore | 1 + protocol/simplex-messaging.md | 166 ++- scripts/resolver/README.md | 256 ++++- scripts/resolver/docker-compose.yml | 6 + scripts/resolver/service/snrc-resolve.py | 477 ++++++++- scripts/resolver/service/test_snrc_resolve.py | 953 ++++++++++++++++++ src/Simplex/Messaging/Agent.hs | 6 +- src/Simplex/Messaging/Agent/Client.hs | 4 +- src/Simplex/Messaging/Client.hs | 23 +- src/Simplex/Messaging/Names/Record.hs | 100 +- src/Simplex/Messaging/Protocol.hs | 54 +- src/Simplex/Messaging/Server.hs | 19 +- src/Simplex/Messaging/Server/Names.hs | 19 +- .../Messaging/Server/Names/HttpResolver.hs | 24 +- src/Simplex/Messaging/Server/Prometheus.hs | 4 +- src/Simplex/Messaging/SimplexName.hs | 36 +- src/Simplex/Messaging/Transport.hs | 21 +- tests/AgentTests/ResolveNameTests.hs | 61 +- tests/NamesResolverServer.hs | 4 +- tests/RSLVTests.hs | 152 ++- tests/SMPNamesTests.hs | 146 ++- 22 files changed, 2359 insertions(+), 200 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c841ec073..85b7522a9 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -301,3 +301,30 @@ jobs: echo "All "$attempts" attempts failed." exit 1 fi + +# ============================= +# Resolver test job +# ============================= + +# The SNRC resolver is Python, so this job needs none of the Haskell toolchain +# above and runs independently of it. + + resolver-test: + name: "resolver (python)" + runs-on: ubuntu-latest + steps: + - name: Clone project + uses: actions/checkout@v3 + + - name: Set up Python + # Matches the runtime stage of scripts/resolver/service/Dockerfile. + uses: actions/setup-python@v5 + with: + python-version: "3.13" + + - name: Install resolver dependencies + # Must match scripts/resolver/service/pyproject.toml. + run: python -m pip install "eth-hash[pycryptodome]>=0.7" + + - name: Test + run: python -m unittest discover -s scripts/resolver/service -v diff --git a/.gitignore b/.gitignore index 9d27c4ccb..9550e48c0 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,4 @@ cabal.project.local~ *.tix .coverage +__pycache__/ diff --git a/protocol/simplex-messaging.md b/protocol/simplex-messaging.md index be7bd448f..a80d211b2 100644 --- a/protocol/simplex-messaging.md +++ b/protocol/simplex-messaging.md @@ -86,7 +86,7 @@ It's designed with the focus on communication security and integrity, under the It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system. -This document describes SMP protocol version 20. Versions 1-5 are discontinued. The version history: +This document describes SMP protocol version 22. Versions 1-5 are discontinued. The version history: - v1: binary protocol encoding - v2: message flags (used to control notifications) @@ -108,6 +108,7 @@ This document describes SMP protocol version 20. Versions 1-5 are discontinued. - v19: service subscriptions to messages (SUBS, NSUBS, SOKS, ENDS, ALLS commands) - v20: public namespaces resolver (RSLV command, RNAME response) — direct or forwarded via PFWD - v21: server public information in handshake +- v22: `RNAME` says whether a name can be registered, not only what it resolves to ## Introduction @@ -1451,53 +1452,150 @@ reads `NameRecord` from. The reference implementation forwards each RSLV to a companion REST resolver process (`scripts/resolver/snrc-resolve.py`) that queries the SNRC contract on Ethereum; alternative backings (different chains, DHT, etc.) are valid as long as they expose the documented HTTP shape (`GET -/resolve/` returning a `NameRecord` on 200, 404 / 400 for unknown names -or TLDs, 502 for upstream RPC failures) or substitute a different transport -while still returning a `NameRecord` matching the encoding below. +/v2/resolve/` returning a `NameRegistration` on 200 for every +registration shape, 400 for unknown TLDs, 502 for upstream failures) or +substitute a different transport returning the same JSON. The resolver API is +versioned separately from this protocol: `/v1/resolve/` returns a bare +`NameRecord` and is what relays before v22 call as `/resolve/`. #### Resolve name command -The `RSLV` command carries the canonical fully-qualified name directly as the -payload (not JSON): +The `RSLV` command carries the query as text, not JSON. A client sends the +hashed form only from v22, and the name itself below it: ```abnf -rslv = %s"RSLV" SP domain ; domain = canonical name as non-space bytes, consuming the remainder of the transmission +rslv = %s"RSLV" SP query +query = domain / hashed ; hashed only from v22 +domain = 1*253 OCTET ; the name as text +hashed = "[" 64HEXDIG "]" tld ; keccak-256 of the second-level label +tld = %s".simplex" / %s".testing" ``` `domain` is the UTF-8 canonical fully-qualified name with the TLD always explicit (e.g. `privacy.simplex`, `test.testing`, `example.com`), bounded to 253 bytes. +**Hashed labels.** The query is a name, or the keccak-256 of a second-level +label in ENS's bracketed form. A label can only be letters, digits and hyphens, +so `[` tells the two apart and no tag is needed. + +Only a second-level name may be hashed. A name with subnames is sent as text: it +is resolved rather than priced, and its record names it anyway. A web TLD has no +registry to key a hash on. + +From v22 a client MUST send the hash. Older routers can only read the name, so a +client on an older session sends the name. A router answering a hashed query +does not know the label's length, so it cannot check a minimum-length policy +either: the client does that, from the pricing it is sent. + +The same form reaches the backing resolver, which is what its HTTP API takes, so +the query is one string end to end. + +A hashed query still answers with the name. The registrar records the plaintext +label when a name is registered, keyed by the hash of that label, so a router can +look up what the hash stands for without ever being told. The router is not +trusted for it: a client MUST check that the record names the name it asked +about, and reject the answer otherwise. A registry that does not record the +label cannot answer a hashed query at all, and the router answers `ERR NAME +RESOLVER` rather than a record it knows the client will reject. An unregistered +name has no recorded label, so it cannot be looked up either. + **Server-side validation.** The names router parses `domain` as a fully-qualified name (TLD required — bare labels are rejected) and forwards it to the configured backing resolver, which is the source of truth for which on-chain registry maps to each TLD. -The names router responds with either an `RNAME` response carrying the resolved -record, or an `ERR NAME` error whose subcode a client iterating across several -configured servers can act on distinctly: +The names router responds with either an `RNAME` response saying what it knows +about the name, or an `ERR NAME` error whose subcode a client iterating across +several configured servers can act on distinctly: | Response | Condition | Client action | |---|---|---| -| `RNAME` | record resolved | use it | -| `ERR NAME NOT_FOUND` | name not registered, unknown TLD, or malformed name | authoritative "no such name" — stop | +| `RNAME` | the router read the registry | use it | +| `ERR NAME NOT_FOUND` | below v22 only: every name that does not resolve. From v22 a router never sends it | stop, and do not read it as registrable | | `ERR NAME NO_RESOLVER` | this router has no resolver (names role not enabled) | skip this server, try the next | -| `ERR NAME RESOLVER ` | transient failure: backing resolver error (upstream 5xx, transport, timeout, decode) | transient — retry or surface, do not treat as "not found" | +| `ERR NAME RESOLVER ` | the router cannot state an answer completely: no registrar or price oracle for the TLD, an unreachable chain, a transport failure, a timeout, a registration it could not date or resolve | surface ``; retry only if it reads as transient | A client SHOULD NOT broadcast a `name` to further servers after a name-capable router has answered (`NOT_FOUND` or `RESOLVER`), since that router has already seen the lookup key; `NO_RESOLVER` discloses nothing about the name beyond the fact that this router cannot resolve, so iterating past it is safe. -#### Name record response +#### Name response -The `RNAME` response carries a JSON-encoded record as the payload: +`RNAME` answers both what a name resolves to and whether it can be registered. ```abnf -rname = %s"RNAME" SP json-bytes ; json-bytes consumes the remainder of the transmission +rname = %s"RNAME" SP nameResponse ``` -`json-bytes` MUST be a UTF-8 JSON object with the following schema: +`nameResponse` is a UTF-8 JSON object consuming the remainder of the +transmission: the `registration` below, and `lastBlockTs`, the block timestamp the +registry was read at. Money is US cents, times are seconds since the Unix epoch, and lengths are +characters. + +| Field | JSON type | Constraints | +|---|---|---| +| `lastBlockTs` | number | block timestamp the answer was read at. Absent only from a v20/v21 router, which sent the record alone | +| `registration` | object | below | + +A router reads the registry through a node of its own, which can lag. A name +registered or changed after `lastBlockTs` still reads as it did before, so a client +MUST NOT treat an answer as current without checking it. + +`registration`'s `type` selects which of the three answers it is. + +| `type` | Meaning | +|---|---| +| `registered` | held by someone until `expires`, renewable by its owner alone until `graceUntil`. It always carries `nameRecord`: where the owner set none, every field is unset and the resolver address is zero | +| `available` | held by nobody and registrable now, at `pricing` | +| `reserved` | held back by the registry and not registered. It carries no price, and a router MUST NOT quote one | + +| Field | On | JSON type | Constraints | +|---|---|---|---| +| `expires` | `registered` | number | absent only from a v20/v21 router, which sent the record alone | +| `graceUntil` | `registered` | number | after `expires`; until here only the owner may renew. Absent on the same condition as `expires` | +| `reservedReason_` | `registered` | string | a reason word, present only when the name is held back as well | +| `nameRecord` | `registered` | object | the record, schema below | +| `pricing` | `available` | object | `registrationPrices`, `basePrice` and `minLabelLength`, below | +| `reservedReason` | `reserved` | string | a reason word | + +| `pricing` field | JSON type | Constraints | +|---|---|---| +| `registrationPrices` | object | label length, as a decimal string, to US cents per year, for the lengths the registry prices specially | +| `basePrice` | number | US cents per year for every other length | +| `minLabelLength` | number | characters; the registry refuses shorter labels | + +A reason word is `internal`, `trademark`, `community`, or a word a later version +reserves under, at most 32 printable ASCII characters. A router truncates an +unknown word to that and otherwise passes it through unchanged. A client MUST +read a word it does not know as unknown and still treat the name as reserved. + +**Computing the price.** In US cents, for a duration in seconds: + +``` +price len duration = tier len * duration / 31536000 +tier len = the entry for len in registrationPrices, or basePrice when there is none +``` + +The registry's minimum registration is 730 days, a contract constant, so it is +specified here rather than sent. `registrationPrices` omits any length below +`minLabelLength`. A client MUST NOT show a quote for a label the registry +refuses: a hashed query carries no length, so only the client can check it. + +Below v22, `RNAME` carries the bare record and nothing else, and every answer +without one is `ERR NAME NOT_FOUND`. A v22 client reads such an answer as +`registered` with no expiry, grace or reservation. + +From v22 a client MUST NOT read `ERR NAME NOT_FOUND` as "registrable": only +`available` says that. + +A router that cannot state an answer completely MUST send `ERR NAME RESOLVER +` rather than answer partially or guess. That covers a TLD with no +registrar or price oracle, an unreachable chain, a timeout, a registration it +could not date or resolve, and any status word it does not recognise. + +`nameRecord` MUST be a UTF-8 JSON object with the following schema: | Field | JSON type | Constraints | |---|---|---| @@ -1514,34 +1612,16 @@ rname = %s"RNAME" SP json-bytes ; json-bytes consumes the remainder of the tra | `owner` | string | `"0x"` followed by 40 lowercase hex characters (20 raw bytes) | | `resolver` | string | `"0x"` followed by 40 lowercase hex characters; the resolver contract address that produced the record | -Text fields (`nickname`, `website`, `location`) use the empty string `""` as -the "unset" sentinel: a backing resolver with no value for the field MUST emit -an empty string, not JSON `null` and not an absent key. Link fields -(`simplexContact`, `simplexChannel`) are arrays, primary link first, and use the -empty array `[]` when unset. Coin fields (`eth`, `btc`, `xmr`, `dot`) use JSON -`null` as the "unset" sentinel and MAY also be absent from the object entirely. +Testnet-vs-mainnet status is derived from the queried TLD, not from the record. -The backing resolver filters records that are expired or otherwise unavailable -(the names router then returns `ERR NAME NOT_FOUND` to the client), so the wire -format carries no expiry field. Testnet-vs-mainnet status is derived from the -queried TLD rather than an in-record flag. +Receivers MUST tolerate extra unknown fields; adding a required field is a +breaking change requiring an SMP version bump. Receivers parse by key name, so +peers MUST NOT rely on a byte-canonical form. -Receivers MUST tolerate extra unknown fields (forward-compatibility for future -field additions). Adding a required field is a breaking change requiring an -SMP version bump. - -**Field order is not significant.** Receivers parse JSON by key name, so object -key order, insignificant whitespace, and number formatting carry no meaning; -records are interpreted by decoded value, never compared byte-for-byte. Peers -MUST NOT rely on a byte-canonical form — a different resolver or server may emit -the same record with different key order or spacing. This order-independence is -what makes the format forward-compatible (see the unknown-field rule above). - -**Wire-size budget.** The names router caps the resolver response it will -accept (`resolver_max_response_bytes`, ≤ 16000 bytes, the default) so the -re-encoded `RNAME` stays within the SMP proxied transmission budget of 16224 -bytes; a response over the cap is rejected as `ERR NAME RESOLVER`. The link -arrays are bounded by this overall budget rather than a fixed per-field count. +The names router caps the resolver response it will accept +(`resolver_max_response_bytes`, at most 16000 bytes) so the re-encoded `RNAME` +stays within the SMP proxied transmission budget of 16224 bytes; a response over +the cap is `ERR NAME RESOLVER`. ## Transport connection with the SMP router diff --git a/scripts/resolver/README.md b/scripts/resolver/README.md index 88fa6fde5..f8f5dde8b 100644 --- a/scripts/resolver/README.md +++ b/scripts/resolver/README.md @@ -62,15 +62,26 @@ curl -s -X POST http://127.0.0.1:8545 \ **2. resolver is healthy:** ```sh curl -s http://127.0.0.1:8000/health | jq -# → {"ok": true, "rpc": "http://reth:8545", "registries": {"testing": "0x…", "simplex": ""}} +# → {"ok": true, "rpc": "http://reth:8545", "registries": {"testing": "0x…", "simplex": ""}, +# "blockNumber": 23400000, "chainLagSeconds": 12} ``` +`chainLagSeconds` is how far the node is behind the wall clock. A resolver that +is reachable and answering can still be hours behind, and every name it reports +is that stale. `null` means the node could not be reached. + **3. resolver resolves a live name** (`foobar.testing` is a populated test name): ```sh curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq # → {"name":"foobar.testing","nickname":"Foo","simplexContact":["https://smp16.simplex.im/a#…"], … } ``` +**4. the route your router will call** (check 3 passes on an older resolver too): +```sh +curl -s http://127.0.0.1:8000/v2/resolve/foobar.testing | jq +# → {"type":"registered","expires":1780000000,"graceUntil":…,"nameRecord":{…}} +``` + **Wire your smp-server:** in its `[NAMES]` section set `resolver_endpoint: http://127.0.0.1:8000` (no auth needed for loopback). @@ -82,7 +93,7 @@ curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq | reth p2p | `:30303` tcp/udp | Ethereum sync (open on firewall) | | nimbus p2p | `:9000` tcp/udp | beacon sync (open on firewall) | | nimbus REST | `127.0.0.1:5052` | beacon API | -| **resolver** | `127.0.0.1:8000` | SNRC REST (`/resolve`, `/health`) | +| **resolver** | `127.0.0.1:8000` | SNRC REST (`/v2/resolve`, `/resolve`, `/health`) | ## Caveats @@ -110,7 +121,57 @@ standalone for local dev (no Docker), via [`uv`](https://docs.astral.sh/uv/): uv run scripts/resolver/service/snrc-resolve.py # defaults to local reth + mainnet .testing ``` -### Response shape +Three routes, versioned separately from the protocol so each only changes when +its own shape does: + +| Route | Called by | Answers | +|---|---|---| +| `/v2/resolve/` | routers from SMP v22 | a `NameRegistration` | +| `/resolve/` | routers before SMP v22 | a name record, flat | +| `/health` | anyone | readiness | + +`/v1/resolve/` is an alias for `/resolve/`. + +### v2: `/v2/resolve/` + +The body is the SMP protocol's `NameResponse`, which the router decodes as is +and forwards; translating the registry's model to it is this resolver's job. It +is the registration below, plus `lastBlockTs`, the timestamp of the block it was read +at: a node that lags answers with names it has not seen registered yet. +Its `type` is `registered`, `available` or `reserved`, and the fields each one +carries are specified once, in the **Name response** section of +[`protocol/simplex-messaging.md`](../../protocol/simplex-messaging.md). It is +the wire format, so it is documented with the wire. + +Two things follow from that and are worth stating here. Expiry and grace belong +to the registration, not to the record: `expires` and `graceUntil` sit beside +`nameRecord`, not inside it. And a name nobody holds is not an error: it +answers 200 with `type: available` and its price, so no status code from this +route means "not registered". + +| Status | Meaning | +|---|---| +| 200 | a registration: `registered`, `available` or `reserved` | +| 400 | `tldNotConfigured`, `notFullyQualified` | +| 502 | `noPriceOracle`, `labelNotRecorded`, an unreadable status, or `upstreamError` | + +Error bodies carry `name` and a fixed `error` code to branch on. Only +`upstreamError` adds a `message`; the v1 route always adds one. + +`labelNotRecorded` means the registrar holds the name but never recorded its +label, so a hashed query cannot be answered with a name. See +[Querying by labelhash](#querying-by-labelhash). + +A subname reports the expiry and grace of the 2LD above it, since that is what +bounds its lifetime. A subname nobody created reports as not registered. + +### v1: `/resolve/` + +What routers before SMP v22 call. Its shape is unrelated to v2's: the record is +flat, and `status`, `expires`, `graceEnds`, `reasonCode` and `reason` sit +alongside its fields. + +#### v1 response shape ```jsonc { @@ -119,28 +180,197 @@ uv run scripts/resolver/service/snrc-resolve.py # defaults to local reth + main "simplexContact": ["https://smp16.simplex.im/a#…", "https://smp11…"], // primary first, fallbacks after "simplexChannel": [], "eth": null, "btc": "bc1q…", "xmr": "4ANz…", "dot": "139G…", - "owner": "0xd83b…", "resolver": "0x80fa…" + "owner": "0xd83b…", "resolver": "0x80fa…", + "status": "registered", // registered | grace | expired | unregistered | unknown + "expires": 1780000000, // Unix seconds; when the registration ends + "graceEnds": 1787776000, // expires + GRACE_PERIOD; last moment the owner can renew + "reasonCode": null, // set when the name is held back as well + "reason": null // set when the name is held back as well } ``` `simplexContact`/`simplexChannel` are arrays (a name can advertise multiple SMP -servers; clients try them in order). On-chain they're a single comma-separated +servers; clients try them in order). On-chain they're a single `;`-separated text record; the resolver splits/trims/drops-empties. Address encodings are canonical per chain (EIP-55 / bech32 / SS58 / Monero-base58). Subnames work identically (`bar.foobar.testing`). -### Status codes +#### v1 registration status and expiry + +A response carries `status`, `expires` and `graceEnds` whenever the resolver +read them, a successful resolve included, so a client that has just resolved a +name already knows when it expires. Both timestamps are Unix seconds, and +`null` when they could not be read. + +| `status` | Meaning | +|---|---| +| `registered` | live; `expires` is when that ends | +| `grace` | lapsed, but only the previous owner may renew it, until `graceEnds` | +| `expired` | lapsed and past grace; anyone may register it | +| `unregistered` | never registered, and free to take | +| `unknown` | no `SNRC_REGISTRAR_` configured, so status could not be read | + +A reservation is orthogonal to the status: a name held back by the registry +carries `reasonCode` and `reason` whether or not it is registered. + +`grace` and `expired` are told apart by the registrar's own `available(id)` +rule, `expires + GRACE_PERIOD < now`. `GRACE_PERIOD` is read from the contract +rather than assumed, and `now` is the latest block's timestamp rather than the +host clock, which the registrar compares against too, so a machine with a wrong +clock cannot misreport a registration. That rule alone is not enough: it also +holds for a name nobody ever registered (`0 + GRACE_PERIOD < now`), so a zero +expiry is what separates *never registered* from *registered and since +released*. + +A subname reports the status of the 2LD above it, which is only as good as the +name it sits under. A subname nobody created answers 404 `unregistered`. + +#### v1 errors + +Every non-2xx body carries two fields: `error` is a fixed code to branch on, +and `message` is a sentence for a human. Match on `error`, never on `message`, +which is free to change. + +```jsonc +{"name": "nope.testing", "error": "unregistered", + "message": "this name has never been registered", + "status": "unregistered", "expires": null, "graceEnds": null} +``` + +The codes are `tldNotConfigured`, `notFullyQualified`, `unregistered`, +`expired`, `noSuchRoute` and `upstreamError`. When the registration is what went +wrong, `error` and `status` hold the same value, so one field is enough to read. + +`upstreamError` says only which exception type the RPC call raised. The text +goes to the resolver's log instead, because `SNRC_RPC` can carry a provider key +and urlopen puts the URL it failed on into the message. It is also the answer +when a registrar, controller or oracle address has no contract behind it: the +empty reply is refused rather than read as zero, which would make every name +look free. + +#### v1 status codes | Status | Meaning | |---|---| -| 200 | resolved | +| 200 | resolved (`status` is `registered` or `grace`, or `unknown` when no registrar is configured) | | 400 | TLD not configured, or not a fully-qualified name | -| 404 | name has no resolver set on the registry | +| 404 | `unregistered` | +| 410 | `expired`: lapsed and past grace, so anyone may take it | | 502 | upstream RPC error / reth not synced | -### Configuring registries +### Querying by labelhash -Defaults to mainnet `.testing` (`0x03f438…`); `.simplex` is unset until -deployed. Override per TLD via env on the `resolver` service in -`docker-compose.yml` (`SNRC_REGISTRY_TESTING` / `SNRC_REGISTRY_SIMPLEX`), or as -env vars for the standalone script. +A client asking whether a name is free is usually about to register it, and +whoever runs the resolver could register it first. To avoid that, send the +keccak hash of the label in ENS's `[<64 hex>]` form instead of the label: + +```sh +# instead of /resolve/acme.testing +curl -s "http://127.0.0.1:8000/resolve/[$(printf acme | keccak-256sum | cut -d' ' -f1)].testing" +``` + +namehash is `keccak(parent || keccak(label))`, so this reaches the same node and +returns the same record. The registrar keys `nameExpires` and `reservedNames` on +the labelhash too, so the status fields do not need the label either. The +resolver learns the name only by guessing the label and hashing it. + +Only the second-level label is a registry key, and `status` decodes a bracket +there at any depth. The record does not: a bracket is decoded only in a +two-label name, so `sub.[].testing` is not a supported query. Subname +labels stay text; a bracket label left of the 2LD is an ordinary label. Routers +from v22 send every 2LD this way, so a registrable name normally never reaches +this service. + +On v2, read `type`: only `available` means the name is free. On v1, read +`status`: a name is free on `unregistered` (404) and on `expired` (410), every +other status means somebody holds it, and a `reasonCode` means the registry will +refuse it whatever the status says. + +The hash must be keccak-256. `openssl dgst -sha3-256` and `sha3sum` compute +SHA3-256, a different function that returns 64 valid-looking hex characters +pointing at the wrong node. + +The resolver lowercases the query before matching, so uppercase hex works too. +Clients that refuse raw brackets in a path can percent-encode them as `%5B` and +`%5D`. + +Brackets cannot collide with a real name: they are invalid in a normalised ENS +name, and a `[<64 hex>]` label is 66 bytes against the registrar's +`maxLabelLength` of 63. A plain `0x…` label is not treated as a hash, since that +is an ordinary, registrable name. + +Only 2LDs can be queried this way, as only a 2LD can be raced for: subnames are +created by the 2LD's owner. A bracket label in a subname is hashed as written, +so it points at a node nobody can own. ENS tooling accepts the bracketed form at +any depth; this resolver does not, on purpose. + +This hides interest in a name and nothing else: the registration itself is +public, and commit-reveal covers that step. A short or well-known label is easy +to guess by hashing candidates, and the reveal publishes the labelhash, so an +operator who logged the query can match it to the name afterwards. + +### What a name costs + +The controller's `prices()` names the price oracle, so no extra configuration is +needed beyond `SNRC_CONTROLLER_`. A `SimplexPriceOracle` exposes its curve +through `prices()`, in US cents per year, which is the unit this API carries. + +An ENS-shaped oracle exposes only `price1Letter()`..`price6Letter()`, in attoUSD +per second, and charges a premium on a lapsed name that it does not expose. A +quote from one is therefore only safe for a name that was never registered: an +`expired` name gets no price rather than one below what the registrar charges. + +**Set `SNRC_CONTROLLER_` wherever `SNRC_REGISTRAR_` is.** Without a +controller there is no oracle, so no name can be priced. + +**Upgrade this service before the routers that query it.** Routers from SMP v22 +call `/v2/resolve`, which an older resolver does not serve. Every name then +answers `ERR NAME RESOLVER "HTTP 404"` until this service is upgraded, while +`/health` still reports it as ready. + +### Why a name is reserved + +v2 carries the controller's reason as `reservedReason`. v1 carries the same word +as `reasonCode`, plus `reason`, an English sentence for a human reading this API. +Clients should branch on the word and phrase it themselves, in the user's +language. + +| `reasonCode` | Meaning | +|---|---| +| `internal` | reserved for SimpleX | +| `trademark` | reserved to protect a trademark | +| `community` | reserved for the community | +| `unknown` | a reason added to the contract after this resolver; still reserved | + +These are `SimplexController.Reason`, where 0 means not reserved. A controller +from before the enum stores a boolean, whose `true` decodes as 1, which is why +1 reads as `internal`, so nothing needs migrating. + +### Configuring addresses + +The resolver reads three contracts, each configured per TLD. + +The **registry** answers who owns a node, and `/resolve` reads the records from +it. The **registrar** (ERC-721) holds `nameExpires` and `GRACE_PERIOD`, which +is where every expiry field comes from, and `labelOf`, which is how a hashed +query is answered with a name. A name registered without recording its label +cannot answer one, and `/v2/resolve` refuses it rather than answer with a name +the client will reject. With no registrar for a TLD, `/resolve` still works and +reports `"status": "unknown"`. The **controller** holds +`reservedNames`, which is where `reasonCode` comes from. With no controller a +held-back name reads as not reserved, and no name can be priced. + +All three default to the mainnet `.testing` deployment. `.simplex` is unset +until it is deployed. + +The controller default is the **proxy**, not `SimplexControllerImpl`. Storage +lives in the proxy, so the implementation address answers nothing. The two +deployment files use different names for that proxy: +`deployments.mainnet.testing.json` records it under the ENS role name +`ETHRegistrarController`, and `verification.mainnet.testing.json` calls it +`SimplexControllerProxy`. Both are the same address, and it is the one used +here. + +To override any of them, set `SNRC_REGISTRY_`, `SNRC_REGISTRAR_` or +`SNRC_CONTROLLER_` on the `resolver` service in `docker-compose.yml`, or +as env vars when you run the script directly. diff --git a/scripts/resolver/docker-compose.yml b/scripts/resolver/docker-compose.yml index 570b63df3..24a90488e 100644 --- a/scripts/resolver/docker-compose.yml +++ b/scripts/resolver/docker-compose.yml @@ -150,6 +150,12 @@ services: # only if you're deploying against a different network or contract. # SNRC_REGISTRY_TESTING: 0x... # SNRC_REGISTRY_SIMPLEX: 0x... + # Registrar and controller, same cascade. Without the registrar `status` + # is "unknown"; without the controller a reserved name is "unregistered". + # SNRC_REGISTRAR_TESTING: 0x... + # SNRC_REGISTRAR_SIMPLEX: 0x... + # SNRC_CONTROLLER_TESTING: 0x... + # SNRC_CONTROLLER_SIMPLEX: 0x... ports: - "127.0.0.1:8000:8000" restart: unless-stopped diff --git a/scripts/resolver/service/snrc-resolve.py b/scripts/resolver/service/snrc-resolve.py index ffddbeb02..42881a2a8 100755 --- a/scripts/resolver/service/snrc-resolve.py +++ b/scripts/resolver/service/snrc-resolve.py @@ -29,6 +29,7 @@ Usage: ./snrc-resolve.py # serve on :8000 curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq . + curl -s 'http://127.0.0.1:8000/resolve/[<64-hex labelhash>].testing' | jq . curl -s http://127.0.0.1:8000/health Environment: @@ -38,6 +39,11 @@ Environment: 0x58fc46996d975c57883564648bda5206d1a0102b) SNRC_REGISTRY_SIMPLEX ENSRegistry for the .simplex deployment (default: empty — TLD not yet deployed) + SNRC_REGISTRAR_ BaseRegistrar (ERC-721) for the TLD; expiry and status + (default: mainnet for .testing, empty for .simplex) + SNRC_CONTROLLER_ SimplexController (proxy) for the TLD; reservations, + and through its `prices()` oracle what registering costs + (default: mainnet for .testing, empty for .simplex) SNRC_PORT Listen port (default: 8000) SNRC_BIND Bind address (default: 0.0.0.0) @@ -61,6 +67,7 @@ import hashlib import json import os import sys +import time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from urllib.parse import unquote, urlparse from urllib.request import Request, urlopen @@ -84,6 +91,31 @@ REGISTRIES = { "simplex": os.environ.get("SNRC_REGISTRY_SIMPLEX", ""), # not deployed yet } +REGISTRARS = { + "testing": os.environ.get("SNRC_REGISTRAR_TESTING", "") + or "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a", # mainnet .testing + "simplex": os.environ.get("SNRC_REGISTRAR_SIMPLEX", ""), # not deployed yet +} + +CONTROLLERS = { + "testing": os.environ.get("SNRC_CONTROLLER_TESTING", "") + # Proxy address, not SimplexControllerImpl: storage is held by the proxy. + # Recorded in deployments.json as ETHRegistrarController. + or "0xeeb9b6bf5fb68fb726005f7ba549c2f4b32f2dad", # mainnet .testing + "simplex": os.environ.get("SNRC_CONTROLLER_SIMPLEX", ""), # not deployed yet +} + +# `reservedNames` holds a SimplexController.Reason; 0 means not reserved. A +# controller from before the enum stores a bool, whose `true` decodes as 1, +# which is why 1 reads as "internal". +RESERVED_REASONS = { + 1: ("internal", "reserved for SimpleX"), + 2: ("trademark", "reserved to protect a trademark"), + 3: ("community", "reserved for the community"), +} +# a Reason added to the contract after this resolver: still reserved, unworded +UNKNOWN_REASON = ("unknown", "reserved") + # SLIP-44 coin types (https://github.com/satoshilabs/slips/blob/master/slip-0044.md) COIN_ETH = 60 COIN_BTC = 0 @@ -92,6 +124,8 @@ COIN_DOT = 354 ZERO_ADDR = "0x0000000000000000000000000000000000000000" +# The registry prices in attoUSD (1e-18 USD); the protocol carries US cents. + # ---------- RPC + ABI helpers (mirrors ens-lookup.py shape) ---------- @@ -123,12 +157,216 @@ def namehash(name: str) -> bytes: return node +# ENS's encoding for a label whose preimage is unknown. Brackets are outside +# the normalised character set, so it cannot collide with a registrable name. +ENCODED_LABELHASH_LEN = 66 # "[" + 64 hex + "]" + + +def is_encoded_labelhash(label: str) -> bool: + return ( + len(label) == ENCODED_LABELHASH_LEN + and label.startswith("[") + and label.endswith("]") + and all(c in "0123456789abcdef" for c in label[1:-1]) + ) + + +def node_of(name: str) -> bytes: + """namehash, decoding a second-level labelhash so `[hash].tld` reaches the + node its name does. Only a second-level name is ever hashed; a bracket + anywhere else is hashed as written.""" + labels = name.split(".") + if len(labels) != 2 or not is_encoded_labelhash(labels[0]): + return namehash(name) + return keccak(namehash(labels[1]) + bytes.fromhex(labels[0][1:-1])) + + +# ---------- Registration status ---------- + + +def head_block(): + """How far behind the node is. Unlike expiry, this is the one thing that has + to be measured against the host clock: a node that stops still has a block.""" + try: + block = rpc("eth_getBlockByNumber", ["latest", False]) + return { + "blockNumber": decode_uint(block["number"]), + "chainLagSeconds": int(time.time()) - decode_uint(block["timestamp"]), + } + except Exception: + return {"blockNumber": None, "chainLagSeconds": None} + + +def chain_now() -> int: + """Expiry is compared against the block timestamp, never the host clock.""" + block = rpc("eth_getBlockByNumber", ["latest", False]) + return decode_uint(block["timestamp"]) + + +def grace_period(registrar: str) -> int: + """A deployment can configure a different window, so it is read on chain.""" + return decode_uint(eth_call(registrar, selector("GRACE_PERIOD()"))) + + +def expiry_status(expires: int, grace: int, now: int) -> str: + """The registrar's `available(id)` is not enough on its own: it is also + true for a name nobody registered, since 0 + GRACE_PERIOD < now.""" + if expires == 0: + return "unregistered" + if expires > now: + return "registered" + if expires + grace >= now: + return "grace" + return "expired" + + +def reservation_reason(tld: str, token: int) -> int: + """The SimplexController.Reason held for the name, 0 when not reserved.""" + controller = CONTROLLERS.get(tld) + if not controller: + return 0 + raw = eth_call(controller, selector("reservedNames(bytes32)") + encode_uint(token)) + return decode_uint(raw) + + +def pricing_params(tld: str): + """What it costs to register a name under this TLD, in US cents, or None + when no controller or price oracle is configured.""" + controller = CONTROLLERS.get(tld) + if not controller: + return None + oracle = decode_address(eth_call(controller, selector("prices()"))) + if oracle == ZERO_ADDR: + return None + try: + return read_oracle_prices(controller, oracle) + except RuntimeError: + # An oracle that does not expose its curve cannot be quoted from. The + # name is still registrable; the price is simply not ours to state. + return None + + +SECONDS_PER_YEAR = 31536000 +ATTO_PER_CENT = 10**16 + + +def read_oracle_prices(controller: str, oracle: str): + """SimplexPriceOracle keeps the curve in US cents per year, the unit the SMP + protocol carries. An ENS-shaped oracle prices in attoUSD per second and + charges a premium on lapsed names that it does not expose, so a quote from + it is only safe for a name that was never registered.""" + try: + base, tiers = decode_prices(eth_call(oracle, selector("prices()"))) + premium_unknown = False + except RuntimeError: + base, tiers = decode_letter_prices(oracle) + premium_unknown = True + min_len = decode_uint(eth_call(controller, selector("minCharLength()"))) + return { + # lengths the registry refuses are left out rather than priced at zero + "registrationPrices": {n: c for n, c in tiers.items() if n >= min_len}, + "basePrice": base, + "minLabelLength": min_len, + "_premiumUnknown": premium_unknown, + } + + +def decode_letter_prices(oracle: str): + """`price1Letter()`..`price6Letter()`, in attoUSD per second. Quotes round + up, so one is never below what the registry charges. An oracle built before + the six-letter tier stops at five, and charges its highest tier for anything + longer, which is what basePrice means here.""" + tiers = {} + for n in range(1, 7): + try: + rate = decode_uint(eth_call(oracle, selector(f"price{n}Letter()"))) + except RuntimeError: + if n <= 5: + raise + break + tiers[n] = ceil_div(rate * SECONDS_PER_YEAR, ATTO_PER_CENT) + return tiers.pop(max(tiers)), tiers + + +def ceil_div(a: int, b: int) -> int: + return -(-a // b) + + +def decode_prices(hex_data: str): + """`prices()` returns the base price and the lengths priced differently.""" + raw = bytes.fromhex(hex_data[2:] if hex_data.startswith("0x") else hex_data) + # a short answer is not a curve: decoding it would quote every name as free + if len(raw) < 96: + raise RuntimeError("prices(): short response") + base = int.from_bytes(raw[:32], "big") + at = int.from_bytes(raw[32:64], "big") + count = int.from_bytes(raw[at:at + 32], "big") + tiers = {} + for i in range(count): + item = at + 32 + i * 64 + length = int.from_bytes(raw[item:item + 32], "big") + tiers[length] = int.from_bytes(raw[item + 32:item + 64], "big") + return base, tiers + + +def name_status(name: str): + labels = name.split(".") + tld = labels[-1] + registrar = REGISTRARS.get(tld) + if not registrar or len(labels) < 2: + return { + "status": "unknown", + # nothing was read, so there is no block to report + "lastBlockTs": None, + "expires": None, + "graceEnds": None, + "reasonCode": None, + "reason": None, + } + + # nameExpires and reservedNames are keyed on uint256(keccak(label)). + # The 2LD's label is that key at any depth. node_of decodes a bracket only + # in a two-label name, so a bracket subname gets a status but no record. + token = label_token(labels[-2]) + expires = decode_uint( + eth_call(registrar, selector("nameExpires(uint256)") + encode_uint(token)) + ) + grace = grace_period(registrar) if expires else 0 + now = chain_now() + status = expiry_status(expires, grace, now) + + # A reservation is orthogonal to the registration: a registered name can be + # held back too. + code = reservation_reason(tld, token) + reason = RESERVED_REASONS.get(code, UNKNOWN_REASON) if code else None + + out = { + "status": status, + # the block this was read at + "lastBlockTs": now, + "expires": expires or None, + "graceEnds": (expires + grace) if expires else None, + "reasonCode": reason[0] if reason else None, + "reason": reason[1] if reason else None, + } + if status in ("unregistered", "expired"): + pricing = pricing_params(tld) + # a lapsed name may carry a premium this resolver cannot read, and a + # quote without it would be below what the registry charges + if pricing and not (status == "expired" and pricing["_premiumUnknown"]): + out.update({k: v for k, v in pricing.items() if not k.startswith("_")}) + return out + + def selector(signature: str) -> str: return "0x" + keccak(signature.encode())[:4].hex() def eth_call(to: str, data: str) -> str: - return rpc("eth_call", [{"to": to, "data": data}, "latest"]) + result = rpc("eth_call", [{"to": to, "data": data}, "latest"]) + if result == "0x": + raise RuntimeError(f"empty return from {to}: no contract at that address?") + return result def decode_address(hex_data: str) -> str: @@ -143,6 +381,42 @@ def decode_bytes(hex_data: str) -> bytes: return raw[64:64 + length] +def registered_label(registrar: str, token: int): + """The plaintext label the registrar recorded at registration, keyed by the + hash of that label. None when the name was registered without + registerWithLabel, so the registrar cannot name it.""" + raw = decode_bytes(eth_call(registrar, selector("labelOf(uint256)") + encode_uint(token))) + return raw.decode("utf-8", errors="replace") if raw else None + + +def canonical_name(name: str): + """The name to answer with: a hashed query does not carry one, so the + registrar's record of the label fills it in. None when it recorded none.""" + labels = name.split(".") + registrar = REGISTRARS.get(labels[-1]) + if not registrar or len(labels) != 2 or not is_encoded_labelhash(labels[0]): + return name + label = registered_label(registrar, label_token(labels[0])) + return label + "." + labels[1] if label else None + + +def label_token(label: str) -> int: + """The registry key for a second-level label, whether it arrived as text or + already hashed.""" + if is_encoded_labelhash(label): + return int(label[1:-1], 16) + return int.from_bytes(keccak(label.encode()), "big") + + +def decode_uint(hex_data: str) -> int: + raw = hex_data[2:] if hex_data.startswith("0x") else hex_data + return int(raw[-64:], 16) if raw else 0 + + +def encode_uint(value: int) -> str: + return value.to_bytes(32, "big").hex() + + def encode_text_call(node: bytes, key: str) -> str: sel = selector("text(bytes32,string)") head = node.hex() + (0x40).to_bytes(32, "big").hex() @@ -390,6 +664,117 @@ def split_links(value: str) -> list: return [item.strip() for item in value.split(LINK_SEPARATOR) if item.strip()] +def upstream_error(subject: dict, e: Exception) -> dict: + """urlopen puts the failing URL into its message and SNRC_RPC can carry a + provider key, so the text goes to the log and only the type to the caller.""" + print(f"upstream error: {type(e).__name__}: {e}", file=sys.stderr) + return { + **subject, + "error": "upstreamError", + "message": f"upstream RPC failed ({type(e).__name__})", + } + + +def name_record(name: str): + """The NameRecord for a registered name. A name with no resolver set still + has one, with every field unset.""" + registry = REGISTRIES[name.rsplit(".", 1)[-1]] + node = node_of(name) + node_hex = node.hex() + resolver_addr = decode_address(eth_call(registry, selector("resolver(bytes32)") + node_hex)) + owner = decode_address(eth_call(registry, selector("owner(bytes32)") + node_hex)) + rec = { + "name": canonical_name(name), + "nickname": "", + "website": "", + "location": "", + "simplexContact": [], + "simplexChannel": [], + "eth": None, + "btc": None, + "xmr": None, + "dot": None, + "owner": owner, + "resolver": resolver_addr, + } + if resolver_addr == ZERO_ADDR: + return rec + texts = {} + for k in TEXT_KEYS: + try: + v = text(resolver_addr, node, k) + except RuntimeError: + v = "" + if v: + texts[k] = v + rec.update( + { + "nickname": texts.get("nickname") or texts.get("name") or texts.get("description") or "", + "website": texts.get("url", ""), + "location": texts.get("location", ""), + "simplexContact": split_links(texts.get("simplex.contact", "")), + "simplexChannel": split_links(texts.get("simplex.channel", "")), + "eth": addr_multicoin(resolver_addr, node, COIN_ETH), + "btc": addr_multicoin(resolver_addr, node, COIN_BTC), + "xmr": addr_multicoin(resolver_addr, node, COIN_XMR), + "dot": addr_multicoin(resolver_addr, node, COIN_DOT), + } + ) + return rec + + +def name_response(reg, registration_body): + """The SMP protocol's NameResponse: the registration and the block read at.""" + return 200, {"lastBlockTs": reg["lastBlockTs"], "registration": registration_body} + + +def registration(name: str): + """The SMP protocol's NameRegistration, which the relay decodes as is. + Translating the contract's model to it is this resolver's job.""" + tld = name.rsplit(".", 1)[-1] + if not REGISTRIES.get(tld): + return 400, {"name": name, "error": "tldNotConfigured"} + reg = name_status(name) + status = reg["status"] + if status in ("registered", "grace"): + rec = name_record(name) + # the client checks that the record names what it asked about, so a + # hashed query the registrar cannot name is refused rather than answered + if rec["name"] is None: + return 502, {"name": name, "error": "labelNotRecorded"} + # a subname inherits the 2LD's status, so only its node's owner says + # whether anyone created it + if len(name.split(".")) > 2 and rec["owner"] == ZERO_ADDR: + # name_status reads pricing only when the name was already + # unregistered, so read it here + status = "unregistered" + pricing = pricing_params(tld) + if pricing: + reg.update({k: v for k, v in pricing.items() if not k.startswith("_")}) + else: + return name_response(reg, { + "type": "registered", + "expires": reg["expires"], + "graceUntil": reg["graceEnds"], + "reservedReason_": reg["reasonCode"], + "nameRecord": rec, + }) + if reg["reasonCode"]: + return name_response(reg, {"type": "reserved", "reservedReason": reg["reasonCode"]}) + if status in ("unregistered", "expired"): + if "basePrice" not in reg: + return 502, {"name": name, "error": "noPriceOracle"} + return name_response(reg, { + "type": "available", + "pricing": { + "registrationPrices": reg["registrationPrices"], + "basePrice": reg["basePrice"], + "minLabelLength": reg["minLabelLength"], + }, + }) + return 502, {"name": name, "error": status} + + def resolve(name: str): tld = name.rsplit(".", 1)[-1] registry = REGISTRIES.get(tld) @@ -397,17 +782,60 @@ def resolve(name: str): configured = [k for k, v in REGISTRIES.items() if v] return 400, { "name": name, - "error": f"TLD '{tld}' is not configured on this resolver", - "configured_tlds": configured, + "error": "tldNotConfigured", + "message": f"TLD '{tld}' is not configured on this resolver", + "configuredTlds": configured, } - node = namehash(name) + node = node_of(name) node_hex = node.hex() + # Before the resolver lookup, so a lapsed name is not reported as noResolver. + reg = name_status(name) + if reg["status"] in ("unregistered", "expired"): + # A name in grace is not here: its record still resolves. + body = { + "name": name, + **reg, + "error": reg["status"], + "message": ( + "this name has never been registered" + if reg["status"] == "unregistered" + else "this registration expired and is open to anyone" + ), + } + return (404 if reg["status"] == "unregistered" else 410), body + resolver_raw = eth_call(registry, selector("resolver(bytes32)") + node_hex) resolver_addr = decode_address(resolver_raw) if resolver_addr == ZERO_ADDR: - return 404, {"name": name, "error": "no resolver set for this name"} + # A registered name always resolves: with no resolver set the record is + # still returned with every field unset, so "taken until " stays + # answerable. For a subname, no owner means nobody created it. + owner = decode_address(eth_call(registry, selector("owner(bytes32)") + node_hex)) + if len(name.split(".")) > 2 and owner == ZERO_ADDR: + return 404, { + "name": name, + **reg, + "status": "unregistered", + "error": "unregistered", + "message": "this subname has never been created", + } + return 200, { + "name": canonical_name(name) or name, + "nickname": "", + "website": "", + "location": "", + "simplexContact": [], + "simplexChannel": [], + "eth": None, + "btc": None, + "xmr": None, + "dot": None, + "owner": owner, + "resolver": ZERO_ADDR, + **reg, + } owner_raw = eth_call(registry, selector("owner(bytes32)") + node_hex) owner = decode_address(owner_raw) @@ -431,7 +859,7 @@ def resolve(name: str): # use the ENSIP-5 dot convention (e.g. "simplex.contact") — only the # resolver's JSON surface camelCases them. return 200, { - "name": name, + "name": canonical_name(name) or name, "nickname": nickname, "website": texts.get("url", ""), "location": texts.get("location", ""), @@ -443,6 +871,7 @@ def resolve(name: str): "dot": addr_multicoin(resolver_addr, node, COIN_DOT), "owner": owner, "resolver": resolver_addr, + **reg, } @@ -454,33 +883,51 @@ class Handler(BaseHTTPRequestHandler): parts = [unquote(p) for p in path.split("/") if p] if parts == ["health"]: - self._respond( - 200, - {"ok": True, "rpc": RPC, "registries": REGISTRIES}, - ) + self._respond(200, {"ok": True, "rpc": RPC, "registries": REGISTRIES, **head_block()}) return + if len(parts) == 3 and parts[0] == "v2" and parts[1] == "resolve": + name = parts[2].strip().lower() + if not name or "." not in name: + self._respond(400, {"name": name, "error": "notFullyQualified"}) + return + try: + status, body = registration(name) + except Exception as e: # surface upstream errors as 502 + status, body = 502, upstream_error({"name": name}, e) + self._respond(status, body) + return + + # /v1/resolve is an alias: relays before SMP v22 call /resolve + if parts[:2] == ["v1", "resolve"] and len(parts) == 3: + parts = ["resolve", parts[2]] + if len(parts) == 2 and parts[0] == "resolve": name = parts[1].strip().lower() if not name or "." not in name: self._respond( 400, { - "error": "expected fully-qualified name, e.g. /resolve/alice.testing", - "got": name, + "name": name, + "error": "notFullyQualified", + "message": "expected a fully-qualified name, e.g. alice.testing", }, ) return try: status, body = resolve(name) except Exception as e: # surface upstream errors as 502 - status, body = 502, {"name": name, "error": f"{type(e).__name__}: {e}"} + status, body = 502, upstream_error({"name": name}, e) self._respond(status, body) return self._respond( 404, - {"error": "not found", "routes": ["/health", "/resolve/"]}, + { + "error": "noSuchRoute", + "message": "not found", + "routes": ["/health", "/v2/resolve/", "/v1/resolve/", "/resolve/"], + }, ) def _respond(self, status: int, body: dict): @@ -505,7 +952,7 @@ def main(): ) for tld, addr in REGISTRIES.items(): sys.stderr.write(f" .{tld:<8s} = {addr or '(not configured)'}\n") - sys.stderr.write(" GET /resolve/ GET /health\n") + sys.stderr.write(" GET /v2/resolve/ GET /v1/resolve/ GET /health\n") try: server.serve_forever() except KeyboardInterrupt: diff --git a/scripts/resolver/service/test_snrc_resolve.py b/scripts/resolver/service/test_snrc_resolve.py index 2bb42f991..4b0555ed1 100644 --- a/scripts/resolver/service/test_snrc_resolve.py +++ b/scripts/resolver/service/test_snrc_resolve.py @@ -4,8 +4,11 @@ Run with `python3 -m unittest scripts/resolver/service/test_snrc_resolve.py`. """ +import contextlib import importlib.util +import io import os +import time import unittest # snrc-resolve.py has a hyphen, so import it via importlib instead of `import`. @@ -17,6 +20,12 @@ snrc = importlib.util.module_from_spec(_SPEC) _SPEC.loader.exec_module(snrc) +def registration(name): + """registration() answers a NameResponse; most tests assert what is in it.""" + status, body = snrc.registration(name) + return status, (body["registration"] if status == 200 else body) + + class SplitLinksTests(unittest.TestCase): """`split_links` decodes the multi-URL convention for simplex.contact / simplex.channel text records. Reuses the same rule the dApp's @@ -82,5 +91,949 @@ class SplitLinksTests(unittest.TestCase): ) +class EncodedLabelhashTests(unittest.TestCase): + # keccak-256("alice"), written out in full wherever a test needs it. + # 9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501 + + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + GRACE = 90 * 86400 + + def setUp(self): + self._saved = (snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": ""} + snrc.chain_now = lambda: int(time.time()) + + def tearDown(self): + snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now = self._saved + + def test_the_encoded_form_is_recognised(self): + self.assertTrue( + snrc.is_encoded_labelhash( + "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + ) + ) + + def test_an_ordinary_label_is_not(self): + self.assertFalse(snrc.is_encoded_labelhash("alice")) + self.assertFalse(snrc.is_encoded_labelhash("[alice]")) + self.assertFalse(snrc.is_encoded_labelhash("9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501")) + + def test_non_hex_between_the_brackets_is_not(self): + self.assertFalse(snrc.is_encoded_labelhash("[" + "z" * 64 + "]")) + # uppercase is rejected because the handler lowercases the whole name + self.assertFalse(snrc.is_encoded_labelhash("[" + "A" * 64 + "]")) + self.assertFalse(snrc.is_encoded_labelhash("[0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]")) + + def test_the_wrong_length_is_not(self): + self.assertFalse(snrc.is_encoded_labelhash("[" + "a" * 63 + "]")) + self.assertFalse(snrc.is_encoded_labelhash("[" + "a" * 65 + "]")) + + def test_hash_and_label_reach_the_same_node(self): + self.assertEqual( + snrc.node_of("alice.testing"), + snrc.node_of( + "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + ".testing" + ), + ) + + def test_a_plain_name_is_unaffected(self): + self.assertEqual(snrc.node_of("alice.testing"), snrc.namehash("alice.testing")) + + def test_a_bracket_subname_label_stays_literal(self): + """Only the 2LD is a key, so a bracket label left of it is hashed as + written.""" + self.assertNotEqual( + snrc.node_of( + "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + ".alice.testing" + ), + snrc.namehash("alice.alice.testing"), + ) + + def test_a_0x_prefixed_label_is_taken_literally(self): + name = "0x9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501.testing" + self.assertEqual(snrc.node_of(name), snrc.namehash(name)) + self.assertNotEqual(snrc.node_of(name), snrc.node_of("alice.testing")) + + def test_a_malformed_bracket_label_falls_back_to_a_literal_name(self): + name = "[nothex].testing" + self.assertEqual(snrc.node_of(name), snrc.namehash(name)) + + def test_status_by_hash_matches_status_by_name(self): + future = int(time.time()) + 86400 + seen = [] + + def eth_call(to, data): + seen.append(data) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + return "0x" + snrc.encode_uint(future) + + snrc.eth_call = eth_call + by_name = snrc.name_status("alice.testing") + by_hash = snrc.name_status( + "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + ".testing" + ) + self.assertEqual(by_name, by_hash) + self.assertEqual(by_name["status"], "registered") + # nothing in either request carried the label itself + self.assertTrue(all("alice".encode().hex() not in d for d in seen)) + + +class NameStatusTests(unittest.TestCase): + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + + GRACE = 90 * 86400 + + def _expiry(self, value): + def eth_call(to, data): + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + self.assertTrue(data.startswith(snrc.selector("nameExpires(uint256)"))) + return "0x" + snrc.encode_uint(value) + + return eth_call + + def _keys(self, status, expires, grace_ends, read_at=-1): + """Every branch answers with the same keys; only some carry values.""" + return { + "status": status, + "lastBlockTs": self.now if read_at == -1 else read_at, + "expires": expires, + "graceEnds": grace_ends, + "reasonCode": None, + "reason": None, + } + + def setUp(self): + self._saved = ( + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + snrc.rpc, + ) + snrc.REGISTRARS = {"testing": self.REGISTRAR} + # Expiry alone; ReservedTests covers a configured controller. + snrc.CONTROLLERS = {"testing": ""} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + + def tearDown(self): + ( + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + snrc.rpc, + ) = self._saved + + def test_now_is_the_latest_blocks_timestamp(self): + # setUp replaced chain_now with the fixture clock; test the real one + real_chain_now = self._saved[3] + snrc.rpc = lambda method, params: {"timestamp": "0x65f1a2c0", "number": "0x123"} + self.assertEqual(real_chain_now(), 0x65F1A2C0) + + def test_status_reads_the_chain_clock_not_the_host_clock(self): + future = int(time.time()) + 3600 + snrc.eth_call = self._expiry(future) + self.assertEqual(snrc.name_status("alice.testing")["status"], "registered") + snrc.chain_now = lambda: future + 3650 * 86400 + self.assertEqual(snrc.name_status("alice.testing")["status"], "expired") + + def test_a_registrar_that_is_not_a_contract_is_an_error_not_a_free_name(self): + """An address with no code answers eth_call with empty data. Read as + zero, that would make every name look free.""" + snrc.eth_call = self._saved[2] # the real one, so its guard runs + snrc.rpc = lambda method, params: "0x" + with self.assertRaises(RuntimeError): + snrc.name_status("alice.testing") + + def test_zero_expiry_means_never_registered(self): + snrc.eth_call = self._expiry(0) + self.assertEqual( + snrc.name_status("alice.testing"), + self._keys("unregistered", None, None), + ) + + def test_recently_expired_is_in_grace_and_says_when_it_ends(self): + past = int(time.time()) - 3600 + snrc.eth_call = self._expiry(past) + self.assertEqual( + snrc.name_status("alice.testing"), + self._keys("grace", past, past + self.GRACE), + ) + + def test_past_the_grace_window_it_is_expired_and_claimable(self): + past = int(time.time()) - self.GRACE - 3600 + snrc.eth_call = self._expiry(past) + self.assertEqual(snrc.name_status("alice.testing")["status"], "expired") + + def test_the_boundary_belongs_to_grace(self): + """The registrar frees a name only when expires + GRACE < now.""" + now = int(time.time()) + snrc.eth_call = self._expiry(now - self.GRACE) + self.assertEqual(snrc.name_status("alice.testing")["status"], "grace") + + def test_future_expiry_is_registered(self): + future = int(time.time()) + 3600 + snrc.eth_call = self._expiry(future) + self.assertEqual( + snrc.name_status("alice.testing"), + self._keys("registered", future, future + self.GRACE), + ) + + def test_never_registered_is_not_confused_with_claimable(self): + """`available(id)` is true for both, since 0 + GRACE < now.""" + snrc.eth_call = self._expiry(0) + self.assertEqual(snrc.name_status("alice.testing")["status"], "unregistered") + self.assertNotEqual(snrc.name_status("alice.testing")["status"], "expired") + + def test_a_subname_reports_the_status_of_its_2ld(self): + future = int(time.time()) + 3600 + seen = [] + + def eth_call(to, data): + seen.append(data) + return "0x" + snrc.encode_uint(future) + + snrc.eth_call = eth_call + self.assertEqual(snrc.name_status("x.alice.testing")["status"], "registered") + # the token asked about is keccak("alice"), not keccak("x") + self.assertTrue(seen[0].endswith(snrc.keccak(b"alice").hex())) + + def test_a_hashed_2ld_is_queried_by_its_hash_at_any_depth(self): + """The token must come from the hash, not from hashing the brackets.""" + seen = [] + + def eth_call(to, data): + seen.append(data) + return "0x" + snrc.encode_uint(0) + + snrc.eth_call = eth_call + hashed = "[" + snrc.keccak(b"alice").hex() + "]" + snrc.name_status("x." + hashed + ".testing") + self.assertTrue(seen[0].endswith(snrc.keccak(b"alice").hex())) + + def test_unconfigured_tld_is_unknown_rather_than_unregistered(self): + snrc.REGISTRARS = {"testing": ""} + snrc.eth_call = lambda *a: self.fail("must not reach the chain") + self.assertEqual( + snrc.name_status("alice.testing"), + self._keys("unknown", None, None, read_at=None), + ) + + def test_every_branch_returns_the_same_keys(self): + keys = { + "status", + "lastBlockTs", + "expires", + "graceEnds", + "reasonCode", + "reason", + } + snrc.eth_call = self._expiry(0) + self.assertEqual(set(snrc.name_status("alice.testing")), keys) + snrc.eth_call = self._expiry(int(time.time()) + 3600) + self.assertEqual(set(snrc.name_status("alice.testing")), keys) + snrc.REGISTRARS = {"testing": ""} + snrc.eth_call = lambda *a: self.fail("must not reach the chain") + self.assertEqual(set(snrc.name_status("alice.testing")), keys) + + +class ReservedTests(unittest.TestCase): + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + + def setUp(self): + self._saved = (snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + snrc.chain_now = lambda: int(time.time()) + + def tearDown(self): + snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now = self._saved + + def _chain(self, expires, reserved): + def eth_call(to, data): + if data.startswith(snrc.selector("reservedNames(bytes32)")): + self.assertEqual(to, self.CONTROLLER) + return "0x" + snrc.encode_uint(1 if reserved else 0) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) # no price oracle, no auction + return "0x" + snrc.encode_uint(expires) + + return eth_call + + def test_unregistered_and_reserved_reports_the_reservation(self): + snrc.eth_call = self._chain(0, True) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "unregistered") + self.assertEqual(reg["reasonCode"], "internal") + + def test_unregistered_and_not_reserved_reads_unregistered(self): + snrc.eth_call = self._chain(0, False) + self.assertEqual(snrc.name_status("acme.testing")["status"], "unregistered") + + def test_a_lapsed_reserved_name_keeps_its_reservation(self): + past = int(time.time()) - 91 * 86400 + snrc.eth_call = self._chain(past, True) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertEqual(reg["reasonCode"], "internal") + + def test_a_live_name_is_registered_even_if_reserved(self): + snrc.eth_call = self._chain(int(time.time()) + 86400, True) + self.assertEqual(snrc.name_status("acme.testing")["status"], "registered") + + def test_a_name_in_grace_belongs_to_its_owner_not_the_reserved_set(self): + snrc.eth_call = self._chain(int(time.time()) - 3600, True) + self.assertEqual(snrc.name_status("acme.testing")["status"], "grace") + + def test_no_controller_configured_means_reserved_is_never_reported(self): + snrc.CONTROLLERS = {"testing": ""} + snrc.eth_call = self._chain(0, True) # reserved on chain, but unread + self.assertEqual(snrc.name_status("acme.testing")["status"], "unregistered") + + def test_reserved_is_asked_by_labelhash_so_a_hashed_query_works(self): + # keccak-256("acme") + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(0, True) + self.assertEqual(snrc.name_status(hashed + ".testing")["reasonCode"], "internal") + + +class ReservedReasonTests(unittest.TestCase): + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + + def setUp(self): + self._saved = ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + snrc.chain_now = lambda: int(time.time()) + + def tearDown(self): + ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) = self._saved + + def _chain(self, expires, reserved): + def eth_call(to, data): + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(1 if reserved else 0) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) # no price oracle, no auction + return "0x" + snrc.encode_uint(expires) + + return eth_call + + def _reserved_as(self, code): + def eth_call(to, data): + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(code) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("prices()")): + return "0x" + snrc.encode_uint(0) + return "0x" + snrc.encode_uint(0) + + return eth_call + + def test_every_enum_value_has_a_code_and_a_sentence(self): + for code, (name, sentence) in snrc.RESERVED_REASONS.items(): + snrc.eth_call = self._reserved_as(code) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], name) + self.assertEqual(reg["reason"], sentence) + + def test_a_trademark_reservation_says_so(self): + snrc.eth_call = self._reserved_as(2) + _, body = snrc.resolve("acme.testing") + self.assertEqual(body["reasonCode"], "trademark") + + def test_a_controller_storing_a_bool_reads_as_internal(self): + """Before the enum `reservedNames` was a bool; its `true` decodes as 1.""" + snrc.eth_call = self._reserved_as(1) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], "internal") + self.assertEqual(reg["reason"], "reserved for SimpleX") + + def test_an_enum_value_this_resolver_predates_is_not_dropped(self): + """A new Reason still reserves the name, and says it is unknown rather + than claiming the chain recorded none.""" + snrc.eth_call = self._reserved_as(99) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["reasonCode"], "unknown") + self.assertEqual(reg["reason"], "reserved") + + def test_a_reserved_name_carries_the_reason(self): + snrc.eth_call = self._chain(0, True) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 404) + self.assertEqual(body["status"], "unregistered") + self.assertEqual(body["reason"], "reserved for SimpleX") + + def test_the_message_does_not_claim_a_trademark(self): + snrc.eth_call = self._chain(0, True) + _, body = snrc.resolve("acme.testing") + self.assertNotIn("trademark", body["message"]) + + def test_an_unregistered_name_has_no_reason(self): + snrc.eth_call = self._chain(0, False) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 404) + self.assertEqual(body["status"], "unregistered") + self.assertIsNone(body["reason"]) + + def test_an_expired_name_has_no_reason(self): + snrc.eth_call = self._chain(1, False) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 410) + self.assertEqual(body["status"], "expired") + self.assertIsNone(body["reason"]) + + def test_a_hashed_query_gets_the_reason_too(self): + snrc.eth_call = self._chain(0, True) + # keccak-256("acme") + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + _, body = snrc.resolve(hashed + ".testing") + self.assertEqual(body["reason"], "reserved for SimpleX") + + +class PricingTests(unittest.TestCase): + """The oracle keeps the curve in US cents per year, and a lapsed name costs + the ordinary price: this registry runs no auction.""" + + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + ORACLE = "0x1e0c9a2b9d1a4c8f7b3e5d6a9c2f4b8e1d7a3c50" + + GRACE = 90 * 86400 + BASE = 200 + EXCEPTIONS = {1: 64000, 2: 16000, 3: 1600, 4: 800, 5: 500} + MIN_LENGTH = 3 + + def setUp(self): + self._saved = ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + + def tearDown(self): + ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) = self._saved + + def _prices_return(self): + words = [snrc.encode_uint(self.BASE), snrc.encode_uint(0x40), + snrc.encode_uint(len(self.EXCEPTIONS))] + for length, cents in self.EXCEPTIONS.items(): + words += [snrc.encode_uint(length), snrc.encode_uint(cents)] + return "0x" + "".join(words) + + def _chain(self, expires, oracle=None, reserved=0): + oracle = self.ORACLE if oracle is None else oracle + self.oracle_calls = [] + + def eth_call(to, data): + if data.startswith(snrc.selector("nameExpires(uint256)")): + return "0x" + snrc.encode_uint(expires) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(reserved) + if data.startswith(snrc.selector("minCharLength()")): + self.assertEqual(to, self.CONTROLLER) + return "0x" + snrc.encode_uint(self.MIN_LENGTH) + if data.startswith(snrc.selector("prices()")): + if to == self.CONTROLLER: + return "0x" + snrc.encode_uint(int(oracle, 16)) + self.oracle_calls.append(data[:10]) + self.assertEqual(to, oracle) + return self._prices_return() + return self.fail("unexpected call " + data[:10]) + + return eth_call + + def _lapsed(self, days_past_grace): + """An expiry whose grace ended `days_past_grace` days ago. The extra + second clears the boundary, which counts as still in grace.""" + return self.now - self.GRACE - 1 - days_past_grace * 86400 + + def test_the_prices_are_the_oracles_cents_per_year(self): + snrc.eth_call = self._chain(self._lapsed(0)) + reg = snrc.name_status("acme.testing") + # 1 and 2 are below minCharLength + self.assertEqual(reg["registrationPrices"], {3: 1600, 4: 800, 5: 500}) + self.assertEqual(reg["basePrice"], self.BASE) + self.assertEqual(reg["minLabelLength"], self.MIN_LENGTH) + + def test_a_lapsed_name_costs_the_ordinary_price(self): + snrc.eth_call = self._chain(self._lapsed(0)) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + + def test_a_controller_with_no_oracle_leaves_the_name_merely_expired(self): + snrc.eth_call = self._chain(self._lapsed(0), oracle=snrc.ZERO_ADDR) + self.assertEqual(snrc.name_status("acme.testing")["status"], "expired") + + def test_a_name_in_grace_never_reaches_the_oracle(self): + snrc.eth_call = self._chain(self.now - 3600) + self.assertEqual(snrc.name_status("acme.testing")["status"], "grace") + self.assertEqual(self.oracle_calls, []) + + def test_a_reserved_lapsed_name_keeps_its_reservation(self): + snrc.eth_call = self._chain(self._lapsed(0), reserved=2) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertEqual(reg["reasonCode"], "trademark") + + def test_resolve_reports_the_prices(self): + snrc.eth_call = self._chain(self._lapsed(1)) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 410) + self.assertEqual(body["status"], "expired") + self.assertEqual(body["basePrice"], self.BASE) + + def test_a_hashed_query_is_priced_too(self): + # keccak-256("acme") + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(self._lapsed(0)) + _, body = snrc.resolve(hashed + ".testing") + self.assertEqual(body["status"], "expired") + self.assertEqual(body["basePrice"], self.BASE) + + +class EnsOracleTests(unittest.TestCase): + """.testing runs an ENS-shaped oracle: it prices in attoUSD per second and + charges a premium on lapsed names that it does not expose.""" + + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + ORACLE = "0x1e0c9a2b9d1a4c8f7b3e5d6a9c2f4b8e1d7a3c50" + GRACE = 90 * 86400 + MIN_LENGTH = 6 + + def setUp(self): + self._saved = (snrc.REGISTRIES, snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + + def tearDown(self): + (snrc.REGISTRIES, snrc.REGISTRARS, snrc.CONTROLLERS, snrc.eth_call, snrc.chain_now) = self._saved + + def _chain(self, expires, letter_cents=0): + def eth_call(to, data): + if data.startswith(snrc.selector("nameExpires(uint256)")): + return "0x" + snrc.encode_uint(expires) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(0) + if data.startswith(snrc.selector("minCharLength()")): + return "0x" + snrc.encode_uint(self.MIN_LENGTH) + if data.startswith(snrc.selector("prices()")): + if to == self.CONTROLLER: + return "0x" + snrc.encode_uint(int(self.ORACLE, 16)) + raise RuntimeError("eth_call returned 0x") # no prices() on this oracle + for n in range(1, 7): + if data.startswith(snrc.selector(f"price{n}Letter()")): + rate = letter_cents * snrc.ATTO_PER_CENT // snrc.SECONDS_PER_YEAR + return "0x" + snrc.encode_uint(rate) + return self.fail("unexpected call " + data[:10]) + + return eth_call + + def test_a_never_registered_name_is_priced_from_the_letter_curve(self): + snrc.eth_call = self._chain(0) + reg = snrc.name_status("ghost.testing") + self.assertEqual(reg["status"], "unregistered") + self.assertEqual(reg["basePrice"], 0) + self.assertEqual(reg["minLabelLength"], self.MIN_LENGTH) + + def test_a_non_zero_letter_curve_converts_to_cents_per_year(self): + snrc.eth_call = self._chain(0, letter_cents=1200) + self.assertEqual(snrc.name_status("ghost.testing")["basePrice"], 1200) + + def test_a_lapsed_name_is_not_priced_because_the_premium_is_unreadable(self): + snrc.eth_call = self._chain(self.now - self.GRACE - 1) + reg = snrc.name_status("acme.testing") + self.assertEqual(reg["status"], "expired") + self.assertNotIn("basePrice", reg) + + +class ErrorCodeTests(unittest.TestCase): + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + + def setUp(self): + self._saved = ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) + snrc.REGISTRIES = {"testing": self.REGISTRY, "simplex": ""} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": ""} + snrc.chain_now = lambda: int(time.time()) + + def tearDown(self): + ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) = self._saved + + def _chain(self, expires, resolver=None): + def eth_call(to, data): + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(90 * 86400) + if data.startswith(snrc.selector("resolver(bytes32)")): + return "0x" + "00" * 12 + (resolver or "00" * 20) + return "0x" + snrc.encode_uint(expires) + + return eth_call + + def test_an_unconfigured_tld_names_the_ones_that_are(self): + status, body = snrc.resolve("alice.nosuchtld") + self.assertEqual(status, 400) + self.assertEqual(body["error"], "tldNotConfigured") + self.assertEqual(body["configuredTlds"], ["testing"]) + self.assertIn("nosuchtld", body["message"]) + + def test_a_registration_problem_reports_the_status_as_the_code(self): + for expires, code in ( + (0, "unregistered"), + (int(time.time()) - 91 * 86400, "expired"), + ): + with self.subTest(code=code): + snrc.eth_call = self._chain(expires) + _, body = snrc.resolve("alice.testing") + self.assertEqual(body["error"], code) + self.assertEqual(body["status"], code) + + def test_a_name_in_grace_still_resolves(self): + snrc.eth_call = self._chain(int(time.time()) - 3600) + status, body = snrc.resolve("alice.testing") + self.assertEqual(status, 200) + self.assertEqual(body["status"], "grace") + self.assertNotIn("error", body) + + def test_a_registered_name_pointing_nowhere_resolves_with_empty_records(self): + snrc.eth_call = self._chain(int(time.time()) + 86400) + status, body = snrc.resolve("alice.testing") + self.assertEqual(status, 200) + self.assertEqual(body["status"], "registered") + self.assertEqual(body["resolver"], snrc.ZERO_ADDR) + self.assertEqual(body["simplexContact"], []) + + def test_every_error_body_carries_both_fields(self): + snrc.eth_call = self._chain(0) + for name in ("alice.nosuchtld", "alice.testing"): + with self.subTest(name=name): + _, body = snrc.resolve(name) + self.assertIsInstance(body["error"], str) + self.assertIsInstance(body["message"], str) + self.assertNotEqual(body["error"], body["message"]) + + def test_an_upstream_failure_does_not_echo_the_exception(self): + with contextlib.redirect_stderr(io.StringIO()) as log: + body = snrc.upstream_error( + {"name": "alice.testing"}, + RuntimeError("http://user:secret@rpc.example/kEy8 refused"), + ) + # the operator still sees the detail in the log + self.assertIn("secret", log.getvalue()) + self.assertEqual(body["error"], "upstreamError") + self.assertIn("RuntimeError", body["message"]) + self.assertNotIn("secret", body["message"]) + self.assertNotIn("kEy8", body["message"]) + + +class RegistrationV2Tests(unittest.TestCase): + """`/v2/resolve` answers with the SMP protocol's NameRegistration, which the + relay decodes as is. The key names are the wire contract, so they are pinned + here: renaming one without the Haskell side is a silent break.""" + + REGISTRY = "0x58fc46996d975c57883564648bda5206d1a0102b" + REGISTRAR = "0xef47eb4384b46c89e4482a677c2cbcbd2a6fd85a" + CONTROLLER = "0x281ca41311c2aa808c917c4674639d7567b75714" + ORACLE = "0x1e0c9a2b9d1a4c8f7b3e5d6a9c2f4b8e1d7a3c50" + OWNER = "0xd83bd7e0e6b8a4c1f2593a7b0c4e8d1a6f9b2c37" + + GRACE = 90 * 86400 + BASE = 200 + EXCEPTIONS = {1: 64000, 2: 16000, 3: 1600, 4: 800, 5: 500} + MIN_LENGTH = 3 + + def setUp(self): + self._saved = ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) + snrc.REGISTRIES = {"testing": self.REGISTRY} + snrc.REGISTRARS = {"testing": self.REGISTRAR} + snrc.CONTROLLERS = {"testing": self.CONTROLLER} + self.now = int(time.time()) + snrc.chain_now = lambda: self.now + + def tearDown(self): + ( + snrc.REGISTRIES, + snrc.REGISTRARS, + snrc.CONTROLLERS, + snrc.eth_call, + snrc.chain_now, + ) = self._saved + + def _prices_return(self): + words = [snrc.encode_uint(self.BASE), snrc.encode_uint(0x40), + snrc.encode_uint(len(self.EXCEPTIONS))] + for length, cents in self.EXCEPTIONS.items(): + words += [snrc.encode_uint(length), snrc.encode_uint(cents)] + return "0x" + "".join(words) + + @staticmethod + def _abi_bytes(value: bytes) -> str: + """head offset, length, then the payload padded to a 32-byte word.""" + pad = (-len(value)) % 32 + return ("0x" + snrc.encode_uint(0x20) + snrc.encode_uint(len(value)) + + (value + b"\x00" * pad).hex()) + + def _chain(self, expires, reserved=0, oracle=None, label=b"acme", owner=None): + """The registry answers a zero resolver, so name_record returns the + empty record a registered name still has. `label` is what the registrar + recorded for the 2LD; b"" means it recorded none. `owner` is the owner of + the queried node; ZERO_ADDR means that node was never created.""" + oracle = self.ORACLE if oracle is None else oracle + owner = self.OWNER if owner is None else owner + + def eth_call(to, data): + if data.startswith(snrc.selector("labelOf(uint256)")): + return self._abi_bytes(label) + if data.startswith(snrc.selector("nameExpires(uint256)")): + return "0x" + snrc.encode_uint(expires) + if data.startswith(snrc.selector("GRACE_PERIOD()")): + return "0x" + snrc.encode_uint(self.GRACE) + if data.startswith(snrc.selector("reservedNames(bytes32)")): + return "0x" + snrc.encode_uint(reserved) + if data.startswith(snrc.selector("minCharLength()")): + return "0x" + snrc.encode_uint(self.MIN_LENGTH) + if data.startswith(snrc.selector("prices()")): + if to == self.CONTROLLER: + return "0x" + snrc.encode_uint(int(oracle, 16)) + return self._prices_return() + if data.startswith(snrc.selector("resolver(bytes32)")): + return "0x" + snrc.encode_uint(0) + if data.startswith(snrc.selector("owner(bytes32)")): + return "0x" + snrc.encode_uint(int(owner, 16)) + return self.fail("unexpected call " + data[:10]) + + return eth_call + + def _lapsed(self, days_past_grace): + return self.now - self.GRACE - 1 - days_past_grace * 86400 + + def test_a_live_name_is_registered_and_carries_its_record(self): + expires = self.now + 3600 + snrc.eth_call = self._chain(expires) + status, body = registration("acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["type"], "registered") + self.assertEqual(body["expires"], expires) + self.assertEqual(body["graceUntil"], expires + self.GRACE) + self.assertIsNone(body["reservedReason_"]) + self.assertEqual(body["nameRecord"]["name"], "acme.testing") + + def test_a_name_in_grace_is_still_registered(self): + expires = self.now - 3600 + snrc.eth_call = self._chain(expires) + _, body = registration("acme.testing") + self.assertEqual(body["type"], "registered") + self.assertGreater(body["graceUntil"], self.now) + + def test_a_registered_name_that_is_held_back_says_so(self): + snrc.eth_call = self._chain(self.now + 3600, reserved=1) + _, body = registration("acme.testing") + self.assertEqual(body["type"], "registered") + self.assertEqual(body["reservedReason_"], "internal") + + def test_an_unregistered_name_is_available_with_its_pricing(self): + snrc.eth_call = self._chain(0) + status, body = registration("acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["type"], "available") + # lengths below minCharLength are unregistrable, so they are not priced + self.assertEqual(body["pricing"]["registrationPrices"], {3: 1600, 4: 800, 5: 500}) + self.assertEqual(body["pricing"]["basePrice"], self.BASE) + self.assertEqual(body["pricing"]["minLabelLength"], self.MIN_LENGTH) + + def test_a_lapsed_name_is_available_at_the_ordinary_price(self): + snrc.eth_call = self._chain(self._lapsed(1)) + _, body = registration("acme.testing") + self.assertEqual(body["type"], "available") + self.assertEqual(body["pricing"]["basePrice"], self.BASE) + + def test_a_held_back_name_is_reserved_and_is_never_priced(self): + snrc.eth_call = self._chain(0, reserved=2) + status, body = registration("acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["type"], "reserved") + self.assertEqual(body["reservedReason"], "trademark") + self.assertNotIn("pricing", body) + + def test_a_hashed_query_answers_the_same_as_the_name(self): + # keccak-256("acme") + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(0) + _, by_name = registration("acme.testing") + _, by_hash = registration(hashed + ".testing") + self.assertEqual(by_name, by_hash) + + def test_an_unconfigured_tld_is_refused_not_answered(self): + snrc.REGISTRIES = {"testing": ""} + snrc.eth_call = lambda *a: self.fail("must not reach the chain") + status, body = registration("acme.testing") + self.assertEqual(status, 400) + self.assertEqual(body["error"], "tldNotConfigured") + + def test_no_price_oracle_is_an_error_not_a_free_name(self): + snrc.eth_call = self._chain(0, oracle=snrc.ZERO_ADDR) + status, body = registration("acme.testing") + self.assertEqual(status, 502) + self.assertEqual(body["error"], "noPriceOracle") + + def test_a_status_it_cannot_read_is_an_error_not_a_registration(self): + snrc.REGISTRARS = {"testing": ""} + snrc.eth_call = self._chain(0) + status, body = registration("acme.testing") + self.assertEqual(status, 502) + self.assertEqual(body["error"], "unknown") + + def test_each_answer_carries_exactly_its_own_fields(self): + """The relay decodes by these names; an extra or missing one is a break.""" + cases = { + "registered": (self._chain(self.now + 3600), + {"type", "expires", "graceUntil", "reservedReason_", "nameRecord"}), + "available": (self._chain(0), {"type", "pricing"}), + "reserved": (self._chain(0, reserved=1), {"type", "reservedReason"}), + } + for expected_type, (chain, keys) in cases.items(): + with self.subTest(type=expected_type): + snrc.eth_call = chain + _, body = registration("acme.testing") + self.assertEqual(body["type"], expected_type) + self.assertEqual(set(body), keys) + def test_a_hashed_query_the_registrar_cannot_name_is_refused(self): + """The client checks the record names what it asked about, so answering + with a record the registrar could not name would only fail there.""" + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(self.now + 3600, label=b"") + status, body = registration(hashed + ".testing") + self.assertEqual(status, 502) + self.assertEqual(body["error"], "labelNotRecorded") + + def test_a_hashed_query_is_answered_with_the_name_the_registrar_recorded(self): + hashed = "[e29dae06ef4c3e336b7538b6d4f52ca1ecec009b1df6fb501320e11b223aeeaf]" + snrc.eth_call = self._chain(self.now + 3600) + status, body = registration(hashed + ".testing") + self.assertEqual(status, 200) + self.assertEqual(body["nameRecord"]["name"], "acme.testing") + def test_a_subname_that_exists_is_registered_with_its_parents_dates(self): + expires = self.now + 3600 + snrc.eth_call = self._chain(expires) + status, body = registration("sub.acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["type"], "registered") + self.assertEqual(body["expires"], expires) + self.assertEqual(body["nameRecord"]["name"], "sub.acme.testing") + + def test_a_subname_nobody_created_is_not_registered(self): + """The registrar only tracks 2LDs, so the parent's registration says + nothing about a child that was never created: its node has no owner.""" + snrc.eth_call = self._chain(self.now + 3600, owner=snrc.ZERO_ADDR) + status, body = registration("sub.acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["type"], "available") + + def test_a_2ld_is_not_subject_to_the_owner_check(self): + """Only a subname can be absent under a registered parent.""" + snrc.eth_call = self._chain(self.now + 3600, owner=snrc.ZERO_ADDR) + _, body = registration("acme.testing") + self.assertEqual(body["type"], "registered") + + def test_v1_does_not_report_an_uncreated_subname_as_registered(self): + """v1 has no availability, so the only honest answer is not-found. The + 2LD case is untouched: a registered name with no resolver still resolves.""" + snrc.eth_call = self._chain(self.now + 3600, owner=snrc.ZERO_ADDR) + status, body = snrc.resolve("sub.acme.testing") + self.assertEqual(status, 404) + self.assertEqual(body["error"], "unregistered") + + def test_v1_still_resolves_a_2ld_with_no_resolver_set(self): + snrc.eth_call = self._chain(self.now + 3600, owner=snrc.ZERO_ADDR) + status, body = snrc.resolve("acme.testing") + self.assertEqual(status, 200) + self.assertEqual(body["resolver"], snrc.ZERO_ADDR) + + def test_the_answer_says_which_block_it_was_read_at(self): + """The resolver is only as current as its node. Without this a client + cannot tell an answer that predates its own registration.""" + snrc.eth_call = self._chain(self.now + 3600) + _, res = snrc.registration("acme.testing") + self.assertEqual(res["lastBlockTs"], self.now) + self.assertEqual(res["registration"]["type"], "registered") + + def test_an_available_name_says_so_too(self): + """This is the path that reads no block otherwise, and the one where + staleness matters most: the name may already be taken.""" + snrc.eth_call = self._chain(0) + _, res = snrc.registration("acme.testing") + self.assertEqual(res["lastBlockTs"], self.now) + self.assertEqual(res["registration"]["type"], "available") + if __name__ == "__main__": unittest.main() diff --git a/src/Simplex/Messaging/Agent.hs b/src/Simplex/Messaging/Agent.hs index 5caaa5be3..e9a7379b4 100644 --- a/src/Simplex/Messaging/Agent.hs +++ b/src/Simplex/Messaging/Agent.hs @@ -228,7 +228,7 @@ import Simplex.Messaging.Protocol ErrorType (AUTH), MsgBody, MsgFlags (..), - NameRecord, + NameResponse, NtfServer, ProtoServerWithAuth (..), ProtocolServer (..), @@ -461,7 +461,7 @@ getConnShortLink c = withAgentEnv c .:. getConnShortLink' c -- | Resolve a SimpleX name (PFWD RSLV). The agent owns server selection: it -- picks a names-capable server (ServerRoles.names) from the user's nameSrvs, so -- chat clients just pass the parsed domain. -resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AE NameRecord +resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AE NameResponse resolveSimplexName c nm userId domain = withAgentEnv c $ resolveSimplexName' c nm userId domain {-# INLINE resolveSimplexName #-} @@ -1270,7 +1270,7 @@ getConnShortLink' c nm userId = \case deleteLocalInvShortLink' :: AgentClient -> ConnShortLink 'CMInvitation -> AM () deleteLocalInvShortLink' c (CSLInvitation _ srv linkId _) = withStore' c $ \db -> deleteInvShortLink db srv linkId -resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameRecord +resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameResponse resolveSimplexName' c nm userId domain = do resolverSrv <- getNextNameServer c userId resolveName c nm userId resolverSrv domain diff --git a/src/Simplex/Messaging/Agent/Client.hs b/src/Simplex/Messaging/Agent/Client.hs index 1f25c4cb5..8cbb4c17b 100644 --- a/src/Simplex/Messaging/Agent/Client.hs +++ b/src/Simplex/Messaging/Agent/Client.hs @@ -272,7 +272,7 @@ import Simplex.Messaging.Protocol NetworkError (..), MsgFlags (..), MsgId, - NameRecord, + NameResponse, NtfServer, NtfServerWithAuth, ProtoServer, @@ -2022,7 +2022,7 @@ getQueueLink c nm userId server lnkId = -- resolver) and falls back to a direct send when the proxy is unavailable -- (faster but exposes the client IP). Mode selection is delegated to -- `sendOrProxySMPCommand`, which honours the network config (SPMNever etc.). -resolveName :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> SimplexDomain -> AM NameRecord +resolveName :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> SimplexDomain -> AM NameResponse resolveName c nm userId server domain = snd <$> sendOrProxySMPCommand c nm userId server "" "RSLV" NoEntity resolveViaProxy resolveDirectly where diff --git a/src/Simplex/Messaging/Client.hs b/src/Simplex/Messaging/Client.hs index 6f5234558..f8f1a4cb9 100644 --- a/src/Simplex/Messaging/Client.hs +++ b/src/Simplex/Messaging/Client.hs @@ -166,7 +166,7 @@ import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, sumTypeJSON import Simplex.Messaging.Protocol import Simplex.Messaging.Protocol.Types import Simplex.Messaging.Server.QueueStore.QueueInfo -import Simplex.Messaging.SimplexName (SimplexDomain) +import Simplex.Messaging.SimplexName (SimplexDomain, fullDomainName) import Simplex.Messaging.TMap (TMap) import qualified Simplex.Messaging.TMap as TM import Simplex.Messaging.Transport @@ -1054,11 +1054,11 @@ proxySMPMessage c nm proxiedRelay spKey sId flags msg = proxyOKSMPCommand c nm p -- through `proxySMPCommand` and pattern-matches the expected RNAME response. -- Version-gated on the destination relay (mirrors `connectSMPProxiedRelay`): -- the client never sends RSLV to a relay that predates names support. -proxyResolveName :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> SimplexDomain -> ExceptT SMPClientError IO (Either ProxyClientError NameRecord) +proxyResolveName :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> SimplexDomain -> ExceptT SMPClientError IO (Either ProxyClientError NameResponse) proxyResolveName c nm proxiedRelay name | prVersion proxiedRelay >= namesSMPVersion = - proxySMPCommand c nm proxiedRelay Nothing NoEntity (RSLV name) >>= \case - Right (RNAME nr) -> pure $ Right nr + proxySMPCommand c nm proxiedRelay Nothing NoEntity (RSLV (NQDomain name)) >>= \case + Right (RNAME reg) | resolvedNameOrNotFound name reg -> pure $ Right reg Right r -> throwE $ unexpectedResponse r Left e -> pure $ Left e | otherwise = throwE $ PCETransportError TEVersion @@ -1066,16 +1066,21 @@ proxyResolveName c nm proxiedRelay name -- | Direct (non-PFWD) name resolution. Exposes the client IP to the resolver; -- callers that want anonymity should use `proxyResolveName` via the standard -- proxy fallback in the agent. RSLV requires no entity ID or authorization --- (see `noAuthCmd` in Protocol.hs). Version-gated on the session here, not the --- encoder, so an old server never receives RSLV. -directResolveName :: SMPClient -> NetworkRequestMode -> SimplexDomain -> ExceptT SMPClientError IO NameRecord +-- (see `noAuthCmd` in Protocol.hs). Gated on the session version, below which +-- the server has no RSLV at all; the encoder gates the query format separately. +directResolveName :: SMPClient -> NetworkRequestMode -> SimplexDomain -> ExceptT SMPClientError IO NameResponse directResolveName c nm name | thVersion (thParams c) >= namesSMPVersion = - sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (RSLV name)) >>= \case - RNAME nr -> pure nr + sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (RSLV (NQDomain name))) >>= \case + RNAME reg | resolvedNameOrNotFound name reg -> pure reg r -> throwE $ unexpectedResponse r | otherwise = throwE $ PCETransportError TEVersion +resolvedNameOrNotFound :: SimplexDomain -> NameResponse -> Bool +resolvedNameOrNotFound d NameResponse {registration} = case registration of + NRRegistered {nameRecord} -> T.toLower (nrName nameRecord) == fullDomainName d + _ -> True + -- | Acknowledge message delivery (server deletes the message). -- -- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#acknowledge-message-delivery diff --git a/src/Simplex/Messaging/Names/Record.hs b/src/Simplex/Messaging/Names/Record.hs index c13d7fccc..461bec16e 100644 --- a/src/Simplex/Messaging/Names/Record.hs +++ b/src/Simplex/Messaging/Names/Record.hs @@ -1,3 +1,6 @@ +{-# LANGUAGE DerivingStrategies #-} +{-# LANGUAGE GeneralizedNewtypeDeriving #-} +{-# LANGUAGE LambdaCase #-} {-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE StrictData #-} @@ -5,13 +8,24 @@ module Simplex.Messaging.Names.Record ( NameRecord (..), + NameResponse (..), + NameRegistration (..), + NamePricing (..), + USDCents (..), + NameReservedReason (..), ) where +import Data.Aeson (FromJSON (..), ToJSON (..)) import qualified Data.Aeson as J import qualified Data.Aeson.TH as JQ +import Data.Int (Int64) +import Data.Map.Strict (Map) import Data.Text (Text) -import Simplex.Messaging.Parsers (defaultJSON, dropPrefix) +import qualified Data.Text as T +import Simplex.Messaging.Encoding.String +import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, taggedObjectJSON) +import Simplex.Messaging.SystemTime (SystemSeconds) -- | Resolved name record returned by the names role. JSON keys match the -- resolver REST output; both FromJSON (resolver -> server) and ToJSON @@ -44,3 +58,87 @@ $( JQ.deriveJSON defaultJSON {J.omitNothingFields = False, J.fieldLabelModifier = dropPrefix "nr"} ''NameRecord ) + +-- | US cents. +newtype USDCents = USDCents Int64 + deriving (Eq, Ord, Show) + deriving newtype (ToJSON, FromJSON) + +-- | What the registry holds for a name, and the block it was read at. +data NameResponse = NameResponse + { -- | absent only from a v20/v21 router, which sent the record alone + lastBlockTs :: Maybe SystemSeconds, + registration :: NameRegistration + } + deriving (Eq, Show) + +-- | What the registry holds for a name. +data NameRegistration + = -- | Held by someone. Always carries a record, empty where none was set. + NRRegistered + { -- | absent only from a v20/v21 router, which sent the record alone + expires :: Maybe SystemSeconds, + -- | unix seconds, > expires: until here only the owner may renew + graceUntil :: Maybe SystemSeconds, + -- | held back as well, which is why it will not free up at expiry + reservedReason_ :: Maybe NameReservedReason, + nameRecord :: NameRecord + } + | -- | Held by nobody, and registrable now. + NRAvailable {pricing :: NamePricing} + | -- | Held back by the registry, and not for sale at its price. + NRReserved {reservedReason :: NameReservedReason} + deriving (Eq, Show) + +-- | Enough to price the name locally, which the router cannot do behind a hash. +data NamePricing = NamePricing + { -- | US cents per year, for the lengths the registry prices specially + registrationPrices :: Map Int USDCents, + -- | US cents per year for every other length + basePrice :: USDCents, + -- | characters; the registry refuses shorter labels + minLabelLength :: Int + } + deriving (Eq, Show) + +-- | Why the registry holds a name back. +data NameReservedReason + = -- | held for SimpleX + NRRInternal + | NRRTrademark + | NRRCommunity + | -- | added to the registry after this version, and still reserved + NRRUnknown Text + deriving (Eq, Show) + +instance TextEncoding NameReservedReason where + textEncode = \case + NRRInternal -> "internal" + NRRTrademark -> "trademark" + NRRCommunity -> "community" + NRRUnknown t -> t + textDecode = Just . reservedReasonOf + +-- | An unknown reason is kept as text, capped: it reaches a client as a word. +-- Capping precedes the match, so what is kept encodes back to what it decoded. +reservedReasonOf :: Text -> NameReservedReason +reservedReasonOf t = case T.take 32 $ T.takeWhile (\c -> c > ' ' && c < '\DEL') t of + "internal" -> NRRInternal + "trademark" -> NRRTrademark + "community" -> NRRCommunity + r -> NRRUnknown r + +instance ToJSON NameReservedReason where + toJSON = textToJSON + toEncoding = textToEncoding + +instance FromJSON NameReservedReason where + parseJSON = textParseJSON "NameReservedReason" + +$(JQ.deriveJSON defaultJSON ''NamePricing) + +-- taggedObjectJSON, not sumTypeJSON: this JSON is the RNAME payload and the +-- resolver contract, so it must not vary with the swift build flag. +$(JQ.deriveJSON (taggedObjectJSON $ dropPrefix "NR") ''NameRegistration) + +$(JQ.deriveJSON defaultJSON ''NameResponse) diff --git a/src/Simplex/Messaging/Protocol.hs b/src/Simplex/Messaging/Protocol.hs index a28d78fe6..e2cbef3a4 100644 --- a/src/Simplex/Messaging/Protocol.hs +++ b/src/Simplex/Messaging/Protocol.hs @@ -80,6 +80,12 @@ module Simplex.Messaging.Protocol ErrorType (..), CommandError (..), ProxyError (..), + NameQuery (..), + NameResponse (..), + NameRegistration (..), + NamePricing (..), + USDCents (..), + NameReservedReason (..), NameErrorType (..), BrokerErrorType (..), NetworkError (..), @@ -265,12 +271,12 @@ import Simplex.Messaging.Agent.Store.DB (Binary (..), FromField (..), ToField (. import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String -import Simplex.Messaging.Names.Record (NameRecord (..)) +import Simplex.Messaging.Names.Record import Simplex.Messaging.Parsers import Simplex.Messaging.Protocol.Types import Simplex.Messaging.Server.QueueStore.QueueInfo import Simplex.Messaging.ServiceScheme -import Simplex.Messaging.SimplexName (SimplexDomain) +import Simplex.Messaging.SimplexName (LabelHash, SimplexDomain (..), SimplexTLD (..), fullDomainName, labelHash) import Simplex.Messaging.Transport import Simplex.Messaging.Transport.Client (TransportHost, TransportHosts (..)) import Simplex.Messaging.Util (bshow, eitherToMaybe, safeDecodeUtf8, (<$?>)) @@ -603,7 +609,7 @@ data Command (p :: Party) where -- - corrId: unique correlation ID between proxy and relay, also used as a nonce to encrypt forwarded transmission RFWD :: EncFwdTransmission -> Command ProxyService -- use CorrId as CbNonce, proxy to relay -- Resolve SimpleX name. - RSLV :: SimplexDomain -> Command Resolver + RSLV :: NameQuery -> Command Resolver deriving instance Show (Command p) @@ -739,8 +745,8 @@ data BrokerMsg where OK :: BrokerMsg ERR :: ErrorType -> BrokerMsg PONG :: BrokerMsg - -- Resolved SimpleX name. - RNAME :: NameRecord -> BrokerMsg + -- What the router knows about a SimpleX name. + RNAME :: NameResponse -> BrokerMsg deriving (Eq, Show) data RcvMessage = RcvMessage @@ -1589,11 +1595,28 @@ data ErrorType DUPLICATE_ -- not part of SMP protocol, used internally deriving (Eq, Show) +-- | What RSLV asks about: a name, or the hash of a second-level label. +data NameQuery = NQDomain SimplexDomain | NQHash LabelHash SimplexTLD + deriving (Eq, Show) + +instance Encoding NameQuery where + smpEncode = \case + NQDomain d -> encodeUtf8 $ fullDomainName d + NQHash h tld -> strEncode h <> strEncode tld + smpP = NQHash <$> strP <*> strP <|> NQDomain <$> strP + +-- | Hashed from v22, except a name with subnames or a web TLD. +hashedQuery :: NameQuery -> NameQuery +hashedQuery q = case q of + NQDomain SimplexDomain {nameTLD, domain, subDomain} + | null subDomain && nameTLD /= TLDWeb -> NQHash (labelHash domain) nameTLD + _ -> q + -- | Name resolution error data NameErrorType = -- | the names role / resolver is not configured on this server NO_RESOLVER - | -- | the name is not registered (resolver returned not-found) + | -- | the name does not resolve; sent only to a session below v22 NOT_FOUND | -- | backing resolver/RPC failure - contains the diagnostic detail RESOLVER {resolverErr :: Text} @@ -1822,7 +1845,7 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where PRXY host auth_ -> e (PRXY_, ' ', host, auth_) PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s) RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s) - RSLV d -> e (RSLV_, ' ', d) + RSLV q -> e (RSLV_, ' ', if v >= nameAvailSMPVersion then hashedQuery q else q) where e :: Encoding a => a -> ByteString e = smpEncode @@ -1929,7 +1952,7 @@ instance ProtocolEncoding SMPVersion ErrorType Cmd where CT SNotifierService NSUBS_ | v >= rcvServiceSMPVersion -> Cmd SNotifierService <$> (NSUBS <$> _smpP <*> smpP) | otherwise -> pure $ Cmd SNotifierService $ NSUBS (-1) mempty - CT SResolver RSLV_ -> Cmd SResolver . RSLV <$> _smpP <* A.takeByteString + CT SResolver RSLV_ -> Cmd SResolver . RSLV <$> _smpP fromProtocolError = fromProtocolError @SMPVersion @ErrorType @BrokerMsg {-# INLINE fromProtocolError #-} @@ -1972,7 +1995,11 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where | v < clientNoticesSMPVersion -> BLOCKED info {notice = Nothing} _ -> err PONG -> e PONG_ - RNAME rec -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode rec) + RNAME res + | v >= nameAvailSMPVersion -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode res) + | otherwise -> case registration res of + NRRegistered {nameRecord} -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode nameRecord) + _ -> e (ERR_, ' ', NAME NOT_FOUND) where e :: Encoding a => a -> ByteString e = smpEncode @@ -2019,8 +2046,12 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where OK_ -> pure OK ERR_ -> ERR <$> _smpP PONG_ -> pure PONG - RNAME_ -> fmap RNAME . J.eitherDecodeStrict . unTail <$?> _smpP + RNAME_ + | v >= nameAvailSMPVersion -> fmap RNAME . J.eitherDecodeStrict . unTail <$?> _smpP + | otherwise -> fmap (RNAME . oldResponse) . J.eitherDecodeStrict . unTail <$?> _smpP where + oldResponse nameRecord = + NameResponse {lastBlockTs = Nothing, registration = NRRegistered {expires = Nothing, graceUntil = Nothing, reservedReason_ = Nothing, nameRecord}} serviceRespP resp | v >= rcvServiceSMPVersion = resp <$> _smpP <*> smpP | otherwise = resp <$> _smpP <*> pure mempty @@ -2042,7 +2073,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where PKEY {} -> noEntityMsg RRES _ -> noEntityMsg ALLS -> noEntityMsg - RNAME _ -> noEntityMsg + RNAME {} -> noEntityMsg -- other broker responses must have queue ID _ | B.null entId -> Left $ CMD NO_ENTITY @@ -2412,3 +2443,4 @@ $(J.deriveJSON defaultJSON ''BlockingInfo) -- run deriveJSON in one TH splice to allow mutual instance $(concat <$> mapM @[] (J.deriveJSON (sumTypeJSON id)) [''ProxyError, ''NameErrorType, ''ErrorType]) + diff --git a/src/Simplex/Messaging/Server.hs b/src/Simplex/Messaging/Server.hs index 16ad58ab3..097cabbbe 100644 --- a/src/Simplex/Messaging/Server.hs +++ b/src/Simplex/Messaging/Server.hs @@ -104,7 +104,6 @@ import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding import Simplex.Messaging.Encoding.String import Simplex.Messaging.Protocol -import Simplex.Messaging.SimplexName (SimplexDomain) import Simplex.Messaging.Server.Control import Simplex.Messaging.Server.Env.STM as Env import Simplex.Messaging.Server.Expiration @@ -1494,15 +1493,19 @@ client Just nenv -> pure (Just nenv) -- Runs on a forked thread so RSLV does not block other commands; -- concurrency is limited by serverResolverConcurrency in forkCmd. - resolveNameMsg :: NamesEnv -> SimplexDomain -> M s BrokerMsg - resolveNameMsg nenv d = do + resolveNameMsg :: VersionSMP -> NamesEnv -> NameQuery -> M s BrokerMsg + resolveNameMsg v nenv q = do st <- asks (rslvStats . serverStats) (selector, msg) <- - liftIO (resolveName nenv d) <&> \case - Right rec -> (rslvSucc, RNAME rec) - Left e@NOT_FOUND -> (rslvNotFound, ERR $ NAME e) + liftIO (resolveName nenv q) <&> \case + Right res -> (if answered (registration res) then rslvSucc else rslvNotFound, RNAME res) Left e -> (rslvResolverErrs, ERR $ NAME e) incStat (selector st) $> msg + where + -- below v22 the encoder answers anything but a record as NAME NOT_FOUND + answered = \case + NRRegistered {} -> True + _ -> v >= nameAvailSMPVersion transportErr :: TransportError -> ErrorType transportErr = PROXY . BROKER . TRANSPORT mkIncProxyStats :: MonadIO m => ProxyStats -> ProxyStats -> OwnServer -> (ProxyStats -> IORef Int) -> m () @@ -1519,7 +1522,7 @@ client Cmd SProxyService (RFWD encBlock) -> (response . (corrId, NoEntity,) =<<) <$> processForwardedCommand encBlock Cmd SResolver (RSLV d) -> rslvNamesEnv >>= \case Nothing -> pure $ response (corrId, NoEntity, ERR (NAME NO_RESOLVER)) - Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity (resolveNameMsg nenv d) + Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity (resolveNameMsg (thVersion thParams') nenv d) Cmd SSenderLink command -> case command of LKEY k -> withQueue $ \q qr -> checkMode QMMessaging qr $ secureQueue_ q k $>> getQueueLink_ q qr LGET -> withQueue $ \q qr -> checkContact qr $ getQueueLink_ q qr @@ -2150,7 +2153,7 @@ client Cmd SResolver (RSLV d) -> lift $ rslvNamesEnv >>= \case Nothing -> pure $ Just (corrId', entId', ERR (NAME NO_RESOLVER)) Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity $ do - msg <- resolveNameMsg nenv d + msg <- resolveNameMsg (thVersion clntTHParams) nenv d either ERR id <$> runExceptT (encodeResp (corrId', entId', msg)) -- INTERNAL because processCommand never returns Nothing for sender commands; -- `fst` drops the empty message only returned for SUB. diff --git a/src/Simplex/Messaging/Server/Names.hs b/src/Simplex/Messaging/Server/Names.hs index 19bae15fc..601519c56 100644 --- a/src/Simplex/Messaging/Server/Names.hs +++ b/src/Simplex/Messaging/Server/Names.hs @@ -20,7 +20,9 @@ import Control.Logger.Simple (logError) import Data.Bifunctor (first) import Data.Maybe (fromMaybe) import qualified Data.Text as T -import Simplex.Messaging.Protocol (NameErrorType (..), NameRecord) +import Data.Text.Encoding (decodeLatin1) +import Simplex.Messaging.Encoding +import Simplex.Messaging.Protocol (NameErrorType (..), NameQuery, NameResponse) import Simplex.Messaging.Server.Names.HttpResolver ( ResolverEnv, ResolverError (..), @@ -30,7 +32,6 @@ import Simplex.Messaging.Server.Names.HttpResolver newResolverEnv, resolveHttp, ) -import Simplex.Messaging.SimplexName (SimplexDomain, fullDomainName) import System.Timeout (timeout) data NamesConfig = NamesConfig @@ -58,9 +59,9 @@ pingEndpoint :: NamesEnv -> IO (Either ResolverError ()) pingEndpoint NamesEnv {resolverEnv, config} = fromMaybe (Left ResolverTimeout) <$> timeout (resolverTimeoutMs config * 1000) (healthHttp resolverEnv) -resolveName :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord) -resolveName env d = do - r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env d)) +resolveName :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameResponse) +resolveName env q = do + r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env q)) case r of Right result -> pure (fromMaybe (Left (RESOLVER "timeout")) result) Left e @@ -69,14 +70,12 @@ resolveName env d = do logError $ "[NAMES] resolver fetch raised " <> T.pack (E.displayException e) pure (Left (RESOLVER "resolver error")) -fetch :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord) -fetch NamesEnv {resolverEnv} d = - first mapResolverError <$> resolveHttp resolverEnv (fullDomainName d) +fetch :: NamesEnv -> NameQuery -> IO (Either NameErrorType NameResponse) +fetch NamesEnv {resolverEnv} q = + first mapResolverError <$> resolveHttp resolverEnv (decodeLatin1 $ smpEncode q) mapResolverError :: ResolverError -> NameErrorType mapResolverError = \case - HttpStatusErr 404 -> NOT_FOUND - HttpStatusErr 400 -> NOT_FOUND HttpStatusErr code -> RESOLVER ("HTTP " <> T.pack (show code)) HttpFailure _ -> RESOLVER "transport failure" BodyTooLarge -> RESOLVER "response too large" diff --git a/src/Simplex/Messaging/Server/Names/HttpResolver.hs b/src/Simplex/Messaging/Server/Names/HttpResolver.hs index 118810a08..0f272bcf4 100644 --- a/src/Simplex/Messaging/Server/Names/HttpResolver.hs +++ b/src/Simplex/Messaging/Server/Names/HttpResolver.hs @@ -8,10 +8,12 @@ -- -- The Python REST resolver (see scripts/resolver/snrc-resolve.py) exposes -- --- GET /resolve/ -> 200 with a NameRecord JSON document --- 404 / 400 for unknown names / TLDs --- 502 for upstream RPC failures --- GET /health -> 200 when the resolver process is ready +-- GET /v2/resolve/ -> 200 with a NameRegistration JSON document, for +-- all three registration shapes; 400 for unknown +-- TLDs, 502 for upstream RPC failures +-- GET /v1/resolve/ -> 200 with a NameRecord, what relays before SMP +-- v22 call as /resolve +-- GET /health -> 200 when the resolver process is ready -- -- Boundary properties: -- * Response body read with `brReadSome maxResponseBytes` — adversarial @@ -57,7 +59,7 @@ import qualified Network.HTTP.Client as HC import Network.HTTP.Client.TLS (tlsManagerSettings) import qualified Network.HTTP.Types as HT import Network.HTTP.Types.URI (urlEncode) -import Simplex.Messaging.Names.Record (NameRecord) +import Simplex.Messaging.Names.Record (NameResponse) data RpcAuth = AuthBearer Text | AuthBasic Text Text @@ -108,14 +110,12 @@ authHeader = \case let encoded = BAE.convertToBase BAE.Base64 (encodeUtf8 u <> ":" <> encodeUtf8 p) :: ByteString in ("Authorization", "Basic " <> encoded) --- | GET /resolve/, decoding the 200 body --- directly into a NameRecord in one pass (no intermediate Aeson Value). The --- name is percent-encoded (every non-unreserved byte per RFC 3986): the --- resolver expects raw labels, so slashes/punctuation must not alter the path. -resolveHttp :: ResolverEnv -> Text -> IO (Either ResolverError NameRecord) -resolveHttp env name = +-- | The query is a name or a bracketed label hash, percent-encoded (every +-- non-unreserved byte per RFC 3986) so it cannot alter the path. +resolveHttp :: ResolverEnv -> Text -> IO (Either ResolverError NameResponse) +resolveHttp env q = (>>= first InvalidJson . J.eitherDecodeStrict . BL.toStrict) - <$> httpGet env ("/resolve/" <> B.unpack (urlEncode True (encodeUtf8 name))) + <$> httpGet env ("/v2/resolve/" <> B.unpack (urlEncode True (encodeUtf8 q))) -- | GET /health; success = reachable with status < 400. The body is -- size-capped but NOT decoded — the probe only checks reachability. diff --git a/src/Simplex/Messaging/Server/Prometheus.hs b/src/Simplex/Messaging/Server/Prometheus.hs index 575f699c6..85d2624fd 100644 --- a/src/Simplex/Messaging/Server/Prometheus.hs +++ b/src/Simplex/Messaging/Server/Prometheus.hs @@ -469,11 +469,11 @@ prometheusMetrics sm rtm ts = \# TYPE simplex_smp_names_reqs counter\n\ \simplex_smp_names_reqs " <> mshow _rslvReqs <> "\n# rslvReqs\n\ \\n\ - \# HELP simplex_smp_names_success NameRecord successfully resolved and returned.\n\ + \# HELP simplex_smp_names_success NameRecord resolved, or availability answered.\n\ \# TYPE simplex_smp_names_success counter\n\ \simplex_smp_names_success " <> mshow _rslvSucc <> "\n# rslvSucc\n\ \\n\ - \# HELP simplex_smp_names_not_found Name not registered (resolver returned 404 / 400).\n\ + \# HELP simplex_smp_names_not_found Answers a client below v22 reads as NOT_FOUND.\n\ \# TYPE simplex_smp_names_not_found counter\n\ \simplex_smp_names_not_found " <> mshow _rslvNotFound <> "\n# rslvNotFound\n\ \\n\ diff --git a/src/Simplex/Messaging/SimplexName.hs b/src/Simplex/Messaging/SimplexName.hs index 2dd0f8645..535682f0c 100644 --- a/src/Simplex/Messaging/SimplexName.hs +++ b/src/Simplex/Messaging/SimplexName.hs @@ -10,14 +10,21 @@ module Simplex.Messaging.SimplexName SimplexTLD (..), SimplexNameType (..), fullDomainName, + LabelHash (..), + labelHash, + boundedNonSpace, shortNameInfoStr, ) where import Control.Applicative (optional, (<|>)) +import Crypto.Hash (Digest, hash) +import Crypto.Hash.Algorithms (Keccak_256) import qualified Data.Aeson.TH as J import qualified Data.Attoparsec.ByteString.Char8 as A import qualified Data.Attoparsec.Text as AT +import qualified Data.ByteArray as BA +import qualified Data.ByteArray.Encoding as BAE import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as B import Data.Char (isDigit) @@ -70,6 +77,20 @@ nameLabelP = do -- (Cyrillic а vs ASCII a hash to different on-chain records). isNameLetter c = c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' +-- | The registry's key for a label: 32 bytes, as labelOf takes it. +newtype LabelHash = LabelHash ByteString + deriving (Eq, Show) + +-- | keccak-256 of the lowercased label, as the registry keys it. +labelHash :: Text -> LabelHash +labelHash label = LabelHash $ BA.convert (hash (encodeUtf8 (T.toLower label)) :: Digest Keccak_256) + +instance StrEncoding LabelHash where + strEncode (LabelHash h) = '[' `B.cons` (BAE.convertToBase BAE.Base16 h `B.snoc` ']') + strP = do + h <- BAE.convertFromBase BAE.Base16 <$?> (A.char '[' *> A.takeWhile (/= ']') <* A.char ']') + if B.length h == 32 then pure $ LabelHash h else fail "bad LabelHash" + -- | Cap the name at 253 bytes (DNS full-domain limit) boundedNonSpace :: A.Parser ByteString boundedNonSpace = do @@ -108,12 +129,15 @@ instance Encoding SimplexDomain where smpP = strP fullDomainName :: SimplexDomain -> Text -fullDomainName SimplexDomain {nameTLD, domain, subDomain} = T.intercalate "." (reverse subDomain ++ [domain] ++ tld') - where - tld' = case nameTLD of - TLDSimplex -> ["simplex"] - TLDTesting -> ["testing"] - TLDWeb -> [] +fullDomainName SimplexDomain {nameTLD, domain, subDomain} = T.intercalate "." (reverse subDomain ++ [domain]) <> decodeLatin1 (strEncode nameTLD) + +instance StrEncoding SimplexTLD where + strEncode = \case + TLDSimplex -> ".simplex" + TLDTesting -> ".testing" + TLDWeb -> "" + strP = + ".simplex" $> TLDSimplex <|> ".testing" $> TLDTesting <|> pure TLDWeb shortNameInfoStr :: SimplexNameInfo -> Text shortNameInfoStr = \case diff --git a/src/Simplex/Messaging/Transport.hs b/src/Simplex/Messaging/Transport.hs index a366e79f7..d2c30d25a 100644 --- a/src/Simplex/Messaging/Transport.hs +++ b/src/Simplex/Messaging/Transport.hs @@ -53,6 +53,7 @@ module Simplex.Messaging.Transport rcvServiceSMPVersion, namesSMPVersion, serverInfoSMPVersion, + nameAvailSMPVersion, simplexMQVersion, smpBlockSize, TransportConfig (..), @@ -175,6 +176,7 @@ smpBlockSize = 16384 -- 19 - service subscriptions to messages (10/20/2025) -- 20 - public namespaces resolver, RSLV command (6/20/2026) -- 21 - server public information in handshake (7/5/2026) +-- 22 - RNAME answers name availability as well as the record (7/25/2026) data SMPVersion @@ -211,6 +213,11 @@ namesSMPVersion = VersionSMP 20 serverInfoSMPVersion :: VersionSMP serverInfoSMPVersion = VersionSMP 21 +-- | RNAME carries availability. A server below this answers RSLV with the +-- record alone, and ERR NAME NOT_FOUND for a name that does not resolve. +nameAvailSMPVersion :: VersionSMP +nameAvailSMPVersion = VersionSMP 22 + minClientSMPRelayVersion :: VersionSMP minClientSMPRelayVersion = VersionSMP 14 @@ -218,20 +225,20 @@ minServerSMPRelayVersion :: VersionSMP minServerSMPRelayVersion = VersionSMP 14 currentClientSMPRelayVersion :: VersionSMP -currentClientSMPRelayVersion = VersionSMP 21 +currentClientSMPRelayVersion = VersionSMP 22 currentServerSMPRelayVersion :: VersionSMP -currentServerSMPRelayVersion = VersionSMP 21 +currentServerSMPRelayVersion = VersionSMP 22 -- Max SMP protocol version to be used in e2e encrypted connection between -- client and server, as defined by SMP proxy. Normally set below the current -- version to prevent client version fingerprinting by the destination relays --- when clients upgrade at different times. Pinned to the current version (20) --- for this release because proxied name resolution is gated on namesSMPVersion --- (20), so the one-version anti-fingerprinting buffer does not apply yet; it --- reappears once the current version advances past 20. +-- when clients upgrade at different times. Pinned to the current version (22) +-- for this release because a proxied RSLV only carries availability from +-- nameAvailSMPVersion (22), so the one-version anti-fingerprinting buffer does +-- not apply yet; it reappears once the current version advances past 22. proxiedSMPRelayVersion :: VersionSMP -proxiedSMPRelayVersion = VersionSMP 20 +proxiedSMPRelayVersion = VersionSMP 22 -- minimal supported protocol version is 14 supportedClientSMPRelayVRange :: VersionRangeSMP diff --git a/tests/AgentTests/ResolveNameTests.hs b/tests/AgentTests/ResolveNameTests.hs index f55faf75f..d8507194e 100644 --- a/tests/AgentTests/ResolveNameTests.hs +++ b/tests/AgentTests/ResolveNameTests.hs @@ -14,7 +14,6 @@ module AgentTests.ResolveNameTests (resolveNameTests) where import AgentTests.FunctionalAPITests (withAgent) import Control.Monad.Except (runExceptT) -import qualified Data.Aeson as J import qualified Data.ByteString.Lazy as LB import Data.List (isInfixOf) import Network.HTTP.Types (Status, status200, status404, status502) @@ -22,7 +21,7 @@ import NamesResolverServer (memCfg, memCfg2, memProxyCfg, withNames) import qualified NamesResolverServer as NRS import SMPAgentClient import SMPClient -import SMPNamesTests (testNameRecord) +import SMPNamesTests (availableBody, registeredBody, testNameRecord) import Simplex.Messaging.Agent (resolveSimplexName) import Simplex.Messaging.Agent.Client (AgentClient) import Simplex.Messaging.Agent.Env.SQLite (InitialAgentServers (..), ServerCfg, ServerRoles (..), presetServerCfg) @@ -71,13 +70,13 @@ withNoNameServers k = withAgent 1 agentCfg (oneSrv (proxySrvCfg testSMPServer)) resolveNameTests :: Spec resolveNameTests = do describe "direct path (SPMNever)" $ - it "404 propagates as SMP host (NAME NOT_FOUND)" testDirectNotFound + it "a resolver error propagates as SMP host (NAME RESOLVER)" testDirectResolverErr describe "proxy path (SPMAlways)" $ - it "404 from resolver propagates via proxy as SMP (NAME NOT_FOUND)" testProxyNotFound + it "a resolver error propagates via proxy as SMP (NAME RESOLVER)" testProxyResolverErr describe "TLDTesting path" $ - it "NAME NOT_FOUND for TLDTesting too" testTestingTldNotFound + it "NAME RESOLVER for TLDTesting too" testTestingTldResolverErr describe "TLDWeb path" $ - it "NAME NOT_FOUND for TLDWeb too" testWebTldNotFound + it "NAME RESOLVER for TLDWeb too" testWebTldResolverErr describe "no resolver configured" $ it "answers NAME NO_RESOLVER" testNoResolver describe "no names servers (names role off everywhere)" $ @@ -86,38 +85,50 @@ resolveNameTests = do it "surfaces as SMP host (NAME (RESOLVER ..))" testBackendError describe "success path" $ it "returns NameRecord" testDirectSuccess + describe "name availability" $ + it "an unregistered name answers as available" testAvailSuccess -testDirectNotFound :: HasCallStack => IO () -testDirectNotFound = +testAvailSuccess :: HasCallStack => IO () +testAvailSuccess = + withDirectResolver (status200, availableBody) $ \c -> do + r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) + case r of + Right SMP.NameResponse {registration = SMP.NRAvailable {}} -> pure () + _ -> expectationFailure $ "expected Right NRAvailable, got: " <> show r + +-- | 404 is a resolver that predates /v2/resolve: no status from that endpoint +-- means "not registered", since an unregistered name answers NRAvailable. +testDirectResolverErr :: HasCallStack => IO () +testDirectResolverErr = withDirectResolver (status404, "{}") $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) case r of - Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure () - _ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r + Left (SMP _ (SMP.NAME (SMP.RESOLVER _))) -> pure () + _ -> expectationFailure $ "expected Left (SMP _ (NAME (RESOLVER _))), got: " <> show r -testProxyNotFound :: HasCallStack => IO () -testProxyNotFound = +testProxyResolverErr :: HasCallStack => IO () +testProxyResolverErr = withProxyAndResolver (status404, "{}") $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) case r of - Left (SMP host (SMP.NAME SMP.NOT_FOUND)) | testPort `isInfixOf` host -> pure () - _ -> expectationFailure $ "expected Left (SMP testPort <> "> (NAME NOT_FOUND)), got: " <> show r + Left (SMP host (SMP.NAME (SMP.RESOLVER _))) | testPort `isInfixOf` host -> pure () + _ -> expectationFailure $ "expected Left (SMP testPort <> "> (NAME (RESOLVER _))), got: " <> show r -testTestingTldNotFound :: HasCallStack => IO () -testTestingTldNotFound = +testTestingTldResolverErr :: HasCallStack => IO () +testTestingTldResolverErr = withDirectResolver (status404, "{}") $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDTesting "bob" []) case r of - Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure () - _ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r + Left (SMP _ (SMP.NAME (SMP.RESOLVER _))) -> pure () + _ -> expectationFailure $ "expected Left (SMP _ (NAME (RESOLVER _))), got: " <> show r -testWebTldNotFound :: HasCallStack => IO () -testWebTldNotFound = +testWebTldResolverErr :: HasCallStack => IO () +testWebTldResolverErr = withDirectResolver (status404, "{}") $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDWeb "example.com" []) case r of - Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure () - _ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r + Left (SMP _ (SMP.NAME (SMP.RESOLVER _))) -> pure () + _ -> expectationFailure $ "expected Left (SMP _ (NAME (RESOLVER _))), got: " <> show r testNoResolver :: HasCallStack => IO () testNoResolver = @@ -145,8 +156,8 @@ testBackendError = testDirectSuccess :: HasCallStack => IO () testDirectSuccess = - withDirectResolver (status200, J.encode testNameRecord) $ \c -> do + withDirectResolver (status200, registeredBody testNameRecord) $ \c -> do r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" []) case r of - Right nr -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right NameRecord, got: " <> show r + Right SMP.NameResponse {registration = SMP.NRRegistered {nameRecord}} -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right NRRegistered, got: " <> show r diff --git a/tests/NamesResolverServer.hs b/tests/NamesResolverServer.hs index a90595206..054d55e40 100644 --- a/tests/NamesResolverServer.hs +++ b/tests/NamesResolverServer.hs @@ -47,10 +47,12 @@ withResolverServerDelayed delayMs handler action = do let (st, body) = handler (pathInfo req) send $ responseLBS st [(hContentType, "application/json")] body +-- | The resolver API is versioned on its own: v2 answers with NameRegistration +-- JSON, which is the only shape the server asks for. resolveResp :: Status -> LB.ByteString -> [Text] -> (Status, LB.ByteString) resolveResp st body = \case ["health"] -> (ok200, "{}") - ("resolve" : _) -> (st, body) + ("v2" : "resolve" : _) -> (st, body) _ -> (notFound404, "{}") testNamesConfig :: Int -> NamesConfig diff --git a/tests/RSLVTests.hs b/tests/RSLVTests.hs index 2416d851e..9263b6953 100644 --- a/tests/RSLVTests.hs +++ b/tests/RSLVTests.hs @@ -12,9 +12,9 @@ module RSLVTests (rslvTests) where import Control.Monad.Trans.Except (ExceptT, runExceptT) -import qualified Data.Aeson as J import qualified Data.ByteString.Char8 as B import qualified Data.ByteString.Lazy as LB +import Data.IORef (IORef, readIORef) import Data.List.NonEmpty (NonEmpty (..)) import Data.Text (Text) import Data.Text.Encoding (encodeUtf8) @@ -26,14 +26,18 @@ import SMPClient import Simplex.Messaging.Client import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Encoding.String (strDecode) -import SMPNamesTests (testNameRecord) +import SMPNamesTests (availableBody, registeredBody, reservedBody, resolved, testNameRecord, testPricing) import Simplex.Messaging.Protocol ( BrokerMsg (..), Cmd (..), Command (..), CorrId (..), ErrorType (..), + NameQuery (..), + NameRegistration (..), + NameResponse (..), NameErrorType (..), + NameReservedReason (..), SParty (..), Transmission, TransmissionForAuth (..), @@ -57,6 +61,11 @@ withResolverServer (st, body) runTest = NRS.withResolverServer (NRS.resolveResp st body) $ \port _ -> withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort (const runTest) +withResolverServerReqs :: (Status, LB.ByteString) -> (IORef [[Text]] -> IO a) -> IO a +withResolverServerReqs (st, body) runTest = + NRS.withResolverServer (NRS.resolveResp st body) $ \port reqs -> + withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort (const (runTest reqs)) + withProxyAndResolver :: (Status, LB.ByteString) -> IO a -> IO a withProxyAndResolver (st, body) runTest = NRS.withResolverServer (NRS.resolveResp st body) $ \port _ -> @@ -65,7 +74,7 @@ withProxyAndResolver (st, body) runTest = sendRslv :: Transport c => THandleSMP c 'TClient -> B.ByteString -> SimplexDomain -> IO (Transmission (Either ErrorType BrokerMsg)) sendRslv h@THandle {params} corrId d = do - let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV d)) + let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV (NQDomain d))) [Right ()] <- tPut h (Right (Nothing, tToSend) :| []) r :| _ <- tGetClient h pure r @@ -73,23 +82,36 @@ sendRslv h@THandle {params} corrId d = do rslvTests :: Spec rslvTests = do describe "RSLV direct (non-forwarded)" $ do - it "resolver replies 404 -> NAME NOT_FOUND (reached, not CMD PROHIBITED)" testRslvBackendNotFound + it "resolver without the v2 route (404) -> NAME RESOLVER, not NOT_FOUND" testRslvBackendNotFound it "resolver replies 502 -> NAME (RESOLVER ..)" testRslvBackendHttpErr it "no names config -> NAME NO_RESOLVER" testRslvDisabled it "refuses to send RSLV on a session below namesSMPVersion" testRslvVersion describe "RSLV forwarded (PFWD)" $ do - it "PFWD-wrapped RSLV reaches resolver via proxy (PCEProtocolError (NAME NOT_FOUND))" testRslvForwarded + it "PFWD-wrapped RSLV reaches resolver via proxy (PCEProtocolError (NAME RESOLVER))" testRslvForwarded it "PFWD-wrapped RSLV success returns RNAME (record JSON frames over the proxy)" testRslvForwardedSuccess describe "RSLV success path (RNAME response)" $ do it "returns RNAME with NameRecord" testRslvSuccess + describe "RSLV availability (RNAME response)" $ do + it "unregistered comes back AVAILABLE" testRslvAvailable + it "reserved comes back with the reason" testRslvReserved + it "PFWD-wrapped availability reaches the resolver" testRslvForwardedAvailable + describe "RSLV below v22" $ do + it "still resolves a name to its record" testRslvOldClientRecord + it "still answers NAME NOT_FOUND for a name that does not resolve" testRslvOldClientNotFound + describe "hashed lookups" $ do + it "RSLV sends the 2LD as its hash" testRslvSendsTheHash + it "a name with subnames is sent as text" testSubnameKeepsItsLabels + it "a record naming a different name is rejected" testRslvWrongName +-- | /v2/resolve answers 200, 400 or 502, so a 404 is a resolver that predates +-- the route, not a name that does not exist. testRslvBackendNotFound :: IO () testRslvBackendNotFound = withResolverServer (status404, "{}") $ testSMPClient @TLS $ \h -> do (corrId, _entId, resp) <- sendRslv h "rs01" (domain "ghost.simplex") corrId `shouldBe` CorrId "rs01" - resp `shouldBe` Right (ERR (NAME NOT_FOUND)) + resp `shouldBe` Right (ERR (NAME (RESOLVER "HTTP 404"))) testRslvBackendHttpErr :: IO () testRslvBackendHttpErr = @@ -107,7 +129,7 @@ testRslvDisabled = testRslvVersion :: IO () testRslvVersion = - withResolverServer (status200, J.encode testNameRecord) $ do + withResolverServer (status200, registeredBody testNameRecord) $ do g <- C.newRandom ts <- getCurrentTime let srv = SMPServer testHost testPort testKeyHash @@ -119,7 +141,7 @@ testRslvVersion = Left (PCETransportError TEVersion) -> pure () _ -> expectationFailure $ "expected Left (PCETransportError TEVersion), got: " <> show r -forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameRecord)) +forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameResponse)) forwardedResolveAlice = do g <- C.newRandom ts <- getCurrentTime @@ -135,25 +157,123 @@ testRslvForwarded :: IO () testRslvForwarded = withProxyAndResolver (status404, "{}") $ forwardedResolveAlice >>= \r -> case r of - Left (PCEProtocolError (SMP.NAME SMP.NOT_FOUND)) -> pure () - _ -> expectationFailure $ "expected Left (PCEProtocolError (NAME NOT_FOUND)), got: " <> show r + Left (PCEProtocolError (SMP.NAME (SMP.RESOLVER _))) -> pure () + _ -> expectationFailure $ "expected Left (PCEProtocolError (NAME (RESOLVER _))), got: " <> show r testRslvForwardedSuccess :: IO () testRslvForwardedSuccess = - withProxyAndResolver (status200, J.encode testNameRecord) $ + withProxyAndResolver (status200, registeredBody testNameRecord) $ forwardedResolveAlice >>= \r -> case r of - Right (Right nr) -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right (Right NameRecord), got: " <> show r + Right (Right NameResponse {registration = NRRegistered {nameRecord}}) -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right (Right NRRegistered), got: " <> show r testRslvSuccess :: IO () testRslvSuccess = - withResolverServer (status200, J.encode testNameRecord) $ + withResolverServer (status200, registeredBody testNameRecord) $ testSMPClient @TLS $ \h -> do (corrId, _entId, resp) <- sendRslv h "rs07" (domain "alice.simplex") corrId `shouldBe` CorrId "rs07" case resp of - Right (RNAME nr) -> nr `shouldBe` testNameRecord - _ -> expectationFailure $ "expected Right (RNAME ..), got: " <> show resp + Right (RNAME NameResponse {registration = NRRegistered {nameRecord}}) -> nameRecord `shouldBe` testNameRecord + _ -> expectationFailure $ "expected Right (RNAME NRRegistered), got: " <> show resp + +testRslvAvailable :: IO () +testRslvAvailable = + withResolverServer (status200, availableBody) $ + testSMPClient @TLS $ \h -> do + (corrId, _entId, resp) <- sendRslv h "na01" (domain "ghost.simplex") + corrId `shouldBe` CorrId "na01" + resp `shouldBe` Right (RNAME (resolved (NRAvailable testPricing))) + +testRslvReserved :: IO () +testRslvReserved = + withResolverServer (status200, reservedBody) $ + testSMPClient @TLS $ \h -> do + (_, _, resp) <- sendRslv h "na03" (domain "acme.simplex") + resp `shouldBe` Right (RNAME (resolved (NRReserved NRRTrademark))) + +-- | A client that predates v22 must see exactly what it saw before: the record +-- for a name that resolves, and NOT_FOUND for one that does not. +oldClient :: IO SMPClient +oldClient = do + g <- C.newRandom + ts <- getCurrentTime + let srv = SMPServer testHost testPort testKeyHash + -- the version just below the gate: a lower ceiling would pass even if + -- the gate were at 20 or 21 + oldCfg = defaultSMPClientConfig {serverVRange = mkVersionRange minServerSMPRelayVersion serverInfoSMPVersion} + pcE <- getProtocolClient g NRMInteractive (1, srv, Nothing) oldCfg [] Nothing ts (\_ -> pure ()) + either (fail . show) pure pcE + +testRslvOldClientRecord :: IO () +testRslvOldClientRecord = + withResolverServer (status200, registeredBody testNameRecord) $ do + pc <- oldClient + r <- runExceptT' (directResolveName pc NRMInteractive (domain "alice.simplex")) + r `shouldBe` NameResponse Nothing (NRRegistered Nothing Nothing Nothing testNameRecord) + +testRslvOldClientNotFound :: IO () +testRslvOldClientNotFound = + withResolverServer (status200, availableBody) $ do + pc <- oldClient + r <- runExceptT (directResolveName pc NRMInteractive (domain "alice.simplex")) + case r of + Left (PCEProtocolError (SMP.NAME SMP.NOT_FOUND)) -> pure () + _ -> expectationFailure $ "expected Left (PCEProtocolError (NAME NOT_FOUND)), got: " <> show r + +testRslvForwardedAvailable :: IO () +testRslvForwardedAvailable = + withProxyAndResolver (status200, availableBody) $ + forwardedResolveAlice >>= \r -> case r of + Right (Right NameResponse {registration = NRAvailable {pricing}}) -> pricing `shouldBe` testPricing + _ -> expectationFailure $ "expected Right (Right NRAvailable), got: " <> show r + +-- keccak-256("alice"), the registry key +aliceHash :: Text +aliceHash = "[9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501]" + +-- | The paths the client asked the resolver for. +resolvePaths :: IORef [[Text]] -> IO [[Text]] +resolvePaths reqs = filter isResolve <$> readIORef reqs + where + isResolve = \case ("v2" : "resolve" : _) -> True; _ -> False + +currentClient :: IO SMPClient +currentClient = do + g <- C.newRandom + ts <- getCurrentTime + let srv = SMPServer testHost testPort testKeyHash + pcE <- getProtocolClient g NRMInteractive (1, srv, Nothing) defaultSMPClientConfig [] Nothing ts (\_ -> pure ()) + either (fail . show) pure pcE + +testRslvSendsTheHash :: IO () +testRslvSendsTheHash = + withResolverServerReqs (status200, registeredBody testNameRecord) $ \reqs -> do + pc <- currentClient + r <- runExceptT' (directResolveName pc NRMInteractive (domain "alice.simplex")) + resolvePaths reqs `shouldReturn` [["v2", "resolve", aliceHash <> ".simplex"]] + -- the client never sent the name, and the record still names it + case r of + NameResponse {registration = NRRegistered {nameRecord}} -> SMP.nrName nameRecord `shouldBe` "alice.simplex" + _ -> expectationFailure $ "expected NRRegistered, got: " <> show r + +testSubnameKeepsItsLabels :: IO () +testSubnameKeepsItsLabels = + withResolverServerReqs (status200, availableBody) $ \reqs -> do + pc <- currentClient + _ <- runExceptT' (directResolveName pc NRMInteractive (domain "x.alice.simplex")) + resolvePaths reqs `shouldReturn` [["v2", "resolve", "x.alice.simplex"]] + +-- a hashed query does not tell the router the name, so the record's own name is +-- checked against the one that was asked for +testRslvWrongName :: IO () +testRslvWrongName = + withResolverServer (status200, registeredBody testNameRecord {SMP.nrName = "mallory.simplex"}) $ do + pc <- currentClient + r <- runExceptT (directResolveName pc NRMInteractive (domain "alice.simplex")) + case r of + Left (PCEUnexpectedResponse _) -> pure () + _ -> expectationFailure $ "expected Left (PCEUnexpectedResponse ..), got: " <> show r runExceptT' :: Show e => ExceptT e IO a -> IO a runExceptT' a = runExceptT a >>= either (fail . show) pure diff --git a/tests/SMPNamesTests.hs b/tests/SMPNamesTests.hs index 0101a40a7..7f28a0b34 100644 --- a/tests/SMPNamesTests.hs +++ b/tests/SMPNamesTests.hs @@ -3,7 +3,7 @@ {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ScopedTypeVariables #-} -module SMPNamesTests (smpNamesTests, testNameRecord) where +module SMPNamesTests (smpNamesTests, testNameRecord, testPricing, registeredBody, availableBody, reservedBody, responseBody, resolved) where import qualified Data.Aeson as J import qualified Data.ByteString.Char8 as B @@ -11,13 +11,14 @@ import qualified Data.ByteString.Lazy as LB import Data.Either (isLeft, isRight) import Data.IORef (readIORef) import Data.List (sort) +import qualified Data.Map.Strict as M import qualified Data.Text as T import Data.Text.Encoding (encodeUtf8) import Network.HTTP.Types (status200, status400, status404, status500, status502) import NamesResolverServer (resolveResp, testNamesConfig, withResolverServer, withResolverServerDelayed) import Simplex.Messaging.Encoding (smpDecode, smpEncode) import Simplex.Messaging.Encoding.String (strDecode) -import Simplex.Messaging.Protocol (ErrorType (..), NameErrorType (..), NameRecord (..)) +import Simplex.Messaging.Protocol (Command (..), ErrorType (..), NameErrorType (..), NamePricing (..), NameQuery (..), NameRecord (..), NameRegistration (..), NameResponse (..), NameReservedReason (..), ProtocolEncoding (..), USDCents (..)) import Simplex.Messaging.Server.Main (validateUrl) import Simplex.Messaging.Server.Names ( NamesConfig (..), @@ -27,7 +28,9 @@ import Simplex.Messaging.Server.Names resolveName, ) import Simplex.Messaging.Server.Names.HttpResolver (ResolverError (..)) -import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..)) +import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..), labelHash) +import Simplex.Messaging.SystemTime (RoundedSystemTime (..)) +import Simplex.Messaging.Transport (nameAvailSMPVersion, serverInfoSMPVersion) import Test.Hspec testNameRecord :: NameRecord @@ -47,12 +50,41 @@ testNameRecord = nrResolver = "0x0202020202020202020202020202020202020202" } +-- | What the resolver serves on /v2/resolve. Spelled out rather than encoded +-- from the Haskell value: the literal JSON is the contract with the resolver. +registeredBody :: NameRecord -> LB.ByteString +registeredBody nameRec = + responseBody $ "{\"type\":\"registered\",\"expires\":1813853483,\"graceUntil\":1821629483,\"reservedReason_\":null,\"nameRecord\":" <> J.encode nameRec <> "}" + +availableBody :: LB.ByteString +availableBody = responseBody "{\"type\":\"available\",\"pricing\":{\"registrationPrices\":{\"3\":12793,\"4\":3198},\"basePrice\":100,\"minLabelLength\":3}}" + +reservedBody :: LB.ByteString +reservedBody = responseBody "{\"type\":\"reserved\",\"reservedReason\":\"trademark\"}" + +-- | The registration, and the block the resolver read it at. +responseBody :: LB.ByteString -> LB.ByteString +responseBody reg = "{\"lastBlockTs\":" <> testReadAt <> ",\"registration\":" <> reg <> "}" + +testReadAt :: LB.ByteString +testReadAt = "1813000000" + +resolved :: NameRegistration -> NameResponse +resolved registration = NameResponse {lastBlockTs = Just (RoundedSystemTime 1813000000), registration} + +-- | What `registeredBody testNameRecord` resolves to. +registeredAlice :: NameRegistration +registeredAlice = + NRRegistered {expires = Just (RoundedSystemTime 1813853483), graceUntil = Just (RoundedSystemTime 1821629483), reservedReason_ = Nothing, nameRecord = testNameRecord} + smpNamesTests :: Spec smpNamesTests = do describe "NameRecord JSON (Protocol)" nameRecordEncodingSpec describe "ErrorType NAME wire encoding" errorWireSpec + describe "RSLV wire encoding" rslvWireSpec describe "Name parsing (SimplexDomain)" parseNameSpec describe "HTTP resolver" resolverSpec + describe "name availability" availabilitySpec describe "Resolver health probe" healthSpec describe "resolver_endpoint validation" validateUrlSpec @@ -101,8 +133,87 @@ errorWireSpec = -- RESOLVER detail may contain spaces - must survive the round-trip smpDecode (smpEncode (NAME (RESOLVER "HTTP 502"))) `shouldBe` Right (NAME (RESOLVER "HTTP 502")) +-- the query format changed at v22, so an older session must still get the name +rslvWireSpec :: Spec +rslvWireSpec = do + it "below v22 carries the name, as it did before" $ + encodeProtocol v20 (RSLV (NQDomain aliceDomain')) `shouldBe` "RSLV " <> smpEncode aliceDomain' + -- keccak-256("alice"), the same constant the resolver's own tests use + it "from v22 carries the 2LD as its hash" $ + encodeProtocol v22 (RSLV (NQDomain aliceDomain')) + `shouldBe` "RSLV [9c0257114eb9399a2985f8e75dad7600c5d89fe3824ffa99ec1c3eb8bf3b0501].simplex" + -- the hashed query has no room for subname labels, so such a name goes as text + it "a name with subnames is not hashed" $ + encodeProtocol v22 (RSLV (NQDomain aliceDomain' {subDomain = ["x"]})) `shouldBe` "RSLV x.alice.simplex" + it "leaves a web name alone: no registry, nothing to key on" $ + encodeProtocol v22 (RSLV (NQDomain webDomain')) `shouldBe` "RSLV example.com" + where + v20 = serverInfoSMPVersion + v22 = nameAvailSMPVersion + aliceDomain' = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + webDomain' = SimplexDomain {nameTLD = TLDWeb, domain = "example.com", subDomain = []} + +availabilitySpec :: Spec +availabilitySpec = do + -- one lookup answers what the name points to, whether it can be taken, and + -- whether it is held back + it "a registered name answers with its record and dates" $ + answers (registeredBody testNameRecord) (resolved registeredAlice) + it "a registered name can be held back too" $ + answers heldBackBody . resolved $ + NRRegistered {expires = Just (RoundedSystemTime 1813853483), graceUntil = Just (RoundedSystemTime 1821629483), reservedReason_ = Just NRRInternal, nameRecord = testNameRecord} + it "an unregistered name answers with the price" $ + answers availableBody (resolved NRAvailable {pricing = testPricing}) + it "reserved carries the reason and no price" $ + answers reservedBody (resolved $ NRReserved NRRTrademark) + -- losing the reservation would offer a name that cannot be registered + it "a reason from a later version still reserves the name" $ + answers (responseBody "{\"type\":\"reserved\",\"reservedReason\":\"seasonal\"}") (resolved $ NRReserved (NRRUnknown "seasonal")) + -- RNAME carries the registration as JSON, so that is the encoding to hold + it "every registration survives the wire" $ + mapM_ + (\a -> J.eitherDecodeStrict (LB.toStrict (J.encode a)) `shouldBe` Right a) + [ registeredAlice, + NRRegistered {expires = Nothing, graceUntil = Nothing, reservedReason_ = Just NRRInternal, nameRecord = testNameRecord}, + NRAvailable {pricing = testPricing}, + NRReserved NRRInternal, + NRReserved NRRTrademark, + NRReserved NRRCommunity, + NRReserved (NRRUnknown "seasonal") + ] + -- one vocabulary: the same word from the resolver and in JSON + it "a reason reads the same in JSON as from the resolver" $ do + J.encode (NRRUnknown "seasonal") `shouldBe` "\"seasonal\"" + J.encode NRRTrademark `shouldBe` "\"trademark\"" + where + heldBackBody = + responseBody $ "{\"type\":\"registered\",\"expires\":1813853483,\"graceUntil\":1821629483,\"reservedReason_\":\"internal\",\"nameRecord\":" <> J.encode testNameRecord <> "}" + answers body a = + withResolverServer (resolveResp status200 body) $ \port _ -> do + env <- newNamesEnv (testNamesConfig port) + resolveName env aliceQuery `shouldReturn` Right a + aliceQuery = NQDomain SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + +-- | The .testing oracle: US cents per year by label length. +testPricing :: NamePricing +testPricing = + NamePricing + { registrationPrices = M.fromList [(3, USDCents 12793), (4, USDCents 3198)], + basePrice = USDCents 100, + minLabelLength = 3 + } + parseNameSpec :: Spec parseNameSpec = do + -- the hashed form is a query, not a name: it has its own type + it "a name is never a hash" $ + parseN ("[" <> T.replicate 64 "b" <> "].simplex") `shouldSatisfy` isLeft + it "a query survives the wire" $ + mapM_ + (\q -> smpDecode (smpEncode q) `shouldBe` Right q) + [ NQDomain d, + NQHash (labelHash "alice") TLDSimplex + ] it "accepts a valid simplex-TLD name" $ case parseN "privacy.simplex" of Right d -> do @@ -138,23 +249,26 @@ parseNameSpec = do where parseN :: T.Text -> Either String SimplexDomain parseN = strDecode . encodeUtf8 + d = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = ["x"]} resolverSpec :: Spec resolverSpec = do - it "returns NameRecord on 200 OK" $ - withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port _ -> do + it "returns the registration on 200 OK" $ + withResolverServer (resolveResp status200 (registeredBody testNameRecord)) $ \port _ -> do env <- newNamesEnv (testNamesConfig port) - resolveName env aliceDomain `shouldReturn` Right testNameRecord + resolveName env aliceDomain `shouldReturn` Right (resolved registeredAlice) - it "returns NOT_FOUND on 404" $ + -- /v2/resolve answers 200, 400 or 502 and never says "no such name": an + -- unregistered name is NRAvailable. So no status maps to NOT_FOUND. + it "returns RESOLVER on 404 (a resolver without the v2 route)" $ withResolverServer (resolveResp status404 "{}") $ \port _ -> do env <- newNamesEnv (testNamesConfig port) - resolveName env aliceDomain `shouldReturn` Left NOT_FOUND + resolveName env aliceDomain `shouldReturn` Left (RESOLVER "HTTP 404") - it "returns NOT_FOUND on 400 (unknown TLD)" $ + it "returns RESOLVER on 400 (TLD not configured)" $ withResolverServer (resolveResp status400 "{}") $ \port _ -> do env <- newNamesEnv (testNamesConfig port) - resolveName env aliceDomain `shouldReturn` Left NOT_FOUND + resolveName env aliceDomain `shouldReturn` Left (RESOLVER "HTTP 400") it "returns RESOLVER on 502 (upstream failure)" $ withResolverServer (resolveResp status502 "{}") $ \port _ -> do @@ -171,31 +285,31 @@ resolverSpec = do env <- newNamesEnv (testNamesConfig port) resolveName env aliceDomain `shouldReturn` Left (RESOLVER "invalid response") - it "returns RESOLVER when JSON parses but isn't a NameRecord shape" $ + it "returns RESOLVER when JSON parses but isn't a NameRegistration shape" $ withResolverServer (resolveResp status200 "{}") $ \port _ -> do env <- newNamesEnv (testNamesConfig port) resolveName env aliceDomain `shouldReturn` Left (RESOLVER "invalid response") it "returns RESOLVER (timeout) when the resolver is slower than resolverTimeoutMs" $ - withResolverServerDelayed 1500 (resolveResp status200 (J.encode testNameRecord)) $ \port _ -> do + withResolverServerDelayed 1500 (resolveResp status200 (registeredBody testNameRecord)) $ \port _ -> do env <- newNamesEnv (testNamesConfig port) {resolverTimeoutMs = 300} resolveName env aliceDomain `shouldReturn` Left (RESOLVER "timeout") it "sends one HTTP request per lookup (no cache)" $ - withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port reqs -> do + withResolverServer (resolveResp status200 (registeredBody testNameRecord)) $ \port reqs -> do env <- newNamesEnv (testNamesConfig port) _ <- resolveName env aliceDomain _ <- resolveName env aliceDomain readIORef reqs >>= \rs -> length rs `shouldBe` 2 it "addresses the resolver with the full canonical domain name" $ - withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port reqs -> do + withResolverServer (resolveResp status200 (registeredBody testNameRecord)) $ \port reqs -> do env <- newNamesEnv (testNamesConfig port) _ <- resolveName env aliceDomain - readIORef reqs `shouldReturn` [["resolve", "alice.simplex"]] + readIORef reqs `shouldReturn` [["v2", "resolve", "alice.simplex"]] where - aliceDomain = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} + aliceDomain = NQDomain SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []} healthSpec :: Spec healthSpec = do