diff --git a/apps/simplex-store-service/README.md b/apps/simplex-store-service/README.md index 28761f0f3e..2390cb10e3 100644 --- a/apps/simplex-store-service/README.md +++ b/apps/simplex-store-service/README.md @@ -701,7 +701,8 @@ With `serve_webapp = on` (all in one), the service listens on both ports, and ea own site: every path outside `/api/` and `/webhooks/` is a file of that site, `/` and `/index.html` its shell. Both ports answer `/api/` and `/webhooks/`, whatever the `Host`. A reverse proxy sends each domain to its own port, as the all-in-one example in [Caddy](#caddy) does. With -`serve_webapp = off`, the service listens on `badges_port` alone, for the API and the webhooks. +`serve_webapp = off`, the service listens on `badges_port` alone, for the API and the webhooks. With +`metrics_port`, it also listens on `127.0.0.1` on that port, for `/metrics` alone. ### Development with a local chain @@ -863,6 +864,106 @@ simplex.domains { The service sets each file's type and `Cache-Control` itself: `no-cache`, and a year for `/assets/*`, so the names site's fonts and images are revalidated rather than kept for a day. +### Metrics + +`[listener] metrics_port` turns on the store's Prometheus metrics. It is off when the key is +absent and must differ from `badges_port` and `names_port`. The service answers `GET /metrics` on +`127.0.0.1:`, whatever `host` is, so only the machine itself reaches it: + +```ini +[listener] +metrics_port = 9300 +``` + +Each scrape counts invoices, badges and, with `[names]`, name orders, transactions, balances and +record edits in the database. With `[names]`, the registrar key's ETH balance, its sponsor +allowance, the node's latest block and the terms links are read from the node at start and then +every minute, and a scrape answers the last read, so a slow or failing node never delays a scrape. +A part that fails keeps its last value, sets `simplex_store_names_chain_read_ok` to 0 and logs a +warning. A port that cannot be listened on is logged as an error and leaves the rest of the +service running. + +| Metric | Value | +|---|---| +| `simplex_store_invoices_open{provider}` | open invoices of badges and names, by provider: `crypto` (BTCPay) or `stripe`; 0 for a configured provider with none | +| `simplex_store_invoices_overdue{provider}` | those of them past their expiry and its 10-minute grace with no money received; the poller expires each once the provider answers, so a count that stays up means the provider cannot be read or the invoice cancelled there | +| `simplex_store_provider_list_ok_timestamp_seconds{provider}` | when the poller last listed the provider's open invoices successfully, every one or two minutes at the default `idle_seconds`; absent until the first list | +| `simplex_store_provider_webhook_timestamp_seconds{provider}` | when the provider last delivered a webhook that verified; absent until the first one | +| `simplex_store_badges_purchases{status}` | badge purchases by status: `acquiring`, `issued`, `superseded` or `failed`, each 0 when it has none | +| `simplex_store_badges_subscriptions{state}` | App Store and Google Play subscriptions paying for badges, by state: `renewing`, `cancelled`, `billingRetry` or `ended`, each 0 when it has none | +| `simplex_store_badges_codes_open` | badge codes paid or free, not revoked or expired, with redemptions left | +| `simplex_store_names_orders{phase}` | name orders in the phase: `unpaid`, `queued`, `committing`, `committed`, `registering`, `registered`, `taken` or `failed`; the queue is `queued` to `registering`, while `unpaid` also counts closed orders kept for 30 days and `failed` every failed order not tried again | +| `simplex_store_names_queue_oldest_change_timestamp_seconds` | the last change, in Unix seconds, of the order unchanged longest among those the service moves on: `queued`, `committing`, `registering`, and `committed` with the page's reveal stored; absent when there is none | +| `simplex_store_names_transactions_pending` | registrar transactions sent and not settled, one per nonce, so a fee replacement counts once | +| `simplex_store_names_transactions_oldest_pending_timestamp_seconds` | when the oldest of them was first signed; absent when none is pending | +| `simplex_store_names_invoices_open` | open invoices of name orders | +| `simplex_store_names_balances_held` | balances reserved by an order paying the difference, until that order is paid or closed | +| `simplex_store_names_balances_held_cents` | the cents of those balances | +| `simplex_store_names_record_edits_pending` | record edits relayed and not settled | +| `simplex_store_names_record_edits_oldest_pending_timestamp_seconds` | when the oldest of them was made; absent when none is pending | +| `simplex_store_names_registrar_balance_wei` | the registrar key's ETH balance, as last read | +| `simplex_store_names_sponsor_allowance_cents` | its allowance left on the sponsor, in cents, rounded down, as last read | +| `simplex_store_names_chain_head_block` | the node's latest block number, as last read | +| `simplex_store_names_chain_head_timestamp_seconds` | that block's time | +| `simplex_store_names_chain_base_fee_wei` | that block's base fee per gas | +| `simplex_store_names_terms_held{kind}` | 1 when the service holds the text of the `general` or `community` terms the name policy links, 0 while it cannot fetch it, so names of that kind cannot be ordered; absent while the policy links none | +| `simplex_store_names_chain_read_timestamp_seconds` | when every part of the read last succeeded | +| `simplex_store_names_chain_read_ok` | 1 when the last read succeeded in every part, 0 when a part failed or none has finished | + +With `[names]`, the service also counts every name on chain, however it was registered, at start and +then every 5 minutes, under the metric names of the former `simplex-names-exporter`. At the latest +block it reads the base registrar's names (`totalSupply`, `tokenByIndex`), keeps those whose +`nameExpires` is after the block's time, takes each label from `labelOf`, then walks the subname +registrar (`subnameHook()`) level by level with `getChildren`, keeping subnames whose `ownerOf` is +not zero. For each name it reads `resolver(node)` on the registry and, on the configured resolver, +its `simplex.contact` and `simplex.channel` texts. The reads run 16 at a time at that one block. A +failed census keeps the last one and is counted. + +| Metric | Value | +|---|---| +| `simplex_names{level,records}` | live names at `level` 2 (`alice.simplex`), 3 (`shop.alice.simplex`) and below, by `records`: `none`, `contact`, `channel`, `both`, or `other_resolver` for a name whose resolver is not the configured one, whose records are not read; names in the 90-day grace period are not counted | +| `simplex_name_length` | histogram of label lengths in characters, by `level`, a bucket for each length 1 to 63 | +| `simplex_names_block` | the block the census read | +| `simplex_names_refresh_timestamp_seconds` | when the census last succeeded | +| `simplex_names_refresh_failures_total` | counter of failed censuses since the service started | + +Every metric is a gauge, but for the census's histogram and counter. A stuck order, transaction or record edit, a node that cannot be read or +has stopped following the chain, terms that cannot be shown and a payment provider that cannot be +read show as: + +``` +time() - simplex_store_names_queue_oldest_change_timestamp_seconds > 1200 +time() - simplex_store_names_transactions_oldest_pending_timestamp_seconds > 900 +time() - simplex_store_names_record_edits_oldest_pending_timestamp_seconds > 900 +simplex_store_names_chain_read_ok == 0 +time() - simplex_store_names_chain_head_timestamp_seconds > 120 +simplex_store_names_terms_held == 0 +time() - simplex_store_provider_list_ok_timestamp_seconds > 600 +simplex_store_invoices_overdue > 0 +``` + +The overdue count can be above 0 for a few seconds between an invoice's grace ending and the next +poller pass, so its alert waits, as with `for: 10m`. Mainnet makes a block every 12 seconds, so a +head older than 2 minutes means the node has fallen behind. + +An order moves on within minutes: its commit and register are each mined and confirmed, and the +register waits about a minute after the commit. An order tried again can wait unchanged up to +`reveal_margin_seconds` (900 by default) for its old commitment to lapse, so the queue alert's +threshold is above it. Size the alerts on the balance and the allowance to +the orders expected before a top-up, with the figures in [Fund the key](#fund-the-key). + +Caddy proxies it on the monitoring site of the host, beside node_exporter, inside that site's +`handle_path /metrics/*` block: + +``` +handle_path /store_service* { + rewrite * /metrics + reverse_proxy http://127.0.0.1:9300 +} +``` + +Prometheus then scrapes `/metrics/store_service` on that site. + ### Terms and community names The controller's name policy (`SimplexNamePolicy`, read from `namePolicy()` once at startup and @@ -1234,6 +1335,7 @@ psql "postgresql://simplex@127.0.0.1:5432/badge_service" -c "SELECT registrar_no #### When funds or the allowance run out +With `metrics_port` set, both are exported before they run out (see [Metrics](#metrics)). `docker compose logs store-service` shows: - `[ERROR …] names sender: nonce : not sent: …`, with the node's `insufficient funds` message, diff --git a/apps/simplex-store-service/docs/names.md b/apps/simplex-store-service/docs/names.md index 4be104d87b..11a52eebe9 100644 --- a/apps/simplex-store-service/docs/names.md +++ b/apps/simplex-store-service/docs/names.md @@ -93,7 +93,7 @@ How a request flows: | Part | Role | Source | |---|---|---| | Startup | `store_service.ini` is parsed, the database lock taken, and the node's chain id and the issuer key checked; then the chat core is started. In its callback the providers, the names environment and the web environment are built, the sites exported, the managed group resolved, and all lanes started together. Without `--service-config` only the chat side and the badge request lanes run. | [`Main.hs:15-28`](../Main.hs#L15-L28), [`Service.hs:280-370`](../src/StoreService/Service.hs#L280-L370) | -| Web API | Warp listeners on `[listener] host`: `badges_port` (8080) always, `names_port` (8081) with `[names]` and `serve_webapp = on`. `/api/*` and both webhooks are answered on every port. Per-client limits a minute: read 60, search 30, preview 20, code 10, records 10, create 5, find 5. Checkouts that reach a provider: 5 a minute per IPv4 /24 or IPv6 /48, and 60 card and 30 BTCPay a minute for all clients together. At most 500 long polls are held at once, each up to 30 s. | [`Web/Server.hs:121-151`](../src/StoreService/Web/Server.hs#L121-L151), [`Web/Server.hs:563-586`](../src/StoreService/Web/Server.hs#L563-L586), [`Config.hs:511-520`](../src/StoreService/Config.hs#L511-L520), [`Web/Names.hs:28-51`](../src/StoreService/Web/Names.hs#L28-L51), [`Web/Http.hs:111-161`](../src/StoreService/Web/Http.hs#L111-L161), [`Web/Http.hs:227-230`](../src/StoreService/Web/Http.hs#L227-L230) | +| Web API | Warp listeners on `[listener] host`: `badges_port` (8080) always, `names_port` (8081) with `[names]` and `serve_webapp = on`; with `metrics_port`, a listener on `127.0.0.1` for `/metrics` alone. `/api/*` and both webhooks are answered on every port but the metrics one. Per-client limits a minute: read 60, search 30, preview 20, code 10, records 10, create 5, find 5. Checkouts that reach a provider: 5 a minute per IPv4 /24 or IPv6 /48, and 60 card and 30 BTCPay a minute for all clients together. At most 500 long polls are held at once, each up to 30 s. | [`Web/Server.hs:121-151`](../src/StoreService/Web/Server.hs#L121-L151), [`Web/Server.hs:563-586`](../src/StoreService/Web/Server.hs#L563-L586), [`Config.hs:511-520`](../src/StoreService/Config.hs#L511-L520), [`Web/Names.hs:28-51`](../src/StoreService/Web/Names.hs#L28-L51), [`Web/Http.hs:111-161`](../src/StoreService/Web/Http.hs#L111-L161), [`Web/Http.hs:227-230`](../src/StoreService/Web/Http.hs#L227-L230) | | Order handlers | An order is placed in one of three ways: a paid invoice, a prepaid code, a spend of another order's balance. The owner's signature, the timestamp, the duration, the terms and an available name are checked, then the chain price for a paid or spent order and the balance for a spend. The order is written in one transaction, in which the same wallet's overdue orders for that name are closed. | [`Web/Names/Orders.hs:86-306`](../src/StoreService/Web/Names/Orders.hs#L86-L306), [`Names/Orders.hs:201-310`](../src/StoreService/Names/Orders.hs#L201-L310), [`Names/Balances.hs:79-106`](../src/StoreService/Names/Balances.hs#L79-L106) | | Registration and owner handlers | The registration view, held on its version; the reveal; Try again; find by the owner's signature; a move to another name. | [`Web/Names/Registration.hs:45-140`](../src/StoreService/Web/Names/Registration.hs#L45-L140), [`Web/Names/Owner.hs:44-152`](../src/StoreService/Web/Names/Owner.hs#L44-L152), [`Names/Reveal.hs:103-145`](../src/StoreService/Names/Reveal.hs#L103-L145), [`Names/View.hs:52-112`](../src/StoreService/Names/View.hs#L52-L112) | | Search handlers | Name status, price, terms, and records with link profiles; the terms text; the link preview. These handlers only read, except that a terms text fetched for the first time is stored. | [`Web/Names/Search.hs:53-154`](../src/StoreService/Web/Names/Search.hs#L53-L154), [`Names/Profiles.hs:236-274`](../src/StoreService/Names/Profiles.hs#L236-L274) | diff --git a/apps/simplex-store-service/src/StoreService/Config.hs b/apps/simplex-store-service/src/StoreService/Config.hs index d8b9a36415..1b10f2dd6c 100644 --- a/apps/simplex-store-service/src/StoreService/Config.hs +++ b/apps/simplex-store-service/src/StoreService/Config.hs @@ -35,7 +35,7 @@ where import qualified Control.Exception as E import Control.Applicative (many, (<|>)) import Control.Logger.Simple (LogLevel (..), logNote, logWarn) -import Control.Monad (forM_, mfilter, unless, when) +import Control.Monad (forM, forM_, mfilter, unless, when) import Data.Attoparsec.Combinator (lookAhead) import Data.Attoparsec.Text (Parser, endOfInput, isEndOfLine, parseOnly, satisfy, skipMany, skipSpace, skipWhile) import Data.Bifunctor (first) @@ -80,7 +80,8 @@ data ListenerConfig = ListenerConfig lNamesStaticDir :: Maybe FilePath, lNamesExportDir :: Maybe FilePath, -- | @Nothing@ unless the names site is served, which takes @[names]@ and @serve_webapp@ on. - lNamesPort :: Maybe Int + lNamesPort :: Maybe Int, + lMetricsPort :: Maybe Int } deriving (Eq, Show) @@ -425,7 +426,7 @@ trailingNoise = skipSpace *> skipMany (comment *> skipSpace) knownSettings :: [(Text, [Text])] knownSettings = -- The retired keys are known, so parseConfig refuses them rather than a warning calling them ignored. - [ ("listener", ["host", "badges_port", "badges_static_dir", "serve_webapp", "badges_export_dir", "trust_forwarded_for"] <> namesListenerKeys <> map fst retiredListenerKeys), + [ ("listener", ["host", "badges_port", "badges_static_dir", "serve_webapp", "badges_export_dir", "trust_forwarded_for", "metrics_port"] <> namesListenerKeys <> map fst retiredListenerKeys), ("btcpay", ["host", "api_key", "store_id", "webhook_secret", "expiry_minutes", "speed_policy", "payment_tolerance"]), ("stripe", ["secret_key", "publishable_key", "webhook_secret", "session_minutes"]), ("poll", ["waiting_seconds", "idle_seconds"]), @@ -471,6 +472,9 @@ parseConfig ini = do let lBadgesExportDir = T.unpack <$> present "listener" "badges_export_dir" lTrustForwardedFor <- bool "listener" "trust_forwarded_for" False (lNamesStaticDir, lNamesExportDir, lNamesPort) <- namesSite lServeWebapp lBadgesPort + lMetricsPort <- optionalPort "metrics_port" + when (any (\p -> p == lBadgesPort || Just p == lNamesPort) lMetricsPort) $ + Left "listener.metrics_port must differ from listener.badges_port and listener.names_port" btc <- btcpaySection str <- stripeSection iss <- issuerSection @@ -510,7 +514,7 @@ parseConfig ini = do Left "[dev] check_store_during_coverage needs accept_unverified_store_receipts or credit_test_purchases = on, as only a dev deployment may ask a store while coverage runs" pure ServiceConfig - { listener = ListenerConfig {lHost, lBadgesPort, lBadgesStaticDir, lServeWebapp, lBadgesExportDir, lTrustForwardedFor, lNamesStaticDir, lNamesExportDir, lNamesPort}, + { listener = ListenerConfig {lHost, lBadgesPort, lBadgesStaticDir, lServeWebapp, lBadgesExportDir, lTrustForwardedFor, lNamesStaticDir, lNamesExportDir, lNamesPort, lMetricsPort}, btcpay = btc, stripe = str, poll = PollConfig {pWaitingSeconds, pIdleSeconds}, @@ -531,9 +535,9 @@ parseConfig ini = do present s k = mfilter (not . T.null) (T.strip <$> look s k) required s k = maybe (Left (T.unpack s <> "." <> T.unpack k <> " is required")) Right (present s k) optional s k d = Right (fromMaybe d (present s k)) - listenerPort k d = do - p <- num "listener" k d - if 1 <= p && p <= 65535 then Right p else Left ("listener." <> T.unpack k <> " must be between 1 and 65535") + listenerPort k d = fromMaybe d <$> optionalPort k + optionalPort k = + optionalNum "listener" k >>= mapM (\p -> if 1 <= p && p <= 65535 then Right p else Left ("listener." <> T.unpack k <> " must be between 1 and 65535")) unusedListenerKey k why = Left ("listener." <> T.unpack k <> " is set but " <> why <> ", so nothing would use it") namesSite serve badgesPort | not (hasSection "names") = case filter (isJust . look "listener") namesListenerKeys of @@ -545,12 +549,11 @@ parseConfig ini = do when (namesPort == Just badgesPort) $ Left "listener.names_port must differ from listener.badges_port" let staticDir = maybe defaultNamesStaticDir T.unpack (present "listener" "names_static_dir") Right (Just staticDir, T.unpack <$> present "listener" "names_export_dir", namesPort) + num s k d = fromMaybe d <$> optionalNum s k -- Integer, because readMaybe at Int wraps silently, reading 2^64+4 as 4. - num s k d = case look s k of - Nothing -> Right d - Just v -> case readMaybe (T.unpack (T.strip v)) of - Just n | n >= toInteger (minBound :: Int), n <= toInteger (maxBound :: Int) -> Right (fromInteger n) - _ -> Left (T.unpack k <> " must be a whole number") + optionalNum s k = forM (look s k) $ \v -> case readMaybe (T.unpack (T.strip v)) of + Just n | n >= toInteger (minBound :: Int), n <= toInteger (maxBound :: Int) -> Right (fromInteger n) + _ -> Left (T.unpack k <> " must be a whole number") bool s k d = case fmap (T.toLower . T.strip) (look s k) of Nothing -> Right d Just "on" -> Right True diff --git a/apps/simplex-store-service/src/StoreService/Metrics.hs b/apps/simplex-store-service/src/StoreService/Metrics.hs new file mode 100644 index 0000000000..777f073e59 --- /dev/null +++ b/apps/simplex-store-service/src/StoreService/Metrics.hs @@ -0,0 +1,267 @@ +{-# LANGUAGE NamedFieldPuns #-} +{-# LANGUAGE OverloadedStrings #-} + +module StoreService.Metrics + ( MetricsEnv (..), + NamesSource (..), + ChainRead (..), + ProviderTimes (..), + CensusRead (..), + noChainRead, + noCensusRead, + readChain, + runChainReads, + readCensus, + serviceText, + namesText, + censusText, + metricsApp, + runMetrics, + ) +where + +import Control.Concurrent (threadDelay) +import Control.Concurrent.STM +import Control.Logger.Simple (logWarn) +import Control.Monad (forever, unless, void) +import qualified Data.ByteString.Lazy as LB +import Data.Either (isRight) +import Data.List (nub, sort) +import Data.Map.Strict (Map) +import qualified Data.Map.Strict as M +import Data.Maybe (fromMaybe) +import Data.String (fromString) +import Data.Text (Text) +import qualified Data.Text as T +import Data.Text.Encoding (encodeUtf8) +import Data.Time.Clock (UTCTime, addUTCTime, getCurrentTime) +import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds) +import Network.HTTP.Types (hContentType, status200) +import Network.Wai (Application, pathInfo, responseLBS) +import qualified Network.Wai.Handler.Warp as Warp +import Simplex.Chat.Badges.Code (maxNameLength) +import Simplex.Chat.Badges.Types (BadgePurchaseStatus (..), SubscriptionState (..)) +import Simplex.Messaging.Agent.Store.Common (DBStore, withTransaction) +import Simplex.Messaging.Encoding.String (textEncode) +import Simplex.Messaging.Util (raceAny_, tshow) +import StoreService.Config (ListenerConfig) +import StoreService.Names.Census (Census (..), CensusName (..), nameCensus, recordsLabel) +import StoreService.Names.Chain (Block (..), ChainClient, latestBlock) +import StoreService.Names.Contracts (Contracts, RegistrarFunds (..), registrarFunds, termsHeld) +import StoreService.Names.Store (NamesCounts (..), namesCounts) +import StoreService.Poller (expiryGrace) +import StoreService.Safely (safely) +import StoreService.Store (BadgeCounts (..), badgeCounts) +import StoreService.Store.Invoices (openInvoiceCounts) +import StoreService.Web.Http (notFound, only) +import StoreService.Web.Server (webSettings) + +data MetricsEnv = MetricsEnv + { meStore :: DBStore, + meProviders :: [Text], + meListedOk :: TVar (Map Text UTCTime), + meWebhookAt :: TVar (Map Text UTCTime), + -- | @Nothing@ when the ini has no @[names]@ section. + meNames :: Maybe NamesSource + } + +data NamesSource = NamesSource {nsChain :: ChainClient, nsContracts :: Contracts} + +-- | Each part keeps its last value while its read fails. +data ChainRead = ChainRead + { crFunds :: Maybe RegistrarFunds, + crHead :: Maybe Block, + crTerms :: [(Text, Bool)], + -- | the last read in which every part succeeded + crReadAt :: Maybe UTCTime, + crOk :: Bool + } + +noChainRead :: ChainRead +noChainRead = ChainRead {crFunds = Nothing, crHead = Nothing, crTerms = [], crReadAt = Nothing, crOk = False} + +-- | By each provider's text encoding. +data ProviderTimes = ProviderTimes {ptListedOk :: Map Text UTCTime, ptWebhookAt :: Map Text UTCTime} + +chainReadInterval :: Int +chainReadInterval = 60 * 1000000 + +metricsHost :: String +metricsHost = "127.0.0.1" + +readChain :: NamesSource -> TVar ChainRead -> IO () +readChain NamesSource {nsChain, nsContracts} v = do + funds <- registrarFunds nsContracts + head' <- latestBlock nsChain + terms <- termsHeld nsContracts + now <- getCurrentTime + let ok = isRight funds && isRight head' && isRight terms + atomically $ modifyTVar' v $ \ChainRead {crFunds, crHead, crTerms, crReadAt} -> + ChainRead + { crFunds = either (const crFunds) Just funds, + crHead = either (const crHead) Just head', + crTerms = either (const crTerms) id terms, + crReadAt = if ok then Just now else crReadAt, + crOk = ok + } + warnIfFailed "the registrar's funds" funds + warnIfFailed "the latest block" head' + warnIfFailed "the terms links" terms + where + warnIfFailed :: Show e => Text -> Either e a -> IO () + warnIfFailed what = either (\e -> logWarn $ "store metrics: " <> what <> " could not be read: " <> tshow e) (const $ pure ()) + +runChainReads :: NamesSource -> TVar ChainRead -> IO () +runChainReads ns v = forever $ do + done <- safely "store metrics: the chain read" (readChain ns v) + unless done $ atomically $ modifyTVar' v $ \cr -> cr {crOk = False} + threadDelay chainReadInterval + +-- | A failed census keeps the last one and is counted. +data CensusRead = CensusRead {crCensus :: Maybe (Census, UTCTime), crFailures :: Int} + +noCensusRead :: CensusRead +noCensusRead = CensusRead {crCensus = Nothing, crFailures = 0} + +censusInterval :: Int +censusInterval = 300 * 1000000 + +readCensus :: NamesSource -> TVar CensusRead -> IO () +readCensus NamesSource {nsChain, nsContracts} v = do + counted <- nameCensus nsChain nsContracts + case counted of + Right census -> do + now <- getCurrentTime + atomically $ modifyTVar' v $ \cr -> cr {crCensus = Just (census, now)} + Left e -> do + atomically $ modifyTVar' v $ \cr -> cr {crFailures = crFailures cr + 1} + logWarn $ "store metrics: the names census failed: " <> tshow e + +runCensus :: NamesSource -> TVar CensusRead -> IO () +runCensus ns v = forever $ do + done <- safely "store metrics: the names census" (readCensus ns v) + unless done $ atomically $ modifyTVar' v $ \cr -> cr {crFailures = crFailures cr + 1} + threadDelay censusInterval + +-- | Open invoices come as (provider, open, overdue). +serviceText :: [Text] -> [(Text, Int, Int)] -> BadgeCounts -> ProviderTimes -> Text +serviceText providers invoices BadgeCounts {bcPurchases, bcSubscriptions, bcOpenCodes} ProviderTimes {ptListedOk, ptWebhookAt} = + T.concat + [ gauge "simplex_store_invoices_open" "Open invoices by provider" [labelled "provider" p (value n) | (p, (n, _)) <- invoiceCounts], + gauge "simplex_store_invoices_overdue" "Open invoices past their expiry and its grace with no money received, which the poller expires once the provider answers" [labelled "provider" p (value n) | (p, (_, n)) <- invoiceCounts], + gauge "simplex_store_provider_list_ok_timestamp_seconds" "When the poller last listed the provider's open invoices successfully" [labelled "provider" p (timestamp t) | (p, t) <- M.toList ptListedOk], + gauge "simplex_store_provider_webhook_timestamp_seconds" "When the provider last delivered a webhook that verified" [labelled "provider" p (timestamp t) | (p, t) <- M.toList ptWebhookAt], + gauge "simplex_store_badges_purchases" "Badge purchases by status" [labelled "status" s (value n) | (s, n) <- withZeros purchaseStatuses bcPurchases], + gauge "simplex_store_badges_subscriptions" "App store subscriptions paying for badges, by state" [labelled "state" s (value n) | (s, n) <- withZeros subscriptionStates bcSubscriptions], + gauge "simplex_store_badges_codes_open" "Badge codes paid or free, not revoked or expired, with redemptions left" (Just $ value bcOpenCodes) + ] + where + invoiceCounts = M.toList $ M.union (M.fromList [(p, (n, overdue)) | (p, n, overdue) <- invoices]) (M.fromList [(p, (0, 0)) | p <- providers]) + +withZeros :: [Text] -> [(Text, Int)] -> [(Text, Int)] +withZeros known counts = M.toList $ M.union (M.fromList counts) (M.fromList [(k, 0) | k <- known]) + +purchaseStatuses :: [Text] +purchaseStatuses = map textEncode [PSAcquiring, PSIssued, PSSuperseded, PSFailed] + +subscriptionStates :: [Text] +subscriptionStates = map textEncode [SSRenewing, SSCancelled, SSBillingRetry, SSEnded] + +namesText :: NamesCounts -> ChainRead -> Text +namesText NamesCounts {ncPhases, ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit} ChainRead {crFunds, crHead, crTerms, crReadAt, crOk} = + T.concat + [ gauge "simplex_store_names_orders" "Name orders by phase" [labelled "phase" (textEncode p) (value $ fromMaybe 0 $ lookup p ncPhases) | p <- [minBound .. maxBound]], + gauge "simplex_store_names_queue_oldest_change_timestamp_seconds" "Last change of the order longest unmoved among those the service moves on: queued, committing, registering, or committed with its reveal" (timestamp <$> ncQueueOldestChange), + gauge "simplex_store_names_transactions_pending" "Registrar transactions sent and not settled, one per nonce" (Just $ value ncPendingTransactions), + gauge "simplex_store_names_transactions_oldest_pending_timestamp_seconds" "When the oldest pending registrar transaction was first signed" (timestamp <$> ncOldestPending), + gauge "simplex_store_names_invoices_open" "Open invoices of name orders" (Just $ value ncOpenInvoices), + gauge "simplex_store_names_balances_held" "Balances reserved by an order paying the difference, until that order is paid or closed" (Just $ value ncHeldBalances), + gauge "simplex_store_names_balances_held_cents" "Cents of the balances orders hold" (Just $ value ncHeldCents), + gauge "simplex_store_names_record_edits_pending" "Record edits relayed and not settled" (Just $ value ncPendingEdits), + gauge "simplex_store_names_record_edits_oldest_pending_timestamp_seconds" "When the oldest pending record edit was made" (timestamp <$> ncOldestEdit), + gauge "simplex_store_names_registrar_balance_wei" "The registrar key's ETH balance, as last read" (value . rfBalanceWei <$> crFunds), + gauge "simplex_store_names_sponsor_allowance_cents" "The registrar key's allowance left on the sponsor, as last read" (value . rfAllowanceCents <$> crFunds), + gauge "simplex_store_names_chain_head_block" "The node's latest block number, as last read" (value . blockNumber <$> crHead), + gauge "simplex_store_names_chain_head_timestamp_seconds" "The node's latest block time, as last read" (value . blockTimestamp <$> crHead), + gauge "simplex_store_names_chain_base_fee_wei" "The node's latest block base fee per gas, as last read" (value . blockBaseFee <$> crHead), + gauge "simplex_store_names_terms_held" "1 when the service holds the text of the terms the name policy links, by kind, as last read" [labelled "kind" k (value $ fromEnum held) | (k, held) <- crTerms], + gauge "simplex_store_names_chain_read_timestamp_seconds" "When every part of the chain read last succeeded" (timestamp <$> crReadAt), + gauge "simplex_store_names_chain_read_ok" "1 when the last chain read succeeded in every part, 0 when a part failed or none was made" (Just $ value $ fromEnum crOk) + ] + +value :: Show a => a -> (Text, Text) +value n = ("", tshow n) + +timestamp :: UTCTime -> (Text, Text) +timestamp t = value (floor $ utcTimeToPOSIXSeconds t :: Integer) + +labelled :: Text -> Text -> (Text, Text) -> (Text, Text) +labelled name v = withLabels [(name, v)] + +withLabels :: [(Text, Text)] -> (Text, Text) -> (Text, Text) +withLabels ls (_, sample) = (labelSet ls, sample) + +labelSet :: [(Text, Text)] -> Text +labelSet ls = "{" <> T.intercalate "," [name <> "=\"" <> escape v <> "\"" | (name, v) <- ls] <> "}" + where + escape = T.replace "\n" "\\n" . T.replace "\"" "\\\"" . T.replace "\\" "\\\\" + +gauge :: Foldable f => Text -> Text -> f (Text, Text) -> Text +gauge = family "gauge" + +family :: Foldable f => Text -> Text -> Text -> f (Text, Text) -> Text +family kind name help samples + | null samples = "" + | otherwise = "# HELP " <> name <> " " <> help <> "\n# TYPE " <> name <> " " <> kind <> "\n" <> foldMap sample samples <> "\n" + where + sample (labels, v) = name <> labels <> " " <> v <> "\n" + +-- | The census of #8114's exporter, under its metric names: names and subnames by level and records set, and their +-- label lengths. Before the first census only the failures are answered. +censusText :: CensusRead -> Text +censusText CensusRead {crCensus, crFailures} = + family "counter" "simplex_names_refresh_failures_total" "Refreshes that failed" (Just $ value crFailures) + <> foldMap counted crCensus + where + counted (Census {cBlock, cNames}, refreshedAt) = + T.concat + [ gauge "simplex_names" "Registered names by level, and by simplex.contact and simplex.channel set" [withLabels [("level", tshow l), ("records", recordsLabel r)] (value $ count l r) | l <- levelsSeen, r <- [minBound .. maxBound]], + lengthHistogram, + gauge "simplex_names_block" "Block the names were read at" (Just $ value cBlock), + gauge "simplex_names_refresh_timestamp_seconds" "Time of the last refresh" (Just $ timestamp refreshedAt) + ] + where + levelsSeen = sort $ nub $ 2 : map cnLevel cNames + count l r = length [() | CensusName {cnLevel, cnRecords} <- cNames, cnLevel == l, cnRecords == r] + lengthHistogram = + "# HELP simplex_name_length Label lengths in characters, by level\n# TYPE simplex_name_length histogram\n" + <> foldMap lengthsAt levelsSeen + <> "\n" + lengthsAt l = + let lengths = [cnLength | CensusName {cnLevel, cnLength} <- cNames, cnLevel == l] + level = [("level", tshow l)] + line suffix ls n = "simplex_name_length" <> suffix <> labelSet ls <> " " <> tshow n <> "\n" + in foldMap (\b -> line "_bucket" (level <> [("le", tshow b <> ".0")]) (length $ filter (<= b) lengths)) [1 .. maxNameLength] + <> line "_bucket" (level <> [("le", "+Inf")]) (length lengths) + <> line "_sum" level (sum lengths) + <> line "_count" level (length lengths) + +metricsApp :: MetricsEnv -> TVar ChainRead -> TVar CensusRead -> Application +metricsApp MetricsEnv {meStore, meProviders, meListedOk, meWebhookAt, meNames} v census req respond = case pathInfo req of + ["metrics"] -> only "GET" req respond $ do + now <- getCurrentTime + (invoices, badges, names) <- withTransaction meStore $ \db -> + (,,) <$> openInvoiceCounts db (addUTCTime (negate expiryGrace) now) <*> badgeCounts db now <*> mapM (const $ namesCounts db) meNames + times <- ProviderTimes <$> readTVarIO meListedOk <*> readTVarIO meWebhookAt + chain <- readTVarIO v + counted <- readTVarIO census + let text = serviceText meProviders invoices badges times <> foldMap (\n -> namesText n chain <> censusText counted) names + respond $ responseLBS status200 [(hContentType, "text/plain; version=0.0.4; charset=utf-8")] $ LB.fromStrict $ encodeUtf8 text + _ -> respond notFound + +runMetrics :: ListenerConfig -> Int -> MetricsEnv -> IO () +runMetrics lc port env@MetricsEnv {meNames} = do + v <- newTVarIO noChainRead + census <- newTVarIO noCensusRead + let serve = safely ("store metrics: the endpoint on port " <> tshow port) $ Warp.runSettings (Warp.setPort port . Warp.setHost (fromString metricsHost) $ webSettings lc) (metricsApp env v census) + raceAny_ $ (void serve >> forever (threadDelay maxBound)) : concat [[runChainReads ns v, runCensus ns census] | Just ns <- [meNames]] diff --git a/apps/simplex-store-service/src/StoreService/Names/Abi.hs b/apps/simplex-store-service/src/StoreService/Names/Abi.hs index 6ad5cb1b94..146553e52b 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Abi.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Abi.hs @@ -1,5 +1,6 @@ {-# LANGUAGE LambdaCase #-} {-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE TupleSections #-} module StoreService.Names.Abi ( AbiValue (..), @@ -21,6 +22,7 @@ module StoreService.Names.Abi decodeBytes32, decodeBytes, decodeString, + decodeLabels, ) where @@ -126,6 +128,18 @@ decodeBytes ret = offsetAt 0 ret >>= bytesAt ret decodeString :: ByteString -> Either AbiError Text decodeString ret = decodeBytes ret >>= first (const AENotUtf8) . decodeUtf8' +-- | A return of a bytes32[] and a string[] of the same length, paired. +decodeLabels :: ByteString -> Either AbiError [(ByteString, Text)] +decodeLabels ret = do + (hashCount, hashesStart) <- offsetAt 0 ret >>= arrayAt + (labelCount, labelsStart) <- offsetAt abiWordSize ret >>= arrayAt + unless (hashCount == labelCount) $ Left AEOffset + hashes <- mapM (\i -> wordAt (hashesStart + i * abiWordSize) ret) [0 .. hashCount - 1] + labels <- mapM (\i -> offsetAt (labelsStart + i * abiWordSize) ret >>= bytesAt ret . (labelsStart +) >>= first (const AENotUtf8) . decodeUtf8') [0 .. labelCount - 1] + pure $ zip hashes labels + where + arrayAt o = (,o + abiWordSize) <$> offsetAt o ret + wordAt :: Int -> ByteString -> Either AbiError ByteString wordAt o ret | o + abiWordSize <= B.length ret = Right $ B.take abiWordSize $ B.drop o ret diff --git a/apps/simplex-store-service/src/StoreService/Names/Census.hs b/apps/simplex-store-service/src/StoreService/Names/Census.hs new file mode 100644 index 0000000000..4e892aeb88 --- /dev/null +++ b/apps/simplex-store-service/src/StoreService/Names/Census.hs @@ -0,0 +1,128 @@ +{-# LANGUAGE NamedFieldPuns #-} +{-# LANGUAGE OverloadedStrings #-} + +module StoreService.Names.Census + ( Census (..), + CensusName (..), + Records (..), + recordsLabel, + nameCensus, + ) +where + +import Control.Concurrent.Async (mapConcurrently) +import Control.Monad.Except (ExceptT (..), runExceptT) +import Data.ByteString (ByteString) +import qualified Data.ByteString as B +import Data.List (unfoldr) +import Data.Text (Text) +import qualified Data.Text as T +import Data.Word (Word64) +import Simplex.Messaging.Crypto (keccak256) +import Simplex.Messaging.Eth.Address (Address, unAddress) +import StoreService.Config (ChainConfig (..)) +import StoreService.Names.Abi +import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError, latestBlock) +import StoreService.Names.Contracts (Contracts, callAt) + +data Records = RNone | RContact | RChannel | RBoth | ROtherResolver + deriving (Eq, Ord, Show, Enum, Bounded) + +recordsLabel :: Records -> Text +recordsLabel r = case r of + RNone -> "none" + RContact -> "contact" + RChannel -> "channel" + RBoth -> "both" + ROtherResolver -> "other_resolver" + +-- | Level 2 is a name such as alice.simplex, 3 its subname; the length is in characters. +data CensusName = CensusName {cnLevel :: Int, cnLength :: Int, cnRecords :: Records} + deriving (Eq, Show) + +data Census = Census {cBlock :: Word64, cNames :: [CensusName]} + deriving (Eq, Show) + +data CensusEnv = CensusEnv + { ceContracts :: Contracts, + ceAt :: BlockTag, + ceRegistry :: Address, + ceSubnames :: Address, + ceResolver :: Address + } + +type Node = (ByteString, Text) + +readsAtOnce :: Int +readsAtOnce = 16 + +childrenPage :: Integer +childrenPage = 100 + +contactKey, channelKey :: ByteString +contactKey = "simplex.contact" +channelKey = "simplex.channel" + +-- | Every live name and subname at the latest block: names while nameExpires is after the block's time, subnames while +-- the subname registrar names an owner. A name whose resolver is not the configured one is counted without its records. +nameCensus :: ChainClient -> Contracts -> IO (Either ChainError Census) +nameCensus c@ChainClient {ccConfig = ChainConfig {cBaseRegistrar, cResolver}} ct = runExceptT $ do + Block {blockNumber, blockTimestamp} <- ExceptT $ latestBlock c + let at = AtBlock blockNumber + registry <- ExceptT $ callAt ct at cBaseRegistrar ("ens()", "") decodeAddress + subnames <- ExceptT $ callAt ct at cBaseRegistrar ("subnameHook()", "") decodeAddress + base <- ExceptT $ callAt ct at cBaseRegistrar ("baseNode()", "") decodeBytes32 + total <- ExceptT $ callAt ct at cBaseRegistrar ("totalSupply()", "") decodeUint256 + let env = CensusEnv {ceContracts = ct, ceAt = at, ceRegistry = registry, ceSubnames = subnames, ceResolver = cResolver} + registrar :: ByteString -> Integer -> (ByteString -> Either AbiError a) -> IO (Either ChainError a) + registrar signature i = readAt env cBaseRegistrar signature (uint256 i) + ids <- readAll [registrar "tokenByIndex(uint256)" i decodeUint256 | i <- [0 .. total - 1]] + expiries <- readAll [registrar "nameExpires(uint256)" i decodeUint256 | i <- ids] + let live = [i | (i, expiry) <- zip ids expiries, expiry > toInteger blockTimestamp] + labels <- readAll [registrar "labelOf(uint256)" i decodeString | i <- live] + Census blockNumber <$> levels env 2 [(keccak256 (base <> uint256 i), label) | (i, label) <- zip live labels] + +levels :: CensusEnv -> Int -> [Node] -> ExceptT ChainError IO [CensusName] +levels env level nodes + | null nodes = pure [] + | otherwise = do + records <- recordsOf env nodes + children <- childrenOf env nodes + (zipWith (\(_, label) r -> CensusName level (T.length label) r) nodes records <>) <$> levels env (level + 1) children + +recordsOf :: CensusEnv -> [Node] -> ExceptT ChainError IO [Records] +recordsOf env@CensusEnv {ceRegistry, ceResolver} nodes = do + resolvers <- readAll [readAt env ceRegistry "resolver(bytes32)" node decodeAddress | (node, _) <- nodes] + readAll [if r == ceResolver then textsOf node else pure (Right ROtherResolver) | ((node, _), r) <- zip nodes resolvers] + where + textsOf node = do + contact <- readAt env ceResolver "text(bytes32,string)" (textArgs node contactKey) decodeString + channel <- readAt env ceResolver "text(bytes32,string)" (textArgs node channelKey) decodeString + pure $ recordsOfTexts <$> contact <*> channel + textArgs node key = abiTuple [Static node, Dynamic (abiBytes key)] + +recordsOfTexts :: Text -> Text -> Records +recordsOfTexts contact channel = case (T.null contact, T.null channel) of + (True, True) -> RNone + (False, True) -> RContact + (True, False) -> RChannel + (False, False) -> RBoth + +-- | A subname the subname registrar no longer names an owner of, as of a lapsed name or a previous owner, is left out. +childrenOf :: CensusEnv -> [Node] -> ExceptT ChainError IO [Node] +childrenOf env@CensusEnv {ceSubnames} parents = do + counts <- readAll [readAt env ceSubnames "childrenLength(bytes32)" node decodeUint256 | (node, _) <- parents] + let pages = [(node, start) | ((node, _), count) <- zip parents counts, start <- [0, childrenPage .. count - 1]] + listed <- readAll [readAt env ceSubnames "getChildren(bytes32,uint256,uint256)" (node <> uint256 start <> uint256 childrenPage) decodeLabels | (node, start) <- pages] + let found = [(keccak256 (node <> labelHash), label) | ((node, _), children) <- zip pages listed, (labelHash, label) <- children] + owners <- readAll [readAt env ceSubnames "ownerOf(uint256)" node decodeAddress | (node, _) <- found] + pure [child | (child, owner) <- zip found owners, not (B.all (== 0) (unAddress owner))] + +readAt :: CensusEnv -> Address -> ByteString -> ByteString -> (ByteString -> Either AbiError a) -> IO (Either ChainError a) +readAt CensusEnv {ceContracts, ceAt} to signature args = callAt ceContracts ceAt to (signature, args) + +readAll :: [IO (Either ChainError a)] -> ExceptT ChainError IO [a] +readAll calls = concat <$> mapM (ExceptT . fmap sequence . mapConcurrently id) (groupsOf readsAtOnce calls) + +groupsOf :: Int -> [a] -> [[a]] +groupsOf n = unfoldr (\xs -> if null xs then Nothing else Just (splitAt n xs)) diff --git a/apps/simplex-store-service/src/StoreService/Names/Chain.hs b/apps/simplex-store-service/src/StoreService/Names/Chain.hs index a6b51e3de7..98c2d37c38 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Chain.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Chain.hs @@ -22,6 +22,7 @@ module StoreService.Names.Chain transactionReceipt, transactionKnown, transactionCount, + registrarBalance, estimateGas, recentTip, signTransaction, @@ -180,6 +181,9 @@ transactionKnown c (TxHash h) = transactionCount :: ChainClient -> Address -> BlockTag -> IO (Either ChainError Word64) transactionCount c a tag = rpc c "eth_getTransactionCount" [J.toJSON a, blockTagParam tag] quantity +registrarBalance :: ChainClient -> IO (Either ChainError Integer) +registrarBalance c@ChainClient {ccRegistrar} = rpc c "eth_getBalance" [J.toJSON ccRegistrar, blockTagParam Latest] natural + blockTagParam :: BlockTag -> J.Value blockTagParam = \case Latest -> "latest" @@ -386,8 +390,13 @@ nullable p = \case v -> Just <$> p v quantity :: forall a. (Integral a, Bounded a) => J.Value -> JT.Parser a -quantity = J.withText "quantity" $ \t -> case T.stripPrefix "0x" t of - Just h | T.length h <= maxQuantityDigits, [(n, "")] <- readHex (T.unpack h), n <= toInteger (maxBound :: a) -> pure $ fromInteger n +quantity = + natural >=> \n -> + if n <= toInteger (maxBound :: a) then pure $ fromInteger n else fail "not a quantity" + +natural :: J.Value -> JT.Parser Integer +natural = J.withText "quantity" $ \t -> case T.stripPrefix "0x" t of + Just h | T.length h <= maxQuantityDigits, [(n, "")] <- readHex (T.unpack h) -> pure n _ -> fail "not a quantity" bytes :: J.Value -> JT.Parser ByteString diff --git a/apps/simplex-store-service/src/StoreService/Names/Contracts.hs b/apps/simplex-store-service/src/StoreService/Names/Contracts.hs index 4f3d15385e..acf14c28a5 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Contracts.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Contracts.hs @@ -11,6 +11,7 @@ module StoreService.Names.Contracts CommitmentAges (..), Terms (..), TermsRefusal (..), + RegistrarFunds (..), newContracts, secondsPerYear, nameStatus, @@ -18,6 +19,7 @@ module StoreService.Names.Contracts textRecord, chainPriceCents, sponsorNonce, + registrarFunds, resolverNonce, namePolicy, forgetNamePolicy, @@ -28,6 +30,7 @@ module StoreService.Names.Contracts termsText, fetchCurrentTerms, runTermsFetch, + termsHeld, policyNonce, acceptedTerms, confirmedTag, @@ -38,6 +41,7 @@ module StoreService.Names.Contracts commitmentAges, together, concurrent, + callAt, ) where @@ -72,7 +76,7 @@ import Simplex.Messaging.SimplexName (LabelHash (..), SimplexTLD) import Simplex.Messaging.Util (catchOwn', tshow) import StoreService.Config (ChainConfig (..), localChainId) import StoreService.Names.Abi -import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError (..), confirmedNumber, ethCallAt, fetchText, latestBlock, nodeChainId) +import StoreService.Names.Chain (Block (..), BlockTag (..), ChainClient (..), ChainError (..), confirmedNumber, ethCallAt, fetchText, latestBlock, nodeChainId, registrarBalance) import StoreService.Names.Registration (channelRecordKey, contactRecordKey, labelNode, nodeBytes) import StoreService.Names.Revert (RevertName (..)) import StoreService.Names.Store (getTermsText, setTermsText) @@ -125,6 +129,9 @@ data Terms = Terms {tmLink :: Text, tmHash :: ByteString} data TermsRefusal = TRLink | TRWait Int deriving (Eq, Show) +data RegistrarFunds = RegistrarFunds {rfBalanceWei :: Integer, rfAllowanceCents :: Integer} + deriving (Eq, Show) + secondsPerYear :: Integer secondsPerYear = 31536000 @@ -210,6 +217,12 @@ chainPriceCents ct@Contracts {ctMinPriceCentsPerYear} len years = fmap (>>= chec sponsorNonce :: Contracts -> Address -> IO (Either ChainError Word64) sponsorNonce ct owner = call ct (cSponsor $ chainConfig ct) ("nonces(address)", addressWord owner) decodeWord64 +registrarFunds :: Contracts -> IO (Either ChainError RegistrarFunds) +registrarFunds ct@Contracts {ctChain = c@ChainClient {ccRegistrar}} = + together $ RegistrarFunds <$> concurrent (registrarBalance c) <*> concurrent allowance + where + allowance = call ct (cSponsor $ chainConfig ct) ("registrarAllowance(address)", addressWord ccRegistrar) $ fmap (`div` attoUsdPerCent) . decodeUint256 + resolverNonce :: Contracts -> Address -> IO (Either ChainError Word64) resolverNonce ct owner = call ct (resolver ct) ("nonces(address)", addressWord owner) decodeWord64 @@ -254,6 +267,14 @@ termsText ct@Contracts {ctTermsTexts, ctFetching} t@Terms {tmHash} = where kept = M.lookup tmHash <$> readIORef ctTermsTexts +-- | Each kind of terms the name policy links, with whether its text is held; nothing is fetched. +termsHeld :: Contracts -> IO (Either ChainError [(Text, Bool)]) +termsHeld ct@Contracts {ctTermsTexts} = termsLinks ct >>= mapM held + where + held (generalLink, communityLink) = do + texts <- readIORef ctTermsTexts + pure [(kind, M.member tmHash texts) | (kind, Just Terms {tmHash}) <- [("general", termsOf generalLink), ("community", termsOf communityLink)]] + fetchCurrentTerms :: Contracts -> IO () fetchCurrentTerms ct = termsLinks ct >>= \case diff --git a/apps/simplex-store-service/src/StoreService/Names/Store.hs b/apps/simplex-store-service/src/StoreService/Names/Store.hs index 805200a613..6508e4e5fa 100644 --- a/apps/simplex-store-service/src/StoreService/Names/Store.hs +++ b/apps/simplex-store-service/src/StoreService/Names/Store.hs @@ -39,6 +39,7 @@ module StoreService.Names.Store EditState (..), RecordEdit (..), NewRecordEdit (..), + NamesCounts (..), findLimit, orderHash, orderLive, @@ -99,6 +100,7 @@ module StoreService.Names.Store relayedEditTimes, getTermsText, setTermsText, + namesCounts, ) where @@ -487,6 +489,19 @@ data NewRecordEdit = NewRecordEdit nreBodyHash :: ByteString } +data NamesCounts = NamesCounts + { ncPhases :: [(NamePhase, Int)], + ncQueueOldestChange :: Maybe UTCTime, + ncPendingTransactions :: Int, + ncOldestPending :: Maybe UTCTime, + ncOpenInvoices :: Int, + ncHeldBalances :: Int, + ncHeldCents :: Int, + ncPendingEdits :: Int, + ncOldestEdit :: Maybe UTCTime + } + deriving (Eq, Show) + findLimit :: Int findLimit = 50 @@ -1568,6 +1583,28 @@ setTermsText db termsHash text = |] (Binary termsHash, Binary text) +namesCounts :: DB.Connection -> IO NamesCounts +namesCounts db = do + ncPhases <- DB.query_ db "SELECT phase, COUNT(*) FROM sx_badge_service_name_registrations GROUP BY phase" + DB.query + db + [sql| + SELECT + (SELECT MIN(updated_at) FROM sx_badge_service_name_registrations WHERE phase IN (?,?,?) OR (phase = ? AND label IS NOT NULL)), + (SELECT COUNT(DISTINCT registrar_nonce) FROM sx_badge_service_name_transactions WHERE status = ?), + (SELECT MIN(sent_at) FROM sx_badge_service_name_transactions WHERE status = ?), + (SELECT COUNT(*) FROM sx_badge_service_name_invoices ni JOIN sx_badge_service_invoices i ON i.invoice_id = ni.invoice_id WHERE i.status = ?), + (SELECT COUNT(*) FROM sx_badge_service_name_balances WHERE reserved_by IS NOT NULL), + (SELECT COALESCE(SUM(amount), 0) FROM sx_badge_service_name_balances WHERE reserved_by IS NOT NULL), + (SELECT COUNT(*) FROM sx_badge_service_name_record_edits WHERE state = ?), + (SELECT MIN(created_at) FROM sx_badge_service_name_record_edits WHERE state = ?) + |] + (PQueued, PCommitting, PRegistering, PCommitted, TSPending, TSPending, invoiceStatusText ISOpen, pendingEdit, pendingEdit) + >>= \case + [(ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit)] -> + pure NamesCounts {ncPhases, ncQueueOldestChange, ncPendingTransactions, ncOldestPending, ncOpenInvoices, ncHeldBalances, ncHeldCents, ncPendingEdits, ncOldestEdit} + _ -> E.throwIO $ StoreDecodeError "names counts: not one row" + int64 :: Word64 -> Int64 int64 = fromIntegral diff --git a/apps/simplex-store-service/src/StoreService/Poller.hs b/apps/simplex-store-service/src/StoreService/Poller.hs index 77b8423770..283e5a53b1 100644 --- a/apps/simplex-store-service/src/StoreService/Poller.hs +++ b/apps/simplex-store-service/src/StoreService/Poller.hs @@ -126,6 +126,8 @@ data PollerEnv = PollerEnv pePoll :: PollConfig, peSkipped :: TVar (Map Text UTCTime), peListedAt :: TVar (Maybe UTCTime), + -- | When each provider's list last succeeded, by its text encoding. + peListedOk :: TVar (Map Text UTCTime), peStrayEvery :: NominalDiffTime, -- When the cancel first failed, for orders older than 72 hours. peCancelFailed :: TVar (Map Text UTCTime), @@ -142,13 +144,14 @@ newPollerEnv :: DBStore -> Waiters -> Maybe Phases -> ReadHints -> [Provider] -> newPollerEnv peStore peWaiters pePhases peHints peProviders pePoll = do peSkipped <- newTVarIO M.empty peListedAt <- newTVarIO Nothing + peListedOk <- newTVarIO M.empty peCancelFailed <- newTVarIO M.empty peReadFailed <- newTVarIO M.empty peCleanedAt <- newTVarIO Nothing peCleanupLeft <- newTVarIO [] peCreditedAt <- newTVarIO Nothing peRates <- newTVarIO M.empty - pure PollerEnv {peStore, peWaiters, pePhases, peHints, peProviders, pePoll, peSkipped, peListedAt, peStrayEvery = strayListInterval, peCancelFailed, peReadFailed, peCleanedAt, peCleanupLeft, peCreditedAt, peRates} + pure PollerEnv {peStore, peWaiters, pePhases, peHints, peProviders, pePoll, peSkipped, peListedAt, peListedOk, peStrayEvery = strayListInterval, peCancelFailed, peReadFailed, peCleanedAt, peCleanupLeft, peCreditedAt, peRates} -- | Read providers before the sweep, and leave out of the sweep every invoice whose read or settlement failed, -- so an invoice with money in it is never expired. After a list that cannot account for every invoice, only overdue @@ -241,13 +244,14 @@ readRows env now rows = -- | The refs of ours the list did not account for, so the sweep does not expire one over money it missed; -- Nothing when it cannot name them all. listPass :: PollerEnv -> UTCTime -> Provider -> IO (Maybe (Set Text)) -listPass env now p = +listPass env@PollerEnv {peListedOk} now p = pListOpen p >>= \case Left (ProviderError e) -> do due <- dueToWarn env now ("list failed: " <> tshow (pProvider p)) when due $ logWarn ("badge poller: " <> tshow (pProvider p) <> " list failed; every invoice waits for the next pass: " <> e) pure Nothing Right ListPass {lpMoved, lpSkipped} -> do + atomically $ modifyTVar' peListedOk $ M.insert (textEncode $ pProvider p) now owners <- mapM (\s -> safelyWith (skipWhat s) SkipUnaccounted (reportSkip env (pProvider p) now s)) lpSkipped reportStrangers env (pProvider p) now [reason | (SkipStranger, (_, reason)) <- zip owners lpSkipped] unsettled <- filterM (\m -> not <$> safely (settleWhat m) (settleMoved env (pProvider p) now m)) lpMoved diff --git a/apps/simplex-store-service/src/StoreService/Service.hs b/apps/simplex-store-service/src/StoreService/Service.hs index 1521711002..da122dd0fd 100644 --- a/apps/simplex-store-service/src/StoreService/Service.hs +++ b/apps/simplex-store-service/src/StoreService/Service.hs @@ -47,9 +47,10 @@ where import StoreService.Catalog (StoreProduct (..), defaultCatalog, storeProduct) import StoreService.Codes (issueFailedText, issueOneCode, singleUse) -import StoreService.Config (AppleStoreConfig (..), BadgeIssuerKey (..), ChainConfig (..), GroupConfig (..), NamesConfig (..), PlayStoreConfig (..), ServiceConfig (..), minNameYears, readServiceConfig) +import StoreService.Config (AppleStoreConfig (..), BadgeIssuerKey (..), ChainConfig (..), GroupConfig (..), ListenerConfig (..), NamesConfig (..), PlayStoreConfig (..), ServiceConfig (..), minNameYears, readServiceConfig) import StoreService.Group (GroupEvent, ensureManagedGroup, fmtTime, groupEvent, hasTracker, joinerAcceptance, refreshTracker, revokeWithTracker, runGroupLane) import StoreService.Group.Command (MoneyCmd (..), badgeTypeP, balanceArgsP, closeArgsP, codeP, maxMonths, maxNameYears, nameCodeP, prepaidCodeP, recordRefundArgsP, textTokenP) +import StoreService.Metrics (MetricsEnv (..), NamesSource (..), runMetrics) import StoreService.Names.Abi (hex) import StoreService.Names.Balances (RefundRecord (..), RefundRefusal (..), recordRefund) import StoreService.Names.Chain (ChainClient, newChainClient, nodeChainId) @@ -65,8 +66,8 @@ import StoreService.Names.Turns (Turns, newTurns) import StoreService.Names.View (addressText) import StoreService.Names.Worker (CloseRefusal (..), Closed (..), closeRegistration, newWorker, runCodeSweep, runWorkers) import StoreService.Options -import StoreService.Poller (expiryGrace, newPollerEnv, newReadHints, runPoller) -import StoreService.Providers (Provider) +import StoreService.Poller (PollerEnv (peListedOk), expiryGrace, newPollerEnv, newReadHints, runPoller) +import StoreService.Providers (Provider (pProvider)) import StoreService.Providers.BTCPay (btcpayProvider) import StoreService.Providers.Stripe (stripeProvider) import StoreService.Store @@ -78,7 +79,7 @@ import StoreService.StoreReceipts.Google (PlayVerifier (..), playStoreVerifier) import StoreService.StoreReceipts.Mock (newMockStoreVerifier) import StoreService.Waiters (Waiters, newWaiters) import StoreService.Web.Http (Bucket, takeFrom) -import StoreService.Web.Server (checkSiteDirs, exportWebapp, newWebEnv, runWebListeners) +import StoreService.Web.Server (WebEnv (weWebhookAt), checkSiteDirs, exportWebapp, newWebEnv, runWebListeners) import Control.Applicative (optional, (<|>)) import Control.Concurrent.STM import Control.Logger.Simple (logError, logNote, logWarn) @@ -119,7 +120,7 @@ import Simplex.Messaging.Agent.Store.Common (DBStore) import qualified Simplex.Messaging.Agent.Store.DB as DB import qualified Simplex.Messaging.Crypto as C import Simplex.Messaging.Crypto.BBS (bbsPublicKey) -import Simplex.Messaging.Encoding.String (strEncode) +import Simplex.Messaging.Encoding.String (strEncode, textEncode) import Simplex.Messaging.SimplexName (LabelHash (..), SimplexTLD) import Simplex.Messaging.Util (catchOwn, catchOwn', raceAny_, safeDecodeUtf8, tshow) import Simplex.Messaging.Version (isCompatible) @@ -328,7 +329,10 @@ storeService opts serviceCfg baseCfg env = do exportWebapp sc pollerEnv <- newPollerEnv chatStore ws (nePhases <$> namesEnv) hints providers (poll sc) nameLanes <- maybe (pure []) namesLanes namesEnv - pure $ [runWebListeners webEnv, runPoller pollerEnv] <> nameLanes + let metricsEnv = MetricsEnv {meStore = chatStore, meProviders = map (textEncode . pProvider) providers, meListedOk = peListedOk pollerEnv, meWebhookAt = weWebhookAt webEnv, meNames = namesSource <$> namesEnv} + namesSource NamesEnv {neChain, neContracts} = NamesSource {nsChain = neChain, nsContracts = neContracts} + metricsLane = [runMetrics (listener sc) port metricsEnv | Just port <- [lMetricsPort $ listener sc]] + pure $ [runWebListeners webEnv, runPoller pollerEnv] <> nameLanes <> metricsLane startNames :: ChatController -> Turns -> Waiters -> [Provider] -> NamesConfig -> IO NamesEnv startNames cc@ChatController {chatStore} turns waiters providers nc@NamesConfig {nTLD, nCodesOnly, nMinPriceCentsPerYear, nChain = ChainConfig {cChainId}} = diff --git a/apps/simplex-store-service/src/StoreService/Store.hs b/apps/simplex-store-service/src/StoreService/Store.hs index feab05accc..23c851898f 100644 --- a/apps/simplex-store-service/src/StoreService/Store.hs +++ b/apps/simplex-store-service/src/StoreService/Store.hs @@ -50,6 +50,8 @@ module StoreService.Store getEditableTrackers, RevokeResult (..), revokeCode, + BadgeCounts (..), + badgeCounts, ) where @@ -65,7 +67,7 @@ import Data.Time.Clock (UTCTime) import Simplex.Chat.Badges (BadgeCredential, BadgeMasterKey (..), BadgeType) import Simplex.Chat.Badges.Ledger import Simplex.Chat.StoreService (StatementCreditType (..), StatementDebitType (..), StatementEntry (..), StatementEntryType (..), SubscriptionState) -import Simplex.Chat.Badges.Types (BadgeCodePaymentStatus, BadgePurchaseStatus (..)) +import Simplex.Chat.Badges.Types (BadgeCodePaymentStatus (..), BadgePurchaseStatus (..)) import Simplex.Chat.PaymentService.Types (CurrencyAmount (..), InvoiceId (..), PaymentProvider (..), PaymentStatus (..)) import Simplex.Chat.Store.Shared (insertedRowId) import Simplex.Messaging.Agent.Store.DB (Binary (..), BoolInt (..)) @@ -647,3 +649,26 @@ getEditableTrackers db sentAfter = ORDER BY badge_code_id |] (Only sentAfter) + +data BadgeCounts = BadgeCounts + { bcPurchases :: [(Text, Int)], + bcSubscriptions :: [(Text, Int)], + bcOpenCodes :: Int + } + deriving (Eq, Show) + +-- | Open codes are paid or free, not revoked, not expired, and have redemptions left. +badgeCounts :: DB.Connection -> UTCTime -> IO BadgeCounts +badgeCounts db now = do + bcPurchases <- DB.query_ db "SELECT status, COUNT(*) FROM sx_badge_service_badge_purchases GROUP BY status ORDER BY status" + bcSubscriptions <- DB.query_ db "SELECT subscription_state, COUNT(*) FROM sx_badge_service_payments WHERE subscription_state IS NOT NULL GROUP BY subscription_state ORDER BY subscription_state" + bcOpenCodes <- + maybeFirstRow' 0 fromOnly $ + DB.query + db + [sql| + SELECT COUNT(*) FROM sx_badge_service_badge_codes + WHERE code_payment_status IN (?,?) AND redeem_count < redeem_limit AND revoked_at IS NULL AND (expires_at IS NULL OR expires_at > ?) + |] + (textEncode CPSPaid, textEncode CPSFree, now) + pure BadgeCounts {bcPurchases, bcSubscriptions, bcOpenCodes} diff --git a/apps/simplex-store-service/src/StoreService/Store/Invoices.hs b/apps/simplex-store-service/src/StoreService/Store/Invoices.hs index fb33b63747..58490b4086 100644 --- a/apps/simplex-store-service/src/StoreService/Store/Invoices.hs +++ b/apps/simplex-store-service/src/StoreService/Store/Invoices.hs @@ -25,6 +25,7 @@ module StoreService.Store.Invoices codeHashExists, OverdueInvoice (..), overdueInvoices, + openInvoiceCounts, expireOverdueInvoice, lockInvoice, unfundedQuery, @@ -267,6 +268,13 @@ qOverdueInvoices = <> unfundedOnly <> "ORDER BY @invoices.created_at" +qOpenInvoiceCounts :: Query +qOpenInvoiceCounts = + mkQuery $ + "SELECT provider, COUNT(*), COALESCE(SUM(CASE WHEN expires_at < ? " + <> unfundedOnly + <> "THEN 1 ELSE 0 END), 0) FROM @invoices WHERE status = 'open' GROUP BY provider ORDER BY provider" + qExpireOverdue :: Query qExpireOverdue = mkQuery $ @@ -617,6 +625,10 @@ overdueInvoices st cutoff = do logError ("store: overdue invoice " <> oiProviderRef <> " names provider " <> providerTxt <> ", which this build does not know, so it is left open") pure Nothing +-- | Open invoices by provider, each with how many of them the sweep would expire at this cutoff. +openInvoiceCounts :: DB.Connection -> UTCTime -> IO [(Text, Int, Int)] +openInvoiceCounts db cutoff = DB.query db qOpenInvoiceCounts (Only (truncateToSecond cutoff)) + -- | Expires the invoice if it is still open, has received no money, and is past the cutoff. expireOverdueInvoice :: DB.Connection -> UTCTime -> InvoiceId -> IO Bool expireOverdueInvoice db cutoff' (InvoiceId i) = (> 0) <$> executeChanging db qExpireOverdue (cutoff, cutoff, i) diff --git a/apps/simplex-store-service/src/StoreService/Web/Server.hs b/apps/simplex-store-service/src/StoreService/Web/Server.hs index 9615c825f8..fda71d5811 100644 --- a/apps/simplex-store-service/src/StoreService/Web/Server.hs +++ b/apps/simplex-store-service/src/StoreService/Web/Server.hs @@ -34,6 +34,7 @@ import StoreService.Web.Names (namesApp, namesRoutes) import StoreService.Web.Names.Orders (nameInvoiceHandler, nameInvoiceView) import StoreService.Web.Names.Registration (heldFor, registrationChangedResponse, versionParam) import Control.Concurrent.Async (race) +import Control.Concurrent.STM (TVar, atomically, modifyTVar', newTVarIO) import qualified Control.Exception as E import Control.Logger.Simple (logError, logNote, logWarn) import Control.Monad (filterM, forM_, void, when) @@ -46,6 +47,8 @@ import qualified Data.ByteString.Lazy as LB import Data.Char (toLower) import Data.Functor ((<&>)) import Data.List (isPrefixOf) +import Data.Map.Strict (Map) +import qualified Data.Map.Strict as M import Data.Maybe (catMaybes, fromMaybe, isJust) import Data.String (fromString) import Data.Text (Text) @@ -80,7 +83,9 @@ data WebEnv = WebEnv -- | @Nothing@ when the ini has no @[names]@ section. weNames :: Maybe NamesEnv, -- | The name orders' part of weNames, which tests set without a chat controller. - weOrders :: Maybe OrdersEnv + weOrders :: Maybe OrdersEnv, + -- | When each provider last delivered a webhook that verified, by its text encoding. + weWebhookAt :: TVar (Map Text UTCTime) } -- | A built site, served when @serve_webapp@ is on. @@ -95,7 +100,8 @@ newWebEnv weStore weConfig weWaiters weHints weProviders weNames = do weHttp <- newHttpEnv lc weBadgesSite <- if lServeWebapp then Just <$> loadSite (keyFills weConfig) lBadgesStaticDir else pure Nothing weNamesSite <- mapM (loadSite (namesFills weConfig)) (lNamesPort *> lNamesStaticDir) - pure WebEnv {weStore, weConfig, weWaiters, weProviders, weHoldMicros = holdMicros, weHints, weHttp, weBadgesSite, weNamesSite, weNames, weOrders = ordersEnv <$> weNames} + weWebhookAt <- newTVarIO M.empty + pure WebEnv {weStore, weConfig, weWaiters, weProviders, weHoldMicros = holdMicros, weHints, weHttp, weBadgesSite, weNamesSite, weNames, weOrders = ordersEnv <$> weNames, weWebhookAt} where lc@ListenerConfig {lServeWebapp, lBadgesStaticDir, lNamesStaticDir, lNamesPort} = listener weConfig loadSite fills dir = Site dir <$> prepareShell dir fills @@ -523,8 +529,9 @@ webhookHandler env@WebEnv {weStore, weHints} provider route req respond = case pVerifyWebhook p now (requestHeaders req) (LB.toStrict body) of Left (WebhookError e) -> refuse e Left (WebhookStale e) -> logRefusal logWarn (route <> " refused: " <> e) >> respond webhookRefused - Right Nothing -> ignore body "nothing this service acts on" - Right (Just ref) -> resolve body ref + Right ref_ -> do + atomically $ modifyTVar' (weWebhookAt env) $ M.insert (textEncode provider) now + maybe (ignore body "nothing this service acts on") (resolve body) ref_ refuse :: Text -> IO ResponseReceived refuse why = logRefusal logNote (route <> " refused: " <> why) >> respond webhookRefused -- anyone can post here, so the lines a flood would write are capped, not the refusals diff --git a/apps/simplex-store-service/store_service.ini.example b/apps/simplex-store-service/store_service.ini.example index cffef4ccde..63a7ef93a6 100644 --- a/apps/simplex-store-service/store_service.ini.example +++ b/apps/simplex-store-service/store_service.ini.example @@ -23,6 +23,8 @@ trust_forwarded_for = on ; names_static_dir = ./apps/simplex-domains/webapp/dist ; names_export_dir = /srv/names-export ; names_port = 8081 +; Prometheus metrics of the store, on 127.0.0.1 whatever host is +; metrics_port = 9300 ; omit this whole section to disable Bitcoin and Monero (the browser shows the provider-unavailable screen) [btcpay] diff --git a/scripts/store-service/store_service.ini.example b/scripts/store-service/store_service.ini.example index c170d5542a..62faa5eb3c 100644 --- a/scripts/store-service/store_service.ini.example +++ b/scripts/store-service/store_service.ini.example @@ -15,6 +15,9 @@ badges_export_dir = /srv/badges-export ; the names site (simplex.domains), with [names] only: the folder it is exported to, which compose ; bind-mounts to ./names for Caddy ;names_export_dir = /srv/names-export +; Prometheus metrics of the store, on 127.0.0.1 whatever host is; Caddy proxies them on the host's +; monitoring site +;metrics_port = 9300 ; on because Caddy proxies /api and /webhooks to this service; without it the rate limiter keys on ; Caddy's address and shares one bucket across all callers trust_forwarded_for = on diff --git a/simplex-chat.cabal b/simplex-chat.cabal index 4e350f6d09..475a687983 100644 --- a/simplex-chat.cabal +++ b/simplex-chat.cabal @@ -460,8 +460,10 @@ executable simplex-store-service StoreService.Config StoreService.Group StoreService.Group.Command + StoreService.Metrics StoreService.Names.Abi StoreService.Names.Balances + StoreService.Names.Census StoreService.Names.Chain StoreService.Names.Codes StoreService.Names.Community @@ -801,6 +803,7 @@ test-suite simplex-chat-test Bots.StoreService.FakeStripe Bots.StoreService.GroupIntegrationTests Bots.StoreService.GroupTests + Bots.StoreService.MetricsTests Bots.StoreService.NameRegistrationTests Bots.StoreService.NamesBalancesTests Bots.StoreService.NamesChainTests @@ -844,8 +847,10 @@ test-suite simplex-chat-test StoreService.Config StoreService.Group StoreService.Group.Command + StoreService.Metrics StoreService.Names.Abi StoreService.Names.Balances + StoreService.Names.Census StoreService.Names.Chain StoreService.Names.Codes StoreService.Names.Community diff --git a/tests/Bots/StoreService/ConfigTests.hs b/tests/Bots/StoreService/ConfigTests.hs index 92e1fc66ce..edf7156058 100644 --- a/tests/Bots/StoreService/ConfigTests.hs +++ b/tests/Bots/StoreService/ConfigTests.hs @@ -712,6 +712,7 @@ namesConfigTests = describe "StoreService names: config" $ do it "reads the names keys of [listener], defaulting to the image's build and port 8081" testNamesListenerKeys it "refuses the names keys of [listener] without [names]" testNamesSiteOnlyWithNames it "refuses a names_port outside 1-65535, on the badges_port, or with serve_webapp off" testNamesPortRules + it "reads metrics_port, off when absent, also without [names], and refuses one outside 1-65535 or on the badges_port or names_port" testMetricsPort it "leaves the names side off when [names] is absent" testNamesAbsent it "refuses [names] without [chain], and [chain] without [names]" testNamesNeedsChain it "refuses years below 2" testNamesYearsBelowTwo @@ -844,6 +845,19 @@ testNamesPortRules = do where refused ks e = parseIni (namesIniWith ks) >>= (`shouldBe` e) . either id (const "parsed") +testMetricsPort :: IO () +testMetricsPort = do + Right off <- parseIni (namesIni namesSection chainSection) + lMetricsPort (listener off) `shouldBe` Nothing + Right on <- parseIni (namesIniWith ["metrics_port = 9300"]) + lMetricsPort (listener on) `shouldBe` Just 9300 + Right badgesOnly <- parseIni (withListenerKeys ["metrics_port = 9300"]) + lMetricsPort (listener badgesOnly) `shouldBe` Just 9300 + forM_ ["0", "65536"] $ \v -> refused ["metrics_port = " <> v] "listener.metrics_port must be between 1 and 65535" + forM_ ["8080", "8081"] $ \v -> refused ["metrics_port = " <> v] "listener.metrics_port must differ from listener.badges_port and listener.names_port" + where + refused ks e = parseIni (namesIniWith ks) >>= (`shouldBe` e) . either id (const "parsed") + -- | namesIni with these lines first in [listener]. namesIniWith :: [T.Text] -> T.Text namesIniWith ks = withListenerKeys ks <> T.unlines (namesSection <> chainSection) diff --git a/tests/Bots/StoreService/FakeEthNode.hs b/tests/Bots/StoreService/FakeEthNode.hs index ae7ecb9445..483cc8600e 100644 --- a/tests/Bots/StoreService/FakeEthNode.hs +++ b/tests/Bots/StoreService/FakeEthNode.hs @@ -62,6 +62,7 @@ module Bots.StoreService.FakeEthNode fakeTip, fakeBaseFee, fakeMaxFeePerGas, + genesisTime, ) where diff --git a/tests/Bots/StoreService/MetricsTests.hs b/tests/Bots/StoreService/MetricsTests.hs new file mode 100644 index 0000000000..1221330c79 --- /dev/null +++ b/tests/Bots/StoreService/MetricsTests.hs @@ -0,0 +1,349 @@ +{-# LANGUAGE NamedFieldPuns #-} +{-# LANGUAGE OverloadedStrings #-} + +module Bots.StoreService.MetricsTests (storeMetricsTests) where + +import Bots.StoreService.FakeEthNode +import Bots.StoreService.NamesContractsTests (minPriceCentsPerYear) +import Bots.StoreService.NamesStoreTests (at, btc, insertInvoiceRow, insertProviderInvoice, newEdit, newReg, order, ownerA, ownerB, paidOrder, price, reveal, setChannel, t0) +import Bots.StoreService.WebTests (digestFixture, statusOf, webGet, webRequestBody, withApp, withServiceStore) +import Control.Concurrent.Async (cancel) +import Control.Concurrent.STM (TVar, newTVarIO, readTVarIO) +import Control.Exception (finally) +import Control.Monad (forM_) +import Crypto.Random (ChaChaDRG) +import qualified Data.Aeson as J +import Data.ByteString (ByteString) +import qualified Data.ByteString as B +import qualified Data.ByteString.Lazy as LB +import qualified Data.Map.Strict as M +import Data.Maybe (fromJust, isJust) +import Data.Text (Text) +import qualified Data.Text as T +import Data.Text.Encoding (decodeUtf8, encodeUtf8) +import Data.Time.Clock (UTCTime, getCurrentTime) +import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds) +import Data.Word (Word64, Word8) +import Network.HTTP.Client (Response, responseBody, responseHeaders) +import Network.HTTP.Types (hContentType) +import Simplex.Chat.Badges.Types (BadgePurchaseStatus (..), SubscriptionState (..)) +import Simplex.Messaging.Agent.Store.Common (withTransaction) +import Simplex.Messaging.Agent.Store.DB (Binary (..)) +import qualified Simplex.Messaging.Agent.Store.DB as DB +import Simplex.Messaging.Crypto (keccak256) +import qualified Simplex.Messaging.Crypto as C +import Simplex.Messaging.Encoding.String (textEncode) +import Simplex.Messaging.Eth.Address (Address, addressFromPrivateKey) +import Simplex.Messaging.SimplexName (SimplexTLD (..)) +import Simplex.Messaging.Util (tshow) +import StoreService.Metrics +import StoreService.Names.Abi (AbiValue (..), abiArray, abiBytes, abiTuple, addressFromBytes, addressWord, selector, uint256) +import StoreService.Names.Census (Census (..)) +import StoreService.Names.Chain (TxHash (..)) +import StoreService.Names.Contracts (newContracts, termsOf, termsText) +import StoreService.Names.Phases (RegistrationId (..)) +import StoreService.Names.Store +import StoreService.Store (badgeCounts) +import StoreService.Store.Invoices (openInvoiceCounts) +import System.Timeout (timeout) +import Test.Hspec + +storeMetricsTests :: Spec +storeMetricsTests = describe "StoreService: metrics" $ do + g <- runIO C.newRandom + it "counts open invoices by provider with those the sweep would expire, badge purchases by status, subscriptions by state and open badge codes, with the providers' last list and webhook, and 0 for a provider, status or state without rows" testServiceCounts + it "counts name orders by phase, the oldest change among the orders the service moves on, pending transactions by nonce, open name invoices, held balances and pending record edits" testNamesCounts + it "reads the registrar's funds, the latest block and which terms texts are held, and keeps each part while a read fails" $ testChainRead g + it "answers GET /metrics with the service metrics alone without names, and with the names metrics while the node holds the chain read" $ testScrape g + it "counts live names and owned subnames by level and records, with their label lengths, and keeps the last census while one fails" $ testCensus g + +testCensus :: TVar ChaChaDRG -> IO () +testCensus g = withServiceStore $ \st -> withFakeEthNode $ \node -> do + c <- fakeChainClient g node + ct <- newContracts st TLDSimplex minPriceCentsPerYear c + let ns = NamesSource {nsChain = c, nsContracts = ct} + answer to signature args = answerCall node to (selector signature <> args) . Returns + str t = abiTuple [Dynamic (abiBytes $ encodeUtf8 t)] + texts n (contact, channel) = + forM_ [("simplex.contact", contact), ("simplex.channel", channel)] $ \(key, text) -> + answer fakeResolver "text(bytes32,string)" (abiTuple [Static n, Dynamic (abiBytes key)]) (str text) + live = genesisTime + 1000 + names :: [(Text, Word64, Maybe (Text, Text))] + names = + [ ("alicebakery", live, Just ("", "")), + ("bobsbakery", live, Just ("https://smp8.simplex.im/a#bob", "")), + ("carolshop", live, Nothing), + ("expiredname", genesisTime, Nothing) + ] + aliceNode = keccak256 (censusBase <> keccak256 "alicebakery") + shopNode = keccak256 (aliceNode <> keccak256 "shop") + oldNode = keccak256 (aliceNode <> keccak256 "old") + answer fakeBaseRegistrar "ens()" "" $ addressWord censusRegistry + answer fakeBaseRegistrar "subnameHook()" "" $ addressWord censusSubnames + answer fakeBaseRegistrar "baseNode()" "" censusBase + answer fakeBaseRegistrar "totalSupply()" "" $ uint256 (length names) + forM_ (zip [0 :: Int ..] names) $ \(i, (label, expiry, records)) -> do + let labelHash = keccak256 (encodeUtf8 label) + n = keccak256 (censusBase <> labelHash) + answer fakeBaseRegistrar "tokenByIndex(uint256)" (uint256 i) labelHash + answer fakeBaseRegistrar "nameExpires(uint256)" labelHash (uint256 expiry) + answer fakeBaseRegistrar "labelOf(uint256)" labelHash (str label) + answer censusRegistry "resolver(bytes32)" n $ addressWord (maybe censusOtherResolver (const fakeResolver) records) + mapM_ (texts n) records + answer censusSubnames "childrenLength(bytes32)" n $ uint256 (if n == aliceNode then 2 else 0 :: Int) + answer censusSubnames "getChildren(bytes32,uint256,uint256)" (aliceNode <> uint256 (0 :: Int) <> uint256 (100 :: Int)) $ + abiTuple [Dynamic (uint256 (2 :: Int) <> keccak256 "shop" <> keccak256 "old"), Dynamic (abiArray [abiBytes "shop", abiBytes "old"])] + answer censusSubnames "ownerOf(uint256)" shopNode $ addressWord censusOwner + answer censusSubnames "ownerOf(uint256)" oldNode $ B.replicate 32 0 + answer censusRegistry "resolver(bytes32)" shopNode $ addressWord fakeResolver + texts shopNode ("https://smp8.simplex.im/a#shop", "https://smp8.simplex.im/c#shop") + answer censusSubnames "childrenLength(bytes32)" shopNode $ uint256 (0 :: Int) + v <- newTVarIO noCensusRead + readCensus ns v + fmap (cBlock . fst) . crCensus <$> readTVarIO v `shouldReturn` Just 1 + let counted = + [ "simplex_names{level=\"2\",records=\"none\"} 1", + "simplex_names{level=\"2\",records=\"contact\"} 1", + "simplex_names{level=\"2\",records=\"channel\"} 0", + "simplex_names{level=\"2\",records=\"both\"} 0", + "simplex_names{level=\"2\",records=\"other_resolver\"} 1", + "simplex_names{level=\"3\",records=\"none\"} 0", + "simplex_names{level=\"3\",records=\"contact\"} 0", + "simplex_names{level=\"3\",records=\"channel\"} 0", + "simplex_names{level=\"3\",records=\"both\"} 1", + "simplex_names{level=\"3\",records=\"other_resolver\"} 0" + ] + lengths = + [ "simplex_name_length_bucket{level=\"2\",le=\"8.0\"} 0", + "simplex_name_length_bucket{level=\"2\",le=\"9.0\"} 1", + "simplex_name_length_bucket{level=\"2\",le=\"10.0\"} 2", + "simplex_name_length_bucket{level=\"2\",le=\"11.0\"} 3", + "simplex_name_length_bucket{level=\"2\",le=\"63.0\"} 3", + "simplex_name_length_bucket{level=\"2\",le=\"+Inf\"} 3", + "simplex_name_length_sum{level=\"2\"} 30", + "simplex_name_length_count{level=\"2\"} 3", + "simplex_name_length_bucket{level=\"3\",le=\"3.0\"} 0", + "simplex_name_length_bucket{level=\"3\",le=\"4.0\"} 1", + "simplex_name_length_sum{level=\"3\"} 4", + "simplex_name_length_count{level=\"3\"} 1", + "simplex_names_block 1", + "simplex_names_refresh_failures_total 0" + ] + censusLines = T.lines . censusText <$> readTVarIO v + filter ("simplex_names{" `T.isPrefixOf`) <$> censusLines `shouldReturn` counted + censusLines >>= \ls -> forM_ lengths $ \l -> (l, l `elem` ls) `shouldBe` (l, True) + setFault node "eth_getBlockByNumber" HttpFailure + readCensus ns v + crFailures <$> readTVarIO v `shouldReturn` 1 + filter ("simplex_names{" `T.isPrefixOf`) <$> censusLines `shouldReturn` counted + +censusRegistry, censusSubnames, censusOtherResolver, censusOwner :: Address +censusRegistry = censusAddress 0x21 +censusSubnames = censusAddress 0x22 +censusOtherResolver = censusAddress 0x23 +censusOwner = censusAddress 0x24 + +censusAddress :: Word8 -> Address +censusAddress = fromJust . addressFromBytes . B.replicate 20 + +censusBase :: ByteString +censusBase = B.replicate 32 0x11 + +testServiceCounts :: IO () +testServiceCounts = withServiceStore $ \st -> do + withTransaction st $ \db -> do + insertProviderInvoice db "crypto" "c-overdue" (at 1) "open" + insertProviderInvoice db "crypto" "c-open" (at 60) "open" + insertProviderInvoice db "crypto" "c-paid" (at 1) "paid" + insertProviderInvoice db "stripe" "s-funded" (at 1) "open" + DB.execute + db + "INSERT INTO sx_badge_service_payments (payment_id, invoice_id, provider, amount, currency, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?)" + ("pay-s" :: Text, "s-funded" :: Text, "stripe" :: Text, price, "usd" :: Text, "pending" :: Text, t0, t0) + forM_ [(1, PSIssued), (2, PSIssued), (3, PSSuperseded)] $ \(n, status) -> + DB.execute + db + "INSERT INTO sx_badge_service_badge_purchases (purchase_key, master_key, initial_badge_type, current_badge_type, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?)" + (Binary $ digestFixture n, Binary $ digestFixture (n + 100), "supporter" :: Text, "supporter" :: Text, textEncode status, t0, t0) + forM_ [("sub-1", SSRenewing), ("sub-2", SSRenewing), ("sub-3", SSBillingRetry)] $ \(paymentId, state) -> + DB.execute + db + "INSERT INTO sx_badge_service_payments (payment_id, provider, status, subscription_state, created_at, updated_at) VALUES (?,?,?,?,?,?)" + (paymentId :: Text, "google" :: Text, "paid" :: Text, textEncode state, t0, t0) + forM_ codes $ \(n, paymentStatus, (redeemLimit, redeemCount), revokedAt, expiresAt) -> + DB.execute + db + "INSERT INTO sx_badge_service_badge_codes (code_hash, badge_type, months, code_payment_status, redeem_limit, redeem_count, revoked_at, expires_at, created_at) VALUES (?,?,?,?,?,?,?,?,?)" + (Binary $ digestFixture n, "supporter" :: Text, 1 :: Int, paymentStatus, redeemLimit, redeemCount, revokedAt, expiresAt, t0) + (invoices, badges) <- withTransaction st $ \db -> (,) <$> openInvoiceCounts db (at 10) <*> badgeCounts db (at 10) + invoices `shouldBe` [("crypto", 2, 1), ("stripe", 1, 0)] + let times = ProviderTimes {ptListedOk = M.fromList [("crypto", at 11), ("stripe", at 12)], ptWebhookAt = M.fromList [("stripe", at 13)]} + typedSamples (serviceText ["crypto", "stripe"] invoices badges times) + `shouldBe` [ "simplex_store_invoices_open{provider=\"crypto\"} 2", + "simplex_store_invoices_open{provider=\"stripe\"} 1", + "simplex_store_invoices_overdue{provider=\"crypto\"} 1", + "simplex_store_invoices_overdue{provider=\"stripe\"} 0", + "simplex_store_provider_list_ok_timestamp_seconds{provider=\"crypto\"} " <> seconds (at 11), + "simplex_store_provider_list_ok_timestamp_seconds{provider=\"stripe\"} " <> seconds (at 12), + "simplex_store_provider_webhook_timestamp_seconds{provider=\"stripe\"} " <> seconds (at 13), + "simplex_store_badges_purchases{status=\"acquiring\"} 0", + "simplex_store_badges_purchases{status=\"failed\"} 0", + "simplex_store_badges_purchases{status=\"issued\"} 2", + "simplex_store_badges_purchases{status=\"superseded\"} 1", + "simplex_store_badges_subscriptions{state=\"billingRetry\"} 1", + "simplex_store_badges_subscriptions{state=\"cancelled\"} 0", + "simplex_store_badges_subscriptions{state=\"ended\"} 0", + "simplex_store_badges_subscriptions{state=\"renewing\"} 2", + "simplex_store_badges_codes_open 2" + ] + closed <- withTransaction st $ \db -> do + DB.execute_ db "UPDATE sx_badge_service_invoices SET status = 'expired' WHERE status = 'open'" + openInvoiceCounts db (at 10) + closed `shouldBe` [] + filter ("simplex_store_invoices_" `T.isPrefixOf`) (typedSamples (serviceText ["crypto", "stripe"] closed badges times)) + `shouldBe` [ "simplex_store_invoices_open{provider=\"crypto\"} 0", + "simplex_store_invoices_open{provider=\"stripe\"} 0", + "simplex_store_invoices_overdue{provider=\"crypto\"} 0", + "simplex_store_invoices_overdue{provider=\"stripe\"} 0" + ] + where + -- open: the first two; then unpaid, used up, revoked, expired + codes :: [(Word8, Text, (Int, Int), Maybe UTCTime, Maybe UTCTime)] + codes = + [ (1, "paid", (1, 0), Nothing, Nothing), + (2, "free", (3, 1), Nothing, Just (at 120)), + (3, "unpaid", (1, 0), Nothing, Nothing), + (4, "paid", (2, 2), Nothing, Nothing), + (5, "paid", (1, 0), Just (at 5), Nothing), + (6, "paid", (1, 0), Nothing, Just (at 5)) + ] + +testNamesCounts :: IO () +testNamesCounts = withServiceStore $ \st -> do + let placedAs status rid n terms = withTransaction st $ \db -> do + insertInvoiceRow db ("inv-" <> rid) (at 60) status + createPaidRegistration db (newReg rid ownerA $ order (rid <> "bakery") n) terms (paidOrder $ "inv-" <> rid) t0 `shouldReturn` Right () + moved rid minute change = withTransaction st $ \db -> do + Just r <- getRegistration db (RegistrationId rid) + updateRegistration db (change r) (at minute) `shouldReturn` Written 2 + placedAs "open" "u" 1 (OTInvoice btc price) + mapM_ (\(rid, n) -> placedAs "paid" rid n (OTInvoice btc price)) [("src", 2), ("q", 3), ("c1", 4), ("c2", 5), ("rg", 6), ("f", 7)] + moved "src" 1 $ \r -> r {nrPhase = PTaken} + moved "f" 1 $ \r -> r {nrPhase = PFailed} + moved "c1" 2 $ \r -> r {nrPhase = PCommitted} + moved "c2" 3 $ \r -> r {nrPhase = PCommitted, nrReveal = Just reveal} + moved "rg" 4 $ \r -> r {nrPhase = PRegistering} + moved "q" 5 $ \r -> r {nrPhase = PQueued} + withTransaction st $ \db -> do + creditBalance db (RegistrationId "src") 6000 t0 + creditBalance db (RegistrationId "f") 2500 t0 + insertInvoiceRow db "inv-badge" (at 60) "open" + placedAs "open" "h" 8 (OTBalance (RegistrationId "src") price 6000 (Just btc)) + withTransaction st $ \db -> do + forM_ [(40, 0, 6), (41, 1, 7), (42, 1, 8), (43, 2, 9)] $ \(n, nonce, minute) -> + insertTransaction db (TxCommit $ RegistrationId "q") NewTransaction {ntxRaw = "raw", ntxHash = TxHash $ digestFixture n, ntxNonce = nonce} (at minute) + recordTransactionOutcome db (TxHash $ digestFixture 40) TSPending TSMined (Just 120) `shouldReturn` True + Just mined <- insertRecordEdit db (newEdit ownerA 1 setChannel) (at 8) + insertRecordEdit db (newEdit ownerB 3 setChannel) (at 9) >>= (`shouldSatisfy` isJust) + settleRecordEdit db mined (ESMined Nothing) `shouldReturn` True + counts <- withTransaction st namesCounts + typedSamples (namesText counts noChainRead) + `shouldBe` [ "simplex_store_names_orders{phase=\"unpaid\"} 2", + "simplex_store_names_orders{phase=\"queued\"} 1", + "simplex_store_names_orders{phase=\"committing\"} 0", + "simplex_store_names_orders{phase=\"committed\"} 2", + "simplex_store_names_orders{phase=\"registering\"} 1", + "simplex_store_names_orders{phase=\"registered\"} 0", + "simplex_store_names_orders{phase=\"taken\"} 1", + "simplex_store_names_orders{phase=\"failed\"} 1", + "simplex_store_names_queue_oldest_change_timestamp_seconds " <> seconds (at 3), + "simplex_store_names_transactions_pending 2", + "simplex_store_names_transactions_oldest_pending_timestamp_seconds " <> seconds (at 7), + "simplex_store_names_invoices_open 2", + "simplex_store_names_balances_held 1", + "simplex_store_names_balances_held_cents 6000", + "simplex_store_names_record_edits_pending 1", + "simplex_store_names_record_edits_oldest_pending_timestamp_seconds " <> seconds (at 9), + "simplex_store_names_chain_read_ok 0" + ] + +testChainRead :: TVar ChaChaDRG -> IO () +testChainRead g = withServiceStore $ \st -> withFakeEthNode $ \node -> do + c <- fakeChainClient g node + ct <- newContracts st TLDSimplex minPriceCentsPerYear c + let ns = NamesSource {nsChain = c, nsContracts = ct} + registrar <- addressFromPrivateKey g fakeRegistrarKey + answerCall node fakeSponsor (selector "registrarAllowance(address)" <> addressWord registrar) $ Returns $ uint256 (123456 * 10 ^ (16 :: Int) + 5 :: Integer) + setFault node "eth_getBalance" $ Answers $ J.String "0x15af1d78b58c40000" + v <- newTVarIO noChainRead + started <- getCurrentTime + readChain ns v + ended <- getCurrentTime + ChainRead {crReadAt} <- readTVarIO v + readAt <- maybe (fail "no chain read succeeded") pure crReadAt + (started <= readAt && readAt <= ended) `shouldBe` True + let held general = ["simplex_store_names_terms_held{kind=\"general\"} " <> general, "simplex_store_names_terms_held{kind=\"community\"} 0"] + chainSamples <$> readTVarIO v + `shouldReturn` [ "simplex_store_names_registrar_balance_wei 25000000000000000000", + "simplex_store_names_sponsor_allowance_cents 123456", + "simplex_store_names_chain_head_block 1", + "simplex_store_names_chain_head_timestamp_seconds " <> tshow genesisTime, + "simplex_store_names_chain_base_fee_wei " <> tshow (fakeBaseFee :: Integer) + ] + <> held "0" + <> ["simplex_store_names_chain_read_timestamp_seconds " <> seconds readAt, "simplex_store_names_chain_read_ok 1"] + termsText ct (fromJust $ termsOf $ fakeTermsLink fakeGeneralTermsText) `shouldReturn` Right "SimpleX general terms v1" + readChain ns v + fetched <- chainSamples <$> readTVarIO v + filter ("simplex_store_names_terms_held" `T.isPrefixOf`) fetched `shouldBe` held "1" + setFault node "eth_getBalance" HttpFailure + readChain ns v + chainSamples <$> readTVarIO v `shouldReturn` init fetched <> ["simplex_store_names_chain_read_ok 0"] + +testScrape :: TVar ChaChaDRG -> IO () +testScrape g = withServiceStore $ \st -> withFakeEthNode $ \node -> do + c <- fakeChainClient g node + ct <- newContracts st TLDSimplex minPriceCentsPerYear c + listedOk <- newTVarIO M.empty + webhookAt <- newTVarIO M.empty + v <- newTVarIO noChainRead + census <- newTVarIO noCensusRead + let badgesOnly = MetricsEnv {meStore = st, meProviders = ["crypto"], meListedOk = listedOk, meWebhookAt = webhookAt, meNames = Nothing} + ns = NamesSource {nsChain = c, nsContracts = ct} + withApp (metricsApp badgesOnly v census) $ \client -> do + r <- webGet client "/metrics" + (statusOf r, lookup hContentType (responseHeaders r)) `shouldBe` (200, Just "text/plain; version=0.0.4; charset=utf-8") + let samples = typedSamples $ bodyText r + samples `shouldContain` ["simplex_store_badges_codes_open 0"] + samples `shouldContain` ["simplex_store_invoices_open{provider=\"crypto\"} 0"] + filter (\l -> any (`T.isPrefixOf` l) ["simplex_store_names_", "simplex_names", "simplex_name_"]) samples `shouldBe` [] + statusOf <$> webGet client "/" `shouldReturn` 404 + statusOf <$> webRequestBody client "POST" "/metrics" [] "" `shouldReturn` 405 + (reads', _) <- pausedAt node "eth_getBalance" (runChainReads ns v) + (`finally` cancel reads') $ withApp (metricsApp badgesOnly {meNames = Just ns} v census) $ \client -> do + r <- timeout scrapeMicros (webGet client "/metrics") >>= maybe (fail "the scrape waited for the node") pure + let samples = typedSamples $ bodyText r + forM_ ["simplex_store_badges_codes_open 0", "simplex_store_names_orders{phase=\"unpaid\"} 0", "simplex_names_refresh_failures_total 0"] $ \s -> + samples `shouldContain` [s] + filter ("simplex_store_names_chain_" `T.isPrefixOf`) samples `shouldBe` ["simplex_store_names_chain_read_ok 0"] + filter ("simplex_names{" `T.isPrefixOf`) samples `shouldBe` [] + +scrapeMicros :: Int +scrapeMicros = 2000000 + +bodyText :: Response LB.ByteString -> Text +bodyText = decodeUtf8 . LB.toStrict . responseBody + +typedSamples :: Text -> [Text] +typedSamples text = [l | l <- ls, not (T.null l || "#" `T.isPrefixOf` l), described (T.takeWhile (`notElem` [' ', '{']) l)] + where + ls = T.lines text + described name = any (("# TYPE " <> name <> " ") `T.isPrefixOf`) ls && any (("# HELP " <> name <> " ") `T.isPrefixOf`) ls + +chainSamples :: ChainRead -> [Text] +chainSamples = filter (\l -> any (`T.isPrefixOf` l) ["simplex_store_names_registrar_", "simplex_store_names_sponsor_", "simplex_store_names_chain_", "simplex_store_names_terms_"]) . typedSamples . namesText noCounts + where + noCounts = NamesCounts {ncPhases = [], ncQueueOldestChange = Nothing, ncPendingTransactions = 0, ncOldestPending = Nothing, ncOpenInvoices = 0, ncHeldBalances = 0, ncHeldCents = 0, ncPendingEdits = 0, ncOldestEdit = Nothing} + +seconds :: UTCTime -> Text +seconds t = tshow (floor $ utcTimeToPOSIXSeconds t :: Integer) diff --git a/tests/Bots/StoreService/NamesCodesTests.hs b/tests/Bots/StoreService/NamesCodesTests.hs index 80b0851403..6ef508901b 100644 --- a/tests/Bots/StoreService/NamesCodesTests.hs +++ b/tests/Bots/StoreService/NamesCodesTests.hs @@ -104,7 +104,7 @@ testCodeStates = do -- The route as namesRoutes registers it, since a NamesEnv needs a running chat controller. withCodeCheck :: (DBStore -> Client -> IO a) -> IO a withCodeCheck action = withServiceStore $ \st -> do - http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing} + http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing} let routes = \case ["api", "code", "check"] -> Just Route {rMethod = "POST", rLimit = codeLimit, rHandler = codeCheckHandler st} _ -> Nothing diff --git a/tests/Bots/StoreService/NamesSearchTests.hs b/tests/Bots/StoreService/NamesSearchTests.hs index a11a88a99a..c4c8252f57 100644 --- a/tests/Bots/StoreService/NamesSearchTests.hs +++ b/tests/Bots/StoreService/NamesSearchTests.hs @@ -102,7 +102,7 @@ withSearch :: TVar ChaChaDRG -> (NamesConfig -> NamesConfig) -> (Contracts -> Co withSearch g adjustConfig adjust action = withServiceStore $ \store -> withContracts g $ \node ct -> do fetched <- newTVarIO [] searchesAtFetch <- newTVarIO [] - http@HttpEnv {heSearches} <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing} + http@HttpEnv {heSearches} <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing} let fetch sl = atomically (readTVar heSearches >>= modifyTVar' searchesAtFetch . (:)) >> fakeFetch fetched sl ps <- newProfiles defaultProfileLimits {fetchTimeout = fetchMicros} monotonicSeconds fetch stubResolver let cfg = adjustConfig $ testNamesConfig node diff --git a/tests/Bots/StoreService/NamesSignatureTests.hs b/tests/Bots/StoreService/NamesSignatureTests.hs index ded9b3ef34..a28bac923f 100644 --- a/tests/Bots/StoreService/NamesSignatureTests.hs +++ b/tests/Bots/StoreService/NamesSignatureTests.hs @@ -119,6 +119,12 @@ namesSignatureTests = describe "StoreService names: signatures" $ do decodeString (uint256 (4096 :: Int) <> B.drop 32 retString) `shouldBe` Left AEOffset decodeString (B.take 80 retString) `shouldBe` Left AEShort decodeString (abiTuple [Dynamic $ abiBytes "\xff"]) `shouldBe` Left AENotUtf8 + it "pairs the bytes32[] and string[] of a subname registrar's children, as viem encodes them, and refuses arrays of different lengths" $ do + decodeLabels (hx "000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000000000000000295b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea69513447e2d376aacd434727887590dd448cda8f2d30c4ace903d31fe209f8ad8000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000473686f70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005636166c3a9000000000000000000000000000000000000000000000000000000") + `shouldBe` Right [(hx "95b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea6", "shop"), (hx "9513447e2d376aacd434727887590dd448cda8f2d30c4ace903d31fe209f8ad8", "café")] + decodeLabels (abiTuple [Dynamic (uint256 (0 :: Int)), Dynamic (uint256 (0 :: Int))]) `shouldBe` Right [] + decodeLabels (hx "00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000195b5b9fbb0d3def5b5033d13f74f6c14f8a5b404b26a9082bbaffd77a3a90ea6000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000473686f70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005636166c3a9000000000000000000000000000000000000000000000000000000") + `shouldBe` Left AEOffset testMessage :: ByteString -> ByteString -> Text -> Text -> IO () testMessage msg expected digest signature = do diff --git a/tests/Bots/StoreService/NamesStoreTests.hs b/tests/Bots/StoreService/NamesStoreTests.hs index ad6cc94207..415147aa83 100644 --- a/tests/Bots/StoreService/NamesStoreTests.hs +++ b/tests/Bots/StoreService/NamesStoreTests.hs @@ -3,7 +3,7 @@ {-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE OverloadedStrings #-} -module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, invoiceStatus, atNonce, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where +module Bots.StoreService.NamesStoreTests (namesStoreTests, at, btc, insertInvoiceRow, insertProviderInvoice, invoiceStatus, atNonce, issue, newEdit, newReg, order, ownerA, ownerB, paidOrder, placeCode, placePaid, price, reg, reveal, setChannel, signature, t0, usable, write) where import Bots.StoreService.FakeEthNode (plainTerms) import Bots.StoreService.WebTests (columnsOf, digestFixture, runMigrations, withServiceStore) @@ -132,11 +132,14 @@ paidOrder :: Text -> PaidOrder paidOrder invId = PaidOrder {poMethod = btc, poPrice = price, poInvoiceId = InvoiceId invId, poProviderRef = "ref-" <> invId} insertInvoiceRow :: DB.Connection -> Text -> UTCTime -> Text -> IO () -insertInvoiceRow db invId expiresAt status = +insertInvoiceRow db = insertProviderInvoice db "crypto" + +insertProviderInvoice :: DB.Connection -> Text -> Text -> UTCTime -> Text -> IO () +insertProviderInvoice db provider invId expiresAt status = DB.execute db "INSERT INTO sx_badge_service_invoices (invoice_id, provider, price, amount, currency, payment_address, payment_crypto_currency, payment_crypto_amount, expires_at, status, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)" - ((invId, "crypto" :: Text, price, price, "usd" :: Text, "bc1qexampleaddress" :: Text, "btc" :: Text, "0.00312" :: Text) :. (expiresAt, status, t0, t0)) + ((invId, provider, price, price, "usd" :: Text, "bc1qexampleaddress" :: Text, "btc" :: Text, "0.00312" :: Text) :. (expiresAt, status, t0, t0)) -- | A paid order whose invoice, named after the registration, expires at the given time. placePaid :: DBStore -> Text -> Address -> OrderFields -> UTCTime -> IO (Either RegistrationRefused ()) diff --git a/tests/Bots/StoreService/NamesWiringTests.hs b/tests/Bots/StoreService/NamesWiringTests.hs index 8f2f753052..f34cd31537 100644 --- a/tests/Bots/StoreService/NamesWiringTests.hs +++ b/tests/Bots/StoreService/NamesWiringTests.hs @@ -118,7 +118,7 @@ probeLimit = Limit {lmName = "probe", lmPerMinute = 1} withProbeRouter :: (Client -> IO a) -> IO a withProbeRouter action = do - http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing} + http <- newHttpEnv ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = "", lServeWebapp = False, lBadgesExportDir = Nothing, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing} let routes = \case ["api", "probe", _] -> Just Route {rMethod = "GET", rLimit = probeLimit, rHandler = \_ respond -> respond (jsonResponse status200 (J.object []))} _ -> Nothing diff --git a/tests/Bots/StoreService/WebTests.hs b/tests/Bots/StoreService/WebTests.hs index 37e8d9fe40..3ffd96bcd9 100644 --- a/tests/Bots/StoreService/WebTests.hs +++ b/tests/Bots/StoreService/WebTests.hs @@ -304,7 +304,7 @@ badgeWebTests = do it "refuses a settlement instant in the future, as a millisecond clock gives" testFutureSettledInstantIsRefused describe "store service poller" $ do it "settles a payment with no webhook delivered anywhere" testSettlesWithNoWebhookAtAll - it "reads only the invoices it is waiting on, and lists on the stray cadence" testPassReadsWhatItAwaits + it "reads only the invoices it is waiting on, and lists on the stray cadence, recording when the provider's list succeeded" testPassReadsWhatItAwaits it "asks the provider nothing when it is waiting on nothing" testIdlePassAsksNothing it "lists instead of reading once too many invoices are open" testManyOpenInvoicesList it "sweeps nothing when no provider can account for the rows" testNoProviderAccountsForNothing @@ -314,7 +314,7 @@ badgeWebTests = do it "passes over a listed invoice this service does not hold" testForeignRefIsPassedOver it "changes nothing when the provider fails, and settles on the next pass" testProviderFailureLosesNothing it "asks a stalled BTCPay once in a pass and again in the next, settling a card invoice listed after its invoices" testStalledBTCPayHoldsNoCardSettlement - it "reads the overdue invoices on their own after a list that timed out" testListTimeoutKeepsTheReads + it "reads the overdue invoices on their own after a list that timed out, recording no successful list" testListTimeoutKeepsTheReads it "reads every invoice of a provider whose reads fail without a timeout" testFailedReadKeepsTheOtherReads it "reads the cadence fresh from the waiters each pass" testCadenceFollowsTheWaiters it "cuts an idle sleep short when a browser arrives during it" testWaiterCutsTheIdleSleepShort @@ -350,7 +350,7 @@ badgeWebTests = do it "pages past a hundred open invoices, a request per hundred" testListPaginates it "stops paging at its ceiling rather than looping on a server that ignores take" testPagingStopsAtTheCeiling describe "store service webhook" $ do - it "verifies a real BTCPay signature over the bytes as received, and queues" testWebhookVerifiesARealSignature + it "verifies a real BTCPay signature over the bytes as received, and queues, recording when only a verified delivery came" testWebhookVerifiesARealSignature it "hands the adapter the raw body, byte for byte" testWebhookPassesTheRawBytes it "queues a read and answers 200 empty, reaching no provider" testWebhookQueuesARead it "settles by the poller's own path, so there is one settlement lane" testWebhookSettlesByThePollerPath @@ -1257,7 +1257,7 @@ prepareStaticDir root = do testServiceConfig :: FilePath -> Bool -> ServiceConfig testServiceConfig staticDir trustForwarded = ServiceConfig - { listener = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = True, lBadgesExportDir = Nothing, lTrustForwardedFor = trustForwarded, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing}, + { listener = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = True, lBadgesExportDir = Nothing, lTrustForwardedFor = trustForwarded, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing}, btcpay = Nothing, stripe = Nothing, poll = PollConfig {pWaitingSeconds = 3, pIdleSeconds = 60}, @@ -1294,7 +1294,7 @@ testExportWebapp = do withTempDirectory "tests/tmp" "badge-export" $ \root -> do staticDir <- prepareStaticDir root let outDir = root "out" - lc = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = False, lBadgesExportDir = Just outDir, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing} + lc = ListenerConfig {lHost = "127.0.0.1", lBadgesPort = 0, lBadgesStaticDir = staticDir, lServeWebapp = False, lBadgesExportDir = Just outDir, lTrustForwardedFor = False, lNamesStaticDir = Nothing, lNamesExportDir = Nothing, lNamesPort = Nothing, lMetricsPort = Nothing} stripeCfg = Just StripeConfig {sSecretKey = "rk_test_x", sPublishableKey = "pk_test_injected", sWebhookSecret = "whsec_x", sSessionMinutes = 60, sHost = "https://api.stripe.com", sTimeoutMicros = providerTimeoutMicros, sCheckoutTimeoutMicros = providerCheckoutTimeoutMicros} exportWebapp (testServiceConfig staticDir False) {listener = lc, stripe = stripeCfg} idx <- LB.readFile (outDir "index.html") @@ -3030,9 +3030,13 @@ testPassReadsWhatItAwaits = bounded "reads what it awaits" $ withStubPoller race b <- seedOpenRef (weStore env) 2 "p-2" someExpiry c <- seedOpenRef (weStore env) 3 "p-3" someExpiry setSignals ref [("p-1", settledSignal), ("p-2", settledSignal), ("p-3", settledSignal)] + passStarted <- getCurrentTime runOnePass poller stubCalls ref `shouldReturn` [StubRead "p-1", StubRead "p-2", StubRead "p-3", StubListOpen] mapM (invoiceStatus (weStore env)) [a, b, c] `shouldReturn` [ISPaid, ISPaid, ISPaid] + listed <- readTVarIO (peListedOk poller) + Map.keys listed `shouldBe` ["crypto"] + all (>= passStarted) listed `shouldBe` True testIdlePassAsksNothing :: IO () testIdlePassAsksNothing = bounded "idle pass" $ withStubPoller raceHold $ \ref poller env _ -> do @@ -3175,6 +3179,7 @@ testListTimeoutKeepsTheReads = bounded "list timeout" $ withStubPoller raceHold runOnePass poller stubCalls ref `shouldReturn` [StubListOpen, StubRead "p-overdue"] invoiceStatus (weStore env) overdue `shouldReturn` ISExpired + readTVarIO (peListedOk poller) `shouldReturn` Map.empty testFailedReadKeepsTheOtherReads :: IO () testFailedReadKeepsTheOtherReads = bounded "failed read" $ withStubPoller raceHold $ \ref poller env _ -> do @@ -3779,14 +3784,18 @@ testWebhookVerifiesARealSignature = bounded "real signature" $ withFakePoller $ Just row <- getInvoice (weStore env) iid let secret = bWebhookSecret (fbConfig fake) body = webhookEvent "InvoiceSettled" (irProviderRef row) + unsigned <- postWebhook client [] body + statusOf unsigned `shouldBe` 400 + readTVarIO (weWebhookAt env) `shouldReturn` Map.empty delivered <- postWebhook client (webhookSigHeader secret body) body statusOf delivered `shouldBe` 200 responseBody delivered `shouldBe` "" + verifiedAt <- readTVarIO (weWebhookAt env) + Map.keys verifiedAt `shouldBe` ["crypto"] tampered <- postWebhook client (webhookSigHeader secret body) (body <> " ") statusOf tampered `shouldBe` 400 responseBody tampered `shouldBe` "" - unsigned <- postWebhook client [] body - statusOf unsigned `shouldBe` 400 + readTVarIO (weWebhookAt env) `shouldReturn` verifiedAt invoiceStatus (weStore env) iid `shouldReturn` ISOpen setInvoiceState fake (irProviderRef row) ["status" .= ("Settled" :: Text), "paymentMethodPaid" .= ("0.32095000" :: Text)] drainHints poller diff --git a/tests/Test.hs b/tests/Test.hs index aa52514625..0654ae6fac 100644 --- a/tests/Test.hs +++ b/tests/Test.hs @@ -9,6 +9,7 @@ import Bots.StoreService.CatalogTests import Bots.StoreService.ConfigTests import Bots.StoreService.GroupIntegrationTests import Bots.StoreService.GroupTests +import Bots.StoreService.MetricsTests import Bots.StoreService.NameRegistrationTests import Bots.StoreService.NamesBalancesTests import Bots.StoreService.NamesLateCreditTests @@ -110,6 +111,7 @@ main = do badgeStripeTests badgeNameRegistrationTests badgeStoreVerifierTests + storeMetricsTests namesConfigTests namesBalancesTests namesLateCreditTests